feat(backend): add moss-tts-cpp text-to-speech backend (#10860)

* feat(backend): add moss-tts-cpp text-to-speech backend

Add a Go + purego backend wrapping the moss-tts.cpp ggml port of the OpenMOSS
MOSS-TTS-Local v1.5 text-to-speech model (GPT-J local transformer decoded through
MOSS-Audio-Tokenizer-v2), producing 48 kHz stereo audio with optional
reference-audio voice cloning. Mirrors the qwen3-tts-cpp backend: dlopen the
static-ggml shared library, bind the moss-tts.cpp C-API via purego, and serve
the gRPC TTS method. A thin C shim holds the pipeline handle and copies engine
PCM into a Go-freeable buffer.

Wires the CI registration: backend-matrix.yml (CPU, CUDA 12/13, Intel SYCL
f16/f32, Vulkan, ROCm, NVIDIA L4T, plus Darwin metal), backend/index.yaml metas
and image entries pointing at mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF, the
root Makefile build targets, and the changed-backends.js path mapping.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: list the moss-tts-cpp backend among the LocalAI-maintained engines

Add moss-tts.cpp to the README "Backends built by us" table, the
Text-to-Speech compatibility table, and the reference-audio voice-cloning
backend list, so the new backend is documented alongside its peers.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(moss-tts-cpp): pin moss-tts.cpp to the squashed single-commit release

moss-tts.cpp history was collapsed to a single commit; repoint MOSSTTS_CPP_VERSION
to ee722b8e9205ee9b1b1c398a4e87e4e393e9be41.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* backend(moss-tts-cpp): add the moss-tts-cpp-development gallery meta

The gallery had the -development image entries but no matching -development
meta anchor (as locate-anything-cpp and depth-anything-cpp have), so the master
build was not installable as a gallery backend. Add moss-tts-cpp-development
mirroring the production meta with the -development capability image names.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
LocalAI [bot]
2026-07-17 09:26:12 +02:00
committed by GitHub
parent 0bd7a29f31
commit 3bb0d1cb49
23 changed files with 1591 additions and 3 deletions

View File

@@ -858,6 +858,19 @@ include:
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'cublas'
cuda-major-version: "12"
cuda-minor-version: "8"
platforms: 'linux/amd64'
tag-latest: 'auto'
tag-suffix: '-gpu-nvidia-cuda-12-moss-tts-cpp'
runs-on: 'ubuntu-latest'
base-image: "ubuntu:24.04"
skip-drivers: 'false'
backend: "moss-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'cublas'
cuda-major-version: "12"
cuda-minor-version: "8"
@@ -1909,6 +1922,19 @@ include:
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'cublas'
cuda-major-version: "13"
cuda-minor-version: "0"
platforms: 'linux/amd64'
tag-latest: 'auto'
tag-suffix: '-gpu-nvidia-cuda-13-moss-tts-cpp'
runs-on: 'ubuntu-latest'
base-image: "ubuntu:24.04"
skip-drivers: 'false'
backend: "moss-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'cublas'
cuda-major-version: "13"
cuda-minor-version: "0"
@@ -1961,6 +1987,19 @@ include:
backend: "qwen3-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
- build-type: 'cublas'
cuda-major-version: "13"
cuda-minor-version: "0"
platforms: 'linux/arm64'
skip-drivers: 'false'
tag-latest: 'auto'
tag-suffix: '-nvidia-l4t-cuda-13-arm64-moss-tts-cpp'
base-image: "ubuntu:24.04"
ubuntu-version: '2404'
runs-on: 'ubuntu-24.04-arm'
backend: "moss-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
- build-type: 'cublas'
cuda-major-version: "13"
cuda-minor-version: "0"
@@ -4529,6 +4568,35 @@ include:
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
# moss-tts-cpp
- build-type: ''
cuda-major-version: ""
cuda-minor-version: ""
platforms: 'linux/amd64'
platform-tag: 'amd64'
tag-latest: 'auto'
tag-suffix: '-cpu-moss-tts-cpp'
runs-on: 'ubuntu-latest'
base-image: "ubuntu:24.04"
skip-drivers: 'false'
backend: "moss-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: ''
cuda-major-version: ""
cuda-minor-version: ""
platforms: 'linux/arm64'
platform-tag: 'arm64'
tag-latest: 'auto'
tag-suffix: '-cpu-moss-tts-cpp'
runs-on: 'ubuntu-24.04-arm'
base-image: "ubuntu:24.04"
skip-drivers: 'false'
backend: "moss-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
# omnivoice-cpp
- build-type: ''
cuda-major-version: ""
@@ -4571,6 +4639,19 @@ include:
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'sycl_f32'
cuda-major-version: ""
cuda-minor-version: ""
platforms: 'linux/amd64'
tag-latest: 'auto'
tag-suffix: '-gpu-intel-sycl-f32-moss-tts-cpp'
runs-on: 'ubuntu-latest'
base-image: "intel/oneapi-basekit:2025.3.0-0-devel-ubuntu24.04"
skip-drivers: 'false'
backend: "moss-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'sycl_f32'
cuda-major-version: ""
cuda-minor-version: ""
@@ -4597,6 +4678,19 @@ include:
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'sycl_f16'
cuda-major-version: ""
cuda-minor-version: ""
platforms: 'linux/amd64'
tag-latest: 'auto'
tag-suffix: '-gpu-intel-sycl-f16-moss-tts-cpp'
runs-on: 'ubuntu-latest'
base-image: "intel/oneapi-basekit:2025.3.0-0-devel-ubuntu24.04"
skip-drivers: 'false'
backend: "moss-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'sycl_f16'
cuda-major-version: ""
cuda-minor-version: ""
@@ -4624,6 +4718,20 @@ include:
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'vulkan'
cuda-major-version: ""
cuda-minor-version: ""
platforms: 'linux/amd64'
platform-tag: 'amd64'
tag-latest: 'auto'
tag-suffix: '-gpu-vulkan-moss-tts-cpp'
runs-on: 'ubuntu-latest'
base-image: "ubuntu:24.04"
skip-drivers: 'false'
backend: "moss-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'vulkan'
cuda-major-version: ""
cuda-minor-version: ""
@@ -4652,6 +4760,20 @@ include:
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'vulkan'
cuda-major-version: ""
cuda-minor-version: ""
platforms: 'linux/arm64'
platform-tag: 'arm64'
tag-latest: 'auto'
tag-suffix: '-gpu-vulkan-moss-tts-cpp'
runs-on: 'ubuntu-24.04-arm'
base-image: "ubuntu:24.04"
skip-drivers: 'false'
backend: "moss-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'vulkan'
cuda-major-version: ""
cuda-minor-version: ""
@@ -4679,6 +4801,19 @@ include:
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2204'
- build-type: 'cublas'
cuda-major-version: "12"
cuda-minor-version: "0"
platforms: 'linux/arm64'
skip-drivers: 'false'
tag-latest: 'auto'
tag-suffix: '-nvidia-l4t-arm64-moss-tts-cpp'
base-image: "nvcr.io/nvidia/l4t-jetpack:r36.4.0"
runs-on: 'ubuntu-24.04-arm'
backend: "moss-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2204'
- build-type: 'cublas'
cuda-major-version: "12"
cuda-minor-version: "0"
@@ -4705,6 +4840,19 @@ include:
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'hipblas'
cuda-major-version: ""
cuda-minor-version: ""
platforms: 'linux/amd64'
tag-latest: 'auto'
tag-suffix: '-gpu-rocm-hipblas-moss-tts-cpp'
base-image: "rocm/dev-ubuntu-24.04:6.4.4"
runs-on: 'ubuntu-latest'
skip-drivers: 'false'
backend: "moss-tts-cpp"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'hipblas'
cuda-major-version: ""
cuda-minor-version: ""
@@ -5622,6 +5770,10 @@ includeDarwin:
tag-suffix: "-metal-darwin-arm64-qwen3-tts-cpp"
build-type: "metal"
lang: "go"
- backend: "moss-tts-cpp"
tag-suffix: "-metal-darwin-arm64-moss-tts-cpp"
build-type: "metal"
lang: "go"
- backend: "omnivoice-cpp"
tag-suffix: "-metal-darwin-arm64-omnivoice-cpp"
build-type: "metal"

View File

@@ -1,5 +1,5 @@
# Disable parallel execution for backend builds
.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/bonsai backends/outetts backends/piper backends/stablediffusion-ggml backends/whisper backends/crispasr backends/parakeet-cpp backends/moss-transcribe-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/cloud-proxy backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/longcat-video backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin
.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/bonsai backends/outetts backends/piper backends/stablediffusion-ggml backends/whisper backends/crispasr backends/parakeet-cpp backends/moss-transcribe-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/cloud-proxy backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/longcat-video backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/moss-tts-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin
GOCMD=go
GOTEST=$(GOCMD) test
@@ -1249,6 +1249,7 @@ BACKEND_DEPTH_ANYTHING_CPP = depth-anything-cpp|golang|.|false|true
BACKEND_VOXTRAL = voxtral|golang|.|false|true
BACKEND_ACESTEP_CPP = acestep-cpp|golang|.|false|true
BACKEND_QWEN3_TTS_CPP = qwen3-tts-cpp|golang|.|false|true
BACKEND_MOSS_TTS_CPP = moss-tts-cpp|golang|.|false|true
BACKEND_OMNIVOICE_CPP = omnivoice-cpp|golang|.|false|true
BACKEND_VIBEVOICE_CPP = vibevoice-cpp|golang|.|false|true
BACKEND_LOCALVQE = localvqe|golang|.|false|true
@@ -1375,6 +1376,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_WHISPERX)))
$(eval $(call generate-docker-build-target,$(BACKEND_ACE_STEP)))
$(eval $(call generate-docker-build-target,$(BACKEND_ACESTEP_CPP)))
$(eval $(call generate-docker-build-target,$(BACKEND_QWEN3_TTS_CPP)))
$(eval $(call generate-docker-build-target,$(BACKEND_MOSS_TTS_CPP)))
$(eval $(call generate-docker-build-target,$(BACKEND_OMNIVOICE_CPP)))
$(eval $(call generate-docker-build-target,$(BACKEND_VIBEVOICE_CPP)))
$(eval $(call generate-docker-build-target,$(BACKEND_LOCALVQE)))
@@ -1394,7 +1396,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_SUPERTONIC)))
docker-save-%: backend-images
docker save local-ai-backend:$* -o backend-images/$*.tar
docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-bonsai docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-longcat-video docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-moss-transcribe-cpp docker-build-privacy-filter
docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-bonsai docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-longcat-video docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-moss-tts-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-moss-transcribe-cpp docker-build-privacy-filter
########################################################
### Mock Backend for E2E Tests

View File

@@ -233,6 +233,7 @@ Most backends wrap a best-in-class upstream engine. A handful of them are native
|---------|-------------|
| [parakeet.cpp](https://github.com/mudler/parakeet.cpp) | C++/GGML port of NVIDIA NeMo Parakeet ASR (tdt/ctc/rnnt/hybrid), with cache-aware streaming transcription |
| [moss-transcribe.cpp](https://github.com/mudler/moss-transcribe.cpp) | C++/GGML port of OpenMOSS MOSS-Transcribe-Diarize: joint long-form transcription, speaker diarization and timestamping in a single pass |
| [moss-tts.cpp](https://github.com/mudler/moss-tts.cpp) | C++/GGML port of the OpenMOSS MOSS-TTS family: text-to-speech (MOSS-TTS-Local v1.5, 48 kHz stereo) with reference-audio voice cloning, through the MOSS-Audio-Tokenizer neural codec |
| [ced.cpp](https://github.com/mudler/ced.cpp) | C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition |
| [voice-detect.cpp](https://github.com/mudler/voice-detect.cpp) | Speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend |
| [voxtral-tts.c](https://github.com/mudler/voxtral-tts.c) | Voxtral Realtime 4B speech-to-text in pure C |

5
backend/go/moss-tts-cpp/.gitignore vendored Normal file
View File

@@ -0,0 +1,5 @@
moss-tts-cpp
*.so
*.dylib
sources/
package/

View File

@@ -0,0 +1,48 @@
cmake_minimum_required(VERSION 3.18)
project(gomosstts-cpp LANGUAGES C CXX)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
set(MOSS_DIR ${CMAKE_CURRENT_SOURCE_DIR}/sources/moss-tts.cpp)
# Override upstream's CMAKE_CUDA_ARCHITECTURES before add_subdirectory.
if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES)
set(CMAKE_CUDA_ARCHITECTURES "75-virtual;80-virtual;86-real;89-real")
endif()
# Add the upstream project. Its own CMakeLists vendors ggml under
# third_party/ggml and builds the moss-tts library (STATIC by default, since we
# do NOT set MOSS_TTS_SHARED). EXCLUDE_FROM_ALL keeps its CLI tool / tests /
# examples from building unless referenced.
add_subdirectory(${MOSS_DIR} mosstts EXCLUDE_FROM_ALL)
add_library(gomosstts-cpp MODULE cpp/mossttscpp.cpp)
target_link_libraries(gomosstts-cpp PRIVATE moss-tts)
target_include_directories(gomosstts-cpp PRIVATE ${MOSS_DIR}/include ${MOSS_DIR}/src)
target_include_directories(gomosstts-cpp SYSTEM PRIVATE ${MOSS_DIR}/third_party/ggml/include)
# Link GPU backends if the upstream ggml created them. moss-tts links these
# PUBLIC when enabled, but we re-list them so the MODULE carries the symbols
# even under static ggml (mirrors qwen3-tts-cpp).
foreach(backend blas cuda hip metal vulkan sycl)
if(TARGET ggml-${backend})
target_link_libraries(gomosstts-cpp PRIVATE ggml-${backend})
if(backend STREQUAL "cuda")
find_package(CUDAToolkit QUIET)
if(CUDAToolkit_FOUND)
target_link_libraries(gomosstts-cpp PRIVATE CUDA::cudart)
endif()
endif()
endif()
endforeach()
if(MSVC)
target_compile_options(gomosstts-cpp PRIVATE /W4 /wd4100 /wd4505)
else()
target_compile_options(gomosstts-cpp PRIVATE -Wall -Wextra
-Wno-unused-parameter -Wno-unused-function)
endif()
set_property(TARGET gomosstts-cpp PROPERTY CXX_STANDARD 17)
set_target_properties(gomosstts-cpp PROPERTIES LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})

View File

@@ -0,0 +1,140 @@
CMAKE_ARGS?=
BUILD_TYPE?=
NATIVE?=false
GOCMD?=go
GO_TAGS?=
JOBS?=$(shell nproc --ignore=1)
# moss-tts.cpp version
MOSSTTS_REPO?=https://github.com/mudler/moss-tts.cpp
MOSSTTS_CPP_VERSION?=ee722b8e9205ee9b1b1c398a4e87e4e393e9be41
SO_TARGET?=libgomosstts-cpp.so
CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF
ifeq ($(NATIVE),false)
CMAKE_ARGS+=-DGGML_NATIVE=OFF
endif
ifeq ($(BUILD_TYPE),cublas)
CMAKE_ARGS+=-DGGML_CUDA=ON
else ifeq ($(BUILD_TYPE),openblas)
CMAKE_ARGS+=-DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS
else ifeq ($(BUILD_TYPE),clblas)
CMAKE_ARGS+=-DGGML_CLBLAST=ON -DCLBlast_DIR=/some/path
else ifeq ($(BUILD_TYPE),hipblas)
# This ggml only understands GGML_HIP (GGML_HIPBLAS was removed upstream),
# so passing GGML_HIPBLAS silently produced a CPU-only build (see #10666).
ROCM_HOME ?= /opt/rocm
ROCM_PATH ?= /opt/rocm
export CXX=$(ROCM_HOME)/llvm/bin/clang++
export CC=$(ROCM_HOME)/llvm/bin/clang
AMDGPU_TARGETS ?= gfx908,gfx90a,gfx942,gfx950,gfx1030,gfx1100,gfx1101,gfx1102,gfx1151,gfx1200,gfx1201
CMAKE_ARGS+=-DGGML_HIP=ON -DAMDGPU_TARGETS=$(AMDGPU_TARGETS)
else ifeq ($(BUILD_TYPE),vulkan)
CMAKE_ARGS+=-DGGML_VULKAN=ON
else ifeq ($(OS),Darwin)
ifneq ($(BUILD_TYPE),metal)
CMAKE_ARGS+=-DGGML_METAL=OFF
else
CMAKE_ARGS+=-DGGML_METAL=ON
CMAKE_ARGS+=-DGGML_METAL_EMBED_LIBRARY=ON
endif
endif
ifeq ($(BUILD_TYPE),sycl_f16)
CMAKE_ARGS+=-DGGML_SYCL=ON \
-DCMAKE_C_COMPILER=icx \
-DCMAKE_CXX_COMPILER=icpx \
-DGGML_SYCL_F16=ON
endif
ifeq ($(BUILD_TYPE),sycl_f32)
CMAKE_ARGS+=-DGGML_SYCL=ON \
-DCMAKE_C_COMPILER=icx \
-DCMAKE_CXX_COMPILER=icpx
endif
sources/moss-tts.cpp:
mkdir -p sources/moss-tts.cpp
cd sources/moss-tts.cpp && \
git init && \
git remote add origin $(MOSSTTS_REPO) && \
git fetch origin && \
git checkout $(MOSSTTS_CPP_VERSION) && \
git submodule update --init --recursive --depth 1 --single-branch
# Detect OS
UNAME_S := $(shell uname -s)
# Only build CPU variants on Linux
ifeq ($(UNAME_S),Linux)
VARIANT_TARGETS = libgomosstts-cpp-avx.so libgomosstts-cpp-avx2.so libgomosstts-cpp-avx512.so libgomosstts-cpp-fallback.so
else
# On non-Linux (e.g., Darwin), build only fallback variant (as a dylib)
VARIANT_TARGETS = libgomosstts-cpp-fallback.dylib
endif
moss-tts-cpp: main.go gomossttscpp.go $(VARIANT_TARGETS)
CGO_ENABLED=0 $(GOCMD) build -tags "$(GO_TAGS)" -o moss-tts-cpp ./
package: moss-tts-cpp
bash package.sh
build: package
clean: purge
rm -rf libgomosstts-cpp*.so libgomosstts-cpp*.dylib package sources/moss-tts.cpp moss-tts-cpp
purge:
rm -rf build*
# Variants must build sequentially
.NOTPARALLEL:
# Build all variants (Linux only)
ifeq ($(UNAME_S),Linux)
libgomosstts-cpp-avx.so: sources/moss-tts.cpp
$(info ${GREEN}I moss-tts-cpp build info:avx${RESET})
SO_TARGET=libgomosstts-cpp-avx.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=on -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_BMI2=off" $(MAKE) libgomosstts-cpp-custom
rm -rf build-libgomosstts-cpp-avx.so
libgomosstts-cpp-avx2.so: sources/moss-tts.cpp
$(info ${GREEN}I moss-tts-cpp build info:avx2${RESET})
SO_TARGET=libgomosstts-cpp-avx2.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=on -DGGML_AVX2=on -DGGML_AVX512=off -DGGML_FMA=on -DGGML_F16C=on -DGGML_BMI2=on" $(MAKE) libgomosstts-cpp-custom
rm -rf build-libgomosstts-cpp-avx2.so
libgomosstts-cpp-avx512.so: sources/moss-tts.cpp
$(info ${GREEN}I moss-tts-cpp build info:avx512${RESET})
SO_TARGET=libgomosstts-cpp-avx512.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=on -DGGML_AVX2=on -DGGML_AVX512=on -DGGML_FMA=on -DGGML_F16C=on -DGGML_BMI2=on" $(MAKE) libgomosstts-cpp-custom
rm -rf build-libgomosstts-cpp-avx512.so
endif
# Build fallback variant (all platforms)
libgomosstts-cpp-fallback.so: sources/moss-tts.cpp
$(info ${GREEN}I moss-tts-cpp build info:fallback${RESET})
SO_TARGET=libgomosstts-cpp-fallback.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=off -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_BMI2=off" $(MAKE) libgomosstts-cpp-custom
rm -rf build-libgomosstts-cpp-fallback.so
# Build fallback variant as a dylib (Darwin)
libgomosstts-cpp-fallback.dylib: sources/moss-tts.cpp
$(info ${GREEN}I moss-tts-cpp build info:fallback (dylib)${RESET})
SO_TARGET=libgomosstts-cpp-fallback.dylib CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=off -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_BMI2=off" $(MAKE) libgomosstts-cpp-custom
rm -rf build-libgomosstts-cpp-fallback.dylib
libgomosstts-cpp-custom: CMakeLists.txt cpp/mossttscpp.cpp cpp/mossttscpp.h
mkdir -p build-$(SO_TARGET) && \
cd build-$(SO_TARGET) && \
cmake .. $(CMAKE_ARGS) && \
cmake --build . --config Release -j$(JOBS) --target gomosstts-cpp && \
cd .. && \
(mv build-$(SO_TARGET)/libgomosstts-cpp.so ./$(SO_TARGET) 2>/dev/null || \
mv build-$(SO_TARGET)/libgomosstts-cpp.dylib ./$(SO_TARGET) 2>/dev/null)
test: moss-tts-cpp
@echo "Running moss-tts-cpp tests..."
bash test.sh
@echo "moss-tts-cpp tests completed."
all: moss-tts-cpp package

View File

@@ -0,0 +1,79 @@
# MOSS-TTS C++ backend
This backend runs the OpenMOSS **MOSS-TTS-Local (v1.5)** GGUF model through
[moss-tts.cpp](https://github.com/mudler/moss-tts.cpp), a from-scratch C++/ggml
port with no Python at inference time. It generates **48 kHz stereo** speech and
supports reference-audio voice cloning.
The engine loads three GGUFs: the local transformer (the model), the
MOSS-Audio-Tokenizer neural codec, and the text tokenizer. It is loaded via
purego (cgo-less `dlopen`) exactly like `qwen3-tts-cpp`.
## Model configuration
The model path points at the local transformer GGUF. The codec and text
tokenizer are auto-discovered as siblings of the model:
- codec: a `*.gguf` whose name contains `audio` + `tokenizer` (or `codec`),
e.g. `moss-audio-tokenizer-v2-f32.gguf`
- tokenizer: the other `*.gguf` whose name contains `tokenizer`,
e.g. `moss-tokenizer-v1_5.gguf`
```yaml
name: moss-tts-cpp
backend: moss-tts-cpp
parameters:
model: moss-tts-local-v1_5-q8_0.gguf
known_usecases:
- tts
tts:
audio_path: voices/default-reference.wav # optional model-wide clone reference
```
Override discovery when the filenames are non-standard:
```yaml
options:
- "codec:moss-audio-tokenizer-v2-f32.gguf"
- "tokenizer:moss-tokenizer-v1_5.gguf"
- "seed:42"
```
GGUFs for the Local v1.5 model (plus its codec and tokenizer) live at
[mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF](https://huggingface.co/mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF).
## Voice cloning
MOSS-TTS-Local has no named speakers; cloning is driven purely by a
reference-audio **path**. Request precedence is: a request `voice` that ends in
a known audio extension (`.wav`, `.flac`, `.mp3`, `.ogg`, `.m4a`), then
`tts.audio_path`. The engine decodes the reference itself, so no client-side
resampling is required.
## API example
```bash
curl http://localhost:8080/v1/audio/speech \
-H 'Content-Type: application/json' \
-d '{
"model": "moss-tts-cpp",
"input": "This request uses a saved reference voice.",
"voice": "/path/to/reference.wav"
}' \
--output speech.wav
```
## Native end-to-end test
The labeled test loads real GGUFs, synthesizes a 48 kHz stereo WAV, and streams
audio:
```bash
make -C backend/go/moss-tts-cpp moss-tts-cpp
MOSSTTS_MODEL=/path/to/moss-tts-local-v1_5-q8_0.gguf \
MOSSTTS_CODEC=/path/to/moss-audio-tokenizer-v2-f32.gguf \
MOSSTTS_TOKENIZER=/path/to/moss-tokenizer-v1_5.gguf \
MOSSTTS_LIBRARY=backend/go/moss-tts-cpp/libgomosstts-cpp-fallback.so \
go test ./backend/go/moss-tts-cpp -ginkgo.label-filter=e2e
```

View File

@@ -0,0 +1,100 @@
package main
import (
"bytes"
"encoding/binary"
"fmt"
"os"
"github.com/go-audio/audio"
"github.com/go-audio/wav"
)
// mossttsSampleRate is the MOSS-TTS-Local v1.5 output rate; the engine also
// reports it per-synthesis via out_sr, which the writers honour.
const mossttsSampleRate = 48000
// mossttsChannels is the MOSS-TTS-Local output layout: 48 kHz stereo, samples
// interleaved [L,R,L,R,...].
const mossttsChannels = 2
// wavHeaderStereo returns a 44-byte WAV header for a streaming 16-bit stereo PCM
// stream at sampleRate, with placeholder (0xFFFFFFFF) sizes since the total
// length is unknown up front. Emitted as the first chunk of TTSStream so the
// HTTP layer receives a self-describing WAV.
func wavHeaderStereo(sampleRate int) []byte {
if sampleRate <= 0 {
sampleRate = mossttsSampleRate
}
const blockAlign = mossttsChannels * 2 // 16-bit stereo
var buf bytes.Buffer
w := func(v any) { _ = binary.Write(&buf, binary.LittleEndian, v) }
buf.WriteString("RIFF")
w(uint32(0xFFFFFFFF))
buf.WriteString("WAVE")
buf.WriteString("fmt ")
w(uint32(16)) // Subchunk1Size
w(uint16(1)) // PCM
w(uint16(mossttsChannels)) // stereo
w(uint32(sampleRate)) // sample rate
w(uint32(sampleRate * blockAlign)) // byte rate = SR * blockAlign
w(uint16(blockAlign)) // block align
w(uint16(16)) // bits per sample
buf.WriteString("data")
w(uint32(0xFFFFFFFF))
return buf.Bytes()
}
// floatToPCM16LE clamps each sample to [-1,1] and encodes it as little-endian
// signed 16-bit PCM. Input may be interleaved stereo; the layout is preserved.
func floatToPCM16LE(samples []float32) []byte {
out := make([]byte, len(samples)*2)
for i, s := range samples {
if s > 1 {
s = 1
} else if s < -1 {
s = -1
}
v := int16(s * 32767)
out[i*2] = byte(v) // #nosec G115 -- intentional little-endian split of a clamped int16
out[i*2+1] = byte(v >> 8) // #nosec G115 -- high byte of the same clamped int16
}
return out
}
// writeWAVStereo writes interleaved [L,R,...] float samples as a finalized
// 16-bit stereo WAV at sampleRate (falling back to the v1.5 default of 48 kHz).
func writeWAVStereo(dst string, samples []float32, sampleRate int) error {
if sampleRate <= 0 {
sampleRate = mossttsSampleRate
}
f, err := os.Create(dst) // #nosec G304 -- dst is the server-chosen output path from the TTS request, not user-traversable
if err != nil {
return fmt.Errorf("moss-tts: create %q: %w", dst, err)
}
enc := wav.NewEncoder(f, sampleRate, 16, mossttsChannels, 1)
ints := make([]int, len(samples))
for i, s := range samples {
if s > 1 {
s = 1
} else if s < -1 {
s = -1
}
ints[i] = int(s * 32767)
}
b := &audio.IntBuffer{
Format: &audio.Format{NumChannels: mossttsChannels, SampleRate: sampleRate},
Data: ints,
SourceBitDepth: 16,
}
if err := enc.Write(b); err != nil {
_ = enc.Close()
_ = f.Close()
return fmt.Errorf("moss-tts: encode WAV: %w", err)
}
if err := enc.Close(); err != nil {
_ = f.Close()
return fmt.Errorf("moss-tts: finalize WAV: %w", err)
}
return f.Close()
}

View File

@@ -0,0 +1,49 @@
package main
import (
"path/filepath"
pb "github.com/mudler/LocalAI/pkg/grpc/proto"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// These specs pin the voice-selection logic in resolveRequest, in particular
// the config-level audio_path (tts.audio_path -> ModelOptions.AudioPath) being
// used as the default voice-cloning reference. MOSS-TTS-Local takes the
// reference as a WAV path (the engine decodes it), so resolveRequest is pure
// path logic and needs no model / C library.
var _ = Describe("resolveRequest voice/clone selection", func() {
var dir, refWav string
BeforeEach(func() {
dir = GinkgoT().TempDir()
refWav = filepath.Join(dir, "ref.wav")
})
It("uses the config audio_path as the clone reference when Voice is empty", func() {
m := &MossTtsCpp{audioPath: refWav}
ref, seed := m.resolveRequest(&pb.TTSRequest{Text: "hi"})
Expect(ref).To(Equal(refWav))
Expect(seed).To(Equal(0)) // zero-value loadOptions seed
})
It("lets a per-request audio Voice override audio_path", func() {
other := filepath.Join(dir, "other.wav")
m := &MossTtsCpp{audioPath: refWav}
ref, _ := m.resolveRequest(&pb.TTSRequest{Text: "hi", Voice: other})
Expect(ref).To(Equal(other))
})
It("does not clone for a bare-token Voice, falling back to audio_path", func() {
m := &MossTtsCpp{audioPath: refWav}
ref, _ := m.resolveRequest(&pb.TTSRequest{Text: "hi", Voice: "serena"})
Expect(ref).To(Equal(refWav))
})
It("reads a per-request seed override from params", func() {
m := &MossTtsCpp{opts: loadOptions{seed: -1}}
_, seed := m.resolveRequest(&pb.TTSRequest{Text: "hi", Params: map[string]string{"seed": "123"}})
Expect(seed).To(Equal(123))
})
})

View File

@@ -0,0 +1,119 @@
#include "mossttscpp.h"
#include "ggml-backend.h"
#include "moss_tts_capi.h"
#include <cstdio>
#include <cstdlib>
#include <cstring>
// Single loaded Local pipeline. base.SingleThread serializes calls on the Go
// side, so a plain global is safe (mirrors qwen3-tts-cpp's g_ctx).
static moss_local *g_local = nullptr;
static void ggml_log_cb(enum ggml_log_level level, const char *log,
void * /*data*/) {
if (!log)
return;
const char *lvl = "?????";
switch (level) {
case GGML_LOG_LEVEL_DEBUG: lvl = "DEBUG"; break;
case GGML_LOG_LEVEL_INFO: lvl = "INFO"; break;
case GGML_LOG_LEVEL_WARN: lvl = "WARN"; break;
case GGML_LOG_LEVEL_ERROR: lvl = "ERROR"; break;
default: break;
}
fprintf(stderr, "[%-5s] %s", lvl, log);
fflush(stderr);
}
int mtl_load(const char *local_path, const char *codec_path,
const char *tokenizer_path) {
ggml_log_set(ggml_log_cb, nullptr);
ggml_backend_load_all();
if (!local_path || local_path[0] == '\0') {
fprintf(stderr, "[moss-tts-cpp] ERROR: local_path is required\n");
return 1;
}
if (!codec_path || codec_path[0] == '\0') {
fprintf(stderr, "[moss-tts-cpp] ERROR: codec_path is required\n");
return 2;
}
if (!tokenizer_path || tokenizer_path[0] == '\0') {
fprintf(stderr, "[moss-tts-cpp] ERROR: tokenizer_path is required\n");
return 3;
}
fprintf(stderr,
"[moss-tts-cpp] Loading local=%s codec=%s tokenizer=%s\n",
local_path, codec_path, tokenizer_path);
g_local = moss_local_load(local_path, codec_path, tokenizer_path);
if (!g_local) {
fprintf(stderr, "[moss-tts-cpp] FATAL: moss_local_load failed\n");
return 4;
}
fprintf(stderr, "[moss-tts-cpp] Model loaded (%s)\n", moss_tts_version());
return 0;
}
float *mtl_tts(const char *text, const char *reference_wav, int seed,
int *out_n, int *out_sr) {
if (out_n)
*out_n = 0;
if (out_sr)
*out_sr = 0;
if (!g_local) {
fprintf(stderr, "[moss-tts-cpp] ERROR: model not loaded\n");
return nullptr;
}
if (!text || text[0] == '\0') {
fprintf(stderr, "[moss-tts-cpp] ERROR: text is required\n");
return nullptr;
}
// An empty reference path means "no cloning": pass NULL so the engine skips
// the clone branch rather than trying to open "".
const char *ref = (reference_wav && reference_wav[0] != '\0')
? reference_wav
: nullptr;
int n = 0, sr = 0;
float *pcm = moss_local_tts(g_local, text, ref, seed, &n, &sr);
if (!pcm || n <= 0) {
fprintf(stderr, "[moss-tts-cpp] ERROR: moss_local_tts failed\n");
if (pcm)
moss_free(pcm);
return nullptr;
}
// Copy into a plain malloc buffer the Go side frees via mtl_pcm_free, then
// release the engine-owned buffer with moss_free (mirrors qwen3-tts-cpp,
// keeping ownership on the C runtime's malloc/free).
size_t bytes = (size_t)n * sizeof(float);
float *buf = (float *)malloc(bytes);
if (!buf) {
fprintf(stderr, "[moss-tts-cpp] ERROR: malloc(%zu) failed\n", bytes);
moss_free(pcm);
return nullptr;
}
memcpy(buf, pcm, bytes);
moss_free(pcm);
if (out_n)
*out_n = n;
if (out_sr)
*out_sr = sr;
return buf;
}
void mtl_pcm_free(float *p) { free(p); }
void mtl_unload(void) {
if (g_local) {
moss_local_free(g_local);
g_local = nullptr;
}
}
const char *mtl_version(void) { return moss_tts_version(); }

View File

@@ -0,0 +1,36 @@
#pragma once
// Thin C shim over moss-tts.cpp's flat C-API (include/moss_tts_capi.h) for the
// MOSS-TTS-Local (v1.5) pipeline. It holds the loaded pipeline as a single
// global handle so the purego bindings on the Go side stay handle-free (mirrors
// the qwen3-tts-cpp qt3_* shim). Only the Local variant is wired here; the
// Delay / Realtime / Nano variants of the upstream API are intentionally
// unused.
extern "C" {
// Load the Local (MossTTSLocal v1.5) pipeline from its three GGUFs: the local
// transformer, the MOSS-Audio-Tokenizer codec, and the text tokenizer. Sets up
// ggml logging + backend registration first. Returns 0 on success, non-zero on
// failure.
int mtl_load(const char *local_path, const char *codec_path,
const char *tokenizer_path);
// Synthesize `text` to a malloc'd interleaved f32 PCM buffer (caller frees via
// mtl_pcm_free). v1.5 output is 48 kHz stereo: *out_n is the total number of
// float samples across both channels (frames * 2) and *out_sr is the sample
// rate. `reference_wav` is an optional path to a clone-reference WAV (may be
// NULL/empty for no cloning); the engine decodes it itself. `seed` < 0 means
// random. Returns NULL on failure (out_n / out_sr set to 0).
float *mtl_tts(const char *text, const char *reference_wav, int seed,
int *out_n, int *out_sr);
// Free a buffer returned by mtl_tts.
void mtl_pcm_free(float *p);
// Release the loaded pipeline.
void mtl_unload(void);
// Upstream engine version string (moss_tts_version()).
const char *mtl_version(void);
}

View File

@@ -0,0 +1,72 @@
package main
import (
"os"
"strings"
"github.com/ebitengine/purego"
pb "github.com/mudler/LocalAI/pkg/grpc/proto"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
func ttsReq(text, voice, dst string) *pb.TTSRequest {
return &pb.TTSRequest{Text: text, Voice: voice, Dst: dst}
}
var _ = Describe("moss-tts-cpp e2e", Label("e2e"), func() {
var loaded bool
BeforeEach(func() {
modelPath := os.Getenv("MOSSTTS_MODEL")
codecPath := os.Getenv("MOSSTTS_CODEC")
tokenizerPath := os.Getenv("MOSSTTS_TOKENIZER")
if modelPath == "" || codecPath == "" || tokenizerPath == "" {
Skip("MOSSTTS_MODEL / MOSSTTS_CODEC / MOSSTTS_TOKENIZER not set; skipping e2e")
}
if !loaded {
lib := os.Getenv("MOSSTTS_LIBRARY")
if lib == "" {
lib = "./libgomosstts-cpp-fallback.so"
}
h, err := purego.Dlopen(lib, purego.RTLD_NOW|purego.RTLD_GLOBAL)
Expect(err).ToNot(HaveOccurred())
purego.RegisterLibFunc(&CppLoad, h, "mtl_load")
purego.RegisterLibFunc(&CppTTS, h, "mtl_tts")
purego.RegisterLibFunc(&CppPCMFree, h, "mtl_pcm_free")
purego.RegisterLibFunc(&CppUnload, h, "mtl_unload")
Expect(CppLoad(modelPath, codecPath, tokenizerPath)).To(Equal(0))
loaded = true
}
})
It("synthesizes a stereo WAV file via TTS", func() {
m := &MossTtsCpp{opts: loadOptions{seed: 42}}
dst := GinkgoT().TempDir() + "/out.wav"
err := m.TTS(ttsReq("Hello world.", "", dst))
Expect(err).ToNot(HaveOccurred())
fi, err := os.Stat(dst)
Expect(err).ToNot(HaveOccurred())
Expect(fi.Size()).To(BeNumerically(">", int64(44)))
})
It("streams audio chunks via TTSStream", func() {
m := &MossTtsCpp{opts: loadOptions{seed: 42}}
results := make(chan []byte, 4096)
done := make(chan error, 1)
go func() { done <- m.TTSStream(ttsReq("Hello there, streaming test.", "", ""), results) }()
var chunks int
var first []byte
for c := range results {
if chunks == 0 {
first = c
}
chunks++
}
Expect(<-done).ToNot(HaveOccurred())
Expect(chunks).To(BeNumerically(">=", 2))
Expect(string(first[0:4])).To(Equal("RIFF"))
Expect(strings.HasPrefix(string(first[8:12]), "WAVE")).To(BeTrue())
})
})

View File

@@ -0,0 +1,224 @@
package main
import (
"fmt"
"os"
"path/filepath"
"strings"
"unsafe"
"github.com/mudler/LocalAI/pkg/grpc/base"
pb "github.com/mudler/LocalAI/pkg/grpc/proto"
"github.com/mudler/xlog"
)
var (
// mtl_load(local_path, codec_path, tokenizer_path) int
CppLoad func(localPath, codecPath, tokenizerPath string) int
// mtl_tts(text, reference_wav, seed, out_n, out_sr) -> float*
CppTTS func(text, referenceWav string, seed int, outN, outSR unsafe.Pointer) uintptr
// mtl_pcm_free(ptr)
CppPCMFree func(ptr uintptr)
// mtl_unload()
CppUnload func()
)
type MossTtsCpp struct {
base.SingleThread
opts loadOptions
// audioPath is the model-config reference voice (tts.audio_path), the
// default clone reference when a request omits an audio Voice.
audioPath string
}
func (m *MossTtsCpp) Load(opts *pb.ModelOptions) error {
model := opts.ModelFile
if model == "" {
model = opts.ModelPath
}
if !filepath.IsAbs(model) && opts.ModelPath != "" {
model = filepath.Join(opts.ModelPath, model)
}
m.opts = parseOptions(opts.Options)
dir := filepath.Dir(model)
// Resolve the codec GGUF (MOSS-Audio-Tokenizer): explicit option, else
// auto-discover an *audio*tokenizer*/codec sibling of the model.
codec := resolveAux(m.opts.codecPath, dir)
if codec == "" {
codec = discoverCodec(dir, model)
}
if codec == "" {
return fmt.Errorf("moss-tts: no codec GGUF found; set option 'codec:<file>'")
}
m.opts.codecPath = codec
// Resolve the text tokenizer GGUF: explicit option, else auto-discover the
// *tokenizer* sibling that is not the audio codec or the model.
tokenizer := resolveAux(m.opts.tokenizerPath, dir)
if tokenizer == "" {
tokenizer = discoverTokenizer(dir, model, codec)
}
if tokenizer == "" {
return fmt.Errorf("moss-tts: no tokenizer GGUF found; set option 'tokenizer:<file>'")
}
m.opts.tokenizerPath = tokenizer
m.audioPath = opts.AudioPath
if m.audioPath != "" && !filepath.IsAbs(m.audioPath) {
m.audioPath = filepath.Join(dir, m.audioPath)
}
xlog.Info("[moss-tts-cpp] Load", "model", model, "codec", codec, "tokenizer", tokenizer)
if rc := CppLoad(model, codec, tokenizer); rc != 0 {
return fmt.Errorf("moss-tts: failed to load model (rc=%d)", rc)
}
return nil
}
// resolveAux resolves an explicitly configured auxiliary GGUF path relative to
// the model directory when it is not absolute. Empty stays empty.
func resolveAux(p, dir string) string {
if p == "" || filepath.IsAbs(p) {
return p
}
return filepath.Join(dir, p)
}
// isAudioCodecName reports whether a GGUF filename denotes the MOSS-Audio-
// Tokenizer codec (e.g. moss-audio-tokenizer-v2-f32.gguf) rather than the text
// tokenizer (moss-tokenizer-v1_5.gguf).
func isAudioCodecName(name string) bool {
n := strings.ToLower(name)
return strings.Contains(n, "codec") ||
(strings.Contains(n, "audio") && strings.Contains(n, "tokenizer"))
}
// discoverCodec returns the first codec GGUF in dir (excluding the model), or "".
func discoverCodec(dir, model string) string {
entries, err := os.ReadDir(dir)
if err != nil {
return ""
}
modelBase := filepath.Base(model)
for _, e := range entries {
name := e.Name()
if name == modelBase || !strings.HasSuffix(strings.ToLower(name), ".gguf") {
continue
}
if isAudioCodecName(name) {
return filepath.Join(dir, name)
}
}
return ""
}
// discoverTokenizer returns the first text-tokenizer GGUF in dir that is neither
// the model nor the audio codec, or "".
func discoverTokenizer(dir, model, codec string) string {
entries, err := os.ReadDir(dir)
if err != nil {
return ""
}
modelBase := filepath.Base(model)
codecBase := filepath.Base(codec)
for _, e := range entries {
name := e.Name()
lower := strings.ToLower(name)
if name == modelBase || name == codecBase || !strings.HasSuffix(lower, ".gguf") {
continue
}
if strings.Contains(lower, "tokenizer") && !isAudioCodecName(name) {
return filepath.Join(dir, name)
}
}
return ""
}
// resolveRequest derives the synthesis inputs from a TTSRequest: the optional
// clone-reference WAV path and the seed. MOSS-TTS-Local drives cloning purely
// from a reference-audio path (the engine decodes it), so there is no
// language / speaker / instruct plumbing here (unlike qwen3-tts-cpp).
func (m *MossTtsCpp) resolveRequest(req *pb.TTSRequest) (refPath string, seed int) {
refPath = resolveVoice(req.Voice)
if refPath == "" && m.audioPath != "" {
// No per-request voice: fall back to the config clone reference.
refPath = m.audioPath
}
if refPath != "" && !filepath.IsAbs(refPath) {
refPath = filepath.Join(filepath.Dir(m.opts.codecPath), refPath)
}
seed = m.opts.seed
if req.Params != nil {
seed = parseInt(req.Params["seed"], seed)
}
return
}
func (m *MossTtsCpp) TTS(req *pb.TTSRequest) error {
if req.Dst == "" {
return fmt.Errorf("moss-tts: TTS requires a destination path")
}
if req.Text == "" {
return fmt.Errorf("moss-tts: TTS requires text")
}
refPath, seed := m.resolveRequest(req)
var n, sr int32
ptr := CppTTS(req.Text, refPath, seed, unsafe.Pointer(&n), unsafe.Pointer(&sr)) // #nosec G103 -- out-params for the purego-bound C-API
if ptr == 0 {
return fmt.Errorf("moss-tts: synthesis failed")
}
// Register the free as soon as we own a non-null buffer, so the n<=0 guard
// below cannot leak it (defensive: the C contract returns NULL on failure).
defer CppPCMFree(ptr)
if n <= 0 {
return fmt.Errorf("moss-tts: synthesis produced no samples")
}
//nolint:govet // C-allocated PCM, copied out before free
src := unsafe.Slice((*float32)(unsafe.Pointer(ptr)), int(n)) // #nosec G103 -- C-allocated PCM, copied out before free
out := make([]float32, int(n))
copy(out, src)
return writeWAVStereo(req.Dst, out, int(sr))
}
// TTSStream synthesizes one-shot (MOSS-TTS-Local has no native streaming C-API)
// and then emits a self-describing stereo WAV: a header chunk followed by the
// interleaved PCM in fixed-size slices, so the HTTP layer still receives a
// streamed WAV (the gRPC TTSStream path never sets Message, so the backend owns
// the header - see core/backend/tts.go:ModelTTSStream).
func (m *MossTtsCpp) TTSStream(req *pb.TTSRequest, results chan []byte) error {
defer close(results)
if req.Text == "" {
return fmt.Errorf("moss-tts: TTSStream requires text")
}
refPath, seed := m.resolveRequest(req)
var n, sr int32
ptr := CppTTS(req.Text, refPath, seed, unsafe.Pointer(&n), unsafe.Pointer(&sr)) // #nosec G103 -- out-params for the purego-bound C-API
if ptr == 0 {
return fmt.Errorf("moss-tts: synthesis failed")
}
defer CppPCMFree(ptr)
if n <= 0 {
return fmt.Errorf("moss-tts: synthesis produced no samples")
}
//nolint:govet // C-allocated PCM, copied out before free
src := unsafe.Slice((*float32)(unsafe.Pointer(ptr)), int(n)) // #nosec G103 -- C-allocated PCM, copied out before free
out := make([]float32, int(n))
copy(out, src)
results <- wavHeaderStereo(int(sr))
const frameChunk = 4096 // interleaved stereo samples per emitted chunk
for off := 0; off < len(out); off += frameChunk {
end := off + frameChunk
if end > len(out) {
end = len(out)
}
results <- floatToPCM16LE(out[off:end])
}
return nil
}

View File

@@ -0,0 +1,52 @@
package main
// Note: this is started internally by LocalAI and a server is allocated for each model
import (
"flag"
"os"
"runtime"
"github.com/ebitengine/purego"
grpc "github.com/mudler/LocalAI/pkg/grpc"
)
var (
addr = flag.String("addr", "localhost:50051", "the address to connect to")
)
type LibFuncs struct {
FuncPtr any
Name string
}
func main() {
libName := os.Getenv("MOSSTTS_LIBRARY")
if libName == "" {
if runtime.GOOS == "darwin" {
libName = "./libgomosstts-cpp-fallback.dylib"
} else {
libName = "./libgomosstts-cpp-fallback.so"
}
}
lib, err := purego.Dlopen(libName, purego.RTLD_NOW|purego.RTLD_GLOBAL)
if err != nil {
panic(err)
}
libFuncs := []LibFuncs{
{&CppLoad, "mtl_load"},
{&CppTTS, "mtl_tts"},
{&CppPCMFree, "mtl_pcm_free"},
{&CppUnload, "mtl_unload"},
}
for _, lf := range libFuncs {
purego.RegisterLibFunc(lf.FuncPtr, lib, lf.Name)
}
flag.Parse()
if err := grpc.StartServer(*addr, &MossTtsCpp{}); err != nil {
panic(err)
}
}

View File

@@ -0,0 +1,124 @@
package main
import (
"bytes"
"encoding/binary"
"os"
"path/filepath"
"testing"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
func TestMossTtsCpp(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "moss-tts-cpp suite")
}
var _ = Describe("resolveVoice", func() {
It("treats an audio path as a clone reference (case-insensitive ext)", func() {
Expect(resolveVoice("/x/ref.WAV")).To(Equal("/x/ref.WAV"))
})
It("recognizes mp3/flac/ogg/m4a", func() {
for _, p := range []string{"a.mp3", "b.flac", "c.ogg", "d.m4a"} {
Expect(resolveVoice(p)).To(Equal(p))
}
})
It("ignores a bare token (no named speakers in MOSS-TTS-Local)", func() {
Expect(resolveVoice("serena")).To(BeEmpty())
})
It("returns empty for empty input", func() {
Expect(resolveVoice(" ")).To(BeEmpty())
})
})
var _ = Describe("parseOptions", func() {
It("extracts codec, tokenizer, seed", func() {
o := parseOptions([]string{
"codec:codec.gguf", "tokenizer:tok.gguf", "seed:7", "unknown:ignored",
})
Expect(o.codecPath).To(Equal("codec.gguf"))
Expect(o.tokenizerPath).To(Equal("tok.gguf"))
Expect(o.seed).To(Equal(7))
})
It("accepts audio_tokenizer / text_tokenizer aliases", func() {
o := parseOptions([]string{"audio_tokenizer:c.gguf", "text_tokenizer:t.gguf"})
Expect(o.codecPath).To(Equal("c.gguf"))
Expect(o.tokenizerPath).To(Equal("t.gguf"))
})
It("defaults seed -1", func() {
Expect(parseOptions(nil).seed).To(Equal(-1))
})
})
var _ = Describe("isAudioCodecName", func() {
DescribeTable("distinguishes the audio codec from the text tokenizer",
func(name string, want bool) {
Expect(isAudioCodecName(name)).To(Equal(want))
},
Entry("audio tokenizer is the codec", "moss-audio-tokenizer-v2-f32.gguf", true),
Entry("codec keyword", "some-codec.gguf", true),
Entry("text tokenizer is not the codec", "moss-tokenizer-v1_5.gguf", false),
Entry("model is not the codec", "moss-tts-local-v1_5-q8_0.gguf", false),
)
})
var _ = Describe("discoverCodec / discoverTokenizer", func() {
var dir, model, codec, tok string
BeforeEach(func() {
dir = GinkgoT().TempDir()
model = filepath.Join(dir, "moss-tts-local-v1_5-q8_0.gguf")
codec = filepath.Join(dir, "moss-audio-tokenizer-v2-f32.gguf")
tok = filepath.Join(dir, "moss-tokenizer-v1_5.gguf")
for _, p := range []string{model, codec, tok} {
Expect(os.WriteFile(p, []byte("x"), 0o644)).To(Succeed())
}
})
It("finds the audio codec, excluding the model", func() {
Expect(discoverCodec(dir, model)).To(Equal(codec))
})
It("finds the text tokenizer, excluding model and codec", func() {
Expect(discoverTokenizer(dir, model, codec)).To(Equal(tok))
})
})
var _ = Describe("wavHeaderStereo", func() {
It("emits a 44-byte streaming WAV header at 48 kHz stereo 16-bit", func() {
h := wavHeaderStereo(48000)
Expect(h).To(HaveLen(44))
Expect(string(h[0:4])).To(Equal("RIFF"))
Expect(string(h[8:12])).To(Equal("WAVE"))
Expect(string(h[12:16])).To(Equal("fmt "))
Expect(string(h[36:40])).To(Equal("data"))
var channels uint16
Expect(binary.Read(bytes.NewReader(h[22:24]), binary.LittleEndian, &channels)).To(Succeed())
Expect(channels).To(Equal(uint16(2)))
var sampleRate uint32
Expect(binary.Read(bytes.NewReader(h[24:28]), binary.LittleEndian, &sampleRate)).To(Succeed())
Expect(sampleRate).To(Equal(uint32(48000)))
})
It("falls back to 48 kHz for a non-positive sample rate", func() {
var sampleRate uint32
h := wavHeaderStereo(0)
Expect(binary.Read(bytes.NewReader(h[24:28]), binary.LittleEndian, &sampleRate)).To(Succeed())
Expect(sampleRate).To(Equal(uint32(48000)))
})
})
var _ = Describe("floatToPCM16LE", func() {
It("clamps and converts float PCM to little-endian int16 bytes", func() {
b := floatToPCM16LE([]float32{0, 1.0, -1.0, 2.0, -2.0})
Expect(b).To(HaveLen(10))
read := func(off int) int16 {
var v int16
_ = binary.Read(bytes.NewReader(b[off:off+2]), binary.LittleEndian, &v)
return v
}
Expect(read(0)).To(Equal(int16(0)))
Expect(read(2)).To(Equal(int16(32767)))
Expect(read(4)).To(Equal(int16(-32767)))
Expect(read(6)).To(Equal(int16(32767))) // clamped from 2.0
Expect(read(8)).To(Equal(int16(-32767))) // clamped from -2.0
})
})

View File

@@ -0,0 +1,78 @@
package main
import (
"strconv"
"strings"
)
// loadOptions holds the parsed model-level options. MOSS-TTS-Local needs three
// GGUFs (local transformer, audio codec, text tokenizer); the model path is the
// local transformer and codec/tokenizer are resolved as siblings or via these
// options.
type loadOptions struct {
codecPath string
tokenizerPath string
seed int
}
func splitOption(o string) (key, value string, ok bool) {
i := strings.Index(o, ":")
if i < 0 {
return "", "", false
}
return strings.TrimSpace(o[:i]), strings.TrimSpace(o[i+1:]), true
}
// parseOptions reads the backend "key:value" option slice. Unknown keys are
// ignored. Default seed is -1 (engine random).
func parseOptions(opts []string) loadOptions {
o := loadOptions{seed: -1}
for _, oo := range opts {
key, value, ok := splitOption(oo)
if !ok {
continue
}
switch key {
case "codec", "audio_tokenizer":
o.codecPath = value
case "tokenizer", "text_tokenizer":
o.tokenizerPath = value
case "seed":
if n, err := strconv.Atoi(value); err == nil {
o.seed = n
}
}
}
return o
}
var refAudioExts = []string{".wav", ".flac", ".mp3", ".ogg", ".m4a"}
// resolveVoice interprets the request Voice field. MOSS-TTS-Local has no named
// speakers, only reference-audio cloning, so a value ending in a known audio
// extension is treated as a clone-reference path and anything else is ignored
// (the caller then falls back to the config audio_path).
func resolveVoice(voice string) (refPath string) {
v := strings.TrimSpace(voice)
if v == "" {
return ""
}
lower := strings.ToLower(v)
for _, ext := range refAudioExts {
if strings.HasSuffix(lower, ext) {
return v
}
}
return ""
}
func parseInt(v string, def int) int {
if v == "" {
return def
}
n, err := strconv.Atoi(v)
if err != nil {
return def
}
return n
}

View File

@@ -0,0 +1,65 @@
#!/bin/bash
# Script to copy the appropriate libraries based on architecture
# This script is used in the final stage of the Dockerfile
set -e
CURDIR=$(dirname "$(realpath $0)")
REPO_ROOT="${CURDIR}/../../.."
# Create lib directory
mkdir -p $CURDIR/package/lib
cp -avf $CURDIR/moss-tts-cpp $CURDIR/package/
cp -fv $CURDIR/libgomosstts-cpp-*.so $CURDIR/package/ 2>/dev/null || true
cp -fv $CURDIR/libgomosstts-cpp-*.dylib $CURDIR/package/ 2>/dev/null || true
cp -fv $CURDIR/run.sh $CURDIR/package/
# Detect architecture and copy appropriate libraries
if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
# x86_64 architecture
echo "Detected x86_64 architecture, copying x86_64 libraries..."
cp -arfLv /lib64/ld-linux-x86-64.so.2 $CURDIR/package/lib/ld.so
cp -arfLv /lib/x86_64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
cp -arfLv /lib/x86_64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
cp -arfLv /lib/x86_64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
cp -arfLv /lib/x86_64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
cp -arfLv /lib/x86_64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
cp -arfLv /lib/x86_64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
cp -arfLv /lib/x86_64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
cp -arfLv /lib/x86_64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
cp -arfLv /lib/x86_64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
cp -arfLv /lib/x86_64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
# ARM64 architecture
echo "Detected ARM64 architecture, copying ARM64 libraries..."
cp -arfLv /lib/ld-linux-aarch64.so.1 $CURDIR/package/lib/ld.so
cp -arfLv /lib/aarch64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
cp -arfLv /lib/aarch64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
cp -arfLv /lib/aarch64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
cp -arfLv /lib/aarch64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
cp -arfLv /lib/aarch64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
cp -arfLv /lib/aarch64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
cp -arfLv /lib/aarch64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
cp -arfLv /lib/aarch64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
cp -arfLv /lib/aarch64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
cp -arfLv /lib/aarch64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
elif [ $(uname -s) = "Darwin" ]; then
echo "Detected Darwin"
else
echo "Error: Could not detect architecture"
exit 1
fi
# Package GPU libraries based on BUILD_TYPE
GPU_LIB_SCRIPT="${REPO_ROOT}/scripts/build/package-gpu-libs.sh"
if [ -f "$GPU_LIB_SCRIPT" ]; then
echo "Packaging GPU libraries for BUILD_TYPE=${BUILD_TYPE:-cpu}..."
source "$GPU_LIB_SCRIPT" "$CURDIR/package/lib"
package_gpu_libs
fi
echo "Packaging completed successfully"
ls -liah $CURDIR/package/
ls -liah $CURDIR/package/lib/

View File

@@ -0,0 +1,57 @@
#!/bin/bash
set -ex
# Get the absolute current dir where the script is located
CURDIR=$(dirname "$(realpath "$0")")
cd /
echo "CPU info:"
if [ "$(uname)" != "Darwin" ]; then
grep -e "model\sname" /proc/cpuinfo | head -1
grep -e "flags" /proc/cpuinfo | head -1
fi
if [ "$(uname)" = "Darwin" ]; then
# macOS: single dylib variant (Metal or Accelerate)
LIBRARY="$CURDIR/libgomosstts-cpp-fallback.dylib"
export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
else
LIBRARY="$CURDIR/libgomosstts-cpp-fallback.so"
if grep -q -e "\savx\s" /proc/cpuinfo ; then
echo "CPU: AVX found OK"
if [ -e "$CURDIR"/libgomosstts-cpp-avx.so ]; then
LIBRARY="$CURDIR/libgomosstts-cpp-avx.so"
fi
fi
if grep -q -e "\savx2\s" /proc/cpuinfo ; then
echo "CPU: AVX2 found OK"
if [ -e "$CURDIR"/libgomosstts-cpp-avx2.so ]; then
LIBRARY="$CURDIR/libgomosstts-cpp-avx2.so"
fi
fi
# Check avx 512
if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
echo "CPU: AVX512F found OK"
if [ -e "$CURDIR"/libgomosstts-cpp-avx512.so ]; then
LIBRARY="$CURDIR/libgomosstts-cpp-avx512.so"
fi
fi
export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
fi
export MOSSTTS_LIBRARY=$LIBRARY
# If there is a lib/ld.so, use it
if [ -f "$CURDIR"/lib/ld.so ]; then
echo "Using lib/ld.so"
echo "Using library: $LIBRARY"
exec "$CURDIR"/lib/ld.so "$CURDIR"/moss-tts-cpp "$@"
fi
echo "Using library: $LIBRARY"
exec "$CURDIR"/moss-tts-cpp "$@"

View File

@@ -0,0 +1,33 @@
#!/bin/bash
set -e
CURDIR=$(dirname "$(realpath $0)")
cd "$CURDIR"
echo "Running moss-tts-cpp backend tests..."
# Auto-download the Local v1.5 model + codec + tokenizer only when
# MOSSTTS_MODEL is not set.
if [ -z "$MOSSTTS_MODEL" ]; then
MODEL_DIR="./moss-tts-models"
mkdir -p "$MODEL_DIR"
REPO_ID="mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF"
BASE_URL="https://huggingface.co/${REPO_ID}/resolve/main"
FILES=( "moss-tts-local-v1_5-q8_0.gguf" "moss-audio-tokenizer-v2-f32.gguf" "moss-tokenizer-v1_5.gguf" )
for file in "${FILES[@]}"; do
dest="${MODEL_DIR}/${file}"
if [ -f "${dest}" ]; then
echo " [skip] ${file}"
else
echo " [download] ${file}..."
curl -L -o "${dest}" "${BASE_URL}/${file}" --progress-bar
fi
done
export MOSSTTS_MODEL="${MODEL_DIR}/moss-tts-local-v1_5-q8_0.gguf"
export MOSSTTS_CODEC="${MODEL_DIR}/moss-audio-tokenizer-v2-f32.gguf"
export MOSSTTS_TOKENIZER="${MODEL_DIR}/moss-tokenizer-v1_5.gguf"
fi
go test -v -timeout 1200s .
echo "All moss-tts-cpp tests passed."

View File

@@ -1129,6 +1129,48 @@
nvidia-l4t: "nvidia-l4t-arm64-qwen3-tts-cpp"
nvidia-l4t-cuda-12: "nvidia-l4t-arm64-qwen3-tts-cpp"
nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-qwen3-tts-cpp"
- &mossttscpp
name: "moss-tts-cpp"
description: |
MOSS-TTS C++ backend using GGML (moss-tts.cpp). Native C++ text-to-speech
for the OpenMOSS MOSS-TTS-Local Transformer v1.5 model (a GPT-J local
transformer + MOSS-Audio-Tokenizer-v2 neural codec), running from a
self-contained GGUF with no Python at inference time. 48kHz stereo output
with reference-audio voice cloning.
urls:
- https://github.com/mudler/moss-tts.cpp
- https://huggingface.co/mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF
tags:
- text-to-speech
- tts
- voice-cloning
alias: "moss-tts-cpp"
capabilities:
default: "cpu-moss-tts-cpp"
nvidia: "cuda12-moss-tts-cpp"
nvidia-cuda-13: "cuda13-moss-tts-cpp"
nvidia-cuda-12: "cuda12-moss-tts-cpp"
intel: "intel-sycl-f16-moss-tts-cpp"
metal: "metal-moss-tts-cpp"
amd: "rocm-moss-tts-cpp"
vulkan: "vulkan-moss-tts-cpp"
nvidia-l4t: "nvidia-l4t-arm64-moss-tts-cpp"
nvidia-l4t-cuda-12: "nvidia-l4t-arm64-moss-tts-cpp"
nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-moss-tts-cpp"
- !!merge <<: *mossttscpp
name: "moss-tts-cpp-development"
capabilities:
default: "cpu-moss-tts-cpp-development"
nvidia: "cuda12-moss-tts-cpp-development"
nvidia-cuda-13: "cuda13-moss-tts-cpp-development"
nvidia-cuda-12: "cuda12-moss-tts-cpp-development"
intel: "intel-sycl-f16-moss-tts-cpp-development"
metal: "metal-moss-tts-cpp-development"
amd: "rocm-moss-tts-cpp-development"
vulkan: "vulkan-moss-tts-cpp-development"
nvidia-l4t: "nvidia-l4t-arm64-moss-tts-cpp-development"
nvidia-l4t-cuda-12: "nvidia-l4t-arm64-moss-tts-cpp-development"
nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-moss-tts-cpp-development"
- &omnivoicecpp
name: "omnivoice-cpp"
description: |
@@ -4611,6 +4653,107 @@
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-nvidia-cuda-13-qwen3-tts-cpp"
mirrors:
- localai/localai-backends:master-gpu-nvidia-cuda-13-qwen3-tts-cpp
## moss-tts-cpp
- !!merge <<: *mossttscpp
name: "nvidia-l4t-arm64-moss-tts-cpp"
uri: "quay.io/go-skynet/local-ai-backends:latest-nvidia-l4t-arm64-moss-tts-cpp"
mirrors:
- localai/localai-backends:latest-nvidia-l4t-arm64-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "nvidia-l4t-arm64-moss-tts-cpp-development"
uri: "quay.io/go-skynet/local-ai-backends:master-nvidia-l4t-arm64-moss-tts-cpp"
mirrors:
- localai/localai-backends:master-nvidia-l4t-arm64-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "cuda13-nvidia-l4t-arm64-moss-tts-cpp"
uri: "quay.io/go-skynet/local-ai-backends:latest-nvidia-l4t-cuda-13-arm64-moss-tts-cpp"
mirrors:
- localai/localai-backends:latest-nvidia-l4t-cuda-13-arm64-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "cuda13-nvidia-l4t-arm64-moss-tts-cpp-development"
uri: "quay.io/go-skynet/local-ai-backends:master-nvidia-l4t-cuda-13-arm64-moss-tts-cpp"
mirrors:
- localai/localai-backends:master-nvidia-l4t-cuda-13-arm64-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "cpu-moss-tts-cpp"
uri: "quay.io/go-skynet/local-ai-backends:latest-cpu-moss-tts-cpp"
mirrors:
- localai/localai-backends:latest-cpu-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "metal-moss-tts-cpp"
uri: "quay.io/go-skynet/local-ai-backends:latest-metal-darwin-arm64-moss-tts-cpp"
mirrors:
- localai/localai-backends:latest-metal-darwin-arm64-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "metal-moss-tts-cpp-development"
uri: "quay.io/go-skynet/local-ai-backends:master-metal-darwin-arm64-moss-tts-cpp"
mirrors:
- localai/localai-backends:master-metal-darwin-arm64-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "cpu-moss-tts-cpp-development"
uri: "quay.io/go-skynet/local-ai-backends:master-cpu-moss-tts-cpp"
mirrors:
- localai/localai-backends:master-cpu-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "cuda12-moss-tts-cpp"
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-moss-tts-cpp"
mirrors:
- localai/localai-backends:latest-gpu-nvidia-cuda-12-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "rocm-moss-tts-cpp"
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-rocm-hipblas-moss-tts-cpp"
mirrors:
- localai/localai-backends:latest-gpu-rocm-hipblas-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "intel-sycl-f32-moss-tts-cpp"
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-intel-sycl-f32-moss-tts-cpp"
mirrors:
- localai/localai-backends:latest-gpu-intel-sycl-f32-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "intel-sycl-f16-moss-tts-cpp"
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-intel-sycl-f16-moss-tts-cpp"
mirrors:
- localai/localai-backends:latest-gpu-intel-sycl-f16-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "vulkan-moss-tts-cpp"
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-vulkan-moss-tts-cpp"
mirrors:
- localai/localai-backends:latest-gpu-vulkan-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "vulkan-moss-tts-cpp-development"
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-vulkan-moss-tts-cpp"
mirrors:
- localai/localai-backends:master-gpu-vulkan-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "cuda12-moss-tts-cpp-development"
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-nvidia-cuda-12-moss-tts-cpp"
mirrors:
- localai/localai-backends:master-gpu-nvidia-cuda-12-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "rocm-moss-tts-cpp-development"
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-rocm-hipblas-moss-tts-cpp"
mirrors:
- localai/localai-backends:master-gpu-rocm-hipblas-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "intel-sycl-f32-moss-tts-cpp-development"
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-intel-sycl-f32-moss-tts-cpp"
mirrors:
- localai/localai-backends:master-gpu-intel-sycl-f32-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "intel-sycl-f16-moss-tts-cpp-development"
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-intel-sycl-f16-moss-tts-cpp"
mirrors:
- localai/localai-backends:master-gpu-intel-sycl-f16-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "cuda13-moss-tts-cpp"
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-13-moss-tts-cpp"
mirrors:
- localai/localai-backends:latest-gpu-nvidia-cuda-13-moss-tts-cpp
- !!merge <<: *mossttscpp
name: "cuda13-moss-tts-cpp-development"
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-nvidia-cuda-13-moss-tts-cpp"
mirrors:
- localai/localai-backends:master-gpu-nvidia-cuda-13-moss-tts-cpp
## omnivoice-cpp
- !!merge <<: *omnivoicecpp
name: "omnivoice-cpp-development"

View File

@@ -127,7 +127,7 @@ When a saved profile is selected, LocalAI supplies both its private WAV and exac
| Backend | Automatically compatible variants |
| --- | --- |
| `chatterbox`, `faster-qwen3-tts`, `fish-speech`, `neutts`, `omnivoice-cpp`, `pocket-tts`, `voxcpm` | Reference-audio cloning models served by these dedicated backends. |
| `chatterbox`, `faster-qwen3-tts`, `fish-speech`, `moss-tts-cpp`, `neutts`, `omnivoice-cpp`, `pocket-tts`, `voxcpm` | Reference-audio cloning models served by these dedicated backends. |
| `qwen-tts`, `qwen3-tts-cpp`, `vllm-omni` | Base or VoiceClone variants. CustomVoice and VoiceDesign variants are not raw reference-audio models. |
| `vibevoice-cpp` | 1.5B reference-WAV variants. The realtime 0.5B preset-prompt model is excluded. |
| `coqui` | XTTS and YourTTS variants. |

View File

@@ -59,6 +59,7 @@ All backends listed here can be installed on demand from the [Backend Gallery]({
| [Chatterbox](https://github.com/resemble-ai/chatterbox) | Production-grade TTS with emotion control | CPU, CUDA 12/13, Metal, Jetson L4T |
| [VibeVoice](https://github.com/microsoft/VibeVoice) | Real-time TTS with voice cloning | CPU, CUDA 12/13, ROCm, Intel SYCL, Metal, Jetson L4T |
| [vibevoice.cpp](https://github.com/mudler/vibevoice.cpp) | Native C++/GGML port of VibeVoice for TTS (voice cloning) and long-form ASR with diarization | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T |
| [moss-tts.cpp](https://github.com/mudler/moss-tts.cpp) | Native C++/GGML port of OpenMOSS MOSS-TTS-Local v1.5: 48 kHz stereo TTS with reference-audio voice cloning | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T |
| [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) | TTS with custom voice, voice design, and voice cloning | CPU, CUDA 12/13, ROCm, Intel SYCL, Metal, Jetson L4T |
| [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp) | Native C++/GGML Qwen3-TTS with streaming, named speakers, and voice design | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T |
| [OmniVoice](https://github.com/ServeurpersoCom/omnivoice.cpp) | Native C++/GGML TTS with voice cloning, voice design, and streaming | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T |

View File

@@ -41,6 +41,14 @@ function inferBackendPath(item) {
if (item.backend === "moss-transcribe-cpp") {
return `backend/go/moss-transcribe-cpp/`;
}
// moss-tts-cpp is a Go backend (Dockerfile.golang) wrapping the moss-tts.cpp
// ggml port via purego, living in backend/go/moss-tts-cpp/. Same
// explicit-branch rationale as parakeet-cpp / ced / moss-transcribe-cpp: the
// generic golang fallthrough would also resolve it, but this documents the
// mapping and guards a future dockerfile-suffix change.
if (item.backend === "moss-tts-cpp") {
return `backend/go/moss-tts-cpp/`;
}
if (item.dockerfile.endsWith("golang")) {
return `backend/go/${item.backend}/`;
}