mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 09:57:57 -04:00
feat(backend): add moss-tts-cpp text-to-speech backend (#10860)
* feat(backend): add moss-tts-cpp text-to-speech backend Add a Go + purego backend wrapping the moss-tts.cpp ggml port of the OpenMOSS MOSS-TTS-Local v1.5 text-to-speech model (GPT-J local transformer decoded through MOSS-Audio-Tokenizer-v2), producing 48 kHz stereo audio with optional reference-audio voice cloning. Mirrors the qwen3-tts-cpp backend: dlopen the static-ggml shared library, bind the moss-tts.cpp C-API via purego, and serve the gRPC TTS method. A thin C shim holds the pipeline handle and copies engine PCM into a Go-freeable buffer. Wires the CI registration: backend-matrix.yml (CPU, CUDA 12/13, Intel SYCL f16/f32, Vulkan, ROCm, NVIDIA L4T, plus Darwin metal), backend/index.yaml metas and image entries pointing at mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF, the root Makefile build targets, and the changed-backends.js path mapping. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: list the moss-tts-cpp backend among the LocalAI-maintained engines Add moss-tts.cpp to the README "Backends built by us" table, the Text-to-Speech compatibility table, and the reference-audio voice-cloning backend list, so the new backend is documented alongside its peers. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * backend(moss-tts-cpp): pin moss-tts.cpp to the squashed single-commit release moss-tts.cpp history was collapsed to a single commit; repoint MOSSTTS_CPP_VERSION to ee722b8e9205ee9b1b1c398a4e87e4e393e9be41. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * backend(moss-tts-cpp): add the moss-tts-cpp-development gallery meta The gallery had the -development image entries but no matching -development meta anchor (as locate-anything-cpp and depth-anything-cpp have), so the master build was not installable as a gallery backend. Add moss-tts-cpp-development mirroring the production meta with the -development capability image names. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
152
.github/backend-matrix.yml
vendored
152
.github/backend-matrix.yml
vendored
@@ -858,6 +858,19 @@ include:
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'cublas'
|
||||
cuda-major-version: "12"
|
||||
cuda-minor-version: "8"
|
||||
platforms: 'linux/amd64'
|
||||
tag-latest: 'auto'
|
||||
tag-suffix: '-gpu-nvidia-cuda-12-moss-tts-cpp'
|
||||
runs-on: 'ubuntu-latest'
|
||||
base-image: "ubuntu:24.04"
|
||||
skip-drivers: 'false'
|
||||
backend: "moss-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'cublas'
|
||||
cuda-major-version: "12"
|
||||
cuda-minor-version: "8"
|
||||
@@ -1909,6 +1922,19 @@ include:
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'cublas'
|
||||
cuda-major-version: "13"
|
||||
cuda-minor-version: "0"
|
||||
platforms: 'linux/amd64'
|
||||
tag-latest: 'auto'
|
||||
tag-suffix: '-gpu-nvidia-cuda-13-moss-tts-cpp'
|
||||
runs-on: 'ubuntu-latest'
|
||||
base-image: "ubuntu:24.04"
|
||||
skip-drivers: 'false'
|
||||
backend: "moss-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'cublas'
|
||||
cuda-major-version: "13"
|
||||
cuda-minor-version: "0"
|
||||
@@ -1961,6 +1987,19 @@ include:
|
||||
backend: "qwen3-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
- build-type: 'cublas'
|
||||
cuda-major-version: "13"
|
||||
cuda-minor-version: "0"
|
||||
platforms: 'linux/arm64'
|
||||
skip-drivers: 'false'
|
||||
tag-latest: 'auto'
|
||||
tag-suffix: '-nvidia-l4t-cuda-13-arm64-moss-tts-cpp'
|
||||
base-image: "ubuntu:24.04"
|
||||
ubuntu-version: '2404'
|
||||
runs-on: 'ubuntu-24.04-arm'
|
||||
backend: "moss-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
- build-type: 'cublas'
|
||||
cuda-major-version: "13"
|
||||
cuda-minor-version: "0"
|
||||
@@ -4529,6 +4568,35 @@ include:
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
# moss-tts-cpp
|
||||
- build-type: ''
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
platforms: 'linux/amd64'
|
||||
platform-tag: 'amd64'
|
||||
tag-latest: 'auto'
|
||||
tag-suffix: '-cpu-moss-tts-cpp'
|
||||
runs-on: 'ubuntu-latest'
|
||||
base-image: "ubuntu:24.04"
|
||||
skip-drivers: 'false'
|
||||
backend: "moss-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: ''
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
platforms: 'linux/arm64'
|
||||
platform-tag: 'arm64'
|
||||
tag-latest: 'auto'
|
||||
tag-suffix: '-cpu-moss-tts-cpp'
|
||||
runs-on: 'ubuntu-24.04-arm'
|
||||
base-image: "ubuntu:24.04"
|
||||
skip-drivers: 'false'
|
||||
backend: "moss-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
# omnivoice-cpp
|
||||
- build-type: ''
|
||||
cuda-major-version: ""
|
||||
@@ -4571,6 +4639,19 @@ include:
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'sycl_f32'
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
platforms: 'linux/amd64'
|
||||
tag-latest: 'auto'
|
||||
tag-suffix: '-gpu-intel-sycl-f32-moss-tts-cpp'
|
||||
runs-on: 'ubuntu-latest'
|
||||
base-image: "intel/oneapi-basekit:2025.3.0-0-devel-ubuntu24.04"
|
||||
skip-drivers: 'false'
|
||||
backend: "moss-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'sycl_f32'
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
@@ -4597,6 +4678,19 @@ include:
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'sycl_f16'
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
platforms: 'linux/amd64'
|
||||
tag-latest: 'auto'
|
||||
tag-suffix: '-gpu-intel-sycl-f16-moss-tts-cpp'
|
||||
runs-on: 'ubuntu-latest'
|
||||
base-image: "intel/oneapi-basekit:2025.3.0-0-devel-ubuntu24.04"
|
||||
skip-drivers: 'false'
|
||||
backend: "moss-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'sycl_f16'
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
@@ -4624,6 +4718,20 @@ include:
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'vulkan'
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
platforms: 'linux/amd64'
|
||||
platform-tag: 'amd64'
|
||||
tag-latest: 'auto'
|
||||
tag-suffix: '-gpu-vulkan-moss-tts-cpp'
|
||||
runs-on: 'ubuntu-latest'
|
||||
base-image: "ubuntu:24.04"
|
||||
skip-drivers: 'false'
|
||||
backend: "moss-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'vulkan'
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
@@ -4652,6 +4760,20 @@ include:
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'vulkan'
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
platforms: 'linux/arm64'
|
||||
platform-tag: 'arm64'
|
||||
tag-latest: 'auto'
|
||||
tag-suffix: '-gpu-vulkan-moss-tts-cpp'
|
||||
runs-on: 'ubuntu-24.04-arm'
|
||||
base-image: "ubuntu:24.04"
|
||||
skip-drivers: 'false'
|
||||
backend: "moss-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'vulkan'
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
@@ -4679,6 +4801,19 @@ include:
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2204'
|
||||
- build-type: 'cublas'
|
||||
cuda-major-version: "12"
|
||||
cuda-minor-version: "0"
|
||||
platforms: 'linux/arm64'
|
||||
skip-drivers: 'false'
|
||||
tag-latest: 'auto'
|
||||
tag-suffix: '-nvidia-l4t-arm64-moss-tts-cpp'
|
||||
base-image: "nvcr.io/nvidia/l4t-jetpack:r36.4.0"
|
||||
runs-on: 'ubuntu-24.04-arm'
|
||||
backend: "moss-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2204'
|
||||
- build-type: 'cublas'
|
||||
cuda-major-version: "12"
|
||||
cuda-minor-version: "0"
|
||||
@@ -4705,6 +4840,19 @@ include:
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'hipblas'
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
platforms: 'linux/amd64'
|
||||
tag-latest: 'auto'
|
||||
tag-suffix: '-gpu-rocm-hipblas-moss-tts-cpp'
|
||||
base-image: "rocm/dev-ubuntu-24.04:6.4.4"
|
||||
runs-on: 'ubuntu-latest'
|
||||
skip-drivers: 'false'
|
||||
backend: "moss-tts-cpp"
|
||||
dockerfile: "./backend/Dockerfile.golang"
|
||||
context: "./"
|
||||
ubuntu-version: '2404'
|
||||
- build-type: 'hipblas'
|
||||
cuda-major-version: ""
|
||||
cuda-minor-version: ""
|
||||
@@ -5622,6 +5770,10 @@ includeDarwin:
|
||||
tag-suffix: "-metal-darwin-arm64-qwen3-tts-cpp"
|
||||
build-type: "metal"
|
||||
lang: "go"
|
||||
- backend: "moss-tts-cpp"
|
||||
tag-suffix: "-metal-darwin-arm64-moss-tts-cpp"
|
||||
build-type: "metal"
|
||||
lang: "go"
|
||||
- backend: "omnivoice-cpp"
|
||||
tag-suffix: "-metal-darwin-arm64-omnivoice-cpp"
|
||||
build-type: "metal"
|
||||
|
||||
6
Makefile
6
Makefile
@@ -1,5 +1,5 @@
|
||||
# Disable parallel execution for backend builds
|
||||
.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/bonsai backends/outetts backends/piper backends/stablediffusion-ggml backends/whisper backends/crispasr backends/parakeet-cpp backends/moss-transcribe-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/cloud-proxy backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/longcat-video backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin
|
||||
.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/bonsai backends/outetts backends/piper backends/stablediffusion-ggml backends/whisper backends/crispasr backends/parakeet-cpp backends/moss-transcribe-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/cloud-proxy backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/longcat-video backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/moss-tts-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin
|
||||
|
||||
GOCMD=go
|
||||
GOTEST=$(GOCMD) test
|
||||
@@ -1249,6 +1249,7 @@ BACKEND_DEPTH_ANYTHING_CPP = depth-anything-cpp|golang|.|false|true
|
||||
BACKEND_VOXTRAL = voxtral|golang|.|false|true
|
||||
BACKEND_ACESTEP_CPP = acestep-cpp|golang|.|false|true
|
||||
BACKEND_QWEN3_TTS_CPP = qwen3-tts-cpp|golang|.|false|true
|
||||
BACKEND_MOSS_TTS_CPP = moss-tts-cpp|golang|.|false|true
|
||||
BACKEND_OMNIVOICE_CPP = omnivoice-cpp|golang|.|false|true
|
||||
BACKEND_VIBEVOICE_CPP = vibevoice-cpp|golang|.|false|true
|
||||
BACKEND_LOCALVQE = localvqe|golang|.|false|true
|
||||
@@ -1375,6 +1376,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_WHISPERX)))
|
||||
$(eval $(call generate-docker-build-target,$(BACKEND_ACE_STEP)))
|
||||
$(eval $(call generate-docker-build-target,$(BACKEND_ACESTEP_CPP)))
|
||||
$(eval $(call generate-docker-build-target,$(BACKEND_QWEN3_TTS_CPP)))
|
||||
$(eval $(call generate-docker-build-target,$(BACKEND_MOSS_TTS_CPP)))
|
||||
$(eval $(call generate-docker-build-target,$(BACKEND_OMNIVOICE_CPP)))
|
||||
$(eval $(call generate-docker-build-target,$(BACKEND_VIBEVOICE_CPP)))
|
||||
$(eval $(call generate-docker-build-target,$(BACKEND_LOCALVQE)))
|
||||
@@ -1394,7 +1396,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_SUPERTONIC)))
|
||||
docker-save-%: backend-images
|
||||
docker save local-ai-backend:$* -o backend-images/$*.tar
|
||||
|
||||
docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-bonsai docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-longcat-video docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-moss-transcribe-cpp docker-build-privacy-filter
|
||||
docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-bonsai docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-longcat-video docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-moss-tts-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-moss-transcribe-cpp docker-build-privacy-filter
|
||||
|
||||
########################################################
|
||||
### Mock Backend for E2E Tests
|
||||
|
||||
@@ -233,6 +233,7 @@ Most backends wrap a best-in-class upstream engine. A handful of them are native
|
||||
|---------|-------------|
|
||||
| [parakeet.cpp](https://github.com/mudler/parakeet.cpp) | C++/GGML port of NVIDIA NeMo Parakeet ASR (tdt/ctc/rnnt/hybrid), with cache-aware streaming transcription |
|
||||
| [moss-transcribe.cpp](https://github.com/mudler/moss-transcribe.cpp) | C++/GGML port of OpenMOSS MOSS-Transcribe-Diarize: joint long-form transcription, speaker diarization and timestamping in a single pass |
|
||||
| [moss-tts.cpp](https://github.com/mudler/moss-tts.cpp) | C++/GGML port of the OpenMOSS MOSS-TTS family: text-to-speech (MOSS-TTS-Local v1.5, 48 kHz stereo) with reference-audio voice cloning, through the MOSS-Audio-Tokenizer neural codec |
|
||||
| [ced.cpp](https://github.com/mudler/ced.cpp) | C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition |
|
||||
| [voice-detect.cpp](https://github.com/mudler/voice-detect.cpp) | Speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend |
|
||||
| [voxtral-tts.c](https://github.com/mudler/voxtral-tts.c) | Voxtral Realtime 4B speech-to-text in pure C |
|
||||
|
||||
5
backend/go/moss-tts-cpp/.gitignore
vendored
Normal file
5
backend/go/moss-tts-cpp/.gitignore
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
moss-tts-cpp
|
||||
*.so
|
||||
*.dylib
|
||||
sources/
|
||||
package/
|
||||
48
backend/go/moss-tts-cpp/CMakeLists.txt
Normal file
48
backend/go/moss-tts-cpp/CMakeLists.txt
Normal file
@@ -0,0 +1,48 @@
|
||||
cmake_minimum_required(VERSION 3.18)
|
||||
project(gomosstts-cpp LANGUAGES C CXX)
|
||||
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
|
||||
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
|
||||
|
||||
set(MOSS_DIR ${CMAKE_CURRENT_SOURCE_DIR}/sources/moss-tts.cpp)
|
||||
|
||||
# Override upstream's CMAKE_CUDA_ARCHITECTURES before add_subdirectory.
|
||||
if(NOT DEFINED CMAKE_CUDA_ARCHITECTURES)
|
||||
set(CMAKE_CUDA_ARCHITECTURES "75-virtual;80-virtual;86-real;89-real")
|
||||
endif()
|
||||
|
||||
# Add the upstream project. Its own CMakeLists vendors ggml under
|
||||
# third_party/ggml and builds the moss-tts library (STATIC by default, since we
|
||||
# do NOT set MOSS_TTS_SHARED). EXCLUDE_FROM_ALL keeps its CLI tool / tests /
|
||||
# examples from building unless referenced.
|
||||
add_subdirectory(${MOSS_DIR} mosstts EXCLUDE_FROM_ALL)
|
||||
|
||||
add_library(gomosstts-cpp MODULE cpp/mossttscpp.cpp)
|
||||
target_link_libraries(gomosstts-cpp PRIVATE moss-tts)
|
||||
|
||||
target_include_directories(gomosstts-cpp PRIVATE ${MOSS_DIR}/include ${MOSS_DIR}/src)
|
||||
target_include_directories(gomosstts-cpp SYSTEM PRIVATE ${MOSS_DIR}/third_party/ggml/include)
|
||||
|
||||
# Link GPU backends if the upstream ggml created them. moss-tts links these
|
||||
# PUBLIC when enabled, but we re-list them so the MODULE carries the symbols
|
||||
# even under static ggml (mirrors qwen3-tts-cpp).
|
||||
foreach(backend blas cuda hip metal vulkan sycl)
|
||||
if(TARGET ggml-${backend})
|
||||
target_link_libraries(gomosstts-cpp PRIVATE ggml-${backend})
|
||||
if(backend STREQUAL "cuda")
|
||||
find_package(CUDAToolkit QUIET)
|
||||
if(CUDAToolkit_FOUND)
|
||||
target_link_libraries(gomosstts-cpp PRIVATE CUDA::cudart)
|
||||
endif()
|
||||
endif()
|
||||
endif()
|
||||
endforeach()
|
||||
|
||||
if(MSVC)
|
||||
target_compile_options(gomosstts-cpp PRIVATE /W4 /wd4100 /wd4505)
|
||||
else()
|
||||
target_compile_options(gomosstts-cpp PRIVATE -Wall -Wextra
|
||||
-Wno-unused-parameter -Wno-unused-function)
|
||||
endif()
|
||||
|
||||
set_property(TARGET gomosstts-cpp PROPERTY CXX_STANDARD 17)
|
||||
set_target_properties(gomosstts-cpp PROPERTIES LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR})
|
||||
140
backend/go/moss-tts-cpp/Makefile
Normal file
140
backend/go/moss-tts-cpp/Makefile
Normal file
@@ -0,0 +1,140 @@
|
||||
CMAKE_ARGS?=
|
||||
BUILD_TYPE?=
|
||||
NATIVE?=false
|
||||
|
||||
GOCMD?=go
|
||||
GO_TAGS?=
|
||||
JOBS?=$(shell nproc --ignore=1)
|
||||
|
||||
# moss-tts.cpp version
|
||||
MOSSTTS_REPO?=https://github.com/mudler/moss-tts.cpp
|
||||
MOSSTTS_CPP_VERSION?=ee722b8e9205ee9b1b1c398a4e87e4e393e9be41
|
||||
SO_TARGET?=libgomosstts-cpp.so
|
||||
|
||||
CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF
|
||||
|
||||
ifeq ($(NATIVE),false)
|
||||
CMAKE_ARGS+=-DGGML_NATIVE=OFF
|
||||
endif
|
||||
|
||||
ifeq ($(BUILD_TYPE),cublas)
|
||||
CMAKE_ARGS+=-DGGML_CUDA=ON
|
||||
else ifeq ($(BUILD_TYPE),openblas)
|
||||
CMAKE_ARGS+=-DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS
|
||||
else ifeq ($(BUILD_TYPE),clblas)
|
||||
CMAKE_ARGS+=-DGGML_CLBLAST=ON -DCLBlast_DIR=/some/path
|
||||
else ifeq ($(BUILD_TYPE),hipblas)
|
||||
# This ggml only understands GGML_HIP (GGML_HIPBLAS was removed upstream),
|
||||
# so passing GGML_HIPBLAS silently produced a CPU-only build (see #10666).
|
||||
ROCM_HOME ?= /opt/rocm
|
||||
ROCM_PATH ?= /opt/rocm
|
||||
export CXX=$(ROCM_HOME)/llvm/bin/clang++
|
||||
export CC=$(ROCM_HOME)/llvm/bin/clang
|
||||
AMDGPU_TARGETS ?= gfx908,gfx90a,gfx942,gfx950,gfx1030,gfx1100,gfx1101,gfx1102,gfx1151,gfx1200,gfx1201
|
||||
CMAKE_ARGS+=-DGGML_HIP=ON -DAMDGPU_TARGETS=$(AMDGPU_TARGETS)
|
||||
else ifeq ($(BUILD_TYPE),vulkan)
|
||||
CMAKE_ARGS+=-DGGML_VULKAN=ON
|
||||
else ifeq ($(OS),Darwin)
|
||||
ifneq ($(BUILD_TYPE),metal)
|
||||
CMAKE_ARGS+=-DGGML_METAL=OFF
|
||||
else
|
||||
CMAKE_ARGS+=-DGGML_METAL=ON
|
||||
CMAKE_ARGS+=-DGGML_METAL_EMBED_LIBRARY=ON
|
||||
endif
|
||||
endif
|
||||
|
||||
ifeq ($(BUILD_TYPE),sycl_f16)
|
||||
CMAKE_ARGS+=-DGGML_SYCL=ON \
|
||||
-DCMAKE_C_COMPILER=icx \
|
||||
-DCMAKE_CXX_COMPILER=icpx \
|
||||
-DGGML_SYCL_F16=ON
|
||||
endif
|
||||
|
||||
ifeq ($(BUILD_TYPE),sycl_f32)
|
||||
CMAKE_ARGS+=-DGGML_SYCL=ON \
|
||||
-DCMAKE_C_COMPILER=icx \
|
||||
-DCMAKE_CXX_COMPILER=icpx
|
||||
endif
|
||||
|
||||
sources/moss-tts.cpp:
|
||||
mkdir -p sources/moss-tts.cpp
|
||||
cd sources/moss-tts.cpp && \
|
||||
git init && \
|
||||
git remote add origin $(MOSSTTS_REPO) && \
|
||||
git fetch origin && \
|
||||
git checkout $(MOSSTTS_CPP_VERSION) && \
|
||||
git submodule update --init --recursive --depth 1 --single-branch
|
||||
|
||||
# Detect OS
|
||||
UNAME_S := $(shell uname -s)
|
||||
|
||||
# Only build CPU variants on Linux
|
||||
ifeq ($(UNAME_S),Linux)
|
||||
VARIANT_TARGETS = libgomosstts-cpp-avx.so libgomosstts-cpp-avx2.so libgomosstts-cpp-avx512.so libgomosstts-cpp-fallback.so
|
||||
else
|
||||
# On non-Linux (e.g., Darwin), build only fallback variant (as a dylib)
|
||||
VARIANT_TARGETS = libgomosstts-cpp-fallback.dylib
|
||||
endif
|
||||
|
||||
moss-tts-cpp: main.go gomossttscpp.go $(VARIANT_TARGETS)
|
||||
CGO_ENABLED=0 $(GOCMD) build -tags "$(GO_TAGS)" -o moss-tts-cpp ./
|
||||
|
||||
package: moss-tts-cpp
|
||||
bash package.sh
|
||||
|
||||
build: package
|
||||
|
||||
clean: purge
|
||||
rm -rf libgomosstts-cpp*.so libgomosstts-cpp*.dylib package sources/moss-tts.cpp moss-tts-cpp
|
||||
|
||||
purge:
|
||||
rm -rf build*
|
||||
|
||||
# Variants must build sequentially
|
||||
.NOTPARALLEL:
|
||||
|
||||
# Build all variants (Linux only)
|
||||
ifeq ($(UNAME_S),Linux)
|
||||
libgomosstts-cpp-avx.so: sources/moss-tts.cpp
|
||||
$(info ${GREEN}I moss-tts-cpp build info:avx${RESET})
|
||||
SO_TARGET=libgomosstts-cpp-avx.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=on -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_BMI2=off" $(MAKE) libgomosstts-cpp-custom
|
||||
rm -rf build-libgomosstts-cpp-avx.so
|
||||
|
||||
libgomosstts-cpp-avx2.so: sources/moss-tts.cpp
|
||||
$(info ${GREEN}I moss-tts-cpp build info:avx2${RESET})
|
||||
SO_TARGET=libgomosstts-cpp-avx2.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=on -DGGML_AVX2=on -DGGML_AVX512=off -DGGML_FMA=on -DGGML_F16C=on -DGGML_BMI2=on" $(MAKE) libgomosstts-cpp-custom
|
||||
rm -rf build-libgomosstts-cpp-avx2.so
|
||||
|
||||
libgomosstts-cpp-avx512.so: sources/moss-tts.cpp
|
||||
$(info ${GREEN}I moss-tts-cpp build info:avx512${RESET})
|
||||
SO_TARGET=libgomosstts-cpp-avx512.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=on -DGGML_AVX2=on -DGGML_AVX512=on -DGGML_FMA=on -DGGML_F16C=on -DGGML_BMI2=on" $(MAKE) libgomosstts-cpp-custom
|
||||
rm -rf build-libgomosstts-cpp-avx512.so
|
||||
endif
|
||||
|
||||
# Build fallback variant (all platforms)
|
||||
libgomosstts-cpp-fallback.so: sources/moss-tts.cpp
|
||||
$(info ${GREEN}I moss-tts-cpp build info:fallback${RESET})
|
||||
SO_TARGET=libgomosstts-cpp-fallback.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=off -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_BMI2=off" $(MAKE) libgomosstts-cpp-custom
|
||||
rm -rf build-libgomosstts-cpp-fallback.so
|
||||
|
||||
# Build fallback variant as a dylib (Darwin)
|
||||
libgomosstts-cpp-fallback.dylib: sources/moss-tts.cpp
|
||||
$(info ${GREEN}I moss-tts-cpp build info:fallback (dylib)${RESET})
|
||||
SO_TARGET=libgomosstts-cpp-fallback.dylib CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=off -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_BMI2=off" $(MAKE) libgomosstts-cpp-custom
|
||||
rm -rf build-libgomosstts-cpp-fallback.dylib
|
||||
|
||||
libgomosstts-cpp-custom: CMakeLists.txt cpp/mossttscpp.cpp cpp/mossttscpp.h
|
||||
mkdir -p build-$(SO_TARGET) && \
|
||||
cd build-$(SO_TARGET) && \
|
||||
cmake .. $(CMAKE_ARGS) && \
|
||||
cmake --build . --config Release -j$(JOBS) --target gomosstts-cpp && \
|
||||
cd .. && \
|
||||
(mv build-$(SO_TARGET)/libgomosstts-cpp.so ./$(SO_TARGET) 2>/dev/null || \
|
||||
mv build-$(SO_TARGET)/libgomosstts-cpp.dylib ./$(SO_TARGET) 2>/dev/null)
|
||||
|
||||
test: moss-tts-cpp
|
||||
@echo "Running moss-tts-cpp tests..."
|
||||
bash test.sh
|
||||
@echo "moss-tts-cpp tests completed."
|
||||
|
||||
all: moss-tts-cpp package
|
||||
79
backend/go/moss-tts-cpp/README.md
Normal file
79
backend/go/moss-tts-cpp/README.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# MOSS-TTS C++ backend
|
||||
|
||||
This backend runs the OpenMOSS **MOSS-TTS-Local (v1.5)** GGUF model through
|
||||
[moss-tts.cpp](https://github.com/mudler/moss-tts.cpp), a from-scratch C++/ggml
|
||||
port with no Python at inference time. It generates **48 kHz stereo** speech and
|
||||
supports reference-audio voice cloning.
|
||||
|
||||
The engine loads three GGUFs: the local transformer (the model), the
|
||||
MOSS-Audio-Tokenizer neural codec, and the text tokenizer. It is loaded via
|
||||
purego (cgo-less `dlopen`) exactly like `qwen3-tts-cpp`.
|
||||
|
||||
## Model configuration
|
||||
|
||||
The model path points at the local transformer GGUF. The codec and text
|
||||
tokenizer are auto-discovered as siblings of the model:
|
||||
|
||||
- codec: a `*.gguf` whose name contains `audio` + `tokenizer` (or `codec`),
|
||||
e.g. `moss-audio-tokenizer-v2-f32.gguf`
|
||||
- tokenizer: the other `*.gguf` whose name contains `tokenizer`,
|
||||
e.g. `moss-tokenizer-v1_5.gguf`
|
||||
|
||||
```yaml
|
||||
name: moss-tts-cpp
|
||||
backend: moss-tts-cpp
|
||||
parameters:
|
||||
model: moss-tts-local-v1_5-q8_0.gguf
|
||||
known_usecases:
|
||||
- tts
|
||||
tts:
|
||||
audio_path: voices/default-reference.wav # optional model-wide clone reference
|
||||
```
|
||||
|
||||
Override discovery when the filenames are non-standard:
|
||||
|
||||
```yaml
|
||||
options:
|
||||
- "codec:moss-audio-tokenizer-v2-f32.gguf"
|
||||
- "tokenizer:moss-tokenizer-v1_5.gguf"
|
||||
- "seed:42"
|
||||
```
|
||||
|
||||
GGUFs for the Local v1.5 model (plus its codec and tokenizer) live at
|
||||
[mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF](https://huggingface.co/mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF).
|
||||
|
||||
## Voice cloning
|
||||
|
||||
MOSS-TTS-Local has no named speakers; cloning is driven purely by a
|
||||
reference-audio **path**. Request precedence is: a request `voice` that ends in
|
||||
a known audio extension (`.wav`, `.flac`, `.mp3`, `.ogg`, `.m4a`), then
|
||||
`tts.audio_path`. The engine decodes the reference itself, so no client-side
|
||||
resampling is required.
|
||||
|
||||
## API example
|
||||
|
||||
```bash
|
||||
curl http://localhost:8080/v1/audio/speech \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "moss-tts-cpp",
|
||||
"input": "This request uses a saved reference voice.",
|
||||
"voice": "/path/to/reference.wav"
|
||||
}' \
|
||||
--output speech.wav
|
||||
```
|
||||
|
||||
## Native end-to-end test
|
||||
|
||||
The labeled test loads real GGUFs, synthesizes a 48 kHz stereo WAV, and streams
|
||||
audio:
|
||||
|
||||
```bash
|
||||
make -C backend/go/moss-tts-cpp moss-tts-cpp
|
||||
|
||||
MOSSTTS_MODEL=/path/to/moss-tts-local-v1_5-q8_0.gguf \
|
||||
MOSSTTS_CODEC=/path/to/moss-audio-tokenizer-v2-f32.gguf \
|
||||
MOSSTTS_TOKENIZER=/path/to/moss-tokenizer-v1_5.gguf \
|
||||
MOSSTTS_LIBRARY=backend/go/moss-tts-cpp/libgomosstts-cpp-fallback.so \
|
||||
go test ./backend/go/moss-tts-cpp -ginkgo.label-filter=e2e
|
||||
```
|
||||
100
backend/go/moss-tts-cpp/audio.go
Normal file
100
backend/go/moss-tts-cpp/audio.go
Normal file
@@ -0,0 +1,100 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/binary"
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"github.com/go-audio/audio"
|
||||
"github.com/go-audio/wav"
|
||||
)
|
||||
|
||||
// mossttsSampleRate is the MOSS-TTS-Local v1.5 output rate; the engine also
|
||||
// reports it per-synthesis via out_sr, which the writers honour.
|
||||
const mossttsSampleRate = 48000
|
||||
|
||||
// mossttsChannels is the MOSS-TTS-Local output layout: 48 kHz stereo, samples
|
||||
// interleaved [L,R,L,R,...].
|
||||
const mossttsChannels = 2
|
||||
|
||||
// wavHeaderStereo returns a 44-byte WAV header for a streaming 16-bit stereo PCM
|
||||
// stream at sampleRate, with placeholder (0xFFFFFFFF) sizes since the total
|
||||
// length is unknown up front. Emitted as the first chunk of TTSStream so the
|
||||
// HTTP layer receives a self-describing WAV.
|
||||
func wavHeaderStereo(sampleRate int) []byte {
|
||||
if sampleRate <= 0 {
|
||||
sampleRate = mossttsSampleRate
|
||||
}
|
||||
const blockAlign = mossttsChannels * 2 // 16-bit stereo
|
||||
var buf bytes.Buffer
|
||||
w := func(v any) { _ = binary.Write(&buf, binary.LittleEndian, v) }
|
||||
buf.WriteString("RIFF")
|
||||
w(uint32(0xFFFFFFFF))
|
||||
buf.WriteString("WAVE")
|
||||
buf.WriteString("fmt ")
|
||||
w(uint32(16)) // Subchunk1Size
|
||||
w(uint16(1)) // PCM
|
||||
w(uint16(mossttsChannels)) // stereo
|
||||
w(uint32(sampleRate)) // sample rate
|
||||
w(uint32(sampleRate * blockAlign)) // byte rate = SR * blockAlign
|
||||
w(uint16(blockAlign)) // block align
|
||||
w(uint16(16)) // bits per sample
|
||||
buf.WriteString("data")
|
||||
w(uint32(0xFFFFFFFF))
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
// floatToPCM16LE clamps each sample to [-1,1] and encodes it as little-endian
|
||||
// signed 16-bit PCM. Input may be interleaved stereo; the layout is preserved.
|
||||
func floatToPCM16LE(samples []float32) []byte {
|
||||
out := make([]byte, len(samples)*2)
|
||||
for i, s := range samples {
|
||||
if s > 1 {
|
||||
s = 1
|
||||
} else if s < -1 {
|
||||
s = -1
|
||||
}
|
||||
v := int16(s * 32767)
|
||||
out[i*2] = byte(v) // #nosec G115 -- intentional little-endian split of a clamped int16
|
||||
out[i*2+1] = byte(v >> 8) // #nosec G115 -- high byte of the same clamped int16
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// writeWAVStereo writes interleaved [L,R,...] float samples as a finalized
|
||||
// 16-bit stereo WAV at sampleRate (falling back to the v1.5 default of 48 kHz).
|
||||
func writeWAVStereo(dst string, samples []float32, sampleRate int) error {
|
||||
if sampleRate <= 0 {
|
||||
sampleRate = mossttsSampleRate
|
||||
}
|
||||
f, err := os.Create(dst) // #nosec G304 -- dst is the server-chosen output path from the TTS request, not user-traversable
|
||||
if err != nil {
|
||||
return fmt.Errorf("moss-tts: create %q: %w", dst, err)
|
||||
}
|
||||
enc := wav.NewEncoder(f, sampleRate, 16, mossttsChannels, 1)
|
||||
ints := make([]int, len(samples))
|
||||
for i, s := range samples {
|
||||
if s > 1 {
|
||||
s = 1
|
||||
} else if s < -1 {
|
||||
s = -1
|
||||
}
|
||||
ints[i] = int(s * 32767)
|
||||
}
|
||||
b := &audio.IntBuffer{
|
||||
Format: &audio.Format{NumChannels: mossttsChannels, SampleRate: sampleRate},
|
||||
Data: ints,
|
||||
SourceBitDepth: 16,
|
||||
}
|
||||
if err := enc.Write(b); err != nil {
|
||||
_ = enc.Close()
|
||||
_ = f.Close()
|
||||
return fmt.Errorf("moss-tts: encode WAV: %w", err)
|
||||
}
|
||||
if err := enc.Close(); err != nil {
|
||||
_ = f.Close()
|
||||
return fmt.Errorf("moss-tts: finalize WAV: %w", err)
|
||||
}
|
||||
return f.Close()
|
||||
}
|
||||
49
backend/go/moss-tts-cpp/audiopath_test.go
Normal file
49
backend/go/moss-tts-cpp/audiopath_test.go
Normal file
@@ -0,0 +1,49 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
|
||||
pb "github.com/mudler/LocalAI/pkg/grpc/proto"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
// These specs pin the voice-selection logic in resolveRequest, in particular
|
||||
// the config-level audio_path (tts.audio_path -> ModelOptions.AudioPath) being
|
||||
// used as the default voice-cloning reference. MOSS-TTS-Local takes the
|
||||
// reference as a WAV path (the engine decodes it), so resolveRequest is pure
|
||||
// path logic and needs no model / C library.
|
||||
var _ = Describe("resolveRequest voice/clone selection", func() {
|
||||
var dir, refWav string
|
||||
|
||||
BeforeEach(func() {
|
||||
dir = GinkgoT().TempDir()
|
||||
refWav = filepath.Join(dir, "ref.wav")
|
||||
})
|
||||
|
||||
It("uses the config audio_path as the clone reference when Voice is empty", func() {
|
||||
m := &MossTtsCpp{audioPath: refWav}
|
||||
ref, seed := m.resolveRequest(&pb.TTSRequest{Text: "hi"})
|
||||
Expect(ref).To(Equal(refWav))
|
||||
Expect(seed).To(Equal(0)) // zero-value loadOptions seed
|
||||
})
|
||||
|
||||
It("lets a per-request audio Voice override audio_path", func() {
|
||||
other := filepath.Join(dir, "other.wav")
|
||||
m := &MossTtsCpp{audioPath: refWav}
|
||||
ref, _ := m.resolveRequest(&pb.TTSRequest{Text: "hi", Voice: other})
|
||||
Expect(ref).To(Equal(other))
|
||||
})
|
||||
|
||||
It("does not clone for a bare-token Voice, falling back to audio_path", func() {
|
||||
m := &MossTtsCpp{audioPath: refWav}
|
||||
ref, _ := m.resolveRequest(&pb.TTSRequest{Text: "hi", Voice: "serena"})
|
||||
Expect(ref).To(Equal(refWav))
|
||||
})
|
||||
|
||||
It("reads a per-request seed override from params", func() {
|
||||
m := &MossTtsCpp{opts: loadOptions{seed: -1}}
|
||||
_, seed := m.resolveRequest(&pb.TTSRequest{Text: "hi", Params: map[string]string{"seed": "123"}})
|
||||
Expect(seed).To(Equal(123))
|
||||
})
|
||||
})
|
||||
119
backend/go/moss-tts-cpp/cpp/mossttscpp.cpp
Normal file
119
backend/go/moss-tts-cpp/cpp/mossttscpp.cpp
Normal file
@@ -0,0 +1,119 @@
|
||||
#include "mossttscpp.h"
|
||||
#include "ggml-backend.h"
|
||||
#include "moss_tts_capi.h"
|
||||
|
||||
#include <cstdio>
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
|
||||
// Single loaded Local pipeline. base.SingleThread serializes calls on the Go
|
||||
// side, so a plain global is safe (mirrors qwen3-tts-cpp's g_ctx).
|
||||
static moss_local *g_local = nullptr;
|
||||
|
||||
static void ggml_log_cb(enum ggml_log_level level, const char *log,
|
||||
void * /*data*/) {
|
||||
if (!log)
|
||||
return;
|
||||
const char *lvl = "?????";
|
||||
switch (level) {
|
||||
case GGML_LOG_LEVEL_DEBUG: lvl = "DEBUG"; break;
|
||||
case GGML_LOG_LEVEL_INFO: lvl = "INFO"; break;
|
||||
case GGML_LOG_LEVEL_WARN: lvl = "WARN"; break;
|
||||
case GGML_LOG_LEVEL_ERROR: lvl = "ERROR"; break;
|
||||
default: break;
|
||||
}
|
||||
fprintf(stderr, "[%-5s] %s", lvl, log);
|
||||
fflush(stderr);
|
||||
}
|
||||
|
||||
int mtl_load(const char *local_path, const char *codec_path,
|
||||
const char *tokenizer_path) {
|
||||
ggml_log_set(ggml_log_cb, nullptr);
|
||||
ggml_backend_load_all();
|
||||
|
||||
if (!local_path || local_path[0] == '\0') {
|
||||
fprintf(stderr, "[moss-tts-cpp] ERROR: local_path is required\n");
|
||||
return 1;
|
||||
}
|
||||
if (!codec_path || codec_path[0] == '\0') {
|
||||
fprintf(stderr, "[moss-tts-cpp] ERROR: codec_path is required\n");
|
||||
return 2;
|
||||
}
|
||||
if (!tokenizer_path || tokenizer_path[0] == '\0') {
|
||||
fprintf(stderr, "[moss-tts-cpp] ERROR: tokenizer_path is required\n");
|
||||
return 3;
|
||||
}
|
||||
|
||||
fprintf(stderr,
|
||||
"[moss-tts-cpp] Loading local=%s codec=%s tokenizer=%s\n",
|
||||
local_path, codec_path, tokenizer_path);
|
||||
|
||||
g_local = moss_local_load(local_path, codec_path, tokenizer_path);
|
||||
if (!g_local) {
|
||||
fprintf(stderr, "[moss-tts-cpp] FATAL: moss_local_load failed\n");
|
||||
return 4;
|
||||
}
|
||||
fprintf(stderr, "[moss-tts-cpp] Model loaded (%s)\n", moss_tts_version());
|
||||
return 0;
|
||||
}
|
||||
|
||||
float *mtl_tts(const char *text, const char *reference_wav, int seed,
|
||||
int *out_n, int *out_sr) {
|
||||
if (out_n)
|
||||
*out_n = 0;
|
||||
if (out_sr)
|
||||
*out_sr = 0;
|
||||
if (!g_local) {
|
||||
fprintf(stderr, "[moss-tts-cpp] ERROR: model not loaded\n");
|
||||
return nullptr;
|
||||
}
|
||||
if (!text || text[0] == '\0') {
|
||||
fprintf(stderr, "[moss-tts-cpp] ERROR: text is required\n");
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
// An empty reference path means "no cloning": pass NULL so the engine skips
|
||||
// the clone branch rather than trying to open "".
|
||||
const char *ref = (reference_wav && reference_wav[0] != '\0')
|
||||
? reference_wav
|
||||
: nullptr;
|
||||
|
||||
int n = 0, sr = 0;
|
||||
float *pcm = moss_local_tts(g_local, text, ref, seed, &n, &sr);
|
||||
if (!pcm || n <= 0) {
|
||||
fprintf(stderr, "[moss-tts-cpp] ERROR: moss_local_tts failed\n");
|
||||
if (pcm)
|
||||
moss_free(pcm);
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
// Copy into a plain malloc buffer the Go side frees via mtl_pcm_free, then
|
||||
// release the engine-owned buffer with moss_free (mirrors qwen3-tts-cpp,
|
||||
// keeping ownership on the C runtime's malloc/free).
|
||||
size_t bytes = (size_t)n * sizeof(float);
|
||||
float *buf = (float *)malloc(bytes);
|
||||
if (!buf) {
|
||||
fprintf(stderr, "[moss-tts-cpp] ERROR: malloc(%zu) failed\n", bytes);
|
||||
moss_free(pcm);
|
||||
return nullptr;
|
||||
}
|
||||
memcpy(buf, pcm, bytes);
|
||||
moss_free(pcm);
|
||||
|
||||
if (out_n)
|
||||
*out_n = n;
|
||||
if (out_sr)
|
||||
*out_sr = sr;
|
||||
return buf;
|
||||
}
|
||||
|
||||
void mtl_pcm_free(float *p) { free(p); }
|
||||
|
||||
void mtl_unload(void) {
|
||||
if (g_local) {
|
||||
moss_local_free(g_local);
|
||||
g_local = nullptr;
|
||||
}
|
||||
}
|
||||
|
||||
const char *mtl_version(void) { return moss_tts_version(); }
|
||||
36
backend/go/moss-tts-cpp/cpp/mossttscpp.h
Normal file
36
backend/go/moss-tts-cpp/cpp/mossttscpp.h
Normal file
@@ -0,0 +1,36 @@
|
||||
#pragma once
|
||||
|
||||
// Thin C shim over moss-tts.cpp's flat C-API (include/moss_tts_capi.h) for the
|
||||
// MOSS-TTS-Local (v1.5) pipeline. It holds the loaded pipeline as a single
|
||||
// global handle so the purego bindings on the Go side stay handle-free (mirrors
|
||||
// the qwen3-tts-cpp qt3_* shim). Only the Local variant is wired here; the
|
||||
// Delay / Realtime / Nano variants of the upstream API are intentionally
|
||||
// unused.
|
||||
|
||||
extern "C" {
|
||||
|
||||
// Load the Local (MossTTSLocal v1.5) pipeline from its three GGUFs: the local
|
||||
// transformer, the MOSS-Audio-Tokenizer codec, and the text tokenizer. Sets up
|
||||
// ggml logging + backend registration first. Returns 0 on success, non-zero on
|
||||
// failure.
|
||||
int mtl_load(const char *local_path, const char *codec_path,
|
||||
const char *tokenizer_path);
|
||||
|
||||
// Synthesize `text` to a malloc'd interleaved f32 PCM buffer (caller frees via
|
||||
// mtl_pcm_free). v1.5 output is 48 kHz stereo: *out_n is the total number of
|
||||
// float samples across both channels (frames * 2) and *out_sr is the sample
|
||||
// rate. `reference_wav` is an optional path to a clone-reference WAV (may be
|
||||
// NULL/empty for no cloning); the engine decodes it itself. `seed` < 0 means
|
||||
// random. Returns NULL on failure (out_n / out_sr set to 0).
|
||||
float *mtl_tts(const char *text, const char *reference_wav, int seed,
|
||||
int *out_n, int *out_sr);
|
||||
|
||||
// Free a buffer returned by mtl_tts.
|
||||
void mtl_pcm_free(float *p);
|
||||
|
||||
// Release the loaded pipeline.
|
||||
void mtl_unload(void);
|
||||
|
||||
// Upstream engine version string (moss_tts_version()).
|
||||
const char *mtl_version(void);
|
||||
}
|
||||
72
backend/go/moss-tts-cpp/e2e_test.go
Normal file
72
backend/go/moss-tts-cpp/e2e_test.go
Normal file
@@ -0,0 +1,72 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"github.com/ebitengine/purego"
|
||||
pb "github.com/mudler/LocalAI/pkg/grpc/proto"
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
func ttsReq(text, voice, dst string) *pb.TTSRequest {
|
||||
return &pb.TTSRequest{Text: text, Voice: voice, Dst: dst}
|
||||
}
|
||||
|
||||
var _ = Describe("moss-tts-cpp e2e", Label("e2e"), func() {
|
||||
var loaded bool
|
||||
|
||||
BeforeEach(func() {
|
||||
modelPath := os.Getenv("MOSSTTS_MODEL")
|
||||
codecPath := os.Getenv("MOSSTTS_CODEC")
|
||||
tokenizerPath := os.Getenv("MOSSTTS_TOKENIZER")
|
||||
if modelPath == "" || codecPath == "" || tokenizerPath == "" {
|
||||
Skip("MOSSTTS_MODEL / MOSSTTS_CODEC / MOSSTTS_TOKENIZER not set; skipping e2e")
|
||||
}
|
||||
if !loaded {
|
||||
lib := os.Getenv("MOSSTTS_LIBRARY")
|
||||
if lib == "" {
|
||||
lib = "./libgomosstts-cpp-fallback.so"
|
||||
}
|
||||
h, err := purego.Dlopen(lib, purego.RTLD_NOW|purego.RTLD_GLOBAL)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
purego.RegisterLibFunc(&CppLoad, h, "mtl_load")
|
||||
purego.RegisterLibFunc(&CppTTS, h, "mtl_tts")
|
||||
purego.RegisterLibFunc(&CppPCMFree, h, "mtl_pcm_free")
|
||||
purego.RegisterLibFunc(&CppUnload, h, "mtl_unload")
|
||||
Expect(CppLoad(modelPath, codecPath, tokenizerPath)).To(Equal(0))
|
||||
loaded = true
|
||||
}
|
||||
})
|
||||
|
||||
It("synthesizes a stereo WAV file via TTS", func() {
|
||||
m := &MossTtsCpp{opts: loadOptions{seed: 42}}
|
||||
dst := GinkgoT().TempDir() + "/out.wav"
|
||||
err := m.TTS(ttsReq("Hello world.", "", dst))
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
fi, err := os.Stat(dst)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(fi.Size()).To(BeNumerically(">", int64(44)))
|
||||
})
|
||||
|
||||
It("streams audio chunks via TTSStream", func() {
|
||||
m := &MossTtsCpp{opts: loadOptions{seed: 42}}
|
||||
results := make(chan []byte, 4096)
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- m.TTSStream(ttsReq("Hello there, streaming test.", "", ""), results) }()
|
||||
|
||||
var chunks int
|
||||
var first []byte
|
||||
for c := range results {
|
||||
if chunks == 0 {
|
||||
first = c
|
||||
}
|
||||
chunks++
|
||||
}
|
||||
Expect(<-done).ToNot(HaveOccurred())
|
||||
Expect(chunks).To(BeNumerically(">=", 2))
|
||||
Expect(string(first[0:4])).To(Equal("RIFF"))
|
||||
Expect(strings.HasPrefix(string(first[8:12]), "WAVE")).To(BeTrue())
|
||||
})
|
||||
})
|
||||
224
backend/go/moss-tts-cpp/gomossttscpp.go
Normal file
224
backend/go/moss-tts-cpp/gomossttscpp.go
Normal file
@@ -0,0 +1,224 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"unsafe"
|
||||
|
||||
"github.com/mudler/LocalAI/pkg/grpc/base"
|
||||
pb "github.com/mudler/LocalAI/pkg/grpc/proto"
|
||||
"github.com/mudler/xlog"
|
||||
)
|
||||
|
||||
var (
|
||||
// mtl_load(local_path, codec_path, tokenizer_path) int
|
||||
CppLoad func(localPath, codecPath, tokenizerPath string) int
|
||||
// mtl_tts(text, reference_wav, seed, out_n, out_sr) -> float*
|
||||
CppTTS func(text, referenceWav string, seed int, outN, outSR unsafe.Pointer) uintptr
|
||||
// mtl_pcm_free(ptr)
|
||||
CppPCMFree func(ptr uintptr)
|
||||
// mtl_unload()
|
||||
CppUnload func()
|
||||
)
|
||||
|
||||
type MossTtsCpp struct {
|
||||
base.SingleThread
|
||||
opts loadOptions
|
||||
// audioPath is the model-config reference voice (tts.audio_path), the
|
||||
// default clone reference when a request omits an audio Voice.
|
||||
audioPath string
|
||||
}
|
||||
|
||||
func (m *MossTtsCpp) Load(opts *pb.ModelOptions) error {
|
||||
model := opts.ModelFile
|
||||
if model == "" {
|
||||
model = opts.ModelPath
|
||||
}
|
||||
if !filepath.IsAbs(model) && opts.ModelPath != "" {
|
||||
model = filepath.Join(opts.ModelPath, model)
|
||||
}
|
||||
|
||||
m.opts = parseOptions(opts.Options)
|
||||
dir := filepath.Dir(model)
|
||||
|
||||
// Resolve the codec GGUF (MOSS-Audio-Tokenizer): explicit option, else
|
||||
// auto-discover an *audio*tokenizer*/codec sibling of the model.
|
||||
codec := resolveAux(m.opts.codecPath, dir)
|
||||
if codec == "" {
|
||||
codec = discoverCodec(dir, model)
|
||||
}
|
||||
if codec == "" {
|
||||
return fmt.Errorf("moss-tts: no codec GGUF found; set option 'codec:<file>'")
|
||||
}
|
||||
m.opts.codecPath = codec
|
||||
|
||||
// Resolve the text tokenizer GGUF: explicit option, else auto-discover the
|
||||
// *tokenizer* sibling that is not the audio codec or the model.
|
||||
tokenizer := resolveAux(m.opts.tokenizerPath, dir)
|
||||
if tokenizer == "" {
|
||||
tokenizer = discoverTokenizer(dir, model, codec)
|
||||
}
|
||||
if tokenizer == "" {
|
||||
return fmt.Errorf("moss-tts: no tokenizer GGUF found; set option 'tokenizer:<file>'")
|
||||
}
|
||||
m.opts.tokenizerPath = tokenizer
|
||||
|
||||
m.audioPath = opts.AudioPath
|
||||
if m.audioPath != "" && !filepath.IsAbs(m.audioPath) {
|
||||
m.audioPath = filepath.Join(dir, m.audioPath)
|
||||
}
|
||||
|
||||
xlog.Info("[moss-tts-cpp] Load", "model", model, "codec", codec, "tokenizer", tokenizer)
|
||||
|
||||
if rc := CppLoad(model, codec, tokenizer); rc != 0 {
|
||||
return fmt.Errorf("moss-tts: failed to load model (rc=%d)", rc)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// resolveAux resolves an explicitly configured auxiliary GGUF path relative to
|
||||
// the model directory when it is not absolute. Empty stays empty.
|
||||
func resolveAux(p, dir string) string {
|
||||
if p == "" || filepath.IsAbs(p) {
|
||||
return p
|
||||
}
|
||||
return filepath.Join(dir, p)
|
||||
}
|
||||
|
||||
// isAudioCodecName reports whether a GGUF filename denotes the MOSS-Audio-
|
||||
// Tokenizer codec (e.g. moss-audio-tokenizer-v2-f32.gguf) rather than the text
|
||||
// tokenizer (moss-tokenizer-v1_5.gguf).
|
||||
func isAudioCodecName(name string) bool {
|
||||
n := strings.ToLower(name)
|
||||
return strings.Contains(n, "codec") ||
|
||||
(strings.Contains(n, "audio") && strings.Contains(n, "tokenizer"))
|
||||
}
|
||||
|
||||
// discoverCodec returns the first codec GGUF in dir (excluding the model), or "".
|
||||
func discoverCodec(dir, model string) string {
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
modelBase := filepath.Base(model)
|
||||
for _, e := range entries {
|
||||
name := e.Name()
|
||||
if name == modelBase || !strings.HasSuffix(strings.ToLower(name), ".gguf") {
|
||||
continue
|
||||
}
|
||||
if isAudioCodecName(name) {
|
||||
return filepath.Join(dir, name)
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// discoverTokenizer returns the first text-tokenizer GGUF in dir that is neither
|
||||
// the model nor the audio codec, or "".
|
||||
func discoverTokenizer(dir, model, codec string) string {
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
modelBase := filepath.Base(model)
|
||||
codecBase := filepath.Base(codec)
|
||||
for _, e := range entries {
|
||||
name := e.Name()
|
||||
lower := strings.ToLower(name)
|
||||
if name == modelBase || name == codecBase || !strings.HasSuffix(lower, ".gguf") {
|
||||
continue
|
||||
}
|
||||
if strings.Contains(lower, "tokenizer") && !isAudioCodecName(name) {
|
||||
return filepath.Join(dir, name)
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// resolveRequest derives the synthesis inputs from a TTSRequest: the optional
|
||||
// clone-reference WAV path and the seed. MOSS-TTS-Local drives cloning purely
|
||||
// from a reference-audio path (the engine decodes it), so there is no
|
||||
// language / speaker / instruct plumbing here (unlike qwen3-tts-cpp).
|
||||
func (m *MossTtsCpp) resolveRequest(req *pb.TTSRequest) (refPath string, seed int) {
|
||||
refPath = resolveVoice(req.Voice)
|
||||
if refPath == "" && m.audioPath != "" {
|
||||
// No per-request voice: fall back to the config clone reference.
|
||||
refPath = m.audioPath
|
||||
}
|
||||
if refPath != "" && !filepath.IsAbs(refPath) {
|
||||
refPath = filepath.Join(filepath.Dir(m.opts.codecPath), refPath)
|
||||
}
|
||||
|
||||
seed = m.opts.seed
|
||||
if req.Params != nil {
|
||||
seed = parseInt(req.Params["seed"], seed)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
func (m *MossTtsCpp) TTS(req *pb.TTSRequest) error {
|
||||
if req.Dst == "" {
|
||||
return fmt.Errorf("moss-tts: TTS requires a destination path")
|
||||
}
|
||||
if req.Text == "" {
|
||||
return fmt.Errorf("moss-tts: TTS requires text")
|
||||
}
|
||||
refPath, seed := m.resolveRequest(req)
|
||||
|
||||
var n, sr int32
|
||||
ptr := CppTTS(req.Text, refPath, seed, unsafe.Pointer(&n), unsafe.Pointer(&sr)) // #nosec G103 -- out-params for the purego-bound C-API
|
||||
if ptr == 0 {
|
||||
return fmt.Errorf("moss-tts: synthesis failed")
|
||||
}
|
||||
// Register the free as soon as we own a non-null buffer, so the n<=0 guard
|
||||
// below cannot leak it (defensive: the C contract returns NULL on failure).
|
||||
defer CppPCMFree(ptr)
|
||||
if n <= 0 {
|
||||
return fmt.Errorf("moss-tts: synthesis produced no samples")
|
||||
}
|
||||
//nolint:govet // C-allocated PCM, copied out before free
|
||||
src := unsafe.Slice((*float32)(unsafe.Pointer(ptr)), int(n)) // #nosec G103 -- C-allocated PCM, copied out before free
|
||||
out := make([]float32, int(n))
|
||||
copy(out, src)
|
||||
return writeWAVStereo(req.Dst, out, int(sr))
|
||||
}
|
||||
|
||||
// TTSStream synthesizes one-shot (MOSS-TTS-Local has no native streaming C-API)
|
||||
// and then emits a self-describing stereo WAV: a header chunk followed by the
|
||||
// interleaved PCM in fixed-size slices, so the HTTP layer still receives a
|
||||
// streamed WAV (the gRPC TTSStream path never sets Message, so the backend owns
|
||||
// the header - see core/backend/tts.go:ModelTTSStream).
|
||||
func (m *MossTtsCpp) TTSStream(req *pb.TTSRequest, results chan []byte) error {
|
||||
defer close(results)
|
||||
if req.Text == "" {
|
||||
return fmt.Errorf("moss-tts: TTSStream requires text")
|
||||
}
|
||||
refPath, seed := m.resolveRequest(req)
|
||||
|
||||
var n, sr int32
|
||||
ptr := CppTTS(req.Text, refPath, seed, unsafe.Pointer(&n), unsafe.Pointer(&sr)) // #nosec G103 -- out-params for the purego-bound C-API
|
||||
if ptr == 0 {
|
||||
return fmt.Errorf("moss-tts: synthesis failed")
|
||||
}
|
||||
defer CppPCMFree(ptr)
|
||||
if n <= 0 {
|
||||
return fmt.Errorf("moss-tts: synthesis produced no samples")
|
||||
}
|
||||
//nolint:govet // C-allocated PCM, copied out before free
|
||||
src := unsafe.Slice((*float32)(unsafe.Pointer(ptr)), int(n)) // #nosec G103 -- C-allocated PCM, copied out before free
|
||||
out := make([]float32, int(n))
|
||||
copy(out, src)
|
||||
|
||||
results <- wavHeaderStereo(int(sr))
|
||||
const frameChunk = 4096 // interleaved stereo samples per emitted chunk
|
||||
for off := 0; off < len(out); off += frameChunk {
|
||||
end := off + frameChunk
|
||||
if end > len(out) {
|
||||
end = len(out)
|
||||
}
|
||||
results <- floatToPCM16LE(out[off:end])
|
||||
}
|
||||
return nil
|
||||
}
|
||||
52
backend/go/moss-tts-cpp/main.go
Normal file
52
backend/go/moss-tts-cpp/main.go
Normal file
@@ -0,0 +1,52 @@
|
||||
package main
|
||||
|
||||
// Note: this is started internally by LocalAI and a server is allocated for each model
|
||||
import (
|
||||
"flag"
|
||||
"os"
|
||||
"runtime"
|
||||
|
||||
"github.com/ebitengine/purego"
|
||||
grpc "github.com/mudler/LocalAI/pkg/grpc"
|
||||
)
|
||||
|
||||
var (
|
||||
addr = flag.String("addr", "localhost:50051", "the address to connect to")
|
||||
)
|
||||
|
||||
type LibFuncs struct {
|
||||
FuncPtr any
|
||||
Name string
|
||||
}
|
||||
|
||||
func main() {
|
||||
libName := os.Getenv("MOSSTTS_LIBRARY")
|
||||
if libName == "" {
|
||||
if runtime.GOOS == "darwin" {
|
||||
libName = "./libgomosstts-cpp-fallback.dylib"
|
||||
} else {
|
||||
libName = "./libgomosstts-cpp-fallback.so"
|
||||
}
|
||||
}
|
||||
|
||||
lib, err := purego.Dlopen(libName, purego.RTLD_NOW|purego.RTLD_GLOBAL)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
libFuncs := []LibFuncs{
|
||||
{&CppLoad, "mtl_load"},
|
||||
{&CppTTS, "mtl_tts"},
|
||||
{&CppPCMFree, "mtl_pcm_free"},
|
||||
{&CppUnload, "mtl_unload"},
|
||||
}
|
||||
for _, lf := range libFuncs {
|
||||
purego.RegisterLibFunc(lf.FuncPtr, lib, lf.Name)
|
||||
}
|
||||
|
||||
flag.Parse()
|
||||
|
||||
if err := grpc.StartServer(*addr, &MossTtsCpp{}); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
124
backend/go/moss-tts-cpp/mossttscpp_test.go
Normal file
124
backend/go/moss-tts-cpp/mossttscpp_test.go
Normal file
@@ -0,0 +1,124 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/binary"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
func TestMossTtsCpp(t *testing.T) {
|
||||
RegisterFailHandler(Fail)
|
||||
RunSpecs(t, "moss-tts-cpp suite")
|
||||
}
|
||||
|
||||
var _ = Describe("resolveVoice", func() {
|
||||
It("treats an audio path as a clone reference (case-insensitive ext)", func() {
|
||||
Expect(resolveVoice("/x/ref.WAV")).To(Equal("/x/ref.WAV"))
|
||||
})
|
||||
It("recognizes mp3/flac/ogg/m4a", func() {
|
||||
for _, p := range []string{"a.mp3", "b.flac", "c.ogg", "d.m4a"} {
|
||||
Expect(resolveVoice(p)).To(Equal(p))
|
||||
}
|
||||
})
|
||||
It("ignores a bare token (no named speakers in MOSS-TTS-Local)", func() {
|
||||
Expect(resolveVoice("serena")).To(BeEmpty())
|
||||
})
|
||||
It("returns empty for empty input", func() {
|
||||
Expect(resolveVoice(" ")).To(BeEmpty())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("parseOptions", func() {
|
||||
It("extracts codec, tokenizer, seed", func() {
|
||||
o := parseOptions([]string{
|
||||
"codec:codec.gguf", "tokenizer:tok.gguf", "seed:7", "unknown:ignored",
|
||||
})
|
||||
Expect(o.codecPath).To(Equal("codec.gguf"))
|
||||
Expect(o.tokenizerPath).To(Equal("tok.gguf"))
|
||||
Expect(o.seed).To(Equal(7))
|
||||
})
|
||||
It("accepts audio_tokenizer / text_tokenizer aliases", func() {
|
||||
o := parseOptions([]string{"audio_tokenizer:c.gguf", "text_tokenizer:t.gguf"})
|
||||
Expect(o.codecPath).To(Equal("c.gguf"))
|
||||
Expect(o.tokenizerPath).To(Equal("t.gguf"))
|
||||
})
|
||||
It("defaults seed -1", func() {
|
||||
Expect(parseOptions(nil).seed).To(Equal(-1))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("isAudioCodecName", func() {
|
||||
DescribeTable("distinguishes the audio codec from the text tokenizer",
|
||||
func(name string, want bool) {
|
||||
Expect(isAudioCodecName(name)).To(Equal(want))
|
||||
},
|
||||
Entry("audio tokenizer is the codec", "moss-audio-tokenizer-v2-f32.gguf", true),
|
||||
Entry("codec keyword", "some-codec.gguf", true),
|
||||
Entry("text tokenizer is not the codec", "moss-tokenizer-v1_5.gguf", false),
|
||||
Entry("model is not the codec", "moss-tts-local-v1_5-q8_0.gguf", false),
|
||||
)
|
||||
})
|
||||
|
||||
var _ = Describe("discoverCodec / discoverTokenizer", func() {
|
||||
var dir, model, codec, tok string
|
||||
BeforeEach(func() {
|
||||
dir = GinkgoT().TempDir()
|
||||
model = filepath.Join(dir, "moss-tts-local-v1_5-q8_0.gguf")
|
||||
codec = filepath.Join(dir, "moss-audio-tokenizer-v2-f32.gguf")
|
||||
tok = filepath.Join(dir, "moss-tokenizer-v1_5.gguf")
|
||||
for _, p := range []string{model, codec, tok} {
|
||||
Expect(os.WriteFile(p, []byte("x"), 0o644)).To(Succeed())
|
||||
}
|
||||
})
|
||||
It("finds the audio codec, excluding the model", func() {
|
||||
Expect(discoverCodec(dir, model)).To(Equal(codec))
|
||||
})
|
||||
It("finds the text tokenizer, excluding model and codec", func() {
|
||||
Expect(discoverTokenizer(dir, model, codec)).To(Equal(tok))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("wavHeaderStereo", func() {
|
||||
It("emits a 44-byte streaming WAV header at 48 kHz stereo 16-bit", func() {
|
||||
h := wavHeaderStereo(48000)
|
||||
Expect(h).To(HaveLen(44))
|
||||
Expect(string(h[0:4])).To(Equal("RIFF"))
|
||||
Expect(string(h[8:12])).To(Equal("WAVE"))
|
||||
Expect(string(h[12:16])).To(Equal("fmt "))
|
||||
Expect(string(h[36:40])).To(Equal("data"))
|
||||
var channels uint16
|
||||
Expect(binary.Read(bytes.NewReader(h[22:24]), binary.LittleEndian, &channels)).To(Succeed())
|
||||
Expect(channels).To(Equal(uint16(2)))
|
||||
var sampleRate uint32
|
||||
Expect(binary.Read(bytes.NewReader(h[24:28]), binary.LittleEndian, &sampleRate)).To(Succeed())
|
||||
Expect(sampleRate).To(Equal(uint32(48000)))
|
||||
})
|
||||
It("falls back to 48 kHz for a non-positive sample rate", func() {
|
||||
var sampleRate uint32
|
||||
h := wavHeaderStereo(0)
|
||||
Expect(binary.Read(bytes.NewReader(h[24:28]), binary.LittleEndian, &sampleRate)).To(Succeed())
|
||||
Expect(sampleRate).To(Equal(uint32(48000)))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("floatToPCM16LE", func() {
|
||||
It("clamps and converts float PCM to little-endian int16 bytes", func() {
|
||||
b := floatToPCM16LE([]float32{0, 1.0, -1.0, 2.0, -2.0})
|
||||
Expect(b).To(HaveLen(10))
|
||||
read := func(off int) int16 {
|
||||
var v int16
|
||||
_ = binary.Read(bytes.NewReader(b[off:off+2]), binary.LittleEndian, &v)
|
||||
return v
|
||||
}
|
||||
Expect(read(0)).To(Equal(int16(0)))
|
||||
Expect(read(2)).To(Equal(int16(32767)))
|
||||
Expect(read(4)).To(Equal(int16(-32767)))
|
||||
Expect(read(6)).To(Equal(int16(32767))) // clamped from 2.0
|
||||
Expect(read(8)).To(Equal(int16(-32767))) // clamped from -2.0
|
||||
})
|
||||
})
|
||||
78
backend/go/moss-tts-cpp/options.go
Normal file
78
backend/go/moss-tts-cpp/options.go
Normal file
@@ -0,0 +1,78 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// loadOptions holds the parsed model-level options. MOSS-TTS-Local needs three
|
||||
// GGUFs (local transformer, audio codec, text tokenizer); the model path is the
|
||||
// local transformer and codec/tokenizer are resolved as siblings or via these
|
||||
// options.
|
||||
type loadOptions struct {
|
||||
codecPath string
|
||||
tokenizerPath string
|
||||
seed int
|
||||
}
|
||||
|
||||
func splitOption(o string) (key, value string, ok bool) {
|
||||
i := strings.Index(o, ":")
|
||||
if i < 0 {
|
||||
return "", "", false
|
||||
}
|
||||
return strings.TrimSpace(o[:i]), strings.TrimSpace(o[i+1:]), true
|
||||
}
|
||||
|
||||
// parseOptions reads the backend "key:value" option slice. Unknown keys are
|
||||
// ignored. Default seed is -1 (engine random).
|
||||
func parseOptions(opts []string) loadOptions {
|
||||
o := loadOptions{seed: -1}
|
||||
for _, oo := range opts {
|
||||
key, value, ok := splitOption(oo)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
switch key {
|
||||
case "codec", "audio_tokenizer":
|
||||
o.codecPath = value
|
||||
case "tokenizer", "text_tokenizer":
|
||||
o.tokenizerPath = value
|
||||
case "seed":
|
||||
if n, err := strconv.Atoi(value); err == nil {
|
||||
o.seed = n
|
||||
}
|
||||
}
|
||||
}
|
||||
return o
|
||||
}
|
||||
|
||||
var refAudioExts = []string{".wav", ".flac", ".mp3", ".ogg", ".m4a"}
|
||||
|
||||
// resolveVoice interprets the request Voice field. MOSS-TTS-Local has no named
|
||||
// speakers, only reference-audio cloning, so a value ending in a known audio
|
||||
// extension is treated as a clone-reference path and anything else is ignored
|
||||
// (the caller then falls back to the config audio_path).
|
||||
func resolveVoice(voice string) (refPath string) {
|
||||
v := strings.TrimSpace(voice)
|
||||
if v == "" {
|
||||
return ""
|
||||
}
|
||||
lower := strings.ToLower(v)
|
||||
for _, ext := range refAudioExts {
|
||||
if strings.HasSuffix(lower, ext) {
|
||||
return v
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func parseInt(v string, def int) int {
|
||||
if v == "" {
|
||||
return def
|
||||
}
|
||||
n, err := strconv.Atoi(v)
|
||||
if err != nil {
|
||||
return def
|
||||
}
|
||||
return n
|
||||
}
|
||||
65
backend/go/moss-tts-cpp/package.sh
Normal file
65
backend/go/moss-tts-cpp/package.sh
Normal file
@@ -0,0 +1,65 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Script to copy the appropriate libraries based on architecture
|
||||
# This script is used in the final stage of the Dockerfile
|
||||
|
||||
set -e
|
||||
|
||||
CURDIR=$(dirname "$(realpath $0)")
|
||||
REPO_ROOT="${CURDIR}/../../.."
|
||||
|
||||
# Create lib directory
|
||||
mkdir -p $CURDIR/package/lib
|
||||
|
||||
cp -avf $CURDIR/moss-tts-cpp $CURDIR/package/
|
||||
cp -fv $CURDIR/libgomosstts-cpp-*.so $CURDIR/package/ 2>/dev/null || true
|
||||
cp -fv $CURDIR/libgomosstts-cpp-*.dylib $CURDIR/package/ 2>/dev/null || true
|
||||
cp -fv $CURDIR/run.sh $CURDIR/package/
|
||||
|
||||
# Detect architecture and copy appropriate libraries
|
||||
if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
|
||||
# x86_64 architecture
|
||||
echo "Detected x86_64 architecture, copying x86_64 libraries..."
|
||||
cp -arfLv /lib64/ld-linux-x86-64.so.2 $CURDIR/package/lib/ld.so
|
||||
cp -arfLv /lib/x86_64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
|
||||
cp -arfLv /lib/x86_64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
|
||||
cp -arfLv /lib/x86_64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
|
||||
cp -arfLv /lib/x86_64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
|
||||
cp -arfLv /lib/x86_64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
|
||||
cp -arfLv /lib/x86_64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
|
||||
cp -arfLv /lib/x86_64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
|
||||
cp -arfLv /lib/x86_64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
|
||||
cp -arfLv /lib/x86_64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
|
||||
cp -arfLv /lib/x86_64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
|
||||
elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
|
||||
# ARM64 architecture
|
||||
echo "Detected ARM64 architecture, copying ARM64 libraries..."
|
||||
cp -arfLv /lib/ld-linux-aarch64.so.1 $CURDIR/package/lib/ld.so
|
||||
cp -arfLv /lib/aarch64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
|
||||
cp -arfLv /lib/aarch64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
|
||||
cp -arfLv /lib/aarch64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
|
||||
cp -arfLv /lib/aarch64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
|
||||
cp -arfLv /lib/aarch64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
|
||||
cp -arfLv /lib/aarch64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
|
||||
cp -arfLv /lib/aarch64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
|
||||
cp -arfLv /lib/aarch64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
|
||||
cp -arfLv /lib/aarch64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
|
||||
cp -arfLv /lib/aarch64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
|
||||
elif [ $(uname -s) = "Darwin" ]; then
|
||||
echo "Detected Darwin"
|
||||
else
|
||||
echo "Error: Could not detect architecture"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Package GPU libraries based on BUILD_TYPE
|
||||
GPU_LIB_SCRIPT="${REPO_ROOT}/scripts/build/package-gpu-libs.sh"
|
||||
if [ -f "$GPU_LIB_SCRIPT" ]; then
|
||||
echo "Packaging GPU libraries for BUILD_TYPE=${BUILD_TYPE:-cpu}..."
|
||||
source "$GPU_LIB_SCRIPT" "$CURDIR/package/lib"
|
||||
package_gpu_libs
|
||||
fi
|
||||
|
||||
echo "Packaging completed successfully"
|
||||
ls -liah $CURDIR/package/
|
||||
ls -liah $CURDIR/package/lib/
|
||||
57
backend/go/moss-tts-cpp/run.sh
Normal file
57
backend/go/moss-tts-cpp/run.sh
Normal file
@@ -0,0 +1,57 @@
|
||||
#!/bin/bash
|
||||
set -ex
|
||||
|
||||
# Get the absolute current dir where the script is located
|
||||
CURDIR=$(dirname "$(realpath "$0")")
|
||||
|
||||
cd /
|
||||
|
||||
echo "CPU info:"
|
||||
if [ "$(uname)" != "Darwin" ]; then
|
||||
grep -e "model\sname" /proc/cpuinfo | head -1
|
||||
grep -e "flags" /proc/cpuinfo | head -1
|
||||
fi
|
||||
|
||||
if [ "$(uname)" = "Darwin" ]; then
|
||||
# macOS: single dylib variant (Metal or Accelerate)
|
||||
LIBRARY="$CURDIR/libgomosstts-cpp-fallback.dylib"
|
||||
export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
|
||||
else
|
||||
LIBRARY="$CURDIR/libgomosstts-cpp-fallback.so"
|
||||
|
||||
if grep -q -e "\savx\s" /proc/cpuinfo ; then
|
||||
echo "CPU: AVX found OK"
|
||||
if [ -e "$CURDIR"/libgomosstts-cpp-avx.so ]; then
|
||||
LIBRARY="$CURDIR/libgomosstts-cpp-avx.so"
|
||||
fi
|
||||
fi
|
||||
|
||||
if grep -q -e "\savx2\s" /proc/cpuinfo ; then
|
||||
echo "CPU: AVX2 found OK"
|
||||
if [ -e "$CURDIR"/libgomosstts-cpp-avx2.so ]; then
|
||||
LIBRARY="$CURDIR/libgomosstts-cpp-avx2.so"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check avx 512
|
||||
if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
|
||||
echo "CPU: AVX512F found OK"
|
||||
if [ -e "$CURDIR"/libgomosstts-cpp-avx512.so ]; then
|
||||
LIBRARY="$CURDIR/libgomosstts-cpp-avx512.so"
|
||||
fi
|
||||
fi
|
||||
|
||||
export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
|
||||
fi
|
||||
|
||||
export MOSSTTS_LIBRARY=$LIBRARY
|
||||
|
||||
# If there is a lib/ld.so, use it
|
||||
if [ -f "$CURDIR"/lib/ld.so ]; then
|
||||
echo "Using lib/ld.so"
|
||||
echo "Using library: $LIBRARY"
|
||||
exec "$CURDIR"/lib/ld.so "$CURDIR"/moss-tts-cpp "$@"
|
||||
fi
|
||||
|
||||
echo "Using library: $LIBRARY"
|
||||
exec "$CURDIR"/moss-tts-cpp "$@"
|
||||
33
backend/go/moss-tts-cpp/test.sh
Normal file
33
backend/go/moss-tts-cpp/test.sh
Normal file
@@ -0,0 +1,33 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
CURDIR=$(dirname "$(realpath $0)")
|
||||
cd "$CURDIR"
|
||||
|
||||
echo "Running moss-tts-cpp backend tests..."
|
||||
|
||||
# Auto-download the Local v1.5 model + codec + tokenizer only when
|
||||
# MOSSTTS_MODEL is not set.
|
||||
if [ -z "$MOSSTTS_MODEL" ]; then
|
||||
MODEL_DIR="./moss-tts-models"
|
||||
mkdir -p "$MODEL_DIR"
|
||||
REPO_ID="mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF"
|
||||
BASE_URL="https://huggingface.co/${REPO_ID}/resolve/main"
|
||||
FILES=( "moss-tts-local-v1_5-q8_0.gguf" "moss-audio-tokenizer-v2-f32.gguf" "moss-tokenizer-v1_5.gguf" )
|
||||
for file in "${FILES[@]}"; do
|
||||
dest="${MODEL_DIR}/${file}"
|
||||
if [ -f "${dest}" ]; then
|
||||
echo " [skip] ${file}"
|
||||
else
|
||||
echo " [download] ${file}..."
|
||||
curl -L -o "${dest}" "${BASE_URL}/${file}" --progress-bar
|
||||
fi
|
||||
done
|
||||
export MOSSTTS_MODEL="${MODEL_DIR}/moss-tts-local-v1_5-q8_0.gguf"
|
||||
export MOSSTTS_CODEC="${MODEL_DIR}/moss-audio-tokenizer-v2-f32.gguf"
|
||||
export MOSSTTS_TOKENIZER="${MODEL_DIR}/moss-tokenizer-v1_5.gguf"
|
||||
fi
|
||||
|
||||
go test -v -timeout 1200s .
|
||||
|
||||
echo "All moss-tts-cpp tests passed."
|
||||
@@ -1129,6 +1129,48 @@
|
||||
nvidia-l4t: "nvidia-l4t-arm64-qwen3-tts-cpp"
|
||||
nvidia-l4t-cuda-12: "nvidia-l4t-arm64-qwen3-tts-cpp"
|
||||
nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-qwen3-tts-cpp"
|
||||
- &mossttscpp
|
||||
name: "moss-tts-cpp"
|
||||
description: |
|
||||
MOSS-TTS C++ backend using GGML (moss-tts.cpp). Native C++ text-to-speech
|
||||
for the OpenMOSS MOSS-TTS-Local Transformer v1.5 model (a GPT-J local
|
||||
transformer + MOSS-Audio-Tokenizer-v2 neural codec), running from a
|
||||
self-contained GGUF with no Python at inference time. 48kHz stereo output
|
||||
with reference-audio voice cloning.
|
||||
urls:
|
||||
- https://github.com/mudler/moss-tts.cpp
|
||||
- https://huggingface.co/mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF
|
||||
tags:
|
||||
- text-to-speech
|
||||
- tts
|
||||
- voice-cloning
|
||||
alias: "moss-tts-cpp"
|
||||
capabilities:
|
||||
default: "cpu-moss-tts-cpp"
|
||||
nvidia: "cuda12-moss-tts-cpp"
|
||||
nvidia-cuda-13: "cuda13-moss-tts-cpp"
|
||||
nvidia-cuda-12: "cuda12-moss-tts-cpp"
|
||||
intel: "intel-sycl-f16-moss-tts-cpp"
|
||||
metal: "metal-moss-tts-cpp"
|
||||
amd: "rocm-moss-tts-cpp"
|
||||
vulkan: "vulkan-moss-tts-cpp"
|
||||
nvidia-l4t: "nvidia-l4t-arm64-moss-tts-cpp"
|
||||
nvidia-l4t-cuda-12: "nvidia-l4t-arm64-moss-tts-cpp"
|
||||
nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-moss-tts-cpp"
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "moss-tts-cpp-development"
|
||||
capabilities:
|
||||
default: "cpu-moss-tts-cpp-development"
|
||||
nvidia: "cuda12-moss-tts-cpp-development"
|
||||
nvidia-cuda-13: "cuda13-moss-tts-cpp-development"
|
||||
nvidia-cuda-12: "cuda12-moss-tts-cpp-development"
|
||||
intel: "intel-sycl-f16-moss-tts-cpp-development"
|
||||
metal: "metal-moss-tts-cpp-development"
|
||||
amd: "rocm-moss-tts-cpp-development"
|
||||
vulkan: "vulkan-moss-tts-cpp-development"
|
||||
nvidia-l4t: "nvidia-l4t-arm64-moss-tts-cpp-development"
|
||||
nvidia-l4t-cuda-12: "nvidia-l4t-arm64-moss-tts-cpp-development"
|
||||
nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-moss-tts-cpp-development"
|
||||
- &omnivoicecpp
|
||||
name: "omnivoice-cpp"
|
||||
description: |
|
||||
@@ -4611,6 +4653,107 @@
|
||||
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-nvidia-cuda-13-qwen3-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:master-gpu-nvidia-cuda-13-qwen3-tts-cpp
|
||||
## moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "nvidia-l4t-arm64-moss-tts-cpp"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:latest-nvidia-l4t-arm64-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:latest-nvidia-l4t-arm64-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "nvidia-l4t-arm64-moss-tts-cpp-development"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:master-nvidia-l4t-arm64-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:master-nvidia-l4t-arm64-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "cuda13-nvidia-l4t-arm64-moss-tts-cpp"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:latest-nvidia-l4t-cuda-13-arm64-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:latest-nvidia-l4t-cuda-13-arm64-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "cuda13-nvidia-l4t-arm64-moss-tts-cpp-development"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:master-nvidia-l4t-cuda-13-arm64-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:master-nvidia-l4t-cuda-13-arm64-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "cpu-moss-tts-cpp"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:latest-cpu-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:latest-cpu-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "metal-moss-tts-cpp"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:latest-metal-darwin-arm64-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:latest-metal-darwin-arm64-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "metal-moss-tts-cpp-development"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:master-metal-darwin-arm64-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:master-metal-darwin-arm64-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "cpu-moss-tts-cpp-development"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:master-cpu-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:master-cpu-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "cuda12-moss-tts-cpp"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:latest-gpu-nvidia-cuda-12-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "rocm-moss-tts-cpp"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-rocm-hipblas-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:latest-gpu-rocm-hipblas-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "intel-sycl-f32-moss-tts-cpp"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-intel-sycl-f32-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:latest-gpu-intel-sycl-f32-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "intel-sycl-f16-moss-tts-cpp"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-intel-sycl-f16-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:latest-gpu-intel-sycl-f16-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "vulkan-moss-tts-cpp"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-vulkan-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:latest-gpu-vulkan-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "vulkan-moss-tts-cpp-development"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-vulkan-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:master-gpu-vulkan-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "cuda12-moss-tts-cpp-development"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-nvidia-cuda-12-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:master-gpu-nvidia-cuda-12-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "rocm-moss-tts-cpp-development"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-rocm-hipblas-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:master-gpu-rocm-hipblas-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "intel-sycl-f32-moss-tts-cpp-development"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-intel-sycl-f32-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:master-gpu-intel-sycl-f32-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "intel-sycl-f16-moss-tts-cpp-development"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-intel-sycl-f16-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:master-gpu-intel-sycl-f16-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "cuda13-moss-tts-cpp"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-13-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:latest-gpu-nvidia-cuda-13-moss-tts-cpp
|
||||
- !!merge <<: *mossttscpp
|
||||
name: "cuda13-moss-tts-cpp-development"
|
||||
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-nvidia-cuda-13-moss-tts-cpp"
|
||||
mirrors:
|
||||
- localai/localai-backends:master-gpu-nvidia-cuda-13-moss-tts-cpp
|
||||
## omnivoice-cpp
|
||||
- !!merge <<: *omnivoicecpp
|
||||
name: "omnivoice-cpp-development"
|
||||
|
||||
@@ -127,7 +127,7 @@ When a saved profile is selected, LocalAI supplies both its private WAV and exac
|
||||
|
||||
| Backend | Automatically compatible variants |
|
||||
| --- | --- |
|
||||
| `chatterbox`, `faster-qwen3-tts`, `fish-speech`, `neutts`, `omnivoice-cpp`, `pocket-tts`, `voxcpm` | Reference-audio cloning models served by these dedicated backends. |
|
||||
| `chatterbox`, `faster-qwen3-tts`, `fish-speech`, `moss-tts-cpp`, `neutts`, `omnivoice-cpp`, `pocket-tts`, `voxcpm` | Reference-audio cloning models served by these dedicated backends. |
|
||||
| `qwen-tts`, `qwen3-tts-cpp`, `vllm-omni` | Base or VoiceClone variants. CustomVoice and VoiceDesign variants are not raw reference-audio models. |
|
||||
| `vibevoice-cpp` | 1.5B reference-WAV variants. The realtime 0.5B preset-prompt model is excluded. |
|
||||
| `coqui` | XTTS and YourTTS variants. |
|
||||
|
||||
@@ -59,6 +59,7 @@ All backends listed here can be installed on demand from the [Backend Gallery]({
|
||||
| [Chatterbox](https://github.com/resemble-ai/chatterbox) | Production-grade TTS with emotion control | CPU, CUDA 12/13, Metal, Jetson L4T |
|
||||
| [VibeVoice](https://github.com/microsoft/VibeVoice) | Real-time TTS with voice cloning | CPU, CUDA 12/13, ROCm, Intel SYCL, Metal, Jetson L4T |
|
||||
| [vibevoice.cpp](https://github.com/mudler/vibevoice.cpp) | Native C++/GGML port of VibeVoice for TTS (voice cloning) and long-form ASR with diarization | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T |
|
||||
| [moss-tts.cpp](https://github.com/mudler/moss-tts.cpp) | Native C++/GGML port of OpenMOSS MOSS-TTS-Local v1.5: 48 kHz stereo TTS with reference-audio voice cloning | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T |
|
||||
| [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) | TTS with custom voice, voice design, and voice cloning | CPU, CUDA 12/13, ROCm, Intel SYCL, Metal, Jetson L4T |
|
||||
| [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp) | Native C++/GGML Qwen3-TTS with streaming, named speakers, and voice design | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T |
|
||||
| [OmniVoice](https://github.com/ServeurpersoCom/omnivoice.cpp) | Native C++/GGML TTS with voice cloning, voice design, and streaming | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T |
|
||||
|
||||
@@ -41,6 +41,14 @@ function inferBackendPath(item) {
|
||||
if (item.backend === "moss-transcribe-cpp") {
|
||||
return `backend/go/moss-transcribe-cpp/`;
|
||||
}
|
||||
// moss-tts-cpp is a Go backend (Dockerfile.golang) wrapping the moss-tts.cpp
|
||||
// ggml port via purego, living in backend/go/moss-tts-cpp/. Same
|
||||
// explicit-branch rationale as parakeet-cpp / ced / moss-transcribe-cpp: the
|
||||
// generic golang fallthrough would also resolve it, but this documents the
|
||||
// mapping and guards a future dockerfile-suffix change.
|
||||
if (item.backend === "moss-tts-cpp") {
|
||||
return `backend/go/moss-tts-cpp/`;
|
||||
}
|
||||
if (item.dockerfile.endsWith("golang")) {
|
||||
return `backend/go/${item.backend}/`;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user