Files
LocalAI/backend/go/qwen3-tts-cpp/Makefile
T
mudler-agentandEttore Di Giacinto f85087f992 feat(tts): stream Qwen text and add assistant preset (#12770)
* feat(tts): stream Qwen text through audio pipelines

Feed cleaned LLM deltas into one persistent Qwen synthesis session.
Return audio before input ends and preserve the live RPC through model
cache, staging and in-flight wrappers. Stop generation on sink failures
or cancellation without replaying speech through a fallback.

Pin the incremental native fork and verify early audio, late text,
finalization and cancellation with lifecycle and real-model checks.

Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* build(tts): use the organization Qwen fork

Fetch the reviewed incremental synthesis commit from localai-org so
backend builds use the organization-maintained fork.

Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): add a Qwen streaming assistant preset

Expose an English pipeline using existing VAD, ASR, LLM and named-voice
TTS models. Enable incremental text and audio delivery, and document
separate component installs because the gallery has no dependency
installer.

Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): use Parakeet TDT v3 for the assistant

Select the existing Parakeet TDT 0.6B v3 transcription entry and update
its prerequisite install command. Keep the preset's English voice and
incremental Qwen synthesis configuration.

Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(kokoros): reject unsupported backend RPCs

Implement the MotionStream and SlotCache methods required by the shared
protobuf service. Return Unimplemented so Kokoros compiles without
advertising support for either operation.

Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(tts): mark cleanup and logging results ignored

Explicitly ignore connection shutdown, test server shutdown and
Ginkgo diagnostic writer errors. These operations do not replace the
stream's inference or sink errors, and errcheck requires their return
values to be acknowledged.

Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(tts): stream buffered clauses in realtime pipelines

Start clause synthesis while LLM text is still arriving when both streaming flags are enabled. Keep explicit clause opt-out and native Qwen sessions, bound automatic queued work, and choose streaming support on resolved failover targets.

Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(tts): bound synthetic voice clone generation

The sine-wave reference checks speaker conditioning and WAV output, but does not provide a meaningful speech EOS target. Limit that smoke request to 128 frames and verify the request limit reaches native synthesis. Leave production sampling defaults unchanged.

Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(tts): require explicit clause buffering opt-in

Keep whole-reply synthesis for nonnative engines when clause_chunking is unset or false. Use the bounded cancellable worker when explicitly enabled, with unary clauses when TTS streaming is disabled. Preserve native Qwen precedence for every clause setting.

Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): add VITS and Piper streaming assistants

Provide comparable full pipeline configurations with shared VAD, Parakeet transcription, and Qwen chat components. Enable clause buffering explicitly and document the separate prerequisite model installs and Talk selection.

Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-10-19 12:30:52 +02:00

141 lines
4.9 KiB
Makefile

CMAKE_ARGS?=
BUILD_TYPE?=
NATIVE?=false
GOCMD?=go
GO_TAGS?=
JOBS?=$(shell nproc --ignore=1)
# Incremental text input uses the persistent-session ABI in this fork.
QWEN3TTS_REPO?=https://github.com/localai-org/qwentts.cpp
QWEN3TTS_CPP_VERSION?=7cdb17cdaf6ef88961365e9b5177a0284501beb9
SO_TARGET?=libgoqwen3ttscpp.so
CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF
ifeq ($(NATIVE),false)
CMAKE_ARGS+=-DGGML_NATIVE=OFF
endif
ifeq ($(BUILD_TYPE),cublas)
CMAKE_ARGS+=-DGGML_CUDA=ON
else ifeq ($(BUILD_TYPE),openblas)
CMAKE_ARGS+=-DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS
else ifeq ($(BUILD_TYPE),clblas)
CMAKE_ARGS+=-DGGML_CLBLAST=ON -DCLBlast_DIR=/some/path
else ifeq ($(BUILD_TYPE),hipblas)
# This ggml only understands GGML_HIP (GGML_HIPBLAS was removed upstream),
# so passing GGML_HIPBLAS silently produced a CPU-only build (see #10666).
ROCM_HOME ?= /opt/rocm
ROCM_PATH ?= /opt/rocm
export CXX=$(ROCM_HOME)/llvm/bin/clang++
export CC=$(ROCM_HOME)/llvm/bin/clang
AMDGPU_TARGETS ?= gfx908,gfx90a,gfx942,gfx950,gfx1030,gfx1100,gfx1101,gfx1102,gfx1151,gfx1200,gfx1201
CMAKE_ARGS+=-DGGML_HIP=ON -DAMDGPU_TARGETS=$(AMDGPU_TARGETS)
else ifeq ($(BUILD_TYPE),vulkan)
CMAKE_ARGS+=-DGGML_VULKAN=ON
else ifeq ($(OS),Darwin)
ifneq ($(BUILD_TYPE),metal)
CMAKE_ARGS+=-DGGML_METAL=OFF
else
CMAKE_ARGS+=-DGGML_METAL=ON
CMAKE_ARGS+=-DGGML_METAL_EMBED_LIBRARY=ON
endif
endif
ifeq ($(BUILD_TYPE),sycl_f16)
CMAKE_ARGS+=-DGGML_SYCL=ON \
-DCMAKE_C_COMPILER=icx \
-DCMAKE_CXX_COMPILER=icpx \
-DGGML_SYCL_F16=ON
endif
ifeq ($(BUILD_TYPE),sycl_f32)
CMAKE_ARGS+=-DGGML_SYCL=ON \
-DCMAKE_C_COMPILER=icx \
-DCMAKE_CXX_COMPILER=icpx
endif
sources/qwentts.cpp:
mkdir -p sources/qwentts.cpp
cd sources/qwentts.cpp && \
git init && \
git remote add origin $(QWEN3TTS_REPO) && \
git fetch origin && \
git checkout $(QWEN3TTS_CPP_VERSION) && \
git submodule update --init --recursive --depth 1 --single-branch
# Detect OS
UNAME_S := $(shell uname -s)
# Only build CPU variants on Linux
ifeq ($(UNAME_S),Linux)
VARIANT_TARGETS = libgoqwen3ttscpp-avx.so libgoqwen3ttscpp-avx2.so libgoqwen3ttscpp-avx512.so libgoqwen3ttscpp-fallback.so
else
# On non-Linux (e.g., Darwin), build only fallback variant (as a dylib)
VARIANT_TARGETS = libgoqwen3ttscpp-fallback.dylib
endif
qwen3-tts-cpp: main.go goqwen3ttscpp.go $(VARIANT_TARGETS)
CGO_ENABLED=0 $(GOCMD) build -tags "$(GO_TAGS)" -o qwen3-tts-cpp ./
package: qwen3-tts-cpp
bash package.sh
build: package
clean: purge
rm -rf libgoqwen3ttscpp*.so libgoqwen3ttscpp*.dylib package sources/qwentts.cpp qwen3-tts-cpp
purge:
rm -rf build*
# Variants must build sequentially
.NOTPARALLEL:
# Build all variants (Linux only)
ifeq ($(UNAME_S),Linux)
libgoqwen3ttscpp-avx.so: sources/qwentts.cpp
$(info ${GREEN}I qwen3-tts-cpp build info:avx${RESET})
SO_TARGET=libgoqwen3ttscpp-avx.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=on -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_BMI2=off" $(MAKE) libgoqwen3ttscpp-custom
rm -rf build-libgoqwen3ttscpp-avx.so
libgoqwen3ttscpp-avx2.so: sources/qwentts.cpp
$(info ${GREEN}I qwen3-tts-cpp build info:avx2${RESET})
SO_TARGET=libgoqwen3ttscpp-avx2.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=on -DGGML_AVX2=on -DGGML_AVX512=off -DGGML_FMA=on -DGGML_F16C=on -DGGML_BMI2=on" $(MAKE) libgoqwen3ttscpp-custom
rm -rf build-libgoqwen3ttscpp-avx2.so
libgoqwen3ttscpp-avx512.so: sources/qwentts.cpp
$(info ${GREEN}I qwen3-tts-cpp build info:avx512${RESET})
SO_TARGET=libgoqwen3ttscpp-avx512.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=on -DGGML_AVX2=on -DGGML_AVX512=on -DGGML_FMA=on -DGGML_F16C=on -DGGML_BMI2=on" $(MAKE) libgoqwen3ttscpp-custom
rm -rf build-libgoqwen3ttscpp-avx512.so
endif
# Build fallback variant (all platforms)
libgoqwen3ttscpp-fallback.so: sources/qwentts.cpp
$(info ${GREEN}I qwen3-tts-cpp build info:fallback${RESET})
SO_TARGET=libgoqwen3ttscpp-fallback.so CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=off -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_BMI2=off" $(MAKE) libgoqwen3ttscpp-custom
rm -rf build-libgoqwen3ttscpp-fallback.so
# Build fallback variant as a dylib (Darwin)
libgoqwen3ttscpp-fallback.dylib: sources/qwentts.cpp
$(info ${GREEN}I qwen3-tts-cpp build info:fallback (dylib)${RESET})
SO_TARGET=libgoqwen3ttscpp-fallback.dylib CMAKE_ARGS="$(CMAKE_ARGS) -DGGML_AVX=off -DGGML_AVX2=off -DGGML_AVX512=off -DGGML_FMA=off -DGGML_F16C=off -DGGML_BMI2=off" $(MAKE) libgoqwen3ttscpp-custom
rm -rf build-libgoqwen3ttscpp-fallback.dylib
libgoqwen3ttscpp-custom: CMakeLists.txt cpp/goqwen3ttscpp.cpp cpp/goqwen3ttscpp.h
mkdir -p build-$(SO_TARGET) && \
cd build-$(SO_TARGET) && \
cmake .. $(CMAKE_ARGS) && \
cmake --build . --config Release -j$(JOBS) --target goqwen3ttscpp && \
cd .. && \
(mv build-$(SO_TARGET)/libgoqwen3ttscpp.so ./$(SO_TARGET) 2>/dev/null || \
mv build-$(SO_TARGET)/libgoqwen3ttscpp.dylib ./$(SO_TARGET) 2>/dev/null)
test: qwen3-tts-cpp
@echo "Running qwen3-tts-cpp tests..."
bash test.sh
@echo "qwen3-tts-cpp tests completed."
all: qwen3-tts-cpp package