mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-13 09:44:26 -04:00
13734ae9fa
feat(backend): Add Sherpa ONNX backend and Omnilingual ASR Adds a new Go backend wrapping sherpa-onnx via purego (no cgo). Same approach as opus/stablediffusion-ggml/whisper — a thin C shim (csrc/shim.c + shim.h → libsherpa-shim.so) wraps the bits purego can't reach directly: nested struct config writes, result-struct field reads, and the streaming TTS callback trampoline. The Go side uses opaque uintptr handles and purego.NewCallback for the TTS callback. Supports: - VAD via sherpa-onnx's Silero VAD - Offline ASR: Whisper, Paraformer, SenseVoice, Omnilingual CTC - Online/streaming ASR: zipformer transducer with endpoint detection (AudioTranscriptionStream emits delta events during decode) - Offline TTS: VITS (LJS, etc.) - Streaming TTS: sherpa-onnx's callback API → PCM chunks on a channel, prefixed by a streaming WAV header Gallery entries: omnilingual-0.3b-ctc-q8-sherpa (1600-language offline ASR), streaming-zipformer-en-sherpa (low-latency streaming ASR), silero-vad-sherpa, vits-ljs-sherpa. E2E coverage: tests/e2e-backends for offline + streaming ASR, tests/e2e for the full realtime pipeline (VAD + STT + TTS). Assisted-by: claude-opus-4-7-1M [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com>
18 lines
455 B
YAML
18 lines
455 B
YAML
---
|
|
name: "sherpa-onnx-vad"
|
|
|
|
config_file: |
|
|
backend: sherpa-onnx
|
|
type: vad
|
|
options:
|
|
# Silero VAD. Defaults mirror upstream sherpa-onnx. Override for
|
|
# faster turn-taking (lower min_silence) or different sample rate
|
|
# derivatives (8 kHz Silero variants).
|
|
- vad.threshold=0.5
|
|
- vad.min_silence=0.5
|
|
- vad.min_speech=0.25
|
|
- vad.window_size=512
|
|
- vad.max_speech=20.0
|
|
- vad.sample_rate=16000
|
|
- vad.buffer_size=60.0
|