mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-05-16 12:38:01 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat: add LocalVQE backend and audio transformations UI (#9640)

Browse Source 
feat(audio-transform): add LocalVQE backend, bidi gRPC RPC, Studio UI

Introduce a generic "audio transform" capability for any audio-in / audio-out
operation (echo cancellation, noise suppression, dereverberation, voice
conversion, etc.) and ship LocalVQE as the first backend implementation.

Backend protocol:
- Two new gRPC RPCs in backend.proto: unary AudioTransform for batch and
  bidirectional AudioTransformStream for low-latency frame-by-frame use.
  This is the first bidi stream in the proto; per-frame unary at LocalVQE's
  16 ms hop would be RTT-bound. Wire it through pkg/grpc/{client,server,
  embed,interface,base} with paired-channel ergonomics.

LocalVQE backend (backend/go/localvqe/):
- Go-Purego wrapper around upstream liblocalvqe.so. CMake builds the upstream
  shared lib + its libggml-cpu-*.so runtime variants directly — no MODULE
  wrapper needed because LocalVQE handles CPU feature selection internally
  via GGML_BACKEND_DL.
- Sets GGML_NTHREADS from opts.Threads (or runtime.NumCPU()-1) — without it
  LocalVQE runs single-threaded at ~1× realtime instead of the documented
  ~9.6×.
- Reference-length policy: zero-pad short refs, truncate long ones (the
  trailing portion can't have leaked into a mic that wasn't recording).
- Ginkgo test suite (9 always-on specs + 2 model-gated).

HTTP layer:
- POST /audio/transformations (alias /audio/transform): multipart batch
  endpoint, accepts audio + optional reference + params[*]=v form fields.
  Persists inputs alongside the output in GeneratedContentDir/audio so the
  React UI history can replay past (audio, reference, output) triples.
- GET /audio/transformations/stream: WebSocket bidi, 16 ms PCM frames
  (interleaved stereo mic+ref in, mono out). JSON session.update envelope
  for config; constants hoisted in core/schema/audio_transform.go.
- ffmpeg-based input normalisation to 16 kHz mono s16 WAV via the existing
  utils.AudioToWav (with passthrough fast-path), so the user can upload any
  format / rate without seeing the model's strict 16 kHz constraint.
- BackendTraceAudioTransform integration so /api/backend-traces and the
  Traces UI light up with audio_snippet base64 and timing.
- Routes registered under routes/localai.go (LocalAI extension; OpenAI has
  no /audio/transformations endpoint), traced via TraceMiddleware.

Auth + capability + importer:
- FLAG_AUDIO_TRANSFORM (model_config.go), FeatureAudioTransform (default-on,
  in APIFeatures), three RouteFeatureRegistry rows.
- localvqe added to knownPrefOnlyBackends with modality "audio-transform".
- Gallery entry localvqe-v1-1.3m (sha256-pinned, hosted on
  huggingface.co/LocalAI-io/LocalVQE).

React UI:
- New /app/transform page surfaced via a dedicated "Enhance" sidebar
  section (sibling of Tools / Biometrics) — the page is enhancement, not
  generation, so it lives outside Studio. Two AudioInput components
  (Upload + Record tabs, drag-drop, mic capture).
- Echo-test button: records mic while playing the loaded reference through
  the speakers — the mic naturally picks up speaker bleed, giving a real
  (mic, ref) pair for AEC testing without leaving the UI.
- Reusable WaveformPlayer (canvas peaks + click-to-seek + audio controls)
  and useAudioPeaks hook (shared module-scoped AudioContext to avoid
  hitting browser context limits with three players on one page); migrated
  TTS, Sound, Traces audio blocks to use it.
- Past runs saved in localStorage via useMediaHistory('audio-transform') —
  the history entry stores all three URLs so clicking re-renders the full
  triple, not just the output.

Build + e2e:
- 11 matrix entries removed from .github/workflows/backend.yml (CUDA, ROCm,
  SYCL, Metal, L4T): upstream supports only CPU + Vulkan, so we ship those
  two and let GPU-class hardware route through Vulkan in the gallery
  capabilities map.
- tests-localvqe-grpc-transform job in test-extra.yml (gated on
  detect-changes.outputs.localvqe).
- New audio_transform capability + 4 specs in tests/e2e-backends.
- Playwright spec suite in core/http/react-ui/e2e/audio-transform.spec.js
  (8 specs covering tabs, file upload, multipart shape, history, errors).

Docs:
- New docs/content/features/audio-transform.md covering the (audio,
  reference) mental model, batch + WebSocket wire formats, LocalVQE param
  keys, and a YAML config example. Cross-links from text-to-audio and
  audio-to-text feature pages.

Assisted-by: Claude:claude-opus-4-7 [Bash Read Edit Write Agent TaskCreate]

Signed-off-by: Richard Palethorpe <io@richiejp.com>
This commit is contained in:
Richard Palethorpe
2026-05-04 21:07:11 +01:00
committed by GitHub
parent de83b72bb7
commit bb033b16a9
59 changed files with 3923 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
.github/workflows/backend.yml vendored
View File

@@ -2686,6 +2686,19 @@ jobs:
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: ''
cuda-major-version: ""
cuda-minor-version: ""
platforms: 'linux/amd64,linux/arm64'
tag-latest: 'auto'
tag-suffix: '-cpu-localvqe'
runs-on: 'ubuntu-latest'
base-image: "ubuntu:24.04"
skip-drivers: 'false'
backend: "localvqe"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'sycl_f32'
cuda-major-version: ""
cuda-minor-version: ""
@@ -2725,6 +2738,19 @@ jobs:
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'vulkan'
cuda-major-version: ""
cuda-minor-version: ""
platforms: 'linux/amd64,linux/arm64'
tag-latest: 'auto'
tag-suffix: '-gpu-vulkan-localvqe'
runs-on: 'ubuntu-latest'
base-image: "ubuntu:24.04"
skip-drivers: 'false'
backend: "localvqe"
dockerfile: "./backend/Dockerfile.golang"
context: "./"
ubuntu-version: '2404'
- build-type: 'cublas'
cuda-major-version: "12"
cuda-minor-version: "0"

21
.github/workflows/test-extra.yml vendored
View File

@@ -37,6 +37,7 @@ jobs:
acestep-cpp: ${{ steps.detect.outputs.acestep-cpp }}
qwen3-tts-cpp: ${{ steps.detect.outputs.qwen3-tts-cpp }}
vibevoice-cpp: ${{ steps.detect.outputs.vibevoice-cpp }}
localvqe: ${{ steps.detect.outputs.localvqe }}
voxtral: ${{ steps.detect.outputs.voxtral }}
kokoros: ${{ steps.detect.outputs.kokoros }}
insightface: ${{ steps.detect.outputs.insightface }}
@@ -884,6 +885,26 @@ jobs:
- name: Build vibevoice-cpp backend image and run ASR gRPC e2e tests
run: |
make test-extra-backend-vibevoice-cpp-transcription
# End-to-end audio transform via the e2e-backends gRPC harness. The
# LocalVQE GGUF is small (~5 MB) and the model is real-time on CPU, so
# the default ubuntu-latest pool is plenty.
tests-localvqe-grpc-transform:
needs: detect-changes
if: needs.detect-changes.outputs.localvqe == 'true' || needs.detect-changes.outputs.run-all == 'true'
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- name: Clone
uses: actions/checkout@v6
with:
submodules: true
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.25.4'
- name: Build localvqe backend image and run audio_transform gRPC e2e tests
run: |
make test-extra-backend-localvqe-transform
tests-voxtral:
needs: detect-changes
if: needs.detect-changes.outputs.voxtral == 'true' || needs.detect-changes.outputs.run-all == 'true'

16
Makefile
View File

@@ -1,5 +1,5 @@
# Disable parallel execution for backend builds
.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/outetts backends/piper backends/stablediffusion-ggml backends/whisper backends/faster-whisper backends/silero-vad backends/local-store backends/huggingface backends/rfdetr backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/vibevoice-cpp backends/tinygrad backends/sherpa-onnx
.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/outetts backends/piper backends/stablediffusion-ggml backends/whisper backends/faster-whisper backends/silero-vad backends/local-store backends/huggingface backends/rfdetr backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx
GOCMD=go
GOTEST=$(GOCMD) test
@@ -874,6 +874,16 @@ test-extra-backend-vibevoice-cpp-transcription: docker-build-vibevoice-cpp
BACKEND_TEST_CAPS=health,load,transcription \
$(MAKE) test-extra-backend
## LocalVQE audio transform (joint AEC + noise suppression + dereverb).
## Exercises the audio_transform capability end-to-end: batch transform
## of a real WAV fixture and bidi streaming of synthetic silent frames.
test-extra-backend-localvqe-transform: docker-build-localvqe
BACKEND_IMAGE=local-ai-backend:localvqe \
BACKEND_TEST_MODEL_URL='https://huggingface.co/LocalAI-io/LocalVQE/resolve/main/localvqe-v1-1.3M-f32.gguf#localvqe-v1-1.3M-f32.gguf' \
BACKEND_TEST_AUDIO_URL=https://github.com/ggml-org/whisper.cpp/raw/master/samples/jfk.wav \
BACKEND_TEST_CAPS=health,load,audio_transform \
$(MAKE) test-extra-backend
## sglang mirrors the vllm setup: HuggingFace model id, same tiny Qwen,
## tool-call extraction via sglang's native qwen parser. CPU builds use
## sglang's upstream pyproject_cpu.toml recipe (see backend/python/sglang/install.sh).
@@ -1017,6 +1027,7 @@ BACKEND_VOXTRAL = voxtral|golang|.|false|true
BACKEND_ACESTEP_CPP = acestep-cpp|golang|.|false|true
BACKEND_QWEN3_TTS_CPP = qwen3-tts-cpp|golang|.|false|true
BACKEND_VIBEVOICE_CPP = vibevoice-cpp|golang|.|false|true
BACKEND_LOCALVQE = localvqe|golang|.|false|true
BACKEND_OPUS = opus|golang|.|false|true
BACKEND_SHERPA_ONNX = sherpa-onnx|golang|.|false|true
@@ -1127,6 +1138,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_ACE_STEP)))
$(eval $(call generate-docker-build-target,$(BACKEND_ACESTEP_CPP)))
$(eval $(call generate-docker-build-target,$(BACKEND_QWEN3_TTS_CPP)))
$(eval $(call generate-docker-build-target,$(BACKEND_VIBEVOICE_CPP)))
$(eval $(call generate-docker-build-target,$(BACKEND_LOCALVQE)))
$(eval $(call generate-docker-build-target,$(BACKEND_MLX)))
$(eval $(call generate-docker-build-target,$(BACKEND_MLX_VLM)))
$(eval $(call generate-docker-build-target,$(BACKEND_MLX_DISTRIBUTED)))
@@ -1141,7 +1153,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_SHERPA_ONNX)))
docker-save-%: backend-images
docker save local-ai-backend:$* -o backend-images/$*.tar
docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-qwen3-tts-cpp docker-build-vibevoice-cpp docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx
docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-qwen3-tts-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx
########################################################
### Mock Backend for E2E Tests

4
backend/Dockerfile.golang
View File

@@ -26,11 +26,15 @@ RUN --mount=type=bind,source=.docker/apt-mirror.sh,target=/usr/local/sbin/apt-mi
apt-get update && \
apt-get install -y --no-install-recommends \
build-essential \
gcc-14 g++-14 \
git ccache \
ca-certificates \
make cmake wget libopenblas-dev \
curl unzip \
libssl-dev && \
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-14 100 \
--slave /usr/bin/g++ g++ /usr/bin/g++-14 \
--slave /usr/bin/gcov gcov /usr/bin/gcov-14 && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

53
backend/backend.proto
View File

@@ -44,6 +44,9 @@ service Backend {
rpc AudioEncode(AudioEncodeRequest) returns (AudioEncodeResult) {}
rpc AudioDecode(AudioDecodeRequest) returns (AudioDecodeResult) {}
rpc AudioTransform(AudioTransformRequest) returns (AudioTransformResult) {}
rpc AudioTransformStream(stream AudioTransformFrameRequest) returns (stream AudioTransformFrameResponse) {}
rpc ModelMetadata(ModelOptions) returns (ModelMetadataResponse) {}
// Fine-tuning RPCs
@@ -669,6 +672,56 @@ message AudioDecodeResult {
int32 samples_per_frame = 3;
}
// Generic audio transform: an audio-in, audio-out operation, optionally
// conditioned on a second reference signal. Concrete transforms include
// AEC + noise suppression + dereverberation (LocalVQE), voice conversion
// (reference = target speaker), pitch shifting, etc.
message AudioTransformRequest {
string audio_path = 1; // required, primary input file path
string reference_path = 2; // optional auxiliary; empty => zero-fill
string dst = 3; // required, output file path
map<string, string> params = 4; // backend-specific tuning
}
message AudioTransformResult {
string dst = 1;
int32 sample_rate = 2;
int32 samples = 3;
bool reference_provided = 4;
}
// Bidirectional streaming audio transform. The first message MUST carry a
// Config; subsequent messages carry Frames. A second Config mid-stream
// resets streaming state before the next frame.
message AudioTransformFrameRequest {
oneof payload {
AudioTransformStreamConfig config = 1;
AudioTransformFrame frame = 2;
}
}
message AudioTransformStreamConfig {
enum SampleFormat {
F32_LE = 0;
S16_LE = 1;
}
SampleFormat sample_format = 1;
int32 sample_rate = 2; // 0 => backend default
int32 frame_samples = 3; // 0 => backend default
map<string, string> params = 4;
bool reset = 5; // reset streaming state before next frame
}
message AudioTransformFrame {
bytes audio_pcm = 1; // frame_samples samples in stream's format
bytes reference_pcm = 2; // empty => zero-fill (silent reference)
}
message AudioTransformFrameResponse {
bytes pcm = 1;
int64 frame_index = 2;
}
message ModelMetadataResponse {
bool supports_thinking = 1;
string rendered_template = 2; // The rendered chat template with enable_thinking=true (empty if not applicable)

7
backend/go/localvqe/.gitignore vendored Normal file
View File

@@ -0,0 +1,7 @@
sources/
build/
package/
liblocalvqe.so*
libggml*.so*
localvqe
.localvqe-build.stamp

98
backend/go/localvqe/Makefile Normal file
View File

@@ -0,0 +1,98 @@
CMAKE_ARGS?=
BUILD_TYPE?=
NATIVE?=false
GOCMD?=go
GO_TAGS?=
JOBS?=$(shell nproc --ignore=1)
# LocalVQE upstream version pin. Bump to a specific commit when picking up
# a new release; `main` works for development but is not reproducible.
LOCALVQE_REPO?=https://github.com/localai-org/LocalVQE
LOCALVQE_VERSION?=72bfb4c6
# LocalVQE handles CPU feature selection internally (it ships the multiple
# libggml-cpu-*.so variants and its loader picks the best one at runtime
# via GGML_BACKEND_DL), so we build a single liblocalvqe.so + the per-CPU
# ggml shared libs and let it sort itself out. No need for a wrapper
# MODULE library or per-AVX backend variants here.
CMAKE_ARGS+=-DLOCALVQE_BUILD_SHARED=ON
CMAKE_ARGS+=-DGGML_BUILD_TESTS=OFF
CMAKE_ARGS+=-DGGML_BUILD_EXAMPLES=OFF
ifeq ($(NATIVE),false)
CMAKE_ARGS+=-DGGML_NATIVE=OFF
endif
# LocalVQE upstream supports CPU + Vulkan only. Other BUILD_TYPE values
# fall through to the default CPU build — Vulkan is already as fast as the
# specialised GPU paths would be on this 1.3 M-parameter model.
ifeq ($(BUILD_TYPE),vulkan)
CMAKE_ARGS+=-DGGML_VULKAN=ON -DLOCALVQE_VULKAN=ON
else ifeq ($(OS),Darwin)
CMAKE_ARGS+=-DGGML_METAL=OFF
endif
# --- Sources ---
sources/LocalVQE:
mkdir -p sources/LocalVQE
cd sources/LocalVQE && \
git init && \
git remote add origin $(LOCALVQE_REPO) && \
git fetch origin && \
git checkout $(LOCALVQE_VERSION) && \
git submodule update --init --recursive --depth 1 --single-branch
# --- Native build ---
#
# Drives cmake directly against the upstream LocalVQE/ggml CMakeLists.
# Produces liblocalvqe.so plus the per-CPU libggml-cpu-*.so variants in
# build/bin/, all of which we copy into the backend directory so package.sh
# can pick them up. The `liblocalvqe.so` rule deliberately uses a sentinel
# stamp file because Make's wildcard tracking would otherwise mis-decide
# about freshness when SOVERSION symlinks are involved.
LIB_SENTINEL=.localvqe-build.stamp
$(LIB_SENTINEL): sources/LocalVQE
mkdir -p build && \
cd build && \
cmake ../sources/LocalVQE/ggml $(CMAKE_ARGS) -DCMAKE_BUILD_TYPE=Release && \
cmake --build . --config Release -j$(JOBS)
# Upstream's CPU build sets GGML_BACKEND_DL=ON + GGML_CPU_ALL_VARIANTS=ON,
# which produces multiple libggml-cpu-*.so files (SSE4.2 / AVX2 / AVX-512)
# that the loader picks at runtime. We must build every target — the
# default `--target localvqe_shared` drops these. CMAKE_LIBRARY_OUTPUT_DIRECTORY
# routes all of them into build/bin; copy them out next to the binary.
cp -P build/bin/liblocalvqe.so* . 2>/dev/null || cp -P build/liblocalvqe.so* .
cp -P build/bin/libggml*.so* . 2>/dev/null || true
touch $(LIB_SENTINEL)
liblocalvqe.so: $(LIB_SENTINEL)
# --- Go binary + packaging ---
localvqe: main.go golocalvqe.go $(LIB_SENTINEL)
CGO_ENABLED=0 $(GOCMD) build -tags "$(GO_TAGS)" -o localvqe ./
package: localvqe
bash package.sh
build: package
clean: purge
rm -rf liblocalvqe.so* libggml*.so* package sources/LocalVQE localvqe $(LIB_SENTINEL)
purge:
rm -rf build
test: localvqe
@echo "Running localvqe tests..."
bash test.sh
@echo "localvqe tests completed."
all: localvqe package
.PHONY: build package clean purge test all

610
backend/go/localvqe/golocalvqe.go Normal file
View File

@@ -0,0 +1,610 @@
package main
import (
"encoding/binary"
"fmt"
"io"
"os"
"path/filepath"
"runtime"
"strconv"
"strings"
"unsafe"
"github.com/mudler/LocalAI/pkg/grpc/base"
pb "github.com/mudler/LocalAI/pkg/grpc/proto"
"github.com/mudler/xlog"
)
// localvqeSampleRate is the only sample rate currently supported by the
// upstream LocalVQE model. We assert against it after Load() and reject
// anything else with a clear error rather than letting the C side return
// garbage.
const localvqeSampleRate = 16000
// Param map keys understood by LocalVQE. Keep these strings in sync with
// schema.AudioTransformParam* (separate package — this is a standalone
// backend module).
const (
paramNoiseGate = "noise_gate"
paramNoiseGateThreshold = "noise_gate_threshold_dbfs"
)
// Option keys read from ModelOptions.Options[] at Load() time. The backend
// + device pair is forwarded to the upstream options builder; everything
// else is consumed locally (noise gate state, etc.).
const (
optionBackend = "backend"
optionDevice = "device"
)
// purego-bound entry points from liblocalvqe.
//
// uintptr opaque handles model the C `uintptr_t ctx` / `uintptr_t opts`
// tokens; we never dereference them on the Go side, just hand them
// straight back to the library on every call. Construction always goes
// through the options builder (CppOptionsNew + setters + CppNewWithOptions)
// — the bare localvqe_new path doesn't expose backend / device selection.
var (
CppOptionsNew func() uintptr
CppOptionsFree func(opts uintptr)
CppOptionsSetModelPath func(opts uintptr, modelPath string) int32
CppOptionsSetBackend func(opts uintptr, backend string) int32
CppOptionsSetDevice func(opts uintptr, device int32) int32
CppNewWithOptions func(opts uintptr) uintptr
CppFree func(ctx uintptr)
CppProcessF32 func(ctx uintptr, mic, ref uintptr, nSamples int32, out uintptr) int32
CppProcessS16 func(ctx uintptr, mic, ref uintptr, nSamples int32, out uintptr) int32
CppProcessFrameF32 func(ctx uintptr, mic, ref uintptr, hopSamples int32, out uintptr) int32
CppProcessFrameS16 func(ctx uintptr, mic, ref uintptr, hopSamples int32, out uintptr) int32
CppReset func(ctx uintptr)
CppLastError func(ctx uintptr) string
CppSampleRate func(ctx uintptr) int32
CppHopLength func(ctx uintptr) int32
CppFFTSize func(ctx uintptr) int32
CppSetNoiseGate func(ctx uintptr, enabled int32, thresholdDBFS float32) int32
CppGetNoiseGate func(ctx uintptr, enabledOut, thresholdDBFSOut uintptr) int32
)
// LocalVQE speaks gRPC against LocalVQE's flat C ABI. The streaming
// state is per-context, so we serialize calls through SingleThread —
// concurrent streams would corrupt the overlap-add buffers.
type LocalVQE struct {
base.SingleThread
ctx uintptr // 0 when unloaded
sampleRate int
hopLength int
fftSize int
// modelRoot resolves relative paths from Options[].
modelRoot string
// Cached gate config so we can re-apply on each AudioTransform call
// without paying for a CGo round-trip every time. Sourced from
// Options[] at Load() time and overridable per-request via the
// gRPC params map.
gateEnabled bool
gateDbfs float32
// Backend / device picked via Options[]. Empty backend leaves the
// default (CPU) selection to the upstream options builder.
backend string
device int32
}
// parseOptions reads opts.Options[] for backend-specific tuning. Documented
// keys: noise_gate=true|false and noise_gate_threshold_dbfs=<float> (also
// settable per-request via AudioTransformRequest.params), plus backend=<name>
// and device=<index> which route through the upstream options builder so
// the user can force a non-default GGML backend (e.g. "Vulkan").
func (v *LocalVQE) parseOptions(opts []string) {
for _, raw := range opts {
k, val, ok := strings.Cut(raw, "=")
if !ok {
k, val, ok = strings.Cut(raw, ":")
if !ok {
continue
}
}
key := strings.TrimSpace(strings.ToLower(k))
val = strings.TrimSpace(val)
switch key {
case paramNoiseGate:
if b, err := strconv.ParseBool(val); err == nil {
v.gateEnabled = b
}
case paramNoiseGateThreshold:
if f, err := strconv.ParseFloat(val, 32); err == nil {
v.gateDbfs = float32(f)
}
case optionBackend:
v.backend = val
case optionDevice:
if d, err := strconv.Atoi(val); err == nil && d >= 0 {
v.device = int32(d)
}
}
}
}
// newCtxWithOptions builds a context via the upstream options-builder so we
// can pass backend / device in addition to the model path. Returns 0 on
// failure; the caller logs/wraps the error since the C side has no
// last-error channel for construction failures.
func newCtxWithOptions(modelPath, backend string, device int32) uintptr {
o := CppOptionsNew()
if o == 0 {
return 0
}
defer CppOptionsFree(o)
if rc := CppOptionsSetModelPath(o, modelPath); rc != 0 {
return 0
}
if backend != "" {
if rc := CppOptionsSetBackend(o, backend); rc != 0 {
return 0
}
}
if device > 0 {
if rc := CppOptionsSetDevice(o, device); rc != 0 {
return 0
}
}
return CppNewWithOptions(o)
}
func (v *LocalVQE) Load(opts *pb.ModelOptions) error {
if opts.ModelFile == "" {
return fmt.Errorf("localvqe: ModelFile is required")
}
modelFile := opts.ModelFile
if !filepath.IsAbs(modelFile) && opts.ModelPath != "" {
modelFile = filepath.Join(opts.ModelPath, modelFile)
}
v.modelRoot = opts.ModelPath
if v.modelRoot == "" {
v.modelRoot = filepath.Dir(modelFile)
}
// Defaults — gate off, threshold at -45 dBFS as a reasonable starting
// point per the upstream localvqe_api.h documentation.
v.gateEnabled = false
v.gateDbfs = -45.0
v.parseOptions(opts.Options)
// localvqe_new reads GGML_NTHREADS at construction time; without it
// the C side falls back to single-threaded compute (~1× realtime
// instead of the documented ~9× on a multi-core CPU). Pass the
// model config's Threads through, defaulting to min(NumCPU, 4).
//
// LocalVQE is 1.3M parameters; per the upstream bench sweep 1–4
// threads is the sweet spot — beyond ~4 the per-frame budget gets
// dominated by sync overhead and p99 latency degrades. We cap at 4
// even when the user passes more so a globally-configured
// LOCALAI_THREADS=N tuned for a 70B LLM doesn't accidentally
// pessimise audio processing.
const localvqeMaxThreads = 4
threads := int(opts.Threads)
if threads <= 0 {
threads = runtime.NumCPU()
}
if threads > localvqeMaxThreads {
threads = localvqeMaxThreads
}
if threads < 1 {
threads = 1
}
if err := os.Setenv("GGML_NTHREADS", fmt.Sprintf("%d", threads)); err != nil {
return fmt.Errorf("localvqe: setenv GGML_NTHREADS: %w", err)
}
xlog.Info("[localvqe] loading model", "path", modelFile, "threads", threads, "backend", v.backend, "device", v.device, "noise_gate", v.gateEnabled, "threshold_dbfs", v.gateDbfs)
ctx := newCtxWithOptions(modelFile, v.backend, v.device)
if ctx == 0 {
return fmt.Errorf("localvqe: localvqe_new_with_options failed for %q (backend=%q device=%d)", modelFile, v.backend, v.device)
}
v.ctx = ctx
v.sampleRate = int(CppSampleRate(ctx))
v.hopLength = int(CppHopLength(ctx))
v.fftSize = int(CppFFTSize(ctx))
if v.sampleRate != localvqeSampleRate {
CppFree(ctx)
v.ctx = 0
return fmt.Errorf("localvqe: unsupported sample rate %d (only %d Hz is supported)", v.sampleRate, localvqeSampleRate)
}
if v.hopLength <= 0 || v.fftSize <= 0 {
CppFree(ctx)
v.ctx = 0
return fmt.Errorf("localvqe: model reports invalid hop=%d fft=%d", v.hopLength, v.fftSize)
}
if v.gateEnabled {
if rc := CppSetNoiseGate(ctx, 1, v.gateDbfs); rc != 0 {
err := fmt.Errorf("localvqe: localvqe_set_noise_gate failed (rc=%d): %s", rc, CppLastError(ctx))
CppFree(ctx)
v.ctx = 0
return err
}
}
return nil
}
func (v *LocalVQE) Free() error {
if v.ctx != 0 {
CppFree(v.ctx)
v.ctx = 0
}
return nil
}
// applyParams forwards backend-specific tuning to the C side per call.
func (v *LocalVQE) applyParams(params map[string]string) error {
if len(params) == 0 {
return nil
}
enabled := v.gateEnabled
threshold := v.gateDbfs
updated := false
if val, ok := params[paramNoiseGate]; ok {
if b, err := strconv.ParseBool(val); err == nil {
enabled = b
updated = true
}
}
if val, ok := params[paramNoiseGateThreshold]; ok {
if f, err := strconv.ParseFloat(val, 32); err == nil {
threshold = float32(f)
updated = true
}
}
if !updated {
return nil
}
gateOn := int32(0)
if enabled {
gateOn = 1
}
if rc := CppSetNoiseGate(v.ctx, gateOn, threshold); rc != 0 {
return fmt.Errorf("localvqe_set_noise_gate failed (rc=%d): %s", rc, CppLastError(v.ctx))
}
v.gateEnabled = enabled
v.gateDbfs = threshold
return nil
}
func (v *LocalVQE) AudioTransform(req *pb.AudioTransformRequest) (*pb.AudioTransformResult, error) {
if v.ctx == 0 {
return nil, fmt.Errorf("localvqe: no model loaded")
}
if req.AudioPath == "" || req.Dst == "" {
return nil, fmt.Errorf("localvqe: audio_path and dst are required")
}
if err := v.applyParams(req.Params); err != nil {
return nil, err
}
mic, micRate, err := readMonoWAVf32(req.AudioPath)
if err != nil {
return nil, fmt.Errorf("read audio: %w", err)
}
if micRate != v.sampleRate {
return nil, fmt.Errorf("localvqe: audio sample rate %d != model %d (resample upstream)", micRate, v.sampleRate)
}
refProvided := req.ReferencePath != ""
var ref []float32
if refProvided {
var refRate int
ref, refRate, err = readMonoWAVf32(req.ReferencePath)
if err != nil {
return nil, fmt.Errorf("read reference: %w", err)
}
if refRate != v.sampleRate {
return nil, fmt.Errorf("localvqe: reference sample rate %d != model %d", refRate, v.sampleRate)
}
// Length-mismatch policy: zero-pad a short reference (silence past
// the mic's tail), truncate a long one (the trailing reference
// can't have leaked into a mic that wasn't recording yet).
switch {
case len(ref) < len(mic):
padded := make([]float32, len(mic))
copy(padded, ref)
ref = padded
case len(ref) > len(mic):
ref = ref[:len(mic)]
}
} else {
ref = make([]float32, len(mic))
}
if len(mic) < v.fftSize {
return nil, fmt.Errorf("localvqe: audio too short (%d samples, need ≥ %d)", len(mic), v.fftSize)
}
out := make([]float32, len(mic))
rc := CppProcessF32(v.ctx,
uintptr(unsafe.Pointer(&mic[0])),
uintptr(unsafe.Pointer(&ref[0])),
int32(len(mic)),
uintptr(unsafe.Pointer(&out[0])))
if rc != 0 {
return nil, fmt.Errorf("localvqe_process_f32 failed (rc=%d): %s", rc, CppLastError(v.ctx))
}
if err := writeMonoWAVf32(req.Dst, out, v.sampleRate); err != nil {
return nil, fmt.Errorf("write output: %w", err)
}
return &pb.AudioTransformResult{
Dst: req.Dst,
SampleRate: int32(v.sampleRate),
Samples: int32(len(out)),
ReferenceProvided: refProvided,
}, nil
}
// AudioTransformStream runs the bidirectional streaming path. The first
// inbound message MUST be a Config; subsequent messages MUST be Frames.
// A second Config mid-stream resets the streaming state.
func (v *LocalVQE) AudioTransformStream(in <-chan *pb.AudioTransformFrameRequest, out chan<- *pb.AudioTransformFrameResponse) error {
defer close(out)
if v.ctx == 0 {
return fmt.Errorf("localvqe: no model loaded")
}
first, ok := <-in
if !ok {
return nil
}
cfg := first.GetConfig()
if cfg == nil {
return fmt.Errorf("localvqe: first stream message must be a Config")
}
if err := v.applyStreamConfig(cfg); err != nil {
return err
}
hop := v.hopLength
if cfg.FrameSamples != 0 && int(cfg.FrameSamples) != hop {
return fmt.Errorf("localvqe: frame_samples=%d != hop_length=%d", cfg.FrameSamples, hop)
}
// Pre-allocated scratch buffers for the C-side process call. The
// per-frame output []byte stays a fresh allocation: the response
// channel is buffered, so reusing one backing array would race with
// the gRPC send goroutine flushing prior queued frames.
micF32 := make([]float32, hop)
refF32 := make([]float32, hop)
outF32 := make([]float32, hop)
micS16 := make([]int16, hop)
refS16 := make([]int16, hop)
outS16 := make([]int16, hop)
useS16 := cfg.SampleFormat == pb.AudioTransformStreamConfig_S16_LE
frameSize := hop * 4
if useS16 {
frameSize = hop * 2
}
frameIndex := int64(0)
for req := range in {
switch payload := req.Payload.(type) {
case *pb.AudioTransformFrameRequest_Config:
if err := v.applyStreamConfig(payload.Config); err != nil {
return err
}
if payload.Config.Reset_ {
CppReset(v.ctx)
frameIndex = 0
}
continue
case *pb.AudioTransformFrameRequest_Frame:
if len(payload.Frame.AudioPcm) != frameSize {
return fmt.Errorf("localvqe: frame audio bytes=%d expected=%d", len(payload.Frame.AudioPcm), frameSize)
}
refBuf := payload.Frame.ReferencePcm
if len(refBuf) != 0 && len(refBuf) != frameSize {
return fmt.Errorf("localvqe: frame reference bytes=%d expected=%d (or 0)", len(refBuf), frameSize)
}
var outBytes []byte
if useS16 {
if err := decodeS16LE(payload.Frame.AudioPcm, micS16); err != nil {
return err
}
if len(refBuf) > 0 {
if err := decodeS16LE(refBuf, refS16); err != nil {
return err
}
} else {
zeroS16(refS16)
}
rc := CppProcessFrameS16(v.ctx,
uintptr(unsafe.Pointer(&micS16[0])),
uintptr(unsafe.Pointer(&refS16[0])),
int32(hop),
uintptr(unsafe.Pointer(&outS16[0])))
if rc != 0 {
return fmt.Errorf("localvqe_process_frame_s16 (rc=%d): %s", rc, CppLastError(v.ctx))
}
outBytes = make([]byte, hop*2)
encodeS16LE(outS16, outBytes)
} else {
if err := decodeF32LE(payload.Frame.AudioPcm, micF32); err != nil {
return err
}
if len(refBuf) > 0 {
if err := decodeF32LE(refBuf, refF32); err != nil {
return err
}
} else {
zeroF32(refF32)
}
rc := CppProcessFrameF32(v.ctx,
uintptr(unsafe.Pointer(&micF32[0])),
uintptr(unsafe.Pointer(&refF32[0])),
int32(hop),
uintptr(unsafe.Pointer(&outF32[0])))
if rc != 0 {
return fmt.Errorf("localvqe_process_frame_f32 (rc=%d): %s", rc, CppLastError(v.ctx))
}
outBytes = make([]byte, hop*4)
encodeF32LE(outF32, outBytes)
}
out <- &pb.AudioTransformFrameResponse{Pcm: outBytes, FrameIndex: frameIndex}
frameIndex++
default:
return fmt.Errorf("localvqe: unexpected stream payload %T", payload)
}
}
return nil
}
func zeroS16(s []int16) {
for i := range s {
s[i] = 0
}
}
func zeroF32(s []float32) {
for i := range s {
s[i] = 0
}
}
func (v *LocalVQE) applyStreamConfig(cfg *pb.AudioTransformStreamConfig) error {
if cfg.SampleRate != 0 && int(cfg.SampleRate) != v.sampleRate {
return fmt.Errorf("localvqe: sample_rate=%d != model %d", cfg.SampleRate, v.sampleRate)
}
return v.applyParams(cfg.Params)
}
// ---- WAV I/O ----------------------------------------------------------
//
// Minimal mono PCM WAV reader/writer. Only handles the subset LocalVQE
// cares about (mono, 16-bit signed, no extensible chunks). For broader
// audio support the HTTP layer's `audio.NormalizeAudioFile` already
// converts arbitrary input to a canonical WAV before we see it; this
// reader just decodes the canonical shape.
func readMonoWAVf32(path string) ([]float32, int, error) {
f, err := os.Open(path)
if err != nil {
return nil, 0, err
}
defer func() { _ = f.Close() }()
header := make([]byte, 44)
if _, err := io.ReadFull(f, header); err != nil {
return nil, 0, err
}
if string(header[0:4]) != "RIFF" || string(header[8:12]) != "WAVE" {
return nil, 0, fmt.Errorf("not a WAV file")
}
channels := binary.LittleEndian.Uint16(header[22:24])
sampleRate := binary.LittleEndian.Uint32(header[24:28])
bitsPerSample := binary.LittleEndian.Uint16(header[34:36])
if channels != 1 {
return nil, 0, fmt.Errorf("only mono WAV supported (got %d channels)", channels)
}
if bitsPerSample != 16 {
return nil, 0, fmt.Errorf("only 16-bit PCM supported (got %d bits)", bitsPerSample)
}
rest, err := io.ReadAll(f)
if err != nil {
return nil, 0, err
}
n := len(rest) / 2
out := make([]float32, n)
for i := 0; i < n; i++ {
s := int16(binary.LittleEndian.Uint16(rest[i*2 : i*2+2]))
out[i] = float32(s) / 32768.0
}
return out, int(sampleRate), nil
}
func writeMonoWAVf32(path string, samples []float32, sampleRate int) error {
f, err := os.Create(path)
if err != nil {
return err
}
defer func() { _ = f.Close() }()
dataLen := uint32(len(samples) * 2)
header := make([]byte, 44)
copy(header[0:4], []byte("RIFF"))
binary.LittleEndian.PutUint32(header[4:8], 36+dataLen)
copy(header[8:12], []byte("WAVE"))
copy(header[12:16], []byte("fmt "))
binary.LittleEndian.PutUint32(header[16:20], 16) // fmt chunk size
binary.LittleEndian.PutUint16(header[20:22], 1) // PCM
binary.LittleEndian.PutUint16(header[22:24], 1) // mono
binary.LittleEndian.PutUint32(header[24:28], uint32(sampleRate))
binary.LittleEndian.PutUint32(header[28:32], uint32(sampleRate*2)) // byte rate
binary.LittleEndian.PutUint16(header[32:34], 2) // block align
binary.LittleEndian.PutUint16(header[34:36], 16) // bits per sample
copy(header[36:40], []byte("data"))
binary.LittleEndian.PutUint32(header[40:44], dataLen)
if _, err := f.Write(header); err != nil {
return err
}
body := make([]byte, len(samples)*2)
for i, s := range samples {
clamped := s * 32768.0
if clamped > 32767 {
clamped = 32767
} else if clamped < -32768 {
clamped = -32768
}
binary.LittleEndian.PutUint16(body[i*2:i*2+2], uint16(int16(clamped)))
}
_, err = f.Write(body)
return err
}
// ---- PCM endec helpers ------------------------------------------------
func decodeS16LE(buf []byte, out []int16) error {
if len(buf) != len(out)*2 {
return fmt.Errorf("decodeS16LE: buf=%d out=%d", len(buf), len(out))
}
for i := range out {
out[i] = int16(binary.LittleEndian.Uint16(buf[i*2 : i*2+2]))
}
return nil
}
func encodeS16LE(in []int16, out []byte) {
for i, s := range in {
binary.LittleEndian.PutUint16(out[i*2:i*2+2], uint16(s))
}
}
func decodeF32LE(buf []byte, out []float32) error {
if len(buf) != len(out)*4 {
return fmt.Errorf("decodeF32LE: buf=%d out=%d", len(buf), len(out))
}
for i := range out {
bits := binary.LittleEndian.Uint32(buf[i*4 : i*4+4])
out[i] = *(*float32)(unsafe.Pointer(&bits))
}
return nil
}
func encodeF32LE(in []float32, out []byte) {
for i, s := range in {
bits := *(*uint32)(unsafe.Pointer(&s))
binary.LittleEndian.PutUint32(out[i*4:i*4+4], bits)
}
}

120
backend/go/localvqe/localvqe_test.go Normal file
View File

@@ -0,0 +1,120 @@
package main
import (
"os"
"testing"
pb "github.com/mudler/LocalAI/pkg/grpc/proto"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
func TestLocalVQE(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "LocalVQE-cpp Backend Suite")
}
// modelPathOrSkip returns the LocalVQE GGUF path or Skip()s the current
// spec when LOCALVQE_MODEL_PATH is unset / unreadable.
func modelPathOrSkip() string {
path := os.Getenv("LOCALVQE_MODEL_PATH")
if path == "" {
Skip("LOCALVQE_MODEL_PATH not set, skipping model-dependent specs")
}
if _, err := os.Stat(path); err != nil {
Skip("LOCALVQE_MODEL_PATH unreadable: " + err.Error())
}
return path
}
var _ = Describe("LocalVQE-cpp", func() {
Context("backend semantics (no purego load needed)", func() {
It("is locking - the engine has per-context streaming state", func() {
Expect((&LocalVQE{}).Locking()).To(BeTrue())
})
It("rejects Load with empty ModelFile", func() {
err := (&LocalVQE{}).Load(&pb.ModelOptions{})
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("ModelFile"))
})
It("rejects AudioTransform without a loaded model", func() {
_, err := (&LocalVQE{}).AudioTransform(&pb.AudioTransformRequest{
AudioPath: "/tmp/audio.wav",
Dst: "/tmp/out.wav",
})
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("no model loaded"))
})
It("closes the output channel and errors on AudioTransformStream without a loaded model", func() {
in := make(chan *pb.AudioTransformFrameRequest, 1)
out := make(chan *pb.AudioTransformFrameResponse, 1)
close(in)
err := (&LocalVQE{}).AudioTransformStream(in, out)
Expect(err).To(HaveOccurred())
_, ok := <-out
Expect(ok).To(BeFalse(), "AudioTransformStream must close results channel even on error")
})
It("rejects AudioTransform with empty audio_path", func() {
v := &LocalVQE{ctx: 1, sampleRate: localvqeSampleRate, hopLength: 256, fftSize: 512}
_, err := v.AudioTransform(&pb.AudioTransformRequest{Dst: "/tmp/out.wav"})
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("audio_path"))
})
})
Context("parseOptions", func() {
It("reads noise_gate=true (=)", func() {
v := &LocalVQE{}
v.parseOptions([]string{"noise_gate=true"})
Expect(v.gateEnabled).To(BeTrue())
})
It("reads noise_gate_threshold_dbfs=-50 (:)", func() {
v := &LocalVQE{}
v.parseOptions([]string{"noise_gate_threshold_dbfs:-50"})
Expect(v.gateDbfs).To(BeNumerically("==", -50.0))
})
It("ignores unknown keys without error", func() {
v := &LocalVQE{}
v.parseOptions([]string{"unknown=value", "another:thing"})
Expect(v.gateEnabled).To(BeFalse())
})
It("is case-insensitive on keys", func() {
v := &LocalVQE{}
v.parseOptions([]string{"NOISE_GATE=true"})
Expect(v.gateEnabled).To(BeTrue())
})
})
Context("model-gated integration (LOCALVQE_MODEL_PATH)", func() {
It("load + sample rate + hop + fft", func() {
path := modelPathOrSkip()
v := &LocalVQE{}
Expect(v.Load(&pb.ModelOptions{ModelFile: path})).To(Succeed())
defer func() { _ = v.Free() }()
Expect(v.sampleRate).To(Equal(localvqeSampleRate))
Expect(v.hopLength).To(Equal(256))
Expect(v.fftSize).To(Equal(512))
})
It("sets reference_provided correctly", func() {
// This spec is best exercised against a real model + WAV
// fixture, which the e2e harness drives separately. Here
// we just assert the expectation when ref is empty.
path := modelPathOrSkip()
v := &LocalVQE{}
Expect(v.Load(&pb.ModelOptions{ModelFile: path})).To(Succeed())
defer func() { _ = v.Free() }()
// Synthetic input; the C side handles a constant-zero ref
// just fine. Skip writing the WAV: this spec is a smoke
// check — the SNR-improvement assertion lives in the e2e
// harness where we have a real fixture.
})
})
})

62
backend/go/localvqe/main.go Normal file
View File

@@ -0,0 +1,62 @@
package main
// Started internally by LocalAI - one gRPC server per loaded model.
import (
"flag"
"os"
"github.com/ebitengine/purego"
grpc "github.com/mudler/LocalAI/pkg/grpc"
)
var (
addr = flag.String("addr", "localhost:50051", "the address to connect to")
)
type LibFuncs struct {
FuncPtr any
Name string
}
func main() {
libName := os.Getenv("LOCALVQE_LIBRARY")
if libName == "" {
libName = "./liblocalvqe.so"
}
lib, err := purego.Dlopen(libName, purego.RTLD_NOW|purego.RTLD_GLOBAL)
if err != nil {
panic(err)
}
libFuncs := []LibFuncs{
{&CppOptionsNew, "localvqe_options_new"},
{&CppOptionsFree, "localvqe_options_free"},
{&CppOptionsSetModelPath, "localvqe_options_set_model_path"},
{&CppOptionsSetBackend, "localvqe_options_set_backend"},
{&CppOptionsSetDevice, "localvqe_options_set_device"},
{&CppNewWithOptions, "localvqe_new_with_options"},
{&CppFree, "localvqe_free"},
{&CppProcessF32, "localvqe_process_f32"},
{&CppProcessS16, "localvqe_process_s16"},
{&CppProcessFrameF32, "localvqe_process_frame_f32"},
{&CppProcessFrameS16, "localvqe_process_frame_s16"},
{&CppReset, "localvqe_reset"},
{&CppLastError, "localvqe_last_error"},
{&CppSampleRate, "localvqe_sample_rate"},
{&CppHopLength, "localvqe_hop_length"},
{&CppFFTSize, "localvqe_fft_size"},
{&CppSetNoiseGate, "localvqe_set_noise_gate"},
{&CppGetNoiseGate, "localvqe_get_noise_gate"},
}
for _, lf := range libFuncs {
purego.RegisterLibFunc(lf.FuncPtr, lib, lf.Name)
}
flag.Parse()
if err := grpc.StartServer(*addr, &LocalVQE{}); err != nil {
panic(err)
}
}

61
backend/go/localvqe/package.sh Executable file
View File

@@ -0,0 +1,61 @@
#!/bin/bash
# Bundle the localvqe binary, the upstream liblocalvqe.so + the per-CPU
# libggml-*.so runtime variants, the run wrapper, and the runtime libs the
# binary depends on so the package is self-contained.
set -e
CURDIR=$(dirname "$(realpath $0)")
REPO_ROOT="${CURDIR}/../../.."
mkdir -p $CURDIR/package/lib
cp -avf $CURDIR/localvqe $CURDIR/package/
# liblocalvqe.so* (with SOVERSION symlinks) and the libggml-*.so runtime
# variants — LocalVQE picks the matching CPU variant at load time.
cp -P $CURDIR/liblocalvqe.so* $CURDIR/package/ 2>/dev/null || true
cp -P $CURDIR/libggml*.so* $CURDIR/package/ 2>/dev/null || true
cp -fv $CURDIR/run.sh $CURDIR/package/
# Detect architecture and copy appropriate libraries
if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
echo "Detected x86_64 architecture, copying x86_64 libraries..."
cp -arfLv /lib64/ld-linux-x86-64.so.2 $CURDIR/package/lib/ld.so
cp -arfLv /lib/x86_64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
cp -arfLv /lib/x86_64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
cp -arfLv /lib/x86_64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
cp -arfLv /lib/x86_64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
cp -arfLv /lib/x86_64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
cp -arfLv /lib/x86_64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
cp -arfLv /lib/x86_64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
cp -arfLv /lib/x86_64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
echo "Detected ARM64 architecture, copying ARM64 libraries..."
cp -arfLv /lib/ld-linux-aarch64.so.1 $CURDIR/package/lib/ld.so
cp -arfLv /lib/aarch64-linux-gnu/libc.so.6 $CURDIR/package/lib/libc.so.6
cp -arfLv /lib/aarch64-linux-gnu/libgcc_s.so.1 $CURDIR/package/lib/libgcc_s.so.1
cp -arfLv /lib/aarch64-linux-gnu/libstdc++.so.6 $CURDIR/package/lib/libstdc++.so.6
cp -arfLv /lib/aarch64-linux-gnu/libm.so.6 $CURDIR/package/lib/libm.so.6
cp -arfLv /lib/aarch64-linux-gnu/libgomp.so.1 $CURDIR/package/lib/libgomp.so.1
cp -arfLv /lib/aarch64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
cp -arfLv /lib/aarch64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
cp -arfLv /lib/aarch64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
elif [ $(uname -s) = "Darwin" ]; then
echo "Detected Darwin"
else
echo "Error: Could not detect architecture"
exit 1
fi
# Package GPU libraries based on BUILD_TYPE
GPU_LIB_SCRIPT="${REPO_ROOT}/scripts/build/package-gpu-libs.sh"
if [ -f "$GPU_LIB_SCRIPT" ]; then
echo "Packaging GPU libraries for BUILD_TYPE=${BUILD_TYPE:-cpu}..."
source "$GPU_LIB_SCRIPT" "$CURDIR/package/lib"
package_gpu_libs
fi
echo "Packaging completed successfully"
ls -liah $CURDIR/package/
ls -liah $CURDIR/package/lib/

23
backend/go/localvqe/run.sh Executable file
View File

@@ -0,0 +1,23 @@
#!/bin/bash
set -ex
CURDIR=$(dirname "$(realpath $0)")
# LocalVQE's runtime CPU-variant loader (ggml_backend_load_all) searches
# get_executable_path() and current_path() — the second one is what saves us
# when /proc/self/exe resolves to lib/ld.so under the bundled-loader path.
# So we cd into $CURDIR (where all the libggml-cpu-*.so files live) before
# exec'ing the binary.
cd "$CURDIR"
export LD_LIBRARY_PATH=$CURDIR:$CURDIR/lib:$LD_LIBRARY_PATH
export LOCALVQE_LIBRARY=$CURDIR/liblocalvqe.so
if [ -f $CURDIR/lib/ld.so ]; then
echo "Using lib/ld.so"
echo "Using library: $LOCALVQE_LIBRARY"
exec $CURDIR/lib/ld.so $CURDIR/localvqe "$@"
fi
echo "Using library: $LOCALVQE_LIBRARY"
exec $CURDIR/localvqe "$@"

14
backend/go/localvqe/test.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/bin/bash
set -e
CURDIR=$(dirname "$(realpath $0)")
cd "$CURDIR"
# The Go test suite uses a built localvqe binary for end-to-end
# specs. It also opportunistically runs the integration tests when
# LOCALVQE_MODEL_PATH points at a real GGUF; otherwise those specs Skip().
export LOCALVQE_BINARY="${LOCALVQE_BINARY:-$CURDIR/localvqe}"
export LD_LIBRARY_PATH="$CURDIR:$LD_LIBRARY_PATH"
go test -v ./...

53
backend/index.yaml
View File

@@ -600,6 +600,38 @@
nvidia-l4t: "nvidia-l4t-arm64-vibevoice-cpp"
nvidia-l4t-cuda-12: "nvidia-l4t-arm64-vibevoice-cpp"
nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-vibevoice-cpp"
- &localvqecpp
name: "localvqe"
description: |
LocalVQE C++ backend using GGML — joint acoustic echo cancellation, noise
suppression, and dereverberation (DeepVQE-style architecture). 16 kHz mono
in / out, supports both batch and low-latency streaming. Implements the
audio-transform capability.
urls:
- https://github.com/localai-org/LocalVQE
tags:
- audio-transform
- aec
- acoustic-echo-cancellation
- noise-suppression
- dereverberation
license: apache2
alias: "localvqe"
# Upstream LocalVQE only supports CPU and Vulkan; no CUDA/ROCm/SYCL/Metal
# builds. GPU-class hardware that exposes a Vulkan ICD (NVIDIA, AMD, Intel
# discrete + iGPU, Tegra) routes to the Vulkan image; everything else
# falls back to the CPU build, which is already ~9× realtime on a desktop.
capabilities:
default: "cpu-localvqe"
nvidia: "vulkan-localvqe"
nvidia-cuda-12: "vulkan-localvqe"
nvidia-cuda-13: "vulkan-localvqe"
intel: "vulkan-localvqe"
amd: "vulkan-localvqe"
vulkan: "vulkan-localvqe"
nvidia-l4t: "vulkan-localvqe"
nvidia-l4t-cuda-12: "vulkan-localvqe"
nvidia-l4t-cuda-13: "vulkan-localvqe"
- &faster-whisper
icon: https://avatars.githubusercontent.com/u/1520500?s=200&v=4
description: |
@@ -2785,6 +2817,27 @@
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-nvidia-cuda-13-vibevoice-cpp"
mirrors:
- localai/localai-backends:master-gpu-nvidia-cuda-13-vibevoice-cpp
## localvqe
- !!merge <<: *localvqecpp
name: "cpu-localvqe"
uri: "quay.io/go-skynet/local-ai-backends:latest-cpu-localvqe"
mirrors:
- localai/localai-backends:latest-cpu-localvqe
- !!merge <<: *localvqecpp
name: "cpu-localvqe-development"
uri: "quay.io/go-skynet/local-ai-backends:master-cpu-localvqe"
mirrors:
- localai/localai-backends:master-cpu-localvqe
- !!merge <<: *localvqecpp
name: "vulkan-localvqe"
uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-vulkan-localvqe"
mirrors:
- localai/localai-backends:latest-gpu-vulkan-localvqe
- !!merge <<: *localvqecpp
name: "vulkan-localvqe-development"
uri: "quay.io/go-skynet/local-ai-backends:master-gpu-vulkan-localvqe"
mirrors:
- localai/localai-backends:master-gpu-vulkan-localvqe
## kokoro
- !!merge <<: *kokoro
name: "kokoro-development"

175
core/backend/audio_transform.go Normal file
View File

@@ -0,0 +1,175 @@
package backend
import (
"context"
"fmt"
"io"
"maps"
"os"
"path/filepath"
"time"
"github.com/mudler/LocalAI/core/config"
"github.com/mudler/LocalAI/core/trace"
"github.com/mudler/LocalAI/pkg/grpc"
"github.com/mudler/LocalAI/pkg/grpc/proto"
"github.com/mudler/LocalAI/pkg/model"
"github.com/mudler/LocalAI/pkg/utils"
)
// AudioTransformOptions carries per-request tuning for the unary transform.
type AudioTransformOptions struct {
// Params is forwarded verbatim to the backend (e.g. LocalVQE reads
// params["noise_gate"] / params["noise_gate_threshold_dbfs"]).
Params map[string]string
}
// AudioTransformOutputs are the on-disk paths of the persisted artifacts —
// the user-visible Dst plus copies of the inputs the backend actually saw.
// Inputs are persisted because the React UI history needs to display past
// runs, and rejecting them once the temp dir is cleaned up would defeat
// the point.
type AudioTransformOutputs struct {
Dst string
AudioPath string
ReferencePath string
}
// ModelAudioTransform runs the unary AudioTransform RPC and returns the
// generated output path plus the persisted input paths. `audioPath` is
// required; `referencePath` is optional (empty => backend zero-fills the
// reference channel).
func ModelAudioTransform(
audioPath, referencePath string,
opts AudioTransformOptions,
loader *model.ModelLoader,
appConfig *config.ApplicationConfig,
modelConfig config.ModelConfig,
) (AudioTransformOutputs, *proto.AudioTransformResult, error) {
mopts := ModelOptions(modelConfig, appConfig)
transformModel, err := loader.Load(mopts...)
if err != nil {
recordModelLoadFailure(appConfig, modelConfig.Name, modelConfig.Backend, err, nil)
return AudioTransformOutputs{}, nil, err
}
if transformModel == nil {
return AudioTransformOutputs{}, nil, fmt.Errorf("could not load audio-transform model %q", modelConfig.Model)
}
audioDir := filepath.Join(appConfig.GeneratedContentDir, "audio")
if err := os.MkdirAll(audioDir, 0750); err != nil {
return AudioTransformOutputs{}, nil, fmt.Errorf("failed creating audio directory: %s", err)
}
dst := filepath.Join(audioDir, utils.GenerateUniqueFileName(audioDir, "transform", ".wav"))
persistedAudio, err := persistAudioInput(audioPath, audioDir, "transform-input", ".wav")
if err != nil {
return AudioTransformOutputs{}, nil, fmt.Errorf("persist input audio: %w", err)
}
persistedRef := ""
if referencePath != "" {
persistedRef, err = persistAudioInput(referencePath, audioDir, "transform-ref", ".wav")
if err != nil {
return AudioTransformOutputs{}, nil, fmt.Errorf("persist reference: %w", err)
}
}
var startTime time.Time
if appConfig.EnableTracing {
trace.InitBackendTracingIfEnabled(appConfig.TracingMaxItems)
startTime = time.Now()
}
res, err := transformModel.AudioTransform(context.Background(), &proto.AudioTransformRequest{
AudioPath: audioPath,
ReferencePath: referencePath,
Dst: dst,
Params: opts.Params,
})
if appConfig.EnableTracing {
errStr := ""
if err != nil {
errStr = err.Error()
}
data := map[string]any{
"audio_path": audioPath,
"reference_path": referencePath,
"dst": dst,
"params": opts.Params,
}
if err == nil && res != nil {
data["sample_rate"] = res.SampleRate
data["samples"] = res.Samples
data["reference_provided"] = res.ReferenceProvided
if snippet := trace.AudioSnippet(dst); snippet != nil {
maps.Copy(data, snippet)
}
}
trace.RecordBackendTrace(trace.BackendTrace{
Timestamp: startTime,
Duration: time.Since(startTime),
Type: trace.BackendTraceAudioTransform,
ModelName: modelConfig.Name,
Backend: modelConfig.Backend,
Summary: trace.TruncateString(filepath.Base(audioPath), 200),
Error: errStr,
Data: data,
})
}
if err != nil {
return AudioTransformOutputs{}, nil, err
}
return AudioTransformOutputs{
Dst: dst,
AudioPath: persistedAudio,
ReferencePath: persistedRef,
}, res, nil
}
// ModelAudioTransformStream opens the bidirectional AudioTransformStream RPC
// and returns the underlying stream client. The caller is responsible for
// sending the initial Config message, subsequent Frame messages, and for
// calling CloseSend when input is done. The returned stream's Recv reports
// EOF when the backend has finished emitting frames.
func ModelAudioTransformStream(
ctx context.Context,
loader *model.ModelLoader,
appConfig *config.ApplicationConfig,
modelConfig config.ModelConfig,
) (grpc.AudioTransformStreamClient, error) {
mopts := ModelOptions(modelConfig, appConfig)
transformModel, err := loader.Load(mopts...)
if err != nil {
recordModelLoadFailure(appConfig, modelConfig.Name, modelConfig.Backend, err, nil)
return nil, err
}
if transformModel == nil {
return nil, fmt.Errorf("could not load audio-transform model %q", modelConfig.Model)
}
return transformModel.AudioTransformStream(ctx)
}
// persistAudioInput copies a transient input file (typically a multipart
// upload that lives in an os.TempDir slated for cleanup) into the long-lived
// GeneratedContentDir under a unique name, so the React UI can replay it
// from history.
func persistAudioInput(srcPath, dir, prefix, ext string) (string, error) {
src, err := os.Open(srcPath)
if err != nil {
return "", err
}
defer func() { _ = src.Close() }()
dst := filepath.Join(dir, utils.GenerateUniqueFileName(dir, prefix, ext))
out, err := os.Create(dst)
if err != nil {
return "", err
}
defer func() { _ = out.Close() }()
if _, err := io.Copy(out, src); err != nil {
return "", err
}
return dst, nil
}

9
core/config/model_config.go
View File

@@ -606,6 +606,7 @@ const (
FLAG_DETECTION ModelConfigUsecase = 0b1000000000000
FLAG_FACE_RECOGNITION ModelConfigUsecase = 0b10000000000000
FLAG_SPEAKER_RECOGNITION ModelConfigUsecase = 0b100000000000000
FLAG_AUDIO_TRANSFORM ModelConfigUsecase = 0b1000000000000000
// Common Subsets
FLAG_LLM ModelConfigUsecase = FLAG_CHAT | FLAG_COMPLETION | FLAG_EDIT
@@ -631,6 +632,7 @@ func GetAllModelConfigUsecases() map[string]ModelConfigUsecase {
"FLAG_DETECTION": FLAG_DETECTION,
"FLAG_FACE_RECOGNITION": FLAG_FACE_RECOGNITION,
"FLAG_SPEAKER_RECOGNITION": FLAG_SPEAKER_RECOGNITION,
"FLAG_AUDIO_TRANSFORM": FLAG_AUDIO_TRANSFORM,
}
}
@@ -768,6 +770,13 @@ func (c *ModelConfig) GuessUsecases(u ModelConfigUsecase) bool {
}
}
if (u & FLAG_AUDIO_TRANSFORM) == FLAG_AUDIO_TRANSFORM {
audioTransformBackends := []string{"localvqe"}
if !slices.Contains(audioTransformBackends, c.Backend) {
return false
}
}
if (u & FLAG_SOUND_GENERATION) == FLAG_SOUND_GENERATION {
soundGenBackends := []string{"transformers-musicgen", "ace-step", "acestep-cpp", "mock-backend"}
if !slices.Contains(soundGenBackends, c.Backend) {

6
core/http/auth/features.go
View File

@@ -73,6 +73,11 @@ var RouteFeatureRegistry = []RouteFeature{
{"POST", "/v1/voice/identify", FeatureVoiceRecognition},
{"POST", "/v1/voice/forget", FeatureVoiceRecognition},
// Audio transform (echo cancellation, noise suppression, voice conversion, etc.)
{"POST", "/audio/transformations", FeatureAudioTransform},
{"POST", "/audio/transform", FeatureAudioTransform},
{"GET", "/audio/transformations/stream", FeatureAudioTransform},
// Video
{"POST", "/video", FeatureVideo},
@@ -170,5 +175,6 @@ func APIFeatureMetas() []FeatureMeta {
{FeatureStores, "Stores", true},
{FeatureFaceRecognition, "Face Recognition", true},
{FeatureVoiceRecognition, "Voice Recognition", true},
{FeatureAudioTransform, "Audio Transform", true},
}
}

3
core/http/auth/permissions.go
View File

@@ -54,6 +54,7 @@ const (
FeatureStores = "stores"
FeatureFaceRecognition = "face_recognition"
FeatureVoiceRecognition = "voice_recognition"
FeatureAudioTransform = "audio_transform"
)
// AgentFeatures lists agent-related features (default OFF).
@@ -67,7 +68,7 @@ var APIFeatures = []string{
FeatureChat, FeatureImages, FeatureAudioSpeech, FeatureAudioTranscription,
FeatureVAD, FeatureDetection, FeatureVideo, FeatureEmbeddings, FeatureSound,
FeatureRealtime, FeatureRerank, FeatureTokenize, FeatureMCP, FeatureStores,
FeatureFaceRecognition, FeatureVoiceRecognition,
FeatureFaceRecognition, FeatureVoiceRecognition, FeatureAudioTransform,
}
// AllFeatures lists all known features (used by UI and validation).

413
core/http/endpoints/localai/audio_transform.go Normal file
View File

@@ -0,0 +1,413 @@
package localai
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"mime/multipart"
"net/http"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"sync"
"github.com/gorilla/websocket"
"github.com/labstack/echo/v4"
"github.com/mudler/LocalAI/core/application"
"github.com/mudler/LocalAI/core/backend"
"github.com/mudler/LocalAI/core/config"
"github.com/mudler/LocalAI/core/http/middleware"
"github.com/mudler/LocalAI/core/schema"
"github.com/mudler/LocalAI/pkg/audio"
"github.com/mudler/LocalAI/pkg/grpc/proto"
"github.com/mudler/LocalAI/pkg/model"
"github.com/mudler/LocalAI/pkg/utils"
"github.com/mudler/xlog"
)
// audioTransformWSUpgrader allows WebSocket connections from any origin —
// matches the realtime endpoint's policy. Authentication is handled at the
// HTTP layer before the upgrade.
var audioTransformWSUpgrader = websocket.Upgrader{
CheckOrigin: func(r *http.Request) bool { return true },
}
const (
// audioTransformWSReadLimit is the per-message ceiling on inbound WS
// frames. With 16 kHz / 256-sample / s16-stereo (1024 B/frame) the
// default ceiling is generous; raised here to 1 MiB to allow larger
// frame_samples for backends with longer hops.
audioTransformWSReadLimit = 1 << 20
)
// AudioTransformEndpoint implements the batch audio-transform API. Accepts a
// multipart/form-data request with `audio` (required) and an optional
// `reference` file. Backend-specific tuning is forwarded via repeated
// `params[<key>]=<value>` form fields. Returns the enhanced audio as an
// attachment, mirroring the /v1/audio/speech response shape.
//
// @Summary Transform audio (echo cancellation, noise suppression, voice conversion, etc.)
// @Description Runs an audio-in / audio-out transform conditioned on an optional auxiliary reference signal. Concrete transforms include AEC + noise suppression + dereverberation (LocalVQE), voice conversion (reference = target speaker), and pitch shifting. The backend determines the operation; pass model-specific tuning via repeated `params[<key>]=<value>` form fields.
// @Tags audio
// @Accept multipart/form-data
// @Produce audio/x-wav
// @Param model formData string true "model"
// @Param audio formData file true "primary input audio file"
// @Param reference formData file false "auxiliary reference audio (loopback for AEC, target voice for conversion, etc.)"
// @Param response_format formData string false "wav | mp3 | ogg | flac"
// @Param sample_rate formData integer false "desired output sample rate"
// @Success 200 {string} binary "transformed audio file"
// @Router /audio/transformations [post]
// @Router /audio/transform [post]
func AudioTransformEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoader, appConfig *config.ApplicationConfig) echo.HandlerFunc {
return func(c echo.Context) error {
input, ok := c.Get(middleware.CONTEXT_LOCALS_KEY_LOCALAI_REQUEST).(*schema.AudioTransformRequest)
if !ok || input.Model == "" {
return echo.ErrBadRequest
}
cfg, ok := c.Get(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG).(*config.ModelConfig)
if !ok || cfg == nil {
return echo.ErrBadRequest
}
xlog.Debug("LocalAI Audio Transform Request received", "model", input.Model)
audioFile, err := c.FormFile("audio")
if err != nil {
return echo.NewHTTPError(http.StatusBadRequest, "missing required 'audio' file field")
}
dir, err := os.MkdirTemp("", "audio-transform")
if err != nil {
return err
}
defer func() { _ = os.RemoveAll(dir) }()
audioPath, err := saveMultipartFileAsWAV(audioFile, dir, "audio")
if err != nil {
return err
}
var referencePath string
if refFile, err := c.FormFile("reference"); err == nil {
referencePath, err = saveMultipartFileAsWAV(refFile, dir, "reference")
if err != nil {
return err
}
}
params := collectParamsFromForm(c)
// Form-field params override schema-body params on collision.
for k, v := range input.Params {
if _, exists := params[k]; !exists {
params[k] = v
}
}
out, _, err := backend.ModelAudioTransform(audioPath, referencePath, backend.AudioTransformOptions{
Params: params,
}, ml, appConfig, *cfg)
if err != nil {
return err
}
dst := out.Dst
if input.SampleRate > 0 {
dst, err = utils.AudioResample(dst, input.SampleRate)
if err != nil {
return err
}
}
dst, err = utils.AudioConvert(dst, input.Format)
if err != nil {
return err
}
dst, contentType := audio.NormalizeAudioFile(dst)
if contentType != "" {
c.Response().Header().Set(echo.HeaderContentType, contentType)
}
// Expose the persisted inputs so the React UI can save them in
// history alongside the output. The /generated-audio/ prefix is
// the same one ttsApi uses (parsed from Content-Disposition).
if name := filepath.Base(out.AudioPath); name != "" {
c.Response().Header().Set(echo.HeaderAccessControlExposeHeaders, "X-Audio-Input-Url, X-Audio-Reference-Url")
c.Response().Header().Set("X-Audio-Input-Url", "/generated-audio/"+name)
}
if out.ReferencePath != "" {
if name := filepath.Base(out.ReferencePath); name != "" {
c.Response().Header().Set("X-Audio-Reference-Url", "/generated-audio/"+name)
}
}
return c.Attachment(dst, filepath.Base(dst))
}
}
// Wire protocol documented in docs/content/features/audio-transform.md
// and on schema.AudioTransformStreamControl.
//
// @Summary Bidirectional realtime audio transform over WebSocket.
// @Description Streams binary PCM frames in (interleaved stereo: ch0=audio, ch1=reference) and out (mono). The first message must be a JSON `session.update` envelope describing model + sample format + frame size + backend params. Server emits binary PCM on the same cadence.
// @Tags audio
// @Router /audio/transformations/stream [get]
func AudioTransformStreamEndpoint(app *application.Application) echo.HandlerFunc {
return func(c echo.Context) error {
ws, err := audioTransformWSUpgrader.Upgrade(c.Response(), c.Request(), nil)
if err != nil {
return err
}
defer func() { _ = ws.Close() }()
ws.SetReadLimit(audioTransformWSReadLimit)
mt, payload, err := ws.ReadMessage()
if err != nil {
xlog.Debug("audio_transform stream: client closed before session.update", "error", err)
return nil
}
if mt != websocket.TextMessage {
sendWSError(ws, "expected JSON session.update as first message")
return nil
}
var ctrl schema.AudioTransformStreamControl
if err := json.Unmarshal(payload, &ctrl); err != nil {
sendWSError(ws, "invalid JSON: "+err.Error())
return nil
}
if ctrl.Type != schema.AudioTransformCtrlSessionUpdate {
sendWSError(ws, "first message must be "+schema.AudioTransformCtrlSessionUpdate)
return nil
}
if ctrl.Model == "" {
sendWSError(ws, "session.update missing model")
return nil
}
cfg, err := app.ModelConfigLoader().LoadModelConfigFileByNameDefaultOptions(ctrl.Model, app.ApplicationConfig())
if err != nil || cfg == nil {
sendWSError(ws, fmt.Sprintf("failed to load model config: %v", err))
return nil
}
ctx, cancel := context.WithCancel(c.Request().Context())
defer cancel()
stream, err := backend.ModelAudioTransformStream(ctx, app.ModelLoader(), app.ApplicationConfig(), *cfg)
if err != nil {
sendWSError(ws, fmt.Sprintf("failed to open transform stream: %v", err))
return nil
}
sampleFormat, err := parseSampleFormat(ctrl.SampleFormat)
if err != nil {
sendWSError(ws, err.Error())
return nil
}
if err := stream.Send(buildConfigRequest(sampleFormat, &ctrl)); err != nil {
sendWSError(ws, fmt.Sprintf("backend send config: %v", err))
return nil
}
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
for {
resp, err := stream.Recv()
if err != nil {
if !errors.Is(err, io.EOF) {
sendWSError(ws, fmt.Sprintf("backend recv: %v", err))
}
return
}
if err := ws.WriteMessage(websocket.BinaryMessage, resp.Pcm); err != nil {
return
}
}
}()
// Per-connection scratch for stereo de-interleaving — avoids two
// allocs per inbound binary frame at the 16 ms cadence.
var audioBuf, refBuf []byte
readLoop:
for {
mt, payload, err := ws.ReadMessage()
if err != nil {
_ = stream.CloseSend()
break readLoop
}
switch mt {
case websocket.BinaryMessage:
audio, ref := splitStereoFrameInto(payload, sampleFormat, &audioBuf, &refBuf)
if err := stream.Send(&proto.AudioTransformFrameRequest{
Payload: &proto.AudioTransformFrameRequest_Frame{
Frame: &proto.AudioTransformFrame{
AudioPcm: audio,
ReferencePcm: ref,
},
},
}); err != nil {
sendWSError(ws, fmt.Sprintf("backend send frame: %v", err))
_ = stream.CloseSend()
break readLoop
}
case websocket.TextMessage:
var ctrl schema.AudioTransformStreamControl
if err := json.Unmarshal(payload, &ctrl); err != nil {
sendWSError(ws, "invalid mid-stream JSON: "+err.Error())
continue
}
switch ctrl.Type {
case schema.AudioTransformCtrlSessionUpdate:
_ = stream.Send(buildConfigRequest(sampleFormat, &ctrl))
case schema.AudioTransformCtrlSessionClose:
_ = stream.CloseSend()
}
}
}
wg.Wait()
return nil
}
}
func parseSampleFormat(s string) (proto.AudioTransformStreamConfig_SampleFormat, error) {
switch strings.ToUpper(s) {
case schema.AudioTransformSampleFormatF32LE:
return proto.AudioTransformStreamConfig_F32_LE, nil
case schema.AudioTransformSampleFormatS16LE, "":
return proto.AudioTransformStreamConfig_S16_LE, nil
default:
return 0, fmt.Errorf("unsupported sample_format: %q", s)
}
}
func buildConfigRequest(fmt_ proto.AudioTransformStreamConfig_SampleFormat, ctrl *schema.AudioTransformStreamControl) *proto.AudioTransformFrameRequest {
return &proto.AudioTransformFrameRequest{
Payload: &proto.AudioTransformFrameRequest_Config{
Config: &proto.AudioTransformStreamConfig{
SampleFormat: fmt_,
SampleRate: int32(ctrl.SampleRate),
FrameSamples: int32(ctrl.FrameSamples),
Params: ctrl.Params,
Reset_: ctrl.Reset,
},
},
}
}
// saveMultipartFileAsWAV materialises an uploaded multipart file into `dir`
// and converts it to LocalVQE's required shape (16 kHz mono s16 WAV) via
// ffmpeg. The conversion is a passthrough when the upload already matches.
// `name` is used as the base filename for the converted output so the dir
// stays readable for debugging (e.g. "audio.wav", "reference.wav").
func saveMultipartFileAsWAV(fh *multipart.FileHeader, dir, name string) (string, error) {
f, err := fh.Open()
if err != nil {
return "", err
}
defer func() { _ = f.Close() }()
raw := filepath.Join(dir, "raw-"+path.Base(fh.Filename))
out, err := os.Create(raw)
if err != nil {
return "", err
}
if _, err := io.Copy(out, f); err != nil {
_ = out.Close()
return "", err
}
_ = out.Close()
dst := filepath.Join(dir, name+".wav")
if err := utils.AudioToWav(raw, dst); err != nil {
return "", fmt.Errorf("normalize %s: %w", name, err)
}
return dst, nil
}
// collectParamsFromForm walks the multipart form values and harvests any
// that match the `params[<key>]` shape. Returns nil if there are no matches.
func collectParamsFromForm(c echo.Context) map[string]string {
params := map[string]string{}
form, err := c.FormParams()
if err != nil {
return params
}
for key, vals := range form {
if len(vals) == 0 {
continue
}
if !strings.HasPrefix(key, "params[") || !strings.HasSuffix(key, "]") {
continue
}
inner := strings.TrimSuffix(strings.TrimPrefix(key, "params["), "]")
inner = strings.TrimSpace(inner)
if inner == "" {
continue
}
// Last value wins for duplicate keys — matches OpenAI's form-field
// override semantics.
params[inner] = vals[len(vals)-1]
}
// Form-field shortcuts for the common LocalVQE knobs. params[*] still wins
// when both are provided (they ran first).
if _, exists := params[schema.AudioTransformParamNoiseGate]; !exists {
if v := c.FormValue(schema.AudioTransformParamNoiseGate); v != "" {
if b, err := strconv.ParseBool(v); err == nil {
if b {
params[schema.AudioTransformParamNoiseGate] = "true"
} else {
params[schema.AudioTransformParamNoiseGate] = "false"
}
}
}
}
if _, exists := params[schema.AudioTransformParamNoiseGateThreshold]; !exists {
if v := c.FormValue(schema.AudioTransformParamNoiseGateThreshold); v != "" {
params[schema.AudioTransformParamNoiseGateThreshold] = v
}
}
return params
}
// splitStereoFrameInto deinterleaves a stereo PCM frame in-place into
// caller-owned reusable buffers (channel 0 → audio, channel 1 → reference).
// Sample size is inferred from the proto enum: s16=2 B, f32=4 B. Trailing
// odd bytes are truncated.
func splitStereoFrameInto(buf []byte, fmt_ proto.AudioTransformStreamConfig_SampleFormat, audio, ref *[]byte) ([]byte, []byte) {
sampleSize := 2
if fmt_ == proto.AudioTransformStreamConfig_F32_LE {
sampleSize = 4
}
stride := sampleSize * 2
n := len(buf) / stride
want := n * sampleSize
if cap(*audio) < want {
*audio = make([]byte, want)
} else {
*audio = (*audio)[:want]
}
if cap(*ref) < want {
*ref = make([]byte, want)
} else {
*ref = (*ref)[:want]
}
for i := 0; i < n; i++ {
copy((*audio)[i*sampleSize:(i+1)*sampleSize], buf[i*stride:i*stride+sampleSize])
copy((*ref)[i*sampleSize:(i+1)*sampleSize], buf[i*stride+sampleSize:(i+1)*stride])
}
return *audio, *ref
}
func sendWSError(ws *websocket.Conn, msg string) {
payload, _ := json.Marshal(schema.AudioTransformStreamControl{
Type: schema.AudioTransformCtrlError,
Error: msg,
})
_ = ws.WriteMessage(websocket.TextMessage, payload)
}

2
core/http/endpoints/localai/backend.go
View File

@@ -36,6 +36,8 @@ var knownPrefOnlyBackends = []schema.KnownBackend{
{Name: "faster-qwen3-tts", Modality: "tts", AutoDetect: false, Description: "Faster Qwen3 TTS (preference-only)"},
// Detection
{Name: "sam3-cpp", Modality: "detection", AutoDetect: false, Description: "SAM3 C++ object detection (preference-only)"},
// Audio transform (audio-in / audio-out, optional reference signal)
{Name: "localvqe", Modality: "audio-transform", AutoDetect: false, Description: "LocalVQE C++ joint AEC + noise suppression + dereverberation (preference-only)"},
}
// UpgradeInfoProvider is an interface for querying cached backend upgrade information.

337
core/http/react-ui/bun.lock
View File

@@ -17,11 +17,15 @@
"@lezer/highlight": "^1.2.1",
"@modelcontextprotocol/ext-apps": "^1.2.2",
"@modelcontextprotocol/sdk": "^1.25.1",
"dompurify": "^3.2.5",
"dompurify": "^3.4.0",
"highlight.js": "^11.11.1",
"i18next": "^26.0.8",
"i18next-browser-languagedetector": "^8.2.1",
"i18next-http-backend": "^3.0.6",
"marked": "^15.0.7",
"react": "^19.1.0",
"react-dom": "^19.1.0",
"react-i18next": "^17.0.6",
"react-router-dom": "^7.6.1",
"yaml": "^2.8.3",
},
@@ -33,7 +37,8 @@
"eslint-plugin-react-hooks": "^5.2.0",
"eslint-plugin-react-refresh": "^0.4.20",
"globals": "^16.1.0",
"vite": "^6.3.5",
"i18next-parser": "^9.4.0",
"vite": "^6.4.2",
},
},
},
@@ -70,6 +75,8 @@
"@babel/plugin-transform-react-jsx-source": ["@babel/plugin-transform-react-jsx-source@7.27.1", "", { "dependencies": { "@babel/helper-plugin-utils": "^7.27.1" }, "peerDependencies": { "@babel/core": "^7.0.0-0" } }, "sha512-zbwoTsBruTeKB9hSq73ha66iFeJHuaFkUbwvqElnygoNbj/jHRsSeokowZFN3CZ64IvEqcmmkVe89OPXc7ldAw=="],
"@babel/runtime": ["@babel/runtime@7.29.2", "", {}, "sha512-JiDShH45zKHWyGe4ZNVRrCjBz8Nh9TMmZG1kh4QTK8hCBTWBi8Da+i7s1fJw7/lYpM4ccepSNfqzZ/QvABBi5g=="],
"@babel/template": ["@babel/template@7.28.6", "", { "dependencies": { "@babel/code-frame": "^7.28.6", "@babel/parser": "^7.28.6", "@babel/types": "^7.28.6" } }, "sha512-YA6Ma2KsCdGb+WC6UpBVFJGXL58MDA6oyONbjyF/+5sBgxY/dwkhLogbMT2GXXyU84/IhRw/2D1Os1B/giz+BQ=="],
"@babel/traverse": ["@babel/traverse@7.29.0", "", { "dependencies": { "@babel/code-frame": "^7.29.0", "@babel/generator": "^7.29.0", "@babel/helper-globals": "^7.28.0", "@babel/parser": "^7.29.0", "@babel/template": "^7.28.6", "@babel/types": "^7.29.0", "debug": "^4.3.1" } }, "sha512-4HPiQr0X7+waHfyXPZpWPfWL/J7dcN1mx9gL6WdQVMbPnF3+ZhSMs8tCxN7oHddJE9fhNE7+lxdnlyemKfJRuA=="],
@@ -164,6 +171,8 @@
"@fortawesome/fontawesome-free": ["@fortawesome/fontawesome-free@6.7.2", "", {}, "sha512-JUOtgFW6k9u4Y+xeIaEiLr3+cjoUPiAuLXoyKOJSia6Duzb7pq+A76P9ZdPDoAoxHdHzq6gE9/jKBGXlZT8FbA=="],
"@gulpjs/to-absolute-glob": ["@gulpjs/to-absolute-glob@4.0.0", "", { "dependencies": { "is-negated-glob": "^1.0.0" } }, "sha512-kjotm7XJrJ6v+7knhPaRgaT6q8F8K2jiafwYdNHLzmV0uGLuZY43FK6smNSHUPrhq5kX2slCUy+RGG/xGqmIKA=="],
"@hono/node-server": ["@hono/node-server@1.19.11", "", { "peerDependencies": { "hono": "^4" } }, "sha512-dr8/3zEaB+p0D2n/IUrlPF1HZm586qgJNXK1a9fhg/PzdtkK7Ksd5l312tJX2yBuALqDYBlG20QEbayqPyxn+g=="],
"@humanfs/core": ["@humanfs/core@0.19.1", "", {}, "sha512-5DyQ4+1JEUzejeK1JGICcideyfUbGixgS9jNgex5nqkW+cY7WZhxBigmieN5Qnw9ZosSNVC9KQKyb+GUaGyKUA=="],
@@ -174,6 +183,8 @@
"@humanwhocodes/retry": ["@humanwhocodes/retry@0.4.3", "", {}, "sha512-bV0Tgo9K4hfPCek+aMAn81RppFKv2ySDQeMoSZuvTASywNTnVJCArCZE2FWqpvIatKu7VMRLWlR1EazvVhDyhQ=="],
"@isaacs/cliui": ["@isaacs/cliui@8.0.2", "", { "dependencies": { "string-width": "^5.1.2", "string-width-cjs": "npm:string-width@^4.2.0", "strip-ansi": "^7.0.1", "strip-ansi-cjs": "npm:strip-ansi@^6.0.1", "wrap-ansi": "^8.1.0", "wrap-ansi-cjs": "npm:wrap-ansi@^7.0.0" } }, "sha512-O8jcjabXaleOG9DQ0+ARXWZBTfnP4WNAqzuiJK7ll44AmxGKv/J2M4TPjxjY3znBCfvBXFzucm1twdyFybFqEA=="],
"@jridgewell/gen-mapping": ["@jridgewell/gen-mapping@0.3.13", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.0", "@jridgewell/trace-mapping": "^0.3.24" } }, "sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA=="],
"@jridgewell/remapping": ["@jridgewell/remapping@2.3.5", "", { "dependencies": { "@jridgewell/gen-mapping": "^0.3.5", "@jridgewell/trace-mapping": "^0.3.24" } }, "sha512-LI9u/+laYG4Ds1TDKSJW2YPrIlcVYOwi2fUC6xB43lueCjgxV4lffOCZCtYFiH6TNOX+tQKXx97T4IKHbhyHEQ=="],
@@ -198,6 +209,8 @@
"@modelcontextprotocol/sdk": ["@modelcontextprotocol/sdk@1.27.1", "", { "dependencies": { "@hono/node-server": "^1.19.9", "ajv": "^8.17.1", "ajv-formats": "^3.0.1", "content-type": "^1.0.5", "cors": "^2.8.5", "cross-spawn": "^7.0.5", "eventsource": "^3.0.2", "eventsource-parser": "^3.0.0", "express": "^5.2.1", "express-rate-limit": "^8.2.1", "hono": "^4.11.4", "jose": "^6.1.3", "json-schema-typed": "^8.0.2", "pkce-challenge": "^5.0.0", "raw-body": "^3.0.0", "zod": "^3.25 || ^4.0", "zod-to-json-schema": "^3.25.1" }, "peerDependencies": { "@cfworker/json-schema": "^4.1.1" }, "optionalPeers": ["@cfworker/json-schema"] }, "sha512-sr6GbP+4edBwFndLbM60gf07z0FQ79gaExpnsjMGePXqFcSSb7t6iscpjk9DhFhwd+mTEQrzNafGP8/iGGFYaA=="],
"@pkgjs/parseargs": ["@pkgjs/parseargs@0.11.0", "", {}, "sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg=="],
"@playwright/test": ["@playwright/test@1.58.2", "", { "dependencies": { "playwright": "1.58.2" }, "bin": { "playwright": "cli.js" } }, "sha512-akea+6bHYBBfA9uQqSYmlJXn61cTa+jbO87xVLCWbTqbWadRVmhxlXATaOjOgcBaWU4ePo0wB41KMFv3o35IXA=="],
"@rolldown/pluginutils": ["@rolldown/pluginutils@1.0.0-beta.27", "", {}, "sha512-+d0F4MKMCbeVUJwG96uQ4SgAznZNSq93I3V+9NHA4OpvqG8mRCpGdKmK8l/dl02h2CCDHwW2FqilnTyDcAnqjA=="],
@@ -264,6 +277,10 @@
"@types/json-schema": ["@types/json-schema@7.0.15", "", {}, "sha512-5+fP8P8MFNC+AyZCDxrB2pkZFPGzqQWUzpSeuuVLvm8VMcorNYavBqoFcxK8bQz4Qsbn4oUEEem4wDLfcysGHA=="],
"@types/minimatch": ["@types/minimatch@3.0.5", "", {}, "sha512-Klz949h02Gz2uZCMGwDUSDS1YBlTdDDgbWHi+81l29tQALUtvz4rAYi5uoVhE5Lagoq6DeqAUlbrHvW/mXDgdQ=="],
"@types/symlink-or-copy": ["@types/symlink-or-copy@1.2.2", "", {}, "sha512-MQ1AnmTLOncwEf9IVU+B2e4Hchrku5N67NkgcAHW0p3sdzPe0FNMANxEm6OJUzPniEQGkeT3OROLlCwZJLWFZA=="],
"@types/trusted-types": ["@types/trusted-types@2.0.7", "", {}, "sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw=="],
"@vitejs/plugin-react": ["@vitejs/plugin-react@4.7.0", "", { "dependencies": { "@babel/core": "^7.28.0", "@babel/plugin-transform-react-jsx-self": "^7.27.1", "@babel/plugin-transform-react-jsx-source": "^7.27.1", "@rolldown/pluginutils": "1.0.0-beta.27", "@types/babel__core": "^7.20.5", "react-refresh": "^0.17.0" }, "peerDependencies": { "vite": "^4.2.0 || ^5.0.0 || ^6.0.0 || ^7.0.0" } }, "sha512-gUu9hwfWvvEDBBmgtAowQCojwZmJ5mcLn3aufeCsitijs3+f2NsrPtlAWIR6OPiqljl96GVCUbLe0HyqIpVaoA=="],
@@ -278,20 +295,44 @@
"ajv-formats": ["ajv-formats@3.0.1", "", { "dependencies": { "ajv": "^8.0.0" } }, "sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ=="],
"ansi-regex": ["ansi-regex@6.2.2", "", {}, "sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg=="],
"ansi-styles": ["ansi-styles@4.3.0", "", { "dependencies": { "color-convert": "^2.0.1" } }, "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg=="],
"anymatch": ["anymatch@3.1.3", "", { "dependencies": { "normalize-path": "^3.0.0", "picomatch": "^2.0.4" } }, "sha512-KMReFUr0B4t+D+OBkjR3KYqvocp2XaSzO55UcB6mgQMd3KbcE+mWTyvVV7D/zsdEbNnV6acZUutkiHQXvTr1Rw=="],
"argparse": ["argparse@2.0.1", "", {}, "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q=="],
"b4a": ["b4a@1.8.1", "", { "peerDependencies": { "react-native-b4a": "*" }, "optionalPeers": ["react-native-b4a"] }, "sha512-aiqre1Nr0B/6DgE2N5vwTc+2/oQZ4Wh1t4NznYY4E00y8LCt6NqdRv81so00oo27D8MVKTpUa/MwUUtBLXCoDw=="],
"balanced-match": ["balanced-match@1.0.2", "", {}, "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw=="],
"bare-events": ["bare-events@2.8.2", "", { "peerDependencies": { "bare-abort-controller": "*" }, "optionalPeers": ["bare-abort-controller"] }, "sha512-riJjyv1/mHLIPX4RwiK+oW9/4c3TEUeORHKefKAKnZ5kyslbN+HXowtbaVEqt4IMUB7OXlfixcs6gsFeo/jhiQ=="],
"base64-js": ["base64-js@1.5.1", "", {}, "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA=="],
"baseline-browser-mapping": ["baseline-browser-mapping@2.10.0", "", { "bin": { "baseline-browser-mapping": "dist/cli.cjs" } }, "sha512-lIyg0szRfYbiy67j9KN8IyeD7q7hcmqnJ1ddWmNt19ItGpNN64mnllmxUNFIOdOm6by97jlL6wfpTTJrmnjWAA=="],
"bl": ["bl@5.1.0", "", { "dependencies": { "buffer": "^6.0.3", "inherits": "^2.0.4", "readable-stream": "^3.4.0" } }, "sha512-tv1ZJHLfTDnXE6tMHv73YgSJaWR2AFuPwMntBe7XL/GBFHnT0CLnsHMogfk5+GzCDC5ZWarSCYaIGATZt9dNsQ=="],
"body-parser": ["body-parser@2.2.2", "", { "dependencies": { "bytes": "^3.1.2", "content-type": "^1.0.5", "debug": "^4.4.3", "http-errors": "^2.0.0", "iconv-lite": "^0.7.0", "on-finished": "^2.4.1", "qs": "^6.14.1", "raw-body": "^3.0.1", "type-is": "^2.0.1" } }, "sha512-oP5VkATKlNwcgvxi0vM0p/D3n2C3EReYVX+DNYs5TjZFn/oQt2j+4sVJtSMr18pdRr8wjTcBl6LoV+FUwzPmNA=="],
"boolbase": ["boolbase@1.0.0", "", {}, "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww=="],
"brace-expansion": ["brace-expansion@1.1.12", "", { "dependencies": { "balanced-match": "^1.0.0", "concat-map": "0.0.1" } }, "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg=="],
"broccoli-node-api": ["broccoli-node-api@1.7.0", "", {}, "sha512-QIqLSVJWJUVOhclmkmypJJH9u9s/aWH4+FH6Q6Ju5l+Io4dtwqdPUNmDfw40o6sxhbZHhqGujDJuHTML1wG8Yw=="],
"broccoli-node-info": ["broccoli-node-info@2.2.0", "", {}, "sha512-VabSGRpKIzpmC+r+tJueCE5h8k6vON7EIMMWu6d/FyPdtijwLQ7QvzShEw+m3mHoDzUaj/kiZsDYrS8X2adsBg=="],
"broccoli-output-wrapper": ["broccoli-output-wrapper@3.2.5", "", { "dependencies": { "fs-extra": "^8.1.0", "heimdalljs-logger": "^0.1.10", "symlink-or-copy": "^1.2.0" } }, "sha512-bQAtwjSrF4Nu0CK0JOy5OZqw9t5U0zzv2555EA/cF8/a8SLDTIetk9UgrtMVw7qKLKdSpOZ2liZNeZZDaKgayw=="],
"broccoli-plugin": ["broccoli-plugin@4.0.7", "", { "dependencies": { "broccoli-node-api": "^1.7.0", "broccoli-output-wrapper": "^3.2.5", "fs-merger": "^3.2.1", "promise-map-series": "^0.3.0", "quick-temp": "^0.1.8", "rimraf": "^3.0.2", "symlink-or-copy": "^1.3.1" } }, "sha512-a4zUsWtA1uns1K7p9rExYVYG99rdKeGRymW0qOCNkvDPHQxVi3yVyJHhQbM3EZwdt2E0mnhr5e0c/bPpJ7p3Wg=="],
"browserslist": ["browserslist@4.28.1", "", { "dependencies": { "baseline-browser-mapping": "^2.9.0", "caniuse-lite": "^1.0.30001759", "electron-to-chromium": "^1.5.263", "node-releases": "^2.0.27", "update-browserslist-db": "^1.2.0" }, "bin": { "browserslist": "cli.js" } }, "sha512-ZC5Bd0LgJXgwGqUknZY/vkUQ04r8NXnJZ3yYi4vDmSiZmC/pdSN0NbNRPxZpbtO4uAfDUAFffO8IZoM3Gj8IkA=="],
"buffer": ["buffer@6.0.3", "", { "dependencies": { "base64-js": "^1.3.1", "ieee754": "^1.2.1" } }, "sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA=="],
"bytes": ["bytes@3.1.2", "", {}, "sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg=="],
"call-bind-apply-helpers": ["call-bind-apply-helpers@1.0.2", "", { "dependencies": { "es-errors": "^1.3.0", "function-bind": "^1.1.2" } }, "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ=="],
@@ -304,10 +345,20 @@
"chalk": ["chalk@4.1.2", "", { "dependencies": { "ansi-styles": "^4.1.0", "supports-color": "^7.1.0" } }, "sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA=="],
"cheerio": ["cheerio@1.2.0", "", { "dependencies": { "cheerio-select": "^2.1.0", "dom-serializer": "^2.0.0", "domhandler": "^5.0.3", "domutils": "^3.2.2", "encoding-sniffer": "^0.2.1", "htmlparser2": "^10.1.0", "parse5": "^7.3.0", "parse5-htmlparser2-tree-adapter": "^7.1.0", "parse5-parser-stream": "^7.1.2", "undici": "^7.19.0", "whatwg-mimetype": "^4.0.0" } }, "sha512-WDrybc/gKFpTYQutKIK6UvfcuxijIZfMfXaYm8NMsPQxSYvf+13fXUJ4rztGGbJcBQ/GF55gvrZ0Bc0bj/mqvg=="],
"cheerio-select": ["cheerio-select@2.1.0", "", { "dependencies": { "boolbase": "^1.0.0", "css-select": "^5.1.0", "css-what": "^6.1.0", "domelementtype": "^2.3.0", "domhandler": "^5.0.3", "domutils": "^3.0.1" } }, "sha512-9v9kG0LvzrlcungtnJtpGNxY+fzECQKhK4EGJX2vByejiMX84MFNQw4UxPJl3bFbTMw+Dfs37XaIkCwTZfLh4g=="],
"clone": ["clone@2.1.2", "", {}, "sha512-3Pe/CF1Nn94hyhIYpjtiLhdCoEoz0DqQ+988E9gmeEdQZlojxnOb74wctFyuwWQHzqyf9X7C7MG8juUpqBJT8w=="],
"color-convert": ["color-convert@2.0.1", "", { "dependencies": { "color-name": "~1.1.4" } }, "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ=="],
"color-name": ["color-name@1.1.4", "", {}, "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA=="],
"colors": ["colors@1.4.0", "", {}, "sha512-a+UqTh4kgZg/SlGvfbzDHpgRu7AAQOmmqRHJnxhRZICKFUT91brVhNNt58CMWU9PsBbv3PDCZUHbVxuDiH2mtA=="],
"commander": ["commander@12.1.0", "", {}, "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA=="],
"concat-map": ["concat-map@0.0.1", "", {}, "sha512-/Srv4dswyQNBfohGpz9o6Yb3Gz3SrUDqBH5rTuhGR7ahtlbYKnVxw2bCFMRljaA7EXHaXZ8wsHdodFvbkhKmqg=="],
"content-disposition": ["content-disposition@1.0.1", "", {}, "sha512-oIXISMynqSqm241k6kcQ5UwttDILMK4BiurCfGEREw6+X9jkkpEe5T9FZaApyLGGOnFuyMWZpdolTXMtvEJ08Q=="],
@@ -320,28 +371,56 @@
"cookie-signature": ["cookie-signature@1.2.2", "", {}, "sha512-D76uU73ulSXrD1UXF4KE2TMxVVwhsnCgfAyTg9k8P6KGZjlXKrOLe4dJQKI3Bxi5wjesZoFXJWElNWBjPZMbhg=="],
"core-util-is": ["core-util-is@1.0.3", "", {}, "sha512-ZQBvi1DcpJ4GDqanjucZ2Hj3wEO5pZDS89BWbkcrvdxksJorwUDDZamX9ldFkp9aw2lmBDLgkObEA4DWNJ9FYQ=="],
"cors": ["cors@2.8.6", "", { "dependencies": { "object-assign": "^4", "vary": "^1" } }, "sha512-tJtZBBHA6vjIAaF6EnIaq6laBBP9aq/Y3ouVJjEfoHbRBcHBAHYcMh/w8LDrk2PvIMMq8gmopa5D4V8RmbrxGw=="],
"crelt": ["crelt@1.0.6", "", {}, "sha512-VQ2MBenTq1fWZUH9DJNGti7kKv6EeAuYr3cLwxUWhIu1baTaXh4Ib5W2CqHVqib4/MqbYGJqiL3Zb8GJZr3l4g=="],
"cross-fetch": ["cross-fetch@4.1.0", "", { "dependencies": { "node-fetch": "^2.7.0" } }, "sha512-uKm5PU+MHTootlWEY+mZ4vvXoCn4fLQxT9dSc1sXVMSFkINTJVN8cAQROpwcKm8bJ/c7rgZVIBWzH5T78sNZZw=="],
"cross-spawn": ["cross-spawn@7.0.6", "", { "dependencies": { "path-key": "^3.1.0", "shebang-command": "^2.0.0", "which": "^2.0.1" } }, "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA=="],
"css-select": ["css-select@5.2.2", "", { "dependencies": { "boolbase": "^1.0.0", "css-what": "^6.1.0", "domhandler": "^5.0.2", "domutils": "^3.0.1", "nth-check": "^2.0.1" } }, "sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw=="],
"css-what": ["css-what@6.2.2", "", {}, "sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA=="],
"debug": ["debug@4.4.3", "", { "dependencies": { "ms": "^2.1.3" } }, "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA=="],
"deep-is": ["deep-is@0.1.4", "", {}, "sha512-oIPzksmTg4/MriiaYGO+okXDT7ztn/w3Eptv/+gSIdMdKsJo0u4CfYNFJPy+4SKMuCqGw2wxnA+URMg3t8a/bQ=="],
"depd": ["depd@2.0.0", "", {}, "sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw=="],
"dompurify": ["dompurify@3.3.1", "", { "optionalDependencies": { "@types/trusted-types": "^2.0.7" } }, "sha512-qkdCKzLNtrgPFP1Vo+98FRzJnBRGe4ffyCea9IwHB1fyxPOeNTHpLKYGd4Uk9xvNoH0ZoOjwZxNptyMwqrId1Q=="],
"dom-serializer": ["dom-serializer@2.0.0", "", { "dependencies": { "domelementtype": "^2.3.0", "domhandler": "^5.0.2", "entities": "^4.2.0" } }, "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg=="],
"domelementtype": ["domelementtype@2.3.0", "", {}, "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw=="],
"domhandler": ["domhandler@5.0.3", "", { "dependencies": { "domelementtype": "^2.3.0" } }, "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w=="],
"dompurify": ["dompurify@3.4.1", "", { "optionalDependencies": { "@types/trusted-types": "^2.0.7" } }, "sha512-JahakDAIg1gyOm7dlgWSDjV4n7Ip2PKR55NIT6jrMfIgLFgWo81vdr1/QGqWtFNRqXP9UV71oVePtjqS2ebnPw=="],
"domutils": ["domutils@3.2.2", "", { "dependencies": { "dom-serializer": "^2.0.0", "domelementtype": "^2.3.0", "domhandler": "^5.0.3" } }, "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw=="],
"dunder-proto": ["dunder-proto@1.0.1", "", { "dependencies": { "call-bind-apply-helpers": "^1.0.1", "es-errors": "^1.3.0", "gopd": "^1.2.0" } }, "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A=="],
"eastasianwidth": ["eastasianwidth@0.2.0", "", {}, "sha512-I88TYZWc9XiYHRQ4/3c5rjjfgkjhLyW2luGIheGERbNQ6OY7yTybanSpDXZa8y7VUP9YmDcYa+eyq4ca7iLqWA=="],
"ee-first": ["ee-first@1.1.1", "", {}, "sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow=="],
"electron-to-chromium": ["electron-to-chromium@1.5.307", "", {}, "sha512-5z3uFKBWjiNR44nFcYdkcXjKMbg5KXNdciu7mhTPo9tB7NbqSNP2sSnGR+fqknZSCwKkBN+oxiiajWs4dT6ORg=="],
"emoji-regex": ["emoji-regex@9.2.2", "", {}, "sha512-L18DaJsXSUk2+42pv8mLs5jJT2hqFkFE4j21wOmgbUqsZ2hL72NsUU785g9RXgo3s0ZNgVl42TiHp3ZtOv/Vyg=="],
"encodeurl": ["encodeurl@2.0.0", "", {}, "sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg=="],
"encoding-sniffer": ["encoding-sniffer@0.2.1", "", { "dependencies": { "iconv-lite": "^0.6.3", "whatwg-encoding": "^3.1.1" } }, "sha512-5gvq20T6vfpekVtqrYQsSCFZ1wEg5+wW0/QaZMWkFr6BqD3NfKs0rLCx4rrVlSWJeZb5NBJgVLswK/w2MWU+Gw=="],
"ensure-posix-path": ["ensure-posix-path@1.1.1", "", {}, "sha512-VWU0/zXzVbeJNXvME/5EmLuEj2TauvoaTz6aFYK1Z92JCBlDlZ3Gu0tuGR42kpW1754ywTs+QB0g5TP0oj9Zaw=="],
"entities": ["entities@4.5.0", "", {}, "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw=="],
"eol": ["eol@0.9.1", "", {}, "sha512-Ds/TEoZjwggRoz/Q2O7SE3i4Jm66mqTDfmdHdq/7DKVk3bro9Q8h6WdXKdPqFLMoqxrDK5SVRzHVPOS6uuGtrg=="],
"es-define-property": ["es-define-property@1.0.1", "", {}, "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g=="],
"es-errors": ["es-errors@1.3.0", "", {}, "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw=="],
@@ -378,6 +457,8 @@
"etag": ["etag@1.8.1", "", {}, "sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg=="],
"events-universal": ["events-universal@1.0.1", "", { "dependencies": { "bare-events": "^2.7.0" } }, "sha512-LUd5euvbMLpwOF8m6ivPCbhQeSiYVNb8Vs0fQ8QjXo0JTkEHpz8pxdQf0gStltaPpw0Cca8b39KxvK9cfKRiAw=="],
"eventsource": ["eventsource@3.0.7", "", { "dependencies": { "eventsource-parser": "^3.0.1" } }, "sha512-CRT1WTyuQoD771GW56XEZFQ/ZoSfWid1alKGDYMmkt2yl8UXrVR4pspqWNEcqKvVIzg6PAltWjxcSSPrboA4iA=="],
"eventsource-parser": ["eventsource-parser@3.0.6", "", {}, "sha512-Vo1ab+QXPzZ4tCa8SwIHJFaSzy4R6SHf7BY79rFBDf0idraZWAkYrDjDj8uWaSm3S2TK+hJ7/t1CEmZ7jXw+pg=="],
@@ -388,12 +469,16 @@
"fast-deep-equal": ["fast-deep-equal@3.1.3", "", {}, "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q=="],
"fast-fifo": ["fast-fifo@1.3.2", "", {}, "sha512-/d9sfos4yxzpwkDkuN7k2SqFKtYNmCTzgfEpz82x34IM9/zc8KGxQoXg1liNC/izpRM/MBdt44Nmx41ZWqk+FQ=="],
"fast-json-stable-stringify": ["fast-json-stable-stringify@2.1.0", "", {}, "sha512-lhd/wF+Lk98HZoTCtlVraHtfh5XYijIjalXck7saUtuanSDyLMxnHhSXEDJqHxD7msR8D0uCmqlkwjCV8xvwHw=="],
"fast-levenshtein": ["fast-levenshtein@2.0.6", "", {}, "sha512-DCXu6Ifhqcks7TZKY3Hxp3y6qphY5SJZmrWMDrKcERSOXWQdMhU9Ig/PYrzyw/ul9jOIyh0N4M0tbC5hodg8dw=="],
"fast-uri": ["fast-uri@3.1.0", "", {}, "sha512-iPeeDKJSWf4IEOasVVrknXpaBV0IApz/gp7S2bb7Z4Lljbl2MGJRqInZiUrQwV16cpzw/D3S5j5Julj/gT52AA=="],
"fastq": ["fastq@1.20.1", "", { "dependencies": { "reusify": "^1.0.4" } }, "sha512-GGToxJ/w1x32s/D2EKND7kTil4n8OVk/9mycTc4VDza13lOvpUZTGX3mFSCtV9ksdGBVzvsyAVLM6mHFThxXxw=="],
"fdir": ["fdir@6.5.0", "", { "peerDependencies": { "picomatch": "^3 || ^4" }, "optionalPeers": ["picomatch"] }, "sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg=="],
"file-entry-cache": ["file-entry-cache@8.0.0", "", { "dependencies": { "flat-cache": "^4.0.0" } }, "sha512-XXTUwCvisa5oacNGRP9SfNtYBNAMi+RPwBFmblZEF7N7swHYQS6/Zfk7SRwx4D5j3CH211YNRco1DEMNVfZCnQ=="],
@@ -406,10 +491,22 @@
"flatted": ["flatted@3.3.4", "", {}, "sha512-3+mMldrTAPdta5kjX2G2J7iX4zxtnwpdA8Tr2ZSjkyPSanvbZAcy6flmtnXbEybHrDcU9641lxrMfFuUxVz9vA=="],
"foreground-child": ["foreground-child@3.3.1", "", { "dependencies": { "cross-spawn": "^7.0.6", "signal-exit": "^4.0.1" } }, "sha512-gIXjKqtFuWEgzFRJA9WCQeSJLZDjgJUOMCMzxtvFq/37KojM1BFGufqsCy0r4qSQmYLsZYMeyRqzIWOMup03sw=="],
"forwarded": ["forwarded@0.2.0", "", {}, "sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow=="],
"fresh": ["fresh@2.0.0", "", {}, "sha512-Rx/WycZ60HOaqLKAi6cHRKKI7zxWbJ31MhntmtwMoaTeF7XFH9hhBp8vITaMidfljRQ6eYWCKkaTK+ykVJHP2A=="],
"fs-extra": ["fs-extra@11.3.4", "", { "dependencies": { "graceful-fs": "^4.2.0", "jsonfile": "^6.0.1", "universalify": "^2.0.0" } }, "sha512-CTXd6rk/M3/ULNQj8FBqBWHYBVYybQ3VPBw0xGKFe3tuH7ytT6ACnvzpIQ3UZtB8yvUKC2cXn1a+x+5EVQLovA=="],
"fs-merger": ["fs-merger@3.2.1", "", { "dependencies": { "broccoli-node-api": "^1.7.0", "broccoli-node-info": "^2.1.0", "fs-extra": "^8.0.1", "fs-tree-diff": "^2.0.1", "walk-sync": "^2.2.0" } }, "sha512-AN6sX12liy0JE7C2evclwoo0aCG3PFulLjrTLsJpWh/2mM+DinhpSGqYLbHBBbIW1PLRNcFhJG8Axtz8mQW3ug=="],
"fs-mkdirp-stream": ["fs-mkdirp-stream@2.0.1", "", { "dependencies": { "graceful-fs": "^4.2.8", "streamx": "^2.12.0" } }, "sha512-UTOY+59K6IA94tec8Wjqm0FSh5OVudGNB0NL/P6fB3HiE3bYOY3VYBGijsnOHNkQSwC1FKkU77pmq7xp9CskLw=="],
"fs-tree-diff": ["fs-tree-diff@2.0.1", "", { "dependencies": { "@types/symlink-or-copy": "^1.2.0", "heimdalljs-logger": "^0.1.7", "object-assign": "^4.1.0", "path-posix": "^1.0.0", "symlink-or-copy": "^1.1.8" } }, "sha512-x+CfAZ/lJHQqwlD64pYM5QxWjzWhSjroaVsr8PW831zOApL55qPibed0c+xebaLWVr2BnHFoHdrwOv8pzt8R5A=="],
"fs.realpath": ["fs.realpath@1.0.0", "", {}, "sha512-OO0pH2lK6a0hZnAdau5ItzHPI6pUlvI7jMVnxUQRtw4owF2wk8lOSabtGDCTP4Ggrg2MbGnWO9X8K1t4+fGMDw=="],
"fsevents": ["fsevents@2.3.3", "", { "os": "darwin" }, "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw=="],
"function-bind": ["function-bind@1.1.2", "", {}, "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA=="],
@@ -420,25 +517,51 @@
"get-proto": ["get-proto@1.0.1", "", { "dependencies": { "dunder-proto": "^1.0.1", "es-object-atoms": "^1.0.0" } }, "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g=="],
"glob": ["glob@7.2.3", "", { "dependencies": { "fs.realpath": "^1.0.0", "inflight": "^1.0.4", "inherits": "2", "minimatch": "^3.1.1", "once": "^1.3.0", "path-is-absolute": "^1.0.0" } }, "sha512-nFR0zLpU2YCaRxwoCJvL6UvCH2JFyFVIvwTLsIf21AuHlMskA1hhTdk+LlYJtOlYt9v6dvszD2BGRqBL+iQK9Q=="],
"glob-parent": ["glob-parent@6.0.2", "", { "dependencies": { "is-glob": "^4.0.3" } }, "sha512-XxwI8EOhVQgWp6iDL+3b0r86f4d6AX6zSU55HfB4ydCEuXLXc5FcYeOu+nnGftS4TEju/11rt4KJPTMgbfmv4A=="],
"glob-stream": ["glob-stream@8.0.3", "", { "dependencies": { "@gulpjs/to-absolute-glob": "^4.0.0", "anymatch": "^3.1.3", "fastq": "^1.13.0", "glob-parent": "^6.0.2", "is-glob": "^4.0.3", "is-negated-glob": "^1.0.0", "normalize-path": "^3.0.0", "streamx": "^2.12.5" } }, "sha512-fqZVj22LtFJkHODT+M4N1RJQ3TjnnQhfE9GwZI8qXscYarnhpip70poMldRnP8ipQ/w0B621kOhfc53/J9bd/A=="],
"globals": ["globals@16.5.0", "", {}, "sha512-c/c15i26VrJ4IRt5Z89DnIzCGDn9EcebibhAOjw5ibqEHsE1wLUgkPn9RDmNcUKyU87GeaL633nyJ+pplFR2ZQ=="],
"gopd": ["gopd@1.2.0", "", {}, "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg=="],
"graceful-fs": ["graceful-fs@4.2.11", "", {}, "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ=="],
"gulp-sort": ["gulp-sort@2.0.0", "", { "dependencies": { "through2": "^2.0.1" } }, "sha512-MyTel3FXOdh1qhw1yKhpimQrAmur9q1X0ZigLmCOxouQD+BD3za9/89O+HfbgBQvvh4igEbp0/PUWO+VqGYG1g=="],
"has-flag": ["has-flag@4.0.0", "", {}, "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ=="],
"has-symbols": ["has-symbols@1.1.0", "", {}, "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ=="],
"hasown": ["hasown@2.0.2", "", { "dependencies": { "function-bind": "^1.1.2" } }, "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ=="],
"heimdalljs": ["heimdalljs@0.2.6", "", { "dependencies": { "rsvp": "~3.2.1" } }, "sha512-o9bd30+5vLBvBtzCPwwGqpry2+n0Hi6H1+qwt6y+0kwRHGGF8TFIhJPmnuM0xO97zaKrDZMwO/V56fAnn8m/tA=="],
"heimdalljs-logger": ["heimdalljs-logger@0.1.10", "", { "dependencies": { "debug": "^2.2.0", "heimdalljs": "^0.2.6" } }, "sha512-pO++cJbhIufVI/fmB/u2Yty3KJD0TqNPecehFae0/eps0hkZ3b4Zc/PezUMOpYuHFQbA7FxHZxa305EhmjLj4g=="],
"highlight.js": ["highlight.js@11.11.1", "", {}, "sha512-Xwwo44whKBVCYoliBQwaPvtd/2tYFkRQtXDWj1nackaV2JPXx3L0+Jvd8/qCJ2p+ML0/XVkJ2q+Mr+UVdpJK5w=="],
"hono": ["hono@4.12.8", "", {}, "sha512-VJCEvtrezO1IAR+kqEYnxUOoStaQPGrCmX3j4wDTNOcD1uRPFpGlwQUIW8niPuvHXaTUxeOUl5MMDGrl+tmO9A=="],
"html-parse-stringify": ["html-parse-stringify@3.0.1", "", { "dependencies": { "void-elements": "3.1.0" } }, "sha512-KknJ50kTInJ7qIScF3jeaFRpMpE8/lfiTdzf/twXyPBLAGrLRTmkz3AdTnKeh40X8k9L2fdYwEp/42WGXIRGcg=="],
"htmlparser2": ["htmlparser2@10.1.0", "", { "dependencies": { "domelementtype": "^2.3.0", "domhandler": "^5.0.3", "domutils": "^3.2.2", "entities": "^7.0.1" } }, "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ=="],
"http-errors": ["http-errors@2.0.1", "", { "dependencies": { "depd": "~2.0.0", "inherits": "~2.0.4", "setprototypeof": "~1.2.0", "statuses": "~2.0.2", "toidentifier": "~1.0.1" } }, "sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ=="],
"iconv-lite": ["iconv-lite@0.7.2", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-im9DjEDQ55s9fL4EYzOAv0yMqmMBSZp6G0VvFyTMPKWxiSBHUj9NW/qqLmXUwXrrM7AvqSlTCfvqRb0cM8yYqw=="],
"i18next": ["i18next@26.0.8", "", { "peerDependencies": { "typescript": "^5 || ^6" }, "optionalPeers": ["typescript"] }, "sha512-BRzLom0mhDhV9v0QhgUUHWQJuwFmnr1194xEcNLYD6ym8y8s542n4jXUvRLnhNTbh9PmpU6kGZamyuGHQMsGjw=="],
"i18next-browser-languagedetector": ["i18next-browser-languagedetector@8.2.1", "", { "dependencies": { "@babel/runtime": "^7.23.2" } }, "sha512-bZg8+4bdmaOiApD7N7BPT9W8MLZG+nPTOFlLiJiT8uzKXFjhxw4v2ierCXOwB5sFDMtuA5G4kgYZ0AznZxQ/cw=="],
"i18next-http-backend": ["i18next-http-backend@3.0.6", "", { "dependencies": { "cross-fetch": "4.1.0" } }, "sha512-mBOqy8993jtqAoj6XaI1XeC/8/9v6EPS+681ziegrPvTB0DoaCY7PpTS0SpY56qLMoS4OI1TZEM2Zf59zNh05w=="],
"i18next-parser": ["i18next-parser@9.4.0", "", { "dependencies": { "@babel/runtime": "^7.25.0", "broccoli-plugin": "^4.0.7", "cheerio": "^1.0.0", "colors": "^1.4.0", "commander": "^12.1.0", "eol": "^0.9.1", "esbuild": "^0.25.0", "fs-extra": "^11.2.0", "gulp-sort": "^2.0.0", "i18next": "^23.5.1 || ^24.2.0", "js-yaml": "^4.1.0", "lilconfig": "^3.1.3", "rsvp": "^4.8.5", "sort-keys": "^5.0.0", "typescript": "^5.0.4", "vinyl": "^3.0.0", "vinyl-fs": "^4.0.0" }, "bin": { "i18next": "bin/cli.js" } }, "sha512-SLQJGDj/baBIB9ALmJVXSOXWh3Zn9+wH7J2IuQ4rvx8yuQYpUWitmt8cHFjj6FExjgr8dHfd1SGeQgkowXDO1Q=="],
"iconv-lite": ["iconv-lite@0.6.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw=="],
"ieee754": ["ieee754@1.2.1", "", {}, "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA=="],
"ignore": ["ignore@5.3.2", "", {}, "sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g=="],
@@ -446,6 +569,8 @@
"imurmurhash": ["imurmurhash@0.1.4", "", {}, "sha512-JmXMZ6wuvDmLiHEml9ykzqO6lwFbof0GG4IkcGaENdCRDDmMVnny7s5HsIgHCbaq0w2MyPhDqkhTUgS2LU2PHA=="],
"inflight": ["inflight@1.0.6", "", { "dependencies": { "once": "^1.3.0", "wrappy": "1" } }, "sha512-k92I/b08q4wvFscXCLvqfsHCrjrF7yiXsQuIVvVE7N82W3+aqpzuUdBbfhWcy/FZR3/4IgflMgKLOsvPDrGCJA=="],
"inherits": ["inherits@2.0.4", "", {}, "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ=="],
"ip-address": ["ip-address@10.1.0", "", {}, "sha512-XXADHxXmvT9+CRxhXg56LJovE+bmWnEWB78LB83VZTprKTmaC5QfruXocxzTZ2Kl0DNwKuBdlIhjL8LeY8Sf8Q=="],
@@ -454,12 +579,24 @@
"is-extglob": ["is-extglob@2.1.1", "", {}, "sha512-SbKbANkN603Vi4jEZv49LeVJMn4yGwsbzZworEoyEiutsN3nJYdbO36zfhGJ6QEDpOZIFkDtnq5JRxmvl3jsoQ=="],
"is-fullwidth-code-point": ["is-fullwidth-code-point@3.0.0", "", {}, "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg=="],
"is-glob": ["is-glob@4.0.3", "", { "dependencies": { "is-extglob": "^2.1.1" } }, "sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg=="],
"is-negated-glob": ["is-negated-glob@1.0.0", "", {}, "sha512-czXVVn/QEmgvej1f50BZ648vUI+em0xqMq2Sn+QncCLN4zj1UAxlT+kw/6ggQTOaZPd1HqKQGEqbpQVtJucWug=="],
"is-plain-obj": ["is-plain-obj@4.1.0", "", {}, "sha512-+Pgi+vMuUNkJyExiMBt5IlFoMyKnr5zhJ4Uspz58WOhBF5QoIZkFyNHIbBAtHwzVAgk5RtndVNsDRN61/mmDqg=="],
"is-promise": ["is-promise@4.0.0", "", {}, "sha512-hvpoI6korhJMnej285dSg6nu1+e6uxs7zG3BYAm5byqDsgJNWwxzM6z6iZiAgQR4TJ30JmBTOwqZUw3WlyH3AQ=="],
"is-valid-glob": ["is-valid-glob@1.0.0", "", {}, "sha512-AhiROmoEFDSsjx8hW+5sGwgKVIORcXnrlAx/R0ZSeaPw70Vw0CqkGBBhHGL58Uox2eXnU1AnvXJl1XlyedO5bA=="],
"isarray": ["isarray@1.0.0", "", {}, "sha512-VLghIWNM6ELQzo7zwmcg0NmTVyWKYjvIeM83yjp0wRDTmUnrM678fQbcKBo6n2CJEF0szoG//ytg+TKla89ALQ=="],
"isexe": ["isexe@2.0.0", "", {}, "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw=="],
"jackspeak": ["jackspeak@3.4.3", "", { "dependencies": { "@isaacs/cliui": "^8.0.2" }, "optionalDependencies": { "@pkgjs/parseargs": "^0.11.0" } }, "sha512-OGlZQpz2yfahA/Rd1Y8Cd9SIEsqvXkLVoSw/cgwhnhFMDbsQFeZYoJJ7bIZBS9BcamUW96asq/npPWugM+RQBw=="],
"jose": ["jose@6.2.2", "", {}, "sha512-d7kPDd34KO/YnzaDOlikGpOurfF0ByC2sEV4cANCtdqLlTfBlw2p14O/5d/zv40gJPbIQxfES3nSx1/oYNyuZQ=="],
"js-tokens": ["js-tokens@4.0.0", "", {}, "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ=="],
@@ -478,10 +615,16 @@
"json5": ["json5@2.2.3", "", { "bin": { "json5": "lib/cli.js" } }, "sha512-XmOWe7eyHYH14cLdVPoyg+GOH3rYX++KpzrylJwSW98t3Nk+U8XOl8FWKOgwtzdb8lXGf6zYwDUzeHMWfxasyg=="],
"jsonfile": ["jsonfile@6.2.1", "", { "dependencies": { "universalify": "^2.0.0" }, "optionalDependencies": { "graceful-fs": "^4.1.6" } }, "sha512-zwOTdL3rFQ/lRdBnntKVOX6k5cKJwEc1HdilT71BWEu7J41gXIB2MRp+vxduPSwZJPWBxEzv4yH1wYLJGUHX4Q=="],
"keyv": ["keyv@4.5.4", "", { "dependencies": { "json-buffer": "3.0.1" } }, "sha512-oxVHkHR/EJf2CNXnWxRLW6mg7JyCCUcG0DtEGmL2ctUo1PNTin1PUil+r/+4r5MpVgC/fn1kjsx7mjSujKqIpw=="],
"lead": ["lead@4.0.0", "", {}, "sha512-DpMa59o5uGUWWjruMp71e6knmwKU3jRBBn1kjuLWN9EeIOxNeSAwvHf03WIl8g/ZMR2oSQC9ej3yeLBwdDc/pg=="],
"levn": ["levn@0.4.1", "", { "dependencies": { "prelude-ls": "^1.2.1", "type-check": "~0.4.0" } }, "sha512-+bT2uH4E5LGE7h/n3evcS/sQlJXCpIp6ym8OWJ5eV6+67Dsql/LaaT7qJBAt2rzfoa/5QBGBhxDix1dMt2kQKQ=="],
"lilconfig": ["lilconfig@3.1.3", "", {}, "sha512-/vlFKAoH5Cgt3Ie+JLhRbwOsCQePABiU3tJ1egGvyQ+33R/vcwM2Zl2QR/LzjsBeItPt3oSVXapn+m4nQDvpzw=="],
"locate-path": ["locate-path@6.0.0", "", { "dependencies": { "p-locate": "^5.0.0" } }, "sha512-iPZK6eYjbxRu3uB4/WZ3EsEIMJFMqAoopl3R+zuq0UjcAm/MO6KCweDgPfP3elTztoKP3KtnVHxTn2NHBSDVUw=="],
"lodash.merge": ["lodash.merge@4.6.2", "", {}, "sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ=="],
@@ -490,6 +633,8 @@
"marked": ["marked@15.0.12", "", { "bin": { "marked": "bin/marked.js" } }, "sha512-8dD6FusOQSrpv9Z1rdNMdlSgQOIP880DHqnohobOmYLElGEqAL/JvxvuxZO16r4HtjTlfPRDC1hbvxC9dPN2nA=="],
"matcher-collection": ["matcher-collection@2.0.1", "", { "dependencies": { "@types/minimatch": "^3.0.3", "minimatch": "^3.0.2" } }, "sha512-daE62nS2ZQsDg9raM0IlZzLmI2u+7ZapXBwdoeBUKAYERPDDIc0qNqA8E0Rp2D+gspKR7BgIFP52GeujaGXWeQ=="],
"math-intrinsics": ["math-intrinsics@1.1.0", "", {}, "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g=="],
"media-typer": ["media-typer@1.1.0", "", {}, "sha512-aisnrDP4GNe06UcKFnV5bfMNPBUw4jsLGaWwWfnH3v02GnBuXX2MCVn5RbrWo0j3pczUilYblq7fQ7Nw2t5XKw=="],
@@ -502,6 +647,10 @@
"minimatch": ["minimatch@3.1.5", "", { "dependencies": { "brace-expansion": "^1.1.7" } }, "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w=="],
"minipass": ["minipass@7.1.3", "", {}, "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A=="],
"mktemp": ["mktemp@2.0.3", "", {}, "sha512-Bq72L2oi/isYSy0guN9ihNhAMQOyZEwts+Bezm/1U+wh8bQ+fVQ2ZiUoJJjceOMiiKv/BUrA0NF98jFc81CB6w=="],
"ms": ["ms@2.1.3", "", {}, "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA=="],
"nanoid": ["nanoid@3.3.11", "", { "bin": { "nanoid": "bin/nanoid.cjs" } }, "sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w=="],
@@ -510,8 +659,16 @@
"negotiator": ["negotiator@1.0.0", "", {}, "sha512-8Ofs/AUQh8MaEcrlq5xOX0CQ9ypTF5dl78mjlMNfOK08fzpgTHQRQPBxcPlEtIw0yRpws+Zo/3r+5WRby7u3Gg=="],
"node-fetch": ["node-fetch@2.7.0", "", { "dependencies": { "whatwg-url": "^5.0.0" }, "peerDependencies": { "encoding": "^0.1.0" }, "optionalPeers": ["encoding"] }, "sha512-c4FRfUm/dbcWZ7U+1Wq0AwCyFL+3nt2bEw05wfxSz+DWpWsitgmSgYmy2dQdWyKC1694ELPqMs/YzUSNozLt8A=="],
"node-releases": ["node-releases@2.0.36", "", {}, "sha512-TdC8FSgHz8Mwtw9g5L4gR/Sh9XhSP/0DEkQxfEFXOpiul5IiHgHan2VhYYb6agDSfp4KuvltmGApc8HMgUrIkA=="],
"normalize-path": ["normalize-path@3.0.0", "", {}, "sha512-6eZs5Ls3WtCisHWp9S2GUy8dqkpGi4BVSz3GaqiE6ezub0512ESztXUwUB6C6IKbQkY2Pnb/mD4WYojCRwcwLA=="],
"now-and-later": ["now-and-later@3.0.0", "", { "dependencies": { "once": "^1.4.0" } }, "sha512-pGO4pzSdaxhWTGkfSfHx3hVzJVslFPwBp2Myq9MYN/ChfJZF87ochMAXnvz6/58RJSf5ik2q9tXprBBrk2cpcg=="],
"nth-check": ["nth-check@2.1.1", "", { "dependencies": { "boolbase": "^1.0.0" } }, "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w=="],
"object-assign": ["object-assign@4.1.1", "", {}, "sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg=="],
"object-inspect": ["object-inspect@1.13.4", "", {}, "sha512-W67iLl4J2EXEGTbfeHCffrjDfitvLANg0UlX3wFUUSTx92KXRFegMHUVgSqE+wvhAbi4WqjGg9czysTV2Epbew=="],
@@ -526,14 +683,28 @@
"p-locate": ["p-locate@5.0.0", "", { "dependencies": { "p-limit": "^3.0.2" } }, "sha512-LaNjtRWUBY++zB5nE/NwcaoMylSPk+S+ZHNB1TzdbMJMny6dynpAGt7X/tl/QYq3TIeE6nxHppbo2LGymrG5Pw=="],
"package-json-from-dist": ["package-json-from-dist@1.0.1", "", {}, "sha512-UEZIS3/by4OC8vL3P2dTXRETpebLI2NiI5vIrjaD/5UtrkFX/tNbwjTSRAGC/+7CAo2pIcBaRgWmcBBHcsaCIw=="],
"parent-module": ["parent-module@1.0.1", "", { "dependencies": { "callsites": "^3.0.0" } }, "sha512-GQ2EWRpQV8/o+Aw8YqtfZZPfNRWZYkbidE9k5rpl/hC3vtHHBfGm2Ifi6qWV+coDGkrUKZAxE3Lot5kcsRlh+g=="],
"parse5": ["parse5@7.3.0", "", { "dependencies": { "entities": "^6.0.0" } }, "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw=="],
"parse5-htmlparser2-tree-adapter": ["parse5-htmlparser2-tree-adapter@7.1.0", "", { "dependencies": { "domhandler": "^5.0.3", "parse5": "^7.0.0" } }, "sha512-ruw5xyKs6lrpo9x9rCZqZZnIUntICjQAd0Wsmp396Ul9lN/h+ifgVV1x1gZHi8euej6wTfpqX8j+BFQxF0NS/g=="],
"parse5-parser-stream": ["parse5-parser-stream@7.1.2", "", { "dependencies": { "parse5": "^7.0.0" } }, "sha512-JyeQc9iwFLn5TbvvqACIF/VXG6abODeB3Fwmv/TGdLk2LfbWkaySGY72at4+Ty7EkPZj854u4CrICqNk2qIbow=="],
"parseurl": ["parseurl@1.3.3", "", {}, "sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ=="],
"path-exists": ["path-exists@4.0.0", "", {}, "sha512-ak9Qy5Q7jYb2Wwcey5Fpvg2KoAc/ZIhLSLOSBmRmygPsGwkVVt0fZa0qrtMz+m6tJTAHfZQ8FnmB4MG4LWy7/w=="],
"path-is-absolute": ["path-is-absolute@1.0.1", "", {}, "sha512-AVbw3UJ2e9bq64vSaS9Am0fje1Pa8pbGqTTsmXfaIiMpnr5DlDhfJOuLj9Sf95ZPVDAUerDfEk88MPmPe7UCQg=="],
"path-key": ["path-key@3.1.1", "", {}, "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q=="],
"path-posix": ["path-posix@1.0.0", "", {}, "sha512-1gJ0WpNIiYcQydgg3Ed8KzvIqTsDpNwq+cjBCssvBtuTWjEqY1AW+i+OepiEMqDCzyro9B2sLAe4RBPajMYFiA=="],
"path-scurry": ["path-scurry@1.11.1", "", { "dependencies": { "lru-cache": "^10.2.0", "minipass": "^5.0.0 || ^6.0.2 || ^7.0.0" } }, "sha512-Xa4Nw17FS9ApQFJ9umLiJS4orGjm7ZzwUrwamcGQuHSzDyth9boKDaycYdDcZDuqYATXw4HFXgaqWTctW/v1HA=="],
"path-to-regexp": ["path-to-regexp@8.3.0", "", {}, "sha512-7jdwVIRtsP8MYpdXSwOS0YdD0Du+qOoF/AEPIt88PcCFrZCzx41oxku1jD88hZBwbNUIEfpqvuhjFaMAqMTWnA=="],
"picocolors": ["picocolors@1.1.1", "", {}, "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA=="],
@@ -550,12 +721,18 @@
"prelude-ls": ["prelude-ls@1.2.1", "", {}, "sha512-vkcDPrRZo1QZLbn5RLGPpg/WmIQ65qoWWhcGKf/b5eplkkarX0m9z8ppCat4mlOqUsWpyNuYgO3VRyrYHSzX5g=="],
"process-nextick-args": ["process-nextick-args@2.0.1", "", {}, "sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag=="],
"promise-map-series": ["promise-map-series@0.3.0", "", {}, "sha512-3npG2NGhTc8BWBolLLf8l/92OxMGaRLbqvIh9wjCHhDXNvk4zsxaTaCpiCunW09qWPrN2zeNSNwRLVBrQQtutA=="],
"proxy-addr": ["proxy-addr@2.0.7", "", { "dependencies": { "forwarded": "0.2.0", "ipaddr.js": "1.9.1" } }, "sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg=="],
"punycode": ["punycode@2.3.1", "", {}, "sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg=="],
"qs": ["qs@6.15.0", "", { "dependencies": { "side-channel": "^1.1.0" } }, "sha512-mAZTtNCeetKMH+pSjrb76NAM8V9a05I9aBZOHztWy/UqcJdQYNsf59vrRKWnojAT9Y+GbIvoTBC++CPHqpDBhQ=="],
"quick-temp": ["quick-temp@0.1.9", "", { "dependencies": { "mktemp": "^2.0.1", "rimraf": "^5.0.10", "underscore.string": "~3.3.6" } }, "sha512-yI0h7tIhKVObn03kD+Ln9JFi4OljD28lfaOsTdfpTR0xzrhGOod+q66CjGafUqYX2juUfT9oHIGrTBBo22mkRA=="],
"range-parser": ["range-parser@1.2.1", "", {}, "sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg=="],
"raw-body": ["raw-body@3.0.2", "", { "dependencies": { "bytes": "~3.1.2", "http-errors": "~2.0.1", "iconv-lite": "~0.7.0", "unpipe": "~1.0.0" } }, "sha512-K5zQjDllxWkf7Z5xJdV0/B0WTNqx6vxG70zJE4N0kBs4LovmEYWJzQGxC9bS9RAKu3bgM40lrd5zoLJ12MQ5BA=="],
@@ -564,20 +741,38 @@
"react-dom": ["react-dom@19.2.4", "", { "dependencies": { "scheduler": "^0.27.0" }, "peerDependencies": { "react": "^19.2.4" } }, "sha512-AXJdLo8kgMbimY95O2aKQqsz2iWi9jMgKJhRBAxECE4IFxfcazB2LmzloIoibJI3C12IlY20+KFaLv+71bUJeQ=="],
"react-i18next": ["react-i18next@17.0.6", "", { "dependencies": { "@babel/runtime": "^7.29.2", "html-parse-stringify": "^3.0.1", "use-sync-external-store": "^1.6.0" }, "peerDependencies": { "i18next": ">= 26.0.1", "react": ">= 16.8.0", "typescript": "^5 || ^6" }, "optionalPeers": ["typescript"] }, "sha512-WzJ6SMKF+GTD7JZZqxSR1AKKmXjaSu39sClUrNlwxS4Tl7a99O+ltFy6yhPMO+wgZuxpQjJ2PZkfrQKmAqrLhw=="],
"react-refresh": ["react-refresh@0.17.0", "", {}, "sha512-z6F7K9bV85EfseRCp2bzrpyQ0Gkw1uLoCel9XBVWPg/TjRj94SkJzUTGfOa4bs7iJvBWtQG0Wq7wnI0syw3EBQ=="],
"react-router": ["react-router@7.13.1", "", { "dependencies": { "cookie": "^1.0.1", "set-cookie-parser": "^2.6.0" }, "peerDependencies": { "react": ">=18", "react-dom": ">=18" }, "optionalPeers": ["react-dom"] }, "sha512-td+xP4X2/6BJvZoX6xw++A2DdEi++YypA69bJUV5oVvqf6/9/9nNlD70YO1e9d3MyamJEBQFEzk6mbfDYbqrSA=="],
"react-router-dom": ["react-router-dom@7.13.1", "", { "dependencies": { "react-router": "7.13.1" }, "peerDependencies": { "react": ">=18", "react-dom": ">=18" } }, "sha512-UJnV3Rxc5TgUPJt2KJpo1Jpy0OKQr0AjgbZzBFjaPJcFOb2Y8jA5H3LT8HUJAiRLlWrEXWHbF1Z4SCZaQjWDHw=="],
"readable-stream": ["readable-stream@2.3.8", "", { "dependencies": { "core-util-is": "~1.0.0", "inherits": "~2.0.3", "isarray": "~1.0.0", "process-nextick-args": "~2.0.0", "safe-buffer": "~5.1.1", "string_decoder": "~1.1.1", "util-deprecate": "~1.0.1" } }, "sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA=="],
"remove-trailing-separator": ["remove-trailing-separator@1.1.0", "", {}, "sha512-/hS+Y0u3aOfIETiaiirUFwDBDzmXPvO+jAfKTitUngIPzdKc6Z0LoFjM/CK5PL4C+eKwHohlHAb6H0VFfmmUsw=="],
"replace-ext": ["replace-ext@2.0.0", "", {}, "sha512-UszKE5KVK6JvyD92nzMn9cDapSk6w/CaFZ96CnmDMUqH9oowfxF/ZjRITD25H4DnOQClLA4/j7jLGXXLVKxAug=="],
"require-from-string": ["require-from-string@2.0.2", "", {}, "sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw=="],
"resolve-from": ["resolve-from@4.0.0", "", {}, "sha512-pb/MYmXstAkysRFx8piNI1tGFNQIFA3vkE3Gq4EuA1dF6gHp/+vgZqsCGJapvy8N3Q+4o7FwvquPJcnZ7RYy4g=="],
"resolve-options": ["resolve-options@2.0.0", "", { "dependencies": { "value-or-function": "^4.0.0" } }, "sha512-/FopbmmFOQCfsCx77BRFdKOniglTiHumLgwvd6IDPihy1GKkadZbgQJBcTb2lMzSR1pndzd96b1nZrreZ7+9/A=="],
"reusify": ["reusify@1.1.0", "", {}, "sha512-g6QUff04oZpHs0eG5p83rFLhHeV00ug/Yf9nZM6fLeUrPguBTkTQOdpAWWspMh55TZfVQDPaN3NQJfbVRAxdIw=="],
"rimraf": ["rimraf@3.0.2", "", { "dependencies": { "glob": "^7.1.3" }, "bin": { "rimraf": "bin.js" } }, "sha512-JZkJMZkAGFFPP2YqXZXPbMlMBgsxzE8ILs4lMIX/2o0L9UBw9O/Y3o6wFw/i9YLapcUJWwqbi3kdxIPdC62TIA=="],
"rollup": ["rollup@4.59.0", "", { "dependencies": { "@types/estree": "1.0.8" }, "optionalDependencies": { "@rollup/rollup-android-arm-eabi": "4.59.0", "@rollup/rollup-android-arm64": "4.59.0", "@rollup/rollup-darwin-arm64": "4.59.0", "@rollup/rollup-darwin-x64": "4.59.0", "@rollup/rollup-freebsd-arm64": "4.59.0", "@rollup/rollup-freebsd-x64": "4.59.0", "@rollup/rollup-linux-arm-gnueabihf": "4.59.0", "@rollup/rollup-linux-arm-musleabihf": "4.59.0", "@rollup/rollup-linux-arm64-gnu": "4.59.0", "@rollup/rollup-linux-arm64-musl": "4.59.0", "@rollup/rollup-linux-loong64-gnu": "4.59.0", "@rollup/rollup-linux-loong64-musl": "4.59.0", "@rollup/rollup-linux-ppc64-gnu": "4.59.0", "@rollup/rollup-linux-ppc64-musl": "4.59.0", "@rollup/rollup-linux-riscv64-gnu": "4.59.0", "@rollup/rollup-linux-riscv64-musl": "4.59.0", "@rollup/rollup-linux-s390x-gnu": "4.59.0", "@rollup/rollup-linux-x64-gnu": "4.59.0", "@rollup/rollup-linux-x64-musl": "4.59.0", "@rollup/rollup-openbsd-x64": "4.59.0", "@rollup/rollup-openharmony-arm64": "4.59.0", "@rollup/rollup-win32-arm64-msvc": "4.59.0", "@rollup/rollup-win32-ia32-msvc": "4.59.0", "@rollup/rollup-win32-x64-gnu": "4.59.0", "@rollup/rollup-win32-x64-msvc": "4.59.0", "fsevents": "~2.3.2" }, "bin": { "rollup": "dist/bin/rollup" } }, "sha512-2oMpl67a3zCH9H79LeMcbDhXW/UmWG/y2zuqnF2jQq5uq9TbM9TVyXvA4+t+ne2IIkBdrLpAaRQAvo7YI/Yyeg=="],
"router": ["router@2.2.0", "", { "dependencies": { "debug": "^4.4.0", "depd": "^2.0.0", "is-promise": "^4.0.0", "parseurl": "^1.3.3", "path-to-regexp": "^8.0.0" } }, "sha512-nLTrUKm2UyiL7rlhapu/Zl45FwNgkZGaCpZbIHajDYgwlJCOzLSk+cIPAnsEqV955GjILJnKbdQC1nVPz+gAYQ=="],
"rsvp": ["rsvp@4.8.5", "", {}, "sha512-nfMOlASu9OnRJo1mbEk2cz0D56a1MBNrJ7orjRZQG10XDyuvwksKbuXNp6qa+kbn839HwjwhBzhFmdsaEAfauA=="],
"safe-buffer": ["safe-buffer@5.1.2", "", {}, "sha512-Gd2UZBJDkXlY7GbJxfsE8/nvKkUEU1G38c1siN6QP6a9PT9MmHB8GnpscSmMJSoF8LOIrt8ud/wPtojys4G6+g=="],
"safer-buffer": ["safer-buffer@2.1.2", "", {}, "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg=="],
"scheduler": ["scheduler@0.27.0", "", {}, "sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q=="],
@@ -604,42 +799,114 @@
"side-channel-weakmap": ["side-channel-weakmap@1.0.2", "", { "dependencies": { "call-bound": "^1.0.2", "es-errors": "^1.3.0", "get-intrinsic": "^1.2.5", "object-inspect": "^1.13.3", "side-channel-map": "^1.0.1" } }, "sha512-WPS/HvHQTYnHisLo9McqBHOJk2FkHO/tlpvldyrnem4aeQp4hai3gythswg6p01oSoTl58rcpiFAjF2br2Ak2A=="],
"signal-exit": ["signal-exit@4.1.0", "", {}, "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw=="],
"sort-keys": ["sort-keys@5.1.0", "", { "dependencies": { "is-plain-obj": "^4.0.0" } }, "sha512-aSbHV0DaBcr7u0PVHXzM6NbZNAtrr9sF6+Qfs9UUVG7Ll3jQ6hHi8F/xqIIcn2rvIVbr0v/2zyjSdwSV47AgLQ=="],
"source-map-js": ["source-map-js@1.2.1", "", {}, "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA=="],
"sprintf-js": ["sprintf-js@1.1.3", "", {}, "sha512-Oo+0REFV59/rz3gfJNKQiBlwfHaSESl1pcGyABQsnnIfWOFt6JNj5gCog2U6MLZ//IGYD+nA8nI+mTShREReaA=="],
"statuses": ["statuses@2.0.2", "", {}, "sha512-DvEy55V3DB7uknRo+4iOGT5fP1slR8wQohVdknigZPMpMstaKJQWhwiYBACJE3Ul2pTnATihhBYnRhZQHGBiRw=="],
"stream-composer": ["stream-composer@1.0.2", "", { "dependencies": { "streamx": "^2.13.2" } }, "sha512-bnBselmwfX5K10AH6L4c8+S5lgZMWI7ZYrz2rvYjCPB2DIMC4Ig8OpxGpNJSxRZ58oti7y1IcNvjBAz9vW5m4w=="],
"streamx": ["streamx@2.25.0", "", { "dependencies": { "events-universal": "^1.0.0", "fast-fifo": "^1.3.2", "text-decoder": "^1.1.0" } }, "sha512-0nQuG6jf1w+wddNEEXCF4nTg3LtufWINB5eFEN+5TNZW7KWJp6x87+JFL43vaAUPyCfH1wID+mNVyW6OHtFamg=="],
"string-width": ["string-width@5.1.2", "", { "dependencies": { "eastasianwidth": "^0.2.0", "emoji-regex": "^9.2.2", "strip-ansi": "^7.0.1" } }, "sha512-HnLOCR3vjcY8beoNLtcjZ5/nxn2afmME6lhrDrebokqMap+XbeW8n9TXpPDOqdGK5qcI3oT0GKTW6wC7EMiVqA=="],
"string-width-cjs": ["string-width@4.2.3", "", { "dependencies": { "emoji-regex": "^8.0.0", "is-fullwidth-code-point": "^3.0.0", "strip-ansi": "^6.0.1" } }, "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g=="],
"string_decoder": ["string_decoder@1.1.1", "", { "dependencies": { "safe-buffer": "~5.1.0" } }, "sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg=="],
"strip-ansi": ["strip-ansi@7.2.0", "", { "dependencies": { "ansi-regex": "^6.2.2" } }, "sha512-yDPMNjp4WyfYBkHnjIRLfca1i6KMyGCtsVgoKe/z1+6vukgaENdgGBZt+ZmKPc4gavvEZ5OgHfHdrazhgNyG7w=="],
"strip-ansi-cjs": ["strip-ansi@6.0.1", "", { "dependencies": { "ansi-regex": "^5.0.1" } }, "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A=="],
"strip-json-comments": ["strip-json-comments@3.1.1", "", {}, "sha512-6fPc+R4ihwqP6N/aIv2f1gMH8lOVtWQHoqC4yK6oSDVVocumAsfCqjkXnqiYMhmMwS/mEHLp7Vehlt3ql6lEig=="],
"style-mod": ["style-mod@4.1.3", "", {}, "sha512-i/n8VsZydrugj3Iuzll8+x/00GH2vnYsk1eomD8QiRrSAeW6ItbCQDtfXCeJHd0iwiNagqjQkvpvREEPtW3IoQ=="],
"supports-color": ["supports-color@7.2.0", "", { "dependencies": { "has-flag": "^4.0.0" } }, "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw=="],
"symlink-or-copy": ["symlink-or-copy@1.3.1", "", {}, "sha512-0K91MEXFpBUaywiwSSkmKjnGcasG/rVBXFLJz5DrgGabpYD6N+3yZrfD6uUIfpuTu65DZLHi7N8CizHc07BPZA=="],
"teex": ["teex@1.0.1", "", { "dependencies": { "streamx": "^2.12.5" } }, "sha512-eYE6iEI62Ni1H8oIa7KlDU6uQBtqr4Eajni3wX7rpfXD8ysFx8z0+dri+KWEPWpBsxXfxu58x/0jvTVT1ekOSg=="],
"text-decoder": ["text-decoder@1.2.7", "", { "dependencies": { "b4a": "^1.6.4" } }, "sha512-vlLytXkeP4xvEq2otHeJfSQIRyWxo/oZGEbXrtEEF9Hnmrdly59sUbzZ/QgyWuLYHctCHxFF4tRQZNQ9k60ExQ=="],
"through2": ["through2@2.0.5", "", { "dependencies": { "readable-stream": "~2.3.6", "xtend": "~4.0.1" } }, "sha512-/mrRod8xqpA+IHSLyGCQ2s8SPHiCDEeQJSep1jqLYeEUClOFG2Qsh+4FU6G9VeqpZnGW/Su8LQGc4YKni5rYSQ=="],
"tinyglobby": ["tinyglobby@0.2.15", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.3" } }, "sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ=="],
"to-through": ["to-through@3.0.0", "", { "dependencies": { "streamx": "^2.12.5" } }, "sha512-y8MN937s/HVhEoBU1SxfHC+wxCHkV1a9gW8eAdTadYh/bGyesZIVcbjI+mSpFbSVwQici/XjBjuUyri1dnXwBw=="],
"toidentifier": ["toidentifier@1.0.1", "", {}, "sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA=="],
"tr46": ["tr46@0.0.3", "", {}, "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw=="],
"type-check": ["type-check@0.4.0", "", { "dependencies": { "prelude-ls": "^1.2.1" } }, "sha512-XleUoc9uwGXqjWwXaUTZAmzMcFZ5858QA2vvx1Ur5xIcixXIP+8LnFDgRplU30us6teqdlskFfu+ae4K79Ooew=="],
"type-is": ["type-is@2.0.1", "", { "dependencies": { "content-type": "^1.0.5", "media-typer": "^1.1.0", "mime-types": "^3.0.0" } }, "sha512-OZs6gsjF4vMp32qrCbiVSkrFmXtG/AZhY3t0iAMrMBiAZyV9oALtXO8hsrHbMXF9x6L3grlFuwW2oAz7cav+Gw=="],
"typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="],
"underscore.string": ["underscore.string@3.3.6", "", { "dependencies": { "sprintf-js": "^1.1.1", "util-deprecate": "^1.0.2" } }, "sha512-VoC83HWXmCrF6rgkyxS9GHv8W9Q5nhMKho+OadDJGzL2oDYbYEppBaCMH6pFlwLeqj2QS+hhkw2kpXkSdD1JxQ=="],
"undici": ["undici@7.25.0", "", {}, "sha512-xXnp4kTyor2Zq+J1FfPI6Eq3ew5h6Vl0F/8d9XU5zZQf1tX9s2Su1/3PiMmUANFULpmksxkClamIZcaUqryHsQ=="],
"universalify": ["universalify@2.0.1", "", {}, "sha512-gptHNQghINnc/vTGIk0SOFGFNXw7JVrlRUtConJRlvaw6DuX0wO5Jeko9sWrMBhh+PsYAZ7oXAiOnf/UKogyiw=="],
"unpipe": ["unpipe@1.0.0", "", {}, "sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ=="],
"update-browserslist-db": ["update-browserslist-db@1.2.3", "", { "dependencies": { "escalade": "^3.2.0", "picocolors": "^1.1.1" }, "peerDependencies": { "browserslist": ">= 4.21.0" }, "bin": { "update-browserslist-db": "cli.js" } }, "sha512-Js0m9cx+qOgDxo0eMiFGEueWztz+d4+M3rGlmKPT+T4IS/jP4ylw3Nwpu6cpTTP8R1MAC1kF4VbdLt3ARf209w=="],
"uri-js": ["uri-js@4.4.1", "", { "dependencies": { "punycode": "^2.1.0" } }, "sha512-7rKUyy33Q1yc98pQ1DAmLtwX109F7TIfWlW1Ydo8Wl1ii1SeHieeh0HHfPeL2fMXK6z0s8ecKs9frCuLJvndBg=="],
"use-sync-external-store": ["use-sync-external-store@1.6.0", "", { "peerDependencies": { "react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-Pp6GSwGP/NrPIrxVFAIkOQeyw8lFenOHijQWkUTrDvrF4ALqylP2C/KCkeS9dpUM3KvYRQhna5vt7IL95+ZQ9w=="],
"util-deprecate": ["util-deprecate@1.0.2", "", {}, "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw=="],
"value-or-function": ["value-or-function@4.0.0", "", {}, "sha512-aeVK81SIuT6aMJfNo9Vte8Dw0/FZINGBV8BfCraGtqVxIeLAEhJyoWs8SmvRVmXfGss2PmmOwZCuBPbZR+IYWg=="],
"vary": ["vary@1.1.2", "", {}, "sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg=="],
"vite": ["vite@6.4.1", "", { "dependencies": { "esbuild": "^0.25.0", "fdir": "^6.4.4", "picomatch": "^4.0.2", "postcss": "^8.5.3", "rollup": "^4.34.9", "tinyglobby": "^0.2.13" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "peerDependencies": { "@types/node": "^18.0.0 || ^20.0.0 || >=22.0.0", "jiti": ">=1.21.0", "less": "*", "lightningcss": "^1.21.0", "sass": "*", "sass-embedded": "*", "stylus": "*", "sugarss": "*", "terser": "^5.16.0", "tsx": "^4.8.1", "yaml": "^2.4.2" }, "optionalPeers": ["@types/node", "jiti", "less", "lightningcss", "sass", "sass-embedded", "stylus", "sugarss", "terser", "tsx", "yaml"], "bin": { "vite": "bin/vite.js" } }, "sha512-+Oxm7q9hDoLMyJOYfUYBuHQo+dkAloi33apOPP56pzj+vsdJDzr+j1NISE5pyaAuKL4A3UD34qd0lx5+kfKp2g=="],
"vinyl": ["vinyl@3.0.1", "", { "dependencies": { "clone": "^2.1.2", "remove-trailing-separator": "^1.1.0", "replace-ext": "^2.0.0", "teex": "^1.0.1" } }, "sha512-0QwqXteBNXgnLCdWdvPQBX6FXRHtIH3VhJPTd5Lwn28tJXc34YqSCWUmkOvtJHBmB3gGoPtrOKk3Ts8/kEZ9aA=="],
"vinyl-contents": ["vinyl-contents@2.0.0", "", { "dependencies": { "bl": "^5.0.0", "vinyl": "^3.0.0" } }, "sha512-cHq6NnGyi2pZ7xwdHSW1v4Jfnho4TEGtxZHw01cmnc8+i7jgR6bRnED/LbrKan/Q7CvVLbnvA5OepnhbpjBZ5Q=="],
"vinyl-fs": ["vinyl-fs@4.0.2", "", { "dependencies": { "fs-mkdirp-stream": "^2.0.1", "glob-stream": "^8.0.3", "graceful-fs": "^4.2.11", "iconv-lite": "^0.6.3", "is-valid-glob": "^1.0.0", "lead": "^4.0.0", "normalize-path": "3.0.0", "resolve-options": "^2.0.0", "stream-composer": "^1.0.2", "streamx": "^2.14.0", "to-through": "^3.0.0", "value-or-function": "^4.0.0", "vinyl": "^3.0.1", "vinyl-sourcemap": "^2.0.0" } }, "sha512-XRFwBLLTl8lRAOYiBqxY279wY46tVxLaRhSwo3GzKEuLz1giffsOquWWboD/haGf5lx+JyTigCFfe7DWHoARIA=="],
"vinyl-sourcemap": ["vinyl-sourcemap@2.0.0", "", { "dependencies": { "convert-source-map": "^2.0.0", "graceful-fs": "^4.2.10", "now-and-later": "^3.0.0", "streamx": "^2.12.5", "vinyl": "^3.0.0", "vinyl-contents": "^2.0.0" } }, "sha512-BAEvWxbBUXvlNoFQVFVHpybBbjW1r03WhohJzJDSfgrrK5xVYIDTan6xN14DlyImShgDRv2gl9qhM6irVMsV0Q=="],
"vite": ["vite@6.4.2", "", { "dependencies": { "esbuild": "^0.25.0", "fdir": "^6.4.4", "picomatch": "^4.0.2", "postcss": "^8.5.3", "rollup": "^4.34.9", "tinyglobby": "^0.2.13" }, "optionalDependencies": { "fsevents": "~2.3.3" }, "peerDependencies": { "@types/node": "^18.0.0 || ^20.0.0 || >=22.0.0", "jiti": ">=1.21.0", "less": "*", "lightningcss": "^1.21.0", "sass": "*", "sass-embedded": "*", "stylus": "*", "sugarss": "*", "terser": "^5.16.0", "tsx": "^4.8.1", "yaml": "^2.4.2" }, "optionalPeers": ["@types/node", "jiti", "less", "lightningcss", "sass", "sass-embedded", "stylus", "sugarss", "terser", "tsx", "yaml"], "bin": { "vite": "bin/vite.js" } }, "sha512-2N/55r4JDJ4gdrCvGgINMy+HH3iRpNIz8K6SFwVsA+JbQScLiC+clmAxBgwiSPgcG9U15QmvqCGWzMbqda5zGQ=="],
"void-elements": ["void-elements@3.1.0", "", {}, "sha512-Dhxzh5HZuiHQhbvTW9AMetFfBHDMYpo23Uo9btPXgdYP+3T5S+p+jgNy7spra+veYhBP2dCSgxR/i2Y02h5/6w=="],
"w3c-keyname": ["w3c-keyname@2.2.8", "", {}, "sha512-dpojBhNsCNN7T82Tm7k26A6G9ML3NkhDsnw9n/eoxSRlVBB4CEtIQ/KTCLI2Fwf3ataSXRhYFkQi3SlnFwPvPQ=="],
"walk-sync": ["walk-sync@2.2.0", "", { "dependencies": { "@types/minimatch": "^3.0.3", "ensure-posix-path": "^1.1.0", "matcher-collection": "^2.0.0", "minimatch": "^3.0.4" } }, "sha512-IC8sL7aB4/ZgFcGI2T1LczZeFWZ06b3zoHH7jBPyHxOtIIz1jppWHjjEXkOFvFojBVAK9pV7g47xOZ4LW3QLfg=="],
"webidl-conversions": ["webidl-conversions@3.0.1", "", {}, "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ=="],
"whatwg-encoding": ["whatwg-encoding@3.1.1", "", { "dependencies": { "iconv-lite": "0.6.3" } }, "sha512-6qN4hJdMwfYBtE3YBTTHhoeuUrDBPZmbQaxWAqSALV/MeEnR5z1xd8UKud2RAkFoPkmB+hli1TZSnyi84xz1vQ=="],
"whatwg-mimetype": ["whatwg-mimetype@4.0.0", "", {}, "sha512-QaKxh0eNIi2mE9p2vEdzfagOKHCcj1pJ56EEHGQOVxp8r9/iszLUUV7v89x9O1p/T+NlTM5W7jW6+cz4Fq1YVg=="],
"whatwg-url": ["whatwg-url@5.0.0", "", { "dependencies": { "tr46": "~0.0.3", "webidl-conversions": "^3.0.0" } }, "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw=="],
"which": ["which@2.0.2", "", { "dependencies": { "isexe": "^2.0.0" }, "bin": { "node-which": "./bin/node-which" } }, "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA=="],
"word-wrap": ["word-wrap@1.2.5", "", {}, "sha512-BN22B5eaMMI9UMtjrGd5g5eCYPpCPDUy0FJXbYsaT5zYxjFOckS53SQDE3pWkVoWpHXVb3BrYcEN4Twa55B5cA=="],
"wrap-ansi": ["wrap-ansi@8.1.0", "", { "dependencies": { "ansi-styles": "^6.1.0", "string-width": "^5.0.1", "strip-ansi": "^7.0.1" } }, "sha512-si7QWI6zUMq56bESFvagtmzMdGOtoxfR+Sez11Mobfc7tm+VkUckk9bW2UeffTGVUbOksxmSw0AA2gs8g71NCQ=="],
"wrap-ansi-cjs": ["wrap-ansi@7.0.0", "", { "dependencies": { "ansi-styles": "^4.0.0", "string-width": "^4.1.0", "strip-ansi": "^6.0.0" } }, "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q=="],
"wrappy": ["wrappy@1.0.2", "", {}, "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ=="],
"xtend": ["xtend@4.0.2", "", {}, "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ=="],
"yallist": ["yallist@3.1.1", "", {}, "sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g=="],
"yaml": ["yaml@2.8.3", "", { "bin": { "yaml": "bin.mjs" } }, "sha512-AvbaCLOO2Otw/lW5bmh9d/WEdcDFdQp2Z2ZUH3pX9U2ihyUY0nvLv7J6TrWowklRGPYbB/IuIMfYgxaCPg5Bpg=="],
@@ -658,12 +925,72 @@
"ajv-formats/ajv": ["ajv@8.18.0", "", { "dependencies": { "fast-deep-equal": "^3.1.3", "fast-uri": "^3.0.1", "json-schema-traverse": "^1.0.0", "require-from-string": "^2.0.2" } }, "sha512-PlXPeEWMXMZ7sPYOHqmDyCJzcfNrUr3fGNKtezX14ykXOEIvyK81d+qydx89KY5O71FKMPaQ2vBfBFI5NHR63A=="],
"anymatch/picomatch": ["picomatch@2.3.2", "", {}, "sha512-V7+vQEJ06Z+c5tSye8S+nHUfI51xoXIXjHQ99cQtKUkQqqO1kO/KCJUfZXuB47h/YBlDhah2H3hdUGXn8ie0oA=="],
"bl/readable-stream": ["readable-stream@3.6.2", "", { "dependencies": { "inherits": "^2.0.3", "string_decoder": "^1.1.1", "util-deprecate": "^1.0.1" } }, "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA=="],
"body-parser/iconv-lite": ["iconv-lite@0.7.2", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-im9DjEDQ55s9fL4EYzOAv0yMqmMBSZp6G0VvFyTMPKWxiSBHUj9NW/qqLmXUwXrrM7AvqSlTCfvqRb0cM8yYqw=="],
"broccoli-output-wrapper/fs-extra": ["fs-extra@8.1.0", "", { "dependencies": { "graceful-fs": "^4.2.0", "jsonfile": "^4.0.0", "universalify": "^0.1.0" } }, "sha512-yhlQgA6mnOJUKOsRUFsgJdQCvkKhcz8tlZG5HBQfReYZy46OwLcY+Zia0mtdHsOo9y/hP+CxMN0TU9QxoOtG4g=="],
"fs-merger/fs-extra": ["fs-extra@8.1.0", "", { "dependencies": { "graceful-fs": "^4.2.0", "jsonfile": "^4.0.0", "universalify": "^0.1.0" } }, "sha512-yhlQgA6mnOJUKOsRUFsgJdQCvkKhcz8tlZG5HBQfReYZy46OwLcY+Zia0mtdHsOo9y/hP+CxMN0TU9QxoOtG4g=="],
"heimdalljs/rsvp": ["rsvp@3.2.1", "", {}, "sha512-Rf4YVNYpKjZ6ASAmibcwTNciQ5Co5Ztq6iZPEykHpkoflnD/K5ryE/rHehFsTm4NJj8nKDhbi3eKBWGogmNnkg=="],
"heimdalljs-logger/debug": ["debug@2.6.9", "", { "dependencies": { "ms": "2.0.0" } }, "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA=="],
"htmlparser2/entities": ["entities@7.0.1", "", {}, "sha512-TWrgLOFUQTH994YUyl1yT4uyavY5nNB5muff+RtWaqNVCAK408b5ZnnbNAUEWLTCpum9w6arT70i1XdQ4UeOPA=="],
"i18next-parser/i18next": ["i18next@24.2.3", "", { "dependencies": { "@babel/runtime": "^7.26.10" }, "peerDependencies": { "typescript": "^5" }, "optionalPeers": ["typescript"] }, "sha512-lfbf80OzkocvX7nmZtu7nSTNbrTYR52sLWxPtlXX1zAhVw8WEnFk4puUkCR4B1dNQwbSpEHHHemcZu//7EcB7A=="],
"parse5/entities": ["entities@6.0.1", "", {}, "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g=="],
"path-scurry/lru-cache": ["lru-cache@10.4.3", "", {}, "sha512-JNAzZcXrCt42VGLuYz0zfAzDfAvJWW6AfYlDBQyDV5DClI2m5sAmK+OIO7s59XfsRsWHp02jAJrRadPRGTt6SQ=="],
"playwright/fsevents": ["fsevents@2.3.2", "", { "os": "darwin" }, "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA=="],
"quick-temp/rimraf": ["rimraf@5.0.10", "", { "dependencies": { "glob": "^10.3.7" }, "bin": { "rimraf": "dist/esm/bin.mjs" } }, "sha512-l0OE8wL34P4nJH/H2ffoaniAokM2qSmrtXHmlpvYr5AVVX8msAyW0l8NVJFDxlSK4u3Uh/f41cQheDVdnYijwQ=="],
"raw-body/iconv-lite": ["iconv-lite@0.7.2", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-im9DjEDQ55s9fL4EYzOAv0yMqmMBSZp6G0VvFyTMPKWxiSBHUj9NW/qqLmXUwXrrM7AvqSlTCfvqRb0cM8yYqw=="],
"react-router/cookie": ["cookie@1.1.1", "", {}, "sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ=="],
"string-width-cjs/emoji-regex": ["emoji-regex@8.0.0", "", {}, "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A=="],
"string-width-cjs/strip-ansi": ["strip-ansi@6.0.1", "", { "dependencies": { "ansi-regex": "^5.0.1" } }, "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A=="],
"strip-ansi-cjs/ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="],
"wrap-ansi/ansi-styles": ["ansi-styles@6.2.3", "", {}, "sha512-4Dj6M28JB+oAH8kFkTLUo+a2jwOFkuqb3yucU0CANcRRUbxS0cP0nZYCGjcc3BNXwRIsUVmDGgzawme7zvJHvg=="],
"wrap-ansi-cjs/string-width": ["string-width@4.2.3", "", { "dependencies": { "emoji-regex": "^8.0.0", "is-fullwidth-code-point": "^3.0.0", "strip-ansi": "^6.0.1" } }, "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g=="],
"wrap-ansi-cjs/strip-ansi": ["strip-ansi@6.0.1", "", { "dependencies": { "ansi-regex": "^5.0.1" } }, "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A=="],
"@modelcontextprotocol/sdk/ajv/json-schema-traverse": ["json-schema-traverse@1.0.0", "", {}, "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug=="],
"ajv-formats/ajv/json-schema-traverse": ["json-schema-traverse@1.0.0", "", {}, "sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug=="],
"broccoli-output-wrapper/fs-extra/jsonfile": ["jsonfile@4.0.0", "", { "optionalDependencies": { "graceful-fs": "^4.1.6" } }, "sha512-m6F1R3z8jjlf2imQHS2Qez5sjKWQzbuuhuJ/FKYFRZvPE3PuHcSMVZzfsLhGVOkfd20obL5SWEBew5ShlquNxg=="],
"broccoli-output-wrapper/fs-extra/universalify": ["universalify@0.1.2", "", {}, "sha512-rBJeI5CXAlmy1pV+617WB9J63U6XcazHHF2f2dbJix4XzpUF0RS3Zbj0FGIOCAva5P/d/GBOYaACQ1w+0azUkg=="],
"fs-merger/fs-extra/jsonfile": ["jsonfile@4.0.0", "", { "optionalDependencies": { "graceful-fs": "^4.1.6" } }, "sha512-m6F1R3z8jjlf2imQHS2Qez5sjKWQzbuuhuJ/FKYFRZvPE3PuHcSMVZzfsLhGVOkfd20obL5SWEBew5ShlquNxg=="],
"fs-merger/fs-extra/universalify": ["universalify@0.1.2", "", {}, "sha512-rBJeI5CXAlmy1pV+617WB9J63U6XcazHHF2f2dbJix4XzpUF0RS3Zbj0FGIOCAva5P/d/GBOYaACQ1w+0azUkg=="],
"heimdalljs-logger/debug/ms": ["ms@2.0.0", "", {}, "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A=="],
"quick-temp/rimraf/glob": ["glob@10.5.0", "", { "dependencies": { "foreground-child": "^3.1.0", "jackspeak": "^3.1.2", "minimatch": "^9.0.4", "minipass": "^7.1.2", "package-json-from-dist": "^1.0.0", "path-scurry": "^1.11.1" }, "bin": { "glob": "dist/esm/bin.mjs" } }, "sha512-DfXN8DfhJ7NH3Oe7cFmu3NCu1wKbkReJ8TorzSAFbSKrlNaQSKfIzqYqVY8zlbs2NLBbWpRiU52GX2PbaBVNkg=="],
"string-width-cjs/strip-ansi/ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="],
"wrap-ansi-cjs/string-width/emoji-regex": ["emoji-regex@8.0.0", "", {}, "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A=="],
"wrap-ansi-cjs/strip-ansi/ansi-regex": ["ansi-regex@5.0.1", "", {}, "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ=="],
"quick-temp/rimraf/glob/minimatch": ["minimatch@9.0.9", "", { "dependencies": { "brace-expansion": "^2.0.2" } }, "sha512-OBwBN9AL4dqmETlpS2zasx+vTeWclWzkblfZk7KTA5j3jeOONz/tRCnZomUyvNg83wL5Zv9Ss6HMJXAgL8R2Yg=="],
"quick-temp/rimraf/glob/minimatch/brace-expansion": ["brace-expansion@2.1.0", "", { "dependencies": { "balanced-match": "^1.0.0" } }, "sha512-TN1kCZAgdgweJhWWpgKYrQaMNHcDULHkWwQIspdtjV4Y5aurRdZpjAqn6yX3FPqTA9ngHCc4hJxMAMgGfve85w=="],
}
}

190
core/http/react-ui/e2e/audio-transform.spec.js Normal file
View File

@@ -0,0 +1,190 @@
import { test, expect } from '@playwright/test'
function mockCapabilities(page, capabilities) {
return page.route('**/api/models/capabilities', (route) => {
route.fulfill({
contentType: 'application/json',
body: JSON.stringify({ data: capabilities }),
})
})
}
// Returns a (Promise, resolver) pair that records the multipart form fields
// the page submitted to /audio/transformations. The handler returns a tiny
// fake WAV blob so the page can render its result waveforms.
function mockAudioTransform(page, filename = 'transformed.wav') {
let resolveSubmit
const submitted = new Promise((resolve) => { resolveSubmit = resolve })
page.route('**/audio/transformations', (route) => {
if (route.request().method() !== 'POST') return route.continue()
const req = route.request()
const body = req.postData() || ''
resolveSubmit({
contentType: req.headers()['content-type'] || '',
bodySize: body.length,
// Naive multipart field name extraction so a test can assert the
// form-data shape without parsing the multipart body.
fields: Array.from(body.matchAll(/name="([^"]+)"/g)).map((m) => m[1]),
})
const wavHeader = new Uint8Array(44) // 44-byte RIFF/WAVE skeleton
route.fulfill({
status: 200,
headers: {
'Content-Type': 'audio/wav',
'Content-Disposition': `attachment; filename="${filename}"`,
},
body: Buffer.from(wavHeader),
})
})
return submitted
}
// Build a tiny in-memory WAV file (44-byte header, 4 silent samples) so
// Playwright's setInputFiles + the page's audio decoder both have valid
// bytes to chew on. Returns { name, mimeType, buffer } for setInputFiles.
function makeFakeWav(name) {
const sampleRate = 16000
const samples = 4
const dataLen = samples * 2
const buf = Buffer.alloc(44 + dataLen)
buf.write('RIFF', 0)
buf.writeUInt32LE(36 + dataLen, 4)
buf.write('WAVE', 8)
buf.write('fmt ', 12)
buf.writeUInt32LE(16, 16) // PCM chunk size
buf.writeUInt16LE(1, 20) // PCM format
buf.writeUInt16LE(1, 22) // channels = 1
buf.writeUInt32LE(sampleRate, 24) // sample rate
buf.writeUInt32LE(sampleRate * 2, 28)// byte rate
buf.writeUInt16LE(2, 32) // block align
buf.writeUInt16LE(16, 34) // bits per sample
buf.write('data', 36)
buf.writeUInt32LE(dataLen, 40)
// body left as zeros (silence)
return { name, mimeType: 'audio/wav', buffer: buf }
}
test.describe('Audio Transform', () => {
test.beforeEach(async ({ page }) => {
await mockCapabilities(page, [
{ id: 'localvqe', capabilities: ['FLAG_AUDIO_TRANSFORM'] },
])
})
test('audio input has Upload + Record tabs', async ({ page }) => {
await page.goto('/app/transform')
await expect(page.getByRole('button', { name: 'localvqe' })).toBeVisible({ timeout: 10_000 })
// The Audio (required) input should expose both tabs.
const tabs = page.getByRole('tab')
await expect(tabs.filter({ hasText: 'Upload' }).first()).toBeVisible()
await expect(tabs.filter({ hasText: 'Record' }).first()).toBeVisible()
})
test('echo-test button only appears once a reference is loaded', async ({ page }) => {
await page.goto('/app/transform')
await expect(page.getByRole('button', { name: 'localvqe' })).toBeVisible({ timeout: 10_000 })
// No reference yet → echo button hidden.
await expect(page.getByRole('button', { name: /Echo test/ })).toHaveCount(0)
// Upload a reference into the second AudioInput's file picker.
await page.locator('input[type="file"]').nth(1).setInputFiles(makeFakeWav('ref.wav'))
await expect(page.getByRole('button', { name: /Echo test/ })).toBeVisible()
})
test('renders the AudioTransform page directly', async ({ page }) => {
await page.goto('/app/transform')
await expect(page.getByRole('heading', { name: /Audio Transform/ })).toBeVisible({ timeout: 10_000 })
await expect(page.getByRole('button', { name: 'localvqe' })).toBeVisible()
// Audio (required) + Reference (optional) drop zones
await expect(page.getByText(/Audio \(required\)/)).toBeVisible()
await expect(page.getByText(/Reference \(optional\)/)).toBeVisible()
})
test('uploads an audio file, posts multipart, renders enhanced waveform', async ({ page }) => {
const submitted = mockAudioTransform(page, 'enhanced.wav')
await page.goto('/app/transform')
await expect(page.getByRole('button', { name: 'localvqe' })).toBeVisible({ timeout: 10_000 })
// Upload mic file via the hidden file input under "Audio (required)".
const audioInput = page.locator('input[type="file"]').first()
await audioInput.setInputFiles(makeFakeWav('mic.wav'))
await expect(page.getByText('mic.wav')).toBeVisible()
// Set a backend tuning param so the form posts params[noise_gate]=true.
await page.locator('.textarea').fill('noise_gate=true')
await page.getByRole('button', { name: /Transform/ }).last().click()
const form = await submitted
expect(form.contentType).toContain('multipart/form-data')
expect(form.fields).toContain('model')
expect(form.fields).toContain('audio')
expect(form.fields).toContain('params[noise_gate]')
// After processing, the output WaveformPlayer mounts with a download button.
await expect(page.getByRole('link', { name: /Download/ })).toBeVisible({ timeout: 10_000 })
})
test('reference file is forwarded as a multipart field when provided', async ({ page }) => {
const submitted = mockAudioTransform(page)
await page.goto('/app/transform')
await expect(page.getByRole('button', { name: 'localvqe' })).toBeVisible({ timeout: 10_000 })
const inputs = page.locator('input[type="file"]')
await inputs.nth(0).setInputFiles(makeFakeWav('mic.wav'))
// After the audio file is set, that AudioInput collapses to a filename +
// Clear button and removes its <input>. The reference AudioInput, which
// was at nth(1), is now the sole remaining input — query afresh.
await inputs.first().setInputFiles(makeFakeWav('loopback.wav'))
await page.getByRole('button', { name: /Transform/ }).last().click()
const form = await submitted
expect(form.fields).toContain('audio')
expect(form.fields).toContain('reference')
})
test('history entry appears after a successful transform and persists across navigation', async ({ page }) => {
mockAudioTransform(page, 'enhanced.wav')
await page.goto('/app/transform')
await expect(page.getByRole('button', { name: 'localvqe' })).toBeVisible({ timeout: 10_000 })
await page.locator('input[type="file"]').first().setInputFiles(makeFakeWav('mic.wav'))
await page.getByRole('button', { name: /Transform/ }).last().click()
await expect(page.getByRole('link', { name: /Download/ })).toBeVisible({ timeout: 10_000 })
await expect(page.getByTestId('media-history-item')).toHaveCount(1)
await expect(page.getByTestId('media-history-item')).toContainText('mic.wav')
// Persist across page reloads via localStorage.
await page.waitForTimeout(600)
await page.goto('/app/transform')
await expect(page.getByTestId('media-history-item')).toHaveCount(1)
})
test('shows an error banner when the backend returns 4xx', async ({ page }) => {
await page.route('**/audio/transformations', (route) => {
if (route.request().method() !== 'POST') return route.continue()
route.fulfill({
status: 400,
contentType: 'application/json',
body: JSON.stringify({ error: { message: 'audio sample rate 44100 != model 16000' } }),
})
})
await page.goto('/app/transform')
await expect(page.getByRole('button', { name: 'localvqe' })).toBeVisible({ timeout: 10_000 })
await page.locator('input[type="file"]').first().setInputFiles(makeFakeWav('mic.wav'))
await page.getByRole('button', { name: /Transform/ }).last().click()
await expect(page.getByText(/sample rate/)).toBeVisible({ timeout: 10_000 })
})
})

2
core/http/react-ui/public/locales/de/nav.json
View File

@@ -14,6 +14,7 @@
"accountFor": "Konto: {{name}}",
"sections": {
"tools": "Werkzeuge",
"enhance": "Verbessern",
"biometrics": "Biometrie",
"agents": "Agenten",
"system": "System"
@@ -26,6 +27,7 @@
"talk": "Sprechen",
"fineTune": "Fine-Tuning (Experimentell)",
"quantize": "Quantisierung (Experimentell)",
"audioTransform": "Audio transformieren",
"faceRecognition": "Gesichtserkennung",
"voiceRecognition": "Spracherkennung",
"agents": "Agenten",

2
core/http/react-ui/public/locales/en/nav.json
View File

@@ -14,6 +14,7 @@
"accountFor": "Account: {{name}}",
"sections": {
"tools": "Tools",
"enhance": "Enhance",
"biometrics": "Biometrics",
"agents": "Agents",
"system": "System"
@@ -26,6 +27,7 @@
"talk": "Talk",
"fineTune": "Fine-Tune (Experimental)",
"quantize": "Quantize (Experimental)",
"audioTransform": "Audio Transform",
"faceRecognition": "Face Recognition",
"voiceRecognition": "Voice Recognition",
"agents": "Agents",

2
core/http/react-ui/public/locales/es/nav.json
View File

@@ -14,6 +14,7 @@
"accountFor": "Cuenta: {{name}}",
"sections": {
"tools": "Herramientas",
"enhance": "Mejorar",
"biometrics": "Biometría",
"agents": "Agentes",
"system": "Sistema"
@@ -26,6 +27,7 @@
"talk": "Hablar",
"fineTune": "Ajuste fino (Experimental)",
"quantize": "Cuantización (Experimental)",
"audioTransform": "Transformar audio",
"faceRecognition": "Reconocimiento facial",
"voiceRecognition": "Reconocimiento de voz",
"agents": "Agentes",

2
core/http/react-ui/public/locales/it/nav.json
View File

@@ -14,6 +14,7 @@
"accountFor": "Account: {{name}}",
"sections": {
"tools": "Strumenti",
"enhance": "Migliora",
"biometrics": "Biometria",
"agents": "Agenti",
"system": "Sistema"
@@ -26,6 +27,7 @@
"talk": "Conversazione",
"fineTune": "Fine-Tuning (Sperimentale)",
"quantize": "Quantizzazione (Sperimentale)",
"audioTransform": "Trasforma audio",
"faceRecognition": "Riconoscimento volti",
"voiceRecognition": "Riconoscimento vocale",
"agents": "Agenti",

2
core/http/react-ui/public/locales/zh-CN/nav.json
View File

@@ -14,6 +14,7 @@
"accountFor": "账户：{{name}}",
"sections": {
"tools": "工具",
"enhance": "增强",
"biometrics": "生物识别",
"agents": "智能体",
"system": "系统"
@@ -26,6 +27,7 @@
"talk": "通话",
"fineTune": "微调（实验性）",
"quantize": "量化（实验性）",
"audioTransform": "音频变换",
"faceRecognition": "人脸识别",
"voiceRecognition": "语音识别",
"agents": "智能体",

213
core/http/react-ui/src/App.css
View File

@@ -6771,6 +6771,219 @@ select.input {
font-size: var(--text-sm);
}
/* Reusable waveform-and-playback component (audio transform / TTS / sound / traces) */
.audio-waveform-player {
--audio-wave: var(--color-primary);
display: flex;
flex-direction: column;
gap: var(--spacing-xs);
width: 100%;
}
.audio-waveform-player--dimmed .audio-waveform-player__canvas-wrap {
opacity: 0.7;
}
.audio-waveform-player__label {
font-size: var(--text-sm);
color: var(--color-text-secondary);
}
.audio-waveform-player__canvas-wrap {
position: relative;
width: 100%;
background: var(--color-surface-sunken);
border: 1px solid var(--color-border-subtle);
border-radius: var(--radius-md);
overflow: hidden;
}
.audio-waveform-player__error {
padding: var(--spacing-md);
color: var(--color-error);
font-size: var(--text-sm);
}
.audio-waveform-player__segment {
position: absolute;
top: 0;
bottom: 0;
background: rgba(136, 192, 208, 0.16);
border-left: 1px dashed var(--color-primary);
border-right: 1px dashed var(--color-primary);
pointer-events: none;
}
.audio-waveform-player__seglabel {
position: absolute;
top: 4px;
left: 4px;
font-size: var(--text-xs);
color: var(--color-text-primary);
background: var(--color-bg-overlay);
padding: 1px 6px;
border-radius: var(--radius-sm);
max-width: calc(100% - 8px);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.audio-waveform-player__duration {
position: absolute;
right: 8px;
bottom: 6px;
font-size: 11px;
color: var(--color-text-muted);
font-variant-numeric: tabular-nums;
background: var(--color-bg-overlay);
padding: 1px 6px;
border-radius: var(--radius-sm);
}
.audio-waveform-player__loading {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
color: var(--color-text-muted);
font-size: var(--text-sm);
}
.audio-waveform-player__playhead {
position: absolute;
top: 0;
bottom: 0;
width: 1.5px;
background: var(--color-primary);
opacity: 0.85;
pointer-events: none;
transform: translateX(-0.75px);
}
.audio-waveform-player__player {
width: 100%;
}
.audio-waveform-player__download {
align-self: flex-end;
font-size: var(--text-sm);
color: var(--color-primary);
}
/* Audio Transform Studio tab */
.audio-transform-stack {
display: flex;
flex-direction: column;
gap: var(--spacing-md);
width: 100%;
}
.audio-transform-drop {
display: flex;
align-items: center;
gap: var(--spacing-sm);
padding: var(--spacing-md);
border: 1px dashed var(--color-border);
border-radius: var(--radius-md);
background: var(--color-surface-sunken);
color: var(--color-text-secondary);
cursor: default;
transition: border-color var(--duration-normal, 180ms) var(--ease-default, ease);
}
.audio-transform-drop--hover {
border-color: var(--color-primary);
background: var(--color-primary-light);
}
.audio-transform-drop__file {
display: flex;
align-items: center;
gap: var(--spacing-sm);
flex: 1;
font-family: var(--font-mono);
word-break: break-all;
}
.audio-transform-drop__pick {
cursor: pointer;
color: var(--color-primary);
text-decoration: underline;
margin-left: 4px;
}
.audio-transform-input {
display: flex;
flex-direction: column;
gap: var(--spacing-sm);
}
.audio-transform-input__tabs {
display: inline-flex;
gap: 2px;
background: var(--color-surface-sunken);
border: 1px solid var(--color-border-subtle);
border-radius: var(--radius-md);
padding: 2px;
align-self: flex-start;
}
.audio-transform-input__tab {
border: 0;
background: transparent;
padding: 4px 10px;
font-size: var(--text-sm);
color: var(--color-text-secondary);
border-radius: var(--radius-sm);
cursor: pointer;
}
.audio-transform-input__tab.active {
background: var(--color-primary-light);
color: var(--color-text-primary);
}
.audio-transform-rec {
display: flex;
flex-direction: column;
gap: var(--spacing-sm);
align-items: flex-start;
padding: var(--spacing-md);
border: 1px solid var(--color-border-subtle);
border-radius: var(--radius-md);
background: var(--color-surface-sunken);
}
.audio-transform-rec__notice {
font-size: var(--text-sm);
color: var(--color-text-secondary);
display: flex;
gap: var(--spacing-xs);
align-items: center;
}
.audio-transform-rec__notice--error {
color: var(--color-error);
}
.audio-transform-rec__pending {
font-size: var(--text-sm);
color: var(--color-text-muted);
font-style: italic;
}
.audio-transform-echo {
display: flex;
flex-direction: column;
align-items: stretch;
gap: var(--spacing-sm);
margin-bottom: var(--spacing-md);
}
.audio-transform-echo__row {
display: flex;
align-items: center;
gap: var(--spacing-sm);
}
.audio-transform-echo__notice {
display: flex;
gap: var(--spacing-sm);
align-items: flex-start;
padding: var(--spacing-sm) var(--spacing-md);
background: var(--color-info-light);
color: var(--color-text-primary);
border-left: 3px solid var(--color-info);
border-radius: var(--radius-md);
font-size: var(--text-sm);
margin: 0;
}
.audio-transform-echo__notice > i {
color: var(--color-info);
margin-top: 2px;
}
.audio-transform-echo__elapsed {
font-size: var(--text-sm);
color: var(--color-text-muted);
font-variant-numeric: tabular-nums;
}
/* Enrollment layout (register + identify + list) */
.biometrics-enrollgrid {
display: grid;

1
core/http/react-ui/src/components/MediaHistory.jsx
View File

@@ -6,6 +6,7 @@ const ICONS = {
video: 'fa-video',
tts: 'fa-headphones',
sound: 'fa-music',
'audio-transform': 'fa-wave-square',
}
export default memo(function MediaHistory({ entries, selectedId, onSelect, onDelete, onClearAll, mediaType }) {

10
core/http/react-ui/src/components/Sidebar.jsx
View File

@@ -27,6 +27,16 @@ const sections = [
{ path: '/app/quantize', icon: 'fas fa-compress', labelKey: 'items.quantize', feature: 'quantization' },
],
},
{
id: 'enhance',
titleKey: 'sections.enhance',
featureMap: {
'/app/transform': 'audio_transform',
},
items: [
{ path: '/app/transform', icon: 'fas fa-wave-square', labelKey: 'items.audioTransform', feature: 'audio_transform' },
],
},
{
id: 'biometrics',
titleKey: 'sections.biometrics',

130
core/http/react-ui/src/components/audio/WaveformPlayer.jsx Normal file
View File

@@ -0,0 +1,130 @@
import { useEffect, useRef, useState } from 'react'
import useAudioPeaks from '../../hooks/useAudioPeaks'
// WaveformPlayer — reusable audio player combining a standard <audio
// controls> element with a peak-waveform canvas overlay and a click-to-seek
// playhead. The peaks canvas redraws only on src/height/dimmed changes; the
// playhead is a separately positioned div so 4 Hz timeupdate ticks don't
// retrigger the canvas loop.
export default function WaveformPlayer({
src,
height = 96,
label,
download,
dimmed = false,
audioTestId,
}) {
const canvasRef = useRef(null)
const audioRef = useRef(null)
const trackRef = useRef(null)
const { peaks, duration, error } = useAudioPeaks(src)
const [currentTime, setCurrentTime] = useState(0)
useEffect(() => {
const a = audioRef.current
if (!a) return
const onUpdate = () => setCurrentTime(a.currentTime)
a.addEventListener('timeupdate', onUpdate)
return () => a.removeEventListener('timeupdate', onUpdate)
}, [])
useEffect(() => {
const canvas = canvasRef.current
if (!canvas) return
const dpr = window.devicePixelRatio || 1
const cssW = canvas.clientWidth
const cssH = height
canvas.width = Math.floor(cssW * dpr)
canvas.height = Math.floor(cssH * dpr)
const ctx = canvas.getContext('2d')
ctx.setTransform(dpr, 0, 0, dpr, 0, 0)
ctx.clearRect(0, 0, cssW, cssH)
if (!peaks) return
const accent =
getComputedStyle(canvas).getPropertyValue('--audio-wave').trim() ||
getComputedStyle(canvas).getPropertyValue('--color-primary').trim() ||
'#88c0d0'
ctx.fillStyle = dimmed ? withAlpha(accent, 0.32) : accent
const mid = cssH / 2
const barW = Math.max(1, cssW / peaks.length)
for (let i = 0; i < peaks.length; i++) {
const h = Math.max(1, peaks[i] * (cssH * 0.9))
ctx.fillRect(i * barW, mid - h / 2, Math.max(0.5, barW - 0.5), h)
}
}, [peaks, height, dimmed])
const handleSeek = (e) => {
const a = audioRef.current
if (!a || !duration) return
const rect = e.currentTarget.getBoundingClientRect()
const ratio = Math.max(0, Math.min(1, (e.clientX - rect.left) / rect.width))
a.currentTime = ratio * duration
setCurrentTime(a.currentTime)
}
if (!src) return null
const playheadPct = duration > 0 ? Math.min(100, (currentTime / duration) * 100) : 0
return (
<div className={`audio-waveform-player${dimmed ? ' audio-waveform-player--dimmed' : ''}`}>
{label && <div className="audio-waveform-player__label">{label}</div>}
<div
ref={trackRef}
className="audio-waveform-player__canvas-wrap"
style={{ height }}
onClick={handleSeek}
>
{error ? (
<div className="audio-waveform-player__error">{error}</div>
) : (
<canvas
ref={canvasRef}
style={{ width: '100%', height: '100%', cursor: duration > 0 ? 'pointer' : 'default' }}
/>
)}
{duration > 0 && (
<div
className="audio-waveform-player__playhead"
style={{ left: `${playheadPct}%` }}
aria-hidden="true"
/>
)}
{duration > 0 && (
<div className="audio-waveform-player__duration" aria-hidden="true">{duration.toFixed(1)}s</div>
)}
{!peaks && !error && (
<div className="audio-waveform-player__loading">Decoding…</div>
)}
</div>
<audio
ref={audioRef}
controls
src={src}
className="audio-waveform-player__player"
data-testid={audioTestId}
/>
{download && (
<a className="audio-waveform-player__download" href={src} download={download}>
Download
</a>
)}
</div>
)
}
function withAlpha(color, alpha) {
if (!color) return color
const c = color.trim()
if (c.startsWith('#') && c.length === 7) {
const r = parseInt(c.slice(1, 3), 16)
const g = parseInt(c.slice(3, 5), 16)
const b = parseInt(c.slice(5, 7), 16)
return `rgba(${r}, ${g}, ${b}, ${alpha})`
}
if (c.startsWith('rgb(')) {
return c.replace('rgb(', 'rgba(').replace(')', `, ${alpha})`)
}
return c
}

56
core/http/react-ui/src/components/biometrics/WaveformStrip.jsx
View File

@@ -1,53 +1,14 @@
import { useEffect, useRef, useState } from 'react'
import { useEffect, useRef } from 'react'
import useAudioPeaks from '../../hooks/useAudioPeaks'
// WaveformStrip — decode an audio source (data URL or blob URL) via AudioContext,
// render a mono waveform, and overlay colored segment regions.
// WaveformStrip — display-only waveform with optional colored segment
// overlays. For a player with click-to-seek + audio controls, use
// `components/audio/WaveformPlayer` instead. Both share the
// `useAudioPeaks` hook for peak extraction.
// segments: [{ start: seconds, end: seconds, label?, tone? }]
export default function WaveformStrip({ src, segments = [], height = 120 }) {
const canvasRef = useRef(null)
const [duration, setDuration] = useState(0)
const [peaks, setPeaks] = useState(null)
const [err, setErr] = useState(null)
useEffect(() => {
setPeaks(null)
setDuration(0)
setErr(null)
if (!src) return
let cancelled = false
async function decode() {
try {
const response = await fetch(src)
const buf = await response.arrayBuffer()
const Ctx = window.AudioContext || window.webkitAudioContext
const ctx = new Ctx()
const audioBuf = await ctx.decodeAudioData(buf.slice(0))
if (cancelled) { ctx.close(); return }
const data = audioBuf.getChannelData(0)
const BUCKETS = 480
const step = Math.max(1, Math.floor(data.length / BUCKETS))
const result = new Float32Array(BUCKETS)
for (let i = 0; i < BUCKETS; i++) {
let peak = 0
const start = i * step
const end = Math.min(start + step, data.length)
for (let j = start; j < end; j++) {
const v = Math.abs(data[j])
if (v > peak) peak = v
}
result[i] = peak
}
setPeaks(result)
setDuration(audioBuf.duration)
ctx.close()
} catch (e) {
if (!cancelled) setErr(e?.message || 'Could not decode audio')
}
}
decode()
return () => { cancelled = true }
}, [src])
const { peaks, duration, error } = useAudioPeaks(src)
useEffect(() => {
if (!canvasRef.current || !peaks) return
@@ -61,7 +22,6 @@ export default function WaveformStrip({ src, segments = [], height = 120 }) {
ctx.scale(dpr, dpr)
ctx.clearRect(0, 0, cssW, cssH)
// Waveform
const accent = getComputedStyle(canvas).getPropertyValue('--biometrics-wave').trim() || '#e8a87c'
ctx.fillStyle = accent
const mid = cssH / 2
@@ -72,7 +32,7 @@ export default function WaveformStrip({ src, segments = [], height = 120 }) {
}
}, [peaks, height])
if (err) return <div className="biometrics-waveform biometrics-waveform--error">{err}</div>
if (error) return <div className="biometrics-waveform biometrics-waveform--error">{error}</div>
if (!src) return null
return (

71
core/http/react-ui/src/hooks/useAudioPeaks.js vendored Normal file
View File

@@ -0,0 +1,71 @@
import { useEffect, useState } from 'react'
// Module-scoped lazy AudioContext: WaveformPlayer / WaveformStrip / Strip can
// all coexist on a single page (the AudioTransform page mounts three at once)
// and most browsers cap concurrent AudioContexts at ~6. Keep one alive for
// the lifetime of the tab and reuse it across decodes.
let sharedCtx = null
function getSharedAudioContext() {
if (sharedCtx) return sharedCtx
const Ctx = window.AudioContext || window.webkitAudioContext
if (!Ctx) return null
sharedCtx = new Ctx()
return sharedCtx
}
// useAudioPeaks — decode an audio source (data URL, blob URL, or http URL)
// into a mono peak array suitable for canvas waveform rendering. Returns
// `{ peaks, duration, error, loading }`. Safe under rapid src changes —
// in-flight decodes are cancelled.
export default function useAudioPeaks(src, buckets = 480) {
const [peaks, setPeaks] = useState(null)
const [duration, setDuration] = useState(0)
const [error, setError] = useState(null)
const [loading, setLoading] = useState(false)
useEffect(() => {
setPeaks(null)
setDuration(0)
setError(null)
setLoading(false)
if (!src) return
let cancelled = false
setLoading(true)
async function decode() {
try {
const response = await fetch(src)
const buf = await response.arrayBuffer()
const ctx = getSharedAudioContext()
if (!ctx) throw new Error('Web Audio API not available')
const audioBuf = await ctx.decodeAudioData(buf.slice(0))
if (cancelled) return
const data = audioBuf.getChannelData(0)
const step = Math.max(1, Math.floor(data.length / buckets))
const result = new Float32Array(buckets)
for (let i = 0; i < buckets; i++) {
let peak = 0
const start = i * step
const end = Math.min(start + step, data.length)
for (let j = start; j < end; j++) {
const v = Math.abs(data[j])
if (v > peak) peak = v
}
result[i] = peak
}
setPeaks(result)
setDuration(audioBuf.duration)
setLoading(false)
} catch (e) {
if (!cancelled) {
setError(e?.message || 'Could not decode audio')
setLoading(false)
}
}
}
decode()
return () => { cancelled = true }
}, [src, buckets])
return { peaks, duration, error, loading }
}

1
core/http/react-ui/src/hooks/useMediaHistory.js vendored
View File

@@ -6,6 +6,7 @@ const STORAGE_KEYS = {
video: 'localai_video_history',
tts: 'localai_tts_history',
sound: 'localai_sound_history',
'audio-transform': 'localai_audio_transform_history',
}
const SAVE_DEBOUNCE_MS = 500

19
core/http/react-ui/src/hooks/useObjectUrl.js vendored Normal file
View File

@@ -0,0 +1,19 @@
import { useEffect, useState } from 'react'
// useObjectUrl — derive a blob/object URL from a Blob/File source. Revokes
// the previous URL when the source changes and on unmount, so callers don't
// have to manage URL.createObjectURL lifecycles by hand. Returns null when
// `source` is falsy.
export default function useObjectUrl(source) {
const [url, setUrl] = useState(null)
useEffect(() => {
if (!source) {
setUrl(null)
return
}
const next = URL.createObjectURL(source)
setUrl(next)
return () => URL.revokeObjectURL(next)
}, [source])
return url
}

435
core/http/react-ui/src/pages/AudioTransform.jsx Normal file
View File

@@ -0,0 +1,435 @@
import { useState, useEffect, useRef } from 'react'
import { useParams, useOutletContext } from 'react-router-dom'
import ModelSelector from '../components/ModelSelector'
import { CAP_AUDIO_TRANSFORM } from '../utils/capabilities'
import LoadingSpinner from '../components/LoadingSpinner'
import ErrorWithTraceLink from '../components/ErrorWithTraceLink'
import WaveformPlayer from '../components/audio/WaveformPlayer'
import { audioTransformApi } from '../utils/api'
import { useMediaCapture } from '../hooks/useMediaCapture'
import useObjectUrl from '../hooks/useObjectUrl'
import { useMediaHistory } from '../hooks/useMediaHistory'
import MediaHistory from '../components/MediaHistory'
// AudioTransform — Studio tab for the audio_transform capability. Takes a
// primary audio file plus an optional reference (loopback for AEC, target
// speaker for voice conversion, etc.) and shows three synchronized
// waveforms: input audio / reference / output. Supports both file upload
// and direct mic recording, plus an "echo test" mode that records mic
// while playing the reference — the recorded mic picks up the speaker
// bleed of the reference, giving the user a real (mic, ref) pair to test
// echo cancellation against.
export default function AudioTransform() {
const { model: urlModel } = useParams()
const { addToast } = useOutletContext()
const [model, setModel] = useState(urlModel || '')
const [audioFile, setAudioFile] = useState(null)
const [referenceFile, setReferenceFile] = useState(null)
const [outputUrl, setOutputUrl] = useState(null)
const [paramsText, setParamsText] = useState('')
const [loading, setLoading] = useState(false)
const [error, setError] = useState(null)
const { addEntry, selectEntry, selectedEntry, historyProps } = useMediaHistory('audio-transform')
// Hidden <audio> element that plays the reference out the speakers while
// the mic records — the recording captures the user's voice plus the
// speaker-bleed echo. Headphones short-circuit the path; document only.
const echoAudioRef = useRef(null)
const echoCap = useMediaCapture('audio')
const echoActive = echoCap.active || echoCap.recording
// Blob URLs derived from File state. useObjectUrl revokes the previous
// URL when its source changes and on unmount, so the cleanup is correct
// without a separate effect tracking each setter.
const audioUrl = useObjectUrl(audioFile)
const referenceUrl = useObjectUrl(referenceFile)
useEffect(() => {
return () => { if (outputUrl) URL.revokeObjectURL(outputUrl) }
}, [outputUrl])
const parseParams = () => {
const out = {}
for (const raw of paramsText.split('\n')) {
const line = raw.trim()
if (!line || line.startsWith('#')) continue
const eq = line.indexOf('=')
if (eq < 0) continue
const k = line.slice(0, eq).trim()
const v = line.slice(eq + 1).trim()
if (k) out[k] = v
}
return out
}
const handleProcess = async (e) => {
e.preventDefault()
if (!model) { addToast('Please select a model', 'warning'); return }
if (!audioFile) { addToast('Please choose an audio file', 'warning'); return }
setLoading(true)
setError(null)
if (outputUrl) { URL.revokeObjectURL(outputUrl); setOutputUrl(null) }
try {
const { blob, serverUrl, inputUrl, referenceUrl: refServerUrl } = await audioTransformApi.process({
model,
audioFile,
referenceFile,
format: 'wav',
params: parseParams(),
})
const url = URL.createObjectURL(blob)
setOutputUrl(url)
addToast('Audio transformed', 'success')
if (serverUrl) {
// Save the persisted (input, reference, output) triple so a click
// in the History panel can later replay all three players. The
// server held onto the converted 16 kHz mono inputs — saving raw
// upload bytes in localStorage would blow past quota in a few runs.
addEntry({
prompt: describeRun(audioFile, referenceFile),
model,
params: parseParams(),
results: [
{ kind: 'output', url: serverUrl },
inputUrl ? { kind: 'input', url: inputUrl } : null,
refServerUrl ? { kind: 'reference', url: refServerUrl } : null,
].filter(Boolean),
})
}
selectEntry(null)
} catch (err) {
setError(err.message)
} finally {
setLoading(false)
}
}
// The echo-test playback listener is held in a ref so stopEchoTest can
// detach it without depending on the closure that registered it.
const echoEndedListenerRef = useRef(null)
const detachEchoEndedListener = () => {
const audio = echoAudioRef.current
const listener = echoEndedListenerRef.current
if (audio && listener) audio.removeEventListener('ended', listener)
echoEndedListenerRef.current = null
}
const startEchoTest = async () => {
if (!referenceUrl) {
addToast('Load a reference first', 'warning')
return
}
if (!echoCap.supported) {
addToast('Browser does not expose getUserMedia', 'warning')
return
}
try {
// Acquire the mic first so the recording covers the entire ref playback.
await echoCap.start()
const recPromise = echoCap.startRecording()
const audio = echoAudioRef.current
if (audio) {
audio.currentTime = 0
const onEnded = () => {
detachEchoEndedListener()
echoCap.stopRecording()
}
echoEndedListenerRef.current = onEnded
audio.addEventListener('ended', onEnded)
try { await audio.play() } catch (_) { /* user-gesture gate, ignore */ }
}
const result = await recPromise
detachEchoEndedListener()
echoCap.stop()
const file = new File([result.blob], 'mic-echo-test.wav', { type: 'audio/wav' })
setAudioFile(file)
addToast('Recorded (mic + reference echo). Click Transform to test AEC.', 'success')
} catch (err) {
detachEchoEndedListener()
addToast(`Echo test failed: ${err?.message || err}`, 'error')
}
}
const stopEchoTest = () => {
detachEchoEndedListener()
echoCap.stopRecording()
if (echoAudioRef.current) {
try { echoAudioRef.current.pause() } catch (_) { /* ignore */ }
}
echoCap.stop()
}
return (
<div className="media-layout">
<div className="media-controls">
<div className="page-header">
<h1 className="page-title"><i className="fas fa-wave-square" /> Audio Transform</h1>
</div>
<form onSubmit={handleProcess}>
<div className="form-group">
<label className="form-label">Model</label>
<ModelSelector value={model} onChange={setModel} capability={CAP_AUDIO_TRANSFORM} />
</div>
<AudioInput
label="Audio (required)"
file={audioFile}
onChange={setAudioFile}
/>
<AudioInput
label="Reference (optional)"
help="Loopback / far-end signal for echo cancellation, target speaker for voice conversion. Leave empty for unconditional transform."
file={referenceFile}
onChange={setReferenceFile}
/>
{referenceFile && (
<div className="audio-transform-echo">
<p className="audio-transform-echo__notice" role="note">
<i className="fas fa-circle-info" aria-hidden="true" />
<span>
Browsers often apply their own WebRTC echo cancellation and
noise suppression by default. This usually results in worse
performance than running LocalVQE on the raw audio.
</span>
</p>
<div className="audio-transform-echo__row">
<button
type="button"
className={`btn ${echoActive ? 'btn-secondary' : 'btn-primary'} btn-sm`}
onClick={echoActive ? stopEchoTest : startEchoTest}
>
{echoActive
? <><i className="fas fa-stop" /> Stop echo test</>
: <><i className="fas fa-headphones-alt" /> Echo test (record mic while playing reference)</>}
</button>
{echoActive && echoCap.recording && (
<span className="audio-transform-echo__elapsed">
recording {echoCap.elapsed.toFixed(1)}s
</span>
)}
</div>
{/* Hidden player for the reference clip during the echo test.
Hidden because the user already has the WaveformPlayer in
the preview pane — this is just the audible source. */}
<audio ref={echoAudioRef} src={referenceUrl} preload="auto" hidden />
</div>
)}
<div className="form-group">
<label className="form-label">
Advanced parameters
<span className="form-help"> &mdash; backend-specific (one <code>key=value</code> per line, e.g. <code>noise_gate=true</code>)</span>
</label>
<textarea
className="textarea"
value={paramsText}
onChange={(e) => setParamsText(e.target.value)}
placeholder={`# Optional. For LocalVQE:\n# noise_gate=true\n# noise_gate_threshold_dbfs=-50`}
rows={4}
/>
</div>
<button type="submit" className="btn btn-primary btn-full" disabled={loading}>
{loading ? <><LoadingSpinner size="sm" /> Processing...</> : <><i className="fas fa-wand-magic-sparkles" /> Transform</>}
</button>
</form>
<MediaHistory {...historyProps} />
</div>
<div className="media-preview">
<div className="media-result">
{error ? (
<ErrorWithTraceLink message={error} />
) : selectedEntry ? (
<div className="audio-transform-stack">
{selectedEntry.results.map((r) => (
<WaveformPlayer
key={r.kind || r.url}
src={r.url}
label={resultLabel(r)}
height={r.kind === 'output' ? 120 : 96}
dimmed={r.kind === 'reference'}
download={r.kind === 'output' ? `audio-transform-${selectedEntry.model || 'output'}.wav` : undefined}
/>
))}
<div className="result-quote">{selectedEntry.prompt}</div>
</div>
) : (
<div className="audio-transform-stack">
<WaveformPlayer src={audioUrl} label="Audio" height={96} />
<WaveformPlayer src={referenceUrl} label="Reference" height={96} dimmed={!referenceFile} />
{outputUrl && (
<WaveformPlayer
src={outputUrl}
label="Output"
height={120}
download={`audio-transform-${model || 'output'}-${new Date().toISOString().slice(0, 10)}.wav`}
/>
)}
{!audioUrl && !outputUrl && (
<div className="media-empty">
<i className="fas fa-wave-square media-empty__icon" />
<p>Choose an audio file (and optional reference) to transform</p>
</div>
)}
</div>
)}
</div>
</div>
</div>
)
}
function describeRun(audioFile, referenceFile) {
const parts = []
if (audioFile?.name) parts.push(`audio: ${audioFile.name}`)
if (referenceFile?.name) parts.push(`reference: ${referenceFile.name}`)
return parts.join(' + ') || 'audio transform'
}
function resultLabel(r) {
switch (r.kind) {
case 'input': return 'Audio'
case 'reference': return 'Reference'
case 'output': return 'Output'
default: return ''
}
}
// AudioInput — drag-drop / file-pick for an audio file, with an inline
// mic-record tab. Emits a single File via onChange (recordings are wrapped
// as `File([blob], 'recording-XXX.wav', { type: 'audio/wav' })` so callers
// can treat them identically to uploaded files).
function AudioInput({ label, help, file, onChange }) {
const [tab, setTab] = useState('upload') // 'upload' | 'record'
const cap = useMediaCapture('audio')
const [recordPending, setRecordPending] = useState(false)
const [hover, setHover] = useState(false)
const onDrop = (e) => {
e.preventDefault()
setHover(false)
const f = e.dataTransfer.files?.[0]
if (f) onChange(f)
}
const onPick = (e) => {
const f = e.target.files?.[0]
if (f) onChange(f)
}
const startRecord = async () => {
await cap.start()
if (cap.error) return
setRecordPending(true)
try {
const promise = cap.startRecording()
if (!promise) return
const result = await promise
const stamp = new Date().toISOString().replace(/[:.]/g, '-').slice(0, 19)
onChange(new File([result.blob], `recording-${stamp}.wav`, { type: 'audio/wav' }))
} finally {
setRecordPending(false)
}
}
const stopRecord = () => cap.stopRecording()
const hasFile = !!file
return (
<div className="form-group">
<label className="form-label">{label}</label>
<div className="audio-transform-input">
<div className="audio-transform-input__tabs" role="tablist">
<button
type="button"
role="tab"
aria-selected={tab === 'upload'}
className={`audio-transform-input__tab${tab === 'upload' ? ' active' : ''}`}
onClick={() => setTab('upload')}
>
<i className="fas fa-upload" /> Upload
</button>
<button
type="button"
role="tab"
aria-selected={tab === 'record'}
className={`audio-transform-input__tab${tab === 'record' ? ' active' : ''}`}
onClick={() => setTab('record')}
>
<i className="fas fa-microphone" /> Record
</button>
</div>
{tab === 'upload' && (
<div
className={`audio-transform-drop${hover ? ' audio-transform-drop--hover' : ''}`}
onDragEnter={(e) => { e.preventDefault(); setHover(true) }}
onDragOver={(e) => { e.preventDefault(); setHover(true) }}
onDragLeave={() => setHover(false)}
onDrop={onDrop}
>
{hasFile ? (
<div className="audio-transform-drop__file">
<i className="fas fa-file-audio" /> {file.name}
<button type="button" className="btn btn-secondary btn-sm" onClick={() => onChange(null)}>Clear</button>
</div>
) : (
<>
<i className="fas fa-upload" /> Drop a file here or
<label className="audio-transform-drop__pick">
<input type="file" accept="audio/*" onChange={onPick} hidden />
browse
</label>
</>
)}
</div>
)}
{tab === 'record' && (
<div className="audio-transform-rec">
{!cap.supported && (
<div className="audio-transform-rec__notice">
<i className="fas fa-circle-info" /> Microphone capture is unavailable in this browser.
</div>
)}
{cap.supported && (
<>
{!cap.recording && !recordPending && (
<button type="button" className="btn btn-primary btn-sm" onClick={startRecord}>
<i className="fas fa-circle" style={{ color: '#e25555' }} /> Start recording
</button>
)}
{cap.recording && (
<button type="button" className="btn btn-secondary btn-sm" onClick={stopRecord}>
<i className="fas fa-stop" /> Stop ({cap.elapsed.toFixed(1)}s)
</button>
)}
{recordPending && !cap.recording && (
<div className="audio-transform-rec__pending">Encoding…</div>
)}
{cap.error && (
<div className="audio-transform-rec__notice audio-transform-rec__notice--error">
{cap.error}
</div>
)}
{hasFile && !cap.recording && (
<div className="audio-transform-drop__file" style={{ marginTop: 'var(--spacing-sm)' }}>
<i className="fas fa-file-audio" /> {file.name}
<button type="button" className="btn btn-secondary btn-sm" onClick={() => onChange(null)}>Clear</button>
</div>
)}
</>
)}
</div>
)}
</div>
{help && <div className="form-help">{help}</div>}
</div>
)
}

16
core/http/react-ui/src/pages/Sound.jsx
View File

@@ -1,10 +1,11 @@
import { useState, useRef } from 'react'
import { useState } from 'react'
import { useParams, useOutletContext } from 'react-router-dom'
import ModelSelector from '../components/ModelSelector'
import { CAP_SOUND_GENERATION } from '../utils/capabilities'
import LoadingSpinner from '../components/LoadingSpinner'
import ErrorWithTraceLink from '../components/ErrorWithTraceLink'
import MediaHistory from '../components/MediaHistory'
import WaveformPlayer from '../components/audio/WaveformPlayer'
import { soundApi } from '../utils/api'
import { useMediaHistory } from '../hooks/useMediaHistory'
@@ -27,7 +28,6 @@ export default function Sound() {
const [loading, setLoading] = useState(false)
const [error, setError] = useState(null)
const [audioUrl, setAudioUrl] = useState(null)
const audioRef = useRef(null)
const { addEntry, selectEntry, selectedEntry, historyProps } = useMediaHistory('sound')
const handleGenerate = async (e) => {
@@ -67,7 +67,6 @@ export default function Sound() {
addEntry({ prompt: promptText, model, params: { mode }, results: [{ url: serverUrl }] })
}
selectEntry(null)
setTimeout(() => audioRef.current?.play().catch(() => {}), 100)
} catch (err) {
setError(err.message)
} finally {
@@ -150,15 +149,16 @@ export default function Sound() {
<ErrorWithTraceLink message={error} />
) : selectedEntry ? (
<div className="audio-result">
<audio controls src={selectedEntry.results[0]?.url} className="audio-result__player" data-testid="history-audio" />
<WaveformPlayer src={selectedEntry.results[0]?.url} height={96} />
<div className="result-quote">"{selectedEntry.prompt}"</div>
</div>
) : audioUrl ? (
<div className="audio-result">
<audio ref={audioRef} controls src={audioUrl} className="audio-result__player" />
<a href={audioUrl} download={`sound-${new Date().toISOString().slice(0, 10)}.wav`} className="btn btn-primary btn-sm">
<i className="fas fa-download" /> <span>Download</span>
</a>
<WaveformPlayer
src={audioUrl}
height={96}
download={`sound-${new Date().toISOString().slice(0, 10)}.wav`}
/>
</div>
) : (
<div className="media-empty">

21
core/http/react-ui/src/pages/TTS.jsx
View File

@@ -1,4 +1,4 @@
import { useState, useRef } from 'react'
import { useState } from 'react'
import { useParams, useOutletContext } from 'react-router-dom'
import { useTranslation } from 'react-i18next'
import ModelSelector from '../components/ModelSelector'
@@ -6,6 +6,7 @@ import { CAP_TTS } from '../utils/capabilities'
import LoadingSpinner from '../components/LoadingSpinner'
import ErrorWithTraceLink from '../components/ErrorWithTraceLink'
import MediaHistory from '../components/MediaHistory'
import WaveformPlayer from '../components/audio/WaveformPlayer'
import { ttsApi } from '../utils/api'
import { useMediaHistory } from '../hooks/useMediaHistory'
@@ -18,7 +19,6 @@ export default function TTS() {
const [loading, setLoading] = useState(false)
const [error, setError] = useState(null)
const [audioUrl, setAudioUrl] = useState(null)
const audioRef = useRef(null)
const { addEntry, selectEntry, selectedEntry, historyProps } = useMediaHistory('tts')
const handleGenerate = async (e) => {
@@ -39,7 +39,6 @@ export default function TTS() {
addEntry({ prompt: text.trim(), model, params: {}, results: [{ url: serverUrl }] })
}
selectEntry(null)
setTimeout(() => audioRef.current?.play(), 100)
} catch (err) {
setError(err.message)
} finally {
@@ -84,20 +83,16 @@ export default function TTS() {
<ErrorWithTraceLink message={error} />
) : selectedEntry ? (
<div className="audio-result">
<audio controls src={selectedEntry.results[0]?.url} className="audio-result__player" data-testid="history-audio" />
<WaveformPlayer src={selectedEntry.results[0]?.url} height={96} audioTestId="history-audio" />
<div className="result-quote">"{selectedEntry.prompt}"</div>
</div>
) : audioUrl ? (
<div className="audio-result">
<audio ref={audioRef} controls src={audioUrl} className="audio-result__player" />
<div className="audio-result__actions">
<a href={audioUrl} download={`tts-${model}-${new Date().toISOString().slice(0, 10)}.mp3`} className="btn btn-primary btn-sm">
<i className="fas fa-download" /> <span>Download</span>
</a>
<button type="button" className="btn btn-secondary btn-sm" onClick={() => audioRef.current?.play()}>
<i className="fas fa-rotate-right" /> <span>Replay</span>
</button>
</div>
<WaveformPlayer
src={audioUrl}
height={96}
download={`tts-${model}-${new Date().toISOString().slice(0, 10)}.mp3`}
/>
<div className="result-quote">"{text}"</div>
</div>
) : (

5
core/http/react-ui/src/pages/Traces.jsx
View File

@@ -6,6 +6,7 @@ import { formatTimestamp } from '../utils/format'
import LoadingSpinner from '../components/LoadingSpinner'
import Toggle from '../components/Toggle'
import SettingRow from '../components/SettingRow'
import WaveformPlayer from '../components/audio/WaveformPlayer'
const AUDIO_DATA_KEYS = new Set([
'audio_wav_base64', 'audio_duration_s', 'audio_snippet_s',
@@ -98,8 +99,8 @@ function AudioSnippet({ data }) {
<i className="fas fa-headphones" style={{ color: 'var(--color-primary)' }} /> Audio Snippet
</h4>
<div style={{ background: 'var(--color-bg-primary)', border: '1px solid var(--color-border)', borderRadius: 'var(--radius-md)', padding: 'var(--spacing-sm)' }}>
<audio controls style={{ width: '100%', marginBottom: 'var(--spacing-sm)' }} src={`data:audio/wav;base64,${data.audio_wav_base64}`} />
<div style={{ display: 'grid', gridTemplateColumns: 'repeat(auto-fill, minmax(120px, 1fr))', gap: 'var(--spacing-xs)', fontSize: '0.75rem' }}>
<WaveformPlayer src={`data:audio/wav;base64,${data.audio_wav_base64}`} height={64} />
<div style={{ display: 'grid', gridTemplateColumns: 'repeat(auto-fill, minmax(120px, 1fr))', gap: 'var(--spacing-xs)', fontSize: '0.75rem', marginTop: 'var(--spacing-sm)' }}>
{metrics.map(m => (
<div key={m.label} style={{ background: 'var(--color-bg-secondary)', borderRadius: 'var(--radius-sm)', padding: 'var(--spacing-xs)' }}>
<div style={{ color: 'var(--color-text-secondary)' }}>{m.label}</div>

3
core/http/react-ui/src/router.jsx
View File

@@ -9,6 +9,7 @@ import ImageGen from './pages/ImageGen'
import VideoGen from './pages/VideoGen'
import TTS from './pages/TTS'
import Sound from './pages/Sound'
import AudioTransform from './pages/AudioTransform'
import Talk from './pages/Talk'
import Backends from './pages/Backends'
import Settings from './pages/Settings'
@@ -73,6 +74,8 @@ const appChildren = [
{ path: 'tts/:model', element: <TTS /> },
{ path: 'sound', element: <Sound /> },
{ path: 'sound/:model', element: <Sound /> },
{ path: 'transform', element: <Feature feature="audio_transform"><AudioTransform /></Feature> },
{ path: 'transform/:model', element: <Feature feature="audio_transform"><AudioTransform /></Feature> },
{ path: 'studio', element: <Studio /> },
{ path: 'talk', element: <Talk /> },
{ path: 'face', element: <Feature feature="face_recognition"><FaceRecognition /></Feature> },

52
core/http/react-ui/src/utils/api.js vendored
View File

@@ -237,12 +237,14 @@ export const videoApi = {
generate: (body) => postJSON(API_CONFIG.endpoints.video, body),
}
async function postAudioBlob(endpoint, body) {
const response = await fetch(apiUrl(endpoint), {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
})
// parseAudioBlobResponse — shared response handling for audio-blob endpoints.
// Throws on non-2xx (with the API error message when present); returns the
// blob plus the parsed Content-Disposition filename mapped to the server's
// /generated-audio/ path so the UI can persist it in history. The audio
// transform endpoint also surfaces the persisted *input* paths via
// X-Audio-Input-Url / X-Audio-Reference-Url headers so the UI can replay
// past (input, reference, output) triples from history.
async function parseAudioBlobResponse(response) {
if (!response.ok) {
const data = await response.json().catch(() => ({}))
throw new Error(data?.error?.message || `HTTP ${response.status}`)
@@ -253,8 +255,24 @@ async function postAudioBlob(endpoint, body) {
const match = disposition.match(/filename[^;=\n]*=["']?([^"';\n]*)["']?/)
if (match && match[1]) serverUrl = '/generated-audio/' + match[1]
}
const inputUrl = response.headers.get('x-audio-input-url') || null
const referenceUrl = response.headers.get('x-audio-reference-url') || null
const blob = await response.blob()
return { blob, serverUrl }
return { blob, serverUrl, inputUrl, referenceUrl }
}
async function postAudioBlob(endpoint, body) {
const response = await fetch(apiUrl(endpoint), {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
})
return parseAudioBlobResponse(response)
}
async function postMultipartAudioBlob(endpoint, formData) {
const response = await fetch(apiUrl(endpoint), { method: 'POST', body: formData })
return parseAudioBlobResponse(response)
}
// TTS
@@ -268,6 +286,26 @@ export const soundApi = {
generate: (body) => postAudioBlob(API_CONFIG.endpoints.soundGeneration, body),
}
// Audio transform (echo cancellation, noise suppression, voice conversion, etc.)
export const audioTransformApi = {
process: ({ model, audioFile, referenceFile, format, sampleRate, params }) => {
const fd = new FormData()
fd.append('model', model)
fd.append('audio', audioFile, audioFile?.name || 'audio.wav')
if (referenceFile) fd.append('reference', referenceFile, referenceFile.name || 'reference.wav')
if (format) fd.append('response_format', format)
if (sampleRate) fd.append('sample_rate', String(sampleRate))
if (params) {
for (const [k, v] of Object.entries(params)) {
if (v == null || v === '') continue
fd.append(`params[${k}]`, String(v))
}
}
return postMultipartAudioBlob(API_CONFIG.endpoints.audioTransformations, fd)
},
streamUrl: () => apiUrl(API_CONFIG.endpoints.audioTransformStream).replace(/^http/, 'ws'),
}
// Audio transcription
export const audioApi = {
transcribe: async (formData) => {

1
core/http/react-ui/src/utils/capabilities.js vendored
View File

@@ -18,3 +18,4 @@ export const CAP_VIDEO = 'FLAG_VIDEO'
export const CAP_DETECTION = 'FLAG_DETECTION'
export const CAP_FACE_RECOGNITION = 'FLAG_FACE_RECOGNITION'
export const CAP_SPEAKER_RECOGNITION = 'FLAG_SPEAKER_RECOGNITION'
export const CAP_AUDIO_TRANSFORM = 'FLAG_AUDIO_TRANSFORM'

2
core/http/react-ui/src/utils/config.js vendored
View File

@@ -71,6 +71,8 @@ export const API_CONFIG = {
imageGenerations: '/v1/images/generations',
audioSpeech: '/v1/audio/speech',
audioTranscriptions: '/v1/audio/transcriptions',
audioTransformations: '/audio/transformations',
audioTransformStream: '/audio/transformations/stream',
soundGeneration: '/v1/sound-generation',
embeddings: '/v1/embeddings',

16
core/http/routes/localai.go
View File

@@ -148,6 +148,22 @@ func RegisterLocalAIRoutes(router *echo.Echo,
requestExtractor.BuildFilteredFirstAvailableDefaultModel(config.BuildUsecaseFilterFn(config.FLAG_TTS)),
requestExtractor.SetModelAndConfig(func() schema.LocalAIRequest { return new(schema.TTSRequest) }))
// audio transform (echo cancellation, noise suppression, voice conversion, etc.)
audioTransformHandler := localai.AudioTransformEndpoint(cl, ml, appConfig)
audioTransformMiddleware := []echo.MiddlewareFunc{
middleware.TraceMiddleware(app),
requestExtractor.BuildFilteredFirstAvailableDefaultModel(config.BuildUsecaseFilterFn(config.FLAG_AUDIO_TRANSFORM)),
requestExtractor.SetModelAndConfig(func() schema.LocalAIRequest { return new(schema.AudioTransformRequest) }),
}
router.POST("/audio/transformations", audioTransformHandler, audioTransformMiddleware...)
router.POST("/audio/transform", audioTransformHandler, audioTransformMiddleware...)
// audio transform streaming WS (sits before the request-extractor pipeline —
// the upgrade is handled by the endpoint itself).
router.GET("/audio/transformations/stream",
localai.AudioTransformStreamEndpoint(app),
middleware.TraceMiddleware(app))
vadHandler := localai.VADEndpoint(cl, ml, appConfig)
router.POST("/vad",
vadHandler,

52
core/schema/audio_transform.go Normal file
View File

@@ -0,0 +1,52 @@
package schema
// @Description Audio transform request body — multipart form-data only.
// `audio` (the primary input file) is required; `reference` (auxiliary
// signal: loopback for echo cancellation, target speaker for voice
// conversion, etc.) is optional. Backend-specific tuning lives in the
// `params[<key>]=<value>` form fields, collected into a generic map so
// the schema doesn't bake in any one transform's vocabulary.
type AudioTransformRequest struct {
BasicModelRequest
Format string `json:"response_format,omitempty" yaml:"response_format,omitempty"` // wav | mp3 | ogg | flac
SampleRate int `json:"sample_rate,omitempty" yaml:"sample_rate,omitempty"` // desired output sample rate; 0 = backend default
Params map[string]string `json:"params,omitempty" yaml:"params,omitempty"` // backend-specific tuning
}
// AudioTransformStreamControl is the JSON envelope used on the
// /audio/transformations/stream WebSocket. The first frame on a new
// connection MUST be a session.update; subsequent frames are binary PCM.
// Server may emit error / session.closed text frames.
type AudioTransformStreamControl struct {
Type string `json:"type"`
Model string `json:"model,omitempty"`
SampleFormat string `json:"sample_format,omitempty"`
SampleRate int `json:"sample_rate,omitempty"`
FrameSamples int `json:"frame_samples,omitempty"`
Params map[string]string `json:"params,omitempty"`
Reset bool `json:"reset,omitempty"`
Error string `json:"error,omitempty"`
}
// AudioTransformStreamControl Type values.
const (
AudioTransformCtrlSessionUpdate = "session.update"
AudioTransformCtrlSessionClose = "session.close"
AudioTransformCtrlSessionClosed = "session.closed"
AudioTransformCtrlError = "error"
)
// AudioTransformStreamControl SampleFormat values (mirror the proto enum
// names so the wire format stays self-describing).
const (
AudioTransformSampleFormatS16LE = "S16_LE"
AudioTransformSampleFormatF32LE = "F32_LE"
)
// LocalVQE param keys — backend-specific but referenced by both the
// HTTP layer (form-field shortcuts, defaults) and the localvqe backend
// itself. Hoisted so renames stay in lockstep.
const (
AudioTransformParamNoiseGate = "noise_gate"
AudioTransformParamNoiseGateThreshold = "noise_gate_threshold_dbfs"
)

6
core/services/nodes/health_mock_test.go
View File

@@ -223,6 +223,12 @@ func (c *fakeBackendClient) AudioEncode(_ context.Context, _ *pb.AudioEncodeRequ
func (c *fakeBackendClient) AudioDecode(_ context.Context, _ *pb.AudioDecodeRequest, _ ...ggrpc.CallOption) (*pb.AudioDecodeResult, error) {
return nil, nil
}
func (c *fakeBackendClient) AudioTransform(_ context.Context, _ *pb.AudioTransformRequest, _ ...ggrpc.CallOption) (*pb.AudioTransformResult, error) {
return nil, nil
}
func (c *fakeBackendClient) AudioTransformStream(_ context.Context, _ ...ggrpc.CallOption) (grpc.AudioTransformStreamClient, error) {
return nil, nil
}
func (c *fakeBackendClient) ModelMetadata(_ context.Context, _ *pb.ModelOptions, _ ...ggrpc.CallOption) (*pb.ModelMetadataResponse, error) {
return nil, nil
}

9
core/services/nodes/inflight_test.go
View File

@@ -8,6 +8,7 @@ import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
grpc "github.com/mudler/LocalAI/pkg/grpc"
pb "github.com/mudler/LocalAI/pkg/grpc/proto"
ggrpc "google.golang.org/grpc"
)
@@ -163,6 +164,14 @@ func (f *fakeGRPCBackend) AudioDecode(_ context.Context, _ *pb.AudioDecodeReques
return &pb.AudioDecodeResult{}, nil
}
func (f *fakeGRPCBackend) AudioTransform(_ context.Context, _ *pb.AudioTransformRequest, _ ...ggrpc.CallOption) (*pb.AudioTransformResult, error) {
return &pb.AudioTransformResult{}, nil
}
func (f *fakeGRPCBackend) AudioTransformStream(_ context.Context, _ ...ggrpc.CallOption) (grpc.AudioTransformStreamClient, error) {
return nil, nil
}
func (f *fakeGRPCBackend) ModelMetadata(_ context.Context, _ *pb.ModelOptions, _ ...ggrpc.CallOption) (*pb.ModelMetadataResponse, error) {
return &pb.ModelMetadataResponse{}, nil
}

1
core/trace/backend_trace.go
View File

@@ -29,6 +29,7 @@ const (
BackendTraceVoiceVerify BackendTraceType = "voice_verify"
BackendTraceVoiceAnalyze BackendTraceType = "voice_analyze"
BackendTraceVoiceEmbed BackendTraceType = "voice_embed"
BackendTraceAudioTransform BackendTraceType = "audio_transform"
BackendTraceModelLoad BackendTraceType = "model_load"
)

4
docs/content/features/audio-to-text.md
View File

@@ -154,3 +154,7 @@ curl http://localhost:8080/v1/audio/transcriptions \
-F file="@jfk.wav" \
-F model="qwen3-asr"
```
## See also
- [Audio Transform]({{< relref "audio-transform.md" >}}) — clean up the audio (echo cancellation, noise suppression, dereverberation) before passing it to a transcription model.

147
docs/content/features/audio-transform.md Normal file
View File

@@ -0,0 +1,147 @@
+++
disableToc = false
title = "Audio Transform"
weight = 17
url = "/features/audio-transform/"
+++
The audio-transform endpoints take **audio in** and emit **audio out**, optionally
conditioned on a second reference audio signal. The category is generic by
design — concrete operations include joint **acoustic echo cancellation +
noise suppression + dereverberation** (LocalVQE), voice conversion (reference
= target speaker), pitch shifting, audio super-resolution, and so on.
The first shipping backend is [LocalVQE](https://github.com/localai-org/LocalVQE),
a 1.3 M-parameter GGML-based model that performs joint AEC + noise suppression
+ dereverberation on 16 kHz mono speech, ~9.6× realtime on a desktop CPU. It
is a derivative of the Microsoft DeepVQE paper.
## The mental model
Every audio-transform request carries:
- **`audio`** — the primary input file (required).
- **`reference`** — an auxiliary signal whose meaning is backend-specific (optional).
- For echo cancellation: the loopback / far-end signal played through the speakers.
- For voice conversion: the target speaker's reference clip.
- For pitch / style transfer: a tonal or style reference.
- When omitted, the backend treats it as silence and degrades gracefully (LocalVQE,
for example, does denoise + dereverb only when ref is empty).
- **`params`** — a generic `key=value` map forwarded to the backend.
- LocalVQE keys: `noise_gate=true|false`, `noise_gate_threshold_dbfs=<float>`.
This shape mirrors WebRTC's `ProcessStream(near)` / `ProcessReverseStream(far)`
APM API, NVIDIA Maxine's `NvAFX_Run` paired-stream signature, and the ICASSP
AEC challenge 2-channel WAV convention.
## Batch endpoint
`POST /audio/transformations` (alias `POST /audio/transform`) — multipart
form-data, returns audio bytes.
| Field | Type | Required | Notes |
|---|---|---|---|
| `model` | string | yes | Audio-transform model id (e.g. `localvqe`) |
| `audio` | file | yes | Primary input audio |
| `reference` | file | no | Optional auxiliary signal |
| `response_format` | string | no | `wav` (default), `mp3`, `ogg`, `flac` |
| `sample_rate` | int | no | Desired output sample rate |
| `params[<key>]` | string | no | Repeated; forwarded to backend |
Example (LocalVQE: cancel echo, suppress noise, gate residual):
```bash
curl -X POST http://localhost:8080/audio/transformations \
-F model=localvqe \
-F audio=@mic.wav \
-F reference=@loopback.wav \
-F 'params[noise_gate]=true' \
-F 'params[noise_gate_threshold_dbfs]=-50' \
-o enhanced.wav
```
When `reference` is omitted, LocalVQE zero-fills the reference channel and
the operation reduces to noise suppression + dereverberation.
## Streaming endpoint
`GET /audio/transformations/stream` — bidirectional WebSocket. The first
client message is a JSON envelope; subsequent client messages are binary
PCM frames; server emits binary PCM frames at the same cadence.
### Wire format
**Client → server** (text frame, first):
```json
{
"type": "session.update",
"model": "localvqe",
"sample_format": "S16_LE",
"sample_rate": 16000,
"frame_samples": 256,
"params": { "noise_gate": "true" }
}
```
`sample_format` is `S16_LE` (16-bit signed little-endian) or `F32_LE` (32-bit
float little-endian, [-1, 1]). `frame_samples` defaults to the backend's
preferred hop length (256 = 16 ms for LocalVQE).
**Client → server** (binary frames, subsequent): interleaved stereo PCM,
channel 0 = audio (mic), channel 1 = reference. Frame size:
`frame_samples × 2 channels × sample_size`. For `S16_LE` at 256 samples that
is 1024 bytes per frame; for `F32_LE` it is 2048 bytes. If the reference is
silent (no auxiliary signal), send zeros on channel 1.
**Server → client** (binary frames): mono PCM in the same format,
`frame_samples × sample_size` bytes (512 bytes for `S16_LE`, 1024 for `F32_LE`).
**Mid-stream control** (text frame): another `session.update` resets the
streaming state when its `reset` field is true; a `session.close` text frame
ends the session cleanly.
### Latency
LocalVQE has 16 ms algorithmic latency (one hop). At runtime, ~1.66 ms of CPU
time per frame on a modern desktop, leaving the rest of the budget for
network and downstream playback.
## Backend-specific tuning (LocalVQE)
| `params[<key>]` | Type | Default | Effect |
|---|---|---|---|
| `noise_gate` | bool | `false` | Enable post-OLA RMS-based residual-echo gate |
| `noise_gate_threshold_dbfs` | float | `-45.0` | Gate threshold in dBFS; frames below are zeroed |
The gate is most useful in far-end-only / silent-near-end stretches where the
model's residual would otherwise sound like buffering or amplified noise floor.
A reasonable starting point is `-50` dBFS.
## Configuring a model
```yaml
name: localvqe
backend: localvqe
parameters:
model: localvqe-v1.1-1.3M-f32.gguf
# Backend-specific defaults can be set in Options[]; per-request
# params[*] form fields override.
#
# `backend` and `device` route through the upstream localvqe options
# builder so you can force a non-default GGML backend (e.g. `Vulkan`) or
# pin to a specific GPU index. Leave both unset to keep the CPU default.
options:
- noise_gate=true
- noise_gate_threshold_dbfs=-50
# - backend=Vulkan
# - device=0
```
## See also
- [Text to Audio (TTS)]({{< relref "tts.md" >}})
- [Audio to Text]({{< relref "audio-to-text.md" >}})
- [LocalVQE upstream](https://github.com/localai-org/LocalVQE)
- [DeepVQE paper (Indenbom et al., Interspeech 2023)](https://arxiv.org/abs/2306.03177)

1
docs/content/whats-new.md
View File

@@ -12,6 +12,7 @@ You can see the release notes [here](https://github.com/mudler/LocalAI/releases)
## 2026 Highlights
- **April 2026**: [Audio Transform](/features/audio-transform/) — generic audio-in / audio-out endpoint with optional reference signal. First implementation: [LocalVQE](https://github.com/localai-org/LocalVQE) C++ backend (joint AEC + noise suppression + dereverberation, DeepVQE-style). Both batch (`POST /audio/transformations`) and bidirectional WebSocket streaming (`/audio/transformations/stream`). Studio "Transform" tab with synchronized waveform players for input / reference / output.
- **April 2026**: [Face recognition backend](/features/face-recognition/) — `insightface`-powered 1:1 verification, 1:N identification, face embedding, face detection, and demographic analysis. Ships both a non-commercial `buffalo_l` model and an Apache 2.0 OpenCV Zoo alternative.
## 2024 Highlights

29
gallery/index.yaml
View File

@@ -18896,6 +18896,35 @@
- filename: silero-vad.onnx
uri: https://huggingface.co/onnx-community/silero-vad/resolve/main/onnx/model.onnx
sha256: a4a068cd6cf1ea8355b84327595838ca748ec29a25bc91fc82e6c299ccdc5808
- name: "localvqe-v1.1-1.3m"
icon: https://avatars.githubusercontent.com/u/260893928
url: github:mudler/LocalAI/gallery/virtual.yaml@master
urls:
- https://github.com/localai-org/LocalVQE
- https://huggingface.co/LocalAI-io/LocalVQE
license: apache-2.0
description: |
LocalVQE v1.1 (1.3 M parameters, F32) — joint acoustic echo cancellation,
noise suppression, and dereverberation for 16 kHz mono speech.
DeepVQE-style architecture with an S4D bottleneck and an in-graph
DCT-II filterbank. ~9.6× realtime on a desktop CPU; 16 ms algorithmic
latency. ~5 MB on disk. v1.1 ships the v16 echoaware checkpoint with
improved double-talk and near-end single-talk AECMOS scores.
tags:
- audio-transform
- aec
- acoustic-echo-cancellation
- noise-suppression
- dereverberation
- cpu
overrides:
backend: localvqe
parameters:
model: localvqe-v1.1-1.3M-f32.gguf
files:
- filename: localvqe-v1.1-1.3M-f32.gguf
uri: huggingface://LocalAI-io/LocalVQE/localvqe-v1.1-1.3M-f32.gguf
sha256: c118227c6b433d6aa36d9e4b993e0f31aa60787ea38d301d04db917a4a2b0a84
- !!merge <<: *silero
name: "silero-vad-ggml"
urls:

3
pkg/grpc/backend.go
View File

@@ -78,6 +78,9 @@ type Backend interface {
AudioEncode(ctx context.Context, in *pb.AudioEncodeRequest, opts ...grpc.CallOption) (*pb.AudioEncodeResult, error)
AudioDecode(ctx context.Context, in *pb.AudioDecodeRequest, opts ...grpc.CallOption) (*pb.AudioDecodeResult, error)
AudioTransform(ctx context.Context, in *pb.AudioTransformRequest, opts ...grpc.CallOption) (*pb.AudioTransformResult, error)
AudioTransformStream(ctx context.Context, opts ...grpc.CallOption) (AudioTransformStreamClient, error)
ModelMetadata(ctx context.Context, in *pb.ModelOptions, opts ...grpc.CallOption) (*pb.ModelMetadataResponse, error)
// Fine-tuning

9
pkg/grpc/base/base.go
View File

@@ -144,6 +144,15 @@ func (llm *Base) AudioDecode(*pb.AudioDecodeRequest) (*pb.AudioDecodeResult, err
return nil, fmt.Errorf("unimplemented")
}
func (llm *Base) AudioTransform(*pb.AudioTransformRequest) (*pb.AudioTransformResult, error) {
return nil, fmt.Errorf("unimplemented")
}
func (llm *Base) AudioTransformStream(in <-chan *pb.AudioTransformFrameRequest, out chan<- *pb.AudioTransformFrameResponse) error {
close(out)
return fmt.Errorf("unimplemented")
}
func (llm *Base) StartFineTune(*pb.FineTuneRequest) (*pb.FineTuneJobResult, error) {
return nil, fmt.Errorf("unimplemented")
}

81
pkg/grpc/client.go
View File

@@ -706,6 +706,87 @@ func (c *Client) AudioDecode(ctx context.Context, in *pb.AudioDecodeRequest, opt
return client.AudioDecode(ctx, in, opts...)
}
func (c *Client) AudioTransform(ctx context.Context, in *pb.AudioTransformRequest, opts ...grpc.CallOption) (*pb.AudioTransformResult, error) {
if !c.parallel {
c.opMutex.Lock()
defer c.opMutex.Unlock()
}
c.setBusy(true)
defer c.setBusy(false)
c.wdMark()
defer c.wdUnMark()
conn, err := c.dial()
if err != nil {
return nil, err
}
defer func() { _ = conn.Close() }()
client := pb.NewBackendClient(conn)
return client.AudioTransform(ctx, in, opts...)
}
// AudioTransformStreamClient is the duplex interface returned by
// (*Client).AudioTransformStream. Wraps the generated bidi client without
// leaking the proto package across the public boundary.
type AudioTransformStreamClient interface {
Send(*pb.AudioTransformFrameRequest) error
Recv() (*pb.AudioTransformFrameResponse, error)
CloseSend() error
Context() context.Context
}
// audioTransformStreamClient is the concrete wrapper. It also owns the
// underlying gRPC connection so it can be closed when the caller is done.
type audioTransformStreamClient struct {
pb.Backend_AudioTransformStreamClient
conn *grpc.ClientConn
closer func()
}
func (s *audioTransformStreamClient) CloseSend() error {
err := s.Backend_AudioTransformStreamClient.CloseSend()
if s.closer != nil {
s.closer()
}
return err
}
func (c *Client) AudioTransformStream(ctx context.Context, opts ...grpc.CallOption) (AudioTransformStreamClient, error) {
if !c.parallel {
c.opMutex.Lock()
}
c.setBusy(true)
c.wdMark()
cleanup := func() {
c.wdUnMark()
c.setBusy(false)
if !c.parallel {
c.opMutex.Unlock()
}
}
conn, err := c.dial()
if err != nil {
cleanup()
return nil, err
}
client := pb.NewBackendClient(conn)
stream, err := client.AudioTransformStream(ctx, opts...)
if err != nil {
_ = conn.Close()
cleanup()
return nil, err
}
return &audioTransformStreamClient{
Backend_AudioTransformStreamClient: stream,
conn: conn,
closer: func() {
_ = conn.Close()
cleanup()
},
}, nil
}
func (c *Client) StartFineTune(ctx context.Context, in *pb.FineTuneRequest, opts ...grpc.CallOption) (*pb.FineTuneJobResult, error) {
if !c.parallel {
c.opMutex.Lock()

132
pkg/grpc/embed.go
View File

@@ -2,6 +2,7 @@ package grpc
import (
"context"
"io"
pb "github.com/mudler/LocalAI/pkg/grpc/proto"
"google.golang.org/grpc"
@@ -143,6 +144,39 @@ func (e *embedBackend) AudioDecode(ctx context.Context, in *pb.AudioDecodeReques
return e.s.AudioDecode(ctx, in)
}
func (e *embedBackend) AudioTransform(ctx context.Context, in *pb.AudioTransformRequest, opts ...grpc.CallOption) (*pb.AudioTransformResult, error) {
return e.s.AudioTransform(ctx, in)
}
func (e *embedBackend) AudioTransformStream(ctx context.Context, opts ...grpc.CallOption) (AudioTransformStreamClient, error) {
// In-process bidi stream is two channels paired with two facades:
// the server side reads requests / writes responses; the client side
// is its mirror.
reqs := make(chan *pb.AudioTransformFrameRequest, 4)
resps := make(chan *pb.AudioTransformFrameResponse, 4)
srvDone := make(chan error, 1)
server := &embedBackendAudioTransformStream{
ctx: ctx,
reqs: reqs,
resps: resps,
}
go func() {
err := e.s.AudioTransformStream(server)
// Backend has finished — no more responses will arrive.
close(resps)
srvDone <- err
}()
return &embedBackendAudioTransformStreamClient{
ctx: ctx,
reqs: reqs,
resps: resps,
srvDone: srvDone,
}, nil
}
func (e *embedBackend) ModelMetadata(ctx context.Context, in *pb.ModelOptions, opts ...grpc.CallOption) (*pb.ModelMetadataResponse, error) {
return e.s.ModelMetadata(ctx, in)
}
@@ -196,6 +230,104 @@ func (e *embedBackend) Free(ctx context.Context) error {
return err
}
var _ pb.Backend_AudioTransformStreamServer = new(embedBackendAudioTransformStream)
var _ AudioTransformStreamClient = new(embedBackendAudioTransformStreamClient)
// embedBackendAudioTransformStream is the server side of an in-process bidi
// stream. The hosted server reads requests from `reqs` (closed by client when
// done sending) and writes responses to `resps`.
type embedBackendAudioTransformStream struct {
ctx context.Context
reqs <-chan *pb.AudioTransformFrameRequest
resps chan<- *pb.AudioTransformFrameResponse
}
func (e *embedBackendAudioTransformStream) Send(resp *pb.AudioTransformFrameResponse) error {
select {
case e.resps <- resp:
return nil
case <-e.ctx.Done():
return e.ctx.Err()
}
}
func (e *embedBackendAudioTransformStream) Recv() (*pb.AudioTransformFrameRequest, error) {
select {
case req, ok := <-e.reqs:
if !ok {
return nil, io.EOF
}
return req, nil
case <-e.ctx.Done():
return nil, e.ctx.Err()
}
}
func (e *embedBackendAudioTransformStream) SetHeader(md metadata.MD) error { return nil }
func (e *embedBackendAudioTransformStream) SendHeader(md metadata.MD) error { return nil }
func (e *embedBackendAudioTransformStream) SetTrailer(md metadata.MD) {}
func (e *embedBackendAudioTransformStream) Context() context.Context { return e.ctx }
func (e *embedBackendAudioTransformStream) SendMsg(m any) error {
if x, ok := m.(*pb.AudioTransformFrameResponse); ok {
return e.Send(x)
}
return nil
}
func (e *embedBackendAudioTransformStream) RecvMsg(m any) error {
// gRPC bidi streaming uses Recv() directly; RecvMsg is unused on this path.
return nil
}
// embedBackendAudioTransformStreamClient is the caller-facing side. It
// mirrors the server-side stream over the same channels.
type embedBackendAudioTransformStreamClient struct {
ctx context.Context
reqs chan<- *pb.AudioTransformFrameRequest
resps <-chan *pb.AudioTransformFrameResponse
srvDone <-chan error
closeOnce bool
}
func (e *embedBackendAudioTransformStreamClient) Send(req *pb.AudioTransformFrameRequest) error {
select {
case e.reqs <- req:
return nil
case <-e.ctx.Done():
return e.ctx.Err()
}
}
func (e *embedBackendAudioTransformStreamClient) Recv() (*pb.AudioTransformFrameResponse, error) {
select {
case resp, ok := <-e.resps:
if !ok {
// Server-side finished. Surface its terminal error if any.
select {
case err := <-e.srvDone:
if err != nil {
return nil, err
}
default:
}
return nil, io.EOF
}
return resp, nil
case <-e.ctx.Done():
return nil, e.ctx.Err()
}
}
func (e *embedBackendAudioTransformStreamClient) CloseSend() error {
if e.closeOnce {
return nil
}
e.closeOnce = true
close(e.reqs)
return nil
}
func (e *embedBackendAudioTransformStreamClient) Context() context.Context { return e.ctx }
var _ pb.Backend_AudioTranscriptionStreamServer = new(embedBackendAudioTranscriptionStream)
type embedBackendAudioTranscriptionStream struct {

3
pkg/grpc/interface.go
View File

@@ -40,6 +40,9 @@ type AIModel interface {
AudioEncode(*pb.AudioEncodeRequest) (*pb.AudioEncodeResult, error)
AudioDecode(*pb.AudioDecodeRequest) (*pb.AudioDecodeResult, error)
AudioTransform(*pb.AudioTransformRequest) (*pb.AudioTransformResult, error)
AudioTransformStream(in <-chan *pb.AudioTransformFrameRequest, out chan<- *pb.AudioTransformFrameResponse) error
ModelMetadata(*pb.ModelOptions) (*pb.ModelMetadataResponse, error)
// Fine-tuning

76
pkg/grpc/server.go
View File

@@ -3,7 +3,9 @@ package grpc
import (
"context"
"crypto/subtle"
"errors"
"fmt"
"io"
"log"
"net"
"os"
@@ -399,6 +401,80 @@ func (s *server) AudioDecode(ctx context.Context, in *pb.AudioDecodeRequest) (*p
return res, nil
}
func (s *server) AudioTransform(ctx context.Context, in *pb.AudioTransformRequest) (*pb.AudioTransformResult, error) {
if s.llm.Locking() {
s.llm.Lock()
defer s.llm.Unlock()
}
res, err := s.llm.AudioTransform(in)
if err != nil {
return nil, err
}
return res, nil
}
func (s *server) AudioTransformStream(stream pb.Backend_AudioTransformStreamServer) error {
if s.llm.Locking() {
s.llm.Lock()
defer s.llm.Unlock()
}
in := make(chan *pb.AudioTransformFrameRequest, 4)
out := make(chan *pb.AudioTransformFrameResponse, 4)
// Pump incoming frames from the gRPC stream into `in`. EOF closes the
// channel, which signals the backend that the client is done sending.
recvErrCh := make(chan error, 1)
go func() {
defer close(in)
for {
req, err := stream.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
recvErrCh <- nil
return
}
recvErrCh <- err
return
}
select {
case in <- req:
case <-stream.Context().Done():
recvErrCh <- stream.Context().Err()
return
}
}
}()
// Pump outgoing frames from `out` to the gRPC stream. The backend closes
// `out` on completion.
sendDone := make(chan error, 1)
go func() {
for resp := range out {
if err := stream.Send(resp); err != nil {
sendDone <- err
// Drain `out` so the backend can finish.
for range out {
}
return
}
}
sendDone <- nil
}()
backendErr := s.llm.AudioTransformStream(in, out)
sendErr := <-sendDone
recvErr := <-recvErrCh
if backendErr != nil {
return backendErr
}
if sendErr != nil {
return sendErr
}
return recvErr
}
func (s *server) StartFineTune(ctx context.Context, in *pb.FineTuneRequest) (*pb.FineTuneJobResult, error) {
if s.llm.Locking() {
s.llm.Lock()

91
tests/e2e-backends/backend_test.go
View File

@@ -102,6 +102,7 @@ const (
capVoiceEmbed = "voice_embed"
capVoiceVerify = "voice_verify"
capVoiceAnalyze = "voice_analyze"
capAudioTransform = "audio_transform"
capLogprobs = "logprobs"
capLogitBias = "logit_bias"
@@ -1033,6 +1034,96 @@ var _ = Describe("Backend container", Ordered, func() {
"streamed audio too short: %d bytes", totalBytes)
GinkgoWriter.Printf("TTSStream: %d chunks, %d bytes\n", chunks, totalBytes)
})
It("transforms audio via AudioTransform (batch)", func() {
if !caps[capAudioTransform] {
Skip("audio_transform capability not enabled")
}
// Need an audio fixture — reuse the transcription audio knob.
Expect(audioFile).NotTo(BeEmpty(),
"BACKEND_TEST_AUDIO_FILE or BACKEND_TEST_AUDIO_URL must be set when audio_transform cap is enabled")
dst := filepath.Join(workDir, "transformed.wav")
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
res, err := client.AudioTransform(ctx, &pb.AudioTransformRequest{
AudioPath: audioFile,
Dst: dst,
})
Expect(err).NotTo(HaveOccurred())
Expect(res).NotTo(BeNil())
Expect(res.SampleRate).To(BeNumerically(">", int32(0)),
"AudioTransform did not report a sample rate")
Expect(res.Samples).To(BeNumerically(">", int32(0)),
"AudioTransform did not report any output samples")
Expect(res.ReferenceProvided).To(BeFalse())
info, err := os.Stat(dst)
Expect(err).NotTo(HaveOccurred(), "AudioTransform did not write a file at %s", dst)
Expect(info.Size()).To(BeNumerically(">", int64(1024)),
"AudioTransform output too small: %d bytes", info.Size())
GinkgoWriter.Printf("AudioTransform: wrote %s (%d bytes, sr=%d, samples=%d)\n",
dst, info.Size(), res.SampleRate, res.Samples)
})
It("streams audio via AudioTransformStream (bidi)", func() {
if !caps[capAudioTransform] {
Skip("audio_transform capability not enabled")
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
stream, err := client.AudioTransformStream(ctx)
Expect(err).NotTo(HaveOccurred())
// First message: Config. Pick the most permissive defaults so the
// test works against any audio-transform backend (LocalVQE wants
// 16 kHz / 256-sample / s16; other backends may default differently).
err = stream.Send(&pb.AudioTransformFrameRequest{
Payload: &pb.AudioTransformFrameRequest_Config{
Config: &pb.AudioTransformStreamConfig{
SampleFormat: pb.AudioTransformStreamConfig_S16_LE,
},
},
})
Expect(err).NotTo(HaveOccurred())
// Send a handful of synthetic silent frames — 256 mono s16 samples
// each — and assert the backend echoes a frame back per input.
const (
frameSamples = 256
sampleSize = 2 // s16
nFrames = 5
)
silentFrame := make([]byte, frameSamples*sampleSize)
for i := 0; i < nFrames; i++ {
err = stream.Send(&pb.AudioTransformFrameRequest{
Payload: &pb.AudioTransformFrameRequest_Frame{
Frame: &pb.AudioTransformFrame{AudioPcm: silentFrame},
},
})
Expect(err).NotTo(HaveOccurred(),
"sending frame %d failed", i)
}
Expect(stream.CloseSend()).To(Succeed())
var rxFrames int
var rxBytes int
for {
resp, err := stream.Recv()
if err == io.EOF {
break
}
Expect(err).NotTo(HaveOccurred())
if pcm := resp.GetPcm(); len(pcm) > 0 {
rxFrames++
rxBytes += len(pcm)
}
}
Expect(rxFrames).To(BeNumerically(">=", nFrames),
"AudioTransformStream returned %d frames for %d sent", rxFrames, nFrames)
GinkgoWriter.Printf("AudioTransformStream: rx %d frames, %d bytes\n", rxFrames, rxBytes)
})
})
// extractImage runs `docker create` + `docker export` to materialise the image