fix(whisper-medusa): pin available portable Python

The standalone Python release used by libbackend does not publish a 3.11.18 artifact. Pin Whisper-Medusa to the available 3.11.13 build and cover the generated download URL.

Assisted-by: Codex:gpt-5 [systematic-debugging]

.github/backend-matrix.yml

@@ -166,6 +166,19 @@ include:
     dockerfile: "./backend/Dockerfile.python"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: ''
+    cuda-major-version: ""
+    cuda-minor-version: ""
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-cpu-whisper-medusa'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'true'
+    backend: "whisper-medusa"
+    dockerfile: "./backend/Dockerfile.python"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: ''
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -626,6 +639,19 @@ include:
     dockerfile: "./backend/Dockerfile.python"
     context: "./"
     ubuntu-version: '2404'
+  - build-type: 'cublas'
+    cuda-major-version: "12"
+    cuda-minor-version: "1"
+    platforms: 'linux/amd64'
+    tag-latest: 'auto'
+    tag-suffix: '-gpu-nvidia-cuda-12-whisper-medusa'
+    runs-on: 'ubuntu-latest'
+    base-image: "ubuntu:24.04"
+    skip-drivers: 'false'
+    backend: "whisper-medusa"
+    dockerfile: "./backend/Dockerfile.python"
+    context: "./"
+    ubuntu-version: '2404'
   - build-type: 'cublas'
     cuda-major-version: "12"
     cuda-minor-version: "8"

.github/workflows/bump_deps.yaml

@@ -110,10 +110,14 @@ jobs:
             variable: "LOCATEANYTHING_VERSION"
             branch: "master"
             file: "backend/go/locate-anything-cpp/Makefile"
-          - repository: "ServeurpersoCom/qwentts.cpp"
-            variable: "QWEN3TTS_CPP_VERSION"
-            branch: "master"
-            file: "backend/go/qwen3-tts-cpp/Makefile"
+          # qwentts.cpp is held, not tracked: upstream master hangs in synthesis
+          # (see the comment on QWEN3TTS_CPP_VERSION in the backend Makefile).
+          # Leaving it here would re-bump the pin back onto the hang every night.
+          # Restore this entry once the upstream fix lands.
+          # - repository: "ServeurpersoCom/qwentts.cpp"
+          #   variable: "QWEN3TTS_CPP_VERSION"
+          #   branch: "master"
+          #   file: "backend/go/qwen3-tts-cpp/Makefile"
           - repository: "ServeurpersoCom/omnivoice.cpp"
             variable: "OMNIVOICE_VERSION"
             branch: "master"

Makefile

@@ -1,5 +1,5 @@
 # Disable parallel execution for backend builds
-.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/bonsai backends/outetts backends/piper backends/stablediffusion-ggml backends/trellis2cpp backends/trellis2cpp-darwin backends/whisper backends/crispasr backends/parakeet-cpp backends/moss-transcribe-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/valkey-store backends/cloud-proxy backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/longcat-video backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/moss-tts-cpp backends/magpie-tts-cpp backends/vllm-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin backends/audio-cpp backends/audio-cpp-darwin
+.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/bonsai backends/outetts backends/piper backends/stablediffusion-ggml backends/trellis2cpp backends/trellis2cpp-darwin backends/whisper backends/crispasr backends/parakeet-cpp backends/moss-transcribe-cpp backends/faster-whisper backends/whisper-medusa backends/silero-vad backends/local-store backends/valkey-store backends/cloud-proxy backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/longcat-video backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/moss-tts-cpp backends/magpie-tts-cpp backends/vllm-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin backends/audio-cpp backends/audio-cpp-darwin
 
 GOCMD=go
 GOTEST=$(GOCMD) test
@@ -611,6 +611,7 @@ prepare-test-extra: protogen-python
 	$(MAKE) -C backend/python/nemo
 	$(MAKE) -C backend/python/voxcpm
 	$(MAKE) -C backend/python/faster-whisper
+	$(MAKE) -C backend/python/whisper-medusa
 	$(MAKE) -C backend/python/whisperx
 	$(MAKE) -C backend/python/ace-step
 	$(MAKE) -C backend/python/trl
@@ -642,6 +643,7 @@ test-extra: prepare-test-extra
 	$(MAKE) -C backend/python/nemo test
 	$(MAKE) -C backend/python/voxcpm test
 	$(MAKE) -C backend/python/faster-whisper test
+	$(MAKE) -C backend/python/whisper-medusa test
 	$(MAKE) -C backend/python/whisperx test
 	$(MAKE) -C backend/python/ace-step test
 	$(MAKE) -C backend/python/trl test
@@ -1317,6 +1319,7 @@ BACKEND_RERANKERS = rerankers|python|.|false|true
 BACKEND_TRANSFORMERS = transformers|python|.|false|true
 BACKEND_OUTETTS = outetts|python|.|false|true
 BACKEND_FASTER_WHISPER = faster-whisper|python|.|false|true
+BACKEND_WHISPER_MEDUSA = whisper-medusa|python|.|false|true
 BACKEND_COQUI = coqui|python|.|false|true
 BACKEND_RFDETR = rfdetr|python|.|false|true
 BACKEND_INSIGHTFACE = insightface|python|.|false|true
@@ -1407,6 +1410,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_RERANKERS)))
 $(eval $(call generate-docker-build-target,$(BACKEND_TRANSFORMERS)))
 $(eval $(call generate-docker-build-target,$(BACKEND_OUTETTS)))
 $(eval $(call generate-docker-build-target,$(BACKEND_FASTER_WHISPER)))
+$(eval $(call generate-docker-build-target,$(BACKEND_WHISPER_MEDUSA)))
 $(eval $(call generate-docker-build-target,$(BACKEND_COQUI)))
 $(eval $(call generate-docker-build-target,$(BACKEND_RFDETR)))
 $(eval $(call generate-docker-build-target,$(BACKEND_INSIGHTFACE)))
@@ -1456,7 +1460,7 @@ $(eval $(call generate-docker-build-target,$(BACKEND_SUPERTONIC)))
 docker-save-%: backend-images
 	docker save local-ai-backend:$* -o backend-images/$*.tar
 
-docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-bonsai docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-longcat-video docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-moss-tts-cpp docker-build-magpie-tts-cpp docker-build-vllm-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-moss-transcribe-cpp docker-build-privacy-filter docker-build-trellis2cpp docker-build-valkey-store docker-build-audio-cpp
+docker-build-backends: docker-build-llama-cpp docker-build-ik-llama-cpp docker-build-turboquant docker-build-bonsai docker-build-ds4 docker-build-rerankers docker-build-vllm docker-build-vllm-omni docker-build-longcat-video docker-build-sglang docker-build-transformers docker-build-outetts docker-build-diffusers docker-build-kokoro docker-build-faster-whisper docker-build-whisper-medusa docker-build-crispasr docker-build-coqui docker-build-chatterbox docker-build-vibevoice docker-build-liquid-audio docker-build-moonshine docker-build-pocket-tts docker-build-qwen-tts docker-build-fish-speech docker-build-faster-qwen3-tts docker-build-qwen-asr docker-build-nemo docker-build-voxcpm docker-build-whisperx docker-build-ace-step docker-build-acestep-cpp docker-build-voxtral docker-build-mlx-distributed docker-build-trl docker-build-llama-cpp-quantization docker-build-tinygrad docker-build-kokoros docker-build-sam3-cpp docker-build-rfdetr-cpp docker-build-qwen3-tts-cpp docker-build-moss-tts-cpp docker-build-magpie-tts-cpp docker-build-vllm-cpp docker-build-omnivoice-cpp docker-build-vibevoice-cpp docker-build-localvqe docker-build-insightface docker-build-speaker-recognition docker-build-sherpa-onnx docker-build-cloud-proxy docker-build-supertonic docker-build-depth-anything-cpp docker-build-moss-transcribe-cpp docker-build-privacy-filter docker-build-trellis2cpp docker-build-valkey-store docker-build-audio-cpp
 
 ########################################################
 ### Mock Backend for E2E Tests

backend/cpp/audio-cpp/Makefile

@@ -9,7 +9,7 @@
 # recipe is a make target (not a prepare.sh) so 'make purge && make' is a clean
 # rebuild and so the bump bot can see the pin.
 
-AUDIO_CPP_VERSION?=f78227c52736a4792a50aa3f82ead7e7385c891b
+AUDIO_CPP_VERSION?=545e29a6f2fde24298cb3b0f07baab4352987ac9
 AUDIO_CPP_REPO?=https://github.com/0xShug0/audio.cpp
 
 CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))

backend/cpp/ik-llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-IK_LLAMA_VERSION?=3f53a059024039358e9fef75b5dc0c99dbcb40f9
+IK_LLAMA_VERSION?=0be97a7a5ad113f33e08729261649ccea2cdc5ff
 LLAMA_REPO?=https://github.com/ikawrakow/ik_llama.cpp
 
 CMAKE_ARGS?=

backend/go/crispasr/Makefile

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
 
 # CrispASR version (release tag)
 CRISPASR_REPO?=https://github.com/CrispStrobe/CrispASR
-CRISPASR_VERSION?=b5211ac635489049ee8ce86a82d69faa18e8d8da
+CRISPASR_VERSION?=66ac7843e319b588f5410051c575affd19424fb3
 SO_TARGET?=libgocrispasr.so
 
 CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

backend/go/qwen3-tts-cpp/Makefile

@@ -7,8 +7,18 @@ GO_TAGS?=
 JOBS?=$(shell nproc --ignore=1)
 
 # qwentts.cpp version
+#
+# Held at 35ebe537 rather than tracking latest: abab6b3 hangs in synthesis.
+# TTS() never returns from the native call, so tests-qwen3-tts-cpp goes from
+# ~5 minutes to the 20 minute Go test timeout. Reproduced on master on
+# 2026-08-01 and again on re-run, and the bump PR (#11241) was merged with
+# this same check already red.
+#
+# The regression is in 35ebe537..abab6b3, three upstream commits whose only
+# functional change is 26dd8adb, "predictor: unroll the frame into one cgraph
+# and sample in standard ops". Restore the bump once that is fixed upstream.
 QWEN3TTS_REPO?=https://github.com/ServeurpersoCom/qwentts.cpp
-QWEN3TTS_CPP_VERSION?=abab6b3bf317cfa1b788efce1d25f4f9239395ad
+QWEN3TTS_CPP_VERSION?=35ebe5376b82a0a59d008586d55bbe623d449011
 SO_TARGET?=libgoqwen3ttscpp.so
 
 CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

backend/index.yaml

@@ -1457,6 +1457,25 @@
     nvidia-cuda-12: "cuda12-faster-whisper"
     nvidia-l4t: "nvidia-l4t-arm64-faster-whisper"
     nvidia-l4t-cuda-12: "nvidia-l4t-arm64-faster-whisper"
+- &whisper-medusa
+  description: |
+    Whisper-Medusa accelerates Whisper transcription by predicting multiple tokens per decoding step.
+    The upstream checkpoints use custom Transformers generation code and accept audio clips up to 30 seconds.
+  urls:
+    - https://github.com/aiola-lab/whisper-medusa
+    - https://huggingface.co/collections/aiola/whisper-medusa
+  tags:
+    - speech-to-text
+    - transcription
+    - Whisper
+    - Medusa
+  license: MIT
+  name: "whisper-medusa"
+  alias: "whisper-medusa"
+  capabilities:
+    default: "cpu-whisper-medusa"
+    nvidia: "cuda12-whisper-medusa"
+    nvidia-cuda-12: "cuda12-whisper-medusa"
 - &moonshine
   description: |
     Moonshine is a fast, accurate, and efficient speech-to-text transcription model using ONNX Runtime.
@@ -6214,6 +6233,34 @@
   uri: "quay.io/go-skynet/local-ai-backends:master-nvidia-l4t-cuda-13-arm64-faster-qwen3-tts"
   mirrors:
     - localai/localai-backends:master-nvidia-l4t-cuda-13-arm64-faster-qwen3-tts
+## whisper-medusa
+- !!merge <<: *whisper-medusa
+  name: "whisper-medusa-development"
+  capabilities:
+    default: "cpu-whisper-medusa-development"
+    nvidia: "cuda12-whisper-medusa-development"
+    nvidia-cuda-12: "cuda12-whisper-medusa-development"
+- !!merge <<: *whisper-medusa
+  name: "cpu-whisper-medusa"
+  uri: "quay.io/go-skynet/local-ai-backends:latest-cpu-whisper-medusa"
+  mirrors:
+    - localai/localai-backends:latest-cpu-whisper-medusa
+- !!merge <<: *whisper-medusa
+  name: "cpu-whisper-medusa-development"
+  uri: "quay.io/go-skynet/local-ai-backends:master-cpu-whisper-medusa"
+  mirrors:
+    - localai/localai-backends:master-cpu-whisper-medusa
+- !!merge <<: *whisper-medusa
+  name: "cuda12-whisper-medusa"
+  uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-whisper-medusa"
+  mirrors:
+    - localai/localai-backends:latest-gpu-nvidia-cuda-12-whisper-medusa
+- !!merge <<: *whisper-medusa
+  name: "cuda12-whisper-medusa-development"
+  uri: "quay.io/go-skynet/local-ai-backends:master-gpu-nvidia-cuda-12-whisper-medusa"
+  mirrors:
+    - localai/localai-backends:master-gpu-nvidia-cuda-12-whisper-medusa
+
 ## qwen-asr
 - !!merge <<: *qwen-asr
   name: "qwen-asr-development"

backend/python/whisper-medusa/Makefile

@@ -0,0 +1,14 @@
+.DEFAULT_GOAL := install
+
+.PHONY: install
+install:
+	bash install.sh
+
+.PHONY: clean
+clean:
+	$(RM) backend_pb2_grpc.py backend_pb2.py
+	rm -rf venv __pycache__
+
+.PHONY: test
+test:
+	python3 -m unittest test_unit.py

backend/python/whisper-medusa/backend.py

@@ -0,0 +1,167 @@
+#!/usr/bin/env python3
+"""LocalAI gRPC backend for aiola Whisper-Medusa speech recognition."""
+
+import argparse
+from concurrent import futures
+import os
+import signal
+import sys
+import time
+
+import backend_pb2
+import backend_pb2_grpc
+import grpc
+
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "common"))
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "common"))
+from grpc_auth import get_auth_interceptors
+from model_utils import resolve_model_reference
+
+
+SAMPLE_RATE = 16000
+MAX_DURATION_SECONDS = 30
+MAX_WORKERS = int(os.environ.get("PYTHON_GRPC_MAX_WORKERS", "1"))
+_ONE_DAY_IN_SECONDS = 60 * 60 * 24
+
+
+def _parse_options(raw_options):
+    options = {}
+    for option in raw_options:
+        if ":" not in option:
+            continue
+        key, value = option.split(":", 1)
+        try:
+            value = int(value)
+        except ValueError:
+            try:
+                value = float(value)
+            except ValueError:
+                pass
+        options[key] = value
+    return options
+
+
+def _prepare_audio(path, torchaudio):
+    waveform, sample_rate = torchaudio.load(path)
+    if waveform.shape[0] > 1:
+        waveform = waveform.mean(dim=0, keepdim=True)
+    if sample_rate != SAMPLE_RATE:
+        waveform = torchaudio.transforms.Resample(sample_rate, SAMPLE_RATE)(waveform)
+        sample_rate = SAMPLE_RATE
+    return waveform, sample_rate
+
+
+class BackendServicer(backend_pb2_grpc.BackendServicer):
+    def __init__(self):
+        self.model = None
+        self.processor = None
+        self.device = None
+        self.options = {}
+
+    def Health(self, request, context):
+        return backend_pb2.Reply(message=b"OK")
+
+    def LoadModel(self, request, context):
+        try:
+            import torch
+            from transformers import WhisperProcessor
+            from whisper_medusa import WhisperMedusaModel
+
+            if request.CUDA and not torch.cuda.is_available():
+                return backend_pb2.Result(success=False, message="CUDA is not available")
+
+            self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+            if hasattr(torch.backends, "mps") and torch.backends.mps.is_available():
+                self.device = torch.device("mps")
+
+            self.options = _parse_options(request.Options)
+            model_path, local_only = resolve_model_reference(
+                request, "aiola/whisper-medusa-linear-libri"
+            )
+            self.model = WhisperMedusaModel.from_pretrained(
+                model_path, local_files_only=local_only
+            ).to(self.device)
+            self.model.eval()
+            self.processor = WhisperProcessor.from_pretrained(
+                model_path, local_files_only=local_only
+            )
+        except Exception as err:
+            print(f"Whisper-Medusa model load failed: {err}", file=sys.stderr)
+            return backend_pb2.Result(success=False, message=str(err))
+
+        return backend_pb2.Result(success=True, message="Model loaded successfully")
+
+    def AudioTranscription(self, request, context):
+        if self.model is None or self.processor is None:
+            return backend_pb2.TranscriptResult(segments=[], text="")
+
+        try:
+            import torch
+            import torchaudio
+
+            waveform, sample_rate = _prepare_audio(request.dst, torchaudio)
+            duration = waveform.shape[-1] / sample_rate
+            if duration > MAX_DURATION_SECONDS:
+                raise ValueError(
+                    f"Whisper-Medusa supports audio clips up to {MAX_DURATION_SECONDS} seconds"
+                )
+
+            language = request.language or str(self.options.get("language", "en"))
+            regulation_start = int(self.options.get("regulation_start", 140))
+            regulation_factor = float(self.options.get("regulation_factor", 1.01))
+            features = self.processor(
+                waveform.squeeze(), return_tensors="pt", sampling_rate=sample_rate
+            ).input_features.to(self.device)
+            with torch.inference_mode():
+                output = self.model.generate(
+                    features,
+                    language=language,
+                    exponential_decay_length_penalty=(
+                        regulation_start,
+                        regulation_factor,
+                    ),
+                )
+            text = self.processor.decode(output[0], skip_special_tokens=True).strip()
+            segment = backend_pb2.TranscriptSegment(
+                id=0,
+                start=0,
+                end=int(duration * 1_000_000_000),
+                text=text,
+            )
+            return backend_pb2.TranscriptResult(segments=[segment], text=text)
+        except Exception as err:
+            print(f"Whisper-Medusa transcription failed: {err}", file=sys.stderr)
+            return backend_pb2.TranscriptResult(segments=[], text="")
+
+
+def serve(address):
+    server = grpc.server(
+        futures.ThreadPoolExecutor(max_workers=MAX_WORKERS),
+        options=[
+            ("grpc.max_send_message_length", 50 * 1024 * 1024),
+            ("grpc.max_receive_message_length", 50 * 1024 * 1024),
+        ],
+        interceptors=get_auth_interceptors(),
+    )
+    backend_pb2_grpc.add_BackendServicer_to_server(BackendServicer(), server)
+    server.add_insecure_port(address)
+    server.start()
+    print(f"Server started. Listening on: {address}", file=sys.stderr)
+
+    def stop_server(_signal, _frame):
+        server.stop(0)
+        sys.exit(0)
+
+    signal.signal(signal.SIGINT, stop_server)
+    signal.signal(signal.SIGTERM, stop_server)
+    try:
+        while True:
+            time.sleep(_ONE_DAY_IN_SECONDS)
+    except KeyboardInterrupt:
+        server.stop(0)
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Run the Whisper-Medusa backend")
+    parser.add_argument("--addr", default="localhost:50051")
+    serve(parser.parse_args().addr)

backend/python/whisper-medusa/install.sh

@@ -0,0 +1,13 @@
+#!/bin/bash
+set -e
+
+backend_dir=$(dirname "$0")
+if [ -d "$backend_dir/common" ]; then
+    source "$backend_dir/common/libbackend.sh"
+else
+    source "$backend_dir/../common/libbackend.sh"
+fi
+
+PYTHON_VERSION="3.11"
+PYTHON_PATCH="13"
+installRequirements

backend/python/whisper-medusa/protogen.sh

@@ -0,0 +1,3 @@
+#!/bin/bash
+set -e
+python3 -m grpc_tools.protoc -I../../ --python_out=. --grpc_python_out=. ../../backend.proto

backend/python/whisper-medusa/requirements-cpu.txt

@@ -0,0 +1,3 @@
+--extra-index-url https://download.pytorch.org/whl/cpu
+torch==2.2.2
+torchaudio==2.2.2

backend/python/whisper-medusa/requirements-cublas12.txt

@@ -0,0 +1,3 @@
+--extra-index-url https://download.pytorch.org/whl/cu121
+torch==2.2.2
+torchaudio==2.2.2

backend/python/whisper-medusa/requirements.txt

@@ -0,0 +1,5 @@
+grpcio==1.71.0
+protobuf
+grpcio-tools
+transformers==4.49.0
+git+https://github.com/aiola-lab/whisper-medusa.git@19819c37ab15db6e68826e406614a2c86fbb946e

backend/python/whisper-medusa/run.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+backend_dir=$(dirname "$0")
+if [ -d "$backend_dir/common" ]; then
+    source "$backend_dir/common/libbackend.sh"
+else
+    source "$backend_dir/../common/libbackend.sh"
+fi
+
+startBackend "$@"

backend/python/whisper-medusa/test.sh

@@ -0,0 +1,26 @@
+#!/bin/bash
+set -e
+python3 -m unittest test_unit.py
+
+tmp_dir=$(mktemp -d)
+trap 'rm -rf "$tmp_dir"' EXIT
+mkdir -p "$tmp_dir/bin"
+cat >"$tmp_dir/bin/curl" <<'EOF'
+#!/bin/bash
+printf '%s\n' "${@: -1}" >"$PORTABLE_PY_URL_CAPTURE"
+exit 22
+EOF
+chmod +x "$tmp_dir/bin/curl"
+
+PORTABLE_PY_URL_CAPTURE="$tmp_dir/url" \
+    ENV_DIR="$tmp_dir/backend" \
+    PORTABLE_PYTHON=true \
+    PATH="$tmp_dir/bin:$PATH" \
+    bash install.sh >/dev/null 2>&1 || true
+
+expected_url="https://github.com/astral-sh/python-build-standalone/releases/download/20250818/cpython-3.11.13+20250818-x86_64-unknown-linux-gnu-install_only.tar.gz"
+actual_url=$(cat "$tmp_dir/url")
+if [ "$actual_url" != "$expected_url" ]; then
+    echo "unexpected portable Python URL: $actual_url" >&2
+    exit 1
+fi

backend/python/whisper-medusa/test_unit.py

@@ -0,0 +1,89 @@
+import importlib.util
+import pathlib
+import sys
+import types
+import unittest
+
+
+class _Message:
+    def __init__(self, **kwargs):
+        self.__dict__.update(kwargs)
+
+
+backend_pb2 = types.ModuleType("backend_pb2")
+for name in ("Reply", "Result", "TranscriptResult", "TranscriptSegment"):
+    setattr(backend_pb2, name, _Message)
+sys.modules["backend_pb2"] = backend_pb2
+
+backend_pb2_grpc = types.ModuleType("backend_pb2_grpc")
+backend_pb2_grpc.BackendServicer = object
+backend_pb2_grpc.add_BackendServicer_to_server = lambda *args: None
+sys.modules["backend_pb2_grpc"] = backend_pb2_grpc
+
+grpc = types.ModuleType("grpc")
+grpc.server = lambda *args, **kwargs: None
+sys.modules["grpc"] = grpc
+
+grpc_auth = types.ModuleType("grpc_auth")
+grpc_auth.get_auth_interceptors = lambda: []
+sys.modules["grpc_auth"] = grpc_auth
+
+model_utils = types.ModuleType("model_utils")
+model_utils.resolve_model_reference = lambda request, default: (request.Model or default, False)
+sys.modules["model_utils"] = model_utils
+
+spec = importlib.util.spec_from_file_location(
+    "whisper_medusa_backend", pathlib.Path(__file__).with_name("backend.py")
+)
+backend = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(backend)
+
+
+class FakeTensor:
+    def __init__(self, channels=1, samples=16000):
+        self.channels = channels
+        self.samples = samples
+        self.shape = (channels, samples)
+        self.mean_calls = []
+
+    def mean(self, dim, keepdim):
+        self.mean_calls.append((dim, keepdim))
+        return FakeTensor(1, self.samples)
+
+    def squeeze(self):
+        return self
+
+    def to(self, device):
+        return self
+
+
+class FakeTorchaudio:
+    class transforms:
+        class Resample:
+            def __init__(self, source, target):
+                self.source = source
+                self.target = target
+
+            def __call__(self, waveform):
+                return FakeTensor(waveform.channels, waveform.samples * self.target // self.source)
+
+    @staticmethod
+    def load(path):
+        return FakeTensor(2, 8000), 8000
+
+
+class BackendHelpersTest(unittest.TestCase):
+    def test_parse_options_converts_supported_scalar_types(self):
+        self.assertEqual(
+            backend._parse_options(["regulation_start:120", "regulation_factor:1.05", "ignored"]),
+            {"regulation_start": 120, "regulation_factor": 1.05},
+        )
+
+    def test_prepare_audio_mixes_to_mono_and_resamples_to_16khz(self):
+        waveform, sample_rate = backend._prepare_audio("clip.wav", FakeTorchaudio)
+        self.assertEqual(sample_rate, 16000)
+        self.assertEqual(waveform.shape, (1, 16000))
+
+
+if __name__ == "__main__":
+    unittest.main()

docs/content/features/backends.md

@@ -130,7 +130,7 @@ For getting started, see the available backends in LocalAI here: https://github.
 LocalAI supports various types of backends:
 
 - **LLM Backends**: For running language models (e.g., llama.cpp, vLLM, vllm.cpp, SGLang, transformers, MLX)
-- **Speech-to-Text Backends**: For transcription, forced alignment and speaker diarization (e.g., whisper.cpp, parakeet.cpp, moss-transcribe.cpp, faster-whisper, NeMo, [audio.cpp]({{%relref "features/audio-cpp" %}}))
+- **Speech-to-Text Backends**: For transcription, forced alignment and speaker diarization (e.g., whisper.cpp, parakeet.cpp, moss-transcribe.cpp, faster-whisper, [Whisper-Medusa]({{%relref "features/whisper-medusa" %}}), NeMo, [audio.cpp]({{%relref "features/audio-cpp" %}}))
 - **Text-to-Speech Backends**: For speech synthesis (e.g., piper, Kokoro, VibeVoice, Qwen3-TTS, [audio.cpp]({{%relref "features/audio-cpp" %}}))
 - **Sound Generation Backends**: For music and audio generation (e.g., ACE-Step, [audio.cpp]({{%relref "features/audio-cpp" %}}))
 - **Sound Classification Backends**: For sound-event classification / audio tagging - identifying everyday sounds like baby cry, glass breaking, alarms (e.g., ced.cpp)

docs/content/features/whisper-medusa.md

@@ -0,0 +1,48 @@
+---
+title: "Whisper-Medusa"
+description: "Run Whisper-Medusa speech-to-text checkpoints with LocalAI"
+---
+
+The `whisper-medusa` backend serves aiola's Whisper-Medusa checkpoints through
+LocalAI's OpenAI-compatible audio transcription endpoint. Whisper-Medusa uses
+multiple decoding heads to predict several tokens per decoding step.
+
+## Model configuration
+
+Install the `whisper-medusa` backend from the backend gallery, then create a
+model configuration such as:
+
+```yaml
+name: whisper-medusa
+backend: whisper-medusa
+parameters:
+  model: aiola/whisper-medusa-linear-libri
+options:
+  - language:en
+  - regulation_start:140
+  - regulation_factor:1.01
+```
+
+Transcribe a clip with the standard endpoint:
+
+```bash
+curl http://localhost:8080/v1/audio/transcriptions \
+  -F file=@audio.wav \
+  -F model=whisper-medusa \
+  -F language=en
+```
+
+`language` in the request overrides the configured default. The regulation
+options control the exponential length penalty passed to upstream generation.
+
+{{% notice warning %}}
+The upstream Whisper-Medusa repository is archived. Its implementation accepts
+clips of at most 30 seconds and expects 16 kHz audio; LocalAI resamples input to
+16 kHz and rejects longer clips. The LibriSpeech checkpoints are optimized for
+English. Use `aiola/whisper-medusa-multilingual` for supported multilingual
+audio.
+{{% /notice %}}
+
+The backend currently ships Linux CPU and NVIDIA CUDA 12 images. It is not
+advertised for macOS, ROCm, Intel GPU, or Jetson because upstream pins PyTorch
+2.2.2 and has not published compatibility for those platforms.

docs/superpowers/plans/2026-08-02-whisper-medusa-backend.md

@@ -0,0 +1,63 @@
+# Whisper-Medusa Backend Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Add a dedicated LocalAI speech-to-text backend for aiola Whisper-Medusa checkpoints.
+
+**Architecture:** A Python gRPC backend owns model loading, audio normalization, and Medusa generation. LocalAI's existing `AudioTranscription` RPC remains unchanged; build, gallery, and documentation surfaces follow the existing Python ASR backend pattern.
+
+**Tech Stack:** Python 3.11, PyTorch, torchaudio, transformers, whisper-medusa, gRPC, YAML, Make.
+
+## Global Constraints
+
+- Accept local model paths and Hugging Face model identifiers through `resolve_model_reference`.
+- Normalize input audio to mono 16 kHz before generation.
+- Default the language to `en` and expose upstream generation regulation options.
+- Document upstream's archived status, 30-second clip limit, and checkpoint language limitations.
+- Build Linux CPU and NVIDIA CUDA 12 images; do not claim unsupported Darwin or ROCm coverage.
+
+---
+
+### Task 1: Backend behavior
+
+**Files:**
+- Create: `backend/python/whisper-medusa/backend.py`
+- Create: `backend/python/whisper-medusa/test_unit.py`
+
+**Interfaces:**
+- Consumes: LocalAI `LoadModel` and `AudioTranscription` protobuf requests.
+- Produces: `BackendServicer`, `_parse_options`, and `_prepare_audio`.
+
+- [ ] Write unit tests for option parsing, mono conversion, resampling, load failure, and transcription.
+- [ ] Run `python -m unittest test_unit.py` and confirm it fails because the backend does not exist.
+- [ ] Implement the minimal gRPC backend and rerun the unit tests to green.
+
+### Task 2: Packaging and registration
+
+**Files:**
+- Create: `backend/python/whisper-medusa/{Makefile,install.sh,protogen.sh,run.sh,test.sh,requirements.txt,requirements-cpu.txt,requirements-cublas12.txt}`
+- Modify: `Makefile`
+- Modify: `.github/backend-matrix.yml`
+- Modify: `backend/index.yaml`
+
+**Interfaces:**
+- Consumes: the Python backend Docker build conventions.
+- Produces: `whisper-medusa` install/build targets and CPU/CUDA backend images.
+
+- [ ] Add packaging scripts and pinned upstream dependency.
+- [ ] Register the backend in Make and backend metadata.
+- [ ] Add Linux amd64 CPU and CUDA 12 CI matrix entries.
+- [ ] Validate Make and YAML parsing.
+
+### Task 3: Documentation and verification
+
+**Files:**
+- Create: `docs/content/features/whisper-medusa.md`
+- Modify: `docs/content/features/backends.md`
+
+**Interfaces:**
+- Produces: user-facing model YAML and limitation guidance.
+
+- [ ] Document setup, configuration, options, and upstream constraints.
+- [ ] Run unit tests, syntax compilation, registration checks, and diff review.
+- [ ] Commit with the required `Assisted-by` trailer, push, and open a PR closing issue #3127.

gallery/index.yaml

@@ -3,25 +3,7 @@
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
     - https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF
-  description: |
-    # DeepSeek-V4-Flash-0731
-
-    Technical Report👁️
-
-    ## Introduction
-
-    **DeepSeek-V4-Flash-0731** is the official release of **DeepSeek-V4-Flash**, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.
-
-    DeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available.
-
-    Notes:
-
-    1. For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the `max` reasoning effort level with `temperature = 1.0, top_p = 0.95`.
-    2. † DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems.
-
-    ## Chat Template
-
-    ...
+  description: "# DeepSeek-V4-Flash-0731\n\nTechnical Report\U0001F441️\n\n## Introduction\n\n**DeepSeek-V4-Flash-0731** is the official release of **DeepSeek-V4-Flash**, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.\n\nDeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available.\n\nNotes:\n\n1. For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the `max` reasoning effort level with `temperature = 1.0, top_p = 0.95`.\n2. † DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems.\n\n## Chat Template\n\n...\n"
   license: "mit"
   tags:
     - llm
@@ -41,8 +23,8 @@
       use_tokenizer_template: true
   files:
     - filename: ds4flash.gguf
-      sha256: ""
       uri: https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF
+      sha256: 7ab5e306300dd16d237d0f5ff07ee968d02be5c47afc87282778f4ec6b0f54a7
 - name: "parable-granite-4.1-3b-claude-fable-5"
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
@@ -1605,7 +1587,7 @@
   files:
     - filename: ds4flash.gguf
       uri: https://huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF
-      sha256: 7d6d1691bc2d02c5a8194afb8bd9b57519343afebf020231c61d9236504b4a5c
+      sha256: efb3e92a73b877e69e1754212e132d90963ce9ced4f0b68f9173723fee7f85a4
 - name: "qwopus3.6-35b-a3b-coder-mtp"
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:

website/content/blog/what-landed-in-localai-4-8.md

@@ -1,14 +1,14 @@
 ---
 title: "What landed in LocalAI 4.8"
-date: 2026-07-28
+date: 2026-08-01
 author: "Ettore Di Giacinto"
 category: "Release"
-tags: ["release", "vllm.cpp", "gallery", "distributed", "performance"]
-summary: "The web interface got 3.48x lighter, gallery entries now install the build your hardware can actually run, and there is a new inference engine in the box. 214 pull requests in thirteen days."
+tags: ["release", "vllm.cpp", "audio.cpp", "3d", "gallery", "distributed", "performance"]
+summary: "A new inference engine, 3D generation, one backend that serves six audio endpoints, and a web interface 3.48x lighter. 321 pull requests in eighteen days."
 extracss: ["blog.css"]
 ---
 
-LocalAI 4.8.0 is out. It took thirteen days and 214 merged pull requests, and the changes you will notice first are the boring ones: the web interface loads faster, model installs stop asking you to pick a quantization, and a cluster no longer reports models as loaded when they are gone.
+LocalAI 4.8.0 is out. It took eighteen days and 321 merged pull requests, and it pulls in two directions at once: three new things LocalAI can do that it could not do before, and a long list of places where it now does the old things without lying to you.
 
 The full notes list everything. This post covers the parts that change what you do day to day, with the pull request numbers so you can read the diffs.
 
@@ -77,6 +77,67 @@ The CPU path is verified end to end against `Qwen3.5-2B-UD-Q8_K_XL.gguf` with th
 
 > The GPU images build and ship, but their runtime behavior has not been through the same e2e gate yet. This is a first release of a young engine: no throughput comparison against upstream vLLM is claimed here, and `llama-cpp` remains the default recommendation for general use. Try it, and please report what breaks.
 
+<figure>
+<video src="/media/vllm-race.mp4" muted loop playsinline preload="none" data-lazy aria-label="vllm.cpp generating tokens"></video>
+<figcaption>vllm.cpp serving a GGUF checkpoint with no Python in the process.</figcaption>
+</figure>
+
+## LocalAI generates 3D models now
+
+This is a new modality rather than a new backend under an existing one, so it goes through the whole stack: a `Generate3D` RPC in `backend.proto`, a `FLAG_3D` capability so the loader knows which backends can serve it, and `POST /v1/3d/generations`.
+
+The first engine behind it is `trellis2cpp`, an image-to-3D backend over TRELLIS.2. You give it an image, you get a GLB back. The web UI has a page for it with a native GLB viewer, so you can turn the result around in the browser instead of downloading it to find out whether it worked, history kept in IndexedDB so a reload does not lose your generations, and previewable print remeshing for output you actually intend to send to a printer ([#10979](https://github.com/mudler/LocalAI/pull/10979)).
+
+<figure>
+<video src="/media/3d-generation.mp4" muted loop playsinline preload="none" data-lazy aria-label="A generated 3D llama rotating in the LocalAI GLB viewer"></video>
+<figcaption>trellis2-4b, 2,502,928 vertices and 5,012,118 triangles, turning in the browser. The remesh slider below it is the print path.</figcaption>
+</figure>
+
+## One backend, six audio endpoints
+
+The usual shape for audio is one backend per model family, which means a process per capability and a config file for each. `audio-cpp` wraps [audio.cpp](https://github.com/0xShug0/audio.cpp), a multi-family ggml audio engine, and inverts that: one backend process serves several unrelated families through a single runtime vocabulary, and works out which family a checkpoint belongs to from the GGUF's own `audiocpp.model_spec.family` metadata key. There is nothing backend-specific to write in the model config.
+
+<div class="tw">
+<table>
+<thead><tr><th>Endpoint</th><th>Families</th></tr></thead>
+<tbody>
+<tr><td><code>/v1/audio/speech</code></td><td>supertonic, chatterbox (voice cloning), irodori-voicedesign</td></tr>
+<tr><td><code>/v1/audio/transcriptions</code></td><td>citrinet, nemotron, forced-aligner</td></tr>
+<tr><td><code>/v1/audio/vad</code></td><td>silero-vad, marblenet-vad</td></tr>
+<tr><td><code>/v1/audio/diarize</code></td><td>sortformer</td></tr>
+<tr><td><code>/audio/transform</code></td><td>htdemucs (4-stem separation), voice conversion, speech to speech</td></tr>
+<tr><td><code>/v1/sound-generation</code></td><td>stable-audio-sfx</td></tr>
+</tbody>
+</table>
+</div>
+
+Thirteen gallery entries ship with it, one per task kind the engine can actually serve. Where it cannot honestly back an RPC it returns `UNIMPLEMENTED` with a reason rather than an empty success, and a failed load is a gRPC error rather than `success: false`, which matters because the loader's greedy backend probe would otherwise pick it for a model it cannot serve.
+
+Two changes reach past the backend itself. `AudioTransformResult` gained a `stems` field, so source separation can return the whole stem set instead of one mixdown. And `/audio/transform` no longer folds every upload to 16 kHz mono: that fold made 4-stem separation unreachable by construction, so it became a per-backend capability, with the existing backends keeping it explicitly and the default for an unregistered backend being to leave the upload alone ([#11141](https://github.com/mudler/LocalAI/pull/11141)).
+
+## Three new backends for speech and small quants
+
+`magpie-tts-cpp` wraps [magpie-tts.cpp](https://github.com/mudler/magpie-tts.cpp), a C++17 and ggml port of NVIDIA Magpie TTS Multilingual 357M with the NanoCodec vocoder embedded. Five voices, nine or more languages, 22.05 kHz mono, out of one self-contained GGUF. The upstream engine is parity-gated against NeMo per component, with a teacher-forced replay maximum absolute difference of 3.6e-5.
+
+`moss-tts-cpp` wraps [moss-tts.cpp](https://github.com/mudler/moss-tts.cpp) and serves MOSS-TTS-Local v1.5 at 48 kHz stereo, with optional reference-audio voice cloning. Images cover CPU, CUDA 12 and 13, Intel SYCL, Vulkan, ROCm, L4T and Darwin Metal. Both landed in [#11115](https://github.com/mudler/LocalAI/pull/11115), [#10860](https://github.com/mudler/LocalAI/pull/10860) and [#10877](https://github.com/mudler/LocalAI/pull/10877).
+
+<figure>
+<video src="/media/magpie.mp4" muted loop playsinline preload="none" data-lazy aria-label="magpie-tts-cpp synthesising multilingual speech"></video>
+<figcaption>magpie-tts-cpp, five voices and nine languages out of a single GGUF.</figcaption>
+</figure>
+
+The `bonsai` backend serves the 1-bit (Q1_0) and ternary (Q2_0) Bonsai quantizations of Qwen3 8B and Qwen3.6-27B, from about 1.15 GB. Stock llama.cpp has no kernels for those formats, so the backend builds against the PrismML fork through a wrapper Makefile that swaps only `LLAMA_REPO` and `LLAMA_VERSION`, reusing the same `grpc-server.cpp` with zero skew patches. Eight gallery entries ship with it. If Q1_0 and Q2_0 reach mainline llama.cpp, this backend retires into a routine version bump ([#10834](https://github.com/mudler/LocalAI/pull/10834), [#10866](https://github.com/mudler/LocalAI/pull/10866)).
+
+## The operations bar became a page
+
+The old operations bar rendered one row per in-flight operation above every page. Queue four model installs and a backend and it took most of the viewport, on every route, until the last one finished. Two things were conflated there: a global "something is happening" signal, which needs one line, and the detail of what is happening, which needs somewhere to put it.
+
+The strip is now one line, permanently, showing a failure first and otherwise the least-advanced running operation, with a `+N more` pill. Its `✕` hides the strip and no longer cancels anything. That is a deliberate behavior change worth knowing about before you click it out of habit: the same glyph used to cancel a 17 GB download in one row and dismiss a message in the next. Cancelling moved to the new page, behind a button that says so.
+
+The Activity page at `/app/activity` is admin only and carries what the strip has to drop: phase, bytes, a derived time remaining, and a per-node breakdown for cluster installs. It also keeps a record. `/api/operations` dropped an operation the moment it succeeded, so if you started a large install and walked away there was no way to find out afterwards whether it finished, failed, or never started. That record is a bounded 50-entry ring, and dismissing a failure moves it into the record instead of deleting it.
+
+Several long-standing UI bugs fell out of the rewrite: retrying a failed removal re-downloaded the model, queued operations rendered as "Installing" with a spinner, a long error message pushed every page about 270px past the viewport and gave the whole app a horizontal scrollbar, and the ETA blanked for every operation whenever any one of them was verifying ([#11163](https://github.com/mudler/LocalAI/pull/11163)).
+
 ## VRAM budgets, per node
 
 You can now cap how much of a card LocalAI is allowed to use, as a percentage or an absolute amount ([#10833](https://github.com/mudler/LocalAI/pull/10833)):
@@ -88,14 +149,6 @@ LOCALAI_VRAM_BUDGET=12GB
 
 Everywhere LocalAI reads VRAM to make an allocation decision it now uses `min(detected, budget)`. Percentages above 100 are rejected and absolute values above physical are clamped, so the ceiling can only ever lower usable VRAM. Standalone it is a hard per-process cap that hardware defaults, context auto-fit, GGUF warnings and the watchdog all inherit. Distributed it is a placement ceiling: the worker reports raw VRAM plus its budget string, the node registry resolves it to a byte ceiling on registration and heartbeat, and the SQL scheduler needed no query change. Admin overrides through `PUT` and `DELETE /api/nodes/:id/vram-budget` survive worker restarts, and the same thing is available as the `set_node_vram_budget` MCP tool. Unset means all detected VRAM, so existing deployments do not change.
 
-## Three new backends for speech and small quants
-
-`magpie-tts-cpp` wraps [magpie-tts.cpp](https://github.com/mudler/magpie-tts.cpp), a C++17 and ggml port of NVIDIA Magpie TTS Multilingual 357M with the NanoCodec vocoder embedded. Five voices, nine or more languages, 22.05 kHz mono, out of one self-contained GGUF. The upstream engine is parity-gated against NeMo per component, with a teacher-forced replay maximum absolute difference of 3.6e-5.
-
-`moss-tts-cpp` wraps [moss-tts.cpp](https://github.com/mudler/moss-tts.cpp) and serves MOSS-TTS-Local v1.5 at 48 kHz stereo, with optional reference-audio voice cloning. Images cover CPU, CUDA 12 and 13, Intel SYCL, Vulkan, ROCm, L4T and Darwin Metal. Both landed in [#11115](https://github.com/mudler/LocalAI/pull/11115), [#10860](https://github.com/mudler/LocalAI/pull/10860) and [#10877](https://github.com/mudler/LocalAI/pull/10877).
-
-The `bonsai` backend serves the 1-bit (Q1_0) and ternary (Q2_0) Bonsai quantizations of Qwen3 8B and Qwen3.6-27B, from about 1.15 GB. Stock llama.cpp has no kernels for those formats, so the backend builds against the PrismML fork through a wrapper Makefile that swaps only `LLAMA_REPO` and `LLAMA_VERSION`, reusing the same `grpc-server.cpp` with zero skew patches. Eight gallery entries ship with it. If Q1_0 and Q2_0 reach mainline llama.cpp, this backend retires into a routine version bump ([#10834](https://github.com/mudler/LocalAI/pull/10834), [#10866](https://github.com/mudler/LocalAI/pull/10866)).
-
 ## Distributed mode stops reaping live backends
 
 A model that showed as loaded on the home page but appeared on no node in the cluster turned out to be four separate bugs, all fixed in this cycle.
@@ -110,7 +163,9 @@ Alongside those, `in_flight` counters no longer leak high and pin a replica's VR
 
 `POST /api/fine-tuning/jobs` accepted `reward_functions[].code`, an inline Python body that ran against a hand-rolled builtin allowlist. That allowlist was not a security boundary. Standard CPython introspection reaches the real `os` module from inside it, which is arbitrary code execution on the host, and the execution happened during a smoke test at job start on an endpoint that is unauthenticated by default.
 
-Inline reward code is now refused unless the operator sets `LOCALAI_TRL_ALLOW_INLINE_REWARD=true` on the backend. Builtin reward functions are unaffected and need no configuration. The documentation no longer describes the allowlist as a sandbox ([#11068](https://github.com/mudler/LocalAI/pull/11068)). This release also picks up hono 4.12.25 for CVE-2026-54290 ([#11023](https://github.com/mudler/LocalAI/pull/11023)).
+Inline reward code is now refused unless the operator sets `LOCALAI_TRL_ALLOW_INLINE_REWARD=true` on the backend. Builtin reward functions are unaffected and need no configuration. The documentation no longer describes the allowlist as a sandbox ([#11068](https://github.com/mudler/LocalAI/pull/11068)).
+
+Two more hardening fixes landed in the same cycle. Tar hardlinks that escape the extraction root are now rejected: the archive extractor pre-scanned members and rejected symlinks, but a tar hardlink entry carries a regular file mode and passed that check, and `Header.Linkname` was never validated, so an archive could link to a path outside the destination ([#11266](https://github.com/mudler/LocalAI/pull/11266)). And a cyclic `$ref` in a JSON-schema grammar is now rejected rather than recursing into a stack overflow ([#11041](https://github.com/mudler/LocalAI/pull/11041)). This release also picks up hono 4.12.25 for CVE-2026-54290 ([#11023](https://github.com/mudler/LocalAI/pull/11023)).
 
 ## The rest, briefly
 
@@ -120,6 +175,10 @@ The traces panel gained a sortable User column, plus client IP and user agent in
 
 Documentation got an onboarding overhaul driven by a per-page audit: one model, `qwen3-4b`, now carries through install, the web UI and a curl call; there is a new walkthrough for building your first agent; and a new runtime-errors reference is keyed on the literal error strings users actually see.
 
-Fifteen people contributed to this release, five of them for the first time. The gallery went from 1,221 entries to 1,476.
+Valkey Search joins the vector store options as the `valkey-store` backend ([#11196](https://github.com/mudler/LocalAI/pull/11196)). `local-ai` can be started on demand through systemd socket activation, taking a listener it inherits rather than binding one, with ordinary `--address` behavior unchanged when no activation listener is present ([#11169](https://github.com/mudler/LocalAI/pull/11169)). Trace history now survives a restart, stored as bounded per-record JSON under the data path with no database dependency ([#11203](https://github.com/mudler/LocalAI/pull/11203)). You can edit saved chat messages in place without triggering a new inference request ([#11189](https://github.com/mudler/LocalAI/pull/11189)). And the Intel SYCL llama.cpp backend is now self-contained, so it runs without a matching oneAPI runtime on the host ([#10991](https://github.com/mudler/LocalAI/pull/10991)).
 
-To upgrade, pull `localai/localai:latest` or re-run the install script. The [full changelog](https://github.com/mudler/LocalAI/compare/v4.7.1...v4.8.0) has the other 180 pull requests.
+This is also the release where localai.io split in two: the project site at the root, and the documentation under `/docs/`. Every URL that was published before still resolves, through 214 generated redirect stubs, because GitHub Pages has no server-side rewrites to do it properly ([#11243](https://github.com/mudler/LocalAI/pull/11243)).
+
+Twenty-four people contributed to this release, eleven of them for the first time. The gallery went from 1,221 entries to 1,505.
+
+To upgrade, pull `localai/localai:latest` or re-run the install script. The [full changelog](https://github.com/mudler/LocalAI/compare/v4.7.1...v4.8.0) has everything this post left out.

website/static/css/blog.css

@@ -99,7 +99,10 @@
 .bp-body hr{margin:2.8rem 0;border:0;border-top:1px solid var(--line2)}
 
 .bp-body figure{margin:2.2rem 0}
-.bp-body figure img{width:100%;border:1px solid var(--line);border-radius:11px;background:var(--deep)}
+/* video matches img: release posts embed the same demo clips the home page
+   and engines pages play, so they have to sit in the same card */
+.bp-body figure img,
+.bp-body figure video{width:100%;border:1px solid var(--line);border-radius:11px;background:var(--deep)}
 .bp-body figcaption{margin-top:.7rem;font-family:'Geist Mono',monospace;font-size:.64rem;
   line-height:1.6;color:var(--faint)}