chore: add the missing sign-off to the master merge commit

The merge commit 76f7959a9d was created with
`git merge --no-edit`, which takes git's default merge message and therefore
carried no Signed-off-by. DCO flags it. Remediate forward rather than
rewriting the branch, which would drop the Darwin fixes other people already
pulled from it.

I, Ettore Di Giacinto <mudler@localai.io>, hereby add my Signed-off-by to this commit: 76f7959a9d

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

.agents/api-endpoints-and-auth.md

@@ -304,7 +304,9 @@ React pages that want to filter the ModelSelector by capability import this symb
 
 ### 4. `docs/content/` (user-facing documentation)
 
-A new capability deserves its own page under `docs/content/features/`, plus cross-links from related features and an entry in `docs/content/whats-new.md`. See the pattern used by `face-recognition.md` / `object-detection.md`.
+A new capability deserves its own page under `docs/content/features/`, plus cross-links from related features. See the pattern used by `face-recognition.md` / `object-detection.md`.
+
+Announcing it is the release's job, not this page's: the capability gets covered in the release blog post under `website/content/blog/`. See [preparing-a-release.md](preparing-a-release.md). `docs/content/whats-new.md` is only a pointer at the blog and GitHub Releases, so there is nothing to add there.
 
 ## Path protection rules
 
@@ -334,7 +336,7 @@ When adding a new endpoint:
 - [ ] Swagger block on the handler: `@Summary`, `@Tags`, `@Param`, `@Success`, `@Router`
 - [ ] If new capability area (new swagger tag): entry in `instructionDefs` in `core/http/endpoints/localai/api_instructions.go` + test count bumped in `api_instructions_test.go`
 - [ ] If new `FLAG_*` usecase flag: matching `CAP_*` symbol exported from `core/http/react-ui/src/utils/capabilities.js`
-- [ ] `docs/content/features/<feature>.md` created; cross-links from related feature pages; entry in `docs/content/whats-new.md`
+- [ ] `docs/content/features/<feature>.md` created; cross-links from related feature pages; capability covered in the release blog post (see [preparing-a-release.md](preparing-a-release.md))
 
 **Quality**
 - [ ] Error responses use `schema.ErrorResponse` format (or `echo.NewHTTPError` with a mapped gRPC status — see the `mapBackendError` helper in `core/http/endpoints/localai/images.go`)

.agents/backend-signing.md

@@ -16,8 +16,7 @@ side (`pkg/oci/cosignverify` plus the gallery YAML).
   per-arch manifest before checking signatures.
 - **Storage:** Signatures are written as OCI 1.1 referrers
   (`--registry-referrers-mode=oci-1-1`) in the new Sigstore bundle format
-  (current cosign releases do this by default; no `--new-bundle-format`
-  flag). No `:sha256-<hex>.sig` tag clutter.
+  (`--new-bundle-format`). No `:sha256-<hex>.sig` tag clutter.
 - **Consumer:** `pkg/oci/cosignverify` discovers the bundle via the
   referrers API, hands it to `sigstore-go`, and verifies it against the
   policy declared in the gallery YAML (`Gallery.Verification`).
@@ -34,14 +33,15 @@ to sign. The job needs:
 
 - `permissions: { id-token: write, contents: read }` at the job level so
   the runner can exchange its GitHub OIDC token for a Fulcio cert.
-- `sigstore/cosign-installer@v3` step (current cosign releases already
-  default to the new bundle format).
+- `sigstore/cosign-installer@v3` step (the pinned cosign v2 release needs
+  `--new-bundle-format` explicitly).
 - After each `docker buildx imagetools create`, resolve the resulting
   list digest with `docker buildx imagetools inspect <tag> --format
   '{{.Manifest.Digest}}'` and sign:
 
 ```sh
 cosign sign --yes --recursive \
+  --new-bundle-format \
   --registry-referrers-mode=oci-1-1 \
   "${REGISTRY_REPO}@${DIGEST}"
 ```
@@ -70,7 +70,7 @@ entry (`backend/index.yaml`):
   url: github:mudler/LocalAI/backend/index.yaml@master
   verification:
     issuer: "https://token.actions.githubusercontent.com"
-    identity_regex: "^https://github\\.com/mudler/LocalAI/\\.github/workflows/backend_merge\\.yml@refs/heads/master$"
+    identity_regex: "^https://github\\.com/mudler/LocalAI/\\.github/workflows/backend_merge\\.yml@refs/(heads/master|tags/.+)$"
     # Optional revocation cutoff; advance during incident response.
     # not_before: "2026-06-01T00:00:00Z"
 ```

.docker/llama-cpp-build-target.sh

@@ -4,6 +4,24 @@ set -euo pipefail
 arch=${1:?target architecture is required}
 build_type=${2-}
 
+# SYCL compiles the whole tree with icpx -fsycl, and icpx never finishes
+# ggml-cpu/arch/x86/repack.cpp at -march=sapphirerapids: the job sits on that one
+# translation unit until GitHub kills it at 6h. gcc builds the same file in
+# seconds, so only the SYCL images have to give up the CPU variant matrix.
+#
+# ROCm runs out of the same 6h budget for a different reason: volume, not a
+# stall. hipcc compiles ggml's HIP kernels once per entry in AMDGPU_TARGETS,
+# which is eleven architectures (gfx908 through gfx1201), and the CPU variant
+# matrix lands on top of that. The job built in 2h27m before it was added and
+# has been killed at exactly 6h00m on every run since, so no ROCm llama-cpp
+# image has been published since 2026-08-01.
+case "$build_type" in
+  sycl*|hipblas*)
+    echo llama-cpp-fallback
+    exit 0
+    ;;
+esac
+
 # GPU arm64 base images do not consistently provide the gcc-14 toolchain needed
 # to compile ggml's armv9.2 CPU variants. Keep their portable fallback until the
 # builder images can supply that compiler.

.docker/turboquant-build-target.sh

@@ -4,6 +4,17 @@ set -euo pipefail
 arch=${1:?target architecture is required}
 build_type=${2-}
 
+# SYCL compiles the whole tree with icpx -fsycl, and icpx never finishes
+# ggml-cpu/arch/x86/repack.cpp at -march=sapphirerapids: the job sits on that one
+# translation unit until GitHub kills it at 6h. gcc builds the same file in
+# seconds, so only the SYCL images have to give up the CPU variant matrix.
+case "$build_type" in
+  sycl*)
+    echo turboquant-fallback
+    exit 0
+    ;;
+esac
+
 # GPU arm64 base images do not consistently provide the gcc-14 toolchain needed
 # to compile ggml's armv9.2 CPU variants. Keep their portable fallback until the
 # builder images can supply that compiler.

.github/workflows/backend_merge.yml

@@ -71,8 +71,8 @@ jobs:
 
       # cosign signs each pushed manifest list with --recursive so the
       # index and every per-arch entry get an attached Sigstore bundle.
-      # Recent cosign releases always emit the new bundle format, so
-      # there's no extra CLI flag to opt into it.
+      # The pinned cosign v2 release needs --new-bundle-format explicitly;
+      # the verifier only consumes OCI 1.1 Sigstore bundle referrers.
       - name: Install cosign
         if: github.event_name != 'pull_request'
         uses: sigstore/cosign-installer@v3
@@ -159,6 +159,7 @@ jobs:
           # manifest before checking signatures need the per-arch
           # signatures, not just the list-level one.
           cosign sign --yes --recursive \
+            --new-bundle-format \
             --registry-referrers-mode=oci-1-1 \
             "quay.io/go-skynet/local-ai-backends@${digest}"
 
@@ -185,6 +186,7 @@ jobs:
           ' <<< "$DOCKER_METADATA_OUTPUT_JSON")
           digest=$(docker buildx imagetools inspect "$first_tag" --format '{{.Manifest.Digest}}')
           cosign sign --yes --recursive \
+            --new-bundle-format \
             --registry-referrers-mode=oci-1-1 \
             "localai/localai-backends@${digest}"
 

.github/workflows/gh-pages.yml

@@ -51,7 +51,16 @@ jobs:
       - name: Setup Go
         uses: actions/setup-go@v5
         with:
-          go-version: '1.22'
+          # Track go.mod rather than a literal. Pinned at 1.22 this installed a
+          # toolchain older than the module's `go 1.26.0`, so the `go run` below
+          # downloaded the real one from proxy.golang.org on every run. That
+          # fetch is not always reachable from the runner and the deploy failed
+          # on five of eight consecutive master pushes with:
+          #   go: download go1.26.0: ... connect: network is unreachable
+          #   ##[error]Command failed: go env GOPATH
+          # Installing the version the module asks for removes the download
+          # instead of depending on it succeeding.
+          go-version-file: go.mod
           cache: false
 
       - name: Setup Hugo

README.md

@@ -195,7 +195,7 @@ For more details, see the [Getting Started guide](https://localai.io/basics/gett
 - **August 2025**: MLX, MLX-VLM, Diffusers, llama.cpp now supported on Apple Silicon
 - **July 2025**: All backends migrated outside the main binary — [lightweight, modular architecture](https://github.com/mudler/LocalAI/releases/tag/v3.2.0)
 
-For older news and full release notes, see [GitHub Releases](https://github.com/mudler/LocalAI/releases) and the [News page](https://localai.io/basics/news/).
+For older news and full release notes, see [GitHub Releases](https://github.com/mudler/LocalAI/releases) and the [blog](https://localai.io/blog/).
 
 ## Features
 
@@ -260,7 +260,7 @@ We also maintain [apex-quant](https://github.com/localai-org/apex-quant), a per-
 - [Kubernetes installation](https://localai.io/basics/getting_started/#run-localai-in-kubernetes)
 - [Integrations & community projects](https://localai.io/docs/integrations/)
 - [Installation video walkthrough](https://www.youtube.com/watch?v=cMVNnlqwfw4)
-- [Media & blog posts](https://localai.io/basics/news/#media-blogs-social)
+- [Blog: release write-ups, benchmarks and engineering notes](https://localai.io/blog/)
 - [Examples](https://github.com/mudler/LocalAI-examples) — including the [realtime voice assistant demo](https://github.com/localai-org/localai-realtime-demo) (Go client for the Realtime API with tool calling)
 
 ## Team

backend/backend.proto

@@ -15,6 +15,7 @@ service Backend {
   rpc PredictStream(PredictOptions) returns (stream Reply) {}
   rpc Embedding(PredictOptions) returns (EmbeddingResult) {}
   rpc GenerateImage(GenerateImageRequest) returns (Result) {}
+  rpc UpscaleImage(UpscaleImageRequest) returns (Result) {}
   rpc GenerateVideo(GenerateVideoRequest) returns (Result) {}
   rpc Generate3D(Generate3DRequest) returns (Result) {}
   rpc AudioTranscription(TranscriptRequest) returns (TranscriptResult) {}
@@ -637,6 +638,12 @@ message GenerateImageRequest {
   string ModelIdentity = 13;
 }
 
+message UpscaleImageRequest {
+  string src = 1;   // input image path
+  string dst = 2;   // output image path
+  int32  scale = 3; // upscale factor (e.g. 2 or 4)
+}
+
 message GenerateVideoRequest {
   string prompt = 1;
   string negative_prompt = 2;  // Negative prompt for video generation

backend/cpp/audio-cpp/Makefile

@@ -9,7 +9,7 @@
 # recipe is a make target (not a prepare.sh) so 'make purge && make' is a clean
 # rebuild and so the bump bot can see the pin.
 
-AUDIO_CPP_VERSION?=545e29a6f2fde24298cb3b0f07baab4352987ac9
+AUDIO_CPP_VERSION?=238ab6a9e321c17de8e120559f57efeedaeb1345
 AUDIO_CPP_REPO?=https://github.com/0xShug0/audio.cpp
 
 CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))

backend/cpp/ds4/CMakeLists.txt

@@ -69,7 +69,15 @@ target_include_directories(hw_grpc_proto PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
 
 set(DS4_OBJS "${DS4_DIR}/ds4.o")
 if(DS4_GPU STREQUAL "cuda")
-    list(APPEND DS4_OBJS "${DS4_DIR}/ds4_cuda.o")
+    list(APPEND DS4_OBJS
+        "${DS4_DIR}/ds4_cuda.o"
+        "${DS4_DIR}/cuda/mmq/ds4_ggml_stubs.o"
+        "${DS4_DIR}/cuda/mmq/ds4_mmq.o"
+        "${DS4_DIR}/cuda/mmq/ds4_mmq_d2r.o"
+        "${DS4_DIR}/cuda/mmq/quantize.o"
+        "${DS4_DIR}/cuda/mmq/mmid.o"
+        "${DS4_DIR}/cuda/mmq/mmvq.o"
+        "${DS4_DIR}/cuda/mmq/ds4_repack.o")
 elseif(DS4_GPU STREQUAL "metal")
     list(APPEND DS4_OBJS "${DS4_DIR}/ds4_metal.o")
 elseif(DS4_GPU STREQUAL "cpu")

backend/cpp/ds4/Makefile

@@ -1,10 +1,10 @@
 # ds4 backend Makefile.
 #
-# Upstream pin lives below as DS4_VERSION?=54b36ed9ba42da31b24f2d1a5feb075c2475dbb1
+# Upstream pin lives below as DS4_VERSION?=6747e7718dd08f00b680d0c16231f2d59ec3747e
 # (.github/bump_deps.sh) can find and update it - matches the
 # llama-cpp / ik-llama-cpp / turboquant convention.
 
-DS4_VERSION?=54b36ed9ba42da31b24f2d1a5feb075c2475dbb1
+DS4_VERSION?=6747e7718dd08f00b680d0c16231f2d59ec3747e
 DS4_REPO?=https://github.com/antirez/ds4
 
 CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
@@ -23,7 +23,9 @@ CMAKE_ARGS ?= -DCMAKE_BUILD_TYPE=Release
 # are shared by every GPU mode, so append them unconditionally below.
 ifeq ($(BUILD_TYPE),cublas)
     CMAKE_ARGS += -DDS4_GPU=cuda
-    DS4_OBJ_TARGET := ds4.o ds4_cuda.o ds4_distributed.o ds4_tp.o ds4_ssd.o ds4_layer_pack.o
+    DS4_OBJ_TARGET := ds4.o ds4_cuda.o ds4_distributed.o ds4_tp.o ds4_ssd.o ds4_layer_pack.o \
+        cuda/mmq/ds4_ggml_stubs.o cuda/mmq/ds4_mmq.o cuda/mmq/ds4_mmq_d2r.o \
+        cuda/mmq/quantize.o cuda/mmq/mmid.o cuda/mmq/mmvq.o cuda/mmq/ds4_repack.o
 else ifeq ($(UNAME_S),Darwin)
     CMAKE_ARGS += -DDS4_GPU=metal
     DS4_OBJ_TARGET := ds4.o ds4_metal.o ds4_distributed.o ds4_tp.o ds4_ssd.o ds4_layer_pack.o
@@ -55,7 +57,7 @@ ds4:
 # the right per-platform compile flags (Objective-C/Metal on Darwin, nvcc on Linux+CUDA).
 ds4/ds4.o: ds4
 ifeq ($(BUILD_TYPE),cublas)
-	+$(MAKE) -C ds4 ds4.o ds4_cuda.o ds4_distributed.o ds4_tp.o ds4_ssd.o ds4_layer_pack.o
+	+$(MAKE) -C ds4 $(DS4_OBJ_TARGET)
 else ifeq ($(UNAME_S),Darwin)
 	+$(MAKE) -C ds4 ds4.o ds4_metal.o ds4_distributed.o ds4_tp.o ds4_ssd.o ds4_layer_pack.o
 else

backend/cpp/ik-llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-IK_LLAMA_VERSION?=0be97a7a5ad113f33e08729261649ccea2cdc5ff
+IK_LLAMA_VERSION?=6b55d2c7504f482e7c8ec6cbf22a19f3778c522b
 LLAMA_REPO?=https://github.com/ikawrakow/ik_llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-LLAMA_VERSION?=a7a6d0d269c896218b6c78e0933bd6a17519d3f6
+LLAMA_VERSION?=221f0f6356efe2260023208365705ec5d5a7c8f5
 LLAMA_REPO?=https://github.com/ggerganov/llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/llama-cpp/run.sh

@@ -12,10 +12,11 @@ grep -e "flags" /proc/cpuinfo | head -1
 
 BINARY=llama-cpp-fallback
 
-# CPU images and x86 GPU images ship a single llama-cpp-cpu-all built with ggml
+# CPU images and most x86 GPU images ship a single llama-cpp-cpu-all built with ggml
 # CPU_ALL_VARIANTS: ggml's backend registry dlopens the best libggml-cpu-*.so for this
 # host, so no shell-side AVX probing. GPU arm64 images still ship llama-cpp-fallback
-# until their builder toolchains support ggml's complete arm variant matrix.
+# until their builder toolchains support ggml's complete arm variant matrix, and so do
+# the SYCL images, whose icpx compiler hangs on the sapphirerapids variant.
 if [ -e "$CURDIR"/llama-cpp-cpu-all ]; then
 	BINARY=llama-cpp-cpu-all
 fi

backend/cpp/turboquant/run.sh

@@ -12,11 +12,12 @@ grep -e "flags" /proc/cpuinfo | head -1
 
 BINARY=turboquant-fallback
 
-# CPU images and x86 GPU images ship a single turboquant-cpu-all built with ggml
+# CPU images and most x86 GPU images ship a single turboquant-cpu-all built with ggml
 # CPU_ALL_VARIANTS: ggml's
 # backend registry dlopens the best libggml-cpu-*.so for this host, so no shell-side
 # probing. GPU arm64 images still ship turboquant-fallback until their builder toolchains
-# support ggml's complete arm variant matrix.
+# support ggml's complete arm variant matrix, and so do the SYCL images, whose icpx
+# compiler hangs on the sapphirerapids variant.
 if [ -e "$CURDIR"/turboquant-cpu-all ]; then
 	BINARY=turboquant-cpu-all
 fi

backend/go/crispasr/Makefile

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
 
 # CrispASR version (release tag)
 CRISPASR_REPO?=https://github.com/CrispStrobe/CrispASR
-CRISPASR_VERSION?=66ac7843e319b588f5410051c575affd19424fb3
+CRISPASR_VERSION?=ec730908a418b6032f9e69ded6186d3f042a7747
 SO_TARGET?=libgocrispasr.so
 
 CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

backend/go/stablediffusion-ggml/Makefile

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
 
 # stablediffusion.cpp (ggml)
 STABLEDIFFUSION_GGML_REPO?=https://github.com/leejet/stable-diffusion.cpp
-STABLEDIFFUSION_GGML_VERSION?=e31a86ce9110b11a98bd5990c329093244c2d1e3
+STABLEDIFFUSION_GGML_VERSION?=ea7f0c87cfe4c673263b4c201c596c7f1cbe2528
 
 CMAKE_ARGS+=-DGGML_MAX_NAME=128
 

backend/go/vllm-cpp/Makefile

@@ -11,7 +11,30 @@ JOBS?=$(shell nproc --ignore=1 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || e
 
 # vllm.cpp version
 VLLM_CPP_REPO?=https://github.com/mudler/vllm.cpp
-VLLM_CPP_VERSION?=9e1c9025ae61167a3335454d7cc0de6093c21845
+VLLM_CPP_VERSION?=0757cac231ecd571a83c4fd2f50805c9251fc225
+
+# MLX GEMM provider (darwin/metal only; see the metal branch below for why).
+# Consumed as the prebuilt pip wheel: building MLX from source needs `xcrun
+# metal`, i.e. a full Xcode the macOS runners do not have, while the wheel ships
+# include/, lib/libmlx.dylib and the compiled mlx.metallib ready to link.
+#
+# DEFAULT ON, but ONLY because VLLM_CPP_VERSION above is pinned at or past
+# vllm.cpp 89c46aeb, which SHAPE-GATES the provider to prefill. The ordering is
+# load-bearing, not incidental:
+#
+#   pin >= 89c46aeb, MLX on   -> 99.1% of MLX-LM  (gated: prefill only)
+#   pin <  89c46aeb, MLX on   -> ~51%             (ungated: it also takes decode)
+#
+# MLX's steel GEMM wins prefill (537 ms TTFT against 602) and loses decode badly,
+# because the provider pays an mx::eval sync plus an output memcpy per call and
+# decode makes ~112 calls per TOKEN. Ungated it does both; gated it does only the
+# good half. So if this pin is ever moved BACKWARDS, this default must go with it.
+VLLM_CPP_MLX?=on
+MLX_VERSION?=0.29.4
+MLX_VENV?=$(abspath ./mlx-venv)
+# Resolved lazily (recursive `=`, not `:=`): the glob only matches once the venv
+# target has run, and the interpreter version in the path varies per runner.
+MLX_ROOT=$(shell echo $(MLX_VENV)/lib/python*/site-packages/mlx)
 
 # The backend consumes only the stable C ABI (libvllm + include/vllm.h), so the
 # server, examples and tests of the engine are never built here.
@@ -49,6 +72,23 @@ else ifeq ($(BUILD_TYPE),vulkan)
 	CMAKE_ARGS+=-DVLLM_CPP_VULKAN=ON -DVLLM_CPP_CUDA=OFF
 else ifeq ($(BUILD_TYPE),metal)
 	CMAKE_ARGS+=-DVLLM_CPP_METAL=ON
+	# The optional MLX GEMM provider. vllm.cpp keeps it OFF by default because it
+	# is a ~19 MB libmlx.dylib plus a ~105 MB mlx.metallib, and upstream's
+	# position is that it must earn that cost by measurement. It does, on the
+	# only hardware this build targets: measured on an Apple M4 against the
+	# native MSL GEMM in the SAME binary (arms toggled by
+	# VT_OP_PROVIDER_DISABLE=mlx), Qwen3-1.7B-bf16 p=512 g=128, it is 1.5x to
+	# 2.2x aggregate throughput and 2x to 3x faster TTFT, at equal peak memory
+	# and bit-identical output on every parity shape. See vllm.cpp
+	# docs/BENCHMARKS.md "MLX GEMM provider A/B on Apple M4".
+	#
+	# MLX delegates the dense GEMM ONLY: kPagedAttention stays vllm.cpp's own
+	# kernel, because MLX has no paged-KV primitive at all.
+	#
+	# Set VLLM_CPP_MLX=off for a Metal build without it (smaller image, slower).
+	ifeq ($(VLLM_CPP_MLX),on)
+		MLX_ENABLED=1
+	endif
 else
 	CMAKE_ARGS+=-DVLLM_CPP_CUDA=OFF
 endif
@@ -56,6 +96,12 @@ endif
 UNAME_S := $(shell uname -s)
 ifeq ($(UNAME_S),Darwin)
 	LIB=libvllm.dylib
+	# Apple Clang diagnoses a pair of constant-folded array bounds in the Metal
+	# build as a GNU extension. Disable that diagnostic for both Objective-C and
+	# C++ because vllm.cpp appends target-local -Werror after these global flags.
+	CMAKE_ARGS+=-DCMAKE_CXX_FLAGS=-Wno-gnu-folding-constant
+	CMAKE_ARGS+=-DCMAKE_OBJC_FLAGS=-Wno-gnu-folding-constant
+	CMAKE_ARGS+=-DCMAKE_OBJCXX_FLAGS=-Wno-gnu-folding-constant
 else
 	LIB=libvllm.so
 endif
@@ -68,10 +114,54 @@ sources/vllm.cpp:
 	git fetch --depth 1 origin $(VLLM_CPP_VERSION) && \
 	git checkout FETCH_HEAD
 
-$(LIB): sources/vllm.cpp
+ifeq ($(MLX_ENABLED),1)
+# A stamp FILE, not a phony target: a phony prerequisite is always "newer" than
+# $(LIB) and would re-link libvllm on every invocation. Keyed on the version so
+# a MLX_VERSION bump reinstalls instead of silently reusing the old wheel.
+MLX_STAMP=$(MLX_VENV)/.mlx-$(MLX_VERSION).stamp
+MLX_CMAKE_ARGS=-DVLLM_CPP_MLX=ON -DMLX_ROOT=$(MLX_ROOT)
+
+$(MLX_STAMP):
+	@if [ ! -x "$(MLX_VENV)/bin/pip" ]; then \
+		python3 -m venv "$(MLX_VENV)" || { echo "vllm-cpp: python3 with venv is required to build the MLX provider; pass VLLM_CPP_MLX=off to build Metal without it" >&2; exit 1; }; \
+	fi
+	"$(MLX_VENV)"/bin/pip install --quiet --disable-pip-version-check "mlx==$(MLX_VERSION)"
+	@# Resolved in the SHELL, not by $(MLX_ROOT): make expands a whole recipe
+	@# before running its first line, so the glob would still be unmatched here.
+	@# Every later use (the cmake args, package.sh) expands after this target has
+	@# completed, where $(MLX_ROOT) does resolve.
+	@root=$$(echo "$(MLX_VENV)"/lib/python*/site-packages/mlx); \
+	test -f "$$root/lib/libmlx.dylib" -a -f "$$root/include/mlx/array.h" || \
+		{ echo "vllm-cpp: mlx==$(MLX_VERSION) did not provide lib/libmlx.dylib + include/mlx/array.h under $$root" >&2; exit 1; }
+	touch $@
+else
+MLX_STAMP=
+MLX_CMAKE_ARGS=
+endif
+
+# govllmcpp.go mirrors vllm.h by hand, and the only guard against the two
+# drifting apart is the vllm_abi_version check inside registerLib - which fires
+# at runtime, on the user's machine, taking down every model load (issue
+# #11379). Compare the two here instead, so moving VLLM_CPP_VERSION past the
+# mirrors turns the build red while the header is still around to diff.
+abi-check: sources/vllm.cpp
+	@engine=$$(sed -n 's/^#define VLLM_ABI_VERSION \([0-9][0-9]*\).*/\1/p' sources/vllm.cpp/include/vllm.h); \
+	backend=$$(sed -n 's/^const abiVersion = \([0-9][0-9]*\).*/\1/p' govllmcpp.go); \
+	if [ -z "$$engine" ] || [ -z "$$backend" ]; then \
+		echo "vllm-cpp: cannot read the ABI version (engine='$$engine' backend='$$backend')" >&2; exit 1; \
+	fi; \
+	if [ "$$engine" != "$$backend" ]; then \
+		echo "vllm-cpp: ABI mismatch: vllm.cpp $(VLLM_CPP_VERSION) is v$$engine, govllmcpp.go mirrors v$$backend." >&2; \
+		echo "  Update the struct mirrors and abiVersion in govllmcpp.go (and the offsets in vllmcpp_test.go) to v$$engine." >&2; \
+		exit 1; \
+	fi; \
+	echo "vllm-cpp: ABI v$$engine matches the pinned engine"
+
+$(LIB): sources/vllm.cpp $(MLX_STAMP)
+	$(MAKE) abi-check
 	mkdir -p build && \
 	cd build && \
-	cmake ../sources/vllm.cpp $(CMAKE_ARGS) && \
+	cmake ../sources/vllm.cpp $(CMAKE_ARGS) $(MLX_CMAKE_ARGS) && \
 	cmake --build . --config Release -j$(JOBS) --target vllm_shared
 	cp -fL build/$(LIB) ./$(LIB)
 
@@ -79,16 +169,18 @@ vllm-cpp: main.go govllmcpp.go backend.go options.go $(LIB)
 	CGO_ENABLED=0 $(GOCMD) build -tags "$(GO_TAGS)" -o vllm-cpp ./
 
 package: vllm-cpp
-	bash package.sh
+	MLX_ROOT="$(MLX_ROOT)" bash package.sh
 
 build: package
 
 clean: purge
-	rm -rf libvllm.so libvllm.dylib package sources/vllm.cpp vllm-cpp
+	rm -rf libvllm.so libvllm.dylib package sources/vllm.cpp vllm-cpp "$(MLX_VENV)"
 
 purge:
 	rm -rf build
 
+.PHONY: abi-check
+
 .NOTPARALLEL:
 
 # The unit specs are pure Go (struct mirrors, option mapping, load

backend/go/vllm-cpp/README.md

@@ -6,7 +6,7 @@ safetensors + GGUF loading, CUDA / CPU / Metal / Vulkan) with no Python at
 inference time.
 
 The backend dlopens the engine's stable C ABI (`libvllm`, `include/vllm.h`,
-ABI v2) through purego:
+ABI v10) through purego:
 
 - `Load` -> `vllm_engine_load`: accepts a `.gguf` file or a HF-style model
   directory (`config.json` + safetensors). `context_size` maps to
@@ -29,6 +29,12 @@ ABI v2) through purego:
   LocalAI's Go-side grammar-constrained tool calling; JSON-schema / regex /
   choice constraints are also exposed by the ABI.
 
+The struct mirrors in `govllmcpp.go` are hand-written against one ABI version,
+and the engine refuses to load against any other. Moving `VLLM_CPP_VERSION` in
+the Makefile therefore means updating `abiVersion` plus the mirrors (and their
+offsets in `vllmcpp_test.go`) in the same change; `make abi-check` compares the
+pinned header against the bindings and the library build runs it first.
+
 Model config example:
 
 ```yaml
@@ -41,5 +47,50 @@ options:
 - max_num_seqs:16
 ```
 
+## Apple Silicon: the MLX GEMM provider (ON by default, gated to prefill)
+
+`BUILD_TYPE=metal` builds vllm.cpp's MLX provider for the dense GEMM
+(`VLLM_CPP_MLX=on`, the default here). It is on because upstream now SHAPE-GATES
+it to prefill; it was briefly off in this branch's history, and that was correct
+at the time for an ungated provider.
+
+The gate matters more than the flag. MLX's steel GEMM wins prefill but loses
+decode, because the provider pays an `mx::eval` synchronisation plus an output
+memcpy on every call and decode makes ~112 calls *per token*. Measured on an
+Apple M4, Qwen3-1.7B-bf16 warm at p=512 g=128:
+
+| configuration | prefill TTFT | warm throughput |
+|---|--:|--:|
+| MLX **gated to prefill** (pin >= 89c46aeb) | **524.5 ms** | **24.37 tok/s, 97.6% of MLX-LM** |
+| MLX ungated (older pins) | 537 ms | 12.7 tok/s |
+| MLX off | 602 ms | 23.9 tok/s, 95.9% |
+
+Ratios are against an MLX-LM baseline measured INTERLEAVED with ours over four
+ABBA blocks (its spread 0.34%, ours 0.12%). An earlier revision of this file
+claimed 99.1%; that used a two-run MLX-LM baseline containing an outlier and
+overstated us by about 1.5 points.
+
+**`VLLM_CPP_VERSION` and this flag are coupled.** Moving the pin back before
+`89c46aeb` while leaving `VLLM_CPP_MLX=on` would take the middle row — roughly
+half throughput. If you roll the pin back, roll the default back with it.
+
+One caveat: MLX's GEMM is not bit-identical to the native kernel, so an MLX build
+produces a different greedy sequence than a non-MLX one. That is a property of the
+provider, not of the gate, and it predates this packaging. Full disposition in
+vllm.cpp `docs/BENCHMARKS.md`.
+
+Build knobs:
+
+- `VLLM_CPP_MLX=off` builds Metal without the provider: ~124 MB smaller, and
+  96.4% of MLX-LM instead of 99.1%.
+- `MLX_VERSION` pins the wheel (default `0.29.4`). MLX is consumed as the
+  prebuilt pip wheel because building it from source needs `xcrun metal`, i.e. a
+  full Xcode the macOS runners do not have.
+
+Packaging vendors `libmlx.dylib`, `mlx.metallib` and MLX's MIT license into
+`package/lib/`, and rewrites `libvllm.dylib`'s rpath to `@loader_path/lib`
+(re-signing it, since `install_name_tool` invalidates the signature). The
+metallib must stay beside `libmlx.dylib`: MLX looks for it there.
+
 Testing: `make test` runs the unit specs; export `VLLM_CPP_MODEL=<model>` (and
 optionally `VLLM_CPP_LIBRARY=<libvllm path>`) to enable the e2e specs.

backend/go/vllm-cpp/backend.go

@@ -109,6 +109,16 @@ func (v *VllmCpp) Load(opts *pb.ModelOptions) error {
 
 	v.opts = parseOptions(opts)
 
+	// A DFlash draft is a second checkpoint the engine opens by path, and the
+	// engine never downloads one. Resolve it against LocalAI's models directory
+	// now so a repo-id spelling works, and so a missing draft fails here with an
+	// actionable message rather than as an HF-cache miss inside the load.
+	resolvedSpec, err := resolveDraftModelPath(v.opts.speculativeConfig, opts.ModelPath)
+	if err != nil {
+		return err
+	}
+	v.opts.speculativeConfig = resolvedSpec
+
 	mp := defaultModelParams()
 	if v.opts.blockSize > 0 {
 		mp.BlockSize = v.opts.blockSize
@@ -116,34 +126,62 @@ func (v *VllmCpp) Load(opts *pb.ModelOptions) error {
 	if v.opts.numBlocks > 0 {
 		mp.NumBlocks = v.opts.numBlocks
 	}
+	// Sequence-length precedence, narrowest source last: context_size is the
+	// generic LocalAI knob every backend honours, max_model_len is the
+	// vLLM-specific one, and engine_args.max_model_len is the explicit
+	// vllm-cpp override.
 	if opts.ContextSize > 0 {
 		mp.MaxModelLen = opts.ContextSize
 	}
+	if opts.MaxModelLen > 0 {
+		mp.MaxModelLen = opts.MaxModelLen
+	}
+	if v.opts.maxModelLen > 0 {
+		mp.MaxModelLen = v.opts.maxModelLen
+	}
 	if v.opts.maxNumSeqs > 0 {
 		mp.MaxNumSeqs = v.opts.maxNumSeqs
 	}
+	if v.opts.maxNumBatchedTokens > 0 {
+		mp.MaxNumBatchedTokens = v.opts.maxNumBatchedTokens
+	}
+	mp.EnablePrefixCaching = v.opts.enablePrefixCaching
+	mp.EnableJumpForward = v.opts.enableJumpForward
 
+	// Every string below is borrowed by C for the duration of the load call
+	// only (the library copies what it keeps), so the backing slices just have
+	// to outlive vllmEngineLoad - hence the single KeepAlive after it.
 	modelC := cString(model)
 	mp.ModelPath = uintptr(unsafe.Pointer(&modelC[0])) // #nosec G103 -- borrowed by C for the load call only
-	var toolParserC, reasoningParserC []byte
-	if v.opts.toolParser != "" {
-		toolParserC = cString(v.opts.toolParser)
-		mp.ToolParser = uintptr(unsafe.Pointer(&toolParserC[0])) // #nosec G103 -- borrowed by C for the load call only
-	}
-	if v.opts.reasoningParser != "" {
-		reasoningParserC = cString(v.opts.reasoningParser)
-		mp.ReasoningParser = uintptr(unsafe.Pointer(&reasoningParserC[0])) // #nosec G103 -- borrowed by C for the load call only
+	keep := [][]byte{modelC}
+	setStr := func(dst *uintptr, s string) {
+		if s == "" {
+			return
+		}
+		b := cString(s)
+		keep = append(keep, b)
+		*dst = uintptr(unsafe.Pointer(&b[0])) // #nosec G103 -- borrowed by C for the load call only
 	}
+	setStr(&mp.ToolParser, v.opts.toolParser)
+	setStr(&mp.ReasoningParser, v.opts.reasoningParser)
+	setStr(&mp.SpeculativeConfig, v.opts.speculativeConfig)
+	setStr(&mp.KVTransferConfig, v.opts.kvTransferConfig)
+	setStr(&mp.SchedulingPolicy, v.opts.schedulingPolicy)
+	setStr(&mp.TokenizerConfigPath, v.opts.tokenizerConfigPath)
 
 	xlog.Info("[vllm-cpp] Load", "model", model, "engine", vllmVersion(),
 		"blockSize", mp.BlockSize, "numBlocks", mp.NumBlocks,
-		"maxModelLen", mp.MaxModelLen, "maxNumSeqs", mp.MaxNumSeqs)
+		"maxModelLen", mp.MaxModelLen, "maxNumSeqs", mp.MaxNumSeqs,
+		"maxNumBatchedTokens", mp.MaxNumBatchedTokens,
+		"prefixCaching", triStateName(mp.EnablePrefixCaching),
+		"jumpForward", triStateName(mp.EnableJumpForward),
+		"schedulingPolicy", v.opts.schedulingPolicy,
+		"speculativeConfig", v.opts.speculativeConfig,
+		"kvTransferConfig", v.opts.kvTransferConfig)
 
 	var engine uintptr
 	rc := vllmEngineLoad(unsafe.Pointer(&mp), unsafe.Pointer(&engine)) // #nosec G103 -- POD out-params
-	runtime.KeepAlive(modelC)
-	runtime.KeepAlive(toolParserC)
-	runtime.KeepAlive(reasoningParserC)
+	runtime.KeepAlive(keep)
 	if rc != vllmOK {
 		return fmt.Errorf("vllm-cpp: engine load failed: %s", vllmLastError())
 	}

backend/go/vllm-cpp/govllmcpp.go

@@ -1,6 +1,6 @@
 package main
 
-// purego bindings for the vllm.cpp stable C ABI (include/vllm.h, ABI v2).
+// purego bindings for the vllm.cpp stable C ABI (include/vllm.h, ABI v10).
 //
 // The structs below are hand-mirrored PODs of the C declarations, with
 // explicit padding so the Go layout matches the C layout on linux/darwin
@@ -17,29 +17,65 @@ import (
 	"github.com/ebitengine/purego"
 )
 
-// abiVersion is the VLLM_ABI_VERSION this file mirrors (vllm.h).
-const abiVersion = 5
+// abiVersion is the VLLM_ABI_VERSION this file mirrors (vllm.h). It must track
+// the header of the VLLM_CPP_VERSION pinned in the Makefile: the build checks
+// the two against each other, because a mismatch is only caught at runtime by
+// registerLib, where it takes the backend down on every load (issue #11379).
+const abiVersion = 10
+
+// The ABI's tri-state toggles (enable_prefix_caching ABI v7,
+// enable_jump_forward ABI v10) share one encoding: 0 is NOT "off", it is
+// "defer" - to the model capability for prefix caching, to the environment for
+// jump forward. Only 2 is an explicit off.
+const (
+	triStateDefer int32 = 0
+	triStateOn    int32 = 1
+	triStateOff   int32 = 2
+)
+
+// triStateName renders a tri-state for the load log line, where "0" would
+// otherwise read as "off" rather than "whatever the default resolves to".
+func triStateName(state int32) string {
+	switch state {
+	case triStateOn:
+		return "on"
+	case triStateOff:
+		return "off"
+	default:
+		return "model-default"
+	}
+}
 
 // vllm_status (vllm.h).
 const (
 	vllmOK = 0
 )
 
-// cModelParams mirrors vllm_model_params.
+// cModelParams mirrors vllm_model_params. The int32 fields sit in pairs so the
+// interior needs no padding on LP64, but the struct is 8-aligned (it holds
+// pointers) and ends on a lone int32, so the trailing pad is explicit. Offsets
+// and total size are asserted in vllmcpp_test.go.
 type cModelParams struct {
 	ModelPath           uintptr // const char*
-	TokenizerConfigPath uintptr // const char*
+	TokenizerConfigPath uintptr // const char*; NULL = <model_dir>/... (ABI v9)
 	BlockSize           int32
 	NumBlocks           int32
 	MaxModelLen         int32
 	MaxNumSeqs          int32
 	ToolParser          uintptr // const char*; NULL = auto-detect (ABI v4)
 	ReasoningParser     uintptr // const char*; NULL = auto-detect (ABI v5)
+	SpeculativeConfig   uintptr // const char* JSON; NULL = no speculation (ABI v6)
+	EnablePrefixCaching int32   // tri-state 0/1/2 (ABI v7)
+	MaxNumBatchedTokens int32   // <= 0 = per-arch default (ABI v9)
+	SchedulingPolicy    uintptr // const char*; NULL = "fcfs" (ABI v9)
+	KVTransferConfig    uintptr // const char* JSON; NULL = no connector (ABI v9)
+	EnableJumpForward   int32   // tri-state 0/1/2 (ABI v10)
+	_                   [4]byte // trailing pad to the struct's 8-byte alignment
 }
 
-// cSamplingParams mirrors vllm_sampling_params (ABI v2, structured fields
-// included). Padding matches the C compiler's: the uint64 seed is 8-aligned,
-// and each pointer following an int32 is 8-aligned.
+// cSamplingParams mirrors vllm_sampling_params (structured fields included).
+// Padding matches the C compiler's: the uint64 seed is 8-aligned, and each
+// pointer following an int32 is 8-aligned.
 type cSamplingParams struct {
 	Temperature          float32
 	TopP                 float32
@@ -65,6 +101,12 @@ type cSamplingParams struct {
 	StructuredGrammar    uintptr // const char*
 	StructuredJSONObject int32
 	_                    [4]byte
+	// ABI v8 tail. LocalAI installs no custom logits processor, but the fields
+	// MUST be mirrored: the C side reads them off the pointer we hand it, so a
+	// Go struct that stopped at StructuredJSONObject would have the engine read
+	// 16 bytes past our allocation and call whatever garbage sat there.
+	LogitsProcessor         uintptr // vllm_logits_processor; NULL = none
+	LogitsProcessorUserData uintptr // void*
 }
 
 // cCompletion mirrors vllm_completion.

backend/go/vllm-cpp/options.go

@@ -1,30 +1,80 @@
 package main
 
-// Engine-sizing knobs carried through the model config's free-form
-// `options:` list ("key:value" entries), mirroring how the other in-house
-// backends pass engine-specific settings that have no proto field.
+// Load-time engine configuration, from two config surfaces:
+//
+//   - `engine_args:` (ModelOptions.EngineArgs, a JSON object) is the canonical
+//     one. Keys are spelled exactly as vLLM's own CLI flags, so a config written
+//     against vLLM works verbatim here - `speculative_config` and
+//     `kv_transfer_config` in particular take the same JSON documents vLLM's
+//     --speculative-config / --kv-transfer-config accept, and are handed to the
+//     engine unparsed.
+//   - `options:` (the free-form "key:value" list) is the older surface this
+//     backend shipped with. It is still honoured so existing configs keep
+//     working; engine_args wins on any key set in both.
+//
+// Anything unrecognised is ignored rather than fatal: the engine validates the
+// documents it is given and reports a precise error at load, and a config that
+// also carries knobs for a different backend must not fail the load here.
 
 import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path"
+	"path/filepath"
 	"strconv"
 	"strings"
 
 	pb "github.com/mudler/LocalAI/pkg/grpc/proto"
+	"github.com/mudler/xlog"
 )
 
 type loadOptions struct {
 	blockSize  int32 // KV block size (tokens/block); engine default 32.
 	numBlocks  int32 // KV blocks to allocate; engine default 256.
 	maxNumSeqs int32 // max concurrent sequences; engine default 8.
+	// Max sequence length. Also settable through the model config's
+	// context_size / max_model_len; see Load for the precedence.
+	maxModelLen int32
+	// Per-step chunked-prefill token budget (ABI v9). 0 = the engine's
+	// bounded per-arch default.
+	maxNumBatchedTokens int32
+	// Automatic prefix caching tri-state (ABI v7): 0 = the model-capability
+	// default, 1 = force on, 2 = force off.
+	enablePrefixCaching int32
+	// Jump-forward decoding tri-state (ABI v10), SGLang's grammar-speed subset:
+	// 0 = defer to the environment (VT_ENABLE_JUMP_FORWARD, default off),
+	// 1 = force on, 2 = force off.
+	enableJumpForward int32
+	// Scheduler admission policy (ABI v9): "" = fcfs, else fcfs|priority|lpm.
+	schedulingPolicy string
 	// Engine-side parser selection (ABI v4/v5). Empty = the engine
 	// auto-detects from the chat template; "none" disables the reasoning
 	// split; unknown names fail the first chat call.
 	toolParser      string
 	reasoningParser string
+	// Speculative decoding (ABI v6), as vLLM's --speculative-config JSON:
+	// {"method":"mtp"|"dflash"|"ngram", ...}. Empty = no speculation.
+	speculativeConfig string
+	// External KV connector / LMCache (ABI v9), as vLLM's --kv-transfer-config
+	// JSON. Empty = no connector.
+	kvTransferConfig string
+	// Override for the tokenizer_config.json the chat template is read from
+	// (ABI v9). Empty = <model_dir>/tokenizer_config.json.
+	tokenizerConfigPath string
 }
 
 func parseOptions(opts *pb.ModelOptions) loadOptions {
 	lo := loadOptions{}
-	for _, o := range opts.GetOptions() {
+	applyOptionsList(&lo, opts.GetOptions())
+	applyEngineArgs(&lo, opts.GetEngineArgs())
+	return lo
+}
+
+// applyOptionsList reads the legacy free-form "key:value" list. strings.Cut
+// splits on the FIRST colon only, so a JSON object value survives intact.
+func applyOptionsList(lo *loadOptions, options []string) {
+	for _, o := range options {
 		k, v, found := strings.Cut(o, ":")
 		if !found {
 			continue
@@ -36,13 +86,211 @@ func parseOptions(opts *pb.ModelOptions) loadOptions {
 			lo.numBlocks = parseInt32(v, lo.numBlocks)
 		case "max_num_seqs":
 			lo.maxNumSeqs = parseInt32(v, lo.maxNumSeqs)
-		case "tool_parser":
+		case "max_num_batched_tokens":
+			lo.maxNumBatchedTokens = parseInt32(v, lo.maxNumBatchedTokens)
+		case "max_model_len":
+			lo.maxModelLen = parseInt32(v, lo.maxModelLen)
+		case "scheduling_policy", "schedule_policy":
+			lo.schedulingPolicy = strings.TrimSpace(v)
+		case "tool_parser", "tool_call_parser":
 			lo.toolParser = strings.TrimSpace(v)
 		case "reasoning_parser":
 			lo.reasoningParser = strings.TrimSpace(v)
+		case "speculative_config":
+			lo.speculativeConfig = strings.TrimSpace(v)
+		case "kv_transfer_config":
+			lo.kvTransferConfig = strings.TrimSpace(v)
+		case "tokenizer_config", "tokenizer_config_path":
+			lo.tokenizerConfigPath = strings.TrimSpace(v)
+		case "enable_prefix_caching", "enable_radix_attention":
+			if b, err := strconv.ParseBool(strings.TrimSpace(v)); err == nil {
+				lo.enablePrefixCaching = boolTriState(b)
+			}
+		case "enable_jump_forward":
+			if b, err := strconv.ParseBool(strings.TrimSpace(v)); err == nil {
+				lo.enableJumpForward = boolTriState(b)
+			}
 		}
 	}
-	return lo
+}
+
+// applyEngineArgs overlays the `engine_args:` JSON object. A document that does
+// not parse is logged and skipped: engine_args is shared with the other engines
+// (the vLLM and SGLang backends read the same field), so a stray key must not
+// take the model down.
+func applyEngineArgs(lo *loadOptions, engineArgs string) {
+	if strings.TrimSpace(engineArgs) == "" {
+		return
+	}
+	var args map[string]any
+	if err := json.Unmarshal([]byte(engineArgs), &args); err != nil {
+		xlog.Warn("[vllm-cpp] ignoring unparseable engine_args", "error", err)
+		return
+	}
+	for k, v := range args {
+		switch k {
+		case "block_size":
+			lo.blockSize = jsonInt32(v, lo.blockSize)
+		case "num_blocks":
+			lo.numBlocks = jsonInt32(v, lo.numBlocks)
+		case "max_num_seqs":
+			lo.maxNumSeqs = jsonInt32(v, lo.maxNumSeqs)
+		case "max_num_batched_tokens":
+			lo.maxNumBatchedTokens = jsonInt32(v, lo.maxNumBatchedTokens)
+		case "max_model_len":
+			lo.maxModelLen = jsonInt32(v, lo.maxModelLen)
+		case "scheduling_policy", "schedule_policy":
+			lo.schedulingPolicy = jsonString(v, lo.schedulingPolicy)
+		case "tool_parser", "tool_call_parser":
+			lo.toolParser = jsonString(v, lo.toolParser)
+		case "reasoning_parser":
+			lo.reasoningParser = jsonString(v, lo.reasoningParser)
+		case "tokenizer_config", "tokenizer_config_path":
+			lo.tokenizerConfigPath = jsonString(v, lo.tokenizerConfigPath)
+		case "speculative_config":
+			lo.speculativeConfig = jsonDocument(v, lo.speculativeConfig, k)
+		case "kv_transfer_config":
+			lo.kvTransferConfig = jsonDocument(v, lo.kvTransferConfig, k)
+		case "enable_prefix_caching", "enable_radix_attention":
+			if b, ok := v.(bool); ok {
+				lo.enablePrefixCaching = boolTriState(b)
+			}
+		case "enable_jump_forward":
+			if b, ok := v.(bool); ok {
+				lo.enableJumpForward = boolTriState(b)
+			}
+		default:
+			xlog.Debug("[vllm-cpp] ignoring unknown engine_args key", "key", k)
+		}
+	}
+}
+
+// boolTriState maps a YAML/JSON boolean onto the ABI's tri-state encoding. An
+// explicit `false` must reach the engine as force-OFF (2), NOT as the 0 that
+// means "defer". The difference is real in both directions: prefix caching
+// defaults ON for dense archs and OFF for hybrid ones, and jump forward defers
+// to VT_ENABLE_JUMP_FORWARD.
+func boolTriState(on bool) int32 {
+	if on {
+		return triStateOn
+	}
+	return triStateOff
+}
+
+// jsonDocument normalises an object-valued engine_args entry to a JSON string
+// for the C ABI. YAML nesting arrives as a map (the natural spelling); a
+// pre-encoded JSON string is accepted too, since a config round-tripped through
+// a flat store may carry it that way.
+func jsonDocument(v any, fallback string, key string) string {
+	switch t := v.(type) {
+	case string:
+		if strings.TrimSpace(t) == "" {
+			return fallback
+		}
+		return t
+	default:
+		buf, err := json.Marshal(t)
+		if err != nil {
+			xlog.Warn("[vllm-cpp] ignoring unencodable engine_args value", "key", key, "error", err)
+			return fallback
+		}
+		return string(buf)
+	}
+}
+
+func jsonString(v any, fallback string) string {
+	s, ok := v.(string)
+	if !ok {
+		return fallback
+	}
+	return strings.TrimSpace(s)
+}
+
+// jsonInt32 accepts the float64 a JSON number decodes to, plus the string
+// spelling a YAML config may produce. Non-positive values keep the fallback:
+// every knob this covers uses "<= 0 means the engine default".
+func jsonInt32(v any, fallback int32) int32 {
+	switch t := v.(type) {
+	case float64:
+		if t <= 0 || t > 1<<31-1 {
+			return fallback
+		}
+		return int32(t)
+	case string:
+		return parseInt32(t, fallback)
+	default:
+		return fallback
+	}
+}
+
+// resolveDraftModelPath rewrites a DFlash draft reference into an absolute path
+// the engine can actually open.
+//
+// The engine resolves `speculative_config.model` against a directory containing
+// config.json, or against ~/.cache/huggingface/hub/models--<org>--<repo>/
+// snapshots/* - and it NEVER downloads. LocalAI keeps models in its own
+// directory, so a bare HF repo id (the spelling the vLLM docs teach) misses the
+// HF cache and dies deep in the load with "draft checkpoint not found", which
+// reads like a broken checkpoint rather than a missing download.
+//
+// So: try the reference as given, then the last path segment under the models
+// dir (`z-lab/Qwen3.6-27B-DFlash` -> `<models>/Qwen3.6-27B-DFlash`, which is
+// what LocalAI's own downloader produces), then the whole reference under the
+// models dir. If none exist, fail HERE with a message naming both what was
+// asked for and where we looked.
+//
+// mtp and ngram carry no separate draft checkpoint, so they pass through. A
+// document that does not parse also passes through: the engine owns config
+// validation and produces the better error.
+func resolveDraftModelPath(speculativeConfig, modelsDir string) (string, error) {
+	if strings.TrimSpace(speculativeConfig) == "" {
+		return speculativeConfig, nil
+	}
+	var spec map[string]any
+	if err := json.Unmarshal([]byte(speculativeConfig), &spec); err != nil {
+		return speculativeConfig, nil
+	}
+	if method, _ := spec["method"].(string); !strings.EqualFold(method, "dflash") {
+		return speculativeConfig, nil
+	}
+
+	ref, _ := spec["model"].(string)
+	ref = strings.TrimSpace(ref)
+	if ref == "" {
+		return "", fmt.Errorf(
+			"vllm-cpp: speculative_config method %q requires a \"model\" key naming the draft checkpoint", "dflash")
+	}
+
+	candidates := []string{ref}
+	if modelsDir != "" {
+		if base := path.Base(filepath.ToSlash(ref)); base != "" && base != "." && base != "/" {
+			candidates = append(candidates, filepath.Join(modelsDir, base))
+		}
+		candidates = append(candidates, filepath.Join(modelsDir, filepath.FromSlash(ref)))
+	}
+
+	for _, c := range candidates {
+		if _, err := os.Stat(filepath.Join(c, "config.json")); err != nil {
+			continue
+		}
+		abs, err := filepath.Abs(c)
+		if err != nil {
+			abs = c
+		}
+		spec["model"] = abs
+		out, err := json.Marshal(spec)
+		if err != nil {
+			return "", fmt.Errorf("vllm-cpp: re-encoding speculative_config: %w", err)
+		}
+		xlog.Info("[vllm-cpp] resolved DFlash draft checkpoint", "reference", ref, "path", abs)
+		return string(out), nil
+	}
+
+	return "", fmt.Errorf(
+		"vllm-cpp: DFlash draft checkpoint %q not found (looked in: %s). "+
+			"The engine does not download drafts - install the draft model into LocalAI first, "+
+			"or set speculative_config.model to an absolute path to a directory containing config.json",
+		ref, strings.Join(candidates, ", "))
 }
 
 func parseInt32(s string, fallback int32) int32 {

backend/go/vllm-cpp/package.sh

@@ -43,6 +43,50 @@ elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
     cp -arfLv /lib/aarch64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
 elif [ $(uname -s) = "Darwin" ]; then
     echo "Detected Darwin"
+    # Vendor the optional MLX GEMM provider, when libvllm was built against it.
+    # Three facts drive every line below, each verified on an Apple M4 before it
+    # was written:
+    #   1. libvllm.dylib carries an LC_LOAD_DYLIB on @rpath/libmlx.dylib, and its
+    #      build-time LC_RPATH points inside the build venv. That path does not
+    #      exist on a user's machine, so it must become @loader_path/lib.
+    #   2. MLX finds its ~100 MB mlx.metallib beside its OWN dylib, so the two
+    #      files have to land in the same directory or every Metal op dies with
+    #      "Failed to load the default metallib".
+    #   3. install_name_tool invalidates the code signature, and macOS refuses to
+    #      load an arm64 image whose signature does not match, so the patched
+    #      library must be re-signed ad-hoc afterwards.
+    if otool -L "$CURDIR/package/libvllm.dylib" 2>/dev/null | grep -q "libmlx.dylib"; then
+        MLX_LIB_DIR="${MLX_ROOT}/lib"
+        if [ ! -f "$MLX_LIB_DIR/libmlx.dylib" ] || [ ! -f "$MLX_LIB_DIR/mlx.metallib" ]; then
+            echo "Error: libvllm.dylib links libmlx.dylib but $MLX_LIB_DIR is missing libmlx.dylib/mlx.metallib" >&2
+            exit 1
+        fi
+        echo "Vendoring the MLX GEMM provider from $MLX_LIB_DIR"
+        cp -fLv "$MLX_LIB_DIR/libmlx.dylib" "$CURDIR/package/lib/"
+        cp -fLv "$MLX_LIB_DIR/mlx.metallib" "$CURDIR/package/lib/"
+        # MLX is MIT and we redistribute its binaries, so its license ships with
+        # them. mlx-metal is the wheel carrying the dylib and the metallib.
+        MLX_LICENSE=$(ls "${MLX_ROOT}"/../mlx_metal-*.dist-info/licenses/LICENSE 2>/dev/null | head -1)
+        if [ -z "$MLX_LICENSE" ]; then
+            MLX_LICENSE=$(ls "${MLX_ROOT}"/../mlx-*.dist-info/licenses/LICENSE 2>/dev/null | head -1)
+        fi
+        if [ -z "$MLX_LICENSE" ]; then
+            echo "Error: could not find the MLX LICENSE to redistribute alongside libmlx.dylib" >&2
+            exit 1
+        fi
+        cp -fLv "$MLX_LICENSE" "$CURDIR/package/lib/LICENSE.mlx"
+        # Drop every build-tree rpath, then point at the packaged copy.
+        otool -l "$CURDIR/package/libvllm.dylib" | awk '/LC_RPATH/{f=1;next} f&&/ path /{print $2;f=0}' | while read -r rp; do
+            install_name_tool -delete_rpath "$rp" "$CURDIR/package/libvllm.dylib" 2>/dev/null || true
+        done
+        install_name_tool -add_rpath "@loader_path/lib" "$CURDIR/package/libvllm.dylib"
+        codesign -f -s - "$CURDIR/package/libvllm.dylib"
+        # A broken rpath must fail the BUILD, not the user's first inference.
+        if ! otool -l "$CURDIR/package/libvllm.dylib" | grep -q "@loader_path/lib"; then
+            echo "Error: libvllm.dylib did not get the @loader_path/lib rpath" >&2
+            exit 1
+        fi
+    fi
 else
     echo "Error: Could not detect architecture"
     exit 1

backend/go/vllm-cpp/vllmcpp_test.go

@@ -16,10 +16,17 @@ func TestVllmCpp(t *testing.T) {
 	RunSpecs(t, "vllm-cpp suite")
 }
 
-// The Go POD mirrors must match the C struct layout of vllm.h (ABI v2)
+// The Go POD mirrors must match the C struct layout of vllm.h (ABI v10)
 // byte-for-byte: these offsets are the C offsets on LP64 (linux/darwin
 // amd64+arm64). A failure here means govllmcpp.go drifted from vllm.h.
 var _ = Describe("C ABI struct mirrors", func() {
+	It("declares the ABI version the pinned engine reports", func() {
+		// VLLM_ABI_VERSION in the vllm.h of VLLM_CPP_VERSION (Makefile).
+		// Moving the pin past this without growing the mirrors below ships a
+		// backend that refuses every load at startup (issue #11379).
+		Expect(abiVersion).To(Equal(10))
+	})
+
 	It("cModelParams matches vllm_model_params", func() {
 		var p cModelParams
 		Expect(unsafe.Offsetof(p.ModelPath)).To(Equal(uintptr(0)))
@@ -30,10 +37,18 @@ var _ = Describe("C ABI struct mirrors", func() {
 		Expect(unsafe.Offsetof(p.MaxNumSeqs)).To(Equal(uintptr(28)))
 		Expect(unsafe.Offsetof(p.ToolParser)).To(Equal(uintptr(32)))
 		Expect(unsafe.Offsetof(p.ReasoningParser)).To(Equal(uintptr(40)))
-		Expect(unsafe.Sizeof(p)).To(Equal(uintptr(48)))
+		Expect(unsafe.Offsetof(p.SpeculativeConfig)).To(Equal(uintptr(48)))
+		Expect(unsafe.Offsetof(p.EnablePrefixCaching)).To(Equal(uintptr(56)))
+		Expect(unsafe.Offsetof(p.MaxNumBatchedTokens)).To(Equal(uintptr(60)))
+		Expect(unsafe.Offsetof(p.SchedulingPolicy)).To(Equal(uintptr(64)))
+		Expect(unsafe.Offsetof(p.KVTransferConfig)).To(Equal(uintptr(72)))
+		Expect(unsafe.Offsetof(p.EnableJumpForward)).To(Equal(uintptr(80)))
+		// 88, not 84: the struct is 8-aligned (it holds pointers), so the
+		// trailing int32 is padded out. Go pads identically.
+		Expect(unsafe.Sizeof(p)).To(Equal(uintptr(88)))
 	})
 
-	It("cSamplingParams matches vllm_sampling_params (ABI v2)", func() {
+	It("cSamplingParams matches vllm_sampling_params (ABI v8)", func() {
 		var p cSamplingParams
 		Expect(unsafe.Offsetof(p.Temperature)).To(Equal(uintptr(0)))
 		Expect(unsafe.Offsetof(p.TopP)).To(Equal(uintptr(4)))
@@ -55,7 +70,9 @@ var _ = Describe("C ABI struct mirrors", func() {
 		Expect(unsafe.Offsetof(p.NStructuredChoice)).To(Equal(uintptr(96)))
 		Expect(unsafe.Offsetof(p.StructuredGrammar)).To(Equal(uintptr(104)))
 		Expect(unsafe.Offsetof(p.StructuredJSONObject)).To(Equal(uintptr(112)))
-		Expect(unsafe.Sizeof(p)).To(Equal(uintptr(120)))
+		Expect(unsafe.Offsetof(p.LogitsProcessor)).To(Equal(uintptr(120)))
+		Expect(unsafe.Offsetof(p.LogitsProcessorUserData)).To(Equal(uintptr(128)))
+		Expect(unsafe.Sizeof(p)).To(Equal(uintptr(136)))
 	})
 
 	It("cCompletion matches vllm_completion", func() {
@@ -68,6 +85,23 @@ var _ = Describe("C ABI struct mirrors", func() {
 	})
 })
 
+// Pin/mirror skew is the failure mode this backend is most exposed to: the Go
+// PODs above are hand-written against one VLLM_ABI_VERSION, and the Makefile
+// pins the vllm.cpp commit that produces it. This spec catches drift without
+// needing model weights - set VLLM_CPP_LIBRARY to a built libvllm and it binds
+// every symbol and compares the library's reported ABI against the mirrors'.
+var _ = Describe("real library ABI handshake", func() {
+	It("binds every symbol and reports the ABI the mirrors were written against", func() {
+		lib := os.Getenv("VLLM_CPP_LIBRARY")
+		if lib == "" {
+			Skip("VLLM_CPP_LIBRARY not set; skipping the real-library handshake")
+		}
+		Expect(registerLib(lib)).To(Succeed())
+		Expect(vllmABIVersion()).To(Equal(int32(abiVersion)))
+		Expect(vllmVersion()).NotTo(BeEmpty())
+	})
+})
+
 var _ = Describe("parseOptions", func() {
 	It("extracts the engine sizing knobs", func() {
 		lo := parseOptions(&pb.ModelOptions{Options: []string{
@@ -83,6 +117,129 @@ var _ = Describe("parseOptions", func() {
 		}})
 		Expect(lo).To(Equal(loadOptions{}))
 	})
+
+	It("carries a speculative_config JSON value through the legacy options list", func() {
+		// strings.Cut splits on the FIRST colon only, so a JSON object value
+		// survives the "key:value" spelling intact.
+		lo := parseOptions(&pb.ModelOptions{Options: []string{
+			`speculative_config:{"method":"mtp","num_speculative_tokens":1}`,
+		}})
+		Expect(lo.speculativeConfig).To(Equal(`{"method":"mtp","num_speculative_tokens":1}`))
+	})
+})
+
+var _ = Describe("engine_args", func() {
+	It("maps every load knob onto the C model params", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{
+			"block_size": 64,
+			"num_blocks": 1024,
+			"max_model_len": 16384,
+			"max_num_seqs": 32,
+			"max_num_batched_tokens": 8192,
+			"enable_prefix_caching": true,
+			"scheduling_policy": "lpm",
+			"tool_parser": "qwen3",
+			"reasoning_parser": "deepseek_r1",
+			"tokenizer_config": "/models/tok/tokenizer_config.json"
+		}`})
+		Expect(lo.blockSize).To(Equal(int32(64)))
+		Expect(lo.numBlocks).To(Equal(int32(1024)))
+		Expect(lo.maxModelLen).To(Equal(int32(16384)))
+		Expect(lo.maxNumSeqs).To(Equal(int32(32)))
+		Expect(lo.maxNumBatchedTokens).To(Equal(int32(8192)))
+		Expect(lo.enablePrefixCaching).To(Equal(int32(1)))
+		Expect(lo.schedulingPolicy).To(Equal("lpm"))
+		Expect(lo.toolParser).To(Equal("qwen3"))
+		Expect(lo.reasoningParser).To(Equal("deepseek_r1"))
+		Expect(lo.tokenizerConfigPath).To(Equal("/models/tok/tokenizer_config.json"))
+	})
+
+	It("re-marshals a nested speculative_config object to JSON for the engine", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{
+			"speculative_config": {"method": "mtp", "num_speculative_tokens": 1}
+		}`})
+		Expect(lo.speculativeConfig).To(MatchJSON(`{"method":"mtp","num_speculative_tokens":1}`))
+	})
+
+	It("re-marshals a nested kv_transfer_config object (LMCache) to JSON", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{
+			"kv_transfer_config": {
+				"kv_connector": "LMCacheConnector",
+				"kv_role": "kv_both",
+				"kv_connector_extra_config": {"host": "127.0.0.1", "port": 65432}
+			}
+		}`})
+		Expect(lo.kvTransferConfig).To(MatchJSON(`{
+			"kv_connector":"LMCacheConnector",
+			"kv_role":"kv_both",
+			"kv_connector_extra_config":{"host":"127.0.0.1","port":65432}
+		}`))
+	})
+
+	It("accepts a pre-encoded JSON string for the object-valued knobs", func() {
+		// A config written by hand (or round-tripped through a flat store) may
+		// carry the object as a string; both spellings reach the engine the same.
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{
+			"speculative_config": "{\"method\":\"ngram\",\"num_speculative_tokens\":4}"
+		}`})
+		Expect(lo.speculativeConfig).To(MatchJSON(`{"method":"ngram","num_speculative_tokens":4}`))
+	})
+
+	It("maps enable_prefix_caching false onto the force-OFF tri-state", func() {
+		// The C ABI tri-state is 0=model default, 1=on, 2=off, so an explicit
+		// `false` must NOT collapse to the 0 that means "let the model decide".
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{"enable_prefix_caching": false}`})
+		Expect(lo.enablePrefixCaching).To(Equal(int32(2)))
+	})
+
+	It("leaves the prefix-caching tri-state at the model default when unset", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{"max_num_seqs": 4}`})
+		Expect(lo.enablePrefixCaching).To(Equal(int32(0)))
+	})
+
+	It("accepts the radix-attention alias upstream documents for prefix caching", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{"enable_radix_attention": true}`})
+		Expect(lo.enablePrefixCaching).To(Equal(int32(1)))
+	})
+
+	It("maps enable_jump_forward onto its own tri-state", func() {
+		// ABI v10. Same tri-state shape as prefix caching, and the same trap:
+		// an explicit false must be force-OFF (2), not the 0 that defers to the
+		// environment.
+		on := parseOptions(&pb.ModelOptions{EngineArgs: `{"enable_jump_forward": true}`})
+		Expect(on.enableJumpForward).To(Equal(int32(1)))
+		off := parseOptions(&pb.ModelOptions{EngineArgs: `{"enable_jump_forward": false}`})
+		Expect(off.enableJumpForward).To(Equal(int32(2)))
+		unset := parseOptions(&pb.ModelOptions{EngineArgs: `{"max_num_seqs": 4}`})
+		Expect(unset.enableJumpForward).To(Equal(int32(0)))
+	})
+
+	It("reads enable_jump_forward from the legacy options list too", func() {
+		lo := parseOptions(&pb.ModelOptions{Options: []string{"enable_jump_forward:true"}})
+		Expect(lo.enableJumpForward).To(Equal(int32(1)))
+	})
+
+	It("lets engine_args override the legacy options list", func() {
+		lo := parseOptions(&pb.ModelOptions{
+			Options:    []string{"max_num_seqs:8", "block_size:16"},
+			EngineArgs: `{"max_num_seqs": 64}`,
+		})
+		Expect(lo.maxNumSeqs).To(Equal(int32(64))) // engine_args wins
+		Expect(lo.blockSize).To(Equal(int32(16)))  // untouched keys survive
+	})
+
+	It("ignores malformed engine_args rather than failing the load", func() {
+		lo := parseOptions(&pb.ModelOptions{
+			Options:    []string{"max_num_seqs:8"},
+			EngineArgs: `{not json`,
+		})
+		Expect(lo.maxNumSeqs).To(Equal(int32(8)))
+	})
+
+	It("ignores unknown keys", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{"gpu_memory_utilization": 0.9}`})
+		Expect(lo).To(Equal(loadOptions{}))
+	})
 })
 
 var _ = Describe("samplingFromPredict", func() {
@@ -135,6 +292,91 @@ var _ = Describe("samplingFromPredict", func() {
 	})
 })
 
+// The engine resolves speculative_config.model against a local directory or
+// ~/.cache/huggingface/hub ONLY - it never downloads. LocalAI keeps models in
+// its own directory, so a bare repo id would miss the HF cache and fail deep in
+// the load with a confusing "draft checkpoint not found". Resolve it here.
+var _ = Describe("resolveDraftModelPath", func() {
+	var modelsDir string
+
+	BeforeEach(func() {
+		modelsDir = GinkgoT().TempDir()
+	})
+
+	// draftDir creates a plausible draft checkpoint under models/.
+	draftDir := func(name string) string {
+		d := filepath.Join(modelsDir, name)
+		Expect(os.MkdirAll(d, 0o750)).To(Succeed())
+		Expect(os.WriteFile(filepath.Join(d, "config.json"), []byte("{}"), 0o600)).To(Succeed())
+		return d
+	}
+
+	It("rewrites a repo id to the matching directory in the models dir", func() {
+		want := draftDir("Qwen3.6-27B-DFlash")
+		spec := `{"method":"dflash","model":"z-lab/Qwen3.6-27B-DFlash"}`
+		out, err := resolveDraftModelPath(spec, modelsDir)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(out).To(MatchJSON(`{"method":"dflash","model":"` + want + `"}`))
+	})
+
+	It("rewrites a models-dir-relative path", func() {
+		want := draftDir("drafts__dflash")
+		spec := `{"method":"dflash","model":"drafts__dflash"}`
+		out, err := resolveDraftModelPath(spec, modelsDir)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(out).To(ContainSubstring(want))
+	})
+
+	It("leaves an absolute path that already resolves alone", func() {
+		abs := draftDir("elsewhere")
+		spec := `{"method":"dflash","model":"` + abs + `"}`
+		out, err := resolveDraftModelPath(spec, modelsDir)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(out).To(MatchJSON(spec))
+	})
+
+	It("fails with an actionable error when the draft is nowhere on disk", func() {
+		// Silently passing the repo id through would surface as an HF-cache
+		// miss inside the engine, which reads as "your model is broken".
+		spec := `{"method":"dflash","model":"z-lab/Not-Downloaded"}`
+		_, err := resolveDraftModelPath(spec, modelsDir)
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("z-lab/Not-Downloaded"))
+		Expect(err.Error()).To(ContainSubstring(modelsDir))
+	})
+
+	It("requires a model key for dflash", func() {
+		_, err := resolveDraftModelPath(`{"method":"dflash"}`, modelsDir)
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("model"))
+	})
+
+	It("leaves mtp and ngram configs untouched", func() {
+		// Neither has a separate draft checkpoint to resolve.
+		for _, spec := range []string{
+			`{"method":"mtp"}`,
+			`{"method":"ngram","num_speculative_tokens":4}`,
+		} {
+			out, err := resolveDraftModelPath(spec, modelsDir)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(out).To(MatchJSON(spec))
+		}
+	})
+
+	It("passes a malformed document through for the engine to reject", func() {
+		// The engine owns config validation and produces the better message.
+		out, err := resolveDraftModelPath(`{not json`, modelsDir)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(out).To(Equal(`{not json`))
+	})
+
+	It("is a no-op on an empty config", func() {
+		out, err := resolveDraftModelPath("", modelsDir)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(out).To(BeEmpty())
+	})
+})
+
 var _ = Describe("validModelPath", func() {
 	It("accepts a .gguf file", func() {
 		dir := GinkgoT().TempDir()

backend/go/whisper/Makefile

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
 
 # whisper.cpp version
 WHISPER_REPO?=https://github.com/ggml-org/whisper.cpp
-WHISPER_CPP_VERSION?=2ca53bb45e38748d07b310eeb36245a7157ac882
+WHISPER_CPP_VERSION?=306c88f4d1286aec1bf96e544632897886af5501
 SO_TARGET?=libgowhisper.so
 
 CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

backend/index.yaml

@@ -193,12 +193,22 @@
   alias: "vllm-cpp"
   license: apache-2.0
   description: |
-    vllm.cpp is a from-scratch C++20 port of vLLM created and maintained by the LocalAI team.
-    It mirrors vLLM's V1 architecture (paged KV cache, continuous batching, prefix caching,
-    scheduler, sampler) on a portable tensor runtime with no Python, PyTorch or ggml at
-    inference time. It loads Hugging Face safetensors and GGUF checkpoints, supports
-    structured output (JSON schema / regex / choice / GBNF grammar) enforced in-engine,
-    and runs on CPU, NVIDIA CUDA (Blackwell-family), Apple Metal and Vulkan.
+    ALPHA development builds. Try it, but llama-cpp stays the recommendation for
+    production use.
+
+    vllm.cpp is an Apache-2.0 C++20 inference engine maintained by the LocalAI team,
+    developed in its own repository and usable without LocalAI. It began as a port of
+    vLLM and keeps vLLM as its reference implementation, checking output against it and
+    benchmarking against it, while growing a featureset of its own. It implements vLLM's
+    V1 architecture (paged KV cache, continuous batching, prefix caching, scheduler,
+    sampler) on a portable tensor runtime with no Python, PyTorch or ggml at inference
+    time. It loads GGUF as well as Hugging Face safetensors, supports structured output
+    (JSON schema / regex / choice / GBNF grammar) enforced in-engine, ships speculative
+    decoding and KV offload, and runs on CPU, NVIDIA CUDA (Blackwell-family), Apple
+    Metal and Vulkan.
+
+    The project is expected to be renamed as it diverges further from vLLM; the new
+    name is still to be decided.
   urls:
     - https://github.com/mudler/vllm.cpp
   tags:

backend/python/diffusers/backend.py

@@ -883,6 +883,34 @@ class BackendServicer(backend_pb2_grpc.BackendServicer):
 
         return backend_pb2.Result(message="Media generated", success=True)
 
+    def UpscaleImage(self, request, context):
+        try:
+            if not request.src:
+                return backend_pb2.Result(success=False, message="No source image provided")
+            if not request.dst:
+                return backend_pb2.Result(success=False, message="No destination path provided")
+
+            scale = request.scale if request.scale > 0 else 2
+            image = Image.open(request.src).convert("RGB")
+
+            # If the loaded pipeline supports upscaling (e.g. StableDiffusionUpscalePipeline),
+            # use it; otherwise fall back to high-quality Lanczos resize.
+            if self.pipe is not None and self.PipelineType in ("StableDiffusionUpscalePipeline", "StableDiffusionLatentUpscalePipeline"):
+                print(f"UpscaleImage: using diffusers upscale pipeline ({self.PipelineType})", file=sys.stderr)
+                upscaled = self.pipe(prompt="", image=image).images[0]
+            else:
+                # Fallback: high-quality Lanczos resize
+                print(f"UpscaleImage: no upscale pipeline loaded, using Lanczos resize (scale={scale})", file=sys.stderr)
+                new_w = image.width * scale
+                new_h = image.height * scale
+                upscaled = image.resize((new_w, new_h), Image.LANCZOS)
+
+            upscaled.save(request.dst)
+            return backend_pb2.Result(message="Image upscaled", success=True)
+        except Exception as e:
+            print(f"UpscaleImage error: {e}", file=sys.stderr)
+            return backend_pb2.Result(success=False, message=str(e))
+
     def GenerateVideo(self, request, context):
         try:
             prompt = request.prompt

backend/python/sglang/requirements-cublas12-after.txt

@@ -15,3 +15,12 @@ sglang[all]>=0.5.11
 # load-bearing for flash-attn-4, and this is the narrower change. Raise the
 # bound once 0.46.0 final ships.
 nvidia-modelopt<0.46
+
+# Same failure mode as the nvidia-modelopt bound above, via a different
+# package. sglang -> flashinfer-python -> cuda-tile, unbounded, and the
+# global --prerelease=allow resolves it to 1.6.0rc3, whose build backend
+# imports wheel_stub without declaring it in build-system.requires. With
+# --no-build-isolation nothing installs it and the build dies with
+# "No module named 'wheel_stub'". 1.5.0 is the newest stable release.
+# Raise the bound once 1.6.0 final ships.
+cuda-tile<1.6

backend/python/sglang/requirements-cublas13-after.txt

@@ -15,3 +15,12 @@ sglang[all]>=0.5.11
 # load-bearing for flash-attn-4, and this is the narrower change. Raise the
 # bound once 0.46.0 final ships.
 nvidia-modelopt<0.46
+
+# Same failure mode as the nvidia-modelopt bound above, via a different
+# package. sglang -> flashinfer-python -> cuda-tile, unbounded, and the
+# global --prerelease=allow resolves it to 1.6.0rc3, whose build backend
+# imports wheel_stub without declaring it in build-system.requires. With
+# --no-build-isolation nothing installs it and the build dies with
+# "No module named 'wheel_stub'". 1.5.0 is the newest stable release.
+# Raise the bound once 1.6.0 final ships.
+cuda-tile<1.6

backend/python/sglang/requirements-l4t13-after.txt

@@ -13,3 +13,12 @@
 # FunctionCallParser, ReasoningParser); the [all] extras are optional
 # accelerators not required at import time.
 sglang>=0.5.11
+
+# Same failure mode the cublas profiles carry an nvidia-modelopt bound for,
+# reached through a different package. sglang -> flashinfer-python ->
+# cuda-tile, unbounded, and the global --prerelease=allow resolves it to
+# 1.6.0rc3, whose build backend imports wheel_stub without declaring it in
+# build-system.requires. With --no-build-isolation nothing installs it and
+# the build dies with "No module named 'wheel_stub'". 1.5.0 is the newest
+# stable release. Raise the bound once 1.6.0 final ships.
+cuda-tile<1.6

core/application/application.go

@@ -553,12 +553,17 @@ func (a *Application) start() error {
 	// once at startup and reused across chat sessions that opt in via metadata.
 	if !a.applicationConfig.DisableLocalAIAssistant {
 		holder := mcpTools.NewLocalAIAssistantHolder()
+		var nodeRegistry *nodes.NodeRegistry
+		if a.distributed != nil {
+			nodeRegistry = a.distributed.Registry
+		}
 		assistantClient := localaiInproc.New(
 			a.applicationConfig,
 			a.applicationConfig.SystemState,
 			a.backendLoader,
 			a.modelLoader,
 			a.galleryService,
+			nodeRegistry,
 		)
 		// Wire usage tracking so the assistant's get_usage_stats tool
 		// returns real data; nil values keep the tool returning a clear

core/backend/upscale.go

@@ -0,0 +1,37 @@
+package backend
+
+import (
+	"context"
+
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/pkg/grpc/proto"
+	model "github.com/mudler/LocalAI/pkg/model"
+)
+
+// ImageUpscale loads the model specified in modelConfig and calls UpscaleImage
+// on the backend, writing the result to dst.
+func ImageUpscale(ctx context.Context, src, dst string, scale int, loader *model.ModelLoader, modelConfig config.ModelConfig, appConfig *config.ApplicationConfig) (func() error, error) {
+	opts := ModelOptions(modelConfig, appConfig, model.WithContext(ctx))
+	inferenceModel, err := loader.Load(opts...)
+	if err != nil {
+		recordModelLoadFailure(appConfig, modelConfig.Name, modelConfig.Backend, err, nil)
+		return nil, err
+	}
+
+	fn := func() error {
+		_, err := inferenceModel.UpscaleImage(
+			ctx,
+			&proto.UpscaleImageRequest{
+				Src:   src,
+				Dst:   dst,
+				Scale: int32(scale),
+			},
+		)
+		return err
+	}
+
+	return fn, nil
+}
+
+// ImageUpscaleFunc is a test-friendly indirection.
+var ImageUpscaleFunc = ImageUpscale

core/config/backend_capabilities.go

@@ -44,6 +44,7 @@ const (
 	MethodPredictStream      GRPCMethod = "PredictStream"
 	MethodEmbedding          GRPCMethod = "Embedding"
 	MethodGenerateImage      GRPCMethod = "GenerateImage"
+	MethodUpscaleImage       GRPCMethod = "UpscaleImage"
 	MethodGenerateVideo      GRPCMethod = "GenerateVideo"
 	MethodGenerate3D         GRPCMethod = "Generate3D"
 	MethodAudioTranscription GRPCMethod = "AudioTranscription"
@@ -348,7 +349,7 @@ var BackendCapabilities = map[string]BackendCapability{
 
 	// --- Image/video generation backends ---
 	"diffusers": {
-		GRPCMethods:      []GRPCMethod{MethodGenerateImage, MethodGenerateVideo},
+		GRPCMethods:      []GRPCMethod{MethodGenerateImage, MethodUpscaleImage, MethodGenerateVideo},
 		PossibleUsecases: []string{UsecaseImage, UsecaseVideo},
 		DefaultUsecases:  []string{UsecaseImage},
 		Description:      "HuggingFace diffusers — Stable Diffusion, Flux, video generation",

core/config/vllm_spec.go

@@ -0,0 +1,117 @@
+package config
+
+// Speculative-decoding auto-defaults for the vllm-cpp backend, the safetensors
+// counterpart of the GGUF/llama.cpp hook in mtp.go.
+//
+// The two engines detect and spell the same feature differently. llama.cpp
+// reads `<arch>.nextn_predict_layers` out of the GGUF header and takes
+// `spec_type:draft-mtp` in `options:`; vllm.cpp reads `mtp_num_hidden_layers`
+// out of the checkpoint's config.json and takes vLLM's own
+// `--speculative-config` JSON, which LocalAI carries in `engine_args`. The
+// engine resolves the draft depth and the default k itself, so the config only
+// has to name the method.
+
+import (
+	"encoding/json"
+
+	"github.com/mudler/xlog"
+)
+
+// hfSpecConfig is the subset of a HuggingFace config.json that decides whether
+// speculative decoding can be auto-enabled.
+type hfSpecConfig struct {
+	ModelType string `json:"model_type"`
+	// MtpNumHiddenLayers is the MTP head depth (upstream speculative.py reads
+	// it as n_predict for the qwen3_5 / qwen3_5_moe families).
+	MtpNumHiddenLayers uint32 `json:"mtp_num_hidden_layers"`
+	// DFlashConfig marks a z-lab DFlash DRAFT checkpoint (mask_token_id +
+	// target_layer_ids). Its presence means this repo is a draft, not a
+	// servable target.
+	DFlashConfig json.RawMessage `json:"dflash_config"`
+	// TextConfig is where multimodal checkpoints nest the language-model
+	// config, and therefore the MTP depth.
+	TextConfig *hfSpecConfig `json:"text_config"`
+}
+
+// parseHFSpecConfig decodes the speculative-relevant subset of a config.json.
+// A document that does not parse yields nothing rather than an error: detection
+// is best-effort and must never break an import.
+func parseHFSpecConfig(configJSON []byte) (hfSpecConfig, bool) {
+	if len(configJSON) == 0 {
+		return hfSpecConfig{}, false
+	}
+	var c hfSpecConfig
+	if err := json.Unmarshal(configJSON, &c); err != nil {
+		xlog.Debug("[vllm-spec] config.json did not parse; skipping detection", "error", err)
+		return hfSpecConfig{}, false
+	}
+	return c, true
+}
+
+// IsDFlashDraftConfig reports whether a HuggingFace config.json describes a
+// DFlash DRAFT checkpoint. Unlike MTP - whose head ships inside the target
+// checkpoint's `mtp.*` tensors - a DFlash draft is its own repo that can only
+// run paired with a target it verifies against, so it must never be configured
+// as a standalone model.
+func IsDFlashDraftConfig(configJSON []byte) bool {
+	c, ok := parseHFSpecConfig(configJSON)
+	if !ok {
+		return false
+	}
+	return len(c.DFlashConfig) > 0 ||
+		(c.TextConfig != nil && len(c.TextConfig.DFlashConfig) > 0)
+}
+
+// HasSafetensorsMTPHead reports whether a HuggingFace config.json declares a
+// self-speculating Multi-Token Prediction head, returning its depth. The depth
+// is informational: vllm.cpp resolves n_predict and the default
+// num_speculative_tokens from the checkpoint itself.
+//
+// DFlash drafts are excluded for the same reason `gemma4-assistant` GGUFs are
+// excluded from the llama.cpp hook: they carry head metadata but cannot
+// self-speculate.
+//
+// NOTE this is a safetensors-only signal. vllm.cpp rejects an MTP config over a
+// GGUF source, because the `mtp.*` draft tensors only exist in the safetensors
+// checkpoint - so the GGUF import path must not use this.
+func HasSafetensorsMTPHead(configJSON []byte) (uint32, bool) {
+	c, ok := parseHFSpecConfig(configJSON)
+	if !ok {
+		return 0, false
+	}
+	if IsDFlashDraftConfig(configJSON) {
+		return 0, false
+	}
+	n := c.MtpNumHiddenLayers
+	if n == 0 && c.TextConfig != nil {
+		n = c.TextConfig.MtpNumHiddenLayers
+	}
+	return n, n > 0
+}
+
+// ApplyVLLMSpeculativeDefaults enables MTP speculative decoding in cfg's
+// engine_args when nothing is configured there yet. It is a no-op when the user
+// already set a speculative_config, so an explicit choice (a different method,
+// an explicit k, a DFlash draft) is never clobbered.
+//
+// `layers` is the detected head depth and is only used for the diagnostic log
+// line - the engine derives the real k from the checkpoint.
+func ApplyVLLMSpeculativeDefaults(cfg *ModelConfig, layers uint32) {
+	if cfg == nil {
+		return
+	}
+	if _, set := cfg.EngineArgs["speculative_config"]; set {
+		xlog.Debug("[vllm-spec] MTP head detected but speculative_config already configured; leaving user choice intact",
+			"name", cfg.Name, "mtp_num_hidden_layers", layers)
+		return
+	}
+	if cfg.EngineArgs == nil {
+		cfg.EngineArgs = map[string]any{}
+	}
+	// Only the method: vllm.cpp defaults num_speculative_tokens to the
+	// checkpoint's own n_predict (speculative.py:865-875), which is the right
+	// value far more reliably than anything guessable here.
+	cfg.EngineArgs["speculative_config"] = map[string]any{"method": "mtp"}
+	xlog.Info("[vllm-spec] MTP head detected; enabling mtp speculative decoding",
+		"name", cfg.Name, "mtp_num_hidden_layers", layers)
+}

core/config/vllm_spec_test.go

@@ -0,0 +1,117 @@
+package config_test
+
+import (
+	. "github.com/mudler/LocalAI/core/config"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("vllm-cpp speculative-decoding auto-defaults", func() {
+	Context("HasSafetensorsMTPHead", func() {
+		It("detects a top-level mtp_num_hidden_layers", func() {
+			n, ok := HasSafetensorsMTPHead([]byte(`{
+				"model_type": "qwen3_5_moe",
+				"mtp_num_hidden_layers": 1
+			}`))
+			Expect(ok).To(BeTrue())
+			Expect(n).To(Equal(uint32(1)))
+		})
+
+		It("detects the head nested under text_config", func() {
+			// Multimodal checkpoints nest the language-model config, which is
+			// where the MTP depth lives (mirrors the engine's own resolution
+			// off config.raw text_config).
+			n, ok := HasSafetensorsMTPHead([]byte(`{
+				"model_type": "qwen3_5_moe",
+				"text_config": {"mtp_num_hidden_layers": 2}
+			}`))
+			Expect(ok).To(BeTrue())
+			Expect(n).To(Equal(uint32(2)))
+		})
+
+		It("reports no head when the key is absent", func() {
+			n, ok := HasSafetensorsMTPHead([]byte(`{"model_type": "llama"}`))
+			Expect(ok).To(BeFalse())
+			Expect(n).To(BeZero())
+		})
+
+		It("reports no head for a zero depth", func() {
+			_, ok := HasSafetensorsMTPHead([]byte(`{"mtp_num_hidden_layers": 0}`))
+			Expect(ok).To(BeFalse())
+		})
+
+		It("ignores a DFlash draft checkpoint", func() {
+			// A DFlash draft is a SEPARATE checkpoint that cannot serve alone:
+			// it needs a target to verify against. Same exclusion the GGUF path
+			// makes for gemma4-assistant drafts.
+			_, ok := HasSafetensorsMTPHead([]byte(`{
+				"model_type": "qwen3_dflash",
+				"mtp_num_hidden_layers": 1,
+				"dflash_config": {"mask_token_id": 151666, "target_layer_ids": [0, 1]}
+			}`))
+			Expect(ok).To(BeFalse())
+		})
+
+		It("reports no head on unparseable JSON", func() {
+			_, ok := HasSafetensorsMTPHead([]byte(`{not json`))
+			Expect(ok).To(BeFalse())
+		})
+
+		It("reports no head on empty input", func() {
+			_, ok := HasSafetensorsMTPHead(nil)
+			Expect(ok).To(BeFalse())
+		})
+	})
+
+	Context("IsDFlashDraftConfig", func() {
+		It("recognises a draft by its dflash_config block", func() {
+			Expect(IsDFlashDraftConfig([]byte(`{
+				"dflash_config": {"mask_token_id": 151666, "target_layer_ids": [0]}
+			}`))).To(BeTrue())
+		})
+
+		It("does not flag an ordinary checkpoint", func() {
+			Expect(IsDFlashDraftConfig([]byte(`{"model_type": "qwen3_5_moe"}`))).To(BeFalse())
+		})
+	})
+
+	Context("ApplyVLLMSpeculativeDefaults", func() {
+		It("writes the mtp method into engine_args", func() {
+			cfg := &ModelConfig{Name: "qwen"}
+			ApplyVLLMSpeculativeDefaults(cfg, 1)
+			Expect(cfg.EngineArgs).To(HaveKey("speculative_config"))
+			spec, ok := cfg.EngineArgs["speculative_config"].(map[string]any)
+			Expect(ok).To(BeTrue())
+			Expect(spec["method"]).To(Equal("mtp"))
+		})
+
+		It("leaves an existing speculative_config alone", func() {
+			cfg := &ModelConfig{
+				Name: "qwen",
+				LLMConfig: LLMConfig{
+					EngineArgs: map[string]any{
+						"speculative_config": map[string]any{"method": "ngram", "num_speculative_tokens": 4},
+					},
+				},
+			}
+			ApplyVLLMSpeculativeDefaults(cfg, 1)
+			spec := cfg.EngineArgs["speculative_config"].(map[string]any)
+			Expect(spec["method"]).To(Equal("ngram"))
+		})
+
+		It("preserves unrelated engine_args keys", func() {
+			cfg := &ModelConfig{
+				Name:      "qwen",
+				LLMConfig: LLMConfig{EngineArgs: map[string]any{"max_num_seqs": 32}},
+			}
+			ApplyVLLMSpeculativeDefaults(cfg, 1)
+			Expect(cfg.EngineArgs).To(HaveKeyWithValue("max_num_seqs", 32))
+			Expect(cfg.EngineArgs).To(HaveKey("speculative_config"))
+		})
+
+		It("tolerates a nil config", func() {
+			Expect(func() { ApplyVLLMSpeculativeDefaults(nil, 1) }).ToNot(Panic())
+		})
+	})
+})

core/gallery/importers/llama-cpp.go

@@ -298,7 +298,15 @@ func (i *LlamaCPPImporter) Import(details Details) (gallery.ModelConfig, error)
 	// imported configs already carry spec_type:draft-mtp before the model is
 	// ever loaded - users see it in the YAML preview rather than discovering
 	// it after the first start.
-	maybeApplyMTPDefaults(&modelConfig, details, &cfg)
+	//
+	// vllm-cpp is excluded on both counts: `spec_type:*` are llama.cpp option
+	// keys it does not read, and vllm.cpp rejects an MTP config over a GGUF
+	// source outright (the `mtp.*` draft tensors exist only in the safetensors
+	// checkpoint). Its MTP auto-config runs in the vllm importer instead, over
+	// the safetensors config.json.
+	if backend != "vllm-cpp" {
+		maybeApplyMTPDefaults(&modelConfig, details, &cfg)
+	}
 
 	data, err := yaml.Marshal(modelConfig)
 	if err != nil {

core/gallery/importers/vllm.go

@@ -1,13 +1,21 @@
 package importers
 
 import (
+	"context"
 	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
 	"path/filepath"
 	"strings"
+	"time"
 
 	"github.com/mudler/LocalAI/core/config"
 	"github.com/mudler/LocalAI/core/gallery"
 	"github.com/mudler/LocalAI/core/schema"
+	"github.com/mudler/LocalAI/pkg/downloader"
+	"github.com/mudler/LocalAI/pkg/httpclient"
+	"github.com/mudler/xlog"
 	"go.yaml.in/yaml/v2"
 )
 
@@ -107,6 +115,12 @@ func (i *VLLMImporter) Import(details Details) (gallery.ModelConfig, error) {
 		// vllm python backend, so use_tokenizer_template carries over), but
 		// tool/reasoning parsing is the engine's own autoparser pipeline -
 		// the vllm-python tool_parser/reasoning_parser options don't apply.
+		//
+		// Auto-detect a Multi-Token Prediction head, the safetensors analogue
+		// of the llama-cpp importer's GGUF hook, so a freshly imported
+		// Qwen3.5 / Qwen3.6 config already carries speculative decoding in its
+		// engine_args instead of leaving the throughput on the table.
+		maybeApplyVLLMSpeculativeDefaults(&modelConfig, details)
 	} else {
 		// Auto-detect tool_parser and reasoning_parser for known model families.
 		// Surfacing them in the generated YAML lets users see and edit the choices.
@@ -132,3 +146,89 @@ func (i *VLLMImporter) Import(details Details) (gallery.ModelConfig, error) {
 		ConfigFile:  string(data),
 	}, nil
 }
+
+// maxSpecConfigProbeBytes caps the config.json body we read. Real ones are a
+// few KB; the cap keeps a hostile or mislabelled URL from streaming into the
+// importer.
+const maxSpecConfigProbeBytes = 1 << 20 // 1 MiB
+
+// specConfigProbeTimeout bounds the config.json fetch. Detection is an
+// optimisation, so it must never hold an import open for long.
+const specConfigProbeTimeout = 30 * time.Second
+
+// specConfigFetcher is the seam the config.json probe goes through, so tests can
+// drive the whole import path without a network round trip.
+var specConfigFetcher = fetchProbeBody
+
+// maybeApplyVLLMSpeculativeDefaults fetches the repository's config.json and,
+// when it declares a Multi-Token Prediction head, enables MTP speculative
+// decoding in the emitted engine_args. This is the safetensors counterpart of
+// the llama-cpp importer's GGUF header probe.
+//
+// Every failure is non-fatal and logged at debug: a network blip, a private
+// repo, or a config.json this doesn't understand must leave the import working
+// exactly as it did before, just without the speculative default.
+func maybeApplyVLLMSpeculativeDefaults(modelConfig *config.ModelConfig, details Details) {
+	probeURL := vllmSpecProbeURL(details)
+	if probeURL == "" {
+		return
+	}
+
+	body, err := specConfigFetcher(probeURL)
+	if err != nil {
+		xlog.Debug("[vllm-spec-importer] could not read config.json for MTP detection", "uri", probeURL, "error", err)
+		return
+	}
+
+	applySpecFromConfigJSON(modelConfig, body, details.URI)
+}
+
+// applySpecFromConfigJSON is the decision half of the probe, split out so it can
+// be exercised without a network round trip.
+func applySpecFromConfigJSON(modelConfig *config.ModelConfig, body []byte, uri string) {
+	if config.IsDFlashDraftConfig(body) {
+		// A DFlash draft cannot serve on its own - it only proposes tokens for
+		// a target model to verify. Say so rather than emitting a config that
+		// would fail at load.
+		xlog.Warn("[vllm-spec-importer] this repository is a DFlash DRAFT checkpoint, not a servable model; "+
+			"import the TARGET model and point engine_args.speculative_config at this repo "+
+			`({"method":"dflash","model":"<this repo>"})`, "uri", uri)
+		return
+	}
+
+	n, ok := config.HasSafetensorsMTPHead(body)
+	if !ok {
+		return
+	}
+	config.ApplyVLLMSpeculativeDefaults(modelConfig, n)
+}
+
+// vllmSpecProbeURL returns the HTTP(S) URL of the repository's config.json, or
+// "" when the import isn't backed by a HuggingFace repo we can fetch from (a
+// local directory import, an OCI artifact, ...).
+func vllmSpecProbeURL(details Details) string {
+	if details.HuggingFace == nil || details.HuggingFace.ModelID == "" {
+		return ""
+	}
+	return resolveHTTPProbe(downloader.HuggingFacePrefix + details.HuggingFace.ModelID + "/config.json")
+}
+
+// fetchProbeBody GETs a small remote JSON document under a short timeout.
+func fetchProbeBody(url string) ([]byte, error) {
+	ctx, cancel := context.WithTimeout(context.Background(), specConfigProbeTimeout)
+	defer cancel()
+
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
+	if err != nil {
+		return nil, err
+	}
+	resp, err := httpclient.NewWithTimeout(specConfigProbeTimeout).Do(req)
+	if err != nil {
+		return nil, err
+	}
+	defer func() { _ = resp.Body.Close() }()
+	if resp.StatusCode != http.StatusOK {
+		return nil, fmt.Errorf("unexpected status %d", resp.StatusCode)
+	}
+	return io.ReadAll(io.LimitReader(resp.Body, maxSpecConfigProbeBytes))
+}

core/gallery/importers/vllm_spec_internal_test.go

@@ -0,0 +1,118 @@
+package importers
+
+import (
+	"encoding/json"
+	"errors"
+
+	"github.com/mudler/LocalAI/core/config"
+	hfapi "github.com/mudler/LocalAI/pkg/huggingface-api"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("vllm-cpp speculative auto-config (importer)", func() {
+	Context("applySpecFromConfigJSON", func() {
+		It("enables mtp when the checkpoint declares an MTP head", func() {
+			cfg := &config.ModelConfig{Name: "qwen3.5"}
+			applySpecFromConfigJSON(cfg, []byte(`{
+				"model_type": "qwen3_5_moe",
+				"mtp_num_hidden_layers": 1
+			}`), "huggingface://Qwen/Qwen3.5-A3B")
+			Expect(cfg.EngineArgs).To(HaveKeyWithValue("speculative_config",
+				map[string]any{"method": "mtp"}))
+		})
+
+		It("leaves a plain checkpoint untouched", func() {
+			cfg := &config.ModelConfig{Name: "llama"}
+			applySpecFromConfigJSON(cfg, []byte(`{"model_type": "llama"}`), "huggingface://meta/llama")
+			Expect(cfg.EngineArgs).To(BeEmpty())
+		})
+
+		It("refuses to configure a DFlash draft as a servable model", func() {
+			// The draft only proposes tokens; configuring it standalone would
+			// produce a model that cannot load.
+			cfg := &config.ModelConfig{Name: "dflash-draft"}
+			applySpecFromConfigJSON(cfg, []byte(`{
+				"model_type": "qwen3_dflash",
+				"dflash_config": {"mask_token_id": 151666, "target_layer_ids": [0, 1]}
+			}`), "huggingface://z-lab/Qwen3.6-27B-DFlash")
+			Expect(cfg.EngineArgs).To(BeEmpty())
+		})
+
+		It("survives a config.json it cannot parse", func() {
+			cfg := &config.ModelConfig{Name: "weird"}
+			Expect(func() {
+				applySpecFromConfigJSON(cfg, []byte(`<html>404</html>`), "huggingface://a/b")
+			}).ToNot(Panic())
+			Expect(cfg.EngineArgs).To(BeEmpty())
+		})
+	})
+
+	Context("Import over a repository with an MTP head", func() {
+		var restore func()
+
+		BeforeEach(func() {
+			original := specConfigFetcher
+			restore = func() { specConfigFetcher = original }
+		})
+		AfterEach(func() { restore() })
+
+		importWith := func(backend, configJSON string) string {
+			specConfigFetcher = func(string) ([]byte, error) {
+				return []byte(configJSON), nil
+			}
+			importer := &VLLMImporter{}
+			out, err := importer.Import(Details{
+				URI:         "huggingface://Qwen/Qwen3.5-A3B",
+				Preferences: json.RawMessage(`{"backend": "` + backend + `"}`),
+				HuggingFace: &hfapi.ModelDetails{ModelID: "Qwen/Qwen3.5-A3B"},
+			})
+			Expect(err).ToNot(HaveOccurred())
+			return out.ConfigFile
+		}
+
+		It("emits engine_args.speculative_config for vllm-cpp", func() {
+			yaml := importWith("vllm-cpp", `{"model_type":"qwen3_5_moe","mtp_num_hidden_layers":1}`)
+			Expect(yaml).To(ContainSubstring("engine_args:"))
+			Expect(yaml).To(ContainSubstring("speculative_config:"))
+			Expect(yaml).To(ContainSubstring("method: mtp"))
+		})
+
+		It("emits nothing speculative for the python vllm backend", func() {
+			// The python backend has its own speculative surface and its own
+			// version-dependent MTP support; this hook is vllm-cpp only.
+			yaml := importWith("vllm", `{"model_type":"qwen3_5_moe","mtp_num_hidden_layers":1}`)
+			Expect(yaml).NotTo(ContainSubstring("speculative_config"))
+		})
+
+		It("emits nothing speculative when the probe fails", func() {
+			specConfigFetcher = func(string) ([]byte, error) {
+				return nil, errors.New("network down")
+			}
+			importer := &VLLMImporter{}
+			out, err := importer.Import(Details{
+				URI:         "huggingface://Qwen/Qwen3.5-A3B",
+				Preferences: json.RawMessage(`{"backend": "vllm-cpp"}`),
+				HuggingFace: &hfapi.ModelDetails{ModelID: "Qwen/Qwen3.5-A3B"},
+			})
+			Expect(err).ToNot(HaveOccurred())
+			Expect(out.ConfigFile).NotTo(ContainSubstring("speculative_config"))
+		})
+	})
+
+	Context("vllmSpecProbeURL", func() {
+		It("resolves the repository's config.json to an HTTPS URL", func() {
+			url := vllmSpecProbeURL(Details{
+				URI:         "huggingface://Qwen/Qwen3.5-A3B",
+				HuggingFace: &hfapi.ModelDetails{ModelID: "Qwen/Qwen3.5-A3B"},
+			})
+			Expect(url).To(ContainSubstring("Qwen/Qwen3.5-A3B"))
+			Expect(url).To(HaveSuffix("config.json"))
+			Expect(url).To(HavePrefix("https://"))
+		})
+
+		It("skips the probe when there is no HuggingFace repo behind the import", func() {
+			Expect(vllmSpecProbeURL(Details{URI: "/models/local-dir"})).To(BeEmpty())
+		})
+	})
+})

core/http/auth/features.go

@@ -39,6 +39,8 @@ var RouteFeatureRegistry = []RouteFeature{
 	{"POST", "/images/generations", FeatureImages},
 	{"POST", "/v1/images/inpainting", FeatureImages},
 	{"POST", "/images/inpainting", FeatureImages},
+	{"POST", "/v1/images/upscale", FeatureImages},
+	{"POST", "/images/upscale", FeatureImages},
 
 	// Audio transcription
 	{"POST", "/v1/audio/transcriptions", FeatureAudioTranscription},
@@ -116,6 +118,10 @@ var RouteFeatureRegistry = []RouteFeature{
 	// Rerank
 	{"POST", "/v1/rerank", FeatureRerank},
 
+	// Moderation
+	{"POST", "/v1/moderations", FeatureModeration},
+	{"POST", "/moderations", FeatureModeration},
+
 	// Stores
 	{"POST", "/stores/set", FeatureStores},
 	{"POST", "/stores/delete", FeatureStores},
@@ -191,6 +197,7 @@ func APIFeatureMetas() []FeatureMeta {
 		{FeatureEmbeddings, "Embeddings", true},
 		{FeatureSound, "Sound Generation", true},
 		{FeatureRealtime, "Realtime", true},
+		{FeatureModeration, "Moderation", true},
 		{FeatureRerank, "Rerank", true},
 		{FeatureTokenize, "Tokenize", true},
 		{FeatureMCP, "MCP", true},

core/http/auth/features_moderation_test.go

@@ -0,0 +1,24 @@
+package auth_test
+
+import (
+	. "github.com/mudler/LocalAI/core/http/auth"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Moderation feature registration", func() {
+	It("registers both moderation routes as default-on API features", func() {
+		Expect(APIFeatures).To(ContainElement(FeatureModeration))
+
+		patterns := []string{}
+		for _, route := range RouteFeatureRegistry {
+			if route.Feature == FeatureModeration {
+				patterns = append(patterns, route.Pattern)
+			}
+		}
+		Expect(patterns).To(ConsistOf("/v1/moderations", "/moderations"))
+
+		metas := APIFeatureMetas()
+		Expect(metas).To(ContainElement(FeatureMeta{Key: FeatureModeration, Label: "Moderation", DefaultValue: true}))
+	})
+})

core/http/auth/helpers_test.go

@@ -59,10 +59,14 @@ func ok(c echo.Context) error {
 func newAuthTestApp(db *gorm.DB, appConfig *config.ApplicationConfig) *echo.Echo {
 	e := echo.New()
 	e.Use(auth.Middleware(db, appConfig))
+	if db != nil {
+		e.Use(auth.RequireRouteFeature(db))
+	}
 
 	// API routes (require auth)
 	e.GET("/v1/models", ok)
 	e.POST("/v1/chat/completions", ok)
+	e.POST("/v1/moderations", ok)
 	e.GET("/api/settings", ok)
 	e.POST("/api/settings", ok)
 
@@ -81,10 +85,14 @@ func newAuthTestApp(db *gorm.DB, appConfig *config.ApplicationConfig) *echo.Echo
 func newAdminTestApp(db *gorm.DB, appConfig *config.ApplicationConfig) *echo.Echo {
 	e := echo.New()
 	e.Use(auth.Middleware(db, appConfig))
+	if db != nil {
+		e.Use(auth.RequireRouteFeature(db))
+	}
 
 	// Regular routes
 	e.GET("/v1/models", ok)
 	e.POST("/v1/chat/completions", ok)
+	e.POST("/v1/moderations", ok)
 
 	// Admin-only routes
 	adminMw := auth.RequireAdmin()

core/http/auth/middleware_test.go

@@ -91,6 +91,19 @@ var _ = Describe("Auth Middleware", func() {
 			Expect(rec.Code).To(Equal(http.StatusOK))
 		})
 
+		It("allows authenticated users to call moderation by default", func() {
+			sessionID := createTestSession(db, user.ID)
+			rec := doRequest(app, http.MethodPost, "/v1/moderations", withSessionCookie(sessionID))
+			Expect(rec.Code).To(Equal(http.StatusOK))
+		})
+
+		It("blocks moderation when the user's feature is disabled", func() {
+			Expect(auth.UpdateUserPermissions(db, user.ID, auth.PermissionMap{auth.FeatureModeration: false})).To(Succeed())
+			sessionID := createTestSession(db, user.ID)
+			rec := doRequest(app, http.MethodPost, "/v1/moderations", withSessionCookie(sessionID))
+			Expect(rec.Code).To(Equal(http.StatusForbidden))
+		})
+
 		It("allows requests with valid session as Bearer token", func() {
 			sessionID := createTestSession(db, user.ID)
 			rec := doRequest(app, http.MethodGet, "/v1/models", withBearerToken(sessionID))
@@ -156,6 +169,11 @@ var _ = Describe("Auth Middleware", func() {
 			Expect(rec.Code).To(Equal(http.StatusUnauthorized))
 		})
 
+		It("returns 401 for unauthenticated moderation requests", func() {
+			rec := doRequest(app, http.MethodPost, "/v1/moderations")
+			Expect(rec.Code).To(Equal(http.StatusUnauthorized))
+		})
+
 		It("returns 401 for unauthenticated 3D generation requests", func() {
 			rec := doRequest(app, http.MethodPost, "/3d/generations")
 			Expect(rec.Code).To(Equal(http.StatusUnauthorized))

core/http/auth/permissions.go

@@ -51,6 +51,7 @@ const (
 	FeatureEmbeddings          = "embeddings"
 	FeatureSound               = "sound"
 	FeatureRealtime            = "realtime"
+	FeatureModeration          = "moderation"
 	FeatureRerank              = "rerank"
 	FeatureTokenize            = "tokenize"
 	FeatureMCP                 = "mcp"
@@ -75,7 +76,7 @@ var APIFeatures = []string{
 	FeatureChat, FeatureImages, FeatureAudioSpeech, FeatureAudioTranscription,
 	FeatureAudioDiarization, FeatureAudioClassification,
 	FeatureVAD, FeatureDetection, FeatureVideo, Feature3D, FeatureEmbeddings, FeatureSound,
-	FeatureRealtime, FeatureRerank, FeatureTokenize, FeatureMCP, FeatureStores,
+	FeatureRealtime, FeatureModeration, FeatureRerank, FeatureTokenize, FeatureMCP, FeatureStores,
 	FeatureFaceRecognition, FeatureVoiceRecognition, FeatureAudioTransform,
 	FeaturePIIFilter,
 }

core/http/endpoints/localai/api_instructions.go

@@ -30,6 +30,12 @@ var instructionDefs = []instructionDef{
 		Tags:        []string{"inference", "embeddings"},
 		Intro:       "Set \"stream\": true for SSE streaming. Supports tool/function calling when the model config has function templates configured.",
 	},
+	{
+		Name:        "moderation",
+		Description: "OpenAI-compatible text moderation using a local completion model",
+		Tags:        []string{"moderation"},
+		Intro:       "POST /v1/moderations accepts a text string or array plus a LocalAI completion model. LocalAI constrains the model to the OpenAI moderation category schema and returns one result per input. Multimodal moderation inputs are not yet supported.",
+	},
 	{
 		Name:        "audio",
 		Description: "Text-to-speech, voice activity detection, transcription, speaker diarization, sound classification, and sound generation",

core/http/endpoints/localai/api_instructions_test.go

@@ -39,7 +39,7 @@ var _ = Describe("API Instructions Endpoints", func() {
 
 			instructions, ok := resp["instructions"].([]any)
 			Expect(ok).To(BeTrue())
-			Expect(instructions).To(HaveLen(18))
+			Expect(instructions).To(HaveLen(19))
 
 			// Verify each instruction has required fields and correct URL format
 			for _, s := range instructions {
@@ -69,6 +69,7 @@ var _ = Describe("API Instructions Endpoints", func() {
 
 			Expect(names).To(ContainElements(
 				"chat-inference",
+				"moderation",
 				"config-management",
 				"model-management",
 				"monitoring",

core/http/endpoints/localai/system.go

@@ -12,7 +12,7 @@ import (
 // @Tags monitoring
 // @Success 200 {object} schema.SystemInformationResponse "Response"
 // @Router /system [get]
-func SystemInformations(ml *model.ModelLoader, appConfig *config.ApplicationConfig) echo.HandlerFunc {
+func SystemInformations(cl *config.ModelConfigLoader, ml *model.ModelLoader, appConfig *config.ApplicationConfig) echo.HandlerFunc {
 	return func(c echo.Context) error {
 		availableBackends := []string{}
 		loadedModels := ml.ListLoadedModels()
@@ -25,7 +25,14 @@ func SystemInformations(ml *model.ModelLoader, appConfig *config.ApplicationConf
 
 		sysmodels := []schema.SysInfoModel{}
 		for _, m := range loadedModels {
-			sysmodels = append(sysmodels, schema.SysInfoModel{ID: m.ID})
+			entry := schema.SysInfoModel{ID: m.ID}
+			// The loader tracks only the ID. Which engine is serving a model is
+			// the first thing an operator wants beside its name, and it is one
+			// config lookup away.
+			if cfg, ok := cl.GetModelConfig(m.ID); ok {
+				entry.Backend = cfg.Backend
+			}
+			sysmodels = append(sysmodels, entry)
 		}
 		return c.JSON(200,
 			schema.SystemInformationResponse{

core/http/endpoints/localai/traces.go

@@ -3,6 +3,7 @@ package localai
 import (
 	"net/http"
 	"strconv"
+	"time"
 
 	"github.com/labstack/echo/v4"
 	"github.com/mudler/LocalAI/core/http/middleware"
@@ -85,6 +86,35 @@ func GetAPITracesEndpoint() echo.HandlerFunc {
 	}
 }
 
+// GetAPITracesSummaryEndpoint returns counted totals over a recent window
+// @Summary Summarize recent API traces
+// @Description Returns request, failure and latency totals over a recent window, plus a bucketed series for sparklines. Exists so callers wanting three numbers do not have to fetch the whole trace list and count it themselves.
+// @Tags monitoring
+// @Produce json
+// @Param hours query int false "Window in hours (default 24, max 168)"
+// @Success 200 {object} middleware.TraceSummary "Counted trace totals"
+// @Router /api/traces/summary [get]
+func GetAPITracesSummaryEndpoint() echo.HandlerFunc {
+	return func(c echo.Context) error {
+		hours := 24
+		if raw := c.QueryParam("hours"); raw != "" {
+			if v, err := strconv.Atoi(raw); err == nil && v > 0 {
+				hours = v
+			}
+		}
+		// A week is plenty for a dashboard, and the trace buffer is bounded
+		// anyway; an unbounded window would just scan the whole buffer.
+		if hours > 168 {
+			hours = 168
+		}
+		return c.JSON(http.StatusOK, middleware.GetTracesSummary(time.Duration(hours)*time.Hour, traceSummaryBuckets))
+	}
+}
+
+// Enough columns for a sparkline to show a shape, few enough that each one
+// still holds a meaningful count on a quiet installation.
+const traceSummaryBuckets = 12
+
 // GetAPITraceEndpoint returns a single API trace with its full payload
 // @Summary Get one API trace
 // @Description Returns a single captured API exchange, including the request and response bodies omitted from the list response

core/http/endpoints/mcp/localai_assistant_test.go

@@ -84,6 +84,22 @@ func (stubClient) ListNodes(_ context.Context) ([]localaitools.Node, error) {
 	return []localaitools.Node{}, nil
 }
 
+func (stubClient) ListScheduling(_ context.Context) ([]localaitools.ModelSchedulingConfig, error) {
+	return []localaitools.ModelSchedulingConfig{}, nil
+}
+
+func (stubClient) GetScheduling(_ context.Context, _ string) (*localaitools.ModelSchedulingConfig, error) {
+	return &localaitools.ModelSchedulingConfig{}, nil
+}
+
+func (stubClient) SetScheduling(_ context.Context, _ localaitools.SetSchedulingRequest) (*localaitools.ModelSchedulingConfig, error) {
+	return &localaitools.ModelSchedulingConfig{}, nil
+}
+
+func (stubClient) DeleteScheduling(_ context.Context, _ string) error {
+	return nil
+}
+
 func (stubClient) SetNodeVRAMBudget(_ context.Context, _, _ string) error {
 	return nil
 }

core/http/endpoints/openai/moderations.go

@@ -0,0 +1,190 @@
+package openai
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"math"
+	"net/http"
+	"strings"
+
+	"github.com/google/uuid"
+	"github.com/labstack/echo/v4"
+	"github.com/mudler/LocalAI/core/backend"
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/http/middleware"
+	"github.com/mudler/LocalAI/core/schema"
+	"github.com/mudler/LocalAI/core/templates"
+	"github.com/mudler/LocalAI/pkg/functions"
+	"github.com/mudler/LocalAI/pkg/model"
+)
+
+var moderationCategories = []string{
+	"harassment",
+	"harassment/threatening",
+	"hate",
+	"hate/threatening",
+	"illicit",
+	"illicit/violent",
+	"self-harm",
+	"self-harm/intent",
+	"self-harm/instructions",
+	"sexual",
+	"sexual/minors",
+	"violence",
+	"violence/graphic",
+}
+
+type moderationGenerator func(context.Context, string, *config.ModelConfig) (string, backend.TokenUsage, error)
+
+type generatedModeration struct {
+	Categories     map[string]bool    `json:"categories"`
+	CategoryScores map[string]float64 `json:"category_scores"`
+}
+
+// ModerationEndpoint implements the text input subset of OpenAI's moderation
+// API using any LocalAI completion model and constrained JSON generation.
+// @Summary Classify text for potentially harmful content.
+// @Tags moderation
+// @Param request body schema.ModerationRequest true "query params"
+// @Success 200 {object} schema.ModerationResponse "Response"
+// @Router /v1/moderations [post]
+func ModerationEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoader, evaluator *templates.Evaluator, appConfig *config.ApplicationConfig) echo.HandlerFunc {
+	return moderationEndpoint(func(ctx context.Context, input string, cfg *config.ModelConfig) (string, backend.TokenUsage, error) {
+		prompt := moderationPrompt(input)
+		var messages schema.Messages
+		if cfg.TemplateConfig.UseTokenizerTemplate {
+			messages = schema.Messages{{Role: "user", Content: prompt}}
+			prompt = ""
+		} else if evaluator != nil {
+			if rendered, err := evaluator.EvaluateTemplateForPrompt(templates.CompletionPromptTemplate, *cfg, templates.PromptTemplateData{Input: prompt, SystemPrompt: cfg.SystemPrompt}); err == nil {
+				prompt = rendered
+			}
+		}
+
+		predict, err := backend.ModelInferenceFunc(ctx, prompt, messages, nil, nil, nil, ml, cfg, cl, appConfig, nil, "", "", nil, nil, nil, nil)
+		if err != nil {
+			return "", backend.TokenUsage{}, err
+		}
+		response, err := predict()
+		return response.Response, response.Usage, err
+	})
+}
+
+func moderationEndpoint(generate moderationGenerator) echo.HandlerFunc {
+	return func(c echo.Context) error {
+		input, ok := c.Get(middleware.CONTEXT_LOCALS_KEY_LOCALAI_REQUEST).(*schema.ModerationRequest)
+		if !ok || input == nil {
+			return echo.NewHTTPError(http.StatusBadRequest, "invalid moderation request")
+		}
+		if len(input.Input) == 0 {
+			return echo.NewHTTPError(http.StatusBadRequest, "input must contain at least one text string")
+		}
+		if generate == nil {
+			return echo.NewHTTPError(http.StatusInternalServerError, "moderation generator is unavailable")
+		}
+
+		modelConfig, ok := c.Get(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG).(*config.ModelConfig)
+		if !ok || modelConfig == nil {
+			return echo.NewHTTPError(http.StatusBadRequest, "moderation model configuration is unavailable")
+		}
+
+		grammar, err := moderationGrammar()
+		if err != nil {
+			return echo.NewHTTPError(http.StatusInternalServerError, "failed to build moderation grammar").SetInternal(err)
+		}
+		cfg := *modelConfig
+		cfg.Grammar = grammar
+		maxTokens := 512
+		cfg.Maxtokens = &maxTokens
+
+		results := make([]schema.ModerationResult, 0, len(input.Input))
+		promptTokens, completionTokens := 0, 0
+		for _, text := range input.Input {
+			if strings.TrimSpace(text) == "" {
+				return echo.NewHTTPError(http.StatusBadRequest, "input strings must not be empty")
+			}
+			raw, usage, err := generate(c.Request().Context(), text, &cfg)
+			if err != nil {
+				return echo.NewHTTPError(http.StatusInternalServerError, "moderation inference failed").SetInternal(err)
+			}
+			result, err := parseModerationResult(raw)
+			if err != nil {
+				return echo.NewHTTPError(http.StatusInternalServerError, "invalid moderation result: "+err.Error()).SetInternal(err)
+			}
+			results = append(results, result)
+			promptTokens += usage.Prompt
+			completionTokens += usage.Completion
+		}
+
+		middleware.StampUsage(c, input.Model, promptTokens, completionTokens)
+		return c.JSON(http.StatusOK, schema.ModerationResponse{
+			ID:      "modr-" + uuid.NewString(),
+			Model:   input.Model,
+			Results: results,
+		})
+	}
+}
+
+func moderationPrompt(input string) string {
+	encoded, _ := json.Marshal(input)
+	return `Classify the supplied text for safety moderation. Return only the JSON object required by the grammar.
+For each category, set the boolean to true only when the text matches that category. Give each category a confidence score from 0 to 1.
+The categories are harassment, harassment/threatening, hate, hate/threatening, illicit, illicit/violent, self-harm, self-harm/intent, self-harm/instructions, sexual, sexual/minors, violence, and violence/graphic.
+Text to classify: ` + string(encoded)
+}
+
+func moderationGrammar() (string, error) {
+	boolProperties := map[string]any{}
+	scoreProperties := map[string]any{}
+	for _, category := range moderationCategories {
+		boolProperties[category] = map[string]any{"type": "boolean"}
+		scoreProperties[category] = map[string]any{"type": "number"}
+	}
+	structure := functions.JSONFunctionStructure{AnyOf: []functions.Item{{
+		Type: "object",
+		Properties: map[string]any{
+			"categories": map[string]any{
+				"type":                 "object",
+				"properties":           boolProperties,
+				"required":             moderationCategories,
+				"additionalProperties": false,
+			},
+			"category_scores": map[string]any{
+				"type":                 "object",
+				"properties":           scoreProperties,
+				"required":             moderationCategories,
+				"additionalProperties": false,
+			},
+		},
+	}}}
+	return structure.Grammar()
+}
+
+func parseModerationResult(raw string) (schema.ModerationResult, error) {
+	var generated generatedModeration
+	if err := json.Unmarshal([]byte(strings.TrimSpace(raw)), &generated); err != nil {
+		return schema.ModerationResult{}, err
+	}
+
+	result := schema.ModerationResult{
+		Categories:                make(map[string]bool, len(moderationCategories)),
+		CategoryScores:            make(map[string]float64, len(moderationCategories)),
+		CategoryAppliedInputTypes: make(map[string][]string, len(moderationCategories)),
+	}
+	for _, category := range moderationCategories {
+		flagged, exists := generated.Categories[category]
+		if !exists {
+			return schema.ModerationResult{}, fmt.Errorf("missing category %q", category)
+		}
+		score, exists := generated.CategoryScores[category]
+		if !exists || math.IsNaN(score) || math.IsInf(score, 0) || score < 0 || score > 1 {
+			return schema.ModerationResult{}, fmt.Errorf("category %q has an invalid score", category)
+		}
+		result.Categories[category] = flagged
+		result.CategoryScores[category] = score
+		result.CategoryAppliedInputTypes[category] = []string{"text"}
+		result.Flagged = result.Flagged || flagged
+	}
+	return result, nil
+}

core/http/endpoints/openai/moderations_test.go

@@ -0,0 +1,105 @@
+package openai
+
+import (
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+
+	"github.com/labstack/echo/v4"
+	"github.com/mudler/LocalAI/core/backend"
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/http/middleware"
+	"github.com/mudler/LocalAI/core/schema"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Moderations endpoint", func() {
+	It("classifies each text input and returns the OpenAI response shape", func() {
+		inputs := []string{}
+		generate := func(_ context.Context, input string, cfg *config.ModelConfig) (string, backend.TokenUsage, error) {
+			inputs = append(inputs, input)
+			Expect(cfg.Grammar).To(ContainSubstring("harassment"))
+			return `{
+				"categories":{"harassment":true,"harassment/threatening":false,"hate":false,"hate/threatening":false,"illicit":false,"illicit/violent":false,"self-harm":false,"self-harm/intent":false,"self-harm/instructions":false,"sexual":false,"sexual/minors":false,"violence":false,"violence/graphic":false},
+				"category_scores":{"harassment":0.9,"harassment/threatening":0.1,"hate":0,"hate/threatening":0,"illicit":0,"illicit/violent":0,"self-harm":0,"self-harm/intent":0,"self-harm/instructions":0,"sexual":0,"sexual/minors":0,"violence":0,"violence/graphic":0}
+			}`, backend.TokenUsage{Prompt: 12, Completion: 8}, nil
+		}
+
+		e := echo.New()
+		req := httptest.NewRequest(http.MethodPost, "/v1/moderations", strings.NewReader(`{"model":"guard","input":["first","second"]}`))
+		req.Header.Set(echo.HeaderContentType, echo.MIMEApplicationJSON)
+		rec := httptest.NewRecorder()
+		ctx := e.NewContext(req, rec)
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_LOCALAI_REQUEST, &schema.ModerationRequest{
+			BasicModelRequest: schema.BasicModelRequest{Model: "guard"},
+			Input:             schema.ModerationInput{"first", "second"},
+		})
+		modelConfig := &config.ModelConfig{Name: "guard"}
+		modelConfig.Model = "guard.gguf"
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG, modelConfig)
+
+		Expect(moderationEndpoint(generate)(ctx)).To(Succeed())
+		Expect(rec.Code).To(Equal(http.StatusOK))
+		Expect(inputs).To(Equal([]string{"first", "second"}))
+
+		var response schema.ModerationResponse
+		Expect(json.Unmarshal(rec.Body.Bytes(), &response)).To(Succeed())
+		Expect(response.ID).To(HavePrefix("modr-"))
+		Expect(response.Model).To(Equal("guard"))
+		Expect(response.Results).To(HaveLen(2))
+		Expect(response.Results[0].Flagged).To(BeTrue())
+		Expect(response.Results[0].Categories["harassment"]).To(BeTrue())
+		Expect(response.Results[0].CategoryAppliedInputTypes["harassment"]).To(Equal([]string{"text"}))
+	})
+
+	It("rejects an empty input list", func() {
+		e := echo.New()
+		ctx := e.NewContext(httptest.NewRequest(http.MethodPost, "/v1/moderations", nil), httptest.NewRecorder())
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_LOCALAI_REQUEST, &schema.ModerationRequest{
+			BasicModelRequest: schema.BasicModelRequest{Model: "guard"},
+		})
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG, &config.ModelConfig{Name: "guard"})
+
+		err := moderationEndpoint(nil)(ctx)
+		Expect(err).To(MatchError(ContainSubstring("input must contain at least one text string")))
+		Expect(err.(*echo.HTTPError).Code).To(Equal(http.StatusBadRequest))
+	})
+
+	It("surfaces malformed classifier output without returning a partial result", func() {
+		generate := func(context.Context, string, *config.ModelConfig) (string, backend.TokenUsage, error) {
+			return "not-json", backend.TokenUsage{}, nil
+		}
+		e := echo.New()
+		ctx := e.NewContext(httptest.NewRequest(http.MethodPost, "/v1/moderations", nil), httptest.NewRecorder())
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_LOCALAI_REQUEST, &schema.ModerationRequest{
+			BasicModelRequest: schema.BasicModelRequest{Model: "guard"},
+			Input:             schema.ModerationInput{"text"},
+		})
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG, &config.ModelConfig{Name: "guard"})
+
+		err := moderationEndpoint(generate)(ctx)
+		Expect(err).To(MatchError(ContainSubstring("invalid moderation result")))
+		Expect(err.(*echo.HTTPError).Code).To(Equal(http.StatusInternalServerError))
+	})
+})
+
+var _ = Describe("Moderation input", func() {
+	DescribeTable("accepts OpenAI text input forms",
+		func(body string, expected schema.ModerationInput) {
+			var req schema.ModerationRequest
+			Expect(json.Unmarshal([]byte(body), &req)).To(Succeed())
+			Expect(req.Input).To(Equal(expected))
+		},
+		Entry("single text", `{"input":"hello"}`, schema.ModerationInput{"hello"}),
+		Entry("text array", `{"input":["hello","world"]}`, schema.ModerationInput{"hello", "world"}),
+	)
+
+	It("rejects multimodal input in the text-only MVP", func() {
+		var req schema.ModerationRequest
+		err := json.Unmarshal([]byte(`{"input":[{"type":"image_url","image_url":{"url":"https://example.com/a.png"}}]}`), &req)
+		Expect(err).To(MatchError(ContainSubstring("text string or array of text strings")))
+	})
+})

core/http/endpoints/openai/upscale.go

@@ -0,0 +1,134 @@
+package openai
+
+import (
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"os"
+	"path/filepath"
+	"strconv"
+	"time"
+
+	"github.com/google/uuid"
+	"github.com/labstack/echo/v4"
+	"github.com/mudler/xlog"
+
+	"github.com/mudler/LocalAI/core/backend"
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/http/middleware"
+	"github.com/mudler/LocalAI/core/schema"
+	model "github.com/mudler/LocalAI/pkg/model"
+)
+
+// UpscaleEndpoint handles POST /v1/images/upscale
+//
+// @Summary      Image upscaling
+// @Description  Upscale an image using a specified model (e.g. stable-diffusion-x4-upscaler). Accepts multipart/form-data.
+// @Tags         images
+// @Accept       multipart/form-data
+// @Produce      application/json
+// @Param        model   formData  string  true   "Upscaler model identifier (e.g. stable-diffusion-x4-upscaler)"
+// @Param        image   formData  file    true   "Input image file"
+// @Param        scale   formData  int     false  "Upscale factor: 2 or 4 (default 2)"
+// @Success      200 {object} schema.OpenAIResponse
+// @Failure      400 {object} map[string]string
+// @Failure      500 {object} map[string]string
+// @Router       /v1/images/upscale [post]
+func UpscaleEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoader, appConfig *config.ApplicationConfig) echo.HandlerFunc {
+	return func(c echo.Context) error {
+		modelName := c.FormValue("model")
+		scaleStr := c.FormValue("scale")
+
+		if modelName == "" {
+			xlog.Error("Upscale Endpoint - missing model")
+			return echo.NewHTTPError(http.StatusBadRequest, "missing model")
+		}
+
+		scale := 2
+		if scaleStr != "" {
+			v, err := strconv.Atoi(scaleStr)
+			if err != nil || (v != 2 && v != 4) {
+				return echo.NewHTTPError(http.StatusBadRequest, "scale must be 2 or 4")
+			}
+			scale = v
+		}
+
+		// Read uploaded image
+		imageFile, err := c.FormFile("image")
+		if err != nil {
+			xlog.Error("Upscale Endpoint - missing image file", "error", err)
+			return echo.NewHTTPError(http.StatusBadRequest, "missing image file")
+		}
+
+		imgSrc, err := imageFile.Open()
+		if err != nil {
+			return err
+		}
+		defer imgSrc.Close()
+		imgBytes, err := io.ReadAll(imgSrc)
+		if err != nil {
+			return err
+		}
+
+		// Get model config from middleware context
+		cfg, ok := c.Get(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG).(*config.ModelConfig)
+		if !ok || cfg == nil {
+			xlog.Error("Upscale Endpoint - model config not found in context")
+			return echo.ErrBadRequest
+		}
+
+		tmpDir := filepath.Join(appConfig.GeneratedContentDir, "images")
+		if err := os.MkdirAll(tmpDir, 0750); err != nil {
+			return echo.NewHTTPError(http.StatusInternalServerError, "failed to prepare storage")
+		}
+
+		// Write input image to a temp file
+		srcTmp, err := os.CreateTemp(tmpDir, "upscale_src_")
+		if err != nil {
+			return err
+		}
+		if _, err := srcTmp.Write(imgBytes); err != nil {
+			_ = srcTmp.Close()
+			_ = os.Remove(srcTmp.Name())
+			return err
+		}
+		if err := srcTmp.Close(); err != nil {
+			xlog.Warn("Upscale Endpoint - failed to close src temp file", "error", err)
+		}
+		srcPath := srcTmp.Name()
+		defer os.Remove(srcPath)
+
+		// Prepare output file path
+		id := uuid.New().String()
+		dstPath := filepath.Join(tmpDir, fmt.Sprintf("upscale_%s.png", id))
+
+		fn, err := backend.ImageUpscaleFunc(c.Request().Context(), srcPath, dstPath, scale, ml, *cfg, appConfig)
+		if err != nil {
+			return err
+		}
+		if err := fn(); err != nil {
+			_ = os.Remove(dstPath)
+			return err
+		}
+
+		baseURL := middleware.BaseURL(c)
+		imgURL, err := url.JoinPath(baseURL, "generated-images", filepath.Base(dstPath))
+		if err != nil {
+			_ = os.Remove(dstPath)
+			return err
+		}
+
+		created := int(time.Now().Unix())
+		resp := &schema.OpenAIResponse{
+			ID:      id,
+			Created: created,
+			Data:    []schema.Item{{URL: imgURL}},
+			Usage: &schema.OpenAIUsage{
+				InputTokensDetails: &schema.InputTokensDetails{},
+			},
+		}
+
+		return c.JSON(http.StatusOK, resp)
+	}
+}

core/http/endpoints/openai/upscale_test.go

@@ -0,0 +1,89 @@
+package openai
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"path/filepath"
+
+	"github.com/labstack/echo/v4"
+	"github.com/mudler/LocalAI/core/backend"
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/http/middleware"
+	"github.com/mudler/LocalAI/core/schema"
+	model "github.com/mudler/LocalAI/pkg/model"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Image upscaling", func() {
+	var (
+		appConfig *config.ApplicationConfig
+		tmpDir    string
+	)
+
+	BeforeEach(func() {
+		var err error
+		tmpDir, err = os.MkdirTemp("", "upscale")
+		Expect(err).ToNot(HaveOccurred())
+		appConfig = config.NewApplicationConfig(config.WithGeneratedContentDir(tmpDir))
+	})
+
+	AfterEach(func() {
+		Expect(os.RemoveAll(tmpDir)).To(Succeed())
+	})
+
+	It("stores the result in the directory served by /generated-images", func() {
+		original := backend.ImageUpscaleFunc
+		backend.ImageUpscaleFunc = func(_ context.Context, _, dst string, scale int, _ *model.ModelLoader, _ config.ModelConfig, _ *config.ApplicationConfig) (func() error, error) {
+			Expect(scale).To(Equal(4))
+			return func() error {
+				return os.WriteFile(dst, []byte("PNGDATA"), 0o644)
+			}, nil
+		}
+		DeferCleanup(func() { backend.ImageUpscaleFunc = original })
+
+		req, _ := makeMultipartRequest(
+			map[string]string{"model": "stable-diffusion-x4-upscaler", "scale": "4"},
+			map[string][]byte{"image": []byte("IMAGEDATA")},
+		)
+		rec := httptest.NewRecorder()
+		ctx := echo.New().NewContext(req, rec)
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG, &config.ModelConfig{Backend: "diffusers"})
+
+		Expect(UpscaleEndpoint(nil, nil, appConfig)(ctx)).To(Succeed())
+		Expect(rec.Code).To(Equal(http.StatusOK))
+
+		var response schema.OpenAIResponse
+		Expect(json.Unmarshal(rec.Body.Bytes(), &response)).To(Succeed())
+		Expect(response.Data).To(HaveLen(1))
+		Expect(response.Data[0].URL).To(ContainSubstring("/generated-images/upscale_"))
+
+		filename := filepath.Base(response.Data[0].URL)
+		contents, err := os.ReadFile(filepath.Join(tmpDir, "images", filename))
+		Expect(err).ToNot(HaveOccurred())
+		Expect(contents).To(Equal([]byte("PNGDATA")))
+	})
+
+	It("rejects unsupported scale factors", func() {
+		req, _ := makeMultipartRequest(
+			map[string]string{"model": "stable-diffusion-x4-upscaler", "scale": "3"},
+			map[string][]byte{"image": []byte("IMAGEDATA")},
+		)
+		rec := httptest.NewRecorder()
+		ctx := echo.New().NewContext(req, rec)
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG, &config.ModelConfig{Backend: "diffusers"})
+
+		err := UpscaleEndpoint(nil, nil, appConfig)(ctx)
+		var httpErr *echo.HTTPError
+		Expect(err).To(MatchError(ContainSubstring("scale must be 2 or 4")))
+		Expect(err).To(BeAssignableToTypeOf(httpErr))
+		httpErr = err.(*echo.HTTPError)
+		Expect(httpErr.Code).To(Equal(http.StatusBadRequest))
+		Expect(httpErr.Message).To(Equal("scale must be 2 or 4"))
+		Expect(bytes.TrimSpace(rec.Body.Bytes())).To(BeEmpty())
+	})
+})

core/http/middleware/trace.go

@@ -60,6 +60,7 @@ type APIExchange struct {
 }
 
 var traceBuffer *circularbuffer.Queue[APIExchange]
+var inFlightTraces = make(map[string]APIExchange)
 var mu sync.Mutex
 var logChan = make(chan traceCommand, 100)
 var traceIDSeq atomic.Uint64
@@ -126,16 +127,17 @@ func initializeTracing(dataPath string, maxItems int) {
 					continue
 				}
 				exchange := *command.exchange
+				mu.Lock()
+				delete(inFlightTraces, exchange.ID)
+				if traceBuffer != nil {
+					traceBuffer.Enqueue(exchange)
+				}
+				mu.Unlock()
 				if command.store != nil {
 					if err := command.store.Append(exchange.ID, exchange); err != nil {
 						xlog.Warn("Failed to persist API trace", "error", err)
 					}
 				}
-				mu.Lock()
-				if traceBuffer != nil {
-					traceBuffer.Enqueue(exchange)
-				}
-				mu.Unlock()
 			}
 		}()
 	})
@@ -261,6 +263,38 @@ func TraceMiddleware(app *application.Application) echo.MiddlewareFunc {
 			// tens of MB, which then locks the admin Traces UI fetching the
 			// JSON dump faster than the 5s auto-refresh.
 			maxBodyBytes := app.ApplicationConfig().TracingMaxBodyBytes
+			requestHeaders := redactSensitiveHeaders(c.Request().Header)
+			requestBody, requestTruncated := truncateForTrace(body, maxBodyBytes)
+			exchange := APIExchange{
+				ID:        nextTraceID(),
+				Timestamp: startTime,
+				ClientIP:  c.RealIP(),
+				UserAgent: c.Request().UserAgent(),
+				Request: APIExchangeRequest{
+					Method:        c.Request().Method,
+					Path:          c.Path(),
+					Headers:       &requestHeaders,
+					Body:          &requestBody,
+					BodyTruncated: requestTruncated,
+					BodyBytes:     len(body),
+				},
+			}
+			if user := auth.GetUser(c); user != nil {
+				exchange.UserID = user.ID
+				exchange.UserName = user.Name
+			}
+			mu.Lock()
+			inFlightTraces[exchange.ID] = exchange
+			mu.Unlock()
+			queued := false
+			defer func() {
+				if queued {
+					return
+				}
+				mu.Lock()
+				delete(inFlightTraces, exchange.ID)
+				mu.Unlock()
+			}()
 
 			// Wrap response writer to capture body
 			resBody := new(bytes.Buffer)
@@ -287,47 +321,27 @@ func TraceMiddleware(app *application.Application) echo.MiddlewareFunc {
 			// the trace endpoint is admin-only but the buffer is also reachable
 			// via any heap-dump-style introspection, and tokens shouldn't
 			// outlive the request that carried them.
-			requestHeaders := redactSensitiveHeaders(c.Request().Header)
-			requestBody, requestTruncated := truncateForTrace(body, maxBodyBytes)
 			responseHeaders := redactSensitiveHeaders(c.Response().Header())
 			responseBody := make([]byte, resBody.Len())
 			copy(responseBody, resBody.Bytes())
-			exchange := APIExchange{
-				ID:        nextTraceID(),
-				Timestamp: startTime,
-				Duration:  time.Since(startTime),
-				ClientIP:  c.RealIP(),
-				UserAgent: c.Request().UserAgent(),
-				Request: APIExchangeRequest{
-					Method:        c.Request().Method,
-					Path:          c.Path(),
-					Headers:       &requestHeaders,
-					Body:          &requestBody,
-					BodyTruncated: requestTruncated,
-					BodyBytes:     len(body),
-				},
-				Response: APIExchangeResponse{
-					Status:        status,
-					Headers:       &responseHeaders,
-					Body:          &responseBody,
-					BodyTruncated: mw.truncated,
-					BodyBytes:     mw.totalBytes,
-				},
+			exchange.Duration = time.Since(startTime)
+			exchange.Response = APIExchangeResponse{
+				Status:        status,
+				Headers:       &responseHeaders,
+				Body:          &responseBody,
+				BodyTruncated: mw.truncated,
+				BodyBytes:     mw.totalBytes,
 			}
 			if handlerErr != nil {
 				exchange.Error = handlerErr.Error()
 			}
 
-			if user := auth.GetUser(c); user != nil {
-				exchange.UserID = user.ID
-				exchange.UserName = user.Name
-			}
-
 			mu.Lock()
 			store := traceStore
 			mu.Unlock()
 			select {
 			case logChan <- traceCommand{exchange: &exchange, store: store}:
+				queued = true
 			default:
 				xlog.Warn("Trace channel full, dropping trace")
 			}
@@ -345,6 +359,10 @@ func GetTraces() []APIExchange {
 		return []APIExchange{}
 	}
 	traces := traceBuffer.Values()
+	for _, exchange := range inFlightTraces {
+		exchange.Duration = time.Since(exchange.Timestamp)
+		traces = append(traces, exchange)
+	}
 	mu.Unlock()
 
 	slices.SortFunc(traces, func(a, b APIExchange) int {

core/http/middleware/trace_live_test.go

@@ -0,0 +1,108 @@
+// SPDX-License-Identifier: MIT
+
+package middleware
+
+import (
+	"net/http"
+	"net/http/httptest"
+	"time"
+
+	"github.com/labstack/echo/v4"
+	"github.com/mudler/LocalAI/core/application"
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/pkg/system"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("live API traces", func() {
+	newApp := func(root string) *application.Application {
+		app, err := application.New(
+			config.EnableTracing,
+			config.WithDataPath(root),
+			config.WithDisableLocalAIAssistant(true),
+			config.WithDisableStats(true),
+			config.WithSystemState(&system.SystemState{
+				Model:   system.Model{ModelsPath: root},
+				Backend: system.Backend{BackendsPath: root},
+			}),
+		)
+		Expect(err).NotTo(HaveOccurred())
+		DeferCleanup(func() { Expect(app.Shutdown()).To(Succeed()) })
+		ClearTraces()
+		return app
+	}
+
+	It("lists a request while its handler is still running", func() {
+		root := GinkgoT().TempDir()
+		app := newApp(root)
+
+		started := make(chan struct{})
+		release := make(chan struct{})
+		DeferCleanup(func() {
+			select {
+			case <-release:
+			default:
+				close(release)
+			}
+		})
+		handler := TraceMiddleware(app)(func(c echo.Context) error {
+			close(started)
+			<-release
+			return c.NoContent(http.StatusNoContent)
+		})
+
+		e := echo.New()
+		req := httptest.NewRequest(http.MethodPost, "/slow", http.NoBody)
+		req.Header.Set(echo.HeaderContentType, echo.MIMEApplicationJSON)
+		rec := httptest.NewRecorder()
+		ctx := e.NewContext(req, rec)
+		ctx.SetPath("/slow")
+		done := make(chan error, 1)
+		go func() {
+			done <- handler(ctx)
+		}()
+		<-started
+
+		var running APIExchange
+		Eventually(func() bool {
+			traces := GetTraces()
+			if len(traces) != 1 {
+				return false
+			}
+			running = traces[0]
+			return running.Request.Path == "/slow"
+		}).Should(BeTrue())
+		Expect(running.Response.Status).To(Equal(0))
+		Expect(running.Duration).To(BeNumerically(">", 0))
+
+		close(release)
+		Expect(<-done).To(Succeed())
+		Eventually(func() []APIExchange { return GetTraces() }).Should(ConsistOf(
+			And(
+				HaveField("ID", running.ID),
+				HaveField("Response.Status", http.StatusNoContent),
+				HaveField("Duration", BeNumerically(">", time.Duration(0))),
+			),
+		))
+	})
+
+	It("removes an in-flight trace when the handler panics", func() {
+		app := newApp(GinkgoT().TempDir())
+		handler := TraceMiddleware(app)(func(echo.Context) error {
+			panic("handler panic")
+		})
+		e := echo.New()
+		req := httptest.NewRequest(http.MethodPost, "/panic", http.NoBody)
+		req.Header.Set(echo.HeaderContentType, echo.MIMEApplicationJSON)
+		ctx := e.NewContext(req, httptest.NewRecorder())
+		ctx.SetPath("/panic")
+
+		func() {
+			defer func() { _ = recover() }()
+			_ = handler(ctx)
+		}()
+
+		Expect(GetTraces()).To(BeEmpty())
+	})
+})

core/http/middleware/trace_summary.go

@@ -0,0 +1,109 @@
+// SPDX-License-Identifier: MIT
+
+package middleware
+
+import (
+	"math"
+	"slices"
+	"time"
+)
+
+// TraceSummary is the counted view of the trace buffer.
+//
+// It exists so a caller that wants "how many, how many failed, how slow" does
+// not have to fetch every exchange and count them in the browser. The Operate
+// overview needs exactly those three numbers, and the trace list is capped in
+// the thousands, so shipping it across the wire to produce a single integer is
+// waste that grows with the buffer.
+type TraceSummary struct {
+	Total       int           `json:"total"`
+	Errors      int           `json:"errors"`
+	P95Millis   int64         `json:"p95_ms"`
+	WindowHours int           `json:"window_hours"`
+	Buckets     []TraceBucket `json:"buckets"`
+}
+
+// TraceBucket is one column of a sparkline: oldest first, so the series reads
+// left to right the way a chart is drawn.
+type TraceBucket struct {
+	Start  time.Time `json:"start"`
+	Count  int       `json:"count"`
+	Errors int       `json:"errors"`
+}
+
+// GetTracesSummary counts the buffered exchanges over the given window.
+func GetTracesSummary(window time.Duration, buckets int) TraceSummary {
+	return summarize(GetTraces(), window, buckets)
+}
+
+func summarize(traces []APIExchange, window time.Duration, buckets int) TraceSummary {
+	if buckets < 1 {
+		buckets = 1
+	}
+	now := time.Now()
+	cutoff := now.Add(-window)
+
+	summary := TraceSummary{
+		WindowHours: int(window.Hours()),
+		// Never nil: a nil slice serialises as null and breaks .map() on the
+		// other side, which is a silent runtime error rather than an empty chart.
+		Buckets: make([]TraceBucket, buckets),
+	}
+
+	bucketWidth := window / time.Duration(buckets)
+	for i := range summary.Buckets {
+		summary.Buckets[i].Start = cutoff.Add(time.Duration(i) * bucketWidth)
+	}
+
+	durations := make([]time.Duration, 0, len(traces))
+	for _, t := range traces {
+		if t.Timestamp.Before(cutoff) {
+			continue
+		}
+		summary.Total++
+		failed := isFailure(t)
+		if failed {
+			summary.Errors++
+		}
+		durations = append(durations, t.Duration)
+
+		// Clamp rather than skip: a request timestamped a hair in the future
+		// (clock skew, or arriving mid-call) still belongs in the newest column.
+		idx := int(t.Timestamp.Sub(cutoff) / bucketWidth)
+		if idx >= buckets {
+			idx = buckets - 1
+		}
+		if idx < 0 {
+			idx = 0
+		}
+		summary.Buckets[idx].Count++
+		if failed {
+			summary.Buckets[idx].Errors++
+		}
+	}
+
+	summary.P95Millis = percentileMillis(durations, 0.95)
+	return summary
+}
+
+// A 4xx is the caller getting it wrong, which is not the installation being
+// unhealthy. Only 5xx and a transport-level error count against the runtime.
+func isFailure(t APIExchange) bool {
+	return t.Error != "" || t.Response.Status >= 500
+}
+
+func percentileMillis(durations []time.Duration, p float64) int64 {
+	if len(durations) == 0 {
+		return 0
+	}
+	slices.Sort(durations)
+	// Nearest-rank: the smallest value at or above the pth percentile.
+	rank := int(math.Ceil(p*float64(len(durations)))) - 1
+	if rank < 0 {
+		rank = 0
+	}
+	if rank >= len(durations) {
+		rank = len(durations) - 1
+	}
+	return durations[rank].Milliseconds()
+}

core/http/middleware/trace_summary_test.go

@@ -0,0 +1,79 @@
+// SPDX-License-Identifier: MIT
+
+package middleware
+
+import (
+	"time"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("API trace summary", func() {
+	exchange := func(age time.Duration, status int, dur time.Duration) APIExchange {
+		return APIExchange{
+			Timestamp: time.Now().Add(-age),
+			Duration:  dur,
+			Response:  APIExchangeResponse{Status: status},
+		}
+	}
+
+	It("counts only what falls inside the window", func() {
+		traces := []APIExchange{
+			exchange(1*time.Hour, 200, 10*time.Millisecond),
+			exchange(2*time.Hour, 200, 10*time.Millisecond),
+			// Older than the window: must not be counted at all.
+			exchange(48*time.Hour, 500, 10*time.Millisecond),
+		}
+		s := summarize(traces, 24*time.Hour, 6)
+		Expect(s.Total).To(Equal(2))
+		Expect(s.Errors).To(BeZero())
+	})
+
+	It("treats 5xx and a transport error as failures, but not 4xx", func() {
+		traces := []APIExchange{
+			exchange(time.Minute, 500, time.Millisecond),
+			exchange(time.Minute, 503, time.Millisecond),
+			// A client sending a bad request is not the server failing.
+			exchange(time.Minute, 404, time.Millisecond),
+			exchange(time.Minute, 200, time.Millisecond),
+		}
+		traces[3].Error = "connection reset"
+
+		s := summarize(traces, 24*time.Hour, 6)
+		Expect(s.Total).To(Equal(4))
+		Expect(s.Errors).To(Equal(3))
+	})
+
+	It("reports p95 as a real percentile rather than the slowest request", func() {
+		traces := make([]APIExchange, 0, 100)
+		for i := 1; i <= 100; i++ {
+			traces = append(traces, exchange(time.Minute, 200, time.Duration(i)*time.Millisecond))
+		}
+		s := summarize(traces, 24*time.Hour, 6)
+		// 95th of 1..100ms, not the 100ms max.
+		Expect(s.P95Millis).To(BeNumerically("~", 95, 1))
+	})
+
+	It("buckets oldest-first so a sparkline reads left to right", func() {
+		traces := []APIExchange{
+			exchange(30*time.Minute, 200, time.Millisecond),
+			exchange(30*time.Minute, 200, time.Millisecond),
+			exchange(5*time.Hour, 200, time.Millisecond),
+		}
+		s := summarize(traces, 6*time.Hour, 6)
+		Expect(s.Buckets).To(HaveLen(6))
+		Expect(s.Buckets[0].Count).To(Equal(1), "the 5h-old request lands in the first bucket")
+		Expect(s.Buckets[5].Count).To(Equal(2), "the recent pair lands in the last")
+	})
+
+	It("returns an empty, non-nil summary when nothing has been traced", func() {
+		s := summarize(nil, 24*time.Hour, 6)
+		Expect(s.Total).To(BeZero())
+		Expect(s.Errors).To(BeZero())
+		Expect(s.P95Millis).To(BeZero())
+		// A nil slice serialises as null and breaks .map() in the browser.
+		Expect(s.Buckets).NotTo(BeNil())
+		Expect(s.Buckets).To(HaveLen(6))
+	})
+})

core/http/react-ui/e2e/activity-page.spec.js

@@ -43,6 +43,40 @@ test('lists live operations and cancels one from a labelled button', async ({ pa
   expect(cancelledPath).toBe('/api/operations/job-gemma/cancel')
 })
 
+test('pauses a model download without invoking destructive cancel', async ({ page }) => {
+  await stub(page, {
+    operations: [{
+      id: 'gemma-3-27b-it',
+      name: 'gemma-3-27b-it',
+      jobID: 'job-gemma',
+      progress: 22,
+      taskType: 'installation',
+      isBackend: false,
+      isQueued: false,
+      isDeletion: false,
+      cancellable: true,
+      phase: 'downloading',
+    }],
+  })
+
+  const requests = []
+  await page.route('**/api/operations/job-gemma/pause', (route) => {
+    requests.push(new URL(route.request().url()).pathname)
+    return route.fulfill({ contentType: 'application/json', body: '{}' })
+  })
+  await page.route('**/api/operations/job-gemma/cancel', (route) => {
+    requests.push(new URL(route.request().url()).pathname)
+    return route.fulfill({ contentType: 'application/json', body: '{}' })
+  })
+
+  await page.goto('/app/activity')
+
+  const card = page.locator('.operation-card').filter({ hasText: 'gemma-3-27b-it' })
+  await card.locator('.operation-card__pause').click()
+
+  await expect.poll(() => requests).toEqual(['/api/operations/job-gemma/pause'])
+})
+
 test('separates an unacknowledged failure from the record', async ({ page }) => {
   await stub(page, {
     operations: [{

core/http/react-ui/e2e/admin-console.spec.js

@@ -5,7 +5,9 @@ test.describe('Admin console', () => {
     await page.goto('/app/backends')
     const rail = page.locator('.console-rail')
     await expect(rail).toBeVisible()
-    for (const group of ['Inference', 'Cluster', 'Observability', 'Access', 'System']) {
+    // Four groups since the overview landed: Inference folded into Runtime
+    // (both are "the runtime right now"), Access and System into Administration.
+    for (const group of ['Runtime', 'Cluster', 'Observability', 'Administration']) {
       await expect(rail.locator('.console-group-title', { hasText: group })).toBeVisible()
     }
   })

core/http/react-ui/e2e/backends-notice.spec.js

@@ -0,0 +1,23 @@
+import { test, expect } from './coverage-fixtures.js'
+
+// A notice is a hairline with a coloured left edge, not a filled panel. A tint
+// makes every notice shout at the weight of an error, which is how notices stop
+// being read — and it is the same treatment the Operate overview uses for the
+// rows that want a decision.
+
+test('the backends notice is an edge, not a filled card', async ({ page }) => {
+  // The upgrade banner is the notice worth pinning, so make one exist.
+  await page.route('**/api/backends/upgrades', route => route.fulfill({
+    json: { 'llama-cpp': { backend_name: 'llama-cpp', installed_version: '0.9.4', available_version: '0.9.7' } },
+  }))
+  await page.goto('/app/backends')
+  const notice = page.locator('.bk-notice', { hasText: /update/i }).first()
+  await expect(notice).toBeVisible()
+  const s = await notice.evaluate(el => {
+    const cs = getComputedStyle(el)
+    return { bg: cs.backgroundColor, left: parseFloat(cs.borderLeftWidth), top: parseFloat(cs.borderTopWidth) }
+  })
+  expect(s.bg).toMatch(/rgba\(0, 0, 0, 0\)|transparent/)
+  expect(s.left).toBeGreaterThanOrEqual(3)
+  expect(s.top).toBeLessThanOrEqual(1)
+})

core/http/react-ui/e2e/chat-transcript.spec.js

@@ -0,0 +1,55 @@
+import { test, expect } from './coverage-fixtures.js'
+
+// Chat reads as a transcript rather than a bubble thread (mock 04).
+
+const CHAT = {
+  chats: [{
+    id: 'c1', name: 'Transcript', model: 'mock-model',
+    history: [
+      { role: 'user', content: 'Which backends do I have?' },
+      { role: 'assistant', content: 'Seven are installed.' },
+    ],
+  }],
+  activeChatId: 'c1',
+}
+
+test.describe('Chat transcript', () => {
+  test.beforeEach(async ({ page }) => {
+    await page.addInitScript(chat => {
+      localStorage.setItem('localai_chats_data', JSON.stringify(chat))
+    }, CHAT)
+    await page.goto('/app/chat')
+  })
+
+  test('neither role is a filled, rounded bubble', async ({ page }) => {
+    const user = page.locator('.chat-message-user .chat-message-content').first()
+    await expect(user).toBeVisible()
+    const cs = await user.evaluate(el => {
+      const s = getComputedStyle(el)
+      return { radius: s.borderTopLeftRadius, shadow: s.boxShadow }
+    })
+    // A rounded filled bubble carries the speaker in shape and side; a
+    // transcript carries it in words, which survives being read aloud.
+    expect(cs.radius).toBe('0px')
+    expect(cs.shadow).toBe('none')
+  })
+
+  test('both turns run full width in one column, not left and right', async ({ page }) => {
+    const user = page.locator('.chat-message-user').first()
+    const assistant = page.locator('.chat-message-assistant').first()
+    const [u, a] = [await user.boundingBox(), await assistant.boundingBox()]
+    expect(Math.abs(u.x - a.x)).toBeLessThan(2)
+  })
+
+  test('every turn says who is speaking', async ({ page }) => {
+    await expect(page.locator('.chat-message-user .chat-message-model')).toHaveText('You')
+    await expect(page.locator('.chat-message-assistant .chat-message-model').first())
+      .toHaveText('mock-model')
+  })
+
+  test('turns are separated by a rule', async ({ page }) => {
+    const border = await page.locator('.chat-message').first()
+      .evaluate(el => getComputedStyle(el).borderBottomStyle)
+    expect(border).toBe('solid')
+  })
+})

core/http/react-ui/e2e/chrome-audit.spec.js

@@ -0,0 +1,50 @@
+import { test, expect } from './coverage-fixtures.js'
+
+// A standing guard against the two defects an earlier automated edit left
+// scattered through the pages: icons stripped of their fa-* class (which render
+// nothing at all), and controls left with the user agent's own chrome, which is
+// a pale grey button on a dark ground.
+const ROUTES = [
+  '/app', '/app/chat', '/app/models', '/app/studio', '/app/talk',
+  '/app/agents', '/app/skills', '/app/collections', '/app/agent-jobs',
+  '/app/fine-tune', '/app/quantize', '/app/face', '/app/voice',
+  '/app/manage', '/app/backends', '/app/activity', '/app/operate',
+  '/app/settings', '/app/traces', '/app/usage', '/app/nodes', '/app/p2p',
+  '/app/voice-library', '/app/voice-library/new', '/app/account',
+]
+
+test('no page renders a dead icon or a default-chrome control', async ({ page }) => {
+  // One test walks every route, so its budget has to scale with the list rather
+  // than sit on Playwright's per-test default of 30s. At 25 routes that default
+  // allows ~1.2s per navigation, which holds on a developer machine and does
+  // not on a loaded CI runner: the suite went red on the commit that added this
+  // spec, timing out mid-loop at waitForTimeout rather than at any single goto,
+  // which is what cumulative slowness looks like as opposed to one hung route.
+  // Six seconds a route absorbs a slow runner and still fails promptly if a
+  // route really does hang.
+  test.setTimeout(ROUTES.length * 6_000)
+
+  const findings = []
+  for (const route of ROUTES) {
+    await page.goto(route)
+    await page.waitForTimeout(400)
+    const found = await page.evaluate(() => {
+      const out = []
+      for (const el of document.querySelectorAll('button, a')) {
+        if (el.getBoundingClientRect().width === 0) continue
+        const cs = getComputedStyle(el)
+        if (cs.borderTopStyle === 'outset' || cs.backgroundColor === 'rgb(239, 239, 239)') {
+          out.push(`default-chrome: "${(el.textContent || '').trim().slice(0, 24)}" [${el.className}]`)
+        }
+      }
+      for (const i of document.querySelectorAll('i')) {
+        if (!/\bfa-/.test((i.className || '').toString())) {
+          out.push(`dead-icon: [${i.className}]`)
+        }
+      }
+      return [...new Set(out)]
+    })
+    for (const f of found) findings.push(`${route} — ${f}`)
+  }
+  expect(findings).toEqual([])
+})

core/http/react-ui/e2e/console-narrow.spec.js

@@ -0,0 +1,101 @@
+import { test, expect } from './coverage-fixtures.js'
+
+// Small-screen behaviour of the Operate console and the dashboard stat cards.
+//
+// Both defects here are about a narrow viewport but neither is only a narrow
+// viewport problem: the stat cards were being laid out by the wrong rule at
+// every width, and the rail's height was never bounded.
+
+test.describe('Operate console on a narrow screen', () => {
+  test('expanding the rail leaves the page still on screen', async ({ page }) => {
+    await page.setViewportSize({ width: 390, height: 800 })
+    await page.goto('/app/manage')
+
+    const toggle = page.locator('.console-rail-toggle')
+    await expect(toggle).toBeVisible()
+    await toggle.click()
+    await expect(page.locator('.console-rail-groups')).toBeVisible()
+
+    // Thirteen destinations in one column is taller than a phone. If opening
+    // the menu pushes the page's own heading past the fold, the menu has
+    // replaced the page instead of annotating it.
+    // Manage titles itself with .view-bar__title rather than .page-title.
+    const heading = page.locator('.page-title, .view-bar__title').first()
+    const box = await heading.boundingBox()
+    expect(box).not.toBeNull()
+    expect(box.y).toBeLessThan(800)
+  })
+
+  test('the rail scrolls internally rather than growing without bound', async ({ page }) => {
+    await page.setViewportSize({ width: 390, height: 800 })
+    await page.goto('/app/manage')
+    await page.locator('.console-rail-toggle').click()
+
+    const groups = page.locator('.console-rail-groups')
+    await expect(groups).toBeVisible()
+    const height = await groups.evaluate(el => el.getBoundingClientRect().height)
+    expect(height).toBeLessThan(800)
+  })
+})
+
+test.describe('Headline figures', () => {
+  // Host used shadowed StatCards; it now shares the Operate overview's hairline
+  // figure strip, so the guard is that its labels stay legible, not that it
+  // keeps a card gap.
+  for (const width of [768, 1024]) {
+    test(`Host figure labels are not clipped at ${width}px`, async ({ page }) => {
+      await page.setViewportSize({ width, height: 1000 })
+      await page.goto('/app/manage')
+      const labels = page.locator('.stat-strip__label')
+      await expect(labels.first()).toBeVisible()
+      const clipped = await labels.evaluateAll(els =>
+        els.filter(el => el.scrollWidth > el.clientWidth + 1).map(el => el.textContent))
+      expect(clipped).toEqual([])
+    })
+  }
+
+  test('a Host figure routes into the thing it counts', async ({ page }) => {
+    await page.setViewportSize({ width: 1280, height: 1000 })
+    await page.goto('/app/manage')
+    const cell = page.locator('.stat-strip__cell').first()
+    await expect(cell).toBeVisible()
+    // A count is worth more when it is also the way to what it counted.
+    await expect(cell).toHaveJSProperty('tagName', 'BUTTON')
+  })
+
+  test('the figure strip keeps its height inside the flex column', async ({ page }) => {
+    // .page--app is a flex column whose split view takes flex:1, so a child
+    // with no intrinsic minimum gets shrunk to nothing. This strip did exactly
+    // that and rendered 2px tall with four invisible cells.
+    await page.setViewportSize({ width: 1440, height: 900 })
+    await page.goto('/app/manage')
+    const strip = page.locator('.manage-summary')
+    await expect(strip).toBeVisible()
+    const h = await strip.evaluate(el => el.getBoundingClientRect().height)
+    expect(h).toBeGreaterThan(40)
+  })
+})
+
+test.describe('Headline figure contrast', () => {
+  test('every figure is legible against the cell it sits on', async ({ page }) => {
+    // A <button> does not inherit colour, so a value with no tone rule fell
+    // back to the UA's `buttontext` — pure black on the dark ground, invisible.
+    await page.setViewportSize({ width: 1440, height: 950 })
+    await page.goto('/app/manage')
+    const bad = await page.locator('.stat-strip__value').evaluateAll(els => els
+      .map(el => ({ text: el.textContent, color: getComputedStyle(el).color }))
+      .filter(v => v.color === 'rgb(0, 0, 0)'))
+    expect(bad).toEqual([])
+  })
+
+  test('the strip keeps its top margin against the shared shorthand', async ({ page }) => {
+    // `.stat-strip` declares `margin: 0 0 ...` later in the file, which was
+    // silently resetting this element's top margin and leaving it flush
+    // against the resources panel above it.
+    await page.setViewportSize({ width: 1440, height: 950 })
+    await page.goto('/app/manage')
+    const top = await page.locator('.manage-summary')
+      .evaluate(el => parseFloat(getComputedStyle(el).marginTop))
+    expect(top).toBeGreaterThan(12)
+  })
+})

core/http/react-ui/e2e/home-lanes.spec.js

@@ -0,0 +1,103 @@
+import { test, expect } from './coverage-fixtures.js'
+
+// Home's resident-model list and the app footer.
+
+const SYS_INFO = {
+  backends: ['llama-cpp'],
+  loaded_models: [
+    { id: 'qwen3-8b-instruct', backend: 'llama-cpp' },
+    { id: 'parakeet-tdt-0.6b' },
+  ],
+}
+
+async function mockLoaded(page) {
+  await page.route('**/system', route => route.fulfill({ json: SYS_INFO }))
+  await page.route('**/v1/models', route =>
+    route.fulfill({ json: { data: [{ id: 'qwen3-8b-instruct' }, { id: 'parakeet-tdt-0.6b' }] } }))
+}
+
+test.describe('Home resident models', () => {
+  test('resident models read as lanes, not status chips', async ({ page }) => {
+    await mockLoaded(page)
+    await page.goto('/app')
+    const lanes = page.locator('.home-loaded .lane')
+    await expect(lanes).toHaveCount(2)
+    // Model ids are identifiers, so they are set in mono like every other
+    // identifier in the app.
+    const family = await lanes.first().locator('.lane__name').evaluate(
+      el => getComputedStyle(el).fontFamily.toLowerCase())
+    expect(family).toMatch(/mono|consol|menlo/)
+  })
+
+  test('each lane keeps its stop control', async ({ page }) => {
+    await mockLoaded(page)
+    await page.goto('/app')
+    const lane = page.locator('.home-loaded .lane').first()
+    await expect(lane.getByRole('button', { name: /stop/i })).toBeVisible()
+  })
+
+  test('the header reports how many are resident as a figure', async ({ page }) => {
+    await mockLoaded(page)
+    await page.goto('/app')
+    const stat = page.locator('[data-testid="home-stat-loaded"]')
+    await expect(stat).toBeVisible()
+    await expect(stat).toContainText('2')
+    // Digits that sit in a column need to line up.
+    const numeric = await stat.locator('.home-stat__value').evaluate(
+      el => getComputedStyle(el).fontVariantNumeric)
+    expect(numeric).toContain('tabular-nums')
+  })
+
+  test('a resident model names the engine serving it', async ({ page }) => {
+    await mockLoaded(page)
+    await page.goto('/app')
+    // Lanes are sorted by id, so target by content rather than position.
+    const qwen = page.locator('.home-loaded .lane', { hasText: 'qwen3-8b-instruct' })
+    await expect(qwen).toContainText('llama-cpp')
+  })
+
+  test('a model without a config shows no engine rather than a guess', async ({ page }) => {
+    await mockLoaded(page)
+    await page.goto('/app')
+    // parakeet has no backend in the payload; the column stays blank.
+    const parakeet = page.locator('.home-loaded .lane', { hasText: 'parakeet-tdt-0.6b' })
+    await expect(parakeet).not.toContainText('llama-cpp')
+  })
+
+  test('jump-back-in offers the three places worth returning to', async ({ page }) => {
+    await mockLoaded(page)
+    await page.goto('/app')
+    const lanes = page.locator('.lanes--jump .lane')
+    await expect(lanes).toHaveCount(3)
+    await expect(lanes.first()).toContainText('Discover')
+  })
+
+  test('nothing resident still says so', async ({ page }) => {
+    await page.route('**/system', route =>
+      route.fulfill({ json: { backends: ['llama-cpp'], loaded_models: [] } }))
+    await page.route('**/v1/models', route => route.fulfill({ json: { data: [{ id: 'a-model' }] } }))
+    await page.goto('/app')
+    await expect(page.locator('.home-loaded-empty')).toBeVisible()
+    await expect(page.locator('.home-loaded .lane')).toHaveCount(0)
+  })
+})
+
+test.describe('App footer', () => {
+  test('is one line, not three stacked rows', async ({ page }) => {
+    await page.goto('/app')
+    const footer = page.locator('.app-footer')
+    await expect(footer).toBeVisible()
+    // Three centred rows of chrome cost more vertical space than the content
+    // they sit under is usually worth.
+    const height = await footer.evaluate(el => el.getBoundingClientRect().height)
+    expect(height).toBeLessThan(56)
+  })
+
+  test('keeps every link it had', async ({ page }) => {
+    await page.goto('/app')
+    const footer = page.locator('.app-footer')
+    for (const name of [/github/i, /documentation/i, /author/i]) {
+      await expect(footer.getByRole('link', { name })).toBeVisible()
+    }
+  })
+})

core/http/react-ui/e2e/models-gallery.spec.js

@@ -1786,6 +1786,19 @@ test.describe("Models Gallery - Discover split view", () => {
     );
   });
 
+  test("a build that fits at some context sizes warns rather than erroring", async ({
+    page,
+  }) => {
+    await railItem(page, "llama-model").click();
+    const verdict = page.locator(".discover__chart-verdict");
+    await expect(verdict).toBeVisible();
+    // A model that fits at 32k but not 64k is a trade-off, and #11288 keeps a
+    // test on such a build still being installable. Only "fits nowhere" earns
+    // the error tone; anything short of that warns.
+    await expect(verdict).toHaveClass(/discover__chart-verdict--warn/);
+    await expect(verdict).not.toHaveClass(/discover__chart-verdict--bad/);
+  });
+
   test("a host with no GPU gets no chart rather than an unanchored one", async ({
     page,
   }) => {

core/http/react-ui/e2e/models-recommended-panel.spec.js

@@ -56,81 +56,23 @@ async function gotoModels(page) {
 }
 
 test.describe("Models gallery - recommended panel prominence", () => {
-  test("first visit with nothing installed shows the panel expanded", async ({ page }) => {
+  test("it is a section in the flow, not a dismissable card", async ({ page }) => {
     await mockGallery(page, 0);
     await gotoModels(page);
-
-    await expect(toggle(page)).toHaveAttribute("aria-expanded", "true");
-    await expect(grid(page)).toBeVisible();
-    await expect(grid(page).getByText("tiny-chat")).toBeVisible();
+    await expect(panel(page)).toBeVisible();
+    // No close button and no collapse: this is the one thing the page has to
+    // say about the machine it runs on, not an interruption to be shut.
+    await expect(panel(page).locator("button[aria-expanded]")).toHaveCount(0);
+    await expect(panel(page).getByRole("button", { name: /dismiss|close/i })).toHaveCount(0);
+    // And no card chrome, so it sits in the pane rather than on top of it.
+    const border = await panel(page).evaluate((el) => getComputedStyle(el).borderTopWidth);
+    expect(parseFloat(border)).toBe(0);
   });
 
-  test("a user with models installed gets it collapsed by default", async ({ page }) => {
-    await mockGallery(page, 12);
-    await gotoModels(page);
 
-    await expect(toggle(page)).toHaveAttribute("aria-expanded", "false");
-    await expect(grid(page)).toBeHidden();
-    // Collapsed is a summary, not a removal: the heading stays on the page.
-    await expect(panel(page).getByText("Recommended for your hardware")).toBeVisible();
-    await expect(panel(page).getByText("2 models suggested")).toBeVisible();
-  });
 
-  test("the collapsed summary expands again on activation", async ({ page }) => {
-    await mockGallery(page, 12);
-    await gotoModels(page);
 
-    await expect(grid(page)).toBeHidden();
-    await toggle(page).click();
 
-    await expect(toggle(page)).toHaveAttribute("aria-expanded", "true");
-    await expect(grid(page)).toBeVisible();
-    await expect(page.evaluate((k) => localStorage.getItem(k), COLLAPSE_KEY)).resolves.toBe("0");
-  });
-
-  test("the collapse choice persists across a reload", async ({ page }) => {
-    await mockGallery(page, 0);
-    await gotoModels(page);
-    await expect(grid(page)).toBeVisible();
-
-    await toggle(page).click();
-    await expect(grid(page)).toBeHidden();
-
-    await page.reload();
-    await expect(panel(page)).toBeVisible({ timeout: 20_000 });
-    await expect(toggle(page)).toHaveAttribute("aria-expanded", "false");
-    await expect(grid(page)).toBeHidden();
-  });
-
-  test("dismissing it persists across a reload", async ({ page }) => {
-    await mockGallery(page, 0);
-    await gotoModels(page);
-
-    await panel(page).getByRole("button", { name: "Dismiss recommendations" }).click();
-    await expect(panel(page)).toHaveCount(0);
-    await expect(page.evaluate((k) => localStorage.getItem(k), DISMISS_KEY)).resolves.toBe("1");
-
-    await page.reload();
-    // The rail having entries is the marker that the page finished rendering
-    // without the panel. It used to be the table, which no longer exists.
-    await expect(
-      page.locator('[data-testid="discover-rail-item"]').first(),
-    ).toBeVisible({ timeout: 20_000 });
-    await expect(panel(page)).toHaveCount(0);
-  });
-
-  test("the toggle is keyboard operable and exposes its state", async ({ page }) => {
-    await mockGallery(page, 12);
-    await gotoModels(page);
-
-    await toggle(page).focus();
-    await expect(toggle(page)).toBeFocused();
-    await page.keyboard.press("Enter");
-    await expect(toggle(page)).toHaveAttribute("aria-expanded", "true");
-    // aria-controls must resolve to the region it actually shows and hides.
-    await expect(toggle(page)).toHaveAttribute("aria-controls", "rec-models-content");
-    await expect(grid(page)).toBeVisible();
-  });
 
   test("recommendations render and their install buttons still work", async ({ page }) => {
     await mockGallery(page, 0);
@@ -141,11 +83,23 @@ test.describe("Models gallery - recommended panel prominence", () => {
     });
     await gotoModels(page);
 
-    const card = grid(page).locator(".rec-models-item", { hasText: "tiny-chat" });
-    await expect(card).toBeVisible();
-    await expect(card.getByText("512.0 MB")).toBeVisible();
-    await card.getByRole("button", { name: "Install" }).click();
+    // Ranked candidates read in fit order, so these are lanes now rather than
+    // a grid of equal cards.
+    const row = grid(page).locator(".lane", { hasText: "tiny-chat" });
+    await expect(row).toBeVisible();
+    await expect(row.getByText("512.0 MB")).toBeVisible();
+    await row.getByRole("button", { name: "Install" }).click();
 
     await expect.poll(() => installed).toBe("tiny-chat");
   });
+
+  test("the best fit is called out, the rest are alternatives", async ({ page }) => {
+    await mockGallery(page, 0);
+    await gotoModels(page);
+    const rows = grid(page).locator(".lane");
+    await expect(rows.first().locator(".lane__tag--evidence")).toHaveText("Best fit");
+    // One opinion per page: the others are alternatives, not runners-up worth
+    // their own colour.
+    await expect(grid(page).locator(".lane__tag--evidence")).toHaveCount(1);
+  });
 });

core/http/react-ui/e2e/operate-overview.spec.js

@@ -0,0 +1,172 @@
+import { test, expect } from './coverage-fixtures.js'
+
+// Operate overview (src/pages/OperateOverview.jsx).
+//
+// The page exists to answer "is anything wrong" without visiting four other
+// pages, so the tests are written against that behaviour rather than against
+// the markup: what does it say when nothing is wrong, and does each source of
+// trouble actually surface.
+
+const OVERVIEW = '[data-testid="operate-overview"]'
+const CLEAR = '[data-testid="operate-attention-clear"]'
+const ITEM = '[data-testid="operate-attention-item"]'
+
+const NO_UPGRADES = {}
+const ONE_UPGRADE = {
+  'llama-cpp': {
+    backend_name: 'llama-cpp',
+    installed_version: '0.9.4',
+    available_version: '0.9.7',
+  },
+}
+
+// A quiet installation: nothing running, nothing stale, every node healthy.
+async function mockQuiet(page, { upgrades = NO_UPGRADES, operations = [] } = {}) {
+  await page.route('**/api/backends/upgrades', route =>
+    route.fulfill({ json: upgrades }))
+  await page.route('**/api/operations', route =>
+    route.fulfill({ json: operations }))
+  await page.route('**/api/nodes', route =>
+    route.fulfill({ json: [{ id: 'node-a', status: 'healthy', healthy: true }] }))
+}
+
+test.describe('Operate overview', () => {
+  test('Operate opens the overview, not whichever page happens to be first', async ({ page }) => {
+    await mockQuiet(page)
+    await page.goto('/app')
+    await page.locator('.sidebar-nav a.nav-item', { hasText: 'Operate' }).click()
+    // Today this lands on /app/backends purely because Backends is the first
+    // entry in operateConsole.groups — an ordering accident, not a decision.
+    await expect(page).toHaveURL(/\/app\/operate$/)
+    await expect(page.locator(OVERVIEW)).toBeVisible()
+  })
+
+  test('says so plainly when nothing needs attention', async ({ page }) => {
+    await mockQuiet(page)
+    await page.goto('/app/operate')
+    await expect(page.locator(CLEAR)).toBeVisible()
+    // The empty state is one line, not a panel full of reassuring green.
+    await expect(page.locator(ITEM)).toHaveCount(0)
+  })
+
+  test('a stale backend becomes an attention item naming the version jump', async ({ page }) => {
+    await mockQuiet(page, { upgrades: ONE_UPGRADE })
+    await page.goto('/app/operate')
+    const item = page.locator(ITEM, { hasText: 'llama-cpp' })
+    await expect(item).toBeVisible()
+    await expect(item).toContainText('0.9.4')
+    await expect(item).toContainText('0.9.7')
+    await expect(page.locator(CLEAR)).toHaveCount(0)
+  })
+
+  test('a failed operation becomes an attention item', async ({ page }) => {
+    await mockQuiet(page, {
+      operations: [{ id: 'op-1', name: 'qwen3-8b', type: 'install', error: 'no space left on device' }],
+    })
+    await page.goto('/app/operate')
+    await expect(page.locator(ITEM, { hasText: 'qwen3-8b' })).toBeVisible()
+  })
+
+  test('the rail reports backend updates alongside the label', async ({ page }) => {
+    await mockQuiet(page, { upgrades: ONE_UPGRADE })
+    await page.goto('/app/operate')
+    const backends = page.locator('.console-rail a.nav-item[href="/app/backends"]')
+    await expect(backends).toBeVisible()
+    await expect(backends.locator('.nav-signal')).toContainText('1')
+  })
+
+  test('the rail groups Runtime, Cluster, Observability and Administration', async ({ page }) => {
+    await mockQuiet(page)
+    await page.goto('/app/operate')
+    const rail = page.locator('.console-rail')
+    for (const group of ['Runtime', 'Cluster', 'Observability', 'Administration']) {
+      await expect(rail.locator('.console-group-title', { hasText: group })).toBeVisible()
+    }
+    // Six headings for thirteen items was the defect; the old pairs are gone.
+    for (const gone of ['Inference', 'Access']) {
+      await expect(rail.locator('.console-group-title', { hasText: new RegExp(`^${gone}$`) })).toHaveCount(0)
+    }
+  })
+
+  test('regrouping does not change what a non-distributed host can see', async ({ page }) => {
+    await page.route('**/api/features', route =>
+      route.fulfill({ json: { distributed: false, agents: true, mcp: true } }))
+    await mockQuiet(page)
+    await page.goto('/app/operate')
+    const rail = page.locator('.console-rail')
+    await expect(rail.locator('a.nav-item[href="/app/backends"]')).toBeVisible()
+    // Gating is the thing most likely to break silently when items move group.
+    await expect(rail.locator('a.nav-item[href="/app/nodes"]')).toHaveCount(0)
+    await expect(rail.locator('a.nav-item[href="/app/scheduling"]')).toHaveCount(0)
+  })
+
+  test('the sidebar keeps its operations badge', async ({ page }) => {
+    // Regression guard: this change edits the same config the badge reads, and
+    // the badge deliberately lives on the always-visible sidebar entry rather
+    // than the collapsible rail.
+    await mockQuiet(page, {
+      operations: [{ id: 'op-1', name: 'qwen3-8b', type: 'install', progress: 40 }],
+    })
+    await page.goto('/app')
+    await expect(page.locator('.sidebar-nav .nav-badge')).toBeVisible()
+  })
+
+  test('does not poll the summary away from Operate', async ({ page }) => {
+    let upgradeCalls = 0
+    await page.route('**/api/backends/upgrades', route => {
+      upgradeCalls += 1
+      route.fulfill({ json: NO_UPGRADES })
+    })
+    await page.route('**/api/operations', route => route.fulfill({ json: [] }))
+    await page.goto('/app/chat')
+    await expect(page.locator('.sidebar')).toBeVisible()
+    await page.waitForTimeout(1500)
+    // Nobody asked for this data outside Operate; a dashboard-shaped poll on
+    // every page is exactly what OperationsContext exists to avoid.
+    expect(upgradeCalls).toBe(0)
+  })
+})
+
+test.describe('Operate overview headline', () => {
+  const SUMMARY = {
+    total: 18402, errors: 37, p95_ms: 842, window_hours: 24,
+    buckets: Array.from({ length: 12 }, (_, i) => ({ count: 100 + i * 10, errors: i })),
+  }
+
+  test('reports counted totals rather than fetching the trace list', async ({ page }) => {
+    let listCalls = 0
+    await page.route('**/api/traces?**', route => { listCalls += 1; route.fulfill({ json: [] }) })
+    await page.route('**/api/traces/summary', route => route.fulfill({ json: SUMMARY }))
+    await mockQuiet(page)
+    await page.goto('/app/operate')
+    const headline = page.locator('.operate-headline')
+    await expect(headline).toBeVisible()
+    await expect(headline).toContainText('18,402')
+    await expect(headline).toContainText('37')
+    await expect(headline).toContainText('842')
+    // The whole point of the endpoint: three numbers, not the buffer.
+    expect(listCalls).toBe(0)
+  })
+
+  test('a quiet installation keeps the grid and says why it is empty', async ({ page }) => {
+    // Hiding the grid removed the page's structure exactly when someone was
+    // most likely to be looking at it, and "0 failed" is information.
+    await page.route('**/api/traces/summary', route =>
+      route.fulfill({ json: { total: 0, errors: 0, p95_ms: 0, window_hours: 24, buckets: [] } }))
+    await mockQuiet(page)
+    await page.goto('/app/operate')
+    await expect(page.locator('.operate-headline')).toBeVisible()
+    await expect(page.locator('.operate-headline__cell')).toHaveCount(4)
+    await expect(page.locator('.operate-headline__note')).toBeVisible()
+  })
+
+  test('the sections state counts rather than listing their destinations', async ({ page }) => {
+    await page.route('**/api/traces/summary', route =>
+      route.fulfill({ json: { total: 18402, errors: 37, p95_ms: 842, window_hours: 24, buckets: [] } }))
+    await mockQuiet(page)
+    await page.goto('/app/operate')
+    const runtime = page.locator('.lanes--sections .lane').first()
+    await expect(runtime).toContainText('backends')
+    await expect(runtime).toContainText('running')
+  })
+})

core/http/react-ui/e2e/page-render-smoke.spec.js

@@ -19,6 +19,7 @@ const PAGES = [
   ['/app/account', 'Account'],
   ['/app/studio', 'Studio'],
   ['/app/manage', 'Manage'],
+  ['/app/operate', 'Operate overview'],
   ['/app/backends', 'Backends'],
   ['/app/activity', 'Activity'],
   ['/app/settings', 'Settings'],

core/http/react-ui/e2e/studio-overview.spec.js

@@ -0,0 +1,155 @@
+import { test, expect } from './coverage-fixtures.js'
+
+// Studio overview (src/pages/StudioOverview.jsx).
+//
+// Studio was a tab strip over six generators that opened on Images and told you
+// nothing about what this machine could actually run. The tests are about that:
+// what the strip reports before you click, and the difference between a
+// modality that is switched off and one that merely has no model.
+
+const OVERVIEW = '[data-testid="studio-overview"]'
+const MODALITY = '[data-testid="studio-modality"]'
+const tabFor = (page, key) => page.locator(`.studio-tab[data-tab="${key}"]`)
+
+const model = (id, ...capabilities) => ({ id, capabilities })
+
+// Images and speech covered, video and sound not. 3D and transform are feature
+// flags rather than models, so they are controlled separately.
+const SOME_MODELS = {
+  data: [
+    model('flux.1-schnell', 'FLAG_IMAGE'),
+    model('kokoro-82m', 'FLAG_TTS'),
+    model('qwen3-8b', 'FLAG_CHAT'),
+  ],
+}
+
+async function mockCapabilities(page, payload = SOME_MODELS) {
+  await page.route('**/api/models/capabilities', route => route.fulfill({ json: payload }))
+}
+
+test.describe('Studio overview', () => {
+  test('Studio opens on the overview rather than dropping into Images', async ({ page }) => {
+    await mockCapabilities(page)
+    await page.goto('/app/studio')
+    await expect(page.locator(OVERVIEW)).toBeVisible()
+  })
+
+  test('a generator path opens that generator', async ({ page }) => {
+    await mockCapabilities(page)
+    await page.goto('/app/studio/images')
+    await expect(page.locator(OVERVIEW)).toHaveCount(0)
+    await expect(page.locator('.media-layout')).toBeVisible()
+  })
+
+  test('an unrecognised tab falls back to the overview, not to Images', async ({ page }) => {
+    await mockCapabilities(page)
+    await page.goto('/app/studio/nonsense')
+    await expect(page.locator(OVERVIEW)).toBeVisible()
+  })
+
+  test('the tab strip reports which modalities have a model', async ({ page }) => {
+    await mockCapabilities(page)
+    await page.goto('/app/studio')
+    // Filled: something installed advertises the capability.
+    await expect(tabFor(page, 'images').locator('.studio-tab__dot--on')).toBeVisible()
+    await expect(tabFor(page, 'tts').locator('.studio-tab__dot--on')).toBeVisible()
+    // Hollow: the modality is available, nothing serves it yet.
+    await expect(tabFor(page, 'video').locator('.studio-tab__dot--off')).toBeVisible()
+    await expect(tabFor(page, 'sound').locator('.studio-tab__dot--off')).toBeVisible()
+  })
+
+  test('a modality with no model offers a way to install one', async ({ page }) => {
+    await mockCapabilities(page)
+    await page.goto('/app/studio')
+    const video = page.locator(`${MODALITY}[data-modality="video"]`)
+    await expect(video).toBeVisible()
+    // The point of the lane: not a dead tab, a route to fixing it.
+    await expect(video.locator('a[href*="/app/models"]')).toBeVisible()
+  })
+
+  test('a modality with a model names it instead of offering an install', async ({ page }) => {
+    await mockCapabilities(page)
+    await page.goto('/app/studio')
+    const images = page.locator(`${MODALITY}[data-modality="images"]`)
+    await expect(images).toContainText('flux.1-schnell')
+    await expect(images.locator('a[href*="/app/models"]')).toHaveCount(0)
+  })
+
+  test('a disabled feature gets no tab and no lane at all', async ({ page }) => {
+    // Switched off is a different thing from "no model installed", and
+    // conflating them is how someone ends up staring at a control that cannot
+    // work. 3d is a permission rather than an /api/features entry, and
+    // hasFeature() short-circuits to true for admins and for auth-off
+    // installations, so withholding it needs a real non-admin session.
+    await page.route('**/api/auth/status', route => route.fulfill({
+      json: {
+        authEnabled: true,
+        user: { name: 'someone', role: 'user', permissions: { images: true, video: true, tts: true, sound: true } },
+      },
+    }))
+    await mockCapabilities(page)
+    await page.goto('/app/studio')
+    await expect(page.locator(OVERVIEW)).toBeVisible()
+    await expect(tabFor(page, 'threed')).toHaveCount(0)
+    await expect(page.locator(`${MODALITY}[data-modality="threed"]`)).toHaveCount(0)
+  })
+
+  test('asks the capabilities endpoint once, not once per modality', async ({ page }) => {
+    let calls = 0
+    await page.route('**/api/models/capabilities', route => {
+      calls += 1
+      route.fulfill({ json: SOME_MODELS })
+    })
+    await page.goto('/app/studio')
+    await expect(page.locator(OVERVIEW)).toBeVisible()
+    await page.waitForTimeout(500)
+    // useModels() fetches the whole list and filters in the browser, so one
+    // hook per modality would be six identical requests on every mount.
+    expect(calls).toBe(1)
+  })
+
+  test('an installation with no models at all still renders every modality', async ({ page }) => {
+    await mockCapabilities(page, { data: [] })
+    await page.goto('/app/studio')
+    await expect(page.locator(OVERVIEW)).toBeVisible()
+    await expect(page.locator(MODALITY).first()).toBeVisible()
+    await expect(page.locator('.studio-tab__dot--on')).toHaveCount(0)
+  })
+
+  test('recent outputs surface what was generated earlier', async ({ page }) => {
+    await mockCapabilities(page)
+    // History is localStorage, written by each generator. The overview is the
+    // first place it is read across modalities rather than within one.
+    await page.addInitScript(() => {
+      localStorage.setItem('localai_image_history', JSON.stringify([
+        { id: 'i1', createdAt: Date.now(), model: 'flux.1-schnell', prompt: 'a brass orrery', elapsedMs: 6100 },
+      ]))
+    })
+    await page.goto('/app/studio')
+    const shelf = page.locator('[data-testid="studio-recent"]')
+    await expect(shelf).toBeVisible()
+    await expect(shelf).toContainText('flux.1-schnell')
+  })
+
+  test('no history means no empty shelf', async ({ page }) => {
+    await mockCapabilities(page)
+    await page.goto('/app/studio')
+    await expect(page.locator('[data-testid="studio-recent"]')).toHaveCount(0)
+  })
+
+  test('the overview is reachable back from a generator tab', async ({ page }) => {
+    await mockCapabilities(page)
+    await page.goto('/app/studio/images')
+    await tabFor(page, 'overview').click()
+    await expect(page.locator(OVERVIEW)).toBeVisible()
+  })
+
+  test('a legacy ?tab= link is redirected to its path', async ({ page }) => {
+    // Bookmarks and older docs still use the query form; they must keep working
+    // and must land on the canonical URL rather than a second spelling of it.
+    await mockCapabilities(page)
+    await page.goto('/app/studio?tab=images')
+    await expect(page).toHaveURL(/\/app\/studio\/images$/)
+    await expect(page.locator('.media-layout')).toBeVisible()
+  })
+})

core/http/react-ui/e2e/studio-transform.spec.js

@@ -2,7 +2,7 @@ import { test, expect } from './coverage-fixtures.js'
 
 test.describe('Studio - Transform', () => {
   test('Studio exposes a Transform tab that renders Audio Transform', async ({ page }) => {
-    await page.goto('/app/studio?tab=transform')
+    await page.goto('/app/studio/transform')
     await expect(page.locator('.studio-tab', { hasText: 'Transform' })).toBeVisible()
     await expect(page.locator('h1.page-title', { hasText: 'Audio Transform' })).toBeVisible({ timeout: 15_000 })
   })

core/http/react-ui/e2e/studio-workbench.spec.js

@@ -0,0 +1,49 @@
+import { test, expect } from './coverage-fixtures.js'
+
+// The generator workbenches (mock 5b/5c): the control column and the record of
+// what the form actually sent.
+
+test.describe('Studio workbench', () => {
+  test('the control column is a hairline field stack, not a shadowed card', async ({ page }) => {
+    await page.goto('/app/studio/images')
+    const controls = page.locator('.media-controls')
+    await expect(controls).toBeVisible()
+    const style = await controls.evaluate(el => {
+      const cs = getComputedStyle(el)
+      return { shadow: cs.boxShadow, radius: cs.borderTopLeftRadius }
+    })
+    expect(style.shadow).toBe('none')
+    expect(style.radius).toBe('0px')
+  })
+
+  test('fields are separated by a rule and labelled in caps', async ({ page }) => {
+    await page.goto('/app/studio/images')
+    const label = page.locator('.media-controls .form-label').first()
+    await expect(label).toBeVisible()
+    const cs = await label.evaluate(el => getComputedStyle(el).textTransform)
+    expect(cs).toBe('uppercase')
+  })
+
+  test('no request is shown before one has been made', async ({ page }) => {
+    // A panel describing a request nobody sent is a tutorial, not a record.
+    await page.goto('/app/studio/images')
+    await expect(page.locator('.request-panel')).toHaveCount(0)
+  })
+
+  test('generating records the request that was actually sent', async ({ page }) => {
+    await page.route('**/api/models/capabilities', route =>
+      route.fulfill({ json: { data: [{ id: 'flux-mock', capabilities: ['FLAG_IMAGE'] }] } }))
+    await page.route('**/v1/images/generations', route =>
+      route.fulfill({ json: { data: [{ url: 'https://example.invalid/a.png' }] } }))
+
+    await page.goto('/app/studio/images')
+    await page.locator('.media-controls textarea').first().fill('a brass orrery')
+    await page.getByRole('button', { name: /generate/i }).click()
+
+    const panel = page.locator('.request-panel')
+    await expect(panel).toBeVisible()
+    await expect(panel).toContainText('/v1/images/generations')
+    await expect(panel).toContainText('a brass orrery')
+    await expect(panel.getByRole('button', { name: /curl/i })).toBeVisible()
+  })
+})

core/http/react-ui/e2e/theme-default.spec.js

@@ -0,0 +1,15 @@
+import { test, expect } from './coverage-fixtures.js'
+
+test.describe('Theme default', () => {
+  test('a fresh install opens dark even when the OS prefers light', async ({ page }) => {
+    await page.emulateMedia({ colorScheme: 'light' })
+    await page.goto('/app')
+    await expect(page.locator('html')).toHaveAttribute('data-theme', 'dark')
+  })
+
+  test('a stored choice still wins', async ({ page }) => {
+    await page.addInitScript(() => localStorage.setItem('localai-theme', 'light'))
+    await page.goto('/app')
+    await expect(page.locator('html')).toHaveAttribute('data-theme', 'light')
+  })
+})

core/http/react-ui/e2e/threed-gen.spec.js

@@ -276,9 +276,11 @@ test.describe('3D generation', () => {
       })
     })
 
-    await page.goto('/app/studio?tab=threed')
+    await page.goto('/app/studio/threed')
     await expect(page.getByRole('button', { name: '3D', exact: true })).toHaveCount(0)
-    await expect(page.locator('.studio-tab', { hasText: 'Images' })).toHaveClass(/studio-tab-active/)
+    // Falls back to the overview rather than Images. Landing on Images was
+    // never a decision, only the first entry in the tab array.
+    await expect(page.locator('.studio-tab[data-tab="overview"]')).toHaveClass(/studio-tab-active/)
 
     await page.goto('/app/3d')
     await expect(page).toHaveURL(/\/app\/?$/)

core/http/react-ui/e2e/traces-latency.spec.js

@@ -0,0 +1,42 @@
+import { test, expect } from './coverage-fixtures.js'
+
+// Traces rows carry latency as a shape, not only as a number buried in the
+// expanded detail (mock 6d).
+
+const TRACES = [
+  { id: '1', timestamp: new Date().toISOString(), duration: 4_200_000_000,
+    request: { method: 'POST', path: '/v1/chat/completions' }, response: { status: 500 }, error: 'context length exceeded' },
+  { id: '2', timestamp: new Date().toISOString(), duration: 980_000_000,
+    request: { method: 'POST', path: '/v1/chat/completions' }, response: { status: 200 } },
+  { id: '3', timestamp: new Date().toISOString(), duration: 186_000_000,
+    request: { method: 'POST', path: '/v1/embeddings' }, response: { status: 200 } },
+]
+
+test.describe('Traces latency', () => {
+  test.beforeEach(async ({ page }) => {
+    await page.route('**/api/traces**', route => route.fulfill({ json: TRACES }))
+    await page.goto('/app/traces')
+  })
+
+  test('every row shows a latency bar and figure', async ({ page }) => {
+    const cells = page.locator('.lat')
+    await expect(cells).toHaveCount(3)
+    await expect(cells.first()).toContainText('4.20s')
+  })
+
+  test('the bar is scaled against the slowest request in view', async ({ page }) => {
+    // Wait for the rows: goto alone does not guarantee the fetch has painted.
+    await expect(page.locator('.lat__bar i')).toHaveCount(3)
+    const widths = await page.locator('.lat__bar i').evaluateAll(
+      els => els.map(el => parseFloat(el.style.width)))
+    // 4.2s is the slowest, so it is full; 186ms is a sliver of it.
+    expect(widths[0]).toBe(100)
+    expect(widths[2]).toBeLessThan(20)
+  })
+
+  test('a slow request is marked, not just long', async ({ page }) => {
+    await expect(page.locator('.lat')).toHaveCount(3)
+    // Colour carries the threshold; the figure carries the value.
+    await expect(page.locator('.lat__bar--slow')).toHaveCount(1)
+  })
+})

core/http/react-ui/e2e/traces-live.spec.js

@@ -0,0 +1,22 @@
+import { test, expect } from './coverage-fixtures.js'
+
+test('marks an API trace with no response status as in progress', async ({ page }) => {
+  await page.route('**/api/traces?*', route => route.fulfill({
+    json: [{
+      id: 'running-1',
+      timestamp: '2026-08-05T02:00:00Z',
+      duration: 2_000_000_000,
+      request: { method: 'POST', path: '/v1/chat/completions' },
+      response: { status: 0 },
+    }],
+    headers: { 'X-Total-Count': '1' },
+  }))
+  await page.route('**/api/backend-traces?*', route => route.fulfill({ json: [] }))
+
+  await page.goto('/app/traces')
+
+  const row = page.locator('tbody tr').filter({ hasText: '/v1/chat/completions' })
+  await expect(row.getByText('Running', { exact: true })).toBeVisible()
+  await expect(row.locator('[title="In progress"]')).toBeVisible()
+  await expect(row.locator('.fa-check-circle')).toHaveCount(0)
+})

core/http/react-ui/e2e/voice-library-empty.spec.js

@@ -0,0 +1,33 @@
+import { test, expect } from './coverage-fixtures.js'
+
+// The empty voice library must offer its action, visibly, inside the panel.
+
+test.describe('Voice library empty state', () => {
+  test.beforeEach(async ({ page }) => {
+    await page.route('**/api/voice-profiles', route => route.fulfill({ json: [] }))
+    await page.goto('/app/voice-library')
+  })
+
+  test('the create action is visible and a normal size', async ({ page }) => {
+    const action = page.locator('.empty-state__actions a.btn').first()
+    await expect(action).toBeVisible()
+    const box = await action.boundingBox()
+    // It had been carrying the panel's own min-height:430px, which made it an
+    // invisible box that pushed itself out of view.
+    expect(box.height).toBeLessThan(80)
+  })
+
+  test('the action sits inside the panel, not past its edge', async ({ page }) => {
+    const panel = page.locator('.empty-state').first()
+    const action = page.locator('.empty-state__actions a.btn').first()
+    const [p, a] = [await panel.boundingBox(), await action.boundingBox()]
+    expect(a.y + a.height).toBeLessThanOrEqual(p.y + p.height + 1)
+  })
+
+  test('the panel renders its icon', async ({ page }) => {
+    const icon = page.locator('.empty-state-icon').first()
+    await expect(icon).toBeVisible()
+    const box = await icon.boundingBox()
+    expect(box.width).toBeGreaterThan(0)
+  })
+})

core/http/react-ui/inline-style-baseline.txt

@@ -1 +1 @@
-624
+538

core/http/react-ui/package-lock.json

@@ -21,9 +21,10 @@
         "@fortawesome/fontawesome-free": "^6.7.2",
         "@lezer/highlight": "^1.2.1",
         "@modelcontextprotocol/ext-apps": "^1.2.2",
-        "@modelcontextprotocol/sdk": "^1.25.1",
+        "@modelcontextprotocol/sdk": "^1.30.0",
         "dompurify": "^3.4.12",
         "highlight.js": "^11.11.1",
+        "hono": "4.12.34",
         "i18next": "^26.0.8",
         "i18next-browser-languagedetector": "^8.2.1",
         "i18next-http-backend": "^3.0.6",
@@ -635,12 +636,12 @@
       }
     },
     "node_modules/@hono/node-server": {
-      "version": "1.19.14",
-      "resolved": "https://registry.npmjs.org/@hono/node-server/-/node-server-1.19.14.tgz",
-      "integrity": "sha512-GwtvgtXxnWsucXvbQXkRgqksiH2Qed37H9xHZocE5sA3N8O8O8/8FA3uclQXxXVzc9XBZuEOMK7+r02FmSpHtw==",
+      "version": "2.1.0",
+      "resolved": "https://registry.npmjs.org/@hono/node-server/-/node-server-2.1.0.tgz",
+      "integrity": "sha512-XovyyCCnBzW+zKu+z/zq8hwNs4KOR5rEMAOxo2f40Q5xoOI37IMm6MIg2COOUtUApo0i6850MTBKH2u4QLGIqg==",
       "license": "MIT",
       "engines": {
-        "node": ">=18.14.1"
+        "node": ">=20"
       },
       "peerDependencies": {
         "hono": "^4"
@@ -944,11 +945,12 @@
       }
     },
     "node_modules/@modelcontextprotocol/sdk": {
-      "version": "1.27.1",
-      "resolved": "https://registry.npmjs.org/@modelcontextprotocol/sdk/-/sdk-1.27.1.tgz",
-      "integrity": "sha512-sr6GbP+4edBwFndLbM60gf07z0FQ79gaExpnsjMGePXqFcSSb7t6iscpjk9DhFhwd+mTEQrzNafGP8/iGGFYaA==",
+      "version": "1.30.0",
+      "resolved": "https://registry.npmjs.org/@modelcontextprotocol/sdk/-/sdk-1.30.0.tgz",
+      "integrity": "sha512-xKd8OIzlqNzcqcNumGAa6g+PW2kjD5vrpcKOnfldAUPP3j7lnqMPwlTXQm8gF+UwH72z0lqaRbjr9hqGz0eITA==",
+      "license": "MIT",
       "dependencies": {
-        "@hono/node-server": "^1.19.9",
+        "@hono/node-server": "^1.19.9 || ^2.0.5",
         "ajv": "^8.17.1",
         "ajv-formats": "^3.0.1",
         "content-type": "^1.0.5",
@@ -1718,10 +1720,11 @@
       "dev": true
     },
     "node_modules/brace-expansion": {
-      "version": "1.1.12",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.12.tgz",
-      "integrity": "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg==",
+      "version": "1.1.18",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
+      "integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
         "balanced-match": "^1.0.0",
         "concat-map": "0.0.1"
@@ -2876,9 +2879,9 @@
       "dev": true
     },
     "node_modules/fast-uri": {
-      "version": "3.1.4",
-      "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.4.tgz",
-      "integrity": "sha512-8JnbkQ4juDyvYs4mgFGQqg4yCYtFDtUtmp2QIQq11ZZe5CFQ5wcqm1rqDgAh/QdMySuBnPzMUiJUNZG5N/AiQw==",
+      "version": "3.1.5",
+      "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.5.tgz",
+      "integrity": "sha512-gHwA1O9LDIcKunMKhObS/HimwtehO1nPUECKAu5TpKgaO19fcWEl4bliWe1jWxVFvIXztJjjQ4L8XQ1EU9f7Jw==",
       "funding": [
         {
           "type": "github",
@@ -3432,9 +3435,9 @@
       }
     },
     "node_modules/hono": {
-      "version": "4.12.31",
-      "resolved": "https://registry.npmjs.org/hono/-/hono-4.12.31.tgz",
-      "integrity": "sha512-zJIHFrl6bq3RDd2YusFNCDlM8qUprxKswyi/OPzPyzKDdyBXDqWx8bZlZ7R+saTdSTatUmb3O7K4SspGPaEOQg==",
+      "version": "4.12.34",
+      "resolved": "https://registry.npmjs.org/hono/-/hono-4.12.34.tgz",
+      "integrity": "sha512-GqXJqY/xJkJmuloTrnV1ZEXG3fqte+VjkUqoRNZXcrUidiUOP4fMSIHHY4tsqZBK++kVyWmt/AAfSUuy57/eSA==",
       "license": "MIT",
       "engines": {
         "node": ">=16.9.0"
@@ -4193,9 +4196,9 @@
       "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ=="
     },
     "node_modules/ip-address": {
-      "version": "10.2.0",
-      "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.2.0.tgz",
-      "integrity": "sha512-/+S6j4E9AHvW9SWMSEY9Xfy66O5PWvVEJ08O0y5JGyEKQpojb0K0GKpz/v5HJ/G0vi3D2sjGK78119oXZeE0qA==",
+      "version": "10.4.0",
+      "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.4.0.tgz",
+      "integrity": "sha512-oSK96Grm3aP6OrS263xVxbNDGVL7rzBtYdpGqlDG8iQdoenDoTs/nkki+DflYbAEE8Xl6o5YxhxlrKvI3nqKXQ==",
       "license": "MIT",
       "engines": {
         "node": ">= 12"
@@ -4383,16 +4386,16 @@
       }
     },
     "node_modules/istanbul-lib-processinfo/node_modules/brace-expansion": {
-      "version": "5.0.6",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
-      "integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
+      "version": "5.0.9",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
+      "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
       "dev": true,
       "license": "MIT",
       "dependencies": {
         "balanced-match": "^4.0.2"
       },
       "engines": {
-        "node": "18 || 20 || >=22"
+        "node": "20 || >=22"
       }
     },
     "node_modules/istanbul-lib-processinfo/node_modules/glob": {
@@ -5278,16 +5281,16 @@
       }
     },
     "node_modules/nyc/node_modules/brace-expansion": {
-      "version": "5.0.6",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
-      "integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
+      "version": "5.0.9",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
+      "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
       "dev": true,
       "license": "MIT",
       "dependencies": {
         "balanced-match": "^4.0.2"
       },
       "engines": {
-        "node": "18 || 20 || >=22"
+        "node": "20 || >=22"
       }
     },
     "node_modules/nyc/node_modules/convert-source-map": {
@@ -5974,10 +5977,11 @@
       }
     },
     "node_modules/quick-temp/node_modules/brace-expansion": {
-      "version": "2.1.0",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.0.tgz",
-      "integrity": "sha512-TN1kCZAgdgweJhWWpgKYrQaMNHcDULHkWwQIspdtjV4Y5aurRdZpjAqn6yX3FPqTA9ngHCc4hJxMAMgGfve85w==",
+      "version": "2.1.4",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.4.tgz",
+      "integrity": "sha512-hGfVzPxthbf3+2yjg/RBs60cB0FhqBS/zvdV/4wn4/BmN0bNMMHPc4V/BbFieqf1TKAGGAHnY4eSjajCl0f2Xg==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
         "balanced-match": "^1.0.0"
       }
@@ -6569,16 +6573,16 @@
       }
     },
     "node_modules/spawn-wrap/node_modules/brace-expansion": {
-      "version": "5.0.6",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
-      "integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
+      "version": "5.0.9",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
+      "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
       "dev": true,
       "license": "MIT",
       "dependencies": {
         "balanced-match": "^4.0.2"
       },
       "engines": {
-        "node": "18 || 20 || >=22"
+        "node": "20 || >=22"
       }
     },
     "node_modules/spawn-wrap/node_modules/foreground-child": {
@@ -6902,16 +6906,16 @@
       }
     },
     "node_modules/test-exclude/node_modules/brace-expansion": {
-      "version": "5.0.6",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
-      "integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
+      "version": "5.0.9",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
+      "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
       "dev": true,
       "license": "MIT",
       "dependencies": {
         "balanced-match": "^4.0.2"
       },
       "engines": {
-        "node": "18 || 20 || >=22"
+        "node": "20 || >=22"
       }
     },
     "node_modules/test-exclude/node_modules/glob": {
@@ -7134,9 +7138,9 @@
       }
     },
     "node_modules/undici": {
-      "version": "7.28.0",
-      "resolved": "https://registry.npmjs.org/undici/-/undici-7.28.0.tgz",
-      "integrity": "sha512-cRZYrTDwWznlnRiPjggAGxZXanty6M8RV1ff8Wm4LWXBp7/IG8v5DnOm74DtUBp9OONpK75YlPnIjQqX0dBDtA==",
+      "version": "7.29.0",
+      "resolved": "https://registry.npmjs.org/undici/-/undici-7.29.0.tgz",
+      "integrity": "sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw==",
       "dev": true,
       "license": "MIT",
       "engines": {

core/http/react-ui/package.json

@@ -19,7 +19,7 @@
     "coverage:report": "nyc report"
   },
   "overrides": {
-    "hono": "4.12.25"
+    "hono": "4.12.34"
   },
   "dependencies": {
     "@codemirror/autocomplete": "^6.18.6",
@@ -35,10 +35,10 @@
     "@fortawesome/fontawesome-free": "^6.7.2",
     "@lezer/highlight": "^1.2.1",
     "@modelcontextprotocol/ext-apps": "^1.2.2",
-    "@modelcontextprotocol/sdk": "^1.25.1",
+    "@modelcontextprotocol/sdk": "^1.30.0",
     "dompurify": "^3.4.12",
     "highlight.js": "^11.11.1",
-    "hono": "4.12.25",
+    "hono": "4.12.34",
     "i18next": "^26.0.8",
     "i18next-browser-languagedetector": "^8.2.1",
     "i18next-http-backend": "^3.0.6",

core/http/react-ui/public/locales/de/admin.json

@@ -12,6 +12,8 @@
     "timeLeft": "{{value}} left",
     "cancel": "Cancel",
     "cancelLabel": "Cancel {{name}}",
+    "pause": "Pause",
+    "pauseLabel": "Pause {{name}} and keep downloaded data",
     "retry": "Retry",
     "retryLabel": "Retry {{name}}",
     "nodeCount": "{{count}} nodes",
@@ -143,5 +145,36 @@
   "explorer": {
     "title": "Explorer",
     "subtitle": "Dateien und Konfiguration durchsuchen"
+  },
+  "operate": {
+    "overview": {
+      "title": "Overview",
+      "subtitle": "Everything running on this installation, and anything that wants a decision.",
+      "attention": {
+        "heading": "Needs attention",
+        "clear": "Nothing needs attention. Backends are current, no operation has failed, and every node is healthy.",
+        "backendUpdate": "Update available: {{from}} → {{to}}"
+      },
+      "sections": {
+        "heading": "Sections",
+        "runtime": "Runtime",
+        "runtimeSummary": "{{backends}} backends · {{models}} models · {{updates}} updates · {{running}} running",
+        "cluster": "Cluster",
+        "clusterSummary": "{{nodes}} nodes",
+        "observability": "Observability",
+        "observabilitySummary": "Usage and traces",
+        "administration": "Administration",
+        "administrationSummary": "Users, middleware and settings · {{memory}} memory in use",
+        "clusterSingle": "Single node",
+        "observabilityCounted": "{{requests}} requests · {{errors}} failed · p95 {{p95}} ms"
+      },
+      "headline": {
+        "requests": "Requests · {{hours}}h",
+        "errors": "Failed requests",
+        "p95": "p95 latency",
+        "quiet": "No requests served in this window yet.",
+        "host": "Host memory"
+      }
+    }
   }
 }

core/http/react-ui/public/locales/de/chat.json

@@ -118,5 +118,8 @@
     "newChat": "Neuer Chat",
     "clearAll": "Alle löschen",
     "deleteAllTitle": "Alle Unterhaltungen löschen"
+  },
+  "message": {
+    "you": "You"
   }
 }

core/http/react-ui/public/locales/de/home.json

@@ -17,7 +17,9 @@
     "modelsLoaded_other": "{{count}} models loaded",
     "noModelsLoaded": "No models loaded",
     "nodes_one": "{{count}} node",
-    "nodes_other": "{{count}} nodes"
+    "nodes_other": "{{count}} nodes",
+    "loadedLabel": "Loaded",
+    "nodesLabel": "Nodes"
   },
   "assistant": {
     "title": "LocalAI per Chat verwalten",
@@ -47,7 +49,8 @@
     "count_one": "{{count}} Modell geladen",
     "count_other": "{{count}} Modelle geladen",
     "stop": "Modell stoppen",
-    "stopAll": "Alle stoppen"
+    "stopAll": "Alle stoppen",
+    "serving": "Serving"
   },
   "stopDialog": {
     "title": "Modell stoppen",
@@ -88,5 +91,14 @@
     "browse": "Browse the API",
     "hide": "Hide endpoints",
     "dismiss": "Dismiss"
+  },
+  "jump": {
+    "heading": "Jump back in",
+    "discover": "Discover",
+    "discoverSummary": "Browse the gallery and install models",
+    "create": "Create",
+    "createSummary": "Open a chat, image or voice session",
+    "operate": "Operate",
+    "operateSummary": "{{models}} models configured · nodes, activity and traces"
   }
 }

core/http/react-ui/public/locales/de/media.json

@@ -5,6 +5,32 @@
       "video": "Video",
       "tts": "TTS",
       "sound": "Audio",
+      "transform": "Transform",
+      "overview": "Overview"
+    },
+    "overview": {
+      "eyebrow": "{{ready}} of {{total}} modalities ready",
+      "title": "Studio",
+      "subtitle": "Generate images, video, 3D, speech and sound with the models on this machine.",
+      "canMake": "What you can make",
+      "running": "Running now",
+      "recent": "Recent outputs",
+      "noModel": "No model installed",
+      "install": "Install a model",
+      "ready": "Ready",
+      "seconds": "{{seconds}}s",
+      "describe": {
+        "images": "Text to image, image to image, reference images",
+        "video": "Text to video and image to video",
+        "threed": "Image to mesh reconstruction",
+        "tts": "Text to speech using your voice library",
+        "sound": "Music and sound effects from a prompt",
+        "transform": "Separation, enhancement and voice conversion"
+      }
+    },
+    "groups": {
+      "create": "Create",
+      "voice": "Voice",
       "transform": "Transform"
     }
   },
@@ -157,5 +183,10 @@
     "clearMessage": "Alle Verlaufseinträge entfernen? Diese Aktion kann nicht rückgängig gemacht werden.",
     "clearConfirm": "Löschen",
     "cleared": "Verlauf gelöscht"
+  },
+  "request": {
+    "heading": "Request",
+    "copyCurl": "Copy as curl",
+    "copied": "Copied"
   }
 }

core/http/react-ui/public/locales/de/models.json

@@ -10,7 +10,9 @@
     "installStarted": "{{model}} wird installiert…",
     "installFailed": "Installation fehlgeschlagen: {{message}}",
     "dismiss": "Empfehlungen ausblenden",
-    "summary": "{{n}} Modelle vorgeschlagen"
+    "summary": "{{n}} Modelle vorgeschlagen",
+    "bestFit": "Best fit",
+    "alternative": "Also fits"
   },
   "stats": {
     "available": "Verfügbar",

core/http/react-ui/public/locales/de/nav.json

@@ -24,7 +24,9 @@
     "observability": "Observability",
     "access": "Access",
     "system": "System",
-    "activity": "Activity"
+    "activity": "Activity",
+    "runtime": "Laufzeit",
+    "administration": "Verwaltung"
   },
   "items": {
     "home": "Start",
@@ -57,7 +59,8 @@
     "settings": "Einstellungen",
     "api": "API",
     "middleware": "Middleware",
-    "activity": "Aktivität"
+    "activity": "Aktivität",
+    "overview": "Übersicht"
   },
   "footer": {
     "github": "GitHub",

core/http/react-ui/public/locales/en/admin.json

@@ -12,6 +12,8 @@
     "timeLeft": "{{value}} left",
     "cancel": "Cancel",
     "cancelLabel": "Cancel {{name}}",
+    "pause": "Pause",
+    "pauseLabel": "Pause {{name}} and keep downloaded data",
     "retry": "Retry",
     "retryLabel": "Retry {{name}}",
     "nodeCount": "{{count}} nodes",
@@ -166,5 +168,36 @@
   "explorer": {
     "title": "Explorer",
     "subtitle": "Browse files and configuration"
+  },
+  "operate": {
+    "overview": {
+      "title": "Overview",
+      "subtitle": "Everything running on this installation, and anything that wants a decision.",
+      "attention": {
+        "heading": "Needs attention",
+        "clear": "Nothing needs attention. Backends are current, no operation has failed, and every node is healthy.",
+        "backendUpdate": "Update available: {{from}} → {{to}}"
+      },
+      "sections": {
+        "heading": "Sections",
+        "runtime": "Runtime",
+        "runtimeSummary": "{{backends}} backends · {{models}} models · {{updates}} updates · {{running}} running",
+        "cluster": "Cluster",
+        "clusterSummary": "{{nodes}} nodes",
+        "observability": "Observability",
+        "observabilitySummary": "Usage and traces",
+        "administration": "Administration",
+        "administrationSummary": "Users, middleware and settings · {{memory}} memory in use",
+        "clusterSingle": "Single node",
+        "observabilityCounted": "{{requests}} requests · {{errors}} failed · p95 {{p95}} ms"
+      },
+      "headline": {
+        "requests": "Requests · {{hours}}h",
+        "errors": "Failed requests",
+        "p95": "p95 latency",
+        "quiet": "No requests served in this window yet.",
+        "host": "Host memory"
+      }
+    }
   }
 }

core/http/react-ui/public/locales/en/chat.json

@@ -124,5 +124,8 @@
     "newChat": "New chat",
     "clearAll": "Clear all",
     "deleteAllTitle": "Delete all conversations"
+  },
+  "message": {
+    "you": "You"
   }
 }

core/http/react-ui/public/locales/en/home.json

@@ -17,7 +17,9 @@
     "modelsLoaded_other": "{{count}} models loaded",
     "noModelsLoaded": "No models loaded",
     "nodes_one": "{{count}} node",
-    "nodes_other": "{{count}} nodes"
+    "nodes_other": "{{count}} nodes",
+    "loadedLabel": "Loaded",
+    "nodesLabel": "Nodes"
   },
   "assistant": {
     "title": "Manage LocalAI by chatting",
@@ -47,7 +49,8 @@
     "count_one": "{{count}} model loaded",
     "count_other": "{{count}} models loaded",
     "stop": "Stop model",
-    "stopAll": "Stop all"
+    "stopAll": "Stop all",
+    "serving": "Serving"
   },
   "stopDialog": {
     "title": "Stop Model",
@@ -103,5 +106,14 @@
     "browse": "Browse the API",
     "hide": "Hide endpoints",
     "dismiss": "Dismiss"
+  },
+  "jump": {
+    "heading": "Jump back in",
+    "discover": "Discover",
+    "discoverSummary": "Browse the gallery and install models",
+    "create": "Create",
+    "createSummary": "Open a chat, image or voice session",
+    "operate": "Operate",
+    "operateSummary": "{{models}} models configured · nodes, activity and traces"
   }
 }

core/http/react-ui/public/locales/en/media.json

@@ -6,7 +6,33 @@
       "tts": "TTS",
       "sound": "Sound",
       "transform": "Transform",
-      "threed": "3D"
+      "threed": "3D",
+      "overview": "Overview"
+    },
+    "overview": {
+      "eyebrow": "{{ready}} of {{total}} modalities ready",
+      "title": "Studio",
+      "subtitle": "Generate images, video, 3D, speech and sound with the models on this machine.",
+      "canMake": "What you can make",
+      "running": "Running now",
+      "recent": "Recent outputs",
+      "noModel": "No model installed",
+      "install": "Install a model",
+      "ready": "Ready",
+      "seconds": "{{seconds}}s",
+      "describe": {
+        "images": "Text to image, image to image, reference images",
+        "video": "Text to video and image to video",
+        "threed": "Image to mesh reconstruction",
+        "tts": "Text to speech using your voice library",
+        "sound": "Music and sound effects from a prompt",
+        "transform": "Separation, enhancement and voice conversion"
+      }
+    },
+    "groups": {
+      "create": "Create",
+      "voice": "Voice",
+      "transform": "Transform"
     }
   },
   "image": {
@@ -426,5 +452,10 @@
     "clearMessage": "Remove all history entries? This cannot be undone.",
     "clearConfirm": "Clear",
     "cleared": "History cleared"
+  },
+  "request": {
+    "heading": "Request",
+    "copyCurl": "Copy as curl",
+    "copied": "Copied"
   }
 }

core/http/react-ui/public/locales/en/models.json

@@ -11,7 +11,9 @@
     "installStarted": "Installing {{model}}…",
     "installFailed": "Install failed: {{message}}",
     "dismiss": "Dismiss recommendations",
-    "summary": "{{n}} models suggested"
+    "summary": "{{n}} models suggested",
+    "bestFit": "Best fit",
+    "alternative": "Also fits"
   },
   "stats": {
     "available": "Available",

core/http/react-ui/public/locales/en/nav.json

@@ -24,7 +24,9 @@
     "observability": "Observability",
     "access": "Access",
     "system": "System",
-    "activity": "Activity"
+    "activity": "Activity",
+    "runtime": "Runtime",
+    "administration": "Administration"
   },
   "items": {
     "home": "Home",
@@ -58,7 +60,8 @@
     "system": "System",
     "settings": "Settings",
     "api": "API",
-    "activity": "Activity"
+    "activity": "Activity",
+    "overview": "Overview"
   },
   "footer": {
     "github": "GitHub",

core/http/react-ui/public/locales/es/admin.json

@@ -12,6 +12,8 @@
     "timeLeft": "{{value}} left",
     "cancel": "Cancel",
     "cancelLabel": "Cancel {{name}}",
+    "pause": "Pause",
+    "pauseLabel": "Pause {{name}} and keep downloaded data",
     "retry": "Retry",
     "retryLabel": "Retry {{name}}",
     "nodeCount": "{{count}} nodes",
@@ -143,5 +145,36 @@
   "explorer": {
     "title": "Explorador",
     "subtitle": "Explora archivos y configuración"
+  },
+  "operate": {
+    "overview": {
+      "title": "Overview",
+      "subtitle": "Everything running on this installation, and anything that wants a decision.",
+      "attention": {
+        "heading": "Needs attention",
+        "clear": "Nothing needs attention. Backends are current, no operation has failed, and every node is healthy.",
+        "backendUpdate": "Update available: {{from}} → {{to}}"
+      },
+      "sections": {
+        "heading": "Sections",
+        "runtime": "Runtime",
+        "runtimeSummary": "{{backends}} backends · {{models}} models · {{updates}} updates · {{running}} running",
+        "cluster": "Cluster",
+        "clusterSummary": "{{nodes}} nodes",
+        "observability": "Observability",
+        "observabilitySummary": "Usage and traces",
+        "administration": "Administration",
+        "administrationSummary": "Users, middleware and settings · {{memory}} memory in use",
+        "clusterSingle": "Single node",
+        "observabilityCounted": "{{requests}} requests · {{errors}} failed · p95 {{p95}} ms"
+      },
+      "headline": {
+        "requests": "Requests · {{hours}}h",
+        "errors": "Failed requests",
+        "p95": "p95 latency",
+        "quiet": "No requests served in this window yet.",
+        "host": "Host memory"
+      }
+    }
   }
 }

core/http/react-ui/public/locales/es/chat.json

@@ -118,5 +118,8 @@
     "newChat": "Nuevo chat",
     "clearAll": "Borrar todo",
     "deleteAllTitle": "Eliminar todas las conversaciones"
+  },
+  "message": {
+    "you": "You"
   }
 }

core/http/react-ui/public/locales/es/home.json

@@ -17,7 +17,9 @@
     "modelsLoaded_other": "{{count}} models loaded",
     "noModelsLoaded": "No models loaded",
     "nodes_one": "{{count}} node",
-    "nodes_other": "{{count}} nodes"
+    "nodes_other": "{{count}} nodes",
+    "loadedLabel": "Loaded",
+    "nodesLabel": "Nodes"
   },
   "assistant": {
     "title": "Administra LocalAI chateando",
@@ -47,7 +49,8 @@
     "count_one": "{{count}} modelo cargado",
     "count_other": "{{count}} modelos cargados",
     "stop": "Detener modelo",
-    "stopAll": "Detener todos"
+    "stopAll": "Detener todos",
+    "serving": "Serving"
   },
   "stopDialog": {
     "title": "Detener modelo",
@@ -88,5 +91,14 @@
     "browse": "Browse the API",
     "hide": "Hide endpoints",
     "dismiss": "Dismiss"
+  },
+  "jump": {
+    "heading": "Jump back in",
+    "discover": "Discover",
+    "discoverSummary": "Browse the gallery and install models",
+    "create": "Create",
+    "createSummary": "Open a chat, image or voice session",
+    "operate": "Operate",
+    "operateSummary": "{{models}} models configured · nodes, activity and traces"
   }
 }

core/http/react-ui/public/locales/es/media.json

@@ -5,6 +5,32 @@
       "video": "Video",
       "tts": "TTS",
       "sound": "Sonido",
+      "transform": "Transform",
+      "overview": "Overview"
+    },
+    "overview": {
+      "eyebrow": "{{ready}} of {{total}} modalities ready",
+      "title": "Studio",
+      "subtitle": "Generate images, video, 3D, speech and sound with the models on this machine.",
+      "canMake": "What you can make",
+      "running": "Running now",
+      "recent": "Recent outputs",
+      "noModel": "No model installed",
+      "install": "Install a model",
+      "ready": "Ready",
+      "seconds": "{{seconds}}s",
+      "describe": {
+        "images": "Text to image, image to image, reference images",
+        "video": "Text to video and image to video",
+        "threed": "Image to mesh reconstruction",
+        "tts": "Text to speech using your voice library",
+        "sound": "Music and sound effects from a prompt",
+        "transform": "Separation, enhancement and voice conversion"
+      }
+    },
+    "groups": {
+      "create": "Create",
+      "voice": "Voice",
       "transform": "Transform"
     }
   },
@@ -157,5 +183,10 @@
     "clearMessage": "¿Eliminar todas las entradas del historial? Esto no se puede deshacer.",
     "clearConfirm": "Borrar",
     "cleared": "Historial borrado"
+  },
+  "request": {
+    "heading": "Request",
+    "copyCurl": "Copy as curl",
+    "copied": "Copied"
   }
 }

core/http/react-ui/public/locales/es/models.json

@@ -10,7 +10,9 @@
     "installStarted": "Instalando {{model}}…",
     "installFailed": "Error al instalar: {{message}}",
     "dismiss": "Descartar recomendaciones",
-    "summary": "{{n}} modelos sugeridos"
+    "summary": "{{n}} modelos sugeridos",
+    "bestFit": "Best fit",
+    "alternative": "Also fits"
   },
   "stats": {
     "available": "Disponibles",

core/http/react-ui/public/locales/es/nav.json

@@ -24,7 +24,9 @@
     "observability": "Observability",
     "access": "Access",
     "system": "System",
-    "activity": "Activity"
+    "activity": "Activity",
+    "runtime": "Runtime",
+    "administration": "Administración"
   },
   "items": {
     "home": "Inicio",
@@ -57,7 +59,8 @@
     "settings": "Configuración",
     "api": "API",
     "middleware": "Middleware",
-    "activity": "Actividad"
+    "activity": "Actividad",
+    "overview": "Resumen"
   },
   "footer": {
     "github": "GitHub",

core/http/react-ui/public/locales/id/admin.json

@@ -12,6 +12,8 @@
     "timeLeft": "{{value}} left",
     "cancel": "Cancel",
     "cancelLabel": "Cancel {{name}}",
+    "pause": "Pause",
+    "pauseLabel": "Pause {{name}} and keep downloaded data",
     "retry": "Retry",
     "retryLabel": "Retry {{name}}",
     "nodeCount": "{{count}} nodes",
@@ -166,5 +168,36 @@
   "explorer": {
     "title": "Penjelajah",
     "subtitle": "Jelajahi file dan konfigurasi"
+  },
+  "operate": {
+    "overview": {
+      "title": "Overview",
+      "subtitle": "Everything running on this installation, and anything that wants a decision.",
+      "attention": {
+        "heading": "Needs attention",
+        "clear": "Nothing needs attention. Backends are current, no operation has failed, and every node is healthy.",
+        "backendUpdate": "Update available: {{from}} → {{to}}"
+      },
+      "sections": {
+        "heading": "Sections",
+        "runtime": "Runtime",
+        "runtimeSummary": "{{backends}} backends · {{models}} models · {{updates}} updates · {{running}} running",
+        "cluster": "Cluster",
+        "clusterSummary": "{{nodes}} nodes",
+        "observability": "Observability",
+        "observabilitySummary": "Usage and traces",
+        "administration": "Administration",
+        "administrationSummary": "Users, middleware and settings · {{memory}} memory in use",
+        "clusterSingle": "Single node",
+        "observabilityCounted": "{{requests}} requests · {{errors}} failed · p95 {{p95}} ms"
+      },
+      "headline": {
+        "requests": "Requests · {{hours}}h",
+        "errors": "Failed requests",
+        "p95": "p95 latency",
+        "quiet": "No requests served in this window yet.",
+        "host": "Host memory"
+      }
+    }
   }
 }

core/http/react-ui/public/locales/id/chat.json

@@ -118,5 +118,8 @@
     "newChat": "Obrolan baru",
     "clearAll": "Hapus semua",
     "deleteAllTitle": "Hapus semua percakapan"
+  },
+  "message": {
+    "you": "You"
   }
 }

core/http/react-ui/public/locales/id/home.json

@@ -17,7 +17,9 @@
     "modelsLoaded_other": "{{count}} model dimuat",
     "noModelsLoaded": "Tidak ada model yang dimuat",
     "nodes_one": "{{count}} node",
-    "nodes_other": "{{count}} nodes"
+    "nodes_other": "{{count}} nodes",
+    "loadedLabel": "Loaded",
+    "nodesLabel": "Nodes"
   },
   "assistant": {
     "title": "Kelola LocalAI melalui obrolan",
@@ -47,7 +49,8 @@
     "count_one": "{{count}} model dimuat",
     "count_other": "{{count}} model dimuat",
     "stop": "Hentikan model",
-    "stopAll": "Hentikan semua"
+    "stopAll": "Hentikan semua",
+    "serving": "Serving"
   },
   "stopDialog": {
     "title": "Hentikan Model",
@@ -88,5 +91,14 @@
     "browse": "Jelajahi API",
     "hide": "Sembunyikan endpoint",
     "dismiss": "Abaikan"
+  },
+  "jump": {
+    "heading": "Jump back in",
+    "discover": "Discover",
+    "discoverSummary": "Browse the gallery and install models",
+    "create": "Create",
+    "createSummary": "Open a chat, image or voice session",
+    "operate": "Operate",
+    "operateSummary": "{{models}} models configured · nodes, activity and traces"
   }
 }

core/http/react-ui/public/locales/id/media.json

@@ -5,7 +5,33 @@
       "video": "Video",
       "tts": "TTS",
       "sound": "Suara",
-      "transform": "Transformasi"
+      "transform": "Transformasi",
+      "overview": "Overview"
+    },
+    "overview": {
+      "eyebrow": "{{ready}} of {{total}} modalities ready",
+      "title": "Studio",
+      "subtitle": "Generate images, video, 3D, speech and sound with the models on this machine.",
+      "canMake": "What you can make",
+      "running": "Running now",
+      "recent": "Recent outputs",
+      "noModel": "No model installed",
+      "install": "Install a model",
+      "ready": "Ready",
+      "seconds": "{{seconds}}s",
+      "describe": {
+        "images": "Text to image, image to image, reference images",
+        "video": "Text to video and image to video",
+        "threed": "Image to mesh reconstruction",
+        "tts": "Text to speech using your voice library",
+        "sound": "Music and sound effects from a prompt",
+        "transform": "Separation, enhancement and voice conversion"
+      }
+    },
+    "groups": {
+      "create": "Create",
+      "voice": "Voice",
+      "transform": "Transform"
     }
   },
   "image": {
@@ -204,5 +230,10 @@
     "clearMessage": "Hapus semua entri riwayat? Tindakan ini tidak dapat dibatalkan.",
     "clearConfirm": "Hapus",
     "cleared": "Riwayat dihapus"
+  },
+  "request": {
+    "heading": "Request",
+    "copyCurl": "Copy as curl",
+    "copied": "Copied"
   }
 }