fix(vllm-cpp): pass Darwin warning flag to Objective-C

The Metal source that triggers Apple Clang's constant-folding diagnostic is compiled as Objective-C. Pass the targeted suppression through CMAKE_OBJC_FLAGS as well as the C++ language flags so vllm.cpp's -Werror no longer promotes it.

Assisted-by: Codex:gpt-5

.agents/api-endpoints-and-auth.md

@@ -304,7 +304,9 @@ React pages that want to filter the ModelSelector by capability import this symb
 
 ### 4. `docs/content/` (user-facing documentation)
 
-A new capability deserves its own page under `docs/content/features/`, plus cross-links from related features and an entry in `docs/content/whats-new.md`. See the pattern used by `face-recognition.md` / `object-detection.md`.
+A new capability deserves its own page under `docs/content/features/`, plus cross-links from related features. See the pattern used by `face-recognition.md` / `object-detection.md`.
+
+Announcing it is the release's job, not this page's: the capability gets covered in the release blog post under `website/content/blog/`. See [preparing-a-release.md](preparing-a-release.md). `docs/content/whats-new.md` is only a pointer at the blog and GitHub Releases, so there is nothing to add there.
 
 ## Path protection rules
 
@@ -334,7 +336,7 @@ When adding a new endpoint:
 - [ ] Swagger block on the handler: `@Summary`, `@Tags`, `@Param`, `@Success`, `@Router`
 - [ ] If new capability area (new swagger tag): entry in `instructionDefs` in `core/http/endpoints/localai/api_instructions.go` + test count bumped in `api_instructions_test.go`
 - [ ] If new `FLAG_*` usecase flag: matching `CAP_*` symbol exported from `core/http/react-ui/src/utils/capabilities.js`
-- [ ] `docs/content/features/<feature>.md` created; cross-links from related feature pages; entry in `docs/content/whats-new.md`
+- [ ] `docs/content/features/<feature>.md` created; cross-links from related feature pages; capability covered in the release blog post (see [preparing-a-release.md](preparing-a-release.md))
 
 **Quality**
 - [ ] Error responses use `schema.ErrorResponse` format (or `echo.NewHTTPError` with a mapped gRPC status — see the `mapBackendError` helper in `core/http/endpoints/localai/images.go`)

.docker/llama-cpp-build-target.sh

@@ -4,6 +4,17 @@ set -euo pipefail
 arch=${1:?target architecture is required}
 build_type=${2-}
 
+# SYCL compiles the whole tree with icpx -fsycl, and icpx never finishes
+# ggml-cpu/arch/x86/repack.cpp at -march=sapphirerapids: the job sits on that one
+# translation unit until GitHub kills it at 6h. gcc builds the same file in
+# seconds, so only the SYCL images have to give up the CPU variant matrix.
+case "$build_type" in
+  sycl*)
+    echo llama-cpp-fallback
+    exit 0
+    ;;
+esac
+
 # GPU arm64 base images do not consistently provide the gcc-14 toolchain needed
 # to compile ggml's armv9.2 CPU variants. Keep their portable fallback until the
 # builder images can supply that compiler.

.docker/turboquant-build-target.sh

@@ -4,6 +4,17 @@ set -euo pipefail
 arch=${1:?target architecture is required}
 build_type=${2-}
 
+# SYCL compiles the whole tree with icpx -fsycl, and icpx never finishes
+# ggml-cpu/arch/x86/repack.cpp at -march=sapphirerapids: the job sits on that one
+# translation unit until GitHub kills it at 6h. gcc builds the same file in
+# seconds, so only the SYCL images have to give up the CPU variant matrix.
+case "$build_type" in
+  sycl*)
+    echo turboquant-fallback
+    exit 0
+    ;;
+esac
+
 # GPU arm64 base images do not consistently provide the gcc-14 toolchain needed
 # to compile ggml's armv9.2 CPU variants. Keep their portable fallback until the
 # builder images can supply that compiler.

.github/workflows/gh-pages.yml

@@ -51,7 +51,16 @@ jobs:
       - name: Setup Go
         uses: actions/setup-go@v5
         with:
-          go-version: '1.22'
+          # Track go.mod rather than a literal. Pinned at 1.22 this installed a
+          # toolchain older than the module's `go 1.26.0`, so the `go run` below
+          # downloaded the real one from proxy.golang.org on every run. That
+          # fetch is not always reachable from the runner and the deploy failed
+          # on five of eight consecutive master pushes with:
+          #   go: download go1.26.0: ... connect: network is unreachable
+          #   ##[error]Command failed: go env GOPATH
+          # Installing the version the module asks for removes the download
+          # instead of depending on it succeeding.
+          go-version-file: go.mod
           cache: false
 
       - name: Setup Hugo

README.md

@@ -195,7 +195,7 @@ For more details, see the [Getting Started guide](https://localai.io/basics/gett
 - **August 2025**: MLX, MLX-VLM, Diffusers, llama.cpp now supported on Apple Silicon
 - **July 2025**: All backends migrated outside the main binary — [lightweight, modular architecture](https://github.com/mudler/LocalAI/releases/tag/v3.2.0)
 
-For older news and full release notes, see [GitHub Releases](https://github.com/mudler/LocalAI/releases) and the [News page](https://localai.io/basics/news/).
+For older news and full release notes, see [GitHub Releases](https://github.com/mudler/LocalAI/releases) and the [blog](https://localai.io/blog/).
 
 ## Features
 
@@ -260,7 +260,7 @@ We also maintain [apex-quant](https://github.com/localai-org/apex-quant), a per-
 - [Kubernetes installation](https://localai.io/basics/getting_started/#run-localai-in-kubernetes)
 - [Integrations & community projects](https://localai.io/docs/integrations/)
 - [Installation video walkthrough](https://www.youtube.com/watch?v=cMVNnlqwfw4)
-- [Media & blog posts](https://localai.io/basics/news/#media-blogs-social)
+- [Blog: release write-ups, benchmarks and engineering notes](https://localai.io/blog/)
 - [Examples](https://github.com/mudler/LocalAI-examples) — including the [realtime voice assistant demo](https://github.com/localai-org/localai-realtime-demo) (Go client for the Realtime API with tool calling)
 
 ## Team

backend/backend.proto

@@ -15,6 +15,7 @@ service Backend {
   rpc PredictStream(PredictOptions) returns (stream Reply) {}
   rpc Embedding(PredictOptions) returns (EmbeddingResult) {}
   rpc GenerateImage(GenerateImageRequest) returns (Result) {}
+  rpc UpscaleImage(UpscaleImageRequest) returns (Result) {}
   rpc GenerateVideo(GenerateVideoRequest) returns (Result) {}
   rpc Generate3D(Generate3DRequest) returns (Result) {}
   rpc AudioTranscription(TranscriptRequest) returns (TranscriptResult) {}
@@ -637,6 +638,12 @@ message GenerateImageRequest {
   string ModelIdentity = 13;
 }
 
+message UpscaleImageRequest {
+  string src = 1;   // input image path
+  string dst = 2;   // output image path
+  int32  scale = 3; // upscale factor (e.g. 2 or 4)
+}
+
 message GenerateVideoRequest {
   string prompt = 1;
   string negative_prompt = 2;  // Negative prompt for video generation

backend/cpp/audio-cpp/Makefile

@@ -9,7 +9,7 @@
 # recipe is a make target (not a prepare.sh) so 'make purge && make' is a clean
 # rebuild and so the bump bot can see the pin.
 
-AUDIO_CPP_VERSION?=5a8312ef7b8aa7cf14e9a24ac568cabd8725d68a
+AUDIO_CPP_VERSION?=4e3aea2fd99aeaa5924e71c51eb2793846045332
 AUDIO_CPP_REPO?=https://github.com/0xShug0/audio.cpp
 
 CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))

backend/cpp/ik-llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-IK_LLAMA_VERSION?=cb9147fd0d9c08a9a84eee5ac405a73f4e10e3e1
+IK_LLAMA_VERSION?=60389410a1ff01f9d37dcc6261db33b3183bdea2
 LLAMA_REPO?=https://github.com/ikawrakow/ik_llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-LLAMA_VERSION?=a7a6d0d269c896218b6c78e0933bd6a17519d3f6
+LLAMA_VERSION?=221f0f6356efe2260023208365705ec5d5a7c8f5
 LLAMA_REPO?=https://github.com/ggerganov/llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/llama-cpp/run.sh

@@ -12,10 +12,11 @@ grep -e "flags" /proc/cpuinfo | head -1
 
 BINARY=llama-cpp-fallback
 
-# CPU images and x86 GPU images ship a single llama-cpp-cpu-all built with ggml
+# CPU images and most x86 GPU images ship a single llama-cpp-cpu-all built with ggml
 # CPU_ALL_VARIANTS: ggml's backend registry dlopens the best libggml-cpu-*.so for this
 # host, so no shell-side AVX probing. GPU arm64 images still ship llama-cpp-fallback
-# until their builder toolchains support ggml's complete arm variant matrix.
+# until their builder toolchains support ggml's complete arm variant matrix, and so do
+# the SYCL images, whose icpx compiler hangs on the sapphirerapids variant.
 if [ -e "$CURDIR"/llama-cpp-cpu-all ]; then
 	BINARY=llama-cpp-cpu-all
 fi

backend/cpp/turboquant/run.sh

@@ -12,11 +12,12 @@ grep -e "flags" /proc/cpuinfo | head -1
 
 BINARY=turboquant-fallback
 
-# CPU images and x86 GPU images ship a single turboquant-cpu-all built with ggml
+# CPU images and most x86 GPU images ship a single turboquant-cpu-all built with ggml
 # CPU_ALL_VARIANTS: ggml's
 # backend registry dlopens the best libggml-cpu-*.so for this host, so no shell-side
 # probing. GPU arm64 images still ship turboquant-fallback until their builder toolchains
-# support ggml's complete arm variant matrix.
+# support ggml's complete arm variant matrix, and so do the SYCL images, whose icpx
+# compiler hangs on the sapphirerapids variant.
 if [ -e "$CURDIR"/turboquant-cpu-all ]; then
 	BINARY=turboquant-cpu-all
 fi

backend/go/crispasr/Makefile

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
 
 # CrispASR version (release tag)
 CRISPASR_REPO?=https://github.com/CrispStrobe/CrispASR
-CRISPASR_VERSION?=fcb79282a6bc52e13d858026c42b24fb6e63c97a
+CRISPASR_VERSION?=fe3caf8e363b27572dbdd1a9d37083f25e6decda
 SO_TARGET?=libgocrispasr.so
 
 CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

backend/go/vllm-cpp/Makefile

@@ -11,7 +11,7 @@ JOBS?=$(shell nproc --ignore=1 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || e
 
 # vllm.cpp version
 VLLM_CPP_REPO?=https://github.com/mudler/vllm.cpp
-VLLM_CPP_VERSION?=9e1c9025ae61167a3335454d7cc0de6093c21845
+VLLM_CPP_VERSION?=a42b8187caff02c570c28e19e4dc2b1d7f55ed14
 
 # The backend consumes only the stable C ABI (libvllm + include/vllm.h), so the
 # server, examples and tests of the engine are never built here.
@@ -56,6 +56,12 @@ endif
 UNAME_S := $(shell uname -s)
 ifeq ($(UNAME_S),Darwin)
 	LIB=libvllm.dylib
+	# Apple Clang diagnoses a pair of constant-folded array bounds in the Metal
+	# build as a GNU extension. Disable that diagnostic for both Objective-C and
+	# C++ because vllm.cpp appends target-local -Werror after these global flags.
+	CMAKE_ARGS+=-DCMAKE_CXX_FLAGS=-Wno-gnu-folding-constant
+	CMAKE_ARGS+=-DCMAKE_OBJC_FLAGS=-Wno-gnu-folding-constant
+	CMAKE_ARGS+=-DCMAKE_OBJCXX_FLAGS=-Wno-gnu-folding-constant
 else
 	LIB=libvllm.so
 endif

backend/go/vllm-cpp/backend.go

@@ -109,6 +109,16 @@ func (v *VllmCpp) Load(opts *pb.ModelOptions) error {
 
 	v.opts = parseOptions(opts)
 
+	// A DFlash draft is a second checkpoint the engine opens by path, and the
+	// engine never downloads one. Resolve it against LocalAI's models directory
+	// now so a repo-id spelling works, and so a missing draft fails here with an
+	// actionable message rather than as an HF-cache miss inside the load.
+	resolvedSpec, err := resolveDraftModelPath(v.opts.speculativeConfig, opts.ModelPath)
+	if err != nil {
+		return err
+	}
+	v.opts.speculativeConfig = resolvedSpec
+
 	mp := defaultModelParams()
 	if v.opts.blockSize > 0 {
 		mp.BlockSize = v.opts.blockSize
@@ -116,34 +126,62 @@ func (v *VllmCpp) Load(opts *pb.ModelOptions) error {
 	if v.opts.numBlocks > 0 {
 		mp.NumBlocks = v.opts.numBlocks
 	}
+	// Sequence-length precedence, narrowest source last: context_size is the
+	// generic LocalAI knob every backend honours, max_model_len is the
+	// vLLM-specific one, and engine_args.max_model_len is the explicit
+	// vllm-cpp override.
 	if opts.ContextSize > 0 {
 		mp.MaxModelLen = opts.ContextSize
 	}
+	if opts.MaxModelLen > 0 {
+		mp.MaxModelLen = opts.MaxModelLen
+	}
+	if v.opts.maxModelLen > 0 {
+		mp.MaxModelLen = v.opts.maxModelLen
+	}
 	if v.opts.maxNumSeqs > 0 {
 		mp.MaxNumSeqs = v.opts.maxNumSeqs
 	}
+	if v.opts.maxNumBatchedTokens > 0 {
+		mp.MaxNumBatchedTokens = v.opts.maxNumBatchedTokens
+	}
+	mp.EnablePrefixCaching = v.opts.enablePrefixCaching
+	mp.EnableJumpForward = v.opts.enableJumpForward
 
+	// Every string below is borrowed by C for the duration of the load call
+	// only (the library copies what it keeps), so the backing slices just have
+	// to outlive vllmEngineLoad - hence the single KeepAlive after it.
 	modelC := cString(model)
 	mp.ModelPath = uintptr(unsafe.Pointer(&modelC[0])) // #nosec G103 -- borrowed by C for the load call only
-	var toolParserC, reasoningParserC []byte
-	if v.opts.toolParser != "" {
-		toolParserC = cString(v.opts.toolParser)
-		mp.ToolParser = uintptr(unsafe.Pointer(&toolParserC[0])) // #nosec G103 -- borrowed by C for the load call only
-	}
-	if v.opts.reasoningParser != "" {
-		reasoningParserC = cString(v.opts.reasoningParser)
-		mp.ReasoningParser = uintptr(unsafe.Pointer(&reasoningParserC[0])) // #nosec G103 -- borrowed by C for the load call only
+	keep := [][]byte{modelC}
+	setStr := func(dst *uintptr, s string) {
+		if s == "" {
+			return
+		}
+		b := cString(s)
+		keep = append(keep, b)
+		*dst = uintptr(unsafe.Pointer(&b[0])) // #nosec G103 -- borrowed by C for the load call only
 	}
+	setStr(&mp.ToolParser, v.opts.toolParser)
+	setStr(&mp.ReasoningParser, v.opts.reasoningParser)
+	setStr(&mp.SpeculativeConfig, v.opts.speculativeConfig)
+	setStr(&mp.KVTransferConfig, v.opts.kvTransferConfig)
+	setStr(&mp.SchedulingPolicy, v.opts.schedulingPolicy)
+	setStr(&mp.TokenizerConfigPath, v.opts.tokenizerConfigPath)
 
 	xlog.Info("[vllm-cpp] Load", "model", model, "engine", vllmVersion(),
 		"blockSize", mp.BlockSize, "numBlocks", mp.NumBlocks,
-		"maxModelLen", mp.MaxModelLen, "maxNumSeqs", mp.MaxNumSeqs)
+		"maxModelLen", mp.MaxModelLen, "maxNumSeqs", mp.MaxNumSeqs,
+		"maxNumBatchedTokens", mp.MaxNumBatchedTokens,
+		"prefixCaching", triStateName(mp.EnablePrefixCaching),
+		"jumpForward", triStateName(mp.EnableJumpForward),
+		"schedulingPolicy", v.opts.schedulingPolicy,
+		"speculativeConfig", v.opts.speculativeConfig,
+		"kvTransferConfig", v.opts.kvTransferConfig)
 
 	var engine uintptr
 	rc := vllmEngineLoad(unsafe.Pointer(&mp), unsafe.Pointer(&engine)) // #nosec G103 -- POD out-params
-	runtime.KeepAlive(modelC)
-	runtime.KeepAlive(toolParserC)
-	runtime.KeepAlive(reasoningParserC)
+	runtime.KeepAlive(keep)
 	if rc != vllmOK {
 		return fmt.Errorf("vllm-cpp: engine load failed: %s", vllmLastError())
 	}

backend/go/vllm-cpp/govllmcpp.go

@@ -1,6 +1,6 @@
 package main
 
-// purego bindings for the vllm.cpp stable C ABI (include/vllm.h, ABI v2).
+// purego bindings for the vllm.cpp stable C ABI (include/vllm.h, ABI v10).
 //
 // The structs below are hand-mirrored PODs of the C declarations, with
 // explicit padding so the Go layout matches the C layout on linux/darwin
@@ -18,23 +18,56 @@ import (
 )
 
 // abiVersion is the VLLM_ABI_VERSION this file mirrors (vllm.h).
-const abiVersion = 5
+const abiVersion = 10
+
+// The ABI's tri-state toggles (enable_prefix_caching ABI v7,
+// enable_jump_forward ABI v10) share one encoding: 0 is NOT "off", it is
+// "defer" - to the model capability for prefix caching, to the environment for
+// jump forward. Only 2 is an explicit off.
+const (
+	triStateDefer int32 = 0
+	triStateOn    int32 = 1
+	triStateOff   int32 = 2
+)
+
+// triStateName renders a tri-state for the load log line, where "0" would
+// otherwise read as "off" rather than "whatever the default resolves to".
+func triStateName(state int32) string {
+	switch state {
+	case triStateOn:
+		return "on"
+	case triStateOff:
+		return "off"
+	default:
+		return "model-default"
+	}
+}
 
 // vllm_status (vllm.h).
 const (
 	vllmOK = 0
 )
 
-// cModelParams mirrors vllm_model_params.
+// cModelParams mirrors vllm_model_params. The int32 fields sit in pairs so the
+// interior needs no padding on LP64, but the struct is 8-aligned (it holds
+// pointers) and ends on a lone int32, so the trailing pad is explicit. Offsets
+// and total size are asserted in vllmcpp_test.go.
 type cModelParams struct {
 	ModelPath           uintptr // const char*
-	TokenizerConfigPath uintptr // const char*
+	TokenizerConfigPath uintptr // const char*; NULL = <model_dir>/... (ABI v9)
 	BlockSize           int32
 	NumBlocks           int32
 	MaxModelLen         int32
 	MaxNumSeqs          int32
 	ToolParser          uintptr // const char*; NULL = auto-detect (ABI v4)
 	ReasoningParser     uintptr // const char*; NULL = auto-detect (ABI v5)
+	SpeculativeConfig   uintptr // const char* JSON; NULL = no speculation (ABI v6)
+	EnablePrefixCaching int32   // tri-state 0/1/2 (ABI v7)
+	MaxNumBatchedTokens int32   // <= 0 = per-arch default (ABI v9)
+	SchedulingPolicy    uintptr // const char*; NULL = "fcfs" (ABI v9)
+	KVTransferConfig    uintptr // const char* JSON; NULL = no connector (ABI v9)
+	EnableJumpForward   int32   // tri-state 0/1/2 (ABI v10)
+	_                   [4]byte // trailing pad to the struct's 8-byte alignment
 }
 
 // cSamplingParams mirrors vllm_sampling_params (ABI v2, structured fields
@@ -65,6 +98,12 @@ type cSamplingParams struct {
 	StructuredGrammar    uintptr // const char*
 	StructuredJSONObject int32
 	_                    [4]byte
+	// ABI v8 tail. LocalAI installs no custom logits processor, but the fields
+	// MUST be mirrored: the C side reads them off the pointer we hand it, so a
+	// Go struct that stopped at StructuredJSONObject would have the engine read
+	// 16 bytes past our allocation and call whatever garbage sat there.
+	LogitsProcessor         uintptr // vllm_logits_processor; NULL = none
+	LogitsProcessorUserData uintptr // void*
 }
 
 // cCompletion mirrors vllm_completion.

backend/go/vllm-cpp/options.go

@@ -1,30 +1,80 @@
 package main
 
-// Engine-sizing knobs carried through the model config's free-form
-// `options:` list ("key:value" entries), mirroring how the other in-house
-// backends pass engine-specific settings that have no proto field.
+// Load-time engine configuration, from two config surfaces:
+//
+//   - `engine_args:` (ModelOptions.EngineArgs, a JSON object) is the canonical
+//     one. Keys are spelled exactly as vLLM's own CLI flags, so a config written
+//     against vLLM works verbatim here - `speculative_config` and
+//     `kv_transfer_config` in particular take the same JSON documents vLLM's
+//     --speculative-config / --kv-transfer-config accept, and are handed to the
+//     engine unparsed.
+//   - `options:` (the free-form "key:value" list) is the older surface this
+//     backend shipped with. It is still honoured so existing configs keep
+//     working; engine_args wins on any key set in both.
+//
+// Anything unrecognised is ignored rather than fatal: the engine validates the
+// documents it is given and reports a precise error at load, and a config that
+// also carries knobs for a different backend must not fail the load here.
 
 import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path"
+	"path/filepath"
 	"strconv"
 	"strings"
 
 	pb "github.com/mudler/LocalAI/pkg/grpc/proto"
+	"github.com/mudler/xlog"
 )
 
 type loadOptions struct {
 	blockSize  int32 // KV block size (tokens/block); engine default 32.
 	numBlocks  int32 // KV blocks to allocate; engine default 256.
 	maxNumSeqs int32 // max concurrent sequences; engine default 8.
+	// Max sequence length. Also settable through the model config's
+	// context_size / max_model_len; see Load for the precedence.
+	maxModelLen int32
+	// Per-step chunked-prefill token budget (ABI v9). 0 = the engine's
+	// bounded per-arch default.
+	maxNumBatchedTokens int32
+	// Automatic prefix caching tri-state (ABI v7): 0 = the model-capability
+	// default, 1 = force on, 2 = force off.
+	enablePrefixCaching int32
+	// Jump-forward decoding tri-state (ABI v10), SGLang's grammar-speed subset:
+	// 0 = defer to the environment (VT_ENABLE_JUMP_FORWARD, default off),
+	// 1 = force on, 2 = force off.
+	enableJumpForward int32
+	// Scheduler admission policy (ABI v9): "" = fcfs, else fcfs|priority|lpm.
+	schedulingPolicy string
 	// Engine-side parser selection (ABI v4/v5). Empty = the engine
 	// auto-detects from the chat template; "none" disables the reasoning
 	// split; unknown names fail the first chat call.
 	toolParser      string
 	reasoningParser string
+	// Speculative decoding (ABI v6), as vLLM's --speculative-config JSON:
+	// {"method":"mtp"|"dflash"|"ngram", ...}. Empty = no speculation.
+	speculativeConfig string
+	// External KV connector / LMCache (ABI v9), as vLLM's --kv-transfer-config
+	// JSON. Empty = no connector.
+	kvTransferConfig string
+	// Override for the tokenizer_config.json the chat template is read from
+	// (ABI v9). Empty = <model_dir>/tokenizer_config.json.
+	tokenizerConfigPath string
 }
 
 func parseOptions(opts *pb.ModelOptions) loadOptions {
 	lo := loadOptions{}
-	for _, o := range opts.GetOptions() {
+	applyOptionsList(&lo, opts.GetOptions())
+	applyEngineArgs(&lo, opts.GetEngineArgs())
+	return lo
+}
+
+// applyOptionsList reads the legacy free-form "key:value" list. strings.Cut
+// splits on the FIRST colon only, so a JSON object value survives intact.
+func applyOptionsList(lo *loadOptions, options []string) {
+	for _, o := range options {
 		k, v, found := strings.Cut(o, ":")
 		if !found {
 			continue
@@ -36,13 +86,211 @@ func parseOptions(opts *pb.ModelOptions) loadOptions {
 			lo.numBlocks = parseInt32(v, lo.numBlocks)
 		case "max_num_seqs":
 			lo.maxNumSeqs = parseInt32(v, lo.maxNumSeqs)
-		case "tool_parser":
+		case "max_num_batched_tokens":
+			lo.maxNumBatchedTokens = parseInt32(v, lo.maxNumBatchedTokens)
+		case "max_model_len":
+			lo.maxModelLen = parseInt32(v, lo.maxModelLen)
+		case "scheduling_policy", "schedule_policy":
+			lo.schedulingPolicy = strings.TrimSpace(v)
+		case "tool_parser", "tool_call_parser":
 			lo.toolParser = strings.TrimSpace(v)
 		case "reasoning_parser":
 			lo.reasoningParser = strings.TrimSpace(v)
+		case "speculative_config":
+			lo.speculativeConfig = strings.TrimSpace(v)
+		case "kv_transfer_config":
+			lo.kvTransferConfig = strings.TrimSpace(v)
+		case "tokenizer_config", "tokenizer_config_path":
+			lo.tokenizerConfigPath = strings.TrimSpace(v)
+		case "enable_prefix_caching", "enable_radix_attention":
+			if b, err := strconv.ParseBool(strings.TrimSpace(v)); err == nil {
+				lo.enablePrefixCaching = boolTriState(b)
+			}
+		case "enable_jump_forward":
+			if b, err := strconv.ParseBool(strings.TrimSpace(v)); err == nil {
+				lo.enableJumpForward = boolTriState(b)
+			}
 		}
 	}
-	return lo
+}
+
+// applyEngineArgs overlays the `engine_args:` JSON object. A document that does
+// not parse is logged and skipped: engine_args is shared with the other engines
+// (the vLLM and SGLang backends read the same field), so a stray key must not
+// take the model down.
+func applyEngineArgs(lo *loadOptions, engineArgs string) {
+	if strings.TrimSpace(engineArgs) == "" {
+		return
+	}
+	var args map[string]any
+	if err := json.Unmarshal([]byte(engineArgs), &args); err != nil {
+		xlog.Warn("[vllm-cpp] ignoring unparseable engine_args", "error", err)
+		return
+	}
+	for k, v := range args {
+		switch k {
+		case "block_size":
+			lo.blockSize = jsonInt32(v, lo.blockSize)
+		case "num_blocks":
+			lo.numBlocks = jsonInt32(v, lo.numBlocks)
+		case "max_num_seqs":
+			lo.maxNumSeqs = jsonInt32(v, lo.maxNumSeqs)
+		case "max_num_batched_tokens":
+			lo.maxNumBatchedTokens = jsonInt32(v, lo.maxNumBatchedTokens)
+		case "max_model_len":
+			lo.maxModelLen = jsonInt32(v, lo.maxModelLen)
+		case "scheduling_policy", "schedule_policy":
+			lo.schedulingPolicy = jsonString(v, lo.schedulingPolicy)
+		case "tool_parser", "tool_call_parser":
+			lo.toolParser = jsonString(v, lo.toolParser)
+		case "reasoning_parser":
+			lo.reasoningParser = jsonString(v, lo.reasoningParser)
+		case "tokenizer_config", "tokenizer_config_path":
+			lo.tokenizerConfigPath = jsonString(v, lo.tokenizerConfigPath)
+		case "speculative_config":
+			lo.speculativeConfig = jsonDocument(v, lo.speculativeConfig, k)
+		case "kv_transfer_config":
+			lo.kvTransferConfig = jsonDocument(v, lo.kvTransferConfig, k)
+		case "enable_prefix_caching", "enable_radix_attention":
+			if b, ok := v.(bool); ok {
+				lo.enablePrefixCaching = boolTriState(b)
+			}
+		case "enable_jump_forward":
+			if b, ok := v.(bool); ok {
+				lo.enableJumpForward = boolTriState(b)
+			}
+		default:
+			xlog.Debug("[vllm-cpp] ignoring unknown engine_args key", "key", k)
+		}
+	}
+}
+
+// boolTriState maps a YAML/JSON boolean onto the ABI's tri-state encoding. An
+// explicit `false` must reach the engine as force-OFF (2), NOT as the 0 that
+// means "defer". The difference is real in both directions: prefix caching
+// defaults ON for dense archs and OFF for hybrid ones, and jump forward defers
+// to VT_ENABLE_JUMP_FORWARD.
+func boolTriState(on bool) int32 {
+	if on {
+		return triStateOn
+	}
+	return triStateOff
+}
+
+// jsonDocument normalises an object-valued engine_args entry to a JSON string
+// for the C ABI. YAML nesting arrives as a map (the natural spelling); a
+// pre-encoded JSON string is accepted too, since a config round-tripped through
+// a flat store may carry it that way.
+func jsonDocument(v any, fallback string, key string) string {
+	switch t := v.(type) {
+	case string:
+		if strings.TrimSpace(t) == "" {
+			return fallback
+		}
+		return t
+	default:
+		buf, err := json.Marshal(t)
+		if err != nil {
+			xlog.Warn("[vllm-cpp] ignoring unencodable engine_args value", "key", key, "error", err)
+			return fallback
+		}
+		return string(buf)
+	}
+}
+
+func jsonString(v any, fallback string) string {
+	s, ok := v.(string)
+	if !ok {
+		return fallback
+	}
+	return strings.TrimSpace(s)
+}
+
+// jsonInt32 accepts the float64 a JSON number decodes to, plus the string
+// spelling a YAML config may produce. Non-positive values keep the fallback:
+// every knob this covers uses "<= 0 means the engine default".
+func jsonInt32(v any, fallback int32) int32 {
+	switch t := v.(type) {
+	case float64:
+		if t <= 0 || t > 1<<31-1 {
+			return fallback
+		}
+		return int32(t)
+	case string:
+		return parseInt32(t, fallback)
+	default:
+		return fallback
+	}
+}
+
+// resolveDraftModelPath rewrites a DFlash draft reference into an absolute path
+// the engine can actually open.
+//
+// The engine resolves `speculative_config.model` against a directory containing
+// config.json, or against ~/.cache/huggingface/hub/models--<org>--<repo>/
+// snapshots/* - and it NEVER downloads. LocalAI keeps models in its own
+// directory, so a bare HF repo id (the spelling the vLLM docs teach) misses the
+// HF cache and dies deep in the load with "draft checkpoint not found", which
+// reads like a broken checkpoint rather than a missing download.
+//
+// So: try the reference as given, then the last path segment under the models
+// dir (`z-lab/Qwen3.6-27B-DFlash` -> `<models>/Qwen3.6-27B-DFlash`, which is
+// what LocalAI's own downloader produces), then the whole reference under the
+// models dir. If none exist, fail HERE with a message naming both what was
+// asked for and where we looked.
+//
+// mtp and ngram carry no separate draft checkpoint, so they pass through. A
+// document that does not parse also passes through: the engine owns config
+// validation and produces the better error.
+func resolveDraftModelPath(speculativeConfig, modelsDir string) (string, error) {
+	if strings.TrimSpace(speculativeConfig) == "" {
+		return speculativeConfig, nil
+	}
+	var spec map[string]any
+	if err := json.Unmarshal([]byte(speculativeConfig), &spec); err != nil {
+		return speculativeConfig, nil
+	}
+	if method, _ := spec["method"].(string); !strings.EqualFold(method, "dflash") {
+		return speculativeConfig, nil
+	}
+
+	ref, _ := spec["model"].(string)
+	ref = strings.TrimSpace(ref)
+	if ref == "" {
+		return "", fmt.Errorf(
+			"vllm-cpp: speculative_config method %q requires a \"model\" key naming the draft checkpoint", "dflash")
+	}
+
+	candidates := []string{ref}
+	if modelsDir != "" {
+		if base := path.Base(filepath.ToSlash(ref)); base != "" && base != "." && base != "/" {
+			candidates = append(candidates, filepath.Join(modelsDir, base))
+		}
+		candidates = append(candidates, filepath.Join(modelsDir, filepath.FromSlash(ref)))
+	}
+
+	for _, c := range candidates {
+		if _, err := os.Stat(filepath.Join(c, "config.json")); err != nil {
+			continue
+		}
+		abs, err := filepath.Abs(c)
+		if err != nil {
+			abs = c
+		}
+		spec["model"] = abs
+		out, err := json.Marshal(spec)
+		if err != nil {
+			return "", fmt.Errorf("vllm-cpp: re-encoding speculative_config: %w", err)
+		}
+		xlog.Info("[vllm-cpp] resolved DFlash draft checkpoint", "reference", ref, "path", abs)
+		return string(out), nil
+	}
+
+	return "", fmt.Errorf(
+		"vllm-cpp: DFlash draft checkpoint %q not found (looked in: %s). "+
+			"The engine does not download drafts - install the draft model into LocalAI first, "+
+			"or set speculative_config.model to an absolute path to a directory containing config.json",
+		ref, strings.Join(candidates, ", "))
 }
 
 func parseInt32(s string, fallback int32) int32 {

backend/go/vllm-cpp/vllmcpp_test.go

@@ -16,7 +16,7 @@ func TestVllmCpp(t *testing.T) {
 	RunSpecs(t, "vllm-cpp suite")
 }
 
-// The Go POD mirrors must match the C struct layout of vllm.h (ABI v2)
+// The Go POD mirrors must match the C struct layout of vllm.h (ABI v9)
 // byte-for-byte: these offsets are the C offsets on LP64 (linux/darwin
 // amd64+arm64). A failure here means govllmcpp.go drifted from vllm.h.
 var _ = Describe("C ABI struct mirrors", func() {
@@ -30,10 +30,18 @@ var _ = Describe("C ABI struct mirrors", func() {
 		Expect(unsafe.Offsetof(p.MaxNumSeqs)).To(Equal(uintptr(28)))
 		Expect(unsafe.Offsetof(p.ToolParser)).To(Equal(uintptr(32)))
 		Expect(unsafe.Offsetof(p.ReasoningParser)).To(Equal(uintptr(40)))
-		Expect(unsafe.Sizeof(p)).To(Equal(uintptr(48)))
+		Expect(unsafe.Offsetof(p.SpeculativeConfig)).To(Equal(uintptr(48)))
+		Expect(unsafe.Offsetof(p.EnablePrefixCaching)).To(Equal(uintptr(56)))
+		Expect(unsafe.Offsetof(p.MaxNumBatchedTokens)).To(Equal(uintptr(60)))
+		Expect(unsafe.Offsetof(p.SchedulingPolicy)).To(Equal(uintptr(64)))
+		Expect(unsafe.Offsetof(p.KVTransferConfig)).To(Equal(uintptr(72)))
+		Expect(unsafe.Offsetof(p.EnableJumpForward)).To(Equal(uintptr(80)))
+		// 88, not 84: the struct is 8-aligned (it holds pointers), so the
+		// trailing int32 is padded out. Go pads identically.
+		Expect(unsafe.Sizeof(p)).To(Equal(uintptr(88)))
 	})
 
-	It("cSamplingParams matches vllm_sampling_params (ABI v2)", func() {
+	It("cSamplingParams matches vllm_sampling_params (ABI v8)", func() {
 		var p cSamplingParams
 		Expect(unsafe.Offsetof(p.Temperature)).To(Equal(uintptr(0)))
 		Expect(unsafe.Offsetof(p.TopP)).To(Equal(uintptr(4)))
@@ -55,7 +63,9 @@ var _ = Describe("C ABI struct mirrors", func() {
 		Expect(unsafe.Offsetof(p.NStructuredChoice)).To(Equal(uintptr(96)))
 		Expect(unsafe.Offsetof(p.StructuredGrammar)).To(Equal(uintptr(104)))
 		Expect(unsafe.Offsetof(p.StructuredJSONObject)).To(Equal(uintptr(112)))
-		Expect(unsafe.Sizeof(p)).To(Equal(uintptr(120)))
+		Expect(unsafe.Offsetof(p.LogitsProcessor)).To(Equal(uintptr(120)))
+		Expect(unsafe.Offsetof(p.LogitsProcessorUserData)).To(Equal(uintptr(128)))
+		Expect(unsafe.Sizeof(p)).To(Equal(uintptr(136)))
 	})
 
 	It("cCompletion matches vllm_completion", func() {
@@ -68,6 +78,23 @@ var _ = Describe("C ABI struct mirrors", func() {
 	})
 })
 
+// Pin/mirror skew is the failure mode this backend is most exposed to: the Go
+// PODs above are hand-written against one VLLM_ABI_VERSION, and the Makefile
+// pins the vllm.cpp commit that produces it. This spec catches drift without
+// needing model weights - set VLLM_CPP_LIBRARY to a built libvllm and it binds
+// every symbol and compares the library's reported ABI against the mirrors'.
+var _ = Describe("real library ABI handshake", func() {
+	It("binds every symbol and reports the ABI the mirrors were written against", func() {
+		lib := os.Getenv("VLLM_CPP_LIBRARY")
+		if lib == "" {
+			Skip("VLLM_CPP_LIBRARY not set; skipping the real-library handshake")
+		}
+		Expect(registerLib(lib)).To(Succeed())
+		Expect(vllmABIVersion()).To(Equal(int32(abiVersion)))
+		Expect(vllmVersion()).NotTo(BeEmpty())
+	})
+})
+
 var _ = Describe("parseOptions", func() {
 	It("extracts the engine sizing knobs", func() {
 		lo := parseOptions(&pb.ModelOptions{Options: []string{
@@ -83,6 +110,129 @@ var _ = Describe("parseOptions", func() {
 		}})
 		Expect(lo).To(Equal(loadOptions{}))
 	})
+
+	It("carries a speculative_config JSON value through the legacy options list", func() {
+		// strings.Cut splits on the FIRST colon only, so a JSON object value
+		// survives the "key:value" spelling intact.
+		lo := parseOptions(&pb.ModelOptions{Options: []string{
+			`speculative_config:{"method":"mtp","num_speculative_tokens":1}`,
+		}})
+		Expect(lo.speculativeConfig).To(Equal(`{"method":"mtp","num_speculative_tokens":1}`))
+	})
+})
+
+var _ = Describe("engine_args", func() {
+	It("maps every load knob onto the C model params", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{
+			"block_size": 64,
+			"num_blocks": 1024,
+			"max_model_len": 16384,
+			"max_num_seqs": 32,
+			"max_num_batched_tokens": 8192,
+			"enable_prefix_caching": true,
+			"scheduling_policy": "lpm",
+			"tool_parser": "qwen3",
+			"reasoning_parser": "deepseek_r1",
+			"tokenizer_config": "/models/tok/tokenizer_config.json"
+		}`})
+		Expect(lo.blockSize).To(Equal(int32(64)))
+		Expect(lo.numBlocks).To(Equal(int32(1024)))
+		Expect(lo.maxModelLen).To(Equal(int32(16384)))
+		Expect(lo.maxNumSeqs).To(Equal(int32(32)))
+		Expect(lo.maxNumBatchedTokens).To(Equal(int32(8192)))
+		Expect(lo.enablePrefixCaching).To(Equal(int32(1)))
+		Expect(lo.schedulingPolicy).To(Equal("lpm"))
+		Expect(lo.toolParser).To(Equal("qwen3"))
+		Expect(lo.reasoningParser).To(Equal("deepseek_r1"))
+		Expect(lo.tokenizerConfigPath).To(Equal("/models/tok/tokenizer_config.json"))
+	})
+
+	It("re-marshals a nested speculative_config object to JSON for the engine", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{
+			"speculative_config": {"method": "mtp", "num_speculative_tokens": 1}
+		}`})
+		Expect(lo.speculativeConfig).To(MatchJSON(`{"method":"mtp","num_speculative_tokens":1}`))
+	})
+
+	It("re-marshals a nested kv_transfer_config object (LMCache) to JSON", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{
+			"kv_transfer_config": {
+				"kv_connector": "LMCacheConnector",
+				"kv_role": "kv_both",
+				"kv_connector_extra_config": {"host": "127.0.0.1", "port": 65432}
+			}
+		}`})
+		Expect(lo.kvTransferConfig).To(MatchJSON(`{
+			"kv_connector":"LMCacheConnector",
+			"kv_role":"kv_both",
+			"kv_connector_extra_config":{"host":"127.0.0.1","port":65432}
+		}`))
+	})
+
+	It("accepts a pre-encoded JSON string for the object-valued knobs", func() {
+		// A config written by hand (or round-tripped through a flat store) may
+		// carry the object as a string; both spellings reach the engine the same.
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{
+			"speculative_config": "{\"method\":\"ngram\",\"num_speculative_tokens\":4}"
+		}`})
+		Expect(lo.speculativeConfig).To(MatchJSON(`{"method":"ngram","num_speculative_tokens":4}`))
+	})
+
+	It("maps enable_prefix_caching false onto the force-OFF tri-state", func() {
+		// The C ABI tri-state is 0=model default, 1=on, 2=off, so an explicit
+		// `false` must NOT collapse to the 0 that means "let the model decide".
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{"enable_prefix_caching": false}`})
+		Expect(lo.enablePrefixCaching).To(Equal(int32(2)))
+	})
+
+	It("leaves the prefix-caching tri-state at the model default when unset", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{"max_num_seqs": 4}`})
+		Expect(lo.enablePrefixCaching).To(Equal(int32(0)))
+	})
+
+	It("accepts the radix-attention alias upstream documents for prefix caching", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{"enable_radix_attention": true}`})
+		Expect(lo.enablePrefixCaching).To(Equal(int32(1)))
+	})
+
+	It("maps enable_jump_forward onto its own tri-state", func() {
+		// ABI v10. Same tri-state shape as prefix caching, and the same trap:
+		// an explicit false must be force-OFF (2), not the 0 that defers to the
+		// environment.
+		on := parseOptions(&pb.ModelOptions{EngineArgs: `{"enable_jump_forward": true}`})
+		Expect(on.enableJumpForward).To(Equal(int32(1)))
+		off := parseOptions(&pb.ModelOptions{EngineArgs: `{"enable_jump_forward": false}`})
+		Expect(off.enableJumpForward).To(Equal(int32(2)))
+		unset := parseOptions(&pb.ModelOptions{EngineArgs: `{"max_num_seqs": 4}`})
+		Expect(unset.enableJumpForward).To(Equal(int32(0)))
+	})
+
+	It("reads enable_jump_forward from the legacy options list too", func() {
+		lo := parseOptions(&pb.ModelOptions{Options: []string{"enable_jump_forward:true"}})
+		Expect(lo.enableJumpForward).To(Equal(int32(1)))
+	})
+
+	It("lets engine_args override the legacy options list", func() {
+		lo := parseOptions(&pb.ModelOptions{
+			Options:    []string{"max_num_seqs:8", "block_size:16"},
+			EngineArgs: `{"max_num_seqs": 64}`,
+		})
+		Expect(lo.maxNumSeqs).To(Equal(int32(64))) // engine_args wins
+		Expect(lo.blockSize).To(Equal(int32(16)))  // untouched keys survive
+	})
+
+	It("ignores malformed engine_args rather than failing the load", func() {
+		lo := parseOptions(&pb.ModelOptions{
+			Options:    []string{"max_num_seqs:8"},
+			EngineArgs: `{not json`,
+		})
+		Expect(lo.maxNumSeqs).To(Equal(int32(8)))
+	})
+
+	It("ignores unknown keys", func() {
+		lo := parseOptions(&pb.ModelOptions{EngineArgs: `{"gpu_memory_utilization": 0.9}`})
+		Expect(lo).To(Equal(loadOptions{}))
+	})
 })
 
 var _ = Describe("samplingFromPredict", func() {
@@ -135,6 +285,91 @@ var _ = Describe("samplingFromPredict", func() {
 	})
 })
 
+// The engine resolves speculative_config.model against a local directory or
+// ~/.cache/huggingface/hub ONLY - it never downloads. LocalAI keeps models in
+// its own directory, so a bare repo id would miss the HF cache and fail deep in
+// the load with a confusing "draft checkpoint not found". Resolve it here.
+var _ = Describe("resolveDraftModelPath", func() {
+	var modelsDir string
+
+	BeforeEach(func() {
+		modelsDir = GinkgoT().TempDir()
+	})
+
+	// draftDir creates a plausible draft checkpoint under models/.
+	draftDir := func(name string) string {
+		d := filepath.Join(modelsDir, name)
+		Expect(os.MkdirAll(d, 0o750)).To(Succeed())
+		Expect(os.WriteFile(filepath.Join(d, "config.json"), []byte("{}"), 0o600)).To(Succeed())
+		return d
+	}
+
+	It("rewrites a repo id to the matching directory in the models dir", func() {
+		want := draftDir("Qwen3.6-27B-DFlash")
+		spec := `{"method":"dflash","model":"z-lab/Qwen3.6-27B-DFlash"}`
+		out, err := resolveDraftModelPath(spec, modelsDir)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(out).To(MatchJSON(`{"method":"dflash","model":"` + want + `"}`))
+	})
+
+	It("rewrites a models-dir-relative path", func() {
+		want := draftDir("drafts__dflash")
+		spec := `{"method":"dflash","model":"drafts__dflash"}`
+		out, err := resolveDraftModelPath(spec, modelsDir)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(out).To(ContainSubstring(want))
+	})
+
+	It("leaves an absolute path that already resolves alone", func() {
+		abs := draftDir("elsewhere")
+		spec := `{"method":"dflash","model":"` + abs + `"}`
+		out, err := resolveDraftModelPath(spec, modelsDir)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(out).To(MatchJSON(spec))
+	})
+
+	It("fails with an actionable error when the draft is nowhere on disk", func() {
+		// Silently passing the repo id through would surface as an HF-cache
+		// miss inside the engine, which reads as "your model is broken".
+		spec := `{"method":"dflash","model":"z-lab/Not-Downloaded"}`
+		_, err := resolveDraftModelPath(spec, modelsDir)
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("z-lab/Not-Downloaded"))
+		Expect(err.Error()).To(ContainSubstring(modelsDir))
+	})
+
+	It("requires a model key for dflash", func() {
+		_, err := resolveDraftModelPath(`{"method":"dflash"}`, modelsDir)
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("model"))
+	})
+
+	It("leaves mtp and ngram configs untouched", func() {
+		// Neither has a separate draft checkpoint to resolve.
+		for _, spec := range []string{
+			`{"method":"mtp"}`,
+			`{"method":"ngram","num_speculative_tokens":4}`,
+		} {
+			out, err := resolveDraftModelPath(spec, modelsDir)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(out).To(MatchJSON(spec))
+		}
+	})
+
+	It("passes a malformed document through for the engine to reject", func() {
+		// The engine owns config validation and produces the better message.
+		out, err := resolveDraftModelPath(`{not json`, modelsDir)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(out).To(Equal(`{not json`))
+	})
+
+	It("is a no-op on an empty config", func() {
+		out, err := resolveDraftModelPath("", modelsDir)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(out).To(BeEmpty())
+	})
+})
+
 var _ = Describe("validModelPath", func() {
 	It("accepts a .gguf file", func() {
 		dir := GinkgoT().TempDir()

backend/go/whisper/Makefile

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
 
 # whisper.cpp version
 WHISPER_REPO?=https://github.com/ggml-org/whisper.cpp
-WHISPER_CPP_VERSION?=2ca53bb45e38748d07b310eeb36245a7157ac882
+WHISPER_CPP_VERSION?=64d57d3df5c8dacee098577257edcaa154bf5ef3
 SO_TARGET?=libgowhisper.so
 
 CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

backend/python/diffusers/backend.py

@@ -883,6 +883,34 @@ class BackendServicer(backend_pb2_grpc.BackendServicer):
 
         return backend_pb2.Result(message="Media generated", success=True)
 
+    def UpscaleImage(self, request, context):
+        try:
+            if not request.src:
+                return backend_pb2.Result(success=False, message="No source image provided")
+            if not request.dst:
+                return backend_pb2.Result(success=False, message="No destination path provided")
+
+            scale = request.scale if request.scale > 0 else 2
+            image = Image.open(request.src).convert("RGB")
+
+            # If the loaded pipeline supports upscaling (e.g. StableDiffusionUpscalePipeline),
+            # use it; otherwise fall back to high-quality Lanczos resize.
+            if self.pipe is not None and self.PipelineType in ("StableDiffusionUpscalePipeline", "StableDiffusionLatentUpscalePipeline"):
+                print(f"UpscaleImage: using diffusers upscale pipeline ({self.PipelineType})", file=sys.stderr)
+                upscaled = self.pipe(prompt="", image=image).images[0]
+            else:
+                # Fallback: high-quality Lanczos resize
+                print(f"UpscaleImage: no upscale pipeline loaded, using Lanczos resize (scale={scale})", file=sys.stderr)
+                new_w = image.width * scale
+                new_h = image.height * scale
+                upscaled = image.resize((new_w, new_h), Image.LANCZOS)
+
+            upscaled.save(request.dst)
+            return backend_pb2.Result(message="Image upscaled", success=True)
+        except Exception as e:
+            print(f"UpscaleImage error: {e}", file=sys.stderr)
+            return backend_pb2.Result(success=False, message=str(e))
+
     def GenerateVideo(self, request, context):
         try:
             prompt = request.prompt

backend/python/sglang/requirements-cublas12-after.txt

@@ -15,3 +15,12 @@ sglang[all]>=0.5.11
 # load-bearing for flash-attn-4, and this is the narrower change. Raise the
 # bound once 0.46.0 final ships.
 nvidia-modelopt<0.46
+
+# Same failure mode as the nvidia-modelopt bound above, via a different
+# package. sglang -> flashinfer-python -> cuda-tile, unbounded, and the
+# global --prerelease=allow resolves it to 1.6.0rc3, whose build backend
+# imports wheel_stub without declaring it in build-system.requires. With
+# --no-build-isolation nothing installs it and the build dies with
+# "No module named 'wheel_stub'". 1.5.0 is the newest stable release.
+# Raise the bound once 1.6.0 final ships.
+cuda-tile<1.6

backend/python/sglang/requirements-cublas13-after.txt

@@ -15,3 +15,12 @@ sglang[all]>=0.5.11
 # load-bearing for flash-attn-4, and this is the narrower change. Raise the
 # bound once 0.46.0 final ships.
 nvidia-modelopt<0.46
+
+# Same failure mode as the nvidia-modelopt bound above, via a different
+# package. sglang -> flashinfer-python -> cuda-tile, unbounded, and the
+# global --prerelease=allow resolves it to 1.6.0rc3, whose build backend
+# imports wheel_stub without declaring it in build-system.requires. With
+# --no-build-isolation nothing installs it and the build dies with
+# "No module named 'wheel_stub'". 1.5.0 is the newest stable release.
+# Raise the bound once 1.6.0 final ships.
+cuda-tile<1.6

backend/python/sglang/requirements-l4t13-after.txt

@@ -13,3 +13,12 @@
 # FunctionCallParser, ReasoningParser); the [all] extras are optional
 # accelerators not required at import time.
 sglang>=0.5.11
+
+# Same failure mode the cublas profiles carry an nvidia-modelopt bound for,
+# reached through a different package. sglang -> flashinfer-python ->
+# cuda-tile, unbounded, and the global --prerelease=allow resolves it to
+# 1.6.0rc3, whose build backend imports wheel_stub without declaring it in
+# build-system.requires. With --no-build-isolation nothing installs it and
+# the build dies with "No module named 'wheel_stub'". 1.5.0 is the newest
+# stable release. Raise the bound once 1.6.0 final ships.
+cuda-tile<1.6

core/application/application.go

@@ -553,12 +553,17 @@ func (a *Application) start() error {
 	// once at startup and reused across chat sessions that opt in via metadata.
 	if !a.applicationConfig.DisableLocalAIAssistant {
 		holder := mcpTools.NewLocalAIAssistantHolder()
+		var nodeRegistry *nodes.NodeRegistry
+		if a.distributed != nil {
+			nodeRegistry = a.distributed.Registry
+		}
 		assistantClient := localaiInproc.New(
 			a.applicationConfig,
 			a.applicationConfig.SystemState,
 			a.backendLoader,
 			a.modelLoader,
 			a.galleryService,
+			nodeRegistry,
 		)
 		// Wire usage tracking so the assistant's get_usage_stats tool
 		// returns real data; nil values keep the tool returning a clear

core/backend/upscale.go

@@ -0,0 +1,37 @@
+package backend
+
+import (
+	"context"
+
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/pkg/grpc/proto"
+	model "github.com/mudler/LocalAI/pkg/model"
+)
+
+// ImageUpscale loads the model specified in modelConfig and calls UpscaleImage
+// on the backend, writing the result to dst.
+func ImageUpscale(ctx context.Context, src, dst string, scale int, loader *model.ModelLoader, modelConfig config.ModelConfig, appConfig *config.ApplicationConfig) (func() error, error) {
+	opts := ModelOptions(modelConfig, appConfig, model.WithContext(ctx))
+	inferenceModel, err := loader.Load(opts...)
+	if err != nil {
+		recordModelLoadFailure(appConfig, modelConfig.Name, modelConfig.Backend, err, nil)
+		return nil, err
+	}
+
+	fn := func() error {
+		_, err := inferenceModel.UpscaleImage(
+			ctx,
+			&proto.UpscaleImageRequest{
+				Src:   src,
+				Dst:   dst,
+				Scale: int32(scale),
+			},
+		)
+		return err
+	}
+
+	return fn, nil
+}
+
+// ImageUpscaleFunc is a test-friendly indirection.
+var ImageUpscaleFunc = ImageUpscale

core/config/backend_capabilities.go

@@ -44,6 +44,7 @@ const (
 	MethodPredictStream      GRPCMethod = "PredictStream"
 	MethodEmbedding          GRPCMethod = "Embedding"
 	MethodGenerateImage      GRPCMethod = "GenerateImage"
+	MethodUpscaleImage       GRPCMethod = "UpscaleImage"
 	MethodGenerateVideo      GRPCMethod = "GenerateVideo"
 	MethodGenerate3D         GRPCMethod = "Generate3D"
 	MethodAudioTranscription GRPCMethod = "AudioTranscription"
@@ -348,7 +349,7 @@ var BackendCapabilities = map[string]BackendCapability{
 
 	// --- Image/video generation backends ---
 	"diffusers": {
-		GRPCMethods:      []GRPCMethod{MethodGenerateImage, MethodGenerateVideo},
+		GRPCMethods:      []GRPCMethod{MethodGenerateImage, MethodUpscaleImage, MethodGenerateVideo},
 		PossibleUsecases: []string{UsecaseImage, UsecaseVideo},
 		DefaultUsecases:  []string{UsecaseImage},
 		Description:      "HuggingFace diffusers — Stable Diffusion, Flux, video generation",

core/config/vllm_spec.go

@@ -0,0 +1,117 @@
+package config
+
+// Speculative-decoding auto-defaults for the vllm-cpp backend, the safetensors
+// counterpart of the GGUF/llama.cpp hook in mtp.go.
+//
+// The two engines detect and spell the same feature differently. llama.cpp
+// reads `<arch>.nextn_predict_layers` out of the GGUF header and takes
+// `spec_type:draft-mtp` in `options:`; vllm.cpp reads `mtp_num_hidden_layers`
+// out of the checkpoint's config.json and takes vLLM's own
+// `--speculative-config` JSON, which LocalAI carries in `engine_args`. The
+// engine resolves the draft depth and the default k itself, so the config only
+// has to name the method.
+
+import (
+	"encoding/json"
+
+	"github.com/mudler/xlog"
+)
+
+// hfSpecConfig is the subset of a HuggingFace config.json that decides whether
+// speculative decoding can be auto-enabled.
+type hfSpecConfig struct {
+	ModelType string `json:"model_type"`
+	// MtpNumHiddenLayers is the MTP head depth (upstream speculative.py reads
+	// it as n_predict for the qwen3_5 / qwen3_5_moe families).
+	MtpNumHiddenLayers uint32 `json:"mtp_num_hidden_layers"`
+	// DFlashConfig marks a z-lab DFlash DRAFT checkpoint (mask_token_id +
+	// target_layer_ids). Its presence means this repo is a draft, not a
+	// servable target.
+	DFlashConfig json.RawMessage `json:"dflash_config"`
+	// TextConfig is where multimodal checkpoints nest the language-model
+	// config, and therefore the MTP depth.
+	TextConfig *hfSpecConfig `json:"text_config"`
+}
+
+// parseHFSpecConfig decodes the speculative-relevant subset of a config.json.
+// A document that does not parse yields nothing rather than an error: detection
+// is best-effort and must never break an import.
+func parseHFSpecConfig(configJSON []byte) (hfSpecConfig, bool) {
+	if len(configJSON) == 0 {
+		return hfSpecConfig{}, false
+	}
+	var c hfSpecConfig
+	if err := json.Unmarshal(configJSON, &c); err != nil {
+		xlog.Debug("[vllm-spec] config.json did not parse; skipping detection", "error", err)
+		return hfSpecConfig{}, false
+	}
+	return c, true
+}
+
+// IsDFlashDraftConfig reports whether a HuggingFace config.json describes a
+// DFlash DRAFT checkpoint. Unlike MTP - whose head ships inside the target
+// checkpoint's `mtp.*` tensors - a DFlash draft is its own repo that can only
+// run paired with a target it verifies against, so it must never be configured
+// as a standalone model.
+func IsDFlashDraftConfig(configJSON []byte) bool {
+	c, ok := parseHFSpecConfig(configJSON)
+	if !ok {
+		return false
+	}
+	return len(c.DFlashConfig) > 0 ||
+		(c.TextConfig != nil && len(c.TextConfig.DFlashConfig) > 0)
+}
+
+// HasSafetensorsMTPHead reports whether a HuggingFace config.json declares a
+// self-speculating Multi-Token Prediction head, returning its depth. The depth
+// is informational: vllm.cpp resolves n_predict and the default
+// num_speculative_tokens from the checkpoint itself.
+//
+// DFlash drafts are excluded for the same reason `gemma4-assistant` GGUFs are
+// excluded from the llama.cpp hook: they carry head metadata but cannot
+// self-speculate.
+//
+// NOTE this is a safetensors-only signal. vllm.cpp rejects an MTP config over a
+// GGUF source, because the `mtp.*` draft tensors only exist in the safetensors
+// checkpoint - so the GGUF import path must not use this.
+func HasSafetensorsMTPHead(configJSON []byte) (uint32, bool) {
+	c, ok := parseHFSpecConfig(configJSON)
+	if !ok {
+		return 0, false
+	}
+	if IsDFlashDraftConfig(configJSON) {
+		return 0, false
+	}
+	n := c.MtpNumHiddenLayers
+	if n == 0 && c.TextConfig != nil {
+		n = c.TextConfig.MtpNumHiddenLayers
+	}
+	return n, n > 0
+}
+
+// ApplyVLLMSpeculativeDefaults enables MTP speculative decoding in cfg's
+// engine_args when nothing is configured there yet. It is a no-op when the user
+// already set a speculative_config, so an explicit choice (a different method,
+// an explicit k, a DFlash draft) is never clobbered.
+//
+// `layers` is the detected head depth and is only used for the diagnostic log
+// line - the engine derives the real k from the checkpoint.
+func ApplyVLLMSpeculativeDefaults(cfg *ModelConfig, layers uint32) {
+	if cfg == nil {
+		return
+	}
+	if _, set := cfg.EngineArgs["speculative_config"]; set {
+		xlog.Debug("[vllm-spec] MTP head detected but speculative_config already configured; leaving user choice intact",
+			"name", cfg.Name, "mtp_num_hidden_layers", layers)
+		return
+	}
+	if cfg.EngineArgs == nil {
+		cfg.EngineArgs = map[string]any{}
+	}
+	// Only the method: vllm.cpp defaults num_speculative_tokens to the
+	// checkpoint's own n_predict (speculative.py:865-875), which is the right
+	// value far more reliably than anything guessable here.
+	cfg.EngineArgs["speculative_config"] = map[string]any{"method": "mtp"}
+	xlog.Info("[vllm-spec] MTP head detected; enabling mtp speculative decoding",
+		"name", cfg.Name, "mtp_num_hidden_layers", layers)
+}

core/config/vllm_spec_test.go

@@ -0,0 +1,117 @@
+package config_test
+
+import (
+	. "github.com/mudler/LocalAI/core/config"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("vllm-cpp speculative-decoding auto-defaults", func() {
+	Context("HasSafetensorsMTPHead", func() {
+		It("detects a top-level mtp_num_hidden_layers", func() {
+			n, ok := HasSafetensorsMTPHead([]byte(`{
+				"model_type": "qwen3_5_moe",
+				"mtp_num_hidden_layers": 1
+			}`))
+			Expect(ok).To(BeTrue())
+			Expect(n).To(Equal(uint32(1)))
+		})
+
+		It("detects the head nested under text_config", func() {
+			// Multimodal checkpoints nest the language-model config, which is
+			// where the MTP depth lives (mirrors the engine's own resolution
+			// off config.raw text_config).
+			n, ok := HasSafetensorsMTPHead([]byte(`{
+				"model_type": "qwen3_5_moe",
+				"text_config": {"mtp_num_hidden_layers": 2}
+			}`))
+			Expect(ok).To(BeTrue())
+			Expect(n).To(Equal(uint32(2)))
+		})
+
+		It("reports no head when the key is absent", func() {
+			n, ok := HasSafetensorsMTPHead([]byte(`{"model_type": "llama"}`))
+			Expect(ok).To(BeFalse())
+			Expect(n).To(BeZero())
+		})
+
+		It("reports no head for a zero depth", func() {
+			_, ok := HasSafetensorsMTPHead([]byte(`{"mtp_num_hidden_layers": 0}`))
+			Expect(ok).To(BeFalse())
+		})
+
+		It("ignores a DFlash draft checkpoint", func() {
+			// A DFlash draft is a SEPARATE checkpoint that cannot serve alone:
+			// it needs a target to verify against. Same exclusion the GGUF path
+			// makes for gemma4-assistant drafts.
+			_, ok := HasSafetensorsMTPHead([]byte(`{
+				"model_type": "qwen3_dflash",
+				"mtp_num_hidden_layers": 1,
+				"dflash_config": {"mask_token_id": 151666, "target_layer_ids": [0, 1]}
+			}`))
+			Expect(ok).To(BeFalse())
+		})
+
+		It("reports no head on unparseable JSON", func() {
+			_, ok := HasSafetensorsMTPHead([]byte(`{not json`))
+			Expect(ok).To(BeFalse())
+		})
+
+		It("reports no head on empty input", func() {
+			_, ok := HasSafetensorsMTPHead(nil)
+			Expect(ok).To(BeFalse())
+		})
+	})
+
+	Context("IsDFlashDraftConfig", func() {
+		It("recognises a draft by its dflash_config block", func() {
+			Expect(IsDFlashDraftConfig([]byte(`{
+				"dflash_config": {"mask_token_id": 151666, "target_layer_ids": [0]}
+			}`))).To(BeTrue())
+		})
+
+		It("does not flag an ordinary checkpoint", func() {
+			Expect(IsDFlashDraftConfig([]byte(`{"model_type": "qwen3_5_moe"}`))).To(BeFalse())
+		})
+	})
+
+	Context("ApplyVLLMSpeculativeDefaults", func() {
+		It("writes the mtp method into engine_args", func() {
+			cfg := &ModelConfig{Name: "qwen"}
+			ApplyVLLMSpeculativeDefaults(cfg, 1)
+			Expect(cfg.EngineArgs).To(HaveKey("speculative_config"))
+			spec, ok := cfg.EngineArgs["speculative_config"].(map[string]any)
+			Expect(ok).To(BeTrue())
+			Expect(spec["method"]).To(Equal("mtp"))
+		})
+
+		It("leaves an existing speculative_config alone", func() {
+			cfg := &ModelConfig{
+				Name: "qwen",
+				LLMConfig: LLMConfig{
+					EngineArgs: map[string]any{
+						"speculative_config": map[string]any{"method": "ngram", "num_speculative_tokens": 4},
+					},
+				},
+			}
+			ApplyVLLMSpeculativeDefaults(cfg, 1)
+			spec := cfg.EngineArgs["speculative_config"].(map[string]any)
+			Expect(spec["method"]).To(Equal("ngram"))
+		})
+
+		It("preserves unrelated engine_args keys", func() {
+			cfg := &ModelConfig{
+				Name:      "qwen",
+				LLMConfig: LLMConfig{EngineArgs: map[string]any{"max_num_seqs": 32}},
+			}
+			ApplyVLLMSpeculativeDefaults(cfg, 1)
+			Expect(cfg.EngineArgs).To(HaveKeyWithValue("max_num_seqs", 32))
+			Expect(cfg.EngineArgs).To(HaveKey("speculative_config"))
+		})
+
+		It("tolerates a nil config", func() {
+			Expect(func() { ApplyVLLMSpeculativeDefaults(nil, 1) }).ToNot(Panic())
+		})
+	})
+})

core/gallery/importers/llama-cpp.go

@@ -298,7 +298,15 @@ func (i *LlamaCPPImporter) Import(details Details) (gallery.ModelConfig, error)
 	// imported configs already carry spec_type:draft-mtp before the model is
 	// ever loaded - users see it in the YAML preview rather than discovering
 	// it after the first start.
-	maybeApplyMTPDefaults(&modelConfig, details, &cfg)
+	//
+	// vllm-cpp is excluded on both counts: `spec_type:*` are llama.cpp option
+	// keys it does not read, and vllm.cpp rejects an MTP config over a GGUF
+	// source outright (the `mtp.*` draft tensors exist only in the safetensors
+	// checkpoint). Its MTP auto-config runs in the vllm importer instead, over
+	// the safetensors config.json.
+	if backend != "vllm-cpp" {
+		maybeApplyMTPDefaults(&modelConfig, details, &cfg)
+	}
 
 	data, err := yaml.Marshal(modelConfig)
 	if err != nil {

core/gallery/importers/vllm.go

@@ -1,13 +1,21 @@
 package importers
 
 import (
+	"context"
 	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
 	"path/filepath"
 	"strings"
+	"time"
 
 	"github.com/mudler/LocalAI/core/config"
 	"github.com/mudler/LocalAI/core/gallery"
 	"github.com/mudler/LocalAI/core/schema"
+	"github.com/mudler/LocalAI/pkg/downloader"
+	"github.com/mudler/LocalAI/pkg/httpclient"
+	"github.com/mudler/xlog"
 	"go.yaml.in/yaml/v2"
 )
 
@@ -107,6 +115,12 @@ func (i *VLLMImporter) Import(details Details) (gallery.ModelConfig, error) {
 		// vllm python backend, so use_tokenizer_template carries over), but
 		// tool/reasoning parsing is the engine's own autoparser pipeline -
 		// the vllm-python tool_parser/reasoning_parser options don't apply.
+		//
+		// Auto-detect a Multi-Token Prediction head, the safetensors analogue
+		// of the llama-cpp importer's GGUF hook, so a freshly imported
+		// Qwen3.5 / Qwen3.6 config already carries speculative decoding in its
+		// engine_args instead of leaving the throughput on the table.
+		maybeApplyVLLMSpeculativeDefaults(&modelConfig, details)
 	} else {
 		// Auto-detect tool_parser and reasoning_parser for known model families.
 		// Surfacing them in the generated YAML lets users see and edit the choices.
@@ -132,3 +146,89 @@ func (i *VLLMImporter) Import(details Details) (gallery.ModelConfig, error) {
 		ConfigFile:  string(data),
 	}, nil
 }
+
+// maxSpecConfigProbeBytes caps the config.json body we read. Real ones are a
+// few KB; the cap keeps a hostile or mislabelled URL from streaming into the
+// importer.
+const maxSpecConfigProbeBytes = 1 << 20 // 1 MiB
+
+// specConfigProbeTimeout bounds the config.json fetch. Detection is an
+// optimisation, so it must never hold an import open for long.
+const specConfigProbeTimeout = 30 * time.Second
+
+// specConfigFetcher is the seam the config.json probe goes through, so tests can
+// drive the whole import path without a network round trip.
+var specConfigFetcher = fetchProbeBody
+
+// maybeApplyVLLMSpeculativeDefaults fetches the repository's config.json and,
+// when it declares a Multi-Token Prediction head, enables MTP speculative
+// decoding in the emitted engine_args. This is the safetensors counterpart of
+// the llama-cpp importer's GGUF header probe.
+//
+// Every failure is non-fatal and logged at debug: a network blip, a private
+// repo, or a config.json this doesn't understand must leave the import working
+// exactly as it did before, just without the speculative default.
+func maybeApplyVLLMSpeculativeDefaults(modelConfig *config.ModelConfig, details Details) {
+	probeURL := vllmSpecProbeURL(details)
+	if probeURL == "" {
+		return
+	}
+
+	body, err := specConfigFetcher(probeURL)
+	if err != nil {
+		xlog.Debug("[vllm-spec-importer] could not read config.json for MTP detection", "uri", probeURL, "error", err)
+		return
+	}
+
+	applySpecFromConfigJSON(modelConfig, body, details.URI)
+}
+
+// applySpecFromConfigJSON is the decision half of the probe, split out so it can
+// be exercised without a network round trip.
+func applySpecFromConfigJSON(modelConfig *config.ModelConfig, body []byte, uri string) {
+	if config.IsDFlashDraftConfig(body) {
+		// A DFlash draft cannot serve on its own - it only proposes tokens for
+		// a target model to verify. Say so rather than emitting a config that
+		// would fail at load.
+		xlog.Warn("[vllm-spec-importer] this repository is a DFlash DRAFT checkpoint, not a servable model; "+
+			"import the TARGET model and point engine_args.speculative_config at this repo "+
+			`({"method":"dflash","model":"<this repo>"})`, "uri", uri)
+		return
+	}
+
+	n, ok := config.HasSafetensorsMTPHead(body)
+	if !ok {
+		return
+	}
+	config.ApplyVLLMSpeculativeDefaults(modelConfig, n)
+}
+
+// vllmSpecProbeURL returns the HTTP(S) URL of the repository's config.json, or
+// "" when the import isn't backed by a HuggingFace repo we can fetch from (a
+// local directory import, an OCI artifact, ...).
+func vllmSpecProbeURL(details Details) string {
+	if details.HuggingFace == nil || details.HuggingFace.ModelID == "" {
+		return ""
+	}
+	return resolveHTTPProbe(downloader.HuggingFacePrefix + details.HuggingFace.ModelID + "/config.json")
+}
+
+// fetchProbeBody GETs a small remote JSON document under a short timeout.
+func fetchProbeBody(url string) ([]byte, error) {
+	ctx, cancel := context.WithTimeout(context.Background(), specConfigProbeTimeout)
+	defer cancel()
+
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
+	if err != nil {
+		return nil, err
+	}
+	resp, err := httpclient.NewWithTimeout(specConfigProbeTimeout).Do(req)
+	if err != nil {
+		return nil, err
+	}
+	defer func() { _ = resp.Body.Close() }()
+	if resp.StatusCode != http.StatusOK {
+		return nil, fmt.Errorf("unexpected status %d", resp.StatusCode)
+	}
+	return io.ReadAll(io.LimitReader(resp.Body, maxSpecConfigProbeBytes))
+}

core/gallery/importers/vllm_spec_internal_test.go

@@ -0,0 +1,118 @@
+package importers
+
+import (
+	"encoding/json"
+	"errors"
+
+	"github.com/mudler/LocalAI/core/config"
+	hfapi "github.com/mudler/LocalAI/pkg/huggingface-api"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("vllm-cpp speculative auto-config (importer)", func() {
+	Context("applySpecFromConfigJSON", func() {
+		It("enables mtp when the checkpoint declares an MTP head", func() {
+			cfg := &config.ModelConfig{Name: "qwen3.5"}
+			applySpecFromConfigJSON(cfg, []byte(`{
+				"model_type": "qwen3_5_moe",
+				"mtp_num_hidden_layers": 1
+			}`), "huggingface://Qwen/Qwen3.5-A3B")
+			Expect(cfg.EngineArgs).To(HaveKeyWithValue("speculative_config",
+				map[string]any{"method": "mtp"}))
+		})
+
+		It("leaves a plain checkpoint untouched", func() {
+			cfg := &config.ModelConfig{Name: "llama"}
+			applySpecFromConfigJSON(cfg, []byte(`{"model_type": "llama"}`), "huggingface://meta/llama")
+			Expect(cfg.EngineArgs).To(BeEmpty())
+		})
+
+		It("refuses to configure a DFlash draft as a servable model", func() {
+			// The draft only proposes tokens; configuring it standalone would
+			// produce a model that cannot load.
+			cfg := &config.ModelConfig{Name: "dflash-draft"}
+			applySpecFromConfigJSON(cfg, []byte(`{
+				"model_type": "qwen3_dflash",
+				"dflash_config": {"mask_token_id": 151666, "target_layer_ids": [0, 1]}
+			}`), "huggingface://z-lab/Qwen3.6-27B-DFlash")
+			Expect(cfg.EngineArgs).To(BeEmpty())
+		})
+
+		It("survives a config.json it cannot parse", func() {
+			cfg := &config.ModelConfig{Name: "weird"}
+			Expect(func() {
+				applySpecFromConfigJSON(cfg, []byte(`<html>404</html>`), "huggingface://a/b")
+			}).ToNot(Panic())
+			Expect(cfg.EngineArgs).To(BeEmpty())
+		})
+	})
+
+	Context("Import over a repository with an MTP head", func() {
+		var restore func()
+
+		BeforeEach(func() {
+			original := specConfigFetcher
+			restore = func() { specConfigFetcher = original }
+		})
+		AfterEach(func() { restore() })
+
+		importWith := func(backend, configJSON string) string {
+			specConfigFetcher = func(string) ([]byte, error) {
+				return []byte(configJSON), nil
+			}
+			importer := &VLLMImporter{}
+			out, err := importer.Import(Details{
+				URI:         "huggingface://Qwen/Qwen3.5-A3B",
+				Preferences: json.RawMessage(`{"backend": "` + backend + `"}`),
+				HuggingFace: &hfapi.ModelDetails{ModelID: "Qwen/Qwen3.5-A3B"},
+			})
+			Expect(err).ToNot(HaveOccurred())
+			return out.ConfigFile
+		}
+
+		It("emits engine_args.speculative_config for vllm-cpp", func() {
+			yaml := importWith("vllm-cpp", `{"model_type":"qwen3_5_moe","mtp_num_hidden_layers":1}`)
+			Expect(yaml).To(ContainSubstring("engine_args:"))
+			Expect(yaml).To(ContainSubstring("speculative_config:"))
+			Expect(yaml).To(ContainSubstring("method: mtp"))
+		})
+
+		It("emits nothing speculative for the python vllm backend", func() {
+			// The python backend has its own speculative surface and its own
+			// version-dependent MTP support; this hook is vllm-cpp only.
+			yaml := importWith("vllm", `{"model_type":"qwen3_5_moe","mtp_num_hidden_layers":1}`)
+			Expect(yaml).NotTo(ContainSubstring("speculative_config"))
+		})
+
+		It("emits nothing speculative when the probe fails", func() {
+			specConfigFetcher = func(string) ([]byte, error) {
+				return nil, errors.New("network down")
+			}
+			importer := &VLLMImporter{}
+			out, err := importer.Import(Details{
+				URI:         "huggingface://Qwen/Qwen3.5-A3B",
+				Preferences: json.RawMessage(`{"backend": "vllm-cpp"}`),
+				HuggingFace: &hfapi.ModelDetails{ModelID: "Qwen/Qwen3.5-A3B"},
+			})
+			Expect(err).ToNot(HaveOccurred())
+			Expect(out.ConfigFile).NotTo(ContainSubstring("speculative_config"))
+		})
+	})
+
+	Context("vllmSpecProbeURL", func() {
+		It("resolves the repository's config.json to an HTTPS URL", func() {
+			url := vllmSpecProbeURL(Details{
+				URI:         "huggingface://Qwen/Qwen3.5-A3B",
+				HuggingFace: &hfapi.ModelDetails{ModelID: "Qwen/Qwen3.5-A3B"},
+			})
+			Expect(url).To(ContainSubstring("Qwen/Qwen3.5-A3B"))
+			Expect(url).To(HaveSuffix("config.json"))
+			Expect(url).To(HavePrefix("https://"))
+		})
+
+		It("skips the probe when there is no HuggingFace repo behind the import", func() {
+			Expect(vllmSpecProbeURL(Details{URI: "/models/local-dir"})).To(BeEmpty())
+		})
+	})
+})

core/http/auth/features.go

@@ -39,6 +39,8 @@ var RouteFeatureRegistry = []RouteFeature{
 	{"POST", "/images/generations", FeatureImages},
 	{"POST", "/v1/images/inpainting", FeatureImages},
 	{"POST", "/images/inpainting", FeatureImages},
+	{"POST", "/v1/images/upscale", FeatureImages},
+	{"POST", "/images/upscale", FeatureImages},
 
 	// Audio transcription
 	{"POST", "/v1/audio/transcriptions", FeatureAudioTranscription},
@@ -116,6 +118,10 @@ var RouteFeatureRegistry = []RouteFeature{
 	// Rerank
 	{"POST", "/v1/rerank", FeatureRerank},
 
+	// Moderation
+	{"POST", "/v1/moderations", FeatureModeration},
+	{"POST", "/moderations", FeatureModeration},
+
 	// Stores
 	{"POST", "/stores/set", FeatureStores},
 	{"POST", "/stores/delete", FeatureStores},
@@ -191,6 +197,7 @@ func APIFeatureMetas() []FeatureMeta {
 		{FeatureEmbeddings, "Embeddings", true},
 		{FeatureSound, "Sound Generation", true},
 		{FeatureRealtime, "Realtime", true},
+		{FeatureModeration, "Moderation", true},
 		{FeatureRerank, "Rerank", true},
 		{FeatureTokenize, "Tokenize", true},
 		{FeatureMCP, "MCP", true},

core/http/auth/features_moderation_test.go

@@ -0,0 +1,24 @@
+package auth_test
+
+import (
+	. "github.com/mudler/LocalAI/core/http/auth"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Moderation feature registration", func() {
+	It("registers both moderation routes as default-on API features", func() {
+		Expect(APIFeatures).To(ContainElement(FeatureModeration))
+
+		patterns := []string{}
+		for _, route := range RouteFeatureRegistry {
+			if route.Feature == FeatureModeration {
+				patterns = append(patterns, route.Pattern)
+			}
+		}
+		Expect(patterns).To(ConsistOf("/v1/moderations", "/moderations"))
+
+		metas := APIFeatureMetas()
+		Expect(metas).To(ContainElement(FeatureMeta{Key: FeatureModeration, Label: "Moderation", DefaultValue: true}))
+	})
+})

core/http/auth/helpers_test.go

@@ -59,10 +59,14 @@ func ok(c echo.Context) error {
 func newAuthTestApp(db *gorm.DB, appConfig *config.ApplicationConfig) *echo.Echo {
 	e := echo.New()
 	e.Use(auth.Middleware(db, appConfig))
+	if db != nil {
+		e.Use(auth.RequireRouteFeature(db))
+	}
 
 	// API routes (require auth)
 	e.GET("/v1/models", ok)
 	e.POST("/v1/chat/completions", ok)
+	e.POST("/v1/moderations", ok)
 	e.GET("/api/settings", ok)
 	e.POST("/api/settings", ok)
 
@@ -81,10 +85,14 @@ func newAuthTestApp(db *gorm.DB, appConfig *config.ApplicationConfig) *echo.Echo
 func newAdminTestApp(db *gorm.DB, appConfig *config.ApplicationConfig) *echo.Echo {
 	e := echo.New()
 	e.Use(auth.Middleware(db, appConfig))
+	if db != nil {
+		e.Use(auth.RequireRouteFeature(db))
+	}
 
 	// Regular routes
 	e.GET("/v1/models", ok)
 	e.POST("/v1/chat/completions", ok)
+	e.POST("/v1/moderations", ok)
 
 	// Admin-only routes
 	adminMw := auth.RequireAdmin()

core/http/auth/middleware_test.go

@@ -91,6 +91,19 @@ var _ = Describe("Auth Middleware", func() {
 			Expect(rec.Code).To(Equal(http.StatusOK))
 		})
 
+		It("allows authenticated users to call moderation by default", func() {
+			sessionID := createTestSession(db, user.ID)
+			rec := doRequest(app, http.MethodPost, "/v1/moderations", withSessionCookie(sessionID))
+			Expect(rec.Code).To(Equal(http.StatusOK))
+		})
+
+		It("blocks moderation when the user's feature is disabled", func() {
+			Expect(auth.UpdateUserPermissions(db, user.ID, auth.PermissionMap{auth.FeatureModeration: false})).To(Succeed())
+			sessionID := createTestSession(db, user.ID)
+			rec := doRequest(app, http.MethodPost, "/v1/moderations", withSessionCookie(sessionID))
+			Expect(rec.Code).To(Equal(http.StatusForbidden))
+		})
+
 		It("allows requests with valid session as Bearer token", func() {
 			sessionID := createTestSession(db, user.ID)
 			rec := doRequest(app, http.MethodGet, "/v1/models", withBearerToken(sessionID))
@@ -156,6 +169,11 @@ var _ = Describe("Auth Middleware", func() {
 			Expect(rec.Code).To(Equal(http.StatusUnauthorized))
 		})
 
+		It("returns 401 for unauthenticated moderation requests", func() {
+			rec := doRequest(app, http.MethodPost, "/v1/moderations")
+			Expect(rec.Code).To(Equal(http.StatusUnauthorized))
+		})
+
 		It("returns 401 for unauthenticated 3D generation requests", func() {
 			rec := doRequest(app, http.MethodPost, "/3d/generations")
 			Expect(rec.Code).To(Equal(http.StatusUnauthorized))

core/http/auth/permissions.go

@@ -51,6 +51,7 @@ const (
 	FeatureEmbeddings          = "embeddings"
 	FeatureSound               = "sound"
 	FeatureRealtime            = "realtime"
+	FeatureModeration          = "moderation"
 	FeatureRerank              = "rerank"
 	FeatureTokenize            = "tokenize"
 	FeatureMCP                 = "mcp"
@@ -75,7 +76,7 @@ var APIFeatures = []string{
 	FeatureChat, FeatureImages, FeatureAudioSpeech, FeatureAudioTranscription,
 	FeatureAudioDiarization, FeatureAudioClassification,
 	FeatureVAD, FeatureDetection, FeatureVideo, Feature3D, FeatureEmbeddings, FeatureSound,
-	FeatureRealtime, FeatureRerank, FeatureTokenize, FeatureMCP, FeatureStores,
+	FeatureRealtime, FeatureModeration, FeatureRerank, FeatureTokenize, FeatureMCP, FeatureStores,
 	FeatureFaceRecognition, FeatureVoiceRecognition, FeatureAudioTransform,
 	FeaturePIIFilter,
 }

core/http/endpoints/localai/api_instructions.go

@@ -30,6 +30,12 @@ var instructionDefs = []instructionDef{
 		Tags:        []string{"inference", "embeddings"},
 		Intro:       "Set \"stream\": true for SSE streaming. Supports tool/function calling when the model config has function templates configured.",
 	},
+	{
+		Name:        "moderation",
+		Description: "OpenAI-compatible text moderation using a local completion model",
+		Tags:        []string{"moderation"},
+		Intro:       "POST /v1/moderations accepts a text string or array plus a LocalAI completion model. LocalAI constrains the model to the OpenAI moderation category schema and returns one result per input. Multimodal moderation inputs are not yet supported.",
+	},
 	{
 		Name:        "audio",
 		Description: "Text-to-speech, voice activity detection, transcription, speaker diarization, sound classification, and sound generation",

core/http/endpoints/localai/api_instructions_test.go

@@ -39,7 +39,7 @@ var _ = Describe("API Instructions Endpoints", func() {
 
 			instructions, ok := resp["instructions"].([]any)
 			Expect(ok).To(BeTrue())
-			Expect(instructions).To(HaveLen(18))
+			Expect(instructions).To(HaveLen(19))
 
 			// Verify each instruction has required fields and correct URL format
 			for _, s := range instructions {
@@ -69,6 +69,7 @@ var _ = Describe("API Instructions Endpoints", func() {
 
 			Expect(names).To(ContainElements(
 				"chat-inference",
+				"moderation",
 				"config-management",
 				"model-management",
 				"monitoring",

core/http/endpoints/mcp/localai_assistant_test.go

@@ -84,6 +84,22 @@ func (stubClient) ListNodes(_ context.Context) ([]localaitools.Node, error) {
 	return []localaitools.Node{}, nil
 }
 
+func (stubClient) ListScheduling(_ context.Context) ([]localaitools.ModelSchedulingConfig, error) {
+	return []localaitools.ModelSchedulingConfig{}, nil
+}
+
+func (stubClient) GetScheduling(_ context.Context, _ string) (*localaitools.ModelSchedulingConfig, error) {
+	return &localaitools.ModelSchedulingConfig{}, nil
+}
+
+func (stubClient) SetScheduling(_ context.Context, _ localaitools.SetSchedulingRequest) (*localaitools.ModelSchedulingConfig, error) {
+	return &localaitools.ModelSchedulingConfig{}, nil
+}
+
+func (stubClient) DeleteScheduling(_ context.Context, _ string) error {
+	return nil
+}
+
 func (stubClient) SetNodeVRAMBudget(_ context.Context, _, _ string) error {
 	return nil
 }

core/http/endpoints/openai/moderations.go

@@ -0,0 +1,190 @@
+package openai
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"math"
+	"net/http"
+	"strings"
+
+	"github.com/google/uuid"
+	"github.com/labstack/echo/v4"
+	"github.com/mudler/LocalAI/core/backend"
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/http/middleware"
+	"github.com/mudler/LocalAI/core/schema"
+	"github.com/mudler/LocalAI/core/templates"
+	"github.com/mudler/LocalAI/pkg/functions"
+	"github.com/mudler/LocalAI/pkg/model"
+)
+
+var moderationCategories = []string{
+	"harassment",
+	"harassment/threatening",
+	"hate",
+	"hate/threatening",
+	"illicit",
+	"illicit/violent",
+	"self-harm",
+	"self-harm/intent",
+	"self-harm/instructions",
+	"sexual",
+	"sexual/minors",
+	"violence",
+	"violence/graphic",
+}
+
+type moderationGenerator func(context.Context, string, *config.ModelConfig) (string, backend.TokenUsage, error)
+
+type generatedModeration struct {
+	Categories     map[string]bool    `json:"categories"`
+	CategoryScores map[string]float64 `json:"category_scores"`
+}
+
+// ModerationEndpoint implements the text input subset of OpenAI's moderation
+// API using any LocalAI completion model and constrained JSON generation.
+// @Summary Classify text for potentially harmful content.
+// @Tags moderation
+// @Param request body schema.ModerationRequest true "query params"
+// @Success 200 {object} schema.ModerationResponse "Response"
+// @Router /v1/moderations [post]
+func ModerationEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoader, evaluator *templates.Evaluator, appConfig *config.ApplicationConfig) echo.HandlerFunc {
+	return moderationEndpoint(func(ctx context.Context, input string, cfg *config.ModelConfig) (string, backend.TokenUsage, error) {
+		prompt := moderationPrompt(input)
+		var messages schema.Messages
+		if cfg.TemplateConfig.UseTokenizerTemplate {
+			messages = schema.Messages{{Role: "user", Content: prompt}}
+			prompt = ""
+		} else if evaluator != nil {
+			if rendered, err := evaluator.EvaluateTemplateForPrompt(templates.CompletionPromptTemplate, *cfg, templates.PromptTemplateData{Input: prompt, SystemPrompt: cfg.SystemPrompt}); err == nil {
+				prompt = rendered
+			}
+		}
+
+		predict, err := backend.ModelInferenceFunc(ctx, prompt, messages, nil, nil, nil, ml, cfg, cl, appConfig, nil, "", "", nil, nil, nil, nil)
+		if err != nil {
+			return "", backend.TokenUsage{}, err
+		}
+		response, err := predict()
+		return response.Response, response.Usage, err
+	})
+}
+
+func moderationEndpoint(generate moderationGenerator) echo.HandlerFunc {
+	return func(c echo.Context) error {
+		input, ok := c.Get(middleware.CONTEXT_LOCALS_KEY_LOCALAI_REQUEST).(*schema.ModerationRequest)
+		if !ok || input == nil {
+			return echo.NewHTTPError(http.StatusBadRequest, "invalid moderation request")
+		}
+		if len(input.Input) == 0 {
+			return echo.NewHTTPError(http.StatusBadRequest, "input must contain at least one text string")
+		}
+		if generate == nil {
+			return echo.NewHTTPError(http.StatusInternalServerError, "moderation generator is unavailable")
+		}
+
+		modelConfig, ok := c.Get(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG).(*config.ModelConfig)
+		if !ok || modelConfig == nil {
+			return echo.NewHTTPError(http.StatusBadRequest, "moderation model configuration is unavailable")
+		}
+
+		grammar, err := moderationGrammar()
+		if err != nil {
+			return echo.NewHTTPError(http.StatusInternalServerError, "failed to build moderation grammar").SetInternal(err)
+		}
+		cfg := *modelConfig
+		cfg.Grammar = grammar
+		maxTokens := 512
+		cfg.Maxtokens = &maxTokens
+
+		results := make([]schema.ModerationResult, 0, len(input.Input))
+		promptTokens, completionTokens := 0, 0
+		for _, text := range input.Input {
+			if strings.TrimSpace(text) == "" {
+				return echo.NewHTTPError(http.StatusBadRequest, "input strings must not be empty")
+			}
+			raw, usage, err := generate(c.Request().Context(), text, &cfg)
+			if err != nil {
+				return echo.NewHTTPError(http.StatusInternalServerError, "moderation inference failed").SetInternal(err)
+			}
+			result, err := parseModerationResult(raw)
+			if err != nil {
+				return echo.NewHTTPError(http.StatusInternalServerError, "invalid moderation result: "+err.Error()).SetInternal(err)
+			}
+			results = append(results, result)
+			promptTokens += usage.Prompt
+			completionTokens += usage.Completion
+		}
+
+		middleware.StampUsage(c, input.Model, promptTokens, completionTokens)
+		return c.JSON(http.StatusOK, schema.ModerationResponse{
+			ID:      "modr-" + uuid.NewString(),
+			Model:   input.Model,
+			Results: results,
+		})
+	}
+}
+
+func moderationPrompt(input string) string {
+	encoded, _ := json.Marshal(input)
+	return `Classify the supplied text for safety moderation. Return only the JSON object required by the grammar.
+For each category, set the boolean to true only when the text matches that category. Give each category a confidence score from 0 to 1.
+The categories are harassment, harassment/threatening, hate, hate/threatening, illicit, illicit/violent, self-harm, self-harm/intent, self-harm/instructions, sexual, sexual/minors, violence, and violence/graphic.
+Text to classify: ` + string(encoded)
+}
+
+func moderationGrammar() (string, error) {
+	boolProperties := map[string]any{}
+	scoreProperties := map[string]any{}
+	for _, category := range moderationCategories {
+		boolProperties[category] = map[string]any{"type": "boolean"}
+		scoreProperties[category] = map[string]any{"type": "number"}
+	}
+	structure := functions.JSONFunctionStructure{AnyOf: []functions.Item{{
+		Type: "object",
+		Properties: map[string]any{
+			"categories": map[string]any{
+				"type":                 "object",
+				"properties":           boolProperties,
+				"required":             moderationCategories,
+				"additionalProperties": false,
+			},
+			"category_scores": map[string]any{
+				"type":                 "object",
+				"properties":           scoreProperties,
+				"required":             moderationCategories,
+				"additionalProperties": false,
+			},
+		},
+	}}}
+	return structure.Grammar()
+}
+
+func parseModerationResult(raw string) (schema.ModerationResult, error) {
+	var generated generatedModeration
+	if err := json.Unmarshal([]byte(strings.TrimSpace(raw)), &generated); err != nil {
+		return schema.ModerationResult{}, err
+	}
+
+	result := schema.ModerationResult{
+		Categories:                make(map[string]bool, len(moderationCategories)),
+		CategoryScores:            make(map[string]float64, len(moderationCategories)),
+		CategoryAppliedInputTypes: make(map[string][]string, len(moderationCategories)),
+	}
+	for _, category := range moderationCategories {
+		flagged, exists := generated.Categories[category]
+		if !exists {
+			return schema.ModerationResult{}, fmt.Errorf("missing category %q", category)
+		}
+		score, exists := generated.CategoryScores[category]
+		if !exists || math.IsNaN(score) || math.IsInf(score, 0) || score < 0 || score > 1 {
+			return schema.ModerationResult{}, fmt.Errorf("category %q has an invalid score", category)
+		}
+		result.Categories[category] = flagged
+		result.CategoryScores[category] = score
+		result.CategoryAppliedInputTypes[category] = []string{"text"}
+		result.Flagged = result.Flagged || flagged
+	}
+	return result, nil
+}

core/http/endpoints/openai/moderations_test.go

@@ -0,0 +1,105 @@
+package openai
+
+import (
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+
+	"github.com/labstack/echo/v4"
+	"github.com/mudler/LocalAI/core/backend"
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/http/middleware"
+	"github.com/mudler/LocalAI/core/schema"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Moderations endpoint", func() {
+	It("classifies each text input and returns the OpenAI response shape", func() {
+		inputs := []string{}
+		generate := func(_ context.Context, input string, cfg *config.ModelConfig) (string, backend.TokenUsage, error) {
+			inputs = append(inputs, input)
+			Expect(cfg.Grammar).To(ContainSubstring("harassment"))
+			return `{
+				"categories":{"harassment":true,"harassment/threatening":false,"hate":false,"hate/threatening":false,"illicit":false,"illicit/violent":false,"self-harm":false,"self-harm/intent":false,"self-harm/instructions":false,"sexual":false,"sexual/minors":false,"violence":false,"violence/graphic":false},
+				"category_scores":{"harassment":0.9,"harassment/threatening":0.1,"hate":0,"hate/threatening":0,"illicit":0,"illicit/violent":0,"self-harm":0,"self-harm/intent":0,"self-harm/instructions":0,"sexual":0,"sexual/minors":0,"violence":0,"violence/graphic":0}
+			}`, backend.TokenUsage{Prompt: 12, Completion: 8}, nil
+		}
+
+		e := echo.New()
+		req := httptest.NewRequest(http.MethodPost, "/v1/moderations", strings.NewReader(`{"model":"guard","input":["first","second"]}`))
+		req.Header.Set(echo.HeaderContentType, echo.MIMEApplicationJSON)
+		rec := httptest.NewRecorder()
+		ctx := e.NewContext(req, rec)
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_LOCALAI_REQUEST, &schema.ModerationRequest{
+			BasicModelRequest: schema.BasicModelRequest{Model: "guard"},
+			Input:             schema.ModerationInput{"first", "second"},
+		})
+		modelConfig := &config.ModelConfig{Name: "guard"}
+		modelConfig.Model = "guard.gguf"
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG, modelConfig)
+
+		Expect(moderationEndpoint(generate)(ctx)).To(Succeed())
+		Expect(rec.Code).To(Equal(http.StatusOK))
+		Expect(inputs).To(Equal([]string{"first", "second"}))
+
+		var response schema.ModerationResponse
+		Expect(json.Unmarshal(rec.Body.Bytes(), &response)).To(Succeed())
+		Expect(response.ID).To(HavePrefix("modr-"))
+		Expect(response.Model).To(Equal("guard"))
+		Expect(response.Results).To(HaveLen(2))
+		Expect(response.Results[0].Flagged).To(BeTrue())
+		Expect(response.Results[0].Categories["harassment"]).To(BeTrue())
+		Expect(response.Results[0].CategoryAppliedInputTypes["harassment"]).To(Equal([]string{"text"}))
+	})
+
+	It("rejects an empty input list", func() {
+		e := echo.New()
+		ctx := e.NewContext(httptest.NewRequest(http.MethodPost, "/v1/moderations", nil), httptest.NewRecorder())
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_LOCALAI_REQUEST, &schema.ModerationRequest{
+			BasicModelRequest: schema.BasicModelRequest{Model: "guard"},
+		})
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG, &config.ModelConfig{Name: "guard"})
+
+		err := moderationEndpoint(nil)(ctx)
+		Expect(err).To(MatchError(ContainSubstring("input must contain at least one text string")))
+		Expect(err.(*echo.HTTPError).Code).To(Equal(http.StatusBadRequest))
+	})
+
+	It("surfaces malformed classifier output without returning a partial result", func() {
+		generate := func(context.Context, string, *config.ModelConfig) (string, backend.TokenUsage, error) {
+			return "not-json", backend.TokenUsage{}, nil
+		}
+		e := echo.New()
+		ctx := e.NewContext(httptest.NewRequest(http.MethodPost, "/v1/moderations", nil), httptest.NewRecorder())
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_LOCALAI_REQUEST, &schema.ModerationRequest{
+			BasicModelRequest: schema.BasicModelRequest{Model: "guard"},
+			Input:             schema.ModerationInput{"text"},
+		})
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG, &config.ModelConfig{Name: "guard"})
+
+		err := moderationEndpoint(generate)(ctx)
+		Expect(err).To(MatchError(ContainSubstring("invalid moderation result")))
+		Expect(err.(*echo.HTTPError).Code).To(Equal(http.StatusInternalServerError))
+	})
+})
+
+var _ = Describe("Moderation input", func() {
+	DescribeTable("accepts OpenAI text input forms",
+		func(body string, expected schema.ModerationInput) {
+			var req schema.ModerationRequest
+			Expect(json.Unmarshal([]byte(body), &req)).To(Succeed())
+			Expect(req.Input).To(Equal(expected))
+		},
+		Entry("single text", `{"input":"hello"}`, schema.ModerationInput{"hello"}),
+		Entry("text array", `{"input":["hello","world"]}`, schema.ModerationInput{"hello", "world"}),
+	)
+
+	It("rejects multimodal input in the text-only MVP", func() {
+		var req schema.ModerationRequest
+		err := json.Unmarshal([]byte(`{"input":[{"type":"image_url","image_url":{"url":"https://example.com/a.png"}}]}`), &req)
+		Expect(err).To(MatchError(ContainSubstring("text string or array of text strings")))
+	})
+})

core/http/endpoints/openai/upscale.go

@@ -0,0 +1,134 @@
+package openai
+
+import (
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"os"
+	"path/filepath"
+	"strconv"
+	"time"
+
+	"github.com/google/uuid"
+	"github.com/labstack/echo/v4"
+	"github.com/mudler/xlog"
+
+	"github.com/mudler/LocalAI/core/backend"
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/http/middleware"
+	"github.com/mudler/LocalAI/core/schema"
+	model "github.com/mudler/LocalAI/pkg/model"
+)
+
+// UpscaleEndpoint handles POST /v1/images/upscale
+//
+// @Summary      Image upscaling
+// @Description  Upscale an image using a specified model (e.g. stable-diffusion-x4-upscaler). Accepts multipart/form-data.
+// @Tags         images
+// @Accept       multipart/form-data
+// @Produce      application/json
+// @Param        model   formData  string  true   "Upscaler model identifier (e.g. stable-diffusion-x4-upscaler)"
+// @Param        image   formData  file    true   "Input image file"
+// @Param        scale   formData  int     false  "Upscale factor: 2 or 4 (default 2)"
+// @Success      200 {object} schema.OpenAIResponse
+// @Failure      400 {object} map[string]string
+// @Failure      500 {object} map[string]string
+// @Router       /v1/images/upscale [post]
+func UpscaleEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoader, appConfig *config.ApplicationConfig) echo.HandlerFunc {
+	return func(c echo.Context) error {
+		modelName := c.FormValue("model")
+		scaleStr := c.FormValue("scale")
+
+		if modelName == "" {
+			xlog.Error("Upscale Endpoint - missing model")
+			return echo.NewHTTPError(http.StatusBadRequest, "missing model")
+		}
+
+		scale := 2
+		if scaleStr != "" {
+			v, err := strconv.Atoi(scaleStr)
+			if err != nil || (v != 2 && v != 4) {
+				return echo.NewHTTPError(http.StatusBadRequest, "scale must be 2 or 4")
+			}
+			scale = v
+		}
+
+		// Read uploaded image
+		imageFile, err := c.FormFile("image")
+		if err != nil {
+			xlog.Error("Upscale Endpoint - missing image file", "error", err)
+			return echo.NewHTTPError(http.StatusBadRequest, "missing image file")
+		}
+
+		imgSrc, err := imageFile.Open()
+		if err != nil {
+			return err
+		}
+		defer imgSrc.Close()
+		imgBytes, err := io.ReadAll(imgSrc)
+		if err != nil {
+			return err
+		}
+
+		// Get model config from middleware context
+		cfg, ok := c.Get(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG).(*config.ModelConfig)
+		if !ok || cfg == nil {
+			xlog.Error("Upscale Endpoint - model config not found in context")
+			return echo.ErrBadRequest
+		}
+
+		tmpDir := filepath.Join(appConfig.GeneratedContentDir, "images")
+		if err := os.MkdirAll(tmpDir, 0750); err != nil {
+			return echo.NewHTTPError(http.StatusInternalServerError, "failed to prepare storage")
+		}
+
+		// Write input image to a temp file
+		srcTmp, err := os.CreateTemp(tmpDir, "upscale_src_")
+		if err != nil {
+			return err
+		}
+		if _, err := srcTmp.Write(imgBytes); err != nil {
+			_ = srcTmp.Close()
+			_ = os.Remove(srcTmp.Name())
+			return err
+		}
+		if err := srcTmp.Close(); err != nil {
+			xlog.Warn("Upscale Endpoint - failed to close src temp file", "error", err)
+		}
+		srcPath := srcTmp.Name()
+		defer os.Remove(srcPath)
+
+		// Prepare output file path
+		id := uuid.New().String()
+		dstPath := filepath.Join(tmpDir, fmt.Sprintf("upscale_%s.png", id))
+
+		fn, err := backend.ImageUpscaleFunc(c.Request().Context(), srcPath, dstPath, scale, ml, *cfg, appConfig)
+		if err != nil {
+			return err
+		}
+		if err := fn(); err != nil {
+			_ = os.Remove(dstPath)
+			return err
+		}
+
+		baseURL := middleware.BaseURL(c)
+		imgURL, err := url.JoinPath(baseURL, "generated-images", filepath.Base(dstPath))
+		if err != nil {
+			_ = os.Remove(dstPath)
+			return err
+		}
+
+		created := int(time.Now().Unix())
+		resp := &schema.OpenAIResponse{
+			ID:      id,
+			Created: created,
+			Data:    []schema.Item{{URL: imgURL}},
+			Usage: &schema.OpenAIUsage{
+				InputTokensDetails: &schema.InputTokensDetails{},
+			},
+		}
+
+		return c.JSON(http.StatusOK, resp)
+	}
+}

core/http/endpoints/openai/upscale_test.go

@@ -0,0 +1,89 @@
+package openai
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"path/filepath"
+
+	"github.com/labstack/echo/v4"
+	"github.com/mudler/LocalAI/core/backend"
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/http/middleware"
+	"github.com/mudler/LocalAI/core/schema"
+	model "github.com/mudler/LocalAI/pkg/model"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Image upscaling", func() {
+	var (
+		appConfig *config.ApplicationConfig
+		tmpDir    string
+	)
+
+	BeforeEach(func() {
+		var err error
+		tmpDir, err = os.MkdirTemp("", "upscale")
+		Expect(err).ToNot(HaveOccurred())
+		appConfig = config.NewApplicationConfig(config.WithGeneratedContentDir(tmpDir))
+	})
+
+	AfterEach(func() {
+		Expect(os.RemoveAll(tmpDir)).To(Succeed())
+	})
+
+	It("stores the result in the directory served by /generated-images", func() {
+		original := backend.ImageUpscaleFunc
+		backend.ImageUpscaleFunc = func(_ context.Context, _, dst string, scale int, _ *model.ModelLoader, _ config.ModelConfig, _ *config.ApplicationConfig) (func() error, error) {
+			Expect(scale).To(Equal(4))
+			return func() error {
+				return os.WriteFile(dst, []byte("PNGDATA"), 0o644)
+			}, nil
+		}
+		DeferCleanup(func() { backend.ImageUpscaleFunc = original })
+
+		req, _ := makeMultipartRequest(
+			map[string]string{"model": "stable-diffusion-x4-upscaler", "scale": "4"},
+			map[string][]byte{"image": []byte("IMAGEDATA")},
+		)
+		rec := httptest.NewRecorder()
+		ctx := echo.New().NewContext(req, rec)
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG, &config.ModelConfig{Backend: "diffusers"})
+
+		Expect(UpscaleEndpoint(nil, nil, appConfig)(ctx)).To(Succeed())
+		Expect(rec.Code).To(Equal(http.StatusOK))
+
+		var response schema.OpenAIResponse
+		Expect(json.Unmarshal(rec.Body.Bytes(), &response)).To(Succeed())
+		Expect(response.Data).To(HaveLen(1))
+		Expect(response.Data[0].URL).To(ContainSubstring("/generated-images/upscale_"))
+
+		filename := filepath.Base(response.Data[0].URL)
+		contents, err := os.ReadFile(filepath.Join(tmpDir, "images", filename))
+		Expect(err).ToNot(HaveOccurred())
+		Expect(contents).To(Equal([]byte("PNGDATA")))
+	})
+
+	It("rejects unsupported scale factors", func() {
+		req, _ := makeMultipartRequest(
+			map[string]string{"model": "stable-diffusion-x4-upscaler", "scale": "3"},
+			map[string][]byte{"image": []byte("IMAGEDATA")},
+		)
+		rec := httptest.NewRecorder()
+		ctx := echo.New().NewContext(req, rec)
+		ctx.Set(middleware.CONTEXT_LOCALS_KEY_MODEL_CONFIG, &config.ModelConfig{Backend: "diffusers"})
+
+		err := UpscaleEndpoint(nil, nil, appConfig)(ctx)
+		var httpErr *echo.HTTPError
+		Expect(err).To(MatchError(ContainSubstring("scale must be 2 or 4")))
+		Expect(err).To(BeAssignableToTypeOf(httpErr))
+		httpErr = err.(*echo.HTTPError)
+		Expect(httpErr.Code).To(Equal(http.StatusBadRequest))
+		Expect(httpErr.Message).To(Equal("scale must be 2 or 4"))
+		Expect(bytes.TrimSpace(rec.Body.Bytes())).To(BeEmpty())
+	})
+})

core/http/react-ui/e2e/activity-page.spec.js

@@ -43,6 +43,40 @@ test('lists live operations and cancels one from a labelled button', async ({ pa
   expect(cancelledPath).toBe('/api/operations/job-gemma/cancel')
 })
 
+test('pauses a model download without invoking destructive cancel', async ({ page }) => {
+  await stub(page, {
+    operations: [{
+      id: 'gemma-3-27b-it',
+      name: 'gemma-3-27b-it',
+      jobID: 'job-gemma',
+      progress: 22,
+      taskType: 'installation',
+      isBackend: false,
+      isQueued: false,
+      isDeletion: false,
+      cancellable: true,
+      phase: 'downloading',
+    }],
+  })
+
+  const requests = []
+  await page.route('**/api/operations/job-gemma/pause', (route) => {
+    requests.push(new URL(route.request().url()).pathname)
+    return route.fulfill({ contentType: 'application/json', body: '{}' })
+  })
+  await page.route('**/api/operations/job-gemma/cancel', (route) => {
+    requests.push(new URL(route.request().url()).pathname)
+    return route.fulfill({ contentType: 'application/json', body: '{}' })
+  })
+
+  await page.goto('/app/activity')
+
+  const card = page.locator('.operation-card').filter({ hasText: 'gemma-3-27b-it' })
+  await card.locator('.operation-card__pause').click()
+
+  await expect.poll(() => requests).toEqual(['/api/operations/job-gemma/pause'])
+})
+
 test('separates an unacknowledged failure from the record', async ({ page }) => {
   await stub(page, {
     operations: [{

core/http/react-ui/e2e/chrome-audit.spec.js

@@ -14,6 +14,16 @@ const ROUTES = [
 ]
 
 test('no page renders a dead icon or a default-chrome control', async ({ page }) => {
+  // One test walks every route, so its budget has to scale with the list rather
+  // than sit on Playwright's per-test default of 30s. At 25 routes that default
+  // allows ~1.2s per navigation, which holds on a developer machine and does
+  // not on a loaded CI runner: the suite went red on the commit that added this
+  // spec, timing out mid-loop at waitForTimeout rather than at any single goto,
+  // which is what cumulative slowness looks like as opposed to one hung route.
+  // Six seconds a route absorbs a slow runner and still fails promptly if a
+  // route really does hang.
+  test.setTimeout(ROUTES.length * 6_000)
+
   const findings = []
   for (const route of ROUTES) {
     await page.goto(route)

core/http/react-ui/package-lock.json

@@ -21,9 +21,10 @@
         "@fortawesome/fontawesome-free": "^6.7.2",
         "@lezer/highlight": "^1.2.1",
         "@modelcontextprotocol/ext-apps": "^1.2.2",
-        "@modelcontextprotocol/sdk": "^1.25.1",
+        "@modelcontextprotocol/sdk": "^1.30.0",
         "dompurify": "^3.4.12",
         "highlight.js": "^11.11.1",
+        "hono": "4.12.25",
         "i18next": "^26.0.8",
         "i18next-browser-languagedetector": "^8.2.1",
         "i18next-http-backend": "^3.0.6",
@@ -944,11 +945,12 @@
       }
     },
     "node_modules/@modelcontextprotocol/sdk": {
-      "version": "1.27.1",
-      "resolved": "https://registry.npmjs.org/@modelcontextprotocol/sdk/-/sdk-1.27.1.tgz",
-      "integrity": "sha512-sr6GbP+4edBwFndLbM60gf07z0FQ79gaExpnsjMGePXqFcSSb7t6iscpjk9DhFhwd+mTEQrzNafGP8/iGGFYaA==",
+      "version": "1.30.0",
+      "resolved": "https://registry.npmjs.org/@modelcontextprotocol/sdk/-/sdk-1.30.0.tgz",
+      "integrity": "sha512-xKd8OIzlqNzcqcNumGAa6g+PW2kjD5vrpcKOnfldAUPP3j7lnqMPwlTXQm8gF+UwH72z0lqaRbjr9hqGz0eITA==",
+      "license": "MIT",
       "dependencies": {
-        "@hono/node-server": "^1.19.9",
+        "@hono/node-server": "^1.19.9 || ^2.0.5",
         "ajv": "^8.17.1",
         "ajv-formats": "^3.0.1",
         "content-type": "^1.0.5",
@@ -1718,10 +1720,11 @@
       "dev": true
     },
     "node_modules/brace-expansion": {
-      "version": "1.1.12",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.12.tgz",
-      "integrity": "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg==",
+      "version": "1.1.18",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
+      "integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
         "balanced-match": "^1.0.0",
         "concat-map": "0.0.1"
@@ -3432,9 +3435,9 @@
       }
     },
     "node_modules/hono": {
-      "version": "4.12.31",
-      "resolved": "https://registry.npmjs.org/hono/-/hono-4.12.31.tgz",
-      "integrity": "sha512-zJIHFrl6bq3RDd2YusFNCDlM8qUprxKswyi/OPzPyzKDdyBXDqWx8bZlZ7R+saTdSTatUmb3O7K4SspGPaEOQg==",
+      "version": "4.12.25",
+      "resolved": "https://registry.npmjs.org/hono/-/hono-4.12.25.tgz",
+      "integrity": "sha512-2NFaIyNVgJmBs/ecmtGzlmluTFs5cHEWGTdu0t1HBwYzoGXOL5nUQBRMXsXWla5i4KkG//QMzVP88m1+I3fdAQ==",
       "license": "MIT",
       "engines": {
         "node": ">=16.9.0"
@@ -4383,16 +4386,16 @@
       }
     },
     "node_modules/istanbul-lib-processinfo/node_modules/brace-expansion": {
-      "version": "5.0.6",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
-      "integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
+      "version": "5.0.9",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
+      "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
       "dev": true,
       "license": "MIT",
       "dependencies": {
         "balanced-match": "^4.0.2"
       },
       "engines": {
-        "node": "18 || 20 || >=22"
+        "node": "20 || >=22"
       }
     },
     "node_modules/istanbul-lib-processinfo/node_modules/glob": {
@@ -5278,16 +5281,16 @@
       }
     },
     "node_modules/nyc/node_modules/brace-expansion": {
-      "version": "5.0.6",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
-      "integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
+      "version": "5.0.9",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
+      "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
       "dev": true,
       "license": "MIT",
       "dependencies": {
         "balanced-match": "^4.0.2"
       },
       "engines": {
-        "node": "18 || 20 || >=22"
+        "node": "20 || >=22"
       }
     },
     "node_modules/nyc/node_modules/convert-source-map": {
@@ -5974,10 +5977,11 @@
       }
     },
     "node_modules/quick-temp/node_modules/brace-expansion": {
-      "version": "2.1.0",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.0.tgz",
-      "integrity": "sha512-TN1kCZAgdgweJhWWpgKYrQaMNHcDULHkWwQIspdtjV4Y5aurRdZpjAqn6yX3FPqTA9ngHCc4hJxMAMgGfve85w==",
+      "version": "2.1.4",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.4.tgz",
+      "integrity": "sha512-hGfVzPxthbf3+2yjg/RBs60cB0FhqBS/zvdV/4wn4/BmN0bNMMHPc4V/BbFieqf1TKAGGAHnY4eSjajCl0f2Xg==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
         "balanced-match": "^1.0.0"
       }
@@ -6569,16 +6573,16 @@
       }
     },
     "node_modules/spawn-wrap/node_modules/brace-expansion": {
-      "version": "5.0.6",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
-      "integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
+      "version": "5.0.9",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
+      "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
       "dev": true,
       "license": "MIT",
       "dependencies": {
         "balanced-match": "^4.0.2"
       },
       "engines": {
-        "node": "18 || 20 || >=22"
+        "node": "20 || >=22"
       }
     },
     "node_modules/spawn-wrap/node_modules/foreground-child": {
@@ -6902,16 +6906,16 @@
       }
     },
     "node_modules/test-exclude/node_modules/brace-expansion": {
-      "version": "5.0.6",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
-      "integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
+      "version": "5.0.9",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
+      "integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
       "dev": true,
       "license": "MIT",
       "dependencies": {
         "balanced-match": "^4.0.2"
       },
       "engines": {
-        "node": "18 || 20 || >=22"
+        "node": "20 || >=22"
       }
     },
     "node_modules/test-exclude/node_modules/glob": {

core/http/react-ui/package.json

@@ -35,7 +35,7 @@
     "@fortawesome/fontawesome-free": "^6.7.2",
     "@lezer/highlight": "^1.2.1",
     "@modelcontextprotocol/ext-apps": "^1.2.2",
-    "@modelcontextprotocol/sdk": "^1.25.1",
+    "@modelcontextprotocol/sdk": "^1.30.0",
     "dompurify": "^3.4.12",
     "highlight.js": "^11.11.1",
     "hono": "4.12.25",

core/http/react-ui/public/locales/de/admin.json

@@ -12,6 +12,8 @@
     "timeLeft": "{{value}} left",
     "cancel": "Cancel",
     "cancelLabel": "Cancel {{name}}",
+    "pause": "Pause",
+    "pauseLabel": "Pause {{name}} and keep downloaded data",
     "retry": "Retry",
     "retryLabel": "Retry {{name}}",
     "nodeCount": "{{count}} nodes",

core/http/react-ui/public/locales/en/admin.json

@@ -12,6 +12,8 @@
     "timeLeft": "{{value}} left",
     "cancel": "Cancel",
     "cancelLabel": "Cancel {{name}}",
+    "pause": "Pause",
+    "pauseLabel": "Pause {{name}} and keep downloaded data",
     "retry": "Retry",
     "retryLabel": "Retry {{name}}",
     "nodeCount": "{{count}} nodes",

core/http/react-ui/public/locales/es/admin.json

@@ -12,6 +12,8 @@
     "timeLeft": "{{value}} left",
     "cancel": "Cancel",
     "cancelLabel": "Cancel {{name}}",
+    "pause": "Pause",
+    "pauseLabel": "Pause {{name}} and keep downloaded data",
     "retry": "Retry",
     "retryLabel": "Retry {{name}}",
     "nodeCount": "{{count}} nodes",

core/http/react-ui/public/locales/id/admin.json

@@ -12,6 +12,8 @@
     "timeLeft": "{{value}} left",
     "cancel": "Cancel",
     "cancelLabel": "Cancel {{name}}",
+    "pause": "Pause",
+    "pauseLabel": "Pause {{name}} and keep downloaded data",
     "retry": "Retry",
     "retryLabel": "Retry {{name}}",
     "nodeCount": "{{count}} nodes",

core/http/react-ui/public/locales/it/admin.json

@@ -12,6 +12,8 @@
     "timeLeft": "{{value}} left",
     "cancel": "Cancel",
     "cancelLabel": "Cancel {{name}}",
+    "pause": "Pause",
+    "pauseLabel": "Pause {{name}} and keep downloaded data",
     "retry": "Retry",
     "retryLabel": "Retry {{name}}",
     "nodeCount": "{{count}} nodes",

core/http/react-ui/public/locales/ko/admin.json

@@ -12,6 +12,8 @@
     "timeLeft": "{{value}} left",
     "cancel": "Cancel",
     "cancelLabel": "Cancel {{name}}",
+    "pause": "Pause",
+    "pauseLabel": "Pause {{name}} and keep downloaded data",
     "retry": "Retry",
     "retryLabel": "Retry {{name}}",
     "nodeCount": "{{count}} nodes",

core/http/react-ui/public/locales/zh-CN/admin.json

@@ -12,6 +12,8 @@
     "timeLeft": "{{value}} left",
     "cancel": "Cancel",
     "cancelLabel": "Cancel {{name}}",
+    "pause": "Pause",
+    "pauseLabel": "Pause {{name}} and keep downloaded data",
     "retry": "Retry",
     "retryLabel": "Retry {{name}}",
     "nodeCount": "{{count}} nodes",

core/http/react-ui/src/components/OperationCard.jsx

@@ -29,7 +29,7 @@ function formatEta(seconds) {
   return `${Math.floor(minutes / 60)}h ${minutes % 60}m`
 }
 
-export default function OperationCard({ operation, onCancel, onDismiss, onRetry }) {
+export default function OperationCard({ operation, onCancel, onPause, onDismiss, onRetry }) {
   const { t } = useTranslation('admin')
   const nodes = Array.isArray(operation.nodes) ? operation.nodes : []
   // Holds only what the user chose. The default has to stay a live
@@ -144,6 +144,16 @@ export default function OperationCard({ operation, onCancel, onDismiss, onRetry
 
         <div className="operation-card__actions">
           {showProgress && <span className="operation-card__pct" aria-hidden="true">{Math.round(operation.progress)}%</span>}
+          {canCancel && (
+            <button
+              type="button"
+              className="btn btn-sm btn-secondary operation-card__pause"
+              onClick={() => onPause?.(operation.jobID)}
+              aria-label={t('activity.pauseLabel', { name })}
+            >
+              {t('activity.pause')}
+            </button>
+          )}
           {canCancel && (
             // A page of cards would otherwise hand a screen reader a list of
             // identical "Cancel" buttons with nothing to tell them apart.

core/http/react-ui/src/contexts/OperationsContext.jsx

@@ -137,6 +137,16 @@ export function OperationsProvider({ children, pollInterval = 1000 }) {
     }
   }, [fetchOperations])
 
+  const pauseOperation = useCallback(async (jobID) => {
+    try {
+      await operationsApi.pause(jobID)
+      cancelledRef.current.set(jobID, Date.now())
+      await fetchOperations()
+    } catch (err) {
+      setError(err.message)
+    }
+  }, [fetchOperations])
+
   // Whether this tab cancelled the job. Read by the strip to tell "the last
   // operation finished" from "the user called it off": both look identical in
   // /api/operations, which lists neither.
@@ -226,6 +236,7 @@ export function OperationsProvider({ children, pollInterval = 1000 }) {
     fetchHistory,
     clearHistory,
     cancelOperation,
+    pauseOperation,
     wasCancelled,
     dismissFailedOp,
     refetch: fetchOperations,

core/http/react-ui/src/pages/Activity.jsx

@@ -83,7 +83,7 @@ export default function Activity() {
   const { t } = useTranslation('admin')
   const outlet = useOutletContext()
   const addToast = outlet?.addToast
-  const { operations, history, fetchHistory, clearHistory, cancelOperation, dismissFailedOp } = useOperations()
+  const { operations, history, fetchHistory, clearHistory, cancelOperation, pauseOperation, dismissFailedOp } = useOperations()
   const [filter, setFilter] = useState('all')
 
   useEffect(() => { fetchHistory() }, [fetchHistory])
@@ -195,7 +195,7 @@ export default function Activity() {
             {t('activity.inProgress')} <span className="activity-section__count">{live.length}</span>
           </h2>
           {live.map((op) => (
-            <OperationCard key={op.jobID || op.id} operation={op} onCancel={cancelOperation} />
+            <OperationCard key={op.jobID || op.id} operation={op} onCancel={cancelOperation} onPause={pauseOperation} />
           ))}
         </section>
       )}

core/http/react-ui/src/utils/api.js

@@ -173,6 +173,7 @@ export const resourcesApi = {
 export const operationsApi = {
   list: () => fetchJSON(API_CONFIG.endpoints.operations),
   cancel: (jobID) => postJSON(API_CONFIG.endpoints.cancelOperation(jobID), {}),
+  pause: (jobID) => postJSON(API_CONFIG.endpoints.pauseOperation(jobID), {}),
   dismiss: (jobID) => postJSON(API_CONFIG.endpoints.dismissOperation(jobID), {}),
   history: () => fetchJSON(API_CONFIG.endpoints.operationsHistory),
   clearHistory: () => fetchJSON(API_CONFIG.endpoints.operationsHistory, { method: 'DELETE' }),

core/http/react-ui/src/utils/config.js

@@ -4,6 +4,7 @@ export const API_CONFIG = {
     operations: '/api/operations',
     operationsHistory: '/api/operations/history',
     cancelOperation: (jobID) => `/api/operations/${jobID}/cancel`,
+    pauseOperation: (jobID) => `/api/operations/${jobID}/pause`,
     dismissOperation: (jobID) => `/api/operations/${jobID}/dismiss`,
 
     // Models gallery

core/http/routes/openai.go

@@ -141,6 +141,20 @@ func RegisterOpenAIRoutes(app *echo.Echo,
 	app.POST("/completions", completionHandler, completionMiddleware...)
 	app.POST("/v1/engines/:model/completions", completionHandler, completionMiddleware...)
 
+	// moderation
+	moderationHandler := openai.ModerationEndpoint(application.ModelConfigLoader(), application.ModelLoader(), application.TemplatesEvaluator(), application.ApplicationConfig())
+	moderationMiddleware := []echo.MiddlewareFunc{
+		nodeHeaderMiddleware,
+		usageMiddleware,
+		traceMiddleware,
+		re.BuildFilteredFirstAvailableDefaultModel(config.BuildUsecaseFilterFn(config.FLAG_COMPLETION)),
+		re.BuildConstantDefaultModelNameMiddleware("gpt-4o"),
+		re.SetModelAndConfig(func() schema.LocalAIRequest { return new(schema.ModerationRequest) }),
+		middleware.AdmissionControl(application.AdmissionLimiter(), application.PIIEvents()),
+	}
+	app.POST("/v1/moderations", moderationHandler, moderationMiddleware...)
+	app.POST("/moderations", moderationHandler, moderationMiddleware...)
+
 	// embeddings
 	embeddingHandler := openai.EmbeddingsEndpoint(application.ModelConfigLoader(), application.ModelLoader(), application.ApplicationConfig())
 	embeddingMiddleware := []echo.MiddlewareFunc{
@@ -254,6 +268,11 @@ func RegisterOpenAIRoutes(app *echo.Echo,
 	app.POST("/v1/images/inpainting", inpaintingHandler, imageMiddleware...)
 	app.POST("/images/inpainting", inpaintingHandler, imageMiddleware...)
 
+	// upscale endpoint - reuse same middleware config as images
+	upscaleHandler := openai.UpscaleEndpoint(application.ModelConfigLoader(), application.ModelLoader(), application.ApplicationConfig())
+	app.POST("/v1/images/upscale", upscaleHandler, imageMiddleware...)
+	app.POST("/images/upscale", upscaleHandler, imageMiddleware...)
+
 	// List models
 	app.GET("/v1/models", openai.ListModelsEndpoint(application.ModelConfigLoader(), application.ModelLoader(), application.ApplicationConfig(), application.AuthDB()))
 	app.GET("/models", openai.ListModelsEndpoint(application.ModelConfigLoader(), application.ModelLoader(), application.ApplicationConfig(), application.AuthDB()))

core/http/routes/ui_api.go

@@ -356,6 +356,24 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 		})
 	}, adminMiddleware)
 
+	// Pause operation endpoint (admin only). Unlike cancel, pause preserves a
+	// partial download so submitting the same install later resumes it.
+	app.POST("/api/operations/:jobID/pause", func(c echo.Context) error {
+		jobID := c.Param("jobID")
+		xlog.Debug("API request to pause operation", "jobID", jobID)
+
+		if err := galleryService.PauseOperation(jobID); err != nil {
+			xlog.Error("Failed to pause operation", "error", err, "jobID", jobID)
+			return c.JSON(http.StatusBadRequest, map[string]any{"error": err.Error()})
+		}
+
+		opcache.DeleteUUID(jobID)
+		return c.JSON(200, map[string]any{
+			"success": true,
+			"message": "Operation paused",
+		})
+	}, adminMiddleware)
+
 	// Dismiss a failed operation (acknowledge the error and remove it from the list)
 	app.POST("/api/operations/:jobID/dismiss", func(c echo.Context) error {
 		jobID := c.Param("jobID")
@@ -970,7 +988,7 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 		uid := id.String()
 		opcache.Set(galleryID, uid)
 
-		ctx, cancelFunc := context.WithCancel(context.Background())
+		ctx, cancelFunc, pauseFunc := galleryop.NewUserCancellableContext(context.Background())
 		op := galleryop.ManagementOp[gallery.GalleryModel, gallery.ModelConfig]{
 			ID:                 uid,
 			GalleryElementName: galleryID,
@@ -979,9 +997,10 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 			BackendGalleries:   appConfig.BackendGalleries,
 			Context:            ctx,
 			CancelFunc:         cancelFunc,
+			PauseFunc:          pauseFunc,
 		}
 		// Store cancellation function immediately so queued operations can be cancelled
-		galleryService.StoreCancellation(uid, cancelFunc)
+		galleryService.StoreCancellationActions(uid, cancelFunc, pauseFunc)
 		galleryService.EnqueueModelOp(op)
 
 		return c.JSON(200, map[string]any{
@@ -1017,7 +1036,7 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 
 		opcache.Set(galleryID, uid)
 
-		ctx, cancelFunc := context.WithCancel(context.Background())
+		ctx, cancelFunc, pauseFunc := galleryop.NewUserCancellableContext(context.Background())
 		op := galleryop.ManagementOp[gallery.GalleryModel, gallery.ModelConfig]{
 			ID:                 uid,
 			Delete:             true,
@@ -1026,9 +1045,10 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 			BackendGalleries:   appConfig.BackendGalleries,
 			Context:            ctx,
 			CancelFunc:         cancelFunc,
+			PauseFunc:          pauseFunc,
 		}
 		// Store cancellation function immediately so queued operations can be cancelled
-		galleryService.StoreCancellation(uid, cancelFunc)
+		galleryService.StoreCancellationActions(uid, cancelFunc, pauseFunc)
 		galleryService.EnqueueModelOp(op)
 		cl.RemoveModelConfig(galleryName)
 
@@ -1423,19 +1443,20 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 		uid := id.String()
 		opcache.SetBackend(backendID, uid)
 
-		ctx, cancelFunc := context.WithCancel(context.Background())
+		ctx, cancelFunc, pauseFunc := galleryop.NewUserCancellableContext(context.Background())
 		op := galleryop.ManagementOp[gallery.GalleryBackend, any]{
 			ID:                 uid,
 			GalleryElementName: backendID,
 			Galleries:          appConfig.BackendGalleries,
 			Context:            ctx,
 			CancelFunc:         cancelFunc,
+			PauseFunc:          pauseFunc,
 			// The React UI's "Reinstall backend" action reuses this route, so
 			// the op must force even when the backend is already installed.
 			Force: true,
 		}
 		// Store cancellation function immediately so queued operations can be cancelled
-		galleryService.StoreCancellation(uid, cancelFunc)
+		galleryService.StoreCancellationActions(uid, cancelFunc, pauseFunc)
 		galleryService.EnqueueBackendOp(op)
 
 		return c.JSON(200, map[string]any{
@@ -1485,19 +1506,20 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 		}
 		opcache.SetBackend(cacheKey, uid)
 
-		ctx, cancelFunc := context.WithCancel(context.Background())
+		ctx, cancelFunc, pauseFunc := galleryop.NewUserCancellableContext(context.Background())
 		op := galleryop.ManagementOp[gallery.GalleryBackend, any]{
 			ID:                 uid,
 			GalleryElementName: req.Name, // May be empty, will be derived during installation
 			Galleries:          appConfig.BackendGalleries,
 			Context:            ctx,
 			CancelFunc:         cancelFunc,
+			PauseFunc:          pauseFunc,
 			ExternalURI:        req.URI,
 			ExternalName:       req.Name,
 			ExternalAlias:      req.Alias,
 		}
 		// Store cancellation function immediately so queued operations can be cancelled
-		galleryService.StoreCancellation(uid, cancelFunc)
+		galleryService.StoreCancellationActions(uid, cancelFunc, pauseFunc)
 		galleryService.EnqueueBackendOp(op)
 
 		return c.JSON(200, map[string]any{
@@ -1533,7 +1555,7 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 
 		opcache.SetBackend(backendID, uid)
 
-		ctx, cancelFunc := context.WithCancel(context.Background())
+		ctx, cancelFunc, pauseFunc := galleryop.NewUserCancellableContext(context.Background())
 		op := galleryop.ManagementOp[gallery.GalleryBackend, any]{
 			ID:                 uid,
 			Delete:             true,
@@ -1541,9 +1563,10 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 			Galleries:          appConfig.BackendGalleries,
 			Context:            ctx,
 			CancelFunc:         cancelFunc,
+			PauseFunc:          pauseFunc,
 		}
 		// Store cancellation function immediately so queued operations can be cancelled
-		galleryService.StoreCancellation(uid, cancelFunc)
+		galleryService.StoreCancellationActions(uid, cancelFunc, pauseFunc)
 		galleryService.EnqueueBackendOp(op)
 
 		return c.JSON(200, map[string]any{
@@ -1652,7 +1675,7 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 		// and the Backends UI can reflect progress on the affected row.
 		opcache.SetBackend(backendName, uid)
 
-		ctx, cancelFunc := context.WithCancel(context.Background())
+		ctx, cancelFunc, pauseFunc := galleryop.NewUserCancellableContext(context.Background())
 		op := galleryop.ManagementOp[gallery.GalleryBackend, any]{
 			ID:                 uid,
 			GalleryElementName: backendName,
@@ -1660,9 +1683,10 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 			Upgrade:            true,
 			Context:            ctx,
 			CancelFunc:         cancelFunc,
+			PauseFunc:          pauseFunc,
 		}
 		// Store cancellation function immediately so queued operations can be cancelled
-		galleryService.StoreCancellation(uid, cancelFunc)
+		galleryService.StoreCancellationActions(uid, cancelFunc, pauseFunc)
 		galleryService.EnqueueBackendOp(op)
 
 		return c.JSON(200, map[string]any{

core/http/routes/ui_api_operations_test.go

@@ -339,6 +339,33 @@ var _ = Describe("/api/operations with node-scoped backend ops", func() {
 		Expect(envelope.Operations[0]).ToNot(HaveKey("isCancelled"))
 	})
 
+	It("pauses through the resume-safe operation callback", func() {
+		state, err := system.GetSystemState(system.WithModelPath(GinkgoT().TempDir()))
+		Expect(err).NotTo(HaveOccurred())
+		appCfg := &config.ApplicationConfig{SystemState: state}
+		galleryService := galleryop.NewGalleryService(appCfg, nil)
+		opcache := galleryop.NewOpCache(galleryService)
+		opcache.Set("localai@gemma", "job-pause")
+
+		var cancelled, paused bool
+		galleryService.StoreCancellationActions(
+			"job-pause",
+			func() { cancelled = true },
+			func() { paused = true },
+		)
+
+		e := echo.New()
+		routes.RegisterUIAPIRoutes(e, nil, nil, appCfg, galleryService, opcache, &application.Application{}, noopMw)
+		req := httptest.NewRequest(http.MethodPost, "/api/operations/job-pause/pause", nil)
+		rec := httptest.NewRecorder()
+		e.ServeHTTP(rec, req)
+
+		Expect(rec.Code).To(Equal(http.StatusOK))
+		Expect(paused).To(BeTrue())
+		Expect(cancelled).To(BeFalse())
+		Expect(opcache.Get("localai@gemma")).To(BeEmpty())
+	})
+
 	It("reports a running removal as a deletion", func() {
 		state, err := system.GetSystemState(system.WithModelPath(GinkgoT().TempDir()))
 		Expect(err).NotTo(HaveOccurred())

core/schema/moderation.go

@@ -0,0 +1,52 @@
+package schema
+
+import (
+	"encoding/json"
+	"fmt"
+)
+
+// ModerationInput accepts the text-only forms supported by the OpenAI
+// moderations API. Multimodal moderation can be added without changing the
+// response contract once LocalAI has a moderation-capable vision path.
+type ModerationInput []string
+
+func (i *ModerationInput) UnmarshalJSON(data []byte) error {
+	var single string
+	if err := json.Unmarshal(data, &single); err == nil {
+		*i = ModerationInput{single}
+		return nil
+	}
+
+	var multiple []string
+	if err := json.Unmarshal(data, &multiple); err == nil {
+		*i = ModerationInput(multiple)
+		return nil
+	}
+
+	return fmt.Errorf("input must be a text string or array of text strings")
+}
+
+func (i ModerationInput) MarshalJSON() ([]byte, error) {
+	if len(i) == 1 {
+		return json.Marshal(i[0])
+	}
+	return json.Marshal([]string(i))
+}
+
+type ModerationRequest struct {
+	BasicModelRequest
+	Input ModerationInput `json:"input"`
+}
+
+type ModerationResult struct {
+	Flagged                   bool                `json:"flagged"`
+	Categories                map[string]bool     `json:"categories"`
+	CategoryScores            map[string]float64  `json:"category_scores"`
+	CategoryAppliedInputTypes map[string][]string `json:"category_applied_input_types"`
+}
+
+type ModerationResponse struct {
+	ID      string             `json:"id"`
+	Model   string             `json:"model"`
+	Results []ModerationResult `json:"results"`
+}

core/services/galleryop/cancel_persist_test.go

@@ -54,6 +54,22 @@ var _ = Describe("GalleryService.CancelOperation persistence", func() {
 		Expect(fresh.GetStatus("op-cancel")).To(BeNil(),
 			"a cancelled op must not hydrate back as active after a restart")
 	})
+
+	It("pauses with the resume-safe callback instead of the destructive cancel callback", func() {
+		svc := galleryop.NewGalleryService(&config.ApplicationConfig{}, nil)
+		var cancelled, paused bool
+		svc.StoreCancellationActions("op-pause", func() { cancelled = true }, func() { paused = true })
+
+		Expect(svc.PauseOperation("op-pause")).To(Succeed())
+		Expect(paused).To(BeTrue())
+		Expect(cancelled).To(BeFalse())
+
+		status := svc.GetStatus("op-pause")
+		Expect(status).ToNot(BeNil())
+		Expect(status.Processed).To(BeTrue())
+		Expect(status.Cancelled).To(BeTrue())
+		Expect(status.Message).To(Equal("paused"))
+	})
 })
 
 // Reproduces "an op orphaned by a replica that died mid-flight stays 'pending'

core/services/galleryop/operation.go

@@ -32,6 +32,7 @@ type ManagementOp[T any, E any] struct {
 	// Context for cancellation support
 	Context    context.Context
 	CancelFunc context.CancelFunc
+	PauseFunc  context.CancelFunc
 
 	// External backend installation parameters (for OCI/URL/path)
 	// These are used when installing backends from external sources rather than galleries
@@ -189,6 +190,7 @@ type GalleryProgressEvent struct {
 // runs the cancel func on whichever replica registered it.
 type GalleryCancelEvent struct {
 	JobID string `json:"id"`
+	Pause bool   `json:"pause,omitempty"`
 }
 
 // NodeStatus values shared between NodeProgress (per-node tick) and the

core/services/galleryop/service.go

@@ -28,7 +28,7 @@ type GalleryService struct {
 	modelManager   ModelManager
 	backendManager BackendManager
 	statuses       map[string]*OpStatus
-	cancellations  map[string]context.CancelFunc
+	cancellations  map[string]cancellationActions
 
 	// Distributed mode (nil when not in distributed mode).
 	// natsClient is the wider MessagingClient (Publisher + subscribe methods)
@@ -67,7 +67,7 @@ func NewGalleryService(appConfig *config.ApplicationConfig, ml *model.ModelLoade
 		modelManager:          NewLocalModelManager(appConfig, ml),
 		backendManager:        NewLocalBackendManager(appConfig, ml),
 		statuses:              make(map[string]*OpStatus),
-		cancellations:         make(map[string]context.CancelFunc),
+		cancellations:         make(map[string]cancellationActions),
 	}
 }
 
@@ -387,6 +387,17 @@ func (g *GalleryService) failStaleStatus(id string) {
 // SubjectGalleryCancelWildcard subscriber and runs it locally. The caller
 // gets a non-error reply so the UI shows the cancel as accepted.
 func (g *GalleryService) CancelOperation(id string) error {
+	return g.stopOperation(id, false)
+}
+
+// PauseOperation stops an in-progress download while preserving its partial
+// file. Re-submitting the same install resumes it through the downloader's
+// existing HTTP Range support.
+func (g *GalleryService) PauseOperation(id string) error {
+	return g.stopOperation(id, true)
+}
+
+func (g *GalleryService) stopOperation(id string, pause bool) error {
 	g.Lock()
 
 	if status, ok := g.statuses[id]; ok && status.Cancelled {
@@ -394,7 +405,7 @@ func (g *GalleryService) CancelOperation(id string) error {
 		return fmt.Errorf("operation %q is already cancelled", id)
 	}
 
-	cancelFunc, localExists := g.cancellations[id]
+	actions, localExists := g.cancellations[id]
 	if localExists {
 		delete(g.cancellations, id)
 	}
@@ -410,12 +421,12 @@ func (g *GalleryService) CancelOperation(id string) error {
 	if status, ok := g.statuses[id]; ok {
 		status.Cancelled = true
 		status.Processed = true
-		status.Message = "cancelled"
+		status.Message = map[bool]string{true: "paused", false: "cancelled"}[pause]
 	} else {
 		g.statuses[id] = &OpStatus{
 			Cancelled:   true,
 			Processed:   true,
-			Message:     "cancelled",
+			Message:     map[bool]string{true: "paused", false: "cancelled"}[pause],
 			Cancellable: false,
 		}
 	}
@@ -435,11 +446,15 @@ func (g *GalleryService) CancelOperation(id string) error {
 	// I/O and user-provided callback after Unlock — the cancel-wildcard
 	// subscriber loops back into applyCancel on this same replica, which
 	// would otherwise deadlock on g.Mutex.
-	if cancelFunc != nil {
-		cancelFunc()
+	stopFunc := actions.cancel
+	if pause {
+		stopFunc = actions.pause
+	}
+	if stopFunc != nil {
+		stopFunc()
 	}
 	if nc != nil {
-		if err := nc.Publish(messaging.SubjectGalleryCancel(id), GalleryCancelEvent{JobID: id}); err != nil {
+		if err := nc.Publish(messaging.SubjectGalleryCancel(id), GalleryCancelEvent{JobID: id, Pause: pause}); err != nil {
 			xlog.Warn("Failed to broadcast gallery cancel", "op_id", id, "error", err)
 		}
 	}
@@ -452,9 +467,9 @@ func (g *GalleryService) CancelOperation(id string) error {
 // run the local cancel func if we have one (no echo via NATS), and reflect
 // the cancellation in the local statuses map. Idempotent: a replica that
 // already cancelled this op locally treats the inbound event as a no-op.
-func (g *GalleryService) applyCancel(id string) {
+func (g *GalleryService) applyCancel(id string, pause bool) {
 	g.Lock()
-	cancelFunc, hasCancel := g.cancellations[id]
+	actions, hasCancel := g.cancellations[id]
 	if hasCancel {
 		delete(g.cancellations, id)
 	}
@@ -465,12 +480,12 @@ func (g *GalleryService) applyCancel(id string) {
 		}
 		status.Cancelled = true
 		status.Processed = true
-		status.Message = "cancelled"
+		status.Message = map[bool]string{true: "paused", false: "cancelled"}[pause]
 	} else {
 		g.statuses[id] = &OpStatus{
 			Cancelled:   true,
 			Processed:   true,
-			Message:     "cancelled",
+			Message:     map[bool]string{true: "paused", false: "cancelled"}[pause],
 			Cancellable: false,
 		}
 	}
@@ -478,8 +493,12 @@ func (g *GalleryService) applyCancel(id string) {
 
 	// Invoke the cancel func after Unlock so a callback that touches
 	// GalleryService doesn't re-enter the mutex.
-	if hasCancel {
-		cancelFunc()
+	stopFunc := actions.cancel
+	if pause {
+		stopFunc = actions.pause
+	}
+	if hasCancel && stopFunc != nil {
+		stopFunc()
 	}
 }
 
@@ -488,23 +507,40 @@ func (g *GalleryService) applyCancel(id string) {
 // distinguish a deliberate user cancel (discard the half-downloaded .partial)
 // from an incidental cancellation such as process shutdown (keep the .partial
 // so the next run resumes via Range instead of restarting from zero).
-func newUserCancellableContext(parent context.Context) (context.Context, context.CancelFunc) {
+// NewUserCancellableContext creates distinct callbacks for destructive cancel
+// and resume-safe pause while sharing one operation context.
+func NewUserCancellableContext(parent context.Context) (context.Context, context.CancelFunc, context.CancelFunc) {
 	ctx, cancelCause := context.WithCancelCause(parent)
-	return ctx, func() { cancelCause(downloader.ErrUserCancelled) }
+	return ctx,
+		func() { cancelCause(downloader.ErrUserCancelled) },
+		func() { cancelCause(context.Canceled) }
 }
 
-// storeCancellation stores a cancellation function for an operation
-func (g *GalleryService) storeCancellation(id string, cancelFunc context.CancelFunc) {
+type cancellationActions struct {
+	cancel context.CancelFunc
+	pause  context.CancelFunc
+}
+
+func (g *GalleryService) storeCancellation(id string, cancelFunc, pauseFunc context.CancelFunc) {
 	g.Lock()
 	defer g.Unlock()
-	g.cancellations[id] = cancelFunc
+	if pauseFunc == nil {
+		pauseFunc = cancelFunc
+	}
+	g.cancellations[id] = cancellationActions{cancel: cancelFunc, pause: pauseFunc}
 }
 
 // StoreCancellation is a public method to store a cancellation function for an operation
 // This allows cancellation functions to be stored immediately when operations are created,
 // enabling cancellation of queued operations that haven't started processing yet.
 func (g *GalleryService) StoreCancellation(id string, cancelFunc context.CancelFunc) {
-	g.storeCancellation(id, cancelFunc)
+	g.storeCancellation(id, cancelFunc, cancelFunc)
+}
+
+// StoreCancellationActions registers distinct destructive-cancel and
+// resume-safe pause callbacks for an operation.
+func (g *GalleryService) StoreCancellationActions(id string, cancelFunc, pauseFunc context.CancelFunc) {
+	g.storeCancellation(id, cancelFunc, pauseFunc)
 }
 
 // removeCancellation removes a cancellation function when operation completes
@@ -554,10 +590,10 @@ func (g *GalleryService) Start(c context.Context, cl *config.ModelConfigLoader,
 			case op := <-g.BackendGalleryChannel:
 				// Create context if not provided
 				if op.Context == nil {
-					op.Context, op.CancelFunc = newUserCancellableContext(c)
-					g.storeCancellation(op.ID, op.CancelFunc)
+					op.Context, op.CancelFunc, op.PauseFunc = NewUserCancellableContext(c)
+					g.storeCancellation(op.ID, op.CancelFunc, op.PauseFunc)
 				} else if op.CancelFunc != nil {
-					g.storeCancellation(op.ID, op.CancelFunc)
+					g.storeCancellation(op.ID, op.CancelFunc, op.PauseFunc)
 				}
 				// Create DB record for distributed tracking
 				if g.galleryStore != nil {
@@ -597,10 +633,10 @@ func (g *GalleryService) Start(c context.Context, cl *config.ModelConfigLoader,
 			case op := <-g.ModelGalleryChannel:
 				// Create context if not provided
 				if op.Context == nil {
-					op.Context, op.CancelFunc = newUserCancellableContext(c)
-					g.storeCancellation(op.ID, op.CancelFunc)
+					op.Context, op.CancelFunc, op.PauseFunc = NewUserCancellableContext(c)
+					g.storeCancellation(op.ID, op.CancelFunc, op.PauseFunc)
 				} else if op.CancelFunc != nil {
-					g.storeCancellation(op.ID, op.CancelFunc)
+					g.storeCancellation(op.ID, op.CancelFunc, op.PauseFunc)
 				}
 				// Create DB record for distributed tracking
 				if g.galleryStore != nil {
@@ -663,7 +699,7 @@ func (g *GalleryService) SubscribeBroadcasts() error {
 		if evt.JobID == "" {
 			return
 		}
-		g.applyCancel(evt.JobID)
+		g.applyCancel(evt.JobID, evt.Pause)
 	})
 	if err != nil {
 		if uerr := progressSub.Unsubscribe(); uerr != nil {

core/services/nodes/health_mock_test.go

@@ -154,6 +154,9 @@ func (c *fakeBackendClient) Predict(_ context.Context, _ *pb.PredictOptions, _ .
 func (c *fakeBackendClient) GenerateImage(_ context.Context, _ *pb.GenerateImageRequest, _ ...ggrpc.CallOption) (*pb.Result, error) {
 	return nil, nil
 }
+func (c *fakeBackendClient) UpscaleImage(_ context.Context, _ *pb.UpscaleImageRequest, _ ...ggrpc.CallOption) (*pb.Result, error) {
+	return nil, nil
+}
 func (c *fakeBackendClient) GenerateVideo(_ context.Context, _ *pb.GenerateVideoRequest, _ ...ggrpc.CallOption) (*pb.Result, error) {
 	return nil, nil
 }

core/services/nodes/inflight.go

@@ -138,6 +138,12 @@ func (c *InFlightTrackingClient) GenerateImage(ctx context.Context, in *pb.Gener
 	return res, c.reconcile(err)
 }
 
+func (c *InFlightTrackingClient) UpscaleImage(ctx context.Context, in *pb.UpscaleImageRequest, opts ...ggrpc.CallOption) (*pb.Result, error) {
+	defer c.track(ctx)()
+	res, err := c.inner.UpscaleImage(ctx, in, opts...)
+	return res, c.reconcile(err)
+}
+
 func (c *InFlightTrackingClient) GenerateVideo(ctx context.Context, in *pb.GenerateVideoRequest, opts ...ggrpc.CallOption) (*pb.Result, error) {
 	defer c.track(ctx)()
 	res, err := c.inner.GenerateVideo(ctx, in, opts...)

core/services/nodes/inflight_test.go

@@ -83,6 +83,10 @@ func (f *fakeGRPCBackend) GenerateImage(_ context.Context, _ *pb.GenerateImageRe
 	return &pb.Result{}, nil
 }
 
+func (f *fakeGRPCBackend) UpscaleImage(_ context.Context, _ *pb.UpscaleImageRequest, _ ...ggrpc.CallOption) (*pb.Result, error) {
+	return &pb.Result{}, nil
+}
+
 func (f *fakeGRPCBackend) GenerateVideo(_ context.Context, _ *pb.GenerateVideoRequest, _ ...ggrpc.CallOption) (*pb.Result, error) {
 	return &pb.Result{}, nil
 }

docs/content/features/authentication.md

@@ -157,7 +157,7 @@ When authentication is enabled, the following endpoints require admin role:
 **Model & Backend Management:**
 - `GET /api/models`, `POST /api/models/install/*`, `POST /api/models/delete/*`
 - `GET /api/backends`, `POST /api/backends/install/*`, `POST /api/backends/delete/*`
-- `GET /api/operations`, `POST /api/operations/*/cancel`, `POST /api/operations/*/dismiss`
+- `GET /api/operations`, `POST /api/operations/*/cancel`, `POST /api/operations/*/pause`, `POST /api/operations/*/dismiss`
 - `GET /api/operations/history`, `DELETE /api/operations/history`
 - `GET /models/available`, `GET /models/galleries`, `GET /models/jobs/*`
 - `GET /backends`, `GET /backends/available`, `GET /backends/galleries`

docs/content/features/constrained_grammars.md

@@ -69,4 +69,5 @@ For more complex grammars, you can define multi-line BNF rules. The grammar pars
 ## Related Features
 
 - [OpenAI Functions]({{%relref "features/openai-functions" %}}) - Function calling with structured outputs
-- [Text Generation]({{%relref "features/text-generation" %}}) - General text generation capabilities
+- [Text Generation]({{%relref "features/text-generation" %}}) - General text generation capabilities
+- [Moderation]({{%relref "features/moderation" %}}) - OpenAI-compatible safety classification whose response is constrained to the moderation schema

docs/content/features/image-generation.md

@@ -164,6 +164,30 @@ By default the RPC devices join the pool and participate in placement; combine w
 ![anime_girl](https://github.com/go-skynet/LocalAI/assets/2420543/8aaca62a-e864-4011-98ae-dcc708103928)
 (Generated with [AnimagineXL](https://huggingface.co/Linaqruf/animagine-xl))
 
+#### Image upscaling
+
+LocalAI can upscale an uploaded image by a factor of 2 or 4 through
+`POST /v1/images/upscale`. Install the included Stable Diffusion x4 upscaler
+gallery model first:
+
+```bash
+local-ai models install stable-diffusion-x4-upscaler
+```
+
+Then send the model name, scale, and image as multipart form fields:
+
+```bash
+curl http://localhost:8080/v1/images/upscale \
+  -F model=stable-diffusion-x4-upscaler \
+  -F scale=4 \
+  -F image=@input.png
+```
+
+The response uses the same format as image generation and returns the generated
+image under `/generated-images`. The `diffusers` backend uses a loaded
+`StableDiffusionUpscalePipeline` or `StableDiffusionLatentUpscalePipeline` when
+configured. Other diffusers pipelines fall back to Lanczos resizing.
+
 #### Model setup
 
 The models will be downloaded the first time you use the backend from `huggingface` automatically.

docs/content/features/moderation.md

@@ -0,0 +1,37 @@
++++
+disableToc = false
+title = "Moderation"
+weight = 65
+url = "/features/moderation/"
++++
+
+LocalAI exposes an OpenAI-compatible text moderation endpoint at
+`POST /v1/moderations`. It uses a local text-generation model with a constrained
+JSON grammar, so no separate moderation service or cloud API is required.
+
+```bash
+curl http://localhost:8080/v1/moderations \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "your-instruct-model",
+    "input": "Text to classify"
+  }'
+```
+
+`input` may be one string or an array of strings. The response contains one
+result per input with `flagged`, `categories`, `category_scores`, and
+`category_applied_input_types` fields. The category names match the OpenAI
+moderation API, including harassment, hate, illicit activity, self-harm,
+sexual content, and violence categories.
+
+The selected model must support text completion. For consistent results, use
+an instruction-tuned model that follows safety-classification prompts well.
+LocalAI constrains the output shape, but the model determines the classification
+quality and confidence scores.
+
+{{% notice note %}}
+
+This first implementation supports text only. OpenAI-style multimodal input
+objects containing images return a validation error.
+
+{{% /notice %}}

docs/content/features/text-generation.md

@@ -918,6 +918,200 @@ options:
 The full list of registered parsers lives in `sglang.srt.function_call`
 and `sglang.srt.parser.reasoning_parser`.
 
+### vllm.cpp
+
+[vllm.cpp](https://github.com/mudler/vllm.cpp) is the LocalAI team's C++ port of
+vLLM: the same continuous-batching scheduler, paged KV cache and prefix caching,
+with no Python at inference time. It consumes either a HuggingFace safetensors
+model directory or a `.gguf` file, and applies the model's chat template,
+tool-call parsing and reasoning split engine-side.
+
+#### Setup
+
+```yaml
+name: vllm-cpp
+backend: vllm-cpp
+parameters:
+  model: "Qwen/Qwen3-4B"
+context_size: 8192
+template:
+  use_tokenizer_template: true
+```
+
+#### Configuring the engine with `engine_args`
+
+The same `engine_args:` map the vLLM and SGLang backends accept is honoured
+here, with keys spelled exactly as vLLM's own CLI flags - so a `speculative_config`
+or `kv_transfer_config` block written for vLLM works verbatim. Unknown keys are
+ignored rather than fatal; the engine validates the documents it is handed and
+reports a precise error at load.
+
+```yaml
+name: qwen35-a3b
+backend: vllm-cpp
+parameters:
+  model: "Qwen/Qwen3.5-A3B"
+context_size: 16384
+template:
+  use_tokenizer_template: true
+engine_args:
+  # KV cache sizing: num_blocks * block_size tokens of cache.
+  block_size: 32
+  num_blocks: 1024
+  # Concurrency and the per-step chunked-prefill token budget.
+  max_num_seqs: 32
+  max_num_batched_tokens: 8192
+  # Automatic prefix caching. Omit to keep the model's own default
+  # (on for dense models, off for hybrid / attention-free ones).
+  enable_prefix_caching: true
+  # Scheduler admission order: fcfs (default), priority, or lpm
+  # (cache-aware longest-prefix-match; needs prefix caching to have any effect).
+  scheduling_policy: lpm
+```
+
+| Key | Meaning | Default |
+|-----|---------|---------|
+| `block_size` | KV-cache block size, in tokens per block | 32 |
+| `num_blocks` | KV-cache blocks to allocate | 256 |
+| `max_model_len` | Max sequence length; also settable as `context_size` / `max_model_len` | model config |
+| `max_num_seqs` | Max concurrent sequences the scheduler admits | 8 |
+| `max_num_batched_tokens` | Per-step chunked-prefill token budget | per-arch (2048 dense, 4096/8192 MoE) |
+| `enable_prefix_caching` | Automatic prefix caching; `enable_radix_attention` is an accepted alias | model default |
+| `enable_jump_forward` | Jump-forward decoding, which emits grammar-forced tokens without a model step. Only affects constrained requests (`grammar`, JSON schema) | off |
+| `scheduling_policy` | `fcfs`, `priority`, or `lpm` | `fcfs` |
+| `tool_parser` / `reasoning_parser` | Force a parser instead of chat-template auto-detection | auto |
+| `tokenizer_config` | Override the `tokenizer_config.json` the chat template is read from | `<model_dir>/tokenizer_config.json` |
+| `speculative_config` | Speculative decoding (see below) | disabled |
+| `kv_transfer_config` | External KV connector / LMCache (see below) | none |
+
+Raising `max_num_batched_tokens` lets more prefill land in a single step, at the
+cost of decode latency for requests queued behind it. The default deliberately
+does not scale with `max_num_seqs`, which is what keeps a large concurrent
+prefill from blowing up the per-step activation on the hybrid architectures.
+
+`enable_prefix_caching` and `enable_jump_forward` are tri-state at the engine
+boundary: omitting the key defers to a default (the model's own capability for
+prefix caching, an environment variable for jump forward), while an explicit
+`false` forces the feature off. Those are genuinely different - prefix caching
+defaults *on* for dense models - so write the key only when you mean to override.
+
+#### Speculative decoding
+
+`speculative_config:` takes the same JSON object as vLLM's
+`--speculative-config`. Three methods are supported.
+
+> **Architecture limit.** At the current engine pin, `mtp` and `dflash` are
+> **Qwen3.5 / Qwen3.6 only**. The engine builds a widened speculative KV cache
+> directly for those families rather than through the model registry, so a
+> speculative config on any other architecture (Llama, GLM, Gemma, Mistral, ...)
+> will not work regardless of checkpoint format. `ngram` needs no draft weights
+> and is not subject to this limit.
+
+> **Format support.** `mtp` and `dflash` now work from a `.gguf` target as well
+> as safetensors. An MTP head is read from the GGUF's `nextn.*` tensors when the
+> file declares `<arch>.nextn_predict_layers`; a GGUF exported WITHOUT the head
+> (converted with `--no-mtp`, or predating llama.cpp's Qwen3.5 MTP support) is
+> refused at load naming that as the reason. A DFlash draft may itself be a
+> `dflash`-arch GGUF, and the target may be a GGUF too. `ngram` needs no draft
+> weights and works on any format.
+
+**MTP** (Multi-Token Prediction) uses a draft head shipped inside the target
+checkpoint's own `mtp.*` tensors, so there is no second model to download. It
+requires a **safetensors** checkpoint - the `mtp.*` tensors do not survive GGUF
+conversion, and an MTP config over a `.gguf` model is rejected at load.
+
+```yaml
+engine_args:
+  speculative_config:
+    method: mtp
+    # Optional; defaults to the checkpoint's own head depth, which is
+    # usually the right value. Must be a multiple of that depth.
+    num_speculative_tokens: 1
+```
+
+**DFlash** uses a separate block-diffusion drafter that proposes a whole block
+of tokens in one non-autoregressive forward pass. Unlike MTP, the draft is its
+own checkpoint, so `model:` is **required**:
+
+```yaml
+engine_args:
+  speculative_config:
+    method: dflash
+    model: z-lab/Qwen3.6-27B-DFlash
+    num_speculative_tokens: 4
+```
+
+The draft shares the *target's* `embed_tokens` and `lm_head`, so both must come
+from the same model family and the target must be safetensors.
+
+**The engine does not download the draft.** `model:` is resolved, in order,
+as a path as given, then as the last path segment under LocalAI's models
+directory (`z-lab/Qwen3.6-27B-DFlash` → `<models>/Qwen3.6-27B-DFlash`, which is
+what LocalAI's own downloader produces), then as the whole reference under the
+models directory. Install the draft into LocalAI first, or give an absolute path
+to a directory containing `config.json`. If none of those resolve, the load
+fails immediately naming every location that was tried, rather than reporting a
+missing checkpoint from inside the engine.
+
+**N-gram** needs no draft model at all - it proposes from the prompt's own
+suffix history. `num_speculative_tokens` is required:
+
+```yaml
+engine_args:
+  speculative_config:
+    method: ngram
+    num_speculative_tokens: 4
+    prompt_lookup_min: 5
+    prompt_lookup_max: 5
+```
+
+> **Auto-configuration on import.** When you import a safetensors repository
+> with `backend: vllm-cpp`, LocalAI reads the checkpoint's `config.json` and, if
+> it declares an MTP head (`mtp_num_hidden_layers`), writes
+> `speculative_config: {method: mtp}` into the generated `engine_args` for you.
+> An explicit `speculative_config` in your own config is never overwritten.
+> Importing a DFlash *draft* repository is refused with a warning: a drafter
+> cannot serve on its own, so import the target model and point
+> `speculative_config.model` at the draft.
+
+#### External KV cache with LMCache
+
+`kv_transfer_config:` takes vLLM's `--kv-transfer-config` JSON and selects an
+external KV-cache connector. The `lm://` LMCache client lets prefill KV be
+stored to and reloaded from a shared `lmcache.v1.server`, so a prefix computed
+by one replica does not have to be recomputed by the next:
+
+```yaml
+engine_args:
+  kv_transfer_config:
+    kv_connector: LMCacheConnector
+    kv_role: kv_both          # required whenever kv_connector is set
+    kv_connector_extra_config:
+      host: 127.0.0.1
+      port: 65432
+```
+
+`kv_role` is one of `kv_producer` (store only), `kv_consumer` (load only), or
+`kv_both`. An unregistered connector name, a missing role, or a malformed
+document fails the load with an explicit error rather than silently running
+without the cache.
+
+#### Legacy `options:` list
+
+Earlier versions configured this backend through the flat `options:` list, and
+those configs keep working. Every key in the table above is still read from
+there in `key:value` form, and `engine_args` wins on any key set in both:
+
+```yaml
+options:
+  - max_num_seqs:32
+  - enable_prefix_caching:true
+```
+
+New configs should prefer `engine_args:`, which is the only place the nested
+`speculative_config` / `kv_transfer_config` documents can be written naturally
+rather than as a single-line JSON string.
+
 ### Transformers
 
 [Transformers](https://huggingface.co/docs/transformers/index) is a State-of-the-art Machine Learning library for PyTorch, TensorFlow, and JAX.

docs/content/operations/activity.md

@@ -170,10 +170,15 @@ Both surfaces are backed by these endpoints. All of them are admin-only when
 | -------- | -------------------------------- | ------------------------------------------------------------------------ |
 | `GET`    | `/api/operations`                | Running, queued and failed operations, least advanced first.              |
 | `POST`   | `/api/operations/{jobID}/cancel` | Cancel a queued operation, or a running install. Not a running removal.  |
+| `POST`   | `/api/operations/{jobID}/pause`  | Pause a queued operation or running download and preserve partial data.  |
 | `POST`   | `/api/operations/{jobID}/dismiss`| Acknowledge a failed operation and move it into the record.               |
 | `GET`    | `/api/operations/history`        | List finished operations, newest first.                                   |
 | `DELETE` | `/api/operations/history`        | Clear the record. Live operations are untouched.                          |
 
+Pausing preserves the download's `.partial` file. Start the same model or
+backend installation again to resume from the saved bytes when the origin
+supports HTTP Range requests. Cancelling intentionally discards partial data.
+
 Both `GET` endpoints wrap their list in an `operations` key rather than
 returning a bare array:
 

docs/content/whats-new.md

@@ -6,26 +6,11 @@ url = '/basics/news/'
 icon = "newspaper"
 +++
 
-Release notes have been now moved completely over Github releases.
+LocalAI news is published in two places, both kept current:
 
-You can see the release notes [here](https://github.com/mudler/LocalAI/releases).
+- **[Blog](https://localai.io/blog/)** for release write-ups, benchmark reports and engineering notes.
+- **[GitHub Releases](https://github.com/mudler/LocalAI/releases)** for the full changelog of every version.
 
-## 2026 Highlights
+For how the project got here, read [LocalAI, from March 2023 to now](https://localai.io/blog/localai-since-march-2023/).
 
-- **July 2026**: [LongCat video and avatar generation](/features/video-generation/) - dedicated CUDA backend for `LongCat-Video` text/image-to-video and `LongCat-Video-Avatar-1.5` speech-driven avatars. Includes multi-segment continuation, portrait and recorded-audio inputs in Studio, and an SDPA CUDA 13 ARM64 build for DGX Spark.
-- **April 2026**: [Audio Transform](/features/audio-transform/) - generic audio-in / audio-out endpoint with optional reference signal. First implementation: [LocalVQE](https://github.com/localai-org/LocalVQE) C++ backend (joint AEC + noise suppression + dereverberation, DeepVQE-style). Both batch (`POST /audio/transformations`) and bidirectional WebSocket streaming (`/audio/transformations/stream`). Studio "Transform" tab with synchronized waveform players for input / reference / output.
-- **April 2026**: [Face recognition backend](/features/face-recognition/) - `insightface`-powered 1:1 verification, 1:N identification, face embedding, face detection, and demographic analysis. Ships both a non-commercial `buffalo_l` model and an Apache 2.0 OpenCV Zoo alternative.
-- **May 2026**: [Speaker diarization](/features/audio-diarization/) - new `/v1/audio/diarization` endpoint returning "who spoke when" segments. Backed by `sherpa-onnx` (pyannote-3.0 + speaker embeddings + clustering) for pure diarization, and `vibevoice-cpp` for diarization bundled with long-form ASR. Supports `json` / `verbose_json` / `rttm` response formats.
-- **June 2026**: [Sound classification](/features/audio-classification/) - new `/v1/audio/classification` endpoint for audio tagging / sound-event classification, returning scored [AudioSet](https://research.google.com/audioset/) labels (baby cry, glass breaking, alarms, ...). Backed by [ced.cpp](https://github.com/localai-org/ced.cpp), a 527-class AudioSet tagger ported to ggml.
-- **June 2026**: [PII analyze / redact API](/features/middleware/#analyze--redact-api) - the PII detection pipeline (NER + restricted-regex pattern tiers) is now a standalone service: `POST /api/pii/analyze` returns detected entity spans and `POST /api/pii/redact` returns the sanitised text (or `400 pii_blocked`), without routing a chat request through the middleware. Events gain an `origin` (`middleware` / `proxy` / `pii_analyze` / `pii_redact`) so `/api/pii/events` can be filtered by source.
-- **July 2026**: [Model capabilities endpoint](/features/api-discovery/#model-capabilities) - `GET /v1/models/capabilities`, an additive superset of `/v1/models` that reports each model's `capabilities` plus its `input_modalities` / `output_modalities` (`text` / `image` / `audio` / `video`). Lets clients route attachments using inferred or explicitly declared model modalities instead of backend-name checks.
-- **June 2026**: Concurrent scoring and PII NER on llama.cpp - the `Score` (router classifier) and `TokenClassify` (PII NER) primitives now ride llama.cpp's server task queue instead of locking the context, so they run concurrently with chat/completion/embedding traffic and with each other. The `known_usecases` restriction that forced dedicated scorer/NER model configs on llama-cpp is lifted, repeated scoring calls reuse the prompt KV cache across candidates, and scoring inputs are no longer capped by the physical batch size.
-
-## 2024 Highlights
-
-- **April 2024**: [Reranker API](https://github.com/mudler/LocalAI/pull/2121)
-- **May 2024**: [Distributed inferencing](https://github.com/mudler/LocalAI/pull/2324), [Decentralized P2P llama.cpp](https://github.com/mudler/LocalAI/pull/2343) - [Docs](https://localai.io/features/distribute/)
-- **July/August 2024**: [P2P Dashboard, Federated mode and AI Swarms](https://github.com/mudler/LocalAI/pull/2723), [P2P Global community pools](https://github.com/mudler/LocalAI/issues/3113), FLUX-1 support, [P2P Explorer](https://explorer.localai.io)
-- **October 2024**: Examples moved to [LocalAI-examples](https://github.com/mudler/LocalAI-examples)
-- **November 2024**: [Voice Activity Detection (VAD)](https://github.com/mudler/LocalAI/pull/4204), [Bark.cpp backend](https://github.com/mudler/LocalAI/pull/4287)
-- **December 2024**: [stablediffusion.cpp backend (ggml)](https://github.com/mudler/LocalAI/pull/4289)
+This page used to carry a hand-maintained highlights list. It drifted against both sources above, so it now points at them instead.

gallery/index.yaml

@@ -1,4 +1,59 @@
 ---
+- &nemotron-3-embed-1b
+  name: "nemotron-3-embed-1b-q4"
+  url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
+  variants:
+    - model: nemotron-3-embed-8b-q4
+  license: openmdw-1.1
+  urls:
+    - https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16
+    - https://huggingface.co/zenmagnets/Nemotron-3-Embed-1B-Q4_K_M-GGUF
+  description: |
+    Nemotron-3-Embed-1B is NVIDIA's multilingual text embedding model for
+    retrieval, semantic search, and RAG. This compact Q4_K_M GGUF produces
+    2,048-dimensional normalized embeddings and supports 36 languages. Prefix
+    retrieval queries with `query: ` and documents with `passage: `.
+  tags:
+    - embeddings
+    - multilingual
+    - retrieval
+    - rag
+    - gguf
+    - cpu
+    - gpu
+  overrides:
+    backend: llama-cpp
+    embeddings: true
+    known_usecases:
+      - embeddings
+    parameters:
+      model: llama-cpp/models/nemotron-3-embed-1b-q4_k_m.gguf
+  files:
+    - filename: llama-cpp/models/nemotron-3-embed-1b-q4_k_m.gguf
+      uri: huggingface://zenmagnets/Nemotron-3-Embed-1B-Q4_K_M-GGUF/nemotron-3-embed-1b-q4_k_m.gguf
+      sha256: 9a74166f51dbc280073748fa199bea49283bd21f7f9280f2dec2b4d975ddfd1d
+- !!merge <<: *nemotron-3-embed-1b
+  name: "nemotron-3-embed-8b-q4"
+  variants: []
+  urls:
+    - https://huggingface.co/nvidia/Nemotron-3-Embed-8B-BF16
+    - https://huggingface.co/Abiray/Nemotron-3-Embed-8B-GGUF
+  description: |
+    Nemotron-3-Embed-8B is NVIDIA's larger multilingual text embedding model
+    for retrieval, semantic search, and RAG. This Q4_K_M GGUF balances retrieval
+    quality with local resource use and supports 36 languages. Prefix retrieval
+    queries with `query: ` and documents with `passage: `.
+  overrides:
+    backend: llama-cpp
+    embeddings: true
+    known_usecases:
+      - embeddings
+    parameters:
+      model: llama-cpp/models/Nemotron-3-Embed-8B-Q4_K_M.gguf
+  files:
+    - filename: llama-cpp/models/Nemotron-3-Embed-8B-Q4_K_M.gguf
+      uri: huggingface://Abiray/Nemotron-3-Embed-8B-GGUF/Nemotron-3-Embed-8B-Q4_K_M.gguf
+      sha256: a2aa29c618da6eed10d9474e72e33188c61e5fd700aed2fe9a1d98abdc90c6fc
 - &grug-27b
   name: "grug-27b"
   variants:
@@ -111,7 +166,8 @@
 - name: "deepseek-v4-flash-0731"
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
-    - https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF
+    - https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
+    - https://huggingface.co/ggml-org/DeepSeek-V4-Flash-0731-GGUF
   description: "# DeepSeek-V4-Flash-0731\n\nTechnical Report\U0001F441️\n\n## Introduction\n\n**DeepSeek-V4-Flash-0731** is the official release of **DeepSeek-V4-Flash**, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.\n\nDeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available.\n\nNotes:\n\n1. For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the `max` reasoning effort level with `temperature = 1.0, top_p = 0.95`.\n2. † DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems.\n\n## Chat Template\n\n...\n"
   license: "mit"
   tags:
@@ -120,20 +176,20 @@
     - deepseek
   icon: https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/logo.svg
   overrides:
-    backend: ds4
+    backend: llama-cpp
     function:
       grammar:
         disable: true
     known_usecases:
       - chat
     parameters:
-      model: ds4flash.gguf
+      model: DeepSeek-V4-Flash-0731-MXFP4.gguf
     template:
       use_tokenizer_template: true
   files:
-    - filename: ds4flash.gguf
-      uri: https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF
-      sha256: ef1940d5a91c294393c60b4083b497a05f6df1174e9a12225843945b228c5d20
+    - filename: DeepSeek-V4-Flash-0731-MXFP4.gguf
+      uri: huggingface://ggml-org/DeepSeek-V4-Flash-0731-GGUF/DeepSeek-V4-Flash-0731-MXFP4.gguf
+      sha256: 65f73494afaf27d3add0751a5b716dd2d3e012c66ae0dbbcc1bf8477f92b3ab7
 - name: instella-moe-16b-a3b-think
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
@@ -255,7 +311,7 @@
   files:
     - filename: llama-cpp/models/Parable-Granite-4.1-3B-Claude-Fable-5-Q4_K_M/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf
       uri: https://huggingface.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF/resolve/main/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf
-      sha256: 67dc7695d92939c713165761f115c9d892fdff74fcbd987c8bb453b9b8ab645d
+      sha256: dbf202638af23e72508d8316577655d24ba2037fda51ce802b8996977e290bce
 - name: "parable-qwen3-4b-claude-fable-5"
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
@@ -289,7 +345,7 @@
   files:
     - filename: llama-cpp/models/Parable-Qwen3-4B-Claude-Fable-5-Q4_K_M/Parable-Qwen3-4B-Claude-Fable-5-GGUF-Q4_K_M.gguf
       uri: https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-GGUF/resolve/main/Parable-Qwen3-4B-Claude-Fable-5-GGUF-Q4_K_M.gguf
-      sha256: c94b06a912aa901f3da5689754577ad534415efafc50dcee3f389594a153bf38
+      sha256: 65cc4824fb78ecaf55afdfcdb6dd2e27e1aa805d289db89eae94d32d450403f0
 - name: "parable-granite-4.1-8b-claude-fable-5"
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
@@ -325,7 +381,7 @@
   files:
     - filename: llama-cpp/models/Parable-Granite-4.1-8B-Claude-Fable-5-Q4_K_M/Parable-Granite-4.1-8B-Claude-Fable-5-GGUF-Q4_K_M.gguf
       uri: https://huggingface.co/AnkitAI/Parable-Granite-4.1-8B-Claude-Fable-5-GGUF/resolve/main/Parable-Granite-4.1-8B-Claude-Fable-5-GGUF-Q4_K_M.gguf
-      sha256: 61a8133c344a0d0a00188395afe33c803e3b973cb4bbfd5ef1fa7110e80bc1c3
+      sha256: 57e464ae3d35253d4351639757dc35e71bab8324d12d49a5870695ce73dc19cf
 - name: "parable-qwen3-8b-claude-fable-5"
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
@@ -359,7 +415,7 @@
   files:
     - filename: llama-cpp/models/Parable-Qwen3-8B-Claude-Fable-5-Q4_K_M/Parable-Qwen3-8B-Claude-Fable-5-GGUF-Q4_K_M.gguf
       uri: https://huggingface.co/AnkitAI/Parable-Qwen3-8B-Claude-Fable-5-GGUF/resolve/main/Parable-Qwen3-8B-Claude-Fable-5-GGUF-Q4_K_M.gguf
-      sha256: 956070afc8023b8665fe450842f7be76b505b53d142460fd9b588222f4e16112
+      sha256: 4532d2379d38a37279866a030e51d419561f9d4d22fee00d2a33647d66f05065
 - &pocket-35b
   name: "pocket-35b"
   variants:
@@ -1331,6 +1387,85 @@
     - filename: llama-cpp/mmproj/Qwythos-9B-v2-MTP-Q4_K_M/mmproj-Qwythos-9B-v2-BF16.gguf
       sha256: 0d1687cb33124c78acab788b342d4a2eaf85b3035e87c3abe4ee9d0b84ddb4f5
       uri: https://huggingface.co/empero-ai/Qwythos-9B-v2-GGUF/resolve/main/mmproj-Qwythos-9B-v2-BF16.gguf
+- &qwen3-6-14b-a3b-fablevibes
+  name: "qwen3.6-14b-a3b-fablevibes"
+  variants:
+    - model: qwen3.6-14b-a3b-fablevibes-q8
+  url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
+  urls:
+    - https://huggingface.co/tvall43/Qwen3.6-14B-A3B-FableVibes
+    - https://huggingface.co/tvall43/Qwen3.6-14B-A3B-FableVibes-GGUF
+  description: |
+    Qwen3.6-14B-A3B-FableVibes is an Apache-2.0 mixture-of-experts reasoning
+    model distilled from Fable 5 and Claude Opus traces, with additional tool
+    calling and coding data. It retains Qwen 3.6 vision support while pruning
+    the 35B-A3B base to a 14B consumer-oriented footprint. This default entry
+    uses the recommended Q4_K_M GGUF quantization and its Q8_0 multimodal
+    projector.
+  license: "apache-2.0"
+  tags:
+    - llm
+    - gguf
+    - cpu
+    - gpu
+    - moe
+    - reasoning
+    - thinking
+    - vision
+    - multimodal
+  last_checked: "2026-08-03"
+  overrides:
+    backend: llama-cpp
+    function:
+      automatic_tool_parsing_fallback: true
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    mmproj: llama-cpp/mmproj/Qwen3.6-14B-A3B-FableVibes/Qwen3.6-14B-A3B-FableVibes-mmproj-Q8_0.gguf
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/Qwen3.6-14B-A3B-FableVibes/Qwen3.6-14B-A3B-FableVibes-Q4_K_M.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Qwen3.6-14B-A3B-FableVibes/Qwen3.6-14B-A3B-FableVibes-Q4_K_M.gguf
+      sha256: 21aa4b0b28090469e8a319c889451df2f1ea6aad27ac3818c8c8a86f86d5bc9e
+      uri: huggingface://tvall43/Qwen3.6-14B-A3B-FableVibes-GGUF/Qwen3.6-14B-A3B-FableVibes-Q4_K_M.gguf
+    - filename: llama-cpp/mmproj/Qwen3.6-14B-A3B-FableVibes/Qwen3.6-14B-A3B-FableVibes-mmproj-Q8_0.gguf
+      sha256: ca27dbf0c65a7232e9458bfdda8bc45efc09ab60e4cc6f58ea0c7b7cc2253257
+      uri: huggingface://tvall43/Qwen3.6-14B-A3B-FableVibes-GGUF/Qwen3.6-14B-A3B-FableVibes-mmproj-Q8_0.gguf
+- !!merge <<: *qwen3-6-14b-a3b-fablevibes
+  name: "qwen3.6-14b-a3b-fablevibes-q8"
+  variants: []
+  description: |
+    Qwen3.6-14B-A3B-FableVibes is an Apache-2.0 mixture-of-experts reasoning
+    model distilled from Fable 5 and Claude Opus traces, with additional tool
+    calling and coding data. This entry uses the near-lossless Q8_0 GGUF
+    quantization and its matching Q8_0 multimodal projector.
+  overrides:
+    backend: llama-cpp
+    function:
+      automatic_tool_parsing_fallback: true
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    mmproj: llama-cpp/mmproj/Qwen3.6-14B-A3B-FableVibes-Q8_0/Qwen3.6-14B-A3B-FableVibes-mmproj-Q8_0.gguf
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/Qwen3.6-14B-A3B-FableVibes-Q8_0/Qwen3.6-14B-A3B-FableVibes-Q8_0.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Qwen3.6-14B-A3B-FableVibes-Q8_0/Qwen3.6-14B-A3B-FableVibes-Q8_0.gguf
+      sha256: ddea86093b863215fa75969d81df494fafdb0e6c4a65af557ed3a710e2238e58
+      uri: huggingface://tvall43/Qwen3.6-14B-A3B-FableVibes-GGUF/Qwen3.6-14B-A3B-FableVibes-Q8_0.gguf
+    - filename: llama-cpp/mmproj/Qwen3.6-14B-A3B-FableVibes-Q8_0/Qwen3.6-14B-A3B-FableVibes-mmproj-Q8_0.gguf
+      sha256: ca27dbf0c65a7232e9458bfdda8bc45efc09ab60e4cc6f58ea0c7b7cc2253257
+      uri: huggingface://tvall43/Qwen3.6-14B-A3B-FableVibes-GGUF/Qwen3.6-14B-A3B-FableVibes-mmproj-Q8_0.gguf
 - name: "qwen3.6-27b-fable-fusion-711-uncensored-heretic-nm-dau-neo-max-mtp"
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
@@ -1874,7 +2009,7 @@
   files:
     - filename: ds4flash.gguf
       uri: https://huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF
-      sha256: 856c407993ccffa9ad52e23fbef8bb7b458c792a52278f4ca7931741b0c20ce2
+      sha256: 1bfdafd1c288eb1b2bcb629ee9e1b7567dcf0abbe4d20995905a3c3465e9bd1e
 - name: "qwopus3.6-35b-a3b-coder-mtp"
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
@@ -8959,6 +9094,34 @@
     - filename: llama-cpp/models/deepseek-ai.DeepSeek-V3.2.Q4_K_M-00029-of-00029.gguf
       sha256: 013af4e9d2f84e484f77c7bae2a02652607f0f0179bd2815ffdf401c3ada5184
       uri: https://huggingface.co/DevQuasar/deepseek-ai.DeepSeek-V3.2-GGUF/resolve/main/deepseek-ai.DeepSeek-V3.2.Q4_K_M-00029-of-00029.gguf
+- name: stable-diffusion-x4-upscaler
+  url: github:mudler/LocalAI/gallery/virtual.yaml@master
+  urls:
+    - https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler
+  description: |
+    Stable Diffusion x4 Upscaler is Stability AI's diffusion-based super-resolution model. It enlarges low-resolution images by four times while reconstructing image detail.
+  license: openrail++
+  tags:
+    - image-upscaling
+    - super-resolution
+    - image-to-image
+    - diffusers
+  last_checked: "2026-07-30"
+  overrides:
+    backend: diffusers
+    f16: true
+    diffusers:
+      pipeline_type: StableDiffusionUpscalePipeline
+    known_usecases:
+      - image
+    parameters:
+      model: stabilityai/stable-diffusion-x4-upscaler
+    artifacts:
+      - name: model
+        target: model
+        source:
+          type: huggingface
+          repo: stabilityai/stable-diffusion-x4-upscaler
 - name: z-image-diffusers
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:

pkg/grpc/backend.go

@@ -72,6 +72,7 @@ type InferenceBackend interface {
 	PredictStream(ctx context.Context, in *pb.PredictOptions, f func(reply *pb.Reply), opts ...grpc.CallOption) error
 	Predict(ctx context.Context, in *pb.PredictOptions, opts ...grpc.CallOption) (*pb.Reply, error)
 	GenerateImage(ctx context.Context, in *pb.GenerateImageRequest, opts ...grpc.CallOption) (*pb.Result, error)
+	UpscaleImage(ctx context.Context, in *pb.UpscaleImageRequest, opts ...grpc.CallOption) (*pb.Result, error)
 	GenerateVideo(ctx context.Context, in *pb.GenerateVideoRequest, opts ...grpc.CallOption) (*pb.Result, error)
 	Generate3D(ctx context.Context, in *pb.Generate3DRequest, opts ...grpc.CallOption) (*pb.Result, error)
 	TTS(ctx context.Context, in *pb.TTSRequest, opts ...grpc.CallOption) (*pb.Result, error)

pkg/grpc/base/base.go

@@ -63,6 +63,10 @@ func (llm *Base) Generate3D(*pb.Generate3DRequest) error {
 	return fmt.Errorf("unimplemented")
 }
 
+func (llm *Base) UpscaleImage(*pb.UpscaleImageRequest) error {
+	return fmt.Errorf("unimplemented")
+}
+
 func (llm *Base) AudioTranscription(context.Context, *pb.TranscriptRequest) (pb.TranscriptResult, error) {
 	return pb.TranscriptResult{}, fmt.Errorf("unimplemented")
 }

pkg/grpc/client.go

@@ -230,6 +230,23 @@ func (c *Client) GenerateImage(ctx context.Context, in *pb.GenerateImageRequest,
 	return client.GenerateImage(ctx, in, opts...)
 }
 
+func (c *Client) UpscaleImage(ctx context.Context, in *pb.UpscaleImageRequest, opts ...grpc.CallOption) (*pb.Result, error) {
+	if !c.parallel {
+		c.opMutex.Lock()
+		defer c.opMutex.Unlock()
+	}
+	c.setBusy(true)
+	defer c.setBusy(false)
+	defer c.wdMark()()
+	conn, err := c.dial()
+	if err != nil {
+		return nil, err
+	}
+	defer conn.Close()
+	client := pb.NewBackendClient(conn)
+	return client.UpscaleImage(ctx, in, opts...)
+}
+
 func (c *Client) GenerateVideo(ctx context.Context, in *pb.GenerateVideoRequest, opts ...grpc.CallOption) (*pb.Result, error) {
 	if !c.parallel {
 		c.opMutex.Lock()

pkg/grpc/embed.go

@@ -49,6 +49,10 @@ func (e *embedBackend) GenerateImage(ctx context.Context, in *pb.GenerateImageRe
 	return e.s.GenerateImage(ctx, in)
 }
 
+func (e *embedBackend) UpscaleImage(ctx context.Context, in *pb.UpscaleImageRequest, opts ...grpc.CallOption) (*pb.Result, error) {
+	return e.s.UpscaleImage(ctx, in)
+}
+
 func (e *embedBackend) GenerateVideo(ctx context.Context, in *pb.GenerateVideoRequest, opts ...grpc.CallOption) (*pb.Result, error) {
 	return e.s.GenerateVideo(ctx, in)
 }

pkg/grpc/interface.go

@@ -17,6 +17,7 @@ type AIModel interface {
 	Free() error
 	Embeddings(*pb.PredictOptions) ([]float32, error)
 	GenerateImage(*pb.GenerateImageRequest) error
+	UpscaleImage(*pb.UpscaleImageRequest) error
 	GenerateVideo(*pb.GenerateVideoRequest) error
 	Generate3D(*pb.Generate3DRequest) error
 	Detect(*pb.DetectOptions) (pb.DetectResponse, error)

pkg/grpc/server.go

@@ -150,6 +150,18 @@ func (s *server) GenerateImage(ctx context.Context, in *pb.GenerateImageRequest)
 	return &pb.Result{Message: "Image generated", Success: true}, nil
 }
 
+func (s *server) UpscaleImage(ctx context.Context, in *pb.UpscaleImageRequest) (*pb.Result, error) {
+	if s.llm.Locking() {
+		s.llm.Lock()
+		defer s.llm.Unlock()
+	}
+	err := s.llm.UpscaleImage(in)
+	if err != nil {
+		return &pb.Result{Message: fmt.Sprintf("Error upscaling image: %s", err.Error()), Success: false}, err
+	}
+	return &pb.Result{Message: "Image upscaled", Success: true}, nil
+}
+
 func (s *server) GenerateVideo(ctx context.Context, in *pb.GenerateVideoRequest) (*pb.Result, error) {
 	if err := s.checkModelIdentity(in); err != nil {
 		return nil, err

pkg/mcp/localaitools/client.go

@@ -64,6 +64,10 @@ type LocalAIClient interface {
 	// ---- System ----
 	SystemInfo(ctx context.Context) (*SystemInfo, error)
 	ListNodes(ctx context.Context) ([]Node, error)
+	ListScheduling(ctx context.Context) ([]ModelSchedulingConfig, error)
+	GetScheduling(ctx context.Context, modelName string) (*ModelSchedulingConfig, error)
+	SetScheduling(ctx context.Context, req SetSchedulingRequest) (*ModelSchedulingConfig, error)
+	DeleteScheduling(ctx context.Context, modelName string) error
 	// SetNodeVRAMBudget sets (or, with an empty budget, clears) a federated
 	// node's VRAM allocation cap as a sticky admin override. Only meaningful
 	// in distributed mode; single-process clients report it as unavailable.

pkg/mcp/localaitools/coverage_test.go

@@ -35,6 +35,8 @@ var toolToHTTPRoute = map[string]string{
 	ToolListKnownBackends:   "GET /backends/known",
 	ToolSystemInfo:          "GET / (welcome JSON)",
 	ToolListNodes:           "GET /api/nodes",
+	ToolListScheduling:      "GET /api/nodes/scheduling",
+	ToolGetScheduling:       "GET /api/nodes/scheduling/:model",
 	ToolVRAMEstimate:        "POST /api/models/vram-estimate",
 	ToolGetBranding:         "GET /api/branding",
 	ToolGetUsageStats:       "GET /api/usage (or /api/usage/all when all=true)",
@@ -60,6 +62,8 @@ var toolToHTTPRoute = map[string]string{
 	ToolCreateVoiceProfile: "POST /api/voice-profiles",
 	ToolDeleteVoiceProfile: "DELETE /api/voice-profiles/:id",
 	ToolSetNodeVRAMBudget:  "PUT /api/nodes/:id/vram-budget",
+	ToolSetScheduling:      "POST /api/nodes/scheduling",
+	ToolDeleteScheduling:   "DELETE /api/nodes/scheduling/:model",
 }
 
 // allKnownTools is the union of expectedFullCatalog (defined in

pkg/mcp/localaitools/dto.go

@@ -115,6 +115,41 @@ type SetNodeVRAMBudgetRequest struct {
 	Budget string `json:"budget,omitempty" jsonschema:"VRAM allocation cap as a percentage (e.g. 80%) or absolute amount (e.g. 12GB). Empty string clears the override."`
 }
 
+// ModelSchedulingConfig is the MCP wire shape for one per-model distributed
+// scheduling rule. Keep this DTO explicit instead of aliasing the node-registry
+// model so the MCP contract only exposes operator-facing scheduling fields.
+type ModelSchedulingConfig struct {
+	ModelName           string  `json:"model_name"`
+	NodeSelector        string  `json:"node_selector,omitempty"`
+	MinReplicas         int     `json:"min_replicas"`
+	MaxReplicas         int     `json:"max_replicas"`
+	SpreadAll           bool    `json:"spread_all,omitempty"`
+	RoutePolicy         string  `json:"route_policy,omitempty"`
+	BalanceAbsThreshold int     `json:"balance_abs_threshold,omitempty"`
+	BalanceRelThreshold float64 `json:"balance_rel_threshold,omitempty"`
+	MinPrefixMatch      float64 `json:"min_prefix_match,omitempty"`
+}
+
+// SetSchedulingRequest is the input for set_scheduling. It mirrors
+// /api/nodes/scheduling so standalone MCP and REST callers preserve the same
+// PATCH-style semantics for the optional prefix-cache routing fields.
+type SetSchedulingRequest struct {
+	ModelName           string            `json:"model_name"                         jsonschema:"Installed model name whose distributed scheduling rule should be created or updated."`
+	NodeSelector        map[string]string `json:"node_selector,omitempty"            jsonschema:"Optional node-label selector. Empty means any healthy backend node."`
+	MinReplicas         int               `json:"min_replicas"                       jsonschema:"Minimum desired replicas. Mutually exclusive with spread_all."`
+	MaxReplicas         int               `json:"max_replicas"                       jsonschema:"Maximum desired replicas. Must be >= min_replicas when non-zero. Mutually exclusive with spread_all."`
+	SpreadAll           bool              `json:"spread_all,omitempty"               jsonschema:"When true, keep one replica on every matching node. Mutually exclusive with min_replicas/max_replicas."`
+	RoutePolicy         *string           `json:"route_policy,omitempty"             jsonschema:"Optional prefix-cache route policy override. Omit to preserve the existing value on updates."`
+	BalanceAbsThreshold *int              `json:"balance_abs_threshold,omitempty"    jsonschema:"Optional absolute imbalance threshold override. Omit to preserve the existing value on updates."`
+	BalanceRelThreshold *float64          `json:"balance_rel_threshold,omitempty"    jsonschema:"Optional relative imbalance threshold override. Omit to preserve the existing value on updates."`
+	MinPrefixMatch      *float64          `json:"min_prefix_match,omitempty"         jsonschema:"Optional minimum prefix match threshold override. Omit to preserve the existing value on updates."`
+}
+
+// DeleteSchedulingRequest identifies the model scheduling rule to remove.
+type DeleteSchedulingRequest struct {
+	ModelName string `json:"model_name" jsonschema:"Installed model name whose scheduling config should be removed."`
+}
+
 // ImportModelURIRequest is the input for import_model_uri. It mirrors the
 // REST surface (`/models/import-uri`) closely so both clients can produce
 // identical responses; the BackendPreference is a flat field rather than the

pkg/mcp/localaitools/fakes_test.go

@@ -42,6 +42,10 @@ type fakeClient struct {
 	upgradeBackend      func(string) (string, error)
 	systemInfo          func() (*SystemInfo, error)
 	listNodes           func() ([]Node, error)
+	listScheduling      func() ([]ModelSchedulingConfig, error)
+	getScheduling       func(string) (*ModelSchedulingConfig, error)
+	setScheduling       func(SetSchedulingRequest) (*ModelSchedulingConfig, error)
+	deleteScheduling    func(string) error
 	setNodeVRAMBudget   func(string, string) error
 	vramEstimate        func(VRAMEstimateRequest) (*vram.EstimateResult, error)
 	toggleModelState    func(string, modeladmin.Action) error
@@ -230,6 +234,38 @@ func (f *fakeClient) ListNodes(_ context.Context) ([]Node, error) {
 	return nil, nil
 }
 
+func (f *fakeClient) ListScheduling(_ context.Context) ([]ModelSchedulingConfig, error) {
+	f.record("ListScheduling", nil)
+	if f.listScheduling != nil {
+		return f.listScheduling()
+	}
+	return []ModelSchedulingConfig{}, nil
+}
+
+func (f *fakeClient) GetScheduling(_ context.Context, modelName string) (*ModelSchedulingConfig, error) {
+	f.record("GetScheduling", modelName)
+	if f.getScheduling != nil {
+		return f.getScheduling(modelName)
+	}
+	return &ModelSchedulingConfig{ModelName: modelName}, nil
+}
+
+func (f *fakeClient) SetScheduling(_ context.Context, req SetSchedulingRequest) (*ModelSchedulingConfig, error) {
+	f.record("SetScheduling", req)
+	if f.setScheduling != nil {
+		return f.setScheduling(req)
+	}
+	return &ModelSchedulingConfig{ModelName: req.ModelName, MinReplicas: req.MinReplicas, MaxReplicas: req.MaxReplicas, SpreadAll: req.SpreadAll}, nil
+}
+
+func (f *fakeClient) DeleteScheduling(_ context.Context, modelName string) error {
+	f.record("DeleteScheduling", modelName)
+	if f.deleteScheduling != nil {
+		return f.deleteScheduling(modelName)
+	}
+	return nil
+}
+
 func (f *fakeClient) SetNodeVRAMBudget(_ context.Context, nodeID, budget string) error {
 	f.record("SetNodeVRAMBudget", []any{nodeID, budget})
 	if f.setNodeVRAMBudget != nil {

pkg/mcp/localaitools/httpapi/client.go

@@ -483,6 +483,49 @@ func (c *Client) ListNodes(ctx context.Context) ([]localaitools.Node, error) {
 	return out, nil
 }
 
+func (c *Client) ListScheduling(ctx context.Context) ([]localaitools.ModelSchedulingConfig, error) {
+	var out []localaitools.ModelSchedulingConfig
+	if err := c.do(ctx, http.MethodGet, routeScheduling, nil, &out); err != nil {
+		if errors.Is(err, ErrHTTPNotFound) {
+			return []localaitools.ModelSchedulingConfig{}, nil
+		}
+		return nil, err
+	}
+	return out, nil
+}
+
+func (c *Client) GetScheduling(ctx context.Context, modelName string) (*localaitools.ModelSchedulingConfig, error) {
+	if modelName == "" {
+		return nil, errors.New("model_name is required")
+	}
+	var out localaitools.ModelSchedulingConfig
+	if err := c.do(ctx, http.MethodGet, routeModelScheduling(modelName), nil, &out); err != nil {
+		if errors.Is(err, ErrHTTPNotFound) {
+			return nil, nil
+		}
+		return nil, err
+	}
+	return &out, nil
+}
+
+func (c *Client) SetScheduling(ctx context.Context, req localaitools.SetSchedulingRequest) (*localaitools.ModelSchedulingConfig, error) {
+	if req.ModelName == "" {
+		return nil, errors.New("model_name is required")
+	}
+	var out localaitools.ModelSchedulingConfig
+	if err := c.do(ctx, http.MethodPost, routeScheduling, req, &out); err != nil {
+		return nil, err
+	}
+	return &out, nil
+}
+
+func (c *Client) DeleteScheduling(ctx context.Context, modelName string) error {
+	if modelName == "" {
+		return errors.New("model_name is required")
+	}
+	return c.do(ctx, http.MethodDelete, routeModelScheduling(modelName), nil, nil)
+}
+
 func (c *Client) SetNodeVRAMBudget(ctx context.Context, nodeID, budget string) error {
 	// PUT with an empty value clears the override server-side (Task 9), so we
 	// use PUT uniformly rather than switching to DELETE for the clear case.

pkg/mcp/localaitools/httpapi/client_test.go

@@ -84,6 +84,43 @@ func fakeLocalAI() *httptest.Server {
 		})
 	})
 
+	mux.HandleFunc("/api/nodes/scheduling", func(w http.ResponseWriter, r *http.Request) {
+		switch r.Method {
+		case http.MethodGet:
+			_ = json.NewEncoder(w).Encode([]map[string]any{{
+				"id":                  "sched-1",
+				"model_name":          "qwen",
+				"min_replicas":        1,
+				"max_replicas":        2,
+				"unsatisfiable_ticks": 1,
+				"unsatisfiable_until": "2026-01-01T00:00:00Z",
+				"created_at":          "2026-01-01T00:00:00Z",
+				"updated_at":          "2026-01-01T00:00:00Z",
+			}})
+		case http.MethodPost:
+			var body map[string]any
+			if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
+				http.Error(w, err.Error(), http.StatusBadRequest)
+				return
+			}
+			body["id"] = "sched-1"
+			_ = json.NewEncoder(w).Encode(body)
+		default:
+			http.Error(w, "method", http.StatusMethodNotAllowed)
+		}
+	})
+
+	mux.HandleFunc("/api/nodes/scheduling/qwen", func(w http.ResponseWriter, r *http.Request) {
+		switch r.Method {
+		case http.MethodGet:
+			_ = json.NewEncoder(w).Encode(map[string]any{"model_name": "qwen", "spread_all": true})
+		case http.MethodDelete:
+			w.WriteHeader(http.StatusNoContent)
+		default:
+			http.Error(w, "method", http.StatusMethodNotAllowed)
+		}
+	})
+
 	return httptest.NewServer(mux)
 }
 
@@ -197,8 +234,51 @@ var _ = Describe("httpapi.Client against the LocalAI admin REST surface", func()
 			Expect(bs[0].Installed).To(BeTrue())
 		})
 	})
+
+	Describe("Scheduling", func() {
+		It("lists scheduling configs", func() {
+			out, err := c.ListScheduling(ctx)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(out).To(HaveLen(1))
+			Expect(out[0].ModelName).To(Equal("qwen"))
+			Expect(out[0].MinReplicas).To(Equal(1))
+			Expect(schedulingJSONKeys(&out[0])).ToNot(Or(
+				HaveKey("id"),
+				HaveKey("unsatisfiable_until"),
+				HaveKey("unsatisfiable_ticks"),
+				HaveKey("created_at"),
+				HaveKey("updated_at"),
+			))
+		})
+
+		It("gets one scheduling config", func() {
+			out, err := c.GetScheduling(ctx, "qwen")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(out.ModelName).To(Equal("qwen"))
+			Expect(out.SpreadAll).To(BeTrue())
+		})
+
+		It("sets a scheduling config", func() {
+			out, err := c.SetScheduling(ctx, localaitools.SetSchedulingRequest{ModelName: "qwen", MinReplicas: 1, MaxReplicas: 2})
+			Expect(err).ToNot(HaveOccurred())
+			Expect(out.ModelName).To(Equal("qwen"))
+			Expect(out.MaxReplicas).To(Equal(2))
+		})
+
+		It("deletes a scheduling config", func() {
+			Expect(c.DeleteScheduling(ctx, "qwen")).To(Succeed())
+		})
+	})
 })
 
+func schedulingJSONKeys(config *localaitools.ModelSchedulingConfig) map[string]any {
+	var out map[string]any
+	b, err := json.Marshal(config)
+	Expect(err).ToNot(HaveOccurred())
+	Expect(json.Unmarshal(b, &out)).To(Succeed())
+	return out
+}
+
 var _ = Describe("Model aliases", func() {
 	Describe("ListAliases", func() {
 		It("passes the GET /api/aliases payload through unchanged", func() {

pkg/mcp/localaitools/httpapi/routes.go

@@ -24,6 +24,7 @@ const (
 	routeBackendsKnown   = "/backends/known"
 	routeBackendsApply   = "/backends/apply"
 	routeNodes           = "/api/nodes"
+	routeScheduling      = "/api/nodes/scheduling"
 	routeVRAMEstimate    = "/api/models/vram-estimate"
 	routeBranding        = "/api/branding"
 	routeSettings        = "/api/settings"
@@ -66,3 +67,7 @@ func routeVoiceProfileDelete(id string) string {
 func routeNodeVRAMBudget(id string) string {
 	return "/api/nodes/" + url.PathEscape(id) + "/vram-budget"
 }
+
+func routeModelScheduling(modelName string) string {
+	return "/api/nodes/scheduling/" + url.PathEscape(modelName)
+}

pkg/mcp/localaitools/inproc/client.go

@@ -23,6 +23,8 @@ import (
 	"github.com/mudler/LocalAI/core/schema"
 	"github.com/mudler/LocalAI/core/services/galleryop"
 	"github.com/mudler/LocalAI/core/services/modeladmin"
+	"github.com/mudler/LocalAI/core/services/nodes"
+	"github.com/mudler/LocalAI/core/services/nodes/prefixcache"
 	"github.com/mudler/LocalAI/core/services/routing/billing"
 	"github.com/mudler/LocalAI/core/services/routing/pii"
 	"github.com/mudler/LocalAI/core/services/routing/router"
@@ -47,6 +49,7 @@ type Client struct {
 	ConfigLoader  *config.ModelConfigLoader
 	ModelLoader   *model.ModelLoader
 	Gallery       *galleryop.GalleryService
+	NodeRegistry  *nodes.NodeRegistry
 	VoiceProfiles *voiceprofile.Store
 
 	// StatsRecorder and FallbackUser are optional — they back the
@@ -75,13 +78,18 @@ type Client struct {
 // except ModelLoader (used only for SystemInfo's loaded-models report and
 // best-effort ShutdownModel calls during config edits) and the stats
 // fields (StatsRecorder, FallbackUser) which gate get_usage_stats.
-func New(appConfig *config.ApplicationConfig, systemState *system.SystemState, cl *config.ModelConfigLoader, ml *model.ModelLoader, gs *galleryop.GalleryService) *Client {
+func New(appConfig *config.ApplicationConfig, systemState *system.SystemState, cl *config.ModelConfigLoader, ml *model.ModelLoader, gs *galleryop.GalleryService, registries ...*nodes.NodeRegistry) *Client {
+	var registry *nodes.NodeRegistry
+	if len(registries) > 0 {
+		registry = registries[0]
+	}
 	return &Client{
 		AppConfig:     appConfig,
 		SystemState:   systemState,
 		ConfigLoader:  cl,
 		ModelLoader:   ml,
 		Gallery:       gs,
+		NodeRegistry:  registry,
 		VoiceProfiles: voiceprofile.NewStore(appConfig.DataPath),
 		modelAdmin:    modeladmin.NewConfigService(cl, appConfig),
 	}
@@ -501,6 +509,121 @@ func (c *Client) ListNodes(_ context.Context) ([]localaitools.Node, error) {
 	return []localaitools.Node{}, nil
 }
 
+func (c *Client) ListScheduling(ctx context.Context) ([]localaitools.ModelSchedulingConfig, error) {
+	if c.NodeRegistry == nil {
+		return []localaitools.ModelSchedulingConfig{}, nil
+	}
+	configs, err := c.NodeRegistry.ListModelSchedulings(ctx)
+	if err != nil {
+		return nil, err
+	}
+	return localaitools.SchedulingConfigsFromNodes(configs), nil
+}
+
+func (c *Client) GetScheduling(ctx context.Context, modelName string) (*localaitools.ModelSchedulingConfig, error) {
+	if modelName == "" {
+		return nil, errors.New("model_name is required")
+	}
+	if c.NodeRegistry == nil {
+		return nil, errors.New("model scheduling is only available in distributed mode")
+	}
+	config, err := c.NodeRegistry.GetModelScheduling(ctx, modelName)
+	if err != nil {
+		return nil, err
+	}
+	if config == nil {
+		return nil, nil
+	}
+	out := localaitools.SchedulingConfigFromNode(*config)
+	return &out, nil
+}
+
+func (c *Client) SetScheduling(ctx context.Context, req localaitools.SetSchedulingRequest) (*localaitools.ModelSchedulingConfig, error) {
+	if req.ModelName == "" {
+		return nil, errors.New("model_name is required")
+	}
+	if c.NodeRegistry == nil {
+		return nil, errors.New("model scheduling is only available in distributed mode")
+	}
+
+	existing, err := c.NodeRegistry.GetModelScheduling(ctx, req.ModelName)
+	if err != nil {
+		return nil, fmt.Errorf("load existing scheduling config: %w", err)
+	}
+	routePolicy := ""
+	absThr := 0
+	relThr := 0.0
+	minMatch := 0.0
+	if existing != nil {
+		routePolicy = existing.RoutePolicy
+		absThr = existing.BalanceAbsThreshold
+		relThr = existing.BalanceRelThreshold
+		minMatch = existing.MinPrefixMatch
+	}
+	if req.RoutePolicy != nil {
+		routePolicy = *req.RoutePolicy
+	}
+	if req.BalanceAbsThreshold != nil {
+		absThr = *req.BalanceAbsThreshold
+	}
+	if req.BalanceRelThreshold != nil {
+		relThr = *req.BalanceRelThreshold
+	}
+	if req.MinPrefixMatch != nil {
+		minMatch = *req.MinPrefixMatch
+	}
+	if req.SpreadAll && (req.MinReplicas != 0 || req.MaxReplicas != 0) {
+		return nil, errors.New("spread_all and min_replicas/max_replicas are mutually exclusive")
+	}
+	if req.MinReplicas < 0 {
+		return nil, errors.New("min_replicas must be >= 0")
+	}
+	if req.MaxReplicas < 0 {
+		return nil, errors.New("max_replicas must be >= 0")
+	}
+	if req.MaxReplicas > 0 && req.MinReplicas > req.MaxReplicas {
+		return nil, errors.New("min_replicas must be <= max_replicas")
+	}
+	if err := prefixcache.ValidateThresholds(routePolicy, absThr, relThr, minMatch); err != nil {
+		return nil, err
+	}
+
+	var selectorJSON string
+	if len(req.NodeSelector) > 0 {
+		b, err := json.Marshal(req.NodeSelector)
+		if err != nil {
+			return nil, fmt.Errorf("invalid node_selector: %w", err)
+		}
+		selectorJSON = string(b)
+	}
+	config := &nodes.ModelSchedulingConfig{
+		ModelName:           req.ModelName,
+		NodeSelector:        selectorJSON,
+		MinReplicas:         req.MinReplicas,
+		MaxReplicas:         req.MaxReplicas,
+		SpreadAll:           req.SpreadAll,
+		RoutePolicy:         routePolicy,
+		BalanceAbsThreshold: absThr,
+		BalanceRelThreshold: relThr,
+		MinPrefixMatch:      minMatch,
+	}
+	if err := c.NodeRegistry.SetModelScheduling(ctx, config); err != nil {
+		return nil, err
+	}
+	out := localaitools.SchedulingConfigFromNode(*config)
+	return &out, nil
+}
+
+func (c *Client) DeleteScheduling(ctx context.Context, modelName string) error {
+	if modelName == "" {
+		return errors.New("model_name is required")
+	}
+	if c.NodeRegistry == nil {
+		return errors.New("model scheduling is only available in distributed mode")
+	}
+	return c.NodeRegistry.DeleteModelScheduling(ctx, modelName)
+}
+
 func (c *Client) SetNodeVRAMBudget(_ context.Context, _, _ string) error {
 	// The node registry is a distributed-mode concern owned by the Application
 	// layer and is not wired into the in-process client (which also returns an

pkg/mcp/localaitools/inproc/client_test.go

@@ -2,6 +2,7 @@ package inproc
 
 import (
 	"context"
+	"encoding/json"
 	"errors"
 	"os"
 	"path/filepath"
@@ -13,8 +14,11 @@ import (
 	"github.com/mudler/LocalAI/core/config"
 	"github.com/mudler/LocalAI/core/gallery"
 	"github.com/mudler/LocalAI/core/services/galleryop"
+	"github.com/mudler/LocalAI/core/services/nodes"
 	localaitools "github.com/mudler/LocalAI/pkg/mcp/localaitools"
 	"github.com/mudler/LocalAI/pkg/system"
+	"gorm.io/driver/sqlite"
+	"gorm.io/gorm"
 )
 
 // Regression spec for the bug we fixed when channel sends were
@@ -124,3 +128,88 @@ var _ = Describe("inproc.Client model aliases", func() {
 		})
 	})
 })
+
+var _ = Describe("inproc.Client model scheduling", func() {
+	var (
+		ctx      context.Context
+		registry *nodes.NodeRegistry
+		c        *Client
+		stringp  = func(s string) *string { return &s }
+		floatp   = func(f float64) *float64 { return &f }
+	)
+
+	BeforeEach(func() {
+		ctx = context.Background()
+		db, err := gorm.Open(sqlite.Open(filepath.Join(GinkgoT().TempDir(), "nodes.db")), &gorm.Config{})
+		Expect(err).ToNot(HaveOccurred())
+		registry, err = nodes.NewNodeRegistry(db)
+		Expect(err).ToNot(HaveOccurred())
+
+		tempDir := GinkgoT().TempDir()
+		systemState, err := system.GetSystemState(system.WithModelPath(tempDir))
+		Expect(err).ToNot(HaveOccurred())
+		appConfig := config.NewApplicationConfig(config.WithSystemState(systemState))
+		c = New(appConfig, systemState, config.NewModelConfigLoader(tempDir), nil, nil, registry)
+	})
+
+	It("sets, lists, gets, merges, and deletes scheduling configs through the registry", func() {
+		created, err := c.SetScheduling(ctx, localaitools.SetSchedulingRequest{
+			ModelName:      "qwen",
+			NodeSelector:   map[string]string{"gpu": "nvidia"},
+			MinReplicas:    1,
+			MaxReplicas:    2,
+			RoutePolicy:    stringp("prefix_cache"),
+			MinPrefixMatch: floatp(0.4),
+		})
+		Expect(err).ToNot(HaveOccurred())
+		Expect(created.ModelName).To(Equal("qwen"))
+		Expect(created.NodeSelector).To(Equal(`{"gpu":"nvidia"}`))
+		Expect(created.RoutePolicy).To(Equal("prefix_cache"))
+		Expect(created.MinPrefixMatch).To(Equal(0.4))
+		Expect(schedulingJSONKeys(created)).ToNot(Or(
+			HaveKey("id"),
+			HaveKey("unsatisfiable_until"),
+			HaveKey("unsatisfiable_ticks"),
+			HaveKey("created_at"),
+			HaveKey("updated_at"),
+		))
+
+		listed, err := c.ListScheduling(ctx)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(listed).To(HaveLen(1))
+		Expect(listed[0].ModelName).To(Equal("qwen"))
+
+		updated, err := c.SetScheduling(ctx, localaitools.SetSchedulingRequest{
+			ModelName:   "qwen",
+			MinReplicas: 2,
+			MaxReplicas: 3,
+		})
+		Expect(err).ToNot(HaveOccurred())
+		Expect(updated.MinReplicas).To(Equal(2))
+		Expect(updated.MaxReplicas).To(Equal(3))
+		Expect(updated.RoutePolicy).To(Equal("prefix_cache"))
+		Expect(updated.MinPrefixMatch).To(Equal(0.4))
+
+		got, err := c.GetScheduling(ctx, "qwen")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(got).ToNot(BeNil())
+		Expect(got.MaxReplicas).To(Equal(3))
+
+		Expect(c.DeleteScheduling(ctx, "qwen")).To(Succeed())
+		missing, err := c.GetScheduling(ctx, "qwen")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(missing).To(BeNil())
+	})
+})
+
+func schedulingJSONKeys(config *localaitools.ModelSchedulingConfig) map[string]any {
+	var out map[string]any
+	Expect(json.Unmarshal([]byte(mustMarshal(config)), &out)).To(Succeed())
+	return out
+}
+
+func mustMarshal(v any) string {
+	b, err := json.Marshal(v)
+	Expect(err).ToNot(HaveOccurred())
+	return string(b)
+}

pkg/mcp/localaitools/prompts/10_safety.md

@@ -2,7 +2,7 @@
 
 These rules are non-negotiable. The user trusts you to operate their server without unintended changes.
 
-1. **Confirm before mutating.** Before calling any of these tools — `install_model`, `import_model_uri`, `delete_model`, `install_backend`, `upgrade_backend`, `edit_model_config`, `reload_models`, `load_model`, `toggle_model_state`, `toggle_model_pinned`, `create_voice_profile`, `delete_voice_profile` — first state in plain language what you are about to do (which tool, which target, which arguments) and wait for the user's explicit confirmation in the next turn. "Yes", "do it", "go ahead", "proceed" all count as confirmation. Anything else does not.
+1. **Confirm before mutating.** Before calling any of these tools — `install_model`, `import_model_uri`, `delete_model`, `install_backend`, `upgrade_backend`, `edit_model_config`, `reload_models`, `load_model`, `toggle_model_state`, `toggle_model_pinned`, `create_voice_profile`, `delete_voice_profile`, `set_node_vram_budget`, `set_scheduling`, `delete_scheduling` — first state in plain language what you are about to do (which tool, which target, which arguments) and wait for the user's explicit confirmation in the next turn. "Yes", "do it", "go ahead", "proceed" all count as confirmation. Anything else does not.
 
 2. **Disambiguate before mutating.** If the user's request is ambiguous (several gallery candidates match, the model name has multiple installed versions, the backend has variants), present the candidates as a numbered list and ask the user to pick before calling any mutating tool.
 

pkg/mcp/localaitools/prompts/20_tools.md

@@ -14,6 +14,8 @@ The MCP `tools/list` endpoint also exposes the full input schema for each of the
 - `vram_estimate` — Estimate VRAM use for a model under a given config.
 - `system_info` — LocalAI version, paths, distributed flag, loaded models, installed backends.
 - `list_nodes` — List federated worker nodes (only useful in distributed mode).
+- `list_scheduling` — List distributed per-model scheduling configs.
+- `get_scheduling` — Read the distributed scheduling config for one model.
 - `list_voice_profiles` — List reusable voice-cloning profiles and their stable TTS voice URIs.
 
 ## Mutating (require user confirmation per safety rule 1)
@@ -30,3 +32,6 @@ The MCP `tools/list` endpoint also exposes the full input schema for each of the
 - `toggle_model_pinned` — Pin or unpin a model (`action`: `pin` or `unpin`).
 - `create_voice_profile` — Save a consent-confirmed base64 PCM-WAV reference and exact transcript for reuse in TTS.
 - `delete_voice_profile` — Permanently delete a saved voice profile by UUID.
+- `set_node_vram_budget` — Set or clear a federated node's VRAM budget override.
+- `set_scheduling` — Create or update a distributed per-model scheduling config.
+- `delete_scheduling` — Remove a distributed per-model scheduling config.

pkg/mcp/localaitools/prompts/skills/manage_distributed_scheduling.md

@@ -0,0 +1,9 @@
+# Skill: Manage distributed scheduling
+
+Use this when the user asks to inspect, set, or remove distributed per-model scheduling rules.
+
+1. Call `system_info` first. If `distributed` is false, explain that scheduling tools only affect distributed deployments.
+2. For inspection, call `list_scheduling` or `get_scheduling` and summarize model name, selector, replica bounds, spread-all, and route policy fields.
+3. Before calling `set_scheduling` or `delete_scheduling`, follow safety rule 1 and wait for explicit confirmation.
+4. For `set_scheduling`, never combine `spread_all: true` with non-zero `min_replicas` or `max_replicas`.
+5. After a confirmed mutation, call `get_scheduling` for that model and summarize the persisted config.

pkg/mcp/localaitools/prompts/skills/system_status.md

@@ -4,11 +4,12 @@ Use this when the user asks "what's installed?", "what's running?", "show status
 
 1. Call `system_info` for version, paths, distributed flag, loaded models, installed backends.
 2. Call `list_installed_models` (no capability filter) for the full installed-model inventory.
-3. If `system_info.distributed` is true, also call `list_nodes` and report worker health.
+3. If `system_info.distributed` is true, also call `list_nodes` and `list_scheduling`, then report worker health and any per-model scheduling rules.
 4. Present a concise summary:
    - **Version & mode** (`distributed: true|false`)
    - **Installed models** (count + list, each with name and capabilities)
    - **Installed backends** (count + list)
    - **Loaded right now** (from `loaded_models`)
    - **Workers** (only when distributed)
+   - **Scheduling rules** (only when distributed)
 5. Do not call mutating tools in this skill.

pkg/mcp/localaitools/scheduling.go

@@ -0,0 +1,25 @@
+package localaitools
+
+import "github.com/mudler/LocalAI/core/services/nodes"
+
+func SchedulingConfigFromNode(config nodes.ModelSchedulingConfig) ModelSchedulingConfig {
+	return ModelSchedulingConfig{
+		ModelName:           config.ModelName,
+		NodeSelector:        config.NodeSelector,
+		MinReplicas:         config.MinReplicas,
+		MaxReplicas:         config.MaxReplicas,
+		SpreadAll:           config.SpreadAll,
+		RoutePolicy:         config.RoutePolicy,
+		BalanceAbsThreshold: config.BalanceAbsThreshold,
+		BalanceRelThreshold: config.BalanceRelThreshold,
+		MinPrefixMatch:      config.MinPrefixMatch,
+	}
+}
+
+func SchedulingConfigsFromNodes(configs []nodes.ModelSchedulingConfig) []ModelSchedulingConfig {
+	out := make([]ModelSchedulingConfig, 0, len(configs))
+	for _, config := range configs {
+		out = append(out, SchedulingConfigFromNode(config))
+	}
+	return out
+}

pkg/mcp/localaitools/server.go

@@ -47,6 +47,7 @@ func NewServer(client LocalAIClient, opts Options) *mcp.Server {
 	registerBackendTools(srv, client, opts)
 	registerConfigTools(srv, client, opts)
 	registerSystemTools(srv, client, opts)
+	registerSchedulingTools(srv, client, opts)
 	registerStateTools(srv, client, opts)
 	registerBrandingTools(srv, client, opts)
 	registerVoiceProfileTools(srv, client, opts)

pkg/mcp/localaitools/server_test.go

@@ -92,6 +92,7 @@ var expectedFullCatalog = sortedStrings(
 	ToolListInstalledModels,
 	ToolListKnownBackends,
 	ToolListNodes,
+	ToolListScheduling,
 	ToolListVoiceProfiles,
 	ToolLoadModel,
 	ToolReloadModels,
@@ -105,6 +106,9 @@ var expectedFullCatalog = sortedStrings(
 	ToolCreateVoiceProfile,
 	ToolDeleteVoiceProfile,
 	ToolSetNodeVRAMBudget,
+	ToolSetScheduling,
+	ToolDeleteScheduling,
+	ToolGetScheduling,
 )
 
 // expectedReadOnlyCatalog is the tool set when DisableMutating=true. Sorted.
@@ -123,6 +127,8 @@ var expectedReadOnlyCatalog = sortedStrings(
 	ToolListInstalledModels,
 	ToolListKnownBackends,
 	ToolListNodes,
+	ToolListScheduling,
+	ToolGetScheduling,
 	ToolListVoiceProfiles,
 	ToolSystemInfo,
 	ToolVRAMEstimate,
@@ -165,6 +171,10 @@ var _ = Describe("Tool dispatch", func() {
 		{ToolListKnownBackends, struct{}{}, "ListKnownBackends"},
 		{ToolSystemInfo, struct{}{}, "SystemInfo"},
 		{ToolListNodes, struct{}{}, "ListNodes"},
+		{ToolListScheduling, struct{}{}, "ListScheduling"},
+		{ToolGetScheduling, DeleteSchedulingRequest{ModelName: "qwen"}, "GetScheduling"},
+		{ToolSetScheduling, SetSchedulingRequest{ModelName: "qwen", MinReplicas: 1, MaxReplicas: 2}, "SetScheduling"},
+		{ToolDeleteScheduling, DeleteSchedulingRequest{ModelName: "qwen"}, "DeleteScheduling"},
 		{ToolListVoiceProfiles, struct{}{}, "ListVoiceProfiles"},
 		{ToolInstallModel, InstallModelRequest{ModelName: "test/foo"}, "InstallModel"},
 		{ToolImportModelURI, ImportModelURIRequest{URI: "Qwen/Qwen3-4B-GGUF"}, "ImportModelURI"},
@@ -219,6 +229,40 @@ var _ = Describe("Tool error surfacing", func() {
 	})
 })
 
+var _ = Describe("Scheduling tool behavior", func() {
+	It("round-trips list/get/set/delete over the MCP surface", func() {
+		fc := &fakeClient{
+			listScheduling: func() ([]ModelSchedulingConfig, error) {
+				return []ModelSchedulingConfig{{ModelName: "qwen", MinReplicas: 1, MaxReplicas: 2}}, nil
+			},
+			getScheduling: func(modelName string) (*ModelSchedulingConfig, error) {
+				return &ModelSchedulingConfig{ModelName: modelName, SpreadAll: true}, nil
+			},
+			setScheduling: func(req SetSchedulingRequest) (*ModelSchedulingConfig, error) {
+				return &ModelSchedulingConfig{ModelName: req.ModelName, MinReplicas: req.MinReplicas, MaxReplicas: req.MaxReplicas}, nil
+			},
+		}
+		ctx, sess, done := connectInMemory(fc, Options{})
+		DeferCleanup(done)
+
+		listRes := callTool(ctx, sess, ToolListScheduling, struct{}{})
+		Expect(listRes.IsError).To(BeFalse(), resultText(listRes))
+		Expect(resultText(listRes)).To(ContainSubstring(`"model_name": "qwen"`))
+
+		getRes := callTool(ctx, sess, ToolGetScheduling, DeleteSchedulingRequest{ModelName: "qwen"})
+		Expect(getRes.IsError).To(BeFalse(), resultText(getRes))
+		Expect(resultText(getRes)).To(ContainSubstring(`"spread_all": true`))
+
+		setRes := callTool(ctx, sess, ToolSetScheduling, SetSchedulingRequest{ModelName: "qwen", MinReplicas: 1, MaxReplicas: 2})
+		Expect(setRes.IsError).To(BeFalse(), resultText(setRes))
+		Expect(resultText(setRes)).To(ContainSubstring(`"max_replicas": 2`))
+
+		deleteRes := callTool(ctx, sess, ToolDeleteScheduling, DeleteSchedulingRequest{ModelName: "qwen"})
+		Expect(deleteRes.IsError).To(BeFalse(), resultText(deleteRes))
+		Expect(resultText(deleteRes)).To(ContainSubstring(`"deleted": "qwen"`))
+	})
+})
+
 var _ = Describe("Argument validation", func() {
 	type validationCase struct {
 		desc string
@@ -235,6 +279,9 @@ var _ = Describe("Argument validation", func() {
 		{"toggle_model_state rejects unknown action", ToolToggleModelState, map[string]any{"name": "foo", "action": "noop"}, "action must be one of"},
 		{"edit_model_config rejects empty patch", ToolEditModelConfig, map[string]any{"name": "foo", "patch": map[string]any{}}, "patch is required"},
 		{"create_voice_profile requires consent", ToolCreateVoiceProfile, CreateVoiceProfileRequest{Name: "Voice", Transcript: "words", AudioBase64: "UklGRg=="}, "consent_confirmed must be true"},
+		{"set_scheduling requires model_name", ToolSetScheduling, SetSchedulingRequest{}, "model_name is required"},
+		{"set_scheduling rejects invalid replica range", ToolSetScheduling, SetSchedulingRequest{ModelName: "qwen", MinReplicas: 3, MaxReplicas: 1}, "min_replicas must be <= max_replicas"},
+		{"delete_scheduling requires model_name", ToolDeleteScheduling, DeleteSchedulingRequest{}, "model_name is required"},
 	}
 
 	for _, c := range cases {

pkg/mcp/localaitools/tools.go

@@ -17,6 +17,8 @@ const (
 	ToolListKnownBackends   = "list_known_backends"
 	ToolSystemInfo          = "system_info"
 	ToolListNodes           = "list_nodes"
+	ToolListScheduling      = "list_scheduling"
+	ToolGetScheduling       = "get_scheduling"
 	ToolVRAMEstimate        = "vram_estimate"
 	ToolGetBranding         = "get_branding"
 	ToolGetUsageStats       = "get_usage_stats"
@@ -42,6 +44,8 @@ const (
 	ToolCreateVoiceProfile = "create_voice_profile"
 	ToolDeleteVoiceProfile = "delete_voice_profile"
 	ToolSetNodeVRAMBudget  = "set_node_vram_budget"
+	ToolSetScheduling      = "set_scheduling"
+	ToolDeleteScheduling   = "delete_scheduling"
 
 	// ToolListAliases is read-only but lives here so the alias tools stay
 	// grouped; the catalog tests assert its read-only placement.

pkg/mcp/localaitools/tools_scheduling.go

@@ -0,0 +1,77 @@
+package localaitools
+
+import (
+	"context"
+
+	"github.com/modelcontextprotocol/go-sdk/mcp"
+)
+
+func registerSchedulingTools(s *mcp.Server, client LocalAIClient, opts Options) {
+	mcp.AddTool(s, &mcp.Tool{
+		Name:        ToolListScheduling,
+		Description: "List distributed per-model scheduling configs (only meaningful in distributed mode).",
+	}, func(ctx context.Context, _ *mcp.CallToolRequest, _ struct{}) (*mcp.CallToolResult, any, error) {
+		configs, err := client.ListScheduling(ctx)
+		if err != nil {
+			return errorResult(err), nil, nil
+		}
+		return jsonResult(configs), nil, nil
+	})
+
+	mcp.AddTool(s, &mcp.Tool{
+		Name:        ToolGetScheduling,
+		Description: "Get the distributed scheduling config for one model, or null when none is configured.",
+	}, func(ctx context.Context, _ *mcp.CallToolRequest, args DeleteSchedulingRequest) (*mcp.CallToolResult, any, error) {
+		if args.ModelName == "" {
+			return errorResultf("model_name is required"), nil, nil
+		}
+		config, err := client.GetScheduling(ctx, args.ModelName)
+		if err != nil {
+			return errorResult(err), nil, nil
+		}
+		return jsonResult(config), nil, nil
+	})
+
+	if opts.DisableMutating {
+		return
+	}
+
+	mcp.AddTool(s, &mcp.Tool{
+		Name:        ToolSetScheduling,
+		Description: "Create or update a distributed per-model scheduling config. Requires user confirmation per safety rule 1.",
+	}, func(ctx context.Context, _ *mcp.CallToolRequest, args SetSchedulingRequest) (*mcp.CallToolResult, any, error) {
+		if args.ModelName == "" {
+			return errorResultf("model_name is required"), nil, nil
+		}
+		if args.SpreadAll && (args.MinReplicas != 0 || args.MaxReplicas != 0) {
+			return errorResultf("spread_all and min_replicas/max_replicas are mutually exclusive"), nil, nil
+		}
+		if args.MinReplicas < 0 {
+			return errorResultf("min_replicas must be >= 0"), nil, nil
+		}
+		if args.MaxReplicas < 0 {
+			return errorResultf("max_replicas must be >= 0"), nil, nil
+		}
+		if args.MaxReplicas > 0 && args.MinReplicas > args.MaxReplicas {
+			return errorResultf("min_replicas must be <= max_replicas"), nil, nil
+		}
+		config, err := client.SetScheduling(ctx, args)
+		if err != nil {
+			return errorResult(err), nil, nil
+		}
+		return jsonResult(config), nil, nil
+	})
+
+	mcp.AddTool(s, &mcp.Tool{
+		Name:        ToolDeleteScheduling,
+		Description: "Delete a distributed per-model scheduling config. Requires user confirmation per safety rule 1.",
+	}, func(ctx context.Context, _ *mcp.CallToolRequest, args DeleteSchedulingRequest) (*mcp.CallToolResult, any, error) {
+		if args.ModelName == "" {
+			return errorResultf("model_name is required"), nil, nil
+		}
+		if err := client.DeleteScheduling(ctx, args.ModelName); err != nil {
+			return errorResult(err), nil, nil
+		}
+		return jsonResult(map[string]string{"deleted": args.ModelName}), nil, nil
+	})
+}