The PII middleware scans the text of every message, but images, audio,
video and files pass to the backend unchecked. For an egress policy
("nothing sensitive leaves the box") that is a hole: a photographed
invoice or an attached PDF reaches a cloud-proxy model without any check.
Add pii.block_unscanned_media. While PII is enabled for a model and the
option is set, a request that carries such content is rejected with the
usual 400 pii_blocked response and an audit event with pattern
__unscanned_media__, before any detector runs. Adapters report the
kinds of unscanned content through a new optional Adapter.Unscanned:
OpenAI chat (non-text content blocks), Anthropic messages (image and
document blocks; tool blocks stay allowed) and Ollama chat (images).
Off by default, so existing configurations behave as before.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
Two gaps in the chat composer:
- The microphone button only opened a file dialog for audio files; there
was no way to speak a message. Add a dictation button: the first click
records from the microphone, the second stops and sends the recording
to /v1/audio/transcriptions with the first installed transcription
model; the text is appended to the message box for review. The button
is hidden without a transcription model or without a microphone API.
The audio attach button keeps its function and gets the music icon.
- A PDF without a text layer (a scan) was rejected with a toast. Render
its first ten pages to PNG with pdf.js and attach them as images, so a
vision model can read them; a toast says how many pages went along.
PDFs that cannot be parsed at all are still rejected.
Strings for all eight locales, docs in features/chat.md, Playwright
specs for both (with negative cases: no transcription model, failed
transcription, unparsable PDF).
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
stable-diffusion.cpp PR #2148 added MOSS-Image support, which needs an
external tokenizer file passed via the --tokenizer CLI arg. The C++ binding
(gosd.cpp) never parsed a tokenizer option nor set ctx_params.tokenizer, so
MOSS-Image could not load through LocalAI.
- Wire the "tokenizer" option in gosd.cpp (parse + assign to ctx_params)
- Resolve "tokenizer:..." paths against the model directory in gosd.go,
the same way *_path options are resolved
- Bump the vendored stable-diffusion.cpp to 68285df (moss-image merge)
- Add moss-image-base and moss-image-sft gallery entries (int8_convrot)
FLUX.3 Image Klein is already in the gallery (added in #12736).
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* chore(model gallery): 🤖 add new models via gallery agent
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(gallery): give the Q6 build a unique name
The Q8 entry already uses embeddinggemma-2. Include the quantization
in the new entry name so both builds remain reachable.
Assisted-by: Codex:gpt-6
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(gallery): configure EmbeddingGemma 2 variants
Serve the Unsloth builds as text embeddings and remove chat settings.\nGroup Q4, Q5, and Q6 with the existing family and pin verified assets.
Assisted-by: Codex:gpt-6
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* feat(llama-cpp): add lazy_mode model option
Expose upstream --lazy-mode (on, auto, off) as a model option, for
example `lazy_mode:on`. It controls on-demand reading of large tensors
such as per-layer embeddings; `on` requires mmap.
The option is also accepted as `lazy-mode` or `lzm`. Invalid values log
a warning and keep the upstream default (auto). The pinned llama.cpp
already provides the lazy_mode field in common_params.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* chore(llama-cpp): bump llama.cpp to c0fd78626405
Update the default llama.cpp pin to the current upstream master. The
changes since the old pin that the backend touches are additive: a new
common_decision_type value, new mtmd audio-stream and download helper
declarations. The two server patches still apply in order, and the
common_params fields and server headers the backend uses are unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* feat(llama-cpp): add slot prompt-cache save and restore API
Add POST /v1/slots/save and POST /v1/slots/restore. They write one
llama.cpp slot's prompt cache (KV state) to a file and load it back,
so a client can warm a long system prompt in a few hundred
milliseconds instead of re-running prefill. Refs #12685.
The new SlotCache gRPC call posts the same slot save and restore
tasks the llama-server /slots endpoint uses. It is off unless the
model sets the new slot_save_path option, so existing models are not
affected. The /slots monitoring endpoint is not needed here: the
backend has no HTTP server.
The file name comes from the caller, so it is checked twice, in Go
and again in the backend. It must be one path component: no empty
name, "/", "\", "..", NUL, control characters or absolute path. The
resolved path, with symbolic links followed, must be a direct child
of slot_save_path. The routes are admin only because they write
files on the backend host.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(llama-cpp): answer 501 when a backend lacks slot cache
A backend that does not implement the SlotCache rpc returns gRPC
UNIMPLEMENTED. The API passed that text through, so the client saw
"unknown method SlotCache". Map the status to a typed error in
core/backend and return 501 with a fixed message instead. Every other
error path is unchanged.
Make the SlotCache comments in backend.proto backend-neutral: slot is
the backend's state slot, filename must stay inside the save directory,
and an empty model_identity means the backend does not check it. Only
comments change, no field or name.
Add tests that the identity sent is the configured model, and for the
error mapping.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(llama-cpp): issue slot cache IDs on the server
The slot save and restore API let the client pick the file name.
Restore could load any file in slot_save_path, and the symlink check
ran before llama.cpp opened the file.
The backend now issues the name. A save returns a random 128-bit ID
(32 lowercase hex characters, read from /dev/urandom) and never
overwrites an existing file. Files live in a 0700 directory per model,
named from the SHA-256 of the model identity, with a manifest.json
that records each save. The manifest is written atomically and a
corrupt one stops both save and restore.
A restore takes the ID, finds it in the manifest, refuses an entry of
another model, builds the path from the entry and checks with lstat
that it is a plain file of the recorded size. The directory mode is
the main defence against another local user, since llama.cpp still
opens the path itself.
Each model keeps at most 32 saves and the oldest is deleted first. A
file over 8 GiB is deleted after the save, because llama.cpp does not
report the size before it writes. Responses carry no host path.
The HTTP bodies change: save takes {model, slot}, restore takes
{model, slot, id}, and an unknown ID returns 404. In the proto,
SlotCacheRequest.filename and SlotCacheResponse.filename become id.
SlotCache no longer calls checkModelIdentity, which reads a field this
request does not have.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(llama-cpp): copy slot_cache files in prepare.sh
grpc-server.cpp includes slot_cache.h, but prepare.sh copies the helper
headers into llama.cpp/tools/grpc-server/ one by one and skipped it. The
llama-cpp, turboquant and bonsai builds share this step, so they all
failed with "slot_cache.h: No such file or directory".
Copy slot_cache.h and slot_cache_test.cpp, which CMakeLists.txt builds
when LLAMA_GRPC_BUILD_TESTS is on.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* chore(deps): bump LocalAGI to pick up the scheduler start/stop fix
LocalAGI's scheduler could hit a nil pointer in run when Start and Stop
raced. That made the core/services/agentpool tests panic at random and
turned tests-linux red.
Move to mudler/LocalAGI@7aa0c8fea0, the merge of
mudler/LocalAGI#519 ("make Start and Stop safe to call concurrently").
The new module version also adds a few indirect requirements to go.mod
and go.sum.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
Add official Q4, Q5, Q6, and Q8 GGUF builds with the F16 vision
projector. Group them for automatic selection and document installation.
Pin downloads to one Hugging Face revision with verified SHA256 values.
Assisted-by: Codex:gpt-6
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
fix(ui): serve canvas HTML previews from a route with their own CSP
External scripts in canvas previews (for example three.js from unpkg)
were blocked. The preview used an iframe with srcdoc, which inherits the
global Content-Security-Policy, and a meta tag can only tighten it.
The UI now posts the HTML to POST /api/canvas/preview, gets a random id,
and loads GET /api/canvas/preview/:id in the iframe. That response sets
its own CSP that allows scripts from unpkg, jsdelivr, cdnjs, the
Tailwind CDN and jQuery. The global CSP is unchanged. The iframe keeps
sandbox="allow-scripts" without allow-same-origin.
Previews are kept in memory for 10 minutes and are never written to
disk. The GET route is public because the id is the capability and a
frame navigation cannot send API key headers.
Assisted-by: Claude:claude-haiku-5-5 [go test] [npm]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* chore(vllm-cpp): bump MLX pin to 0.32.4
The macOS runner's pip index no longer offers mlx==0.29.4 (it lists only
0.32.4), so the darwin vllm-cpp build fails at the MLX venv step. Move the
pin to 0.32.4 and update the README default to match.
Not verified on macOS: the Makefile was only dry-run on Linux.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(vllm-cpp): pin MLX 0.32.0, the version the engine builds against
The macOS build fails against mlx 0.32.4:
metal_mlx_provider.mm:147:5: error: no matching member function
for call to 'set_data'
The vllm.cpp Metal provider calls the five-argument
array::set_data(Buffer, size, strides, flags, Deleter). The 0.32.4
headers only declare set_data(allocator::Data) and the Data overload
with size, strides, flags and offset; 0.32.0 still has the Buffer one.
The engine's own CI installs 'mlx==0.32.0' (ci.yml line 1798,
release.yml line 360), so pin that and update the README default.
Not verified on macOS: only the Makefile dry-run and the wheel
headers were checked on Linux.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(vllm-cpp): install the MLX 0.32.0 wheel by URL and hash
The macOS runner's pip index offers only mlx 0.32.4, so
'pip install mlx==0.32.0' fails with "Could not find a version that
satisfies the requirement". 0.32.4 cannot be used: its headers lack the
five-argument array::set_data the engine calls.
Install the 0.32.0 wheel directly by URL with a sha256 check. The wheel
is mlx-metal (py3-none-macosx_14_0_arm64) from PyPI, sha256
5b64b20ac24b0c401f489de01e8209edc4d372125201f19314e6f39e385322aa. It
holds mlx/include, mlx/lib/libmlx.dylib and mlx/lib/mlx.metallib, which
is all the build and package.sh use, and it is not tied to a CPython
version. The URL and hash are Makefile variables; the stamp stays keyed
on MLX_VERSION.
Not verified on macOS: only the Makefile dry-run, the wheel contents and
its sha256 were checked on Linux.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Add the distilled and base FLUX.3 Image Klein models for the
stablediffusion-ggml backend, each in Q4_K_M and Q8_0. They use the
Qwen3-VL 4B text encoder and the FLUX.2 VAE.
Klein runs with 16 steps and CFG 1.0, Klein Base with 20 steps and
CFG 3.5, both with the euler sampler. The sd.cpp pin on master already
contains FLUX.3 support.
Assisted-by: Claude Code:claude-sonnet-5-5
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
A decision request (/v1/systemone, /v1/decisions) sends Score with
question_type "systemone". A backend built before native decisions ignores
that field and runs plain scoring, so the caller saw "Score was not enabled"
(500) or "candidates must be non-empty" (400). The gallery "latest" image can
lag behind the server, and the upgrade check compares only tag, version and
digest, so /backends/upgrades did not list the backend.
Recognise those two errors for a decision request, return 501 with a message
that says the backend is older than the server and must be upgraded, and mark
the backend stale. The upgrade check now reports a marked backend with a
`reason`, and refreshes at once. Reinstalling the backend drops the mark; if
the new image is still old, the next request marks it again. Model loading is
not affected.
Refreshing the "latest" backend image is a separate publishing step and is
not part of this change.
Refs #12693
Assisted-by: Claude Code:claude-sonnet-5-5 [go vet] [go test]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(audio): use captions as the default YuE2 style
Studio sends a caption, but YuE2 requires a style request option.
Default audio-cpp style from caption while preserving explicit options
and the caller's parameter map. Document the Studio fields.
Assisted-by: Codex:gpt-6
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(audio-cpp): map captions to style in backend
Move the YuE2 caption fallback into the C++ request builder as requested.
Preserve explicit style options, including empty values, and move the
regression cases from the Go layer into the backend suite.
Assisted-by: Codex:GPT-6
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Bump LLAMA_VERSION to b76b13379d3b (upstream 0.7.0 plus later
commits). This is the same one-line change as the bot PR #12725.
Add the moe_cache_mib model option. It sets upstream's
moe_cache_size, a GPU cache for MoE experts kept in host memory,
from a size in MiB. It is off by default. Invalid, negative or
non-numeric values are ignored with a log line. Forks that predate
the field ignore the option, using the same prepare.sh probe as the
other fork-skew switches.
Document the option, and cover the MiB to bytes conversion with a
standalone unit test.
Assisted-by: Claude:claude-sonnet-5-5 g++
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
fix(ui): fold the "No reasoning" chip into the effort picker
The chat composer had two controls for one idea: the effort picker
with a "None" level, and a separate "No reasoning" chip. The chip
also hid reasoning traces that a model sends despite
reasoning_effort "none", which "None" did not. The chip disabled the
picker while it was on.
Choosing "None" now hides those traces and stays selectable, so the
chip, its toasts and the /reasoning slash action are removed. A chat
saved with the old reasoningOff flag loads with effort "none", so it
keeps hiding reasoning.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* ⬆️ Update ggml-org/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(llama-cpp): read the rendered chat prompt through str()
llama.cpp now returns the chat prompt as a common_chat_prompt instead of
a std::string. Convert it with a small helper that still accepts the plain
string used by older forks that share this source.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The chat could not set reasoning_effort or turn reasoning traces off.
Add an effort menu and a "No reasoning" toggle to the composer chips,
and the matching /effort and /reasoning slash actions. Both are stored
per chat. The effort menu defaults to the model default, which sends no
reasoning_effort, so existing chats behave as before.
Chosen effort goes out as the per-request reasoning_effort field. The
toggle sends "none", which the server maps to enable_thinking=false, and
also drops any reasoning the model still returns, so the thread shows
only the answer.
Assisted-by: Claude:claude-sonnet-5-5 eslint playwright
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(http): stop listing the models directory on embeddings and bearer requests
Every embeddings request read the whole models directory before the body
was parsed, even when the body named its model. On a network-backed
models directory this cost seconds per request.
Embeddings now binds the body first and looks up a default model only
when the body has none, like chat. The pre-bind default lookup and the
gpt-4o constant default are gone from that route.
The first-available default lookup skips the directory when the filter
cannot accept an unconfigured file. Every usecase filter rejects files
without a config, so the scan never added results for them. Match-all,
nil and name filters keep the scan.
The bearer token check is now an exact test: a configured name, or one
stat of that name in the models directory. The value is a literal, not a
regular expression, and a name that is not a local relative path never
matches a file.
Assisted-by: Claude Code:claude-haiku-5-5
* perf(config): cache GGUF header reads for model defaults
Model defaults re-parsed the GGUF header on every SetDefaults call, which
runs per request and at startup reload. Put a bounded LRU in front of the
parser, keyed by absolute path and validated by size and modification
time. The cached value is a snapshot of the fields the defaults use, not
the parser output.
A changed size or mtime is a miss. Missing files and parse errors are not
cached, so the existing fallback is unchanged. LOCALAI_DISABLE_GUESSING,
the empty-path guard, the alias early return, the panic recovery and the
context-size default are all kept in front of the cache.
Assisted-by: Claude:claude-haiku-5-5
* test(http): cover listing-free default lookup on every usecase route
Assisted-by: Claude Code:claude-haiku-5-5
* feat(config): persist the GGUF header cache per instance
The header cache kept parsed GGUF results only in memory, so every restart
parsed each model file again before its defaults were known. Write the cache
to a per-instance file in the data path and load it at startup. A restarted
frontend with a fixed instance ID starts warm.
The file is versioned and records the estimator inputs (gguf-parser version,
candidate contexts, auto context size, offload count). A mismatch on any of
them makes the file ignored. Each entry keeps path, size and mtime, so a
changed model file is a miss on load. Writes run on one background saver,
debounced, through a temp file and rename. The entry bound applies to the
file as well. Without a data path the cache stays in memory only.
Documented as a per-instance exception in the model configuration docs.
Assisted-by: Claude:claude-haiku-5-5
* docs(embeddings): document model selection and the no-model error
Assisted-by: Claude Code:claude-haiku-5-5
* refactor(config): split the header cache into a generic cache
The GGUF header cache mixed two things: the mechanics (bounded LRU keyed
by path and identity, per-instance file, debounced atomic save, tolerant
load) and the GGUF fields. Move the mechanics into headerCache[S], driven
by a headerCacheSpec[S] that names the file prefix, schema, fingerprint,
parser and the snapshot encode/decode pair. The GGUF cache becomes one
instance of it.
File names, the identity check, LRU size, debounce, atomic write and the
corruption handling are unchanged. The on-disk JSON keeps the "estimator"
key so existing files still load.
Assisted-by: Claude Code:claude-haiku-5-5
* feat(config): share one GGUF header cache file between frontends
Every frontend used to write its own header cache file, named after its
instance ID, so a header read by one frontend was never reused by another.
The cache now uses one file, header-cache.json, in the data path, for all
frontends and for standalone mode. The instance ID no longer changes the
file name.
Frontends share the file in three ways. A save first merges the file's
entries in, so one frontend's write does not drop another's entries. A
miss reloads the file when its modification time has changed, and a hit
never reads it. Writes go to a temp file that is renamed into place, so a
reader never sees a partial file.
Two frontends saving at the same moment can still drop each other's last
entries. That costs one re-parse, never a wrong value, because every
entry is checked against its model file before use.
The shared file is valid only when every frontend mounts the same data
volume. The distributed-mode and model-configuration docs state this mode
and its volume limit.
Assisted-by: Claude:claude-haiku-5-5
* feat(config): give an instance ID its own header cache file
Assisted-by: Claude Code:claude-haiku-5-5
* style(config): gofmt the distributed config struct
Assisted-by: Claude Code:claude-haiku-5-5
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Fluid Speech Mini is a compact English dictation model by FluidVoice,
built on NVIDIA Parakeet TDT 0.6B v3 and Moondream Parakeet Ultra. It
matches Parakeet Ultra accuracy (4.53% WER on the Open ASR Leaderboard's
8 public English sets) at about a quarter of the download size (347 MB),
and is trained to keep transcribing the right speaker when another voice
is in the room.
The GGUF is a stock parakeet.cpp TDT checkpoint (Q2_K/Q4_K mixed
precision) that loads in the existing runtime on CPU and CUDA without
engine changes. Verified locally: parakeet-cli loads the model and
transcribes a test clip correctly.
Gallery entry uses the parakeet-cpp backend with the model file sourced
from huggingface://FluidVoice-Labs/fluid-speech-mini-gguf.
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Warn before the standalone pool starts root agents with user auth enabled.
Count root agents, collections, skills, and configured skill repositories.
Document the auth transition, bootstrap rules, and agent key behavior.
Addresses the warning and documentation step of #12665.
Admin visibility and data migration remain separate work.
Assisted-by: Codex:GPT-6
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* feat(stablediffusion-ggml): run MiniMax-H3 video with reference conditioning
Bump stable-diffusion.cpp to 2988060 and follow its tiling rename
(tile_size_x/y -> tile_size_w/h, rel_size_x/y -> rel_size_w/h).
GenerateVideo now passes params["ref_image"] and the request audio to
the MiniMax-H3 Ref2VA inputs (ref_images, ref_audios). The audio must
be a WAV file. params["ref_video"] is refused, since the proto cannot
carry a frame sequence yet.
stable-diffusion.cpp cannot tell the FL2VA and REF2VA DiTs apart, and a
reference input on FL2VA renders a wrong video without an error. The
backend now reads the video_partition option, or the model file name,
and fails early on that mix. Keyframes plus references are refused too.
Add gallery entries for the FL2VA and REF2VA Q4_K_M sets. They do not
enable any bf16 dequantisation.
Assisted-by: Claude:claude-sonnet-5-5 go vet gofmt
* feat(stablediffusion-ggml): take several reference images, videos and audio clips
Phase 1 passed one reference image and one reference audio clip to
stable-diffusion.cpp and refused ref_video. Add repeated reference fields
to GenerateVideoRequest (ref_images, ref_videos, ref_audios) so the
MiniMax-H3 Ref2VA path can use as many references as upstream accepts. A
reference video is a directory of frames with an fps and an optional
paired audio clip.
The /video endpoint stages the new inputs through the same path as
start_image, end_image and audio. Frames of one reference video are
written to a private directory under zero-padded names so the backend can
read them in order. The input count and frame count per request are
bounded.
The backend merges the new fields with params.ref_image, params.ref_video
and audio, which keep working. The FL2VA partition refuses any reference
input, REF2VA accepts all of them, and keyframes stay exclusive with
references. Reference audio is still WAV only.
Assisted-by: Claude:claude-sonnet-5-5 [Bash]
* feat(gallery): offer MiniMax-H3 by partition, DiT layout and format
Name each MiniMax-H3 entry after its partition (fl2va or ref2va), DiT
layout (unpruned or pruned) and format. Add the vllm.cpp builds that its
model page lists: pruned Q8_0 (renders), pruned Q6_K, Q5_0, Q4_K, Q3_K and
Q2_K (load only), and the pruned FL2VA NVFP4 file (loads only). The
earlier names minimax-h3-fl2va-q4 and minimax-h3-ref2va-q4 stay as
aliases. The two stable-diffusion.cpp entries become
minimax-h3-fl2va-unpruned-q4-ggml and minimax-h3-ref2va-pruned-q4-ggml,
and the latter now lists video input.
Fix two bad sources. The stable-diffusion.cpp entries took the VAEs from
a path that does not exist in the GGUF repository; they now use the
Comfy-Org copies. The vllm.cpp entries paired the fp16 video VAE weights
with a config from a different directory; they now use the official
weights and config from FL2VA/video_vae/source/.
Drop video_dequant_bf16 from every vllm.cpp entry. Expanding a quantised
DiT to bf16 is the suspected cause of a host out-of-memory error, so the
quantised weights stay resident. The model page records its renders with
the expansion on, and each description says so.
Assisted-by: Claude:claude-sonnet-5-5 [Bash]
* feat(gallery): add pruned FL2VA Q4 MiniMax-H3 for sd.cpp
Add minimax-h3-fl2va-pruned-q4-ggml, the 11.4 GB pruned FL2VA
counterpart of the existing pruned REF2VA entry.
Assisted-by: Claude Code:claude-sonnet-5-5 [rc]
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Document the failed and cancelled outcomes of a response.create that
waits for the preceding input_audio_buffer.commit.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A client response.create waits for the in-flight commit before the LLM
starts. Two exits left the client without a usable outcome:
- When the wait was cancelled (response.cancel, a newer response or
teardown) the response returned with no event at all.
- When the commit produced no user item (transcription error, empty
transcript, voice gate rejection) the LLM ran on a history that
lacked the turn.
Send response.created and response.done with status cancelled for the
first case. For the second, do not run the LLM: send response.done with
status failed and an input_not_committed error. The commit still
reports its own cause. A response.create with no commit in flight is
unchanged. Add specs for STT error, empty transcript, cancel and
supersede during the wait.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A client response.create waited on a separate mutex-guarded channel
that only the last client commit recorded, so it ignored in-flight VAD
commits. The commit chain already ends in session.commitTail, whose
done channel closes after every earlier commit has finished.
Read the tail under commitOrderMu when the response is issued and wait
on it. This drops the extra mutex, the clientCommitDone field and the
slotSink callback of issueCommit, and the response is now ordered
against both commit producers. Fix the commitSlot.done comment: it
closes when the commit function returns, not when the user item is
appended. Add a spec for commit, commit, response ordering.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
