Bump LLAMA_VERSION to b76b13379d3b (upstream 0.7.0 plus later
commits). This is the same one-line change as the bot PR #12725.
Add the moe_cache_mib model option. It sets upstream's
moe_cache_size, a GPU cache for MoE experts kept in host memory,
from a size in MiB. It is off by default. Invalid, negative or
non-numeric values are ignored with a log line. Forks that predate
the field ignore the option, using the same prepare.sh probe as the
other fork-skew switches.
Document the option, and cover the MiB to bytes conversion with a
standalone unit test.
Assisted-by: Claude:claude-sonnet-5-5 g++
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* ⬆️ Update ggml-org/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(llama-cpp): read the rendered chat prompt through str()
llama.cpp now returns the chat prompt as a common_chat_prompt instead of
a std::string. Convert it with a small helper that still accepts the plain
string used by older forks that share this source.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* feat(stablediffusion-ggml): run MiniMax-H3 video with reference conditioning
Bump stable-diffusion.cpp to 2988060 and follow its tiling rename
(tile_size_x/y -> tile_size_w/h, rel_size_x/y -> rel_size_w/h).
GenerateVideo now passes params["ref_image"] and the request audio to
the MiniMax-H3 Ref2VA inputs (ref_images, ref_audios). The audio must
be a WAV file. params["ref_video"] is refused, since the proto cannot
carry a frame sequence yet.
stable-diffusion.cpp cannot tell the FL2VA and REF2VA DiTs apart, and a
reference input on FL2VA renders a wrong video without an error. The
backend now reads the video_partition option, or the model file name,
and fails early on that mix. Keyframes plus references are refused too.
Add gallery entries for the FL2VA and REF2VA Q4_K_M sets. They do not
enable any bf16 dequantisation.
Assisted-by: Claude:claude-sonnet-5-5 go vet gofmt
* feat(stablediffusion-ggml): take several reference images, videos and audio clips
Phase 1 passed one reference image and one reference audio clip to
stable-diffusion.cpp and refused ref_video. Add repeated reference fields
to GenerateVideoRequest (ref_images, ref_videos, ref_audios) so the
MiniMax-H3 Ref2VA path can use as many references as upstream accepts. A
reference video is a directory of frames with an fps and an optional
paired audio clip.
The /video endpoint stages the new inputs through the same path as
start_image, end_image and audio. Frames of one reference video are
written to a private directory under zero-padded names so the backend can
read them in order. The input count and frame count per request are
bounded.
The backend merges the new fields with params.ref_image, params.ref_video
and audio, which keep working. The FL2VA partition refuses any reference
input, REF2VA accepts all of them, and keyframes stay exclusive with
references. Reference audio is still WAV only.
Assisted-by: Claude:claude-sonnet-5-5 [Bash]
* feat(gallery): offer MiniMax-H3 by partition, DiT layout and format
Name each MiniMax-H3 entry after its partition (fl2va or ref2va), DiT
layout (unpruned or pruned) and format. Add the vllm.cpp builds that its
model page lists: pruned Q8_0 (renders), pruned Q6_K, Q5_0, Q4_K, Q3_K and
Q2_K (load only), and the pruned FL2VA NVFP4 file (loads only). The
earlier names minimax-h3-fl2va-q4 and minimax-h3-ref2va-q4 stay as
aliases. The two stable-diffusion.cpp entries become
minimax-h3-fl2va-unpruned-q4-ggml and minimax-h3-ref2va-pruned-q4-ggml,
and the latter now lists video input.
Fix two bad sources. The stable-diffusion.cpp entries took the VAEs from
a path that does not exist in the GGUF repository; they now use the
Comfy-Org copies. The vllm.cpp entries paired the fp16 video VAE weights
with a config from a different directory; they now use the official
weights and config from FL2VA/video_vae/source/.
Drop video_dequant_bf16 from every vllm.cpp entry. Expanding a quantised
DiT to bf16 is the suspected cause of a host out-of-memory error, so the
quantised weights stay resident. The model page records its renders with
the expansion on, and each description says so.
Assisted-by: Claude:claude-sonnet-5-5 [Bash]
* feat(gallery): add pruned FL2VA Q4 MiniMax-H3 for sd.cpp
Add minimax-h3-fl2va-pruned-q4-ggml, the 11.4 GB pruned FL2VA
counterpart of the existing pruned REF2VA entry.
Assisted-by: Claude Code:claude-sonnet-5-5 [rc]
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Remove a stray blank line in backend.proto and group the imports in
pkg/motion/validation.go. Decide the motion usecase guess from the
backend capability registry instead of a hardcoded backend name, so
variant builds of the backend resolve too. Declare the weights licence
in the gallery entry, as stated on the model card.
Assisted-by: Claude Code:claude-sonnet-5-5
Add the GEM-X native backend with live pose and root displacement output, bounded duplex WebSocket streaming, session authorization, metrics and usage accounting. Share the embedded bidirectional RPC channel adapter across streaming backends.
Include backend build and gallery integration, API documentation, schemas and tests.
Assisted-by: Codex:gpt-6
Signed-off-by: Richard Palethorpe <io@richiejp.com>
(cherry picked from commit 016dc3d71a)
With DOT, EUCLID or MANHATTAN the backend returns Qdrant's raw scores,
not similarity-like values. Router, face and voice stores compare
against cosine thresholds, so log a warning at load when another
metric is configured, and document the behaviour on the stores page.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
StoresSet sent the whole batch in one Upsert under a single
request_timeout_ms, so a large batch could hit the timeout even though
every point was fine. Send at most 500 points per request and apply
the timeout to each request.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A TLS 1.3 floor rejects Qdrant endpoints that only speak TLS 1.2, such
as some managed deployments and proxies. Use TLS 1.2 as the minimum.
When Load fails after the client was created, it closed the client but
left it set, so Free closed it a second time. Clear it after closing.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The sd_embed branch dropped negative_prompt for every pipeline, which
discarded the user's negative prompt for Flux, where no negative embeds
are produced. Only drop it when negative_prompt_embeds is present, and
move the check to a helper that has unit tests.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* ⬆️ Update ggml-org/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(llama-cpp): preserve chat session state
Newer llama.cpp parsers return template state outside the request JSON.
Pass that state to both inference tasks and response readers so grammar,
tool calls, and reasoning remain available in both chat paths.
Detect the parser API during preparation to preserve older fork builds.
Assisted-by: Codex:gpt-6
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* test(omnivoice): expose ignored duration controls
Compile the real wrapper with a model-free engine fixture. Four failing
assertions expose ignored duration, postprocessing, and validation.
Assisted-by: nib:gpt-6-astra
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(omnivoice): add bounded duration controls
Validate request controls before reference reads or native output.
Preserve engine defaults when controls are omitted. Bound forced
single-shot duration to 30 seconds as wrapper resource policy.
Exercise the real native boundary, streaming rejection, and defaults
with synthetic PCM. Document quantization and postprocessing limits.
Assisted-by: nib:gpt-6-astra
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* test(omnivoice): require fixture-free ordinary tests
Expose the ordinary suite regression before isolating native fixtures.
Require selected native tests to reject a missing fixture explicitly.
Assisted-by: nib:gpt-6-astra
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(omnivoice): isolate native fixture tests
Ordinary Go tests must run without a compiled native fixture.
Select fixture specs explicitly in the dedicated gate while keeping
parser tests ordinary and native checks mandatory.
Assisted-by: nib:gpt-6-astra
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
test: satisfy lint checks for upscale and diagnostics
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Update stable-diffusion.cpp to its merged Iris support commit and add the original F32 Iris weights with the Qwen3-VL Q4_K_M text encoder to the model gallery.
Assisted-by: nib:gpt-5.6-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
