* feat(gallery): add Ornith 1.5 397B variants
Add the official Q4_K_M and Q8_0 GGUF builds with their shared BF16 vision projector.
Assisted-by: Codex:gpt-5
* feat(gallery): resolve Ornith variant ordering\n\nKeep the 35B entries from master next to the 397B variants.\n\nAssisted-by: Codex:gpt-5
---------
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* feat(gallery): add Qwen3.8 OBLITERATED variants
Add Q4_K_M and Q8_0 llama.cpp builds with the shared BF16 vision projector.
Assisted-by: Codex:gpt-5
* fix(tests): implement node liveness stub
NodeCommandSender now requires PingNode. The endpoint test stub must
implement it before the package can compile.
Assisted-by: Codex:gpt-5 [Codex]
* fix(distributed): restore node liveness tests
The router now probes models.running before it schedules work. The E2E
workers only mocked backend.install, so every test node appeared offline.
The endpoint test double also missed the new PingNode method and stopped
the Linux, Apple, and lint jobs during compilation.
Mock the existing worker reply in both distributed fixtures and keep the
endpoint test double aligned with NodeCommandSender.
Assisted-by: Codex:gpt-5 [golangci-lint]
* fix(tests): check node liveness replies
The liveness test subscriptions ignored setup and reply errors.
Errcheck rejected each branch that carried them.
Assisted-by: Codex:gpt-5 [golangci-lint]
---------
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the 2B and 4B distilled models alongside the 9B model. Their compact GGUF builds make the Qwen3.8 reasoning distill practical on smaller hosts.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
LiquidAI now publishes official DSpark draft sidecars for its 2.6B target. Pair Q4 and Q8 targets with matching draft choices so LocalAI can use speculative decoding across different memory budgets.
Assisted-by: Codex:gpt-5 [systematic-debugging]
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The Qwopus variant pair was added twice, which makes the later pair unreachable and fails the gallery invariant on master.
Keep the first pair that LocalAI already resolves and remove the dead duplicate declarations.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The gallery ranker reads serving features from tags instead of backend options. Mark two entries that enable draft MTP so ranking can identify them correctly.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Expose the upstream Q8_0 build and the MTP drafter for the existing Gemma 4 12B agentic entry. The variants let LocalAI select higher fidelity or faster speculative decoding from the same model family.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the official Q4_K_M and Q8_0 GGUF builds with the shared BF16 vision projector.
Assisted-by: Codex:gpt-5 [Hugging Face API]
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add Tencent’s compact translation model in Q4 and Q8 GGUF formats. The linked variants let LocalAI select the best build that fits each host.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add Q4_K_M and Q8_0 GGUF builds for Tencent UI-Mate-9B. Include the f16 vision projector so llama.cpp can process desktop screenshots.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The official GGUF repository publishes both Q4_K_M and Q8_0 builds, but the gallery only exposed Q4_K_M. Link the higher-quality Q8_0 build so capable hosts can select it automatically.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the 2B and 4B reasoning distillations in Q4_K_M and Q8_0 formats. These sizes extend the existing Qwen3.8 family to compact and edge hosts.
Assisted-by: Codex:gpt-5 [web]
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the smaller 3.69-bit mixed quantization to the existing Qwen3.8 27B variant group. Enable its embedded MTP head so compatible hosts can prefer speculative decoding.
Assisted-by: Codex:gpt-5 [Web]
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add Q4_K_M and Q8_0 llama.cpp builds with the shared vision projector. The MIT-licensed agentic coding model is absent from the current gallery.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add Q4_K_M and Q8_0 llama.cpp builds for BAAI AREX-Turbo. The compact research agent is absent from the current gallery.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Adds eight curated vllm-cpp entries to the model gallery. Until now the
backend had gallery coverage only for MiniMax-H3 video, so serving text on
it meant hand-writing engine_args.
The flagship tier is what vllm.cpp gates its correctness and speed claims
on: Qwen3.6-27B and Qwen3.6-35B-A3B in NVFP4, each with a speculative
sibling (MTP on both, DFlash on the 27B). Qwen3-Coder-30B-A3B covers
agentic tool use, and Qwen3-4B / Qwen3-0.6B in bf16 are the entries that
run where NVFP4 cannot, CPU included.
Three details are load-bearing rather than incidental:
- The 27B entries pin revision 890bdef7. That repository was later
re-quantized in place from NVFP4 to FP8 W8A8 under the same name, so an
unpinned entry resolves to different weights and reports nothing.
- Qwen3-Coder names tool_parser: qwen3_coder explicitly. Its dialect is
byte-identical on the wire to step3p5's, so chat-template sniffing
cannot separate them and auto-detection picks wrong.
- enable_prefix_caching is deliberately left unset everywhere. It defaults
on for dense models and off for the GDN hybrids, and that per-model
default is the right answer.
num_blocks is sized per model from its real KV footprint rather than
copied between entries, which ranges from 20 KiB/token on the 35B to
144 KiB/token on the 4B.
Docs: adds features/vllm-cpp.md covering installation, the model table,
the pinning rationale and how to choose between the speculative variants,
and cross-links it from the existing engine_args reference. It also
records that the CUDA images are built for Blackwell only, which is
narrower than vllm.cpp's own ten-architecture release and makes an
otherwise cryptic "no kernel image is available" failure legible.
Verified: gallery suite green; all eight decode and validate as a
ModelConfig. qwen3-0.6b-vllm-cpp confirmed end to end on a real cluster,
chat plus engine-parsed tool_calls. The NVFP4 entries are not yet
runtime-verified: no available node has kernels for them.
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Bash] [Edit]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Add LiquidAI’s compact edge model in Q4_K_M and Q8_0 builds. The
variant pair lets LocalAI choose the highest-quality build that fits.
Assisted-by: Codex:gpt-5.4
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the official Q4_K_M and Q8_0 GGUF files with the shared vision projector. Include an MTP variant for speculative decoding.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the smaller Fara1.5 computer-use model alongside the existing 9B and 27B gallery entries. Offer Q4_K_M and Q8_0 builds so LocalAI can select for memory or quality.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The DeepSeek V4 Flash base entry points at a Hugging Face repository page instead of a GGUF object. When variant probing cannot rank a concrete build, the base fallback downloads no usable model weights.
Use the validated IQ2XXS object and checksum already shipped by the q2 variant. Pin that payload in the gallery resolution test.
Assisted-by: Codex:gpt-5.6
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the publisher-recommended Q4_K_M build and a near-lossless Q8_0 variant for the 3B formal-logic reasoning model.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the official llama.cpp GGUF builds for NVIDIA Nemotron 3.5 Lightning 30B-A3B, linking Q4_K_M, NVFP4, and Q8_0 as installable variants.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
