The vllm-cpp CUDA images were built for Blackwell only: 120a;121a on
amd64 and 121a alone on arm64. vllm.cpp's own release archive builds ten
architectures, so LocalAI shipped one or two of them.
The failure mode is the problem. An unlisted card is not slower, it dies
at the first request with "no kernel image is available for execution on
the device", long after `backends install` reported success. That covers
A100, A10/3090, L4/4090/RTX 6000 Ada, H100/H200, B200, B300, Jetson Orin
and Jetson Thor, and it is how a Jetson Thor node was found serving
nothing at all.
amd64 now builds 80;86;89;90a;100a;103a;120a;121a and arm64 builds
87;90a;100a;110;121a, split by where the silicon exists: Jetson is
arm64-only, desktop 120a is amd64-only, and 90a/100a are on both because
of GH200/GB200.
Triton-AOT stays ON for both, which the old comment said was impossible.
It is not, at the version we pin: only maintainer REGEN needs a single
arch, while the BUILDER path embeds every vendored cubin tree and selects
by exact SM, so 87/103a/110/120a take the portable CUDA kernels and can
never load a neighbouring cubin. Upstream ships its ten-SM archive that
way.
The CUDA 13 guard now covers both branches rather than amd64 alone. arm64
needs compute_121a just as much, and CI already builds it with 13.
Cost is smaller than the arch count suggests, because gencode is
per-source: fp4-mma still resolves to 120a;121a, and the CUTLASS
scaled-mm kernels to one arch each, so the added architectures do not
multiply the expensive translation units.
Verified: flag generation checked for both branches, CUDA 12 still
refused, CPU build untouched; both arch lists expanded through vllm.cpp's
own vt_cuda_gencode_options and per-feature arch gating, and all six
vendored Triton trees confirmed intact, at the exact pinned commit. A
real compile is CI-only: there is no CUDA toolchain on the dev box.
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Bash] [Edit]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* ⬆️ Update ggml-org/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(llama-cpp): refresh the TTS patch
The llama.cpp update moved and changed the generated-audio pipeline. Refresh the carried patch so backend builds can apply it to the new revision.
Assisted-by: Codex:gpt-5.4
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* fix(mcp): surface configured server failures
Keep model-configured MCP servers visible when discovery or connection setup fails, propagate status through distributed discovery, and let the Chat UI show actionable errors while retrying unavailable servers.
Add model-editor metadata for remote and stdio configuration and document the expected format, deployment networking boundary, and alternate MCP scopes.
Assisted-by: Codex:gpt-5 Ordino golangci-lint
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* build(compose): match CUDA development image
Configure the API image with the cublas, CUDA 13, auth-tagged build settings used by the local development Makefile invocation, including the 24-way Docker build.
Assisted-by: Codex:gpt-5 Ordino
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* revert: keep host build settings out of compose
The CUDA development deployment is managed from ~/docker/localai, not the repository example Compose file. Restore the generic example and keep machine-specific build settings in the host deployment.
Assisted-by: Codex:gpt-5 Ordino
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* fix(docker): exclude local agent artifacts
Keep Claude worktrees and locally installed verification tools out of the Docker build context. These host-only directories added roughly 1.9 GB to every root image build.
Assisted-by: Codex:gpt-5 Ordino
Signed-off-by: Richard Palethorpe <io@richiejp.com>
---------
Signed-off-by: Richard Palethorpe <io@richiejp.com>
Forward BUILD_TYPE=metal to PARAKEET_GGML_METAL so macOS backend artifacts include Metal support.
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: fierywaters <fierywaters13@gmail.com>
Add the smaller Fara1.5 computer-use model alongside the existing 9B and 27B gallery entries. Offer Q4_K_M and Q8_0 builds so LocalAI can select for memory or quality.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The DeepSeek V4 Flash base entry points at a Hugging Face repository page instead of a GGUF object. When variant probing cannot rank a concrete build, the base fallback downloads no usable model weights.
Use the validated IQ2XXS object and checksum already shipped by the q2 variant. Pin that payload in the gallery resolution test.
Assisted-by: Codex:gpt-5.6
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the publisher-recommended Q4_K_M build and a near-lossless Q8_0 variant for the 3B formal-logic reasoning model.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the official llama.cpp GGUF builds for NVIDIA Nemotron 3.5 Lightning 30B-A3B, linking Q4_K_M, NVFP4, and Q8_0 as installable variants.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The validated gallery checksum changed in #11459, but its dedicated regression assertion kept the previous value and now fails the master test suite on Linux and macOS.
Assisted-by: Codex:gpt-5.4
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* ⬆️ Update ikawrakow/ik_llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(gallery): align Higgs Audio checksum test
The validated gallery checksum changed in #11459, but its dedicated regression assertion kept the previous value and now fails the master test suite on Linux and macOS.
Assisted-by: Codex:gpt-5.6
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
LOCALAI_WEBRTC_ICE_INTERFACES was silently ignored whenever
LOCALAI_WEBRTC_UDP_PORT was set. Every interface was gathered regardless
of the allow-list, so a browser was handed the docker0/veth addresses the
setting exists to suppress, and the connection established on a good pair
and then dropped when consent checks failed on the unreachable ones.
Two things combine to cause it. A mux built over a wildcard socket makes
pion derive its host candidates by enumerating interfaces itself, with a
nil filter and loopback included. Independently, the muxed gathering path
in pion/ice never consults SetInterfaceFilter, so setting it has no effect
there either.
Bind one socket per admitted interface address via NewMultiUDPMuxFromPort,
which takes the filter, instead of one wildcard socket. All the sockets
share the same port, so the firewall requirement is still a single rule.
Networks are pinned to UDP4 to match the socket family this replaces.
An allow-list that matches no address on the host now reports the
misconfiguration rather than binding nothing and leaving signaling to
succeed while no candidate is ever advertised.
Two tests: one asserts an unmatched allow-list is an error, and one gathers
against a real peer connection and asserts no address outside the allowed
interface appears (skipped on single-interface hosts).
Assisted-by: Claude:claude-opus-5 go vet gofmt
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
The gallery checksum did not match the Q8_0 object currently published by audio-cpp. Use the SHA256 reported by both Hugging Face LFS metadata and the live x-linked-etag so model installation can verify the download.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Eight header controls across seven pages had two or three elements' classes
collapsed into one string. The wrapper or the icon ended up wearing the
button classes, and the buttons themselves were left with no class at all,
so they rendered in the browser's own chrome. Reported on Agent Jobs; the
grep found the rest.
`fas` does not draw anything by itself: it sets
`font-family: "Font Awesome 6 Free"` and weight 900 on whatever carries it,
and the `fa-*` class supplies the glyph via ::before. So
<button className="btn btn-primary fas fa-plus">
renders its own label "New Task" in the icon font, and
<div className="hstack btn btn-primary btn-sm fas fa-edit btn-secondary fa-arrow-left">
<button>Edit</button>
<button>Back</button>
</div>
styles the flex wrapper as a button that is both primary and secondary,
points two glyphs at one ::before, and leaves both real buttons bare.
Fixed, all of them keeping the correct `<i>` child they already had:
- AgentJobs, AgentTaskDetails (x2), AgentCreate - icon classes off the
button.
- AgentTaskDetails, AgentJobDetails - wrapper back to plain `hstack`, and
the two buttons inside each get the variants the wrapper had been
holding. Back is secondary and leads, Edit/Cancel is the emphatic one
and trails, matching every other detail header.
- VoiceLibrary, VoiceProfileCreate - the title `<i>` had swallowed the
action link's classes, so "Create voice" and "Back to library" were
unstyled anchors. Back was also drawing a "+" because it had inherited
fa-plus while its own fa-arrow-left sat up in the title.
- P2P - a stray fa-circle-info on the title icon.
The ninth instance was ImportModel, where this class of bug was first
found. #11461 rewrote that file and landed first, so nothing is left to fix
there.
Guarded by e2e/class-hygiene.spec.js, which reads the source rather than
walking routes: several of these pages need agent or voice data before they
render a header, so a route walk would skip exactly the pages that had the
bug. It fails on an icon-font class outside an `<i>`/`<span>`, on two glyphs
or two button variants on one element, and on a layout wrapper that is also
a button. Font Awesome modifiers (fa-spin, fa-fw, sizes) are excluded, so
the `fa-spinner fa-spin` idiom stays legal.
e2e: 428 passed.
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] [Playwright]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The import page took the new palette in #11305 but kept its old layout, so
it stayed a 760px column with the primary action detached from the form it
submits. Two of the problems were outright bugs.
The Import button carried no className at all, so the page's single most
important control fell through to the user-agent button: system chrome,
wrong radius, no design-system focus ring. The YAML button carried
`fas fa-save fa-upload`, which sets Font Awesome as the button's own font
family (its label text inherits it) and points two glyph classes at one
::before.
On the layout: `page--narrow` is documented for "forms / single-record edit
views", and in Advanced mode this page held a URI field, a six-section
format guide, ten modality chips, nine preference fields, a key-value
repeater and a YAML editor at `calc(100vh - 400px)`. The width was the
symptom; one column was the disease.
- `page--medium` with a work column and a format reference beside it.
The reference answers the only question a first-time admin has and used
to sit behind a chevron, closed by default. Below 1024px it becomes a
disclosure rather than disappearing.
- The source field is the hero: monospace, because it holds something you
paste, and it carries its own Import button. That removes the hidden
aria-hidden submit button that existed only because the real action sat
outside the form.
- Simple and Advanced are gone. They were ~80% the same surface, and the
overlap cost a mode switch, a localStorage key and a three-button
Keep/Discard/Cancel dialog whose only job was protecting state that
switching modes would hide. One form with a collapsible options panel
hides nothing, so none of it is needed. What genuinely differs is the
kind of input, which is now the two tabs: a source, or YAML.
- The size/VRAM estimate reports under the field that produced it instead
of as a banner above the page header, and an import in flight gets the
progress, phase and byte counts the poller already returned and the old
status card threw away.
- ModalityChips resolves its labels through the same `modality.*` keys as
the dropdown it filters. It hardcoded English shorthand, so one modality
carried two names on one screen ("Speech" on the chip, "Speech
recognition" on the group it scrolled to) and seven locales had neither.
Its inline styles and its pill radius move onto the design system.
- Three inline styles go, including both conditional-padding hacks; the
only one left is the progress bar's runtime width. Baseline 538 -> 535.
Docs updated in the same change: the WebUI section described a Simple and an
Advanced mode and told the reader to "Toggle to Advanced Mode".
e2e: 426 passed. The mode-switch suite is replaced by one covering the tabs
and the disclosure, and a new layout suite pins the width, the styled
primary action, the absence of an icon-font button, the reference column at
both widths, and the estimate's position.
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] [Playwright]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(model): surface backend startup exits
Preserve the local backend process exit code and bounded stderr diagnostic when the process dies before its gRPC service becomes ready.
Fixes#9050
Assisted-by: Codex:gpt-5
* fix(model): satisfy startup diagnostic checks
Assisted-by: Codex:gpt-5.6 [Codex]
---------
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* fix(ci): parse current vllm-metal version pins
vllm-metal renamed its installer pin from vllm_v to VLLM_VERSION, breaking both the nightly bumper and the Darwin backend installer after a bump. Share a strict parser that accepts both formats and cover the transition with shell regressions.
Assisted-by: Codex:gpt-5 [Codex]
* fix(ci): parse scoped vllm-metal pins
The pinned vllm-metal installer declares its version as a local shell variable. Accept that optional declaration while retaining strict validation of the assignment and semantic version.
Assisted-by: Codex:gpt-5.4
---------
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add Q4_K_M and Q6_K llama.cpp builds with the matching vision projector and verified Hugging Face artifact hashes.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the text-only IQ2_XXS build with the model card's required Jinja, reasoning parser, KV cache, and sampling defaults.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Expose the existing audio.cpp Higgs support as an installable Q8 gallery model and document voice cloning and licensing constraints.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the official dynamic and 17 GB K-quant GGUF builds with the shared multimodal projector and optional DFlash acceleration.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* feat(realtime): add shared WebRTC UDP port
Allow realtime WebRTC peer connections to reuse one configurable UDP mux, and surface listener bind failures through signaling.
Assisted-by: Codex:gpt-5
* test(realtime): keep UDP mux alive during bind check
The returned SettingEngine owns the UDP listener. Retain it through the duplicate-bind assertion so macOS cannot finalize the listener early and make the exclusivity check spuriously pass.
Assisted-by: Codex:gpt-5 [systematic-debugging]
* test(realtime): use IPv4 for UDP mux checks
Match the socket family used by the WebRTC UDP mux so macOS does not allocate an IPv6 probe that can coexist with the IPv4 listener.\n\nAssisted-by: Codex:gpt-5
---------
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add official multimodal GGUF builds for the new BigBang-v1 scientific and agentic reasoning model, including Q4_K_M and Q6_K variants with MTP enabled.\n\nAssisted-by: Codex:gpt-5 [Hugging Face API]
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
MiniMax-H3 ships two independently-served DiT partitions and each refuses the
other's tasks. The gallery had only the FL2VA one (t2va + fl2va); this adds the
Ref2VA checkpoint, which is what serves REFERENCE conditioning - a reference
image, a reference clip, or reference audio prepended as their own blocks.
It is a genuinely separate 19.9 GB download, not a mode flag, which is why it
needs its own entry. Everything except the DiT is shared with the FL2VA entry.
Q4_K_M specifically, and the description says why: the NVFP4 Ref2VA weights
render a multicolour patch grid. Three upstream investigations attributed that to
the ref2va reference-row path before the checkpoint was varied; on Q4_K_M the
identical code renders coherently (period-16 seam 1.13 against 2.28 broken,
VAE-input latent adjacent-cell cosine 0.8526 against 0.06 white). Pointing users
at NVFP4 here would hand them that dead end.
known_input_modalities carries image, video AND audio, unlike the FL2VA entry:
the ABI's ref2va slots are ref_image (PPM), ref_video (a directory of
frame_%06d.ppm) and ref_audio (16-bit PCM WAV), and the backend's
checkPartitionConditioning already routes all three.
Checksum verified two ways: sha256 computed over the downloaded 19,864,208,064
byte file, and HuggingFace's x-linked-etag for the same URI - both
17925612821ea3037ffaf5f7f9789f5460e87025385bd45e9ec6c7d536684d56.
Not verified here: an end-to-end render through LocalAI itself. The render was
proven with vllm.cpp's own driver on this exact checkpoint; the backend path is
the same library seam but has not been exercised with these weights.
Assisted-by: Claude Code:claude-opus-5 [ClaudeCode]
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
