The WebSocket log handler writes its "initial" batch before it calls
Subscribe, so a line appended the instant that batch arrives lands in the
circular buffer with no subscriber to receive it. Three backend-logs specs
append exactly there and then wait out a 5s read deadline; once a gorilla
read hits its deadline the connection is unusable, so the spec cannot retry.
`--focus='Worker WebSocket log streaming' --repeat=25` failed on attempt 17
with nothing else running, which is far too often to wire into CI.
Add BackendLogStore.SubscriberCount, resolving a model ID by the same
exact-key and replica-prefix rules Subscribe uses, and have the specs poll it
until the handler has attached. Nothing in production calls it and no
assertion is weakened; the handler's own snapshot/subscribe window is left as
it is, being a production streaming question rather than a test one.
Verified with 60 repeats of the WebSocket specs and three consecutive
--randomize-all runs of the whole distributed suite, all at
--flake-attempts 1: 239 of 240 specs pass in about 80 seconds.
Assisted-by: Claude Opus 5 [claude-code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A failed CREATE DATABASE panics out of the assertion before closeDB runs,
leaking a pgx pool per attempt. With --flake-attempts 5 that exhausts
postgres:16-alpine's 100 connection slots, at which point the cleanup path's
own Expect fails the spec and one hiccup cascades across the suite. Scope the
admin handle so the panic unwinds through defer closeDB, and let cleanup use a
fallible tryAdminDB that reports rather than asserts.
Register DeferCleanup immediately after CREATE so a later failure cannot leave
the database behind, and warn on TestInfra that the container handles are now
suite-wide.
Assisted-by: Claude Opus 5 [claude-code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Starting a Postgres and a NATS container per spec cost roughly 48 minutes of
startup across the 213 specs behind SetupInfra, which is why this suite was
never wired into CI. Containers move to BeforeSuite and isolation comes from
CREATE DATABASE, which the dbName argument already described.
Assisted-by: Claude Opus 5 [claude-code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Install 404s because the gallery still points at mmproj-...-f16.gguf.
HF only ships ...-F16.gguf now, with a different sha256.
Signed-off-by: lei_lei <96427312+leilei3167@users.noreply.github.com>
Fixes#11673: on macOS the DMG launcher appeared to launch nothing. After
installing, the app sat in the menu bar with no window, nothing listening
on localhost:8080, and empty log files, because nothing ever started the
server unless the unrelated 'start on system boot' option was enabled.
- Start the LocalAI server automatically when the launcher opens and right
after a fresh install. The new auto_start_server config key defaults to
enabled and gets a settings checkbox; the legacy auto_start key was never
honored nor exposed, so every existing launcher.json carries an
unintentional false and is deliberately left behind.
- Fix the welcome window suppressing itself: its 'don't show this again'
checkbox was initialized with the inverted value, and SetChecked fired
the change callback which persisted ShowWelcome=false on the very first
showing.
- Surface auto-start failures through the systray startup-error dialog,
since there is no visible window during auto-start.
- Pass --app-version to fyne package so the app stops reporting itself as
version 0.0.0 in the About box.
- Document the first-launch flow (menu bar app, auto-start, WebUI URL) in
the macOS getting-started page.
- Repair two launcher specs that never ran in CI: a *bool matched against
BeTrue and a /tmp assertion that trips on Linux where the test tempdir
itself lives under /tmp.
Assisted-by: Claude Code:claude-fable-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add three llama.cpp-compatible mixed quantizations from ISTA DASLab. These builds give Qwen3.8-27B users an 8.4 to 10.1 GB weight tier with the shared vision projector.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add Q4_K_M and Q8_0 MTP variants with the shared vision projector.
The publisher recommends these builds for faster Qwen3.8 generation.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the 3B, 8B, and 30B safetensors checkpoints as one vLLM variant family so LocalAI can select the largest build that fits. Configure the parsers and sampling defaults recommended for Granite reasoning and tool calls.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The Makefile already had a hipblas branch, but no CI row built it and
the gallery's `amd:` mapping stayed commented out. On an AMD host the
capability lookup found no `amd` key and fell back to `default`, so
these users silently ran the CPU build.
Add the hipblas row to the backend matrix and the two gallery entries
it publishes, then point `amd:` at them.
Drop `-DGGML_HIPBLAS=ON` while here. `SD_HIPBLAS` sets `GGML_HIP`
itself, and `GGML_HIPBLAS` is the name ggml used before the rename, so
the flag only produced an unused-variable warning. Add gfx1151 to the
local target list to match the value the workflows pass in.
Assisted-by: Claude Code:claude-opus-5[1m]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Tencent released three WeMM sizes with direct Sentence Transformers support. Add each safetensor repository so users can select the quality and resource tradeoff.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
OpenAI GA clients send multipart or raw SDP requests. They expect a bare
SDP answer. LocalAI only accepted its legacy JSON envelope, so signaling
failed before media setup.
Keep the JSON contract for existing clients. Accept both GA request
shapes and choose the matching response format.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the Q4 and Q8 GGUF builds with the shared vision projector.\nThe variant pair lets LocalAI select the build that fits available memory.\n\nAssisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add complete vLLM and SGLang entries with their exact tool parsers. Preserve an explicit zero temperature in both backend adapters.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Node placement and replica rules could only name a model, so an operator
who pinned "llama3" to the GPU tier had to rewrite the rule whenever a
different model took over that job. An alias already gives a stable name
for whichever model serves it, and a rule on that name makes it a
deployment slot: repoint the alias and the placement follows.
A rule keeps the name the operator chose. Reads resolve that name through
the config loader to the model the rule governs, so the reconciler counts,
schedules and trims replicas of the target, and the router finds an
alias-keyed rule from the target it is already routing. An alias that
resolves to nothing governs nothing loadable, so the reconciler skips it
and the write paths refuse it.
A replica is shared by every name that resolves to it, so only one rule
can decide where it runs. The REST and MCP write paths reject a rule whose
target another rule already governs. A pair that arrives some other way,
such as a seed file or an alias repointed onto a model that already has a
rule, resolves in favour of the rule named after the model itself and then
the oldest, and the rest are listed as shadowed.
The eviction guard is the exception: it matches rules to replicas in raw
SQL inside a locking transaction and cannot resolve an alias. It reads a
stored target that the reconciler refreshes each tick, and falls back to
the rule's own name when that target is empty.
Assisted-by: Claude:claude-opus-5 golangci-lint eslint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(ui): move node labels into the scheduling selector field
The scheduling page kept a node-label browser open above the rules
whether or not anyone was writing one, while the field that actually
needs labels, the rule's node selector, was two bare text inputs with no
hint of what the cluster reports.
The browser is gone. The selector's key input now completes against the
label keys the cluster uses, and the value input offers only the values
that key takes. The roster already loads for the page, so the
suggestions cost no request, and a roster that fails to load costs the
admin the hints and nothing else.
Suggestions stay suggestions: a key no node reports yet still commits as
typed, which is how an admin writes a rule before labelling the nodes
for it.
Assisted-by: Claude:claude-opus-5 golangci-lint eslint playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(distributed): size model fit against the cluster, not the frontend
The models page asked the frontend how much memory a model may occupy.
In distributed mode the frontend is usually a GPU-less pod while every
model runs on a worker, so a fleet of GPU nodes was told it could only
run the smallest CPU build. The variant picker's fits flag and its
auto-selection came from the same place, as did the hardware
recommendations.
The registry now reports the largest single healthy backend node. The
largest node, not the fleet total: a model loads into one node, so four
16GB workers are not a home for a 40GB model. An operator-set VRAM
budget caps a node's contribution, because the scheduler refuses a load
above that ceiling anyway, and a GPU node beats a CPU node holding more
system RAM.
GET /api/resources and GET /api/models carry this as an additional
cluster object. Their aggregate and ram fields keep reporting the
frontend's own hardware, which is what the resource monitor shows.
Variant selection judges backends against the union of the capabilities
present in the cluster, the way backend discovery already did.
Every path degrades to the local host: no cluster object in single-node
mode, and none when the registry cannot be read, so a hiccup narrows the
answer back to single-node behaviour rather than marking the whole
catalog too large.
The verdicts now name the node they belong to, since a model fits
somewhere or nowhere.
Assisted-by: Claude:claude-opus-5 golangci-lint eslint playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* ⬆️ Update ggml-org/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(llama-cpp): follow upstream MTMD APIs
The dependency update adds MTMD initialization options to prompt and
bitmap helpers. The gRPC adapter now passes the server options through
each affected path.
The update also replaces the per-layer MoE regex helper. Preparation
probes both APIs because older forks still reuse this adapter.
Assisted-by: Codex:gpt-5
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
go-m1cpu v0.1.6 runs its cgo initialiser from a package init(), where
getFrequency() dereferences the CFTypeRef returned by
IORegistryEntryCreateCFProperty without a NULL check. On Apple M5 the
pmgr IORegistry node does not expose voltage-states5-sram /
voltage-states1-sram in the shape v0.1.6 expects, so the call returns
NULL and CFDataGetLength(NULL) faults before main() runs. Every command
dies, including local-ai --version.
The package is linked indirectly: cmd/local-ai reaches
gopsutil/v3/{process,disk}, which pull in gopsutil/v3/cpu on darwin,
which calls m1cpu.IsAppleSilicon() and m1cpu.PCoreHz().
v0.2.2 adds the missing NULL guard and moves the IORegistry probe out of
init() behind a lazy sync.Once. The exported Go API is unchanged and the
non-darwin stub is byte-identical, so gopsutil/v3 compiles against it
untouched and no other platform is affected.
Bumping gopsutil/v3 is not an alternative: v3.24.5 is the final v3
release, so the v3 line will never carry this fix.
Fixes#11735
Assisted-by: Claude:claude-opus-5
Signed-off-by: Szymon Podeszwa <2962046+sz-po@users.noreply.github.com>
* ⬆️ Update mudler/vllm.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(vllm-cpp): mirror ABI v23
The new engine pin reports ABI v23 and appends mmproj_path to
vllm_model_params. LocalAI still declares v21, so the build-time ABI
guard rejects every backend build.
Grow the Go mirror by the appended pointer and update its offset checks.
ABI v23 adds a video function but does not change the mirrored text
structs.
Assisted-by: Codex:gpt-5.6 [systematic-debugging]
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
/api/tags appends :latest to untagged names, but chat and the other
model endpoints looked the tagged name up as-is and 404'd.
Signed-off-by: lei_lei <96427312+leilei3167@users.noreply.github.com>
