go-m1cpu v0.1.6 runs its cgo initialiser from a package init(), where
getFrequency() dereferences the CFTypeRef returned by
IORegistryEntryCreateCFProperty without a NULL check. On Apple M5 the
pmgr IORegistry node does not expose voltage-states5-sram /
voltage-states1-sram in the shape v0.1.6 expects, so the call returns
NULL and CFDataGetLength(NULL) faults before main() runs. Every command
dies, including local-ai --version.
The package is linked indirectly: cmd/local-ai reaches
gopsutil/v3/{process,disk}, which pull in gopsutil/v3/cpu on darwin,
which calls m1cpu.IsAppleSilicon() and m1cpu.PCoreHz().
v0.2.2 adds the missing NULL guard and moves the IORegistry probe out of
init() behind a lazy sync.Once. The exported Go API is unchanged and the
non-darwin stub is byte-identical, so gopsutil/v3 compiles against it
untouched and no other platform is affected.
Bumping gopsutil/v3 is not an alternative: v3.24.5 is the final v3
release, so the v3 line will never carry this fix.
Fixes#11735
Assisted-by: Claude:claude-opus-5
Signed-off-by: Szymon Podeszwa <2962046+sz-po@users.noreply.github.com>
* ⬆️ Update mudler/vllm.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(vllm-cpp): mirror ABI v23
The new engine pin reports ABI v23 and appends mmproj_path to
vllm_model_params. LocalAI still declares v21, so the build-time ABI
guard rejects every backend build.
Grow the Go mirror by the appended pointer and update its offset checks.
ABI v23 adds a video function but does not change the mirrored text
structs.
Assisted-by: Codex:gpt-5.6 [systematic-debugging]
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
/api/tags appends :latest to untagged names, but chat and the other
model endpoints looked the tagged name up as-is and 404'd.
Signed-off-by: lei_lei <96427312+leilei3167@users.noreply.github.com>
* feat(gallery): add Ornith 1.5 397B variants
Add the official Q4_K_M and Q8_0 GGUF builds with their shared BF16 vision projector.
Assisted-by: Codex:gpt-5
* feat(gallery): resolve Ornith variant ordering\n\nKeep the 35B entries from master next to the 397B variants.\n\nAssisted-by: Codex:gpt-5
---------
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* ⬆️ Update ggml-org/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(llama-cpp): adapt to the common JSON API
The llama.cpp bump replaces its nlohmann JSON alias with common_json. Update the gRPC adapter for the new exception, iterator, conversion, and container APIs.
Assisted-by: Codex:gpt-5.6 [systematic-debugging]
* fix(turboquant): adapt the JSON exception type
The shared gRPC source now follows the upstream common_json API. The
TurboQuant fork still exposes nlohmann JSON and cannot compile the new
exception type.
Translate that exception in the fork-specific source patch so both
llama.cpp variants compile from the shared adapter.
Assisted-by: Codex:gpt-5.6 [systematic-debugging]
* fix(bonsai): adapt the JSON exception type
The shared gRPC source uses upstream's common_json wrapper. The Bonsai fork still exposes nlohmann JSON and cannot compile that exception type.\n\nTranslate the exception in the fork-specific preparation step and verify that repeated preparation stays idempotent.\n\nAssisted-by: Codex:gpt-5.6 [systematic-debugging]
* fix(llama-cpp): let prepare register gRPC
The score patch duplicated the gRPC CMake registration that prepare.sh already owns. Its stale context rejects the current upstream tools file on Darwin before compilation starts.
Assisted-by: Codex:gpt-5
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Those backends only forwarded the flag when it was "true", so "false"
never reached apply_chat_template and Qwen3 kept thinking on.
Signed-off-by: lei_lei <96427312+leilei3167@users.noreply.github.com>
The scheduler's liveness probe asks a worker a question over NATS and
reads "no responders" as proof the worker is gone. That is only sound
when every worker in the fleet subscribes to the subject asked.
It asked models.running, which arrived in 4.6. A 4.5 worker is alive and
serving, answers backend.list, and never subscribes to models.running,
so the probe condemned it on every scheduling attempt and marked it
unhealthy. A model pinned to such a node by its selector could then
never be placed at all: on this cluster an embedding model pinned to the
one Apple node was unschedulable for exactly this reason, while that
node's log showed it handling backend.list throughout.
Ask backend.list, which has been in the worker protocol far longer, and
treat a worker that answers anything as alive. Only a node that reports
no responders on every subject is absent, so adding a newer subject here
can never condemn an older worker.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
A model's revision is published by administration and checked against on
every inference request. Those were computed by separate code: the
request path resolves through the loader, while each publisher hashed
whatever ModelConfig it happened to hold. By then SetDefaults had folded
in the GGUF guess and app-level options, so the published value was one
no request would ever carry and the model became unroutable until the
row was deleted by hand.
Fixing the publishers one at a time did not hold. Three rounds each
found another: the startup resync, then a saved edit and a toggle, then
a rename and the peer-change path.
ModelConfigLoader.RevisionFor is now the only way to obtain a revision,
and the raw hash is unexported, so a caller outside this package cannot
hash a config it holds. A publisher and a request agree by construction
rather than by two implementations happening to match.
The request path no longer falls back to hashing its merged config
either: an unstamped config is routed without a revision rather than
with a wrong one.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
When no node the selector allows has a free slot, scheduling falls back
to evicting the least-recently-used idle model. That eviction searched
every healthy node, so it freed a slot on a node the selector forbids
and the model was then placed there: pinned to one class of hardware and
running on another.
An unrelated model pays for it. On this cluster an embedding model
pinned to Apple hardware could not reach its only matching node, so each
attempt evicted a large language model from an Nvidia node, failed to
start there anyway, and left the evicted model to reload. Repeated, that
reads as one replica bouncing between nodes.
Eviction is now restricted to the candidate set the selector produced.
With no selector the candidate set is nil and eviction stays global.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
The abandoned-load sweeper treated a replica row with no load job as
abandoned. Only the request path creates load jobs; the reconciler's own
scale-up loads a replica without one. So any scale-up that ran past the
five-minute grace period was deleted mid-transfer, which for a
multi-gigabyte checkpoint is every time. The replica never finished
anywhere, and the reconciler kept re-placing it, so it looked like one
replica hopping between nodes instead of a model reaching its replica
count.
A row with no job is now reclaimed only once its node stops being
healthy, which is the case the sweeper was written for: a worker that
dropped out mid-transfer. A job that failed or stopped heartbeating
still proves abandonment on its own. Every uncertain case leaves the
slot held.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
Only backend workers subscribe to backend.list. ListBackends asked every
node that was not pending, offline or draining, so an agent worker could
only answer "no responders", which the error handling reads as a node
that has gone away. Every poll of the backends view therefore marked
each agent node unhealthy, and its next heartbeat marked it healthy
again.
While unhealthy the node is not schedulable, so this also cost agent
capacity for as long as each flap lasted.
Skip non-backend workers, as the backend-op fan-out already does for the
same reason. A backend worker that does not answer is still marked
unhealthy: that one really is gone.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
The revision was computed after SetDefaults, which folds in things that
are not persisted configuration: the GGUF guess, the hardware defaults,
and app-level options such as threads.
The GGUF guess is the damaging one. It parses the model file to fill in
values like context size, and when that parse fails it falls back to a
different default. Whether a multi-gigabyte file on network storage
parses at a given moment is not a property of the configuration, so one
unchanged YAML produced two different revisions depending on when it was
read. The controller rejected every request carrying the other one, and
the model stayed unroutable until the stored value happened to match
again. This is why it never reproduced against a model directory with no
weights in it: the guess is skipped there and both values agree.
The app-level defaults are the same class of bug with a slower fuse:
changing threads in the settings UI changed every model's revision and
made every model unroutable.
The revision is now stamped when the file is parsed, before any defaults
are applied, so it is a function of the file alone.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
Two code paths computed a model's revision. Inference resolves the
config through the loader, which applies SetDefaults a second time.
Everything that publishes a revision hashed the stored config instead,
with SetDefaults applied once.
SetDefaults is not idempotent for every model: it re-runs the GGUF guess
and the hardware defaults, both of which read state the stored config
does not carry. Where the two disagree, a publisher wrote a revision no
request would ever carry, and the model became unroutable the moment it
was published. On this cluster the startup resync republished one such
value and every request for that model was then rejected against it.
The publishers now resolve the revision through the loader, exactly as a
request does, so there is one definition rather than two that agree only
when SetDefaults happens to be idempotent. This covers the startup
resync, a saved config edit, and enabling or disabling a model.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
quay.io redirects blob downloads to pre-signed S3/Akamai URLs that
expire after about 10 minutes. On a slow connection a multi-GiB
backend layer cannot finish inside that window, so the connection
drops mid-stream on every attempt. The retry added for #10577
restarted each attempt from byte zero, which replayed the same
failure until the budget ran out and the install failed with
"unexpected EOF".
A retry now keeps the bytes already on disk and re-requests the
blob with "Range: bytes=N-". Each request goes back to the
registry, so it gets a fresh redirect URL and auth token. The
retry budget only counts attempts that made no forward progress,
so a slow link that keeps advancing keeps downloading. A resumed
file is spliced from separate responses and bypasses the digest
check in layer.Compressed(), so the assembled file is re-verified
against the layer digest before it is trusted; on a mismatch the
download starts over through the verified reader.
Fixes#10577
Assisted-by: Claude Code:claude-fable-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* feat(gallery): add Qwen3.8 OBLITERATED variants
Add Q4_K_M and Q8_0 llama.cpp builds with the shared BF16 vision projector.
Assisted-by: Codex:gpt-5
* fix(tests): implement node liveness stub
NodeCommandSender now requires PingNode. The endpoint test stub must
implement it before the package can compile.
Assisted-by: Codex:gpt-5 [Codex]
* fix(distributed): restore node liveness tests
The router now probes models.running before it schedules work. The E2E
workers only mocked backend.install, so every test node appeared offline.
The endpoint test double also missed the new PingNode method and stopped
the Linux, Apple, and lint jobs during compilation.
Mock the existing worker reply in both distributed fixtures and keep the
endpoint test double aligned with NodeCommandSender.
Assisted-by: Codex:gpt-5 [golangci-lint]
* fix(tests): check node liveness replies
The liveness test subscriptions ignored setup and reply errors.
Errcheck rejected each branch that carried them.
Assisted-by: Codex:gpt-5 [golangci-lint]
---------
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* fix(distributed): restore node liveness tests
The router now probes models.running before it schedules work. The E2E
workers only mocked backend.install, so every test node appeared offline.
The endpoint test double also missed the new PingNode method and stopped
the Linux, Apple, and lint jobs during compilation.
Mock the existing worker reply in both distributed fixtures and keep the
endpoint test double aligned with NodeCommandSender.
Assisted-by: Codex:gpt-5 [golangci-lint]
* fix(tests): check node liveness replies
The liveness test subscriptions ignored setup and reply errors.
Errcheck rejected each branch that carried them.
Assisted-by: Codex:gpt-5 [golangci-lint]
---------
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* fix(ci): bound Discord release summaries
The release model can return more than Discord's 2,000-character
message limit. Discord then rejects the entire release notification.
Ask the model for a smaller response and truncate extracted content to
1,800 characters before the notification step. The smaller bound leaves
room below Discord's hard limit when model output varies.
Assisted-by: Codex:gpt-5
* fix(tests): implement node liveness stub
NodeCommandSender now requires PingNode. The endpoint test stub must
implement it before the package can compile.
Assisted-by: Codex:gpt-5 [Codex]
---------
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The resync added in 3953448f6 ran before LoadModelConfigsFromPath, so it
read an empty loader, reconciled nothing and reported success. The
symptom was a stored revision that stayed stale across restarts while
the log showed no complaint, which is exactly what the resync was meant
to prevent.
Move the call after the configs are loaded, and refuse to treat an empty
loader as a clean run: reconciling zero models is indistinguishable from
reconciling correctly, and that is what hid the mis-ordered call.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
The controller pins a model's replicas to a stored revision and rejects
any request carrying a different one. Nothing ever re-derived that value
from the configuration on disk: it moved only on an edit, a gallery
install, or a peer's change broadcast. An inference request may only
establish a revision, never replace one.
So any other way for the two to diverge left the model permanently
unroutable. A configuration edited while a frontend was down lands
there, and so does a change in what the revision is computed over: an
upgrade that alters the hashed form leaves every stored revision
describing a configuration that no longer exists. The only recovery was
deleting the row by hand, which is not something a cluster should need.
Each frontend now reconciles the stored revisions against the loaded
configurations at startup and republishes the ones that disagree. Only
those: republishing quarantines every replica loaded under the old
revision, so doing it for a model that did not drift would unload a
healthy replica for nothing. A model with no stored revision has never
been served and is left for its first request to establish.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
A replica row in staging or loading holds its slot, because slot
allocation counts every state except unloading. Nothing ever reclaimed
such a row: every reconciler pass and the router's eviction query filter
state = "loaded", and the per-model probe skips rows without an address,
which is exactly what a row that never finished loading has.
So a worker that dropped out mid-transfer left a row that pinned the
only replica slot for that model on that node. Scheduling then found no
free slot and eviction found nothing it was allowed to evict, and the
request failed with "no replica slot on <node> and eviction failed: all
models busy". The state persisted until an operator intervened.
The reconciler now reclaims a row stuck before serving when no load job
is driving it. Ownership is decided by the job's LastProgress heartbeat,
not by elapsed time: staging a large checkpoint legitimately runs for a
long while without touching the replica row, so a deadline would either
be a model-size cliff or reclaim a healthy transfer. That heartbeat is
the same signal job takeover already trusts. Any error reading the job
leaves the slot held, because holding one for another pass costs a
scheduling opportunity while a wrong reclaim restarts a multi-gigabyte
transfer.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
A node's status comes from its HTTP heartbeat. Backend installs travel
over NATS. The two are independent, so a worker that dies stops
answering on the bus at once but stays healthy in the database until its
heartbeat ages out. Inside that window the scheduler picked a node it
could not reach, and the request failed with "no responders available"
rather than moving to a node that was up.
The scheduler now probes the node it selected and, when nothing answers,
marks it unhealthy and selects again. The demotion is what makes the
retry terminate: the next selection reads only healthy nodes. It also
tells the other frontends what this one learned, so the cluster does not
rediscover a dead worker one failed request at a time.
Only nats.ErrNoResponders counts as absent. A worker that answers slowly
stays eligible, because dropping it would cost capacity that is really
there. The probe reuses the models.running subject: a new subject would
go unanswered by workers that have not been upgraded, and every one of
them would then look dead.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
Picks up "surface option errors instead of discarding them". New records
the error from applying its options and Run returns it, so a process
whose state directory cannot be created reports the real cause instead
of failing later inside os.MkdirAll("") with an empty path.
LocalAI already resolves that directory itself, so this covers the other
process.New call sites rather than changing behaviour on the backend
start path.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5
process.New applies its options and discards the error they return. When
WithTemporaryStateDir could not create a directory, StateDir stayed
empty and every later option went unapplied, so the failure surfaced
from Run as "mkdir : no such file or directory" naming no path.
That message cost a full day of diagnosis on a worker whose volume was
full: the real errno was ENOSPC and nothing reported it.
The loader now creates the directory itself and returns the underlying
error with the path attached.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
