* fix(backends): list backends runnable on worker nodes in distributed mode
GET /backends/available filtered the gallery against the system state of
the host serving the request. In a distributed deployment that host is the
controller, which typically has no GPU, while the GPUs live on worker
nodes. Any meta backend whose capabilities map lacks a "default" (or "cpu")
key was therefore dropped from the listing entirely — longcat-video,
vllm-omni, ltx-video, parakeet, edgetam and qwentts were invisible in the
UI even though installing them by name on a GPU worker worked fine.
Workers now report their own meta-backend capability at registration and
the controller persists it on the node row. The controller cannot derive
it: OS-dependent capabilities (metal, darwin-x86, nvidia-l4t) and the CUDA
runtime refinements are only observable on the worker. Nodes registered
before this field existed fall back to a coarse capability derived from
their GPU vendor and VRAM.
Backend discovery then evaluates compatibility as the union over healthy
backend nodes, so a backend runnable on any node is offered while one no
node can run stays hidden. Each remote capability is evaluated through a
capability-pinned system state, otherwise a forced capability on the
controller image (LOCALAI_FORCE_META_BACKEND_CAPABILITY or
/run/localai/capability) would silently override every worker's verdict.
With no registered nodes the listing is byte-for-byte what it was, so
single-node deployments are unaffected.
Also fixes the same-root-cause misclassification in /api/operations, which
used the capability-filtered listing to decide whether an operation was a
backend or a model install. A GPU-only backend installing on a worker is
still a backend operation on the controller, so that lookup is now
unfiltered.
Assisted-by: Claude:claude-opus-4-8 golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(backends): union worker capabilities in backend discovery
Implementation for the specs added in the previous commit, plus the two
remaining discovery endpoints.
Capability-filtered backend discovery evaluated compatibility against the
system state of the host serving the request. In a distributed deployment
that host is the controller, which typically has no GPU, while the GPUs
live on worker nodes. Any meta backend whose capabilities map lacks a
"default" (or "cpu") key was dropped entirely — longcat-video, vllm-omni,
ltx-video, parakeet, edgetam and qwentts were invisible in the UI even
though installing them by name on a GPU worker worked fine.
Workers now report their own meta-backend capability at registration and
the controller persists it on the node row. The controller cannot derive
it: OS-dependent capabilities (metal, darwin-x86, nvidia-l4t) and the CUDA
runtime refinements are only observable on the worker. Nodes registered
before this field existed fall back to a coarse capability derived from
their GPU vendor and VRAM.
Discovery then evaluates compatibility as the union over healthy backend
nodes, so a backend runnable on any node is offered while one no node can
run stays hidden. Each remote capability is evaluated through a
capability-pinned system state, otherwise a forced capability on the
controller image (LOCALAI_FORCE_META_BACKEND_CAPABILITY or
/run/localai/capability) would silently override every worker's verdict.
With no registered nodes the listing is byte-for-byte what it was, so
single-node deployments are unaffected.
Four surfaces shared this root cause and are all routed through the same
helper now:
- GET /backends/available
- GET /api/fine-tuning/backends
- GET /api/quantization/backends
- /api/operations backend-vs-model classification, which additionally
had no reason to filter by capability at all: a GPU-only backend
installing on a worker is still a backend operation on the
controller, so that lookup is now unfiltered.
Assisted-by: Claude:claude-opus-4-8 golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
