Add Q4_K_M, Q6_K, and Q8_0 builds for llama.cpp with pinned assets.
Group the builds and document installation, reasoning, and tool setup.
Assisted-by: Codex:gpt-6
(cherry picked from commit f9e7f0bfbe, mudler/LocalAI#12706)
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add AtomicChat MiniCPM-V 4.7 35B-A3B Q4 and Q6 builds for llama.cpp.
Include every weight shard and the F16 vision projector.
Assisted-by: Codex:gpt-6
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
(cherry picked from commit eff464d58a, mudler/LocalAI#12690)
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add Kujira's mixed IQ2 Saluki fine-tune for llama.cpp with plain and MTP
decoding. Pin the weights and document the text-only entries.
Assisted-by: Codex:gpt-6
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
(cherry picked from commit 24f23a66aa, mudler/LocalAI#12688)
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add Q4 and IQ4 GGUF builds with a BF16 vision projector and pinned
Hugging Face downloads. Keep the original release available.
Assisted-by: Codex:gpt-6
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
(cherry picked from commit 9c9e18450e, mudler/LocalAI#12670)
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Offer Maya S, S24, M, and L builds through the existing GLM group.
Pin all weight shards and the vision projector to an HF revision.
Document the standard llama.cpp configuration and its limits.
Assisted-by: Codex:gpt-6
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
(cherry picked from commit 4aab47ba66, mudler/LocalAI#12666)
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Offer the Saluki IQ2 weights with an MTP head for llama.cpp. Pin both
assets and document how to select the variant.
Assisted-by: Codex:GPT-6
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
(cherry picked from commit c745b96781, mudler/LocalAI#12662)
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* ⬆️ Update ggml-org/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(llama-cpp): adapt to the multimodal rerank signature
Upstream llama.cpp added query_files and doc_files parameters to
format_prompt_rerank() (ggml-org/llama.cpp#29625). The gRPC rerank
call passed the init options in their place, so grpc-server.cpp no
longer compiled against the new pin.
prepare.sh now probes server-common.h for query_files and writes
LOCALAI_HAS_RERANK_FILES into llama_compat.h. When it is set, the
rerank call passes two empty media lists. The rerank API is text
only, so behaviour is unchanged. Forks that still use the old
signature (turboquant, bonsai) compile as before.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The chat settings sheet, the Talk session sheet and the agent and task
preview sheets set up their focus handling in an effect that depended on
onClose. Every parent passes onClose as an inline arrow, so each re-render
of the page gave it a new identity: the effect cleanup sent focus back to
the opener and the effect ran again and focused the Close button.
Any edit in a sheet re-renders the page, so focus left the control being
used. In the chat sheet, typing in the system prompt moved focus to Close
after the first key, and a following space activated Close and dismissed
the sheet; an open native dropdown in the sheet closed by itself.
Keep the latest onClose in a ref and run the focus effect once per open.
Escape still calls the current onClose; initial focus on Close, the Tab
trap and restoring focus to the opener are unchanged.
The e2e specs type key by key (or switch preview tabs) and assert that
focus stays put; they fail without this change.
Docs-Not-Needed: bug fix, no user-facing behaviour change beyond the fix
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit 773cae7259)
Agent jobs (/api/agent/jobs, local and on distributed agent workers) run
cogito against a LocalAI client that sends no max_tokens of its own, so
every completion gets the client's fixed default of 16384. max_iterations
bounds the number of turns and loop_detection repeated tool calls, but
neither stops one completion that keeps generating, and 16384 is the same
for a small tool-picking model and a reasoning model that thinks at length.
Add agent.max_generation_tokens to the model config. When set, both job
paths apply it to the client with SetMaxTokens; requests that carry their
own cap still win, so cogito's length retry can raise it. Unset keeps
today's behaviour.
The specs run each job path against an MCP fixture and a recording LLM
endpoint and check the max_tokens the completions carry; they fail when
either call site drops the cap.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit 5600a1cabe)
The vLLM and SGLang backends no longer stop at 200 tokens when a request
has no max_tokens. Generation is now bounded by the context window.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
- ApplyReasoningEffort keeps an operator's reasoning.disable for high, xhigh and max
- sglang: _configured_budget() feeds the strict-thinking warning; tested with
effort_budgets alone
Signed-off-by: pos-ei-don <1822533+pos-ei-don@users.noreply.github.com>
(cherry picked from commit 45a960ca1f)
- warn on an effort_budgets entry without a token count (high=)
- update comments: enable_thinking=false still wins over a listed level
- document that precedence in text-generation.md
- cover xhigh/max and the empty-entry warning in the tests
Signed-off-by: pos-ei-don <1822533+pos-ei-don@users.noreply.github.com>
(cherry picked from commit 0bd723a652)
The Go side already forwards the request's effort as
Metadata["reasoning_effort"], but the sglang backend only read
enable_thinking: every level got the same model-level thinking_budget,
so low/medium/high produced identical reasoning.
- New model option `effort_budgets:low=…;medium=…;high=…` picks the
budget of the request's level; unlisted levels and requests without an
effort keep `thinking_budget`. Parsing reuses _parse_thinking_budget.
- A level listed in effort_budgets turns reasoning on for that request,
so levels the Go side does not know cannot silently disable it.
- ApplyReasoningEffort now treats `xhigh` and `max` as thinking levels.
Before, they left DisableReasoning unset, so with a model default of
"off" asking for more reasoning produced none.
Signed-off-by: pos-ei-don <1822533+pos-ei-don@users.noreply.github.com>
(cherry picked from commit 879c686ab1)
Mock the request constructor so fake parser tests can run on older
vLLM profiles without ParserManager. Document the streaming version
requirement and fallback behavior.
Signed-off-by: pos-ei-don <1822533+pos-ei-don@users.noreply.github.com>
(cherry picked from commit ff0130fa89)
With a reasoning parser configured, the streaming path sent every raw
text delta as ChatDelta.content. LocalAI uses the backend's ChatDeltas as
they are, so with thinking enabled the reasoning arrived as
delta.content, and the final chunk then repeated the whole answer and
the reasoning a second time. Non-streaming requests were split correctly.
Use vLLM's composed parser (ParserManager.get_parser, vLLM >= 0.30), the
one vLLM's own OpenAI server streams with: one instance per request,
built with the request's chat_template_kwargs (enable_thinking), fed
each delta via parse_delta. Reasoning, content and tool calls go out in
their own fields as they are produced; the final chunk carries only
token counts and logprobs.
If the parser raises mid-stream, the rest of the output is buffered and
the final chunk sends what the full-text parse has beyond what was
already streamed (suffixes per field and per tool-call index). Where the
full parse disagrees with what went out, nothing is guessed and the
mismatch is logged.
Only active when a reasoning_parser is set; without one, or on a vLLM
without ParserManager, streaming is unchanged.
Signed-off-by: pos-ei-don <1822533+pos-ei-don@users.noreply.github.com>
(cherry picked from commit 5392eb813e)
Both Python backends start from a fixed default (sglang max_new_tokens=200,
vLLM max_tokens=200) and only replace it when the request carries Tokens.
OpenAI clients that leave max_tokens out (several agent frameworks do) were
silently truncated after 200 tokens, and the response still said
finish_reason "stop": the Go side only reports "length" against a budget
the client requested, so it cannot see a backend-side cap.
Use None instead, which allows generation up to the remaining context.
That is what sglang's own OpenAI server does when max_tokens is omitted
(the scheduler turns None into the remaining request capacity; dropping
the key would select SamplingParams' own default of 128), and vLLM treats
max_tokens=None the same way. A requested max_tokens still wins.
Tests check both cases and that the None default passes each library's own
parameter validation.
Signed-off-by: pos-ei-don <1822533+pos-ei-don@users.noreply.github.com>
(cherry picked from commit 5bdab54d97)
Accept response output text when clients replay it as request input.
Keep assistant history in both prompt content fields and preserve user
message boundaries when merging adjacent assistant messages.
Assisted-by: Codex:GPT-6
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
(cherry picked from commit 1cc5336aaa)
Restoring a snapshot after a failed stop lost the request lifecycle events
that happened while the stop was in progress. A request completing during
the stop found no tracked request, and the restore then resurrected it,
leaving the backend marked busy. A request starting during the stop was
overwritten by the restore, so completing an older request marked the
backend idle while the new one still ran, allowing an unsafe eviction.
An eviction now detaches only the backend identity (model, process, size
estimate) from the registry and marks the address as pending. Request
state stays live, so callbacks keep updating it during the stop. Idle and
busy checks skip pending addresses so they neither re-evict nor wipe that
state. When the stop settles, a backend that still runs is registered
again; otherwise its leftover request state is cleared. A replacement
registered on the address meanwhile is left alone.
Regression tests cover a request completing and a request starting inside
a failing force stop. The idle and busy failure tests now prove re-tracking
through a second stop attempt instead of sampling the registry, which
raced with the pending window.
Assisted-by: Claude:claude-opus-5.5 go vet golangci-lint
Signed-off-by: Hugues CHARBONNIER <hugues.charbonnier@gmail.com>
(cherry picked from commit 9a35647ec0)
An eviction untracks the backend before it stops it. When the stop failed,
the error was only logged: the backend kept running and kept its VRAM, but
the watchdog had already forgotten it. Group and LRU enforcement then saw no
conflict, and the next load started next to it and ran out of memory.
Stopping a model takes its exclusive lifecycle lock, which a model that is
still loading holds for the whole load. The stop then times out ("waiting to
shut down model: context deadline exceeded"), and the watchdog no longer
knows the model is there.
The tracking state is now captured before the backend is untracked, and put
back when the stop fails. This applies to every eviction path that untracks
first: group, LRU, idle timeout and busy timeout. A model that is already
gone (ErrModelNotFound) still counts as stopped, and a replacement
registered on the same address in the meantime is left alone. EvictedCount
only counts backends that really stopped, and NeedMore is set when one did
not, so the loader keeps retrying instead of treating the room as made. The
idle and busy checks retry the stop on their next tick.
The state is only put back while the loader still holds the model at that
address (ModelLoader.HasModelAtAddress). deleteProcess drops a remote-only
model, or the local half of a local+remote placement, before it reports a
failed remote unload; tracking that model again would leave a phantom entry
counted as loaded and as a group conflict. The memory reclaimer applies the
same rule and untracks a model the loader already dropped.
The untrack-before-stop ordering dates from #7535, so this is not a
regression of 4.12.0.
Assisted-by: Claude:claude-sonnet-5.5 go vet
Signed-off-by: Hugues CHARBONNIER <hugues.charbonnier@gmail.com>
(cherry picked from commit 97390f7754)
When --threads / LOCALAI_THREADS is unset, LocalAI uses the host's
physical core count. Inside a container with a CPU limit (docker --cpus,
a Kubernetes limits.cpu) that starts one thread per host core inside a
smaller CFS quota, so the backend is throttled for most of each period.
Since Go 1.25 the default GOMAXPROCS already follows the affinity mask and
the cgroup CPU limit. Use the smaller of the physical core count and
GOMAXPROCS as the default. Hosts without a limit are unchanged.
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: manunicholasjacob <manunicholasjacob@gmail.com>
(cherry picked from commit aca800d003)
The menu opens upwards from the sidebar footer and is a child of the
scrollable sidebar. On short windows it was taller than the space above
the trigger, so its top (the first entries) was clipped by the sidebar.
Cap the menu height to the viewport and let it scroll instead.
Signed-off-by: gangna2s-arch <281744444+gangna2s-arch@users.noreply.github.com>
(cherry picked from commit 9a564ef79d)
Describe stop strings, greedy decoding, and seed handling after sampling
parameters are validated during construction.
Signed-off-by: pos-ei-don <1822533+pos-ei-don@users.noreply.github.com>
(cherry picked from commit 05af7b072c)
_build_sampling_params constructed SamplingParams and then assigned the
request values with setattr. protobuf repeated fields (StopPrompts) stay
RepeatedScalarContainer that way, and __post_init__ never sees them:
- vLLM's argument checks assert `isinstance(self.stop, list)` and the
msgpack encoder that ships the params to the engine core cannot
serialise the container, so every request with `stop` failed
(HTTP 500, EngineGenerateError), streaming or not.
- the stop-string hold-back buffer (output_text_buffer_length) was never
set, so the start of a multi-character stop string could be streamed;
seed -1 was not mapped to None.
Collect the values, turn repeated fields into lists and construct
SamplingParams once, as vLLM's OpenAI server does. Values vLLM rejects
(e.g. a penalty out of range, an empty stop string) now fail with vLLM's
validation error instead of being set unchecked.
Signed-off-by: pos-ei-don <1822533+pos-ei-don@users.noreply.github.com>
(cherry picked from commit 48d11fcf74)
The sidebar and the Build overview show the Add-ons and Fine-tuning entries
only when the server reports those switches on. With no add-on owner
running, the server reports the add-ons switch off, and the test server
has the fine-tuning switch off, so the two specs failed on master.
The two tests are about those entries being shown, so they set the switches
on and keep the server's other flags.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The spec "refuses a platform the matrix does not enable" ran package.sh
with windows/amd64 and expected the text "not enabled". That text comes
from the matrix check. The tool now refuses windows/amd64 earlier, in
its format check, with: platform "windows/amd64" is not os/arch of
linux or darwin on amd64 or arm64. The tool still fails and writes no
layout, but the substring is gone, so the add-on packages run failed in
validate package rules.
The format check accepts only linux and darwin on amd64 and arm64, and
Tables lists and enables all four of them. No platform passes the
format check and is missing from the matrix, so the spec cannot reach
the matrix refusal. Assert that the output names the refused platform
and that it gives one of the refusal reasons of the tool. The not this
host check stays as it was.
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* feat(config): add a limit for concurrent backend operations
Add --max-concurrent-backend-operations
(LOCALAI_MAX_CONCURRENT_BACKEND_OPERATIONS) and the runtime setting
max_concurrent_backend_operations. It sets how many backend install,
upgrade and delete operations run at the same time. The default is 1,
which keeps the current behaviour. Values below 1 become 1.
The runtime value is also kept in an atomic, so the scheduler that reads
it never races with a settings change. The setting shows in the Settings
page under Backends. This commit only adds the setting; the next one
uses it.
Assisted-by: Claude:claude-sonnet-5-5 go-vet
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(galleryop): run backend operations in a bounded pool
One goroutine read both gallery channels and ran each operation inline,
so one slow backend operation held every operation behind it, model
operations included. In distributed mode the manager also visited the
nodes one at a time and waited for each up to the install timeout.
Backend operations now go through a scheduler that runs up to
max_concurrent_backend_operations of them at once. Operations start in
the order they arrive. Install, upgrade and delete of one backend share
a key and never overlap; an operation that waits for its backend does
not hold a slot, so independent operations pass it. Cancelling a queued
operation drops it at once. The limit is read on every scheduling step,
so a runtime change applies without a restart. With the default of 1 the
behaviour is the old single worker. Model operations get their own loop
and stay serial.
In distributed mode the manager sends one operation to its nodes in
parallel, up to the same limit, and keeps the result in node order. A
per (node, backend) lock keeps a live operation and the reconciler from
sending the same operation to a node together; the reconciler skips a
row that is locked and checks that the row still exists before it sends.
Timeouts are unchanged.
The download progress timers were plain package variables. They are
atomics now, because operations can run in parallel.
Assisted-by: Claude:claude-sonnet-5-5 go-vet go-test
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The workflow listed addons, pkg/addons, scripts/addons and
gallery/addons, but not tests/addons. The validate job runs the
packaging contract test in tests/addons/packaging through
scripts/addons/package_test.sh. A change to that test did not start the
workflow, so commit 48478a89c (#12884), which fixed a failing spec
there, was never exercised.
Add tests/addons/packaging/** to the push and pull_request path lists.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Add Q4 and Q8 GGUF builds with the F16 vision projector. Link the builds
as variants and document installation.
Assisted-by: Codex:gpt-6
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
⬆️ Checksum updates in the model library
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
* feat(grpc): add a Forecast RPC for time-series models
Add Forecast to the backend service with columnar request and response
messages: flat row-major arrays, NaN for missing values, and one message
per series so a caller can send series of different shapes in one call.
The response keeps the request order.
Wire the RPC through the server, client, embedded client, connection
evicting client, in-flight tracking client and the base backend, which
answers unimplemented. core/backend gains a Forecast helper that stamps
the model identity and records a trace, like the other single-purpose
RPCs.
Assisted-by: Claude:claude-sonnet-5-5 go-vet
* feat(config): add the forecast usecase
FLAG_FORECAST marks a model as a time-series forecaster. The timestral-cpp
backend declares it as its default usecase, so a config with that backend
needs no extra lines. The backend is also listed as a non-text backend, so
a forecast model is never offered as a chat model. It shows up in the
gallery filter and in the capability list.
Assisted-by: Claude:claude-sonnet-5-5 go-vet
* feat(api): add POST /v1/forecast
Take one or more numeric series and return quantile trajectories for the
next prediction_length steps. Targets are [variates][steps] or a flat
array for one variate; JSON null marks a missing value. Past and known
covariates and categorical codes are checked for length before the
backend sees them. Quantiles default to 0.1, 0.5 and 0.9 in the backend.
The route is gated by a new default-on forecast API feature. The endpoint
is listed in the swagger spec, the /api/instructions index, the UI model
filter and the capability symbols.
Assisted-by: Claude:claude-sonnet-5-5 go-vet
* feat(backends): add the timestral-cpp forecast backend
Wrap the timestral.cpp C ABI with purego, the way the other ggml
backends do: libtimestral is built from a pinned commit with ggml
embedded, and the Go binary loads it at start.
Forecast validates the counts against the array lengths, groups the
series by shape, makes one library call per group and puts the results
back in the order the caller sent them. Quantiles default to 0.1, 0.5
and 0.9, and a level the model does not have is refused with the list of
valid ones. A mutex guards the handle because the library is not
reentrant. Request size is bounded by two model options.
The unit specs use a fake library. The integration specs and a gRPC
spec run against the 5M F32 model that test.sh downloads and checks
against a fixed SHA-256.
Assisted-by: Claude:claude-sonnet-5-5 go-vet
* build(backends): register timestral-cpp in the build and the library
Add the CPU image to the backend matrix for amd64 and arm64, plus an
Apple silicon entry that is not built or tested yet. The image has no GPU
keys because the engine runs on the CPU only. Wire the make targets, the
path filter and the dependency bump bot.
The library entries were generated with backendcatalog. The images are
not published yet, so the release entries are unpinned until a
maintainer runs --repin. The gallery reader test now counts the names in
the library instead of a fixed number, so adding a backend does not
break it.
Assisted-by: Claude:claude-sonnet-5-5 go-vet
* feat(gallery): add the Timestral forecast models
Add Timestral 5M, 28M, 123M and 209M as F32 entries, each with its F16
build as a variant. Q8_0 and the K-quants are left out until the quality
question is settled. Each entry runs on the timestral-cpp backend, on
the CPU only.
The weights use Mistral's Modified MIT licence, which bars use by
companies above a monthly revenue threshold. The first paragraph after
the summary says so, the licence file is installed next to the weights,
and the revenue-restricted-license tag makes the entries filterable.
The memory each build needs has not been measured, so no size is set.
Assisted-by: Claude:claude-sonnet-5-5
* docs(forecast): document /v1/forecast and the Timestral models
Add a feature page with the request and response shapes, the error
cases, the model options and the licence note. List the engine in the
README, the compatibility table and the backend categories.
Assisted-by: Claude:claude-sonnet-5-5
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
test(ui): stub the add-on routes in the add-on e2e fixtures
The add-on owner now runs as its own workload. With no owner running the
server reports the add-ons feature off and answers 5xx on
/api/addons/builtin and /api/addons/pins. The add-on pages are gated on
that flag, and the navigation shell calls both routes, which the browser
logs as failed requests. Specs that draw add-on pages assert no console
errors, so they failed on master.
Stub those routes on the browser context: the feature flag is switched on
with the server's other flags kept, and the pin routes answer with an empty
set. A spec's own page routes still take precedence. Studio's old-address
and 404 specs call the same stub, since they expect add-ons to be on. The
server behaviour is unchanged.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Tables now enables the four platforms the CI hosts run on (#12879), so the
spec that expected the host platform to be refused as not enabled failed on
the add-on packages run for 88aa56518. The refused platform is now one the
matrix never lists.
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
A development add-on entry now names a moving tag, sets mutable and has
no digest, the same shape a development backend entry has. A release
entry stays pinned by digest.
One helper, catalog.Pin, turns a mutable entry into a digest at install:
it resolves the tag, checks the digest form and builds the version from
the first 12 hex digits. The backend library installer and the add-on
installer both call it. The add-on installer then verifies the signature
against the resolved digest and records that digest in the receipt. Kind,
platform and install label checks stay in each caller.
The official index lists the development entries as <id>-development,
which the publish workflow already moves in its promote job. The catalog
command renders the development channel as mutable entries. The update
check reports no state for a mutable entry until it resolves the tag too.
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
All add-ons now publish to ADDON_OCI_REPOSITORY as tags of one
repository. Each tag starts with the add-on id: <id>-sha-<short sha> for
a commit, <id>-<version> for a release, <id>-development and
<id>-latest for the channel aliases.
The publish step, the alias promotion and the catalog renderer add the
prefix, so the per-add-on repository step of the workflow is gone. A new
addonpkg target command prints the reference an add-on publishes to
without a network request. The workflow policy now refuses a derived
repository.
The host parser already reads a tag reference and a repository with a
nested path, so both the new form and the old per-repository form
install. A test covers each.
The artifact fields in gallery/addons/index.yaml were changed by hand to
the tag form. The digests are unchanged and stay until the add-ons are
published again.
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The plan job of the add-on package workflow runs the platform matrix
for every add-on. On a publishing event it fails closed when an add-on
has no enabled platform. The Tables platform file listed three
platforms, all with enabled: false, so every master push failed
the plan and nothing was published or signed.
Enable the same four platforms as the other first-party add-ons
(linux/amd64, linux/arm64, darwin/arm64, darwin/amd64), each with its
native runner and the executable-start check. The fail-closed check in
the workflow is unchanged.
The tests assumed Tables was disabled. The matrix spec now requires at
least one enabled platform in the checked-in Tables file. The platform
spec covers Tables like the other add-ons. The shell suite builds a
disabled copy of the Tables file for the two fail-closed cases.
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The add-on Browse list showed backend entries next to add-ons, with
their own install flow. Backends belong on the Backends page, where
the library chips already cover them.
Ask the library for add-ons only and drop any entry that still says
it is a backend. Entries with no kind stay add-ons. Remove the
backend-only install flow from the Browse view.
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The disable-backend-install switch (#12754) makes InstallBackend and
UpgradeBackend read d.systemState.DisableBackendInstall. The test
fixtures build the manager as a struct literal or through
NewDistributedBackendManager with an empty ApplicationConfig, so
systemState is nil and every install and upgrade spec panics with a nil
pointer dereference.
Set a zero-value SystemState in each fixture. That keeps installs
allowed, which is what these specs expect.
Assisted-by: Claude Code:claude-haiku-5-5
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): generate and load a static carrier key set
A split deployment needs the owner and the frontends to share
certificates and a signing key that outlive a process. The loopback
carrier makes fresh keys at every start, so it cannot serve that.
GenerateCarrierKeys writes a CA certificate, an owner leaf and a
frontend leaf, and the frontend signing key pair. It drops the CA key,
so nobody can issue a third leaf, and it refuses to overwrite any file.
LoadCarrierConfig reads only the files of one role and refuses key
files that group or other users can access.
The owner never forwards, but NewCarrier still requires a signing key,
so the owner config gets a throwaway key in memory instead of the
frontends' private key.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): add util addons-carrier-keys
An operator who runs the add-on owner and its frontends as separate
workloads needs a way to make the shared key set. The command wraps
GenerateCarrierKeys: --out names the directory and --owner-host, which
can repeat, sets the names the owner certificate is valid for. It prints
which files belong on the owner and which on the frontends.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): add the owner and frontend roles to local-ai run
Every local-ai run process is its own add-on owner, so a distributed
deployment cannot scale its frontends: replicas that share a data
volume fight over the owner lock.
--addons-role selects combined (today's behaviour, the default), owner
or frontend. The owner and frontend roles also take --addons-owner-address
and --addons-carrier-dir, and the owner takes --addons-owner-listen. They
use the fixed carrier identities and the key set from the carrier key
directory. The frontend role has no add-on root, so it never creates the
add-ons folder. A bad role setup stops startup with a message that names
the flag.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): keep the frontend role away from owner state
A frontend holds no add-on state, but it still wrote the library
sources file of the data path, which is the owner's file. The frontend
role now skips the library option and reads only the galleries named by
the flags, and logs a warning when it has no account database, because
its pins are then kept per frontend.
The add-on manager rejects a relative root even for a forwarding
frontend, so the frontend config names the absolute path where an owner
would keep its state. Nothing creates or opens it, and a spec checks
that the data path stays empty.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* test(addons): run the owner and frontends as separate applications
The split roles are only useful if an owner and its frontends work as
separate processes that share nothing but the carrier keys and the owner
address. A spec now builds them as separate Application values with
their own data paths and keys from GenerateCarrierKeys.
It covers a frontend that lists apps and reads add-on status through the
owner, two frontends on one key set, a frontend that starts while the
owner is down and answers unavailable, a second owner refused on the
same root, and an owner that restarts on the same root and address and
still serves the installed add-on to the same frontend.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(addons): describe running the owner as its own workload
An operator who runs several frontend replicas had no documented way to
stop each one from being its own add-on owner. The add-ons page now has
a section for the owner and frontend roles: one owner replica with its
own volume and a recreate-style rollout, a stable owner address, key
generation and per-role mounts, a Compose example, what a restart and an
owner outage look like, the models-path limit, and the frontend limits.
The singleton section names the roles and the distributed-state mode,
and the distributed mode page links to the new section.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): gate the addons feature in /api/features
One function, Application.AddonsFeature, decides whether the add-on
side is on. A combined or owner process answers true when add-ons are
enabled and the owner manager exists. A frontend answers true when the
owner answered the carrier's audience call within 2 s, and keeps that
answer for 10 s so /api/features does not dial the owner per request.
/api/features reports the result as "addons", and fine_tuning and
quantization take the same value, because both are add-ons. The
frontend cache is per-instance state.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): answer 503 on every add-on route when add-ons are off
With --addons-enabled=false the real add-on, fine-tuning and
quantization routes and services are not registered. One catch-all per
prefix answers the typed unavailable response of an absent owner, for
any path under it, so a client cannot tell which routes exist. The
response is the one the gateway already writes (gateway.WriteUnavailable
over typedError), not a second format.
With add-ons on, the same routes sit behind RequireAddonsFeature, which
reads Application.AddonsFeature, so a frontend that cannot reach the
owner gives the same answer. The per-user feature middleware still runs
after the gate. No endpoint is added, so the swagger, /api/instructions
and RouteFeatureRegistry surfaces do not change.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): build no carrier for a frontend with add-ons off
addonConfigFor already returned the zero config before it read a role
flag when add-ons are disabled. Say so on the way out: when a role,
owner address, listen address or carrier directory flag is set, log one
info line that the flags are ignored. A frontend with
--addons-enabled=false needs no owner address and no key directory,
loads no keys and builds no carrier.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(ui): hide add-on entries when the addons feature is off
The UI reads features.addons from /api/features. When it is false:
the sidebar shows no Add-ons entry and no pinned apps, the 404 page
lists no add-on destination, the shell does not call /api/addons/pins
or /api/addons/builtin, and /app/addons, /app/addons/*, /app/fine-tune
and /app/quantize lead back to /app. The Build hub tabs for add-ons,
fine-tuning and quantization were already keyed on features and now
follow the gate. An absent key (an older server) counts as on.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(addons): describe the addons feature gate
The --addons-enabled row now says the UI hides the add-on entries and
every add-on, fine-tuning and quantization route answers 503. A new
subsection describes the addons key of /api/features, the 10 s frontend
cache as per-instance state, and that a frontend with add-ons off needs
no owner address or key directory. The fine-tuning and quantization
pages say they are off with --addons-enabled=false.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(addons): give a frontend no add-on state root
OwnerConfig.validate required an absolute root for every role, so a
frontend named a path under the data directory that nothing was allowed
to open. Only the owner has a state root now: validate checks it when
the instance is the owner, and the frontend role leaves Root empty.
Nothing on a frontend reads or creates it.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(addons): make the carrier signing key optional for the owner
NewCarrier required an Ed25519 signing key of every instance, so the
owner role loaded a throwaway key in memory that it never used. The
owner never signs an outbound request, so it now needs none:
NewCarrier accepts an empty key when the instance is the owner and
still requires a whole key from a frontend, and a malformed key is
refused for both. LoadCarrierConfig no longer generates one. The owner
still verifies frontend signatures with frontend-sign.pub.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* Keep the owner address out of the add-on identity file
The owner address is runtime configuration, so storing it and comparing
the whole file made a moved owner refuse its own root. The identity file
now holds the owner ID and storage ID only. An older file that still has
an address loads, and the address is ignored. A different owner ID or
storage ID is still refused.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(addons): state that an existing root loads under the owner role
The owner identity no longer stores its address, so a root written by a
single node loads under the owner role when the owner and storage
identity match. Say so, and keep the caveat about absolute paths in the
journal.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(addons): refuse library-source routes on a frontend
A frontend must keep no owner state, but the library-source routes, the
ApplyLibraries call and the startup official-libraries step still opened
and wrote addons/sources.json in its data path.
Mark the frontend role in the add-on config, also when add-ons are off.
On a frontend, LibrarySources returns the unavailable error,
ApplyLibraries does nothing and the startup step skips the store. The
six /api/library/sources routes answer 503 with the body that the add-on
feature gate writes, and an X-LocalAI-Reason header names the cause.
Reads are refused too: the carrier has no verb for them. Owner and
combined roles are unchanged.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(addons): put the reason in the unavailable body
WriteUnavailable wrote the plain text "addon request failed", and the
library-source refusal put its reason in an X-LocalAI-Reason header.
Write the LocalAI error envelope instead, with a message, a type and the
code "unavailable", and keep the 503 status. The message names the
cause: add-ons are off on this instance, the add-on owner is not
reachable, or library sources are managed on the owner. Remove the
header.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): serve library calls on the add-on owner
Add the carrier verbs a frontend needs to work on the library through
the owner: library-list, library-add, library-update, library-enable,
library-disable, library-remove, library-backend-resolve and
library-backend-releases. A call carries the operation in one
LibraryCall field of the wire request, so an old owner rejects it as an
invalid binding. Every verb but the three reads is a mutation, and the
reply is an envelope of data or a typed error, so a frontend gives the
status the owner's own routes give.
The owner serves them through gateway.LibraryAdmin, implemented in the
application on the same LocalAdmin as the owner's HTTP routes. It runs
the owner's administrator check before any store call.
Add two peer-authenticated reads that take no actor and leave the
audience reply unchanged: GET /addons/capabilities (protocol, calls and
the owner's OS and architecture) and GET /addons/library/snapshot (the
gallery-relevant source list and its version).
Split the backend install: Installer.Resolve is the owner's half
(resolve, pick the meta backend for the asker's capability, pin by
digest) and returns a Plan; the node that installs runs the collision
check and the gallery install itself. Move the source administration
behind library.Admin and add library.ReconcileInfos for a source list
read from the owner.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): forward library routes from a frontend to the owner
A frontend refused the library-source routes and did not serve the
backend library routes. Forward them instead, and keep the owner the
only writer of the sources.
The library-source routes work on library.Admin: the local store on a
combined or owner process, the carrier on a frontend. The frontend
checks the calling account (a legacy API key is refused with 401), and
the owner checks the administrator right again before any store call.
With add-ons off, a frontend answers these routes with a 503.
A frontend asks the owner for its source list every 60 seconds through
the peer-authenticated snapshot call and rebuilds its model and backend
gallery lists, and the official-libraries switch, only when the version
changed. If the owner does not answer, it keeps the last lists and marks
them stale. Changing official_libraries on a frontend is refused with
409.
The backend library install is split. The owner resolves the entry and
pins the image; the frontend runs the gallery install on its own node,
with its own size limit and strict mode. A frontend refuses library
calls to an owner that does not serve them, or that runs on another OS
or architecture.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(addons): document library operations on a frontend
Replace the frontend limits that said a frontend refuses the library
routes with what happens now: the library sources and the backend
library go through the owner, the gallery lists follow the owner every
60 seconds, official_libraries is set on the owner, and the owner and
the frontends must run on one OS and architecture.
Add specs for the split owner: a frontend lists sources through the
owner, a source added on a frontend reaches the owner and then the other
frontend after a refresh, a write with the owner down answers 503 with
the error envelope, official_libraries answers 409 on a frontend and a
change on the owner reaches the frontend's gallery lists, the lists are
kept and marked stale while the owner is down, a platform mismatch and a
legacy API key are refused, and a frontend installs a backend into its
own path from the owner's plan.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): check the shared models path with a probe
Exports write into the models path of the owner, and frontends load
models from their own path. A wrong mount fails silently: the export
succeeds and the model never appears.
The owner writes a probe file with a random nonce into its models path
at start. A frontend fetches the nonce over the carrier and checks at
start, then every 60 seconds, that the same file is visible at the same
absolute path. On a mismatch it logs an error, reports model_exports
false and blocks its export and import routes. The owner refuses an
export or import while its own probe is missing, and keeps running if
it cannot write the probe. An unreachable or older owner gives
"unknown" and leaves exports on.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(api): describe the library refusal and forwarding
The library-source and backend-library routes forward to the add-on
owner on a frontend and answer 503 when the owner cannot serve them.
Add the 503 to their swagger annotations, regenerate the swagger files,
and say in /api/instructions that a frontend forwards these routes.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): let frontends run the backend update check through the owner
In named-user mode a frontend has no user to act as, so the background
backend update check failed there. Add one read-only system call for
the release lookup. The owner accepts it only on the peer-authenticated
route from a pinned frontend, only for backend-releases, and without an
actor. Calls through /addons/forward cannot carry the mark, and every
user-facing call still goes through the administrator check.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(mcp): expose the backend update check as a tool
Add list_backend_library_updates, a read-only tool for
GET /backends/library/updates. The in-process client reads the
library installer, and the HTTP client calls the route. The six
library source routes stay out of the catalog: the assistant has no
per-call actor and no approval card for library changes.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(api): add export status routes for fine-tuning and quantization
The add-on UIs could not know that model exports are blocked until a user
hit the refusal of the export route. Add GET /api/fine-tuning/status and
GET /api/quantization/status. Each returns {"model_exports": bool}, from
the same check that drives the model_exports key of /api/features.
The routes sit behind the add-ons gate, so they answer 503 when add-ons
are off. They are listed in the swagger spec, the instructions registry,
the route-feature registry and the feature docs.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(addons): disable exports when the models probe fails
The fine-tuning and quantization pages now read the new status route
and turn off the export form, the checkpoint export button and the
import button when model_exports is false. A short reason shows next
to the control. A failed or slow answer leaves the controls on, as the
server does for an unknown check.
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* test(cli): use a globally routable IPv6 address in the public bind spec
2001:db8::/32 is the IPv6 documentation prefix and is now classed as
non-public, so the bind is trusted and the spec failed. Use a public
address from 2606:4700:4700::/48 instead.
* fix(addons): rename installFlow.js so it no longer clashes with InstallFlow.jsx
installFlow.js and InstallFlow.jsx differ only by case. On a
case-insensitive filesystem the bundler can resolve
`import FlowStatus from './InstallFlow'` to the .js module, which has
no default export, and the macOS build fails with MISSING_EXPORT.
Rename the module to installAndTurnOn.js, after its main export, and
update its two importers. No code changes.
Assisted-by: Claude Code:claude-sonnet-5-5
* fix(addons): keep Reload on the page of an add-on that is turned off
The clearer empty states for a stopped add-on dropped the Reload button
from the unavailable page, so a user who turned the add-on back on had
no way to mount it again without leaving the page. Render the button in
the actions of the off state, next to Turn on.
* test(ui): serve the Voice add-on in the diarization profiles spec
The remembered voices link now opens the page of the Voice add-on, which
the core build no longer draws. Install the built Voice bundle next to
Studio, as the other specs do for their add-ons, and build it in the
global setup.
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
