* chore(deps): bump cogito to v0.11 ahead of the nib harness
nib is the agent harness that becomes 'local-ai chat'. It requires cogito
v0.11, so pull that bump forward on its own: minimal version selection would
apply it to LocalAI anyway, and both repos use cogito and cogito/clients.
Landing it separately keeps the harness change reviewable.
nib itself is not pinned yet. Nothing in LocalAI imports it, and 'go mod
tidy' runs as a goreleaser before-hook in CI, so an unimported require line
does not survive. It lands with its first importer.
No LocalAI call site needed a change. Both cogito.WithMaxAttempts callers
guard the argument above zero, so v0.11's new clamp is unreachable, and
LocalAI's Multimedia values implement only URL(), so v0.11's new
TypedMultimedia routing treats them as images exactly as v0.10 did.
Binary size (cmd/local-ai): 200,301,381 -> 200,336,045 bytes (+34,664).
A throwaway probe that links nib measured 201,042,243 bytes (+740,862 over
the pre-change baseline).
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(chat): resolve and seed the agent state directory
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(chat): write the agent config atomically and tighten its modes
Replacing config.yaml in place truncated it first, so an interrupted write
would have destroyed the api_key nib keeps in the same file. Stage through a
sibling temp file and rename over the target instead, and match nib's 0700
directory mode.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(chat): probe the endpoint and classify failures
Probe lists what a LocalAI endpoint advertises and separates the two
failures that need different advice: nothing listening, and rejected
credentials.
go-openai reports a rejected key as one of two concrete types depending
on the error body, and both occur against a real LocalAI. The normal
error handler sends an OpenAI error envelope, which arrives as
*openai.APIError; the opaque-errors handler replies with a bare status
and no body, which arrives as *openai.RequestError. Classifying on only
one of them misses half the cases, so the status is read from either.
A cancelled probe is not reported as an unreachable server, because it
learned nothing about the endpoint, and neither is a reply that could
not be parsed, because something did answer. Both would otherwise send
the user off to start a server that may already be running.
The model list is returned verbatim and in server order. LocalAI lists
whatever it finds in the models directory, including stray archives and
dotfiles, and deciding which advertised ids are real belongs to whoever
presents them.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(chat): resolve the model from flag, config, or the server
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* test(chat): pin that model resolution sorts a copy of the caller's slice
The sort spec asserted only on what the chooser was offered, so replacing the
defensive copy with an in-place sort of req.Available still passed all 37
specs. Assert the input slice's order after the call, so the guarantee cannot
be dropped silently by a later refactor.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(chat): offer to start a server when none is reachable
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(chat): bound the server wait and pin readiness and stop semantics
Set cmd.WaitDelay so a backend subprocess holding the child's stderr pipe
cannot block cmd.Wait forever, which would leave exited unclosed, burn the
whole shutdown grace on a clean exit, and leak the waiter goroutine.
Two test gaps closed alongside it: the readiness spec now counts polls, so
treating 503 as ready is observable, and Stop's single-interrupt contract is
pinned by giving StartedServer interrupt/kill hooks that a spec can count.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* refactor(chat): drive Stop through one process interface, hide exec plumbing
Two independent interrupt/kill func fields plus a nil check admitted wirings
no test could distinguish: the pair swapped, so a SIGKILL would strand the
backends SIGINT exists to let local-ai run clean up, or kill left nil, so a
wedged server never escalates. One two-method interface that *os.Process
already satisfies leaves nothing to swap and nothing to nil.
Also translate exec.ErrWaitDelay, whose text names an os/exec struct field,
into what the user can act on. os/exec only substitutes that sentinel when the
process exited without an error of its own, so no exit status is swallowed.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(chat): replace the REPL with the built-in agent
local-ai chat is now the nib agent harness compiled into the binary: tool
use behind an approval gate, sub-agents, MCP, plugins, and skills, all
auto-configured against the local server.
The REPL goes with it. Its model listing and its 401 classifier were
duplicates of the ones Probe now owns, and the classifier was the version
that misreads a bare 401 with no OpenAI error envelope, so keeping either
would leave the package with two divergent answers to the same question.
github.com/mudler/nib lands in go.mod in this commit rather than earlier:
go mod tidy runs as a goreleaser before-hook on every PR, so a require
line with no importer is stripped before it reaches CI.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* refactor(chat): split the pre-agent phase out of Run and pin it
Everything before the handoff is testable and nothing after it is: once
app.Run owns the terminal there is no seam left. prepare draws that line,
takes interactivity as a parameter so the prompts can be driven over a
pipe, and hands Run the state dir, the model, and any server it started.
The questions move onto one prompter that owns its buffered reader. A
fresh bufio.Reader per question reads ahead and discards what it buffered,
so the model choice typed behind an answer to "start a server?" was lost
and the next question saw EOF.
choose answers with a list index and refuses an empty offer, so a value
that was never on the list cannot reach ResolveModel, which persists it
and starts every later run against it.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(chat): bound each server check with a deadline
Nothing bounded the model listing, so pointing chat at an address that
accepts the connection and then never replies left the user with no
output and no offer to start a server.
The budget is context.WithTimeout rather than a cancel plus a timer.
Probe deliberately refuses to call an endpoint unreachable on a
context.Canceled, since a caller who gave up learned nothing about the
server, and only honours a deadline. A cancel-based budget therefore
expires as the one error that suppresses ErrUnreachable, exactly for the
hung servers the offer exists to rescue.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(chat): tell the user when their model choice cannot be saved
The choice is meant to be asked for once. When saving it fails the user
is silently asked again on the next run, and the only trace was an
xlog.Warn: the agent runs at log level error, and a --log-level=error run
swallows it entirely.
ModelRequest gains Notify for exactly this class of problem, one that is
worth telling the user about but not worth failing over, and the chat
wiring points it at the same writer the question was asked on.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(chat): stop a session's server when the process is signalled
A server started for the session is stopped by a deferred call, and a
signal skips deferred calls: a SIGTERM between the spawn and the exit
left 'local-ai run' reparented to init with nothing left that knew to
shut it down. Ctrl+C was already safe, but only incidentally, because the
child shares this process' foreground process group.
A signal handler rather than Pdeathsig on the child. Pdeathsig is
Linux-only and, in Go, is delivered when the OS thread that forked exits
rather than when the process does, so it can fire on a healthy parent.
Setpgid would break the Ctrl+C that works today by taking the child out
of the foreground group.
SIGHUP joins SIGINT and SIGTERM: a terminal program whose terminal is
gone has nobody left to talk to. The same context is what cancels the
agent, which nib leaves to its embedder.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(chat): only skip the server checks for work that stays local
Two argument shapes were classified wrongly. Every 'mcp ...' invocation
counted as management, so 'local-ai chat mcp --stdio', which serves the
agent over MCP and needs a model like any other session, was handed an
empty one. And --init, whose shell snippet a user pastes into an rc file
long before any server exists, went the other way: it demanded a running
server to print a static string.
The mcp split is asked of nib's own IsMCPManageSubcommand rather than
restated here, so a verb added upstream cannot drift out of this list.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(chat): exit with the agent's status instead of reporting it twice
nib writes what went wrong to stderr and returns nothing but an exit
code, so returning that error unchanged had main log "Error running the
application error=exit status 1" underneath the message the user had just
read. The refusal to render the full-screen interface into a pipe is the
one they meet in practice: it names --cli, and burying that hides the fix.
ExitCodeError says "already reported, exit with this status". main
honours it and prints nothing more, so a piped or redirected chat still
fails a script the way it should.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* style(chat): route interactive chatter through one writer helper
The prompts and notices all write to a terminal, where a failed write is
not worth failing the session over and the read that follows the question
reports the real problem. say says that once instead of five discarded
error returns.
The command's one-line help comes along: chat is no longer "an
interactive chat session".
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(chat): record why the agent gets this process' streams
Injecting them is what makes nib refuse to draw its full-screen interface
into a pipe and name --cli, instead of rendering onto a terminal the
caller may not own. The tradeoff is worth stating where the wiring is.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(chat): stop the session's server on cancellation, not on the way out
The deferred Stop is reached only if the agent returns, and cancelling
the context does not make it: nib hands the TUI to bubbletea without the
context, so what actually unwinds a running session today is bubbletea's
own SIGINT and SIGTERM handler. SIGHUP has no such backstop, and
registering for it removed the default disposition that used to end the
process outright, so kill -HUP left a live TUI with a cancelled context
and the started server still running.
runSession watches the context alongside the agent and stops the server
the moment it is cancelled, so the guarantee no longer depends on what
the agent does with cancellation. Stop is idempotent, so the deferred
call stays correct and free.
The doc comment on shutdownContext described the mechanism it was
supposed to work by rather than the one that does. Corrected, bubbletea's
handler included.
ResolveModel now checks the chooser's answer against what it offered.
The shipped chooser answers by list index and cannot be wrong, but
ModelChooser is exported, the answer is persisted, and every later run
starts against it, so the invariant belongs at the consumer.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* chore(chat): bump nib to v0.5.1
v0.5.1 carries four fixes that matter to 'local-ai chat':
- --init now names the embedder's command, so the emitted widget invokes
'local-ai chat' rather than a bare 'nib' the user does not have.
- A piped CLI session that succeeds exits 0 instead of failing with EOF.
- EOF at a tool-approval prompt denies the call rather than approving it,
and the session exits 3 (app.ExitCodeApprovalNoInput) so a script can tell
"answered" from "refused to act" without reading stdout. Read-only tools
are unaffected and still run. ExitStatus already unwraps app.ExitError,
so the code propagates with no change here.
- RunTUI passes the context to bubbletea and gives up bubbletea's own signal
handler, which makes shutdownContext the single owner of the signal and
stops a SIGHUP leaving a wedged TUI behind.
Verified against a live server on 127.0.0.1:8080: the three --init shells,
a piped prompt exiting 0, a denied 'touch' that left no file and exited 3,
a read-only 'ls' that still ran and exited 0, and a SIGHUP that unwound a
TUI running under a pty.
Two comment blocks in run.go described the old TUI behavior and are now
wrong, so they are corrected in the same change. No behavior change: both
shutdownContext and runSession are untouched, and stopping the server on
cancellation is still worth keeping independent of how promptly nib unwinds.
One known gap, not addressed here. The widget --init now emits runs
'output=$(local-ai chat --height 50%)', and runAgent injects Stdout
unconditionally, so under $(...) nib refuses the TUI for a non-terminal
stream. This is the cost the runAgent comment already anticipated, now that
the snippets no longer hardcode standalone nib. Ctrl+Space should not be
documented until that is decided.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(chat): let nib own stdout, so the Ctrl+Space widget works
The widget 'local-ai chat --init' emits runs
'output=$(local-ai chat --height 50%)', which puts a pipe on stdout by
construction. runAgent injected os.Stdout unconditionally, and nib refuses
every mode but --cli when a stream it was handed is not a terminal, so
Ctrl+Space printed "Re-run with --cli to use the injected streams" and
inserted nothing. Verified against a pty before and after.
nib reads a nil stream as "not injected" and falls back to the process
stream, which is how an embedder asks for nib's own behavior. That is what
stdout needs: the interface renders on /dev/tty but writes the chosen
command to stdout even when stdout is a pipe, and that write is the whole
of the shell-capture idiom.
Stdin is deliberately left injected. A piped or redirected stdin really is
ignored by the interface, so the refusal is the honest answer there, and it
is the one users meet: 'echo q | local-ai chat' still says to re-run with
--cli, once, exit 1. Nilling stdin the way stdout is nilled would delete
that silently. Stderr is not gated by nib at all and is unchanged.
One case does change and cannot be kept: 'local-ai chat > out.txt' from a
terminal no longer refuses, because it is indistinguishable from the
widget. It renders on /dev/tty and writes the capture line to the file,
which is what standalone nib does.
The app.Options literal moves into agentOptions so the decision is
reachable from a spec rather than being a detail of a function that takes
the terminal. Both sides of the asymmetry are pinned: reinstating
'Stdout: opts.Out' fails "hands nib nothing for the process stdout", and
nilling stdin fails "hands the process stdin over".
Also rewrites the last comments describing the pre-v0.5.1 behavior.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(chat): say what the stream refusal actually keys on
Two comments still called it the refusal to render the interface "into a
pipe". That was true when both stdin and stdout were injected, but a pipe on
stdout no longer refuses, so the wording now points at precisely the case
that was un-refused to make Ctrl+Space work. Only a stdin that cannot be
read triggers it, and both comments now say so and name the command a user
meets it with, 'echo q | local-ai chat'.
The agentOptions doc also said a "file a caller chose" stays injected and
refused, which reads as though 'local-ai chat > out.txt' still refuses. It
does not: a shell redirect arrives as os.Stdout and is nil-ed like the
widget's pipe, because the two differ only in being a regular file rather
than a FIFO and nib's gate does not look at that. What stays injected is a
writer an in-process caller chose for itself. Says that now, in the doc and
in the spec comment that had the same ambiguity.
Comments only. No behavior change.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs: local-ai chat is now the built-in terminal agent
`local-ai chat` was a plain chat prompt and is now an agent that runs
shell commands behind an approval gate, so the pages that described a
REPL were wrong rather than merely thin.
Adds a Terminal agent feature page at /features/terminal-agent covering
the approval gate, piped runs and their exit codes, Ctrl+Space, model
resolution, state directory, and the pass-through management commands
(including the `--yes` caveat that leaves a plugin installed but
disabled in a script).
The three-way "looking for something else" notice becomes four-way and
moves into an agentic-routing shortcode. Four hand-kept copies of the
same paragraph is what produced the drift the new page would otherwise
have added to; the shortcode takes `current=` so each page still marks
itself, and errors the build on a name that is not one of the four.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* website: the agent is in the binary, not a second install
The nib section sold a separate tool you also install, with a GitHub
link as the only way in, which is now the wrong order: the agent ships
compiled into local-ai, and the standalone binary is the second reason
to care rather than the first.
Leads with `local-ai chat`, keeps nib as the SSH-anywhere story, and
adds a docs CTA pointing at the new Terminal agent page. id="nib" is
left alone because localai.io/#nib is linked from outside.
The two credits on the demo clip named nib as the thing that drove the
machine; they now credit the agent in LocalAI, which is the same agent.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* website: fix the exit keys, the plugin warning, and the redirect gap
Three claims on the chat-agent pages that the code does not back.
try-it-out told readers to press Ctrl+D. nib has no Ctrl+D handler: the
full-screen interface quits on Esc or Ctrl+C, and Ctrl+D is only an exit
in --cli, where it arrives as ordinary tty EOF. That sentence had
replaced the removed /exit and /quit text, so the page was left with no
working way to leave a session. Document both modes, since they differ.
The plugin warning said nothing tells you the install stopped short. It
does: the command prints that the plugin was left disabled. What it does
not do is say so in its exit code, which is 0 either way. That is the
part a script cannot work around, and it is the reason to pass --yes.
Overstating it in the paragraph that gives the advice only makes the
advice easier to dismiss.
Redirecting stdout no longer refuses; the interface goes to /dev/tty and
only the yanked command reaches the file. It is what lets the Ctrl+Space
widget capture a command at all, since a redirect and out=$(...) are the
same thing to the stream gate. It was documented nowhere. A non-terminal
stdin is still refused, and the new text says which of the two it is.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(chat): make the CLI flags outrank the agent config file
local-ai chat routed --endpoint, --model, --api-key, --trace-dir and --yolo
through nib's app.Options.Defaults. Defaults are seeds: they sit beneath the
config file, so the file silently undoes them. That made the flags accepted and
inert, and not in an edge case, since EnsureStateDir writes base_url on the
first run and the interactive picker writes model, so from the second run on
the file carried a value for both.
Observed against a live server: with base_url: http://127.0.0.1:9999/v1 in the
config and --endpoint http://127.0.0.1:8080 on the command line, the probe hit
8080 and every agent turn posted to 9999. With model: gemma-4-e2b-it-qat-q4_0
in the config, --model lfm2.5-8b-a1b was ignored on the wire.
nib v0.6.0 adds app.Options.Overrides, applied above the config file and above
the bare environment block. Move the whole block there: all five values are
decisions this invocation already made on the user's behalf, and a flag the
config file can undo is not a flag. Nothing is left in Defaults, because
LocalAI's one genuine seed, the initial base_url, is written into the config
file by EnsureStateDir rather than handed to nib.
Two limits come with the channel and are documented on agentOptions rather than
worked around. An override can only raise a field, since nib cannot tell "set
to the zero value" from "not set", so --yolo can turn approval off but nothing
on the command line turns it back on over an approval_mode: auto in the file.
And nib's own NIB_TRACE_DIR and NIB_YOLO are resolved after the config load and
still outrank these, deliberately, upstream.
The existing spec pinned that the right values reach app.Options, which they
always did, which is exactly why it could not see nib discarding them. The new
specs resolve the config the way app.Run resolves it, against a real config
file that disagrees with every flag, and one asserts Defaults stays empty.
docs/content/features/terminal-agent.md already documented --model as winning
over the saved model; that was false before this change and is true now, so no
docs edit was needed.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(chat): document intentional config file read
Assisted-by: Codex:gpt-5 [gosec]
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The post was written against the first draft of the release notes, when the
cycle stood at 214 PRs over thirteen days. It closed at 321 PRs over eighteen
days, and three of the larger user-facing changes landed after it was written.
- Correct the counts throughout: 321 PRs, eighteen days, 24 contributors
(11 first-time), gallery 1,221 to 1,505.
- Add sections for the three new capabilities: 3D generation as a modality
(Generate3D, FLAG_3D, /v1/3d/generations, trellis2cpp), audio.cpp serving
six audio endpoints from one process, and the operations bar becoming the
Activity page.
- Cover the two further hardening fixes (tar hardlink escape, cyclic $ref
stack overflow) alongside the TRL one.
- Note the Valkey store, systemd socket activation, persistent trace history,
in-place chat edits, the self-contained SYCL backend and the site split.
- Group the new-engine sections together rather than splitting them across
the operational ones.
Embeds the existing vllm-race and magpie clips, and adds a 3D generation clip
cut from the demo recording to the conventions in .agents/preparing-a-release.md
(no audio track, 14s, named for the feature). blog.css styled figure img but
not figure video, so a clip in a post rendered outside the card; both selectors
now share the rule.
Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
tests-qwen3-tts-cpp has been failing on master since 2026-07-31. The suite
loads every component fine and then stops: TTS() never returns from the
native call, so a job that takes ~5 minutes runs into the 20 minute Go test
timeout instead.
goroutine 74 [syscall, 19 minutes]:
github.com/ebitengine/purego.RegisterFunc.func4
qwen3-tts-cpp.(*Qwen3TtsCpp).TTS goqwen3ttscpp.go:154
qwen3-tts-cpp.init.func2.4 e2e_test.go:90
Not a flake: reproduced on master and again on an explicit re-run.
Bisected across this cycle's eight qwentts.cpp bumps by their own check:
10832, 10850, 10902, 10964, 11006, 11039 and 11127 all pass in ~5 minutes;
11241 (abab6b3) fails at 1h58m. That PR was merged with this check already
red, which is how the hang reached master.
35ebe537..abab6b3 is three upstream commits, and the only functional one is
26dd8adb, "predictor: unroll the frame into one cgraph and sample in standard
ops", which is consistent with a generation loop that never reaches its stop
condition.
Hold the pin at the last known-good commit. The bump entry is commented out
rather than left in place, because it tracks upstream master and would put
the hang straight back on the next nightly run. Both spots carry a pointer to
the other so the hold is discoverable, and restoring it is uncommenting four
lines once upstream is fixed.
Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Select the CPU_ALL_VARIANTS target for x86 GPU images so partial offload uses runtime-selected host kernels. Keep GPU arm64 builds on the portable fallback until their toolchains consistently provide gcc-14.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* ⬆️ Update ggml-org/llama.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(llama-cpp): drop merged MiniMax-M3 patch
The bumped llama.cpp revision includes the MiniMax-M3 parser and template detection, so the carried patch now rejects during backend preparation. Remove the obsolete patch while retaining the independent score-task patch.
Assisted-by: Codex:gpt-5 [systematic-debugging]
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Add the new 9B Fara computer-use model alongside its existing 27B sibling, with Q4_K_M and Q8_0 llama.cpp variants plus the required vision projector.
Assisted-by: Codex:gpt-5 [web]
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Build the runtime CPU variant set alongside x86 GPU backends so partial offload uses the host's SIMD kernels instead of the scalar fallback. Keep arm64 GPU images on the portable binary until their builders consistently provide gcc-14.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* fix(utils): reject tar hardlinks that escape the extraction root
ExtractArchive pre-scans archive members and rejects symlinks, but tar
hardlink entries carry a regular file mode and so pass that check.
Header.Linkname was never validated, so an archive could create a link
to a path outside the destination directory.
Validate Linkname with the same path check already applied to member
names. Hardlinks that resolve inside the extraction root still extract,
so ordinary archives are unaffected.
pkg/oci/image.go already resolves tar.TypeLink targets before using
them; this brings the archive extraction path in line with it.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Zelys-DFKH <zelys@dfkhelper.com>
* test(utils): cover hardlink overwrite and in-root hardlinks
The existing hardlink test names a link target two levels above the
extraction root, so its final assertion checked a path the link never
resolved to and could not fail. Point the target one level up instead,
at the path that assertion already names.
Add two cases. The first uses a .tar.gz, where ExtractArchive binds a
Tar config with OverwriteExisting set, and follows the link entry with a
regular entry of the same name. Before the fix that pair linked to a
file outside the root and then truncated it through the link, which the
plain .tar case does not reach. The second extracts a hardlink whose
target is an earlier member of the same archive, covering the claim that
ordinary archives are unaffected.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Zelys-DFKH <zelys@dfkhelper.com>
---------
Signed-off-by: Zelys-DFKH <zelys@dfkhelper.com>
Use the case-sensitive Hugging Face filenames and refresh the linked SHA256 values for both gallery variants.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Detect text-to-speech MLX repositories during model import and emit a TTS-ready mlx-audio configuration. Expose mlx-audio in the backend preference dropdown for repositories without complete metadata.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* feat(sycl): make the intel llama.cpp backend self-contained on any host
The SYCL backend shipped an incomplete oneAPI runtime AND relied on a
host-provided GPU driver, so it only ran inside the build container. On a
bare host it died with "libze_loader.so.1 / libdnnl.so.3: cannot open
shared object file", and even with the host's Intel driver installed it
SIGSEGV'd during SYCL init when the host driver was built against a newer
glibc than the backend's bundled loader (rolling-release distros).
package_intel_libs now bundles the complete, coherent oneAPI runtime
(the missing MKL ILP64 / sycl_blas / tbb_thread + oneDNN + the dlopen'd
UR adapters, plus a sweep of the backend binaries' own direct deps) and
the Intel GPU userspace driver (libze_intel_gpu + libigdrcl + IGC + gmm)
with its OpenCL ICD manifest, mirroring how package_vulkan_libs bundles
Mesa. run.sh points the Level Zero and OpenCL loaders at the bundled
driver, and install-base-deps.sh installs it in the SYCL build image.
Bundling the driver is safe across kernels because it talks to the host
i915/xe via the stable DRM UAPI (unlike NVIDIA's kernel-locked
userspace).
Validated on Arch (glibc 2.43, i915): the backend loads and runs on an
Iris Xe with no host Intel packages installed.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
* fix(sycl): install a driver that exists, and let the user choose their own
The driver install added earlier in this branch asked apt for
intel-level-zero-gpu, which is not a package in Ubuntu 24.04. apt fails
outright on an unknown name, so neither driver was installed, nothing was there
to copy, and the images carried no driver at all.
It now comes from Intel's own repository, which has 25.18 for this Ubuntu
release, against 23.43 from late 2023 in the Ubuntu archive. The archive driver
does not know any card released since, so a machine with a recent Intel GPU
would end up carrying a driver that cannot drive it. Anything that goes wrong
during that install fails the build on purpose: an unreachable repository is a
passing problem that a retry fixes, while quietly carrying a different driver,
or none, is a difference nobody would notice until a user reports an idle GPU.
run.sh used to overwrite whatever driver the user had chosen. Level Zero uses
only the driver it is given, so on a machine with a card too new for the
carried driver, the GPU would go unused with no way back. Both that setting and
the OpenCL one are now left alone when already set, and the docs say how to
point a backend at the machine's own driver.
The OpenCL setting also used to be applied whenever the backend held a driver
list, even when the driver it named had not been copied, which leaves OpenCL
with nothing instead of falling back to the machine's own driver. It now
requires the copied driver to be present, and the packaging leaves out the list
entry of any driver it did not copy. The oneAPI images list a processor-only
OpenCL library, which was being carried with nothing behind it.
Two more corrections in the packaging. The scan for libraries a program is
linked against only looked at files named llama-cpp-*, so turboquant and bonsai,
which are also built for Intel GPUs, were left with the incomplete set of
libraries this branch set out to fix; it now looks at every program in the
directory. And a build that should carry a driver but ends up without one now
says so, which is what a stale prebuilt base image looks like: such a backend
still runs on a machine that has its own driver, so nothing fails and the only
other symptom is a user reporting an idle GPU.
Backends now also ask the driver to report how much graphics memory is free,
without which llama.cpp reads zero on an integrated GPU, since such a chip
shares the system memory instead of having its own. turboquant and bonsai get
the same run.sh handling as llama.cpp.
The driver is only carried by the builds that start through run.sh, because
run.sh is what points Level Zero and OpenCL at it. The Python backends for
Intel GPUs start differently and would never load it, so they keep using the
machine's own driver rather than carrying several hundred megabytes they cannot
use.
Checked in a container on Ubuntu 24.04: the install brings driver 25.18 with
the files where the packaging expects them, an unreachable repository fails the
build, and the copied set resolves on its own once the machine's Intel packages
are moved away.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
* fix(ci): rebuild every Linux backend when the GPU packaging script changes
scripts/build/package-gpu-libs.sh decides which GPU libraries end up inside an
image. The filter that builds the backend matrix listed it as an input of the
Python images only, so changing it rebuilt no Go and no C++ backend, even
though those run it from their own package.sh. A packaging fix aimed at the
Intel llama.cpp backend could merge and reach no image, which is the same
failure this rule was written to prevent.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
* fix(sycl): carry only the driver Level Zero uses, not the OpenCL one
llama.cpp reaches an Intel GPU through Level Zero, which hands the driver
programs that are already compiled and so needs only the back end of the
graphics compiler. The OpenCL driver can be handed source code instead, so it
needs the compiler's front end as well, and that arrives with its own copy of
clang. Carrying it cost about 139 MB in every backend built for Intel GPUs, and
took the carried set from 123 MB to 261 MB.
Nothing here takes that path. No LocalAI code selects an OpenCL device, each
backend image holds one backend, and the documentation never described OpenCL
as a way to run models: the only mentions are a stale clblas row in the
BUILD_TYPE table, for a llama.cpp backend that no longer exists and that no
build matrix entry uses, and the sycl-ls troubleshooting hint. Before this
branch the packaging carried the OpenCL loader and adapter but no driver, so
the path could not work in a released image either. There is nobody to keep
working.
The driver list that OpenCL reads is no longer carried, and run.sh no longer
sets OCL_ICD_VENDORS, so OpenCL inside a container keeps using whatever the
image provides rather than being pointed at a directory with no driver in it.
Checked in a container against the real 25.18 driver: the carried set is 123 MB
with nothing unresolved, and Level Zero still reports the GPU with the
machine's own Intel packages moved out of the way. Neither the Level Zero
driver nor the compiler back end names the front end or clang among the
libraries it opens by name, so the leaner set is complete for this path.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
---------
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
* feat(chat): edit saved conversation messages
Add inline edit, save, and cancel controls for stored user and assistant messages without triggering inference. Preserve structured message attachments and cancel edits when streaming starts.
Assisted-by: Codex:gpt-5
* test(chat): preserve seeded conversation on reload
The saved-message edit test reloads the page to verify persistence, but its init script was replacing localStorage with the original fixture on every navigation. Seed only an empty store so reloads exercise the data written by the application.
Assisted-by: Codex:gpt-5 [Codex]
---------
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Direct repository-root users to the supported make docs target and document the equivalent direct Hugo invocation from docs/.
Fixes#10062
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The page reads as a list of features with nothing joining them, so two
things did not land.
The engines section never said these are the backends LocalAI loads. The
runtime section describes a core that pulls each engine in on demand, and
the engines section describes engines written from scratch, and nothing on
the page connected the two sentences. Readers were taking parakeet.cpp and
the rest for unrelated side projects by the same people. The lede now says
whose backends they are before it says anything else.
APEX was used as a known term on first appearance, in a section that opened
onto a benchmark table. Nothing said what it is or why it follows the
engines. It now opens by placing itself in the stack: the engine decides how
fast a model runs, the weights decide whether it runs at all, and APEX is
the second of those. Then the numbers.
Also drops "Most backends wrap somebody else's engine. These do not", which
is the machine-written antithesis shape, and fixes a list that broke its own
parallel halfway through.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m]
The star, fork, contributor and release counts were typed into the templates
by hand, so they only moved when somebody remembered. They had already
drifted: stars read 48,042 against 48,067, forks 4,314 against 4,320, and
contributors 224 against 225.
They move to website/data/stats.yaml, which .github/ci/refresh-site-counters.sh
rewrites from the GitHub API, run weekly by a new workflow. The contributors
and releases endpoints never report a total, so the script asks for one item
per page and reads the count out of the Link header. It refuses to write a
zero or a non-number, which is what a rate-limited or failed call looks like,
and the workflow commits only when a number actually moved. The Discord count
has no API behind it, so the script reads the existing value back and carries
it through.
The engine count was wrong in a second way. The hero said 18, the section
heading said "Eighteen engines", the timeline said "Nineteen engines of our
own", and the /engines/ page derived 19 from the data file. All of them now
derive from that same file, so they cannot disagree again.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m]
Add Q4_K_M and Q8_0 llama.cpp entries for the newly released Qwopus3.6-27B Fusion reasoning and coding merge, with MTP enabled.
Assisted-by: Codex:gpt-5 [Hugging Face API]
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The timeline set six 15rem columns in a flex row with overflow-x:auto, which
needs 90rem and so scrolled sideways on any normal laptop. It is a wrapping
grid now, and the rule that carries the dots moves from the container onto
each item so a wrapped row still gets a line above it. Column gap is zero and
the items carry their own right padding, so the rule stays continuous.
Integrations move from a card grid to a reel. Any single integration is a weak
signal and the whole moving line is the strong one, so the count is doing the
argument. It pauses on hover and on keyboard focus, since the names are links.
The list grows from 8 to 26: Open WebUI, Dify, LibreChat, RAGFlow, Continue,
big-AGI, Nextcloud, Frigate, promptfoo, Mods, TypingMind, baibot, k8sgpt-
operator and others. Each was admitted only after opening that project's own
repository or docs and reading the line that names LocalAI. The ones that
failed that test are listed in the data file so nobody re-adds them.
The blog cards were hand-written, which is how one of them came to advertise
"Porting vLLM to C++", a post that does not exist, and how all three linked to
the blog index instead of an article. They range over the posts now.
The section intro used the "a changelog tells you what moved, these posts show
you what it does" shape, which is the standard machine-written antithesis. It
states what the posts contain instead, including the perplexity regression
that APEX costs, because publishing the price is the actual claim.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m]
The band led with three sentences of hedging and printed a commit count
next to each employer, so a one-commit entry beside a large name read as
weakness rather than as the modest, true claim it was. It now opens on the
contributor count, sets the employers as a sentence instead of a pill wall,
and keeps the caveat to one line. The counts stay in ecosystem.yaml, since
they are the provenance for the list and anyone re-checking it needs them.
Two "coverage" cards were not about this project. The modelslab.com piece
reviews Frikallo/parakeet.cpp, an unrelated project of the same name, and
the snailtext.app benchmark measures Parakeet through ONNX Runtime without
mentioning LocalAI at all. Both are removed, along with the contributor
card that duplicated the band's opening line.
Press was four posts from one vendor, which read as the whole of the
coverage rather than one enthusiastic outlet. SUSE collapses to a single
series entry, and Pulumi, Semaphore and Spectro Cloud join it. Each was
opened and checked against the project before being added. K8sGPT and
LlamaIndex join the integrations; both document LocalAI as a backend.
The quotes move above the lists so the section opens on its strongest
line, which is somebody else's. The hero gains a GitHub call to action,
the APEX collection link was returning 404 and is corrected, and the
footer no longer describes the site as a design mock.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m]
Follow-up to 42541dd4f, which routed lint to arc-runner-set. Both of its
jobs failed there in one second (run 30637392862): the runner image has
git, curl, unzip, tar, ldd and python3, but not make, and build-scripts
additionally needs gcc because the packaging-script tests compile a
throwaway binary and inspect it with ldd.
golangci-lint needs make twice over: `make protogen-go` (which also wants
curl + unzip to fetch protoc) and `make lint` itself. So both jobs go back
to ubuntu-latest.
gh-pages.yml stays on arc-runner-set and is unaffected: it uses no make and
no C toolchain, and setup-go / actions-hugo fetch their own toolchains.
The preflight steps stay. They cost about a second on the hosted pool, and
they are what turned this into a one-second named failure instead of an
opaque one midway through a build. When the runner image gains make + gcc,
re-routing is one runs-on line per job. Any such re-route must stay
push-only: lint also runs on pull_request, and fork PRs execute untrusted
code that must not reach a persistent self-hosted runner.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [ClaudeCode]
The GitHub-hosted runner pool is shared per ACCOUNT, not per repo, so a
burst in one repo starves every other. On 2026-07-31 it reached zero
scheduled jobs for 35 consecutive minutes with 39 jobs queued, while
arc-runner-set completed 12 jobs without interruption across the same
window. Actions was healthy globally (other public repos were scheduling
normally), so this is an account-level throttle we cannot fix from inside
the workflows, only route around.
Site publishing and lint are small, run on nearly every commit, and gain
nothing from waiting behind a saturated hosted queue, so both move to
arc-runner-set, the label already proven in generate_intel_image.yaml.
lint.yml is routed for PUSH ONLY, and this is the important part: that
workflow also triggers on pull_request, and a fork PR executes untrusted
contributor code. Running that on a persistent self-hosted runner would be
a real compromise vector, so anything that is not a push to mudler/LocalAI
stays on the ephemeral hosted pool. gh-pages.yml needs no such clause: it
triggers only on push-to-master and workflow_dispatch, so it never runs
pull-request code. Both carry a repository guard so forks, which have no
such runner label, fall back to hosted instead of queueing forever.
Neither workflow uses sudo or apt, and both fetch their own toolchains via
setup-go / actions-hugo. A self-hosted image can still be leaner than the
hosted one, so each lint job opens with a preflight that names the missing
tool (curl/unzip/make for protoc and lint; gcc/ldd/python3 for the
packaging-script tests) rather than failing opaquely mid-build. Reverting
is one runs-on expression per job.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [ClaudeCode]
Signed backend verification performs separate registry requests for manifests and referrers. Reuse LocalAI's version-aware User-Agent there so the full install flow is attributable to LocalAI.
Assisted-by: Codex:gpt-5
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Adds the "who turns up around this project" section, split into three lists
because the evidence behind each one is a different strength and collapsing
them into a single logo wall would overclaim.
Contributors 21 companies whose engineers have commits here. Evidence is
the commit history plus the employer on that person's public
GitHub profile, so it is a claim about the person. Commit
counts are shown next to each name, including the ones that
are a single patch, because hiding that would be the whole
problem.
Integrations six projects that reference LocalAI in their own repository
or documentation, which anyone can verify without asking us.
Press four SUSE Communities articles about running LocalAI.
Names are set in type rather than fetched as logos. A logo reads as
endorsement, and a one-line typo fix from somebody who happens to work at a
large company does not support that, quite apart from what their trademark
policy says about it.
ADOPTERS.md is the mechanism for the stronger claim. An organisation that
wants to be listed as a user opens a pull request adding itself, which is both
the evidence and the permission, and is publicly auditable afterwards. The
file says plainly what the website does and does not claim, so the next person
to ask "can we add some big names" has the answer in the repository.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write] [WebSearch]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* feat(website): split the site, move docs to /docs, add a landing page
The Hugo docs site has always been localai.io itself, which left nowhere to
explain what LocalAI is or show what the team builds. This adds a separate
marketing site at the root and moves the documentation under /docs/.
Docs:
The existing site keeps its content tree and its Relearn theme, and now
builds with baseURL <root>/docs/. Its _index.md, which held a hand written
landing page, becomes a real documentation home.
Every previously published URL keeps working. GitHub Pages has no server
side rewrites, so .github/ci/gen-redirects.sh walks the built docs output
and leaves a meta refresh plus a canonical link at each old root path. It
covers bare .html files too, which is what keeps /gallery.html alive, and
it never overwrites a path the marketing site already owns.
Website:
A second Hugo site under website/ with its own layouts and no external
theme, so the marketing side does not have to fight Relearn's home rooted
menu and asset pipeline. CI builds both and merges them into one Pages
artifact.
The design is derived from the project logo rather than invented: the navy
of the triangle, the cyan of the llama, the purple of the speed bars. Those
offset bars became the motion signature. The background renders a real
depth-anything.cpp depth map as contour lines and switches to a
locate-anything.cpp style detection overlay over the engines section.
Also included: an /engines/ index driven entirely by data/engines.yaml, a
/blog/ section with five posts written from the release notes and the
engine benchmark suites, install.sh and a Kubernetes manifest since the
site advertises both, and a rule in .agents/ that release preparation now
includes a blog post and demo clips.
Every figure on the site is derived from the repository or the GitHub API,
not from memory. Correcting them against their sources found one error in
README.md: voxtral-tts.c is text to speech, not speech to text.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write] [Agent]
* feat(website): add a star history chart, rewrite the history post in first person
The history post read like a changelog written by a committee. It is now in
Ettore's voice, first person, with the admissions left in.
The numbers paragraph in particular read like a directory listing. It now says
what the figures mean rather than which file they came from.
Adds an interactive star history chart, built from the GitHub stargazers API
rather than embedded from a third party, so the page makes no external request
and cannot break when someone else's service is down. The four releases the
post is organised around are marked on the curve, and the labels stack into
rows because three of them land within two months of each other.
The API stops paginating at 40,000 items, so the curve is measured up to
December 2025 and the segment from there to today's total is drawn dashed,
labelled as an estimate in the caption and in the tooltip. It is a straight
line between two known points, and the chart says so rather than implying it
is data.
Also drops "marketing site" from the README heading and everywhere else it
appeared, and calls it the main site instead.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit] [Write]
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
