677 Commits

Author	SHA1	Message	Date
Richard Palethorpe	49ef40a187	feat(classifier/VAD): support voice control on low power devices (#10804) ... * feat(llama-cpp): route Score through the slot loop Score previously bypassed the slot loop with a direct llama_decode: a conflict guard aborted the whole process if scoring raced generation, the config validator had to reject score alongside chat/completion/embeddings, and every candidate re-decoded the full shared prompt. Add SERVER_TASK_TYPE_SCORE to the (patched) upstream server so score tasks are scheduled like any other slot work: generation and scoring serialize naturally, the shared prompt is decoded once per call, and the slot's prompt cache carries the conversation prefix across calls. Context checkpoints at the score boundary and at the cache-divergence point keep SWA/hybrid/recurrent models (e.g. LFM2.5) from re-prefilling the whole prompt per candidate: warm-turn scoring on a 6-option set drops from ~8s to ~0.5s on a desktop CPU. The conflict guard and the validation split are removed; declaring score with generation usecases on one config is now supported and shares the slot cache. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(realtime): classifier wire types and pipeline config Wire types and YAML config for realtime classifier mode: sessions carry a localai_classifier extension (options with canned replies/tool calls, softmax threshold, normalization, history trimming, fallback modes, and a deterministic wake-word address gate), mirrored by pipeline.classifier in the model YAML and surfaced in the config-meta registry. The localai.classifier.result server event reports the full score distribution per turn. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(realtime): classifier response flow Classifier-mode responses: instead of autoregressive generation, each user turn is prefill-scored against the option list (router.ScoreClassifier prompt/candidate shapes over the Score primitive) and the winning option's canned reply and tool call are emitted through the existing response machinery. Below-threshold turns take the configured fallback (none / canned reply / generate); empty transcripts and unaddressed turns (wake word not mentioned) skip scoring entirely. The scoring probe defaults to the latest user message only — small scorers echo canned replies from prior turns back as the top option otherwise. Built for hardware that can afford prompt processing but not decode: with slot-based Score the option list stays KV-cached across turns, so a turn costs roughly one forward pass over the new words. session_update_error events now carry the validation cause instead of a generic message. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(realtime): bound the VAD tick's scan window and buffer retention The VAD tick loop re-scanned the entire input buffer every 300ms and only trimmed it on zero-segment ticks or commits. Audio that keeps producing segments without a committing pause (steady noise a mic pipeline lets through, music, continuous speech) grew the buffer toward the 100MB cap with each tick rescanning all of it — O(n^2), measured at ~3.3ms of silero per buffered second: past ~90s retained, ticks run back to back and pin ~4 cores until the stream stops. Silero's recurrent state only carries a few hundred ms of context, so rescanning old audio buys nothing. Clip the slice handed to the VAD to the largest silence the commit test can need to measure (server_vad silence window or the semantic eagerness fallback) plus a warm-up margin, and rebase the returned segment times so every downstream consumer keeps whole-buffer coordinates. An open turn whose clipped window is all silence now commits (the silence outran the window) instead of being discarded as no-speech. Independently, retain at most 90s of raw buffer, rebasing the live-feed and EOU cursors on trim — this also bounds the previously unbounded VAD-error path. Turn boundaries are otherwise unchanged: no forced commits, no new coordinator states. pipeline.turn_detection.vad_window_sec can widen the scan window; values below the automatic floor are ignored. The tick body is extracted into vadTick so specs can drive turn detection synchronously (same shape as classifySoundWindow); the babble reproduction that pinned 4 cores now plateaus under 10% of one core. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(backend): let per-model threads override the global default ModelOptions overrode a set per-model threads value with the app-level --threads whenever the latter was non-zero — and WithThreads defaults it to the physical core count, so it always was. The YAML threads: knob has been dead config: a tiny VAD model could never opt down from the global pool size. SetDefaults already fills an unset per-model value from the app config, which is the intended precedence; resolve threads through a helper that honors it (explicit threads: 0 still means unset). Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * chore(gallery): single-thread the silero VAD Silero is a ~2MB recurrent model with no exploitable graph parallelism: measured per-call latency is identical at 1 and 10 ORT threads, while every extra pool thread just spin-waits between the realtime loop's frequent tiny inferences. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * docs(realtime): classifier mode, VAD scan window, threads precedence Document the realtime classifier mode (options, threshold guidance, wake-word address gate, empty-transcript handling), the VAD scan window and 90s buffer retention (pipeline.turn_detection.vad_window_sec), the per-model threads precedence, and the M3 classifier note in the realtime state-machine design doc. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * perf(llama-cpp): score all candidates in one batched decode One scoring call is now a single SERVER_TASK_TYPE_SCORE task: the slot decodes the shared prefix (prompt + longest common candidate token prefix) once, then forks one sequence per candidate off it (metadata-only for the unified KV cache, copy-on-write for recurrent state) and decodes every candidate's unique tail in one llama_decode. Previously each candidate was its own task that restored the boundary checkpoint and re-decoded its full tail sequentially, paying per-candidate task and decode overhead. The context reserves SERVER_SCORE_FORK_SEQS extra sequence ids (and recurrent-state cells) beyond the parallel slots via the new common_params::n_seq_score_forks. Forking requires the unified KV cache (already this backend's default) since per-sequence streams would shrink n_ctx_seq; an explicit kv_unified:false disables forking and Score calls that need it fail cleanly. Candidates beyond the fork/output budget decode in successive chunks. Wire contract and scores are unchanged: per-token logprobs are stitched from the shared region and the forked tails. Verified bitwise deterministic call-to-call and independent of candidate order (no cross-fork leakage via equal-length candidate swap); ranking matches the per-candidate implementation on the drone battery (winner softmax 0.99996 vs 0.99997), and >16-candidate chunking, prefix-of-another and empty candidates all pass. Measured on a desktop CPU: warm /api/score calls 0.52s -> 0.23s; warm realtime classifier turns 196-303ms. The 9-candidate drone turn decodes ~17 unique tail tokens in one batch instead of nine sequential ~220ms checkpoint-restore tasks. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(realtime): gate scoring capacity by model usecase Reserve llama.cpp scoring slots only for models that explicitly declare the score usecase, while allowing score to coexist with chat and completion. Reject incompatible unified-KV settings and classifier activation on models without scoring capacity. Propagate application defaults when resolving realtime and preload pipeline stages so unset thread counts are resolved consistently without overriding explicit model settings. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(ci): honor APT mirrors in the prebuilt llama-cpp compile step The builder-prebuilt path installs gcc-14 with apt directly and ignored the APT_MIRROR/APT_PORTS_MIRROR build args the from-source path already honors, so an ubuntu mirror outage broke every arm64 backend build. Pass the args into the stage and run apt-mirror.sh (already in the build context via COPY . /LocalAI) before the apt step. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(realtime): classifier argument slots via constrained completion Hybrid classify-then-complete: a classifier option's canned tool call can declare typed argument slots (number | enum | string, with defaults and prompt hints) referenced as "{{name}}" in the arguments template. When the option wins, the slots are filled by a short grammar-constrained completion that continues the exact scoring prompt — rendered by the same cached ScoreClassifier, so the llama.cpp prompt cache is already warm — with the chosen route JSON re-opened at the first slot field. A GBNF grammar pins the field skeleton and frees only the values; temperature 0, a couple dozen tokens at most (~300ms on a desktop CPU for two slots). Slot declarations and hints ride the option descriptions in the shared system prompt, informing scoring and the fill alike at no per-turn token cost. The localai.classifier.result event carries the final arguments and a fill_latency_ms. On inference failure the slots' defaults apply; a slot without a default fails the response (or falls through with fallback.mode: generate). Slot filling requires completion alongside score in the scoring model's known_usecases. Verified end-to-end on the Pi drone demo: "fly forward three meters" in distance mode classifies forward and infers {"distance": 3, "units": "meters"} in ~310ms, and the drone flies exactly 3 units. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(realtime): splice filled slot values into classifier replies A classifier option's spoken reply can now reference its tool's argument slots ("Going forward {{distance}} {{units}}."): the values inferred by the slot-fill completion — or the recovery defaults — are spliced into the reply as plain text before it is emitted, so what the assistant says confirms what it actually inferred. Placeholders without a value stay literal, and options without slots are untouched. FillToolArguments now returns the raw slot values alongside the spliced arguments JSON to make the reply templating possible. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(realtime): harden classifier slot completion Reserve context for constrained slot filling, size completions from their encoded output, and encode enum grammar literals as valid JSON. Reject empty enum values and cover the failure modes with regression tests. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(realtime): prewarm the classifier scoring prompt on registration Swapping a session's classifier option list (a voice-switched command mode, for instance) made the next turns pay a full re-prefill of the new option-list prompt — measured 2.4s vs 0.3s warm on a desktop CPU, and worse: on hybrid-memory models like LFM2.5, whose state cannot be partially rewound (llama.cpp can only restore checkpoints), *every* probe change re-prefilled from scratch whenever the last checkpoint missed the probe boundary, so even same-list turns intermittently cost full prefills. Registering an option list (pipeline seed or session.update) now fires a best-effort background prewarm: two throwaway scores with distinct probes. The first prefills the new option-list prompt; the second, diverging exactly where per-turn probe text starts, plants the backend's rewind point (KV checkpoint) at the stable-prefix boundary that every real turn reuses. The prewarm hides behind the canned mode-switch reply — by the time it finishes speaking, the cache is warm. Idempotent per option set, detached from the registering request's lifetime. Measured on the drone demo (LFM2.5-1.2B, desktop CPU): first turn after a mode switch 2374ms -> 340ms; intermittent same-list full prefills (1.3-2.1s) all -> under 0.5s. For clients that swap lists frequently, options: [parallel:2] on the scoring model additionally keeps one slot per list via prefix-similarity routing (+26MB RSS, unified KV). Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * perf(llama-cpp): checkpoint scoring at the caller-declared stable prefix Hybrid-memory models (LFM2.5 shortconv, Qwen3.5 deltanet — where new small models are headed) cannot rewind their state, so any prompt-cache reuse that needs a rewind falls back to a full re-prefill. For classifier scoring that meant every probe change re-processed the whole option-list prompt: the server's checkpoints were placed reactively (at wherever the previous task happened to diverge), so a checkpoint past the next divergence was erased rather than restored — measured as intermittent 2-10s turns on prompts with a 95%+ common prefix. The classifier now computes the probe-invariant prompt prefix once (the byte-wise common prefix of two synthetic probe renders) and declares its length with every Score request; the server maps it to a token boundary and forces a KV checkpoint exactly there on each score prefill. That checkpoint sits at or before every future divergence under the same option list, so it always survives and always restores — repeat scoring costs probe+candidates regardless of how the probe changes. Also: - prewarm reruns on every option-list registration instead of memoizing per list: with boundary checkpoints a redundant rewarm costs two probe-sized decodes, while skipping one after a slot eviction (three lists sharing fewer slots evict in LRU cascades) silently moves a full re-prefill onto the user's next turn - new llama.cpp backend option rs_seq:N exposes bounded recurrent-state rollback outside speculative decoding; measured impractical for deltanet-scale states (65GB for 64 snapshots on Qwen3.5-4B) but cheap insurance for small-state models - docs: the multi-list recipe (parallel:N + sps:0.5 — the default slot similarity threshold funnels distinct lists onto one slot) Measured on the drone demo (LFM2.5-1.2B scorer, desktop CPU), steady state: every turn 285-421ms including mode switches, vs 2.4s post-switch and intermittent 1.3-2.9s re-prefills before. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(realtime): align classifier cache guidance Document the single-score prewarm behavior and clean the vendored score patch formatting. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(llama-cpp): guard score task for fork backends TurboQuant and Bonsai reuse the primary gRPC server against llama.cpp forks that do not carry LocalAI's slot-based Score patches. Compile the Score integration only for the patched primary backend and return UNIMPLEMENTED from fork builds instead of referencing absent task types and common_params fields. Assisted-by: Codex:gpt-5 [gh] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(dev): generate gRPC code before commit lint The coverage phase regenerates ignored protobuf bindings, but lint runs first and can fail against missing or stale output. Generate the pinned bindings before lint so the gate always type-checks the current schema. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-07-29 12:50:22 +02:00
localai-org-maint-bot	034df6ceb1	fix(worker): report RAM alongside GPU memory (#11167) ... * fix(worker): report RAM alongside GPU memory Assisted-by: Codex:gpt-5 * feat(ui): show worker RAM on node views Assisted-by: Codex:gpt-5 --------- Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-07-28 23:55:35 +02:00
mudler's LocalAI [bot]	0f7186f214	feat(ui): replace the stacked operations bar with a one-line strip and an Activity page (#11163) ... * feat(ui): record finished gallery operations in a bounded history ring The operations panel drops an operation the moment it succeeds, so a user who steps away cannot tell whether an install finished, failed or was never started. OpCache now keeps the last 50 terminal operations, recorded from the point where an op leaves the cache. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): pin the history ring's dedupe, outcome order and start stamp Review of the history ring found four gaps. The dedupe guard and the bounded seen set were unreachable through the exported API and so had no coverage; an in-package spec file now drives opHistory directly. The outcome switch claimed an ordering was load bearing that nothing pinned, so an errored op that never reached Processed now has a spec. Two behaviour fixes come with it. StartedAt was the zero time for ops recovered from the store or replicated from a peer, since neither path stamps a start time, which would have rendered as a two-millennia duration; it now falls back to the finish time. Reusing a cache key with a fresh job ID orphaned the previous stamp, so Set and SetBackend now drop it. The comment on the outcome switch described a state the code cannot be in: CancelOperation sets Cancelled and Processed synchronously before the handler removes the entry, so status.Cancelled already covers the cancel endpoint. The !Processed clause stays for the dismiss endpoint firing on an in-flight op, and the comments now say so. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): record operations that end on a peer replica The NATS end event is the only signal a replica gets for an install another replica ran. Record from applyEnd too, deduped by job ID so the originating replica does not record its own broadcast twice. Three start-stamp defects in the same path go with it. applyEnd now drops the stamp unconditionally, since recordTerminal only cleans up on the path where it found a cache key and an end event can overtake the local Set. applyStart drops the stamp of the job whose cache key it replaces, which a peer-driven retry previously stranded. And recordTerminal reads the stamp once instead of testing Exists and then reading, so a concurrent record for the same job can no longer delete the stamp between the two and let the zero time overwrite the finish-time fallback, which the Activity page would render as a two-millennia run. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): do not guess the outcome of a peer operation with no local status A replica that restarts mid-operation hydrates its OpCache keys from PostgreSQL, but gallery statuses are in-memory only and come back empty. The end broadcast then landed on recordTerminal's nil-status branch, which reads a missing status as queued-and-removed and filed a successful install as cancelled. That reading is right locally and wrong on the peer path, where a missing status means the outcome was never held here. recordTerminal now takes the source of the terminal event and records nothing when the peer path finds no status, restoring what the replica did before the end event started recording. The local path is unchanged. Also move the ApplyEndForTest seam to the conventional export_test.go, and stop the dedupe spec from claiming to guard the ring's seen set: the local delete removes the status keys, so the broadcast that follows returns before reaching it. An in-package spec that calls recordTerminal twice does the pinning. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(api): add GET and DELETE /api/operations/history Admin gated like the rest of the operations API. The live /api/operations payload is unchanged so the one second poll stays small. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): expose operation history through OperationsContext Fetched on demand and when the live list shrinks, never on the one second poll interval. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): detect operation departure by identity and ignore committing ops in the ETA gate Refetching history on a shrinking live count missed a completion that coincided with a start, which is the common case during a batch install. Track the live job IDs instead, so any departure triggers the refetch regardless of how the count moved. An operation that has finished downloading stays live at currentBytes == totalBytes for the whole commit and install phase and can never produce an estimate, so counting it in the all-or-nothing gate blanked every other operation's time remaining for as long as it lasted. Only operations still moving bytes get a vote. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): let only downloading operations gate the time remaining estimate Verifying pins an operation flat below its total for the whole sha256 pass: the AfterDownload hook reports completedBytes plus the finished file against a total summed over every file, then hashes synchronously without emitting progress. Files download sequentially, so a 15 shard model enters that window 14 times, and a byte comparison cannot see it because the counter is genuinely below the total throughout. Gating on phase closes resolving, verifying, committing and persisting in one predicate, so a quiet neighbour no longer blanks every other operation's estimate for minutes at a time. The byte clauses stay: a producer can report downloading with bytes already at the total. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): collapse the operations bar to a single line Four concurrent installs used to take four rows above every page. The strip now shows one operation, failure first, with a counter linking to Activity. The close button hides the strip and no longer cancels an install. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): keep the operations strip from widening the page and from muting a failure A long install error made the strip report a 1600px minimum width, which sized main-content to fit and gave every page under it a horizontal scrollbar. Inline-size containment plus shrinkable detail and bytes cells keep it inside the viewport. Hiding is no longer able to swallow the hidden job's own failure, a completed removal or staging says so instead of claiming an install, a cancelling operation renders as cancelling, and the live region no longer covers the per-second percentage. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): shrink main-content instead of containing the strip, and expose progress min-width on .main-content is what actually lets a long install error shrink, and unlike inline-size containment it has no browser support floor and no latent collapse if the strip ever lands in a shrink-to-fit context. It matches what .app-layout-chat .main-content already does, and it clears pre-existing horizontal overflow on narrow viewports as a side effect. The progress track is now a labelled progressbar, so assistive tech can read the value on demand rather than losing it to the aria-hidden that stopped the live region re-announcing every poll. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add the live operation card for the Activity page Carries the detail the one-line strip has to drop: phase, bytes, the per-node breakdown for cluster installs, and a labelled Cancel button. Cancelling is destructive, so it gets a labelled button rather than a glyph. A cancelling operation drops its progress bar and its time estimate, the same call the strip makes: a percentage still climbing under "Cancelling" reads as the cancel not having taken. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): give the operation card a verb, live node disclosure and its per-node detail The card carried no verb, so an install, a removal and a staging op rendered as spinner plus name plus kind tag and were indistinguishable. It now runs the same verb and icon chain as the one-line strip, which is what stops the page that is meant to carry more detail from carrying less. The auto-expand default was evaluated once at mount. An operation is listed as soon as it is admitted but its nodes are filled in only when the fan-out starts reporting, so a card mounted at creation latched on the empty list and stayed collapsed. The default is a live expression now, and state holds only an explicit choice. Also: an optional onRetry gates a Retry button, so the page can own the install reconstruction without the card ever showing a control with nothing behind it; the disclosure moved above the region it controls and gained aria-controls; the toggle is gated at more than one node so the count is never "1 nodes"; an unmapped node status is passed through instead of being relabelled "Queued"; error text is clamped with the full string in the title; and file_name plus the per-node progress bar are rendered again, reviving three CSS rules that had gone dead along with the detail they styled. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add the Activity page Live operations, unacknowledged failures and the record of what finished, at /app/activity in the Operate console. Cancelling an install now lives here behind a labelled button rather than on the strip, and a failed install can be retried: the retry dismisses the failure first so it still reaches the record, then reissues the model, backend or node-scoped backend install. The sidebar Operate entry carries the operation count. The console rail is only rendered on an Operate route and can be collapsed, so a badge there could vanish while operations were still running. Two follow-ups from review fold in here: a failed removal or staging job no longer reports a failed install on either the card or the strip, and the card's error text can shrink so one unbroken token cannot widen the card. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): dismiss operations by job, and stop the Activity page contradicting itself Dismissing resolved the job by display id, but /api/operations strips the "node:<nodeID>:" prefix before emitting, so a local install and a node-scoped install of one backend arrive as two jobs sharing one id. Dismissing by id retired whichever came first. That defeated the guarantee retry was built around: with the wrong job dismissed, the reinstall overwrote the acted-on failure's opcache entry in place, bypassing recordTerminal, while an unrelated failure vanished from Needs attention. dismissFailedOp, the card's dismiss control and the strip now all pass the jobID, which is what the endpoint takes. A filter matching nothing rendered the "nothing has ever run" empty state while the header counted the records the filter had hidden. The empty state is now gated on the All chip and a narrowed view gets its own message plus a way back; the header counts the instance rather than the chip, so selecting Backends no longer reports "Nothing running" over running model installs. Also: the summary drops a zero clause instead of rendering "0 needs attention" on the happy path and pluralises both counts; a record duration is floored at "< 1s" and rejected above a day, so a zero-value start stamp cannot render a span of millennia and a zero span cannot render "installed in" with nothing after it; a deletion cancelled mid-flight reports the cancellation rather than claiming it was removed; and the retry variant comment names the fix instead of calling the gap closed. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: document the Activity page and the operations history endpoints Adds an Activity page under Operations covering the one-line operations strip, the /app/activity sections and filters, per-operation cancel, retry and dismiss, the in-memory 50-entry record, and the sidebar count. Documents GET and DELETE /api/operations/history, and fills the gap in the admin-only endpoint list, which also omitted the pre-existing POST /api/operations/:jobID/dismiss. Corrects the distributed-mode install-watching section: the per-node breakdown now lives on the Activity page rather than on the strip, which rolls a fan-out up into a single phrase. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: correct nine details in the Activity page documentation The operations strip never renders a file name: its detail line is the error, the node roll-up, the target node, the phase or the queued note. Drops the stale clause in the distributed-mode section, where the per-node bullet is now the only place a file name is described. Scopes the phase vocabulary to artifact-backed gallery models, since a plain GGUF install emits no phase. Corrects the per-node list: the toggle exists for any fan-out of two or more workers and the four-node threshold only governs whether it starts open, while the N nodes tag needs more than one node. Notes that a cancelled operation can sit in the live section reading Cancelling, that cluster staging never reaches the record, and that Clear history appears only when the record has something in it. Names the operations response envelope, with a JSON example, so callers do not index a bare array, and stops describing the icon-only dismiss control as a labelled button. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: drop the unreachable Cancelling state and scope the byte claims An operation can only report isCancelled while it is unprocessed, but every writer of Cancelled sets Processed in the same breath, on the peer path as much as the local one, and the cache evicts cancelled entries before the handler sees them. The state cannot reach the page, so the live section is described again as running or queued operations. Byte counts come from the artifact bridge alone, the same producer as the phase, so a plain GGUF install, a removal and a backend install report none. Scopes both to artifact-backed gallery models and leaves the verb, the name and the percentage as what every operation shows. A worker backend install reports its bytes through fields the operations payload does not carry, so the distributed section now describes the percentage and the node roll-up, with per-file counts pointed at the per-node detail. Also: staging jobs carry no error, so they never reach Needs attention and Retry never had a staging case to exclude; an install that involves workers is no longer called node-scoped, which this page uses for node-targeted installs; and the record timestamps carry nanoseconds. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: state only the verb and the name as unconditional on the strip The percentage is as conditional as the bytes were: it renders only for a running operation that has reported progress, so a queued operation, a failed one and a removal never carry it. A removal in particular sits at progress zero for its whole visible life, since the delete path reports none and its completion is filtered out. Both the strip and the card paragraphs now lead with what always shows and list the rest as conditions. The Cluster chip matches on a node list that finished operations do not carry, so a fan-out install leaves the chip once it reaches the record. Scoped that claim to the live sections. Two more of the same shape, found by re-reading each clause alone: the strip also appears for a failure, which is not running, and the four-second hold only applies when nothing replaces the operation that just finished. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): stop reporting a cancelled install as installed, and make queued real Three defects that all trace to one root cause: `isCancelled: true` is unreachable from /api/operations. Every writer of Cancelled=true also sets Processed=true, the handler skips Processed && Cancelled, and OpCache.GetStatus evicts a cancelled op before the handler iterates it. Cancelling the last running operation put a green "Installed model X" on the strip for four seconds: the completion hold was guarded by `!previous.isCancelled`, which is dead. A cancellation deletes the operation server side, so the strip sees exactly what it sees on a completion, and nothing in the payload separates the two. The signal now comes from the side that issued the cancel: the operations context remembers the job IDs it cancelled (pruned after a minute) and the strip asks before it holds anything. A cancelled operation goes as soon as it stops; the record already reports it as cancelled. isQueued was set only when the gallery status was missing, but markQueued publishes a "queued" status at admission, so a queued op has a status for its whole queued life and the state was unreachable outside a microsecond window. Every operation waiting behind a running install rendered as "Installing model X" with a spinner. The queued phase is now the signal, via an exported PhaseQueued and a nil-safe OpStatus.IsQueued() next to the writer. With those two fixed, the Cancelling state has no way to be entered: cancelling is instantaneous from the API's point of view. Its branches, CSS, locale key and the isCancelled field itself are removed rather than left for a future reader to assume they work. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): keep a removal a removal, and say what an install is doing OpStatus.Deletion was set once, at admission, and lost on the next status write: UpdateStatus replaces the whole status and only carried Nodes forward. Every later writer (the worker's first write, the progress ticks, the failure path) leaves the field at its zero value, so the flag survived only the queued window, and both surfaces test isQueued first. The reachable consequence is that a failed removal reported itself as a failed install, which is exactly the shape the Activity page offers Retry for, and Retry installs: pressing it on a removal that failed re-downloaded the model. A running delete also rendered as "Installing model X" with a spinner, and a successful one as "Installed model X". Carry Deletion forward the way Nodes already is. A job is a delete or an install for its whole life; an unset flag means "no new information", not "this is an install". Pinned by Go specs on both the service and /api/operations: the existing Playwright specs were green only because they stubbed a payload the server could not emit. Also restore the operation's own status message on the Activity card. Phases and byte counters exist only on the managed-artifact path, so a legacy files: gallery model and every backend install rendered a sub-row with nothing in it but the verb. The strip stays terse on purpose. And give the strip's name a min-width floor: overflow: hidden zeroes its automatic minimum, so a long error squeezed the name down to "mod…" and the identity of the thing that broke was the first thing lost. primaryOperation is made module-private: its comment claimed the Activity page selected the same operation, but that page shows all of them, partitioned into failed and running, and never imported it. Assisted-by: Claude Code:Opus 5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(activity): read the operations record from PostgreSQL The Activity page's record of finished installs and removals was a 50-entry in-memory ring per frontend replica. In distributed mode that is the wrong place for it: each replica keeps its own copy, a replica added by a scale-out or a rolling deploy starts empty and never backfills, and "Clear history" clears only the replica that served the request, so the record reappears on the next poll routed elsewhere. The data is already in gallery_operations. Read it from there. GalleryStore gains ListTerminal and ClearTerminal, sharing a lifted terminalStatuses set with CleanOld so there is one definition of "finished". ListTerminal orders by updated_at, when the operation reached its terminal status, because the record reports what finished and when. OpCache.History and ClearHistory dispatch on whether a store is wired, so the HTTP handlers and the OpRecord JSON shape are unchanged and the page needed no change. A failed store read falls back to the local ring rather than blanking the page, and ClearHistory empties the ring as well so a database blip cannot resurrect a record the admin just cleared. The name derivation in recordTerminal is lifted into operationDisplayName and used by both paths, so the ring and the store cannot name the same operation differently. Also fixes a pre-existing bug the store path made visible: the backend channel hardcoded op_type "backend_install" even for a removal, while the model channel derives model_install/model_delete from op.Delete. Both channels carry the same ManagementOp, whose Delete field the backend handler already branches on, so the backend channel now derives backend_delete the same way. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): keep a cancelled operation cancelled, and report a failed clear Review follow-up on the store-backed Activity record. A cancelled install was recorded as a failure. The cancel handler persists "cancelled" synchronously, then the handler goroutine unwinds with the context error and Start hands that to updateError unconditionally, which overwrote the row with "failed: context canceled". The page rendered a cancelled install as a red failure card offering Retry, with a raw context error as the reason. Fixed in GalleryStore rather than in Start, because an operation finishes once and the paths that retire one are not mutually exclusive: UpdateStatus now refuses to rewrite a row that already reached a terminal status. That also pins updated_at to when the operation really finished, which is the key the record is ordered by, and Create's upsert now freezes the same columns so a worker dequeuing an operation the admin cancelled while it was queued cannot reopen it as pending. ClearHistory returned nothing, so a failed delete logged a warning while the handler still answered 200. The admin watched the record clear and come back on the next fetch with nothing said about why. It now returns the error, the DELETE handler answers 500, and the store is cleared before the local ring so a failure leaves the fallback record intact rather than faking an empty one. Hydrate is the only reader that decides from op_type whether an operation is a removal, and it tested for "model_delete" exactly, so the backend_delete added in the previous commit hydrated as an install: a replica restarting during a backend removal rendered "Installing backend X". Both discriminations now go through IsDeleteOpType/IsBackendOpType so a fifth op_type cannot silently read as an install in whichever consumer was missed. Also: the backend channel now persists Cancellable as !op.Delete, matching the model channel; IsBackend falls back to the op_type prefix, since is_backend_op is only written by UpsertCacheKey and the rows needing the name fallback were reporting backend operations as models; and an unrecognized terminal status is logged rather than quietly filed as a success, which is what the comment already claimed. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): keep a reaped operation correctable by its real outcome The terminal-status freeze added in the previous commit was too wide. It froze "failed" alongside "completed" and "cancelled", and the stale reaper writes "failed" onto operations that are still going to run. The gallery worker is a single goroutine consuming both channels serially, so an operation queued behind a large download sits in "pending" with nothing bumping updated_at, and ReapStaleOperations gives up on it after 30 minutes. That used to be self-healing: the worker dequeued it, Create reset the row to "pending", and the operation reported its real outcome. With the freeze the row stayed "failed" forever while the install ran and succeeded underneath it: a red failure card offering Retry for a model that is installed, omitted from ListActive so no replica hydrates it, and no longer deduped cluster-wide by FindDuplicate. Freeze on ("completed", "cancelled") instead. That is all the cancelled-install fix ever needed, and it leaves a failure correctable by what actually happened. The set is separate from terminalStatuses, which ListTerminal, ClearTerminal and CleanOld all still want in full, because the two mean different things: a failure can be superseded by a real outcome, a completion or a cancellation is the real outcome. UpdateStatus now writes the error column unconditionally, so a corrected outcome drops the previous attempt's reason rather than being recorded as completed while still carrying "stale operation reaped" as its error. Also adds the route-level spec for the 500 branch of DELETE /api/operations/history, and trims a comment that credited the persisted cancellable column with more than it survives long enough to do. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): offer Cancel in the phase that can honour it The cancellable flag was set at both ends of an operation's life and was wrong at both, in opposite directions. A queued operation is cancellable whatever it is. EnqueueModelOp and EnqueueBackendOp select on the operation context, so cancelling one that is still waiting releases the delivery goroutine and abandonQueued retires it: the worker never sees it, nothing is downloaded, nothing is deleted. markQueued nevertheless wrote Cancellable: !deletion, so a queued removal reported cancellable: false and the UI hid the Cancel button in the one window where pressing it both works and leaves no trace. A removal queued behind a large install was stuck there until the install finished. A running removal is not cancellable at all. DeleteModel and DeleteBackend take no context, and modelHandler only checks the operation context after the call returns, so a "cancelled" verdict would land after the model was already gone. Both handlers nevertheless wrote Cancellable: true unconditionally at entry, ahead of the op.Delete branch, offering a Cancel button the server cannot honour. So the queued phase is more cancellable than the running phase, which is the reverse of the usual shape. markQueued now reports true unconditionally, and the handler-entry writes report !op.Delete. Both sites carry a comment saying why, because reading either one alone suggests the other is a bug. GalleryStore.Create keeps !op.Delete: it runs at dequeue, so its value already describes the running phase. Its comment now says so. Specs cover queued removal, queued install, running removal and running install through the handlers, plus the queued-removal case through /api/operations where the flag is consumed, plus the behaviour the whole asymmetry rests on: a removal cancelled while queued never reaches the worker and deletes nothing. No existing spec asserted the old values. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Write] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): clamp the installer message, and add a real-binary e2e spec Running the page against a real local-ai showed the legacy installer message wrapping to three lines and dominating the card: it embeds an absolute file path, so it is both long and a single unbreakable token. One line, ellipsised, full text in the title, matching what the error string already does. The spec that found it runs with no route stubbing at all. Every other spec here stubs /api/operations, which is how a payload the server cannot emit (isDeletion true on a live operation) stayed green through a full review while the UI rendered a removal as an install. It is skipped unless LOCALAI_REAL_BINARY is set, so CI is unaffected. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-28 19:33:48 +02:00
mudler's LocalAI [bot]	05ff401de8	fix(test): stop the backend-trace specs racing the lossy trace channel (#11146) ... RecordBackendTrace does a non-blocking send onto a 100-slot channel and drops when it is full, so tracing never stalls inference. The payload bounding specs pushed all 200 traces in one tight loop, which overruns that channel on a loaded machine: entries are dropped for good and the Eventually waiting for 200 can never be satisfied, no matter the timeout. CI hit this on master at 0a8a7fbb, settling at 158/200. Feed the traces in chunks of 50, draining after each, so the channel is never overrun and the count stays exact. Reproduced with 60 busy loops on a 20-core box at GOMAXPROCS=2: 0/12 runs passed before, 12/12 after. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-27 22:38:23 +02:00
Tai An	a7fa678d83	fix(tts): forward the OpenAI speed field to the backend (#11097) (#11120) ... * fix(tts): forward the OpenAI speed field to the backend (#11097) /v1/audio/speech accepted the documented OpenAI `speed` field and then dropped it: schema.TTSRequest had no Speed member, so the value never reached proto.TTSRequest and the request returned 200 with an unchanged playback rate. Accept speed and normalise it into the existing per-request params map, which core/backend forwards verbatim to the backend. An explicit params["speed"] still wins, and a value outside the documented 0.25-4.0 range is now rejected with 400 instead of being silently ignored. Signed-off-by: Anai-Guo <antai12232931@outlook.com> * fix(tts): distinguish explicit speed=0 from an omitted field Make TTSRequest.Speed a *float32 so an explicit `"speed": 0` (invalid, below the documented 0.25 minimum) is rejected with 400 instead of being treated as unset and silently defaulted. An omitted field stays nil and leaves the backend default untouched. Add a request-boundary regression that distinguishes an omitted speed from an explicit zero, addressing review feedback. Signed-off-by: Anai-Guo <antai12232931@outlook.com> * docs: drop the speed field from the TTS docs Per review: no backend consumes params.speed today, so documenting it would be misleading. The API-level plumbing and validation stay. Signed-off-by: Anai-Guo <antai12232931@outlook.com> --------- Signed-off-by: Anai-Guo <antai12232931@outlook.com>	2026-07-27 19:03:32 +02:00
Anupam Mediratta	3698361510	fix: upgrade hono to 4.12.25 (CVE-2026-54290) (#11023) ... * fix: CVE-2026-54290 security vulnerability Automated dependency upgrade by OrbisAI Security Signed-off-by: orbisai0security <mediratta@gmail.com> Signed-off-by: Anupam Mediratta <mediratta@gmail.com> * fix(deps): override hono transitive dep to eliminate CVE-2026-54290 Add package.json `overrides` field to force hono@4.12.25 across the entire dependency graph, including the transitive copy pulled in by @modelcontextprotocol/sdk. Previously bun.lock retained a scoped `@modelcontextprotocol/sdk/hono` entry resolved to the vulnerable hono@4.12.8; the override removes that entry so only the patched version ships. Assisted-by: Claude Code:claude-sonnet-4-6 Signed-off-by: Anupam Mediratta <mediratta@gmail.com> --------- Signed-off-by: orbisai0security <mediratta@gmail.com> Signed-off-by: Anupam Mediratta <mediratta@gmail.com>	2026-07-27 19:02:37 +02:00
dependabot[bot]	4f883c86a9	chore(deps): bump postcss from 8.5.15 to 8.5.23 in /core/http/react-ui in the npm_and_yarn group across 1 directory (#11106) ... chore(deps): bump postcss Bumps the npm_and_yarn group with 1 update in the /core/http/react-ui directory: [postcss](https://github.com/postcss/postcss). Updates `postcss` from 8.5.15 to 8.5.23 - [Release notes](https://github.com/postcss/postcss/releases) - [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md) - [Commits](https://github.com/postcss/postcss/compare/8.5.15...8.5.23) --- updated-dependencies: - dependency-name: postcss dependency-version: 8.5.23 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-07-26 23:13:42 +02:00
Tai An	53006bb8e1	fix(realtime): accept legacy 'modalities' alias for output_modalities (fixes #11103) (#11104) ... * fix(realtime): accept legacy 'modalities' alias for output_modalities OpenAI's Realtime *beta* used the field name `modalities`; the GA field is `output_modalities`. LocalAI only binds `output_modalities`, so a client sending the still-common beta field `modalities: ["text"]` has it silently dropped by encoding/json and the session falls back to audio: TTS runs and the client receives large response.output_audio.* frames even though it asked for text-only. Accept `modalities` as an alias on both session.update (RealtimeSession) and response.create (ResponseCreateParams). The GA `output_modalities` wins when both are present, so GA clients are unaffected. Applied at the two existing resolution points via a small modalitiesWithAlias helper. Fixes #11103 Signed-off-by: Anai-Guo <antai12232931@anaiguo.com> * test(realtime): add JSON-boundary regression for modalities alias Decode representative session.update and response.create payloads that carry only the legacy beta `modalities` key and assert the effective output modality resolves to text (not audio), reproducing the exact expressions used in updateSession and triggerResponseAtTurn. This guards against a wrong JSON tag or a missed call site letting encoding/json drop the alias silently. Also document output_modalities (and the accepted legacy modalities alias) for text-only sessions in the realtime feature docs. Signed-off-by: Tai An <antai12232931@outlook.com> --------- Signed-off-by: Anai-Guo <antai12232931@anaiguo.com> Signed-off-by: Tai An <antai12232931@outlook.com> Co-authored-by: Anai-Guo <antai12232931@anaiguo.com>	2026-07-26 23:09:30 +02:00
mudler's LocalAI [bot]	5d57c08c6e	feat(distributed): cache staged-artifact hashes and publish the model load lifecycle (#11121) ... feat(distributed): cache staged-artifact hashes and publish load lifecycle Every load request re-hashed every staged artifact on the controller (probeExisting and the upload path both re-read the full file), which for a large multi-file model on NAS-backed storage is minutes of pure re-reading per request even when nothing changed - observed as ~9 minutes of "Upload skipped (file already exists with matching hash)" before every avatar generation. Cache the local hash in the same .sha256 sidecar the worker-side transfer server already maintains, invalidated whenever the sidecar is older than the file. The whole staging+loading phase was also invisible: the NodeModel row was only written after LoadModel succeeded, so /api/nodes and the UI showed nothing while a cold load spent 10+ minutes staging - indistinguishable from nothing happening. Publish the lifecycle instead: "staging" as soon as the node is chosen, "loading" when the checkpoint load starts, and the existing "loaded" on success, with the row removed on any failure so a dead load does not leave a phantom replica. The early row also reserves the replica slot against concurrent schedulers. The nodes view already renders non-loaded states on model chips; style "staging" like "loading". Audited every state-filtered registry/router query: eviction, routing, reconciler and idle-model queries all filter state='loaded' explicitly, so the new transitional rows are visible to observability surfaces but inert to scheduling decisions (except slot occupancy, intentionally). Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-25 16:05:44 +02:00
mudler's LocalAI [bot]	2d889e61a6	feat(backend): add magpie-tts-cpp text-to-speech backend (#11115) ... * feat(backend): add magpie-tts-cpp text-to-speech backend Add a Go + purego backend wrapping the magpie-tts.cpp ggml port of NVIDIA's Magpie TTS Multilingual 357M (encoder + autoregressive decoder over NanoCodec tokens), producing 22.05 kHz mono audio in 5 baked voices (Aria, Jason, John, Leo, Sofia; case-insensitive names or indices 0-4) across 9+ languages from a single self-contained GGUF. Mirrors qwen3-tts-cpp / moss-tts-cpp: dlopen the static-ggml shared library, bind the flat magpie_tts_capi_* C-API via purego (no local C shim needed, the upstream .so exports it directly), and serve the gRPC TTS + TTSStream methods behind base.SingleThread (the C context is not reentrant across synthesize calls). The backend CMakeLists translates the Makefile's -DGGML_{CUDA,METAL,VULKAN,HIP} flags into upstream's MAGPIE_GGML_* toggles (upstream FORCE-overwrites the ggml cache entries from those), pinned to magpie-tts.cpp v0.1.1 (e3f3dd1ebe22b64e7405f93b519f2d1930712568), which statically links ggml into libmagpie-tts.so (ldd shows only system libs). Wires the full registration: backend-matrix.yml (CPU amd64/arm64, CUDA 12/13, Intel SYCL f16/f32, Vulkan amd64/arm64, ROCm, NVIDIA L4T + L4T CUDA 13, and Darwin metal), backend/index.yaml metas and image entries, the root Makefile build targets, the changed-backends backend-filter path mapping, the bump_deps auto-bump matrix, a test-extra per-backend smoke job, the /backends/known pref-only importer entry, the backend capabilities map (TTS + TTSStream, no voice cloning), and the README / compatibility-table docs rows. Verified locally: unit + e2e Ginkgo suites pass against the real q8_0 GGUF (22.05 kHz mono WAV, RMS > 0.01), a live gRPC LoadModel + TTS round-trip returns valid non-silent audio, and the pre-commit gates (make lint, make test-coverage-check) pass, run manually with LOCALAI_TEST_HTTP_PORT overriding the locally-occupied 9090. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * gallery: add magpie-tts-cpp model entries (q8_0 + f16) Add the Magpie TTS Multilingual 357M GGUFs from mudler/magpie-tts.cpp-gguf to the model gallery: q8_0 (~624 MB, near-lossless, fastest decode, recommended) with an f16 (~784 MB) variant, both served by the magpie-tts-cpp backend. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * magpie-tts-cpp: bump pin to rewritten upstream v0.1.1 SHA Upstream history was rewritten to purge accidentally committed build artifacts; v0.1.1 now resolves to 6f7696cf. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-25 08:38:48 +02:00
mudler's LocalAI [bot]	05e16e0fa8	chore: remove local pre-commit gates (#11116) ... Remove the versioned pre-commit hook and its installer while retaining CI coverage and conformance checks. Assisted-by: Codex:gpt-5 Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-25 01:19:32 +02:00
mudler's LocalAI [bot]	6584db992f	fix(nodes): never schedule a model onto a node that cannot store it (#11054) ... * fix(nodes): never schedule a model onto a node that cannot store it A worker whose models filesystem was 100% full kept advertising `status: healthy`, stayed a scheduling candidate, was picked to host a 70 GB video model, accepted the staging request, transferred ~17 GB and only then failed: staging .../whisper-large-v3/model.fp32-00001-of-00002.safetensors: upload to node b7bacbf4-... failed with status 500: writing file: /models/longcat-video-avatar-1.5/...: no space left on device The node was at 937G/937G/0-avail. Total elapsed before the truth surfaced: 16 minutes, for a decision that could never have succeeded. The worker health signal only ever proved liveness. `/readyz` (WorkerReadiness/NATSReadiness) checks the NATS link; `status: healthy` in the registry is driven by heartbeat recency. Node capacity carried VRAM and RAM but no disk figure at all, and the router compared model size against VRAM only — nothing anywhere looked at free space on the filesystem that staging actually writes to. Report it, then use it: - Workers now measure the filesystem backing their MODELS directory (not `/` -- staged weights land in the models path, and that mount is very often separate) and report `total_disk`/`available_disk` on registration and on every heartbeat. Free disk moves faster than VRAM under staging traffic, so the per-heartbeat refresh matters. - The SmartRouter drops nodes that cannot store the model before it picks one. The requirement comes from `modelPayloadBytes` -- the same local paths `stageModelFiles` uploads, already computed for the size-derived load budget -- plus a 5% / 1 GiB margin, rather than a fixed percentage of the node's disk. A percentage threshold would take a small-but-usable node out of rotation for models it could hold, and on a homogeneous cluster would strand every node at once. - When no node fits, scheduling fails immediately with an error naming the requirement and each node's free space, instead of picking one and discovering it mid-transfer. Two deliberate non-changes. Low disk does not mark a node `unhealthy`: the check is per model, so a node too small for one model stays a valid target for smaller ones. And `total_disk == 0` means "does not report disk" (pre-upgrade worker, or a failed stat), not "full" -- such nodes pass through untouched so a rolling upgrade never empties the candidate pool. A genuinely full node is distinguishable: non-zero total, zero available. Registry read failures are logged and scheduling continues unfiltered; a database hiccup must not wedge a cluster. Free space is surfaced on the node detail page next to VRAM, since the incident's signature was a node that looked entirely healthy. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] * feat(nodes): make the disk-headroom check operator-controllable The admission check added in the previous commit had no off switch. A scheduler-side veto with no escape hatch is a liability: our size estimate can be wrong (deduplicating or compressing filesystems, a backend that fetches its own weights rather than loading the staged copy), and an operator who hits that has no way out but a downgrade. Add one knob with two surfaces that share a single source of truth: - `--distributed-disk-headroom-check` / `LOCALAI_DISTRIBUTED_DISK_HEADROOM_CHECK` (default true), following the `--distributed-prefix-cache` pattern for a default-on distributed feature. - `distributed_disk_headroom_check` in the runtime-settings registry, so it can be flipped without a restart from `POST /api/settings` and from Settings -> Distributed in the WebUI. Both write `DistributedConfig.DiskHeadroomDisabled`, and the SmartRouter reads that member LIVE on every scheduling decision through a closure over the application config rather than a value snapshotted at construction. Env/CLI sets the boot value, the runtime setting overrides it live, last write wins, and there is exactly one member to read. Snapshotting would have made the runtime toggle a no-op until restart. Disabled means WARN, not SKIP. Selection goes back to ignoring free disk -- byte for byte the pre-check behaviour -- but the check still runs, and when it would have rejected every node it says so, naming the knob that suppressed it. Going quiet when switched off would reproduce the exact condition that made the original incident expensive: a cluster doing something that could not work and saying nothing. Disabling is also logged once at startup. Warning only on the total-rejection case keeps it actionable rather than chatty on a heterogeneous cluster. Also fixes a false positive in the check itself: shared-models mode (LOCALAI_DISTRIBUTED_SHARED_MODELS) stages nothing at all -- every node already mounts this models directory at this path -- so demanding the full checkpoint size of free space per node would have rejected a cluster that needs no new bytes. The check is skipped there entirely. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-23 00:03:21 +02:00
mudler's LocalAI [bot]	f317da7c0f	fix(galleryop): make admitted operations queryable and survive a failed op (#11044) ... Two lifecycle defects observed on a 2-replica distributed cluster. The install endpoints mint a job UUID, hand the operation to an unbuffered channel, and answer HTTP 200 immediately. The gallery worker is a single goroutine that processes operations serially, and the first status write happens inside modelHandler/backendHandler — i.e. only once the worker actually starts the work. An operation queued behind a running install therefore had no status at all: GET /models/jobs/<uuid> answered "could not find any status for ID" and GET /models/jobs did not list it, so the endpoint reported success for work nothing could observe. On the paths that sent directly rather than from a goroutine, the same unbuffered channel blocked the HTTP handler for the whole duration of the in-flight install, which is how a replica came to accept no /models/apply at all while /readyz stayed green. Admission now goes through EnqueueModelOp/EnqueueBackendOp, which publish a "queued" status before handing the operation over, so a job ID is queryable from the instant it is handed out. Delivery selects on the operation's context, so cancelling a still-queued operation releases the delivery goroutine instead of stranding it on a send that will never be received, and an operation the worker never accepts becomes a terminal failure rather than a silent leak. The worker also had no panic containment. A panic in any handler propagated out of the single consumer goroutine and killed the process, taking every queued operation with it; it is now contained to the operation that caused it. The two ignored galleryStore.Create errors are logged, and the model and backend delete endpoints now run under the same ID they hand back — they previously ran under an empty ID and returned a status URL for a job that could never have a status. Second, an operation orphaned by a controller replaced mid-download kept reporting phase=downloading, processed=false, error=none while nothing was downloading. The PostgreSQL side does recover on its own (FindDuplicate ignores rows untouched for 30 minutes and CleanStale marks them failed), but the reaper only ever corrected the database. The in-memory statuses map that GET /models/jobs/<id> and /api/operations actually read was never corrected, so every replica kept serving the frozen tick indefinitely. ReapStaleOperations now reconciles the in-memory copy with the reap. Note that operation ownership is still not tracked: gallery_operations has a FrontendID column that nothing writes, so a live operation and one whose owner died are distinguished only by a 30-minute staleness timeout. Narrowing that window needs a lease/heartbeat mechanism and is out of scope here. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-23 00:02:23 +02:00
mudler's LocalAI [bot]	ff299df453	perf(http): gzip responses, cache hashed assets, bound the trace endpoints (#11056) ... Three measured HTTP-layer regressions on a live deployment, fixed together because they all shape the bytes on the wire. 1. No compression. The server sent no Content-Encoding regardless of what the client asked for, confirmed with curl straight at 127.0.0.1:8080 so it was not an ingress artefact. Adds gzip middleware, on by default and configurable via LOCALAI_DISABLE_HTTP_COMPRESSION and LOCALAI_HTTP_COMPRESSION_MIN_LENGTH (default 1024 bytes so tiny bodies are not wastefully wrapped). Streaming routes are skipped explicitly: an SSE Accept header, a WebSocket upgrade, and the completion / SSE / log-tail path prefixes, because whether a completion request streams is decided by the request body, which the middleware runs too early to see. Already-compressed formats (woff2, png, mp4, ...) are skipped too; gzip made those marginally larger. Measured over the embedded React build: JS+CSS 2815 KB raw to 808 KB gzipped (3.48x). 2. No cache headers on content-hashed assets. Vite hashes the filenames, so a given /assets/ URL can never change content, yet they shipped with no Cache-Control, ETag or Last-Modified, and the browser re-fetched the whole bundle on every navigation with no conditional request available. /assets/* now carries public, max-age=31536000, immutable. index.html stays no-cache so a deploy is picked up, and the unhashed locale JSONs get a short TTL rather than the immutable one. 3. Unbounded trace endpoints. /api/traces returned 21,033,606 bytes in 4.65s and /api/backend-traces 3,471,682 bytes in 1.50s, and the admin UI polls both every few seconds. The ring buffer holds up to 1024 entries, each embedding full input_text payloads. Both list endpoints now take limit / offset / full, default to 50 entries, and strip the heavy fields (request and response bodies plus headers for API traces, body and data for backend traces) unless full=true. Every trace gets a process-lifetime ID and GET /api/traces/{id} and /api/backend-traces/{id} serve the full record, which is what the UI fetches when a row is expanded. The list body stays a JSON array; paging metadata rides in X-Total-Count, X-Trace-Offset and X-Trace-Limit. Reproducing the live shape in a test, the polled payload goes from 21,131,097 bytes to 7,201 bytes. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-22 22:51:25 +02:00
mudler's LocalAI [bot]	7a8db9b1f1	fix(ollama): set ContextSize via the embedded LLMConfig so the package builds (#11049) ... The num_ctx clamping specs added in #11032 construct their fixture with `config.ModelConfig{ContextSize: &existing}`, but ContextSize is not a direct field of ModelConfig: it belongs to LLMConfig, which ModelConfig embeds inline. Go allows reading a promoted field but not setting one in a composite literal, so the test file has never compiled: helpers_internal_test.go:33:31: unknown field ContextSize in struct literal of type "github.com/mudler/LocalAI/core/config".ModelConfig This broke `make lint` on master from bf19758e0 onward, and because the typecheck failure takes down the whole package it also reds tests-linux and tests-apple on every PR branched after that commit. Use the same literal form the rest of the tree already uses for this field (see core/backend/options_internal_test.go). Worth noting the specs were not merely uncompiled but inert: #11032 is a DoS fix (an unauthenticated client raising the context ceiling drives KV-cache allocation), and its regression guard was never actually running. Verified the restored specs are functional by stubbing out the ceiling clamp, which fails the "does not let an oversized num_ctx raise an existing context ceiling" spec as intended. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-22 16:25:56 +02:00
dependabot[bot]	f92410b20b	chore(deps): bump fast-uri from 3.1.2 to 3.1.4 in /core/http/react-ui in the npm_and_yarn group across 1 directory (#11043) ... chore(deps): bump fast-uri Bumps the npm_and_yarn group with 1 update in the /core/http/react-ui directory: [fast-uri](https://github.com/fastify/fast-uri). Updates `fast-uri` from 3.1.2 to 3.1.4 - [Release notes](https://github.com/fastify/fast-uri/releases) - [Commits](https://github.com/fastify/fast-uri/compare/v3.1.2...v3.1.4) --- updated-dependencies: - dependency-name: fast-uri dependency-version: 3.1.4 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-07-22 15:55:10 +02:00
mudler's LocalAI [bot]	54f531f452	fix(mcp): bound MCP session connect so an unreachable server can't hang the widget (#10880) (#10884) ... Establishing an MCP session held the session-cache mutex across client.Connect with no per-connect timeout. An unreachable remote server (bounded only by the 360s httpClient timeout) or a stdio server whose initialize handshake never completes therefore blocked the caller and, because the mutex was held, every other MCP request for that model too. In the UI this shows up as the MCP "Servers" widget spinning forever. It is most visible for cloud-proxy models: their chat path bails out before the MCP tool block, so it never warms the session cache in the background. The widget's /v1/mcp/servers/<model> call is then the first and only code that connects synchronously, in the request foreground. The session, once established, stays bound to the shared context (it is cancelled later via the cached cancel func on eviction/shutdown), so we can't pass a WithTimeout context to Connect: firing the timeout would tear a healthy session down, and cancelling the shared context would also kill sibling servers that already connected. Instead connectMCP runs Connect on the shared context in a goroutine and stops waiting after the discovery timeout, returning an error for that one server without disturbing the others. A stalled goroutine is reaped when the model's sessions are cancelled. Applied to both SessionsFromMCPConfig and NamedSessionsFromMCPConfig. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-22 15:49:40 +02:00
Tai An	d7020708f2	fix(completions): reject empty PromptStrings in streaming to avoid index-out-of-range panic (#11028) ... * fix(completions): reject empty PromptStrings in streaming to avoid index-out-of-range panic The streaming branch of CompletionEndpoint only guarded len(config.PromptStrings) > 1 before unconditionally reading config.PromptStrings[0]. A completion request whose prompt field is an empty array, an array of non-strings, or omitted leaves PromptStrings with length 0, so PromptStrings[0] panics with index out of range and crashes the handler goroutine. Guard for exactly one prompt string instead, returning a clean error for the 0-length case as well as the pre-existing multi-prompt case. Signed-off-by: Tai An <antai12232931@outlook.com> * fix(completions): return 400 for malformed streaming prompt Reject streaming completion requests whose prompt does not resolve to exactly one string (omitted prompt, empty array, or a multi-element array) with an HTTP 400 before writing any SSE headers, instead of returning a plain error that Echo surfaces as a 500. Extract the guard into validateStreamingPromptStrings and cover the three reported payloads with a regression test. Fixes #11021 Signed-off-by: Tai An <antai12232931@outlook.com> --------- Signed-off-by: Tai An <antai12232931@outlook.com>	2026-07-22 09:29:11 +02:00
Tai An	bf19758e05	fix(ollama): cap num_ctx so it cannot wrap negative when cast to int32 (#11032) ... * fix(ollama): cap num_ctx so it cannot wrap negative when cast to int32 applyOllamaOptions copied a client-supplied options.num_ctx straight into cfg.ContextSize with only a > 0 check. That value is later cast to int32 before it reaches the backend (core/backend/options.go), so a num_ctx above math.MaxInt32 silently wrapped into a negative context size that was then sent to the LoadModel gRPC call. Both /api/chat and /api/generate share applyOllamaOptions, so both endpoints were affected. Cap num_ctx at math.MaxInt32 so the later cast stays positive, and add internal regression coverage for the overflow, in-range, and unset cases. num_ctx remains an intentional user override, so this does not re-impose the hardware-aware auto context clamp; that policy choice is left to maintainers. Fixes #11022 Signed-off-by: Tai An <antai12232931@outlook.com> * fix(ollama): clamp num_ctx to model context ceiling, not just int32 Per review on #11032: capping only at math.MaxInt32 still let an unauthenticated request replace the hardware/model-derived context limit with ~2.1B tokens, so a real backend could attempt a catastrophic KV-cache allocation. Treat any existing positive cfg.ContextSize as the server ceiling and clamp num_ctx down to it (smaller values still honored), while retaining the int32-safe bound when no smaller ceiling exists. Shared by /api/chat and /api/generate via applyOllamaOptions. Add regression coverage proving num_ctx=2,000,000,000 cannot replace an existing 4096/8192 ceiling. Signed-off-by: Tai An <antai12232931@outlook.com> --------- Signed-off-by: Tai An <antai12232931@outlook.com>	2026-07-22 09:28:02 +02:00
dependabot[bot]	4ce67ccb84	chore(deps): bump body-parser from 2.2.2 to 2.3.0 in /core/http/react-ui in the npm_and_yarn group across 1 directory (#11016) ... chore(deps): bump body-parser Bumps the npm_and_yarn group with 1 update in the /core/http/react-ui directory: [body-parser](https://github.com/expressjs/body-parser). Updates `body-parser` from 2.2.2 to 2.3.0 - [Release notes](https://github.com/expressjs/body-parser/releases) - [Changelog](https://github.com/expressjs/body-parser/blob/master/HISTORY.md) - [Commits](https://github.com/expressjs/body-parser/compare/v2.2.2...v2.3.0) --- updated-dependencies: - dependency-name: body-parser dependency-version: 2.3.0 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-07-21 15:35:01 +02:00
dependabot[bot]	2a8eb5a04b	chore(deps): bump the npm_and_yarn group across 1 directory with 5 updates (#11011) ... Bumps the npm_and_yarn group with 5 updates in the /core/http/react-ui directory: | Package | From | To | | --- | --- | --- | | [dompurify](https://github.com/cure53/DOMPurify) | `3.4.0` | `3.4.11` | | [hono](https://github.com/honojs/hono) | `4.12.18` | `4.12.31` | | [qs](https://github.com/ljharb/qs) | `6.15.0` | `6.15.3` | | [react-router](https://github.com/remix-run/react-router/tree/HEAD/packages/react-router) | `7.13.1` | `7.18.1` | | [undici](https://github.com/nodejs/undici) | `7.25.0` | `7.28.0` | Updates `dompurify` from 3.4.0 to 3.4.11 - [Release notes](https://github.com/cure53/DOMPurify/releases) - [Commits](https://github.com/cure53/DOMPurify/compare/3.4.0...3.4.11) Updates `hono` from 4.12.18 to 4.12.31 - [Release notes](https://github.com/honojs/hono/releases) - [Commits](https://github.com/honojs/hono/compare/v4.12.18...v4.12.31) Updates `qs` from 6.15.0 to 6.15.3 - [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md) - [Commits](https://github.com/ljharb/qs/compare/v6.15.0...v6.15.3) Updates `react-router` from 7.13.1 to 7.18.1 - [Release notes](https://github.com/remix-run/react-router/releases) - [Changelog](https://github.com/remix-run/react-router/blob/react-router@7.18.1/packages/react-router/CHANGELOG.md) - [Commits](https://github.com/remix-run/react-router/commits/react-router@7.18.1/packages/react-router) Updates `undici` from 7.25.0 to 7.28.0 - [Release notes](https://github.com/nodejs/undici/releases) - [Commits](https://github.com/nodejs/undici/compare/v7.25.0...v7.28.0) --- updated-dependencies: - dependency-name: dompurify dependency-version: 3.4.11 dependency-type: direct:production dependency-group: npm_and_yarn - dependency-name: hono dependency-version: 4.12.31 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: qs dependency-version: 6.15.3 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: react-router dependency-version: 7.18.1 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: undici dependency-version: 7.28.0 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2026-07-21 09:40:42 +02:00
mudler's LocalAI [bot]	d7e04dcc32	fix(openresponses): make responses visible and cancellable across replicas (#11000) ... In distributed mode the Open Responses store is process-local: a sync.OnceValue over a map behind an RWMutex. With several frontend replicas behind a round-robin load balancer, every request that lands on a replica other than the creator misses. Measured on a live 2-replica cluster (#10993): the same response id returns 200 on the creating replica and 404 on its peer, and a cancel on the peer returns 404 without ever invoking CancelFunc, so generation runs to completion on the other replica while the caller is told the response does not exist. previous_response_id chaining fails through the same lookup. Split the state by what can actually cross a process boundary: - Replicated: response metadata (request, response resource, owner, expiry, stream/background flags) via syncstate.SyncedMap, the same component finetune, quantization and agent tasks already use. A local miss in Get/FindItem now falls back to it and returns a read-only remote view, so polling and chaining resolve on any replica. - Delegated: cancellation. context.CancelFunc is a function pointer and exists only in the creating process, so a cancel that lands elsewhere is broadcast on responses.<id>.cancel and applied by whichever replica holds the function. The broadcast is fire-and-forget rather than request/reply: if the owner crashed or was scaled down nobody answers, and the handler must not block on a reply that will never come. The replicated status moves to cancelled either way, which is truthful, since a dead owner's generation died with its process. - Refused: streaming resume. The resume buffer is a byte log plus a live notification channel and cannot be replicated without shipping every token over the bus. A resume that reaches the wrong replica now returns HTTP 409 naming the owning replica via the new ErrResponseNotLocal, instead of an empty event list that looks like a finished stream. It is deliberately distinct from ErrOffsetLost, which means the owner's buffer evicted the requested events. Standalone deployments never call EnableDistributed and keep exactly the previous process-local behaviour. Fixes #10993 Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-20 23:06:33 +02:00
mudler's LocalAI [bot]	65bdbc4ee3	fix(http): make /readyz reflect startup readiness, plus gitignore and coverage-ratchet fixes (#10989) ... * fix(http): make /readyz reflect startup readiness instead of always 200 /readyz was registered as a static handler returning 200 unconditionally, so it carried no information: it was green whenever it could be reached at all. Readiness could not distinguish "serving" from "still starting", and any future change that started the HTTP listener earlier would silently turn the probe into a lie. Track startup completion on the Application (atomic flag, flipped at the very end of New() on the success path only) and have the readiness handler consult it per request, returning 503 with a small JSON body while startup is in progress. A nil readiness source fails open so embedders keep the historical behaviour. /healthz is deliberately left readiness-independent. Liveness and readiness answer different questions, and failing liveness during a long preload makes an orchestrator restart the pod mid-download so the preload never finishes. This matters because since #10949 the startup preload materializes HuggingFace artifacts for managed backends: tens of GB for a large model (31 GB observed on a live cluster). Both probes stay in quietPaths and stay exempt from auth. Note the listener is still started only after New() returns, so today the not-ready state is not observable over HTTP. Moving the listener earlier is a separate, deliberate decision and is not made here. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * chore(gitignore): anchor the mock-backend pattern so its source dir is traversable The bare `mock-backend` pattern matched the *directory* tests/e2e/mock-backend/, not just the binary built into it. Git will not descend into an ignored directory even for tracked files, so `git add tests/e2e/mock-backend/main.go` required -f. This was hit while working on #10970. Anchor it to the artifact's full path. The built binary stays ignored (it is also covered by tests/e2e/mock-backend/.gitignore) while the source directory becomes traversable again. Verified with `git check-ignore -v`: a new source file under tests/e2e/mock-backend/ is no longer ignored, and the binary produced by `make build-mock-backend` still is. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * chore(coverage): raise the coverage ratchet from 48.5% to 54.2% The committed baseline had drifted well below reality: it still read 48.5% while a full instrumented run measures 54.2%. A stale-low baseline makes the gate meaningless — coverage could regress by more than 5 percentage points and still pass. Raising a ratchet is a deliberate act, not something to fold into an unrelated fix, so it gets its own commit. The headroom was earned by tests landed in #10946, #10947, #10948, #10949, #10956, #10967, #10968, #10970 and #10975. Measured with `make test-coverage` on this branch (the same instrumented run `make test-coverage-baseline` uses: ginkgo over ./pkg and ./core plus the in-process tests/e2e suite, --covermode=atomic, --coverpkg over core/... and pkg/..., generated protobuf excluded). The run completed with exit 0 and zero spec failures; the total was then written with the exact command the test-coverage-baseline target uses: go tool cover -func=coverage/coverage.out \ | awk '/^total:/{gsub(/%/,"",$NF); print $NF}' > coverage-baseline.txt Verified afterwards with scripts/coverage-check.sh, which reports OK. Note the measured figure includes the readiness specs added earlier on this branch, so it is a demonstrated floor rather than an estimate. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-20 21:45:27 +02:00
mudler's LocalAI [bot]	83a0f16a21	feat(gallery): let one gallery entry offer several builds of the same model (#10943) ... * feat(system): expose raw detected capability for model meta resolution Model meta gallery entries express hardware fallback through candidate ordering rather than a capability map, so they need the undecorated detected capability string without Capability's default/cpu fallback chain. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(system): drop duplicate capability accessor, cover DetectedCapability ReportedCapability was added with a body identical to the existing DetectedCapability. Keep one accessor and move the specs onto it, since DetectedCapability had no direct coverage of its no-fallback behavior. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): parse IEC binary size suffixes (KiB..PiB) ParseSizeString accepted only SI suffixes, so a "20GiB" floor was rejected outright. Model and VRAM sizes are conventionally quoted in IEC units, and silently reading GiB as GB would understate a floor by about 7%. Purely additive: these inputs previously returned an unknown-suffix error. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add Candidate type for meta model entries Candidate is one option in a meta entry's ordered variant list. It names a concrete gallery entry and declares when that entry suits the host. EffectiveMinVRAM resolves the VRAM floor, letting an authored min_vram win over a nightly-inferred one. An unparseable floor errors instead of being treated as absent: swallowing a typo would turn a constrained candidate into an unconstrained one and select a too-large variant rather than fail loudly. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add hardware-aware model variant resolver Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): allow gallery model entries to declare variant candidates A gallery entry with a non-empty candidates list is a meta entry: it names an ordered list of concrete entries and resolves to the first one the host can satisfy, instead of describing model files directly. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): resolve meta model entries to hardware-appropriate variants at install Meta gallery entries carry an ordered candidate list; at install time the first candidate the host satisfies is resolved and its payload installed under the meta's name, so the model keeps a stable name regardless of which variant backs it. The resolution is recorded in the installed gallery config so a reinstall honors a prior pin and operators can see the backing variant. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): key meta pin recall on the installed name and detach resolved entries Six review findings on the meta-entry install path. Pin recall was keyed on the gallery entry name while applyModel writes the record under the install name (req.Name when supplied), so a meta installed under a custom name with a pin lost that pin on reinstall and was silently re-resolved onto a different variant, possibly swapping its backend. Compute the install name with applyModel's own precedence before the recall. ResolveMetaModel returned a shallow struct copy, so the resolved entry's Overrides aliased the gallery entry's map and the install path's in-place mergo merge wrote the caller's request into the shared catalog. Detach Overrides, ConfigFile, AdditionalFiles, URLs and Tags. Not exploitable today only because this path re-unmarshals the gallery per call, which is a property nobody should have to rely on. Also: overlay the meta's name onto the persisted config for meta installs so the gallery file no longer records the variant's name; move the pinned-VRAM warning below the variant validation so a pin naming a nonexistent entry does not warn about VRAM before failing for an unrelated reason; and stop seeding config.URLs in the config_file branch, which duplicated every declared URL. Add seven network-free specs driving InstallModelFromGallery with a meta entry: variant payload wins over the meta's legacy url fallback, the resolution record round-trips to disk, a pin is recorded and honored on reinstall including under a custom install name, and the resolved entry does not alias the gallery's maps. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): deep-copy meta overrides and make two specs functional ResolveMetaModel detached the resolved entry's Overrides and ConfigFile with maps.Clone, which only copies the top level. Gallery overrides are nested in practice (parameters.model is near-universal) and the install path merges the caller's request with mergo.WithOverride, which recurses into nested maps and overwrites them in place, so the gallery entry's own inner maps were still reachable and still got rewritten by the last caller to install. Copy both maps all the way down instead, recursing through the container shapes a YAML decoder produces. ConfigFile is not mutated on the install path today, but it carries the same kind of nested payload and leaving it shallowly cloned would invite the bug back. Also fix two specs that passed whether or not their target fix was present: - "does not write the caller's overrides back into the gallery entry" re-read the catalog from disk, which re-unmarshals fresh structs and so cannot observe in-memory aliasing. It now asserts against the in-memory gallery entry and drives the real mergo merge. - "round-trips the resolution record to disk under the meta's name" asserted a name that is already correct in the config_file branch. It now drives the url branch via a file:// fixture, where the meta-name overlay actually applies. Both were verified red by reverting their fix. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(gallery): lint meta model entry invariants in index.yaml Adds Ginkgo specs that parse the shipped gallery/index.yaml and enforce the invariants that keep meta entries safe: a legacy url fallback equal to the final candidate's url, references only to existing non-meta entries, a min_vram floor on every candidate but the last-resort one, a capability drawn only from the vocabulary the system can report, and descending VRAM floors within a capability group. The capability check is the only compensating control for a typo there. Candidate matching is a case-sensitive exact comparison against SystemState.DetectedCapability(), so an unknown value never matches and falls through silently instead of erroring. The vocabulary therefore mirrors the raw return set of getSystemCapabilities(), which notably excludes "cpu": that is a fallback key inside Capability(capMap) on the meta backend path, never a reported capability. A CPU-only host reports "default". These pass vacuously until the pilot meta entry lands; the guard is intentionally in place before the thing it guards. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(gallery): close coverage gaps in the meta entry lint The ordering invariant grouped candidates by capability and asserted floors descend within a group. A candidate with an EMPTY capability matches every host, so it does not belong in its own group: it dominates every later candidate whose floor is at or above its own, across capability groups. Track a running minimum floor over the unconditional candidates instead, which subsumes the old same-group check for the empty capability. Every spec skipped non-meta entries, so with zero meta entries in the index all five bodies were no-ops. Aligning GalleryModel.IsMeta() with GalleryBackend.IsMeta(), whose semantics are deliberately opposite, would have made all of them pass while checking nothing. Extract each invariant into a helper over a slice of entries returning the violations it finds, and cover those helpers with synthetic fixtures so the logic stays tested at zero meta entries. The index-driven specs are now a thin application of already proven logic. Also assert the index parses non-empty, report every violation in one run rather than aborting on the first, and parse the index once for the suite. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(gallery): add nightly denormalization of meta model candidates Fills the read-only backend, quantization and inferred_min_vram fields on meta gallery candidates and opens a PR, modeled on the existing checksum_checker job. Computing these needs network access, so it happens nightly rather than at install time. An authored min_vram is never modified: a human who measured a real load knows more than a pre-download estimate does. The index is rewritten via yaml.Node rather than a document round-trip. A full round-trip reflows all ~26k lines of gallery/index.yaml, which would bury the computed values and make the nightly PR unreviewable. The rewrite touches only the three derived keys, so authored styling survives and a run that computes nothing leaves the file untouched. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ci): keep the gallery denormalize diff reviewable and self-healing The nightly denormalization job edits YAML nodes instead of round-tripping structs so its PR stays small enough for a human to review, but the write path undid that: yaml.Marshal re-encoded the node tree at yaml.v3's default 4-space indent and dropped the leading document marker, reflowing roughly 6000 lines around the handful of real changes. Encode through yaml.NewEncoder at the index's authored 2-space indent and restore the header. A write that changes three fields now changes three lines. Stale inferred_min_vram values were also never cleared. Both skip paths (an authored min_vram is present, or the candidate is the last resort) returned before touching the field, so a candidate that gained a floor or became the last resort after a reorder kept an inferred value that EffectiveMinVRAM reported as a real constraint, failing the meta lint with no way for the job to self-heal. Clear the field before both skips. The workflow discarded a whole night's work on any single failure: the program exits 1 when a candidate cannot be estimated, which aborted the job before the PR step, so one unreachable candidate blocked every other refresh indefinitely. Capture the status, open the PR with what was computed, mark the PR body as partial, and fail the run afterwards so the problem still surfaces. Also preserve the index's existing file mode instead of forcing 0644, and drop the redundant //go:build ignore tag, since Go already skips dot directories and the sibling modelslist.go carries no tag. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add nanbeige4.1-3b meta entry with hardware-resolved variants Adds the first real meta entry to the gallery index. It resolves to the Q8_0 build on hosts with at least 6GiB of VRAM and to the Q4_K_M build everywhere else, installing either payload under the stable name nanbeige4.1-3b. The entry carries a url equal to its final candidate's url. LocalAI releases that predate candidates support parse the index non-strictly and drop the key silently, so without that url they would list the entry and install nothing. A regression spec parses the index the way those releases do and asserts every meta entry stays installable for them. Also teaches core/schema/gallery-model.schema.json about candidates. The schema sets additionalProperties: false at the top level, so an author following CONTRIBUTING.md and adding the yaml-language-server comment would otherwise get a validation error on this entry. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): make candidate entries complete, installable entries Reworks hardware-resolved gallery variants after a design pivot. There is no longer a separate "meta" entry kind. A gallery entry is a normal, complete entry that may additionally carry candidates:, a list of hardware-gated upgrades over itself, and the entry is itself the last-resort candidate. The previous design relied on a bare url: as the fallback for LocalAI releases that predate candidates support. That fallback is empty in practice: none of the 80 gallery/*.yaml files carry a top-level files:, and 1216 of 1281 index entries carry their payload in the index entry itself, so a url alone yields a config template with nothing to download. Since every released LocalAI reads gallery/index.yaml live from master, merging a payload-less entry would have shown every existing user a model that installs to a broken state. Making the entry its own base candidate removes the problem at the root: old clients drop the candidates key and install the entry exactly as they do today. Resolution order is now explicit pin, then capability plus VRAM over the declared upgrades, then the entry itself. The entry ALWAYS installs: when its own min_vram or capability is unmet the installer warns and installs it anyway, because there is nothing below it and refusing would make the gallery behave worse the newer the client is. A pin naming the entry's own name is valid and is how an operator declines an upgrade. IsMeta() becomes HasCandidates(), ResolveMetaModel becomes ResolveVariant, and the persisted meta_name record key becomes entry_name. GalleryBackend.IsMeta() is a separate concept and is untouched. The lint drops the three rules the pivot makes wrong (url equality with the final candidate, no inline payload, unconstrained final candidate) and gains one: the entry's own floor must sit strictly below every candidate's, since a base that outranks a candidate makes that candidate unreachable. The pilot entry is now the existing nanbeige4.1-3b-q4, which gains a 2GiB floor of its own and a single 6GiB upgrade to nanbeige4.1-3b-q8, replacing the separate nanbeige4.1-3b entry added in d0d441bb4. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): select model variants by hardware fit, not authored order Gallery entries could already carry a list of alternatives, but selection was an authored, ordered, first-match policy: every candidate declared a `capability` string and the VRAM floors had to descend in a hand-tuned order. That pushed hardware knowledge onto whoever edits the gallery and made ordering load-bearing, so a reordered list silently changed what users installed. None of it was necessary. SystemState.IsBackendCompatible already derives hardware support from a backend name alone: it knows MLX and metal are Darwin-only, CUDA is NVIDIA-only, ROCm AMD-only, SYCL Intel-only. Selection can read that instead of asking authors to restate it. Authoring is now just a list of names: - name: qwen3.6-27b min_memory: 4GiB variants: - model: qwen3.6-27b-mlx-8bit - model: qwen3.6-27b-gguf-q8 min_memory: 28GiB and all the intelligence moved into the selector. Given a host it drops the variants whose backend cannot run here, drops those whose known memory requirement exceeds what the host has, and takes the LARGEST of what is left, because a bigger footprint is a higher quality quantization of the same model. A variant of unknown size is kept, since nothing proves it does not fit, but it ranks last so a proven fit always beats a guess. An explicit pin still wins outright, and if nothing survives the entry installs its own payload: the base always installs, this never refuses. Available memory is VRAM when a GPU was detected and system RAM otherwise, read through xsysinfo so a cgroup limit is honored and a container gets its own limit rather than the node's RAM. Capability disappears entirely, from the types, the schema and the lint. VRAM and RAM collapse into one `min_memory`, because a model's footprint is roughly the same wherever it lives and one figure is compared against whichever applies. The lint rules about ordering, the capability vocabulary and floor relationships are deleted with the hazards they described; what remains is that every variant names an entry that exists and does not itself declare variants, plus that any memory figure actually parses. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * gallery: size model variants with a live probe, drop the nightly denormalizer Selection needs each variant's size to decide whether it fits and to rank largest-first. That figure was written into the index by a nightly job, which made the gallery carry a derived value that could drift from the entry it was derived from. Derive it at install time instead. pkg/vram already sizes a model without downloading it, and the gallery UI already uses it: a remote GGUF header range-fetch, then an HTTP HEAD for the content length, then any declared size:. It caches its results, so reuse it rather than writing a second probing path. A probe failure must never fail an install, so an unprobeable variant is treated as unknown: it survives the memory filter, because nothing proves it does not fit, and it ranks last, so a known-good fit always beats a guess. If every probe fails, selection still terminates on the base entry. The probe is injected through ResolveEnv rather than called directly, for the same reason the backend compatibility check is: specs pin an exact size, or an exact failure, without reaching the network. With that in place three things are dead weight and go: - The nightly job and the fields it populated. Variant.Backend was redundant because the backend is resolved live from the referenced entry during selection, and Quantization was display-only that nothing read. - min_memory on the base entry. The base always installs and its floor could only warn, so it could not change any outcome. - The lint rules and schema entries for both. min_memory on individual variants stays, as the override for when the probed size is wrong. An authored figure now suppresses the probe entirely rather than merely outranking it, so it costs no round trip. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): expose model variants for selection over API, CLI and MCP A gallery entry may carry `variants:`, alternative builds of the same model. Selection already worked at install time, but nothing could see what an entry offered or ask for a specific build, so the feature was undrivable. Listing: `GET /api/models` now reports `variants` and `auto_variant` for the entries that declare variants. Each variant carries its resolved backend, its measured size and whether it fits this host. `auto_variant` is what installing without a choice would pick right now. The new gallery.DescribeVariants runs the same variantOptions + SelectVariant pass the installer runs, so the reported default cannot drift from what installing actually does, and HostResolveEnv is extracted so both derive the host and share pkg/vram's probe cache from one place. Performance: an entry that declares no variants returns early without touching the probe, so the ~1280 ordinary entries cost exactly what they cost before. Selection: `variant` is accepted on POST /models/apply, as a query param on POST /api/models/install/:id, on the gallery apply file/string request, as `local-ai models install --variant`, and as a parameter on the install_model MCP tool (both the httpapi and inproc clients). Empty means auto-select. An unknown variant name now fails the install naming what was requested. This closes a real hole: an entry declaring no variants short-circuits before selection runs, so a requested variant was previously dropped silently and the install reported success. startup.InstallModels ends in a variadic model list, so install options could not be appended to it; InstallModelsWithOptions is added alongside and InstallModels delegates to it. No caller signature changed. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): drop the redundant variant min_memory field Variant.MinMemory was an authored override for when the live probe misreads a variant's footprint. It duplicated an existing field: probeEntryMemory already passes the entry's declared size: into EstimateModelMultiContext, whose cascade prefers that declared size over its own guesswork. Correcting size: on the referenced entry fixes the figure for every consumer rather than only for variant selection, so min_memory shadowed the right answer. A variant is now nothing but a name. Its effective size is exactly the probe result, and an unknown stays unknown: it survives the filter and ranks last. EffectiveMemory loses its error return along with the field. The authored string was the only thing that could fail to parse, so the error had no remaining source and was propagating dead nil-checks through SelectVariant, DescribeVariants and the pin warning. Selection behaviour is unchanged. The specs covering probe-derived sizing, ranking, filtering, the unknown-size path, pin recall, entry/variant metadata split and deep-copy isolation all survive; the three install specs that needed a definite size now declare it through the referenced entry's own size:, which exercises the documented escape hatch directly. gallery/index.yaml is untouched: no entry ever carried the key. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): rank the entry's own build against its variants Variant selection pulled the declaring entry's own payload, the base, out of the candidate set and consulted it only once every declared variant had been rejected. Two real failures followed. A variant whose size the probe cannot determine deliberately survives the memory filter, because nothing proves it does not fit. As the only survivor it then won outright on any host, however small: a 2GiB machine installed an unmeasured variant in preference to the 4GiB build the entry itself ships, with no warning. 241 of the 1280 current index entries carry no files and no size, which is exactly that shape. "Largest wins" also broke whenever the base was the largest. An author writing a Q8 entry that offers a Q4 downgrade for small hosts, a natural shape that nothing in the lint, schema or docs discourages, had the Q4 installed on every large host instead. Make the base an ordinary participant. It is still exempt from both filters, so selection always terminates on something installable, but it is now ranked against the variants: a proven fit first and largest, then the base, then any variant whose size nothing could measure. Both failures disappear together. The base is probed for its size accordingly, which it was not before, because an unsized base would lose every contest to an unmeasurable variant. FellBackToBase is kept but narrowed to "no declared variant survived", rather than "the base was chosen", since the base now also wins on merit and that is not worth warning about. A recalled variant pin also became a permanent install failure. A pin the caller supplies on this request must stay fatal, but one recalled from ._gallery_<name>.yaml can be invalidated by any later gallery edit, and failing on it turned one rename into a model that could never be reinstalled or upgraded again short of deleting a dotfile the user has never heard of. A stale recalled pin is now dropped with a warning naming it, and selection runs as if it had never been recorded. Also drop the last textual reference to two abandoned designs from the DetectedCapability comment, correct the documented variants JSON example, which showed a memory_bytes of 0 that omitempty makes impossible, and remove an em dash from the install skill. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): budget variant memory from RAM when a GPU reports no VRAM Variant selection read its memory budget from VRAM whenever a GPU capability was detected, and from system RAM only when none was. Apple Silicon satisfies the first branch and fails the premise: arm64 macs report the metal capability unconditionally, without probing anything, while TotalAvailableVRAM has no discrete VRAM pool to find and returns zero. The budget therefore came out as zero on every Mac. Zero drops every variant carrying a known size, so the base build was installed on all of them however much memory the machine had. The feature was inert on the platform, and silently: falling back to the base is a legitimate outcome, so nothing looked wrong. Take VRAM only when it is actually a number, and fall back to RAM otherwise. On a unified-memory host RAM is not an approximation of the budget, it is the budget, since the GPU shares it. A discrete GPU whose VRAM could not be read also lands on RAM, which overstates what the card holds but understates nothing the host has; the previous zero understated both. An unreadable RAM figure still yields zero and still installs the base, so a genuinely unknown host is not talked into a larger download. This is what turned tests-apple red: "installs a fitting variant's payload under the entry's own name" asserts on selection, and the runner resolved to the base because its budget was zero. The added specs pin the branch directly rather than relying on a macOS runner to notice again. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add a model variant picker to the models gallery PR #10943 shipped the server side: a gallery entry may declare `variants:`, `GET /api/models` attaches `variants` and `auto_variant` to declaring entries, and `POST /api/models/install/:id` accepts a `variant` query parameter. Nothing in the UI consumed any of it, so the feature was not reachable from the browser. This wires it up. modelsApi.install takes an optional second argument and appends an encoded `?variant=` only when one is given, so every existing call site keeps sending exactly the request it sent before. On the models table, an entry that declares variants gets a split button. The primary Install still installs the auto-selected build, because auto is the default and the point of the feature; the chevron opens a menu for a deliberate override. It follows the Backends.jsx precedent: one shared Popover re-anchored per row, rendering .action-menu items, which brings Escape, outside-click and focus return along with it. An entry that declares no variants renders exactly as it did before. A variant that does not fit is dimmed but stays selectable, since the server honors an explicit choice with a warning rather than refusing it. memory_bytes is omitempty on the wire, so an absent key means the size is unknown and never zero. A single helper guards both the menu and the detail row, because formatBytes would otherwise render a falsy value as "0 B", which reads as "needs nothing". The expanded detail row gains a Variants section listing each build's backend, size, whether it fits, which is the entry's own build, and which one auto-selection would pick, built from the existing DetailRow helper and .badge classes. Eight Playwright specs cover the picker, including that plain Install sends no variant parameter and that choosing one sends it. One pre-existing assertion was scoped with .first(): the Variants section legitimately adds more llama-cpp badges to the detail row, which tripped strict mode. UI line coverage 49.42% -> 49.36% against a 40.0 baseline and 0.8pp tolerance; branch coverage rose 72.04% -> 72.66%. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): describe model variants from a companion endpoint Variant description probes each referenced entry's weight files over the network: an HTTP HEAD plus a ranged GET, serial, five seconds per probe with no aggregate deadline. Running it inline in GET /api/models made one listing cost (entries x variants) round trips. The Manage page fetches with items=9999, so at 200 declaring entries that is ~1000 serial probes, minutes of a blocked handler and gigabytes of range traffic for a single page load. Only one entry declares variants today, but the feature exists so that many will. Follow the precedent already set for VRAM estimates. The listing now reports only has_variants, a length check on loaded metadata that touches nothing, and GET /api/models/variants/:id returns the description for one entry, mirroring estimate/:id in route shape, auth and error handling. DescribeVariants itself is unchanged; only its caller moved. The picker fetches lazily at the two points where a user asks to see variants, opening the split-button menu and expanding the detail row, and caches per entry for the page session. An entry declaring no variants issues no request at all. A spec counts real HTTP hits on the weight files, so it goes red if description becomes reachable from the listing path again through any caller. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): filter the model gallery to entries that declare variants The gallery is heading towards showing parent entries and hiding the individual builds they reference, so a user sees one row per model rather than six quantizations of it. Adoption is a single entry today, so defaulting to that would leave a one-row gallery. This ships the migration-phase inverse instead: the default is untouched, and a toggle narrows the list to only the entries that declare variants. It previews the end state and changes nothing until someone asks for it. The filter is server-side, next to term/tag/backend/capability and above the pagination arithmetic. The listing paginates at 9 items, so narrowing on the client would leave totalPages and availableModels describing the unfiltered set and hand the user empty pages. It selects on HasVariants(), which reads already-loaded metadata, so it issues no variant probes. The parameter is named has_variants after the listing field it selects on, and is compared against "true" like the other boolean query params (all_users, save_checkpoint), so has_variants=false reads as absent. With it omitted the response is byte-for-byte what it was before. The control is the shared Toggle component, matching the fitsFilter toggle already on this page: same wrapper class, same icon and label shape, same localStorage persistence. Unlike fitsFilter it resets to page 1 on change, which a server-side filter has to do. Stacking the toggle with a tag or backend filter easily yields nothing while one entry declares variants, so the empty state now names the variants filter as the cause rather than leaving a user to conclude the gallery is broken. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): render gallery model descriptions as Markdown Gallery descriptions are Markdown, but the React UI dumped them raw, so a model whose description opens with an ATX heading showed a literal "# Qwen3.6-27B [](https://chat.qwen.ai)" in the list. Full-description areas now render through renderMarkdown (marked + DOMPurify), matching how Backends.jsx and the Manage detail panels already handle the same content: - Models.jsx expanded detail row - VoiceLibrary.jsx voice detail header The truncated one-line previews must not render block Markdown: a leading "#" would become an <h1> and wreck the row height and rhythm. They get a new stripMarkdown() helper instead, which reduces Markdown to a single line of readable plain text. It is used for the cell text and for the title tooltip, since a tooltip full of "[](url)" is no better than a cell full of it: - Models.jsx gallery table description cell - Manage.jsx model and backend resource-row descriptions stripMarkdown walks marked's lexer output rather than running regexes over the source, so what it strips is by construction what renderMarkdown would have rendered, and it needs no new dependency. Output lands in JSX text nodes, so React escapes it; no new dangerouslySetInnerHTML beyond the two full-description sites, both of which run DOMPurify. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): strip Markdown from the backends table description cell Commit b35d630cf fixed this for gallery models but left the Backends admin page with the same asymmetry: its detail panel renders the description through renderMarkdown, while the collapsed table row dumped the raw gallery string into both the cell body and the title tooltip. That is user-visible. 40 of the 949 entries in backend/index.yaml carry Markdown - insightface uses inline code backticks, others use lists and links - and backend descriptions also contain embedded newlines, so the one-line cell showed literal syntax. The cell now runs stripMarkdown over the description once and uses the result for the text and the title, matching Models.jsx and the ResourceRowDesc component in Manage.jsx. The '-' placeholder is preserved, and now also fires when a description reduces to nothing after stripping. The detail panel is untouched and no new dangerouslySetInnerHTML is introduced: stripMarkdown output lands in a JSX text node, so React escapes it. Three Playwright specs cover it: a description with a heading, inline code and a link renders as clean text with no literal syntax and no block element in the cell, the title tooltip carries the same stripped text, and a backend without a description still shows the placeholder. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ui(models): polish the variant detail view and scope rendered Markdown The gallery detail pane rendered every field through the same two-column label/value row, including the description. Multi-paragraph prose in a value cell ran eight rows tall at the top of the pane on a ~1200px measure, breaking the grid's rhythm exactly where the eye enters. Move it into its own full-width block above the table, capped at a 68ch measure, keeping the label. Rendered Markdown had no scoped typography anywhere in the app, so a description opening with `#` inherited the browser default 2em inside a 13px surface while a `##` further down was indistinguishable from body text. Add a reusable .markdown-body block mapping h1-h6, paragraphs, lists, links, code, blockquotes, images and tables onto the existing type scale, and apply it to every renderMarkdown() consumer: the models detail, the backends detail, both Manage details and the voice library detail. Rebalance the variants list so the name leads. Backend and size drop from badge/secondary weight to muted metadata; the FITS badge goes entirely, since it was true of nearly every row and so said nothing, while the variant that does not fit keeps a warning badge and a dimmed name. AUTO-SELECTED stays marked because it answers what a plain Install produces. Rows share the parent's grid tracks via subgrid so name, backend, size and status line up down the list instead of raggedly following name length. Finally, make each variant row actionable. It looked like a list of choices but was inert text, with per-variant install hidden behind the split-button chevron elsewhere; each row is now a button onto the existing handleInstall(modelId, variant) path, with hover, keyboard focus and a disabled state while an install is in flight. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): collapse the listing to one row per model The listing supported has_variants=true, which narrowed to entries that DECLARE variants. With adoption at three entries that showed three rows, which is useless; it was always a placeholder. Replace it with the view that is actually useful: the deduplicated gallery. Show every entry installable in its own right and nothing twice, which means the parents plus every entry nobody references, and hide only the builds another entry already offers as a variant, since those are reachable through their parent. The parameter is renamed to collapse_variants accordingly: the filter is no longer a predicate on a row's own metadata but a view over the whole gallery. Default stays off, so the response with the parameter absent is unchanged. VariantReferencedIDs never reports an entry that declares variants of its own, so parents are always visible. That guarantees every hidden entry has a visible entry offering it, and no chain can strand a row. Variant resolution already refuses to install such a reference, but the listing has to stay coherent in the presence of a gallery that has one rather than silently swallowing entries. Self-references and dangling references hide nothing. The referenced set is computed over the whole gallery rather than over what the other filters left, so an entry is hidden because a parent offers it and never because of what the user searched for. The pass is over metadata already in memory: it resolves nothing over the network and triggers no variant description or size probe, so the listing's zero-probe contract still holds. The UI toggle keeps its behaviour (persistence, page reset, clear filters) and becomes "One row per model", which says what the user gets. Its localStorage key moves too, since the stored value meant a different filter. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): show the collapsed model listing by default The gallery listing is what a user reaches for to answer "what can I install". Answering that with several rows for the same model, one per build, makes the reader do the deduplication the collapsed view already does, so the collapsed view is the one to land on. The UI now asks for collapse_variants=true unless the toggle says otherwise. The server default is deliberately untouched: a request with the parameter absent still returns the full listing, because other API clients depend on that response and collapsing it under them would be a breaking change. Opting out omits the parameter rather than sending false, so it asks for exactly the listing everyone else gets. The stored preference changes vocabulary from '1'/'0' to 'on'/'off'. The previous build wrote it from an effect that runs on mount, so a stored '0' recorded that the page had been opened rather than that anyone chose the expanded view, and honouring it would pin every earlier visitor to a default they never picked. Only the new vocabulary counts as a choice; a legacy '1' meant the collapsed view and is what the new default gives anyway, so no earlier deliberate choice is lost. Collapsing being the default also changes what the empty state may say about it. An opted-into filter can be named as the cause of an empty result; a default cannot, so the filters keep the top line and the collapsed view drops to a hint below it, shown only once filters are narrowing the set. For the same reason "Clear filters" now restores the collapsed default instead of switching it off, and the toggle alone no longer counts as a filter worth offering to clear. The label stays "One row per model": it describes the view the user is looking at rather than an action, so it reads the same whether it is opted into or out of. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * gallery: group alternative builds of the same weights under variants Sweep the gallery for entries that are alternative builds of the same weights (different quantization, precision, or runtime format) and declare them as variants of a single parent row, so the listing offers one row per model instead of one row per quantization and the installer picks the largest build that this host can actually run. 41 families over 95 entries, turning 54 entries into variants. The parent is the bare-named entry wherever one exists, so nothing changes about what any existing entry installs. Ranking already selects the largest fitting build regardless of which entry is nominally the parent, so the parent only decides the pathological case where nothing fits. For the ten families that have no bare-named entry, the smallest build is the parent, since that is the one that has to install when nothing fits. Grouping was verified against the actual model filenames rather than the entry names alone. Different parameter sizes, languages, finetunes, and products that merely share a name prefix are left as separate rows: the qwen3.6 APEX and pi-tune finetunes, the DFlash and MTP speculative-decoding pairings, English-only versus multilingual Whisper, the QAT versus non-QAT Gemma 4 weights, and the abliterated FLUX build are all distinct models. Six parents define YAML anchors that other entries pull in with a merge key, which would have handed their variants to every merging child. For the two depth-anything anchors that would have made fourteen unrelated entries advertise the base model's builds as their own. All 26 merging children therefore carry an explicit empty variants list, which overrides the merged key and is equivalent to the key being absent. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): rank model variants by host backend preference Variant auto-selection filtered candidates by whether their backend can run on the host, then ranked the survivors by size alone. The backend never influenced the choice beyond that gate, so a Mac offered both an MLX build and a llama.cpp build kept neither filtered and installed whichever was larger, leaving the native accelerated runtime unused. The same held for CUDA against CPU on NVIDIA and ROCm against Vulkan on AMD. Rank by the host's backend preference between the fit tier and size: fit stays a filter, preference decides among the builds the host can equally hold, and size still separates builds on equally preferred runtimes. The preference data stays in one declarative table in pkg/system, now read by a prefix lookup instead of a switch, so adding a capability or reordering one host's runtimes is a one-line edit and the gallery's ranking code carries no per-backend branching. MLX joins the metal rule ahead of metal itself, which is inert for the existing alias-resolution consumer because no alias group holds a candidate named for mlx. An unrecognised backend, an unrecognised capability and an absent preference list all collapse to the previous size-only ordering rather than erroring or dropping candidates. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): rank variants by engine name, not backend build tag Variant auto-selection ranked candidates with SystemState.BackendPreferenceTokens, but that function and the variant ranker speak different vocabularies. BackendPreferenceTokens returns BUILD TAGS ("cuda", "rocm", "sycl", "vulkan", "metal", "cpu"). It exists to match installed backend build directory names like "llama-cpp-cuda-12" during alias resolution in ListSystemBackends. Variant ranking instead matches a gallery entry's `backend:` value, which is an ENGINE NAME: "llama-cpp", "vllm", "vllm-omni", "sglang", "mlx" and the rest. No engine name in gallery/index.yaml contains "cuda", "rocm", "sycl" or "vulkan". preferenceRank matches by substring, so on an NVIDIA host the tokens [cuda, vulkan, cpu] matched neither "llama-cpp" nor "vllm", every candidate scored identically and size alone decided. The NVIDIA, AMD, Intel, darwin-x86 and vulkan rules were all inert. Only metal appeared to work, and only because the token "mlx" happens to equal an engine name. The mismatch does not error, it silently deletes the feature. Separate the two vocabularies. backendBuildTagPreferenceRules keeps the build tags and its original output for every capability, including metal, whose "mlx" token is removed again; its alias-resolution consumer is byte-identical to before. engineNamePreferenceRules is new, holds engine names, and is read by the new EnginePreferenceTokens, which HostResolveEnv wires into the renamed ResolveEnv.EnginePreference. Both tables sit adjacent under one block comment naming each vocabulary and each consumer, and share one lookup helper so their semantics cannot drift. On NVIDIA the order is vLLM, then SGLang, then llama-cpp: vLLM is the throughput engine and a model published with a vLLM build is published that way because that build is the one worth running. AMD and Intel get the same order, since rocm and intel builds of both serving engines ship. Metal prefers mlx over llama-cpp. Vulkan prefers llama-cpp, the only LLM engine with a Vulkan build. darwin-x86 and unknown capabilities are deliberately absent rather than guessed at, degrading to the size-only ordering that predates preference. preferenceRank stays generic and names no engine and no capability, so adding a runtime remains a one-line table edit. Specs pin the NVIDIA and metal rules through the live table and the real HostResolveEnv wiring, so emptying the engine table or wiring the build tag source back in both go red. A regression table asserts BackendPreferenceTokens' original output per capability, and mirrored locks assert neither table carries the other's vocabulary. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: record that variant selection ranks by engine before size A gallery entry can now declare variants, and selection ranks the builds a host can run by engine preference before size. Nothing told a contributor adding a backend that engineNamePreferenceRules exists, so a new engine would silently rank below every known one and lose to whatever build happened to be larger on hosts where it should have won. Document the step where a backend is added, warn against the sibling backendBuildTagPreferenceRules table (build tags, not engine names: the wrong table matches nothing, scores every candidate equally and disables the preference without erroring), and index it from AGENTS.md. Fix the authoring and user docs, which still claimed the largest surviving build wins. An author grouping builds under one entry has to be able to predict what a user gets, and size alone no longer decides it. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(cli,mcp): describe variant auto-selection as preference before size The CLI flag help and the install_model tool schema both still said auto-selection takes the largest build that runs. Ranking now puts engine preference ahead of size, so on NVIDIA a vLLM build wins over a larger llama.cpp one. An assistant reading the old schema would tell users the wrong thing. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): prefer llama.cpp over GPU serving engines on hosts with no GPU engineNamePreferenceRules had no row for the "default" capability, which getSystemCapabilities() returns both when no GPU is detected and when a GPU is present but under the 4 GiB VRAM floor. A missing row yields an empty preference list, which preferenceRank reads as "score everything equally", collapsing variant selection to size alone. That would be harmless if the hardware filter dropped GPU serving engines on such a host, but it does not. IsBackendCompatible derives support from the engine NAME, and "vllm" and "sglang" contain none of the darwin, cuda, rocm or sycl tokens it keys on, so they fall through to its closing "return true". A vLLM variant therefore survives on a CPU-only box and wins whenever its build is the larger of the two on offer: the machine installs vLLM in preference to llama.cpp. darwin-x86 had the identical hole. It was documented as a deliberate omission because nothing accelerates on an Intel Mac, which is true about acceleration and wrong about consequence: with every engine tied, download size decides. Add rows for both putting llama-cpp first. The GPU engines are enumerated behind it rather than left unmatched: an unmatched engine already ranks below every listed one, so llama.cpp would win either way, but unmatched engines also tie with each other and let size decide among them. Naming them fixes that order. MLX is left off the darwin-x86 row on purpose so it ranks last, since IsBackendCompatible admits darwin-tokened engines on that capability even though MLX needs Apple silicon. Preference orders survivors and never filters, so a model published only as a vLLM build is still installed on a host with no GPU; there is a spec for it. Surveyed every other value getSystemCapabilities() can return. nvidia, amd, intel and vulkan have rows; the l4t and cuda-refined values reach the nvidia row by prefix; "apple" and "" cannot reach the vendor fallthrough because the darwin and no-GPU branches return earlier. default and darwin-x86 were the only live holes. BackendPreferenceTokens and its build-tag table are untouched, and preferenceRank stays generic, naming no engine and no capability. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * gallery: prefer speculative-decoding builds when they fit Rank serving features between engine preference and size, so a host that can hold a DFlash or MTP build of a model's weights installs it instead of the plain build. Both answer faster for the same output, so whenever one survives the filters there is no reason to take the plain build. Precedence is now fit, then engine, then serving feature, then size. Engine outranks the feature deliberately: a serving feature makes the right engine faster, it does not make a wrong engine right, so a plain vLLM build still beats a DFlash llama.cpp build on NVIDIA. Fit outranks both, and a drafter pairing is strictly larger than the plain build, so the existing size filter drops it on a host too small for it before this axis is consulted. The order lives in a third preference table in pkg/system, alongside the build tag and engine name tables. It is the odd one of the three: not keyed by capability, because no hardware prefers a plain build over an equivalent faster one, and matched against whole segments of a gallery ENTRY NAME rather than as a substring of a backend value. Nothing on a gallery entry declares a serving feature, and tags are not a usable substitute: gemma-4-e2b-it:sglang-mtp carries an mtp tag while ornith-1.0-9b-mtp and qwen3.6-27b-nvfp4-mtp carry none. Entry names are author-supplied free text, unlike the closed engine vocabulary, so a short marker can turn up inside an unrelated word and whole segment matching is what keeps smtp-assistant from ranking as an MTP build. The block comment over the tables now documents all three together and states what each is matched against; the ranking code names no feature, so adding one stays a one-line edit to the table. 29c49203b rejected these entries as serving configurations rather than alternative builds of the same weights. The definition is now "alternative ways to serve the same model", which includes them, so regroup 14 entries under 12 parents. Judged by the files each entry points at: the qwen3.6, qwen3.5, qwen3 and deepseek pairings are the base GGUF plus a drafter, the gemma-4 QAT MTP entries are the same QAT weights at a different quantization plus an MTP drafter, and the two sglang MTP entries describe themselves as the same model served with speculative decoding. Left separate: qwen3.6-27b-mtp-pi-tune, a finetune with its own weights, and every entry whose base model LocalAI does not ship as its own row, which is the whole Qwopus line plus gemmable-4-12b-mtp, mimo-7b-mtp:sglang and qwen3.5-4b-dflash. None of the twelve parents defines a YAML anchor, so no variants key can leak through a merge key and no empty override was needed this time. The index was edited by line insertion only. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test: check env restore errors in capability and variant specs errcheck flagged ten unchecked os.Setenv and os.Unsetenv returns in the specs added while the pre-commit hook was being skipped. Restoring an env var is exactly the place a silent failure leaks state into the next spec, so assert on it rather than suppressing the linter. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): make the mtp tag authoritative for serving-feature ranking Variant auto-selection ranks survivors by fit, then engine, then serving feature, then size. The serving-feature lookup read only whole alphanumeric segments of a variant's entry name, because tags were inconsistent: every dflash entry carried a dflash tag, but only 7 of 20 MTP entries carried an mtp tag. Tag the 13 untagged MTP entries, then teach the lookup to read tags as well as names. A tag is now the authoritative signal and is compared whole and case-insensitively, which is safe precisely because a tag is a deliberate declaration rather than free text: there is no word-inside-a-word failure mode, so the segment splitting the name half needs is unnecessary there. The name check stays as a fallback rather than being replaced. Switching to tags only would have regressed the six already-grouped entries on the day it shipped, and would depend on tagging discipline that does not exist yet. The lookup still names no feature, so adding one remains a one-line edit to servingFeaturePreferenceTokens. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): make a declared tag the sole serving-feature signal Variant auto-selection ranks survivors by fit, then engine, then serving feature, then size. The serving-feature lookup recognised a speculative build by either a declared tag or a whole segment of its entry name. Drop the name half: a tag is now the only signal. A name is author-supplied free text and a naming convention is not a contract, so reading a marker out of one infers a capability nobody declared. The gallery already had the failure in it: the four NVFP4 entries name MTP-bearing weights while setting no option that enables speculative decoding, and being live variants they were winning the feature axis without answering any faster. overrides.options was considered as the replacement and rejected. It carries spec_type:draft-mtp / spec_type:draft-dflash, which is what actually turns the feature on, but that spelling is llama.cpp's config vocabulary: ds4 spells the same feature mtp_path and sglang spells it speculative_algorithm in a referenced config. Keying a cross-backend ranking decision on one backend's option syntax would rank the other backends' builds as plain. Options are the curation-time check instead, and never reach the selection logic. With no fallback left, tag correctness is load bearing, so audit every entry against the rule "tagged when the entry configures that feature, in whatever vocabulary its backend uses". Three entries configure MTP untagged and gain the tag (hy3, glm-5.2, qwythos-9b-claude-mythos-5-1m, all spec_type:draft-mtp with no marker in their names). Four carry the tag while configuring nothing and lose it: qwen3.6-27b-nvfp4-mtp, qwen3.6-35b-a3b-nvfp4-mtp, qwopus3.6-27b-coder-mtp-nvfp4 and qwopus3.6-27b-v2-mtp-nvfp4, whose only option is use_jinja:true. The dflash side was checked independently rather than assumed consistent: all five dflash entries declare spec_type:draft-dflash and all five are tagged, so it needed no edits. Four entries keep a tag that a literal spec_type-only reading would strip, because they configure MTP through a different backend: deepseek-v4-flash-q2-mtp via ds4's mtp_path/mtp_draft, and the three sglang entries via speculative_algorithm in their referenced configs. Stripping those would contradict the reason spec_type was rejected as the signal and would demote four genuinely faster builds to plain. The index was edited by line insertion and deletion only, never round-tripped through a serializer. A resolved-tag diff across all 1272 named entries, taken after merge keys are applied, shows exactly these 7 changing and no entry gaining or losing a tag through an anchor. The two specs that pinned the name fallback are inverted rather than deleted, since a name silently promoting a build is the regression worth guarding. The whole-token guard survives on the tag path, where smtp must still not match mtp. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): make deepseek-v4-flash variant targets installable Clicking install on deepseek-v4-flash failed with "invalid gallery model". The parent entry is fine, but all four entries it was grouped with declared neither url: nor config_file:, and applyModel needs one of the two to have anything to build a config from. They carry urls: (plural), the informational HuggingFace link list, which is a different field. None of the four was ever independently installable, so grouping them routed a previously-working install into a broken entry. Give each the url: the parent already resolves through. virtual.yaml is a no-op base, and applyModel passes overrides to InstallModel separately from the fetched config, so backend: ds4, the parameters and the ssd/mtp options all still land exactly as authored. This is the same pattern the parent and many other GGUF entries in the index already use. Add the lint rule that should have caught this. checkVariantReferences only proved a target exists and is not itself a parent, which is structural validity: an entry can exist, declare no variants, and still be uninstallable. checkVariantTargetsInstallable mirrors applyModel's precondition instead, and names the parent, the target and the missing fields, because whoever hits it is reading a gallery entry and has no reason to know applyModel exists. The two index-driven resolution specs live in their own Ordered container: an Ordered container stops at its first failure, so sharing one with the lint rules let a lint breach skip them silently. Nine further entries gallery-wide have the same defect and are unrelated to variants, so they are broken installs that predate this branch. They are left alone here rather than buried in a regression fix, and widening the rule to cover every entry is deferred with them so the gate can ratchet up in one step instead of needing a skip list. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): install entries with no url or config_file on an empty base applyModel had three branches: fetch a base config from url:, build one from an inline config_file:, or fail with "invalid gallery model". An entry declaring neither is now installed on an empty base config, with overrides: and files: supplying everything. This is what the ~345 entries pointing at gallery/virtual.yaml were already getting. That stub is five lines carrying name, description and license. description and license are overwritten from the gallery entry immediately after the fetch, and the name never reaches disk because InstallModel prefers the install name. Crucially applyModel passes model.Overrides to InstallModel as a separate argument rather than merging it into the fetched config, so nothing an author writes depends on that base existing. The fetch bought a round trip to GitHub and nothing else. That makes f4ef80173 the wrong fix, so it is unwound. The four url: lines it added to the deepseek-v4-flash variants are reverted: they are a pointless network fetch now, and the family installs without them. Relaxing the branch would hide a real authoring mistake, so a payload rule replaces the base-config rule. An entry with no url, no config_file, no overrides and no files installs nothing and would leave an empty model directory while reporting success, so it is refused by name. The caller's request counts toward the payload, because its overrides and files are merged into the install exactly as the entry's own are. urls: (plural) is the informational link list and does not count, which is what the four entries that shipped broken had and why they were still uninstallable. checkVariantTargetsInstallable asserted every variant target declares a url: or a config_file:, which is no longer true and would now reject correct authoring. checkEntriesInstallSomething pins what survives instead, and covers every entry rather than only variant targets: the hazard is a half-written stanza and a parent can be one as easily as a target. The old rule was scoped to targets precisely because nine unrelated entries would have failed a gallery-wide version; those nine are valid now, so the deferred ratchet happens here in one step. 1280 entries, zero violations. Those nine (aurore-reveil_koto-small-7b-it, lfm2-1.2b, the six liquidai_lfm2 entries and deepseek-v4-pro-q2-ssd) become installable for free. Each carries overrides: and files:, and one of them is driven through the real install path in a spec. The no-fetch spec is paired rather than bare: an assertion that nothing was fetched proves nothing unless something could have been, so a control runs the same fixture with a url: pointing at a base config that is not there and asserts the install fails. Only then does the identical fixture without the url passing mean the read was skipped. Follow-up, deliberately not here: the ~345 entries still naming virtual.yaml can drop their url:. That is 345 index edits with their own risk, and mixing them in would bury this change. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ui(models): let search bypass the variant collapse, drop the toggle The models page collapsed the gallery to one row per model by default and offered a toggle to see every individual build. Because the collapse composed with the search term, a build another entry offers as a variant could not be found by typing its name, so the toggle was the only way to reach those builds in the UI. A user who typed a name they knew existed got "no models found", which reads as "that model does not exist". Collapse is for browsing; search is for finding. An explicit search term now bypasses the collapse in the listing handler, so a name lookup returns matching entries whether or not a parent offers them. The term is trimmed once at the top of the handler, so whitespace is neither a search nor a bypass; previously an untrimmed blank term also narrowed the listing to whatever contained a space. Tag and backend deliberately do not bypass: they refine a listing the user is still reading rather than name an entry already known to exist. That makes the toggle redundant, so it goes, along with its i18n strings in all six locales, its localStorage persistence, its participation in "Clear filters" and the empty-state hint telling users to turn it off. The hint was doubly stale: it pointed at a control that no longer exists, and it was untrue exactly when a user has a search term, since searching now sees every build. The page always requests the collapsed listing. The stored preference key is left inert rather than cleaned up: nothing reads it, so a user who had the toggle off simply gets the collapsed view. collapse_variants stays on the API, off by default, because other clients want either view and the UI dropping its control is no reason to remove a working parameter. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): give the models gallery filter form a deliberate structure The filter area had accreted controls into one undifferentiated flow. The "Fits in GPU" toggle and the backend select were direct children of .filter-bar, the same wrapping container as the 18 taxonomy chips, so their position was decided by how many chips happened to wrap at the current width rather than by any layout intent. At narrow widths they were pushed past the right edge of that container's horizontal scroll and became unreachable entirely. Restructure into three bands inside the house .filter-bar-group wrapper that components/FilterBar.jsx already uses on Backends and the System tabs: 1. query scope: search plus the backend select 2. taxonomy: the chip row, alone, free to wrap 3. refinements: fits-in-GPU and context size, under a hairline rule The backend select leads the chips rather than trailing them because picking a backend disables the use cases that backend cannot serve, so it gates the row below it. Fits-in-GPU and context size share a band because they are one control group: the context size is the length the VRAM estimate is computed at, and that estimate is what the fits filter tests against. Chips had no visible keyboard focus indicator. The global focus ring is wrapped in :where(), so it carries the specificity of a bare :focus-visible, ties with .filter-btn and loses on source order, leaving focused chips showing their resting drop shadow. Restate the ring where it outranks both resting and hover. Also: aria-pressed on the chips, a real label association and aria-valuetext on the context slider (it steps over an index, so it announced "2"), disabled chip styling moved off inline styles, a prefers-reduced-motion block for the chip transition, and the hard-coded English "Context:" moved into all seven locales. No behaviour change: same filters, same state, same requests. Page reset on change, localStorage persistence and "Clear filters" verified unchanged. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): let the models recommendations panel fade into the background The "Recommended for your hardware" strip rendered at full height on every visit regardless of how many models were already installed, costing 186px at 1600px wide (287px at 1100px, where its cards wrapped to two rows) and pushing the first gallery row to y=554 / y=703. Make its prominence track how much the user still needs it. The panel now defaults to a one-line summary once anything is installed, and both the collapse choice and the existing dismissal persist: collapsed = explicit user choice, if one exists : installedCount > 0 The preference is three-valued on purpose. A boolean cannot tell "the user expanded it" apart from "the user has never chosen", and those need opposite handling when the installed count later crosses zero: someone who deliberately opened the panel on an empty instance should not have it collapse out from under them when their first model finishes installing. Collapsed keeps the card, icon, title and a suggestion count, so the panel is recovered by clicking what you are already looking at rather than by hunting. Expanded is unchanged, because for a user with nothing installed it was never the problem. Collapsed reclaims 145px at 1600 and 420, and 246px at 1100. Models.jsx gains a statsLoaded flag: stats initializes to installed:0, so reading it before the fetch resolves would render expanded and collapse a frame later, which is exactly the layout shove this removes. The dismissal key moves to the page's localai-models-* convention; the old localai_rec_models_dismissed is still read, never written, so an existing dismissal is honoured rather than resurrected by the rename. Accessibility: the disclosure is a real button whose accessible name is the visible title alone, with state on aria-expanded and aria-controls resolving in both states, because the grid is hidden via the hidden attribute rather than unmounted. That also keeps the four install buttons out of the tab order while collapsed. The app's global focus ring applies; no per-component outline is added, per the warning in App.css. Reveal animates opacity and transform only, never height, and both it and the chevron rotation are disabled under prefers-reduced-motion. Only en had a recommended block, so the other six locales were falling back to English for the whole panel. Translated the complete block rather than adding one orphaned key to files that would still render the title in English. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(downloader): recover from a leftover .partial on non-HTTP URIs An interrupted download leaves a `<file>.partial` behind. The partial handling in DownloadFileWithContext gated resume on `err == nil && uri.LooksLikeHTTPURL()`, so for any URI that is not literally http(s) the branch fell through to `else if !errors.Is(err, os.ErrNotExist)`, which with a nil err is true. The download then failed with an error wrapping nil: failed to check file ".../Ternary-Bonsai-27B-Q2_g64.gguf" existence: <nil> Every gallery file URI uses `huggingface://`, so a single interrupted download made that model permanently uninstallable until someone deleted the partial by hand. The `<nil>` in the message compounded it by pointing debugging at a filesystem failure that never happened. Restructure the handling as an explicit switch over the four real states: partial exists and is resumable, partial exists and is not resumable (discard and restart, as already done for an HTTP server without range support), no partial, and a genuine stat failure. The error branch is now only reachable with a non-nil error, names the path that was actually stat'd, and wraps with %w. Discarding is required for correctness and not merely convenience: the writer opens the partial with O_APPEND, so an un-resumed download would concatenate a fresh body onto stale bytes. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): tell the models gallery's variant rows apart, and let browsing see every build Both variant surfaces rendered name, backend and size. For two builds of one model that is close to no information: a variant exists precisely because the same weights are offered another way, so the backend usually matches and the sizes usually land within a few hundred megabytes. Comparing ternary-bonsai-27b-pq2 against ternary-bonsai-27b-q2-g64 meant reading two names that differ by a suffix nobody has defined anywhere in the UI. Report the quantization and the serving features on VariantView, and derive both server-side from the referenced entry rather than parsing names in the browser, so every client reads the same format out of the same file the installer will hand the backend. Quantization comes from overrides.parameters.model first, falling back to the file list. That order is load bearing: entries routinely ship a vision tower alongside the language model at a different quantization, so reading the file list first reports the mmproj's format. Matching walks `-` and `.` delimited segments right to left; `_` deliberately does not split, because it separates the parts INSIDE a quant token and splitting on it reports Q4 for a Q4_K_M build. A second, looser pass takes a segment's `_`-delimited tail, which catches the gemma-4-E2B_q4_0-it.gguf style; it runs second so a precise match can never lose to a fuzzy one further right in the name. An entry naming no format reports nothing, which is the honest answer for a backend served from a directory of weights. Features are the same tag-against-vocabulary match servingFeatureRank already ranks on, over the same host preference list. A build can therefore never be shown as faster than one selection did not actually reward, nor rewarded without being shown; a spec pins that agreement rather than trusting it. The compact dropdown gets the quantization on its meta line and the bare feature token. The detail row, which has the room, gets the quantization as its own monospaced column so precision lines up down the list, and the feature spelled out, because DFLASH names nothing to a user who has not met it. The referenced entry's description stays out of both: the detail row already renders the parent's prose above the table, and a second block per variant would push a three-variant list past a screen to restate what the columns now say precisely. The collapse toggle comes back. 462583f38 dropped it once search bypassed the collapse, on the reasoning that nothing was unreachable any more. That holds for finding a build whose name you know and does not hold for browsing: no sequence of actions enumerated the 68 builds the default view hides. Collapse is for browsing and search is for finding, and the toggle was the browsing half. It goes in the refinements band 0d4823362 established, not back among the taxonomy chips where its position depended on how many chips happened to wrap. It leads that band because it decides how many rows the other two refine over, and because unlike fits-in-GPU it is unconditional: a host with no GPU still browses. The search bypass is untouched and re-checked by a spec in the toggle's default state, since restoring the control must not restore the dead end it replaced. The empty-state hint returns but only without a search term, because a term bypasses the collapse and the hint would otherwise point at a control that cannot change the result. The stored preference reads 'on'/'off' only: an older build wrote '1'/'0' from an effect that ran on mount, so those record that the page was opened, not that anyone chose a view. Also fixes a latent flake it exposed. The collapse_variants spec compared whole response bodies byte for byte, and the listing envelope carries live host telemetry that drifts between two calls milliseconds apart, so it was asserting on the machine's memory pressure. It now compares everything the parameter governs -- the entries, their serialization and the paging -- and is green 25/25 where it was failing about one run in three. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): let the models gallery show a variant's full details The variant list in an entry's expanded detail row says how the builds differ: name, backend, quantization, size, and the auto-selected, base and serving-feature markers. It cannot say what any one of them is. A variant's own description, tags, license, source links and file list are unreachable anywhere in the UI, because while the collapse is on a variant has no gallery row of its own. Give each variant row an info control that reveals its entry, rendered by the same ModelDetail a top-level row gets, so a field added to the detail view appears here too. variantData is withheld from the nested render: a variant may declare variants of its own, and recursing would nest a picker inside a picker two levels deep already. An inline disclosure rather than a modal. The control sits inside a table row that is already expanded, inside a variant list within that; a dialog opened from there stacks a dismissal on a dismissal for a handful of extra fields about the entry the user is already reading, and breaks the page's own expand idiom. The third level is carried by an inset and a left rule instead of another card. The entry is fetched by exact name from the listing, once, on first use. The listing already returns every field the detail view renders, and a search term bypasses the variant collapse server-side, so no new endpoint is needed and neither the listing nor DescribeVariants gains any work. Expanding a row costs nothing; a variant nobody opens costs nothing. A name the listing no longer returns is stated, not blanked: an empty panel reads as a rendering fault rather than as a lookup that came back empty. The control is a sibling of the install button, not a descendant, so asking about a build can never install it. The variant list keeps its content-sized columns via a trailing filler track instead of max-content sizing, so the rows are unchanged while the panel spanning them gets the pane width its file table needs. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): let search respect the collapse instead of switching it off The models listing collapsed to one row per model, and an explicit search term turned that off wholesale. Searching while collapsed therefore answered with the individual builds a parent already offers, which are exactly the rows the view the user asked for has no place for: typing "mtp" returned qwen3.6-27b-nvfp4-mtp, a row that is invisible the moment the box is cleared. The bypass was the right shape of fix for the wrong half of the problem. What a search must not do is answer "no models found" for a build the gallery does hold; that does not require abandoning the grouping the user asked for. So the term is now matched against every entry either way, hidden builds included, and the collapse decides how a match is reported rather than which matches exist. Collapsing stops being a filter that drops rows and becomes a substitution: a match on a build another entry offers is reported as that entry, the one installable in its own right. Nothing becomes unfindable and nothing comes back that the requested view cannot show. Substitution happens after search, tag and backend, so every filter is judged against the build that really carries the name, tag or backend rather than against a parent that merely offers it; the other order would let backend=vllm match a parent whose own backend is something else. The price is that the surfaced row shows the parent's own metadata while the match was on a variant, which is what grouping means, and the alternative is claiming the gallery holds no such build. It happens before the count and the page math, so both describe the rows actually handed out rather than the matches that produced them. A parent already in the result keeps its own position and absorbs its matching variants there, which is what leaves the browsing listing ordered exactly as it was; a parent surfaced only by a variant takes the position of the first variant that surfaced it. Either way it appears once, however many of its builds matched and whether or not it matched itself. Search preserves gallery order rather than scoring, so a surfaced parent has a real position rather than an invented one. VariantParents never reports an entry that declares variants of its own, so a parent is never itself hidden and one hop always lands on a visible row. The handler follows exactly one anyway: refusing the second is what makes a gallery the linter would have rejected terminate rather than loop. The empty-state hint pointing at the toggle goes with it for every server-side filter. Substitution means a match is always reported as some row, so the collapse can no longer be why a term, a chip or a backend came back empty, and naming it there sends the user to a control that cannot change the result. It survives for the fits filter alone, which runs in the browser after the substitution and judges the surfaced entry's own size: there the build that fits really can be filtered out along with a parent that does not. Searching a build's exact name while collapsed now answers with its parent, so the result no longer contains the string the user typed. That is intended, and the row is the one they can act on, but it is a real rough edge: nothing on the row explains the connection. Closing it properly means reporting which variant matched so the UI can say so, which the listing does not do today. ResetGalleryModelCache is added for tests. The model cache is a package global keyed by nothing, so a background refresh one spec triggers can land in the middle of the next and answer it with the previous spec's gallery; the extra specs here made that fail about one run in five. It waits for the in-flight refresh to publish before clearing, since clearing alone only narrows the window. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-20 18:43:02 +02:00
mudler's LocalAI [bot]	9d82c37f98	fix(distributed): backend discovery hid worker-installed backends behind the controller's filesystem (#10967) ... fix(distributed): backend discovery hid worker-installed backends Backend discovery endpoints filter on installed-state, which on a distributed controller derives from the controller's own filesystem. A backend lives on the worker node that runs it, so every backend an admin installed on a GPU worker read as "not installed" and vanished from the listing. #10947 fixed the sibling capability filter on the same endpoints, so a fine-tuning-capable GPU worker now made the backend listable while the installed-state filter still dropped it: the dropdown stayed empty. The controller cannot derive this locally, but it already aggregates the per-node view that GET /backends renders, so discovery reuses the active BackendManager rather than growing a second path. Three surfaces shared the root cause and route through the same helper now: - GET /backends/available (Installed is now cluster-wide) - GET /api/fine-tuning/backends - GET /api/quantization/backends The response stays a boolean rather than an installed-on-N-of-M count: per-node install state is already served by GET /backends nodes[], and per-node control by POST /api/nodes/:id/backends/install, so a summary is all these dropdowns need. A nil provider (single-node) leaves the local filesystem as the only source and reproduces today's listing exactly, and a registry error degrades to that same listing instead of blanking the catalog. Assisted-by: Claude:claude-opus-4-8 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-20 01:09:37 +02:00
mudler's LocalAI [bot]	f735cb24c0	fix(worker): reap deleted backends and stop models that live on a worker (#10956) ... * fix(worker): reap deleted backends and stop models that live on a worker Three related backend-lifecycle defects, all reachable from the same production incident on a Jetson/Thor worker: a deleted backend's gRPC process survived ~40 minutes with its directory removed from disk, a later model load was routed to that orphan and failed with a certifi path pointing into the deleted directory, and the admin could not stop the model because the frontend reported it as not loaded. 1. backend.delete orphaned the process it claimed to delete ------------------------------------------------------------ s.processes is keyed by `modelID#replicaIndex` (buildProcessKey), so the backend name never appeared in a key and was recorded nowhere on the process. backend.delete resolved its target via isRunning/stopBackend, whose prefix path only matches a bare *modelID* - a delete keyed on a backend name resolved to zero keys, the stop silently no-op'd, and the files were removed out from under a live process. The install fast path then handed that orphan back out: it returns any live process for the (model, replica) slot without checking which backend started it, so a reinstalled variant inherited the deleted backend's port. - Record backendName on backendProcess, threaded installBackend -> startBackend. - Add resolveProcessKeysForBackend, matching the recorded name and resolving alias <-> concrete via ListSystemBackends *before* DeleteBackendFromSystem erases the metadata that carries the alias. Alias resolution failure degrades to name-only matching so a delete never fails on it. - backend.stop goes through resolveStopTargets, which accepts a backend name, a model name, or an exact modelID#replica key. Its payload field is named "backend" but is published with all three meanings: the admin UI sends a backend name, UnloadRemoteModel sends a model name, and the router's abandoned-load reap (#10948) sends an exact replica key. Narrowing it to backend names alone would strand the latter two. backend.delete stays strict - its identifier is unambiguously a backend. - Gate the install fast path on processMatchesBackend so a slot held by a different backend is restarted rather than reused. Processes with no recorded name (pre-upgrade) are accepted, so rollout does not restart every running backend. - stopBackendExact reports a real stop failure - the process still being alive afterwards, which is precisely what finishBackendStop already detects to keep the entry and its port reserved - and backend.delete no longer replies success when it knew about a process and could not kill it. "No process was running" stays a success but is logged, so the orphan case is visible rather than silent. 2. /backend/shutdown reported a running model as missing --------------------------------------------------------- ModelLoader.deleteProcess short-circuits on a miss in this replica's in-memory store. In distributed mode the authoritative record of "is this model loaded" is the shared node registry: a frontend replica that never served the model itself (load balancer picked a peer, or the replica restarted) has no local entry. The remote unload path that pkg/model documents ("when ShutdownModel is called for a model with no local process, UnloadRemoteModel is called") sat behind that short-circuit, unreachable in exactly the case it exists for. #10865 reworked this function but kept the short-circuit at the top, so the gap survived that refactor. - deleteProcess consults the remote unloader on a local-store miss, via a shared unloadRemote helper so this branch and the existing no-local-process branch both prefer #10865's RemoteModelContextUnloader, preserving force propagation across the distributed boundary. - UnloadRemoteModelContext reports ErrRemoteModelNotLoaded when no node has the model; it previously returned nil, making a no-op stop indistinguishable from a real one. The converse case (nodes have it, none could be stopped) already errors since #10865 joined the per-node failures, so that half of the original fix was dropped as redundant. - Only when the model is absent locally AND cluster-wide does the endpoint report not-found, now 404 naming both scopes rather than a bare 500. - modelNotFoundErr becomes the exported ErrModelNotFound so the HTTP layer can map it without string matching; watchdog's identity comparison becomes errors.Is. 3. Coverage for the bounded Free() that #10865 shipped untested ---------------------------------------------------------------- The original branch also bounded the pre-stop Free(), but #10865 landed that fix first (workerBackendFreeTimeout, applied in both stopBackendExact and handleModelUnload). That production change is therefore DROPPED here as superseded - master's version is strictly better, since it also releases the supervisor mutex across the call and keeps the port reserved until termination completes. What #10865 did not ship is a test, and the bound is load-bearing: the router-side reap in #10948 sends backend.stop for an abandoned load, and against a wedged backend an unbounded Free would swallow that stop before it reached the process. Nothing failed if the bound regressed. The spec stands up a real gRPC backend server whose Free handler never returns - what a Python backend looks like when its single worker thread (PYTHON_GRPC_MAX_WORKERS=1 on 37 backends) is occupied by a stuck LoadModel. A stub socket is not sufficient and was tried first: without a completed HTTP/2 handshake, gRPC's own ~20s connect timeout ends the call, so that version passed against the very bug it targets. With the connection READY, only the caller's deadline can end it, so the spec hangs to its 60s limit if the timeout is removed and passes with it. Its fixture process is deliberately never started. go-processmanager v0.1.1 writes Process.pid from readPID() without synchronization, so a live process races its own monitor goroutine under -race - reproducible with a bare Run()+Stop() and unrelated to this spec. Since scripts/model-lifecycle-conformance.sh runs this package with -race and is fail-closed, starting one would turn that gate red on an upstream defect. An unstarted process still proves the point: the stop is reached and the slot released, which is exactly what an unbounded Free prevents. Verified: make lint (new-from-merge-base origin/master) reports 0 issues; scripts/model-lifecycle-conformance.sh passes all three stages including the FizzBee liveness check (1458 states, IsLive: true). Assisted-by: Claude:claude-opus-4-8 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): keep remote unload idempotent, ask presence separately 2035a4d25 made UnloadRemoteModel return ErrRemoteModelNotLoaded when no node holds the model, so ShutdownModel could answer 404 instead of a misleading 500. That narrowed a shared adapter contract to serve one caller and broke the documented idempotent-unload guarantee, which CI caught on PR #10956: [FAIL] Node Backend Lifecycle (NATS-driven) > NATS backend.stop events should be no-op for models not on any node [Distributed] Expected success, but got: model not loaded on any node The spec name states the contract outright. The matching unit assertion was updated in that commit; this e2e one was missed because it lives under tests/e2e/ with no build tags and does not run in package-scoped test runs. Caller audit - who breaks when an idempotent unload becomes an error: - pkg/model/watchdog.go:902 (LRU memory reclaimer) is the serious one. It untracks a model ONLY when shutdown returns nil or ErrModelNotFound. A new error type means the model is never untracked, so the reclaimer keeps re-selecting the same entry and never reclaims - a live wedge whenever a local store entry outlives the remote model. - core/services/galleryop/managers_local.go:43 (DeleteModel) would warn on every deletion of an already-unloaded model. - core/services/modeladmin/{state,config,remote_sync}.go stop instances best-effort against models that are frequently not loaded. - deleteProcess itself: the no-local-process branch returns the unload result directly, so a stale local entry for a model no longer on any node turned a previously-successful cleanup into a failure. Only ShutdownModel wants the distinction, and only on the local-store-miss path. So the distinction moves to the caller instead of the contract: - UnloadRemoteModel/UnloadRemoteModelContext return nil again when no node has the model, and ErrRemoteModelNotLoaded is removed. - New optional RemoteModelPresenceChecker (HasRemoteModel) answers the question directly. deleteProcess consults it BEFORE unloading, because an idempotent unload cannot report afterwards whether anything was stopped. Absent locally AND cluster-wide is the only case that reports 404. - A failed registry lookup is surfaced rather than reported as absence: an unreachable registry is not evidence a model is gone, and answering a confident 404 off a failed lookup is how an operator gets told a running model does not exist. - Unloaders that predate the extension keep working - deleteProcess attempts the unload rather than refusing it - and compile-time assertions in the nodes package now pin all three optional interfaces, since both are consumed by runtime type assertion where drift degrades behavior silently instead of failing the build. The contract is now pinned at both levels that disagreed, each spec pointing at the other: "with no nodes returns nil" in unloader_test.go and "should be no-op for models not on any node" in node_lifecycle_test.go. Verified: full distributed e2e suite 233 passed / 0 failed (the suite that failed 232/1 in CI); pkg/model and core/services/nodes green; make lint new-from-merge-base reports 0 issues. Assisted-by: Claude:claude-opus-4-8 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): drop replica rows when a worker stops a backend A worker returns a stopped backend's gRPC port to its allocator as soon as the process is confirmed dead, and hands it to the next backend that starts. The controller's NodeModel row for the old address survives, and both SmartRouter.probeHealth and the HealthMonitor per-model probe verify liveness, not identity, so once an unrelated backend binds the recycled port the stale row passes every check and the request is served by the wrong backend instead of failing. backend.delete is newly able to trigger this: before #10956 a delete never actually stopped a process, so it never recycled a port. backend.upgrade has the identical gap and always did — upgradeBackend force-stops every process using the binary and starts none back up, while DistributedBackendManager.UpgradeBackend never removes rows. model.unload is the one path that gets this right today: it calls RemoveAllNodeModelReplicas straight after StopBackend. Report the process keys the worker terminated on the delete and upgrade replies, and drop the matching rows in RemoteUnloaderAdapter, which already holds a ModelLocator with RemoveNodeModel. All three call sites funnel through that adapter, so no new interface, DB migration, or proto change is needed. A key is reported only once its process is confirmed gone, so the list stays trustworthy on the partial-failure replies too. Old workers never populate the new fields. ReportsStoppedProcesses tells "stopped nothing" apart from "does not report", so an old worker's silence falls back to the pre-existing probe-based staleness recovery instead of being mistaken for a completed cleanup. Quarantine released ports for a short window as an interlock covering the NATS round-trip between the worker freeing the port and the controller dropping the row. It is deliberately not derived from HealthCheckInterval: that cadence is operator-tunable and the per-model reaper can be disabled outright, so coupling a worker-local constant to it would be silently wrong on some clusters. Eager row removal is the fix; the delay only closes the handoff gap. Identity verification in probeHealth was considered and rejected: Health and Status carry no backend identity, so it needs a proto change plus an implementation in 36 Python and 4 C++ Health servicers, it is fail-open for any backend not yet rebuilt, and the probeCache short-circuit means it would not even execute during the 30s window where the misroute happens. Fixes #10952 Refs #10954, #10956 Assisted-by: Claude:claude-opus-4-8 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(deps): bump go-processmanager, assert real backend termination go-processmanager wrote Process.PID from readPID() with no synchronization while its own monitor goroutine cleared the same field on exit, so a bare Run()+Stop() tripped the race detector without any concurrent access from the caller. LocalAI hit this on every backend stop. Upstream fixed it in a94e2b7 by guarding PID with a mutex and adding CurrentPID() as a race-safe accessor. The exported field was kept to avoid a breaking change but is now deprecated: a direct read still races the monitor. No tag carries the fix yet, so pin the pseudo-version. GetGRPCPID reads through CurrentPID() instead of the field. The accessor returns the same string under an RLock, so the empty-PID and strconv error paths are unchanged; it is the only direct field read in the tree. With the race gone, the Free-timeout spec no longer has to leave its fixture process unstarted. It now runs a real child and asserts the child genuinely exits, which is exactly what the earlier workaround gave up: the spec could show the stop was reached and the slot released, but not that SIGTERM ever landed. Termination is observed through Done(), which closes only once the library has waited on the child. The pidfile-based liveness helpers cannot serve here, because Stop() deletes the pidfile while releasing the handle and so reports "not alive" even if no signal was sent. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-20 01:09:22 +02:00
zjuzhongwen	864c84f48b	chore: fix some comments to improve readability (#10960) ... Signed-off-by: zjuzhongwen <zjuzhongwen@outlook.com>	2026-07-20 00:37:40 +02:00
mudler's LocalAI [bot]	b19afb192a	fix(distributed): backend discovery hid GPU-only backends behind the controller's capability (#10947) ... * fix(backends): list backends runnable on worker nodes in distributed mode GET /backends/available filtered the gallery against the system state of the host serving the request. In a distributed deployment that host is the controller, which typically has no GPU, while the GPUs live on worker nodes. Any meta backend whose capabilities map lacks a "default" (or "cpu") key was therefore dropped from the listing entirely — longcat-video, vllm-omni, ltx-video, parakeet, edgetam and qwentts were invisible in the UI even though installing them by name on a GPU worker worked fine. Workers now report their own meta-backend capability at registration and the controller persists it on the node row. The controller cannot derive it: OS-dependent capabilities (metal, darwin-x86, nvidia-l4t) and the CUDA runtime refinements are only observable on the worker. Nodes registered before this field existed fall back to a coarse capability derived from their GPU vendor and VRAM. Backend discovery then evaluates compatibility as the union over healthy backend nodes, so a backend runnable on any node is offered while one no node can run stays hidden. Each remote capability is evaluated through a capability-pinned system state, otherwise a forced capability on the controller image (LOCALAI_FORCE_META_BACKEND_CAPABILITY or /run/localai/capability) would silently override every worker's verdict. With no registered nodes the listing is byte-for-byte what it was, so single-node deployments are unaffected. Also fixes the same-root-cause misclassification in /api/operations, which used the capability-filtered listing to decide whether an operation was a backend or a model install. A GPU-only backend installing on a worker is still a backend operation on the controller, so that lookup is now unfiltered. Assisted-by: Claude:claude-opus-4-8 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(backends): union worker capabilities in backend discovery Implementation for the specs added in the previous commit, plus the two remaining discovery endpoints. Capability-filtered backend discovery evaluated compatibility against the system state of the host serving the request. In a distributed deployment that host is the controller, which typically has no GPU, while the GPUs live on worker nodes. Any meta backend whose capabilities map lacks a "default" (or "cpu") key was dropped entirely — longcat-video, vllm-omni, ltx-video, parakeet, edgetam and qwentts were invisible in the UI even though installing them by name on a GPU worker worked fine. Workers now report their own meta-backend capability at registration and the controller persists it on the node row. The controller cannot derive it: OS-dependent capabilities (metal, darwin-x86, nvidia-l4t) and the CUDA runtime refinements are only observable on the worker. Nodes registered before this field existed fall back to a coarse capability derived from their GPU vendor and VRAM. Discovery then evaluates compatibility as the union over healthy backend nodes, so a backend runnable on any node is offered while one no node can run stays hidden. Each remote capability is evaluated through a capability-pinned system state, otherwise a forced capability on the controller image (LOCALAI_FORCE_META_BACKEND_CAPABILITY or /run/localai/capability) would silently override every worker's verdict. With no registered nodes the listing is byte-for-byte what it was, so single-node deployments are unaffected. Four surfaces shared this root cause and are all routed through the same helper now: - GET /backends/available - GET /api/fine-tuning/backends - GET /api/quantization/backends - /api/operations backend-vs-model classification, which additionally had no reason to filter by capability at all: a GPU-only backend installing on a worker is still a backend operation on the controller, so that lookup is now unfiltered. Assisted-by: Claude:claude-opus-4-8 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-19 07:53:46 +00:00
localai-org-maint-bot	0389495388	fix(webui): use relative asset base so fonts and lazy chunks honor X-Forwarded-Prefix (#10889) (#10904) ... The Vite build emitted path-absolute asset URLs (base: '/'). index.html entry scripts and the favicon were rewritten to include the reverse-proxy prefix in serveIndex, but two reference kinds are not in index.html and so bypassed that rewrite: - CSS `url()` font references (e.g. Font Awesome .woff2), which the browser resolves relative to the stylesheet and which `<base href>` never affects - lazily-imported route chunks, whose preload base came from the absolute Vite base Under a subpath mount (X-Forwarded-Prefix: /llm/) both were fetched from the origin root, 404ing — missing-glyph "tofu" icons and broken lazy-loaded pages. Switch Vite to a relative base ('./') so every generated URL resolves against the file that references it: CSS fonts and route chunks now load from `/llm/assets/...`, and index.html's now-relative entry refs resolve via the `<base href>` serveIndex already injects on every response. Root deployments are unaffected. The existing path-absolute rewrite in app.go still covers the public `/favicon.svg`. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-18 08:38:26 +02:00
localai-org-maint-bot	4be6e22b5f	feat(webui): surface user, client IP and user agent in API traces (#10886, #10887) (#10907) ... The Operate → Traces "API Traces" panel already recorded who made each request (user_id/user_name) but never showed it, and did not capture the caller's network identity at all. Operators asked to see the requesting user (#10886) and the client IP + user agent (#10887) so a trace can be attributed to who/what issued it. Backend: add ClientIP and UserAgent to APIExchange and populate them from echo's c.RealIP() (honours X-Forwarded-For / X-Real-IP behind a trusted proxy) and the request's User-Agent header. Both are omitempty and the /api/traces swagger response is map[string]any, so this is additive. UI: add a sortable "User" column to the API traces table and a metadata block (User / Client IP / User Agent) at the top of the expanded row detail. Fields render only when present, so older buffered traces and unauthenticated/local requests degrade cleanly. Adds an e2e spec covering the new column value and the expanded metadata. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-17 23:47:31 +02:00
localai-org-maint-bot	bf484c5181	feat(webui): show date alongside time in the Traces view (#10888) (#10905) ... The Operate -> Traces table rendered the request time with the time of day only, so entries that span more than one day were ambiguous. Add a formatDateTime helper (localized date + existing time-with-millis) and use it for the Traces "Time" column, keeping the cell on a single line. The shared formatTimestamp used by the log views is unchanged. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-17 23:47:10 +02:00
walcz-de	6ccb1130d8	fix(agent-ui): reset streamed text at generation boundaries in agent chat (#10664) ... One agent turn runs several internal LLM generations (tool selection, reasoning, final answer) that all emit stream_event deltas over the same per-agent SSE channel. The chat page accumulated every 'content' delta into a single live bubble and ignored the 'done' boundary events, so the internal generations' text (e.g. the English tool-selection rationale) merged with — and visually corrupted — the streamed final answer. Reset the accumulated content/reasoning on 'done': each generation gets a clean live bubble, and the authoritative full answer still arrives via the final json_message event as before. Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>	2026-07-17 15:26:50 +02:00
LocalAI [bot]	c1a891662c	refactor(settings): single declarative registry for runtime settings (fixes the #10845 bug class) (#10864) ... * feat(settings): add declarative runtime-settings field registry One fieldSpec row per RuntimeSettings field, with a reflection completeness spec so a field added without a registry row is a red test instead of a silently-dropped setting (the #10845 bug class). Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-fable-5 * refactor(settings): drive ToRuntimeSettings/ApplyRuntimeSettings from the field registry Behavior-preserving: ~350 hand-written per-field lines become two loops over runtimeSettingsFields, gated by a To->Apply->To round-trip spec. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-fable-5 * feat(settings): baseline-driven startup merge for persisted runtime settings ApplyRuntimeSettingsAtStartup compares the live config against DefaultRuntimeBaseline (option-less-run defaults incl. kong-injected flag defaults) instead of per-field == 0 guards. Fixes persisted lru_eviction_max_retries, tracing_max_items, agent_job_retention_days, memory_reclaimer_threshold, galleries and autoload flags being silently ignored at boot. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-fable-5 * fix(settings): registry-driven startup merge, applied before consumers loadRuntimeSettingsFromFile becomes a thin wrapper over ApplyRuntimeSettingsAtStartup and runs at the top of New(), before model configs capture app-level defaults. WithThreads stops eagerly resolving 0 so a persisted thread count survives restart while LOCALAI_THREADS still wins (#10845); the physical-core fallback moves after the merge. Also: run.go now injects the memory-reclaimer threshold unconditionally so the option-less boot matches DefaultRuntimeBaseline and a UI-saved threshold survives restart. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-fable-5 * refactor(settings): file watcher delegates to the registry merge; shared API-key merge Manual edits to runtime_settings.json now behave like a boot-time load (env still wins) instead of the inverted diverged-from-startup guard that ignored most manual edits. MergeAPIKeys dedups env keys in one place for the endpoint and the watcher. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-fable-5 * docs(settings): document unified runtime-settings precedence Document the single env/CLI > runtime_settings.json > defaults rule, applied identically at boot, on POST /api/settings, and on manual file edits, plus the two known limitations (default-valued env vars are indistinguishable from unset; API-changed fields hot-apply on the next restart only). Also add a completion debug log when the watcher applies runtime_settings.json. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-fable-5 * test(settings): reset the global VRAM cap leaked by the round-trip spec The round-trip spec applies vram_budget=12GiB, whose post-loop hook installs a process-global default cap; without a reset every spec ordered after it runs under that phantom budget. Also drop a stale enumeration in the ApplyRuntimeSettings doc comment. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-fable-5 --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-16 22:39:59 +02:00
Tai An	dc2cc4da43	fix(audio-transform): serialize WebSocket writes to avoid concurrent-write panic (#10857) ... * fix(audio-transform): serialize WebSocket writes to avoid concurrent-write panic AudioTransformStreamEndpoint writes to the same Gorilla WebSocket connection from two goroutines: the backend-forwarding goroutine emits binary PCM frames (and can call sendWSError on a backend recv error), while the read loop calls sendWSError for malformed mid-stream JSON or a backend send failure. Gorilla WebSocket permits only one concurrent writer, so these writers race and can panic with "concurrent write to websocket connection", resetting the client session; a -race build reports the data race directly. Wrap the connection in a lockedConn that serializes WriteMessage behind a mutex, mirroring the existing lockedConn used by the openresponses WebSocket endpoint. Reads stay on the single read loop, so only writes need the lock. Fixes #10844 Signed-off-by: Tai An <antai12232931@outlook.com> * chore: empty commit to re-trigger checks Signed-off-by: Anai-Guo <antai12232931@anaiguo.com> --------- Signed-off-by: Tai An <antai12232931@outlook.com> Signed-off-by: Anai-Guo <antai12232931@anaiguo.com> Co-authored-by: Anai-Guo <antai12232931@anaiguo.com>	2026-07-16 16:25:31 +01:00
LocalAI [bot]	8cec22c3b7	feat(vram): per-node VRAM allocation budget (LOCALAI_VRAM_BUDGET) (#10833) ... * feat(vram): add vrambudget primitive for per-node VRAM caps Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): apply default VRAM budget in xsysinfo aggregate getters Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): wire LOCALAI_VRAM_BUDGET flag to xsysinfo default budget Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): persist VRAM budget via runtime settings with live apply Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(vram): reset process-global VRAM budget after runtime-settings spec Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): add VRAM budget field to Settings page Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): store and enforce per-node VRAM budget in the node registry Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): apply per-node VRAM budget in router hardware defaults Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): report worker VRAM budget in node registration The distributed worker now reports its operator-set VRAM budget string (LOCALAI_VRAM_BUDGET) to the server on registration. The worker keeps reporting RAW total/available VRAM and never sets the xsysinfo process-global budget (that stays standalone-only); the server resolves and enforces the budget uniformly (Task 6). Also closes a Task 6 gap: on re-registration, a struct Updates zero-skips an empty budget, so a worker that dropped LOCALAI_VRAM_BUDGET left the stale cap in place. For non-admin-override nodes the budget columns are now force-written (map Updates) even when empty, so removing the env var clears the cap; admin overrides are preserved unchanged. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * style(vram): drop em dash from worker-clear comment Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): add node VRAM budget admin endpoints Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): add node VRAM budget control to the node UI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): expose set_node_vram_budget MCP admin tool Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(vram): document LOCALAI_VRAM_BUDGET and node VRAM budget UI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(vram): avoid double-applying VRAM budget in GetResourceAggregateInfo The GPU-branch aggregate returned by GetResourceInfo is sourced from GetGPUAggregateInfo, which already caps total/free/used against the process-wide VRAM budget. GetResourceAggregateInfo then applied the budget a second time. For an absolute budget this is idempotent, but for a percentage budget b.Apply resolves the ceiling as a fraction of its input total, so a second pass yields P*(P*T) instead of P*T and distorts UsagePercent (read by the memory reclaimer in pkg/model/watchdog.go). Remove the redundant second application so the budget is applied exactly once, against the raw physical totals, upstream in GetGPUAggregateInfo. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(vram): implement SetNodeVRAMBudget on mcp assistant test stub The LocalAIClient interface gained SetNodeVRAMBudget; the stubClient in core/http/endpoints/mcp used by the assistant tests is a separate implementer and needs the method too (broke golangci-lint typecheck and both test jobs). Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-15 09:58:45 +02:00
LocalAI [bot]	3601174ce0	fix(distributed): make per-node backend upgrade actually upgrade (#10838) ... * test(core/http): make the suite's HTTP port overridable app_test.go and openresponses_test.go hardcoded 127.0.0.1:9090. When another service already listens on 9090 the suite does not fail fast: the server goroutine logs the bind error and the specs then poll whatever is squatting the port until Eventually times out. On machines where 9090 is permanently taken this makes the pre-commit coverage gate impossible to pass. Introduce testHTTPAddr, defaulting to 127.0.0.1:9090 (what CI has always used) and overridable via LOCALAI_TEST_HTTP_PORT for local runs. Assisted-by: Claude:claude-fable-5 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): make per-node backend upgrade actually upgrade The node detail page's Upgrade button reused the node-scoped install path (POST /api/nodes/:id/backends/install). That fires NATS backend.install with force=false, and the worker's install handler is deliberately "ensure installed": when the backend binary already exists on disk it short-circuits without touching the gallery. Since only an installed backend can be upgraded, the whole chain was a guaranteed successful no-op - the UI then toasted "backend upgraded" without even waiting for the async job. Route upgrades through the real force-reinstall path instead: - BackendManager.UpgradeBackend now receives the ManagementOp (like InstallBackend already did) so implementations can honor op.TargetNodeID. - DistributedBackendManager.UpgradeBackend scopes the backend.upgrade fan-out to op.TargetNodeID when set, and errors when the target node does not report the backend as installed. - New POST /api/nodes/:id/backends/upgrade endpoint enqueues an Upgrade=true node-scoped op (async 202 + jobID, mirroring install). - NodeDetail UI calls the new endpoint and reports the dispatch ("Upgrading ... on this node...") instead of claiming success; the Operations panel tracks the actual job. Verified against a live local cluster (NATS + Postgres + two workers): the target worker stops the running process, force-reinstalls from the gallery and re-downloads the OCI image; the second worker receives no backend.upgrade event; upgrading a backend missing from the target node fails the job with a clear error. Assisted-by: Claude:claude-fable-5 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-15 09:16:55 +02:00
LocalAI [bot]	bcc41219f7	feat: materialize Hugging Face model artifacts (#10825) ... * feat(config): add model artifact source contract Assisted-by: Codex:GPT-5 [Codex] * feat(downloader): add authenticated raw-byte progress Assisted-by: Codex:GPT-5 [Codex] * feat(huggingface): resolve immutable snapshot manifests Assisted-by: Codex:GPT-5 [Codex] * feat(models): add artifact storage primitives Assisted-by: Codex:GPT-5 [Codex] * feat(models): materialize pinned Hugging Face snapshots Assisted-by: Codex:GPT-5 [Codex] * feat(models): bind managed snapshots at runtime Assisted-by: Codex:GPT-5 [Codex] * feat(gallery): materialize model artifacts during install Assisted-by: Codex:GPT-5 [Codex] * feat(gallery): declare managed Hugging Face artifacts Assisted-by: Codex:GPT-5 [Codex] * feat(models): preload managed model artifacts Assisted-by: Codex:GPT-5 [Codex] * fix(gallery): retain shared artifact caches on delete Assisted-by: Codex:GPT-5 [Codex] * feat(models): report artifact acquisition progress Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): load managed models from ModelFile Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): load staged speech model snapshots Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): use staged snapshots in engine backends Assisted-by: Codex:GPT-5 [Codex] * test(distributed): cover staged artifact snapshots Assisted-by: Codex:GPT-5 [Codex] * docs: explain managed model artifacts Assisted-by: Codex:GPT-5 [Codex] * docs: add product design context Assisted-by: Codex:GPT-5 [Codex] * feat(ui): show model artifact download progress Assisted-by: Codex:GPT-5 [Codex] * Eagerly materialize Hugging Face artifacts Materialize HF-backed model references as managed GGUF artifacts during load, with lazy download retained only as fallback. Assisted-by: Codex:GPT-5 [shell] * Refactor HF downloads through a shared executor Assisted-by: Codex:GPT-5 [shell] * drop Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-15 01:09:33 +02:00
Ijas	a5aa56db81	fix: preserve uploaded file content when regenerating a non-last answer (#10819) ... handleRegenerate rebuilt the outbound message from the display-only message.files metadata ({name, type: 'file'|'image'|..., content}), which doesn't carry the base64/textContent payload sendMessage's file-building loop expects. As a result, regenerating any answer whose own question had an attachment silently dropped that attachment from the resent message. This wasn't fork-specific, but forking a chat and then regenerating an earlier (now non-last) answer is the natural way to hit it. Fix by reusing the original message's already-assembled `content` verbatim (it already has the file text / image_url / audio_url / video_url parts embedded from the first send) instead of trying to reconstruct it from lossy display metadata. Fixes #10806 Assisted-by: Claude:claude-sonnet-5 Signed-off-by: ajuijas <189517297+ajuijas@users.noreply.github.com> Co-authored-by: ajuijas <189517297+ajuijas@users.noreply.github.com>	2026-07-14 11:24:42 +02:00
Dedy F. Setyawan	b10e330590	feat(react-ui): localize MediaHistory, AudioTransform, and Sound components (#10802) ... - Replace hardcoded text with useTranslation hook in UI components - Add localization support for both English (en) and Indonesian (id) locales Signed-off-by: Dedy F. Setyawan <dedyfajars@gmail.com>	2026-07-13 11:51:23 +00:00
LocalAI [bot]	4056283aa4	[voice] feat: add managed voice cloning profiles (#10799) ... * feat(ui): add voice library workflow Give administrators a production-ready flow to record or upload consented reference audio, manage reusable profiles, inspect API usage, discover compatible models, and hand a saved voice directly to text-to-speech. Assisted-by: Codex:gpt-5 * feat(voice): add managed voice cloning profiles Make reusable reference voices manageable through the admin API instead of requiring model-directory and YAML edits. Discover compatible installed and gallery models from server-side backend capabilities, retain explicit model configuration controls, and stage saved references for supported backends. Expose profile management through REST and MCP, document backend-specific behavior, and cover the workflow from profile creation through real Qwen3-TTS synthesis. Harden the agent-job HTTP test against completion racing cancellation. Assisted-by: Codex:gpt-5 --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-13 09:54:46 +02:00
LocalAI [bot]	b00422e45f	feat(backends): add LongCat video and avatar generation (#10792) ... * feat(backends): add LongCat video and avatar generation Assisted-by: Codex:GPT-5 [apply_patch] [exec_command] [web] * refactor(config): declare model I/O modalities Make model configs declare input and output modalities so capability discovery no longer branches on backend or checkpoint names. Complete the LongCat gallery and user documentation, make the SDPA patch apply to the pinned upstream revision, and stabilize the Agent Jobs race exposed by the required hook. Assisted-by: Codex:GPT-5 [web] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-12 23:58:46 +02:00
Dedy F. Setyawan	294487eb61	fix(ui): prevent table container from breaking flexbox layout (#10754) ... Signed-off-by: Dedy F. Setyawan <dedyfajars@gmail.com>	2026-07-09 23:21:07 +02:00
LocalAI [bot]	40dae953f4	feat: interleaved thinking with tool calls (reasoning_content alias + Anthropic thinking blocks) (#10744) ... * feat(schema): accept reasoning_content as inbound alias for reasoning Interleaved-thinking clients (cogito, vLLM/DeepSeek-style) emit reasoning_content on assistant turns. Accept it as an inbound alias so reasoning survives the tool-result loop; canonical reasoning wins when both are present. Emission is unchanged (still reasoning). Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(schema): pin interleaved reasoning+tool_calls round-trip Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(openai): pin reachedTokenBudget truncation detection Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(anthropic): add thinking and signature fields to content blocks Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(anthropic): parse inbound thinking blocks into reasoning Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(anthropic): emit thinking blocks with synthetic signature on tool turns Extract buildAnthropicContentBlocks so non-streaming content assembly is unit-testable, and prepend a thinking block (with an opaque synthetic signature) before text/tool_use blocks when the request opts into thinking. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(anthropic): stream thinking_delta and signature_delta before tool_use Extract anthropicStreamSequence so the streaming block order is unit-testable, and emit content_block_start(thinking) -> thinking_delta -> signature_delta -> content_block_stop before the tool_use block sequence when thinking is enabled. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add interleaved thinking with tool calls guide Add a features guide describing interleaved thinking: an assistant turn carrying reasoning and tool_calls together, the reasoning-round-trip contract (including the reasoning_content inbound alias and Anthropic thinking blocks with a synthetic signature), per-backend enablement (reasoning_format for llama.cpp, reasoning_parser/tool_call_parser for vLLM/SGLang plus the vLLM auto-config hook), a worked request/response example, and known limitations. Cross-link from model-configuration, text-generation, and openai-functions. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-08 16:45:43 +00:00
LocalAI [bot]	a0ed395cf9	fix(auth): accept EC/PS/EdDSA-signed OIDC ID tokens, not just RS256 (#10736) ... The OIDC verifier was built with a bare oidc.Config{ClientID: ...}, so go-oidc applied its default of accepting RS256-signed ID tokens only. An identity provider configured with an EC signing key (e.g. Authentik) issues ES256-signed tokens, and the callback failed verification with: failed to verify ID token: oidc: malformed jwt: unexpected signature algorithm "HS256"; expected ["RS256"] surfacing to the user as HTTP 500 "failed to fetch user info" (#10677; the underlying cause became visible after the logging fix in #10679). Set SupportedSigningAlgs to the standard asymmetric algorithms (RS256/384/512, ES256/384/512, PS256/384/512, EdDSA). All are verified against the provider's published JWKS. HS256 is intentionally excluded: it is symmetric and would validate against the client secret, a different and security-sensitive trust model. Tested with a functional spec that signs an ES256 ID token and confirms it verifies with the configured algorithms and is rejected under go-oidc's RS256-only default (using oidc.StaticKeySet, no network). Closes #10677 Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-08 15:24:29 +02:00
LocalAI [bot]	cd65a1f645	fix(transcription): honor model-config language/translate + OpenAI language form field (#10731) ... fix(transcription): honor model-config language/translate, add language form field The /v1/audio/transcriptions endpoint read only input.Language / input.Translate from the parsed request, and the request middleware never populates those from a multipart upload -- nor did it read a `language` form field. As a result the model config's parameters.language / parameters.translate (a valid PredictionOptions field under `parameters:`) were silently ignored, and multilingual models like canary defaulted to translating into English even when the YAML set language: ru, translate: false (#10655). Resolve both with clear precedence: the request form field wins, then any language on the parsed request, then the model config default. This also makes the endpoint honor OpenAI's `language` form parameter, which was not read before. Applies to both the streaming and non-streaming paths (the resolved values are built into the shared TranscriptionRequest). Note this ensures the language/translate flags reach the backend; whether a given engine acts on them is up to the backend. Closes #10655 Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-07 21:16:25 +00:00
LocalAI [bot]	97175f4b5a	feat(model): debounce model loads after a failure to stop retry-storms (#10728) ... A client that keeps polling a model whose load fails (e.g. a backend that crashes deterministically on init) triggered a fresh backend start on every request: request -> load -> crash in ~10s -> 500, repeat on the next poll. Each attempt could leak GPU/CUDA state, and under LOCALAI_SINGLE_ACTIVE_BACKEND it kept stealing the active slot from healthy models. The existing loading-coalesce map only dedups *concurrent* loads, so sequential polls were never covered. Track load failures per modelID in ModelLoader. After a load fails, refuse fresh load triggers for that model until a cooldown elapses, returning a typed ModelLoadCooldownError that the HTTP layer maps to 503 with a Retry-After header. The cooldown grows exponentially per consecutive failure (base, doubling, capped at 5m) and resets on a successful load. The coalesced follower-retry of an in-flight burst bypasses the gate, so a genuinely concurrent burst still gets its one retry -- only new, independent triggers are refused, matching the report's "refuse new load-triggers" wording. Configurable via --model-load-failure-cooldown / LOCALAI_MODEL_LOAD_FAILURE_COOLDOWN (default 10s, 0 disables), plumbed through ApplicationConfig and applied unconditionally at startup. Closes #10719 Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-07 20:55:24 +00:00
LocalAI [bot]	bd3f2588fd	fix(ui): center the home empty-state wizard (#10691) ... The no-models getting-started wizard (`.home-wizard`) rendered left-aligned instead of centered. `.home-page` is a column flexbox with the default `align-items: stretch`; a child with `max-width: 48rem` cannot be stretched past its max-width, so it falls back to the cross-start (left) edge. The populated home branch never exposed this because its children are full-width. Add `margin: 0 auto` to `.home-wizard` so the max-width block centers horizontally, for both the admin getting-started wizard and the non-admin no-models hero. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-05 13:12:09 +02:00
LocalAI [bot]	b0959d4756	feat(api): add GET /v1/models/capabilities endpoint (#10687) ... Additive superset of /v1/models that enriches each model entry with the capabilities it supports plus its input/output modalities (text / image / audio / video). Clients that only understand /v1/models are unaffected -- they simply never call the new route. Audio and video *input* are derived from the model's multimodal limits (vLLM limit_mm_per_prompt), which no single usecase FLAG expresses. That gap is exactly why a plain capability list is insufficient and this enriched endpoint exists: an attachment router can now decide whether an image/audio/video file can go to the active model directly, or must be converted/transcribed first. Capability derivation lives in core/config as the single source of truth (ModelConfig.Capabilities / InputModalities / OutputModalities / VisionSupported / ...); the Ollama capability surface now delegates to it instead of keeping a parallel copy. Vision is gated on chat/completion capability so a MediaMarker hydrated onto a non-chat model (e.g. a pure ASR/TTS backend) no longer reports a false vision capability. Read-only listing: no new FLAG_* flag, reuses the existing `models` swagger tag, and intentionally exposes no MCP admin tool (there is nothing to manage conversationally). Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-05 08:51:55 +02:00
LocalAI [bot]	9e41be4bfb	fix(auth): log the real cause of OIDC/OAuth user-info failures (#10679) ... The OAuth callback discarded the error returned by user-info resolution before sending the generic 500, so real failures were completely opaque in the logs: ID-token verification errors (e.g. issuer/audience mismatch behind a reverse proxy), a missing id_token, claim-parse errors, or a rejecting GitHub userinfo endpoint all collapsed into "failed to fetch user info" with nothing logged. Log the wrapped cause with xlog.Error (provider + error), matching the code-exchange step just above it. The client-facing message is unchanged, so no internal detail leaks to the browser. Refs #10677 Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-04 19:33:53 +02:00
Richard Palethorpe	eb32cd9073	feat(realtime): eager blocking pipeline warm-up + /backend/load API (#10662) ... Realtime sessions previously lazy-loaded each pipeline sub-model (VAD, transcription, LLM, TTS) on first use, so every cold session paid a per-request model-load stall and load errors only surfaced mid-stream. Warm the whole pipeline eagerly and blockingly at session start (including the voice-gate speaker-recognition model, which an enforced gate blocks each utterance on; compaction's summary_model stays lazy since it only runs off the response path): - Add backend.PreloadModel / PreloadModelByName as the single load path for every modality (no transcription special-case; backend-omitted configs are deprecated). - The realtime session blocks on Model.Warmup and returns a model_load_error to the client if any stage fails to load; updateSession warms in the background. Opt out per pipeline with pipeline.disable_warmup, exposed as a UI toggle via the config-metadata registry. Add a LocalAI-native POST /backend/load (and /v1/backend/load) that pre-loads a model -- expanding realtime pipelines into their sub-models -- as the inverse of /backend/shutdown. There is one preload engine (backend.PreloadStages): the realtime Warmup methods, /backend/load and the --load-to-memory startup flag all use it, so --load-to-memory now also expands pipeline models and records load-failure traces. Pipeline sub-model alias resolution is likewise shared (ModelConfigLoader.LoadResolvedModelConfig). Surface the endpoint everywhere an admin manages models: - MCP admin tool load_model (httpapi + inproc clients, safety/catalog prompts, catalog/dispatch tests). - "Load into memory" action in the React models UI. - Swagger regenerated; docs moved to the general backend-monitor page since it is not realtime-specific. Fix a Traces UI crash ("json: unsupported value: -Inf"): audio-snippet RMS/peak now floor at a finite dBFS, and backend-trace data is sanitized to drop non-finite floats before marshaling. The sanitizer is copy-on-write -- it runs on every RecordBackendTrace, so containers are only re-allocated on the paths that actually changed. Migrate core/http/openresponses_test.go onto the prebuilt mock-backend the rest of the http suite already uses -- it was the last spec still pointing at a real HuggingFace model, so it 404'd wherever no vision backend was built -- and fix its item_reference specs to send the spec's "id" field instead of "item_id", which the handler never accepted. Assisted-by: Claude:claude-opus-4-8 Claude Code Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-07-03 18:00:37 +02:00