mirror of
https://github.com/mudler/LocalAI.git
synced 2026-08-01 02:49:51 -04:00
feat/vllm-cpp-engine-args
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
mudler's LocalAI [bot]
|
0f7186f214 |
feat(ui): replace the stacked operations bar with a one-line strip and an Activity page (#11163)
* feat(ui): record finished gallery operations in a bounded history ring The operations panel drops an operation the moment it succeeds, so a user who steps away cannot tell whether an install finished, failed or was never started. OpCache now keeps the last 50 terminal operations, recorded from the point where an op leaves the cache. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): pin the history ring's dedupe, outcome order and start stamp Review of the history ring found four gaps. The dedupe guard and the bounded seen set were unreachable through the exported API and so had no coverage; an in-package spec file now drives opHistory directly. The outcome switch claimed an ordering was load bearing that nothing pinned, so an errored op that never reached Processed now has a spec. Two behaviour fixes come with it. StartedAt was the zero time for ops recovered from the store or replicated from a peer, since neither path stamps a start time, which would have rendered as a two-millennia duration; it now falls back to the finish time. Reusing a cache key with a fresh job ID orphaned the previous stamp, so Set and SetBackend now drop it. The comment on the outcome switch described a state the code cannot be in: CancelOperation sets Cancelled and Processed synchronously before the handler removes the entry, so status.Cancelled already covers the cancel endpoint. The !Processed clause stays for the dismiss endpoint firing on an in-flight op, and the comments now say so. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): record operations that end on a peer replica The NATS end event is the only signal a replica gets for an install another replica ran. Record from applyEnd too, deduped by job ID so the originating replica does not record its own broadcast twice. Three start-stamp defects in the same path go with it. applyEnd now drops the stamp unconditionally, since recordTerminal only cleans up on the path where it found a cache key and an end event can overtake the local Set. applyStart drops the stamp of the job whose cache key it replaces, which a peer-driven retry previously stranded. And recordTerminal reads the stamp once instead of testing Exists and then reading, so a concurrent record for the same job can no longer delete the stamp between the two and let the zero time overwrite the finish-time fallback, which the Activity page would render as a two-millennia run. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): do not guess the outcome of a peer operation with no local status A replica that restarts mid-operation hydrates its OpCache keys from PostgreSQL, but gallery statuses are in-memory only and come back empty. The end broadcast then landed on recordTerminal's nil-status branch, which reads a missing status as queued-and-removed and filed a successful install as cancelled. That reading is right locally and wrong on the peer path, where a missing status means the outcome was never held here. recordTerminal now takes the source of the terminal event and records nothing when the peer path finds no status, restoring what the replica did before the end event started recording. The local path is unchanged. Also move the ApplyEndForTest seam to the conventional export_test.go, and stop the dedupe spec from claiming to guard the ring's seen set: the local delete removes the status keys, so the broadcast that follows returns before reaching it. An in-package spec that calls recordTerminal twice does the pinning. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(api): add GET and DELETE /api/operations/history Admin gated like the rest of the operations API. The live /api/operations payload is unchanged so the one second poll stays small. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): expose operation history through OperationsContext Fetched on demand and when the live list shrinks, never on the one second poll interval. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): detect operation departure by identity and ignore committing ops in the ETA gate Refetching history on a shrinking live count missed a completion that coincided with a start, which is the common case during a batch install. Track the live job IDs instead, so any departure triggers the refetch regardless of how the count moved. An operation that has finished downloading stays live at currentBytes == totalBytes for the whole commit and install phase and can never produce an estimate, so counting it in the all-or-nothing gate blanked every other operation's time remaining for as long as it lasted. Only operations still moving bytes get a vote. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): let only downloading operations gate the time remaining estimate Verifying pins an operation flat below its total for the whole sha256 pass: the AfterDownload hook reports completedBytes plus the finished file against a total summed over every file, then hashes synchronously without emitting progress. Files download sequentially, so a 15 shard model enters that window 14 times, and a byte comparison cannot see it because the counter is genuinely below the total throughout. Gating on phase closes resolving, verifying, committing and persisting in one predicate, so a quiet neighbour no longer blanks every other operation's estimate for minutes at a time. The byte clauses stay: a producer can report downloading with bytes already at the total. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): collapse the operations bar to a single line Four concurrent installs used to take four rows above every page. The strip now shows one operation, failure first, with a counter linking to Activity. The close button hides the strip and no longer cancels an install. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): keep the operations strip from widening the page and from muting a failure A long install error made the strip report a 1600px minimum width, which sized main-content to fit and gave every page under it a horizontal scrollbar. Inline-size containment plus shrinkable detail and bytes cells keep it inside the viewport. Hiding is no longer able to swallow the hidden job's own failure, a completed removal or staging says so instead of claiming an install, a cancelling operation renders as cancelling, and the live region no longer covers the per-second percentage. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): shrink main-content instead of containing the strip, and expose progress min-width on .main-content is what actually lets a long install error shrink, and unlike inline-size containment it has no browser support floor and no latent collapse if the strip ever lands in a shrink-to-fit context. It matches what .app-layout-chat .main-content already does, and it clears pre-existing horizontal overflow on narrow viewports as a side effect. The progress track is now a labelled progressbar, so assistive tech can read the value on demand rather than losing it to the aria-hidden that stopped the live region re-announcing every poll. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add the live operation card for the Activity page Carries the detail the one-line strip has to drop: phase, bytes, the per-node breakdown for cluster installs, and a labelled Cancel button. Cancelling is destructive, so it gets a labelled button rather than a glyph. A cancelling operation drops its progress bar and its time estimate, the same call the strip makes: a percentage still climbing under "Cancelling" reads as the cancel not having taken. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): give the operation card a verb, live node disclosure and its per-node detail The card carried no verb, so an install, a removal and a staging op rendered as spinner plus name plus kind tag and were indistinguishable. It now runs the same verb and icon chain as the one-line strip, which is what stops the page that is meant to carry more detail from carrying less. The auto-expand default was evaluated once at mount. An operation is listed as soon as it is admitted but its nodes are filled in only when the fan-out starts reporting, so a card mounted at creation latched on the empty list and stayed collapsed. The default is a live expression now, and state holds only an explicit choice. Also: an optional onRetry gates a Retry button, so the page can own the install reconstruction without the card ever showing a control with nothing behind it; the disclosure moved above the region it controls and gained aria-controls; the toggle is gated at more than one node so the count is never "1 nodes"; an unmapped node status is passed through instead of being relabelled "Queued"; error text is clamped with the full string in the title; and file_name plus the per-node progress bar are rendered again, reviving three CSS rules that had gone dead along with the detail they styled. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add the Activity page Live operations, unacknowledged failures and the record of what finished, at /app/activity in the Operate console. Cancelling an install now lives here behind a labelled button rather than on the strip, and a failed install can be retried: the retry dismisses the failure first so it still reaches the record, then reissues the model, backend or node-scoped backend install. The sidebar Operate entry carries the operation count. The console rail is only rendered on an Operate route and can be collapsed, so a badge there could vanish while operations were still running. Two follow-ups from review fold in here: a failed removal or staging job no longer reports a failed install on either the card or the strip, and the card's error text can shrink so one unbroken token cannot widen the card. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): dismiss operations by job, and stop the Activity page contradicting itself Dismissing resolved the job by display id, but /api/operations strips the "node:<nodeID>:" prefix before emitting, so a local install and a node-scoped install of one backend arrive as two jobs sharing one id. Dismissing by id retired whichever came first. That defeated the guarantee retry was built around: with the wrong job dismissed, the reinstall overwrote the acted-on failure's opcache entry in place, bypassing recordTerminal, while an unrelated failure vanished from Needs attention. dismissFailedOp, the card's dismiss control and the strip now all pass the jobID, which is what the endpoint takes. A filter matching nothing rendered the "nothing has ever run" empty state while the header counted the records the filter had hidden. The empty state is now gated on the All chip and a narrowed view gets its own message plus a way back; the header counts the instance rather than the chip, so selecting Backends no longer reports "Nothing running" over running model installs. Also: the summary drops a zero clause instead of rendering "0 needs attention" on the happy path and pluralises both counts; a record duration is floored at "< 1s" and rejected above a day, so a zero-value start stamp cannot render a span of millennia and a zero span cannot render "installed in" with nothing after it; a deletion cancelled mid-flight reports the cancellation rather than claiming it was removed; and the retry variant comment names the fix instead of calling the gap closed. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: document the Activity page and the operations history endpoints Adds an Activity page under Operations covering the one-line operations strip, the /app/activity sections and filters, per-operation cancel, retry and dismiss, the in-memory 50-entry record, and the sidebar count. Documents GET and DELETE /api/operations/history, and fills the gap in the admin-only endpoint list, which also omitted the pre-existing POST /api/operations/:jobID/dismiss. Corrects the distributed-mode install-watching section: the per-node breakdown now lives on the Activity page rather than on the strip, which rolls a fan-out up into a single phrase. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: correct nine details in the Activity page documentation The operations strip never renders a file name: its detail line is the error, the node roll-up, the target node, the phase or the queued note. Drops the stale clause in the distributed-mode section, where the per-node bullet is now the only place a file name is described. Scopes the phase vocabulary to artifact-backed gallery models, since a plain GGUF install emits no phase. Corrects the per-node list: the toggle exists for any fan-out of two or more workers and the four-node threshold only governs whether it starts open, while the N nodes tag needs more than one node. Notes that a cancelled operation can sit in the live section reading Cancelling, that cluster staging never reaches the record, and that Clear history appears only when the record has something in it. Names the operations response envelope, with a JSON example, so callers do not index a bare array, and stops describing the icon-only dismiss control as a labelled button. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: drop the unreachable Cancelling state and scope the byte claims An operation can only report isCancelled while it is unprocessed, but every writer of Cancelled sets Processed in the same breath, on the peer path as much as the local one, and the cache evicts cancelled entries before the handler sees them. The state cannot reach the page, so the live section is described again as running or queued operations. Byte counts come from the artifact bridge alone, the same producer as the phase, so a plain GGUF install, a removal and a backend install report none. Scopes both to artifact-backed gallery models and leaves the verb, the name and the percentage as what every operation shows. A worker backend install reports its bytes through fields the operations payload does not carry, so the distributed section now describes the percentage and the node roll-up, with per-file counts pointed at the per-node detail. Also: staging jobs carry no error, so they never reach Needs attention and Retry never had a staging case to exclude; an install that involves workers is no longer called node-scoped, which this page uses for node-targeted installs; and the record timestamps carry nanoseconds. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: state only the verb and the name as unconditional on the strip The percentage is as conditional as the bytes were: it renders only for a running operation that has reported progress, so a queued operation, a failed one and a removal never carry it. A removal in particular sits at progress zero for its whole visible life, since the delete path reports none and its completion is filtered out. Both the strip and the card paragraphs now lead with what always shows and list the rest as conditions. The Cluster chip matches on a node list that finished operations do not carry, so a fan-out install leaves the chip once it reaches the record. Scoped that claim to the live sections. Two more of the same shape, found by re-reading each clause alone: the strip also appears for a failure, which is not running, and the four-second hold only applies when nothing replaces the operation that just finished. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): stop reporting a cancelled install as installed, and make queued real Three defects that all trace to one root cause: `isCancelled: true` is unreachable from /api/operations. Every writer of Cancelled=true also sets Processed=true, the handler skips Processed && Cancelled, and OpCache.GetStatus evicts a cancelled op before the handler iterates it. Cancelling the last running operation put a green "Installed model X" on the strip for four seconds: the completion hold was guarded by `!previous.isCancelled`, which is dead. A cancellation deletes the operation server side, so the strip sees exactly what it sees on a completion, and nothing in the payload separates the two. The signal now comes from the side that issued the cancel: the operations context remembers the job IDs it cancelled (pruned after a minute) and the strip asks before it holds anything. A cancelled operation goes as soon as it stops; the record already reports it as cancelled. isQueued was set only when the gallery status was missing, but markQueued publishes a "queued" status at admission, so a queued op has a status for its whole queued life and the state was unreachable outside a microsecond window. Every operation waiting behind a running install rendered as "Installing model X" with a spinner. The queued phase is now the signal, via an exported PhaseQueued and a nil-safe OpStatus.IsQueued() next to the writer. With those two fixed, the Cancelling state has no way to be entered: cancelling is instantaneous from the API's point of view. Its branches, CSS, locale key and the isCancelled field itself are removed rather than left for a future reader to assume they work. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): keep a removal a removal, and say what an install is doing OpStatus.Deletion was set once, at admission, and lost on the next status write: UpdateStatus replaces the whole status and only carried Nodes forward. Every later writer (the worker's first write, the progress ticks, the failure path) leaves the field at its zero value, so the flag survived only the queued window, and both surfaces test isQueued first. The reachable consequence is that a failed removal reported itself as a failed install, which is exactly the shape the Activity page offers Retry for, and Retry installs: pressing it on a removal that failed re-downloaded the model. A running delete also rendered as "Installing model X" with a spinner, and a successful one as "Installed model X". Carry Deletion forward the way Nodes already is. A job is a delete or an install for its whole life; an unset flag means "no new information", not "this is an install". Pinned by Go specs on both the service and /api/operations: the existing Playwright specs were green only because they stubbed a payload the server could not emit. Also restore the operation's own status message on the Activity card. Phases and byte counters exist only on the managed-artifact path, so a legacy files: gallery model and every backend install rendered a sub-row with nothing in it but the verb. The strip stays terse on purpose. And give the strip's name a min-width floor: overflow: hidden zeroes its automatic minimum, so a long error squeezed the name down to "mod…" and the identity of the thing that broke was the first thing lost. primaryOperation is made module-private: its comment claimed the Activity page selected the same operation, but that page shows all of them, partitioned into failed and running, and never imported it. Assisted-by: Claude Code:Opus 5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(activity): read the operations record from PostgreSQL The Activity page's record of finished installs and removals was a 50-entry in-memory ring per frontend replica. In distributed mode that is the wrong place for it: each replica keeps its own copy, a replica added by a scale-out or a rolling deploy starts empty and never backfills, and "Clear history" clears only the replica that served the request, so the record reappears on the next poll routed elsewhere. The data is already in gallery_operations. Read it from there. GalleryStore gains ListTerminal and ClearTerminal, sharing a lifted terminalStatuses set with CleanOld so there is one definition of "finished". ListTerminal orders by updated_at, when the operation reached its terminal status, because the record reports what finished and when. OpCache.History and ClearHistory dispatch on whether a store is wired, so the HTTP handlers and the OpRecord JSON shape are unchanged and the page needed no change. A failed store read falls back to the local ring rather than blanking the page, and ClearHistory empties the ring as well so a database blip cannot resurrect a record the admin just cleared. The name derivation in recordTerminal is lifted into operationDisplayName and used by both paths, so the ring and the store cannot name the same operation differently. Also fixes a pre-existing bug the store path made visible: the backend channel hardcoded op_type "backend_install" even for a removal, while the model channel derives model_install/model_delete from op.Delete. Both channels carry the same ManagementOp, whose Delete field the backend handler already branches on, so the backend channel now derives backend_delete the same way. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): keep a cancelled operation cancelled, and report a failed clear Review follow-up on the store-backed Activity record. A cancelled install was recorded as a failure. The cancel handler persists "cancelled" synchronously, then the handler goroutine unwinds with the context error and Start hands that to updateError unconditionally, which overwrote the row with "failed: context canceled". The page rendered a cancelled install as a red failure card offering Retry, with a raw context error as the reason. Fixed in GalleryStore rather than in Start, because an operation finishes once and the paths that retire one are not mutually exclusive: UpdateStatus now refuses to rewrite a row that already reached a terminal status. That also pins updated_at to when the operation really finished, which is the key the record is ordered by, and Create's upsert now freezes the same columns so a worker dequeuing an operation the admin cancelled while it was queued cannot reopen it as pending. ClearHistory returned nothing, so a failed delete logged a warning while the handler still answered 200. The admin watched the record clear and come back on the next fetch with nothing said about why. It now returns the error, the DELETE handler answers 500, and the store is cleared before the local ring so a failure leaves the fallback record intact rather than faking an empty one. Hydrate is the only reader that decides from op_type whether an operation is a removal, and it tested for "model_delete" exactly, so the backend_delete added in the previous commit hydrated as an install: a replica restarting during a backend removal rendered "Installing backend X". Both discriminations now go through IsDeleteOpType/IsBackendOpType so a fifth op_type cannot silently read as an install in whichever consumer was missed. Also: the backend channel now persists Cancellable as !op.Delete, matching the model channel; IsBackend falls back to the op_type prefix, since is_backend_op is only written by UpsertCacheKey and the rows needing the name fallback were reporting backend operations as models; and an unrecognized terminal status is logged rather than quietly filed as a success, which is what the comment already claimed. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): keep a reaped operation correctable by its real outcome The terminal-status freeze added in the previous commit was too wide. It froze "failed" alongside "completed" and "cancelled", and the stale reaper writes "failed" onto operations that are still going to run. The gallery worker is a single goroutine consuming both channels serially, so an operation queued behind a large download sits in "pending" with nothing bumping updated_at, and ReapStaleOperations gives up on it after 30 minutes. That used to be self-healing: the worker dequeued it, Create reset the row to "pending", and the operation reported its real outcome. With the freeze the row stayed "failed" forever while the install ran and succeeded underneath it: a red failure card offering Retry for a model that is installed, omitted from ListActive so no replica hydrates it, and no longer deduped cluster-wide by FindDuplicate. Freeze on ("completed", "cancelled") instead. That is all the cancelled-install fix ever needed, and it leaves a failure correctable by what actually happened. The set is separate from terminalStatuses, which ListTerminal, ClearTerminal and CleanOld all still want in full, because the two mean different things: a failure can be superseded by a real outcome, a completion or a cancellation is the real outcome. UpdateStatus now writes the error column unconditionally, so a corrected outcome drops the previous attempt's reason rather than being recorded as completed while still carrying "stale operation reaped" as its error. Also adds the route-level spec for the 500 branch of DELETE /api/operations/history, and trims a comment that credited the persisted cancellable column with more than it survives long enough to do. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): offer Cancel in the phase that can honour it The cancellable flag was set at both ends of an operation's life and was wrong at both, in opposite directions. A queued operation is cancellable whatever it is. EnqueueModelOp and EnqueueBackendOp select on the operation context, so cancelling one that is still waiting releases the delivery goroutine and abandonQueued retires it: the worker never sees it, nothing is downloaded, nothing is deleted. markQueued nevertheless wrote Cancellable: !deletion, so a queued removal reported cancellable: false and the UI hid the Cancel button in the one window where pressing it both works and leaves no trace. A removal queued behind a large install was stuck there until the install finished. A running removal is not cancellable at all. DeleteModel and DeleteBackend take no context, and modelHandler only checks the operation context after the call returns, so a "cancelled" verdict would land after the model was already gone. Both handlers nevertheless wrote Cancellable: true unconditionally at entry, ahead of the op.Delete branch, offering a Cancel button the server cannot honour. So the queued phase is more cancellable than the running phase, which is the reverse of the usual shape. markQueued now reports true unconditionally, and the handler-entry writes report !op.Delete. Both sites carry a comment saying why, because reading either one alone suggests the other is a bug. GalleryStore.Create keeps !op.Delete: it runs at dequeue, so its value already describes the running phase. Its comment now says so. Specs cover queued removal, queued install, running removal and running install through the handlers, plus the queued-removal case through /api/operations where the flag is consumed, plus the behaviour the whole asymmetry rests on: a removal cancelled while queued never reaches the worker and deletes nothing. No existing spec asserted the old values. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Write] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): clamp the installer message, and add a real-binary e2e spec Running the page against a real local-ai showed the legacy installer message wrapping to three lines and dominating the card: it embeds an absolute file path, so it is both long and a single unbreakable token. One line, ellipsised, full text in the title, matching what the error string already does. The spec that found it runs with no route stubbing at all. Every other spec here stubs /api/operations, which is how a payload the server cannot emit (isDeletion true on a live operation) stayed green through a full review while the UI rendered a removal as an install. It is skipped unless LOCALAI_REAL_BINARY is set, so CI is unaffected. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
83a0f16a21 |
feat(gallery): let one gallery entry offer several builds of the same model (#10943)
* feat(system): expose raw detected capability for model meta resolution Model meta gallery entries express hardware fallback through candidate ordering rather than a capability map, so they need the undecorated detected capability string without Capability's default/cpu fallback chain. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(system): drop duplicate capability accessor, cover DetectedCapability ReportedCapability was added with a body identical to the existing DetectedCapability. Keep one accessor and move the specs onto it, since DetectedCapability had no direct coverage of its no-fallback behavior. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): parse IEC binary size suffixes (KiB..PiB) ParseSizeString accepted only SI suffixes, so a "20GiB" floor was rejected outright. Model and VRAM sizes are conventionally quoted in IEC units, and silently reading GiB as GB would understate a floor by about 7%. Purely additive: these inputs previously returned an unknown-suffix error. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add Candidate type for meta model entries Candidate is one option in a meta entry's ordered variant list. It names a concrete gallery entry and declares when that entry suits the host. EffectiveMinVRAM resolves the VRAM floor, letting an authored min_vram win over a nightly-inferred one. An unparseable floor errors instead of being treated as absent: swallowing a typo would turn a constrained candidate into an unconstrained one and select a too-large variant rather than fail loudly. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add hardware-aware model variant resolver Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): allow gallery model entries to declare variant candidates A gallery entry with a non-empty candidates list is a meta entry: it names an ordered list of concrete entries and resolves to the first one the host can satisfy, instead of describing model files directly. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): resolve meta model entries to hardware-appropriate variants at install Meta gallery entries carry an ordered candidate list; at install time the first candidate the host satisfies is resolved and its payload installed under the meta's name, so the model keeps a stable name regardless of which variant backs it. The resolution is recorded in the installed gallery config so a reinstall honors a prior pin and operators can see the backing variant. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): key meta pin recall on the installed name and detach resolved entries Six review findings on the meta-entry install path. Pin recall was keyed on the gallery entry name while applyModel writes the record under the install name (req.Name when supplied), so a meta installed under a custom name with a pin lost that pin on reinstall and was silently re-resolved onto a different variant, possibly swapping its backend. Compute the install name with applyModel's own precedence before the recall. ResolveMetaModel returned a shallow struct copy, so the resolved entry's Overrides aliased the gallery entry's map and the install path's in-place mergo merge wrote the caller's request into the shared catalog. Detach Overrides, ConfigFile, AdditionalFiles, URLs and Tags. Not exploitable today only because this path re-unmarshals the gallery per call, which is a property nobody should have to rely on. Also: overlay the meta's name onto the persisted config for meta installs so the gallery file no longer records the variant's name; move the pinned-VRAM warning below the variant validation so a pin naming a nonexistent entry does not warn about VRAM before failing for an unrelated reason; and stop seeding config.URLs in the config_file branch, which duplicated every declared URL. Add seven network-free specs driving InstallModelFromGallery with a meta entry: variant payload wins over the meta's legacy url fallback, the resolution record round-trips to disk, a pin is recorded and honored on reinstall including under a custom install name, and the resolved entry does not alias the gallery's maps. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): deep-copy meta overrides and make two specs functional ResolveMetaModel detached the resolved entry's Overrides and ConfigFile with maps.Clone, which only copies the top level. Gallery overrides are nested in practice (parameters.model is near-universal) and the install path merges the caller's request with mergo.WithOverride, which recurses into nested maps and overwrites them in place, so the gallery entry's own inner maps were still reachable and still got rewritten by the last caller to install. Copy both maps all the way down instead, recursing through the container shapes a YAML decoder produces. ConfigFile is not mutated on the install path today, but it carries the same kind of nested payload and leaving it shallowly cloned would invite the bug back. Also fix two specs that passed whether or not their target fix was present: - "does not write the caller's overrides back into the gallery entry" re-read the catalog from disk, which re-unmarshals fresh structs and so cannot observe in-memory aliasing. It now asserts against the in-memory gallery entry and drives the real mergo merge. - "round-trips the resolution record to disk under the meta's name" asserted a name that is already correct in the config_file branch. It now drives the url branch via a file:// fixture, where the meta-name overlay actually applies. Both were verified red by reverting their fix. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(gallery): lint meta model entry invariants in index.yaml Adds Ginkgo specs that parse the shipped gallery/index.yaml and enforce the invariants that keep meta entries safe: a legacy url fallback equal to the final candidate's url, references only to existing non-meta entries, a min_vram floor on every candidate but the last-resort one, a capability drawn only from the vocabulary the system can report, and descending VRAM floors within a capability group. The capability check is the only compensating control for a typo there. Candidate matching is a case-sensitive exact comparison against SystemState.DetectedCapability(), so an unknown value never matches and falls through silently instead of erroring. The vocabulary therefore mirrors the raw return set of getSystemCapabilities(), which notably excludes "cpu": that is a fallback key inside Capability(capMap) on the meta backend path, never a reported capability. A CPU-only host reports "default". These pass vacuously until the pilot meta entry lands; the guard is intentionally in place before the thing it guards. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(gallery): close coverage gaps in the meta entry lint The ordering invariant grouped candidates by capability and asserted floors descend within a group. A candidate with an EMPTY capability matches every host, so it does not belong in its own group: it dominates every later candidate whose floor is at or above its own, across capability groups. Track a running minimum floor over the unconditional candidates instead, which subsumes the old same-group check for the empty capability. Every spec skipped non-meta entries, so with zero meta entries in the index all five bodies were no-ops. Aligning GalleryModel.IsMeta() with GalleryBackend.IsMeta(), whose semantics are deliberately opposite, would have made all of them pass while checking nothing. Extract each invariant into a helper over a slice of entries returning the violations it finds, and cover those helpers with synthetic fixtures so the logic stays tested at zero meta entries. The index-driven specs are now a thin application of already proven logic. Also assert the index parses non-empty, report every violation in one run rather than aborting on the first, and parse the index once for the suite. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(gallery): add nightly denormalization of meta model candidates Fills the read-only backend, quantization and inferred_min_vram fields on meta gallery candidates and opens a PR, modeled on the existing checksum_checker job. Computing these needs network access, so it happens nightly rather than at install time. An authored min_vram is never modified: a human who measured a real load knows more than a pre-download estimate does. The index is rewritten via yaml.Node rather than a document round-trip. A full round-trip reflows all ~26k lines of gallery/index.yaml, which would bury the computed values and make the nightly PR unreviewable. The rewrite touches only the three derived keys, so authored styling survives and a run that computes nothing leaves the file untouched. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ci): keep the gallery denormalize diff reviewable and self-healing The nightly denormalization job edits YAML nodes instead of round-tripping structs so its PR stays small enough for a human to review, but the write path undid that: yaml.Marshal re-encoded the node tree at yaml.v3's default 4-space indent and dropped the leading document marker, reflowing roughly 6000 lines around the handful of real changes. Encode through yaml.NewEncoder at the index's authored 2-space indent and restore the header. A write that changes three fields now changes three lines. Stale inferred_min_vram values were also never cleared. Both skip paths (an authored min_vram is present, or the candidate is the last resort) returned before touching the field, so a candidate that gained a floor or became the last resort after a reorder kept an inferred value that EffectiveMinVRAM reported as a real constraint, failing the meta lint with no way for the job to self-heal. Clear the field before both skips. The workflow discarded a whole night's work on any single failure: the program exits 1 when a candidate cannot be estimated, which aborted the job before the PR step, so one unreachable candidate blocked every other refresh indefinitely. Capture the status, open the PR with what was computed, mark the PR body as partial, and fail the run afterwards so the problem still surfaces. Also preserve the index's existing file mode instead of forcing 0644, and drop the redundant //go:build ignore tag, since Go already skips dot directories and the sibling modelslist.go carries no tag. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add nanbeige4.1-3b meta entry with hardware-resolved variants Adds the first real meta entry to the gallery index. It resolves to the Q8_0 build on hosts with at least 6GiB of VRAM and to the Q4_K_M build everywhere else, installing either payload under the stable name nanbeige4.1-3b. The entry carries a url equal to its final candidate's url. LocalAI releases that predate candidates support parse the index non-strictly and drop the key silently, so without that url they would list the entry and install nothing. A regression spec parses the index the way those releases do and asserts every meta entry stays installable for them. Also teaches core/schema/gallery-model.schema.json about candidates. The schema sets additionalProperties: false at the top level, so an author following CONTRIBUTING.md and adding the yaml-language-server comment would otherwise get a validation error on this entry. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): make candidate entries complete, installable entries Reworks hardware-resolved gallery variants after a design pivot. There is no longer a separate "meta" entry kind. A gallery entry is a normal, complete entry that may additionally carry candidates:, a list of hardware-gated upgrades over itself, and the entry is itself the last-resort candidate. The previous design relied on a bare url: as the fallback for LocalAI releases that predate candidates support. That fallback is empty in practice: none of the 80 gallery/*.yaml files carry a top-level files:, and 1216 of 1281 index entries carry their payload in the index entry itself, so a url alone yields a config template with nothing to download. Since every released LocalAI reads gallery/index.yaml live from master, merging a payload-less entry would have shown every existing user a model that installs to a broken state. Making the entry its own base candidate removes the problem at the root: old clients drop the candidates key and install the entry exactly as they do today. Resolution order is now explicit pin, then capability plus VRAM over the declared upgrades, then the entry itself. The entry ALWAYS installs: when its own min_vram or capability is unmet the installer warns and installs it anyway, because there is nothing below it and refusing would make the gallery behave worse the newer the client is. A pin naming the entry's own name is valid and is how an operator declines an upgrade. IsMeta() becomes HasCandidates(), ResolveMetaModel becomes ResolveVariant, and the persisted meta_name record key becomes entry_name. GalleryBackend.IsMeta() is a separate concept and is untouched. The lint drops the three rules the pivot makes wrong (url equality with the final candidate, no inline payload, unconstrained final candidate) and gains one: the entry's own floor must sit strictly below every candidate's, since a base that outranks a candidate makes that candidate unreachable. The pilot entry is now the existing nanbeige4.1-3b-q4, which gains a 2GiB floor of its own and a single 6GiB upgrade to nanbeige4.1-3b-q8, replacing the separate nanbeige4.1-3b entry added in |
||
|
LocalAI [bot]
|
bcc41219f7 |
feat: materialize Hugging Face model artifacts (#10825)
* feat(config): add model artifact source contract Assisted-by: Codex:GPT-5 [Codex] * feat(downloader): add authenticated raw-byte progress Assisted-by: Codex:GPT-5 [Codex] * feat(huggingface): resolve immutable snapshot manifests Assisted-by: Codex:GPT-5 [Codex] * feat(models): add artifact storage primitives Assisted-by: Codex:GPT-5 [Codex] * feat(models): materialize pinned Hugging Face snapshots Assisted-by: Codex:GPT-5 [Codex] * feat(models): bind managed snapshots at runtime Assisted-by: Codex:GPT-5 [Codex] * feat(gallery): materialize model artifacts during install Assisted-by: Codex:GPT-5 [Codex] * feat(gallery): declare managed Hugging Face artifacts Assisted-by: Codex:GPT-5 [Codex] * feat(models): preload managed model artifacts Assisted-by: Codex:GPT-5 [Codex] * fix(gallery): retain shared artifact caches on delete Assisted-by: Codex:GPT-5 [Codex] * feat(models): report artifact acquisition progress Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): load managed models from ModelFile Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): load staged speech model snapshots Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): use staged snapshots in engine backends Assisted-by: Codex:GPT-5 [Codex] * test(distributed): cover staged artifact snapshots Assisted-by: Codex:GPT-5 [Codex] * docs: explain managed model artifacts Assisted-by: Codex:GPT-5 [Codex] * docs: add product design context Assisted-by: Codex:GPT-5 [Codex] * feat(ui): show model artifact download progress Assisted-by: Codex:GPT-5 [Codex] * Eagerly materialize Hugging Face artifacts Materialize HF-backed model references as managed GGUF artifacts during load, with lazy download retained only as fallback. Assisted-by: Codex:GPT-5 [shell] * Refactor HF downloads through a shared executor Assisted-by: Codex:GPT-5 [shell] * drop Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
LocalAI [bot]
|
6eea3ef2ac |
fix(backends): make backend install ops idempotent unless forced (#10643)
* fix(backends): make backend install ops idempotent unless forced POST /backends/apply hardcoded force=true through LocalBackendManager.InstallBackend, so applying an already-installed backend re-downloaded and re-extracted the whole artifact every time. API clients that ensure a backend exists at startup paid a full OCI image pull on every boot. Backend install ops now default to non-forced — an installed, runnable backend short-circuits (the orphaned-meta reinstall path in InstallBackendFromGallery is preserved) — and reinstall stays available: - ManagementOp gains a Force field; the local manager passes it through instead of hardcoding true. - /backends/apply accepts an optional "force" boolean in the body. - The React UI install route keeps forcing, since its button doubles as the explicit "Reinstall backend" action. Distributed installs already behaved this way (workers skip when the binary exists unless force is set); this aligns single-node behavior. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(backends): don't force-reinstall LOCALAI_EXTERNAL_BACKENDS on boot The startup loop for LOCALAI_EXTERNAL_BACKENDS runs InstallExternalBackend for each listed backend on every boot, and its gallery-name path hardcoded force=true — so every start re-downloaded and re-extracted each listed backend's OCI image even when it was installed and runnable. Supervising apps that list several backends paid several full OCI pulls per launch. Give InstallExternalBackend an explicit force parameter (it only affects the gallery-name fallback; URI installs always write) and pass: - false from the boot loop and `local-ai backends install` (idempotent ensure — `backends upgrade` is the refresh path), - op.Force from the local manager's external-URI op, - the request's force on the worker install path and true on its upgrade path (behavior unchanged). Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
LocalAI [bot]
|
92dea961c2 |
fix: distributed backend reinstall/upgrade UI stuck on 'reinstalling' (#10214)
* fix(galleryop): self-evict terminal ops from OpCache.GetStatus The processingBackends map (the UI 'reinstalling' spinner source) only cleared an op when a client polled /api/backends/job/:uid. The Manage-page Reinstall and Upgrade buttons never poll, so completed installs leaked into processingBackends forever and the backend card spun 'reinstalling' even though the install had finished. Evict terminal ops on the list read instead; DeleteUUID already broadcasts the eviction so peer replicas converge. Reproduced on a live 5-node distributed cluster: 5 backends sat in processingBackends with underlying jobs reporting completed:true,progress:100. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(nodes): clear pending backend ops behind offline/draining nodes ListDuePendingBackendOps filters status=healthy, so a backend op queued against a node that went offline (stale heartbeat) or draining (admin action) was never retried, aged out, or deleted - it leaked forever and kept the UI operation spinning. Add DeleteStalePendingBackendOps and run it each reconcile pass: draining nodes are cleared immediately (model rows already purged), offline nodes once their heartbeat is older than a grace window (blip protection). Reproduced on a live cluster: orphaned llama-cpp install rows targeting an offline (nvidia-thor) and a draining (mac-mini-m4) node sat at attempts=0 indefinitely. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(nodes): stream per-node progress during backend upgrade The install dispatch subscribed to a per-op progress subject and streamed per-node download ticks; the upgrade dispatch did a bare 15-minute blocking NATS round-trip with no subscription, so the UI showed progress:0 the whole time (the 'reinstalling but nothing happens' report on a slow node). Thread the op ID through BackendManager.UpgradeBackend -> the distributed manager -> the adapter, and have the adapter subscribe to the per-op progress subject before the request (extracted into a shared subscribeProgress helper reused by install/upgrade/force-fallback). The worker's upgradeBackend now creates the same DebouncedInstallProgressPublisher installBackend uses. An upgrade is a force-reinstall, so it reuses SubjectNodeBackendInstallProgress rather than minting a new subject - no new NATS permission, no new rolling-update compat surface. Reconciler-driven retries pass empty opID/onProgress and stay on the silent path. Reproduced on a live cluster: upgrade of llama-cpp-development on agx-orin-slow sat at progress:0 for 4+ minutes with no per-node feedback. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(galleryop): persist cancellation + periodically reap orphaned ops Two distributed gaps surfaced when a replica was killed mid-upgrade on a live cluster, leaving the backend stuck 'processing' in the UI forever: 1. CancelOperation flipped the in-memory status to cancelled and broadcast a NATS event but never persisted the terminal status. On the next replica restart the still-active row re-hydrated straight back into processingBackends and the UI spun again. It now calls store.Cancel(id) so the cancel survives a restart. 2. CleanStale (which marks abandoned active ops failed) only ran once on startup, so an op orphaned AFTER startup - its owning replica's foreground handler goroutine gone - was never reaped until the next restart. Add GalleryService.ReapStaleOperations and run it on a 15m ticker (CleanStale now returns the reaped count for observability). Neither is covered by the OpCache self-evict fix: an orphaned op never reaches Processed, so it would never self-evict. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(review): address self-review findings on the distributed install fixes Three findings from an adversarial review of this branch: 1. CRITICAL - OpCache.GetStatus crashed under concurrent load. m.Map() returns the live internal map by reference, so deleting from it on the read path was an unsynchronized write to a map four HTTP handlers poll every ~1s -> a 'concurrent map writes' fatal. Rewritten to iterate a Keys() snapshot, build a fresh result map, and apply evictions via the locked DeleteUUID after the loop. Added a -race concurrency regression guard. 2. HIGH - GetStatus evicted failed ops too, hiding them from /api/operations and breaking the dismiss-failed-op flow (the panel keeps Error != nil ops so the admin can read the error and click Dismiss). Eviction now fires only for terminal ops with Error == nil (success/cancelled); failures are retained. 3. MEDIUM - DeleteStalePendingBackendOps missed StatusUnhealthy nodes. A node marked unhealthy on a NATS ErrNoResponders never transitions to offline (health.go skips re-marking it), so its pending ops leaked exactly like the offline case. Unhealthy is now reaped via the same stale-heartbeat grace path (a fresh-heartbeat node is recovering and keeps its op). Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(review-2): don't evict the still-installing soft-path; don't spin on failed ops Second review pass found two issues: 1. MEDIUM (Go) - OpCache.GetStatus evicted the ErrWorkerStillInstalling soft-path op. That op is deliberately Processed=true with no error to show a yellow in-progress state when a worker timed out the NATS round-trip but is still installing in the background; the reconciler confirms the real outcome later. Evicting it (and broadcasting OpEnd + marking the DB completed) hid an install that may still fail. Eviction is now scoped to a clean success (progress 100 + 'completed', matching the job-poll's historical condition) or a cancellation - the soft-path (progress != 100) and failures are kept. 2. MEDIUM (React) - the Backends gallery card rendered ANY operation as an 'Installing...' spinner, so a failed op (now intentionally kept in the list for the OperationsBar error + Dismiss) spun forever. Exclude errored ops from the card spinner, mirroring Models.jsx (isInstalling already excludes op.error). The error + Dismiss still surface in the global OperationsBar. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): refresh Manage backends table when an operation settles The Manage backends table fetched installed backends only on mount/after delete and checked upgrades only on tab activation. After a reinstall/upgrade completed neither re-ran, so the installed-version cell and the 'update available' badge stayed stale until the user switched tabs - the op looked like it 'did nothing'. Watch the operations list (via useOperations) and re-fetch installed backends + available upgrades whenever the count settles, mirroring the operations.length watch Backends.jsx already uses. Consolidates the prior tab-activation upgrades check into the same effect. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
LocalAI [bot]
|
8d6548c0b9 |
fix(distributed): sync gallery OpCache + caches across frontend replicas (#9983)
When the LocalAI frontend deployment is scaled past one replica, the UI's
/api/operations poll round-robins between pods. Each pod kept the OpCache
(galleryID->jobID), OpStatus map, and the post-install in-memory caches
(ModelConfigLoader, UpgradeChecker) purely in-process. Reads never
consulted PostgreSQL or NATS even though writes already published to PG.
Symptoms:
- A user installing a model on replica A saw the operation card flicker
in and out as the load balancer alternated.
- The Models page re-fetched the whole gallery on every flicker because
useEffect([operations.length]) re-fires when the count changes.
- A chat completion that landed on replica B after the install completed
on replica A failed to find the new model — B's ModelConfigLoader was
still the old one because nothing told it to reload from disk.
- The UpgradeChecker 6-hour cache stayed stale on peer replicas after a
backend upgrade, so /api/backends/upgrades kept surfacing an upgrade
that had already shipped.
Mirror the jobs Dispatcher pattern for gallery ops:
- OpCache learns SetMessagingClient/SetGalleryStore + a Start(ctx) that
hydrates from PostgreSQL and subscribes to gallery.opcache.{start,end}.
Set/SetBackend now upsert cache_key + is_backend_op on the gallery_
operations row and broadcast OpCacheEvent so peers merge it in. The
hydrate path uses a new GalleryStore.ListActive() (status in {pending,
downloading, processing} and updated within 30 min).
- GalleryService.SubscribeBroadcasts wires a SubjectGalleryProgress-
Wildcard subscriber that calls a new lock-light mergeStatus into the
local statuses map, plus a SubjectGalleryCancelWildcard subscriber that
runs the locally-registered cancel func. Hydrate() restores active rows
from PostgreSQL on startup so a freshly-started replica is not
observably empty mid-install. CancelOperation tolerates the cancel func
living on a different replica and publishes anyway.
- modelHandler and backendHandler publish on the new
SubjectCacheInvalidateModels / SubjectCacheInvalidateBackends after
a successful install/delete/upgrade. SubscribeBroadcasts wires peers
to refresh: OnModelsChanged (re-runs LoadModelConfigsFromPath) and
OnBackendOpCompleted (re-triggers UpgradeChecker). The originating
replica reloads inline so it never enters the broadcast handler.
- OpStatus.Error (an error interface) flat-marshalled to "{}" over JSON,
so a failed install replicated to a peer arrived with a nil error and
the UI's failure banner never appeared. Add MarshalJSON/UnmarshalJSON
via an opStatusWire shim that round-trips Error as a string.
- UpdateStatus and CancelOperation now drop the mutex before publishing
to NATS or persisting to PostgreSQL. The wildcard subscriber's
mergeStatus loops back into the same service on the publishing replica
and would deadlock otherwise; this also prevents future PG round-trips
from stalling concurrent readers on every progress tick.
Tests cover the OpStatus error round-trip, OpCache propagation through a
shared in-memory bus, OpCache PostgreSQL hydration (active-only),
GalleryService progress + cancel broadcast, Nodes preservation across a
peer's bare progress tick, GalleryService hydration from PG, and the
two cache-invalidation broadcasts (models + backends). 44 specs total
in galleryop; routes/operations specs and jobs/agents suites still pass.
Assisted-by: claude-code:claude-opus-4-7
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
|
||
|
LocalAI [bot]
|
a0f3e26245 |
fix(distributed): make admin backend installs resilient and observable (#9958)
* feat(distributed): add configurable NATS backend install/upgrade timeouts Adds BackendInstallTimeout and BackendUpgradeTimeout to DistributedConfig with 15m defaults, following the existing MCPToolTimeout / WorkerWaitTimeout pattern. These will replace the hardcoded literals in RemoteUnloaderAdapter so admin-driven backend installs across the cluster survive long OCI image pulls that previously timed out at 3m. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * style(distributed): gofmt alignment after timeout fields Re-aligns the Validate() negative-duration map and the Default* const block so the new BackendInstall/UpgradeTimeout entries do not leave the surrounding columns mis-padded. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(cli): surface LOCALAI_NATS_BACKEND_INSTALL_TIMEOUT and _UPGRADE_TIMEOUT Parses the two new env vars on the run CLI and threads them through the existing AppOption builder so DistributedConfig picks them up. Invalid duration strings now fail loudly at startup rather than silently falling back to the default. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(distributed): inject NATS install/upgrade timeouts into RemoteUnloaderAdapter Removes the hardcoded 3m / 15m literals from RemoteUnloaderAdapter and threads in DistributedConfig.BackendInstallTimeoutOrDefault() and BackendUpgradeTimeoutOrDefault() at construction. Install now defaults to 15m (was 3m); cold OCI image pulls on Jetson Wi-Fi routinely blew past the old ceiling. Scripted messaging client captures the timeout so tests can assert the configured value actually reaches the NATS request. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(distributed): introduce galleryop.ErrWorkerStillInstalling sentinel When the NATS request-reply for backend.install (or .upgrade) times out the worker is almost always still pulling the OCI image. Wrap the timeout in a typed sentinel so the manager above can distinguish "worker hung" from "worker still working" and leave the pending_backend_ops row in place for the reconciler to confirm via backend.list. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(distributed): treat NATS install timeout as in-progress, not failure When a worker times out replying to backend.install but the install is still running on the worker, enqueueAndDrainBackendOp now reports a running_on_worker status and pushes NextRetryAt out by the install timeout so the reconciler does not immediately re-fire another install while the worker is still pulling the image. The pending_backend_ops row stays in place for the next reconciler pass to confirm via backend.list. InstallBackend wraps the result in galleryop.ErrWorkerStillInstalling so callers can branch (galleryop renders yellow in-progress instead of red error). UpgradeBackend uses the same wrap. Adds RemoteUnloaderAdapter.InstallTimeout() so the manager can push NextRetryAt by the configured timeout without reaching into a private field, and NodeRegistry.RecordPendingBackendOpInFlight as the soft cousin of RecordPendingBackendOpFailure. Also includes incidental gofmt-driven struct-field alignment in registry.go on lines unrelated to the change (touched files are re-formatted to canonical form per project policy). Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): don't increment Attempts on in-flight install timeout An in-flight timeout (worker still pulling the OCI image) is not a failed attempt, it's a delayed one. Incrementing Attempts let genuinely-progressing slow installs (e.g. 30 GB CUDA images on Wi-Fi) trip the reconciler's maxPendingBackendOpAttempts cap and dead-letter the queue row while the worker was still legitimately working. RecordPendingBackendOpInFlight now only updates LastError and NextRetryAt. Also documents "running_on_worker" in the NodeOpStatus.Status enum comment so Task 6 implementers see the full surface. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(galleryop): surface ErrWorkerStillInstalling as non-error OpStatus When the distributed backend manager returns an error that wraps ErrWorkerStillInstalling, backendHandler now completes the op with a "still installing in background" message rather than marking it as a red failure. Admin UI sees a yellow in-progress state; reconciler confirms completion on its next pass. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(distributed): end-to-end install-timeout-then-reconcile Wires Task 1-6 end-to-end so any seam mismatch surfaces in CI rather than during a real cluster install. NATS times out, the queue row stays alive with running_on_worker status, the worker eventually reports the backend installed via backend.list, the manager surfaces it via ListBackends. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(distributed): document LOCALAI_NATS_BACKEND_INSTALL_TIMEOUT / _UPGRADE_TIMEOUT Add the two new operator-tunable env vars to the Frontend Configuration table in the distributed-mode docs. Explains the 15m default, when to raise it (slow links pulling multi-GB OCI images), and the new "still installing in background" admin-UI state when the round-trip times out but the worker is still working. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(distributed): clear pending install rows when backend.list confirms DistributedBackendManager.ListBackends now proactively clears pending_backend_ops install rows whose (nodeID, backend) is reported installed by backend.list. Operator UI updates immediately instead of waiting up to installTimeout (default 15m) for the next reconciler tick after NextRetryAt. Only install rows are cleared; upgrade and delete intents are not satisfied by presence in backend.list and continue to drain through their normal reconciler paths. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(messaging): add BackendInstallProgressEvent wire type and subject New NATS subject nodes.<nodeID>.backend.install.<opID>.progress lets the worker publish transient progress events (file, current/total bytes, percentage, phase) while a long-running install pulls its OCI image. BackendInstallRequest gains an optional OpID field so the worker knows which subject to publish on. Transient pub/sub (not JetStream): the install reply remains ground truth for success/failure; dropped progress events are tolerable. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * style(messaging): drop em-dash from BackendInstallProgress test comment Per project convention (no em-dashes anywhere). Comment substance is unchanged. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(distributed): worker publishes debounced install progress over NATS When BackendInstallRequest.OpID is set, the worker's backend.install handler wires a debounced publisher (250ms window) into the gallery download callback. Each tick becomes a BackendInstallProgressEvent on nodes.<nodeID>.backend.install.<opID>.progress; the publisher always emits a final event on Flush so the UI sees the terminal percentage. Old masters that do not set OpID continue to run silent installs: no behavior change for them. Lock ordering: the publisher releases its mutex before calling messaging.Publish so a slow network never stalls the install loop. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(distributed): RemoteUnloaderAdapter subscribes to install progress InstallBackend gains opID + onProgress parameters. When both are set, the adapter subscribes to nodes.<nodeID>.backend.install.<opID>.progress BEFORE publishing the install request, decodes each message into the caller's onProgress callback in a goroutine (so a slow callback never stalls the NATS reader thread), and unsubscribes after RequestJSON returns. When onProgress is nil OR opID is empty (the reconciler retry path), subscription is skipped entirely - silent installs cost nothing extra. Subscribe failure is logged at Warn and the install proceeds without progress streaming; the NATS round-trip still owns terminal status. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(distributed): forward backend install progress into galleryop OpStatus DistributedBackendManager.InstallBackend now passes the gallery op ID and a progress bridge into the adapter call. Each BackendInstallProgressEvent from the worker becomes a galleryop.ProgressCallback tick - which the existing backendHandler already turns into OpStatus.UpdateStatus, so the admin UI/SSE polling sees per-byte progress for distributed installs without any UI-side change. UpgradeBackend is intentionally left silent for now: its wire request (BackendUpgradeRequest) does not carry OpID, and rolling-update fallback is the rarer path. Will be picked up in a follow-up if the worker upgrade path also gets a progress channel. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(distributed): InstallBackend tolerates silent (pre-Phase-2) workers A worker on pre-Phase-2 code never publishes progress events. The new master subscribes optimistically; this spec pins that a silent worker still produces a green install with no progressCb ticks. The install reply is the source of truth for terminal state; the progress stream is a best-effort UX enrichment. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(distributed): document install progress streaming Note the new nodes.<nodeID>.backend.install.<opID>.progress subject and the silent-worker compatibility behavior so operators know to expect real-time progress and what happens on a mixed-version cluster. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(distributed): note progress-event ordering trade-off in InstallBackend Document near the goroutine dispatch why ordering at the consumer is best-effort, why it rarely matters in practice (worker debounce >> goroutine jitter), and what a future hardening pass would look like (Seq field + stale-by-seq drop). Stops the next reader from accidentally "fixing" the goroutine pool away. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(galleryop): add NodeProgress + OpStatus.Nodes for per-node breakdown Adds the data model the UI needs to render an expandable per-node breakdown of a fanned-out backend install. NodeProgress carries node identity (ID + name), per-node status (queued / running_on_worker / success / error / downloading), the current file + bytes + percentage from the Phase 2 progress stream, and any per-node error. OpStatus.Nodes is the slice the /api/operations handler will surface in a follow-up. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(galleryop): UpdateNodeProgress merges per-node ticks by NodeID GalleryService.UpdateNodeProgress(opID, nodeID, np) merges a NodeProgress into OpStatus.Nodes (keyed by NodeID, no duplicates) and mirrors the latest tick into the aggregate Progress / FileName / DownloadedFileSize / TotalFileSize fields so the legacy single-bar OperationsBar view keeps working unchanged alongside the new per-node breakdown. Concurrent-safe via the existing g.Mutex. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(distributed): write per-node OpStatus entries during install fan-out DistributedBackendManager now accepts a nodeProgressSink and feeds it two streams: 1. enqueueAndDrainBackendOp emits a per-node terminal entry on each status it appends to BackendOpResult (queued, success, error, running_on_worker). The opID is threaded through the function so the sink gets the right gallery op identity. 2. The install apply closure fans each BackendInstallProgressEvent into the sink as a downloading entry, alongside the legacy progressCb path so the aggregate single-bar view stays correct. Production wiring passes the GalleryService (which implements UpdateNodeProgress via Task 2) as the sink. Single-node tests pass nil. DeleteBackend and UpgradeBackend pass an empty opID so the sink path no-ops for ops that aren't gallery-tracked the same way as Install. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(operations): expose per-node breakdown on /api/operations When an operation's OpStatus has Nodes entries (populated by the Phase 4 progress sink wiring), surface them as a "nodes" array on the /api/operations response, sorted by node_name for stable rendering. Backward compatible: legacy clients ignore the field; ops without any node entries (single-node mode, model installs) omit the array entirely thanks to the empty-slice guard. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): per-node breakdown in OperationsBar When an install op fans out to more than one worker, the operations bar now shows a "N nodes" chevron that expands into a per-node list. Each row carries the node's status (color-coded pill), the current file being downloaded, byte counts, percentage, and a thin per-node progress bar. Yellow "Worker busy" pill marks running_on_worker status with a tooltip explaining the NATS round-trip timed out but the worker is still installing in the background. Backward compatible: ops without a nodes field (legacy or single-node mode) render as before. State for expand/collapse is local to the component, keyed by jobID/id - reload starts collapsed. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(distributed): document per-node breakdown in the operations bar Adds a short subsection covering the expandable "N nodes" chevron in the OperationsBar admin UI, the meaning of each status pill, and how it relates to the /api/operations nodes array. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(galleryop): UpdateStatus preserves Nodes when caller sends none Real-world bug surfaced by the Phase 4 multi-worker smoke test: the nodes[] array in /api/operations flickered between a single node at a time on a 2-worker install. Root cause: the Phase 2 progress bridge also calls the legacy progressCb -> UpdateStatus(&OpStatus{...}) on every tick. UpdateStatus then overwrote the entire status pointer, wiping the Nodes slice that UpdateNodeProgress had just merged in. Fix: in UpdateStatus, if the incoming op has an empty Nodes slice, carry forward the previous status's Nodes before storing. Callers that explicitly populate Nodes still win (their slice replaces the prior one, no merge across the two code paths). Two regression specs added pinning both directions of the contract. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(distributed): strip implementation details from user-facing docs Trim the new install/upgrade timeout rows and the install-progress sections to focus on what the operator sees and tunes. Drops: - the NATS subject names and pub/sub mechanics - "round-trip" / reconciler / backend.list jargon - /api/operations polling cadence - "pre-2026-05-22" version references Reframes the breakdown text around the admin UI (Operations Bar, chevron, status pills, "Worker busy" tooltip). Implementation context lives in the agent notes and code comments. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(config): move DistributedConfig.Validate flag names to constants The negative-duration check map was a wall of literal kebab-case strings that had to stay in sync with the kong-derived CLI flag names manually. Move them to a Flag* const block alongside the existing Default* block so a rename of either the Go field or the CLI naming convention forces a compile error rather than silent drift. Sole consumer today is Validate; the constants are exported so future operator-facing surfaces (e.g. error messages on other validation paths) can reference them by name instead of repeating the literals. Tests pin both the literal values (so a future "let's just rename this" doesn't accidentally regress the CLI flag) and the negative- duration error message for the new BackendInstall / BackendUpgrade fields. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(distributed): extract NodeStatus and Phase enums to constants Sweep for the same literal-string-as-identifier pattern called out on the Validate flag names: the per-node install status enum ("queued" | "downloading" | "running_on_worker" | "success" | "error") appeared as raw literals across managers_distributed.go (10+ sites, including 3 separate `n.Status == "running_on_worker"` checks), operation.go, and the test suite. Same shape for the Phase enum ("resolving" | "downloading" | "extracting" | "starting") in the worker-side progress publisher. Promote both to exported const blocks: - galleryop.NodeStatus{Queued,Downloading,RunningOnWorker,Success,Error} shared between galleryop.NodeProgress.Status (the wire field) and nodes.NodeOpStatus.Status (the in-process per-node summary) - messaging.Phase{Resolving,Downloading,Extracting,Starting} shared between the worker publisher and any future consumer that needs to switch on phase Tests pin both the literal values (so a future "let's just rename" doesn't silently change the JSON wire) and use the constants in setup (so the producer side stays drift-protected). Wire-format assertions on the /api/operations JSON output keep their literals deliberately, so the constant value can never silently diverge from what the UI receives. Out of scope for this PR (separate cleanup): the finetune and quantization job-status enums have the same anti-pattern with 14+ literal sites each, but predate this PR's work. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
LocalAI [bot]
|
a39e025d64 |
fix(nodes): make per-node backend install async via gallery job queue (#9928)
* feat(galleryop): add TargetNodeID to ManagementOp for single-node installs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(galleryop): add NodeScopedKey helpers for per-node opcache rows Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(galleryop): use strings.Cut for NodeScopedKey parsing, reject empty nodeID Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(nodes): scope DistributedBackendManager.InstallBackend to single node via TargetNodeID Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(http): make /api/nodes/:id/backends/install async via gallery service job queue The handler previously called unloader.InstallBackend synchronously and blocked the browser for up to 3 minutes waiting on the NATS reply. It now enqueues a TargetNodeID-scoped ManagementOp on BackendGalleryChannel and returns HTTP 202 + jobID immediately, matching /api/backends/install/:id. The opcache key is built via NodeScopedKey(nodeID, backend) so concurrent installs of the same backend across different nodes do not stomp each other. galleryService/opcache/appConfig are threaded through RegisterNodeAdminRoutes for this. Assisted-by: Claude:opus-4-7 [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(http): log malformed backend_galleries override and stop test drain goroutine Assisted-by: Claude:opus-4-7 [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(api): expose nodeID for node-scoped backend ops in /api/operations Node-scoped backend installs land in opcache under "node:<nodeID>:<backend>" keys. Without splitting that prefix back out, the operations panel renders the full key as the display name and has no structured way to label which worker an install is targeting. Detect the prefix, surface nodeID as its own response field, and reduce the display name back to the bare backend slug. Bare (non-scoped) ops are left untouched so legacy installs do not gain a misleading empty nodeID. Assisted-by: Claude:opus-4-7 [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(react-ui): poll job status for node-targeted backend installs Assisted-by: Claude:opus-4-7 [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(react-ui): make NodeInstallPicker state updates pure and surface cancellations as errors Assisted-by: Claude:opus-4-7 [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(react-ui): clarify async semantics in handleInstallOnTarget Assisted-by: Claude:opus-4-7 [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(http): use statusUrl casing for node install response to match codebase precedent Assisted-by: Claude:opus-4-7 [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
8ab0744458 |
feat: backend versioning, upgrade detection and auto-upgrade (#9315)
* feat: add backend versioning data model foundation Add Version, URI, and Digest fields to BackendMetadata for tracking installed backend versions and enabling upgrade detection. Add Version field to GalleryBackend. Add UpgradeAvailable/AvailableVersion fields to SystemBackend. Implement GetImageDigest() for lightweight OCI digest lookups via remote.Head. Record version, URI, and digest at install time in InstallBackend() and propagate version through meta backends. * feat: add backend upgrade detection and execution logic Add CheckBackendUpgrades() to compare installed backend versions/digests against gallery entries, and UpgradeBackend() to perform atomic upgrades with backup-based rollback on failure. Includes Agent A's data model changes (Version/URI/Digest fields, GetImageDigest). * feat: add AutoUpgradeBackends config and runtime settings Add configuration and runtime settings for backend auto-upgrade: - RuntimeSettings field for dynamic config via API/JSON - ApplicationConfig field, option func, and roundtrip conversion - CLI flag with LOCALAI_AUTO_UPGRADE_BACKENDS env var - Config file watcher support for runtime_settings.json - Tests for ToRuntimeSettings, ApplyRuntimeSettings, and roundtrip * feat(ui): add backend version display and upgrade support - Add upgrade check/trigger API endpoints to config and api module - Backends page: version badge, upgrade indicator, upgrade button - Manage page: version in metadata, context-aware upgrade/reinstall button - Settings page: auto-upgrade backends toggle * feat: add upgrade checker service, API endpoints, and CLI command - UpgradeChecker background service: checks every 6h, auto-upgrades when enabled - API endpoints: GET /backends/upgrades, POST /backends/upgrades/check, POST /backends/upgrade/:name - CLI: `localai backends upgrade` command, version display in `backends list` - BackendManager interface: add UpgradeBackend and CheckUpgrades methods - Wire upgrade op through GalleryService backend handler - Distributed mode: fan-out upgrade to worker nodes via NATS * fix: use advisory lock for upgrade checker in distributed mode In distributed mode with multiple frontend instances, use PostgreSQL advisory lock (KeyBackendUpgradeCheck) so only one instance runs periodic upgrade checks and auto-upgrades. Prevents duplicate upgrade operations across replicas. Standalone mode is unchanged (simple ticker loop). * test: add e2e tests for backend upgrade API - Test GET /api/backends/upgrades returns 200 (even with no upgrade checker) - Test POST /api/backends/upgrade/:name accepts request and returns job ID - Test full upgrade flow: trigger upgrade via API, wait for job completion, verify run.sh updated to v2 and metadata.json has version 2.0.0 - Test POST /api/backends/upgrades/check returns 200 - Fix nil check for applicationInstance in upgrade API routes |
||
|
Ettore Di Giacinto
|
59108fbe32 |
feat: add distributed mode (#9124)
* feat: add distributed mode (experimental) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix data races, mutexes, transactions Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactorings Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix events and tool stream in agent chat Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * use ginkgo Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(cron): compute correctly time boundaries avoiding re-triggering Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * enhancements, refactorings Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * do not flood of healthy checks Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * do not list obvious backends as text backends Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * tests fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Drop redundant healthcheck Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * enhancements, refactorings Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |