212 Commits

Include renames

Author	SHA1	Message	Date
Ettore Di Giacinto	e25293f7bc	feat(cluster): make a peer prove which replica it is ... GET /api/cluster/peer authenticated with the deployment's shared registration token and took the dialling replica's id from ?id= on trust. Every worker holds that token, so anything holding it could open a peer link as any replica: relay through it to every worker tunnel that replica owns, displace a real replica's inbound link by declaring its id, and point the roughly 31 GiB per-session receive window at one replica. Validating the id against the instances table does not fix this, because the attack declares a real replica's id. So the route now checks two credentials and needs both. The shared token still says the dialler belongs to this deployment; a new per-replica credential says which replica it is. The credential follows the per-node worker credential rather than inventing a second mechanism: crypto/rand.Text, stored only as a hex SHA-256, compared in constant time, with no fallback to the shared token. It differs in the stronger direction. A worker's credential is minted by the frontend and handed over once; a replica writes its own instances row, so it mints its own secret, publishes only the hash in the same statement that publishes its address, and never sends the plaintext anywhere but the peer dial. A peer that presents no credential is refused, not waved through. An old replica and an attacker holding the shared token send the same request, so accepting the first accepts the second; there is no safe downgrade here, only a quiet one. The refusal is made loud instead, on both sides, naming the upgrade rather than the network. On the documented frontend-first order a new replica still dials an old one; an old replica cannot dial a new one, which costs relayed requests that land on a not-yet-restarted replica and surfaces as no route, never as absence. A rejected peer gets its own sentinel, ErrPeerRejected, whose unwrap chain carries ErrPeerUnreachable as well and no absence sentinel at all. Keeping the older sentinel means no existing consumer changes behaviour; the cause stays out of the chain, so absence cannot escape through it and nothing can read an authorization failure as a worker that went away. One consequence beyond the fix: a replica with no advertised address has no instances row, so it now cannot dial out either. It was already unreachable inward. The startup error and the docs say so. Registry.Register, NewMembership, NewPeerPool, PeerHandler and RegisterClusterRoutes all gained required arguments, so the identity cannot be dropped without a compile failure. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-20 03:05:35 +00:00
Ettore Di Giacinto	016686a3db	chore(distributed): take the nats-io modules out of the build ... Distributed mode has not dialled a message broker since the control plane moved onto the workers' own outward tunnels and every fan-out family moved onto PostgreSQL LISTEN/NOTIFY. What was left was the dependency itself, and the code that existed only to feed it. Dropped from go.mod: nats-io/jwt/v2, nats-io/nats.go, nats-io/nkeys, nats-io/nuid and testcontainers-go/modules/nats, along with the fourteen indirect requires that only the NATS testcontainer pulled in. go.sum carries no nats line either, so the removal is not the partial kind where the require goes and the checksum stays. Deleted with them: pkg/natsauth in full, the broker client's remaining options and TLS files, the per-node JWT minting on both the register and the approve path, and the natsauth.Config parameter threaded through the node routes. The credential manager is renamed and stripped rather than deleted, because it still holds the tunnel token that every re-registration rotates. The bus flags stay accepted and ignored, and are now hidden, on every command that had them, so an existing unit file, compose file or Helm values file still starts on the day of the upgrade. What is not kept is the validation that REQUIRED one: a distributed frontend started with no bus URL is no longer fatal. The TLS paths lose type:"existingfile" deliberately, so a certificate deleted along with the broker cannot fail a startup. One operator-visible behaviour change: --nats-require-auth no longer makes an agent worker wait through admin approval. Ask for that wait with --distributed-require-auth, which already implied it. It is documented in the migration section and pinned from both sides. A deployment now needs PostgreSQL and the frontends' own HTTP listener, and nothing else. coverage-baseline.txt moves from 54.2 to 62.0. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-20 03:05:35 +00:00
Ettore Di Giacinto	c97a524013	refactor(distributed): move job and agent fan-out onto the PostgreSQL carrier ... Five of the six families whose subscriber is an open HTTP response rather than a process-lifetime cache now travel on pgbus: jobs.<id>.progress, jobs.<id>.result, jobs.<id>.cancel, agent.<name>.events.<user> and responses.<id>.cancel. Both ends of each move together, so there is no state where a publisher is on one carrier and its subscriber on the other. agent.<name>.cancel does NOT move, and the plan was wrong about why. Its only subscriber in the tree is the agent worker, which has no database and so cannot join the PostgreSQL carrier at all. Publishing that cancel on pgbus would have lost every cancel of a worker-run agent while returning nil, which reports a cancel that reached nobody as a cancel that was sent. EventBridge now names its cancel carrier separately, a frontend replica sets it to the carrier the worker reads, and it stays there until a cancel rides the worker's tunnel like every other verb addressed to a worker. The carrier drops at 256 rather than blocking, which is not safe on its own for a result: a lost result has no successor message. It is not the only path. The claiming replica persists the terminal line before it releases the claim, and an open progress stream re-reads the job row once after subscribing and then periodically, so a dropped terminal broadcast costs promptness and never the answer. Both per-request subscriptions close in a defer instead of on one return path, and pgbus grows Subscribers() so the leak they would otherwise cause can be asserted. It has no other symptom: only the first subscriber of a channel issues a LISTEN, so a leaked filter just adds one closure per notification for every stream the replica has ever served. Subscribe now issues its LISTEN before it registers, which makes that count a readiness signal rather than a figure to compare against itself. Two rules that were stated at several sites and pinned at none are now one each. The re-broadcaster is built beside the dispatcher and the bridge and handed to the dispatch loop, so no line is left that can point it at a carrier nobody subscribes to while every spec stays green. The set of statuses a job never leaves is one exported set that the SSE bridge and the store both read. The last hand-written subject filter in production code became messaging.SubjectAgentEventsWildcard. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-20 03:05:34 +00:00
Ettore Di Giacinto	94bfbd2221	feat(distributed): carry the state.*.delta families on PostgreSQL ... syncstate.Config held one carrier field typed as the NATS client, so a pgbus.Bus could not be handed to a SyncedMap at all: it satisfies messaging.Broadcaster and not MessagingClient. The durable re-hydration path built for the responses map therefore had a NATS-only consumer and nothing in the build said so. The field becomes Bus messaging.Broadcaster, SubscribeJSON moves to its own file and relaxes its parameter to Broadcaster, and the four adopters fan out over PostgreSQL LISTEN/NOTIFY: fine-tune jobs, quantization jobs, agent tasks with their per-tenant children, and Open Responses metadata. A new spec proves it on a real database, over two Bus instances on two pinned listener connections: a Set and a Delete carry, a payload past the 8000-byte notification cap comes back byte identical through the spill row, two families sharing one LISTEN channel stay separate, and a terminated listener re-hydrates a row written while it was gone. The five sites that each chose a carrier for an adopter are collapsed into one DistributedServices.Broadcast() accessor. Five field reads were five chances to leave one family on NATS with nothing failing, because messaging.Client satisfies Broadcaster too. The accessor also refuses to hand out a nil pgbus.Bus wrapped in a non-nil interface, which every adopter would read as "broadcast" and dereference on the first Set. SetTaskSyncNATS and SetJobSyncNATS are renamed to SetTaskSyncBus and SetJobSyncBus so a missed wiring site fails to compile. The response metadata table gains a retention of its own, defaulting to 24 hours. It inherited the Open Responses store TTL, which defaults to 0 meaning no expiration. Zero is defensible for a map that dies with the process and is not for a table: the table grew for the life of the deployment and a restarting replica re-hydrated every response the cluster had ever created. A row that names its own expiry is still judged on that column alone, and "this row is dead" now has one SQL spelling that PurgeExpired deletes by and ListUnexpired is the negation of, so a hydrate cannot resurrect what a sweep has already retired. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-20 03:05:34 +00:00
Ettore Di Giacinto	fa6ac21ca3	feat(distributed): give responses.metadata something to re-hydrate from ... The responses.metadata SyncedMap had no durable Store, so its reconnect re-hydrate replaced nothing. That was survivable while responses converged through deltas on a broker that mostly stayed up. It is not survivable on a carrier whose listener is one pinned PostgreSQL session: every response created while the subscription was down stays invisible on that replica forever, and the symptom is a 404 from one replica and a 200 from another for the same response_id. State that must survive a gap now lives in a response_metadata table, and the notification only says it changed. The map writes through on a Set and reads the table on hydrate, on reconnect and on reconcile, so the gap closes instead of becoming permanent. The row carries the whole projection as JSON rather than one column per field. A column-per-field schema would be a second definition of what a peer may act on, and the two would drift the first time syncedResponse gained a field: the map would broadcast the new field and hydrate without it, so a replica that had reconnected would serve a different response body from one that had not, with nothing failing anywhere. Only PayloadJSON is ever decoded; owner_replica and owner are indexed copies for an operator reading the table by hand. A missing row and an unreachable database are different facts. Every store and adapter method returns a driver failure as an error and never as an empty result, and syncstate replaces nothing when its source errors, so an outage leaves the map holding what it had rather than blanking it into a cluster-wide 404. Liveness is the database's clock, spelled expires_at IS NULL OR expires_at > now(), because every replica hydrating from this table must agree on which rows are live and a Go-side cutoff makes that a property of whichever process asked. The test container shares the host clock, so no behavioural spec can tell the two apart; the statement shape is pinned instead. The constructor refuses a non-PostgreSQL handle, because an unguarded now() on the single-binary path reads as a missing migration. A ticker sweeps expired rows every five minutes on each replica, and Close waits for it rather than racing it. Note that the sweep removes nothing while LOCALAI_OPEN_RESPONSES_STORE_TTL is 0, which is the default: with no TTL nothing ever expires and the table grows for the life of the deployment. The docs say so plainly. EnableDistributed takes the store positionally and last, so a call site that forgets it fails to compile rather than silently restoring the deltas-only map this change exists to replace. A nil store there is refused by name: it is reached only from the distributed branch of route registration, so it is a wiring bug and not a deployment shape. What still never leaves the owning replica is unchanged: the resume buffer and the CancelFunc. The write-through is one row per response state change, not one per generated token. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-20 03:05:34 +00:00
Ettore Di Giacinto	2f3d5b2e74	feat(distributed): make MCP execution and discovery a selection ... mcp.tools.execute and mcp.discovery were the only NATS subjects that combined a queue group with a reply, and no carrier in this design provides both. They never needed one: a queue group is a way of choosing a subscriber, and choosing is a query. The frontend now lists the approved, non-draining agent nodes, asks the node_connections table in one joined statement which of those tunnels a live replica holds, prefers one this replica holds so the call skips the relay hop, and issues an ordinary control RPC on the path task 4 already mounted. A peer-held tunnel is reached through the relay. That is a choice a broker's hidden balancing could not make. The selection reads presence and nothing else. It is filtered only on node type and on the two statuses an operator controls, never on a health verdict written on another clock, because refusing a worker that is connected and answering is the same defect as picking one that is gone. An empty fleet answers ErrNoAgentWorker, which is deliberately neither ErrWorkerUnroutable nor anything cluster.IsWorkerAnswer accepts: nothing was asked of any worker, so no reap guard may act on it. A reply carrying an Error is the worker's own answer and is returned unchanged; it is never offered to a second worker, which would turn "this MCP server rejected your arguments" into "the fleet is broken" and could run a tool twice. A call that never reached a worker is retried against a different pick, at most three times, and whatever error is finally returned is returned unwrapped so its identity survives the loop. MCP prompts and resources now answer 501 in distributed mode instead of an empty 200. They are served only from sessions the frontend holds, and in distributed mode it holds none. That gap predates the removal of the bus and is not closed by it; this only stops it being silent. Agent workers keep every other subject, including nodes.<id>.backend.stop. Their minted JWT loses the two MCP subjects and keeps a non-empty allow list, because NATS reads an empty one as no restriction at all. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-20 03:05:34 +00:00
Ettore Di Giacinto	e29a4e5d15	feat(cluster): reach every worker through its tunnel, never its address ... The tunnel, the fence, the registry and the relay were all built and none of them carried a byte: every dial from the frontend still went to the address a worker registered. This is where that stops. One WorkerDialer resolves where a worker's tunnel is held, opens a stream on it locally or relays through the owning replica, and hands back a conn past both handshakes; gRPC, the file stager's HTTP client and the log-streaming WebSocket are all pointed at it. A worker's address stops being somewhere to connect to and becomes the name of which backend process a stream is for. It still appears in URLs, logs and errors, because that is what identifies the process; what it no longer decides is where the bytes go. Nothing falls back to dialling it. BackendClientFactory now has exactly one method, NewClientForNode, and returns an error where there is no way to reach the worker. The direct-dial constructor was removed rather than kept beside it, because leaving one on the interface keeps the bypass one word away from every call site that holds an address, which is all of them. The second construction path is closed too. DistributedModelStore built remote models with a nil client, and pkg/model.Model.GRPC then dialled the raw address lazily on first use - reached in production by ShutdownModel's Free and by the backend monitor's Status. Those models now carry the tunnel-backed client, and a model that cannot be given one is logged and not listed. Four conditions stay unmixable, and one path produces absence: the dialer answers ErrNoConnection only where Owner's liveness join did. A peer that will not answer, a stale ownership row, a worker's own refusal and a missing relay path are each reported as themselves. This matters because nodes ACTS on absence, and the collapse would have it reclaim the models of a worker that is connected and busy. That is not hypothetical. Writing the mutation for it exposed the bug in this change's own first draft: probeHealth returned bare false when it could not build a client, and tryWarmPath deletes the replica row on a false probe. A frontend whose dialer broke would have emptied node_models for the whole deployment while every model kept running. probeHealth now returns alive and probed separately, the reconciler gets a ProbeUnknown outcome that neither advances nor clears a failure streak, and the health monitor skips rather than counting a miss. Task 5 left the relay's open timeout at a fixed 15s and said so: no operator has the information to set it, because the number that matters is the original client's remaining budget, which is invisible on the relay side. The dialer has that budget, so it now states it in the relay request frame and the owner takes the smaller of the two. It can only shorten - a patient client must not be able to park a relay goroutine and a stream slot on a worker that stopped accepting. Zero is written as no budget at all, since on the far side the number zero is a caller with nothing left and would refuse healthy traffic. Seven mutations, each reddening a named spec: peer-unreachable as absence; the local-failure guard dropped; max instead of min on the budget; the nil-client model restored; ProbeUnknown falling through to the reaper; OwnerRow instead of Owner; probed collapsed into alive. The first budget spec passed for the wrong reason - a handshake deadline, not the relay - and was replaced by three that each assert one link, including one where the spec plays the owning replica and reads the budget out of the frame instead of inferring it from a clock. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-20 03:05:34 +00:00
Ettore Di Giacinto	629381f3d2	feat(cluster): open the door a worker dials its tunnel through ... A worker needs no inbound port: it dials GET /api/cluster/connect, the connection becomes one multiplexed yamux session, and the frontend opens a stream on it per request. This adds the endpoint that accepts that dial and attaches it to the tunnel registry. The dial is authenticated against the NODE's own stored token hash rather than the deployment's registration token. That is the mechanism, not yet the isolation, since a worker still registers by presenting the shared token; what it rules out is the shortcut of comparing against the configured value, which would have to be unpicked the day workers get their own secrets. Every refusal happens BEFORE the WebSocket upgrade, so a dialer reads an HTTP status rather than a handshake error. The route is registered in every deployment, single-binary ones included, which is what puts it in front of the route-coverage test that holds that rule in place; with no node registry it refuses every dial, and tells a credentialed one the frontend has no cluster rather than that its token is wrong. A lookup that FAILED is answered as a failure. Reporting a database that could not be read as "unauthorized" would send a worker re-registering, throwing away the identity its tunnel and loaded models are keyed by. Wires the tunnel registry in core/application/distributed.go and hands it to the membership loop. Without that call the re-claim after a replica is reaped had no production caller and could never run. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Opus 5 [claude-code]	2026-09-20 03:05:34 +00:00
Ettore Di Giacinto	3b19575f81	feat(cluster): give phase 1 a call site, and prove it against real replicas ... Tasks 1 to 5 built an instances table, a splice, both halves of a peer link and an epoch fence, and nothing in the tree called any of it: no replica registered, no route was mounted, no sweeper ran. Proving phase 1 end to end therefore had to start by wiring it. A frontend in distributed mode now publishes the address its peers dial, heartbeats it, and sweeps replicas that stopped answering along with the connection rows they owned, in one pass so the two can never disagree about who is alive. It serves the peer link and owns the sessions peers dial in, refusing streams on them until phase 2 installs a relay: a session nobody accepts on does not fail a peer's Open, it hangs it. The address is the one peers use, not the one the process binds, and it is derived from the route to PostgreSQL. That derivation only holds while the database is remote, so LOCALAI_DISTRIBUTED_ADVERTISE_ADDR sets it explicitly and a replica that can determine neither warns and keeps serving rather than failing to start. Three e2e scenarios run against real local-ai processes, real PostgreSQL and real dials: replicas publish addresses that can actually be connected to; a sibling opens a stream over the peer link and is refused without the cluster token; and a killed replica is reported unreachable, never absent, loses the claim it held, and takes no worker with it. Each was verified by mutation: eight injected defects, each failing the scenario that claims to catch it. Also moves RegisterClusterRoutes to core/http/routes beside every other registrar, folds AutoMigrate and the epoch sequence into one cluster.Migrate, and turns the peer route's auth-coverage spec into a real assertion: it drives the request through the actual auth middleware instead of comparing two string constants, which the old spec would have passed even with the exemption deleted. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-20 03:05:34 +00:00
Richard Palethorpe	2facfc0d88	feat: Add kimodo.cpp and 3D animation API/UI (#12095) ... * fix(vulkan): preserve host ICD discovery for packaged backends Add bundled Mesa manifests through VK_ADD_DRIVER_FILES instead of replacing the system driver list. Merge inherited and model-specific additive paths while preserving explicit operator overrides, with regression coverage. Assisted-by: Codex:gpt-5 golangci-lint Assisted-by: Codex:gpt-5.6-sol Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(3d): add Kimodo CPU and Vulkan animation backend Introduce a distinct animation capability and model-described 3D operations, with a typed /3d/animate API, RPC transport, distributed media staging, permissions, and tracing. Add a persistent kimodo.cpp adapter, skeleton GLB export, CPU/Vulkan packages, model and backend galleries, importer support, CI builds, and documentation. Adapt Studio inputs to each model and provide real-time skeleton playback, seeking, and history. Cover backend validation, packaging, API behavior, importer inventories, distributed staging, and Studio workflows. Validate real-model CPU/Vulkan generation and deploy the integration to the local QA instance. Assisted-by: Codex:gpt-5 golangci-lint Assisted-by: Codex:gpt-5.6-sol Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(kimodocpp): adopt monolithic encoders and resident inference Update upstream for resident weights, packed execution paths, and cached motion graphs. Default to all 32 text layers while retaining configurable streaming and legacy bundle support. Use monolithic Q8_0 encoders by default and offer all six published quantizations through the gallery and importer. Refresh pinned hashes, tests, and documentation; remove the obsolete thread patch and ensure cached source checkouts follow the upstream pin. Validated CPU and Vulkan generation, lower-bit streaming, gallery/importer suites, packaging, lint, and cold/warm Studio generation on localai-dev. Assisted-by: Codex:gpt-5 golangci-lint Assisted-by: Codex:gpt-5.6-sol Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-09-18 06:12:03 +01:00
Ettore Di Giacinto	d55474a149	feat(audio): list available TTS voices ... Clients cannot discover the named voices that an installed TTS model accepts without consulting backend-specific documentation. Expose voice metadata through the audio API and let custom model configs declare their own catalog. Assisted-by: Codex:gpt-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-11 21:55:21 +00:00
mudler's LocalAI [bot]andEttore Di Giacinto	29899cd1e0	fix(ui): size model fit against the cluster and move node labels into the selector (#11765) ... * fix(ui): move node labels into the scheduling selector field The scheduling page kept a node-label browser open above the rules whether or not anyone was writing one, while the field that actually needs labels, the rule's node selector, was two bare text inputs with no hint of what the cluster reports. The browser is gone. The selector's key input now completes against the label keys the cluster uses, and the value input offers only the values that key takes. The roster already loads for the page, so the suggestions cost no request, and a roster that fails to load costs the admin the hints and nothing else. Suggestions stay suggestions: a key no node reports yet still commits as typed, which is how an admin writes a rule before labelling the nodes for it. Assisted-by: Claude:claude-opus-5 golangci-lint eslint playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): size model fit against the cluster, not the frontend The models page asked the frontend how much memory a model may occupy. In distributed mode the frontend is usually a GPU-less pod while every model runs on a worker, so a fleet of GPU nodes was told it could only run the smallest CPU build. The variant picker's fits flag and its auto-selection came from the same place, as did the hardware recommendations. The registry now reports the largest single healthy backend node. The largest node, not the fleet total: a model loads into one node, so four 16GB workers are not a home for a 40GB model. An operator-set VRAM budget caps a node's contribution, because the scheduler refuses a load above that ceiling anyway, and a GPU node beats a CPU node holding more system RAM. GET /api/resources and GET /api/models carry this as an additional cluster object. Their aggregate and ram fields keep reporting the frontend's own hardware, which is what the resource monitor shows. Variant selection judges backends against the union of the capabilities present in the cluster, the way backend discovery already did. Every path degrades to the local host: no cluster object in single-node mode, and none when the registry cannot be read, so a hiccup narrows the answer back to single-node behaviour rather than marking the whole catalog too large. The verdicts now name the node they belong to, since a model fits somewhere or nowhere. Assisted-by: Claude:claude-opus-5 golangci-lint eslint playwright Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-08-28 22:57:35 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	82c191afad	fix(distributed): keep model replicas config-consistent (#11664) ... * docs: design configurable copy buffering Document the context-aware copy buffer option and its validation plan. Assisted-by: Codex:gpt-5 * docs: design durable distributed staging operations Assisted-by: Codex:gpt-5 * docs: design distributed model config revisions Assisted-by: Codex:GPT-5 [apply_patch] [exec_command] * feat(config): add stable model revisions Hash typed model configuration and effective protobuf options deterministically for distributed revision comparisons. Assisted-by: Codex:GPT-5 [apply_patch] [exec_command] * feat(worker): acknowledge exact model stops Assisted-by: Codex:GPT-5 [apply_patch] [exec_command] * feat(nodes): track model config revisions Assisted-by: Codex:GPT-5 [apply_patch] * fix(distributed): retry quarantined model cleanup Stop quarantined replicas by exact process identity, retain failed cleanup as durable capped retries, and compare-and-delete only the claimed registry row. Process one sufficiently leased row at a time so multiple frontends cannot duplicate slow cleanup work. Assisted-by: Codex:gpt-5 * fix(distributed): bind loads to config revisions Assisted-by: Codex: GPT-5 [OpenAI Codex] * fix(modeladmin): apply config revisions consistently Route model edits, patches, state changes, deletion, and peer refreshes through the same revision lifecycle. Quarantine stale replicas before exact cleanup and report durable pending cleanup without failing successful config writes. Assisted-by: Codex: GPT-5 [OpenAI Codex] * feat(distributed): expose model config revision state Document replica revision observability and durable cleanup behavior. Keep pending cleanup explicit in model mutation responses and verify endpoint contracts expose revision state without serialized load options. Assisted-by: Codex:GPT-5 [OpenAI Codex] * test(distributed): cover model revision convergence Exercise cross-frontend quarantine, stale replay rejection, exact cleanup retry, worker re-registration, and current-generation replica convergence against the distributed PostgreSQL harness. Assisted-by: Codex:gpt-5 * fix(distributed): pass config revision CI checks Keep configured gallery sources out of authoritative runtime snapshots only after validating their real schema, and harden rollback snapshots against symlink races and non-regular files. Assisted-by: Codex: GPT-5 [OpenAI Codex] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-08-22 22:44:03 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	a0252ad6a1	fix(distributed): keep staging operations stable (#11663) ... * docs: design configurable copy buffering Document the context-aware copy buffer option and its validation plan. Assisted-by: Codex:gpt-5 * docs: design durable distributed staging operations Assisted-by: Codex:gpt-5 * fix(distributed): merge durable staging operations Use active model load jobs as the durable operations baseline and overlay replica-local staging progress without duplication. Preserve tracker-only operations when the registry cannot be read. Assisted-by: Codex:gpt-5 --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-08-21 20:08:47 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	0ab632b6bd	fix(auth): protect HTTP routes by default (#11602) ... * fix(auth): default to protected HTTP routes Use a method-aware registry for the small anonymous bootstrap surface. Unknown routes now require credentials instead of inheriting fail-open path classification. Keep node self-service routes behind their registration-token middleware. Global auth no longer rejects valid worker credentials first. Assisted-by: Codex:gpt-5 * docs(auth): document public HTTP surface Assisted-by: Codex:gpt-5 * test(auth): align route coverage with default denial Assisted-by: Codex:gpt-5 --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-08-19 20:49:19 +02:00
localai-org-maint-botandlocalai-org-maint-bot	0761bd02c7	feat(chat): add end-to-end context compression (#11556) ... * feat(config): add context compression policy Define the opt-in model configuration contract before the chat middleware consumes it. Document each policy field so later request handling does not invent a second schema.\n\nRefs #9534\n\nAssisted-by: Codex:gpt-5 * fix(config): register compression fields The model editor metadata gate rejects new config fields without descriptions and suitable controls. Register the compression policy so operators can edit its six fields safely. Assisted-by: Codex:gpt-5 [monitoring-prs] * feat(chat): compress long contexts Long conversations currently fail once they reach the model context window. The opt-in policy now summarizes complete older turns before primary inference and preserves the newest tool chains. Both OpenAI and MCP chat routes share the same transformation. Usage metadata and metrics expose each compression event. Refs #9534 Assisted-by: Codex:gpt-5 --------- Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-08-18 11:31:03 +00:00
Richard Palethorpe	d10374f849	feat(router): make KNN a first-class classifier with a persisted, curated corpus (#10652) ... * feat(router): make KNN a first-class classifier with a persisted, curated corpus Add `classifier: knn` — similarity-weighted voting over labelled example prompts. Unlike score/colbert it needs no classifier model: label knowledge lives in a corpus seeded and curated through the admin API, so routing decisions are deterministic, auditable, and grounded in graded experience rather than a model's opinion. Epistemic gate: corpus entries below knn.similarity_threshold cannot vote; when none clears it the classifier activates no labels and the router uses the fallback — a prompt unlike all labelled experience is treated as undecidable, not guessed. Decisions record nearest_similarity (also on fallback rows) so admins can see how far the nearest labelled experience was; the Routing tab explains out-of-corpus fallbacks and shows per-label corpus counts. Persistence: one JSONL file per router under <data path>/router-corpus (text, labels, vector, embedder fingerprint). The file is the source of truth; the local-store index is rebuilt from it at classifier build time and stays a pure in-memory index. Entries recorded under a different embedding model re-embed on load. Also corrects the docs' false claim that local-store collections persist — the embedding cache never survived restarts (and still doesn't); the corpus does. Corpus input is API-only by design (entries may contain example user content): POST /api/router/{name}/corpus seeds (labels validated against declared policies, embedded server-side, indexed immediately), GET .../corpus/stats inspects — label counts only, entry texts are never returned by any surface — DELETE .../corpus wipes. Admin-gated like the sibling router endpoints, and exposed as MCP tools (seed_router_corpus / get_router_corpus_stats / clear_router_corpus) in both the httpapi and inproc clients with coverage-test route mappings. Plumbing: VectorStore gains SearchK (top-K was hardcoded to 1); local-store gets InsertBatch/Delete as optional fast paths; RouterConfig gains a knn block (embedding_model, k, similarity_threshold, vote_threshold, store_name) with meta-registry fields; the classifier dropdown now offers knn and the previously-missing colbert; embedding_cache is ignored (with a warning) for knn — it IS an embedding-KNN lookup; the stale /api/instructions intelligent-routing entry is rewritten (it described a classifier that no longer exists); swagger regenerated. Tests: KNN vote/gate specs with hand-computed vote shares, corpus manager suite (restart reload without re-embedding, fingerprint re-embed, dedupe, hostile store names), middleware specs (corpus routing, gate fallback, config validation, cache-wrap refusal), corpus endpoint specs pinning the texts-never-returned contract, MCP catalog + route-mapping gates, and a Playwright spec for corpus stats and the out-of-corpus decision detail. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(router): name consulted corpus neighbours in knn decisions Every knn decision (decision log rows and the /api/router/decide response) now carries neighbors: the K retrieved corpus entries by descending similarity - including ones below the epistemic gate, which is what makes fallback decisions diagnosable - each as {id, similarity, labels}. The id is the entry's content hash (first 8 bytes of the SHA-256 of its text, hex): stable across reseeds and re-embeds, and text-free, so an external platform that seeded the corpus can recompute text->id on its own copy and bucket decisions by corpus region (per- region reliability accounting) without corpus text ever leaving the server. A corrupt index payload surfaces as an id-less neighbour at a real similarity instead of disappearing. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * refactor(router): deduplicate knn plumbing and cut corpus hot-path waste Post-review cleanup of the knn-first-class-router branch; no behaviour changes on the API surface. Reuse/altitude: - RouterKNNConfig.ResolvedStoreName is now the single source of the router-corpus-<name> default (was hand-derived in four files). - corpus.ResolveKNNRouter + corpus.Seed carry the shared model resolution and seed validation; the REST endpoints and the assistant MCP client are thin transport adapters over them, with sentinel errors mapped to HTTP statuses at the echo boundary. - middleware.NewClassifierDeps assembles the classifier dependency set once for all five entry points (OpenAI, Anthropic, realtime, decide, corpus) instead of five hand-copied literals. - router.AllClassifiers feeds both the status endpoint and the unknown-classifier error, ending the classifier-list drift. - Per-classifier requirements moved out of validateRouterPolicies into their buildClassifier arms; the knn arm owns its embedding_cache opt-out instead of a name-check in the shared wrap tail. - adminOnly replaces four inline copies of the admin gate in the middleware routes. - localVectorStore.Search delegates to SearchK (identical traces). Efficiency: - Manager.Add embeds outside the manager mutex and appends to the JSONL file (O(new) instead of O(corpus) rewrite); a torn tail from a crash mid-append is tolerated on read and repaired on next write. - Stats memoises per store keyed on the file's stat fingerprint and no longer takes the manager mutex, so the 5s status poll stops parsing vector-laden JSONL and stops blocking behind seeds. - KNN Classify decodes each neighbour payload once (was twice) and builds refs and votes in a single pass with one fallback return. - Corpus file writes fsync before rename/close. - The corpus manager is built eagerly in newApplication (sync.Once dropped); test helper dead branch removed. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(router): bind knn corpus vectors to an embedder fingerprint and fail closed on mismatch Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * chore(mcp): align corpus tool prompts and the mutating-tool safety list Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(proto,backend): report embedding shape from the llama-cpp backend Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(embeddings): Go-side pooling — mean/last/decayed_mean with half-life Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(embeddings): accept chat messages[] and per-request pooling on /v1/embeddings Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * chore(middleware): name the failing fields when post-merge validation 400s An intermittent post-merge validation failure surfaced as an opaque 400 during integration (pooling scheme mismatch that no client had sent). Log the model, the request's pooling override, and the merged config's pooling fields at the failure point so the next occurrence identifies whether the request or the stored config carried the bad value. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(embeddings): scheme override must not inherit the config's half-life A model config defaulting to decayed_mean pooling carries pooling_half_life_tokens; a request overriding the scheme to mean/last without its own half-life inherited that value, and post-merge validation rejected the pair the server itself had assembled. Zero the inherited half-life when the overridden scheme is not decayed_mean; a request that explicitly pairs a half-life with a non-decayed scheme still 400s. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix embedding pooling validation and router bounds Declare backend embedding layouts and reject incompatible pooling modes. Reset local-store dimensions after a full clear, validate KNN thresholds, and add real backend and store integration coverage. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * ci: run local-store integration tests Build and install the local-store backend in the Linux test job, then run the existing store integration suite so new specs are discovered automatically. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-08-18 09:37:43 +02:00
3 people	88edd7fc7f	fix(distributed): run cold model loads as durable jobs instead of holding the advisory lock (#11514) ... * fix(advisorylock): set statement_timeout alongside lock_timeout WithLockCtx already overrides a deployment-wide lock_timeout on its dedicated connection so a blocking pg_advisory_lock() waits its turn instead of failing with 55P03. statement_timeout aborts that exact same statement independently, with SQLSTATE 57014, and was not overridden. Production roles commonly carry statement_timeout=60s. Any guarded section longer than that (a cold model load stages for tens of minutes) therefore killed every concurrent waiter: advisorylock: acquiring lock 9003261067483446873: ERROR: canceling statement due to statement timeout (SQLSTATE 57014) Derive it from the same context budget as lock_timeout, with a matching RESET so the pooled connection is returned clean. Assisted-by: Claude Opus 5 [claude-code] * feat(distributed): add ModelLoadJob, the durable cold-load record A cold load in distributed mode is a long-running background job, but it was modelled as a synchronous side effect of an inference request: the whole of it (backend install, multi-GB staging, checkpoint load) ran inside the per-model advisory lock. Loading a 35.7 GB GGUF held that lock for ~20 minutes, so every concurrent request for the same model blocked on pg_advisory_lock and died at the role's 60s statement_timeout. Introduce the row that lets the lock shrink to a decision. Exactly one ModelLoadJob may be active per tracking key; that uniqueness — not the lifetime of a lock — is what de-duplicates concurrent loaders across replicas. ClaimLoadJob does its read-then-write under the advisory lock and nothing else: no network, file or gRPC I/O inside the guarded section, so a claim costs milliseconds no matter how long the resulting load takes. LastProgress is a heartbeat rather than a byte counter. A checkpoint load legitimately moves zero bytes for many minutes, so a reaper keyed on byte movement would reclaim a healthy job mid-load; byte progress stays the concern of load_deadline.go. A job whose heartbeat stops for longer than the orphan window is reclaimable, so a replica killed mid-load cannot wedge a model permanently. Failed jobs keep their row for a short grace so an immediately-following request reports the real cause instead of silently starting a fresh load of a model that just failed. No caller yet — the router moves onto this in the next commit. Assisted-by: Claude Opus 5 [claude-code] * refactor(distributed): run cold loads as jobs, outside the advisory lock Route wrapped the entire cold load — node selection, backend install, multi-GB staging and the remote LoadModel — in the per-model advisory lock. The lock's job is to de-duplicate concurrent loaders, a decision that takes milliseconds; holding it for the tens of minutes the resulting work takes is what turned a dedup mechanism into a cluster-wide outage for that model. Split it into a claim and a run. The claim is the only thing left inside the lock. The run is a background job owned by the claiming replica and bounded by the same progress-extended deadline as before; every other request for that model — local or on another replica — attaches as a waiter and is served the moment the model is ready, with no duplicate load and no lock contention. Waiters share one broadcast rather than an ordered queue: they all want the identical outcome, so ordering them would add fairness machinery that changes no result. The local channel wakes same-replica waiters instantly and a 2s DB poll is the authority, because a waiter on another replica has no channel to close. On wake a waiter re-runs the warm path rather than trusting the signal — the model may have been evicted in between. A waiter whose client disconnects returns immediately and the job keeps running; it belongs to the job record, not to the request. A failure is recorded on the row so every waiter reports the real cause, and the row survives briefly so the next request does not read "no job" as "not loading" and start a duplicate load of a model that just failed. The runner heartbeats the row on a fixed interval whether or not bytes are moving, which is what keeps a legitimately silent checkpoint load from being reclaimed as an orphan. Phase (installing/staging/loading) and placement ride to the heartbeat on the context, the same seam load_deadline.go already uses, so single-host paths are untouched. Non-distributed mode (no DB) keeps the inline load exactly as it was. Assisted-by: Claude Opus 5 [claude-code] * feat(distributed): bound the wait for a loading model and answer with progress A request whose model is cold-loading now attaches to the running job and is served the moment the model is ready. That wait has to be bounded: a held HTTP request cannot survive real infrastructure, and an ingress or LB idle timeout kills a twenty-minute request regardless of what LocalAI does. New LOCALAI_MODEL_LOAD_WAIT (default 60s) bounds the CALLER, never the load — the job keeps running either way. On expiry the request gets 503 with Retry-After and a structured body naming the model, the node, the phase, byte progress and an ETA. The `error` envelope keeps OpenAI clients working; `loading` is additive so they ignore it. The ETA comes from the job's own observed rate and is omitted rather than guessed until enough bytes have moved for that rate to mean anything: a confidently wrong ETA on a twenty-minute wait is worse than none. Retry-After is that ETA when known, clamped to [5s, 300s], and the wait budget otherwise. LOCALAI_MODEL_LOAD_WAIT=0 waits unbounded, for deployments with no proxy in front. Zero in the config struct still means "unset, use the default", so the CLI records the operator's zero as ModelLoadWaitUnbounded rather than losing the distinction. The distributed branch of ModelLoader.loadModel wrapped the router's error with %s, which flattened it to a string. Use %w: the typed error is what the HTTP layer keys the 503 off. Assisted-by: Claude Opus 5 [claude-code] * feat(api): add GET /api/models/{id}/load-status A client that receives 503 while a model stages onto a worker needs somewhere to poll. This returns the same `loading` object the 503 carries — phase, node, byte progress and ETA — or 404 when no load is running. Read-only and observability-shaped, so it is deliberately neither admin-gated nor feature-gated: it explains a 503 the caller just received, and hiding that behind a per-modality feature would make the explanation for a failed image request depend on chat permissions. It also gets no MCP tool, since there is nothing here an admin would manage conversationally. Registered on the surfaces from .agents/api-endpoints-and-auth.md: the swagger block (existing `models` tag, so /api/instructions needs no new area), the endpoint discovery maps in RegisterLocalAIRoutes, regenerated swagger, and the distributed-mode docs page. No FLAG_* usecase is involved, so capabilities.js is unchanged. Assisted-by: Claude Opus 5 [claude-code] * feat(ui): show cold-load progress in Chat and retry when the model is ready A chat request for a model that is still staging onto a worker now gets a 503 carrying live progress instead of an error. Render it: the composer shows the phase (installing / staging / loading), the node, the percent and the ETA, then polls load-status and re-sends the request the moment the model is ready. Reuses the staging progress idiom the page already had rather than inventing a second one — the two sources are folded into one loadProgress, with the load job winning because it is authoritative across frontend replicas and knows the phase, where the staging operation only knows about a byte transfer this replica happens to be performing. Waiting is bounded (three send attempts, ~30 min of polling each), so a load that never finishes still surfaces as an error rather than as a spinner nobody questions. An aborted generation stops the polling too. Assisted-by: Claude Opus 5 [claude-code] * fix(distributed): check warm-path cleanup errors The router moved legacy cleanup calls onto newly linted lines. Report cleanup failures while preserving the fallback to a cold load. Assisted-by: Codex:gpt-5 [golangci-lint] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-08-15 13:20:11 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	7b9167eaad	feat(llama-cpp): serve Qwen3-TTS through the llama.cpp backend (#11392) ... * fix(config): do not read a TTS speaker-encoder mmproj as vision support Qwen3-TTS on llama-cpp ships an mmproj holding the speaker encoder and code predictor. VisionSupported() treated any non-empty MMProj as proof of image input, so every such model would be advertised as vision-capable. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): add TTS request option parsing helper Validates text and speaker reference presence and strictly parses the top_k / top_p per-request params, in a header with no llama.cpp or gRPC dependencies so the standalone C++ unit test gate picks it up. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): range-check the TTS top_k and top_p request params Format validation alone let NaN, infinity and out-of-range values through. The consumer copies both values into the audio generation input unconditionally and only guards its separate sampler assignment with "> 0", a test NaN also fails, so a NaN reached llama.cpp with the guard never firing. top_k must now be >= 0 and top_p must fall within 0.0 to 1.0 inclusive, with the bound written as a negated in-range test so NaN is rejected rather than silently accepted. Also cover the two checks the suite could not previously kill: the whole-string check in the float parser and the int32 range check. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(llama-cpp): bump pin to f9e832c10 and carry the TTS server task Picks up ggml-org/llama.cpp#26254 (Qwen3-TTS via mtmd) and #26536 (the short-input audio chunk fix). Adds 0002-add-server-task-type-tts.patch, the server-side half of the still-draft #26603, so TTS runs through the slot scheduler instead of racing it. Remove that patch when #26603 merges. The patch is rebased on top of the score patch: its tokenize-switch hunk collided with the SERVER_TASK_TYPE_SCORE case, and its lone SRV_WRN call passes no variadic argument, which the macro cannot expand. The score patch itself needed no refresh. Also fixes fallout from the bump in grpc-server.cpp: upstream dropped the per-slot n_ctx argument from server_schema::eval_llama_cmpl_schema. Only the schema branch loses it, since forks predating the server-schema split still expect the old argument list. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): implement the TTS and TTSStream RPCs Both were declared in backend.proto but unimplemented. They now submit a SERVER_TASK_TYPE_TTS task and drain the response reader, the same shape PredictStream uses. The streaming path emits a leading sample_rate message and then raw PCM, because ModelTTSStream builds the WAV header itself; the non-streaming path emits a complete WAV to the requested dst. The streamed samples are converted from the pipeline's float32 to signed 16-bit first. MTMD_HELPER_GEN_AUDIO_OUTTYPE_PCM hands back floats, while the header ModelTTSStream writes announces 16-bit samples, so shipping the floats verbatim would decode as noise. prepare.sh and CMakeLists.txt now stage tts_request_options.h alongside the other grpc-server helpers, and register its standalone test with ctest the way passthrough_options_test is registered. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): mask non-codec tokens for Qwen3-TTS generation The Qwen3-TTS gen-audio pipeline maps a sampled backbone token to a codebook row with an unchecked subtraction, in mtmd-helper-gen.cpp: inp.code0 = sampled - codec_0; For ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF the vocab is 155008 tokens, <|codec_0|> is 151936 and the codec codes end at 153983. The model's own tokenizer.ggml.suppress_tokens holds 1023 ids covering 153984..155007, every special above the codec range except <|codec_eos_token|> (154086) which stays reachable as the stop token. Nothing masks the text range 0..151935, so the backbone can sample a text token at any step, the subtraction goes negative, and ggml_compute_forward_get_rows aborts the whole backend process on GGML_ASSERT(i01 >= 0 && i01 < ne01). Complete the mask upstream started: bias every token below <|codec_0|> to -INFINITY for TTS tasks so only codec codes and the codec EOS remain reachable. The biases are appended to task.params.sampling.logit_bias, which common_sampler_init already merges with the model's suppress tokens into one llama_sampler_init_logit_bias, so no sampler is added to the chain. Measured cost is 0.082 ms per sampled token and 1.16 MB, set against a forward pass in the multi-millisecond range. It lands in launch_slot_with_task rather than in a route handler so that llama.cpp's own POST /tts and LocalAI's TTS/TTSStream RPCs are both covered, and <|codec_0|> is resolved from the vocab rather than hardcoded so a model without it is left alone. This is reproducible with upstream's own llama-tts and no LocalAI code loaded, aborting at frame 55 on Q4_K_M and frame 71 on Q8_0, so it is neither a quantization artifact nor an artifact of the gRPC adapter. Two further defects in the same draft pipeline still prevent end-to-end audio; they are independent of this one and are recorded in the task report for an upstream bug report. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(llama-cpp): bump pin to 9de0fcf2b and drop the TTS codec mask Upstream fixed the Qwen3-TTS abort in ggml-org/llama.cpp c8e03ce81 ("mtmd/ggml: add ggml_build_forward_order", #26649), landed one hour after the previous pin. ggml_build_forward_expand marks a tensor and all its ancestors for compute, so using it as a pure ordering hint defeated ggml_build_forward_select and made GEN_WAV calls execute the GEN_CODE branch against a stale inp_code0, hitting the get_rows bound assert in ggml_compute_forward_get_rows. That single defect accounts for every abort seen on this model, so 0003-mask-non-codec-tokens-for-tts.patch is removed rather than rebased. The mask changed the observed behavior, but it was perturbing a graph ordering bug rather than fixing a sampling one: at the new pin the whole path works without it. Keeping it would have meant carrying a 152k-entry logit bias, and rebasing it on every pin bump, for no benefit. Verified at 9de0fcf2b with only 0001 and 0002 applied, which both apply clean with no fuzz and needed no rebase: non-streaming HTTP 200, 410924 bytes, 8.56 s RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, mono 24000 Hz streaming HTTP 200, 560684 bytes, 11.68 s, exactly one RIFF at byte 0, same format, which also exercises the float32-to-s16 conversion at runtime for the first time Pristine unpatched llama-tts at the same pin now also completes, 130 frames to a valid WAV, where it aborted at frame 55 before. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): clear the TTS slot sequence between requests Only the first TTS request in a backend process succeeded. Every later one failed instantly, in about 0.13 s, with "TTS prompt processing failed" from step_prompt, regardless of streaming or non-streaming and regardless of the text. With LOCALAI_SINGLE_ACTIVE_BACKEND=true the process is kept alive between requests, so a deployment would have served exactly one utterance per backend start. The cause is missing KV hygiene, not anything in the gRPC adapter. TTS slots never enter the shared batch: pre_decode() returns early for them and process_tts_slots() drives them instead, so they skip the prompt-cache bookkeeping that clears a slot's sequence between requests. Nothing in the gen-audio path makes up for it: mtmd_helper_gen_audio_reset only clears host-side buffers, and the pipeline always decodes from position 0 into the sequence identified by slot.id. So the second task on a slot writes positions 0..N over the first task's tokens and llama_decode fails. Fix is one call to slot.prompt_clear(), the same helper the normal path uses, in the SERVER_TASK_TYPE_TTS branch of launch_slot_with_task before set_input. It goes into 0002 rather than a new patch file because it is a defect in the code that patch introduces, and the header now records it as ours so we know whether it still needs carrying if #26603 merges without it. Verified in one backend process, different text on every request: three consecutive non-streaming requests, three consecutive streaming requests, and an interleaved non-streaming, streaming, non-streaming, streaming run. All ten returned HTTP 200 with RIFF ... WAVE audio, Microsoft PCM, 16 bit, mono 24000 Hz, the streamed ones carrying exactly one RIFF header at byte 0, and every output measured as real speech rather than silence or a truncated fragment. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): expose max_frames for TTS requests The Qwen3-TTS backbone does not always emit <|codec_eos_token|>, and when it does not, generation runs to upstream's 512-frame n_predict default. At the model's 12.5 Hz frame rate that is 40.96 s of audio, which a short input can trigger: one request in this session produced 40.96 s for a ten-word sentence. prepareTTSTask hardcoded n_predict to -1, so callers had no way to bound it. Add a max_frames key alongside top_k and top_p, parsed with the same strict whole-string parsing so a typo is an error rather than a silently truncated value, and rejected with a field-naming message when negative. 0 keeps the existing sentinel convention and means unset, so a request that omits it behaves exactly as before. Named max_frames rather than n_predict because frames are what the parameter means at a TTS endpoint: one frame is 0.08 s of audio. The 512-frame default is deliberately unchanged. Lowering it would truncate legitimately long inputs, which is a worse failure than an occasionally overlong one. Verified end to end on one text of thirty words: max_frames=25 HTTP 200, 96044 bytes, 2.00 s, exactly 25 frames max_frames=50 HTTP 200, 192044 bytes, 4.00 s, exactly 50 frames no max_frames HTTP 200, 572204 bytes, 11.92 s, stopped at its own codec EOS after 149 frames, unchanged behavior max_frames=-1 InvalidArgument "max_frames must be >= 0, got \"-1\"" max_frames=many InvalidArgument "max_frames must be an integer, got \"many\"" Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): send the TTS sample rate up front, and tidy three review items Four items from the Task 4 review. Streaming first-byte latency. TTSStream sent the sample-rate reply only once the first audio result arrived, and a chunk needs a whole 72-frame window, roughly 5.8 s of audio and far longer in wall time on CPU. The Go side blocks on that reply before it can emit the WAV header, so a streaming client sat at zero bytes for the whole stretch. The rate is a property of the loaded model and is available synchronously from mtmd_gen_audio_get_info, so it now goes out immediately after post_task and the rate_sent bookkeeping is gone. Measured on a warm model, first byte drops from 30.48 s to 0.014 s, and the output is still a valid WAV with exactly one RIFF header at byte 0. Unchecked close. The non-streaming path ignored ofstream::close(), so a failure that only surfaces on flush was reported as success while leaving a truncated file at dst. It now returns INTERNAL like the other write failures. Wrong comment on set_lang. gen_audio::inp::get() already maps a stored blank to nullptr, so our guard is behavior-preserving, not behavior-fixing. The comment claimed otherwise; the code was right. Repetition penalty. penalty_last_n = -1 is inert at this pin, because llama_sampler_init_penalties clamps it with std::max(penalty_last_n, 0) and then builds a disabled sampler, so the 1.05 penalty never applies. Upstream's README attributes looping to a missing repeat_penalty, so it was worth testing as a root-cause fix for the model running to the frame cap. Dropping the line lets the sampling default of 64 apply, which was confirmed in the sampler chain trace as penalty_last_n = 64 with repeat_penalty = 1.050. Over 15 uncapped short requests each way it did not help: 0 of 15 ran to the cap with the penalty inert, 1 of 15 with it active. Both lines are therefore kept for parity with upstream's draft, and a comment now records that the pair is inert and why, so the next reader does not believe a penalty is applied. max_frames remains the way to bound output. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * build(llama-cpp): let unpatched forks opt out of the TTS task turboquant and bonsai copy grpc-server.cpp into llama.cpp forks that do not carry our patches. disable-tts-task.sh injects the same kind of preprocessor switch disable-score-task.sh already uses, so those builds answer UNIMPLEMENTED rather than failing to compile. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(config): keep a TTS speaker-encoder projector out of vision detection Task 1 exempted a declared-TTS model's mmproj from VisionSupported, but the first real gallery entry with an mmproj still came back vision-capable through two paths the earlier fix did not close. GuessUsecases has no FLAG_VISION branch, so it falls through to true for any chat-ish model. That is not just a wrong answer at the call site: syncKnownUsecasesFromString rewrites KnownUsecaseStrings from HasUsecases, and the loader calls it more than once per config file, so the guessed FLAG_VISION is written out and parsed back into KnownUsecases as if the operator had declared it. Give GuessUsecases a FLAG_VISION branch that defers to the same explicit signals VisionSupported uses. Second, llama.cpp builds an mtmd context for the speaker-encoder projector and reports its media marker on the first chat probe, which resurrected vision after the model had been used once. Apply the same declared-TTS exemption to MediaMarker that the mmproj check already had. Verified against the qwen3-tts-llamacpp-q4 gallery entry: no vision capability and no image input modality, before load, after a TTS request, and after a chat probe. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add Qwen3-TTS entries for the llama-cpp backend Two entries over upstream's own GGUF conversion, Q8_0 and Q4_K_M, each pairing a backbone with the Q8_0 projector. Named to sit alongside the existing qwen3-tts-cpp entries rather than replace them. Also tags the llama-cpp backend text-to-speech / TTS so the backend browser surfaces the capability. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: cover Qwen3-TTS on the llama-cpp backend Adds the gallery variants, the two-file mmproj configuration, the required voice reference, and the language and sampling knobs. Also corrects the streaming-support list, which named only voxcpm. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(config): register llama-cpp as a TTS and voice-cloning backend The branch taught the llama-cpp backend to serve Qwen3-TTS and shipped two gallery entries for it, but never told the capability table. llama-cpp still declared only the text RPCs and usecases, so: - VoiceCloningForModel returned nil at the capability check, before it ever reached the model's own tts.voice_cloning override, and /tts answered 400 "selected model does not support reference-audio voice cloning" for any localai://voice-profiles/... voice. No model YAML could opt back in. - GET /api/backends/usecases did not list tts for llama-cpp, so the gallery greyed out the TTS filter for the entries this branch adds. - The React TTS page saw voice_cloning: null and kept both models out of the Voice Library. Add the TTS RPCs and usecase, and the reference-audio contract. The contract needs narrowing, because the per-backend switch in VoiceCloningForModel ends in a permissive default: an unnarrowed entry would have advertised reference-audio cloning on every GGUF chat model in the gallery. Narrow on the declared TTS usecase rather than the model name. The TTS checkpoints are the only llama-cpp models carrying known_usecases: [tts]; name matching would have to guess at third-party repacks, and "base", the substring the neighbouring Qwen and vLLM cases key on, is a routine word in text-model names. The check reads the declared bit directly instead of going through HasUsecases, which falls through to GuessUsecases and would hand the decision to a heuristic that never had a llama.cpp TTS model in mind. DefaultUsecases stays [chat]: a bare GGUF served by llama.cpp is a chat model, and both the gallery filter and the importer read that field. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): declare what nemotron-3-nano-omni actually accepts The entry is backend: vllm-omni with known_usecases: [chat, completion], no mmproj and no media marker, so it used to report vision only through the blanket GuessUsecases fallthrough that the vision branch in this branch removed. Nemotron 3 Nano Omni is a multimodal understanding model: image, video and audio in, text out. Declaring that is what the sibling vllm-omni-qwen3-omni-30b already does. known_usecases gains vision only. FLAG_VIDEO is video GENERATION, an output modality, and this model generates none; video and audio input belong in known_input_modalities, which is where AudioInputSupported and VideoInputSupported read them from. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(importers): import a Qwen3-TTS GGUF repo as TTS, not chat The llama-cpp importer hardcodes known_usecases: [chat] and assigns any mmproj-matching file as a vision projector, so ggml-org/Qwen3-TTS-12Hz-1.7B- Base-GGUF imported as a chat model with vision. Both fields were wrong, and the model was unreachable from /tts and from the Voice Library. Filenames cannot fix this. A Qwen3-TTS repo has the exact shape of a vision repo, one backbone GGUF plus one mmproj-*.gguf, so the projector's own header is the only honest signal: mtmd writes clip.has_gen_audio_encoder for the projectors it can drive as a speech pipeline and refuses to build one without it. Probe the selected mmproj for that flag, reusing the range-fetch the MTP detection already does, and declare tts when it is set. The mmproj assignment then stops reading as vision on its own, since a declared-TTS model already exempts its projector from vision detection. The probe is best-effort like the MTP one: a network blip leaves the chat default in place rather than failing the import. Verified against the real artifacts on disk: the Qwen3-TTS projector reports gen-audio, its backbone does not. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): stop non-TTS models crashing on the new pin Two regressions, both hit every ordinary llama-cpp model and neither was caught locally because every test on this branch loaded a TTS model. The first is a null dereference. server_slot::tts_ctx::reset() called mtmd_helper_gen_audio_reset() unconditionally, but the gen-audio pipeline is only allocated for models carrying a gen-audio mmproj, and upstream's implementation reads ctx->pipeline before null-checking anything. Since server_slot::reset() runs during slot initialization for every model, any non-TTS model segfaulted the backend the moment it loaded. Guard the call on the is_supported() predicate already defined beside it, and keep the plain field resets unconditional. The second is unrelated to TTS and came in with the pin bump. PredictOptions.Penalty is a bare proto float, so a caller that names no repetition penalty sends 0 rather than omitting the field. Since 9de0fcf2b, common_sampler_init() rejects a non-positive penalty_repeat outright because it would divide logits by zero, turning every such request into "Failed to initialize samplers". Treat 0 as unset and leave llama.cpp's own neutral default in place. Verified with the same suite CI runs, which is what caught both: tests/e2e-backends passes 6 of 6 including the load and predict specs that were red. Qwen3-TTS still synthesises on both paths, 24 kHz mono 16-bit WAV with exactly one RIFF header on the streamed output. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-08-10 10:18:47 +02:00
localai-org-maint-botandlocalai-org-maint-bot	133c546c3f	feat(api): add text moderation endpoint (#11316) ... * feat(api): add text moderation endpoint Add an OpenAI-compatible /v1/moderations endpoint backed by constrained local text generation. Register its auth and discovery surfaces, document the text-only MVP, and cover response shaping and access control. Assisted-by: Codex:gpt-5 * test(mcp): update assistant client stub Keep the LocalAI Assistant holder test stub aligned with the scheduling methods added to LocalAIClient so repository-wide type checking succeeds.\n\nAssisted-by: Codex:gpt-5 --------- Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-08-03 18:03:46 +02:00
Peteandlocalai-org-maint-bot	8a68f3571c	feat(api): add POST /v1/images/upscale endpoint (#10227) ... * feat(api): add POST /v1/images/upscale endpoint Add a new image upscaling endpoint that accepts a source image and returns an upscaled version. Supports selectable upscaler models (e.g. realesrgan) and a configurable scale factor (2x or 4x). - backend.proto: add UpscaleImage RPC and UpscaleImageRequest message - pkg/grpc: implement UpscaleImage in Backend interface, client, server and embed shim - core/backend/upscale.go: new backend helper (mirrors ImageGeneration) - core/http/endpoints/openai/upscale.go: new multipart/form-data handler - core/http/routes/openai.go: register POST /v1/images/upscale - core/http/auth/features.go: gate upscale routes under FeatureImages - backend/python/diffusers/backend.py: implement UpscaleImage — uses diffusers upscale pipeline when loaded, falls back to Lanczos resize * fix(grpc): add UpscaleImage stub to Base backend All Go backends embedding Base now satisfy the AIModel interface without needing to implement UpscaleImage explicitly. * fix(images): complete upscale endpoint integration Store generated upscales under the served images directory, validate scale factors, document and advertise the endpoint, and add a functional Stable Diffusion x4 gallery model. Assisted-by: Codex:gpt-5 --------- Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-08-03 15:27:22 +02:00
localai-org-maint-botandlocalai-org-maint-bot	fd4ec083b9	feat(downloads): add resume-safe pause action (#11222) ... Give gallery operations distinct pause and cancel paths. Pause preserves partial download data so reinstalling the same model or backend resumes through HTTP Range, while cancel keeps its destructive semantics. Surface the action in the Activity UI and document the API behavior. Assisted-by: Codex:gpt-5 Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-08-03 15:25:23 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	5cb0c1a872	feat(ui): close the gap between the shipped UI and the design mocks (#11307) ... * feat(ui): give the Operate overview real numbers and traces a latency shape First two items from a component-by-component comparison against the mocks. The pattern that audit found: everything newly built matched, everything pre-existing got the palette but not the layout, and an "absent rather than empty" rule hid most of the overview exactly when someone was looking at an idle installation. **The headline grid is always rendered**, including at zero, with a fourth cell for host memory. Hiding it removed the page's structure precisely when it was most likely to be read, and "0 failed" is information — an absent panel is not. The quiet case is now said in a line underneath instead of by showing nothing. **The sections state counts** rather than listing their destinations: backends, models, updates and running operations instead of the words "Usage and traces". That needed installed backend and model counts in the summary context, which are two more cheap reads on the poll that was already running. **Traces rows carry latency as a bar as well as a figure**, scaled against the slowest request currently in view and turning amber past two seconds. The table had no latency column at all — the number was buried in the expanded detail, so the shape of the tail was invisible while scanning. Scaling against the view rather than an absolute ceiling is deliberate: what matters when reading a page of traces is which of these are the outliers, and an absolute scale flattens every row on a fast installation into nothing. Full e2e suite: 409 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): name the engine on Home's resident models, and add jump-back-in Third item from the mock comparison. The mock showed each resident model with the engine serving it. /system carried only the id, so the audit recorded this as blocked on a server field — but the config loader is already in scope where that response is built, so it is one lookup. SysInfoModel gains an optional `backend`, resolved from the model's config and omitted rather than guessed when there is none (a loose file, or a config since removed). Home renders the column blank in that case; the test pins both halves of that. Memory per model stays out. It is not one lookup — it would mean asking each backend process — and inventing a number beside a real one is worse than leaving the column off. "Jump back in" is the block the mock had and Home did not. The quick-links row above it is a set of first-run actions; these are the three places someone returns to, each stated with what it currently holds rather than as a bare label. Go: routes suite passes. Full e2e suite: 412 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): rank the recommended models as lanes instead of equal cards The hardware recommendations were a grid of equally-weighted cards. The list is already sorted by fit, and a grid throws that order away: three cards side by side say "pick one", when the page has actually formed an opinion about which one. They are lanes now, read top to bottom in fit order, with the leader carrying the single amber "Best fit" label and the rest marked "Also fits". One opinion per page — the alternatives are alternatives, not runners-up each worth their own colour, which is how a strip of coloured badges ends up meaning nothing. Below 720px the size and VRAM columns drop and the lane keeps the name and the install action, which are the two things a narrow screen needs. The existing panel spec moves off .rec-models-item onto .lane rather than being deleted; dismissal, collapse, keyboard operation and install all still pass unchanged, and there is a new assertion that exactly one row is called out. Full e2e suite: 413 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): drop capsule chips app-wide, and un-break the empty voice library **Pills are gone.** A capsule radius reads as a tag floating on the surface, which fights a system whose structure is hairlines and square corners — and with chips on Discover, Host, Activity and the biometrics pages, "some pages have pills" was the real inconsistency rather than any one page. Sixteen selectors move to the small radius: filter buttons, tab pills, activity and biometrics chips, file and count badges, the jump-to-latest control, the nav badge. Round *buttons* keep their circle — .lightbox__nav and .home-send-btn are circles, not capsules — as do every progress track, status dot and avatar, which are round because they are round, not because they are tags. **The empty voice library was unusable.** `.voice-library-empty` sets min-height: 430px, border: 0 and background: transparent — a description of the empty PANEL — and it had been attached to the action instead. The create button was therefore a 430px transparent box that pushed itself out of the panel and could not be seen. Moved onto the container it describes, which now centres its action rather than letting it fall off the bottom. Same class-mangling shape as the Agents header fixed earlier. Full e2e suite: 416 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): put Host's headline figures on the shared hairline strip Host had shadowed, clickable StatCards above a page that already has a rail, a pane and a tab bar — a second dashboard language on one screen, and a different one again from the figures inside its own detail pane. The Operate overview's figure grid is generalised into a shared `.stat-strip` and Host adopts it, so the two pages read as one system: same cell, same figure scale, same tone vocabulary, and the same hairline grid the split-view StatGrid already uses. The cells stay clickable and still route into the tab and filter they describe, because a count is worth more when it is also the way to the thing counted. Tone is spent only where the number means something — running and updates when non-zero — since a strip where every cell is coloured has no emphasis left. Two bugs made on the way, both now covered: - The first version put `<button>` elements inside a `<dl>` with `<dt>`/`<dd>` inside the buttons. Neither is valid, the browser re-parents both, and the cells collapsed. These cells are a set of controls, so a plain container of buttons is also the honest markup. - Even correct, the strip rendered 2px tall: `.page--app` is a flex column whose split view takes flex:1, so a child with no intrinsic minimum is shrunk away. The old cards survived only because `.stat-card` carried min-height:96px. The strip now declines to shrink, with a test pinning it. The stat-card specs are retargeted rather than deleted: they were written to guard a class collision on a page that no longer uses cards, so they now guard the strip's labels and its height. Full e2e suite: 417 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): make Backends notices an edge rather than a filled card The install and upgrade banners were tinted cards with a full border. A filled panel makes every notice shout at the weight of an error, which is how notices stop being read — and Backends shows one on most visits, so it was shouting routinely. They are now a hairline with a coloured left edge, the same treatment the Operate overview gives rows that want a decision, so "this needs you" looks the same wherever it appears. Counts in the notice take the monospace tabular figures the rest of the console uses. Also drops the last inline style on the page, and refreshes the inline-style baseline, which has read 624 against a real count since #11288 landed. The gate exits 0 either way, so nothing was failing — but a baseline 86 above the truth would have let that many inline styles back in unnoticed. Now at 538, which tightens the ratchet rather than loosening it. The spec creates the upgrade it asserts on rather than skipping when the mock has no notice: a test that skips is a test that proves nothing. Full e2e suite: 418 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): finish the mock parity list, and stop hiding the recommendations The last two items from the audit, plus a correction. **Discover's use-case shelf is lanes.** These are a list of ways in, read in order; a grid of equal cards asks the reader to compare them, which is not the choice on offer. **The request panel reaches every generator.** Video, 3D, Sound and Audio FX join Images and Speech, so each one teaches its own endpoint rather than two of six doing it. Audio FX records the fields that shape the request rather than the bytes, since its payload is multipart. **Recommendations no longer collapse themselves.** They were folded away by default once anything was installed. That is the page's one opinion about this host, and an opinion hidden by default is one the reader never gets. Someone who disagrees can still collapse it and that choice is remembered — the difference is that we no longer make it for them. Three specs asserted the old default and now assert the new one. The use-case heading also sat a line's width from the text it introduces, so the two read as one paragraph. It has air under it now, and the shelf is separated from the recommendations above it. Two tests removed rather than kept: a generator loop whose only real assertion was `expect(endpoint.length).toBeGreaterThan(0)`, and an earlier card-gap guard that could only skip. A test that cannot fail is worse than no test, because it reads as coverage. Full e2e suite: 418 passed, 4 skipped. Inline styles at baseline. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): make the Host figures legible and give the strip its spacing back Three defects introduced by the Host redesign, all found by looking at the running app rather than by the suite. **The figures were invisible.** "Running now" and "Updates available" rendered pure black on the dark ground. Two causes compounding: the `--muted` tone alias never landed, because the source rule has extra spaces before its brace and the exact-match edit missed it silently; and a `<button>` does not inherit colour, so with no tone rule the value fell back to the user agent's `buttontext`. Both fixed, and a test now fails on any figure computing to pure black. **The strip sat flush against the resources panel.** `.stat-strip` declares `margin: 0 0 ...` and is declared later in the file than `.manage-summary`, so the shorthand quietly won and the top margin became zero. Raised to `.stat-strip.manage-summary` so it beats the shorthand on specificity rather than on declaration order, which is the kind of thing that breaks again the next time a rule moves. **Discover's use-case heading had a doubled gap.** `.zero-pane` is a flex column that already separates its children; adding a margin on top of the gap stacked the two. The margin is gone and the heading keeps only its own breathing room. Full e2e suite: 420 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): make Studio's tabs path segments rather than a query parameter `/app/studio?tab=images` reads like a filter applied to a page. It is navigation: a different generator, with its own state and its own deep link. It is now `/app/studio/images`, with the overview at `/app/studio`. Legacy `?tab=` links are redirected once to the path form, replacing the history entry so Back does not bounce between two spellings of the same place. Bookmarks and older links keep working and land on the canonical URL rather than a second version of it, which is the part worth having a test for. The nine `?tab=` references were all in specs, none in docs, so the migration is contained. They move to paths, and a new spec pins the redirect. Full e2e suite: 421 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): make the hardware recommendation a section, not a dismissable card It was a bordered card with a collapse control and a close button, sitting inside a pane that is otherwise hairline sections. Two problems: it read as something bolted onto the page rather than part of it, and treating it as an interruption to be shut is the wrong frame for the one thing the page has to say about the machine it is running on. It is now a plain section with the same heading treatment as the shelves below it. The collapse state, the dismissal, their storage keys and the legacy key read for backwards compatibility all go with it, along with the installedCount prop that existed only to pick a default collapse. Five specs described behaviour that no longer exists and are removed rather than adjusted — collapsing, dismissing, persistence of both, and the toggle's keyboard handling. One new spec asserts the replacement contract: no control with aria-expanded, no dismiss, and no card border. Full e2e suite: 416 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): restore every stripped icon and every default-chrome control You reported two broken icons. They were not two: an earlier automated edit had stripped the `fa-*` class from twenty `<i>` elements across eleven pages, and an `<i>` with no icon class renders nothing at all. Settings' save button, the voice-profile back link, and eighteen others — agent row actions, task and job buttons, import and create actions — were all drawing empty space. Each is restored from its own context rather than a blanket icon: the agent row gets pause/play, pen, comments, file-export and trash; the fine-tune toggle swaps plus for xmark as it opens; the P2P documentation link gets the external-link glyph. The same edit left controls without their classes. Fine-tune's "Import config" was rendering in the browser's own chrome, and `.p2p-cmd__copy` set a border but no background, so it fell back to `buttonface` — a pale grey chip on a dark command block. FineTune's "New job" also had its icon classes folded into the button's className, the same mangling already fixed on the Agents header. Rather than fix the reported two and wait for the next report, this adds a standing audit: twenty-five routes are walked and the test fails on any visible control rendering with user-agent chrome, or any `<i>` without an `fa-*` class. It found the three remaining cases after the first sweep, and it is the reason the next one cannot ship quietly. Full e2e suite: 417 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-08-03 12:16:09 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	58ea2f5d79	feat(ui): give Operate and Studio a front door, and fix two layout regressions (#11305) ... * feat(ui): give Operate a front door and fold six rail groups into four Opening Operate ran firstVisiblePath() and landed on Backends, because Backends happens to be written first in operateConsole.groups. The section that should answer "is anything wrong" opened on a package manager, and nothing was reported until you visited it. Adds /app/operate. Its one irreplaceable block is "Needs attention", which is empty when nothing is wrong and says so in a line rather than rendering a reassuring green panel. It collects stale backends, failed operations and unhealthy nodes. Everything else on the page is a summary you could already assemble by visiting four others. The rail regroups from six headings to four: Inference and Activity were both "the runtime right now", Access and System were both administration. No destination is removed and no gate changes, so isConsoleItemVisible and consolePaths are untouched. Overview leads the first group, which is what makes firstVisiblePath() return it without knowing it exists. Rail entries now carry a signal beside the label. This does not replace the sidebar badge and is not built as if it does: the badge stays on the always-visible sidebar entry for the reason recorded in Sidebar.jsx, that the rail exists only on Operate routes and can be collapsed. The signals are orientation while inside Operate, so they are aria-hidden and nothing urgent depends on them alone. OperateSummaryContext polls once for the whole console, following OperationsContext, which exists because per-consumer setInterval against one endpoint was the defect it fixed. It is mounted by ConsoleLayout for the Operate console only, so "poll only while in Operate" needs no route check. Built on usePolling, so it pauses on a hidden tab. Operations are read from OperationsContext rather than polled a second time, and each source degrades to no-signal on its own so one dead endpoint cannot blank the rest. It reads the cached GET /api/backends/upgrades and never the POST that forces a real registry check. Traces and Usage get no signal yet: /api/traces returns the list, so a count would mean fetching every trace to render one number. A counts endpoint is the honest fix and is scoped separately. Full e2e suite green (369 passed, 4 skipped), including a render-smoke entry for the new route. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): open Studio on what this machine can actually make Studio was a tab strip over six generators that opened on Images, which was never a decision, only the first entry in BASE_TABS. Nothing said which modalities this installation could run, so the way to learn that video had no model was to pick the tab and find an empty select. Adds an overview tab and makes it the fallback. Explicit tabs still win, so existing deep links keep working; anything unrecognised or gated now lands on the overview rather than Images. Each tab carries a dot: filled when an installed model advertises that modality, hollow when nothing serves it. That is the feature in one detail, turning the strip from navigation into a report of what the machine can do before anything is clicked. The dot is aria-hidden because the overview states the same facts in words and the dots change as models load. Two kinds of unavailable, which had to stop looking alike: - switched off, via a permission: no tab and no lane, unchanged - available with no model: a lane, and a route to installing one Studio now owns one MODALITIES table so the tab strip and the overview cannot disagree about what exists, and calls useModels() once, unfiltered, grouping in the browser. useModels(capability) fetches the whole list and filters locally, so a hook per modality would have been six identical requests to /api/models/capabilities on every mount. There is a test for that. Recent outputs read every localStorage store through a new readAllMediaHistory(), which avoids mounting five hooks that carry save timers the overview has no use for. 3D is read separately through use3DHistory rather than folded in: its entries are GLB blobs in IndexedDB, so they cannot come from the same synchronous read. Typical cost is the median of this machine's own history, not a guess, and renders as a dash when there is nothing to go on. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): stop the stat cards and the console rail breaking on small screens Two unrelated causes behind one report that /app/manage looks wrong when the window is narrow. The stat cards were being laid out by the wrong rule at every width. Two different components both claimed `.stat-grid`: the dashboard card strip that holds .stat-card children, and the detail-pane StatGrid the split views introduced further down App.css. Being later, the second won every shared property, so the cards got its 120px columns and its 1px hairline gap in place of their own 180px columns and spacing-md. Four cards were packed onto a row that fits two, labels wrapped to three lines and clipped, and the icon crowded the value. Renamed the strip to `.stat-cards`, after the children it actually holds, which also removes the mismatch of a `.stat-grid` container full of `.stat-card`s. The split-view component keeps `.stat-grid` and its BEM parts. The expanded console rail had no bounded height. Thirteen destinations stacked in one column is taller than a phone, so opening the menu pushed the page's own heading past the fold: the menu replaced the page rather than annotating it. Capped at 55vh with internal scrolling below 768px, so the content behind stays reachable. Both are asserted on behaviour rather than markup: no stat-card label may be clipped, the card gap must not be the detail pane's hairline, and expanding the rail must leave the page heading on screen. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): retemper the palette to localai.io and add the lane primitive The token half of the style transfer, plus the shared list idiom the two overviews had each grown their own copy of. theme.css moves from Nord to the website's palette, variable names preserved so every consumer moves with it: ground #13171f -> #0d1117, accent frost cyan #88c0d0 -> action blue #4f8cff, success sage -> mint #56d6a4, warning -> the amber #f1b95d the site spends only on the thing asking for a decision. Eyebrows go mint. Dividers become an opaque #29384a hairline rather than alpha over a varying surface, which is what makes stacked surfaces read crisply on the site. Light is derived, not inverted. The site ships one theme and never had to answer this, but the app does: blue darkens to #2f62d8, mint to #0d8b60 and amber to #8a5d0b, all clearing 4.5:1 on a cool paper ground, where the dark-mode values sit near 2:1. Same three roles, different values. Three files restate the palette because CSS variables cannot reach them: cmTheme.js (the whole CodeMirror theme), VoiceVisualizer and WaveformPlayer (canvas). Left alone they would have quietly kept the app half-Nord. The `.lane` primitive replaces the near-identical row CSS that OperateOverview and StudioOverview had each written: a full-bleed row on a hairline that insets on hover, with no card and no shadow. Callers supply only the column template. Both pages now use it, along with `.lane-head` for section rhythm and a `.page-pad` container for top-level pages outside a console shell — without which Studio sat flush against the sidebar with its eyebrow clipped. Studio's tab strip wraps rather than running off the edge at narrow widths. Full e2e suite: 386 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): put Home's resident models on lanes and give the footer one line Home's status line was three chips saying a thing was true. It now reports figures: how many models are resident, how many nodes are healthy, what share of memory is in use, set in tabular monospace so the digits line up. A chip answers whether; a figure answers how much, which is what someone opening the page at a glance is after. Resident models move from status chips to lanes, with the id set in a new `.lane__name--id` because an id is something you might type or paste and the UI face makes it read as a label. /api/system-information carries only the id, so there is deliberately no backend or memory column: inventing one would mean a server change this does not make. The footer was three centred rows and cost the bottom sixth of every page for chrome. It is one line now, version left and links right, wrapping to centred when the viewport is too narrow to hold both. Every link it had, it keeps. Full e2e suite: 392 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): correct three contrast failures and stop a guaranteed-404 poll A contrast audit of the new palette found three values below WCAG AA, one of which the previous commit message claimed was fine: - White on the #4f8cff button is 3.22:1, which is large-text only. The website does exactly this, but a button label in an app is not large text, so the label goes to dark ink at 5.88:1. Light mode keeps white, which is 5.44:1 on its darker blue. - Light-mode success was 4.08:1 on paper, not the 4.5 claimed. Darkened to #0a734f, 5.56:1. - Nord red was already 4.28:1 on raised surfaces, a pre-existing miss carried over unexamined. Lifted to #c96f78, 5.02:1. Lanes gain the two states they were missing: a 44px target on coarse pointers, matching what EntityRail already does so the two list idioms feel the same under a thumb, and a reduced-motion variant that keeps the background feedback while dropping the hover inset, which is a position change. The Operate summary no longer asks for /api/nodes on a single-node install. The cluster API answers 503 when distributed mode is off, so it was a guaranteed miss every fifteen seconds; it is now gated on useDistributedMode, the same condition the rail already uses for the Nodes entry. Full e2e suite: 392 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): restore the gap between overview blocks, and stop claiming zero nodes Two defects a design review surfaced. `.lane-head:first-child { margin-top: 0 }` was meant to stop the first block on a page carrying a top margin. But every <section> makes its lane-head a first child, so the reset applied to all of them and the gap between blocks vanished: "Sections" sat flush against the attention row above it. The header supplies its own bottom margin, so a uniform top margin is correct everywhere. The Cluster summary read "0 nodes" on a single-node install, which looks like a fault when the cluster API is simply switched off. It now says "Single node". Full e2e suite: 392 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): open dark by default, and stop clipping the collapsed sidebar footer Dark is the identity rather than a preference: localai.io ships one theme and it is this one, so an install should look like LocalAI before anyone has chosen anything. The OS setting no longer selects light on first load. The toggle still does, and a stored choice wins forever after, which the tests assert both ways. The collapsed sidebar footer stacked its controls but kept the expanded row's inline padding, so their edges were clipped against the 51px rail. Full e2e suite: 394 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(api): count traces server-side and give the Operate overview real totals The overview's headline block had no source. /api/traces returns the trace list, so "37 errors in 24h" meant fetching every buffered exchange to count it in the browser — waste that grows with the buffer, to produce three integers. Adds GET /api/traces/summary: totals, failures, p95 and a bucketed series for sparklines, over a window that defaults to 24 hours and is capped at a week. Deliberate calls, each with a spec: - A 4xx is the caller getting it wrong, not the installation being unhealthy, so only 5xx and transport errors count as failures. - p95 is a nearest-rank percentile rather than the slowest request, which is what a max would report and what makes latency panels lie. - Buckets are oldest-first so a sparkline reads left to right, and the slice is never nil: nil serialises as null and breaks .map() on the other side, which is a silent runtime error rather than an empty chart. - Exchanges outside the window are not counted at all. The route is registered before /api/traces/:id so "summary" is not captured as a trace ID. On the client, Traces and Usage gain the rail signals they were shipped without, the Observability section summary now states counts instead of listing its destinations, and an installation that has served nothing says so rather than showing three zeroes dressed as telemetry. Sparkline is a bare stroke with an emphasised endpoint and no axes: the figure above it already states the value, so its only job is the shape. Go: 185 middleware specs pass. Full e2e suite: 396 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): stop the memory chart calling a trade-off an error The VRAM-by-context chart rendered any build over the limit in error red, and escalated the verdict to the error tone as soon as two context sizes crossed it. But an over-limit build still installs — #11288 keeps a test on exactly that — so red overstates what is happening. A model that fits at 32k and not 64k is a trade-off, not a fault. Over-limit bars and the limit line now use the warning tone, which is the constraint colour used everywhere else in this branch: know what you are doing, not you may not. The error tone is reserved for "fits nowhere", where the model genuinely cannot run on this host. Full e2e suite: 397 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): give the new surfaces orchestrated motion Uses the reveal system already in the codebase rather than adding a library: pageReveal, .reveal-stagger and staggerStyle() were built for exactly this, and anime.js would be ~17KB duplicating four lines of CSS for list reveals. The overview's headline figures, attention rows and section lanes stagger in, as do Studio's modality lanes and recent outputs, so a page assembles in the order it is read instead of appearing all at once. Two additions beyond stagger. Rail signals transition on opacity when a poll lands, so a number changing reads as an update rather than a jump cut, and it stays on the compositor so it cannot reflow the rail. The attention block animates its left edge in — the one thing on the page that should announce itself, and on the border rather than the text so nothing moves under a reader. Both are dropped entirely under prefers-reduced-motion, alongside the lane hover inset already handled. Full e2e suite: 397 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): put the generators on a hairline field stack and record the request The workbench treatment from the mocks, applied where it costs least: both changes land on shared surfaces, so all six generators get them at once rather than drifting apart page by page. The control column stops being a shadowed card of boxed groups and becomes a hairline field stack — the panel is the page's left half, not an object floating on it — with uppercase micro-labels matching the eyebrow treatment used elsewhere. Because .media-controls is shared, Images, Video, 3D, Speech, Sound and Audio FX all move together. RequestPanel shows the request the form actually built, with a copy-as-curl. LocalAI is API-first and Studio is the best place in the app to teach its own endpoints: the form stops being a black box, and a result worth keeping can be reproduced from a shell without reverse-engineering which fields the page sent. It records what was sent rather than what the form currently holds, and renders nothing until a request has been made — a panel describing a request nobody made is a tutorial, not a record. Wired into Images and Speech. Full e2e suite: 401 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): make Chat a transcript instead of a bubble thread Rounded, filled, asymmetric bubbles fight a system built on hairlines, and they carry the speaker in shape and side rather than in words. The assistant side had already given up its bubble; this finishes the job. Both roles now run full width down one column, separated by a rule, each with a mono role label. The user turn keeps a left edge in the action tone so the two are still told apart at a glance, without a fill or a corner radius. The avatars go: the accent and the label carry the speaker, so the glyph was decoration once neither side had a bubble. Saying who is speaking in words rather than in geometry is also what survives being read aloud, printed, or looked at by someone who cannot pick the sides apart by colour. Full e2e suite: 404 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): dress the API reference in LocalAI's palette The Swagger page was the last surface still shipping in someone else's colours, which is conspicuous now that everything it links from is dark. Swagger UI has no theming hook, so rather than fork it we serve our own index ahead of the library's wildcard and restate the palette over its stylesheet. The library's own bundle and assets are still what load, so a swagger-ui upgrade cannot silently break the page — this is a skin, not a fork. Two things needed real care. Swagger tints the entire operation row per method via .opblock.opblock-post and friends, so the palette had to match that specificity rather than reach for !important; the method now lives on one edge instead of washing across the row, because a page where every row is a status colour has no status colour left. And the filled method chip put white on pale green, which was the least readable thing on the page — it is an outlined mono chip now, carrying the method in its border and text. Palette values are copied from theme.css rather than referenced: this page is served by Go and never sees the app's CSS. The comment says so, and says to keep them in step. Go: routes and middleware suites pass. Full e2e suite: 405 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): make tall split-view pages reachable, repair the Agents header, scale titles Three things found by actually using the app rather than measuring it. **Host was unusable.** The shell above a split view is overflow:hidden so the document cannot grow, which left anything taller than the viewport simply unreachable — and Host stacks a resources card, four stat cards and a tab bar above its split, so the bottom of the pane fell off at every window height with nothing to scroll. Every sweep I ran for this was horizontal, which is why it kept coming back clean. The page now scrolls inside the pinned shell. The pane keeps its own scroller: letting it grow instead pushes the document taller and stretches the rail to match, which is the regression e2e/discover-height.spec.js exists to catch, and which the first version of this fix duly caused. **The Agents header controls were unstyled** — "Create Agent" was rendering with the browser's default chrome. The markup had been mangled at some point: six unrelated classes merged into one string on the link, and the label and button left with none at all and empty icons. Repaired, with the inline flex replaced by a shared .header-actions class. **Page titles take the editorial scale from the site**: larger, tracked at -0.04em, on a line height near 1, so a two-word title reads as a statement rather than a label. The typeface is unchanged — DESIGN.md keeps the existing type system — so the whole difference is scale, tracking and leading, which is where the site gets its voice from. This was the biggest reason the running app still did not look like the mocks. Full e2e suite: 404 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-08-03 00:16:08 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	74b7ea2829	feat(ui): replace the gallery and inventory tables with a rail and a detail pane (#11288) ... * feat(ui): rename the Install Models nav entry to Discover "Install Models" named the action rather than the destination, and it was the only multi-word entry in a rail of one-word ones (Home, Chat, Studio, Talk, Build, Operate). A bare "Models" was the obvious fix but it collides with the installed-models view under Host, which is a different page for a different job. "Discover" keeps the rhythm and says what the page is for. The icon moves from a download arrow to a compass for the same reason: the page is browsed before it is installed from. Translated in all seven locales rather than left to fall back, so a locale switch does not leave the entry in English. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * feat(ui): replace the gallery table with a rail and a detail pane The eight-column table was not the real problem; the click-to-expand row underneath it was. Variants, files and a VRAM estimate never fitted inside a <tr>, so they were pushed into a drawer that could hold one model at a time, could not be linked to, and had no room to say anything useful. The gallery is now a rail to scan and a pane that answers. The pane has two states and no third: with nothing selected it is the discovery page, and with a model selected it is that model's detail. Selection lives in the URL, so a model is linkable and Back steps out of the detail instead of off the page. The rail groups by capability while browsing and flattens to results the moment a term is typed. That is a rule rather than a toggle: once someone has said what they are looking for, the buckets are between them and the answer, and making the user choose would be handing them our problem. The detail pane plots VRAM against context length with the host's own limit drawn across it. This is new information, not a restyle. A single number invites "so will it run?", and the honest answer is usually "yes, up to a 32k context", which is a shape rather than a number. The estimates were already fetched for every context size, so it costs no new request. Backends that take no context length say so instead of being given a meaningless chart, and a host with no GPU gets no chart at all rather than bars with nothing to compare against. The split-button variant menu goes with the actions column. The pane lists every build with its backend, quantization, size, fit and a details disclosure, each installable, which is what the dropdown was a cramped substitute for. Its tests move onto that list; the three contracts it alone carried (fetch-once caching, the loading state, an unfit build staying installable) are backfilled against the pane. RecommendedModels moves inside the pane, where it has the width to argue for a model instead of listing one, and keeps its own dismissal and collapse. Rail entries carry no description. Two lines is the budget and the second is better spent on whether the thing will run; the stripped-Markdown contract moves to the pane's lede, tooltip included. e2e: 123 passing across models-gallery, navigation, recommended-panel, model-artifact-operation, operations-strip and page-render-smoke. Inline styles in Models.jsx drop from 82 to 41. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * refactor(ui): extract the split view into shared components Discover shipped its rail, pane and detail header as private functions inside Models.jsx. Backends and Host have the same defect and want the same shape, so leaving them there guarantees three rails that drift. SplitView, EntityRail, DetailHeader and StatGrid now live under components/split/. EntityRail is deliberately data-driven: a surface maps its own entity onto { id, name, icon, meta, stripe, groupId } and keeps its vocabulary to itself, which is what stops the rail learning about models, backends and loaded state all at once. The CSS moves with it. What was .discover__rail is .entity-rail, .discover__ pane is .split-view__pane and so on, because a class named after one page is a lie on the next two. Only what is genuinely Discover's stays behind the old prefix: the shelves, the hero and the VRAM-by-context chart. Two additions the shared rail needs and Discover did not: a state stripe, for surfaces read by condition before they are read by name, and an empty label. Discover passes neither. No behaviour change. e2e 100 passing across models-gallery, navigation and models-recommended-panel. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * feat(ui): put the backend gallery on the split view Same defect as the model gallery, so the same shape: a seven-column table over a click-to-expand row that was the only place the repository, licence, tags and links could go. The rail groups backends by the use case they serve, sharing Discover's taxonomy on purpose: a backend is the runtime a use case needs, so "vision" ought to mean the same thing one level down. It flattens on a query for the same reason it does on Discover. The zero state is the one real departure. A backend's fitness is not free memory, it is the accelerator and platform it was built for, so the pane leads with what this host is, then what is not installed yet, then whether anything installed has gone stale. The table listed 37 runtimes and left "which of these can even run here" entirely to the reader. Distribution moves into the pane, which is the one thing a row could never carry: which nodes hold a copy and which do not, with the install-on-more control next to it rather than squeezed against a chip. The distributed and target-node action logic is unchanged, including the guard that keeps a hardware-specific build off the fan-out path. The split-button popover loses its per-row anchoring because there are no rows; one pane, one anchor. Selection lives in ?backend=, preserving the ?target= scope rather than clobbering it. e2e: 139 passing across models-gallery, navigation, backends-management, models-recommended-panel, nodes-per-node-backend-actions, page-render-smoke, operations-strip and model-artifact-operation. The backends spec gains six split-view tests; its three description-cell tests move onto the pane lede. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * feat(ui): put the Host inventory on the split view The last of the three surfaces, and the one that is not a catalog. Both tabs had the same click-to-expand row, so the shell transfers; what does not transfer is the zero state, because there is nothing to discover in your own inventory. With nothing selected the pane reports what is happening: how many models are loaded, what failed, what has an update, and which models are holding VRAM right now. Every number was already on the page. None of them had been assembled into one statement, so "what is going on" was a question the tabs could not answer however long you looked at them. The rail buckets by state rather than capability - Running, Idle, Disabled for models; Update available, Installed for backends - which is the opposite of the galleries and deliberately so: nobody opens Host wondering which of their models does vision. Entries carry a state stripe for the same reason. Load and Stop are promoted out of the kebab, because that is what an operator came for; the rest stays behind the menu rather than diluting it. Adopted, pinned and alias badges follow the model into the pane: they are facts about the thing, not about its state, and the rail line is spent on state. Deliberately NOT done: folding the two tabs into one rail, as the mock had it. It costs five URL parameters, the manage-tab localStorage key and the stat-card shortcuts, all of which are live deep-links today. The tabs stay as the group selector; merging them is a follow-up with its own migration. e2e: full suite 355 passing. New host-split-view spec; alias-template, manage-logs-link, manage-action-menu-position and model-editor-back-nav move off `.table` and the row kebab onto the rail and the pane. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * polish(ui): accessibility and consistency pass over the three split views Findings from a pass over what the previous four commits actually shipped, rather than what they were supposed to. The rail was not a listbox. ARIA lets a listbox contain options and groups, and nothing else, but each group's collapse control is a button that has to sit inside the scroller with the entries it folds. It is now a labelled group of buttons, which is the honest description; selection is announced with aria-current and the arrow keys are unaffected. Every entry was its own tab stop, so tabbing past a forty-entry rail to reach the pane took forty keystrokes. Roving tabindex makes the rail one stop, and arrowing now moves focus with the selection instead of leaving it behind on an entry Tab can no longer reach. The rail rounds its corners with overflow:hidden, which was clipping the focus ring off the first and last entries entirely. Inset outlines fix it. A 30px row is fine under a mouse and too small under a thumb, so coarse pointers get a 44px target without costing density on a desktop. One slot said three different things: "9 models loaded" on Discover, "12 loaded" on Backends, "3 of 9" on Host. All three lists are a page of a larger set, so all three now say so the same way. Also removed: an emptyLabel prop on EntityRail that nothing passed, its dead CSS rule, and MODELS_COLSPAN and ResourceRowDesc, which died with the tables. e2e: full suite 355 passing. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * fix(ui): correct three defects only a real gallery exposed Running the branch against a live instance with 1,595 models and 1,017 backends, rather than against mocked fixtures, surfaced three things the e2e suite could not. Grouping did nothing. The rails matched on the use-case keys the filter chips send (`chat`, `tts`, `transcript`), but those are a server-side vocabulary the handler maps onto entries. What entries actually carry is free-form and inconsistent: models come back tagged `llm`, `gguf`, `vision`, `coding`, and backends `LLM`, `text-to-text`, `audio-transcription`. Nothing matched, so every model landed in "Everything else" and the feature was decorative. Grouping now lives in utils/entityGroups.js, shared by both galleries, matching case-insensitively against the vocabulary the API really uses, with the entry's backend as a fallback signal - a backend named `whisper` is a speech backend whatever its tags say. Order is specific before general and that is load-bearing: a vision model is tagged `llm` too, so testing text first would swallow it. The zero state claimed GPU memory on a machine with no GPU. The resources endpoint reports system RAM in the same field when gpu_count is 0, so the hero read "84.4 GB of GPU memory" next to the recommendations panel correctly saying "No GPU detected". The number was never wrong, only its label; it now says system memory unless a GPU is actually present. The page title still said "Install Models" under a nav entry saying Discover. Also: the keyboard test named the model it expected to arrive at, which made it a hostage of the grouping table and broke the moment the buckets were fixed. It now asserts that the selection moves and returns. e2e: full suite 355 passing. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * fix(ui): the filters and the rail were fighting over the same job Four things you find odd on Discover, and they turn out to be one mistake seen from four sides. The rail grouped the current page. The listing is paginated at nine rows, so those bucket headers described nine entries out of 1,595, and turning a page reshuffled the sections under the reader. The structure was never stable because it was computed over the wrong set. The chips were redundant for the same reason, seen from the other side. They send tag= and filter all 1,595 server-side. The rail grouped nine of them client-side by the same axis. Two controls for one job, and the weaker one was the one this branch added, so it goes. Grouping stays only on Host, where the list is complete, local, and bucketed by state rather than capability. The search bar felt odd because it sat in a full-width band while the thing it narrowed was a 290px rail below and to the left. The whole band now lives in the rail column: search, backend, use cases, refinements, then the list it narrows. One column to say what you want, one to show what you got. Nineteen chips do not fit at that width, so they fold into a disclosure that states the selection. A disclosure and not a popover, deliberately: picking use cases is multi-select and interleaves with the backend select and the toggles below, and a popover dismisses itself the moment you touch either. The header held two counts and two buttons at arm's length from all of it. The counts were the third statement of the same number on one screen, after the rail's "9 of 1,247" and the pane's own headline, so they go. The buttons move into the pane's zero state, which is the surface that answers "what do I do here". Also: the two first-run empty states wore .loading-center, which is display:flex in the default row direction because it exists to centre one spinner. With four children that put the icon, the heading, the sentence and the buttons on a single line with no gap. They are now a proper full-height empty state. e2e: full suite 353 passing. Grouping tests are replaced by ones asserting the rail stays flat; chip tests open the disclosure first; two filter-layout tests that asserted the old three-band arrangement now assert the column. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * polish(ui): make Discover a full-height view, group the chips, name the refinements Four things, all of them the same complaint: the page read as a document with controls scattered on it rather than as one view. The header is fused. A title block with its own padding, a subtitle and two counts made the split view look like an attachment to a document that happened to sit below it. It is now a slim bar carrying the title, the count and the two page-level actions, and the split fills the rest of the window. Rail and pane scroll independently, so the filters and the pane's headline stay put while a long list moves under them. The chips group. Nineteen in a flat row is a lot to scan even behind a disclosure, and they already belong to the four families the rest of the UI speaks, so they are bucketed by those. "All" sits on its own above them without a heading, because it is a reset rather than a use case. The refinements stop looking dumped. When the band became a column they were three controls left where they landed; they now read as a named section with one control per row. The zero state suggests again. It had decayed into a "Browsing / 9 of 1,247 / select a model" line that restated the count for the third time on one screen. It now offers the four use cases as tiles that set the filter, which is the shelf idea from the mock without inventing curation or paying for a second fetch. Two bugs found by looking at it rather than at the tests: the disclosure was clamped to 190px, which cut it off partway through its third section so two of the five never appeared at all; and the creation actions rendered twice, once in the new bar and once in the pane hero a few pixels away. e2e: full suite 353 passing. The chip-row test now holds its contract across the per-family rows rather than a single one, and additionally asserts every family is present and non-empty. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * fix(ui): pin the split view's height so a long detail scrolls the pane Selecting a model with a long description grew the whole page and dragged the rail down with it, which is the opposite of what "full height" was supposed to buy. The flex chain was right and the ceiling was missing. .app-layout and .main-content are min-height:100dvh, which is a floor: flex distributes free space but nothing caps growth, so a pane taller than the viewport expanded the column, the document scrolled, and the rail stretched to match. height:100% on the pane then resolved against an auto-height parent and did nothing. The chat route already solves this by pinning .main-content to 100dvh. The same treatment now applies to any route containing a .page--app, selected with :has() so the shell does not have to learn which pages happen to be split views. Below the stacking breakpoint the pin is lifted, because two stacked halves in two short scrollers is worse than a page that scrolls. Measured on a live instance: document height stays at the viewport across selection (950px either side) and the pane overflows internally instead. Adds discover-height.spec.js, which asserts the page height and the rail height are unchanged by selection and that the pane is the thing that scrolls. The existing specs could not have caught this: they mock short descriptions, and the bug only appears when the pane has more content than the viewport holds. e2e: full suite 355 passing. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * feat(ui): give Backends and Host the full-height view, and fix the Update button Backends now matches Discover: the header fuses into a slim bar carrying the title, the count and the page-level actions, the filters move into the rail column where they narrow the rail and nothing else, and the split fills the window. Its seven chips fit at rail width, so unlike Discover's nineteen they need no disclosure. Host gets the bar and the height; its resource monitor, summary cards and tabs stay above the split, because those are read once while the rail and the pane are worked in. Two things the height change surfaced. The console layout is a flex row with align-items:flex-start, so its body sizes to content. Right for the pages it was built for, wrong for a split view, which needs a ceiling to scroll inside: without it the Backends rail ran past the viewport and over the footer. Pinned with :has() so only split-view routes are affected. The filters vanished when nothing matched. Both galleries swapped the whole shell for an empty state, which took the search box and the chips with it, so the page said "try adjusting your search or filters" while offering neither. The shell now stays and the empty state moves into the pane. Also fixes the Update control on Host, which had no className at all and rendered as bare text, next to a status span that had picked up btn classes and two copies of `fas` and so rendered as a button you cannot press. They have swapped appearances back. e2e: full suite 355 passing. The render-smoke selector learns .view-bar__title, since the pages it checks no longer all use PageHeader. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * fix(ui): keep the view mounted while searching, and bring rail grouping back Searching replaced the whole view with a loader. The search box lives in the rail column, so every debounced refetch unmounted the field being typed into and dropped its focus with it. The list, the filters and the pane went too. The shell now stays and the rail says it is busy: a sweep bar under its header and the stale list dimmed, so the eye knows the answer is being replaced without losing its place. A cold start still gets the skeleton, because there is nothing to keep. The condition for that is "nothing has loaded yet", not "the list is empty". Those differ exactly when someone is editing a query that matched nothing, and getting it wrong there would unmount the view on the keystroke after a no-results search - the worst possible moment. Grouping comes back on both galleries. It was removed because nine rows could not fill five buckets, so a page turn rebuilt the rail's whole structure. That was a symptom of the page size rather than of grouping: the rail now asks for 30 rows instead of 9 (Backends 60 instead of 21), which is enough for the sections to read as structure and turns five times fewer pages. The order of the sections is fixed, so what changes between pages is membership, not arrangement. Grouped while browsing, flat while searching, as before: once a term is typed the buckets stand between the reader and the answer. Also gives GalleryLoader a class and a testid instead of six inline style declarations on a bare div, which is why nothing could select it. e2e: full suite 359 passing, including a new spec asserting the search box keeps its focus and its value across a refetch, and that a cold start still shows the skeleton. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * perf(gallery): stop invalidating the VRAM estimate caches on every request Searching or turning a page felt slow. It was not the search and not the listing: /api/models answers in 3-9ms. It was the VRAM estimate, which the gallery asks for once per row, and which took ~2.3s every single time however often the same model was asked about. pkg/vram already caches what makes that expensive - the remote content-length probes, the GGUF metadata reads and the HF repo sizes. Those caches key on a gallery generation counter, and AvailableGalleryModelsCached triggered a background refresh on every call, with each refresh bumping the counter. One page view is one listing request plus thirty estimate requests, each of which re-read the gallery and started another refresh, so the generation moved constantly and every cache entry was stale before it could ever be read. The caches were dead in production. Three changes, each doing one thing: A refresh interval. The cached list is still served immediately; this only decides how often re-fetching from upstream is worth starting. Five minutes, as a package variable so tests can drive it without waiting. A generation bump only when the gallery actually changed. An unchanged gallery re-fetched on schedule must not throw away work that is still valid, which is the difference between an estimate costing nothing and costing a network round trip. A separate "loaded" flag. The cache engaged on `cached != nil`, so a gallery that legitimately holds nothing read as never-loaded and took the blocking path on every call, bumping the generation each time. Found by the test for the interval, which could not pass while this was true. Measured against a live instance with 1,595 models: one estimate, repeated 2.3s -> 2ms a page of 30, in parallel 10s -> 0.04s A first, genuinely unseen model still costs its remote probe. That is inherent; what changed is that it is now paid once per model per gallery version rather than once per request. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * perf(gallery): warm VRAM estimates at startup, and stop the UI waiting on them Two halves of the same complaint: the gallery stalls on VRAM estimation. Server side, the estimates are now warmed in the background at startup. Estimating an entry nobody has asked about costs a remote probe of its weight files, and the gallery needs one per row, so the first visitor was paying for the whole page. The warm-up walks the gallery in the order the UI lists it, so the first page is ready before anyone reaches it. It is bounded and it never blocks: 300 entries at 4 at a time by default, on its own goroutine, stopping with the server's context. Warming the whole gallery would be thousands of probes on every boot, which is rude to the upstream and slow to finish; warming nothing leaves the first page paying two seconds a row. Anything past the limit still warms itself on first view. LOCALAI_VRAM_WARM_LIMIT=0 turns it off for an air-gapped host, LOCALAI_VRAM_WARM_CONCURRENCY=1 slows it for a metered link. Client side, the page no longer waits on estimates it does not need yet. It fired one request per row at once; a browser allows about six connections per host, so thirty estimates took every slot and the request behind a click - the variant list, an install - queued behind work nobody asked for. That is the freeze: the list was already usable, and the UI was busy fetching sizes. Four at a time leaves room for the interactive request to overtake, and a row whose estimate is still in flight says "sizing…" rather than leaving a blank where a number will appear. buildEstimateInput moves to core/gallery as EstimateInput, since the handler and the warmer both need it. Measured against 1,595 models, from a cold boot: page 1, 30 estimates in parallel 10s -> 0.04s full warm-up (299 of 300 entries) 3m, in the background Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * chore: untrack data/.local_user_id and ignore the runtime data dir `local-ai run` writes its instance state under ./data when started from the repo root, which is exactly what a contributor testing a build does. The identity file ended up committed on this branch by a `git add -A` while verifying the gallery changes against a live instance. Anchored, so it matches the runtime directory at the repo root and not a `data` directory nested inside some package. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-08-02 19:28:36 +02:00
Adiraandlocalai-org-maint-bot	ef724a3c9d	feat(api): add /v1/detokenize endpoint (#9620) ... * feat(api): add /v1/detokenize endpoint Closes #1649. Mirror of the existing /v1/tokenize path, requested by @benniekiss in the issue thread for "complete API workflow" use cases that need to turn token IDs back into text without local processing. - Add Detokenize gRPC RPC with DetokenizeRequest{tokens} / DetokenizeResponse{content} messages. - Implement in the llama.cpp backend using common_token_to_piece, the same primitive TokenizeString already uses internally. - Other backends inherit the default Unimplemented from base.Base, in line with how Detect, Rerank, etc. are gated per-backend. - Wire up the Go gRPC interface, server, client, and in-process embed wrapper alongside their TokenizeString counterparts. - Add the schema types, ModelDetokenize wrapper, HTTP handler, route registration, RouteFeatureRegistry entry (gated by FeatureTokenize so no new feature flag is needed), and the discovery map entry under ai_functions. - Regenerated swagger reflects the new endpoint and types. - Update authentication.md to list /v1/detokenize alongside /v1/tokenize. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com> * test(e2e): add mock backend tests for /v1/detokenize Add Detokenize to the mock gRPC backend and wire up two e2e tests in the MockBackend suite: one that posts known token IDs and asserts a non-empty content response, and a round-trip that tokenizes first then detokenizes the returned IDs. Addresses reviewer feedback on #9620. Assisted-by: Claude:claude-sonnet-4-6 Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com> * fix(kokoros): implement detokenize in the Rust backend service The Detokenize RPC added in this PR grows the tonic-generated Backend trait. Unlike the other languages there is nothing to inherit a default from — Rust trait impls must list every method — so backend/rust/kokoros failed to compile: error[E0046]: not all trait items implemented, missing: `detokenize` --> src/service.rs:72:1 72 | impl Backend for KokorosService { Go backends pick up the Unimplemented default from base.Base, and the generated C++/Python servicer bases default to UNIMPLEMENTED, which is why the Rust backend was the only one that broke. kokoros is the sole Rust crate in the tree, so this is the full extent of the fallout. Return Status::unimplemented("Not supported"), matching how this same file already gates tokenize_string and ~20 other unsupported RPCs. Fixes the tests-kokoros and backend-jobs-singlearch-4 (-cpu-kokoros) failures on the previous head. Assisted-by: Claude:claude-opus-5 cargo Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com> --------- Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com> Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>	2026-07-30 16:01:47 +02:00
localai-org-maint-botandDaria Korenieva	9bfd71387b	feat(stores): add Valkey Search vector store backend (#11196) ... * feat: add Valkey Search vector store backend Add a new built-in Go gRPC store backend 'valkey-store' that implements the four Stores RPCs (Set/Get/Delete/Find) against the Valkey Search module (FT.*) using the pure-Go github.com/valkey-io/valkey-go client. It is selected via the existing per-request 'backend' field on /stores, so there is no proto or HTTP API change, and it mirrors the in-memory local-store while adding persistence across restarts and opt-in HNSW. Each vector is a Valkey HASH keyed by hex(little-endian float32); the index is created lazily on first Set (FLAT+COSINE by default), cosine similarity is derived as 1-distance, and namespaces get a collision-resistant token. Includes unit tests (valkey-go mock) and env-gated integration tests against valkey/valkey-bundle, plus build/matrix/gallery wiring and docs. Assisted-by: Kiro:claude-opus-4.8 golangci-lint Signed-off-by: Daria Korenieva <daric2612@gmail.com> * Address review feedback: recover persisted index dimension, harden Find - Load now recovers the persisted vector DIM from FT.INFO (not just index existence), so a post-restart Set/Find validates against the real DIM instead of silently re-learning a wrong one and dropping mismatched vectors from the index. This also restores Find's dimension check after a restart. - StoresFind treats a dropped/missing index as an empty store (empty result, no error) and clears the stale indexCreated flag, matching local-store's empty-store behaviour. - StoresSet reuses checkDims for its per-key length check so the four RPCs share one dimension-guard implementation. - Add unit tests for FT.INFO dimension recovery, loadIndexState, and the dropped-index Find path. Assisted-by: Kiro:claude-opus-4.8 Signed-off-by: Daria Korenieva <daric2612@gmail.com> * Address review feedback: TLS ServerName/CA, Find nil-check, config fail-fast Addresses external review comments on the valkey-store backend: - StoresFind now rejects a nil/empty query Key before dereferencing it, so a malformed gRPC request can no longer panic the backend. - TLS: derive ServerName (SNI) from the VALKEY_ADDR host so certificate verification works for IP-addressed endpoints, and add VALKEY_TLS_CA_CERT (custom CA bundle) and VALKEY_TLS_SKIP_VERIFY (testing-only) knobs. - Config integer parsing now fails fast on a malformed value (e.g. VALKEY_HNSW_M=1x6) instead of silently defaulting, matching the fail-fast behaviour of the index-algo/distance-metric validation. - Add VALKEY_DB (SELECT n) support for logical-DB isolation. - Cap the human-readable part of a namespace token at 64 chars so a very long model name cannot produce an unbounded key prefix / index name (the appended short hash keeps distinct namespaces collision-free). - Document the KNN-query injection-safety invariant (fields are constants) and why StoresGet uses a single aggregate DoMulti deadline for reads. - Unit tests for the Find nil/empty-key guard, fail-fast HNSW parsing, and VALKEY_DB parsing/validation; docs + .env updated for the new vars. Assisted-by: Kiro:claude-opus-4.8 golangci-lint Signed-off-by: Daria Korenieva <daric2612@gmail.com> * Address review feedback: configure valkey-store via model config richiejp asked that the valkey-store backend take its configuration from a model config rather than process-wide VALKEY_* environment variables, so multiple stores can each have their own Valkey config within one LocalAI process. This removes every env access from the backend and routes config through the model-config seam every other backend uses. - config.go: loadConfig(opts *pb.ModelOptions) now parses the model config `options:` list (key:value strings, split on the first ':') instead of os.Getenv. Option keys mirror the old VALKEY_* names without the prefix (addr, index_algo, distance_metric, ...). Defaults, fail-fast validation and the mandatory client name are unchanged. - store.go: Load threads opts into loadConfig; TLS comments/errors renamed off the VALKEY_* names. - core/backend/stores.go: StoreBackend and NewVectorStore take a *config.ModelConfigLoader, resolve the per-store ModelConfig by store name, and pass its Options (and Backend when unset) to the backend via WithLoadGRPCLoadModelOpts. No config -> default backend + built-in defaults, preserving the zero-config experience. - Endpoints/routes/application: thread the config loader to StoreBackend. - Unit + integration tests: configure via options; the integration test passes addr through the model-config path (VALKEY_ADDR is now only the test harness locating the server). - docs + .env: document the model-config options, drop the env var table. Assisted-by: Kiro:claude-opus-4.8 Signed-off-by: Daria Korenieva <daric2612@gmail.com> * Remove valkey-store informational comment from .env The backend is configured via model config, not env vars — the comment was unnecessary noise in .env. The configuration is already documented in docs/content/features/stores.md. Signed-off-by: Daria Korenieva <daric2612@gmail.com> * feat(valkey-store): gate Load on NamespacePrefix to refuse autoload probing Mirror local-store's pattern: reject model names without store.NamespacePrefix so the model loader's greedy autoload probe cannot bind an arbitrary model name to the vector store backend (the #9287 failure mode). Also adds unit tests for the gate covering: prefixed namespace, prefix alone, unprefixed model name, empty model, and nil opts. Signed-off-by: Daria Korenieva <daric2612@gmail.com> * feat(valkey-store): add username_env/password_env credential indirection Add support for resolving Valkey credentials from environment variables named in the model config, mirroring cloud-proxy's api_key_env pattern. This keeps secrets out of model YAML files and lets distinct store configs each reference their own credentials. Options: username_env / password_env name the env var holding the value. The direct username / password options still work and take precedence when both are set (backward compatible). Includes 5 unit tests and updated stores.md documentation. Signed-off-by: Daria Korenieva <daric2612@gmail.com> * fix: correct rebase artifacts in backend-matrix.yml and Makefile Fix two issues introduced by the conflict-resolution script during the rebase onto master: 1. .github/backend-matrix.yml: valkey-store entries were merged INTO the cloud-proxy entries (duplicate keys in same YAML map items) instead of being separate list items. This broke cloud-proxy Linux builds and the cloud-proxy darwin entry lost its build-type/lang. Fixed by making them standalone entries and restoring cloud-proxy exactly as on master. 2. Makefile: duplicated .NOTPARALLEL and docker-build-backends lines. Collapsed to single lines that are master's current content plus the valkey-store additions. Also adds the three optional pickups from #10801: - /valkey-store in .gitignore (the built binary) - valkey-store row in docs/content/reference/compatibility-table.md - valkey-store line in backend/README.md Signed-off-by: Daria Korenieva <daric2612@gmail.com> --------- Signed-off-by: Daria Korenieva <daric2612@gmail.com> Co-authored-by: Daria Korenieva <daric2612@gmail.com>	2026-07-29 20:12:29 +02:00
Richard Palethorpeandlocalai-org-maint-bot	9058a2bb46	feat: Add 3d generation UI/API and trellis2cpp backend (#10979) ... * feat(3d): add Generate3D RPC, FLAG_3D capability, and /v1/3d/generations endpoint Adds the plumbing for image-conditioned 3D asset generation (binary glTF / GLB output), modeled on the video generation path: - backend.proto: Generate3D RPC + Generate3DRequest (staged image src, glb dst, seed/step/cfg_scale/texture_steps, quality and background enums, params map for backend-specific extras) - pkg/grpc: thread Generate3D through client, server, embed, base and the backend interfaces; connection-evicting and distributed-node wrappers (in-flight tracking + file staging) included - core/config: FLAG_3D usecase (guessed only for the trellis2cpp backend), '3d' canonical usecase string mapped to the Generate3D method, and a '3d' output modality - REST: POST /v1/3d/generations (+ unversioned alias) returning OpenAIResponse with a /generated-3d URL or b64_json; conditioning image accepted as URL, base64, or data URI; quality/background validated at the edge; .glb served as model/gltf-binary - auth: '3d' route feature (default ON); /api/instructions entry Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(trellis2cpp): add the trellis2.cpp image-to-3D backend Wraps localai-org/trellis2cpp (C++/GGML port of Microsoft TRELLIS.2, pbr-textures branch) as a Go+purego backend, following the stablediffusion-ggml pattern: - backend/go/trellis2cpp: purego bindings to the flat C ABI (v9, asserted at startup), eager pipeline load with model-set validation (refuses non-trellis GGUFs; degrades coarse/geometry-only/textured exactly like the upstream demo), Generate3D via t2_generate + t2_bake_glb writing a binary glTF to dst. Weight-free unit tests cover resolution/validation/param mapping — CI never downloads the multi-GB GGUF set or runs inference. - CPU SIMD variants build into per-variant directories (the shared libggml sonames collide across variants, unlike sd-ggml's flat renamed-.so scheme); run.sh picks one via /proc/cpuinfo. - CI wiring: backend-matrix entries (cpu, cuda12/13, vulkan amd64+arm64, l4t, l4t-cuda13, darwin metal), index.yaml meta + latest/master image entries, bump_deps tracking of the pbr-textures branch, changed-backends.js mapping, top-level Makefile targets. - Importer: auto-detects trellis GGUF repos/URIs (registered before llama-cpp so the .gguf match isn't stolen) and expands any trellis URI to the full 10-file component set spanning the three LocalAI-io HF repos. - Gallery: trellis2-4b (full PBR + 1024 cascade) and trellis2-4b-geometry (512 untextured) with verified sha256s. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(ui): 3D generation page with native GLB viewer and IndexedDB history Adds a Studio tab + /app/3d page for the new image-to-3D endpoint: - GlbViewer ports the trellis2cpp demo's dependency-free WebGL2 renderer (quaternion trackball, metallic-roughness PBR, ACES, hidden-line wireframe with a bounded index budget) and pairs it with a minimal GLB parser for the two forms t2_bake_glb emits — dense vertex-PBR (linear COLOR_0 + _METALLIC_ROUGHNESS, uploaded as normalized integers) and the opt-in UV-atlas textured form. Parsing happens before any GL so stats and errors render without WebGL2. - use3DHistory stores past generations (params, input thumbnail, and the GLB blob itself) in IndexedDB with keep-newest-20 eviction — GLBs are multi-MB binaries localStorage can't hold — and the page offers a download button for the active GLB. - Wiring: CAP_3D capability constant (FLAG_3D — the exact string /api/models/capabilities serves), threeDApi, router entries, Studio tab, vite dev proxy, en locale keys. - e2e: render-smoke entry plus a focused spec that feeds a real one-triangle vertex-PBR GLB through the parser/viewer and exercises IndexedDB persistence, selection, deletion, and API errors. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(3d): address API correctness and UX issues Keep 3D generation on the LocalAI-specific /3d/generations route and ensure authentication and permissions cover it. Propagate distributed transfer failures, publish a portable ARM64 backend image, honor importer overrides, and align discovery, upload validation, and touch controls. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(3d): add previewable print remeshing Add a single-detail CGAL Alpha Wrap workflow for existing Trellis GLBs, including PBR reprojection, API documentation, tracing, and an in-browser preview before download. Allow the remesh route to enforce its 512 MiB upload cap independently of the smaller global default so generated high-resolution meshes can be processed. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * build(trellis2cpp): centralize remesh dependency pins Assisted-by: Codex:GPT-5 [apply_patch] [exec_command] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(kokoros): implement Generate3D stub for new proto RPC The Generate3D RPC added to backend.proto for the trellis2cpp backend made tonic's generated Backend trait require generate3_d, breaking the kokoros-grpc build. Return unimplemented like the other unsupported modalities. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com> Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>	2026-07-29 16:15:04 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	0f7186f214	feat(ui): replace the stacked operations bar with a one-line strip and an Activity page (#11163) ... * feat(ui): record finished gallery operations in a bounded history ring The operations panel drops an operation the moment it succeeds, so a user who steps away cannot tell whether an install finished, failed or was never started. OpCache now keeps the last 50 terminal operations, recorded from the point where an op leaves the cache. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): pin the history ring's dedupe, outcome order and start stamp Review of the history ring found four gaps. The dedupe guard and the bounded seen set were unreachable through the exported API and so had no coverage; an in-package spec file now drives opHistory directly. The outcome switch claimed an ordering was load bearing that nothing pinned, so an errored op that never reached Processed now has a spec. Two behaviour fixes come with it. StartedAt was the zero time for ops recovered from the store or replicated from a peer, since neither path stamps a start time, which would have rendered as a two-millennia duration; it now falls back to the finish time. Reusing a cache key with a fresh job ID orphaned the previous stamp, so Set and SetBackend now drop it. The comment on the outcome switch described a state the code cannot be in: CancelOperation sets Cancelled and Processed synchronously before the handler removes the entry, so status.Cancelled already covers the cancel endpoint. The !Processed clause stays for the dismiss endpoint firing on an in-flight op, and the comments now say so. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): record operations that end on a peer replica The NATS end event is the only signal a replica gets for an install another replica ran. Record from applyEnd too, deduped by job ID so the originating replica does not record its own broadcast twice. Three start-stamp defects in the same path go with it. applyEnd now drops the stamp unconditionally, since recordTerminal only cleans up on the path where it found a cache key and an end event can overtake the local Set. applyStart drops the stamp of the job whose cache key it replaces, which a peer-driven retry previously stranded. And recordTerminal reads the stamp once instead of testing Exists and then reading, so a concurrent record for the same job can no longer delete the stamp between the two and let the zero time overwrite the finish-time fallback, which the Activity page would render as a two-millennia run. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): do not guess the outcome of a peer operation with no local status A replica that restarts mid-operation hydrates its OpCache keys from PostgreSQL, but gallery statuses are in-memory only and come back empty. The end broadcast then landed on recordTerminal's nil-status branch, which reads a missing status as queued-and-removed and filed a successful install as cancelled. That reading is right locally and wrong on the peer path, where a missing status means the outcome was never held here. recordTerminal now takes the source of the terminal event and records nothing when the peer path finds no status, restoring what the replica did before the end event started recording. The local path is unchanged. Also move the ApplyEndForTest seam to the conventional export_test.go, and stop the dedupe spec from claiming to guard the ring's seen set: the local delete removes the status keys, so the broadcast that follows returns before reaching it. An in-package spec that calls recordTerminal twice does the pinning. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(api): add GET and DELETE /api/operations/history Admin gated like the rest of the operations API. The live /api/operations payload is unchanged so the one second poll stays small. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): expose operation history through OperationsContext Fetched on demand and when the live list shrinks, never on the one second poll interval. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): detect operation departure by identity and ignore committing ops in the ETA gate Refetching history on a shrinking live count missed a completion that coincided with a start, which is the common case during a batch install. Track the live job IDs instead, so any departure triggers the refetch regardless of how the count moved. An operation that has finished downloading stays live at currentBytes == totalBytes for the whole commit and install phase and can never produce an estimate, so counting it in the all-or-nothing gate blanked every other operation's time remaining for as long as it lasted. Only operations still moving bytes get a vote. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): let only downloading operations gate the time remaining estimate Verifying pins an operation flat below its total for the whole sha256 pass: the AfterDownload hook reports completedBytes plus the finished file against a total summed over every file, then hashes synchronously without emitting progress. Files download sequentially, so a 15 shard model enters that window 14 times, and a byte comparison cannot see it because the counter is genuinely below the total throughout. Gating on phase closes resolving, verifying, committing and persisting in one predicate, so a quiet neighbour no longer blanks every other operation's estimate for minutes at a time. The byte clauses stay: a producer can report downloading with bytes already at the total. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): collapse the operations bar to a single line Four concurrent installs used to take four rows above every page. The strip now shows one operation, failure first, with a counter linking to Activity. The close button hides the strip and no longer cancels an install. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): keep the operations strip from widening the page and from muting a failure A long install error made the strip report a 1600px minimum width, which sized main-content to fit and gave every page under it a horizontal scrollbar. Inline-size containment plus shrinkable detail and bytes cells keep it inside the viewport. Hiding is no longer able to swallow the hidden job's own failure, a completed removal or staging says so instead of claiming an install, a cancelling operation renders as cancelling, and the live region no longer covers the per-second percentage. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): shrink main-content instead of containing the strip, and expose progress min-width on .main-content is what actually lets a long install error shrink, and unlike inline-size containment it has no browser support floor and no latent collapse if the strip ever lands in a shrink-to-fit context. It matches what .app-layout-chat .main-content already does, and it clears pre-existing horizontal overflow on narrow viewports as a side effect. The progress track is now a labelled progressbar, so assistive tech can read the value on demand rather than losing it to the aria-hidden that stopped the live region re-announcing every poll. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add the live operation card for the Activity page Carries the detail the one-line strip has to drop: phase, bytes, the per-node breakdown for cluster installs, and a labelled Cancel button. Cancelling is destructive, so it gets a labelled button rather than a glyph. A cancelling operation drops its progress bar and its time estimate, the same call the strip makes: a percentage still climbing under "Cancelling" reads as the cancel not having taken. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): give the operation card a verb, live node disclosure and its per-node detail The card carried no verb, so an install, a removal and a staging op rendered as spinner plus name plus kind tag and were indistinguishable. It now runs the same verb and icon chain as the one-line strip, which is what stops the page that is meant to carry more detail from carrying less. The auto-expand default was evaluated once at mount. An operation is listed as soon as it is admitted but its nodes are filled in only when the fan-out starts reporting, so a card mounted at creation latched on the empty list and stayed collapsed. The default is a live expression now, and state holds only an explicit choice. Also: an optional onRetry gates a Retry button, so the page can own the install reconstruction without the card ever showing a control with nothing behind it; the disclosure moved above the region it controls and gained aria-controls; the toggle is gated at more than one node so the count is never "1 nodes"; an unmapped node status is passed through instead of being relabelled "Queued"; error text is clamped with the full string in the title; and file_name plus the per-node progress bar are rendered again, reviving three CSS rules that had gone dead along with the detail they styled. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add the Activity page Live operations, unacknowledged failures and the record of what finished, at /app/activity in the Operate console. Cancelling an install now lives here behind a labelled button rather than on the strip, and a failed install can be retried: the retry dismisses the failure first so it still reaches the record, then reissues the model, backend or node-scoped backend install. The sidebar Operate entry carries the operation count. The console rail is only rendered on an Operate route and can be collapsed, so a badge there could vanish while operations were still running. Two follow-ups from review fold in here: a failed removal or staging job no longer reports a failed install on either the card or the strip, and the card's error text can shrink so one unbroken token cannot widen the card. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): dismiss operations by job, and stop the Activity page contradicting itself Dismissing resolved the job by display id, but /api/operations strips the "node:<nodeID>:" prefix before emitting, so a local install and a node-scoped install of one backend arrive as two jobs sharing one id. Dismissing by id retired whichever came first. That defeated the guarantee retry was built around: with the wrong job dismissed, the reinstall overwrote the acted-on failure's opcache entry in place, bypassing recordTerminal, while an unrelated failure vanished from Needs attention. dismissFailedOp, the card's dismiss control and the strip now all pass the jobID, which is what the endpoint takes. A filter matching nothing rendered the "nothing has ever run" empty state while the header counted the records the filter had hidden. The empty state is now gated on the All chip and a narrowed view gets its own message plus a way back; the header counts the instance rather than the chip, so selecting Backends no longer reports "Nothing running" over running model installs. Also: the summary drops a zero clause instead of rendering "0 needs attention" on the happy path and pluralises both counts; a record duration is floored at "< 1s" and rejected above a day, so a zero-value start stamp cannot render a span of millennia and a zero span cannot render "installed in" with nothing after it; a deletion cancelled mid-flight reports the cancellation rather than claiming it was removed; and the retry variant comment names the fix instead of calling the gap closed. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: document the Activity page and the operations history endpoints Adds an Activity page under Operations covering the one-line operations strip, the /app/activity sections and filters, per-operation cancel, retry and dismiss, the in-memory 50-entry record, and the sidebar count. Documents GET and DELETE /api/operations/history, and fills the gap in the admin-only endpoint list, which also omitted the pre-existing POST /api/operations/:jobID/dismiss. Corrects the distributed-mode install-watching section: the per-node breakdown now lives on the Activity page rather than on the strip, which rolls a fan-out up into a single phrase. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: correct nine details in the Activity page documentation The operations strip never renders a file name: its detail line is the error, the node roll-up, the target node, the phase or the queued note. Drops the stale clause in the distributed-mode section, where the per-node bullet is now the only place a file name is described. Scopes the phase vocabulary to artifact-backed gallery models, since a plain GGUF install emits no phase. Corrects the per-node list: the toggle exists for any fan-out of two or more workers and the four-node threshold only governs whether it starts open, while the N nodes tag needs more than one node. Notes that a cancelled operation can sit in the live section reading Cancelling, that cluster staging never reaches the record, and that Clear history appears only when the record has something in it. Names the operations response envelope, with a JSON example, so callers do not index a bare array, and stops describing the icon-only dismiss control as a labelled button. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: drop the unreachable Cancelling state and scope the byte claims An operation can only report isCancelled while it is unprocessed, but every writer of Cancelled sets Processed in the same breath, on the peer path as much as the local one, and the cache evicts cancelled entries before the handler sees them. The state cannot reach the page, so the live section is described again as running or queued operations. Byte counts come from the artifact bridge alone, the same producer as the phase, so a plain GGUF install, a removal and a backend install report none. Scopes both to artifact-backed gallery models and leaves the verb, the name and the percentage as what every operation shows. A worker backend install reports its bytes through fields the operations payload does not carry, so the distributed section now describes the percentage and the node roll-up, with per-file counts pointed at the per-node detail. Also: staging jobs carry no error, so they never reach Needs attention and Retry never had a staging case to exclude; an install that involves workers is no longer called node-scoped, which this page uses for node-targeted installs; and the record timestamps carry nanoseconds. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: state only the verb and the name as unconditional on the strip The percentage is as conditional as the bytes were: it renders only for a running operation that has reported progress, so a queued operation, a failed one and a removal never carry it. A removal in particular sits at progress zero for its whole visible life, since the delete path reports none and its completion is filtered out. Both the strip and the card paragraphs now lead with what always shows and list the rest as conditions. The Cluster chip matches on a node list that finished operations do not carry, so a fan-out install leaves the chip once it reaches the record. Scoped that claim to the live sections. Two more of the same shape, found by re-reading each clause alone: the strip also appears for a failure, which is not running, and the four-second hold only applies when nothing replaces the operation that just finished. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): stop reporting a cancelled install as installed, and make queued real Three defects that all trace to one root cause: `isCancelled: true` is unreachable from /api/operations. Every writer of Cancelled=true also sets Processed=true, the handler skips Processed && Cancelled, and OpCache.GetStatus evicts a cancelled op before the handler iterates it. Cancelling the last running operation put a green "Installed model X" on the strip for four seconds: the completion hold was guarded by `!previous.isCancelled`, which is dead. A cancellation deletes the operation server side, so the strip sees exactly what it sees on a completion, and nothing in the payload separates the two. The signal now comes from the side that issued the cancel: the operations context remembers the job IDs it cancelled (pruned after a minute) and the strip asks before it holds anything. A cancelled operation goes as soon as it stops; the record already reports it as cancelled. isQueued was set only when the gallery status was missing, but markQueued publishes a "queued" status at admission, so a queued op has a status for its whole queued life and the state was unreachable outside a microsecond window. Every operation waiting behind a running install rendered as "Installing model X" with a spinner. The queued phase is now the signal, via an exported PhaseQueued and a nil-safe OpStatus.IsQueued() next to the writer. With those two fixed, the Cancelling state has no way to be entered: cancelling is instantaneous from the API's point of view. Its branches, CSS, locale key and the isCancelled field itself are removed rather than left for a future reader to assume they work. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): keep a removal a removal, and say what an install is doing OpStatus.Deletion was set once, at admission, and lost on the next status write: UpdateStatus replaces the whole status and only carried Nodes forward. Every later writer (the worker's first write, the progress ticks, the failure path) leaves the field at its zero value, so the flag survived only the queued window, and both surfaces test isQueued first. The reachable consequence is that a failed removal reported itself as a failed install, which is exactly the shape the Activity page offers Retry for, and Retry installs: pressing it on a removal that failed re-downloaded the model. A running delete also rendered as "Installing model X" with a spinner, and a successful one as "Installed model X". Carry Deletion forward the way Nodes already is. A job is a delete or an install for its whole life; an unset flag means "no new information", not "this is an install". Pinned by Go specs on both the service and /api/operations: the existing Playwright specs were green only because they stubbed a payload the server could not emit. Also restore the operation's own status message on the Activity card. Phases and byte counters exist only on the managed-artifact path, so a legacy files: gallery model and every backend install rendered a sub-row with nothing in it but the verb. The strip stays terse on purpose. And give the strip's name a min-width floor: overflow: hidden zeroes its automatic minimum, so a long error squeezed the name down to "mod…" and the identity of the thing that broke was the first thing lost. primaryOperation is made module-private: its comment claimed the Activity page selected the same operation, but that page shows all of them, partitioned into failed and running, and never imported it. Assisted-by: Claude Code:Opus 5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(activity): read the operations record from PostgreSQL The Activity page's record of finished installs and removals was a 50-entry in-memory ring per frontend replica. In distributed mode that is the wrong place for it: each replica keeps its own copy, a replica added by a scale-out or a rolling deploy starts empty and never backfills, and "Clear history" clears only the replica that served the request, so the record reappears on the next poll routed elsewhere. The data is already in gallery_operations. Read it from there. GalleryStore gains ListTerminal and ClearTerminal, sharing a lifted terminalStatuses set with CleanOld so there is one definition of "finished". ListTerminal orders by updated_at, when the operation reached its terminal status, because the record reports what finished and when. OpCache.History and ClearHistory dispatch on whether a store is wired, so the HTTP handlers and the OpRecord JSON shape are unchanged and the page needed no change. A failed store read falls back to the local ring rather than blanking the page, and ClearHistory empties the ring as well so a database blip cannot resurrect a record the admin just cleared. The name derivation in recordTerminal is lifted into operationDisplayName and used by both paths, so the ring and the store cannot name the same operation differently. Also fixes a pre-existing bug the store path made visible: the backend channel hardcoded op_type "backend_install" even for a removal, while the model channel derives model_install/model_delete from op.Delete. Both channels carry the same ManagementOp, whose Delete field the backend handler already branches on, so the backend channel now derives backend_delete the same way. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): keep a cancelled operation cancelled, and report a failed clear Review follow-up on the store-backed Activity record. A cancelled install was recorded as a failure. The cancel handler persists "cancelled" synchronously, then the handler goroutine unwinds with the context error and Start hands that to updateError unconditionally, which overwrote the row with "failed: context canceled". The page rendered a cancelled install as a red failure card offering Retry, with a raw context error as the reason. Fixed in GalleryStore rather than in Start, because an operation finishes once and the paths that retire one are not mutually exclusive: UpdateStatus now refuses to rewrite a row that already reached a terminal status. That also pins updated_at to when the operation really finished, which is the key the record is ordered by, and Create's upsert now freezes the same columns so a worker dequeuing an operation the admin cancelled while it was queued cannot reopen it as pending. ClearHistory returned nothing, so a failed delete logged a warning while the handler still answered 200. The admin watched the record clear and come back on the next fetch with nothing said about why. It now returns the error, the DELETE handler answers 500, and the store is cleared before the local ring so a failure leaves the fallback record intact rather than faking an empty one. Hydrate is the only reader that decides from op_type whether an operation is a removal, and it tested for "model_delete" exactly, so the backend_delete added in the previous commit hydrated as an install: a replica restarting during a backend removal rendered "Installing backend X". Both discriminations now go through IsDeleteOpType/IsBackendOpType so a fifth op_type cannot silently read as an install in whichever consumer was missed. Also: the backend channel now persists Cancellable as !op.Delete, matching the model channel; IsBackend falls back to the op_type prefix, since is_backend_op is only written by UpsertCacheKey and the rows needing the name fallback were reporting backend operations as models; and an unrecognized terminal status is logged rather than quietly filed as a success, which is what the comment already claimed. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): keep a reaped operation correctable by its real outcome The terminal-status freeze added in the previous commit was too wide. It froze "failed" alongside "completed" and "cancelled", and the stale reaper writes "failed" onto operations that are still going to run. The gallery worker is a single goroutine consuming both channels serially, so an operation queued behind a large download sits in "pending" with nothing bumping updated_at, and ReapStaleOperations gives up on it after 30 minutes. That used to be self-healing: the worker dequeued it, Create reset the row to "pending", and the operation reported its real outcome. With the freeze the row stayed "failed" forever while the install ran and succeeded underneath it: a red failure card offering Retry for a model that is installed, omitted from ListActive so no replica hydrates it, and no longer deduped cluster-wide by FindDuplicate. Freeze on ("completed", "cancelled") instead. That is all the cancelled-install fix ever needed, and it leaves a failure correctable by what actually happened. The set is separate from terminalStatuses, which ListTerminal, ClearTerminal and CleanOld all still want in full, because the two mean different things: a failure can be superseded by a real outcome, a completion or a cancellation is the real outcome. UpdateStatus now writes the error column unconditionally, so a corrected outcome drops the previous attempt's reason rather than being recorded as completed while still carrying "stale operation reaped" as its error. Also adds the route-level spec for the 500 branch of DELETE /api/operations/history, and trims a comment that credited the persisted cancellable column with more than it survives long enough to do. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): offer Cancel in the phase that can honour it The cancellable flag was set at both ends of an operation's life and was wrong at both, in opposite directions. A queued operation is cancellable whatever it is. EnqueueModelOp and EnqueueBackendOp select on the operation context, so cancelling one that is still waiting releases the delivery goroutine and abandonQueued retires it: the worker never sees it, nothing is downloaded, nothing is deleted. markQueued nevertheless wrote Cancellable: !deletion, so a queued removal reported cancellable: false and the UI hid the Cancel button in the one window where pressing it both works and leaves no trace. A removal queued behind a large install was stuck there until the install finished. A running removal is not cancellable at all. DeleteModel and DeleteBackend take no context, and modelHandler only checks the operation context after the call returns, so a "cancelled" verdict would land after the model was already gone. Both handlers nevertheless wrote Cancellable: true unconditionally at entry, ahead of the op.Delete branch, offering a Cancel button the server cannot honour. So the queued phase is more cancellable than the running phase, which is the reverse of the usual shape. markQueued now reports true unconditionally, and the handler-entry writes report !op.Delete. Both sites carry a comment saying why, because reading either one alone suggests the other is a bug. GalleryStore.Create keeps !op.Delete: it runs at dequeue, so its value already describes the running phase. Its comment now says so. Specs cover queued removal, queued install, running removal and running install through the handlers, plus the queued-removal case through /api/operations where the flag is consumed, plus the behaviour the whole asymmetry rests on: a removal cancelled while queued never reaches the worker and deletes nothing. No existing spec asserted the old values. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Write] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(activity): clamp the installer message, and add a real-binary e2e spec Running the page against a real local-ai showed the legacy installer message wrapping to three lines and dominating the card: it embeds an absolute file path, so it is both long and a single unbreakable token. One line, ellipsised, full text in the title, matching what the error string already does. The spec that found it runs with no route stubbing at all. Every other spec here stubs /api/operations, which is how a payload the server cannot emit (isDeletion true on a live operation) stayed green through a full review while the UI rendered a removal as an install. It is skipped unless LOCALAI_REAL_BINARY is set, so CI is unaffected. Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-28 19:33:48 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	f317da7c0f	fix(galleryop): make admitted operations queryable and survive a failed op (#11044) ... Two lifecycle defects observed on a 2-replica distributed cluster. The install endpoints mint a job UUID, hand the operation to an unbuffered channel, and answer HTTP 200 immediately. The gallery worker is a single goroutine that processes operations serially, and the first status write happens inside modelHandler/backendHandler — i.e. only once the worker actually starts the work. An operation queued behind a running install therefore had no status at all: GET /models/jobs/<uuid> answered "could not find any status for ID" and GET /models/jobs did not list it, so the endpoint reported success for work nothing could observe. On the paths that sent directly rather than from a goroutine, the same unbuffered channel blocked the HTTP handler for the whole duration of the in-flight install, which is how a replica came to accept no /models/apply at all while /readyz stayed green. Admission now goes through EnqueueModelOp/EnqueueBackendOp, which publish a "queued" status before handing the operation over, so a job ID is queryable from the instant it is handed out. Delivery selects on the operation's context, so cancelling a still-queued operation releases the delivery goroutine instead of stranding it on a send that will never be received, and an operation the worker never accepts becomes a terminal failure rather than a silent leak. The worker also had no panic containment. A panic in any handler propagated out of the single consumer goroutine and killed the process, taking every queued operation with it; it is now contained to the operation that caused it. The two ignored galleryStore.Create errors are logged, and the model and backend delete endpoints now run under the same ID they hand back — they previously ran under an empty ID and returned a status URL for a job that could never have a status. Second, an operation orphaned by a controller replaced mid-download kept reporting phase=downloading, processed=false, error=none while nothing was downloading. The PostgreSQL side does recover on its own (FindDuplicate ignores rows untouched for 30 minutes and CleanStale marks them failed), but the reaper only ever corrected the database. The in-memory statuses map that GET /models/jobs/<id> and /api/operations actually read was never corrected, so every replica kept serving the frozen tick indefinitely. ReapStaleOperations now reconciles the in-memory copy with the reap. Note that operation ownership is still not tracked: gallery_operations has a FrontendID column that nothing writes, so a live operation and one whose owner died are distinguished only by a 30-minute staleness timeout. Narrowing that window needs a lease/heartbeat mechanism and is out of scope here. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-23 00:02:23 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	ff299df453	perf(http): gzip responses, cache hashed assets, bound the trace endpoints (#11056) ... Three measured HTTP-layer regressions on a live deployment, fixed together because they all shape the bytes on the wire. 1. No compression. The server sent no Content-Encoding regardless of what the client asked for, confirmed with curl straight at 127.0.0.1:8080 so it was not an ingress artefact. Adds gzip middleware, on by default and configurable via LOCALAI_DISABLE_HTTP_COMPRESSION and LOCALAI_HTTP_COMPRESSION_MIN_LENGTH (default 1024 bytes so tiny bodies are not wastefully wrapped). Streaming routes are skipped explicitly: an SSE Accept header, a WebSocket upgrade, and the completion / SSE / log-tail path prefixes, because whether a completion request streams is decided by the request body, which the middleware runs too early to see. Already-compressed formats (woff2, png, mp4, ...) are skipped too; gzip made those marginally larger. Measured over the embedded React build: JS+CSS 2815 KB raw to 808 KB gzipped (3.48x). 2. No cache headers on content-hashed assets. Vite hashes the filenames, so a given /assets/ URL can never change content, yet they shipped with no Cache-Control, ETag or Last-Modified, and the browser re-fetched the whole bundle on every navigation with no conditional request available. /assets/* now carries public, max-age=31536000, immutable. index.html stays no-cache so a deploy is picked up, and the unhashed locale JSONs get a short TTL rather than the immutable one. 3. Unbounded trace endpoints. /api/traces returned 21,033,606 bytes in 4.65s and /api/backend-traces 3,471,682 bytes in 1.50s, and the admin UI polls both every few seconds. The ring buffer holds up to 1024 entries, each embedding full input_text payloads. Both list endpoints now take limit / offset / full, default to 50 entries, and strip the heavy fields (request and response bodies plus headers for API traces, body and data for backend traces) unless full=true. Every trace gets a process-lifetime ID and GET /api/traces/{id} and /api/backend-traces/{id} serve the full record, which is what the UI fetches when a row is expanded. The list body stays a JSON array; paging metadata rides in X-Total-Count, X-Trace-Offset and X-Trace-Limit. Reproducing the live shape in a test, the polled payload goes from 21,131,097 bytes to 7,201 bytes. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-22 22:51:25 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	d7e04dcc32	fix(openresponses): make responses visible and cancellable across replicas (#11000) ... In distributed mode the Open Responses store is process-local: a sync.OnceValue over a map behind an RWMutex. With several frontend replicas behind a round-robin load balancer, every request that lands on a replica other than the creator misses. Measured on a live 2-replica cluster (#10993): the same response id returns 200 on the creating replica and 404 on its peer, and a cancel on the peer returns 404 without ever invoking CancelFunc, so generation runs to completion on the other replica while the caller is told the response does not exist. previous_response_id chaining fails through the same lookup. Split the state by what can actually cross a process boundary: - Replicated: response metadata (request, response resource, owner, expiry, stream/background flags) via syncstate.SyncedMap, the same component finetune, quantization and agent tasks already use. A local miss in Get/FindItem now falls back to it and returns a read-only remote view, so polling and chaining resolve on any replica. - Delegated: cancellation. context.CancelFunc is a function pointer and exists only in the creating process, so a cancel that lands elsewhere is broadcast on responses.<id>.cancel and applied by whichever replica holds the function. The broadcast is fire-and-forget rather than request/reply: if the owner crashed or was scaled down nobody answers, and the handler must not block on a reply that will never come. The replicated status moves to cancelled either way, which is truthful, since a dead owner's generation died with its process. - Refused: streaming resume. The resume buffer is a byte log plus a live notification channel and cannot be replicated without shipping every token over the bus. A resume that reaches the wrong replica now returns HTTP 409 naming the owning replica via the new ErrResponseNotLocal, instead of an empty event list that looks like a finished stream. It is deliberately distinct from ErrOffsetLost, which means the owner's buffer evicted the requested events. Standalone deployments never call EnableDistributed and keep exactly the previous process-local behaviour. Fixes #10993 Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-20 23:06:33 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	65bdbc4ee3	fix(http): make /readyz reflect startup readiness, plus gitignore and coverage-ratchet fixes (#10989) ... * fix(http): make /readyz reflect startup readiness instead of always 200 /readyz was registered as a static handler returning 200 unconditionally, so it carried no information: it was green whenever it could be reached at all. Readiness could not distinguish "serving" from "still starting", and any future change that started the HTTP listener earlier would silently turn the probe into a lie. Track startup completion on the Application (atomic flag, flipped at the very end of New() on the success path only) and have the readiness handler consult it per request, returning 503 with a small JSON body while startup is in progress. A nil readiness source fails open so embedders keep the historical behaviour. /healthz is deliberately left readiness-independent. Liveness and readiness answer different questions, and failing liveness during a long preload makes an orchestrator restart the pod mid-download so the preload never finishes. This matters because since #10949 the startup preload materializes HuggingFace artifacts for managed backends: tens of GB for a large model (31 GB observed on a live cluster). Both probes stay in quietPaths and stay exempt from auth. Note the listener is still started only after New() returns, so today the not-ready state is not observable over HTTP. Moving the listener earlier is a separate, deliberate decision and is not made here. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * chore(gitignore): anchor the mock-backend pattern so its source dir is traversable The bare `mock-backend` pattern matched the *directory* tests/e2e/mock-backend/, not just the binary built into it. Git will not descend into an ignored directory even for tracked files, so `git add tests/e2e/mock-backend/main.go` required -f. This was hit while working on #10970. Anchor it to the artifact's full path. The built binary stays ignored (it is also covered by tests/e2e/mock-backend/.gitignore) while the source directory becomes traversable again. Verified with `git check-ignore -v`: a new source file under tests/e2e/mock-backend/ is no longer ignored, and the binary produced by `make build-mock-backend` still is. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * chore(coverage): raise the coverage ratchet from 48.5% to 54.2% The committed baseline had drifted well below reality: it still read 48.5% while a full instrumented run measures 54.2%. A stale-low baseline makes the gate meaningless — coverage could regress by more than 5 percentage points and still pass. Raising a ratchet is a deliberate act, not something to fold into an unrelated fix, so it gets its own commit. The headroom was earned by tests landed in #10946, #10947, #10948, #10949, #10956, #10967, #10968, #10970 and #10975. Measured with `make test-coverage` on this branch (the same instrumented run `make test-coverage-baseline` uses: ginkgo over ./pkg and ./core plus the in-process tests/e2e suite, --covermode=atomic, --coverpkg over core/... and pkg/..., generated protobuf excluded). The run completed with exit 0 and zero spec failures; the total was then written with the exact command the test-coverage-baseline target uses: go tool cover -func=coverage/coverage.out \ | awk '/^total:/{gsub(/%/,"",$NF); print $NF}' > coverage-baseline.txt Verified afterwards with scripts/coverage-check.sh, which reports OK. Note the measured figure includes the readiness specs added earlier on this branch, so it is a demonstrated floor rather than an estimate. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-20 21:45:27 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	83a0f16a21	feat(gallery): let one gallery entry offer several builds of the same model (#10943) ... * feat(system): expose raw detected capability for model meta resolution Model meta gallery entries express hardware fallback through candidate ordering rather than a capability map, so they need the undecorated detected capability string without Capability's default/cpu fallback chain. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(system): drop duplicate capability accessor, cover DetectedCapability ReportedCapability was added with a body identical to the existing DetectedCapability. Keep one accessor and move the specs onto it, since DetectedCapability had no direct coverage of its no-fallback behavior. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): parse IEC binary size suffixes (KiB..PiB) ParseSizeString accepted only SI suffixes, so a "20GiB" floor was rejected outright. Model and VRAM sizes are conventionally quoted in IEC units, and silently reading GiB as GB would understate a floor by about 7%. Purely additive: these inputs previously returned an unknown-suffix error. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add Candidate type for meta model entries Candidate is one option in a meta entry's ordered variant list. It names a concrete gallery entry and declares when that entry suits the host. EffectiveMinVRAM resolves the VRAM floor, letting an authored min_vram win over a nightly-inferred one. An unparseable floor errors instead of being treated as absent: swallowing a typo would turn a constrained candidate into an unconstrained one and select a too-large variant rather than fail loudly. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add hardware-aware model variant resolver Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): allow gallery model entries to declare variant candidates A gallery entry with a non-empty candidates list is a meta entry: it names an ordered list of concrete entries and resolves to the first one the host can satisfy, instead of describing model files directly. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): resolve meta model entries to hardware-appropriate variants at install Meta gallery entries carry an ordered candidate list; at install time the first candidate the host satisfies is resolved and its payload installed under the meta's name, so the model keeps a stable name regardless of which variant backs it. The resolution is recorded in the installed gallery config so a reinstall honors a prior pin and operators can see the backing variant. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): key meta pin recall on the installed name and detach resolved entries Six review findings on the meta-entry install path. Pin recall was keyed on the gallery entry name while applyModel writes the record under the install name (req.Name when supplied), so a meta installed under a custom name with a pin lost that pin on reinstall and was silently re-resolved onto a different variant, possibly swapping its backend. Compute the install name with applyModel's own precedence before the recall. ResolveMetaModel returned a shallow struct copy, so the resolved entry's Overrides aliased the gallery entry's map and the install path's in-place mergo merge wrote the caller's request into the shared catalog. Detach Overrides, ConfigFile, AdditionalFiles, URLs and Tags. Not exploitable today only because this path re-unmarshals the gallery per call, which is a property nobody should have to rely on. Also: overlay the meta's name onto the persisted config for meta installs so the gallery file no longer records the variant's name; move the pinned-VRAM warning below the variant validation so a pin naming a nonexistent entry does not warn about VRAM before failing for an unrelated reason; and stop seeding config.URLs in the config_file branch, which duplicated every declared URL. Add seven network-free specs driving InstallModelFromGallery with a meta entry: variant payload wins over the meta's legacy url fallback, the resolution record round-trips to disk, a pin is recorded and honored on reinstall including under a custom install name, and the resolved entry does not alias the gallery's maps. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): deep-copy meta overrides and make two specs functional ResolveMetaModel detached the resolved entry's Overrides and ConfigFile with maps.Clone, which only copies the top level. Gallery overrides are nested in practice (parameters.model is near-universal) and the install path merges the caller's request with mergo.WithOverride, which recurses into nested maps and overwrites them in place, so the gallery entry's own inner maps were still reachable and still got rewritten by the last caller to install. Copy both maps all the way down instead, recursing through the container shapes a YAML decoder produces. ConfigFile is not mutated on the install path today, but it carries the same kind of nested payload and leaving it shallowly cloned would invite the bug back. Also fix two specs that passed whether or not their target fix was present: - "does not write the caller's overrides back into the gallery entry" re-read the catalog from disk, which re-unmarshals fresh structs and so cannot observe in-memory aliasing. It now asserts against the in-memory gallery entry and drives the real mergo merge. - "round-trips the resolution record to disk under the meta's name" asserted a name that is already correct in the config_file branch. It now drives the url branch via a file:// fixture, where the meta-name overlay actually applies. Both were verified red by reverting their fix. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(gallery): lint meta model entry invariants in index.yaml Adds Ginkgo specs that parse the shipped gallery/index.yaml and enforce the invariants that keep meta entries safe: a legacy url fallback equal to the final candidate's url, references only to existing non-meta entries, a min_vram floor on every candidate but the last-resort one, a capability drawn only from the vocabulary the system can report, and descending VRAM floors within a capability group. The capability check is the only compensating control for a typo there. Candidate matching is a case-sensitive exact comparison against SystemState.DetectedCapability(), so an unknown value never matches and falls through silently instead of erroring. The vocabulary therefore mirrors the raw return set of getSystemCapabilities(), which notably excludes "cpu": that is a fallback key inside Capability(capMap) on the meta backend path, never a reported capability. A CPU-only host reports "default". These pass vacuously until the pilot meta entry lands; the guard is intentionally in place before the thing it guards. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(gallery): close coverage gaps in the meta entry lint The ordering invariant grouped candidates by capability and asserted floors descend within a group. A candidate with an EMPTY capability matches every host, so it does not belong in its own group: it dominates every later candidate whose floor is at or above its own, across capability groups. Track a running minimum floor over the unconditional candidates instead, which subsumes the old same-group check for the empty capability. Every spec skipped non-meta entries, so with zero meta entries in the index all five bodies were no-ops. Aligning GalleryModel.IsMeta() with GalleryBackend.IsMeta(), whose semantics are deliberately opposite, would have made all of them pass while checking nothing. Extract each invariant into a helper over a slice of entries returning the violations it finds, and cover those helpers with synthetic fixtures so the logic stays tested at zero meta entries. The index-driven specs are now a thin application of already proven logic. Also assert the index parses non-empty, report every violation in one run rather than aborting on the first, and parse the index once for the suite. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(gallery): add nightly denormalization of meta model candidates Fills the read-only backend, quantization and inferred_min_vram fields on meta gallery candidates and opens a PR, modeled on the existing checksum_checker job. Computing these needs network access, so it happens nightly rather than at install time. An authored min_vram is never modified: a human who measured a real load knows more than a pre-download estimate does. The index is rewritten via yaml.Node rather than a document round-trip. A full round-trip reflows all ~26k lines of gallery/index.yaml, which would bury the computed values and make the nightly PR unreviewable. The rewrite touches only the three derived keys, so authored styling survives and a run that computes nothing leaves the file untouched. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ci): keep the gallery denormalize diff reviewable and self-healing The nightly denormalization job edits YAML nodes instead of round-tripping structs so its PR stays small enough for a human to review, but the write path undid that: yaml.Marshal re-encoded the node tree at yaml.v3's default 4-space indent and dropped the leading document marker, reflowing roughly 6000 lines around the handful of real changes. Encode through yaml.NewEncoder at the index's authored 2-space indent and restore the header. A write that changes three fields now changes three lines. Stale inferred_min_vram values were also never cleared. Both skip paths (an authored min_vram is present, or the candidate is the last resort) returned before touching the field, so a candidate that gained a floor or became the last resort after a reorder kept an inferred value that EffectiveMinVRAM reported as a real constraint, failing the meta lint with no way for the job to self-heal. Clear the field before both skips. The workflow discarded a whole night's work on any single failure: the program exits 1 when a candidate cannot be estimated, which aborted the job before the PR step, so one unreachable candidate blocked every other refresh indefinitely. Capture the status, open the PR with what was computed, mark the PR body as partial, and fail the run afterwards so the problem still surfaces. Also preserve the index's existing file mode instead of forcing 0644, and drop the redundant //go:build ignore tag, since Go already skips dot directories and the sibling modelslist.go carries no tag. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add nanbeige4.1-3b meta entry with hardware-resolved variants Adds the first real meta entry to the gallery index. It resolves to the Q8_0 build on hosts with at least 6GiB of VRAM and to the Q4_K_M build everywhere else, installing either payload under the stable name nanbeige4.1-3b. The entry carries a url equal to its final candidate's url. LocalAI releases that predate candidates support parse the index non-strictly and drop the key silently, so without that url they would list the entry and install nothing. A regression spec parses the index the way those releases do and asserts every meta entry stays installable for them. Also teaches core/schema/gallery-model.schema.json about candidates. The schema sets additionalProperties: false at the top level, so an author following CONTRIBUTING.md and adding the yaml-language-server comment would otherwise get a validation error on this entry. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): make candidate entries complete, installable entries Reworks hardware-resolved gallery variants after a design pivot. There is no longer a separate "meta" entry kind. A gallery entry is a normal, complete entry that may additionally carry candidates:, a list of hardware-gated upgrades over itself, and the entry is itself the last-resort candidate. The previous design relied on a bare url: as the fallback for LocalAI releases that predate candidates support. That fallback is empty in practice: none of the 80 gallery/*.yaml files carry a top-level files:, and 1216 of 1281 index entries carry their payload in the index entry itself, so a url alone yields a config template with nothing to download. Since every released LocalAI reads gallery/index.yaml live from master, merging a payload-less entry would have shown every existing user a model that installs to a broken state. Making the entry its own base candidate removes the problem at the root: old clients drop the candidates key and install the entry exactly as they do today. Resolution order is now explicit pin, then capability plus VRAM over the declared upgrades, then the entry itself. The entry ALWAYS installs: when its own min_vram or capability is unmet the installer warns and installs it anyway, because there is nothing below it and refusing would make the gallery behave worse the newer the client is. A pin naming the entry's own name is valid and is how an operator declines an upgrade. IsMeta() becomes HasCandidates(), ResolveMetaModel becomes ResolveVariant, and the persisted meta_name record key becomes entry_name. GalleryBackend.IsMeta() is a separate concept and is untouched. The lint drops the three rules the pivot makes wrong (url equality with the final candidate, no inline payload, unconstrained final candidate) and gains one: the entry's own floor must sit strictly below every candidate's, since a base that outranks a candidate makes that candidate unreachable. The pilot entry is now the existing nanbeige4.1-3b-q4, which gains a 2GiB floor of its own and a single 6GiB upgrade to nanbeige4.1-3b-q8, replacing the separate nanbeige4.1-3b entry added in d0d441bb4. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): select model variants by hardware fit, not authored order Gallery entries could already carry a list of alternatives, but selection was an authored, ordered, first-match policy: every candidate declared a `capability` string and the VRAM floors had to descend in a hand-tuned order. That pushed hardware knowledge onto whoever edits the gallery and made ordering load-bearing, so a reordered list silently changed what users installed. None of it was necessary. SystemState.IsBackendCompatible already derives hardware support from a backend name alone: it knows MLX and metal are Darwin-only, CUDA is NVIDIA-only, ROCm AMD-only, SYCL Intel-only. Selection can read that instead of asking authors to restate it. Authoring is now just a list of names: - name: qwen3.6-27b min_memory: 4GiB variants: - model: qwen3.6-27b-mlx-8bit - model: qwen3.6-27b-gguf-q8 min_memory: 28GiB and all the intelligence moved into the selector. Given a host it drops the variants whose backend cannot run here, drops those whose known memory requirement exceeds what the host has, and takes the LARGEST of what is left, because a bigger footprint is a higher quality quantization of the same model. A variant of unknown size is kept, since nothing proves it does not fit, but it ranks last so a proven fit always beats a guess. An explicit pin still wins outright, and if nothing survives the entry installs its own payload: the base always installs, this never refuses. Available memory is VRAM when a GPU was detected and system RAM otherwise, read through xsysinfo so a cgroup limit is honored and a container gets its own limit rather than the node's RAM. Capability disappears entirely, from the types, the schema and the lint. VRAM and RAM collapse into one `min_memory`, because a model's footprint is roughly the same wherever it lives and one figure is compared against whichever applies. The lint rules about ordering, the capability vocabulary and floor relationships are deleted with the hazards they described; what remains is that every variant names an entry that exists and does not itself declare variants, plus that any memory figure actually parses. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * gallery: size model variants with a live probe, drop the nightly denormalizer Selection needs each variant's size to decide whether it fits and to rank largest-first. That figure was written into the index by a nightly job, which made the gallery carry a derived value that could drift from the entry it was derived from. Derive it at install time instead. pkg/vram already sizes a model without downloading it, and the gallery UI already uses it: a remote GGUF header range-fetch, then an HTTP HEAD for the content length, then any declared size:. It caches its results, so reuse it rather than writing a second probing path. A probe failure must never fail an install, so an unprobeable variant is treated as unknown: it survives the memory filter, because nothing proves it does not fit, and it ranks last, so a known-good fit always beats a guess. If every probe fails, selection still terminates on the base entry. The probe is injected through ResolveEnv rather than called directly, for the same reason the backend compatibility check is: specs pin an exact size, or an exact failure, without reaching the network. With that in place three things are dead weight and go: - The nightly job and the fields it populated. Variant.Backend was redundant because the backend is resolved live from the referenced entry during selection, and Quantization was display-only that nothing read. - min_memory on the base entry. The base always installs and its floor could only warn, so it could not change any outcome. - The lint rules and schema entries for both. min_memory on individual variants stays, as the override for when the probed size is wrong. An authored figure now suppresses the probe entirely rather than merely outranking it, so it costs no round trip. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): expose model variants for selection over API, CLI and MCP A gallery entry may carry `variants:`, alternative builds of the same model. Selection already worked at install time, but nothing could see what an entry offered or ask for a specific build, so the feature was undrivable. Listing: `GET /api/models` now reports `variants` and `auto_variant` for the entries that declare variants. Each variant carries its resolved backend, its measured size and whether it fits this host. `auto_variant` is what installing without a choice would pick right now. The new gallery.DescribeVariants runs the same variantOptions + SelectVariant pass the installer runs, so the reported default cannot drift from what installing actually does, and HostResolveEnv is extracted so both derive the host and share pkg/vram's probe cache from one place. Performance: an entry that declares no variants returns early without touching the probe, so the ~1280 ordinary entries cost exactly what they cost before. Selection: `variant` is accepted on POST /models/apply, as a query param on POST /api/models/install/:id, on the gallery apply file/string request, as `local-ai models install --variant`, and as a parameter on the install_model MCP tool (both the httpapi and inproc clients). Empty means auto-select. An unknown variant name now fails the install naming what was requested. This closes a real hole: an entry declaring no variants short-circuits before selection runs, so a requested variant was previously dropped silently and the install reported success. startup.InstallModels ends in a variadic model list, so install options could not be appended to it; InstallModelsWithOptions is added alongside and InstallModels delegates to it. No caller signature changed. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): drop the redundant variant min_memory field Variant.MinMemory was an authored override for when the live probe misreads a variant's footprint. It duplicated an existing field: probeEntryMemory already passes the entry's declared size: into EstimateModelMultiContext, whose cascade prefers that declared size over its own guesswork. Correcting size: on the referenced entry fixes the figure for every consumer rather than only for variant selection, so min_memory shadowed the right answer. A variant is now nothing but a name. Its effective size is exactly the probe result, and an unknown stays unknown: it survives the filter and ranks last. EffectiveMemory loses its error return along with the field. The authored string was the only thing that could fail to parse, so the error had no remaining source and was propagating dead nil-checks through SelectVariant, DescribeVariants and the pin warning. Selection behaviour is unchanged. The specs covering probe-derived sizing, ranking, filtering, the unknown-size path, pin recall, entry/variant metadata split and deep-copy isolation all survive; the three install specs that needed a definite size now declare it through the referenced entry's own size:, which exercises the documented escape hatch directly. gallery/index.yaml is untouched: no entry ever carried the key. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): rank the entry's own build against its variants Variant selection pulled the declaring entry's own payload, the base, out of the candidate set and consulted it only once every declared variant had been rejected. Two real failures followed. A variant whose size the probe cannot determine deliberately survives the memory filter, because nothing proves it does not fit. As the only survivor it then won outright on any host, however small: a 2GiB machine installed an unmeasured variant in preference to the 4GiB build the entry itself ships, with no warning. 241 of the 1280 current index entries carry no files and no size, which is exactly that shape. "Largest wins" also broke whenever the base was the largest. An author writing a Q8 entry that offers a Q4 downgrade for small hosts, a natural shape that nothing in the lint, schema or docs discourages, had the Q4 installed on every large host instead. Make the base an ordinary participant. It is still exempt from both filters, so selection always terminates on something installable, but it is now ranked against the variants: a proven fit first and largest, then the base, then any variant whose size nothing could measure. Both failures disappear together. The base is probed for its size accordingly, which it was not before, because an unsized base would lose every contest to an unmeasurable variant. FellBackToBase is kept but narrowed to "no declared variant survived", rather than "the base was chosen", since the base now also wins on merit and that is not worth warning about. A recalled variant pin also became a permanent install failure. A pin the caller supplies on this request must stay fatal, but one recalled from ._gallery_<name>.yaml can be invalidated by any later gallery edit, and failing on it turned one rename into a model that could never be reinstalled or upgraded again short of deleting a dotfile the user has never heard of. A stale recalled pin is now dropped with a warning naming it, and selection runs as if it had never been recorded. Also drop the last textual reference to two abandoned designs from the DetectedCapability comment, correct the documented variants JSON example, which showed a memory_bytes of 0 that omitempty makes impossible, and remove an em dash from the install skill. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): budget variant memory from RAM when a GPU reports no VRAM Variant selection read its memory budget from VRAM whenever a GPU capability was detected, and from system RAM only when none was. Apple Silicon satisfies the first branch and fails the premise: arm64 macs report the metal capability unconditionally, without probing anything, while TotalAvailableVRAM has no discrete VRAM pool to find and returns zero. The budget therefore came out as zero on every Mac. Zero drops every variant carrying a known size, so the base build was installed on all of them however much memory the machine had. The feature was inert on the platform, and silently: falling back to the base is a legitimate outcome, so nothing looked wrong. Take VRAM only when it is actually a number, and fall back to RAM otherwise. On a unified-memory host RAM is not an approximation of the budget, it is the budget, since the GPU shares it. A discrete GPU whose VRAM could not be read also lands on RAM, which overstates what the card holds but understates nothing the host has; the previous zero understated both. An unreadable RAM figure still yields zero and still installs the base, so a genuinely unknown host is not talked into a larger download. This is what turned tests-apple red: "installs a fitting variant's payload under the entry's own name" asserts on selection, and the runner resolved to the base because its budget was zero. The added specs pin the branch directly rather than relying on a macOS runner to notice again. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add a model variant picker to the models gallery PR #10943 shipped the server side: a gallery entry may declare `variants:`, `GET /api/models` attaches `variants` and `auto_variant` to declaring entries, and `POST /api/models/install/:id` accepts a `variant` query parameter. Nothing in the UI consumed any of it, so the feature was not reachable from the browser. This wires it up. modelsApi.install takes an optional second argument and appends an encoded `?variant=` only when one is given, so every existing call site keeps sending exactly the request it sent before. On the models table, an entry that declares variants gets a split button. The primary Install still installs the auto-selected build, because auto is the default and the point of the feature; the chevron opens a menu for a deliberate override. It follows the Backends.jsx precedent: one shared Popover re-anchored per row, rendering .action-menu items, which brings Escape, outside-click and focus return along with it. An entry that declares no variants renders exactly as it did before. A variant that does not fit is dimmed but stays selectable, since the server honors an explicit choice with a warning rather than refusing it. memory_bytes is omitempty on the wire, so an absent key means the size is unknown and never zero. A single helper guards both the menu and the detail row, because formatBytes would otherwise render a falsy value as "0 B", which reads as "needs nothing". The expanded detail row gains a Variants section listing each build's backend, size, whether it fits, which is the entry's own build, and which one auto-selection would pick, built from the existing DetailRow helper and .badge classes. Eight Playwright specs cover the picker, including that plain Install sends no variant parameter and that choosing one sends it. One pre-existing assertion was scoped with .first(): the Variants section legitimately adds more llama-cpp badges to the detail row, which tripped strict mode. UI line coverage 49.42% -> 49.36% against a 40.0 baseline and 0.8pp tolerance; branch coverage rose 72.04% -> 72.66%. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): describe model variants from a companion endpoint Variant description probes each referenced entry's weight files over the network: an HTTP HEAD plus a ranged GET, serial, five seconds per probe with no aggregate deadline. Running it inline in GET /api/models made one listing cost (entries x variants) round trips. The Manage page fetches with items=9999, so at 200 declaring entries that is ~1000 serial probes, minutes of a blocked handler and gigabytes of range traffic for a single page load. Only one entry declares variants today, but the feature exists so that many will. Follow the precedent already set for VRAM estimates. The listing now reports only has_variants, a length check on loaded metadata that touches nothing, and GET /api/models/variants/:id returns the description for one entry, mirroring estimate/:id in route shape, auth and error handling. DescribeVariants itself is unchanged; only its caller moved. The picker fetches lazily at the two points where a user asks to see variants, opening the split-button menu and expanding the detail row, and caches per entry for the page session. An entry declaring no variants issues no request at all. A spec counts real HTTP hits on the weight files, so it goes red if description becomes reachable from the listing path again through any caller. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): filter the model gallery to entries that declare variants The gallery is heading towards showing parent entries and hiding the individual builds they reference, so a user sees one row per model rather than six quantizations of it. Adoption is a single entry today, so defaulting to that would leave a one-row gallery. This ships the migration-phase inverse instead: the default is untouched, and a toggle narrows the list to only the entries that declare variants. It previews the end state and changes nothing until someone asks for it. The filter is server-side, next to term/tag/backend/capability and above the pagination arithmetic. The listing paginates at 9 items, so narrowing on the client would leave totalPages and availableModels describing the unfiltered set and hand the user empty pages. It selects on HasVariants(), which reads already-loaded metadata, so it issues no variant probes. The parameter is named has_variants after the listing field it selects on, and is compared against "true" like the other boolean query params (all_users, save_checkpoint), so has_variants=false reads as absent. With it omitted the response is byte-for-byte what it was before. The control is the shared Toggle component, matching the fitsFilter toggle already on this page: same wrapper class, same icon and label shape, same localStorage persistence. Unlike fitsFilter it resets to page 1 on change, which a server-side filter has to do. Stacking the toggle with a tag or backend filter easily yields nothing while one entry declares variants, so the empty state now names the variants filter as the cause rather than leaving a user to conclude the gallery is broken. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): render gallery model descriptions as Markdown Gallery descriptions are Markdown, but the React UI dumped them raw, so a model whose description opens with an ATX heading showed a literal "# Qwen3.6-27B [](https://chat.qwen.ai)" in the list. Full-description areas now render through renderMarkdown (marked + DOMPurify), matching how Backends.jsx and the Manage detail panels already handle the same content: - Models.jsx expanded detail row - VoiceLibrary.jsx voice detail header The truncated one-line previews must not render block Markdown: a leading "#" would become an <h1> and wreck the row height and rhythm. They get a new stripMarkdown() helper instead, which reduces Markdown to a single line of readable plain text. It is used for the cell text and for the title tooltip, since a tooltip full of "[](url)" is no better than a cell full of it: - Models.jsx gallery table description cell - Manage.jsx model and backend resource-row descriptions stripMarkdown walks marked's lexer output rather than running regexes over the source, so what it strips is by construction what renderMarkdown would have rendered, and it needs no new dependency. Output lands in JSX text nodes, so React escapes it; no new dangerouslySetInnerHTML beyond the two full-description sites, both of which run DOMPurify. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ui): strip Markdown from the backends table description cell Commit b35d630cf fixed this for gallery models but left the Backends admin page with the same asymmetry: its detail panel renders the description through renderMarkdown, while the collapsed table row dumped the raw gallery string into both the cell body and the title tooltip. That is user-visible. 40 of the 949 entries in backend/index.yaml carry Markdown - insightface uses inline code backticks, others use lists and links - and backend descriptions also contain embedded newlines, so the one-line cell showed literal syntax. The cell now runs stripMarkdown over the description once and uses the result for the text and the title, matching Models.jsx and the ResourceRowDesc component in Manage.jsx. The '-' placeholder is preserved, and now also fires when a description reduces to nothing after stripping. The detail panel is untouched and no new dangerouslySetInnerHTML is introduced: stripMarkdown output lands in a JSX text node, so React escapes it. Three Playwright specs cover it: a description with a heading, inline code and a link renders as clean text with no literal syntax and no block element in the cell, the title tooltip carries the same stripped text, and a backend without a description still shows the placeholder. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ui(models): polish the variant detail view and scope rendered Markdown The gallery detail pane rendered every field through the same two-column label/value row, including the description. Multi-paragraph prose in a value cell ran eight rows tall at the top of the pane on a ~1200px measure, breaking the grid's rhythm exactly where the eye enters. Move it into its own full-width block above the table, capped at a 68ch measure, keeping the label. Rendered Markdown had no scoped typography anywhere in the app, so a description opening with `#` inherited the browser default 2em inside a 13px surface while a `##` further down was indistinguishable from body text. Add a reusable .markdown-body block mapping h1-h6, paragraphs, lists, links, code, blockquotes, images and tables onto the existing type scale, and apply it to every renderMarkdown() consumer: the models detail, the backends detail, both Manage details and the voice library detail. Rebalance the variants list so the name leads. Backend and size drop from badge/secondary weight to muted metadata; the FITS badge goes entirely, since it was true of nearly every row and so said nothing, while the variant that does not fit keeps a warning badge and a dimmed name. AUTO-SELECTED stays marked because it answers what a plain Install produces. Rows share the parent's grid tracks via subgrid so name, backend, size and status line up down the list instead of raggedly following name length. Finally, make each variant row actionable. It looked like a list of choices but was inert text, with per-variant install hidden behind the split-button chevron elsewhere; each row is now a button onto the existing handleInstall(modelId, variant) path, with hover, keyboard focus and a disabled state while an install is in flight. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): collapse the listing to one row per model The listing supported has_variants=true, which narrowed to entries that DECLARE variants. With adoption at three entries that showed three rows, which is useless; it was always a placeholder. Replace it with the view that is actually useful: the deduplicated gallery. Show every entry installable in its own right and nothing twice, which means the parents plus every entry nobody references, and hide only the builds another entry already offers as a variant, since those are reachable through their parent. The parameter is renamed to collapse_variants accordingly: the filter is no longer a predicate on a row's own metadata but a view over the whole gallery. Default stays off, so the response with the parameter absent is unchanged. VariantReferencedIDs never reports an entry that declares variants of its own, so parents are always visible. That guarantees every hidden entry has a visible entry offering it, and no chain can strand a row. Variant resolution already refuses to install such a reference, but the listing has to stay coherent in the presence of a gallery that has one rather than silently swallowing entries. Self-references and dangling references hide nothing. The referenced set is computed over the whole gallery rather than over what the other filters left, so an entry is hidden because a parent offers it and never because of what the user searched for. The pass is over metadata already in memory: it resolves nothing over the network and triggers no variant description or size probe, so the listing's zero-probe contract still holds. The UI toggle keeps its behaviour (persistence, page reset, clear filters) and becomes "One row per model", which says what the user gets. Its localStorage key moves too, since the stored value meant a different filter. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): show the collapsed model listing by default The gallery listing is what a user reaches for to answer "what can I install". Answering that with several rows for the same model, one per build, makes the reader do the deduplication the collapsed view already does, so the collapsed view is the one to land on. The UI now asks for collapse_variants=true unless the toggle says otherwise. The server default is deliberately untouched: a request with the parameter absent still returns the full listing, because other API clients depend on that response and collapsing it under them would be a breaking change. Opting out omits the parameter rather than sending false, so it asks for exactly the listing everyone else gets. The stored preference changes vocabulary from '1'/'0' to 'on'/'off'. The previous build wrote it from an effect that runs on mount, so a stored '0' recorded that the page had been opened rather than that anyone chose the expanded view, and honouring it would pin every earlier visitor to a default they never picked. Only the new vocabulary counts as a choice; a legacy '1' meant the collapsed view and is what the new default gives anyway, so no earlier deliberate choice is lost. Collapsing being the default also changes what the empty state may say about it. An opted-into filter can be named as the cause of an empty result; a default cannot, so the filters keep the top line and the collapsed view drops to a hint below it, shown only once filters are narrowing the set. For the same reason "Clear filters" now restores the collapsed default instead of switching it off, and the toggle alone no longer counts as a filter worth offering to clear. The label stays "One row per model": it describes the view the user is looking at rather than an action, so it reads the same whether it is opted into or out of. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * gallery: group alternative builds of the same weights under variants Sweep the gallery for entries that are alternative builds of the same weights (different quantization, precision, or runtime format) and declare them as variants of a single parent row, so the listing offers one row per model instead of one row per quantization and the installer picks the largest build that this host can actually run. 41 families over 95 entries, turning 54 entries into variants. The parent is the bare-named entry wherever one exists, so nothing changes about what any existing entry installs. Ranking already selects the largest fitting build regardless of which entry is nominally the parent, so the parent only decides the pathological case where nothing fits. For the ten families that have no bare-named entry, the smallest build is the parent, since that is the one that has to install when nothing fits. Grouping was verified against the actual model filenames rather than the entry names alone. Different parameter sizes, languages, finetunes, and products that merely share a name prefix are left as separate rows: the qwen3.6 APEX and pi-tune finetunes, the DFlash and MTP speculative-decoding pairings, English-only versus multilingual Whisper, the QAT versus non-QAT Gemma 4 weights, and the abliterated FLUX build are all distinct models. Six parents define YAML anchors that other entries pull in with a merge key, which would have handed their variants to every merging child. For the two depth-anything anchors that would have made fourteen unrelated entries advertise the base model's builds as their own. All 26 merging children therefore carry an explicit empty variants list, which overrides the merged key and is equivalent to the key being absent. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): rank model variants by host backend preference Variant auto-selection filtered candidates by whether their backend can run on the host, then ranked the survivors by size alone. The backend never influenced the choice beyond that gate, so a Mac offered both an MLX build and a llama.cpp build kept neither filtered and installed whichever was larger, leaving the native accelerated runtime unused. The same held for CUDA against CPU on NVIDIA and ROCm against Vulkan on AMD. Rank by the host's backend preference between the fit tier and size: fit stays a filter, preference decides among the builds the host can equally hold, and size still separates builds on equally preferred runtimes. The preference data stays in one declarative table in pkg/system, now read by a prefix lookup instead of a switch, so adding a capability or reordering one host's runtimes is a one-line edit and the gallery's ranking code carries no per-backend branching. MLX joins the metal rule ahead of metal itself, which is inert for the existing alias-resolution consumer because no alias group holds a candidate named for mlx. An unrecognised backend, an unrecognised capability and an absent preference list all collapse to the previous size-only ordering rather than erroring or dropping candidates. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): rank variants by engine name, not backend build tag Variant auto-selection ranked candidates with SystemState.BackendPreferenceTokens, but that function and the variant ranker speak different vocabularies. BackendPreferenceTokens returns BUILD TAGS ("cuda", "rocm", "sycl", "vulkan", "metal", "cpu"). It exists to match installed backend build directory names like "llama-cpp-cuda-12" during alias resolution in ListSystemBackends. Variant ranking instead matches a gallery entry's `backend:` value, which is an ENGINE NAME: "llama-cpp", "vllm", "vllm-omni", "sglang", "mlx" and the rest. No engine name in gallery/index.yaml contains "cuda", "rocm", "sycl" or "vulkan". preferenceRank matches by substring, so on an NVIDIA host the tokens [cuda, vulkan, cpu] matched neither "llama-cpp" nor "vllm", every candidate scored identically and size alone decided. The NVIDIA, AMD, Intel, darwin-x86 and vulkan rules were all inert. Only metal appeared to work, and only because the token "mlx" happens to equal an engine name. The mismatch does not error, it silently deletes the feature. Separate the two vocabularies. backendBuildTagPreferenceRules keeps the build tags and its original output for every capability, including metal, whose "mlx" token is removed again; its alias-resolution consumer is byte-identical to before. engineNamePreferenceRules is new, holds engine names, and is read by the new EnginePreferenceTokens, which HostResolveEnv wires into the renamed ResolveEnv.EnginePreference. Both tables sit adjacent under one block comment naming each vocabulary and each consumer, and share one lookup helper so their semantics cannot drift. On NVIDIA the order is vLLM, then SGLang, then llama-cpp: vLLM is the throughput engine and a model published with a vLLM build is published that way because that build is the one worth running. AMD and Intel get the same order, since rocm and intel builds of both serving engines ship. Metal prefers mlx over llama-cpp. Vulkan prefers llama-cpp, the only LLM engine with a Vulkan build. darwin-x86 and unknown capabilities are deliberately absent rather than guessed at, degrading to the size-only ordering that predates preference. preferenceRank stays generic and names no engine and no capability, so adding a runtime remains a one-line table edit. Specs pin the NVIDIA and metal rules through the live table and the real HostResolveEnv wiring, so emptying the engine table or wiring the build tag source back in both go red. A regression table asserts BackendPreferenceTokens' original output per capability, and mirrored locks assert neither table carries the other's vocabulary. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: record that variant selection ranks by engine before size A gallery entry can now declare variants, and selection ranks the builds a host can run by engine preference before size. Nothing told a contributor adding a backend that engineNamePreferenceRules exists, so a new engine would silently rank below every known one and lose to whatever build happened to be larger on hosts where it should have won. Document the step where a backend is added, warn against the sibling backendBuildTagPreferenceRules table (build tags, not engine names: the wrong table matches nothing, scores every candidate equally and disables the preference without erroring), and index it from AGENTS.md. Fix the authoring and user docs, which still claimed the largest surviving build wins. An author grouping builds under one entry has to be able to predict what a user gets, and size alone no longer decides it. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(cli,mcp): describe variant auto-selection as preference before size The CLI flag help and the install_model tool schema both still said auto-selection takes the largest build that runs. Ranking now puts engine preference ahead of size, so on NVIDIA a vLLM build wins over a larger llama.cpp one. An assistant reading the old schema would tell users the wrong thing. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): prefer llama.cpp over GPU serving engines on hosts with no GPU engineNamePreferenceRules had no row for the "default" capability, which getSystemCapabilities() returns both when no GPU is detected and when a GPU is present but under the 4 GiB VRAM floor. A missing row yields an empty preference list, which preferenceRank reads as "score everything equally", collapsing variant selection to size alone. That would be harmless if the hardware filter dropped GPU serving engines on such a host, but it does not. IsBackendCompatible derives support from the engine NAME, and "vllm" and "sglang" contain none of the darwin, cuda, rocm or sycl tokens it keys on, so they fall through to its closing "return true". A vLLM variant therefore survives on a CPU-only box and wins whenever its build is the larger of the two on offer: the machine installs vLLM in preference to llama.cpp. darwin-x86 had the identical hole. It was documented as a deliberate omission because nothing accelerates on an Intel Mac, which is true about acceleration and wrong about consequence: with every engine tied, download size decides. Add rows for both putting llama-cpp first. The GPU engines are enumerated behind it rather than left unmatched: an unmatched engine already ranks below every listed one, so llama.cpp would win either way, but unmatched engines also tie with each other and let size decide among them. Naming them fixes that order. MLX is left off the darwin-x86 row on purpose so it ranks last, since IsBackendCompatible admits darwin-tokened engines on that capability even though MLX needs Apple silicon. Preference orders survivors and never filters, so a model published only as a vLLM build is still installed on a host with no GPU; there is a spec for it. Surveyed every other value getSystemCapabilities() can return. nvidia, amd, intel and vulkan have rows; the l4t and cuda-refined values reach the nvidia row by prefix; "apple" and "" cannot reach the vendor fallthrough because the darwin and no-GPU branches return earlier. default and darwin-x86 were the only live holes. BackendPreferenceTokens and its build-tag table are untouched, and preferenceRank stays generic, naming no engine and no capability. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * gallery: prefer speculative-decoding builds when they fit Rank serving features between engine preference and size, so a host that can hold a DFlash or MTP build of a model's weights installs it instead of the plain build. Both answer faster for the same output, so whenever one survives the filters there is no reason to take the plain build. Precedence is now fit, then engine, then serving feature, then size. Engine outranks the feature deliberately: a serving feature makes the right engine faster, it does not make a wrong engine right, so a plain vLLM build still beats a DFlash llama.cpp build on NVIDIA. Fit outranks both, and a drafter pairing is strictly larger than the plain build, so the existing size filter drops it on a host too small for it before this axis is consulted. The order lives in a third preference table in pkg/system, alongside the build tag and engine name tables. It is the odd one of the three: not keyed by capability, because no hardware prefers a plain build over an equivalent faster one, and matched against whole segments of a gallery ENTRY NAME rather than as a substring of a backend value. Nothing on a gallery entry declares a serving feature, and tags are not a usable substitute: gemma-4-e2b-it:sglang-mtp carries an mtp tag while ornith-1.0-9b-mtp and qwen3.6-27b-nvfp4-mtp carry none. Entry names are author-supplied free text, unlike the closed engine vocabulary, so a short marker can turn up inside an unrelated word and whole segment matching is what keeps smtp-assistant from ranking as an MTP build. The block comment over the tables now documents all three together and states what each is matched against; the ranking code names no feature, so adding one stays a one-line edit to the table. 29c49203b rejected these entries as serving configurations rather than alternative builds of the same weights. The definition is now "alternative ways to serve the same model", which includes them, so regroup 14 entries under 12 parents. Judged by the files each entry points at: the qwen3.6, qwen3.5, qwen3 and deepseek pairings are the base GGUF plus a drafter, the gemma-4 QAT MTP entries are the same QAT weights at a different quantization plus an MTP drafter, and the two sglang MTP entries describe themselves as the same model served with speculative decoding. Left separate: qwen3.6-27b-mtp-pi-tune, a finetune with its own weights, and every entry whose base model LocalAI does not ship as its own row, which is the whole Qwopus line plus gemmable-4-12b-mtp, mimo-7b-mtp:sglang and qwen3.5-4b-dflash. None of the twelve parents defines a YAML anchor, so no variants key can leak through a merge key and no empty override was needed this time. The index was edited by line insertion only. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test: check env restore errors in capability and variant specs errcheck flagged ten unchecked os.Setenv and os.Unsetenv returns in the specs added while the pre-commit hook was being skipped. Restoring an env var is exactly the place a silent failure leaks state into the next spec, so assert on it rather than suppressing the linter. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): make the mtp tag authoritative for serving-feature ranking Variant auto-selection ranks survivors by fit, then engine, then serving feature, then size. The serving-feature lookup read only whole alphanumeric segments of a variant's entry name, because tags were inconsistent: every dflash entry carried a dflash tag, but only 7 of 20 MTP entries carried an mtp tag. Tag the 13 untagged MTP entries, then teach the lookup to read tags as well as names. A tag is now the authoritative signal and is compared whole and case-insensitively, which is safe precisely because a tag is a deliberate declaration rather than free text: there is no word-inside-a-word failure mode, so the segment splitting the name half needs is unnecessary there. The name check stays as a fallback rather than being replaced. Switching to tags only would have regressed the six already-grouped entries on the day it shipped, and would depend on tagging discipline that does not exist yet. The lookup still names no feature, so adding one remains a one-line edit to servingFeaturePreferenceTokens. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): make a declared tag the sole serving-feature signal Variant auto-selection ranks survivors by fit, then engine, then serving feature, then size. The serving-feature lookup recognised a speculative build by either a declared tag or a whole segment of its entry name. Drop the name half: a tag is now the only signal. A name is author-supplied free text and a naming convention is not a contract, so reading a marker out of one infers a capability nobody declared. The gallery already had the failure in it: the four NVFP4 entries name MTP-bearing weights while setting no option that enables speculative decoding, and being live variants they were winning the feature axis without answering any faster. overrides.options was considered as the replacement and rejected. It carries spec_type:draft-mtp / spec_type:draft-dflash, which is what actually turns the feature on, but that spelling is llama.cpp's config vocabulary: ds4 spells the same feature mtp_path and sglang spells it speculative_algorithm in a referenced config. Keying a cross-backend ranking decision on one backend's option syntax would rank the other backends' builds as plain. Options are the curation-time check instead, and never reach the selection logic. With no fallback left, tag correctness is load bearing, so audit every entry against the rule "tagged when the entry configures that feature, in whatever vocabulary its backend uses". Three entries configure MTP untagged and gain the tag (hy3, glm-5.2, qwythos-9b-claude-mythos-5-1m, all spec_type:draft-mtp with no marker in their names). Four carry the tag while configuring nothing and lose it: qwen3.6-27b-nvfp4-mtp, qwen3.6-35b-a3b-nvfp4-mtp, qwopus3.6-27b-coder-mtp-nvfp4 and qwopus3.6-27b-v2-mtp-nvfp4, whose only option is use_jinja:true. The dflash side was checked independently rather than assumed consistent: all five dflash entries declare spec_type:draft-dflash and all five are tagged, so it needed no edits. Four entries keep a tag that a literal spec_type-only reading would strip, because they configure MTP through a different backend: deepseek-v4-flash-q2-mtp via ds4's mtp_path/mtp_draft, and the three sglang entries via speculative_algorithm in their referenced configs. Stripping those would contradict the reason spec_type was rejected as the signal and would demote four genuinely faster builds to plain. The index was edited by line insertion and deletion only, never round-tripped through a serializer. A resolved-tag diff across all 1272 named entries, taken after merge keys are applied, shows exactly these 7 changing and no entry gaining or losing a tag through an anchor. The two specs that pinned the name fallback are inverted rather than deleted, since a name silently promoting a build is the regression worth guarding. The whole-token guard survives on the tag path, where smtp must still not match mtp. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): make deepseek-v4-flash variant targets installable Clicking install on deepseek-v4-flash failed with "invalid gallery model". The parent entry is fine, but all four entries it was grouped with declared neither url: nor config_file:, and applyModel needs one of the two to have anything to build a config from. They carry urls: (plural), the informational HuggingFace link list, which is a different field. None of the four was ever independently installable, so grouping them routed a previously-working install into a broken entry. Give each the url: the parent already resolves through. virtual.yaml is a no-op base, and applyModel passes overrides to InstallModel separately from the fetched config, so backend: ds4, the parameters and the ssd/mtp options all still land exactly as authored. This is the same pattern the parent and many other GGUF entries in the index already use. Add the lint rule that should have caught this. checkVariantReferences only proved a target exists and is not itself a parent, which is structural validity: an entry can exist, declare no variants, and still be uninstallable. checkVariantTargetsInstallable mirrors applyModel's precondition instead, and names the parent, the target and the missing fields, because whoever hits it is reading a gallery entry and has no reason to know applyModel exists. The two index-driven resolution specs live in their own Ordered container: an Ordered container stops at its first failure, so sharing one with the lint rules let a lint breach skip them silently. Nine further entries gallery-wide have the same defect and are unrelated to variants, so they are broken installs that predate this branch. They are left alone here rather than buried in a regression fix, and widening the rule to cover every entry is deferred with them so the gate can ratchet up in one step instead of needing a skip list. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): install entries with no url or config_file on an empty base applyModel had three branches: fetch a base config from url:, build one from an inline config_file:, or fail with "invalid gallery model". An entry declaring neither is now installed on an empty base config, with overrides: and files: supplying everything. This is what the ~345 entries pointing at gallery/virtual.yaml were already getting. That stub is five lines carrying name, description and license. description and license are overwritten from the gallery entry immediately after the fetch, and the name never reaches disk because InstallModel prefers the install name. Crucially applyModel passes model.Overrides to InstallModel as a separate argument rather than merging it into the fetched config, so nothing an author writes depends on that base existing. The fetch bought a round trip to GitHub and nothing else. That makes f4ef80173 the wrong fix, so it is unwound. The four url: lines it added to the deepseek-v4-flash variants are reverted: they are a pointless network fetch now, and the family installs without them. Relaxing the branch would hide a real authoring mistake, so a payload rule replaces the base-config rule. An entry with no url, no config_file, no overrides and no files installs nothing and would leave an empty model directory while reporting success, so it is refused by name. The caller's request counts toward the payload, because its overrides and files are merged into the install exactly as the entry's own are. urls: (plural) is the informational link list and does not count, which is what the four entries that shipped broken had and why they were still uninstallable. checkVariantTargetsInstallable asserted every variant target declares a url: or a config_file:, which is no longer true and would now reject correct authoring. checkEntriesInstallSomething pins what survives instead, and covers every entry rather than only variant targets: the hazard is a half-written stanza and a parent can be one as easily as a target. The old rule was scoped to targets precisely because nine unrelated entries would have failed a gallery-wide version; those nine are valid now, so the deferred ratchet happens here in one step. 1280 entries, zero violations. Those nine (aurore-reveil_koto-small-7b-it, lfm2-1.2b, the six liquidai_lfm2 entries and deepseek-v4-pro-q2-ssd) become installable for free. Each carries overrides: and files:, and one of them is driven through the real install path in a spec. The no-fetch spec is paired rather than bare: an assertion that nothing was fetched proves nothing unless something could have been, so a control runs the same fixture with a url: pointing at a base config that is not there and asserts the install fails. Only then does the identical fixture without the url passing mean the read was skipped. Follow-up, deliberately not here: the ~345 entries still naming virtual.yaml can drop their url:. That is 345 index edits with their own risk, and mixing them in would bury this change. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ui(models): let search bypass the variant collapse, drop the toggle The models page collapsed the gallery to one row per model by default and offered a toggle to see every individual build. Because the collapse composed with the search term, a build another entry offers as a variant could not be found by typing its name, so the toggle was the only way to reach those builds in the UI. A user who typed a name they knew existed got "no models found", which reads as "that model does not exist". Collapse is for browsing; search is for finding. An explicit search term now bypasses the collapse in the listing handler, so a name lookup returns matching entries whether or not a parent offers them. The term is trimmed once at the top of the handler, so whitespace is neither a search nor a bypass; previously an untrimmed blank term also narrowed the listing to whatever contained a space. Tag and backend deliberately do not bypass: they refine a listing the user is still reading rather than name an entry already known to exist. That makes the toggle redundant, so it goes, along with its i18n strings in all six locales, its localStorage persistence, its participation in "Clear filters" and the empty-state hint telling users to turn it off. The hint was doubly stale: it pointed at a control that no longer exists, and it was untrue exactly when a user has a search term, since searching now sees every build. The page always requests the collapsed listing. The stored preference key is left inert rather than cleaned up: nothing reads it, so a user who had the toggle off simply gets the collapsed view. collapse_variants stays on the API, off by default, because other clients want either view and the UI dropping its control is no reason to remove a working parameter. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): give the models gallery filter form a deliberate structure The filter area had accreted controls into one undifferentiated flow. The "Fits in GPU" toggle and the backend select were direct children of .filter-bar, the same wrapping container as the 18 taxonomy chips, so their position was decided by how many chips happened to wrap at the current width rather than by any layout intent. At narrow widths they were pushed past the right edge of that container's horizontal scroll and became unreachable entirely. Restructure into three bands inside the house .filter-bar-group wrapper that components/FilterBar.jsx already uses on Backends and the System tabs: 1. query scope: search plus the backend select 2. taxonomy: the chip row, alone, free to wrap 3. refinements: fits-in-GPU and context size, under a hairline rule The backend select leads the chips rather than trailing them because picking a backend disables the use cases that backend cannot serve, so it gates the row below it. Fits-in-GPU and context size share a band because they are one control group: the context size is the length the VRAM estimate is computed at, and that estimate is what the fits filter tests against. Chips had no visible keyboard focus indicator. The global focus ring is wrapped in :where(), so it carries the specificity of a bare :focus-visible, ties with .filter-btn and loses on source order, leaving focused chips showing their resting drop shadow. Restate the ring where it outranks both resting and hover. Also: aria-pressed on the chips, a real label association and aria-valuetext on the context slider (it steps over an index, so it announced "2"), disabled chip styling moved off inline styles, a prefers-reduced-motion block for the chip transition, and the hard-coded English "Context:" moved into all seven locales. No behaviour change: same filters, same state, same requests. Page reset on change, localStorage persistence and "Clear filters" verified unchanged. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): let the models recommendations panel fade into the background The "Recommended for your hardware" strip rendered at full height on every visit regardless of how many models were already installed, costing 186px at 1600px wide (287px at 1100px, where its cards wrapped to two rows) and pushing the first gallery row to y=554 / y=703. Make its prominence track how much the user still needs it. The panel now defaults to a one-line summary once anything is installed, and both the collapse choice and the existing dismissal persist: collapsed = explicit user choice, if one exists : installedCount > 0 The preference is three-valued on purpose. A boolean cannot tell "the user expanded it" apart from "the user has never chosen", and those need opposite handling when the installed count later crosses zero: someone who deliberately opened the panel on an empty instance should not have it collapse out from under them when their first model finishes installing. Collapsed keeps the card, icon, title and a suggestion count, so the panel is recovered by clicking what you are already looking at rather than by hunting. Expanded is unchanged, because for a user with nothing installed it was never the problem. Collapsed reclaims 145px at 1600 and 420, and 246px at 1100. Models.jsx gains a statsLoaded flag: stats initializes to installed:0, so reading it before the fetch resolves would render expanded and collapse a frame later, which is exactly the layout shove this removes. The dismissal key moves to the page's localai-models-* convention; the old localai_rec_models_dismissed is still read, never written, so an existing dismissal is honoured rather than resurrected by the rename. Accessibility: the disclosure is a real button whose accessible name is the visible title alone, with state on aria-expanded and aria-controls resolving in both states, because the grid is hidden via the hidden attribute rather than unmounted. That also keeps the four install buttons out of the tab order while collapsed. The app's global focus ring applies; no per-component outline is added, per the warning in App.css. Reveal animates opacity and transform only, never height, and both it and the chevron rotation are disabled under prefers-reduced-motion. Only en had a recommended block, so the other six locales were falling back to English for the whole panel. Translated the complete block rather than adding one orphaned key to files that would still render the title in English. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(downloader): recover from a leftover .partial on non-HTTP URIs An interrupted download leaves a `<file>.partial` behind. The partial handling in DownloadFileWithContext gated resume on `err == nil && uri.LooksLikeHTTPURL()`, so for any URI that is not literally http(s) the branch fell through to `else if !errors.Is(err, os.ErrNotExist)`, which with a nil err is true. The download then failed with an error wrapping nil: failed to check file ".../Ternary-Bonsai-27B-Q2_g64.gguf" existence: <nil> Every gallery file URI uses `huggingface://`, so a single interrupted download made that model permanently uninstallable until someone deleted the partial by hand. The `<nil>` in the message compounded it by pointing debugging at a filesystem failure that never happened. Restructure the handling as an explicit switch over the four real states: partial exists and is resumable, partial exists and is not resumable (discard and restart, as already done for an HTTP server without range support), no partial, and a genuine stat failure. The error branch is now only reachable with a non-nil error, names the path that was actually stat'd, and wraps with %w. Discarding is required for correctness and not merely convenience: the writer opens the partial with O_APPEND, so an un-resumed download would concatenate a fresh body onto stale bytes. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): tell the models gallery's variant rows apart, and let browsing see every build Both variant surfaces rendered name, backend and size. For two builds of one model that is close to no information: a variant exists precisely because the same weights are offered another way, so the backend usually matches and the sizes usually land within a few hundred megabytes. Comparing ternary-bonsai-27b-pq2 against ternary-bonsai-27b-q2-g64 meant reading two names that differ by a suffix nobody has defined anywhere in the UI. Report the quantization and the serving features on VariantView, and derive both server-side from the referenced entry rather than parsing names in the browser, so every client reads the same format out of the same file the installer will hand the backend. Quantization comes from overrides.parameters.model first, falling back to the file list. That order is load bearing: entries routinely ship a vision tower alongside the language model at a different quantization, so reading the file list first reports the mmproj's format. Matching walks `-` and `.` delimited segments right to left; `_` deliberately does not split, because it separates the parts INSIDE a quant token and splitting on it reports Q4 for a Q4_K_M build. A second, looser pass takes a segment's `_`-delimited tail, which catches the gemma-4-E2B_q4_0-it.gguf style; it runs second so a precise match can never lose to a fuzzy one further right in the name. An entry naming no format reports nothing, which is the honest answer for a backend served from a directory of weights. Features are the same tag-against-vocabulary match servingFeatureRank already ranks on, over the same host preference list. A build can therefore never be shown as faster than one selection did not actually reward, nor rewarded without being shown; a spec pins that agreement rather than trusting it. The compact dropdown gets the quantization on its meta line and the bare feature token. The detail row, which has the room, gets the quantization as its own monospaced column so precision lines up down the list, and the feature spelled out, because DFLASH names nothing to a user who has not met it. The referenced entry's description stays out of both: the detail row already renders the parent's prose above the table, and a second block per variant would push a three-variant list past a screen to restate what the columns now say precisely. The collapse toggle comes back. 462583f38 dropped it once search bypassed the collapse, on the reasoning that nothing was unreachable any more. That holds for finding a build whose name you know and does not hold for browsing: no sequence of actions enumerated the 68 builds the default view hides. Collapse is for browsing and search is for finding, and the toggle was the browsing half. It goes in the refinements band 0d4823362 established, not back among the taxonomy chips where its position depended on how many chips happened to wrap. It leads that band because it decides how many rows the other two refine over, and because unlike fits-in-GPU it is unconditional: a host with no GPU still browses. The search bypass is untouched and re-checked by a spec in the toggle's default state, since restoring the control must not restore the dead end it replaced. The empty-state hint returns but only without a search term, because a term bypasses the collapse and the hint would otherwise point at a control that cannot change the result. The stored preference reads 'on'/'off' only: an older build wrote '1'/'0' from an effect that ran on mount, so those record that the page was opened, not that anyone chose a view. Also fixes a latent flake it exposed. The collapse_variants spec compared whole response bodies byte for byte, and the listing envelope carries live host telemetry that drifts between two calls milliseconds apart, so it was asserting on the machine's memory pressure. It now compares everything the parameter governs -- the entries, their serialization and the paging -- and is green 25/25 where it was failing about one run in three. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): let the models gallery show a variant's full details The variant list in an entry's expanded detail row says how the builds differ: name, backend, quantization, size, and the auto-selected, base and serving-feature markers. It cannot say what any one of them is. A variant's own description, tags, license, source links and file list are unreachable anywhere in the UI, because while the collapse is on a variant has no gallery row of its own. Give each variant row an info control that reveals its entry, rendered by the same ModelDetail a top-level row gets, so a field added to the detail view appears here too. variantData is withheld from the nested render: a variant may declare variants of its own, and recursing would nest a picker inside a picker two levels deep already. An inline disclosure rather than a modal. The control sits inside a table row that is already expanded, inside a variant list within that; a dialog opened from there stacks a dismissal on a dismissal for a handful of extra fields about the entry the user is already reading, and breaks the page's own expand idiom. The third level is carried by an inset and a left rule instead of another card. The entry is fetched by exact name from the listing, once, on first use. The listing already returns every field the detail view renders, and a search term bypasses the variant collapse server-side, so no new endpoint is needed and neither the listing nor DescribeVariants gains any work. Expanding a row costs nothing; a variant nobody opens costs nothing. A name the listing no longer returns is stated, not blanked: an empty panel reads as a rendering fault rather than as a lookup that came back empty. The control is a sibling of the install button, not a descendant, so asking about a build can never install it. The variant list keeps its content-sized columns via a trailing filler track instead of max-content sizing, so the rows are unchanged while the panel spanning them gets the pane width its file table needs. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): let search respect the collapse instead of switching it off The models listing collapsed to one row per model, and an explicit search term turned that off wholesale. Searching while collapsed therefore answered with the individual builds a parent already offers, which are exactly the rows the view the user asked for has no place for: typing "mtp" returned qwen3.6-27b-nvfp4-mtp, a row that is invisible the moment the box is cleared. The bypass was the right shape of fix for the wrong half of the problem. What a search must not do is answer "no models found" for a build the gallery does hold; that does not require abandoning the grouping the user asked for. So the term is now matched against every entry either way, hidden builds included, and the collapse decides how a match is reported rather than which matches exist. Collapsing stops being a filter that drops rows and becomes a substitution: a match on a build another entry offers is reported as that entry, the one installable in its own right. Nothing becomes unfindable and nothing comes back that the requested view cannot show. Substitution happens after search, tag and backend, so every filter is judged against the build that really carries the name, tag or backend rather than against a parent that merely offers it; the other order would let backend=vllm match a parent whose own backend is something else. The price is that the surfaced row shows the parent's own metadata while the match was on a variant, which is what grouping means, and the alternative is claiming the gallery holds no such build. It happens before the count and the page math, so both describe the rows actually handed out rather than the matches that produced them. A parent already in the result keeps its own position and absorbs its matching variants there, which is what leaves the browsing listing ordered exactly as it was; a parent surfaced only by a variant takes the position of the first variant that surfaced it. Either way it appears once, however many of its builds matched and whether or not it matched itself. Search preserves gallery order rather than scoring, so a surfaced parent has a real position rather than an invented one. VariantParents never reports an entry that declares variants of its own, so a parent is never itself hidden and one hop always lands on a visible row. The handler follows exactly one anyway: refusing the second is what makes a gallery the linter would have rejected terminate rather than loop. The empty-state hint pointing at the toggle goes with it for every server-side filter. Substitution means a match is always reported as some row, so the collapse can no longer be why a term, a chip or a backend came back empty, and naming it there sends the user to a control that cannot change the result. It survives for the fits filter alone, which runs in the browser after the substitution and judges the surfaced entry's own size: there the build that fits really can be filtered out along with a parent that does not. Searching a build's exact name while collapsed now answers with its parent, so the result no longer contains the string the user typed. That is intended, and the row is the one they can act on, but it is a real rough edge: nothing on the row explains the connection. Closing it properly means reporting which variant matched so the UI can say so, which the listing does not do today. ResetGalleryModelCache is added for tests. The model cache is a package global keyed by nothing, so a background refresh one spec triggers can land in the middle of the next and answer it with the previous spec's gallery; the extra specs here made that fail about one run in five. It waits for the in-flight refresh to publish before clearing, since clearing alone only narrows the window. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-20 18:43:02 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	9d82c37f98	fix(distributed): backend discovery hid worker-installed backends behind the controller's filesystem (#10967) ... fix(distributed): backend discovery hid worker-installed backends Backend discovery endpoints filter on installed-state, which on a distributed controller derives from the controller's own filesystem. A backend lives on the worker node that runs it, so every backend an admin installed on a GPU worker read as "not installed" and vanished from the listing. #10947 fixed the sibling capability filter on the same endpoints, so a fine-tuning-capable GPU worker now made the backend listable while the installed-state filter still dropped it: the dropdown stayed empty. The controller cannot derive this locally, but it already aggregates the per-node view that GET /backends renders, so discovery reuses the active BackendManager rather than growing a second path. Three surfaces shared the root cause and route through the same helper now: - GET /backends/available (Installed is now cluster-wide) - GET /api/fine-tuning/backends - GET /api/quantization/backends The response stays a boolean rather than an installed-on-N-of-M count: per-node install state is already served by GET /backends nodes[], and per-node control by POST /api/nodes/:id/backends/install, so a summary is all these dropdowns need. A nil provider (single-node) leaves the local filesystem as the only source and reproduces today's listing exactly, and a registry error degrades to that same listing instead of blanking the catalog. Assisted-by: Claude:claude-opus-4-8 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-20 01:09:37 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	b19afb192a	fix(distributed): backend discovery hid GPU-only backends behind the controller's capability (#10947) ... * fix(backends): list backends runnable on worker nodes in distributed mode GET /backends/available filtered the gallery against the system state of the host serving the request. In a distributed deployment that host is the controller, which typically has no GPU, while the GPUs live on worker nodes. Any meta backend whose capabilities map lacks a "default" (or "cpu") key was therefore dropped from the listing entirely — longcat-video, vllm-omni, ltx-video, parakeet, edgetam and qwentts were invisible in the UI even though installing them by name on a GPU worker worked fine. Workers now report their own meta-backend capability at registration and the controller persists it on the node row. The controller cannot derive it: OS-dependent capabilities (metal, darwin-x86, nvidia-l4t) and the CUDA runtime refinements are only observable on the worker. Nodes registered before this field existed fall back to a coarse capability derived from their GPU vendor and VRAM. Backend discovery then evaluates compatibility as the union over healthy backend nodes, so a backend runnable on any node is offered while one no node can run stays hidden. Each remote capability is evaluated through a capability-pinned system state, otherwise a forced capability on the controller image (LOCALAI_FORCE_META_BACKEND_CAPABILITY or /run/localai/capability) would silently override every worker's verdict. With no registered nodes the listing is byte-for-byte what it was, so single-node deployments are unaffected. Also fixes the same-root-cause misclassification in /api/operations, which used the capability-filtered listing to decide whether an operation was a backend or a model install. A GPU-only backend installing on a worker is still a backend operation on the controller, so that lookup is now unfiltered. Assisted-by: Claude:claude-opus-4-8 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(backends): union worker capabilities in backend discovery Implementation for the specs added in the previous commit, plus the two remaining discovery endpoints. Capability-filtered backend discovery evaluated compatibility against the system state of the host serving the request. In a distributed deployment that host is the controller, which typically has no GPU, while the GPUs live on worker nodes. Any meta backend whose capabilities map lacks a "default" (or "cpu") key was dropped entirely — longcat-video, vllm-omni, ltx-video, parakeet, edgetam and qwentts were invisible in the UI even though installing them by name on a GPU worker worked fine. Workers now report their own meta-backend capability at registration and the controller persists it on the node row. The controller cannot derive it: OS-dependent capabilities (metal, darwin-x86, nvidia-l4t) and the CUDA runtime refinements are only observable on the worker. Nodes registered before this field existed fall back to a coarse capability derived from their GPU vendor and VRAM. Discovery then evaluates compatibility as the union over healthy backend nodes, so a backend runnable on any node is offered while one no node can run stays hidden. Each remote capability is evaluated through a capability-pinned system state, otherwise a forced capability on the controller image (LOCALAI_FORCE_META_BACKEND_CAPABILITY or /run/localai/capability) would silently override every worker's verdict. With no registered nodes the listing is byte-for-byte what it was, so single-node deployments are unaffected. Four surfaces shared this root cause and are all routed through the same helper now: - GET /backends/available - GET /api/fine-tuning/backends - GET /api/quantization/backends - /api/operations backend-vs-model classification, which additionally had no reason to filter by capability at all: a GPU-only backend installing on a worker is still a backend operation on the controller, so that lookup is now unfiltered. Assisted-by: Claude:claude-opus-4-8 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-19 07:53:46 +00:00
LocalAI [bot]andEttore Di Giacinto	8cec22c3b7	feat(vram): per-node VRAM allocation budget (LOCALAI_VRAM_BUDGET) (#10833) ... * feat(vram): add vrambudget primitive for per-node VRAM caps Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): apply default VRAM budget in xsysinfo aggregate getters Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): wire LOCALAI_VRAM_BUDGET flag to xsysinfo default budget Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): persist VRAM budget via runtime settings with live apply Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(vram): reset process-global VRAM budget after runtime-settings spec Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): add VRAM budget field to Settings page Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): store and enforce per-node VRAM budget in the node registry Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): apply per-node VRAM budget in router hardware defaults Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): report worker VRAM budget in node registration The distributed worker now reports its operator-set VRAM budget string (LOCALAI_VRAM_BUDGET) to the server on registration. The worker keeps reporting RAW total/available VRAM and never sets the xsysinfo process-global budget (that stays standalone-only); the server resolves and enforces the budget uniformly (Task 6). Also closes a Task 6 gap: on re-registration, a struct Updates zero-skips an empty budget, so a worker that dropped LOCALAI_VRAM_BUDGET left the stale cap in place. For non-admin-override nodes the budget columns are now force-written (map Updates) even when empty, so removing the env var clears the cap; admin overrides are preserved unchanged. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * style(vram): drop em dash from worker-clear comment Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): add node VRAM budget admin endpoints Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): add node VRAM budget control to the node UI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): expose set_node_vram_budget MCP admin tool Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(vram): document LOCALAI_VRAM_BUDGET and node VRAM budget UI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(vram): avoid double-applying VRAM budget in GetResourceAggregateInfo The GPU-branch aggregate returned by GetResourceInfo is sourced from GetGPUAggregateInfo, which already caps total/free/used against the process-wide VRAM budget. GetResourceAggregateInfo then applied the budget a second time. For an absolute budget this is idempotent, but for a percentage budget b.Apply resolves the ceiling as a fraction of its input total, so a second pass yields P*(P*T) instead of P*T and distorts UsagePercent (read by the memory reclaimer in pkg/model/watchdog.go). Remove the redundant second application so the budget is applied exactly once, against the raw physical totals, upstream in GetGPUAggregateInfo. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(vram): implement SetNodeVRAMBudget on mcp assistant test stub The LocalAIClient interface gained SetNodeVRAMBudget; the stubClient in core/http/endpoints/mcp used by the assistant tests is a separate implementer and needs the method too (broke golangci-lint typecheck and both test jobs). Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-15 09:58:45 +02:00
LocalAI [bot]andEttore Di Giacinto	3601174ce0	fix(distributed): make per-node backend upgrade actually upgrade (#10838) ... * test(core/http): make the suite's HTTP port overridable app_test.go and openresponses_test.go hardcoded 127.0.0.1:9090. When another service already listens on 9090 the suite does not fail fast: the server goroutine logs the bind error and the specs then poll whatever is squatting the port until Eventually times out. On machines where 9090 is permanently taken this makes the pre-commit coverage gate impossible to pass. Introduce testHTTPAddr, defaulting to 127.0.0.1:9090 (what CI has always used) and overridable via LOCALAI_TEST_HTTP_PORT for local runs. Assisted-by: Claude:claude-fable-5 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): make per-node backend upgrade actually upgrade The node detail page's Upgrade button reused the node-scoped install path (POST /api/nodes/:id/backends/install). That fires NATS backend.install with force=false, and the worker's install handler is deliberately "ensure installed": when the backend binary already exists on disk it short-circuits without touching the gallery. Since only an installed backend can be upgraded, the whole chain was a guaranteed successful no-op - the UI then toasted "backend upgraded" without even waiting for the async job. Route upgrades through the real force-reinstall path instead: - BackendManager.UpgradeBackend now receives the ManagementOp (like InstallBackend already did) so implementations can honor op.TargetNodeID. - DistributedBackendManager.UpgradeBackend scopes the backend.upgrade fan-out to op.TargetNodeID when set, and errors when the target node does not report the backend as installed. - New POST /api/nodes/:id/backends/upgrade endpoint enqueues an Upgrade=true node-scoped op (async 202 + jobID, mirroring install). - NodeDetail UI calls the new endpoint and reports the dispatch ("Upgrading ... on this node...") instead of claiming success; the Operations panel tracks the actual job. Verified against a live local cluster (NATS + Postgres + two workers): the target worker stops the running process, force-reinstalls from the gallery and re-downloads the OCI image; the second worker receives no backend.upgrade event; upgrading a backend missing from the target node fails the job with a clear error. Assisted-by: Claude:claude-fable-5 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-15 09:16:55 +02:00
LocalAI [bot]andEttore Di Giacinto	bcc41219f7	feat: materialize Hugging Face model artifacts (#10825) ... * feat(config): add model artifact source contract Assisted-by: Codex:GPT-5 [Codex] * feat(downloader): add authenticated raw-byte progress Assisted-by: Codex:GPT-5 [Codex] * feat(huggingface): resolve immutable snapshot manifests Assisted-by: Codex:GPT-5 [Codex] * feat(models): add artifact storage primitives Assisted-by: Codex:GPT-5 [Codex] * feat(models): materialize pinned Hugging Face snapshots Assisted-by: Codex:GPT-5 [Codex] * feat(models): bind managed snapshots at runtime Assisted-by: Codex:GPT-5 [Codex] * feat(gallery): materialize model artifacts during install Assisted-by: Codex:GPT-5 [Codex] * feat(gallery): declare managed Hugging Face artifacts Assisted-by: Codex:GPT-5 [Codex] * feat(models): preload managed model artifacts Assisted-by: Codex:GPT-5 [Codex] * fix(gallery): retain shared artifact caches on delete Assisted-by: Codex:GPT-5 [Codex] * feat(models): report artifact acquisition progress Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): load managed models from ModelFile Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): load staged speech model snapshots Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): use staged snapshots in engine backends Assisted-by: Codex:GPT-5 [Codex] * test(distributed): cover staged artifact snapshots Assisted-by: Codex:GPT-5 [Codex] * docs: explain managed model artifacts Assisted-by: Codex:GPT-5 [Codex] * docs: add product design context Assisted-by: Codex:GPT-5 [Codex] * feat(ui): show model artifact download progress Assisted-by: Codex:GPT-5 [Codex] * Eagerly materialize Hugging Face artifacts Materialize HF-backed model references as managed GGUF artifacts during load, with lazy download retained only as fallback. Assisted-by: Codex:GPT-5 [shell] * Refactor HF downloads through a shared executor Assisted-by: Codex:GPT-5 [shell] * drop Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-15 01:09:33 +02:00
LocalAI [bot]andEttore Di Giacinto	4056283aa4	[voice] feat: add managed voice cloning profiles (#10799) ... * feat(ui): add voice library workflow Give administrators a production-ready flow to record or upload consented reference audio, manage reusable profiles, inspect API usage, discover compatible models, and hand a saved voice directly to text-to-speech. Assisted-by: Codex:gpt-5 * feat(voice): add managed voice cloning profiles Make reusable reference voices manageable through the admin API instead of requiring model-directory and YAML edits. Discover compatible installed and gallery models from server-side backend capabilities, retain explicit model configuration controls, and stage saved references for supported backends. Expose profile management through REST and MCP, document backend-specific behavior, and cover the workflow from profile creation through real Qwen3-TTS synthesis. Harden the agent-job HTTP test against completion racing cancellation. Assisted-by: Codex:gpt-5 --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-13 09:54:46 +02:00
LocalAI [bot]andEttore Di Giacinto	b0959d4756	feat(api): add GET /v1/models/capabilities endpoint (#10687) ... Additive superset of /v1/models that enriches each model entry with the capabilities it supports plus its input/output modalities (text / image / audio / video). Clients that only understand /v1/models are unaffected -- they simply never call the new route. Audio and video *input* are derived from the model's multimodal limits (vLLM limit_mm_per_prompt), which no single usecase FLAG expresses. That gap is exactly why a plain capability list is insufficient and this enriched endpoint exists: an attachment router can now decide whether an image/audio/video file can go to the active model directly, or must be converted/transcribed first. Capability derivation lives in core/config as the single source of truth (ModelConfig.Capabilities / InputModalities / OutputModalities / VisionSupported / ...); the Ollama capability surface now delegates to it instead of keeping a parallel copy. Vision is gated on chat/completion capability so a MediaMarker hydrated onto a non-chat model (e.g. a pure ASR/TTS backend) no longer reports a false vision capability. Read-only listing: no new FLAG_* flag, reuses the existing `models` swagger tag, and intentionally exposes no MCP admin tool (there is nothing to manage conversationally). Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-05 08:51:55 +02:00
Richard Palethorpe	eb32cd9073	feat(realtime): eager blocking pipeline warm-up + /backend/load API (#10662) ... Realtime sessions previously lazy-loaded each pipeline sub-model (VAD, transcription, LLM, TTS) on first use, so every cold session paid a per-request model-load stall and load errors only surfaced mid-stream. Warm the whole pipeline eagerly and blockingly at session start (including the voice-gate speaker-recognition model, which an enforced gate blocks each utterance on; compaction's summary_model stays lazy since it only runs off the response path): - Add backend.PreloadModel / PreloadModelByName as the single load path for every modality (no transcription special-case; backend-omitted configs are deprecated). - The realtime session blocks on Model.Warmup and returns a model_load_error to the client if any stage fails to load; updateSession warms in the background. Opt out per pipeline with pipeline.disable_warmup, exposed as a UI toggle via the config-metadata registry. Add a LocalAI-native POST /backend/load (and /v1/backend/load) that pre-loads a model -- expanding realtime pipelines into their sub-models -- as the inverse of /backend/shutdown. There is one preload engine (backend.PreloadStages): the realtime Warmup methods, /backend/load and the --load-to-memory startup flag all use it, so --load-to-memory now also expands pipeline models and records load-failure traces. Pipeline sub-model alias resolution is likewise shared (ModelConfigLoader.LoadResolvedModelConfig). Surface the endpoint everywhere an admin manages models: - MCP admin tool load_model (httpapi + inproc clients, safety/catalog prompts, catalog/dispatch tests). - "Load into memory" action in the React models UI. - Swagger regenerated; docs moved to the general backend-monitor page since it is not realtime-specific. Fix a Traces UI crash ("json: unsupported value: -Inf"): audio-snippet RMS/peak now floor at a finite dBFS, and backend-trace data is sanitized to drop non-finite floats before marshaling. The sanitizer is copy-on-write -- it runs on every RecordBackendTrace, so containers are only re-allocated on the paths that actually changed. Migrate core/http/openresponses_test.go onto the prebuilt mock-backend the rest of the http suite already uses -- it was the last spec still pointing at a real HuggingFace model, so it 404'd wherever no vision backend was built -- and fix its item_reference specs to send the spec's "id" field instead of "item_id", which the handler never accepted. Assisted-by: Claude:claude-opus-4-8 Claude Code Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-07-03 18:00:37 +02:00
LocalAI [bot]andEttore Di Giacinto	6eea3ef2ac	fix(backends): make backend install ops idempotent unless forced (#10643) ... * fix(backends): make backend install ops idempotent unless forced POST /backends/apply hardcoded force=true through LocalBackendManager.InstallBackend, so applying an already-installed backend re-downloaded and re-extracted the whole artifact every time. API clients that ensure a backend exists at startup paid a full OCI image pull on every boot. Backend install ops now default to non-forced — an installed, runnable backend short-circuits (the orphaned-meta reinstall path in InstallBackendFromGallery is preserved) — and reinstall stays available: - ManagementOp gains a Force field; the local manager passes it through instead of hardcoding true. - /backends/apply accepts an optional "force" boolean in the body. - The React UI install route keeps forcing, since its button doubles as the explicit "Reinstall backend" action. Distributed installs already behaved this way (workers skip when the binary exists unless force is set); this aligns single-node behavior. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(backends): don't force-reinstall LOCALAI_EXTERNAL_BACKENDS on boot The startup loop for LOCALAI_EXTERNAL_BACKENDS runs InstallExternalBackend for each listed backend on every boot, and its gallery-name path hardcoded force=true — so every start re-downloaded and re-extracted each listed backend's OCI image even when it was installed and runnable. Supervising apps that list several backends paid several full OCI pulls per launch. Give InstallExternalBackend an explicit force parameter (it only affects the gallery-name fallback; URI installs always write) and pass: - false from the boot loop and `local-ai backends install` (idempotent ensure — `backends upgrade` is the refresh path), - op.Force from the local manager's external-URI op, - the request's force on the worker install path and true on its upgrade path (behavior unchanged). Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-02 19:16:29 +02:00
LocalAI [bot]andEttore Di Giacinto	91885c2c7e	fix(distributed): return empty backend list for agent nodes instead of failing backend.list (#10545) (#10565) ... Opening an AGENT-type worker node's detail page errored with "failed to list backends on node" / NATS "nodes.<id>.backend.list: no responders available". Agent workers only subscribe to agent.*, jobs.*, mcp.* and <prefix>.backend.stop; they never subscribe to backend.list, so the per-node ListBackendsOnNodeEndpoint request had no responder and timed out. The aggregate cluster-wide list already guards this in managers_distributed.go (skip nodes whose NodeType is set and not "backend"). The single-node endpoint lacked the same guard. Thread the NodeRegistry into ListBackendsOnNodeEndpoint and short-circuit to an empty (non-nil) list for non-backend node types before issuing the doomed NATS request, mirroring the aggregate-list gate so both views stay consistent. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-06-28 01:22:48 +02:00
LocalAI [bot]andEttore Di Giacinto	64150ca7ab	fix(distributed): broadcast admin model-config changes across replicas (#10540) ... In distributed mode the admin model endpoints (/models/edit, /models/import, /models/toggle-state and the PATCH config-json endpoint) wrote the YAML to the shared models dir but reloaded only the local replica's in-memory ModelConfigLoader. With multiple frontend replicas behind one service, a save landed on whichever replica handled the request; peers kept serving their stale in-memory view, so a load-balanced request was a coin-flip between old and new config (a created alias visible on one replica and missing on the other, an edited alias target diverging, etc.). The NATS cache-invalidation channel (SubjectCacheInvalidateModels + OnModelsChanged) already existed for the gallery install/delete path; these admin endpoints simply never published on it. Wire them up via a new GalleryService.BroadcastModelsChanged helper (no-op in standalone mode). Also fix delete propagation: LoadModelConfigsFromPath is additive and never drops an entry whose file is gone, so the subscriber hook (which only reloaded from disk) could not propagate a removal. ApplyRemoteChange now honors the event op - pruning the element on "delete" and reloading otherwise - and shuts down any running instance of the affected model so the new config takes effect. This closes the same latent gap on the gallery delete path. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-06-27 01:36:57 +02:00
LocalAI [bot]andEttore Di Giacinto	fe4f425fb5	fix: correct scheme/host on self-referential URLs behind an HTTPS reverse proxy (#10482) (#10504) ... * fix(http): harden BaseURL proxy scheme/host detection Split comma-separated X-Forwarded-Proto and honor the RFC 7239 Forwarded header so generated links use https behind common reverse-proxy setups. Refs #10482 Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(http): honor explicit external base URL in BaseURL When _external_base_url is set in the request context it dictates the origin (scheme+host+port); the proxy path prefix is still appended. Refs #10482 Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(config): generalize LOCALAI_BASE_URL to ExternalBaseURL LOCALAI_BASE_URL now sets a single instance-wide external base URL used for OAuth callbacks and all self-referential links. A Pre middleware stamps it into the request context for middleware.BaseURL. Refs #10482 Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: document LOCALAI_BASE_URL and reverse-proxy headers Refs #10482 Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(http): cover parseForwarded edge cases; clarify base-url flag group Adds direct unit coverage for quoted/malformed/multi-element Forwarded headers and regroups the external base URL flag away from auth-only. Refs #10482 Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-06-25 08:10:59 +02:00
LocalAI [bot]andEttore Di Giacinto	95b058e1c5	feat(ui): restructure Cluster Nodes view (pulse + panel roster + detail page) (#10447) ... * chore: gitignore SDD scratch directory Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * feat(nodes): add GET /api/nodes/models cluster-wide loaded-models endpoint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * feat(ui): add nodesApi.allModels() for cluster-wide model roster Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * feat(ui): move Scheduling to its own page and nav item Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * feat(ui): replace nodes stat-card strip with cluster pulse + attention callout Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * feat(ui): node-panel roster with inline model chips and segmented filter Replace the Nodes table with a full-width node-panel roster that shows each backend node's running-model chips without an expand click, plus an All/Backend/Agent segmented filter. Per-node detail (models, backends, labels, capacity) moves to the node detail page. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * feat(ui): add deep-linkable node detail page at /app/nodes/:id Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * fix(ui): remove em-dash from CapacityEditor comment; align detail spec backend mock Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * chore(ui): nodes page cleanup, hover/chip polish, docs for restructured cluster view Nodes.jsx dead-code sweep confirmed clean (no StatCard/table/expand state/scheduling-form leftovers). Two App.css polish fixes: move the node-panel hover border-color onto the bordered element so hover gives real feedback, and add the missing .model-chip__state rule the ModelChip component already emits. Update distributed-mode docs prose to describe the restructured cluster view (cluster pulse, attention callout, node-panel roster with inline model chips, All/Backend/Agent filter, node detail page at /app/nodes/:id, Scheduling as its own page). Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * chore(ui): drop unused gpuVendorLabel export from nodeStatus Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-06-22 18:24:29 +02:00
LocalAI [bot]andEttore Di Giacinto	600dafd20b	feat(ced): sound-event classification backend (CED audio tagger) (#10425) ... * feat(ced): sketch sound-classification backend (CED audio tagger) Wires ced.cpp (CED, 527-class AudioSet sound-event tagger; baby cry, footsteps, glass, alarms, dog bark) into LocalAI as a Go/purego backend. SKETCH (backend skeleton real; core REST wiring + CI/gallery is a checklist in DESIGN.md): - backend/backend.proto: new SoundDetection rpc + SoundClass messages (run `make protogen-go` to regenerate pkg/grpc/proto). - backend/go/ced: main.go (purego dlopen libced.so + ced_capi.h), goced.go (Ced gRPC backend: Load + SoundDetection), Makefile (clone-at-pin CED_VERSION, ggml static-PIC shared build), run.sh, package.sh, .gitignore. - DESIGN.md: REST /v1/audio/classification wiring (handler/route/capability registration checklist), gallery/index + CI registration, and a scoping note for the realtime/websocket live-recognition path (sliding-window classify over the existing ws transport + voicegate; the ced C-API per-PCM entry point is already window-friendly). Backend code does not compile until protogen-go regenerates the pb types and a libced.so is built (Makefile clones+builds it). Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ced): REST /v1/audio/classification endpoint + capability registration Wires the ced sound-event classification backend (AudioSet audio tagger) end to end through the REST surface, mirroring the transcription path. - Handler: core/http/endpoints/openai/sound_classification.go parses the multipart audio upload, temp-files it, resolves the model config and calls the SoundDetection RPC; returns {model, detections[]} JSON. - Backend wrapper: core/backend/sound_classification.go (ModelSoundDetection) loads the model and normalizes the proto response into schema types. - Schema: core/schema/sound_classification.go (SoundClassificationResult). - gRPC layer: SoundDetection wired through the LocalAI wrapper (interface, Backend client, Client, embed, server, base default) so the loader-typed client exposes the RPC; proto regenerated via make protogen-go. - Route: POST /v1/audio/classification (+ /audio/classification alias) with the audio/multipart default-model middleware in routes/openai.go. - Capability surfaces: swagger @Tags/@Router on the handler; FLAG_SOUND_ CLASSIFICATION usecase flag + UsecaseSoundClassification + UsecaseInfoMap + GuessUsecases + ModalityGroups + GetAllModelConfigUsecases; meta usecase option; /api/instructions audio area updated; auth RouteFeatureRegistry + FeatureAudioClassification (APIFeatures, default ON) + FeatureMetas; UI usecaseFilters, capabilities.js CAP_SOUND_CLASSIFICATION, Models.jsx filter + i18n; docs page features/audio-classification.md + whats-new + crosslink. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ced): realtime sound-event detection over the websocket API When a realtime pipeline configures a sound-classification model, each VAD-committed utterance (the same window the transcription path produces) is also run through the CED sound-event classifier and the scored AudioSet tags are emitted as a new server event. No new backend rpc is needed: the SoundDetection gRPC method already exists on this branch. - config: add Pipeline.SoundDetection (yaml/json sound_detection,omitempty) beside Transcription/VAD. - realtime: add Model.SoundDetection(ctx, audio, topK, threshold) to the ModelInterface; implement it on wrappedModel and transcriptOnlyModel by calling backend.ModelSoundDetection with the session's sound-classification model config (mirrors how Transcribe dispatches). Load the optional config in newModel / newTranscriptionOnlyModel; nil config keeps it additive. - types: add ConversationItemSoundDetectionEvent (item_id, content_index, detections[]{label,score,index}) with type conversation.item.sound_detection, its ServerEventType constant and MarshalJSON, mirroring the transcription completed event. - realtime: add emitSoundDetection (unary path: classify the committed window, build the event, t.SendEvent) and wire it at the utterance-commit hook right after emitTranscription; gated on session.SoundDetectionEnabled (resolved from Pipeline.SoundDetection at session setup, defaults top_k=5, threshold=0). Its error is logged via xlog but never aborts the turn. - test: Ginkgo specs for emitSoundDetection (tags emitted, empty detections, classifier error) plus a SoundDetection method on the fakeModel double. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ced): implement SoundDetection in nodes backend test doubles The SoundDetection method added to the grpc backend interface left two test doubles (fakeBackendClient, fakeGRPCBackend) incomplete, so core/services/nodes failed to compile under `go vet`/`go test` (go build missed it: the doubles live in _test.go). Add the method to both, mirroring their existing Detect mock. Repairs CI for the nodes package. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ced): decouple realtime sound detection from VAD (sound-only sessions) Sound-event detection must activate on sounds, not speech, so it no longer runs through the voice VAD/transcription path. A sound-detection-only pipeline (sound_detection set, no transcription/LLM) now: - is accepted by prepareRealtimeConfig (sound_detection counts as a pipeline stage), - builds a lightweight model via newSoundDetectionOnlyModel (no VAD/STT/LLM/TTS loaded), and - defaults the session to turn_detection none (no VAD) with no transcription stage, so the client drives windowing via input_audio_buffer.commit (option A: client-side sliding window). The per-PCM C-API already supports arbitrary windows. commitUtterance gains a sound-only branch: it emits the conversation.item.sound_detection event (scored AudioSet tags) and stops - no transcription, no LLM response. generateResponse is now guarded on a transcription stage being present, so a sound-only turn never invokes the LLM. Existing transcription/VAD sessions are unchanged (additive). Added a commitUtterance sound-only Ginkgo spec asserting it emits the sound event and neither transcribes nor generates a response. go vet + golangci-lint (new-from-merge-base) clean; openai suite green. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ced): register sound-classification backend in gallery + CI Mechanical backend-image registration for the ced sound-event classifier, mirroring the parakeet-cpp Go/purego backend everywhere it is wired up. - .github/backend-matrix.yml: add the ced build matrix, field-for-field copies of the parakeet-cpp entries (cpu amd64/arm64, cublas cuda 12/13 amd64, l4t cuda-13 arm64, l4t-jetpack cuda-12 arm64, sycl f32/f16, vulkan amd64/arm64, rocm hipblas, and the metal darwin entry), changing only backend and tag-suffix. dockerfile stays ./backend/Dockerfile.golang. - backend/index.yaml: add the &ced meta anchor (capabilities map per platform) plus ced-development and the per-arch image entries, each uri/mirror tag-suffix matching the matrix exactly. The model gallery (GGUF) entry is intentionally deferred pending the HuggingFace publish (TODO note inline). - scripts/changed-backends.js: add an explicit item.backend === "ced" branch in inferBackendPath mapping to backend/go/ced/, same mechanism and ordering as the parakeet-cpp branch (before the generic golang fallthrough). - .github/workflows/bump_deps.yaml: register mudler/ced.cpp -> CED_VERSION in backend/go/ced/Makefile so the daily bot bumps the pin. - swagger/{docs.go,swagger.json,swagger.yaml}: regenerated via make swagger so the existing /v1/audio/classification annotations land in the generated spec. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ced): server-side windowing for realtime sound detection (option B) Adds an optional server-driven sliding-window classifier so a sound-only realtime client only has to stream audio (no input_audio_buffer.commit): - Pipeline.sound_detection_window_ms / sound_detection_hop_ms config knobs. When both > 0 on a sound-only session, the server classifies the last window of streamed audio every hop and emits a conversation.item.sound_ detection event; the input buffer is trimmed to one window so a long stream stays bounded. When unset, the session stays client-driven (option A). Runs independent of VAD (sound events are not speech). - handleSoundWindow (ticker) + classifySoundWindow (one tick, extracted so it is unit-testable) + writeWindowWAV, which declares the true InputSampleRate (NewWAVHeaderWithRate) so the classifier resamples correctly. Goroutine is started after toggleVAD and torn down with the session (close + wg.Wait). - Register pipeline.sound_detection (+window_ms/hop_ms) in the config meta registry; the earlier realtime commit added pipeline.sound_detection without a registry entry, failing TestAllFieldsHaveRegistryEntries. This fixes that and covers the two new knobs. Tests: classifySoundWindow emits an event + trims the buffer to one window, no-ops on too-little audio; writeWindowWAV declares the given sample rate. go build/vet + golangci-lint (new-from-merge-base) clean; config + openai suites green. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ced): add ced-base GGUF model gallery entries (f16 + q8_0) The ced-base weights are now published at mudler/ced-base-gguf (Apache-2.0, converted from mispeech/ced-base). Adds gallery/ced.yaml (backend: ced + known_usecases: sound_classification) and two gallery/index.yaml entries (ced-base-f16 default, ced-base-q8 smallest) with sha256-pinned files, and removes the now-resolved TODO from backend/index.yaml. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ced): add tiny/mini/small GGUF model gallery entries Publishes the rest of the CED family (same architecture, metadata-driven port verified end-to-end on ced-tiny) to mudler/ced-{tiny,mini,small}-gguf and adds their f16 + q8_0 gallery entries: ced-tiny (5.5M, edge/Pi-class) f16 11MB / q8_0 6MB ced-mini (9.6M) f16 19MB / q8_0 11MB ced-small (22M) f16 42MB / q8_0 23MB All sha256-pinned. ced-base remains the accuracy default. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(ced): point gallery entries at the consolidated mudler/ced-gguf repo All CED quantizations (tiny/mini/small/base, f16/q8_0) now live in a single HuggingFace repo, mudler/ced-gguf, instead of per-model repos. Repoint the 8 gallery model entries' urls + file uris accordingly. sha256 and filenames are unchanged. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(ced): bump CED_VERSION to the short-clip fix Pin the ced backend to ced.cpp 99c6ed3, which fixes a crash on any clip shorter than target_length (~10.11s): time_pos_embed was added at its full 63-frame grid instead of being sliced to the clip's actual time grid, tripping ggml_can_repeat in ggml_add. Surfaced by the live realtime e2e (sub-10s windows) and gated with a short-clip parity test upstream. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(ced): list ced.cpp as a LocalAI-team engine + backend-guide directive - README.md: add ced.cpp to the "native C/C++/GGML engines developed and maintained by the LocalAI project" table. - docs/content/features/backends.md: add a Sound Classification backend category (sound-event classification / audio tagging) listing ced.cpp. - .agents/adding-backends.md: add a "Documenting the backend" section and two verification-checklist items requiring new backends to be documented in the backends.md category list, and in-house native engines to be added to the README maintained-engines table. This directive was missing. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(ced): repin CED_VERSION to the v0.1.0 release commit ced.cpp history was squashed into a single release commit (tagged v0.1.0), so the previous pin (99c6ed3) no longer exists upstream. Pin to c04ac14, the v0.1.0 release commit, so the backend builds against a commit that exists. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ced): silence gosec G304/G103 + govet unsafeptr on audited paths - sound_classification.go: os.Create(dst) where dst = temp dir + path.Base of the upload (no traversal). #nosec G304, matching the depth-anything-cpp handler. - goced.go: reading a NUL-terminated C string from a libced-owned buffer. #nosec G103 (gosec) + //nolint:govet (golangci-lint's unsafeptr check), since the uintptr is a C-owned malloc'd buffer, not Go-GC memory. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-06-22 01:00:28 +02:00
LocalAI [bot]andEttore Di Giacinto	9565db5f94	feat(models): model aliases - redirect a model name to another configured model (#10414) ... * feat(config): add model alias field and self-validation Add ModelConfig.Alias (yaml: alias), IsAlias(), and an alias short-circuit at the top of Validate() that rejects self-reference and forbids setting backend/parameters.model on a pure-redirect alias. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(config): resolve and validate model alias targets in the loader Assisted-by: Claude:opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(middleware): resolve model aliases and stamp requested/served identity Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(modeladmin): reject alias configs with invalid targets on create/edit Validate alias targets at create/swap entry points (ImportModelEndpoint, EditYAML, PatchConfig) so a dangling, chained, or disabled alias target is rejected at save time rather than surfacing as a runtime error. Assisted-by: Claude:opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(api): add GET /api/aliases to list model aliases Adds an admin-gated read-only endpoint that lists every model alias config as {name, target} pairs, backed by the loader's existing GetAllModelsConfigs(). Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(mcp): add set_alias and list_aliases tools Expose model-alias management over the LocalAI Assistant MCP surface: list_aliases (read-only, GET /api/aliases) and set_alias (mutating). SetAlias is swap-first: PATCH /api/models/config-json/:name swaps an existing alias's target (validated, non-destructive) and a 404 falls back to POST /models/import to create a fresh {name, alias} config. The inproc client mirrors this via ConfigService.PatchConfig + a create path modeled on ImportModelEndpoint. Deletion reuses delete_model. Assisted-by: Claude:claude-opus-4 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * style(mcp): replace em dashes in alias tool comments Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(config-meta): expose alias as a model-select field Add an 'alias' section to DefaultSections() and an 'alias' field override in DefaultRegistry() so the schema-driven React editor renders the new top-level ModelConfig.Alias field as a model picker in its own section. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add alias template card and Manage alias badge Add an 'Alias / Routing' template to the create-flow gallery that seeds a minimal name + alias config, and a read-only 'alias -> target' badge on the Manage Models tab. The capabilities row payload does not carry the alias field, so the badge resolves targets from GET /api/aliases looked up by name. Assisted-by: Claude:claude-opus-4 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: document model aliases Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(swagger): regenerate for GET /api/aliases Adds the /api/aliases path and AliasInfo schema generated from the ListAliasesEndpoint annotation. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(localai): check os.RemoveAll error in aliases_test Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: correct alias conversion docs and advertise /api/aliases in instructions Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(mcp): write alias config 0600 to satisfy gosec G306 The inproc createAlias path wrote the alias YAML with 0644, which gosec flags as a new G306 finding on the PR. The LocalAI process is the sole reader/writer of model configs, so 0600 is correct and keeps the scan clean. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-06-20 22:38:42 +02:00
Richard Palethorpe	3fa7b2955c	feat(pii): NER tier engine — privacy-filter.cpp backend + NER-centric PII filter (#10360) ... Squashed feat/pii-ner-tier-engine rebased onto master (was 45 commits; see backup/pii-ner-tier-engine-prerebase). Net change: - privacy-filter.cpp: standalone GGML engine for the openai-privacy-filter PII/NER token classifier, wired as a LocalAI gRPC backend (CPU/CUDA/Vulkan). TokenClassify moves off the patched llama.cpp path onto this backend. - PII filter reworked to be NER-centric (encoder/NER detection tier scanning whole conversations as one document), with a recreated bounded restricted- regex secret-matching pattern detector tier alongside it (per-model pii_detection.builtins / .patterns + core/services/routing/piipattern). - Detection labelled by source (ner vs pattern); backend trace / confidence / debug observability; analyze/redact exposed as a synchronous API. - Instance-wide default detector policy + per-usecase default-on; request filtering extended to completions, embeddings, edits & Ollama. - React UI: NER-centric PII editor, detector-models table, pattern/builtins editor, middleware default-policy UI. - Gallery: privacy-filter-multilingual token-classify model + NER install filter; token_classify known_usecase; batch sized to context for NER models. privacy-filter backend registered in the backend gallery (cpu/vulkan/cuda-13 meta + image entries with a capabilities map) matching its CI matrix jobs, and an /import-model auto-detect importer (PrivacyFilterImporter, narrow privacy-filter GGUF detection) replacing the prior pref-only registration. Reconciled against master's independent evolution: - Dropped master's PIIPatternOverrides feature (global-pattern runtime overrides + /api/pii/patterns API + runtime_settings.json persistence). The per-model NER + pattern-detector design supersedes it; it was built on the global redactor pattern set this branch replaced. - Reverted the llama.cpp Score carry-patch (0006-server-task-type-score): removed the patch and restored master's grpc-server.cpp Score RPC (direct llama_decode, slot-loop bypass) and LLAMA_VERSION pin, plus master's model_config validation forbidding score + chat/completion/embeddings on llama-cpp. token_classify is unaffected (it runs on the privacy-filter backend, not llama-cpp). Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-06-18 11:45:22 +01:00