1210 Commits

Include renames

Author	SHA1	Message	Date
Ettore Di Giacinto	f2cc5a25cb	fix(grpc): let the transport answer through the wrappers, not only past gRPC ... Re-review round 2. One blocking defect, and it was the concern I filed myself last round and mis-scoped as a future trap. It was live, and it sat on the most destructive reaping path of the five. RouteResult.Client is an InFlightTrackingClient, over a FileStagingClient when a stager is configured. model_router puts that on the cached remote model and pkg/model's checkIsLoaded asks IT whether the transport failed. Both wrappers embed grpc.Backend, which does not declare LastDialError, so the type assertion read nil and the guard added last round fell straight through to the old eviction. That eviction sends backend.stop over NATS to every node holding the model and deletes every replica row, where the other sites delete one. The spec covering it built a bare client by hand, which is why it passed while production did not. This is the third time in this task a correct fix was disarmed one layer out, so the fix is a mechanism rather than two methods. BackendUnwrapper is one line per decorator, LastDialErrorOf walks the chain, and both consumers now call it instead of each keeping its own assertion. One implementation, no per-caller policy to get wrong. Sweeping every type that embeds or holds a grpc.Backend found a third decorator the review had not named, and it is itself a reaping consumer of the same collapsed signal. ConnectionEvictingClient is built for remote models in initializers.go and its evict callback runs ShutdownModel; it fires during INFERENCE rather than on a health check, so a tunnel blip mid-request was enough to stop a model that was loaded and serving. It consults the transport first now. A locally spawned backend has no custom transport, so that path is unchanged byte for byte. Everything else touching a Backend is a consumer rather than a decorator; there is no fourth. The probe cache joiner shape is pinned. It was the right design last round with nothing holding it: the mutation back to a closed-over variable passed all 602 specs in the package. Eight goroutines coalesced on a probe that blocks on a channel now assert every joiner gets the leader's REASON and not just its answer, which is the difference between a leader declining to reap and its seven joiners reaping on the leader's own observation. The LastDialError scope note claimed an exactness it does not have at checkIsLoaded, which reads a shared long-lived client after releasing opMutex. It now says which caller is not exact, why the imprecision is accepted there, and what making it exact would cost. The four-outcome table in the docs still said a worker with no live owner is treated as absent and rescheduled, contradicting the code and the paragraph nine lines below it. None of those outcomes is absence any more, and the table says so, names the fifth, and points at the heartbeat as the thing that does decide presence. Five mutations, each reddening named specs, including the two the reviewer found surviving. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	4945e44595	fix(cluster): make "no route" a condition of its own, and let it out of the package ... Review round 1 on task 6. Five blocking findings, all with the same root: the conditions the dialer kept apart were erased one layer out, because every one of them arrived at core/services/nodes as a gRPC codes.Unavailable, which is also what a backend process that died produces. Four call sites acted on that by deleting a replica row, one of them after a single failed probe. The fifth condition is ErrNoRoute: this replica could not get a request to a worker's backend, and no claim at all about the worker. A worker's presence is its HEARTBEAT, which nodes owns; a route is a separate fact that cluster owns, and the two now differ. They differ in normal operation, not exotically: a worker that has not dialled its tunnel yet after a frontend-first upgrade is unroutable on every request while it heartbeats and serves. Two properties, both mutation-tested. Every failure to resolve or open a route carries ErrNoRoute, so a consumer has one check to make. No failure carries an absence sentinel: routeFailure is the single place that rule lives, and it keeps ErrNoConnection and ErrInstanceNotFound in the message and out of the unwrap chain, the guarantee unreachableError already made for peers. Everything else stays matchable, so ErrNotOwner and ErrPeerUnreachable are unchanged for anyone who can act on them. A worker's own refusal carries no umbrella, because a worker that answers has demonstrated it is there and that is the only real evidence on the path. Crossing the boundary needed a value, not a code. NewClientWithDialer wraps the dialer and records each outcome; LastDialError hands it back behind a narrow interface, and nodes.unroutable turns it into ErrWorkerUnroutable with the cluster sentinels still in the chain. A spec asserts a dial failing with ErrNoRoute plus ErrPeerUnreachable arrives matching all three and matching neither absence sentinel. The sweep found a fourth site the review had not named: pkg/model checkIsLoaded evicts a remote model on a connection error, and a tunnel dial failure is one. Four other reap sites were cleared with reasons - inflight and the worker authoritative pass reap only on semantic answers, scale-down is driven by last_used, abandoned loads decide on the node's heartbeat. Every fixed site also grew the opposite spec, so the new check cannot pass by never reaping. probeCache carries the reason through singleflight rather than a closed-over variable. A variable is only written by the goroutine that runs the probe, so the leader would correctly decline to reap while every joiner reaped on the leader's own observation; a mutation reproduces exactly that. The docs sentence promising LOCALAI_WORKER_TUNNEL=false restores direct dialling is gone. There is no such path, so it said the operator could take a worker dark and call it a rollback. Replaced with the upgrade order that is actually safe. The deadline spec the reviewer found vacuous now waits on the dial context's own Done channel before touching the stream, so the armed deadline has really expired; the mutation that survived for the reviewer reddens it. Nine mutations, each reddening a named spec, including both halves of isAbsenceClaim independently. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	d1b33140da	feat(cluster): reach every worker through its tunnel, never its address ... The tunnel, the fence, the registry and the relay were all built and none of them carried a byte: every dial from the frontend still went to the address a worker registered. This is where that stops. One WorkerDialer resolves where a worker's tunnel is held, opens a stream on it locally or relays through the owning replica, and hands back a conn past both handshakes; gRPC, the file stager's HTTP client and the log-streaming WebSocket are all pointed at it. A worker's address stops being somewhere to connect to and becomes the name of which backend process a stream is for. It still appears in URLs, logs and errors, because that is what identifies the process; what it no longer decides is where the bytes go. Nothing falls back to dialling it. BackendClientFactory now has exactly one method, NewClientForNode, and returns an error where there is no way to reach the worker. The direct-dial constructor was removed rather than kept beside it, because leaving one on the interface keeps the bypass one word away from every call site that holds an address, which is all of them. The second construction path is closed too. DistributedModelStore built remote models with a nil client, and pkg/model.Model.GRPC then dialled the raw address lazily on first use - reached in production by ShutdownModel's Free and by the backend monitor's Status. Those models now carry the tunnel-backed client, and a model that cannot be given one is logged and not listed. Four conditions stay unmixable, and one path produces absence: the dialer answers ErrNoConnection only where Owner's liveness join did. A peer that will not answer, a stale ownership row, a worker's own refusal and a missing relay path are each reported as themselves. This matters because nodes ACTS on absence, and the collapse would have it reclaim the models of a worker that is connected and busy. That is not hypothetical. Writing the mutation for it exposed the bug in this change's own first draft: probeHealth returned bare false when it could not build a client, and tryWarmPath deletes the replica row on a false probe. A frontend whose dialer broke would have emptied node_models for the whole deployment while every model kept running. probeHealth now returns alive and probed separately, the reconciler gets a ProbeUnknown outcome that neither advances nor clears a failure streak, and the health monitor skips rather than counting a miss. Task 5 left the relay's open timeout at a fixed 15s and said so: no operator has the information to set it, because the number that matters is the original client's remaining budget, which is invisible on the relay side. The dialer has that budget, so it now states it in the relay request frame and the owner takes the smaller of the two. It can only shorten - a patient client must not be able to park a relay goroutine and a stream slot on a worker that stopped accepting. Zero is written as no budget at all, since on the far side the number zero is a caller with nothing left and would refuse healthy traffic. Seven mutations, each reddening a named spec: peer-unreachable as absence; the local-failure guard dropped; max instead of min on the budget; the nil-client model restored; ProbeUnknown falling through to the reaper; OwnerRow instead of Owner; probed collapsed into alive. The first budget spec passed for the wrong reason - a handshake deadline, not the relay - and was replaced by three that each assert one link, including one where the spec plays the owning replica and reads the budget out of the frame instead of inferring it from a clock. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	5da440ae3f	fix(cluster): express the splice policy once, and pin the relay's budgets ... Review round 1 on task 5. Eight non-blocking items, all addressed. The classifier read Remote in two predicates with a report-by-default fallthrough behind them, so reverting either read left the whole suite green: the error reached the same answer down the other path. A correctness argument that rests on mutation evidence cannot afford a shape that cannot be mutated in pieces, so the two predicates collapse into one muxVerdict deciding each error type once. Falsifying either Remote read now reddens exactly one spec. Three claims the comments made loudly and nothing tested: - clearing the header read deadline before the splice. Deleting the clear left all 49 focused specs green, while in production it is the difference between a relayed response that streams for an hour and one that dies after fifteen seconds of quiet; - the open budget bounding the open and nothing after it; - closing the worker-side stream when the acceptance reply cannot be written, which leaks one stream on the worker per failure. All three are pinned now. The first two share a spec that sets both budgets to 50ms and then watches the conversation outlive them by ten times, which is an assertion about an event that must not happen and so is the one wait a channel cannot replace. The third drives the relay with a peer stream that delivers a request and then fails every write, because no pair of live yamux sessions can be made to fail that write on cue. The disjoint-vocabulary argument was specced for the accepted frame only. Both refusal directions are covered now, and asserted as "not one of the other hop's sentinels" rather than merely "an error", since reading a relay refusal with the tunnel's reader always errors and the question is whether it errors as the wrong thing. The open budget stays non-configurable, and says so: the number that matters is how long the original client will wait, which is not known on this side and is not something a deployment-wide constant can stand in for. The honest fix is the caller's remaining budget travelling in the request frame, which belongs to the dialler that has the budget. Two comments corrected: nothing deadlines the peer stream after the clear, so the tunnelled protocol's own deadlines cannot be what justifies clearing it; and the membership sweep deletes departed replicas but reports only how many, so identifying them is work that would have to be done, not knowledge waiting to be plumbed. Recorded at muxVerdict: a remote RST that does not ride a typeWindowUpdate frame yields the bare sentinel and is still silenced, which is unreachable between two go-yamux peers but keeps the new rule from reading as unconditional. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Opus 5 [claude-code]	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	f358a00bc2	feat(cluster): relay a peer's stream onto a worker tunnel held here ... A worker holds ONE tunnel and it lands on ONE frontend replica, so with N replicas behind a load balancer roughly (N-1)/N of requests arrive somewhere that cannot reach the worker directly. This is the piece that carries them: the SessionStore stream handler reads which worker a peer's stream is for, opens a stream on the tunnel this replica holds, and splices the two. Splice has had no production caller since phase 1. It has one now, and being the first caller it settles the two endings phase 1 deliberately left open, both of which read as normal termination until now: - a peer-initiated *StreamError{Remote: true}, which yamux builds only from an RST frame the far side sent (stream.go:432-449); a reset this side asks for carries Remote: false, and Splice never resets anything, its own Close sending a FIN; - a graceful ErrRemoteGoAway, which handleGoAway returns for code goAwayNormal (session.go:829-833) and close hands unwrapped to every live stream (session.go:328-337). Both truncate whatever was in flight. Reporting them as normal termination is how a half-finished inference comes to look like a short one that completed, so both are now reported; the local forms stay silent, because those are the teardown Splice provokes itself. The decision cannot live in a caller reading Splice's result, since a result already mapped to nil carries nothing left to reclassify, so it lives at the classifier with the reasoning beside it. The relay logs it at debug: a client cancelling a relayed request produces one per cancellation, and the truncation is separately visible to the frontend's own gRPC or HTTP client. The relay hop gets its own request and reply frames. They have to be distinct from the worker tunnel's, because a relayed stream carries both hops' frames back to back, and a vocabulary shared between them would let a reader applied to the wrong hop hand back a plausible sentinel belonging to the other. Its three refusals stay apart for the reason the worker's three do: ErrNotOwner is a routing fact and the caller should resolve the owner again; unavailable is infrastructure at this replica and a retry is worth something; bad-request is the caller's bug. None of them is, or may be built over, an absence error. One hop, always. A stream naming a worker this replica does not hold is refused, never resolved and relayed onward, so a stale ownership row cannot become a loop between two replicas each certain the other holds the worker. PeerPool is constructed and closed alongside SessionStore, so both halves of the peer mesh now have an owner and a shutdown. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	489550d317	fix(worker): make the tunnel credential's node-type gate actually structural ... Re-review follow-up, three items. Two are the overclaiming-comment class again, and the first is that class with a real defect underneath it. attachTunnelToken said "enforcement is therefore structural": an ineligible node never gets a credential, so its hash stays empty and the tunnel route's empty-hash branch does the refusing. That was true for a node that had always been an agent and false for one that had not. Register upserts by NAME, so a backend node re-registering as an agent keeps its ID, and Register's struct Updates zero-skips the credential column while writing the new node_type. The early return left the credential the node earned as a backend sitting on a row that is now an agent, and ConnectHandler never looks at node_type. Fixed by making the claim true rather than by softening it, because the mint-site gate was chosen precisely on the grounds that it was structural: an ineligible node now has its column CLEARED, unconditionally, so the invariant does not depend on what the row happened to contain. A spec pins it and was red before the change. Same shape as the Register-upserts-by-name hazard already carried forward: a name is not an identity. Second, loopbackHost claimed to be the only host any tunnel stream is ever dialled on. It is not: fixedService dials whatever Run built it from, which is this worker's own LOCALAI_HTTP_ADDR, and loopbackAddr rewrites only a wildcard bind, so an operator who binds the file-transfer server to a routable address gets a routable dial. The property that matters is narrower and is what the comment says now: the frontend cannot STEER the dial. The grpc tag builds its address from a constant and a validated port with nothing from the wire reaching the dialler, and the http tag ignores its target entirely. Worth stating exactly rather than summarising, because the argument about what a stream can reach rests on knowing which hosts are reachable, and an overstatement at that site is what would let someone conclude the constant alone is doing the work. Third, a spec named "without allocating it" measured no allocation. It now asserts the mechanism the defence actually rests on, that the reader consumes the two length bytes and not one byte of the body, through a counting reader. The input carries a body on purpose: against input that ends after the header the assertion would pass with the limit check deleted. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Opus 5 [claude-code]	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	4ec400d29d	fix(worker): spec the tunnel's routing table, which was the SSRF boundary ... Review follow-up. One blocking finding and seven others. The blocking one first, and it is this project's recurring shape: the untested path. loopbackService is the function whose comment calls the discarded host "the security property this function exists for", and nothing tested it. The reviewer replaced its body with a dial of whatever the frontend named, no port range, and all 131 specs passed. Every spec installed the permissive test dialler, so the real routing table was exercised nowhere. It now has specs, and the property is stated as reachability rather than as a property of the code: a listener on 127.0.0.2 that only the frontend's target names must NOT be reached. Plus the port-range table, fixedService, loopbackAddr, tunnelEndpoint, and the table itself, which moved out of Run into tunnelServices so it can be built without starting a worker. One spec drives a real stream through that table over the wire, so the routing rules are exercised end to end at least once rather than only in isolation. The reviewer's mutation now reddens ten specs, and six narrower ones redden between two and four each, so no spec is riding on another. The shape changed too, not only the coverage. The dial address is built from a loopbackHost constant and strconv.Itoa of a validated int, so nothing derived from the wire reaches DialContext at all: restoring the hole takes ADDING a data flow, not deleting a check. And a taxonomy fix found while specifying it. A port outside this worker's allocator range was reported as unavailable, which tells a frontend to retry something that can never work. It is a bad request now, and a backend that is merely not listening yet stays unavailable, which is the retryable one. Agent nodes no longer get a tunnel credential. Nothing dials into an agent worker, so a tunnel replaces nothing for it and no client would open one, and the gate is at the mint site rather than in the handler: with no credential minted the hash stays empty and the existing empty-hash refusal covers it, so enforcement is structural. Two comments and one doc paragraph said an anonymous registrant gets a "working" credential. With auto-approve off the node is pending and the credential is inert, which is the distinction this same change argues three files away to justify minting for pending nodes at all. A refusal reason over the frame limit was cut on a byte boundary and could split a rune. It cuts on a rune boundary now, and the code survives truncation, which is what keeps a refusal classifiable. Also: the pending-node spec asserted only that a credential was non-empty, so a credential derived from the shared token passed it; it now pins per-node-ness the way the headline spec does. The tunnel handler's citations into nodes.go were stale before this branch landed, having been written against a file the same commit was editing, and are by function name now. The static-NATS path says plainly that an externally forced rotation locks it out until restart, and where that gets fixed. tunnelproto gained direct specs, including that a read failure is never reported as a refusal. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Opus 5 [claude-code]	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	099e2b00af	feat(worker): dial, hold and serve the tunnel, on a credential of its own ... The worker end of the tunnel. It dials wss://<register-to>/api/cluster/connect, holds one yamux session as the CLIENT, and serves every stream the frontend opens on it. Nothing dials into the worker, which is the point: no inbound port, no reachable address. Each stream opens with a length-prefixed frame naming a tag and a target, and the worker answers before either side speaks the tunnelled protocol. The reply is sent on every stream, not only on refusal, because the protocols carried here are client-speaks-first and a reply sent only sometimes would arrive interleaved with a response body. Two tags today: grpc reaches a backend process, and only on 127.0.0.1 within this worker's own backend port range, because a tunnel terminates inside the worker and letting the frontend name a host would make every worker a proxy into its own LAN; http reaches the worker's file-transfer server, whose address the frontend is not asked about. An unknown tag, an unreachable local service and an unparseable request are three refusals and stay three on the wire. A frontend gives up on the first and retries the second. Each is answered AND the stream is ended: a worker that says why and leaves the stream open has parked the caller on a request nobody will answer, and a deadline on the far side cannot tell that from a slow worker. The specs assert the stream ends rather than that an error occurred, which is what phase 1 shipped in three places and held in none. Reconnects double from 500ms to a 30s ceiling, each wait drawn between half the interval and all of it, and the interval returns to its floor only after a session that LASTED. Resetting on connect is how a rolling restart, where every dial succeeds and dies moments later, becomes a retry storm against the first replica back up. Nothing is assumed to survive a reconnect: the credential is read at dial time, never captured. And the credential is now real. The tunnel endpoint advertised authenticating a worker against its own secret, but registration stored the hash of the shared registration token, so a leak plus a known node ID still opened a tunnel. Registration now mints a per-node secret, returns the plaintext once as tunnel_token, and stores only its SHA-256 in a new column; the endpoint compares against that and does not fall back to the old one. Rotating on every registration follows from storing only the hash, since a re-registering worker cannot be told the secret it already holds; its live tunnel is unaffected, because the credential is checked when a tunnel is dialled and never again. Unlike the agent API key and the NATS JWT next to it, the credential IS issued to a node awaiting approval: the tunnel route re-reads the node's status on every dial and refuses a pending one, so it is inert until an admin acts, and withholding it would strand every worker that registers exactly once. A node that has not registered since this change cannot tunnel, and the column cannot be back-filled because the plaintext only ever existed in the response that minted it. The boot warning that said tunnels need LOCALAI_REGISTRATION_TOKEN is replaced: it was true while the tunnel authenticated against that token's hash, and says the wrong thing now. What is still true, and is what it warns about instead, is that without one, registration itself is unauthenticated. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	2a9f771a52	fix(testutil): clear the maintenance timeouts at connection startup ... The guard added last commit was circular. It cleared the maintenance database's timeouts by executing SET statement_timeout = 0 on a connection that had already inherited that database's bound, so the statement clearing the bound ran under the bound it was clearing. Under the white-box spec's deliberate 1ms that gave it 1ms, and it failed roughly once in fifty at 8-way concurrency with SQLSTATE 57014. The guard against invisible load-dependent flakes had become one. The clearing is now delivered as a connection startup option, options=-c statement_timeout=0 -c lock_timeout=0 on the maintenance DSN, so there is no statement left to abort. Raising the imposed bound would only have bought headroom and left the circularity in place. pgx puts every URL query parameter into settings, options is absent from notRuntimeParams so it becomes a runtime parameter, and runtime parameters are copied into the startup message (pgconn/config.go:340-378, 606-617; pgconn/pgconn.go:382-388). The spec now discriminates on pg_settings.reset_val, the value in force when the connection started: 0 for a startup option, 1ms for a session SET. A first attempt using a deliberately slow first statement did NOT discriminate, because under the circular design the SET is itself the first statement, so by the time a spec runs anything the session is already unbounded. Reinstating the circular clearing now reddens the spec deterministically rather than intermittently. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	27f9e78c8a	fix(testutil): stop the shared-database change from disarming two regressions ... Two advisory-lock specs named their database by literal, ALTER DATABASE testdb. Once the test helper started handing every spec its own database on a shared server, that statement landed on the maintenance database and did nothing to the one the spec was holding, so both specs went green having never reproduced the condition they exist for. They regress a model-load advisory-lock wedge that has already shipped to production once, so the previous commit's de-flaking silently disarmed a regression test for a real deployed bug. Both sites now read the name back with current_database() and, more importantly, assert the override actually landed before relying on it. A literal name can go stale again; an assertion that the setting is in force cannot pass while it is not. Removing either production override now fails the matching spec with the real 55P03 and 57014 again. That literal also meant every CREATE DATABASE and every DROP ... WITH (FORCE) ran under the 300ms bound it set on the maintenance database, which is a new load-dependent single-spec flake inside the change that was meant to remove one. The helper's maintenance connections now pin one connection and clear both timeouts on it, so no setting a spec makes can bound them, and a white-box spec imposes the leak deliberately and proves it does not reach them. Also pins the reclaimOne gate deferral the previous commit added without a test, by panicking inside the re-claim's own claim statement, and drops the per-dial empty-token log line to debug now that the boot warning says it once. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	15cb4be76e	fix(cluster): harden the worker tunnel, and stop starting a database per spec ... Review follow-up. Twelve findings, none blocking, grouped here by what they protect. Panics. The handler now recovers between the WebSocket upgrade and the hand-off, the way the peer link next door already did: net/http recovers the panic but leaves the hijacked socket open, so without this a worker keeps a session this replica has no entry for and will never detach. The claim gate in Attach and reclaimOne is now released with defer, so a panic under Claim cannot wedge one node's gate for the life of the process. SetTunnels gained the nil-receiver guard its sibling Stop has. Operability. A deployment with no registration token stores an empty token_hash on every worker, so every tunnel dial 401s forever on a frontend that looks correctly configured. That now warns at startup, logs its own line rather than sharing the "wrong token" one, and is stated in the docs together with the fact that setting the token later needs the workers to register again. Authorization. A node still awaiting admin approval is refused with 403. The rest of /api/node/ gates on nothing, but the two places that hand a node something durable, its API key and its NATS credential, both refuse a pending one, and a tunnel is that kind of grant. Draining and unhealthy nodes keep their tunnels on purpose. Comments that claimed more than the code. The global auth middleware does run on this path and then declines to reject; the future per-node secret only lands without a change here if it lands in TokenHash; the empty-hash guard is defensive rather than deciding; ClusterPathPrefix is no longer only replica-to-replica; the docs no longer say a reaped replica re-claims unconditionally. And the test harness. SetupTestDB started a PostgreSQL container per BeforeEach with a readiness deadline it asserted on, which is one chance per spec to fail one spec inside its setup, anywhere, never twice in the same place: the shape of the flake seen twice here and never reproduced. It now starts one container per process and creates a database per call, which is the pattern tests/e2e already proved. Isolation is unchanged and is now asserted for the first time. All 69 call sites are untouched; the eleven consumer packages run 1404 specs green, and jobs went from 34.3s to 3.3s, agents from 13.8s to 1.9s, cluster from 97.4s to 37.5s. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Opus 5 [claude-code]	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	4d26e648bc	feat(cluster): open the door a worker dials its tunnel through ... A worker needs no inbound port: it dials GET /api/cluster/connect, the connection becomes one multiplexed yamux session, and the frontend opens a stream on it per request. This adds the endpoint that accepts that dial and attaches it to the tunnel registry. The dial is authenticated against the NODE's own stored token hash rather than the deployment's registration token. That is the mechanism, not yet the isolation, since a worker still registers by presenting the shared token; what it rules out is the shortcut of comparing against the configured value, which would have to be unpicked the day workers get their own secrets. Every refusal happens BEFORE the WebSocket upgrade, so a dialer reads an HTTP status rather than a handshake error. The route is registered in every deployment, single-binary ones included, which is what puts it in front of the route-coverage test that holds that rule in place; with no node registry it refuses every dial, and tells a credentialed one the frontend has no cluster rather than that its token is wrong. A lookup that FAILED is answered as a failure. Reporting a database that could not be read as "unauthorized" would send a worker re-registering, throwing away the identity its tunnel and loaded models are keyed by. Wires the tunnel registry in core/application/distributed.go and hands it to the membership loop. Without that call the re-claim after a replica is reaped had no production caller and could never run. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Opus 5 [claude-code]	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	9e4df19045	fix(cluster): keep a session close out of the per-node claim gate ... The gate is justified by being held for one claim round trip, and Attach held it across the close of the session it superseded. Closing a yamux session closes the underlying conn and then waits for both its send and recv loops to exit, and the send loop can be inside a write bounded only by ConnectionWriteTimeout, so that is a wait on other goroutines. It must not stand between a worker re-dialling this node and its claim. The gate is now released after the store and before the close, which also makes Attach match reclaimOne, where it has always been released explicitly on every path. This is safe because a superseded session is no longer reachable from the map by the time it is closed: the next re-dial replaces an entry that already names the new session. Pin the re-claim half of the gate too. A worker that re-dials between a re-claim's commit and its record leaves the row carrying the re-dial's epoch while the entry carries the re-claim's, so the attachment holding the socket releases an epoch the row does not have and the row outlives it, with nothing to sweep it while this replica is alive. Only Attach's half of the serialisation was asserted; keying the two apart left every spec green. Also take the test hook's action under the lock that guards whether it has fired. It was written from the spec's goroutine and read from whichever goroutine issued the statement, which is a race in the harness that pins the serialisation specs. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	2466e42906	fix(cluster): make a claim and its record indivisible per node ... Two Attach calls for one node both claim, and PostgreSQL serialises the two upserts, but nothing ordered the two map writes against the two commits. The entry left installed could be the one whose claim lost the row, and its Detach then released an epoch the row does not carry, so the release matched nothing and the row survived the socket. Nothing swept that row. This replica is alive and heartbeating, so ReapStale leaves its rows alone, and no reconnect is coming for a worker that has gone. Owner kept naming this replica as the live owner of a tunnel it no longer held, and every dialer sent here was answered ErrNotOwner, which is the relay into a replica that cannot serve the request that this phase exists to prevent. Claims for one node now pass through a gate, so claim and record are indivisible. It is per node rather than one lock over the registry, the way PeerPool locks per peer: the claim is a database round trip, and a slow one for a single worker must not hold up Open for every other. Detach is not gated, because it takes no context and must never park behind an in-flight database call, and it changes no epoch. Reclaim takes the same gate, which makes its claim the newest one for that node, so it records the epoch on whatever attachment is installed rather than only on the one it listed. Refusing to record onto an attachment that replaced the listed one would leave that row with nothing able to release it. The interleave the gate does not cover is Detach, and a claim whose attachment detached while it was in flight is now released again rather than left behind. Also: restore Start's doc comment, which SetTunnels had swallowed; keep reaping other replicas when this one fails to rebuild its own row, rather than skipping the sweep along with the re-claim; scope the comment about an unnoticed dead socket to the keepalive of the session whoever accepted the tunnel built, since the worker session config does not exist yet; and pin the sortedness of Held, the nil-session refusal, and both re-claim interleaves with specs. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	032725c81b	feat(cluster): hold worker tunnels, and re-claim them after a sweep ... Phase 1 left the connection fence with a table and no sockets behind it. This adds the registry that holds them: Attach claims the node and then stores the session, Open hands out a stream over the tunnel this replica holds, Detach releases the claim it was handed, and Held names what this process is carrying. The claim is written before the session is stored. A claimant that installs itself and only then finds it cannot claim has, for that window, published a tunnel no row records, so Held names it while a peer asking Owner is told the worker is connected nowhere. ErrNotOwner is produced at one place, the map miss. It is a routing fact: some other replica may hold that worker perfectly well. A broken socket under a held entry is returned as itself, because answering "not held here" would send a dialer looking elsewhere for a worker this replica is holding. Epochs are compared for equality and never ordered. Claim guarantees an epoch is unique and never reissued; it does not guarantee the later claim draws the larger number, because the sequence value on the insert path is drawn before the row lock. The membership loop now re-claims on re-register, which closes the hole phase 1 named in ReapStale. A replica that stalls long enough is swept by a peer, losing its instance row and, in the same transaction, every connection it owned; Register rebuilds the instance row and nothing else, so without this it serves workers that every other replica reports as connected nowhere. Re-claiming draws a fresh epoch, so an attachment carries two: the token Attach handed back, which is what Detach matches and which never moves, and the epoch of the row currently held, which is what Release is given. Collapsing them would leave the re-claimed row outliving the socket with no caller able to remove it. A tunnel whose session is already closed is skipped rather than claimed back, because claiming is an upsert and would take the row from whoever holds the worker now. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	df7cebf642	fix(cluster): make the ownership comments say only what holds ... Review round 1 on the joined Owner read. The behaviour was accepted; three comments claimed more than the code delivered, one spec pinned less than its doc promised, and one pre-existing spec ranked epochs. instanceIsLive said every reader of instance liveness uses it, which was false: ReapStale spelled the complement by hand. The complement is now written as NOT (instanceIsLive), so "stale" is exactly "not live", including how each side treats a NULL last_seen, and the sentence is true. Inverting that predicate fails 3 reaper specs, so the routing is held. The Select("node_connections.*") in Owner was justified by a SELECT * hazard that cannot occur: with a join present and nothing selected, gorm expands the model's own columns table-qualified (callbacks.BuildQuerySQL), and the suite is green with the Select removed. It stays, because the projection should be a property of this query, and the comment now says that instead. Owner gained the dialect guard Claim has. now() and make_interval are PostgreSQL, so on the SQLite single-binary path it failed with "no such function: now", which reads as a missing migration; that regression already shipped once in phase 1. The refusal is deliberately not ErrNoConnection: a deployment with no cluster has no answer about ownership, and reporting absence would let a caller conclude the worker is not connected. A spec in the non-PostgreSQL block holds all three properties. The new specs aged rows by ten minutes, which any window between zero and ten minutes satisfies, so nothing tied Owner's window to the one the sweeper uses. They now age to just past InstanceLiveness, and a sibling ages to half of it and must still resolve. Widening the window tenfold fails 2 specs, narrowing it tenfold fails 1; before this both were silent. The concurrent-claim spec asserted the stored epoch was the highest handed out, and justified it with claims drawing their epoch after the row lock, which contradicts Claim's own doc: the insert path draws nextval while the tuple is built. It now asserts the stored epoch is one of the epochs handed out, and ranks nothing. OwnerRow's doc justified the function with a sweeper that does not call it. ReapStale deletes orphans with a set difference; the callers are this package's specs and one e2e assertion. It says that, and states plainly that a caller needing to know who owns a node in order to dial it wants Owner. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	a275147145	feat(cluster): resolve tunnel ownership against a live owner ... OwnerRow is a bare row read of node_connections. A connection row outlives the replica that wrote it: a replica that dies stops heartbeating, but its rows survive until a peer's sweep removes them, which is up to InstanceLiveness plus one InstanceHeartbeat later. For that whole window the table names a process that is gone. The next component phase 2 builds is the relaying dialer, and a dialer reading OwnerRow would relay into a corpse for roughly 35 seconds after every replica death, then report the worker as unreachable when it is in fact absent, which is the distinction the phase 1 end-to-end specs pinned. Owner is the resolving read: one statement joining instances, returning ErrNoConnection when the row is missing OR its owner is not live. Both cases are one answer on purpose, since both mean no replica here holds this tunnel; they differ only in which sweep has run. It is one statement, not a row read followed by an instance lookup, because between two statements the owner can die and the caller would act on an owner the second read would have rejected. OwnerRow stays, unjoined, for readers that need the row itself, and a spec holds the two apart: with an aged-out owner, OwnerRow still names it and Owner refuses, so neither can quietly become the other. The liveness predicate is now one string, instanceIsLive, shared by Live and by Owner's join. Two spellings of one fact drift, and this drift would show as a relay to a replica one query calls dead and another calls alive. It is table-qualified so it is unambiguous inside the join, and the cutoff stays on the database clock, so replica clock skew cannot widen or narrow the window. Both mutations were run. Dropping the liveness predicate from the join fails 3 specs, the aged-owner one among them. Replacing the database clock with a Go-side time.Now() fails 1: the aged-owner specs still pass, because the two clocks agree on one host, and only the recorded-SQL spec sees the literal timestamp. That is why that spec exists. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	3daf889f5e	fix(cluster): share one lock order, and correct the phase 1 comments ... ReapStale deleted from instances then node_connections while Deregister took them the other way round, both inside one transaction and both running concurrently by design: a replica shuts down while a peer sweeps it. Opposite orders let each hold the row the other waits for. PostgreSQL breaks the cycle by aborting one side, so the cost today is a warning rather than lost data, but the inversion costs nothing to remove. Deregister now deletes the instance row first. That is the order ReapStale is forced into anyway, since its connection delete asks which instance rows survived, so the sweeper is the fixed side. Both functions say the order is deliberate and shared, and name the other. A spec records the statements each path issues and asserts they delete from the same two tables in the same order; racing two transactions until they really deadlock would be flaky and could pass for the wrong reason. The rest is comment and spec accuracy, deferred from the phase 1 task reviews: - co-location does not imply loopback. Compose's usual host=postgres resolves to a bridge address and discovery works there; it is a DSN that NAMES localhost that yields a loopback source address. Corrected in the DiscoverAdvertisedAddr doc and in the spec comment that repeated it. - unroutableReason labelled every scoped address "link-local", including the class the check exists for, and formatted the IP with %s, which drops the %iface, so the reported address was not the one being rejected. Split into two cases, both rendered with their zone. CheckAdvertisedAddr passed zone "" and net.ParseIP rejects fe80::1%eth0, so a scoped literal looked like a name and collected no warning at all; the zone is now split off before parsing. - Splice's "Both callers satisfy it" claimed callers that still do not exist. It now names the two stream types the wake-on-Close property was verified against and says a phase 2 caller over anything else has to check it. - restored, short, why a socket-level ECONNRESET stays reported while a yamux reset does not: the yamux endings are the teardown Splice's own Close provokes, and whether an aborted request is routine is the relay's policy. - the real-yamux spec's far.Read had no deadline, so a stall parked the suite rather than failing it. - gorilla's SetWriteDeadline is conn.go:796, not 787. - ClusterPathPrefix is no longer derived from: the peer route spells its path out, because core/services/cluster must not import core/http/auth. The comment now points at the spec that holds them together instead of claiming a derivation the move removed. - the epoch spec asserted e2 > e1, an ordering Claim's doc tells callers not to rely on. It asserts uniqueness, which is what the fence guarantees, and is named for that. A sibling spec still described the epoch as incrementing in SQL when it is drawn from a sequence. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	bb4719e67d	fix(cluster): hold the guarantees phase 1's comments were claiming ... Review found the recurring class: assertions that a wrong implementation also satisfies. The "refuse promptly, never park the peer" guarantee was stated in three places and tested in none. Removing the Close from the no-relay branch left the whole cluster suite green, because the specs asserted only that some error arrived and yamux reports a read deadline as ErrTimeout: a parked stream satisfied that as well as a refused one. Both specs now require an ENDING, EOF or a reset, inside a deadline short enough that parking is unmistakable, and both go red when the Close is removed. Deregistration existed only in a comment. Membership.Stop ended the loop and left the row behind, so every clean rolling restart had peers dialling a corpse for the full liveness window; the shutdown comment described the opposite. Registry.Deregister deletes the row and the connections that replica owned, in one transaction, for the reason the sweeper does both, and an e2e spec pins departure inside a budget shorter than the liveness window so it cannot pass on the sweeper doing the work. Before: the spec times out with both replicas still live. After: 3.6s. The configured advertised address bypassed every check discovery makes, so the one value most likely to be copied between hosts, 127.0.0.1, was taken verbatim and would make every peer dial itself. Both paths now share one rejection rule: unparseable is refused, "this host" is warned about once and honoured, because a single-host deployment uses it correctly. Two comments claimed more than the code does. The sweeper said a stalled replica recovers via re-register; only its instance row does, while the connections another replica reaped stay gone and the sockets stay held here - phase 2 must re-claim, on re-register, every connection a replica still holds locally. And Owner became OwnerRow, documenting that the owner it names may be dead for up to InstanceLiveness plus a heartbeat and that any caller acting on it must join instances itself, so the deferred constraint lives at the call site rather than in a report; the plain name is left free for the joining version. Minors: warn once when the peer link mounts with no registration token, so an operator sees the cause rather than 401s; Stop no longer blocks forever when Start was never called; corrected the NewRegistry migration doc and an e2e comment that described a 6s window as "throughout". Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	dca8ce7149	feat(cluster): give phase 1 a call site, and prove it against real replicas ... Tasks 1 to 5 built an instances table, a splice, both halves of a peer link and an epoch fence, and nothing in the tree called any of it: no replica registered, no route was mounted, no sweeper ran. Proving phase 1 end to end therefore had to start by wiring it. A frontend in distributed mode now publishes the address its peers dial, heartbeats it, and sweeps replicas that stopped answering along with the connection rows they owned, in one pass so the two can never disagree about who is alive. It serves the peer link and owns the sessions peers dial in, refusing streams on them until phase 2 installs a relay: a session nobody accepts on does not fail a peer's Open, it hangs it. The address is the one peers use, not the one the process binds, and it is derived from the route to PostgreSQL. That derivation only holds while the database is remote, so LOCALAI_DISTRIBUTED_ADVERTISE_ADDR sets it explicitly and a replica that can determine neither warns and keeps serving rather than failing to start. Three e2e scenarios run against real local-ai processes, real PostgreSQL and real dials: replicas publish addresses that can actually be connected to; a sibling opens a stream over the peer link and is refused without the cluster token; and a killed replica is reported unreachable, never absent, loses the claim it held, and takes no worker with it. Each was verified by mutation: eight injected defects, each failing the scenario that claims to catch it. Also moves RegisterClusterRoutes to core/http/routes beside every other registrar, folds AutoMigrate and the epoch sequence into one cluster.Migrate, and turns the peer route's auth-coverage spec into a real assertion: it drives the request through the actual auth middleware instead of comparing two string constants, which the old spec would have passed even with the exemption deleted. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	e9514334fa	fix(cluster): keep the connection schema migratable on SQLite ... The NodeConnection model carried `default:now()`, which is PostgreSQL syntax reaching the DDL, so AutoMigrate failed on the single-binary SQLite path and took every SQLite caller of nodes.NewNodeRegistry down with it. Stamp the database clock as an expression inside Claim instead, the way Register already does, and leave the column plain. CREATE SEQUENCE is Postgres-only for the same reason, so it is skipped on another dialect, and Claim refuses that dialect outright: a fence that cannot draw a token must say so rather than fail later as a missing function. Also correct a claim the previous commit made in both the doc comment and its message. An epoch is unique and never reissued, but it is not ordered: the insert path draws its sequence value before taking the row lock, so a claim that inserts after a release can be handed a lower number than one already issued. Uniqueness is what Release needs, since it matches by equality; callers must never compare epochs for order. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	6f53701f4c	fix(cluster): draw connection epochs from a sequence so none is ever reused ... Release deletes the row, so a per-row `epoch + 1` restarted the numbering at 1 for the next claim. A replica could then be handed an epoch it already held: claim w1 at epoch 1, lose the link silently, watch another replica claim and release, reclaim and be handed 1 again, and its delayed cleanup for the first dead link would match the live claim and delete it. The fence has to be unique per node over time, not per row lifetime. Every claim now draws nextval from a dedicated sequence on both the insert and the conflict paths, so an epoch is never issued twice. The draw still happens after the row lock on the conflict path, so the winning claim still holds the highest epoch handed out. Also drop last_seen. Nothing maintained it and it was always equal to connected_at, but an indexed column named that way invites a second liveness clock; whether the owner is alive is Instance.LastSeen, and whether a claim is current is the epoch. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	edc7313f3b	feat(cluster): fence worker-connection ownership with a monotonic epoch ... A worker whose link is silently broken reconnects to another replica while the old owner's socket has not yet noticed. Without a fence both believe they own it. Claim is a single atomic upsert returning the new epoch, and a release must match both owner and epoch so a stale owner cannot delete a live claim. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	393797e7e3	refactor(cluster): make the cluster service a leaf and blame the caller's deadline ... The peer link's WebSocket adapter and route constant lived in core/http/endpoints/cluster, so the dialler in core/services/cluster had to import an HTTP endpoints package to reach them. That pulled echo, core/http/auth and core/config into a package whose doc says it is deliberately free of such dependencies, and it made core/services/nodes reach an endpoints package transitively. It also has no way forward: the worker-connect handler needs the tunnel registry and the node token store, both of which are cycles from there. Move WebsocketConn and PeerPath into core/services/cluster and let the endpoints package import it, which is the direction the rest of core/http flows. The route and the auth exemption still cannot drift apart, now asserted where both are visible rather than by a const reference across the boundary, and the assertion is stronger than the one it replaces: it pins the route under the prefix instead of pinning the prefix's spelling. Also guard the fresh-dial path with ctx.Err(), mirroring the cached path. A caller with a 300ms deadline dialling a live, listening peer was told the peer was unreachable, which would be enough for one impatient client to get a healthy replica routed around once the relay consults these errors. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	c7cb7e1ca5	feat(cluster): dial and pool yamux links to peer replicas ... Distinguishes a peer missing from the registry from a peer that will not answer: the second must never be readable as node absence, or a network hiccup between replicas evicts healthy workers. The distinction is a property of the error type rather than of the call sites. The unreachable error formats its cause into its message and keeps it out of its unwrap chain, so an ErrInstanceNotFound picked up on the dial path cannot reach a caller's absence check. One yamux session is cached per peer and re-dialled when OpenStream on it fails, which is how both a dead transport and a graceful remote go-away arrive. A reset of one stream never reaches the pool, so an abandoned request cannot cost every other worker its link. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	9fdd0bef55	fix(cluster): own the peer auth prefix in auth, and pin what the specs claimed ... The prefix constant moves to core/http/auth beside the check that uses it, and the endpoints package derives its route from there. Seven sibling endpoint packages already import auth, so the previous direction would have deadlocked the build as soon as this one registered in RouteFeatureRegistry, and it was dragging echo, gorilla/websocket and yamux into unrelated service packages. Four properties were argued in comments and held by nothing. Flipping the empty-token check to fail open, making SetWriteDeadline a no-op, returning a zero-length read for a zero-length message, and dropping the recover around the callback all left the suite green. Each now fails a spec that asserts the behaviour rather than the setter's return value. SetWriteDeadline takes the write mutex because gorilla keeps that deadline in a plain struct field applied at the next flush; SetReadDeadline must not take the read mutex, since it goes straight to the net.Conn and would otherwise block behind the read it exists to unblock. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	73da8089d6	fix(cluster): stop reading a bare EOF as a clean ending ... A dead yamux session does not always arrive wrapped. Session.close publishes shutdownErr and closes shutdownCh before it force-closes the streams, so a Write or Close landing in that window gets the raw cause back instead (session.go:507-510, 528-533), and for a peer that vanished the raw cause is a bare io.EOF. The generic io.EOF clause then reported the dead session as a clean completion. Remove the clause. A clean read-side EOF never reached it anyway: io.Copy consumes that and reports nil, and neither *yamux.Stream nor *net.TCPConn takes a WriteTo/ReadFrom path that would hand one back. Every existing spec still passes, the io.EOF entry in the normal-termination table included, which is what showed the branch was dead for legitimate endings and live only for the bug. Add a spec driving a real yamux session end to end. Every mux shape until now was a synthesized error, which is exactly why a race inside the real library stayed invisible. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	c2162f712a	feat(cluster): accept authenticated peer links on /api/cluster/peer ... Upgrades to a WebSocket, wraps it as a yamux server session and hands it to the caller. Rejects before upgrading so an unauthenticated dial sees a 401 rather than a WebSocket error, which is what the route-coverage test asserts. The adapter keeps the reader of a partially consumed message across Read calls. yamux reads through a 4 KiB bufio.Reader, so a small-payload test cannot see a dropped message tail; the framing specs drive the adapter directly with buffers smaller than the message. An empty configured token authorizes nobody here, unlike the worker file transfer server's check: this route is registered in every deployment, so failing open would publish an unauthenticated mux. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	a0deca6385	fix(cluster): report a dead yamux session instead of swallowing it ... Matching yamux errors with errors.Is was too broad. Session.close hands every live stream ErrStreamReset wrapped around whatever killed the connection, so a keepalive timeout, a broken TCP connection or a peer that simply vanished all matched, and a relayed request that died reported a clean ending. Nothing upstream would have retried or logged it. Match the plain sentinels by identity, since only identity separates a stream that was reset from the wrapped form that means the session died. Treat a StreamError as a per-stream reset, and a GoAwayError as normal only when it carries the no-error code, read off ErrRemoteGoAway because the constant is unexported. ErrSessionShutdown needs no entry of its own; it is a GoAwayError with that code. Order matters as much as the matching: session death wraps its cause, which is routinely io.EOF or a closed socket, so the mux checks run before the generic endings. Reversing them alone puts a vanished peer back to nil. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	c05246a366	fix(cluster): treat a yamux teardown in Splice as a normal ending ... go-yamux/v5 matches none of its errors against net.ErrClosed, so the classifier reported an ordinary teardown as a failure: when the session has gone away, the FIN that Splice's own Close writes returns ErrSessionShutdown, and a stream torn down under a live copy surfaces as ErrStreamClosed or a reset. Splice owns that Close, so it owns the errors it produces; the sentinels are named here rather than injected by the caller, which would make a forgotten classifier reintroduce the same bug silently. Cover the error half of the contract, which no in-memory pipe could reach: a scripted stream now feeds Splice a genuine transport failure and each closed-stream ending in turn. Replacing the tail of Splice with "return nil" passed every previous spec. Also assert that Splice does not return until the second direction has finished, rename a spec that promised a leak check it never made, and correct two comments that claimed more than the code did. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	6caceb4b1d	feat(cluster): add the bidirectional splice used by the relay and tunnel ... Returns on the first direction to finish and closes both sides so the other unblocks; a sequential copy deadlocks on any protocol where the far side speaks first. EOF and use-of-closed are normal termination, not errors. The fourth spec covers a peer that stops reading mid-body, the case where a copy is parked in Write rather than in Read. The other three tear down an idle splice and pass even against a Splice that closes only one side. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	42be69d469	fix(cluster): stamp liveness on the database clock and refuse undialable addresses ... DiscoverAdvertisedAddr promised to return an error rather than a fallback no peer can dial, but only rejected an unspecified address. With PostgreSQL on the same host or pod as a replica, which is compose, single-node and any sidecar layout, the route to it is loopback, so every replica advertised 127.0.0.1 and a peer dialling that reached itself. Loopback, link-local and zoned source addresses are now rejected with an error naming the remedy, and a port outside 1-65535 is rejected before it becomes an undialable address. Liveness was also measured on each replica's own clock: Register and Heartbeat stamped last_seen from the Go process, and Live compared those rows against the reading replica's time.Now(). Skew therefore shrank or stretched the window by writerBehind+readerAhead, evicting healthy peers or keeping dead ones. Both sides now use the database clock, which is the one clock every replica demonstrably shares. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	c72baa4ad9	feat(cluster): record frontend replicas in a shared instances table ... Replicas need to find each other to relay worker traffic, and nothing in the tree recorded a replica's address. The advertised address is discovered by opening a UDP socket toward PostgreSQL and reading back the local address, which yields the interface every replica demonstrably shares without asking an operator to configure one. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	1e5f5dbe52	test(distributed): correct the claims the e2e comments make ... Review of the whole branch found five comments that would send a reader to the wrong place, plus three smaller inaccuracies. Nothing here changes behaviour. The KNOWN RACE note on both backend-log WebSocket handlers said the fix needs an atomic snapshot-plus-subscribe "under the store lock". It does not: BackendLogStore.mu guards only the buffers map, and AppendLine enqueues and fans out under the per-buffer buf.mu. Whoever took the store lock would ship and the race would survive, so both notes now name buf.mu and say what s.mu does and does not exclude. Two comments in the cluster harness quoted Eventually(c.FrontendAlive) .Should(BeFalse()). FrontendAlive takes an index, so Gomega rejects that with "requested 1 arguments but received 0". Both now quote the closure form the specs actually use, and say why the closure is needed. proveHealthCheckingIsAlive claimed to prove the health monitor ran for the whole preceding window. It proves the monitor was alive at the end of it, and inferring backwards needs any wedge to be sticky. In the peer-replica-death spec that inverts: health checks are single-flighted by a session-scoped pg_try_advisory_lock, the spec SIGKILLs the replica that may hold it, and until Postgres reaps the session the survivor acquires nothing and checks nothing silently. Consistently(healthy) can then pass because nothing was checking, with the positive control still succeeding once the lock frees. The doc now states what is proven, names that gap, and says the assertion is a floor rather than a proof. The Makefile still called DISTRIBUTED_TEST_FLAKES a retry count, which is what seeded that error into the two docs just corrected against it, and the workflow called the 15s window a reconcile tick when the mechanism is HealthCheckInterval in the node health monitor. Also: the cluster suite measured 509.1s / 509.8s / 512.3s, so about 8m30s and not the 8m39s/8m40s three files claimed; the dead-worker spec title implied two independent detectors when both probes read one advisory-lock-serialised verdict out of the same row; and the sanitizeDBName length assertion used <= 50, which an empty string also satisfies, where the invariant for an over-long input is exactly 50. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:12 +00:00
Ettore Di Giacinto	1dc7998520	test(distributed): scope the log-subscriber wait and mark the race it works around ... Three corrections from review of the previous commit. The lock-order comment on SubscriberCount claimed no path takes s.mu and a buffer lock together. Subscribe does exactly that, holding s.mu.RLock across replica registrations that take buf.mu. State the rule that is actually true — s.mu precedes any buffer lock, so counting after releasing it preserves the order — and say what follows from it: the total is a sample, not a snapshot. waitForLogSubscriber read as general-purpose but unblocks on the first registered subscription. Subscribe attaches the exact-key buffer and each replica buffer one at a time, so for a replicated model the count goes positive while later replicas are still unattached and the race survives. Rename it waitForSingleLogSubscriber, document that it holds only where Subscribe resolves to one buffer, and assert on exactly 1: misuse then fails loudly on the count rather than going quietly back to being flaky. Taking the expected count as a parameter was the alternative, but that makes callers predict a store-internal number and an under-count fails the same silent way as the original bug. The snapshot-then-subscribe race had no artifact outside a report, and review found a second site carrying it. Mark both handlers identically, including the point that swapping the two calls duplicates rather than drops and so is not the fix. The race itself is left alone; this branch stays test infrastructure. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 03:05:11 +00:00
localai-org-maint-botandEttore Di Giacinto	a95da0a46c	fix(gallery): read the models dir once per gallery listing (#12283) ... The cached gallery listing refreshed each entry's installed flag with one os.Stat per entry, under the cache's global write lock. A gallery holds about 1,900 entries. On a models directory on SMB, one refresh took about 14s. The listing and every row's VRAM estimate run this refresh, and the lock serialized them, so the models page took minutes to load. The installed check now lists the models directory once and looks up each entry in that listing. On the same SMB share the listing takes about 75ms. The answers match os.Stat: a symlink counts only when its target exists, and names with a path separator still use os.Stat. The listing runs before the lock is taken, so the lock covers only the flag updates. Concurrent callers on a cold cache now share one upstream load. Before this, each caller fetched the gallery index and the configs itself. Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-26 18:53:52 +02:00
Plamen K. Kosseffandlocalai-org-maint-bot	6cfc99196d	feat(gallery): read metadata for system-path backends, enabling variant aliases (#12141) ... * feat(gallery): read metadata for system-path backends, enabling variant aliases Problem: - ListSystemBackends only read metadata.json for user-managed backends; the system-path scan (LOCALAI_BACKENDS_SYSTEM_PATH) was a bare directory walk with Metadata hardcoded nil - system-packaged backends (distro packages installing several accelerator builds of one backend) could not declare aliases or meta indirection at all, while gallery-installed backends could - surfaced while packaging LocalAI for Gentoo: the packages install cpu-/rocm-/vulkan-audio-cpp as system backends aliased to audio-cpp, which the server ignored Change: - scan each root separately, clean the system collection against the user-managed one, merge, then build and resolve — precedence lives in one explicit step - alias candidates carry their own metadata: the resolved alias entry can never pair one installation's executable with another's metadata, and it reports the chosen candidate's origin (IsSystem) - deterministic resolution: entries build in sorted name order and candidates sort by name at the resolution site, independent of scan order Precedence (user-managed always wins): - a user-managed backend hides a same-named system backend entirely - a user-managed variant takes over its whole alias family: the alias resolves among user-managed variants only and the system family's concrete names disappear — family versions move together, and a stale system variant may not work with newer models, so it must not stay reachable - a system variant's alias never hijacks a name that exists as a user-managed backend Tests: Ginkgo regressions for system-path aliasing, same-name hiding, family takeover, and the full metadata permutation matrix of cross-root name collisions (both directions, with and without metadata on each side). Docs: new "Backend Directory Format" section (run.sh, metadata.json, alias resolution — previously undocumented for user-managed backends too) and "System-Provided Backends" with the precedence rules. Assisted-by: Claude:claude-fable-5 Signed-off-by: Plamen K. Kosseff <p.kosseff@gmail.com> * fix(gallery): preserve managed meta backends A system alias can replace a user-managed meta backend during discovery. Protect meta entries with the same precedence guard as concrete backends. Add a regression test and clarify the documented precedence. Assisted-by: Codex:GPT-6 Signed-off-by: Plamen K. Kosseff <p.kosseff@gmail.com> --------- Signed-off-by: Plamen K. Kosseff <p.kosseff@gmail.com> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-09-26 11:33:51 +00:00
mudler-agentandEttore Di Giacinto	42c58a5838	feat(vllm-cpp): unify decision pipeline through Score RPC with vllm_decide ABI v29 (#12247) ... * feat(vllm-cpp): unify decision pipeline through Score RPC with vllm_decide ABI v29 Replace the model-specific SystemOne gRPC approach with a generic Score RPC extension. The pre-existing Score RPC (previously unused by any backend) now carries question_type and response_json fields: - question_type="systemone" routes kev/laya decision-pipeline requests through the unified vllm_decide C ABI (v29), returning the full response JSON in response_json. - question_type empty routes cua-s1-forms candidate scoring through the same vllm_decide ABI, returning CandidateScore probabilities. The vllm-cpp backend's Score() method calls vllm_decide and dispatches by architecture internally. The /v1/systemone HTTP endpoint checks whether the model's backend supports Score; if so, it forwards the raw request JSON and returns the backend response as-is. Other backends fall through to the existing NER-based path. This mirrors the vllm.cpp C ABI refactor (PR #3301) that replaced vllm_systemone + vllm_score with a single vllm_decide function. The purego bindings bump abiVersion from 27 to 29 and resolve vllm_decide and vllm_decide_free symbols. Also fixes validModelPath to accept cua-s1-forms.json and rl_agent_config.json alongside config.json, matching the engine's model_loader.cpp config-filename ordering. AI-Assisted: true Assisted-by: Maki:regolo/glm5.2 [maki] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore: ⬆️ Update mudler/vllm.cpp to e28ec46c6 (fix macOS -Werror build) Bumps vllm.cpp to e28ec46c6 which fixes a -Wnull-conversion error in qwen3_5.cpp:12483 that broke the macOS Metal CI build under -Werror. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-25 23:27:27 +02:00
leilei3167	a95ef3e7f3	fix(backend): re-probe MediaMarker after cold vision model load (#12254) ... llama.cpp picks a new random media marker per backend process. LocalAI cached the first probe on the model config and skipped later probes when MediaMarker was non-empty, so after SINGLE_ACTIVE eviction/reload the prompt still used the stale marker and mtmd_tokenize failed (0 markers vs 1 bitmap). Re-probe whenever the model was not already resident before Load, while still skipping the RPC on warm cache hits. Fixes #12246 Assisted-by: Cursor:composer-2.5 Signed-off-by: leilei3167 <imleilei123@gmail.com>	2026-09-25 09:16:33 +02:00
mudler-agentandEttore Di Giacinto	543fb4bd24	fix(gallery): verification follow-ups for oci:// galleries (#12243) ... * fix(gallery): verification follow-ups for oci:// galleries Follow-ups from the post-merge review of #12238 and #12239. Only a policy decision is a refusal now. cosignverify wraps ErrPolicyRejected around a failed signature check, an identity or source-repository mismatch, a not_before cutoff and a missing or unparseable bundle. A TUF, registry or network failure during verification, or a timeout, is an outage: the gallery falls back to the copy verified under the current policy, as it does when the registry is down. An oci:// gallery with a verification block, or any oci:// gallery under strict integrity, is no longer answered by an https://, github: or file:// mirror. Such a mirror is ignored with a warning, because nothing can check its signature. The index of an HTTP gallery, whose policy only covers its backend images, is cached under the URL-only name again, so no unchecked body is stored under a policy-keyed name. The in-memory index cache key now includes the policy. After a runtime policy change the index is fetched again, and entries with a relative url install again. The registry digest lookups after install and upgrade, and in the upgrade check, run only for real registry references (new URI.LooksLikeRegistryOCI), not for ollama:// or ocifile://. The refusal message names strict integrity when that is the cause, and the gallery name is no longer repeated. Specs pin the URL-only cache name for galleries without a policy, a fixed key for a fixed policy, and that every GalleryVerification field changes the key. The docs describe refusal, outage, mirrors and strict integrity. Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): reset listings on gallery changes, classify referrer outages Review follow-ups for this PR. The React UI lists from AvailableGalleryModelsCached, which is keyed by nothing. A gallery change through the settings API or a runtime_settings.json edit now drops that listing when the model or backend gallery configuration differs. Before, the UI kept the old list, with local paths into the old policy's tree, until the next background refresh, or for good when the new policy refused the gallery. In cosignverify, a referrer the registry fails to serve now makes the lookup an outage whatever other referrers failed and in any order, since the unread one may be the valid signature. An invalid policy (Validate in NewVerifier, an unparseable not_before) is ErrPolicyRejected, because no fetch can make it usable. The docs say that only an oci:// gallery with a verification block skips non-OCI mirrors, and list an unusable policy as a refusal. Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-24 21:20:21 +02:00
mudler-agentandEttore Di Giacinto	f5c4083d7e	fix(gallery): tie oci:// gallery caches to the verification policy (#12239) ... The unpacked oci:// gallery cache and the last known good copy of the index were keyed on the gallery URL only. After an operator tightened a gallery's verification policy (added source_repository, moved not_before forward), content verified under the older policy, or under none, was still served for up to an hour from the unpacked cache, and indefinitely from the last known good copy while fetches failed. A fetch refused by signature verification also fell back to that last known good copy, so a refusal became a silent downgrade. Turning strict integrity on did not stop an unverified cached copy from being served either. Name both caches by the URL plus a stable hash of the policy. A gallery without a policy keeps its old URL-only name, so existing caches stay usable. A fetch refused by the policy, or by strict integrity, is now reported and never answered with a cached copy; a network failure still falls back, but only to a copy verified under the current policy. The strict integrity check runs before the cache is read. Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-24 15:43:51 +02:00
mudler-agentandEttore Di Giacinto	3278b4a2ed	fix(gallery): strip the oci:// scheme before every registry lookup (#12238) ... A backend installed from an oci://host/repo:tag or oci://host/repo@sha256 URI was downloaded correctly, but the digest lookup that follows the install (and the one after an upgrade) passed the raw URI to the registry client. The client does not know the oci:// scheme: with a port in the host it failed to parse the reference, without one it read "oci" as the registry host and queried https://oci/v2/. The install still succeeded, so the only trace was a warning and an empty digest in metadata.json, which made the next upgrade check report an upgrade for no reason. Add downloader.URI.OCIReference and route every consumer that hands an OCI URI to a registry client through it: the install and upgrade digest lookups, the upgrade check, the OCI download path, the oci:// gallery fetch and the llama.cpp importer. Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-24 15:43:23 +02:00
mudler-agentandEttore Di Giacinto	be0671c635	feat(gallery): optionally pin the signing certificate's source repository (#12235) ... * feat(cosignverify): optionally pin the certificate's source repository Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): source_repository in the verification policy Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(gallery): when source_repository is checked; test the issuer Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-24 09:04:55 +02:00
3 people	21c5495a99	feat(vllm-cpp): add GLiNER2.5 NER via TokenClassify (#12140) ... * feat(vllm-cpp): add GLiNER2.5 NER via TokenClassify Wire the vllm-cpp backend to the C ABI NER surface (vllm_gliner_ner, ABI v27) so LocalAI can serve zero-shot named entity recognition through the existing TokenClassify gRPC method. backend.go: TokenClassify method on *VllmCpp calls vllm_gliner_ner with the text and labels, copies the C-owned entity array into protobuf TokenClassifyEntity messages, and frees the result. govllmcpp.go: cNerEntity and cNerResult Go POD mirrors matching the C structs; vllmGlinerNer and vllmNerResultFree purego bindings; abiVersion bumped 26 -> 27. options.go: ner_labels, ner_threshold, ner_max_width parsed from engine_args. pkg/grpc: ClassifyModel interface and TokenClassify server handler (follows the Embedding locking pattern). core/config: vllm-cpp backend declares MethodTokenClassify and UsecaseTokenClassify. docs/content/features/vllm-cpp.md: NER section documenting the engine_args keys and the host-forward contract. Assisted-by: MAKI:regolo/glm5.2 [maki] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(vllm-cpp): correct NER pointer lint directive Use the govet directive for the C-owned NER array, matching the other purego pointer conversions. The array remains valid until its deferred free; the misspelled directive caused CI to flag this conversion. Assisted-by: Codex:gpt-6 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vllm-cpp): add kev-compatible SystemOne API endpoints Add POST /v1/systemone, /v1/systemone/permute, and /v1/systemone/separate to LocalAI, mirroring the kev project's structured-extraction API. Each endpoint runs zero-shot NER over the rendered state text and builds kev-compatible answers for three question types: noul (binary entity presence), choice (pick one option), and score (pick one level). The TokenClassifyRequest proto gains a `repeated string labels` field so each question can supply its own labels at inference time, and TokenClassifier gains TokenClassifyWithLabels for per-call label selection. The vllm-cpp backend uses request labels when non-empty, falling back to configured ner_labels then the built-in defaults. Helpers (renderState, softmax, choiceConfidence, scoreConfidence, r2) are ported from kev/api.py and mirrored in vllm.cpp's api_server.cpp so both servers produce the same answer shape. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(vllm-cpp): suppress gosec G404 on seeded permutation RNG The SystemOne permute endpoint uses math/rand with a caller-supplied seed for reproducible option permutations, matching kev's random.seed. gosec flags this as G404 (weak RNG). Add #nosec with a comment naming the intent: this is reproducibility, not cryptography. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(vllm-cpp): bump vllm.cpp pin to GLiNER2.5 merge commit Advance VLLM_CPP_VERSION from f3cd97e to 5058268d, the commit that landed GLiNER2.5 zero-shot NER support (PR #3224) in vllm.cpp. This brings the DeBERTa v2 encoder, GLiNER2 boundary head, C ABI NER functions, and server endpoints into the LocalAI vllm-cpp backend. The ABI version (27) and Go struct mirrors already match. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(vllm-cpp): use instruction text as NER label in SystemOne handler The SystemOne handler was passing question IDs as NER labels for noul questions and bare key names for choice questions, so the model never matched any entities. Port the label mapping from vllm.cpp's ParseSystemOneBody: - noul: use the rendered instructions field (with instr alias) as the NER label, not the question ID - choice: use optionText(name, desc) — "name: description" or "name" when the description is null/empty — not the bare key - score: already correct (rendered criteria text) - permute: shuffle indices and build parallel key/label arrays so the NER call uses the optionText labels while the response is keyed by the original option names Also add the instructions field to the SystemOneQuestion schema struct (accepted alongside the instr backward-compat alias). Verified end-to-end against the real GLiNER2.5 model: noul questions now find "Apple Inc. is" (organization, 0.999) and "Tim Cook is" (person, 0.852) where they previously returned zero entities. Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-09-23 12:31:28 +02:00
3 people	636c43baa5	fix(openai): return HTTP errors for pre-stream failures and report per-slot context (#12204) ... * fix(capabilities): report the per-request context with split KV slots With parallel slots and kv_unified:false, llama.cpp gives each slot n_ctx/n_parallel, padded up to a multiple of 256. /models/capabilities still reported the full n_ctx. A client that budgets a request against context_size then overflows at a fraction of it. EffectiveRequestContextSize returns the per-slot size in that case and the full context otherwise. With the unified KV cache, the grpc-server default, one request may use all of n_ctx. The capabilities endpoint and the router's prompt trimmer now use it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(openai): return an HTTP error when a stream fails before any chunk A streamed chat request set the SSE headers, then waited for the backend. When the backend failed before the first token, LocalAI sent a 200 with a `data: {"error":...}` chunk and [DONE]. Clients that do not parse error chunks saw an empty reply. cogito's LocalAI client was one of them: nib users got "streaming decision produced no content" instead of the context overflow that caused it. Nothing has been written at that point, so the handler now returns the error as a normal HTTP response. A failure after the first chunk keeps the in-stream error chunk. A prompt that exceeds the context is now a 400 on both paths, as in the OpenAI API and llama-server, and no longer a 500. The message is kept whole, because clients read the token counts from it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): check the error from closing the response body golangci-lint's errcheck flags the unchecked resp.Body.Close in the new pre-stream error helper. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-22 17:19:58 +02:00
Richard Palethorpe	7034d353b1	feat(kimodo): Add observability hooks (#12184) ... feat(kimodocpp): add API and backend request observability Capture animation requests, phase timings, output metadata, and failures in traces. Record correct API error statuses and cover completed, running, failed, and disabled tracing. Assisted-by: Codex:GPT-6 Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-09-22 07:06:32 +00:00
Richard Palethorpe	66e3a06ce6	fix(auth): require validated header credentials for CSRF exemption (#12185) ... Track successful header authentication before allowing cross-site requests to bypass CSRF checks. Arbitrary headers on unauthenticated servers and cookie-authenticated requests no longer grant an exemption. Share the production CSRF middleware with multipart tests, add regression coverage for credential sources, and document the exemption behavior. Assisted-by: Codex:gpt-6 golangci-lint Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-09-22 04:35:09 +01:00
localai-org-maint-botandEttore Di Giacinto	c0b7e64973	feat(ui): show running models and host gauges on single-node installs (#12189) ... On a single-node install nothing in Operate listed the models loaded on this machine or let an admin stop one. The System page that did was retired in #11548, and its replacements (the Nodes workbench) only work in distributed mode. The Nodes page also mis-detected single-node mode: the cluster routes are not registered there, so /api/nodes answers 404, but only 503 was treated as "distributed off", which sent every single-node install to the empty worker-registration card. The rail hid the entry anyway. Nodes route on a single node becomes "This machine": - the Nodes page's VRAM / RAM / CPU / models-disk gauges, fed from this host by mapping /api/resources onto the worker heartbeat fields - a memory bar splitting host RAM by running model - a running-models table (backend, RSS, CPU share, uptime, PID) with search, sorting, logs and a confirmed Stop - the distributed setup behind an "Add machines" button The Operate overview gains a "Running now" preview (heaviest five, with Stop) on single node and a pointer to Nodes > Running models on a cluster. The rail shows "This machine" in Runtime with a running count. Backend, additive only: - /system: each loaded model carries a `process` block (pid, rss_bytes, memory_percent, cpu_percent, started_at). A sampler keeps one gopsutil handle per PID so CPU is the share since the previous poll rather than the lifetime average; it is omitted on the first reading. - /api/resources: host `cpu` and models-path `disk`, the same readings workers send in their heartbeat. Also fixes the fleet tables widening the page on phones: the headers' absolutely positioned sr-only labels escaped the scroll wrapper. Assisted-by: Claude:claude-opus-5 [Playwright] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-21 19:12:35 +02:00
3 people	36d800ebd1	fix(api): report an alias's target in /v1/models/capabilities (#12183) ... An alias config is a pure redirect with no backend of its own, so the capabilities listing described it from its stub: no capabilities, no modalities, and the default 4096 context_size. Clients that size their context budget from this endpoint (nib, for one) then compacted every turn against a model that really serves 100k. Resolve the alias and report the target's capabilities, modalities and context_size under the alias's id. A dangling or chained alias now reports no enrichment instead of defaults no model runs with. Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-21 10:51:17 +02:00
3 people	237cf456fd	feat(gallery): galleries published as OCI artifacts (#12167) ... * feat(oci): pull ORAS artifacts into a directory Galleries published to an OCI registry arrive as an ORAS artifact whose layers carry their tree path in org.opencontainers.image.title. Nothing in pkg/oci could read a non-image manifest, so add PullArtifact: it lays the layers out under a destination directory and returns the manifest digest so callers can pin and verify signatures on it. A registry is remote input and this writes files, so the manifest is validated before any byte is fetched. A title that is absolute, escapes the destination or is missing is refused, an artifact of an unexpected artifactType is refused, and both the layer count and the total size are capped with caller-settable limits. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * feat(gallery): fetch and verify galleries published as OCI artifacts A gallery whose URL is oci://host/repo:tag is now pulled as an ORAS artifact and its index.yaml read from the unpacked tree, alongside the existing HTTP and file sources. Mirrors, the per-candidate timeout, the failure cooldown and the last known good copy work the same for both schemes, so a gallery can mirror an OCI primary with an HTTP fallback. When the gallery declares a verification policy the artifact reference is resolved to its manifest digest, the publisher signature is checked against that digest, and only then is the same digest pulled. Nothing unverified is written to disk or parsed. With strict integrity mode on, an OCI gallery without a verification policy is refused instead of warned about, which is what the flag already does for backend installs. The pull lands in a staging directory that is renamed into the cache only once the whole tree is on disk and the index reads back as an index, so a failed or interrupted pull leaves nothing a later fetch would serve. A fetch that finds a fresh cache entry does not contact the registry. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * feat(gallery): resolve entry urls relative to the gallery root A gallery published as a self-contained tree, which is what an OCI gallery is, names its base configs by their place in that tree. Such a url reached the HTTP client verbatim and no entry of the gallery could be installed. A scheme-less entry url now resolves against the root the index came from: the unpacked artifact for an oci:// gallery, and the directory of the index URL otherwise. The relative path may not climb out of that root, reusing the guard the artifact puller already applies to layer titles, and an entry that does is dropped from the listing rather than failing the whole gallery. An entry that names its own scheme is untouched, so .ref indirection and local github: resolution behave as before. Reading an entry of an OCI gallery also needs the unpacked artifact directory as its trusted root, since the cache is deliberately a sibling of the models directory the downloader otherwise confines a file read to. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5 [Claude Code] * fix(oci): restrict artifact file permissions Create artifact files with owner-only permissions to satisfy G302. Document the existing path validation for the G304 scanner finding. Check the permissions of downloaded layers in the artifact test. Assisted-by: Codex:gpt-6 gosec * fix(oci): confine artifact writes to their root Use root-relative filesystem operations to reject escapes through existing directory symlinks. Retain owner-only permissions and add a symlink escape regression test, without suppressing the path traversal finding. Assisted-by: Codex:gpt-6 --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-09-21 09:23:05 +02:00