mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-15 15:52:31 -04:00
Ettore Di Giacinto
49b3d22974
Ten NATS subscriptions on the worker become ten HTTP routes under
/v1/control/, served on the loopback HTTP server the worker already runs
and reached only through the tunnel's existing `http` stream tag.
The carrier is the tag that already exists rather than a new one. A new
tag would have had to invent correlation, per-request deadlines,
unbounded payloads and a progress stream, and each of those is a place
this branch has already put a defect. It would also have added a fifth
entry to the worker's stream-refusal vocabulary, which decides what a
frontend reaps on and took eight fixes to settle. Riding `http` means a
control RPC to a worker another replica holds takes the same relay the
inference path takes, which is the path that has been measured.
The request and reply DTOs are untouched, so a body on a control route
is byte-for-byte what the corresponding subject carried. No subject was
deleted: agent workers still subscribe to nodes.<id>.backend.stop.
Install and upgrade stream. They answer application/x-ndjson: zero or
more {"progress":...} lines carrying the same event the per-op NATS
subject carried, then exactly one {"reply":...} line, always last. That
deletes the 8000-byte notification cap structurally instead of
reproducing it on a new carrier: a progress line is written into the
response the caller is already reading, so there is nothing to size and
no subscribe-before-request window. The debouncer is shared with the
NATS publisher rather than forked, so the ~4/s tick bound is one fact.
A verb's own failure is a 200 with Error set, never a 5xx. The frontend
maps a transport failure onto "no route to that worker", which nothing
may act on, and the worker's answer onto evidence a reap guard may act
on; answering 500 for a failed install would put the worker's verdict
in the bucket reserved for a broken link. Only a request that could not
be read or routed is non-2xx.
Control RPCs carry the caller's budget. r.Context() replaces four
context.Background() calls at the gallery-install sites, and the one
pre-existing fixed timeout on model.unload is now derived from the
caller's context so a shorter budget is honoured. No timeout is invented.
The inner `go func()` in the install and upgrade handlers is deleted
rather than nested: it existed because one subscription served every
install, and over HTTP each request already has its own goroutine.
Per-backend serialization stays lockBackend, which is what actually
prevented two requests racing the gallery directory.
Bounds against a boundary the worker now serves: every body is capped at
8 MiB before any decode; the 404 echoes at most 128 bytes of the request
path, cut on a rune boundary so a half rune cannot travel downstream as
a replacement character; non-POST is refused before the body is read so
a probe cannot fire a command; the streaming responses set nosniff.
The routes mount through nodes.AuthenticatedRoutes, which hands the
registrar a private mux and puts the whole prefix behind the same
constant-time bearer check as the file routes. The worker's HTTP server
now takes the supervisor as a required parameter, so there is no way to
start it without the control plane mounted.
Assisted-by: Claude Opus 5 [claude-code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
72 lines
2.8 KiB
Go
72 lines
2.8 KiB
Go
// Package workerctl names the HTTP control plane a worker serves to the
|
|
// frontends that manage it.
|
|
//
|
|
// It is a leaf on the standard library alone, and deliberately so: the worker
|
|
// registers these paths and the frontend calls them, so both sides must agree
|
|
// on the literals without either importing the other's package.
|
|
package workerctl
|
|
|
|
import "encoding/json"
|
|
|
|
// Prefix is the one path prefix the worker mounts its whole control plane
|
|
// under. Everything the frontend may command a worker to do lives below it,
|
|
// which is what lets the worker put the control plane behind a single
|
|
// authentication check instead of one per verb.
|
|
const Prefix = "/v1/control/"
|
|
|
|
// The control verbs. Each replaces one NATS subject; the request and reply
|
|
// bodies are the messaging DTOs those subjects already carried, unchanged, so
|
|
// a worker still reachable over NATS and one reachable over the tunnel answer
|
|
// with the same bytes.
|
|
const (
|
|
PathBackendInstall = "/v1/control/backend/install"
|
|
PathBackendUpgrade = "/v1/control/backend/upgrade"
|
|
PathBackendList = "/v1/control/backend/list"
|
|
PathBackendStop = "/v1/control/backend/stop"
|
|
PathBackendDelete = "/v1/control/backend/delete"
|
|
PathModelStop = "/v1/control/model/stop"
|
|
PathModelUnload = "/v1/control/model/unload"
|
|
PathModelDelete = "/v1/control/model/delete"
|
|
PathModelsRunning = "/v1/control/models/running"
|
|
PathNodeStop = "/v1/control/node/stop"
|
|
)
|
|
|
|
// AllPaths returns every control verb's path.
|
|
//
|
|
// It exists so a spec can assert a property of the whole set rather than of a
|
|
// list it re-types, which would go stale the moment a verb is added.
|
|
func AllPaths() []string {
|
|
return []string{
|
|
PathBackendInstall,
|
|
PathBackendUpgrade,
|
|
PathBackendList,
|
|
PathBackendStop,
|
|
PathBackendDelete,
|
|
PathModelStop,
|
|
PathModelUnload,
|
|
PathModelDelete,
|
|
PathModelsRunning,
|
|
PathNodeStop,
|
|
}
|
|
}
|
|
|
|
// Envelope is one line of a streaming control response.
|
|
//
|
|
// Exactly one of the two is set. Zero or more Progress lines are followed by
|
|
// exactly ONE Reply line, and the Reply line is the last thing on the body.
|
|
// That ordering is the contract: it is what lets the frontend stop reading, and
|
|
// it is what replaces the subscribe-before-request dance the NATS carrier
|
|
// needed, since progress and reply now share one response and nothing can
|
|
// arrive before the caller is listening.
|
|
//
|
|
// Progress carrying the reply's own bytes is also why the 8000-byte
|
|
// notification cap that bounded the NATS progress subject has no analogue here:
|
|
// a line is written into the response body the caller is already reading.
|
|
type Envelope struct {
|
|
Progress json.RawMessage `json:"progress,omitempty"`
|
|
Reply json.RawMessage `json:"reply,omitempty"`
|
|
}
|
|
|
|
// ContentTypeStream is the media type of a streaming control response.
|
|
const ContentTypeStream = "application/x-ndjson"
|