mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-12 22:33:54 -04:00
Ettore Di Giacinto
cf619fc91b
Distributed mode needs an operator to run a NATS cluster. This adds the carrier that replaces its fan-out half, so a deployment eventually needs PostgreSQL and its own HTTP listener and nothing else. pgbus holds one PostgreSQL session per replica, pinned for the life of the process because LISTEN registrations belong to one backend session and a pooled handle would lose them on the next checkout. Publishes go out on the pool with pg_notify. Subjects map onto a channel by their first token, from a closed set of roots. A subject outside the set is refused at publish AND at subscribe rather than mapped to a channel of its own: a channel name is capped at 63 bytes, and one LISTEN per job id would be unbounded. Refused rather than dropped, because a subject that goes nowhere and reports nothing is the class of defect this work exists to remove. PostgreSQL refuses a notify payload of 8000 bytes or more, and several subjects on this bus exceed that in normal operation: a job result carries a whole LLM output, a gallery progress event carries one entry per node. Those are written to a row and the notification carries the id. What is measured against the cap is the ENCODED notification, not the caller's payload, because the subject and the envelope travel too. The filter grammar is not respelled here. Subscribe asks messaging.ValidFilter and delivery asks messaging.SubjectMatches, which makes this the first production caller of a matcher that had only test doubles. New refuses a DSN that names a different database from the pool: that pairing publishes successfully, delivers nothing, on every replica, and reports no error anywhere. Nothing publishes on it and nothing subscribes yet. The construction is wired anyway, because the DSN has exactly one legitimate source and a setting that decides whether any broadcast is delivered should not be invented by whichever call site is migrated first. Delivery is at-most-once, like NATS core. Nothing downstream may read a message it did not receive as evidence about a node: a carrier that cannot deliver is not a worker that is gone. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
63 lines
2.7 KiB
Go
63 lines
2.7 KiB
Go
package messaging
|
|
|
|
import "time"
|
|
|
|
// Publisher publishes JSON-encoded messages to NATS subjects.
|
|
type Publisher interface {
|
|
Publish(subject string, data any) error
|
|
}
|
|
|
|
// Subscription represents a NATS subscription that can be unsubscribed.
|
|
type Subscription interface {
|
|
Unsubscribe() error
|
|
}
|
|
|
|
// MessagingClient is the full interface for NATS messaging operations.
|
|
// Consumers should depend on this interface rather than the concrete Client
|
|
// for testability.
|
|
type MessagingClient interface {
|
|
Publisher
|
|
Subscribe(subject string, handler func([]byte)) (Subscription, error)
|
|
QueueSubscribe(subject, queue string, handler func([]byte)) (Subscription, error)
|
|
QueueSubscribeReply(subject, queue string, handler func(data []byte, reply func([]byte))) (Subscription, error)
|
|
SubscribeReply(subject string, handler func(data []byte, reply func([]byte))) (Subscription, error)
|
|
Request(subject string, data []byte, timeout time.Duration) ([]byte, error)
|
|
IsConnected() bool
|
|
Close()
|
|
}
|
|
|
|
// Broadcaster is the fan-out half of the messaging surface: a publish reaches
|
|
// every subscriber on every replica. It exists so a call site can be moved onto
|
|
// the PostgreSQL carrier without waiting for the request/reply and queue-group
|
|
// halves to be retired, because both *messaging.Client and *pgbus.Bus satisfy
|
|
// it.
|
|
//
|
|
// IsConnected and Close are deliberately NOT here, and their absence is stated
|
|
// rather than left to be inferred, because a reader who knows MessagingClient
|
|
// assumes the smaller interface simply forgot them.
|
|
//
|
|
// IsConnected has no production consumer: its only non-test occurrences are its
|
|
// implementations and the MessagingClient line itself. It is asserted by specs
|
|
// and read by logging, and a carrier's consumers must not branch on it. "The
|
|
// carrier is down" is not one of the four conditions a node's state can be in,
|
|
// and no code may turn it into evidence that a worker is absent.
|
|
//
|
|
// Close DOES have real callers, and they hold a CONCRETE type rather than this
|
|
// interface, which is why the interface can omit it. Before any of them is
|
|
// narrowed from MessagingClient to Broadcaster, re-run
|
|
//
|
|
// grep -rn 'Close()' --include='*.go' . | grep -v _test
|
|
//
|
|
// because that grep is the only thing standing between "the interface does not
|
|
// need it" and a lifecycle that silently stops running.
|
|
type Broadcaster interface {
|
|
Publisher
|
|
Subscribe(subject string, handler func([]byte)) (Subscription, error)
|
|
}
|
|
|
|
// The concrete NATS client is one of the two carriers this interface exists to
|
|
// make interchangeable. Asserted here rather than left to the first adopter, so
|
|
// a change to Client's signatures fails to compile in the package that owns the
|
|
// interface instead of in whichever call site is migrated next.
|
|
var _ Broadcaster = (*Client)(nil)
|