mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-14 07:07:33 -04:00
Ettore Di Giacinto
43f7a5d108
The responses.metadata SyncedMap had no durable Store, so its reconnect re-hydrate replaced nothing. That was survivable while responses converged through deltas on a broker that mostly stayed up. It is not survivable on a carrier whose listener is one pinned PostgreSQL session: every response created while the subscription was down stays invisible on that replica forever, and the symptom is a 404 from one replica and a 200 from another for the same response_id. State that must survive a gap now lives in a response_metadata table, and the notification only says it changed. The map writes through on a Set and reads the table on hydrate, on reconnect and on reconcile, so the gap closes instead of becoming permanent. The row carries the whole projection as JSON rather than one column per field. A column-per-field schema would be a second definition of what a peer may act on, and the two would drift the first time syncedResponse gained a field: the map would broadcast the new field and hydrate without it, so a replica that had reconnected would serve a different response body from one that had not, with nothing failing anywhere. Only PayloadJSON is ever decoded; owner_replica and owner are indexed copies for an operator reading the table by hand. A missing row and an unreachable database are different facts. Every store and adapter method returns a driver failure as an error and never as an empty result, and syncstate replaces nothing when its source errors, so an outage leaves the map holding what it had rather than blanking it into a cluster-wide 404. Liveness is the database's clock, spelled expires_at IS NULL OR expires_at > now(), because every replica hydrating from this table must agree on which rows are live and a Go-side cutoff makes that a property of whichever process asked. The test container shares the host clock, so no behavioural spec can tell the two apart; the statement shape is pinned instead. The constructor refuses a non-PostgreSQL handle, because an unguarded now() on the single-binary path reads as a missing migration. A ticker sweeps expired rows every five minutes on each replica, and Close waits for it rather than racing it. Note that the sweep removes nothing while LOCALAI_OPEN_RESPONSES_STORE_TTL is 0, which is the default: with no TTL nothing ever expires and the table grows for the life of the deployment. The docs say so plainly. EnableDistributed takes the store positionally and last, so a call site that forgets it fails to compile rather than silently restoring the deltas-only map this change exists to replace. A nil store there is refused by name: it is reached only from the distributed branch of route registration, so it is a wiring bug and not a deployment shape. What still never leaves the owning replica is unchanged: the resume buffer and the CancelFunc. The write-through is one row per response state change, not one per generated token. Assisted-by: Claude Opus 5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
60 lines
1.4 KiB
Go
60 lines
1.4 KiB
Go
package distributed
|
|
|
|
import (
|
|
"fmt"
|
|
|
|
"github.com/mudler/xlog"
|
|
"gorm.io/gorm"
|
|
)
|
|
|
|
// Stores holds all Phase 4 distributed stores.
|
|
type Stores struct {
|
|
Gallery *GalleryStore
|
|
FineTune *FineTuneStore
|
|
Quant *QuantStore
|
|
Skills *SkillStore
|
|
|
|
// Responses is the durable backing for the responses.metadata SyncedMap. It
|
|
// is what a replica re-hydrates from after its listener misses a delta;
|
|
// without it a response created during the gap is invisible on this replica
|
|
// forever and the same response_id answers 404 here and 200 on a peer.
|
|
Responses *ResponseMetadataStore
|
|
}
|
|
|
|
// InitStores creates and migrates all Phase 4 distributed stores.
|
|
func InitStores(db *gorm.DB) (*Stores, error) {
|
|
gallery, err := NewGalleryStore(db)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("gallery store: %w", err)
|
|
}
|
|
|
|
ft, err := NewFineTuneStore(db)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("fine-tune store: %w", err)
|
|
}
|
|
|
|
quant, err := NewQuantStore(db)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("quantization store: %w", err)
|
|
}
|
|
|
|
skills, err := NewSkillStore(db)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("skills store: %w", err)
|
|
}
|
|
|
|
responses, err := NewResponseMetadataStore(db)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("response metadata store: %w", err)
|
|
}
|
|
|
|
xlog.Info("Distributed stores initialized (Gallery, FineTune, Quant, Skills, Responses)")
|
|
return &Stores{
|
|
Gallery: gallery,
|
|
FineTune: ft,
|
|
Quant: quant,
|
|
Skills: skills,
|
|
Responses: responses,
|
|
}, nil
|
|
}
|