mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-10 07:47:29 -04:00
8db1d165dd2e864dd014b6b63c072114bf106cf9
292
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 mudler-agentandEttore Di Giacinto
|
6343a2dc6f |
feat(voice-detect): report the encoder family so audio-registered voices are fingerprinted (#12499)
* feat(voice-detect): report the encoder family so audio-registered voices are fingerprinted
A voice registered from audio through the voice-detect backend had no
encoder fingerprint, so the parakeet-cpp backend could not tell whether
it was comparable with the loaded speaker model and could only fall back
to the file-name rule.
The voice-detect backend now binds the three new libvoicedetect accessors
with a symbol probe (an older library still loads and reports nothing),
copies the borrowed strings at once and never frees them. It also hashes
the model file once at load. VoiceEmbedResponse gains two optional
fields, encoder_family and encoder_weights ("sha256:<hex>", empty when
the model is not a plain file).
/v1/voice/register stores them as encoder_family and a new
encoder_weights field in the registry entry; model keeps the encoder
name, so the 1:N identify filter by name is unchanged for old entries.
A voice with a family is sent to the backend whatever its file name, and
the backend decides by family. /v1/voice/identify compares the family
when both the stored voice and the probe have one. Old entries load
without the fields and stay unfingerprinted.
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(voice-detect): say how a voice with weights but no family is treated
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(voice-detect): mark the model file open as intended for gosec
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
|
||
|
mudler-agentandEttore Di Giacinto
|
895d50385f |
fix(distributed): converge model configs across frontends (#12558)
* fix(galleryop): announce model changes before the preload After a gallery install or delete, the replica that ran it replaced its config loader, then preloaded every installed model, and only then published the models invalidation. The preload does remote lookups and checksums for each model, so on a large models directory peers learned about the change minutes after the originator listed it. When the preload failed or the operation was cancelled, the event was never sent. Publish the invalidation, and apply the delete lifecycle, as soon as the loader holds the new set. The preload still runs afterwards with its own error handling. Its failure is reported on the operation, but it no longer rolls back a deletion that peers have already applied. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): resync model configs from the models directory Frontends refresh their model configs only when a models invalidation arrives on NATS. NATS keeps no history, so a frontend that is disconnected when the message is published never applies the change. It keeps serving the old config, for example an alias that points at the previous model, until some later change happens to touch it. Each frontend now reruns the peer reconcile against the shared models directory after every NATS reconnect, and every --model-config-resync-interval (default 30s) when a config file changed. The pass names no model, so only models whose file changed get a revision transition, and an unchanged directory costs one read of the config files. The reconcile replaced the whole loader with a parse of the models directory, which dropped models loaded with --config-file and published a deletion revision for them. Configs defined outside the directory are now kept, both there and after a gallery install. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(nodes): stop stale frontends from retargeting alias rules A scheduling rule keyed by an alias derives its target from the alias mapping of the frontend that reads it. Every frontend keeps its own copy of the model configs, so after an alias is repointed a frontend that has not reloaded it still resolves the old target. Two frontends then rewrote the rule's stored target_model against each other on alternate reconciler ticks, and the outdated one scaled up the model the alias used to point at. The registry already records the accepted config revision of each model. A frontend now derives a rule's target from its own alias mapping only when its config revision for the rule's name matches that record. Otherwise it keeps the stored target_model: it neither writes the column nor reconciles replicas of the old target. The check reads the database only for a rule whose stored and derived targets differ. With no accepted revision on record, the old behaviour stays. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agent
|
cd604b5c5b |
fix(distributed): bound, stop and cancel model loads with leases and worker operations (#12524)
Fence model load jobs by generation, lease them on the database clock, bound the work on the worker with operations and a process-group watchdog, and add one stop path with a load-cancel API. See the pull request for the design, the rolling upgrade notes and the test evidence. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-botandEttore Di Giacinto
|
217038d4fe |
fix(agents): share the collection list across frontends (#12536)
With the PostgreSQL vector engine, each frontend kept the collections it had opened in an in-memory map and answered "collection not found" for any other. A collection created through one frontend was unknown to the others until they restarted, and each frontend listed a different set. Wrap the in-process collections backend in distributed mode so that the database is the source of truth: - lists come from the registry, - a lookup miss checks the registry before it returns 404, and opens a collection that exists there (once per name, even under concurrency), - a cached collection that left the registry is dropped and closed, with a re-check at most every 5 seconds, - create and reset publish an event on the existing collection invalidation subject, so other frontends re-check at once. Without the postgres engine, or outside distributed mode, nothing changes. Assisted-by: Claude:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
|
e2faec2bf2 |
fix(distributed): free only the named model on model.unload (#12537)
The worker answered model.unload by calling Free on the first backend process in its map, whatever model the request named. Every idle scale-down or LRU eviction of one model could therefore empty another model's backend on the same node. LocalAI still counted that model as loaded, so its next request failed. A parakeet diarization model then returned 501 "speaker profiles require a loaded speaker encoder" until someone reloaded it by hand. Resolve the target from the model name (all replicas), prefer an address when the request carries one, and free nothing for an unknown model. Also let parakeet-cpp Diarize check the diarization model before the speaker-profile capability. A backend with nothing loaded now answers FailedPrecondition, which LocalAI treats as a stale replica and reloads, instead of a final Unimplemented. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
fmterrors
|
1cd96e496b |
chore: fix some function names in comment (#12517)
Signed-off-by: fmterrors <fmterrors@outlook.com> |
||
|
Ettore Di Giacinto
|
1d117292b1 |
feat(voice): name registered speakers through a bundle speaker component
Naming read only speaker_model:, so a parakeet-cpp bundle that sets speaker_component:voice never received the registered voices, in live transcription or in diarization. Resolve speaker_component: when speaker_model: is not set. A component is not an encoder file, so no file-name tag is derived for it: voices with a hash or family identity and untagged voices reach the backend, and a voice with only a file-name tag matches through the new optional speaker_tag: alias. speaker_model: still wins, including when it points at the bundle file. The bundle gallery entries do not declare speaker_recognition: that usecase means the VoiceEmbed and VoiceVerify RPCs, which the backend does not serve, and it selects the default model for /v1/voice/*. The pinning test now says so. Assisted-by: Claude Code:claude-sonnet-5-5 go golangci-lint |
||
|
Ettore Di Giacinto
|
a2c498359b |
fix(voice): keep one voice store per embedding dimension
The in-memory local-store rejects vectors of another size than the ones it holds. With 192-value voices registered, adding a 256-value voice failed, and identifying with a 256-value probe returned an error. The registry now keeps one store per embedding dimension. The first dimension seen uses the configured store name, so single-encoder instances are unchanged. Later dimensions use "<name>-<dim>". Identify searches only the store of the probe size and returns no match when no voice of that size exists. Forget finds the store from the stored embedding. A name may hold one voice per encoder. Assisted-by: Claude Code:claude-sonnet-5-5 golangci-lint |
||
|
mudler-agentandEttore Di Giacinto
|
79a7631cc5 |
feat(parakeet-cpp): encoder fingerprint for speaker naming, VAD trim and word filter options, pin bump (#12491)
* chore(parakeet-cpp): bump parakeet.cpp to 2de154c Brings in the speaker registry encoder fingerprint, the VAD segment trim and the opt-in word filter, a fix for a per-call thread count that stayed set on the process-wide backend after a Silero VAD pass, and bundle components loaded from memory. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(parakeet-cpp): encoder fingerprint for speaker naming, vad_trim and guard_* options Speaker naming. A registered voice now carries the encoder that made it: the embedding family (voicedetect:<arch>:<name>:<dim>) and the sha256 of the weights. The backend reports the family of the loaded speaker model in Status, voice enrollment from speaker_profiles stores it as encoder_family (old entries load without it), and the registry sent to parakeet.cpp is built with parakeet_capi_speaker_registry_add_embedding_fp. The library then refuses a registry of another encoder family and the error names both families; another quantization of the same family only warns. A voice with only a weights hash gets the loaded family when the hashes are equal. Voices without a fingerprint (registered from audio: libvoicedetect cannot report one) keep the file-name rule and are used with a warning. The library cannot mix them with fingerprinted voices in one registry, so a request that has any uses the old registry for all. speaker_strict:true drops them instead. A library without the symbols behaves as before. Transcription. vad_trim (seconds, 0 keeps the whole cuts) goes through the VAD options JSON, so it reaches /v1/vad and the segmenter. The guard_* options guard_min_local_conf, guard_local_radius and guard_drop_punct_only turn on the word filter through parakeet_capi_transcribe_path_json_with, or through the segmenter with vad:true. They are off by default, bad values fail the load, and a library without the symbol fails it with a clear message. The dropped word count is logged at debug level. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Stefan Walcz
|
5f5feea15a |
fix(agentpool): keep the conversation in the agent web chat (#12410)
* feat(agents): keep the web chat history between messages The agent chat endpoint ran every message as a fresh job (ag.Ask(WithText(message))), so a follow-up such as "add two days to item 3 and recalculate" never saw the answer it referred to. Agents then rebuilt their reply from scratch instead of changing it. Use the agent's own conversation tracker, as the Telegram and Slack connectors already do: send the earlier turns with the new message and record successful answers. Failed, cancelled or empty runs are not recorded, and the tracker drops a conversation after the agent's last_message_duration of inactivity. The distributed (NATS) chat path is unchanged. Assisted-by: Claude:claude-opus-5-5 ginkgo Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(agents): keep each web chat conversation's history separate Review on this PR: the first version kept the history in the agent's conversation tracker under one fixed key per agent. The web UI keeps several conversations per agent (New Chat, switching, Clear) and did not send which one a message belongs to, so the hidden history mixed them: a new chat received the previous chat's turns, and Clear only cleared the screen. The client now sends the earlier turns of the conversation it is showing as `history` with POST /api/agents/:name/chat, and the server keeps no web chat history of its own. Each conversation only ever sees its own turns; New Chat and Clear start without history. The server uses only user and assistant turns with text, bounded to the most recent 40 turns and 64,000 characters. `history` is optional, so existing callers keep the previous behaviour (no history); the distributed (NATS) path does not forward it yet. Tests: two conversations of one agent stay apart, an empty history (New Chat or Clear) starts fresh, system/tool/empty turns are dropped, the bounds keep the most recent turns; the UI helper that builds the history from the visible messages has node --test coverage. Docs: features/agents.md describes `history` and the web UI behaviour. Assisted-by: Claude:claude-opus-5-5 ginkgo Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> --------- Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> |
||
|
mudler-agentandEttore Di Giacinto
|
d3724eef50 |
fix(nodes): stage dedicated diarization audio (#12474)
Dedicated diarization forwards frontend audio paths to remote workers, which cannot read those temporary files. Stage the input before the RPC and release it afterward, following the transcription lifecycle. Clone the request so staging does not change caller-owned data. Cover input bytes, request fields, cleanup, and error propagation in tests. Document distributed diarization staging on the existing feature page. Assisted-by: nib:gpt-6-astra Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
99043b442c |
feat(router): route with native decision models (#12449)
* fix(schema): preserve SystemOne image inputs Assisted-by: OpenAI * test(schema): follow Ginkgo conventions for decision inputs Assisted-by: OpenAI * feat(llama-cpp): dispatch native decisions through Score Upgrade the stock dependency and reconcile Score/TTS patches. Reuse native decision parsing, tasks, formatting and response-reader cleanup; preserve ordinary scoring admission and guard older dependencies. Assisted-by: OpenAI * refactor(systemone): share request and model validation Assisted-by: OpenAI:gpt-5 * fix(systemone): preserve HTTP wire-byte validation limit Keep structural validation separate from the serialized internal request bound so HTML escaping cannot reject valid HTTP payloads. Assisted-by: OpenAI:gpt-5 * feat(systemone): bound images and account native decisions Preserve public wire limits independently from router serialization. Reject unsupported NER images, map native request/capability errors, and stamp explicit usage once. Advertise decisions for stock llama-cpp. Assisted-by: OpenAI * fix(systemone): record usage on registered native route Exercise real registration and billing with a mock native backend. Reject empty native responses, malformed image URLs, trailing JSON, and wire overflow including whitespace. Assisted-by: OpenAI * feat(router): add lazy native decision transport Bind named models through internal ModelSystemOne calls with shared validation and bounded abandoned operations. Remove request and echoed-error contents from decision traces. Assisted-by: OpenAI:gpt-5 * feat(router): classify overlapping policies with native decisions Ask independent noul questions, validate probabilities and preserve first-superset routing. Wire the central factory with config-sensitive invalidation and cancellation-safe resolution. Document native framing and bounded operation limits. Assisted-by: OpenAI:gpt-5 * feat(gallery): add pinned Julia-1 native decision model Add a separate text-only llama-cpp Q8 entry with pinned Apache-2.0 source provenance and checksum. Installed using the gallery installer and exercised choice, score and noul on CPU. Assisted-by: OpenAI * test(router): verify native decisions through central factory Add an opt-in real-model Ginkgo integration covering the native Go loader and C++ transport, token usage, independent overlapping labels, and candidate selection. Document owned-server execution and the intentionally non-quality threshold. Assisted-by: Codex:gpt-5 * fix(llama-cpp): align upstream pin and preserve decision signatures Advance to bed0a856 without losing the automated upstream bump. Detect full-request fill_task support at compile time and forward every question for Nimble framing while retaining the earlier native signature. Preserve reconciled SCORE/TTS patches; add standalone compatibility coverage. Assisted-by: Codex:gpt-5 * feat(gallery): add native decision family defaults Pin Laya, Kev-4B, lev, OpenJev and Nimble artifacts. Verify Laya/Kev/lev gallery installs and CPU contracts on both native pins; clearly mark OpenJev/Nimble runtime validation pending and their noncommercial licenses. Assisted-by: OpenAI * docs(decisions): clarify integrated Nimble prerequisite Record the exact combined backend pin while retaining pending OpenJev and Nimble installation/runtime validation status. Assisted-by: Codex:gpt-5 * fix(gallery): indent native decision model sequences Match repository yamllint indentation for Laya, Kev, lev and OpenJev list fields. Parsed gallery data is unchanged; reproduce CI gallery lint failure before the whitespace-only fix and pass the same command afterward. Assisted-by: Codex:gpt-5 * docs(decisions): record OpenJev and Nimble CPU validation Record gallery installation, checksum/metadata verification and multiquestion native smoke results on bed0a856. Retain noncommercial and text-only limitations without accuracy or deterministic-output claims. Assisted-by: OpenAI * fix(ui): expose native Decisions router classifiers Select classifier models using metadata-driven capability routing, retain tuned thresholds, and validate native decision selections before saving. Cover both native backends and create/save/reopen in the real React editor. Assisted-by: Codex:gpt-5 * fix(router): exclude aliases from native decision discovery Check the originally named config before advertising native Decisions eligibility. Retain target capability inheritance for ordinary generation aliases. Exercise the actual capabilities endpoint with native models on both backends, aliases, and disabled models. Assisted-by: Codex:gpt-5 * feat(systemone): share bounded multimodal input validation Preserve text wire limits while admitting bounded PNG/JPEG decision input. Share collection and header validation across internal and public callers and keep the native runner response budget independent. Assisted-by: OpenAI:API-assistant * fix(systemone): bound admission lifetimes and validate complete images Retain shared admission leases through actual work completion, including abandoned internal operations. Decode bounded image pixels, cap public native responses before usage stamping, and preserve oversized malformed text status precedence. Assisted-by: OpenAI:API-assistant * fix(router): classify images before media fetching Preserve ordered structured probes for native decisions. Defer OpenAI media preparation until routing selects the served model, so rejected decision URLs cannot trigger downloads before shared validation. Guard direct image collection with context-aware shared admission. Keep text classifiers and embedding caches from discarding image input. Retain fail-closed classifier configuration and runtime fallback policy. Add middleware, typed-content, admission, cancellation and cache tests. Assisted-by: OpenAI:API-assistant * fix(router): bound extraction before serialization Check probe budgets before copying text or marshaling message state. Count JSON escaping so oversized internal inputs fail before allocation. Preserve typed Anthropic blocks through selected-model conversion and fallback. Keep retry coverage in Ginkgo without global test registration. Assisted-by: OpenAI * fix(router): bound supported probe serialization Arbitrary structs can bypass the probe budget through pointer marshalers, string tags, and promoted fields. Accept concrete chat schema types and plain JSON values instead of emulating arbitrary struct serialization. Budget escaped direct prompts before marshaling so raw length cannot hide serialized expansion. Preserve runtime fallback and reject oversized input before invoking the decision runner. Add Ginkgo allocation, boundary, and marshaler invocation regressions. Six-package tests, three-package race tests, and full-T2 delta lint pass. Assisted-by: OpenAI:GPT-5 golangci-lint * feat(decisions): enable bounded OpenJev images Validate native decision images before permissive media parsing and pixel allocation. Require both decision image support and a vision projector; missing or audio-only projectors cannot silently become text decisions. Pin the OpenJev Q8 projector and document its license and disk footprint. Add native safety tests, canonical limit parity, gallery and load-option checks, and a reproducible CPU direct-RPC contrasting-image smoke. Assisted-by: OpenAI:GPT-5 * fix(decisions): reject incomplete image streams stb accepts corrupt PNG Adler checksums and truncated JPEG scans. Use bounded zlib validation and strict libjpeg decoding before parsing. Keep dimension and aggregate pixel checks ahead of decoder allocations. Wire decoder dependencies into native builds and runtime packaging. Add regressions for appended EOI and embedded marker bypasses. Assisted-by: OpenAI:GPT-5 * fix(ci): gate native decision image validation Run the decoder security tests outside the stdlib-only native suite. Fetch vendor headers at the backend pin and provision decoder dependencies. Gate Go limit parity and production CMake wiring without model downloads. Assisted-by: OpenAI:GPT-5 * test(decisions): cover multimodal public API paths Exercise shared image contracts through the registered HTTP routes and external mock backend. Add opt-in cached gallery installation and real OpenJev image decisions through SystemOne and both routing APIs. Assisted-by: Codex:gpt-5 * test(decisions): assert isolation and cache bypass Observe external RPC calls and compare complete classifier history. Winner-only and cache-miss checks could hide dropped history or cache use. Give real inference its own application and model directory so shared backend mappings and loaded processes cannot affect mixed suite order. Assisted-by: OpenAI:ChatGPT * test(decisions): isolate fixture globals Disable optional global services in the isolated HTTP fixture and register cleanup before setup assertions. Verify meter provider identity survives fixture creation and destruction. Snapshot observed usage before assertions so failures cannot retain the mutex. Require a successful usage stamp before checking error responses. Assisted-by: Codex:gpt-5 golangci-lint * fix(application): honor optional telemetry controls Skip failover gauge registration when metrics are disabled. Register against the application meter rather than looking up the global provider. Allow embedders to retain the bounded routing log without billing stats. Keep the existing default when stats are disabled. The isolated HTTP fixture uses this option without losing its native router assertions. Assisted-by: Codex:gpt-5 golangci-lint --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
e4fa051ee3 |
refactor(distributed): put the NATS-only paths behind interfaces (#12395)
* feat(messaging): add shared subject rules Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(messaging): cover BroadcastRoots, ControlRoots and SubjectRoot Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(messaging): add Broadcaster and enforce subject rules in every carrier Broadcaster is the fan-out half of MessagingClient. The NATS client and the in-memory FakeBus now refuse a subject outside the served roots and any wildcard other than a whole single token, and FakeBus shares MatchSubject instead of its own copy. FakeBus Unsubscribe now removes its own subscription instead of the first one with the same subject. A shared conformance suite in messagingtest runs against both carriers. The distributed e2e specs that used invented test.* subjects, and the one that subscribed with a > filter, now use subjects from subjects.go. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: depend on Broadcaster where only publish and subscribe are used Narrowed to messaging.Broadcaster: nodes/staging_progress.go, nodes/install_progress_publisher.go, galleryop/operation.go, galleryop/service.go, agentpool/user_services.go, agentpool/agent_jobs.go, openresponses/store.go, openresponses/sync.go, syncstate/syncstate.go, finetune/service.go, quantization/service.go and failover/distsync/distsync.go. SubscribeJSON now takes a Broadcaster because it only calls Subscribe, which lets the narrowed consumers use it. Stayed wide: worker/supervisor.go, because its client field also serves the SubscribeReply handlers in worker/lifecycle.go. The request/reply, queue and wiring files (nodes/unloader.go, nodes/file_stager_s3.go, jobs/dispatcher.go, agents/dispatcher.go, agents/events.go, worker/file_staging.go, cli/agent_worker.go, http/app.go) are unchanged by design. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): name the no-route condition and confine the carrier error Consumers matched nats.ErrNoResponders, which names an absence, to demote a node. They now match ErrNoRoute, the control path maps the carrier's failure onto it, and timeouts and worker refusals are pinned as not being no-route. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(nodes): state which FileStager implementations return ErrNoRoute Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): build backend clients through one node-aware seam Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: describe the distributed transport seams Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: correct comments that overclaim after the seams refactor Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(agent-worker): refuse an unserved LOCALAI_AGENT_SUBJECT at startup The messaging client now refuses a subject whose root no carrier serves. An agent worker started with a custom LOCALAI_AGENT_SUBJECT such as tenant-a.agent.execute used to start and then wait on a subject the frontend never publishes to. After the subject rules landed it exited at subscribe time with an error that did not name the setting. Behaviour change: the worker now checks LOCALAI_AGENT_SUBJECT before it registers or connects, and exits with an error that names the variable and says to use a served subject under the agent root, for example agent.execute. The served roots are not widened: a custom root was never delivered by the frontend, and a wider set would reopen the drift the subject rules exist to close. The flag help and the agent worker docs state the constraint. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(nodes): pin the reactions to ErrNoRoute Three callers react to ErrNoRoute and had no spec: the reconciler's upgrade drain falls back to the legacy forced install, the reconciler marks the node unhealthy when a pending op has no route, and the backend-op fan-out marks the node unhealthy. Each spec drives the real caller with a scripted no-responders reply and reads the result from the registry or the recorded requests. A fourth spec pins the other side: a pending op that times out leaves the node healthy and only counts the attempt, so mapping timeouts onto ErrNoRoute would fail here. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(messaging): pin client subject checks and fail the carrier suite in CI Add specs that call Publish, Request, Subscribe, QueueSubscribe, SubscribeReply and QueueSubscribeReply on a client with no connection. Each call must return ErrUnservedSubject for bogus.thing and ErrUnsupportedWildcard for jobs.>. This proves that the subject check runs before the connection is used, and needs no server. The NATS conformance suite is the only check that runs the subject rules against a real carrier. Before this change it skipped without output when Docker was missing. Now it fails when CI is set, so a Linux runner without Docker cannot hide it. It still skips on local runs and on macOS CI, which has no Docker. Add SubjectNodeBackendInstallProgress to the list of constructors that must build served subjects, and ask contributors to extend the list. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: state what ErrNoRoute may change, and group the distributed guides The seams note said MarkUnhealthy was the only state change allowed on ErrNoRoute. A pending backend op still records the failed attempt, counts toward the reconciler's retry limit and is dead-lettered after the maximum attempts. The note now says that MarkUnhealthy is the only change to the node's own state, and that the per-op accounting is not a verdict about the node. The note also documents that the NATS conformance run fails under CI when Docker is missing. The distributed-seams row moves next to the distributed-state row in the topics table. The liveness ping spec header now says no route is a reason to skip the worker, not proof that the worker is gone. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): give the backend client factory the node id Mechanical: the method gains a nodeID parameter and the eight test fakes are updated. No behaviour change. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): drop the optional node-aware factory The node id is now in the main method, so the optional interface and its helper had no behaviour of their own. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): dial backend probes through the client factory Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): dial workers' file servers through a per-node dialer Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(http): proxy backend logs through the per-node worker dialer The admin backend-logs proxy (list, lines and the WebSocket stream) now reaches a worker through the same per-node dialer as the HTTP file stager, so every frontend-to-worker dial goes through one seam. The shared direct dialer keeps alive for 15s where the proxy used 30s. Harmless for requests bounded at 15s. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(http): keep the backend-logs proxy independent of the admin connection The proxy request had no context before the dialer change and is bounded only by its 15s timeout. Keep it that way. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: move the worker control payloads to workerctl Mechanical move of the request and reply structs, the install progress event and the file payloads out of messaging. The verbs no longer belong to one carrier. No alias is left behind. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(worker): serve the lifecycle verbs through a controlServer The worker registers one handler per verb and a NATS server maps each verb to its subject. Registration errors now name the verb. node.stop is served with SubscribeReply, which is identical on the wire because the handler never replies. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(worker): report install progress through the control sink Install and upgrade now emit download progress through the sink the control server hands them. The debounce and the terminal flush stay in the handler path, built over that sink by the new nodes.NewDebouncedInstallProgressSink, which replaces NewDebouncedInstallProgressPublisher. The subject and payload on the wire are unchanged. The supervisor no longer holds the bus, and installFn and upgradeFn let specs drive both verbs without a gallery. The malformed-request log lines are restored for install, upgrade, backend.delete, model.unload, model.stop and model.delete, with the reply bytes unchanged. The signal adapter is renamed noReply, which also lets worker.go import os/signal without an alias again. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(worker): serve the file-staging verbs through a controlServer An empty list-dir answer is now {} rather than {"files":null}, because the typed reply omits an empty Files slice. The frontend decodes both to a nil slice in nodes/file_stager_s3.go. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(messaging): add WorkQueue and the NATS producer This is the producer side of the competing-consumer seam. The work kinds map one to one to today's subjects and queue groups: task to jobs.new and mcp-ci to jobs.mcp-ci.new (both in group workers), agent-run to agent.execute (group agent-workers). Enqueue publishes the payload as Publish does today, with one JSON marshal. FakeBus now records queue groups and keeps reply handlers so later specs can pin and drive them. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(messaging): add the NATS WorkConsumer An in-flight limit of one runs the handler inline on the delivery goroutine, as the MCP CI consumer does today. Any other limit spawns per delivery, as the agent consumer does. Queue groups are unchanged. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: publish queued work through WorkQueue The job dispatcher, the agent pool and the agent scheduler enqueue through messaging.WorkQueue; the NATS implementation publishes to the same subjects as before. DistributedServices builds the queue next to the NATS client and hands it to the dispatcher and the agent pool, whose distributed mode switch now reads a non-nil WorkQueue. The unused AgentPoolService.SetNATSClient is removed. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: consume queued work through WorkConsumer The agent dispatcher and the MCP CI consumer register through messaging.WorkConsumer. The NATS implementation keeps the inline one-at-a-time model for MCP CI and the per-delivery model for agent runs. handleMCPCIJob reports on the events publisher the carrier hands it instead of a captured client. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: delete the consumers nothing in production reached jobs.new has a producer and no production consumer, and the agent dispatcher's Dispatch was only called from tests. Publishing jobs.new is unchanged. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(mcp): send MCP requests to agent workers through AgentControl Timeouts still honour only the deadline, not cancellation, exactly as today. The NATS no-responders error maps to ErrNoRoute and a timeout does not. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(agent-worker): serve MCP requests and backend.stop through agentRPCServer The agent worker's MCP tool and discovery reply subscriptions and its backend stop listener move behind an unexported agentRPCServer interface, served on NATS by nodes.NATSAgentRPCServer. The handlers become typed mcp.ToolHandler and mcp.DiscoveryHandler values that answer every failure with a reply carrying Error. Queue group (agent-workers), inline execution on the delivery goroutine, the background handler context, the unmarshal error reply texts and the reply-less backend stop subscription are unchanged. The backend stop handler takes the decoded backend name, so it can still close that backend's MCP sessions. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(messaging): remove helpers that only tests used BroadcastRoots, ControlRoots and SubjectRoot had no production caller. The roots spec now asserts every served root through ValidateSubject instead. MatchSubject moves back into the test support package, the only place that used it, with its table. NATSAgentRPCServer drops the subscription list it stored and never read, and NewNATSAgentRPCServer gets a doc comment. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(mcp): round trip the agent RPC server over a real NATS server One spec sends a tool request and a discovery request through NATSAgentControl to NATSAgentRPCServer and checks that the handlers see the decoded requests and the replies come back. It also puts an undecodable body on the tool subject and checks the server answers with an unmarshal error instead of leaving the requester to time out. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: describe the distributed transport seams The developer note now lists the final seams: fan-out, queues, both halves of the control verbs and of agent RPC, and the dial. It records the open items a second carrier has to handle. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test: pin the in-flight limit each queue consumer asks for The work queue specs pin what Consume does for a given limit, but nothing pinned which limit each production consumer passes. Changing the agent worker's MCP CI limit from 1 to 0 would have let MCP CI jobs run concurrently on each worker with every test green. Move the MCP CI Consume call into startMCPCIConsumer with the same wiring and pin that it asks for (WorkMCPCI, 1). Pin that NATSDispatcher.Start asks for (WorkAgentRun, maxConcurrent) for several limits. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: remove helpers the branch left without a caller SubjectJobCancelWildcard lost its last subscriber when the frontend stopped listening on jobs.*.cancel; the NATS permissions and conformance suite spell the subject out, so nothing reads the constant. decodeBackendStopRequest returned a stopAll flag that production dropped and only a test read. decodeBackendStop is now the single decoder with the same semantics: an empty body is stop-all, an empty Backend is stop-all, malformed JSON is an error. stopBackends still derives stop-all from Backend, so no reply changes. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(messaging): keep an explicitly empty agent queue a plain subscription Before the work queue seam the agent worker passed LOCALAI_AGENT_QUEUE straight to QueueSubscribe, so an explicitly empty value made a plain subscription and every agent worker ran every agent run. WithAgentRunRoute replaced an empty queue with agent-workers, which silently changed that. Keep the queue as given once the option is applied. An empty subject still falls back to agent.execute, since it never had a meaning of its own. The flag default stays agent-workers, so only an explicitly empty value reaches this. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: correct comments and record the PR B notes Fix the recordingFactory comment (it also records the parallel flag), document that a negative maxInFlight is unbounded and that Unsubscribe from a handler deadlocks, and say a permanently undecodable payload returns nil. Record controlHandler's undecodable return as a kept exception, and add the second carrier notes to the developer note: the reconciler has no ClientFactory option, the logs proxy honours HTTP_PROXY, verbs one carrier serves need an opt-out, terminal replies come from the result event, and agent runs publish through the NATS-bound EventBridge, which is not an additive change. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
9eb5a9e61d |
feat(audio): remember speakers from diarization (#12414)
* feat(schema): validate portable speaker profiles Add the versioned profile schema for explicit speaker enrollment. Validate compatibility against separately supplied loaded-encoder metadata. Reject unusable speakers, invalid vectors, and inconsistent clean spans. This slice does not change HTTP routes, backend integration, or the UI. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(parakeet): export profiles with transcripts Export opt-in speaker profiles and trusted encoder metadata. Replay registrations by ID so duplicate display names keep independent vectors. Use one profile-capable diarization for slots, names, and clean spans. Assign timestamped ASR words to those slots without a second diarization. Preserve legacy opt-out and no-ASR behavior, and propagate failures. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(audio): enroll portable speaker profiles Gate profile exports with voice-recognition permission and validate registration against metadata from the loaded encoder. Preserve audio enrollment and independent registrations with duplicate display names. Exclude diarization and registration exchanges before API trace capture so persisted traces cannot retain profile vectors or JSON audio. Defer candidate dimensions to trusted loaded metadata. Sort candidates by registration ID so incompatible profiles cannot suppress legacy voices through registry iteration order. Keep portable identity checks closed when trusted metadata is unavailable. Test persisted traces, explicit slot zero, and selection through offline and live transport. Document privacy and the ephemeral registry lifecycle. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): remember speakers from diarization Add a Studio page for diarization and opt-in speaker profiles. Preview clean intervals from the original recording before explicit registration. Join profiles by raw speaker labels, preserve duplicate names, and relabel turns only after a successful save. Discard stale results when the model or recording changes. Share registration metadata with voice management without storing vectors or recordings from this flow. Document permissions and the global, ephemeral registry. Cover enrollment, permissions, previews, and asynchronous races with mocked Playwright tests. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: clarify HTTP speaker enrollment support Replace the stale enrollment limitation with the current HTTP workflow. Distinguish native transport from explicit registration and link its docs. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(parakeet): pin merged speaker profile support Use the merged commit from mudler/parakeet.cpp#80. Its tree matches the previously accepted native pin. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add diarization enrollment setup example Connect the existing gallery modes to the speaker enrollment workflow. Show installation, private profile export, explicit raw-slot registration, and later recognition without another export. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(blog): explain diarization speaker profiles Put the diarization walkthrough on the LocalAI website in the feature PR. Cover the three gallery modes, explicit enrollment, and privacy limits. Link setup instructions and keep availability conditional on feature support. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(blog): focus diarization on everyday use Explain what users can do with recordings before the setup steps. Replace the technical walkthrough with a short Studio guide and link readers to the existing reference for model names and developer use. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(blog): lead with speaker capabilities Present speaker recognition through everyday uses and a short UI flow. Keep technical reference details in the existing documentation. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(diarization): satisfy Go lint checks Avoid copying protobuf message state when extending backend status, check the multipart reader close result, and document the focused testing.T lint exemptions. Assisted-by: nib:gpt-5.6-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-botandEttore Di Giacinto
|
2ae6cae70d |
feat(parakeet-cpp): name speakers from the shared voice registry (#12382)
* feat(voice): list registered voices and record which encoder made them The voice registry could register, identify and forget but not list, and it did not remember which speaker encoder produced an embedding. Add Metadata.Model and Registry.List, answered from the index the store registry already keeps for Forget. Needed so a backend can be given the registered voices that match its own speaker encoder. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(voice): store the encoder model with a registered voice Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(voice): pick the registered voices that match a speaker model Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(proto): carry known voices and speaker names on diarize and live messages Assisted-by: Claude:claude-haiku-4-5 [Claude Code] * feat(diarization): name speakers from the voice registry When a diarization model has a speaker_model option, the endpoint sends the registered voices made by that encoder to the backend. The backend's name and name_score come back as extra fields next to the normalized SPEAKER_NN speaker, and the speakers summary carries the first name seen for each speaker. RTTM output and results without names are unchanged. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(live): pass registered voices to a live session and surface speaker names Live sessions now send the registered voices that match the model's speaker_model to the backend, and each speaker segment carries the name the backend matched. The realtime segment event gains an optional speaker_name field. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(parakeet-cpp): load a speaker model and build per-request voice registries Adds the speaker bindings (ABI v9 and v10, probed separately), the speaker_model, speaker_threshold and speaker_margin options, and a per-request registry builder over the known voices. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(parakeet-cpp): name the speakers in Diarize from the known voices Diarize builds a per-request speaker registry from the known voices when a speaker model is loaded, calls the named C functions, and puts each slot's registered name and score on the segments. The registry is freed on every path. A library without ABI 10 reports Unimplemented instead of dropping the names. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(parakeet-cpp): name speakers in the live scene stream The live scene stream now begins with a known-voice registry when a speaker model is loaded and the live config carries voices, and each closed speaker segment takes its slot's current name from the feed's names map. A segment that closes before its slot is identified has an empty name. The registry is freed after the stream, on every path. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(gallery): speaker naming entries and docs for parakeet-cpp Add three gallery entries that load the WeSpeaker ResNet34 speaker model next to the diarization or realtime scene models, and document speaker names in the voice recognition, diarization, audio to text and realtime pages. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * fix(parakeet-cpp): skip an unusable registered voice instead of failing the request A registered voice with the wrong embedding size, or one the C side refused, failed the whole diarization request, so one legacy voice broke the model for every user. Skip such voices with a warning that does not carry the voice name, and take the plain path when none is left. Also map an exact 0 speaker threshold or margin to a tiny positive value, since the C side reads 0 as "use the default", and fix a stale comment about which contexts Free() walks. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * fix(diarization): warn once per model about voices from another encoder; document the privacy limit The different-encoder warning fired on every request. Log it once per feature and speaker model, then at debug level. Document that the global voice registry lets any caller of a speaker_model model learn matching names, and that skipped wrong-sized voices are logged. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * chore(parakeet-cpp): bump parakeet.cpp to 8c8cec0 (C-API v10) and check speaker naming against the real library The pin moves from 623a968 to 8c8cec0, which brings in everything merged in parakeet.cpp since: the voice identification change (C-API v9, #78) and raw-embedding enroll plus diarize-only speaker naming (C-API v10, #79). New real-library specs (gated on PARAKEET_BACKEND_TEST_SPEAKER_MODEL, _DIAR_MODEL, _WAV and, for the live path, _STREAM_MODEL) name the two speakers of two_speakers.wav from a committed pair of WeSpeaker embeddings, with the voices passed in reversed order. They also check that the float32 threshold reaches C through purego. The shared test loader now registers the v9/v10 and scene symbols as main.go does. The rebase onto origin/master had no conflicts. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agent
|
1d7023be1a |
test(agentpool): pin the standalone agent contract (#12378)
Adds a Ginkgo contract suite for the standalone (LocalAGI-backed) agent service in core/services/agentpool, with a fake OpenAI-compatible LLM and a harness that boots a real AgentPoolService. It pins CRUD, per-user isolation, export and import, pause and resume, the chat and SSE event contract, status and observables, persistence and the raw-key agent lookup. No production code changes. Specs that pin a known gap are named known gap or known defect, so later phases flip them on purpose. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5-5 [Claude Code] |
||
|
Ettore Di Giacinto
|
9c156656bd |
Merge PR #12285: feat(failover): serve a model name from a chain of local and remote targets
Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
590512d9eb |
feat(distributed): report worker version and show models in node inspector (#12328)
* ci: bump Hugo from 0.146.3 to 0.166.0 The hugo-theme-relearn submodule was bumped to 9.1.x in #12096, which requires Hugo >= 0.165.0. The pinned 0.146.3 broke the docs site build with a template error in alias.html that could not evaluate the Locale field on langs.Language. Bump HUGO_VERSION to 0.166.0 (latest stable) to satisfy the theme minimum and resolve the alias.html template error. Assisted-by: nib:claude-sonnet-4.5 [bash] [read] [edit] * feat(distributed): report worker version and show models in node inspector Workers now send their LocalAI build version and git commit at registration. The controller stores them on BackendNode and exposes them through the existing node list/detail API responses. The node inspector side pane now fetches and renders the list of loaded models (name, state, in-flight) instead of showing only a count, matching what the node detail page already displays. --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
50c284fdcc |
fix: make the remaining VerifyPath checks effective (#12326)
utils.VerifyPath joins its argument onto the base path, so a path that the caller already joined always passes. Several callers gave it joined paths, and their checks could not fail: - modeladmin (config view, patch, edit, pin and state): the config file path from the loader. A config loaded from outside the models directory (--models-config-file) could be pinned, and the pin wrote the outside file. The patch and state paths stopped later, in the mutation snapshot, with a different error. - core/backend/tts.go: the model path joined onto the models path. - The trellis2cpp and stablediffusion-ggml backends: option paths (*_path) joined onto the model path. A "../" value outside the model directory was accepted. Add utils.VerifyResolvedPath for a full path. modeladmin and tts use it. The backends now check the relative option value before they join it. A rename in modeladmin checks the new relative name. For models from a config file outside the models directory, the admin API and web UI now return ErrPathNotTrusted for view, edit, pin, and enable or disable. The docs describe this. Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
97ad8f1d70 |
fix(distributed): stage the files a model install declares (#12309)
* fix(distributed): stage every shard of a split GGUF A split GGUF is configured by its first shard only. llama.cpp opens the other "-0000N-of-0000M.gguf" files from the same directory by name. The router staged only the configured path, so the worker received shard 1 and the load failed with "failed to load GGUF split". The router now stages the remaining shards next to the first one. A missing shard fails the load and names the file. The file count for progress and the payload size also include all shards. The payload size feeds the load deadline and the disk-headroom check. For a 111 GB model whose first shard is 10 MB, both were sized for less than 1 GB. Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): stage the files a model install declares Replace the split GGUF file name matching with the model's own file list. A gallery install or an import records every file of the model in ._gallery_<name>.yaml (files:), and a config can list more under download_files:. The router now stages all of these files, not only the files that the config's path fields name. This includes the other shards of a split GGUF, which llama.cpp opens by name. The application gives the router a resolver that reads the two lists. The resolver looks up the files by model name when it stages them, so a replica that the reconciler loads from saved load options gets the same files. backend.proto does not change. A declared file that is missing on the frontend is skipped with a warning. The load deadline and the disk headroom check include the declared files. Assisted-by: Claude:claude-opus-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
0565fc06af |
Merge PR #12302: chore(deps): bump LocalAGI to 7e0947d (no-RAG-DB crash fix, tool filters, per-collection models)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Stefan Walcz
|
950c271710 |
[router] fix: re-seed the knn corpus index when the vector store comes back empty (#12267)
fix(router): re-seed the knn corpus index when the vector store comes back empty The corpus manager records a store as synced by file fingerprint and embedding fingerprint. The local-store backend behind it is an in-memory gRPC process the model loader may evict (active-backend cap, memory pressure) or the idle watchdog may kill, and relaunch on the next request — empty. The file is unchanged, so EnsureLoaded returned early and the router went blind: every probe fell back with similarity 0 while corpus/stats kept reporting the full count. Measured on a production router (LOCALAI_MAX_ACTIVE_BACKENDS=6, four resident models + two router stores): loading any further backend evicted a store, and the idle watchdog killed both after 15 minutes; /stores/find returned 0 hits against a 100-line corpus file whose stored vectors matched fresh embeddings with cosine 1.000. Two parts, because the knn classifier is built once and cached (GetOrBuildClassifier), so the sync at build time is otherwise the only one for the process lifetime: - corpus.Manager remembers one vector it inserted (probe) and, on the synced path, asks the live index for it. A miss means the index was relaunched — fall through and re-seed from the file (no re-embedding). - The router middleware wraps the knn classifier's store so every lookup runs EnsureLoaded first; the loader gets the raw store, so its probe never re-enters the wrapper. A sync error fails the lookup closed, like the build-time load. Specs: corpus package (relaunched empty store is re-seeded under an unchanged file), middleware (relaunched index behind the cached classifier is re-seeded instead of falling back; the spec is red without the wrapper). The test fake now answers Search for inserted vectors. Folds in the maintainer's follow-up (router-corpus-reseed-after-store-relaunch): reviewed and accepted. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> |
||
|
|
52a6d62bbc |
fix: return correct HTTP status codes for saturation and no-nodes-available (#12113)
* feat: return 429 when backends are saturated When backends are at capacity (per-model max_concurrent or the process-wide --max-concurrent-backend-requests ceiling), the response was 503. The OpenAI SDK, litellm, and most agent harnesses key on 429 for rate-limit backoff and treat 503 as a hard error. Both saturation paths now return 429 with the existing Retry-After header and type: "rate_limit_error" in the JSON body. The per-model admission middleware keeps admission_rejected as the code field so existing alerts that match on it still fire. Non-saturation 503s are unchanged: model cold-loading (with progress body), model-load failure cooldown, PII detector fail-closed, and classifier unavailable. These mean "not ready" rather than "busy". Assisted-by: AGENT:regolo/glm5.2 [TOOL] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: return 503 when scheduler has no available nodes When the scheduler cannot find any healthy node to serve a model — all nodes are full and eviction cannot free a slot, or a node_selector excludes every candidate — the error fell through to 500. A 500 tells clients something is broken when the condition is transient and retryable. The router now wraps these errors with a new ErrNoAvailableNodes sentinel. The HTTP error handler maps it to 503 via applyNoAvailableNodes, following the same pattern as applyBackendAdmission (429). Unrelated scheduler errors (DB timeouts, registry lookups) still return 500. Three return sites are wrapped: - resolveSelectorCandidates: selector matches zero healthy nodes - scheduleNewModel eviction-busy: all models have in-flight requests - scheduleNewModel eviction-failed: eviction itself errored The existing scheduleAndLoad wrapper ("no available nodes: %w") preserves the sentinel through the chain via errors.Is, as does ModelRouterAdapter. Assisted-by: AGENT:regolo/glm5.2 [TOOL] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(http): use Ginkgo for admission tests Replace forbidden testing.T calls with Ginkgo and Gomega so the lint check accepts the admission handler tests. Assisted-by: Codex:GPT-6 forbidigo * fix(middleware): show the recorded status for admission rejections The admission audit row now records 429, but the Middleware page still printed a hard-coded 503. Read the status from the event, and update the two package comments that still said 503. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5-5 [Claude Code] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> |
||
|
Tai AnandEttore Di Giacinto
|
5b10d36b5f |
fix(quantization): pin the producing backend on imported quantized models (#11875) (#11879)
* fix(quantization): pin the producing backend on imported quantized models (#11875) ImportModel hands the copied GGUF to importers.ImportLocalPath, which detects the file format and defaults every GGUF to `backend: llama-cpp`. For a model this service just produced with a backend stock llama.cpp cannot read, the generated config names an engine that cannot load the file, and the import silently registers an unloadable model. Correcting `backend:` by hand makes the same file work. The job record already carries the backend that served StartQuantization, so carry it into the config instead of keeping the detected default. The gallery publishes a quantizer as a release channel of the engine that runs its output ("llama-cpp-quantization" is llama.cpp's quantizer, whose GGUF is served by "llama-cpp"), so the channel suffix is stripped to get the serving backend. A backend that both quantizes and serves ("rocmfp4") carries no suffix and passes through unchanged, as do pinned hardware variants ("rocm-rocmfp4"), which are valid values for a config's backend field. An empty job backend leaves the detected default in place. Also replace the importer's generic "Fine-tuned model (GGUF)" description for this path: the model was quantized, not fine-tuned, and the job knows the type. Signed-off-by: Tai An <antai12232931@outlook.com> * style: restore trailing newline in service.go for gofmt Signed-off-by: Anai Guo <antai12232931@outlook.com> * style(quantization): restore trailing newline in service.go gofmt requires the file to end with a newline; the previous style commit did not actually add it. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5-5 [Claude Code] --------- Signed-off-by: Tai An <antai12232931@outlook.com> Signed-off-by: Anai Guo <antai12232931@outlook.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
84e2fc5eac |
feat(agents): support tool lists and required tool in distributed mode
LocalAGI 7e0947d added allowed_tools/excluded_tools and the required_tool_before_finish gate. Single-node agents get them through LocalAGI's runtime, but the distributed executor drives cogito directly and its static config meta did not list the fields, so the agent form hid them and the worker ignored them. The distributed config now parses the tool lists from a JSON array or a comma/newline separated string, and the meta entries match LocalAGI's. The executor filters the knowledge base, skill and MCP tools (MCP via cogito.WithMCPToolFilter) before the model sees them, and re-prompts the model when it answers before the required tool returned "ok": true, up to the configured number of reminders. LocalAGI keeps its filter and gate helpers unexported, so a minimal copy lives in core/services/agents/toolpolicy.go. A spec compares the meta entries with LocalAGI's to catch drift. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5-5 [Claude Code] |
||
|
Ettore Di Giacinto
|
00bd5ee121 |
fix(failover): skip a target that has been edited into a chain
A chain is checked for nested chains when it is saved, but not when one of its targets is later edited into a chain. Requests then served the inner chain's config as the target, which has no backend and triggers backend auto-detection. Mark such a target missing so no plan picks it, and skip it without a trip in the HTTP path when it is pinned. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5-5 [Claude Code] |
||
|
Ettore Di Giacinto
|
8474296cb1 |
fix(failover): keep the aliased model of a warm target loaded
The warm list held target names as the chain lists them. For an alias target that is the alias, but the preloader loads the alias stub (no backend, no model) and the eviction guard compares against loaded model names, which never include an alias. A warm alias target was neither preloaded nor protected from eviction. Report the model that serves each warm target instead. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5-5 [Claude Code] |
||
|
Ettore Di Giacinto
|
1b03600437 |
fix(failover): drop the echo of an own target publish
A manager snapshots a target publish when it queues it and applies every echo the sync layer sends back. With one recovery probe, a success moves a target from down to recovering to healthy under one lock and queues two publishes. The echo of "recovering" then arrived after the target was healthy, rolled it back, switched the chain away with reason trip and restarted the dwell timer. Tag each target snapshot with the publishing manager and ignore own echoes. The manager already holds that state or a newer one. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5-5 [Claude Code] |
||
|
Ettore Di Giacinto
|
7804e4591c |
fix(syncstate): do not keep a value in memory when its Store write fails
SyncedMap.Set and Delete changed memory before they wrote the Store. When the write failed they returned the error, but the unpersisted value stayed in memory. Callers that re-read the map then applied it again. The failover manager rolled back a failed pin, but its periodic pin re-sync read the pin back from the map and re-applied it on that frontend until the database came back. Write the Store first and change memory only after it succeeds. A failed Set or Delete now leaves memory and peers as they were, so the map always matches what the Store holds. This also stops a failed create of a fine-tune, quantization or agent task from leaving an orphan entry that the API had reported as failed. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5-5 [Claude Code] |
||
|
Ettore Di Giacinto
|
55f7d5bfdf |
fix(e2e): wire proxy api_key_env lookup into the e2e in-process app
|
||
|
Ettore Di Giacinto
|
4a7c96f533 |
fix(advisorylock): bound HeldLock.TryAcquire on a hung database
The failover prober gate calls TryAcquire with the application context, which never ends. A database that stopped answering blocked the scheduler goroutine for good, with the lock's mutex held. Bound the session open and the lock query with the same timeout as Verify. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
5ecaf8f792 |
fix(failover): count a chain switch once per cluster
Every frontend recorded localai_failover_switches_total for the switches it adopted from the leader, so a cluster of N frontends counted each switch N times. Only the leader, which decides the switch, records it. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
4f801dfc99 |
fix(failover): roll back a pin that could not be shared
Pin and Unpin apply the change locally first for read-your-writes. When the shared write then failed, the local pin stayed, so this frontend served a target the others did not. Restore the previous pin on error, unless a newer change arrived meanwhile. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
ed9951cf7c |
fix(failover): re-sync pins on every frontend after a missed delta
A NATS reconnect re-hydrates the pins map from the DB without OnApply, so a frontend that missed an unpin kept serving the old pin and flip-flopped with the leader's republish. Every frontend now reconciles the manager's pins with the shared set every ten ticks and after a reconnect, and the pins map re-reads the DB every 30 s to repair a delta dropped without a reconnect. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
e61bd2d78e |
fix(failover): do not trip targets on a gRPC message size limit
ResourceExhausted is retried and trips the target, which is right for a rate limit or an out-of-memory backend. A payload over the gRPC message cap is ResourceExhausted too, but every target rejects it the same way, so it tripped the whole chain. Classify it as a request error. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
5ed67be0bf |
fix(failover): skip a target that answers HTTP 501 instead of failing the chain
The non-OpenAI endpoints (depth, detection, face_*, voice_*, images, video, 3d) map a backend's gRPC Unimplemented to an echo 501 without the gRPC status. The retry loop did not see a capability gap, and IsRetryable is false for 501, so the client got 501 and the next target was never tried. Treat a returned or written 501 as a capability gap. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
452a3a0cbe |
fix(docs): correct warm-toggle and Reconcile-without-Store claims
The failover chain editor leaves the warm toggle enabled on every row (the model list has no backend field to gate on) and relies on the server warning instead, so the docs describing it as disabled for remote targets were wrong. Separately, syncstate's hydrate() returns early with no Store or Loader, so a Reconcile tick is a no-op rather than one that empties the map — correct that claim everywhere it was repeated (contributor guide, distsync comment, design spec). No behavior change. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
882f51fd62 |
fix(failover): trip a rate-limited or exhausted target instead of skipping it
Treating ResourceExhausted as a capability gap skipped the target without counting a failure, so a target that stays rate limited or out of memory kept its traffic. It is now an ordinary retryable failure: the request moves to the next target and the exhausted one trips. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
6df1767133 |
fix(localai-proxy): keep rerank, stream errors and chat intact through the proxy
Rerank no longer sends top_n 0, which the upstream rejects. A mid-stream upstream error frame now fails the call instead of ending it as a short success. Temperature 0 is forwarded. An upstream 429 becomes ResourceExhausted, which failover skips like Unimplemented. A localai-proxy config sends its own name upstream when upstream_model is unset, and a chat proxy defaults to the tokenizer template so chat reaches the upstream as messages. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
509cca35e0 |
feat(grpc): serve Rerank from Go backends and skip Unimplemented targets
Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
e59fb854ee |
fix(failover): free the leader lock soon after the leader's host dies
A leader whose host died without closing its connection kept the advisory lock for about two hours of OS keepalive defaults, and no other frontend could probe. The lock session now sets short TCP keepalives and tcp_user_timeout, so the server drops it within about 30 seconds. Shutdown now closes the lock for good, so a tick that runs after it cannot take the lock back. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
b5792e4d17 |
fix(failover): keep the probe leader until its session ends
A lock taken per tick passed between frontends on almost every tick, so several frontends probed at once and each change of leader re-sent the warm set and all state. The leader now holds a dedicated PostgreSQL session with the advisory lock and keeps it until it shuts down or the session dies. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
d30c33a074 |
feat(failover): run one prober per cluster and pin warm targets on workers
Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
7b88674fc5 |
feat(failover): sync pins, target health and chain state over NATS
Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
5f98dfcf95 |
fix(failover): keep shared pins for chains this frontend has not loaded yet
A pin arrives from the sync layer once. Dropping it when the chain or target is unknown here left this frontend routing differently from the cluster whenever its config lagged or a chain was re-created. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
65c22c53bc |
feat(failover): share state through a sync hook and gate probes on a leader
Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-bot
|
e62854c340 |
fix(failover): pass credential lookup from CLI
Pass the API key environment lookup through ApplicationConfig to satisfy core configuration lint. Keep credential resolution dynamic and exclude the callback from serialization. Handle the five close results reported by errcheck. Assisted-by: Codex:gpt-6 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
f7f12f8203 |
fix(failover): warn about warm on a remote target, align the spec
The spec promised a load-time warning when a chain marks a remote target warm, where the flag does nothing; the loader now logs it. The remote-backend test moves into ModelConfig.IsRemoteProxy so the loader and the failover manager agree on what is remote. The spec now says what ships: a load blocked by pinned warm targets proceeds over the limit after eviction retries, without an error that names them. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
27e5aba60e |
fix(failover): do not follow redirects in remote probes
Go resends custom headers such as x-api-key when it follows a redirect, also to another host, so a redirecting upstream could receive the target's API key elsewhere. The probe client now treats a redirect as the response, which fails the probe as a non-2xx status. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
2ecbaab010 |
fix(failover): send remote targets their own upstream model
A chain request reached a cloud-proxy target with the client's model, the chain name, whenever the target set no upstream_model: passthrough forwards the body's model and translate falls back to it. The upstream answered 404, which neither retries nor trips, while the liveness probe, which checks the target's own name, kept passing. PrepareTarget now sets the upstream model of a remote target to proxy.upstream_model or the target name, the same name the probe uses. The request pipeline and realtime chain stages both call it. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |