mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 09:08:54 -04:00
49b2b151b773122a72d44da965b31d136cd394fc
328
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ettore Di Giacinto
|
1d8d3e3421 |
feat(backends): install a backend from a library entry
A library can now list backends. An entry of kind backend names the image of the backend gallery by digest, and POST /backends/library/install installs it on the local node. The gallery stays the source of truth and its install path is the one used: the library resolves the entry, takes the signature policy from the library, pins the image by digest and calls the same install as the gallery. Installing a name that came from a gallery, another library or by hand is refused and changes nothing. The signature policy of a backend library names the signer with an anchored identityRegex, so images signed by the merge workflow on master and on tags both verify. A pattern that is not anchored is refused. An install is bounded by a size limit per kind: add-ons keep 512 MiB and a backend image defaults to 8 GiB, set with --addons-backend-max-bytes. The limit reads the manifest after the signature check and before any layer is fetched. The metadata of an installed backend records the library it came from. scripts/backends/cmd/backendcatalog renders entries from backend/index.yaml and pins them against the registry. The pilot is the CPU build of local-store, in gallery/backends/index.yaml. The library is registered on the first start only with --addons-backend-library, which is off by default. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
484813f856 |
feat(addons): add a kind to library entries and read unknown kinds tolerantly
A library entry now has an optional kind: addon (the default when it is absent), backend or model. Every index written before kinds existed stays valid and its entries are add-ons. A reader keeps the entries of the kinds it knows and skips the others without reading their fields, so a newer kind never makes an older host refuse the whole index. The refresh result and the library state report how many entries were skipped and of which kinds, and the refresh logs a warning. A kind that is not a plain kind name still makes the index malformed. Search lists add-ons unless the caller asks for a kind, and Resolve returns add-ons only, so an add-on install cannot be pointed at a backend image. A library policy may name the signer with an anchored identityRegex instead of an exact identity, so builds from master and from tags both verify. An add-on install refuses a policy that carries one. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
c24de4fc21 |
test(addons): move the remaining add-on tests to Ginkgo
The built-in catalog, tool arguments, loopback carrier, host operation contract, packaging, application, config, CLI and endpoint tests used the testing package calls that forbidigo rejects. They now run as Ginkgo specs. The endpoint tests also drop the unchecked Close errors that errcheck reports when the auth tag is on. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
50a0b5c8cd |
test(addons): move the pin store tests to Ginkgo
The pin service, file store, sweep and shared store contract tests used the testing package calls that forbidigo rejects. The store contract is now a function that registers its specs for each store, so the file store, the memory-only store and the database store run the same cases. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
82e83bf253 |
test(addons): move the host operation tests to Ginkgo
The hostops tests used the testing package calls that forbidigo rejects. They now run as Ginkgo specs under one suite bootstrap. Every check and every per-operation case is kept. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
579ed93daf |
test(addons): discard the close error of the held listener
errcheck flags the deferred Close of the listener that keeps the port busy. The test has no use for that error, so the defer drops it explicitly. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
78fa0b93a8 |
feat(addons): list the tables add-on in the first-party index
Add the Tables entry to gallery/addons/index.yaml next to the six existing ones. It comes from addonpkg catalog over a locally built linux/amd64 package, so the digest pins that image. The official index test now expects seven add-ons. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
4edf30cda0 |
test(addons): expect Studio and Talk in the official index
The official index now lists six add-ons. Update the test that decodes it so it accepts Studio and Talk. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
3569da24c5 |
feat(addons): register an official add-on library by default
Add gallery/addons/index.yaml, a YAML index of the four first-party add-ons (fine-tuning, quantization, voice, face) in the library format. The entries come from addonpkg catalog over locally built linux/amd64 packages, so the digests pin those images. On the first start, when the owner has no sources.json, register the raw GitHub URL of that index on master as the "official" library and write the file. The file is then the record: a library the admin removes stays removed, and an existing file is never changed. --addons-official-library=false skips the default. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
2009506233 |
fix(addons): read the loopback port file through os.Root
The persisted port file was read with a path built from the add-on root, which gosec reports as a file inclusion through a variable. Open it through os.Root so the name cannot leave the root, even by a symbolic link, and require a regular file with no group or other access. A looser file now fails closed instead of being trusted. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
02a707558e |
feat(addons): ship Studio and Talk as add-ons instead of built-in pages
Studio and Talk were compiled into the web bundle and always listed in the Add-ons drawer. They are add-ons now, like Voice, Face, fine-tuning and quantization: a person installs and removes them, the drawer lists them only while they are installed, and they are packaged the same way. Move the pages, their components, hooks and helpers into addons/studio and addons/talk. Each has a manifest, a UI bundle, a release platform matrix and a catalog template, and is built by the compat mode of the shared builder. The core API stays where it is: the pages call it with the signed-in person's session. The history keeps its browser storage keys. The unit tests of the moved helpers move with them. In the core, drop the two pages, the Create group of the sidebar (Chat joins Home and Models in the old layout), the built-in catalog entries (the catalog is empty) and the talk locale namespace. The old addresses (/app/studio, /app/talk and the generator links) redirect to the add-on pages, with the model as ?model=. The Home /studio command is offered only while the Studio add-on is installed and opens it. The release workflow and its tests now list seven add-ons. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
222d8e64b3 |
fix(addons): keep the loopback owner address stable across restarts
The single-node loopback transport took a new ephemeral port at each start. The owner stores its address in its identity file and refuses to start when the address differs, so every start after the first failed with "invalid addon owner configuration". Choose the port once, keep it in the add-on state root (loopback-port, mode 0600) and reuse it. If the port is taken at a later start, stop with an error that names the port and --addons-enabled=false, because another port would break the identity check. The keys and certificate stay in memory. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
49995e4aff |
feat(addons): run the add-on owner in the normal binary
The owner was only enabled when an embedding host set the add-on config, so a normal run answered every add-on request as unavailable. Add --addons-enabled (default true) and --addons-ownership-mode (default named-user, or local-owner; anything else stops startup). The node is its own owner: state lives under the data path, and the carrier uses loopback keys made at each start and kept in memory. --addons-enabled=false turns the owner off completely. Named-user mode needs the account database. With authentication off the owner stays off and a warning says how to use local-owner. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
6e34303348 |
fix(addons): kill an add-on only after three missed health probes
One slow 100 ms probe killed a busy add-on and dropped the work it was serving. Require three misses in a row. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
bab102e630 |
fix(addons): release the manager lock while a release is fetched
Submit held the state lock across the catalog lookup and the package download, so the apps list and status reads stalled for the whole install. Serialize Submit and Reconcile with their own lock and drop the state lock around the fetch only. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
faa74b7a92 |
fix(addons): refuse a reinstall that would migrate retained data unbacked
An upgrade takes a verified backup before the new package migrates the data. A reinstall over retained data has no running generation to stop, so the new package would migrate it without one. Refuse a reinstall whose data format differs from the retained one. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
ed6e677640 |
fix(gosec): tighten file modes and annotate false positives
Write job state, configs and imported models with mode 0600, clean the paths that are read from disk, and mark the route tables, the header name and the retry jitter as reviewed false positives. Assisted-by: Claude Code:claude-sonnet-5-5 gosec Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
1604a67c1f |
feat(addons): add the add-on runtime, navigation and first-party add-ons
Add the add-on runtime to LocalAI: manifests, the host operation broker, admin gating by account role, the add-on UI bundle builder and the SDK for add-on pages. Move the navigation to Chat, Models, Add-ons and Settings, with pinned add-ons in the sidebar and an Add-ons drawer. Ship the first-party add-ons (voice, face, fine-tuning, quantization) as packages for linux/amd64, linux/arm64 and darwin/arm64. Intel macOS (darwin/amd64) is enabled in the platform files. Squashed from the review branch. The tree is unchanged; the pre-squash history is kept on a local backup branch. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Stefan Walczandlocalai-org-maint-bot
|
d768f1f973 |
feat(pii): apply PII middleware to the Open Responses API (#12708)
* feat(pii): apply PII middleware to the Open Responses API
The per-model PII filter ran on /v1/chat/completions, /v1/completions,
/v1/messages and the Ollama routes, but not on POST /v1/responses. A
model whose policy blocks or masks a secret on the chat route served the
same content unfiltered when it was sent through the Responses API.
Add a piiadapter.OpenResponses adapter and wire pii.RequestMiddleware
into the POST /v1/responses and /responses chain, after the model config
is resolved and merged and before the handler converts input items into
prompt messages. The adapter covers instructions, input as a string, and
in an input item array the text of message and reasoning items (string
content or input_text/output_text parts, including typeless
{role, content} items), function_call arguments and function_call_output
output; structured tool output is scanned as the JSON the handler
forwards. Image, file, audio and video parts are left untouched.
The Responses WebSocket mode (GET /v1/responses) is not HTTP middleware
and cannot filter, so a response.create for a model with PII filtering
enabled is now refused with pii_unsupported_transport instead of being
served unfiltered. The docs list the new route and this limit.
Tests: adapter specs for every covered field, each with a clean-text
counter-check; a route spec driving the real middleware chain with an
in-process pattern detector (block on string input, content part,
instructions and tool output; mask; clean input passes unchanged); and
WebSocket specs proving the refusal and that models without a PII
policy are still served.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
* fix(pii): scan complete Responses tool output arrays
The Responses handler forwards non-string tool output as JSON. Scan
that full encoding so string arrays and extra fields cannot bypass PII
filtering. Cover array scans, redaction, clean values and route blocking.
Document that stored history is resolved after request filtering.
Assisted-by: Codex:gpt-6
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
* feat(pii): report unscanned media on the Open Responses route
Extend block_unscanned_media to /v1/responses: input_image, input_file
and input_audio parts inside message items, and the same types as bare
input items, are reported as unscanned content. Tool output stays
allowed, as it is scanned as JSON text.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
---------
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
|
||
|
mudler-agentandEttore Di Giacinto
|
dce91ba7aa |
fix(backends): prefer installed development builds (#12782)
* fix(ci): capture and surface ginkgo errors in run-coverage.sh The coverage script ran ginkgo per test root with `|| fail=1`, which swallowed the exit code. Combined with `make --output-sync=target`, which buffers all output and flushes at the end, the actual ginkgo error was invisible in interleaved CI logs. Every test suite passed but the script still exited 1, and the error message was buried in 22 000 lines of buffered output. Each ginkgo invocation now captures combined stdout+stderr to a per-root log file. On failure the last 80 lines are printed to stderr so the error is visible in the CI log. On success the log is replayed to stdout so test output remains visible. The build flags (--tags, --coverpkg) are quoted correctly so values containing spaces (e.g. --tags="debug auth") are preserved through eval. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ci): create output dir before writing ginkgo logs The previous commit added per-root log files but forgot to create the output directory before the first ginkgo invocation. On CI the coverage/ dir did not exist yet, so the shell redirect to cover-pkg.out.log failed with "No such file or directory" before ginkgo even ran. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(core/http): add canvas preview route to auth coverage allowlist Commit |
||
|
|
36307005a9 |
feat(chat-history): server-side chat history with tombstones (follow-up to #9902) (#12663)
* feat(schema): add Conversation schema for chat history (#9432) Persist WebUI chat conversations server-side so browser refresh, private windows, or device changes preserve user history (issue #9432). This commit adds the on-disk representation: - Conversation holds id, name, model, opaque history, MCP / sampling settings, and timestamps. - ConversationsFile is the per-user list serialised to JSON. History is stored as json.RawMessage so the server stays agnostic to React message shapes (user / assistant / thinking / tool_call / tool_result mixed with text / image_url / audio_url / file attachments). Assisted-by: Claude:claude-opus-4-7 Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * feat(chathistory): add file-based per-user conversation store (#9432) File-backed persister for chat history. Each user's conversations live under {baseDir}/{userID}/conversations.json (anonymous/ when auth is disabled). Design choices: - In-memory cache backed by sync.Mutex so concurrent saves don't interleave writes. - Atomic write via tmp file + os.Rename so a crash mid-save never leaves a corrupted history. - ID validation regex blocks path-traversal payloads at the store boundary, in addition to the auth context constraint. Tests cover round-trip persistence across instances, user isolation, unsafe-ID rejection, bulk replace migration, and the anonymous fallback path. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * feat(api): expose chat conversation CRUD endpoints (#9432) Wire the chathistory store into the HTTP layer. New endpoints under /api/conversations: - GET /api/conversations list - POST /api/conversations upsert (id in body) - DELETE /api/conversations delete all - PUT /api/conversations/bulk replace entire set (localStorage migration) - GET /api/conversations/:id fetch one - PUT /api/conversations/:id upsert (id in path; path id wins over body) - DELETE /api/conversations/:id delete one All endpoints scope queries to the authenticated user via getUserID(c) - callers cannot impersonate other users by passing a user_id in the body. The endpoints are gated behind the new chat_history feature permission (default ON in APIFeatures), registered through the existing RouteFeatureRegistry so the unified feature middleware picks them up automatically. Application.ChatHistoryStore() returns nil when DisableWebUI is set or no persistence path is configured, in which case route registration is skipped entirely rather than registering handlers that always 503. Also adds the chat-history instruction entry and Swagger tag so the endpoint surfaces in /api/instructions and /swagger. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * feat(react-ui): sync chat history to server with localStorage fallback (#9432) useChat now probes /api/conversations on mount. When the endpoint responds 200, the server becomes the authoritative source: the hook merges remote conversations into local state and pushes per-chat PUT updates on each debounced save. When the endpoint 404s - older LocalAI deploys or the feature disabled - the hook silently keeps the old localStorage-only behaviour, so the change is backward-compatible. Migration: when the server is reachable but empty and the browser has local conversations with history, the hook fires a single PUT /api/conversations/bulk to seed the server. The atomic bulk endpoint avoids partial-upload states where a retry would skip already-uploaded entries. Delete operations propagate to the server when it's available; failures are silently swallowed so the local UI stays responsive on transient network errors. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * test(chathistory): rewrite tests in Ginkgo/Gomega The initial test file used the stdlib testing package (t.Fatalf / t.Errorf), which is forbidden by the forbidigo linter rule in .golangci.yml — LocalAI mandates Ginkgo/Gomega for all Go tests (see .agents/coding-style.md). Restructured the same six scenarios into Describe / Context / It blocks, with a chathistory_suite_test.go bootstrap that registers the Ginkgo fail handler. Test coverage is unchanged: basic CRUD round-trip, cross- instance persistence, user isolation, unsafe ID rejection (now via DescribeTable), bulk replace overwrite, and the anonymous fallback path. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * refactor(chathistory): switch store to GORM-backed authDB Addresses mudler's review on #9902. Chat history was the only per-user state in LocalAI sitting in a separate filesystem store instead of the shared GORM database — AgentStore and JobStore both live there. Move it onto Application.authDB so per-user state stays in one place. * ConversationRecord (GORM model) with composite primary key (user_id, conv_id) — React mints IDs locally with no global coordination so per-user uniqueness is the natural shape. * Store.New(db *gorm.DB) replaces Store.New(baseDir string). Save / Get / List / Delete / DeleteAll / ReplaceAll keep their signatures so HTTP handlers and the React UI are untouched. * ReplaceAll runs in a transaction with an Unscoped() pre-clear so a future retention sweep cannot resurrect a stale ID on re-upload. * gorm.DeletedAt in the schema means a future retention pruner is a one-line query — addresses the second half of the review (unbounded in-memory map). The map is gone: reads go straight to the DB. Trade-off worth flagging in the PR reply: chat history persistence now requires auth.Enabled = true (that's what initialises Application.authDB). With auth disabled the React UI transparently falls back to localStorage, the same path the original PR took on 404. Consistent with AgentStore / JobStore; if a "chat history always on" mode is preferred we can pull DB init out of auth.InitDB into a shared helper. Assisted-by: Claude:claude-opus-4-7 [Read Edit Bash] Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * refactor(schema): drop now-unused ConversationsFile envelope ConversationsFile was the JSON wrapper for the old file-based chat history store — a {Conversations: [...], UpdatedAt: ...} pair serialised to disk under {baseDir}/{userID}/conversations.json. The previous commit replaced that store with a GORM-backed one that writes rows directly as ConversationRecord, leaving the envelope without a consumer. Assisted-by: Claude:claude-opus-4-7 [Read Edit Bash] Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * test(chathistory): switch tests to testutil.SetupTestDB Mirror the AgentStore test setup: spin up a PostgreSQL 16 container via testutil.SetupTestDB(), construct a Store on top of it, exercise CRUD round-trips. The previous file-based fixtures (TempDir, filepath.Join checks) are no longer meaningful now that the store talks GORM. Tests are skipped on macOS (testcontainers needs Docker) and run on CI, same as the other GORM-backed suites in the repo. Also reframed two cases: * "unsafe IDs" → "malformed IDs" — the DB-backed store doesn't care about path traversal, but idRegex still rejects whitespace / control characters so IDs stay safe in logs and HTTP responses. * "anonymous user" no longer pins a directory path (no directory exists anymore); instead it checks the round-trip plus the per-user isolation guarantee. Added a DeleteAll round-trip to cover the bulk-clear endpoint. Assisted-by: Claude:claude-opus-4-7 [Read Edit Bash] Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * test(chathistory): cover the conversation HTTP handlers The seven /api/conversations handlers had no tests. Inject the *chathistory.Store into the handlers instead of the whole Application and move the route table into RegisterChatHistoryRoutes, so a spec can drive the real registration and middleware chain (auth.Middleware plus the chat_history feature gate) against an in-memory SQLite auth database. The new specs cover 401 without credentials on every route, 403 when an admin revoked chat_history, the CRUD round trip, path id over body id, 400 on malformed ids and bodies, per-user isolation, bulk replace and delete-all, and that a nil store registers nothing (the UI then falls back to localStorage). The handlers' nil-store branches are gone: the routes are only registered with a store, so those branches could never run. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(chathistory): make deletions final with tombstones and versions A conversation deleted on one device came back as soon as a client with a stale copy wrote it again: Delete soft-deleted the row, and the next Save went through GORM's upsert, which cleared deleted_at and resurrected it for every device. The lookup that recovered CreatedAt also ran outside any transaction, so concurrent writers raced on it. The server is now authoritative over deletions: * Delete turns the row into a tombstone: content dropped, deleted_at set. It is idempotent, and deleting an id the server has not seen yet still records a tombstone, so a delete that overtakes the UI's in-flight debounced save of the same chat still wins. * Save runs in a transaction, refuses a tombstoned id with ErrDeleted (HTTP 410 Gone), and bumps the version with a single UPDATE ... SET version = version + 1 guarded by deleted_at IS NULL, so concurrent writers each count and a delete committed between the lookup and the write still wins. Two concurrent first writes of the same id retry once (ON CONFLICT DO NOTHING, then update). * Every row carries a server-assigned version (1 on the first save, +1 per write including the delete), returned to the client. A non-zero version tells the UI the server has seen a conversation, which is how it can tell a never-synced draft from a server-side deletion. * GET /api/conversations returns the tombstones under "deleted". * DeleteAll and ReplaceAll leave tombstones too. ReplaceAll no longer hard-deletes everything first; it upserts the payload, skips ids that were deleted (reported in "skipped"), and tombstones the rest. * createdAt/updatedAt are stored as the client sends them (only zero values are stamped), so clients compare updatedAt from their own clocks rather than against the server's. The store specs now run against PostgreSQL (testcontainers) and, with the auth tag, SQLite through auth.InitDB, since both back the auth database. The API specs cover a stale client reconnecting after a remote deletion. Swagger is regenerated with swag v1.16.6 (json.RawMessage fields get swaggertype), and the feature is documented, including that it needs user auth. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(react-ui): let the server decide which chats were deleted mergeRemoteAndLocal kept every chat that was only in the browser and let the server copy win for every shared id. A stale browser therefore uploaded chats deleted on another device as if they were new drafts, and a message sent while the list request was still in flight was replaced by the older server copy and then lost from localStorage too. The first load now reconciles with the server's tombstones and versions: * a chat the server lists under "deleted" is dropped locally; * a chat on both sides keeps the copy with the newer updatedAt, so an edit made during the load wins and is uploaded afterwards; * a chat only in the browser is uploaded only if it was never synced (no version). With a version the server has seen it and no longer has it, so it is dropped instead of resurrected (this also keeps one account's cached chats out of another account on a shared browser); * deletes made while the list is loading are queued and sent once it arrives. Saves record the version the server returns, and a 410 on save drops the chat locally instead of retrying it forever. Chats received from the server count as already sent, so adopting a server copy no longer echoes it straight back. The bulk localStorage migration is gone: it replaced the whole server set from a possibly stale browser, and the per-chat saves already upload never-synced drafts through the tombstone check. The Playwright spec drives each case against a stubbed /api/conversations that follows the server contract. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(chathistory): keep the LocalAI Assistant toggle on synced chats master's createNewChat sets localaiAssistant, but serializeChat did not send it and schema.Conversation had no field for it, so a chat with the Assistant mode on came back with it off after a reload from the server (and the local cache dropped it too). Add the field on both sides. The other fields createNewChat sets were already covered; a handler spec now round-trips all of them, and the UI spec checks the toggle on the way up and in the local cache. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(react-ui): scope chat cache per account, retry unconfirmed deletes Two gaps in the client half of server-side chat history, both found in review: The browser cache was one shared localStorage key. A draft that one account could not upload (a failed first save) stayed in it, and after that account signed out and another signed in, the reconcile kept the unversioned draft and the save pass uploaded it into the second account. The cache now lives under localai_chats_data:<user id>, Chat remounts its state when the account changes, and nothing is synced without a signed-in account. The shared key is left for browser-only use without auth and is not uploaded when signed in, because nothing tells which account it belongs to. Deletes were dropped locally and their DELETE errors swallowed; only deletes made while the first list was loading were queued, and not persisted. A delete made while the server was unreachable therefore came back with the next list. Deletes (single, delete all, and deletes on the Home page) are now kept as pending deletes in the account's cache until the server confirms them, override the server list in the reconcile, and are sent again on the next change, on the online event and on the next load. While a delete-all is pending no conversation is uploaded, so a chat started after it is not caught by it. The Playwright specs replay both review scenarios (account switch after a failed first save; delete and delete-all while offline) with counter-checks that the same account keeps and uploads its own draft and that a confirmed delete is neither re-sent nor resurrected. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(react-ui): persist cleared chats immediately Navigation cancels the debounced cache write and can restore unsynced chats after Clear all. Save the cleared list with its pending deletion before returning, and cover immediate navigation in the browser spec. Assisted-by: Codex:GPT-6 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * test(react-ui): make the chat history sync specs deterministic - "clear all persists before navigation" opened a single chat, and the menu offers no Clear all for one chat, so the spec timed out in every run. It now opens two unsynced chats and checks that neither survives. Reverting 03cd9c115's save in clearAllChats turns it red again. - "a delete the server confirmed is not sent again" reopened the page as soon as the server had the request, before the page had the answer; the unconfirmed delete was then rightly sent again (4 of 5 runs). It now waits until the cache holds no pending delete. Docs-Not-Needed: test-only change. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(chathistory): keep the chat's reasoning effort on the server reasoningEffort (#12700, #12716) is part of the chat the UI syncs since the master merge, but schema.Conversation had no field for it, so the server dropped it and a reload from the server reset the setting. Add the field. The handler spec that round-trips every chat field now includes reasoningEffort. A schema test reads serializeChat in useChat.js and fails for any key the UI sends that schema.Conversation cannot hold, so the next such drift shows up as a failing test rather than in review (it reports reasoningEffort on the previous head). Swagger and docs updated. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * test(chat): use Ginkgo for the schema drift guard The new drift guard uses testing methods forbidden by repository lint. Keep its field coverage and fail-fast parser checks in Ginkgo/Gomega. Assisted-by: nib:gpt-6.1-sol --------- Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> Co-authored-by: TLoE419 <tloemizuchizu@gmail.com> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> Co-authored-by: mudler-agent <mudler-bot@c3os.io> |
||
|
mudler-agentandEttore Di Giacinto
|
f85087f992 |
feat(tts): stream Qwen text and add assistant preset (#12770)
* feat(tts): stream Qwen text through audio pipelines Feed cleaned LLM deltas into one persistent Qwen synthesis session. Return audio before input ends and preserve the live RPC through model cache, staging and in-flight wrappers. Stop generation on sink failures or cancellation without replaying speech through a fallback. Pin the incremental native fork and verify early audio, late text, finalization and cancellation with lifecycle and real-model checks. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * build(tts): use the organization Qwen fork Fetch the reviewed incremental synthesis commit from localai-org so backend builds use the organization-maintained fork. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add a Qwen streaming assistant preset Expose an English pipeline using existing VAD, ASR, LLM and named-voice TTS models. Enable incremental text and audio delivery, and document separate component installs because the gallery has no dependency installer. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): use Parakeet TDT v3 for the assistant Select the existing Parakeet TDT 0.6B v3 transcription entry and update its prerequisite install command. Keep the preset's English voice and incremental Qwen synthesis configuration. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(kokoros): reject unsupported backend RPCs Implement the MotionStream and SlotCache methods required by the shared protobuf service. Return Unimplemented so Kokoros compiles without advertising support for either operation. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(tts): mark cleanup and logging results ignored Explicitly ignore connection shutdown, test server shutdown and Ginkgo diagnostic writer errors. These operations do not replace the stream's inference or sink errors, and errcheck requires their return values to be acknowledged. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(tts): stream buffered clauses in realtime pipelines Start clause synthesis while LLM text is still arriving when both streaming flags are enabled. Keep explicit clause opt-out and native Qwen sessions, bound automatic queued work, and choose streaming support on resolved failover targets. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(tts): bound synthetic voice clone generation The sine-wave reference checks speaker conditioning and WAV output, but does not provide a meaningful speech EOS target. Limit that smoke request to 128 frames and verify the request limit reaches native synthesis. Leave production sampling defaults unchanged. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(tts): require explicit clause buffering opt-in Keep whole-reply synthesis for nonnative engines when clause_chunking is unset or false. Use the bounded cancellable worker when explicitly enabled, with unary clauses when TTS streaming is disabled. Preserve native Qwen precedence for every clause setting. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add VITS and Piper streaming assistants Provide comparable full pipeline configurations with shared VAD, Parakeet transcription, and Qwen chat components. Enable clause buffering explicitly and document the separate prerequisite model installs and Talk selection. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Stefan Walcz
|
f4505dbb5d |
feat(pii): optionally block content the detectors cannot inspect (#12755)
The PII middleware scans the text of every message, but images, audio,
video and files pass to the backend unchecked. For an egress policy
("nothing sensitive leaves the box") that is a hole: a photographed
invoice or an attached PDF reaches a cloud-proxy model without any check.
Add pii.block_unscanned_media. While PII is enabled for a model and the
option is set, a request that carries such content is rejected with the
usual 400 pii_blocked response and an audit event with pattern
__unscanned_media__, before any detector runs. Adapters report the
kinds of unscanned content through a new optional Adapter.Unscanned:
OpenAI chat (non-text content blocks), Anthropic messages (image and
document blocks; tool blocks stay allowed) and Ollama chat (images).
Off by default, so existing configurations behave as before.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
|
||
|
|
e715af1def |
feat(llama-cpp): add slot prompt-cache save and restore API (#12730)
* feat(llama-cpp): add slot prompt-cache save and restore API Add POST /v1/slots/save and POST /v1/slots/restore. They write one llama.cpp slot's prompt cache (KV state) to a file and load it back, so a client can warm a long system prompt in a few hundred milliseconds instead of re-running prefill. Refs #12685. The new SlotCache gRPC call posts the same slot save and restore tasks the llama-server /slots endpoint uses. It is off unless the model sets the new slot_save_path option, so existing models are not affected. The /slots monitoring endpoint is not needed here: the backend has no HTTP server. The file name comes from the caller, so it is checked twice, in Go and again in the backend. It must be one path component: no empty name, "/", "\", "..", NUL, control characters or absolute path. The resolved path, with symbolic links followed, must be a direct child of slot_save_path. The routes are admin only because they write files on the backend host. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): answer 501 when a backend lacks slot cache A backend that does not implement the SlotCache rpc returns gRPC UNIMPLEMENTED. The API passed that text through, so the client saw "unknown method SlotCache". Map the status to a typed error in core/backend and return 501 with a fixed message instead. Every other error path is unchanged. Make the SlotCache comments in backend.proto backend-neutral: slot is the backend's state slot, filename must stay inside the save directory, and an empty model_identity means the backend does not check it. Only comments change, no field or name. Add tests that the identity sent is the configured model, and for the error mapping. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): issue slot cache IDs on the server The slot save and restore API let the client pick the file name. Restore could load any file in slot_save_path, and the symlink check ran before llama.cpp opened the file. The backend now issues the name. A save returns a random 128-bit ID (32 lowercase hex characters, read from /dev/urandom) and never overwrites an existing file. Files live in a 0700 directory per model, named from the SHA-256 of the model identity, with a manifest.json that records each save. The manifest is written atomically and a corrupt one stops both save and restore. A restore takes the ID, finds it in the manifest, refuses an entry of another model, builds the path from the entry and checks with lstat that it is a plain file of the recorded size. The directory mode is the main defence against another local user, since llama.cpp still opens the path itself. Each model keeps at most 32 saves and the oldest is deleted first. A file over 8 GiB is deleted after the save, because llama.cpp does not report the size before it writes. Responses carry no host path. The HTTP bodies change: save takes {model, slot}, restore takes {model, slot, id}, and an unknown ID returns 404. In the proto, SlotCacheRequest.filename and SlotCacheResponse.filename become id. SlotCache no longer calls checkModelIdentity, which reads a field this request does not have. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): copy slot_cache files in prepare.sh grpc-server.cpp includes slot_cache.h, but prepare.sh copies the helper headers into llama.cpp/tools/grpc-server/ one by one and skipped it. The llama-cpp, turboquant and bonsai builds share this step, so they all failed with "slot_cache.h: No such file or directory". Copy slot_cache.h and slot_cache_test.cpp, which CMakeLists.txt builds when LLAMA_GRPC_BUILD_TESTS is on. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(deps): bump LocalAGI to pick up the scheduler start/stop fix LocalAGI's scheduler could hit a nil pointer in run when Start and Stop raced. That made the core/services/agentpool tests panic at random and turned tests-linux red. Move to mudler/LocalAGI@7aa0c8fea0, the merge of mudler/LocalAGI#519 ("make Start and Stop safe to call concurrently"). The new module version also adds a few indirect requirements to go.mod and go.sum. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com> |
||
|
mudler-agentandEttore Di Giacinto
|
c7ce0c602c |
fix: stop listing the models directory on request paths and cache GGUF header reads (#12683)
* fix(http): stop listing the models directory on embeddings and bearer requests Every embeddings request read the whole models directory before the body was parsed, even when the body named its model. On a network-backed models directory this cost seconds per request. Embeddings now binds the body first and looks up a default model only when the body has none, like chat. The pre-bind default lookup and the gpt-4o constant default are gone from that route. The first-available default lookup skips the directory when the filter cannot accept an unconfigured file. Every usecase filter rejects files without a config, so the scan never added results for them. Match-all, nil and name filters keep the scan. The bearer token check is now an exact test: a configured name, or one stat of that name in the models directory. The value is a literal, not a regular expression, and a name that is not a local relative path never matches a file. Assisted-by: Claude Code:claude-haiku-5-5 * perf(config): cache GGUF header reads for model defaults Model defaults re-parsed the GGUF header on every SetDefaults call, which runs per request and at startup reload. Put a bounded LRU in front of the parser, keyed by absolute path and validated by size and modification time. The cached value is a snapshot of the fields the defaults use, not the parser output. A changed size or mtime is a miss. Missing files and parse errors are not cached, so the existing fallback is unchanged. LOCALAI_DISABLE_GUESSING, the empty-path guard, the alias early return, the panic recovery and the context-size default are all kept in front of the cache. Assisted-by: Claude:claude-haiku-5-5 * test(http): cover listing-free default lookup on every usecase route Assisted-by: Claude Code:claude-haiku-5-5 * feat(config): persist the GGUF header cache per instance The header cache kept parsed GGUF results only in memory, so every restart parsed each model file again before its defaults were known. Write the cache to a per-instance file in the data path and load it at startup. A restarted frontend with a fixed instance ID starts warm. The file is versioned and records the estimator inputs (gguf-parser version, candidate contexts, auto context size, offload count). A mismatch on any of them makes the file ignored. Each entry keeps path, size and mtime, so a changed model file is a miss on load. Writes run on one background saver, debounced, through a temp file and rename. The entry bound applies to the file as well. Without a data path the cache stays in memory only. Documented as a per-instance exception in the model configuration docs. Assisted-by: Claude:claude-haiku-5-5 * docs(embeddings): document model selection and the no-model error Assisted-by: Claude Code:claude-haiku-5-5 * refactor(config): split the header cache into a generic cache The GGUF header cache mixed two things: the mechanics (bounded LRU keyed by path and identity, per-instance file, debounced atomic save, tolerant load) and the GGUF fields. Move the mechanics into headerCache[S], driven by a headerCacheSpec[S] that names the file prefix, schema, fingerprint, parser and the snapshot encode/decode pair. The GGUF cache becomes one instance of it. File names, the identity check, LRU size, debounce, atomic write and the corruption handling are unchanged. The on-disk JSON keeps the "estimator" key so existing files still load. Assisted-by: Claude Code:claude-haiku-5-5 * feat(config): share one GGUF header cache file between frontends Every frontend used to write its own header cache file, named after its instance ID, so a header read by one frontend was never reused by another. The cache now uses one file, header-cache.json, in the data path, for all frontends and for standalone mode. The instance ID no longer changes the file name. Frontends share the file in three ways. A save first merges the file's entries in, so one frontend's write does not drop another's entries. A miss reloads the file when its modification time has changed, and a hit never reads it. Writes go to a temp file that is renamed into place, so a reader never sees a partial file. Two frontends saving at the same moment can still drop each other's last entries. That costs one re-parse, never a wrong value, because every entry is checked against its model file before use. The shared file is valid only when every frontend mounts the same data volume. The distributed-mode and model-configuration docs state this mode and its volume limit. Assisted-by: Claude:claude-haiku-5-5 * feat(config): give an instance ID its own header cache file Assisted-by: Claude Code:claude-haiku-5-5 * style(config): gofmt the distributed config struct Assisted-by: Claude Code:claude-haiku-5-5 --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-botandlocalai-org-maint-bot
|
f02d5244da |
fix(agents): warn about unowned data with auth (#12669)
Warn before the standalone pool starts root agents with user auth enabled. Count root agents, collections, skills, and configured skill repositories. Document the auth transition, bootstrap rules, and agent key behavior. Addresses the warning and documentation step of #12665. Admin visibility and data migration remain separate work. Assisted-by: Codex:GPT-6 Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> |
||
|
Richard Palethorpe
|
3de91d48d9 |
feat(motion): add GEM-X backend and duplex streaming API
Add the GEM-X native backend with live pose and root displacement output, bounded duplex WebSocket streaming, session authorization, metrics and usage accounting. Share the embedded bidirectional RPC channel adapter across streaming backends.
Include backend build and gallery integration, API documentation, schemas and tests.
Assisted-by: Codex:gpt-6
Signed-off-by: Richard Palethorpe <io@richiejp.com>
(cherry picked from commit
|
||
|
Ettore Di Giacinto
|
f420bfcc26 |
fix(pii): give extend_to_previous_word its own settings order
It shared order 215 with extend_to_next_word, so the settings UI had no defined order between the two. Move it to 216 and shift the protected terms fields. Pin that a number in front of a hit is never taken. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Stefan Walcz
|
e3341d21a0 |
feat(pii): extend a detection over the previous word (extend_to_previous_word)
The mirror of extend_to_next_word. A name model often tags the surname with
high confidence and scores the first name in front of it below min_score,
so "Gast Greta Waldmeister" became "Gast Greta [REDACTED:ner:LASTNAME]".
extend_to_previous_word lists entity groups whose accepted detections also
cover the word directly before them on the same line. The preceding word
must consist of letters (plus '-' and the apostrophe): a word with digits -
an amount, a date, a room number - is never taken, nor is a protected term,
and the search never crosses a line break.
Docs: operations/middleware.md and advanced/model-configuration.md.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit
|
||
|
Ettore Di Giacinto
|
a0e844d611 |
fix(pii): keep hyphen and apostrophe words whole around protected terms
A protected term that is one half of "Anna-Lena" or "O'Brien" is not a whole word: shielding it left the term in the clear next to a masked rest. Hyphen and apostrophe joiners between letters now count as in-word, as they do for the word extensions. When a detection spanned the placeholder and the name after it, the space beside the protected term stayed inside the mask. Trim whitespace from the surviving pieces. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
c7ca54159e |
fix(pii): cache protected matchers by term list and match Unicode spaces
Look up the compiled matcher by comparing the raw term list instead of normalising, sorting and joining it on every call, compile outside the lock, and evict the least recently used entry instead of the oldest. Whitespace inside a term now also matches no-break, narrow no-break and other Unicode spaces, so "Hotel Seeblick" written with a no-break space is still protected. Close the term file with the repo's errcheck-clean idiom, and note that the multi-segment path scans the joined document. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Stefan Walcz
|
a7f47a3e4d |
fix(pii): bound the protected matcher cache
Term-file reloads retain each compiled matcher indefinitely. Limit the
cache to 32 lists and verify that evicted lists still match on reload.
Assisted-by: Codex:GPT-6
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit
|
||
|
Stefan Walcz
|
e16b9d04cc |
feat(pii): protected terms that pass the NER filter unchanged
A name model reads many business names as surnames. Measured with
privacy-filter-multilingual on 24 synthetic German receipts and
invoices: the supplier name ("Hotel Seeblick", "Café Lindenhof", ...)
was masked as LASTNAME (0.78-0.98) or FIRSTNAME in 9 of 12 receipts
that carried one. The supplier is exactly what the upstream model needs
to categorise a receipt, and no score floor helps at those confidences.
pii_detection.protected_terms (inline) and
pii_detection.protected_terms_files (files inside the models path, one
term per line, '#' comments, re-read when size or mtime change) list
values that must pass unchanged. Each whole-word, case-insensitive
occurrence is replaced by a neutral placeholder before the detector
runs; every detection is mapped back onto the original text with the
placeholder parts cut out, so a protected term is never masked or
blocked, the detector never sees it as context, and event offsets keep
addressing the original text. A word extension (extend_to_next_word)
is dropped rather than run into a protected term.
Terms shorter than three characters are ignored. A missing or
unreadable file is logged and skipped: fewer protected terms means more
masking, never less. Paths are verified to stay inside the models path.
Opt-in, empty by default. Covered by Ginkgo specs in
core/services/routing/pii/protected_test.go (detector-visible text,
exact restoration, straddling detections, whole-word/case/line-wrap
matching, the segment path, file loading and reload), a YAML round-trip
spec, registry entries and the middleware/model-configuration docs.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit
|
||
|
Stefan Walcz
|
9b13408c94 |
feat(pii): extend a detection over the next word (extend_to_next_word)
Address models tend to tag a postal code with high confidence but miss
or under-score the town right after it. Measured with
privacy-filter-multilingual on 24 synthetic German invoices: the postal
code was masked every time, while the town after it stayed in the clear
in 6 of 18 cases (4 not detected at all, 2 tagged CITY at 0.43/0.47,
below the configured floor). Masking the code alone leaves the pair half
redacted: "[REDACTED:ner:ZIPCODE] Ludwigshafen".
pii_detection.extend_to_next_word lists entity groups whose accepted
detections also cover the word that directly follows them on the same
line, with the same action and group. Horizontal whitespace is skipped,
a line break ends the search, and nothing is extended when no word
follows, so a hit never grows over bare whitespace or onto the next
line (or across the segment separator on the multi-message path).
The option is opt-in and empty by default; existing detector configs
behave exactly as before. Covered by Ginkgo specs in
core/services/routing/pii/extend_test.go (fail without the change), a
YAML round-trip spec, a registry entry for the model editor, and the
middleware and model-configuration docs.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit
|
||
|
mudler-agentandEttore Di Giacinto
|
99d6b35cb5 |
perf(http): skip discovery for configured requests (#12628)
Exact configured requests need no loose-file enumeration. Reuse the synchronized config lookup without adding a cache. Misses retain regex discovery and raw-weight fallback with ALWAYS_INCLUDE. Keep general listing and bearer discovery unchanged. Preserve request validation, alias resolution, disabled-target checks, and model access. Record diagnostics only for work performed. Assisted-by: nib:gpt-6-astra Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
0e0e91f95c |
feat(diagnostics): add opt-in profiling (#12610)
* feat(diagnostics): add private opt-in profiler Keep profiling on a dedicated loopback listener rather than the public API. Validate settings and bind synchronously so startup reports configuration and socket failures. Own sampling and shutdown for the run process. Cover private handlers, blocked cmdline access, sampling cleanup, and serving failures with focused Ginkgo tests. Assisted-by: nib:gpt-6-astra * feat(diagnostics): add sanitized phase timing Carry generated request and reload identities through private contexts. Keep timing disabled without allocations and emit only validated fields at Info level when enabled. Cover identity isolation, nested durations, inheritance, and sanitized logging with focused race-tested Ginkgo specs. Assisted-by: nib:gpt-6-astra * fix(diagnostics): retain request observations Reuse an existing request observation across recorders to avoid splitting correlation. Keep the original context, identity, and sink without new allocations. Preserve nil-recorder behavior and fresh reload identities. Verify the regression fails before the fix and passes with the full Diagnostics race suite afterward. Assisted-by: nib:gpt-6-astra * feat(cli): own opt-in diagnostics lifecycle Expose profiling and phase timing flags without enabling them for debug. Bind the private profiler before application initialization and propagate serving failures through the run lifecycle. Bound signal cleanup and preserve preload-only backend startup. Cover flag parsing, preload isolation, cancellation, and once-only shutdown. Document initialization and shutdown cancellation limitations. Assisted-by: nib:gpt-6-astra * feat(config): measure filter and reload phases Separate request lock waits from filtering and held work. Aggregate reload filesystem and parsing durations without logging inside the config lock. Preserve caller option execution under the lock after metadata collection. Extract only the internal diagnostics option before waiting for the lock. Keep direct callers uninstrumented unless they supply diagnostics options. Verify focused Diagnostics examples with the race detector, including option ordering, gated contention, errors, panic propagation, and parity. Assisted-by: nib:gpt-6-astra * fix(config): separate early reload observation Opaque callbacks cannot select timing before their execution point. Carry the startup recorder explicitly through loader construction. Apply late options once under the lock and honor composed last writes. Document early and late phase boundaries and observation identities. Report malformed gallery syntax as an internal parse failure without changing strict or non-strict loader returns. Assisted-by: nib:gpt-6-astra * fix(config): preserve per-model option callbacks The resolved snapshot suppresses existing callbacks during model defaults, even when diagnostics are disabled. Pass original options through the existing defaults pathway without adding diagnostics-driven replay. Test stateful options with two models in enabled and disabled modes. Preserve three calls and thread defaults of two and three. Assisted-by: nib:gpt-6-astra * feat(model): observe discovery phases Carry request diagnostics through config and loose-file discovery to separate directory enumeration from in-memory filtering and fallback. Keep existing APIs as background-context wrappers and preserve discovery policies, ordering, duplicates, errors, and filesystem operation counts. Add focused parity, privacy, phase, and disabled-allocation tests. The focused diagnostics race gate passes for model and galleryop. Assisted-by: nib:gpt-6-astra * feat(http): observe extraction phases Generate diagnostic identity before request discovery, independently of client correlation headers. End each extraction segment before inference so downstream latency does not inflate discovery measurements. Carry observations through discovery helpers and rebuilt request contexts without changing cancellation or model selection. Cover disabled parity, error outcomes, privacy, concurrent identity isolation and route contexts. Assisted-by: nib:gpt-6-astra * fix(http): retain diagnostics across failover Failover restores its entry request before retrying. Initialize the observation before that capture so direct extraction retains one ID. Keep extraction spans per attempt and end them before downstream work. Exercise two real targets without default middleware, including replay, shared identity, phase boundaries, and timing-off response parity. Assisted-by: nib:gpt-6-astra * feat(model): observe initialization and routing Mark loader entry before cache and admission checks. Measure only the ModelRouter callback, without attributing transport or worker latency. Test inherited identity and sanitized events with fake backends. Exercise the actual Echo router to keep profiles private across debug and timing options. Cover capture shutdown, forced close, and lifecycle failures. Assisted-by: nib:gpt-6-astra * docs(diagnostics): guide bounded investigations Document private profiling, sampling ownership, and phase boundaries so operators can interpret captures without assuming request attribution. Add Ginkgo drift checks for flags, defaults, phases, links, and capture warnings. These checks do not certify the operator prose. The focused documentation race test passes after four expected failures. The consolidated Diagnostics race gate passes across all eight packages. This gate does not validate the full suite, auth, or production UI. Assisted-by: nib:gpt-6-astra * refactor(cli): restore inline run lifecycle Remove diagnostic startup hooks so application startup stays readable in Run. Keep the profiler as an optional sidecar with bounded cleanup and an error watcher, without a general lifecycle runner. Replace hook simulations with real CLI startup checks and component capture tests. Document that application cleanup can exceed the private listener budget. The eight-package Diagnostics race gate passes. A cleanup mutation fails the real CLI listener-release regression as expected. Assisted-by: nib:gpt-6-astra * fix(cli): retain concurrent startup errors Cancellation must not hide a public API bind failure. Join the public serve result, cancellation cause, and HTTP shutdown errors inline. Ignore ErrServerClosed only after intentional shutdown. Exercise real startup with an occupied API port and a failed private listener. Isolate the Linux socket fixture in a subprocess and wait for cancellation acknowledgment without timing sleeps. Assisted-by: nib:gpt-6-astra * refactor(http): make request context primary Move context preparation into the request extractor. Diagnostics remain an optional addition, not the entry point for request processing. Preserve context initialization before failover captures the request. Assisted-by: nib:gpt-6-astra --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
6343a2dc6f |
feat(voice-detect): report the encoder family so audio-registered voices are fingerprinted (#12499)
* feat(voice-detect): report the encoder family so audio-registered voices are fingerprinted
A voice registered from audio through the voice-detect backend had no
encoder fingerprint, so the parakeet-cpp backend could not tell whether
it was comparable with the loaded speaker model and could only fall back
to the file-name rule.
The voice-detect backend now binds the three new libvoicedetect accessors
with a symbol probe (an older library still loads and reports nothing),
copies the borrowed strings at once and never frees them. It also hashes
the model file once at load. VoiceEmbedResponse gains two optional
fields, encoder_family and encoder_weights ("sha256:<hex>", empty when
the model is not a plain file).
/v1/voice/register stores them as encoder_family and a new
encoder_weights field in the registry entry; model keeps the encoder
name, so the 1:N identify filter by name is unchanged for old entries.
A voice with a family is sent to the backend whatever its file name, and
the backend decides by family. /v1/voice/identify compares the family
when both the stored voice and the probe have one. Old entries load
without the fields and stay unfingerprinted.
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(voice-detect): say how a voice with weights but no family is treated
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(voice-detect): mark the model file open as intended for gosec
Assisted-by: Claude:claude-sonnet-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
|
||
|
mudler-agentandEttore Di Giacinto
|
895d50385f |
fix(distributed): converge model configs across frontends (#12558)
* fix(galleryop): announce model changes before the preload After a gallery install or delete, the replica that ran it replaced its config loader, then preloaded every installed model, and only then published the models invalidation. The preload does remote lookups and checksums for each model, so on a large models directory peers learned about the change minutes after the originator listed it. When the preload failed or the operation was cancelled, the event was never sent. Publish the invalidation, and apply the delete lifecycle, as soon as the loader holds the new set. The preload still runs afterwards with its own error handling. Its failure is reported on the operation, but it no longer rolls back a deletion that peers have already applied. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): resync model configs from the models directory Frontends refresh their model configs only when a models invalidation arrives on NATS. NATS keeps no history, so a frontend that is disconnected when the message is published never applies the change. It keeps serving the old config, for example an alias that points at the previous model, until some later change happens to touch it. Each frontend now reruns the peer reconcile against the shared models directory after every NATS reconnect, and every --model-config-resync-interval (default 30s) when a config file changed. The pass names no model, so only models whose file changed get a revision transition, and an unchanged directory costs one read of the config files. The reconcile replaced the whole loader with a parse of the models directory, which dropped models loaded with --config-file and published a deletion revision for them. Configs defined outside the directory are now kept, both there and after a gallery install. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(nodes): stop stale frontends from retargeting alias rules A scheduling rule keyed by an alias derives its target from the alias mapping of the frontend that reads it. Every frontend keeps its own copy of the model configs, so after an alias is repointed a frontend that has not reloaded it still resolves the old target. Two frontends then rewrote the rule's stored target_model against each other on alternate reconciler ticks, and the outdated one scaled up the model the alias used to point at. The registry already records the accepted config revision of each model. A frontend now derives a rule's target from its own alias mapping only when its config revision for the rule's name matches that record. Otherwise it keeps the stored target_model: it neither writes the column nor reconciles replicas of the old target. The check reads the database only for a rule whose stored and derived targets differ. With no accepted revision on record, the old behaviour stays. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agent
|
cd604b5c5b |
fix(distributed): bound, stop and cancel model loads with leases and worker operations (#12524)
Fence model load jobs by generation, lease them on the database clock, bound the work on the worker with operations and a process-group watchdog, and add one stop path with a load-cancel API. See the pull request for the design, the rolling upgrade notes and the test evidence. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-botandEttore Di Giacinto
|
217038d4fe |
fix(agents): share the collection list across frontends (#12536)
With the PostgreSQL vector engine, each frontend kept the collections it had opened in an in-memory map and answered "collection not found" for any other. A collection created through one frontend was unknown to the others until they restarted, and each frontend listed a different set. Wrap the in-process collections backend in distributed mode so that the database is the source of truth: - lists come from the registry, - a lookup miss checks the registry before it returns 404, and opens a collection that exists there (once per name, even under concurrency), - a cached collection that left the registry is dropped and closed, with a re-check at most every 5 seconds, - create and reset publish an event on the existing collection invalidation subject, so other frontends re-check at once. Without the postgres engine, or outside distributed mode, nothing changes. Assisted-by: Claude:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
|
e2faec2bf2 |
fix(distributed): free only the named model on model.unload (#12537)
The worker answered model.unload by calling Free on the first backend process in its map, whatever model the request named. Every idle scale-down or LRU eviction of one model could therefore empty another model's backend on the same node. LocalAI still counted that model as loaded, so its next request failed. A parakeet diarization model then returned 501 "speaker profiles require a loaded speaker encoder" until someone reloaded it by hand. Resolve the target from the model name (all replicas), prefer an address when the request carries one, and free nothing for an unknown model. Also let parakeet-cpp Diarize check the diarization model before the speaker-profile capability. A backend with nothing loaded now answers FailedPrecondition, which LocalAI treats as a stale replica and reloads, instead of a final Unimplemented. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
fmterrors
|
1cd96e496b |
chore: fix some function names in comment (#12517)
Signed-off-by: fmterrors <fmterrors@outlook.com> |
||
|
Ettore Di Giacinto
|
1d117292b1 |
feat(voice): name registered speakers through a bundle speaker component
Naming read only speaker_model:, so a parakeet-cpp bundle that sets speaker_component:voice never received the registered voices, in live transcription or in diarization. Resolve speaker_component: when speaker_model: is not set. A component is not an encoder file, so no file-name tag is derived for it: voices with a hash or family identity and untagged voices reach the backend, and a voice with only a file-name tag matches through the new optional speaker_tag: alias. speaker_model: still wins, including when it points at the bundle file. The bundle gallery entries do not declare speaker_recognition: that usecase means the VoiceEmbed and VoiceVerify RPCs, which the backend does not serve, and it selects the default model for /v1/voice/*. The pinning test now says so. Assisted-by: Claude Code:claude-sonnet-5-5 go golangci-lint |
||
|
Ettore Di Giacinto
|
a2c498359b |
fix(voice): keep one voice store per embedding dimension
The in-memory local-store rejects vectors of another size than the ones it holds. With 192-value voices registered, adding a 256-value voice failed, and identifying with a 256-value probe returned an error. The registry now keeps one store per embedding dimension. The first dimension seen uses the configured store name, so single-encoder instances are unchanged. Later dimensions use "<name>-<dim>". Identify searches only the store of the probe size and returns no match when no voice of that size exists. Forget finds the store from the stored embedding. A name may hold one voice per encoder. Assisted-by: Claude Code:claude-sonnet-5-5 golangci-lint |
||
|
mudler-agentandEttore Di Giacinto
|
79a7631cc5 |
feat(parakeet-cpp): encoder fingerprint for speaker naming, VAD trim and word filter options, pin bump (#12491)
* chore(parakeet-cpp): bump parakeet.cpp to 2de154c Brings in the speaker registry encoder fingerprint, the VAD segment trim and the opt-in word filter, a fix for a per-call thread count that stayed set on the process-wide backend after a Silero VAD pass, and bundle components loaded from memory. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(parakeet-cpp): encoder fingerprint for speaker naming, vad_trim and guard_* options Speaker naming. A registered voice now carries the encoder that made it: the embedding family (voicedetect:<arch>:<name>:<dim>) and the sha256 of the weights. The backend reports the family of the loaded speaker model in Status, voice enrollment from speaker_profiles stores it as encoder_family (old entries load without it), and the registry sent to parakeet.cpp is built with parakeet_capi_speaker_registry_add_embedding_fp. The library then refuses a registry of another encoder family and the error names both families; another quantization of the same family only warns. A voice with only a weights hash gets the loaded family when the hashes are equal. Voices without a fingerprint (registered from audio: libvoicedetect cannot report one) keep the file-name rule and are used with a warning. The library cannot mix them with fingerprinted voices in one registry, so a request that has any uses the old registry for all. speaker_strict:true drops them instead. A library without the symbols behaves as before. Transcription. vad_trim (seconds, 0 keeps the whole cuts) goes through the VAD options JSON, so it reaches /v1/vad and the segmenter. The guard_* options guard_min_local_conf, guard_local_radius and guard_drop_punct_only turn on the word filter through parakeet_capi_transcribe_path_json_with, or through the segmenter with vad:true. They are off by default, bad values fail the load, and a library without the symbol fails it with a clear message. The dropped word count is logged at debug level. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Stefan Walcz
|
5f5feea15a |
fix(agentpool): keep the conversation in the agent web chat (#12410)
* feat(agents): keep the web chat history between messages The agent chat endpoint ran every message as a fresh job (ag.Ask(WithText(message))), so a follow-up such as "add two days to item 3 and recalculate" never saw the answer it referred to. Agents then rebuilt their reply from scratch instead of changing it. Use the agent's own conversation tracker, as the Telegram and Slack connectors already do: send the earlier turns with the new message and record successful answers. Failed, cancelled or empty runs are not recorded, and the tracker drops a conversation after the agent's last_message_duration of inactivity. The distributed (NATS) chat path is unchanged. Assisted-by: Claude:claude-opus-5-5 ginkgo Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(agents): keep each web chat conversation's history separate Review on this PR: the first version kept the history in the agent's conversation tracker under one fixed key per agent. The web UI keeps several conversations per agent (New Chat, switching, Clear) and did not send which one a message belongs to, so the hidden history mixed them: a new chat received the previous chat's turns, and Clear only cleared the screen. The client now sends the earlier turns of the conversation it is showing as `history` with POST /api/agents/:name/chat, and the server keeps no web chat history of its own. Each conversation only ever sees its own turns; New Chat and Clear start without history. The server uses only user and assistant turns with text, bounded to the most recent 40 turns and 64,000 characters. `history` is optional, so existing callers keep the previous behaviour (no history); the distributed (NATS) path does not forward it yet. Tests: two conversations of one agent stay apart, an empty history (New Chat or Clear) starts fresh, system/tool/empty turns are dropped, the bounds keep the most recent turns; the UI helper that builds the history from the visible messages has node --test coverage. Docs: features/agents.md describes `history` and the web UI behaviour. Assisted-by: Claude:claude-opus-5-5 ginkgo Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> --------- Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> |
||
|
mudler-agentandEttore Di Giacinto
|
d3724eef50 |
fix(nodes): stage dedicated diarization audio (#12474)
Dedicated diarization forwards frontend audio paths to remote workers, which cannot read those temporary files. Stage the input before the RPC and release it afterward, following the transcription lifecycle. Clone the request so staging does not change caller-owned data. Cover input bytes, request fields, cleanup, and error propagation in tests. Document distributed diarization staging on the existing feature page. Assisted-by: nib:gpt-6-astra Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
99043b442c |
feat(router): route with native decision models (#12449)
* fix(schema): preserve SystemOne image inputs Assisted-by: OpenAI * test(schema): follow Ginkgo conventions for decision inputs Assisted-by: OpenAI * feat(llama-cpp): dispatch native decisions through Score Upgrade the stock dependency and reconcile Score/TTS patches. Reuse native decision parsing, tasks, formatting and response-reader cleanup; preserve ordinary scoring admission and guard older dependencies. Assisted-by: OpenAI * refactor(systemone): share request and model validation Assisted-by: OpenAI:gpt-5 * fix(systemone): preserve HTTP wire-byte validation limit Keep structural validation separate from the serialized internal request bound so HTML escaping cannot reject valid HTTP payloads. Assisted-by: OpenAI:gpt-5 * feat(systemone): bound images and account native decisions Preserve public wire limits independently from router serialization. Reject unsupported NER images, map native request/capability errors, and stamp explicit usage once. Advertise decisions for stock llama-cpp. Assisted-by: OpenAI * fix(systemone): record usage on registered native route Exercise real registration and billing with a mock native backend. Reject empty native responses, malformed image URLs, trailing JSON, and wire overflow including whitespace. Assisted-by: OpenAI * feat(router): add lazy native decision transport Bind named models through internal ModelSystemOne calls with shared validation and bounded abandoned operations. Remove request and echoed-error contents from decision traces. Assisted-by: OpenAI:gpt-5 * feat(router): classify overlapping policies with native decisions Ask independent noul questions, validate probabilities and preserve first-superset routing. Wire the central factory with config-sensitive invalidation and cancellation-safe resolution. Document native framing and bounded operation limits. Assisted-by: OpenAI:gpt-5 * feat(gallery): add pinned Julia-1 native decision model Add a separate text-only llama-cpp Q8 entry with pinned Apache-2.0 source provenance and checksum. Installed using the gallery installer and exercised choice, score and noul on CPU. Assisted-by: OpenAI * test(router): verify native decisions through central factory Add an opt-in real-model Ginkgo integration covering the native Go loader and C++ transport, token usage, independent overlapping labels, and candidate selection. Document owned-server execution and the intentionally non-quality threshold. Assisted-by: Codex:gpt-5 * fix(llama-cpp): align upstream pin and preserve decision signatures Advance to bed0a856 without losing the automated upstream bump. Detect full-request fill_task support at compile time and forward every question for Nimble framing while retaining the earlier native signature. Preserve reconciled SCORE/TTS patches; add standalone compatibility coverage. Assisted-by: Codex:gpt-5 * feat(gallery): add native decision family defaults Pin Laya, Kev-4B, lev, OpenJev and Nimble artifacts. Verify Laya/Kev/lev gallery installs and CPU contracts on both native pins; clearly mark OpenJev/Nimble runtime validation pending and their noncommercial licenses. Assisted-by: OpenAI * docs(decisions): clarify integrated Nimble prerequisite Record the exact combined backend pin while retaining pending OpenJev and Nimble installation/runtime validation status. Assisted-by: Codex:gpt-5 * fix(gallery): indent native decision model sequences Match repository yamllint indentation for Laya, Kev, lev and OpenJev list fields. Parsed gallery data is unchanged; reproduce CI gallery lint failure before the whitespace-only fix and pass the same command afterward. Assisted-by: Codex:gpt-5 * docs(decisions): record OpenJev and Nimble CPU validation Record gallery installation, checksum/metadata verification and multiquestion native smoke results on bed0a856. Retain noncommercial and text-only limitations without accuracy or deterministic-output claims. Assisted-by: OpenAI * fix(ui): expose native Decisions router classifiers Select classifier models using metadata-driven capability routing, retain tuned thresholds, and validate native decision selections before saving. Cover both native backends and create/save/reopen in the real React editor. Assisted-by: Codex:gpt-5 * fix(router): exclude aliases from native decision discovery Check the originally named config before advertising native Decisions eligibility. Retain target capability inheritance for ordinary generation aliases. Exercise the actual capabilities endpoint with native models on both backends, aliases, and disabled models. Assisted-by: Codex:gpt-5 * feat(systemone): share bounded multimodal input validation Preserve text wire limits while admitting bounded PNG/JPEG decision input. Share collection and header validation across internal and public callers and keep the native runner response budget independent. Assisted-by: OpenAI:API-assistant * fix(systemone): bound admission lifetimes and validate complete images Retain shared admission leases through actual work completion, including abandoned internal operations. Decode bounded image pixels, cap public native responses before usage stamping, and preserve oversized malformed text status precedence. Assisted-by: OpenAI:API-assistant * fix(router): classify images before media fetching Preserve ordered structured probes for native decisions. Defer OpenAI media preparation until routing selects the served model, so rejected decision URLs cannot trigger downloads before shared validation. Guard direct image collection with context-aware shared admission. Keep text classifiers and embedding caches from discarding image input. Retain fail-closed classifier configuration and runtime fallback policy. Add middleware, typed-content, admission, cancellation and cache tests. Assisted-by: OpenAI:API-assistant * fix(router): bound extraction before serialization Check probe budgets before copying text or marshaling message state. Count JSON escaping so oversized internal inputs fail before allocation. Preserve typed Anthropic blocks through selected-model conversion and fallback. Keep retry coverage in Ginkgo without global test registration. Assisted-by: OpenAI * fix(router): bound supported probe serialization Arbitrary structs can bypass the probe budget through pointer marshalers, string tags, and promoted fields. Accept concrete chat schema types and plain JSON values instead of emulating arbitrary struct serialization. Budget escaped direct prompts before marshaling so raw length cannot hide serialized expansion. Preserve runtime fallback and reject oversized input before invoking the decision runner. Add Ginkgo allocation, boundary, and marshaler invocation regressions. Six-package tests, three-package race tests, and full-T2 delta lint pass. Assisted-by: OpenAI:GPT-5 golangci-lint * feat(decisions): enable bounded OpenJev images Validate native decision images before permissive media parsing and pixel allocation. Require both decision image support and a vision projector; missing or audio-only projectors cannot silently become text decisions. Pin the OpenJev Q8 projector and document its license and disk footprint. Add native safety tests, canonical limit parity, gallery and load-option checks, and a reproducible CPU direct-RPC contrasting-image smoke. Assisted-by: OpenAI:GPT-5 * fix(decisions): reject incomplete image streams stb accepts corrupt PNG Adler checksums and truncated JPEG scans. Use bounded zlib validation and strict libjpeg decoding before parsing. Keep dimension and aggregate pixel checks ahead of decoder allocations. Wire decoder dependencies into native builds and runtime packaging. Add regressions for appended EOI and embedded marker bypasses. Assisted-by: OpenAI:GPT-5 * fix(ci): gate native decision image validation Run the decoder security tests outside the stdlib-only native suite. Fetch vendor headers at the backend pin and provision decoder dependencies. Gate Go limit parity and production CMake wiring without model downloads. Assisted-by: OpenAI:GPT-5 * test(decisions): cover multimodal public API paths Exercise shared image contracts through the registered HTTP routes and external mock backend. Add opt-in cached gallery installation and real OpenJev image decisions through SystemOne and both routing APIs. Assisted-by: Codex:gpt-5 * test(decisions): assert isolation and cache bypass Observe external RPC calls and compare complete classifier history. Winner-only and cache-miss checks could hide dropped history or cache use. Give real inference its own application and model directory so shared backend mappings and loaded processes cannot affect mixed suite order. Assisted-by: OpenAI:ChatGPT * test(decisions): isolate fixture globals Disable optional global services in the isolated HTTP fixture and register cleanup before setup assertions. Verify meter provider identity survives fixture creation and destruction. Snapshot observed usage before assertions so failures cannot retain the mutex. Require a successful usage stamp before checking error responses. Assisted-by: Codex:gpt-5 golangci-lint * fix(application): honor optional telemetry controls Skip failover gauge registration when metrics are disabled. Register against the application meter rather than looking up the global provider. Allow embedders to retain the bounded routing log without billing stats. Keep the existing default when stats are disabled. The isolated HTTP fixture uses this option without losing its native router assertions. Assisted-by: Codex:gpt-5 golangci-lint --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
e4fa051ee3 |
refactor(distributed): put the NATS-only paths behind interfaces (#12395)
* feat(messaging): add shared subject rules Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(messaging): cover BroadcastRoots, ControlRoots and SubjectRoot Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(messaging): add Broadcaster and enforce subject rules in every carrier Broadcaster is the fan-out half of MessagingClient. The NATS client and the in-memory FakeBus now refuse a subject outside the served roots and any wildcard other than a whole single token, and FakeBus shares MatchSubject instead of its own copy. FakeBus Unsubscribe now removes its own subscription instead of the first one with the same subject. A shared conformance suite in messagingtest runs against both carriers. The distributed e2e specs that used invented test.* subjects, and the one that subscribed with a > filter, now use subjects from subjects.go. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: depend on Broadcaster where only publish and subscribe are used Narrowed to messaging.Broadcaster: nodes/staging_progress.go, nodes/install_progress_publisher.go, galleryop/operation.go, galleryop/service.go, agentpool/user_services.go, agentpool/agent_jobs.go, openresponses/store.go, openresponses/sync.go, syncstate/syncstate.go, finetune/service.go, quantization/service.go and failover/distsync/distsync.go. SubscribeJSON now takes a Broadcaster because it only calls Subscribe, which lets the narrowed consumers use it. Stayed wide: worker/supervisor.go, because its client field also serves the SubscribeReply handlers in worker/lifecycle.go. The request/reply, queue and wiring files (nodes/unloader.go, nodes/file_stager_s3.go, jobs/dispatcher.go, agents/dispatcher.go, agents/events.go, worker/file_staging.go, cli/agent_worker.go, http/app.go) are unchanged by design. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): name the no-route condition and confine the carrier error Consumers matched nats.ErrNoResponders, which names an absence, to demote a node. They now match ErrNoRoute, the control path maps the carrier's failure onto it, and timeouts and worker refusals are pinned as not being no-route. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(nodes): state which FileStager implementations return ErrNoRoute Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): build backend clients through one node-aware seam Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: describe the distributed transport seams Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: correct comments that overclaim after the seams refactor Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(agent-worker): refuse an unserved LOCALAI_AGENT_SUBJECT at startup The messaging client now refuses a subject whose root no carrier serves. An agent worker started with a custom LOCALAI_AGENT_SUBJECT such as tenant-a.agent.execute used to start and then wait on a subject the frontend never publishes to. After the subject rules landed it exited at subscribe time with an error that did not name the setting. Behaviour change: the worker now checks LOCALAI_AGENT_SUBJECT before it registers or connects, and exits with an error that names the variable and says to use a served subject under the agent root, for example agent.execute. The served roots are not widened: a custom root was never delivered by the frontend, and a wider set would reopen the drift the subject rules exist to close. The flag help and the agent worker docs state the constraint. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(nodes): pin the reactions to ErrNoRoute Three callers react to ErrNoRoute and had no spec: the reconciler's upgrade drain falls back to the legacy forced install, the reconciler marks the node unhealthy when a pending op has no route, and the backend-op fan-out marks the node unhealthy. Each spec drives the real caller with a scripted no-responders reply and reads the result from the registry or the recorded requests. A fourth spec pins the other side: a pending op that times out leaves the node healthy and only counts the attempt, so mapping timeouts onto ErrNoRoute would fail here. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(messaging): pin client subject checks and fail the carrier suite in CI Add specs that call Publish, Request, Subscribe, QueueSubscribe, SubscribeReply and QueueSubscribeReply on a client with no connection. Each call must return ErrUnservedSubject for bogus.thing and ErrUnsupportedWildcard for jobs.>. This proves that the subject check runs before the connection is used, and needs no server. The NATS conformance suite is the only check that runs the subject rules against a real carrier. Before this change it skipped without output when Docker was missing. Now it fails when CI is set, so a Linux runner without Docker cannot hide it. It still skips on local runs and on macOS CI, which has no Docker. Add SubjectNodeBackendInstallProgress to the list of constructors that must build served subjects, and ask contributors to extend the list. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: state what ErrNoRoute may change, and group the distributed guides The seams note said MarkUnhealthy was the only state change allowed on ErrNoRoute. A pending backend op still records the failed attempt, counts toward the reconciler's retry limit and is dead-lettered after the maximum attempts. The note now says that MarkUnhealthy is the only change to the node's own state, and that the per-op accounting is not a verdict about the node. The note also documents that the NATS conformance run fails under CI when Docker is missing. The distributed-seams row moves next to the distributed-state row in the topics table. The liveness ping spec header now says no route is a reason to skip the worker, not proof that the worker is gone. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): give the backend client factory the node id Mechanical: the method gains a nodeID parameter and the eight test fakes are updated. No behaviour change. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): drop the optional node-aware factory The node id is now in the main method, so the optional interface and its helper had no behaviour of their own. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): dial backend probes through the client factory Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(nodes): dial workers' file servers through a per-node dialer Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(http): proxy backend logs through the per-node worker dialer The admin backend-logs proxy (list, lines and the WebSocket stream) now reaches a worker through the same per-node dialer as the HTTP file stager, so every frontend-to-worker dial goes through one seam. The shared direct dialer keeps alive for 15s where the proxy used 30s. Harmless for requests bounded at 15s. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(http): keep the backend-logs proxy independent of the admin connection The proxy request had no context before the dialer change and is bounded only by its 15s timeout. Keep it that way. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: move the worker control payloads to workerctl Mechanical move of the request and reply structs, the install progress event and the file payloads out of messaging. The verbs no longer belong to one carrier. No alias is left behind. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(worker): serve the lifecycle verbs through a controlServer The worker registers one handler per verb and a NATS server maps each verb to its subject. Registration errors now name the verb. node.stop is served with SubscribeReply, which is identical on the wire because the handler never replies. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(worker): report install progress through the control sink Install and upgrade now emit download progress through the sink the control server hands them. The debounce and the terminal flush stay in the handler path, built over that sink by the new nodes.NewDebouncedInstallProgressSink, which replaces NewDebouncedInstallProgressPublisher. The subject and payload on the wire are unchanged. The supervisor no longer holds the bus, and installFn and upgradeFn let specs drive both verbs without a gallery. The malformed-request log lines are restored for install, upgrade, backend.delete, model.unload, model.stop and model.delete, with the reply bytes unchanged. The signal adapter is renamed noReply, which also lets worker.go import os/signal without an alias again. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(worker): serve the file-staging verbs through a controlServer An empty list-dir answer is now {} rather than {"files":null}, because the typed reply omits an empty Files slice. The frontend decodes both to a nil slice in nodes/file_stager_s3.go. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(messaging): add WorkQueue and the NATS producer This is the producer side of the competing-consumer seam. The work kinds map one to one to today's subjects and queue groups: task to jobs.new and mcp-ci to jobs.mcp-ci.new (both in group workers), agent-run to agent.execute (group agent-workers). Enqueue publishes the payload as Publish does today, with one JSON marshal. FakeBus now records queue groups and keeps reply handlers so later specs can pin and drive them. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(messaging): add the NATS WorkConsumer An in-flight limit of one runs the handler inline on the delivery goroutine, as the MCP CI consumer does today. Any other limit spawns per delivery, as the agent consumer does. Queue groups are unchanged. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: publish queued work through WorkQueue The job dispatcher, the agent pool and the agent scheduler enqueue through messaging.WorkQueue; the NATS implementation publishes to the same subjects as before. DistributedServices builds the queue next to the NATS client and hands it to the dispatcher and the agent pool, whose distributed mode switch now reads a non-nil WorkQueue. The unused AgentPoolService.SetNATSClient is removed. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: consume queued work through WorkConsumer The agent dispatcher and the MCP CI consumer register through messaging.WorkConsumer. The NATS implementation keeps the inline one-at-a-time model for MCP CI and the per-delivery model for agent runs. handleMCPCIJob reports on the events publisher the carrier hands it instead of a captured client. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: delete the consumers nothing in production reached jobs.new has a producer and no production consumer, and the agent dispatcher's Dispatch was only called from tests. Publishing jobs.new is unchanged. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(mcp): send MCP requests to agent workers through AgentControl Timeouts still honour only the deadline, not cancellation, exactly as today. The NATS no-responders error maps to ErrNoRoute and a timeout does not. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(agent-worker): serve MCP requests and backend.stop through agentRPCServer The agent worker's MCP tool and discovery reply subscriptions and its backend stop listener move behind an unexported agentRPCServer interface, served on NATS by nodes.NATSAgentRPCServer. The handlers become typed mcp.ToolHandler and mcp.DiscoveryHandler values that answer every failure with a reply carrying Error. Queue group (agent-workers), inline execution on the delivery goroutine, the background handler context, the unmarshal error reply texts and the reply-less backend stop subscription are unchanged. The backend stop handler takes the decoded backend name, so it can still close that backend's MCP sessions. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(messaging): remove helpers that only tests used BroadcastRoots, ControlRoots and SubjectRoot had no production caller. The roots spec now asserts every served root through ValidateSubject instead. MatchSubject moves back into the test support package, the only place that used it, with its table. NATSAgentRPCServer drops the subscription list it stored and never read, and NewNATSAgentRPCServer gets a doc comment. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(mcp): round trip the agent RPC server over a real NATS server One spec sends a tool request and a discovery request through NATSAgentControl to NATSAgentRPCServer and checks that the handlers see the decoded requests and the replies come back. It also puts an undecodable body on the tool subject and checks the server answers with an unmarshal error instead of leaving the requester to time out. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: describe the distributed transport seams The developer note now lists the final seams: fan-out, queues, both halves of the control verbs and of agent RPC, and the dial. It records the open items a second carrier has to handle. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test: pin the in-flight limit each queue consumer asks for The work queue specs pin what Consume does for a given limit, but nothing pinned which limit each production consumer passes. Changing the agent worker's MCP CI limit from 1 to 0 would have let MCP CI jobs run concurrently on each worker with every test green. Move the MCP CI Consume call into startMCPCIConsumer with the same wiring and pin that it asks for (WorkMCPCI, 1). Pin that NATSDispatcher.Start asks for (WorkAgentRun, maxConcurrent) for several limits. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor: remove helpers the branch left without a caller SubjectJobCancelWildcard lost its last subscriber when the frontend stopped listening on jobs.*.cancel; the NATS permissions and conformance suite spell the subject out, so nothing reads the constant. decodeBackendStopRequest returned a stopAll flag that production dropped and only a test read. decodeBackendStop is now the single decoder with the same semantics: an empty body is stop-all, an empty Backend is stop-all, malformed JSON is an error. stopBackends still derives stop-all from Backend, so no reply changes. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(messaging): keep an explicitly empty agent queue a plain subscription Before the work queue seam the agent worker passed LOCALAI_AGENT_QUEUE straight to QueueSubscribe, so an explicitly empty value made a plain subscription and every agent worker ran every agent run. WithAgentRunRoute replaced an empty queue with agent-workers, which silently changed that. Keep the queue as given once the option is applied. An empty subject still falls back to agent.execute, since it never had a meaning of its own. The flag default stays agent-workers, so only an explicitly empty value reaches this. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: correct comments and record the PR B notes Fix the recordingFactory comment (it also records the parallel flag), document that a negative maxInFlight is unbounded and that Unsubscribe from a handler deadlocks, and say a permanently undecodable payload returns nil. Record controlHandler's undecodable return as a kept exception, and add the second carrier notes to the developer note: the reconciler has no ClientFactory option, the logs proxy honours HTTP_PROXY, verbs one carrier serves need an opt-out, terminal replies come from the result event, and agent runs publish through the NATS-bound EventBridge, which is not an additive change. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
9eb5a9e61d |
feat(audio): remember speakers from diarization (#12414)
* feat(schema): validate portable speaker profiles Add the versioned profile schema for explicit speaker enrollment. Validate compatibility against separately supplied loaded-encoder metadata. Reject unusable speakers, invalid vectors, and inconsistent clean spans. This slice does not change HTTP routes, backend integration, or the UI. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(parakeet): export profiles with transcripts Export opt-in speaker profiles and trusted encoder metadata. Replay registrations by ID so duplicate display names keep independent vectors. Use one profile-capable diarization for slots, names, and clean spans. Assign timestamped ASR words to those slots without a second diarization. Preserve legacy opt-out and no-ASR behavior, and propagate failures. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(audio): enroll portable speaker profiles Gate profile exports with voice-recognition permission and validate registration against metadata from the loaded encoder. Preserve audio enrollment and independent registrations with duplicate display names. Exclude diarization and registration exchanges before API trace capture so persisted traces cannot retain profile vectors or JSON audio. Defer candidate dimensions to trusted loaded metadata. Sort candidates by registration ID so incompatible profiles cannot suppress legacy voices through registry iteration order. Keep portable identity checks closed when trusted metadata is unavailable. Test persisted traces, explicit slot zero, and selection through offline and live transport. Document privacy and the ephemeral registry lifecycle. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): remember speakers from diarization Add a Studio page for diarization and opt-in speaker profiles. Preview clean intervals from the original recording before explicit registration. Join profiles by raw speaker labels, preserve duplicate names, and relabel turns only after a successful save. Discard stale results when the model or recording changes. Share registration metadata with voice management without storing vectors or recordings from this flow. Document permissions and the global, ephemeral registry. Cover enrollment, permissions, previews, and asynchronous races with mocked Playwright tests. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: clarify HTTP speaker enrollment support Replace the stale enrollment limitation with the current HTTP workflow. Distinguish native transport from explicit registration and link its docs. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(parakeet): pin merged speaker profile support Use the merged commit from mudler/parakeet.cpp#80. Its tree matches the previously accepted native pin. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add diarization enrollment setup example Connect the existing gallery modes to the speaker enrollment workflow. Show installation, private profile export, explicit raw-slot registration, and later recognition without another export. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(blog): explain diarization speaker profiles Put the diarization walkthrough on the LocalAI website in the feature PR. Cover the three gallery modes, explicit enrollment, and privacy limits. Link setup instructions and keep availability conditional on feature support. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(blog): focus diarization on everyday use Explain what users can do with recordings before the setup steps. Replace the technical walkthrough with a short Studio guide and link readers to the existing reference for model names and developer use. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(blog): lead with speaker capabilities Present speaker recognition through everyday uses and a short UI flow. Keep technical reference details in the existing documentation. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(diarization): satisfy Go lint checks Avoid copying protobuf message state when extending backend status, check the multipart reader close result, and document the focused testing.T lint exemptions. Assisted-by: nib:gpt-5.6-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |