mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 17:18:49 -04:00
02a707558e410ee3394be527e708aaadee64213e
1373
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ettore Di Giacinto
|
02a707558e |
feat(addons): ship Studio and Talk as add-ons instead of built-in pages
Studio and Talk were compiled into the web bundle and always listed in the Add-ons drawer. They are add-ons now, like Voice, Face, fine-tuning and quantization: a person installs and removes them, the drawer lists them only while they are installed, and they are packaged the same way. Move the pages, their components, hooks and helpers into addons/studio and addons/talk. Each has a manifest, a UI bundle, a release platform matrix and a catalog template, and is built by the compat mode of the shared builder. The core API stays where it is: the pages call it with the signed-in person's session. The history keeps its browser storage keys. The unit tests of the moved helpers move with them. In the core, drop the two pages, the Create group of the sidebar (Chat joins Home and Models in the old layout), the built-in catalog entries (the catalog is empty) and the talk locale namespace. The old addresses (/app/studio, /app/talk and the generator links) redirect to the add-on pages, with the model as ?model=. The Home /studio command is offered only while the Studio add-on is installed and opens it. The release workflow and its tests now list seven add-ons. Assisted-by: Claude Code:claude-sonnet-5-5 go node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
c6bb76939b |
feat(addons): let add-on pages keep the host modules they were written against
A page written for the web UI imports React, the router, i18next and a few shared components. The add-on builder refuses a bundle that holds a host file, so moving such a page into an add-on meant rewriting it. Add a compat mode to the shared builder. It answers those imports from a directory of stand-ins that read the SDK, bundles the dynamic import of the page code into the one output file, and collects the style sheets the page imports. A relative import that is not in the add-on's own source resolves to the file at the same path under the compat directory. Widen the SDK for these pages, without a version change: more host components (spinner, modal, lightbox, model picker) and hooks (models, resources, voice profiles, operations, MCP client), a style on icons, the icon names the pages draw, sdk.navigation.openApp to open a page of the app (only addresses under /app, and the way back as the only state) and sdk.portal. The shell passes the query string to the page. The stand-ins cover the router (the tab in the path, the model in the query), the translator over the host locale files, the core API calls through sdk.core.fetch, and byte copies of the pure helpers, with tests that keep the copies equal to the originals. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
222d8e64b3 |
fix(addons): keep the loopback owner address stable across restarts
The single-node loopback transport took a new ephemeral port at each start. The owner stores its address in its identity file and refuses to start when the address differs, so every start after the first failed with "invalid addon owner configuration". Choose the port once, keep it in the add-on state root (loopback-port, mode 0600) and reuse it. If the port is taken at a later start, stop with an error that names the port and --addons-enabled=false, because another port would break the identity check. The keys and certificate stay in memory. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
49995e4aff |
feat(addons): run the add-on owner in the normal binary
The owner was only enabled when an embedding host set the add-on config, so a normal run answered every add-on request as unavailable. Add --addons-enabled (default true) and --addons-ownership-mode (default named-user, or local-owner; anything else stops startup). The node is its own owner: state lives under the data path, and the carrier uses loopback keys made at each start and kept in memory. --addons-enabled=false turns the owner off completely. Named-user mode needs the account database. With authentication off the owner stays off and a warning says how to use local-owner. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
01a3ce41ff |
fix(ui): show the server's reason when an add-on request is refused
A 403 or 400 answer showed a generic message and hid the reason in the response body, so a refusal by the browser guard could not be told from a missing permission. Show error.message from the body as the server sent it, and keep the generic text when the body has none. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
66e5407c88 |
fix(addons): accept same-host browser requests that carry no fetch metadata
Browsers do not send Sec-Fetch-* headers over plain HTTP on an address that is not localhost, so every add-on request from a LAN address was refused as cross-site. Keep refusing a Sec-Fetch-Site value other than same-origin. When the header is absent, accept the request only if its Origin, or failing that its Referer, names the same host and port as the request. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
e5710a5af0 |
feat(ui): move settings under the operate hub
The Operate hub already has Settings as a tab, so the sidebar no longer shows a separate Settings entry. The /app/settings route stays, and the hub tab still reaches it. The navigation test now expects Chat, Models and Add-ons as the primary items. Assisted-by: Claude Code:claude-sonnet-5-5 node Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
6e34303348 |
fix(addons): kill an add-on only after three missed health probes
One slow 100 ms probe killed a busy add-on and dropped the work it was serving. Require three misses in a row. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
bab102e630 |
fix(addons): release the manager lock while a release is fetched
Submit held the state lock across the catalog lookup and the package download, so the apps list and status reads stalled for the whole install. Serialize Submit and Reconcile with their own lock and drop the state lock around the fetch only. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
faa74b7a92 |
fix(addons): refuse a reinstall that would migrate retained data unbacked
An upgrade takes a verified backup before the new package migrates the data. A reinstall over retained data has no running generation to stop, so the new package would migrate it without one. Refuse a reinstall whose data format differs from the retained one. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
2de5ce7cae |
fix(addons): keep API keys off recovery mutations and same-site cookies
POST /api/addons/dispatch forwarded the recovery and update mutations to the owner for any administrator key, which skipped the browser-session rule of their own routes. Refuse those actions there. Also apply the same-origin rule to the legacy token cookie, not only the session cookie, because the global CSRF check lets same-site requests through. Do not fail the start of LocalAI when an add-on library cannot be refreshed; log it and keep the installed add-ons up. Assisted-by: Claude Code:claude-sonnet-5-5 go Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
ed6e677640 |
fix(gosec): tighten file modes and annotate false positives
Write job state, configs and imported models with mode 0600, clean the paths that are read from disk, and mark the route tables, the header name and the retry jitter as reviewed false positives. Assisted-by: Claude Code:claude-sonnet-5-5 gosec Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
1604a67c1f |
feat(addons): add the add-on runtime, navigation and first-party add-ons
Add the add-on runtime to LocalAI: manifests, the host operation broker, admin gating by account role, the add-on UI bundle builder and the SDK for add-on pages. Move the navigation to Chat, Models, Add-ons and Settings, with pinned add-ons in the sidebar and an Add-ons drawer. Ship the first-party add-ons (voice, face, fine-tuning, quantization) as packages for linux/amd64, linux/arm64 and darwin/arm64. Intel macOS (darwin/amd64) is enabled in the platform files. Squashed from the review branch. The tree is unchanged; the pre-squash history is kept on a local backup branch. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Stefan Walcz
|
6fc2bc4967 |
fix(chat-history): create the store after the auth database is opened (#12788)
The conversation routes are only registered when the chat history store exists, and the store needs the auth database. It was created in start(), which New() runs before it opens the auth database, so the store was always nil and /api/conversations answered 404 with auth enabled; the React UI fell back to localStorage and nothing was synced between devices. The route tests register the routes with a store directly and never went through New(). Create the store right after the auth database is opened, and cover it through New(): with auth the store exists, without auth or with the WebUI disabled it stays nil. Docs-Not-Needed: bug fix; the documented server-side chat history now works as described Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> |
||
|
mudler-agentandEttore Di Giacinto
|
e684fdf084 |
fix(ui): reuse Home picker for Sound expansion (#12790)
* fix(ui): reuse Home picker for Sound expansion Use the shared model chip and listbox for Sound expansion instead of a native select. Keep explicit Sound selection and Backend default independent from Home model preferences. Preserve keyboard navigation and expose the selected value through an accessible description. Hide runtime badges without loaded-model data. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(ui): select Sound expansion through shared picker Update media history coverage to use the model button and listbox. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
0d77a4023c |
fix(sound): enumerate expansion keys (#12787)
List the existing keyscale vocabulary in the expansion tool schema. Keep optional keys omitted when unspecified and retain server validation. Quote rejected keys in validation errors, with a bounded prefix, so a failed request exposes the invalid notation without other arguments. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Stefan Walczandlocalai-org-maint-bot
|
9c0266e506 |
fix(openresponses): keep server-side MCP tool items on synchronous paths (#12711)
* fix(openresponses): keep server-side MCP tool items on synchronous paths When a model runs MCP tools server-side, /v1/responses streams or builds function_call and function_call_output items for each call, but the two synchronous paths dropped them from the final response: - non-streaming: the output items are local to each MCP iteration, which recurses, so only the last iteration's items were returned and stored. - streaming: response.completed and the stored response were rebuilt from the last iteration (reasoning, message, unexecuted tool calls) only. The background paths already keep them. Without the trail, a follow-up with previous_response_id, or a client that rebuilds history from the completed output, never shows the model which tools ran and what they returned. Measured with gemma4-26b behind a model with mcp.auto: on 12 follow-up questions that need fresh tool data, the model called the tool 0/12 times when the history carried only text (and invented numbers), and 12/12 times when the earlier tool calls and results were included. The non-streaming handler now passes the items of earlier iterations into the recursive call; the streaming handler collects each finished MCP iteration's reasoning, function_call and function_call_output items and puts them in front of the final items in response.completed and in the store. Requests without server-side tools are unchanged. No new state: the items travel in the already replicated stored response. Tests: an in-memory MCP server behind the real LocalToolExecutor and a stubbed backend. For both synchronous paths the completed and the stored output are function_call, function_call_output, message (with reasoning in front when the model reasoned before the call), and a follow-up through ResponsesEndpoint with previous_response_id hands the backend the tool call and its result. Counter-checks: requests without tools still return a single message, and both background paths are pinned as they were. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(responses): preserve MCP trail associations Keep tool result call IDs when replaying stored responses. Clear consumed streaming state so the iteration limit cannot duplicate the final round. Add regression coverage for replay and iteration exhaustion. Assisted-by: Codex:GPT-6 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> --------- Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> |
||
|
Stefan Walcz
|
95034f1147 |
feat(ui): offer model MCP servers to non-admin chat users (#12660)
The chat page derived MCP availability from GET
/api/models/config-json/:name, which is admin-only. With auth enabled a
regular user holding the `mcp` feature (default ON) therefore never saw
the MCP Servers/Prompts/Resources tabs, although every endpoint behind
them (/v1/mcp/servers|prompts|resources/:model) is gated only by that
feature.
/api/models/capabilities, which the chat already loads for its model
picker and which honours the per-user model allowlist, now reports
`mcp: true` for models whose config declares remote or stdio MCP servers
(never when MCP is disabled on the instance). For non-admins the chat
combines that flag with hasFeature('mcp') and clears a stale MCP
selection otherwise. Admins keep the config-json path unchanged; the
model info panel and automatic context size stay admin-only on purpose,
since they need the full model config.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
|
||
|
Stefan Walcz
|
b56b903c92 |
feat(config): let model aliases carry an opt-in profile block (system prompt, MCP, client tools) (#12661)
* feat(config): let model aliases carry an opt-in profile block An alias can now act as a profile of its target through an explicit `profile:` block carrying system_prompt, mcp (remote/stdio servers, plus the new mcp.auto) and the new client_tools. ResolveAlias applies only that block on top of a copy of the target, while Name, Model, Backend and the persisted revision stay the target's. The loader key is therefore unchanged and every profile runs on the target's backend process; today a persona needs a separate model config, which loads the weights a second time. The block is opt-in so existing aliases never change: top-level fields on an alias stay ignored, and an alias without a profile block resolves to exactly the target. A profile block on a non-alias config is rejected. A profile with its own MCP servers caches its sessions under the alias name (MCPSessionKey), because the cache slot keyed on the target's name is invalidated whenever its config hash changes and two server sets would tear each other's sessions down. mcp.auto enables all configured servers on /v1/chat/completions when the request does not name any in metadata.mcp_servers, which plain OpenAI clients cannot send. client_tools: drop discards client tools and tool_choice before the request is merged (chat completions, responses, Anthropic messages), so only the model's own MCP tools are offered. Both keys work on any model and inside an alias's profile block. Refs #12548 Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(openai): only auto-enable model MCP when MCP is enabled and permitted mcp.auto turns a plain /v1/chat/completions request into a server-side MCP request. It now applies the same controls as the dedicated MCP routes before any server is selected: the instance switch (LOCALAI_DISABLE_MCP) and, with auth on, the caller's "mcp" feature, checked with auth.HasFeatureAccess against the auth database that the chat route now passes to ChatEndpoint. A denied request is answered as a plain chat, without contacting a server or adding tools, rather than rejected: the client did not ask for MCP, it only reached a model that offers it. Endpoint specs drive ChatEndpoint against a counting MCP fixture server: MCP disabled and a user with chat but without mcp see no tools and no connection; MCP enabled and a user with mcp get the fixture tool. The user specs need the auth build tag (SQLite auth database). Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(mcp): resolve profile aliases in the MCP discovery endpoints GET /v1/mcp/servers/:model, /v1/mcp/prompts/:model and /v1/mcp/resources/:model looked the model up with GetModelConfig, which returns the alias stub. A profile alias carries its servers in its profile block, so the stub has none of its own and the endpoints reported an empty list for an alias whose chat requests do use tools. Resolve the alias the way the request middleware does before reading the MCP config, and open the sessions under MCPSessionKey so discovery shares the profile's session cache slot instead of tearing down the target's. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * feat(config): opt-in evidence check for profile aliases A profile with server-side MCP tools (profile.mcp, mcp.auto) runs its tools on the server and returns only the answer text. On the next turn the client sends the history back without those tool calls and results, and the model then tends to skip the tool and answer from nothing. In a measurement on a follow-up question, 0 of 12 turns called the tool and the answers carried invented numbers (e.g. "0" where the true value was 9); with the tool trail kept in the history the tool was called 12 of 12 times. A prompt instruction does not reliably prevent this, so the check sits at the output. New profile keys: profile: evidence_check: numbers: true # enables the check note: "..." # optional, defaults to an English note Rule, per request and stateless: when the check is on, no MCP tool call succeeded on the server during this request (a call that returned an error delivers no evidence and does not count), and the final answer contains a number (\b\d+([.,]\d+)?\b) that occurs in none of the request's input (the profile's system prompt and every message as received, before compression), the note is appended after a blank line. "3,5" and "3.5" count as the same number. In a streamed response the note is one extra content delta before the finish chunk. Choices that hand tool calls to the client are not checked. The setting is only read from a profile block, so plain models and aliases behave as before; a note without numbers: true is rejected at validation. Tests (Ginkgo): the number rule (invented number, number from the input, no numbers, comma/dot decimals, system prompt as evidence, tool ran), the overlay into the resolved config with plain alias, target and a top-level key as counter-checks, validation, and ChatEndpoint specs on a stubbed backend for both paths: note on an invented number, none for a number from the request, none for a profile without the check, none after a successful MCP tool call against an in-process MCP server, and the note again after a failed tool call (arguments that are not JSON). Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> --------- Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> |
||
|
Stefan Walczandlocalai-org-maint-bot
|
d768f1f973 |
feat(pii): apply PII middleware to the Open Responses API (#12708)
* feat(pii): apply PII middleware to the Open Responses API
The per-model PII filter ran on /v1/chat/completions, /v1/completions,
/v1/messages and the Ollama routes, but not on POST /v1/responses. A
model whose policy blocks or masks a secret on the chat route served the
same content unfiltered when it was sent through the Responses API.
Add a piiadapter.OpenResponses adapter and wire pii.RequestMiddleware
into the POST /v1/responses and /responses chain, after the model config
is resolved and merged and before the handler converts input items into
prompt messages. The adapter covers instructions, input as a string, and
in an input item array the text of message and reasoning items (string
content or input_text/output_text parts, including typeless
{role, content} items), function_call arguments and function_call_output
output; structured tool output is scanned as the JSON the handler
forwards. Image, file, audio and video parts are left untouched.
The Responses WebSocket mode (GET /v1/responses) is not HTTP middleware
and cannot filter, so a response.create for a model with PII filtering
enabled is now refused with pii_unsupported_transport instead of being
served unfiltered. The docs list the new route and this limit.
Tests: adapter specs for every covered field, each with a clean-text
counter-check; a route spec driving the real middleware chain with an
in-process pattern detector (block on string input, content part,
instructions and tool output; mask; clean input passes unchanged); and
WebSocket specs proving the refusal and that models without a PII
policy are still served.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
* fix(pii): scan complete Responses tool output arrays
The Responses handler forwards non-string tool output as JSON. Scan
that full encoding so string arrays and extra fields cannot bypass PII
filtering. Cover array scans, redaction, clean values and route blocking.
Document that stored history is resolved after request filtering.
Assisted-by: Codex:gpt-6
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
* feat(pii): report unscanned media on the Open Responses route
Extend block_unscanned_media to /v1/responses: input_image, input_file
and input_audio parts inside message items, and the same types as bare
input items, are reported as unscanned content. Tool output stays
allowed, as it is scanned as JSON text.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
---------
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
|
||
|
mudler-agentandEttore Di Giacinto
|
0de4a7e0d9 |
fix(ui): restore per-node installed backend removal (#12785)
* fix(ci): capture and surface ginkgo errors in run-coverage.sh
The coverage script ran ginkgo per test root with `|| fail=1`, which
swallowed the exit code. Combined with `make --output-sync=target`,
which buffers all output and flushes at the end, the actual ginkgo
error was invisible in interleaved CI logs. Every test suite passed
but the script still exited 1, and the error message was buried in
22 000 lines of buffered output.
Each ginkgo invocation now captures combined stdout+stderr to a
per-root log file. On failure the last 80 lines are printed to
stderr so the error is visible in the CI log. On success the log is
replayed to stdout so test output remains visible.
The build flags (--tags, --coverpkg) are quoted correctly so values
containing spaces (e.g. --tags="debug auth") are preserved through
eval.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(ci): create output dir before writing ginkgo logs
The previous commit added per-root log files but forgot to create
the output directory before the first ginkgo invocation. On CI the
coverage/ dir did not exist yet, so the shell redirect to
cover-pkg.out.log failed with "No such file or directory" before
ginkgo even ran.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* test(core/http): add canvas preview route to auth coverage allowlist
Commit
|
||
|
mudler-agentandEttore Di Giacinto
|
2f4eaf1fad |
fix(sound): require tool calls for expansion (#12783)
Require one expand_sound call instead of parsing assistant prose. Reject unsupported templates and invalid arguments before audio generation. Keep installed configuration unchanged and preserve explicit controls. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
dce91ba7aa |
fix(backends): prefer installed development builds (#12782)
* fix(ci): capture and surface ginkgo errors in run-coverage.sh The coverage script ran ginkgo per test root with `|| fail=1`, which swallowed the exit code. Combined with `make --output-sync=target`, which buffers all output and flushes at the end, the actual ginkgo error was invisible in interleaved CI logs. Every test suite passed but the script still exited 1, and the error message was buried in 22 000 lines of buffered output. Each ginkgo invocation now captures combined stdout+stderr to a per-root log file. On failure the last 80 lines are printed to stderr so the error is visible in the CI log. On success the log is replayed to stdout so test output remains visible. The build flags (--tags, --coverpkg) are quoted correctly so values containing spaces (e.g. --tags="debug auth") are preserved through eval. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ci): create output dir before writing ginkgo logs The previous commit added per-root log files but forgot to create the output directory before the first ginkgo invocation. On CI the coverage/ dir did not exist yet, so the shell redirect to cover-pkg.out.log failed with "No such file or directory" before ginkgo even ran. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(core/http): add canvas preview route to auth coverage allowlist Commit |
||
|
|
36307005a9 |
feat(chat-history): server-side chat history with tombstones (follow-up to #9902) (#12663)
* feat(schema): add Conversation schema for chat history (#9432) Persist WebUI chat conversations server-side so browser refresh, private windows, or device changes preserve user history (issue #9432). This commit adds the on-disk representation: - Conversation holds id, name, model, opaque history, MCP / sampling settings, and timestamps. - ConversationsFile is the per-user list serialised to JSON. History is stored as json.RawMessage so the server stays agnostic to React message shapes (user / assistant / thinking / tool_call / tool_result mixed with text / image_url / audio_url / file attachments). Assisted-by: Claude:claude-opus-4-7 Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * feat(chathistory): add file-based per-user conversation store (#9432) File-backed persister for chat history. Each user's conversations live under {baseDir}/{userID}/conversations.json (anonymous/ when auth is disabled). Design choices: - In-memory cache backed by sync.Mutex so concurrent saves don't interleave writes. - Atomic write via tmp file + os.Rename so a crash mid-save never leaves a corrupted history. - ID validation regex blocks path-traversal payloads at the store boundary, in addition to the auth context constraint. Tests cover round-trip persistence across instances, user isolation, unsafe-ID rejection, bulk replace migration, and the anonymous fallback path. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * feat(api): expose chat conversation CRUD endpoints (#9432) Wire the chathistory store into the HTTP layer. New endpoints under /api/conversations: - GET /api/conversations list - POST /api/conversations upsert (id in body) - DELETE /api/conversations delete all - PUT /api/conversations/bulk replace entire set (localStorage migration) - GET /api/conversations/:id fetch one - PUT /api/conversations/:id upsert (id in path; path id wins over body) - DELETE /api/conversations/:id delete one All endpoints scope queries to the authenticated user via getUserID(c) - callers cannot impersonate other users by passing a user_id in the body. The endpoints are gated behind the new chat_history feature permission (default ON in APIFeatures), registered through the existing RouteFeatureRegistry so the unified feature middleware picks them up automatically. Application.ChatHistoryStore() returns nil when DisableWebUI is set or no persistence path is configured, in which case route registration is skipped entirely rather than registering handlers that always 503. Also adds the chat-history instruction entry and Swagger tag so the endpoint surfaces in /api/instructions and /swagger. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * feat(react-ui): sync chat history to server with localStorage fallback (#9432) useChat now probes /api/conversations on mount. When the endpoint responds 200, the server becomes the authoritative source: the hook merges remote conversations into local state and pushes per-chat PUT updates on each debounced save. When the endpoint 404s - older LocalAI deploys or the feature disabled - the hook silently keeps the old localStorage-only behaviour, so the change is backward-compatible. Migration: when the server is reachable but empty and the browser has local conversations with history, the hook fires a single PUT /api/conversations/bulk to seed the server. The atomic bulk endpoint avoids partial-upload states where a retry would skip already-uploaded entries. Delete operations propagate to the server when it's available; failures are silently swallowed so the local UI stays responsive on transient network errors. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * test(chathistory): rewrite tests in Ginkgo/Gomega The initial test file used the stdlib testing package (t.Fatalf / t.Errorf), which is forbidden by the forbidigo linter rule in .golangci.yml — LocalAI mandates Ginkgo/Gomega for all Go tests (see .agents/coding-style.md). Restructured the same six scenarios into Describe / Context / It blocks, with a chathistory_suite_test.go bootstrap that registers the Ginkgo fail handler. Test coverage is unchanged: basic CRUD round-trip, cross- instance persistence, user isolation, unsafe ID rejection (now via DescribeTable), bulk replace overwrite, and the anonymous fallback path. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * refactor(chathistory): switch store to GORM-backed authDB Addresses mudler's review on #9902. Chat history was the only per-user state in LocalAI sitting in a separate filesystem store instead of the shared GORM database — AgentStore and JobStore both live there. Move it onto Application.authDB so per-user state stays in one place. * ConversationRecord (GORM model) with composite primary key (user_id, conv_id) — React mints IDs locally with no global coordination so per-user uniqueness is the natural shape. * Store.New(db *gorm.DB) replaces Store.New(baseDir string). Save / Get / List / Delete / DeleteAll / ReplaceAll keep their signatures so HTTP handlers and the React UI are untouched. * ReplaceAll runs in a transaction with an Unscoped() pre-clear so a future retention sweep cannot resurrect a stale ID on re-upload. * gorm.DeletedAt in the schema means a future retention pruner is a one-line query — addresses the second half of the review (unbounded in-memory map). The map is gone: reads go straight to the DB. Trade-off worth flagging in the PR reply: chat history persistence now requires auth.Enabled = true (that's what initialises Application.authDB). With auth disabled the React UI transparently falls back to localStorage, the same path the original PR took on 404. Consistent with AgentStore / JobStore; if a "chat history always on" mode is preferred we can pull DB init out of auth.InitDB into a shared helper. Assisted-by: Claude:claude-opus-4-7 [Read Edit Bash] Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * refactor(schema): drop now-unused ConversationsFile envelope ConversationsFile was the JSON wrapper for the old file-based chat history store — a {Conversations: [...], UpdatedAt: ...} pair serialised to disk under {baseDir}/{userID}/conversations.json. The previous commit replaced that store with a GORM-backed one that writes rows directly as ConversationRecord, leaving the envelope without a consumer. Assisted-by: Claude:claude-opus-4-7 [Read Edit Bash] Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * test(chathistory): switch tests to testutil.SetupTestDB Mirror the AgentStore test setup: spin up a PostgreSQL 16 container via testutil.SetupTestDB(), construct a Store on top of it, exercise CRUD round-trips. The previous file-based fixtures (TempDir, filepath.Join checks) are no longer meaningful now that the store talks GORM. Tests are skipped on macOS (testcontainers needs Docker) and run on CI, same as the other GORM-backed suites in the repo. Also reframed two cases: * "unsafe IDs" → "malformed IDs" — the DB-backed store doesn't care about path traversal, but idRegex still rejects whitespace / control characters so IDs stay safe in logs and HTTP responses. * "anonymous user" no longer pins a directory path (no directory exists anymore); instead it checks the round-trip plus the per-user isolation guarantee. Added a DeleteAll round-trip to cover the bulk-clear endpoint. Assisted-by: Claude:claude-opus-4-7 [Read Edit Bash] Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> * test(chathistory): cover the conversation HTTP handlers The seven /api/conversations handlers had no tests. Inject the *chathistory.Store into the handlers instead of the whole Application and move the route table into RegisterChatHistoryRoutes, so a spec can drive the real registration and middleware chain (auth.Middleware plus the chat_history feature gate) against an in-memory SQLite auth database. The new specs cover 401 without credentials on every route, 403 when an admin revoked chat_history, the CRUD round trip, path id over body id, 400 on malformed ids and bodies, per-user isolation, bulk replace and delete-all, and that a nil store registers nothing (the UI then falls back to localStorage). The handlers' nil-store branches are gone: the routes are only registered with a store, so those branches could never run. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(chathistory): make deletions final with tombstones and versions A conversation deleted on one device came back as soon as a client with a stale copy wrote it again: Delete soft-deleted the row, and the next Save went through GORM's upsert, which cleared deleted_at and resurrected it for every device. The lookup that recovered CreatedAt also ran outside any transaction, so concurrent writers raced on it. The server is now authoritative over deletions: * Delete turns the row into a tombstone: content dropped, deleted_at set. It is idempotent, and deleting an id the server has not seen yet still records a tombstone, so a delete that overtakes the UI's in-flight debounced save of the same chat still wins. * Save runs in a transaction, refuses a tombstoned id with ErrDeleted (HTTP 410 Gone), and bumps the version with a single UPDATE ... SET version = version + 1 guarded by deleted_at IS NULL, so concurrent writers each count and a delete committed between the lookup and the write still wins. Two concurrent first writes of the same id retry once (ON CONFLICT DO NOTHING, then update). * Every row carries a server-assigned version (1 on the first save, +1 per write including the delete), returned to the client. A non-zero version tells the UI the server has seen a conversation, which is how it can tell a never-synced draft from a server-side deletion. * GET /api/conversations returns the tombstones under "deleted". * DeleteAll and ReplaceAll leave tombstones too. ReplaceAll no longer hard-deletes everything first; it upserts the payload, skips ids that were deleted (reported in "skipped"), and tombstones the rest. * createdAt/updatedAt are stored as the client sends them (only zero values are stamped), so clients compare updatedAt from their own clocks rather than against the server's. The store specs now run against PostgreSQL (testcontainers) and, with the auth tag, SQLite through auth.InitDB, since both back the auth database. The API specs cover a stale client reconnecting after a remote deletion. Swagger is regenerated with swag v1.16.6 (json.RawMessage fields get swaggertype), and the feature is documented, including that it needs user auth. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(react-ui): let the server decide which chats were deleted mergeRemoteAndLocal kept every chat that was only in the browser and let the server copy win for every shared id. A stale browser therefore uploaded chats deleted on another device as if they were new drafts, and a message sent while the list request was still in flight was replaced by the older server copy and then lost from localStorage too. The first load now reconciles with the server's tombstones and versions: * a chat the server lists under "deleted" is dropped locally; * a chat on both sides keeps the copy with the newer updatedAt, so an edit made during the load wins and is uploaded afterwards; * a chat only in the browser is uploaded only if it was never synced (no version). With a version the server has seen it and no longer has it, so it is dropped instead of resurrected (this also keeps one account's cached chats out of another account on a shared browser); * deletes made while the list is loading are queued and sent once it arrives. Saves record the version the server returns, and a 410 on save drops the chat locally instead of retrying it forever. Chats received from the server count as already sent, so adopting a server copy no longer echoes it straight back. The bulk localStorage migration is gone: it replaced the whole server set from a possibly stale browser, and the per-chat saves already upload never-synced drafts through the tombstone check. The Playwright spec drives each case against a stubbed /api/conversations that follows the server contract. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(chathistory): keep the LocalAI Assistant toggle on synced chats master's createNewChat sets localaiAssistant, but serializeChat did not send it and schema.Conversation had no field for it, so a chat with the Assistant mode on came back with it off after a reload from the server (and the local cache dropped it too). Add the field on both sides. The other fields createNewChat sets were already covered; a handler spec now round-trips all of them, and the UI spec checks the toggle on the way up and in the local cache. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(react-ui): scope chat cache per account, retry unconfirmed deletes Two gaps in the client half of server-side chat history, both found in review: The browser cache was one shared localStorage key. A draft that one account could not upload (a failed first save) stayed in it, and after that account signed out and another signed in, the reconcile kept the unversioned draft and the save pass uploaded it into the second account. The cache now lives under localai_chats_data:<user id>, Chat remounts its state when the account changes, and nothing is synced without a signed-in account. The shared key is left for browser-only use without auth and is not uploaded when signed in, because nothing tells which account it belongs to. Deletes were dropped locally and their DELETE errors swallowed; only deletes made while the first list was loading were queued, and not persisted. A delete made while the server was unreachable therefore came back with the next list. Deletes (single, delete all, and deletes on the Home page) are now kept as pending deletes in the account's cache until the server confirms them, override the server list in the reconcile, and are sent again on the next change, on the online event and on the next load. While a delete-all is pending no conversation is uploaded, so a chat started after it is not caught by it. The Playwright specs replay both review scenarios (account switch after a failed first save; delete and delete-all while offline) with counter-checks that the same account keeps and uploads its own draft and that a confirmed delete is neither re-sent nor resurrected. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(react-ui): persist cleared chats immediately Navigation cancels the debounced cache write and can restore unsynced chats after Clear all. Save the cleared list with its pending deletion before returning, and cover immediate navigation in the browser spec. Assisted-by: Codex:GPT-6 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * test(react-ui): make the chat history sync specs deterministic - "clear all persists before navigation" opened a single chat, and the menu offers no Clear all for one chat, so the spec timed out in every run. It now opens two unsynced chats and checks that neither survives. Reverting 03cd9c115's save in clearAllChats turns it red again. - "a delete the server confirmed is not sent again" reopened the page as soon as the server had the request, before the page had the answer; the unconfirmed delete was then rightly sent again (4 of 5 runs). It now waits until the cache holds no pending delete. Docs-Not-Needed: test-only change. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(chathistory): keep the chat's reasoning effort on the server reasoningEffort (#12700, #12716) is part of the chat the UI syncs since the master merge, but schema.Conversation had no field for it, so the server dropped it and a reload from the server reset the setting. Add the field. The handler spec that round-trips every chat field now includes reasoningEffort. A schema test reads serializeChat in useChat.js and fails for any key the UI sends that schema.Conversation cannot hold, so the next such drift shows up as a failing test rather than in review (it reports reasoningEffort on the previous head). Swagger and docs updated. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * test(chat): use Ginkgo for the schema drift guard The new drift guard uses testing methods forbidden by repository lint. Keep its field coverage and fail-fast parser checks in Ginkgo/Gomega. Assisted-by: nib:gpt-6.1-sol --------- Signed-off-by: TLoE419 <tloemizuchizu@gmail.com> Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> Co-authored-by: TLoE419 <tloemizuchizu@gmail.com> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> Co-authored-by: mudler-agent <mudler-bot@c3os.io> |
||
|
mudler-agentandEttore Di Giacinto
|
ae8092e1c6 |
feat(sound): expand Simple prompts with chat (#12772)
* feat(sound): expand simple prompts with chat models Allow callers to select an installed chat model for sound prompt expansion. Validate its JSON before audio generation and preserve explicit instrumental and language controls. Keep backend defaults when expansion is omitted, including native Python ACE-Step sampling. Assisted-by: Codex:gpt-6 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(sound): map expansion by loaded audio family Map neutral expanded fields where the resolved audio family is known. Keep unmarked requests unchanged and reject unsupported expansion before inference. Clear YuE2 and HeartMuLa transcripts to prevent lyric fallback. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(sound): choose a prompt expansion model Require an explicit Simple choice and remember it separately from chat. Clear unavailable choices instead of silently switching language models. Preserve the choice in history and restore it when editing a past result. Keep Backend default and Advanced requests compatible with existing APIs. Document family support and align the published optional request schema. Verify selection, request fields, persistence and history in the browser. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(sound): authorize both generation models Check the resolved audio model and explicit expansion model before loading either. Permission lookup failures stop generation. Assisted-by: Codex:gpt-6 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(sound): enforce JSON during prompt expansion Constrain expansion output with the existing JSON grammar before decoding. Require a llama-cpp chat model so unsupported backends cannot ignore it. Keep strict field validation and fail before generating audio. Assisted-by: Codex:gpt-6 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
f85087f992 |
feat(tts): stream Qwen text and add assistant preset (#12770)
* feat(tts): stream Qwen text through audio pipelines Feed cleaned LLM deltas into one persistent Qwen synthesis session. Return audio before input ends and preserve the live RPC through model cache, staging and in-flight wrappers. Stop generation on sink failures or cancellation without replaying speech through a fallback. Pin the incremental native fork and verify early audio, late text, finalization and cancellation with lifecycle and real-model checks. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * build(tts): use the organization Qwen fork Fetch the reviewed incremental synthesis commit from localai-org so backend builds use the organization-maintained fork. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add a Qwen streaming assistant preset Expose an English pipeline using existing VAD, ASR, LLM and named-voice TTS models. Enable incremental text and audio delivery, and document separate component installs because the gallery has no dependency installer. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): use Parakeet TDT v3 for the assistant Select the existing Parakeet TDT 0.6B v3 transcription entry and update its prerequisite install command. Keep the preset's English voice and incremental Qwen synthesis configuration. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(kokoros): reject unsupported backend RPCs Implement the MotionStream and SlotCache methods required by the shared protobuf service. Return Unimplemented so Kokoros compiles without advertising support for either operation. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(tts): mark cleanup and logging results ignored Explicitly ignore connection shutdown, test server shutdown and Ginkgo diagnostic writer errors. These operations do not replace the stream's inference or sink errors, and errcheck requires their return values to be acknowledged. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(tts): stream buffered clauses in realtime pipelines Start clause synthesis while LLM text is still arriving when both streaming flags are enabled. Keep explicit clause opt-out and native Qwen sessions, bound automatic queued work, and choose streaming support on resolved failover targets. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(tts): bound synthetic voice clone generation The sine-wave reference checks speaker conditioning and WAV output, but does not provide a meaningful speech EOS target. Limit that smoke request to 128 frames and verify the request limit reaches native synthesis. Leave production sampling defaults unchanged. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(tts): require explicit clause buffering opt-in Keep whole-reply synthesis for nonnative engines when clause_chunking is unset or false. Use the bounded cancellable worker when explicitly enabled, with unary clauses when TTS streaming is disabled. Preserve native Qwen precedence for every clause setting. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add VITS and Piper streaming assistants Provide comparable full pipeline configurations with shared VAD, Parakeet transcription, and Qwen chat components. Enable clause buffering explicitly and document the separate prerequisite model installs and Talk selection. Assisted-by: Codex:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
48f358dd95 |
fix(ci): capture and surface ginkgo errors in run-coverage.sh (#12765)
* fix(ci): capture and surface ginkgo errors in run-coverage.sh
The coverage script ran ginkgo per test root with `|| fail=1`, which
swallowed the exit code. Combined with `make --output-sync=target`,
which buffers all output and flushes at the end, the actual ginkgo
error was invisible in interleaved CI logs. Every test suite passed
but the script still exited 1, and the error message was buried in
22 000 lines of buffered output.
Each ginkgo invocation now captures combined stdout+stderr to a
per-root log file. On failure the last 80 lines are printed to
stderr so the error is visible in the CI log. On success the log is
replayed to stdout so test output remains visible.
The build flags (--tags, --coverpkg) are quoted correctly so values
containing spaces (e.g. --tags="debug auth") are preserved through
eval.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(ci): create output dir before writing ginkgo logs
The previous commit added per-root log files but forgot to create
the output directory before the first ginkgo invocation. On CI the
coverage/ dir did not exist yet, so the shell redirect to
cover-pkg.out.log failed with "No such file or directory" before
ginkgo even ran.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* test(core/http): add canvas preview route to auth coverage allowlist
Commit
|
||
|
Stefan Walcz
|
f4505dbb5d |
feat(pii): optionally block content the detectors cannot inspect (#12755)
The PII middleware scans the text of every message, but images, audio,
video and files pass to the backend unchecked. For an egress policy
("nothing sensitive leaves the box") that is a hole: a photographed
invoice or an attached PDF reaches a cloud-proxy model without any check.
Add pii.block_unscanned_media. While PII is enabled for a model and the
option is set, a request that carries such content is rejected with the
usual 400 pii_blocked response and an audit event with pattern
__unscanned_media__, before any detector runs. Adapters report the
kinds of unscanned content through a new optional Adapter.Unscanned:
OpenAI chat (non-text content blocks), Anthropic messages (image and
document blocks; tool blocks stay allowed) and Ollama chat (images).
Off by default, so existing configurations behave as before.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
|
||
|
Stefan Walcz
|
92500b4389 |
feat(ui): dictate in the chat composer and read scanned PDFs as images (#12756)
Two gaps in the chat composer: - The microphone button only opened a file dialog for audio files; there was no way to speak a message. Add a dictation button: the first click records from the microphone, the second stops and sends the recording to /v1/audio/transcriptions with the first installed transcription model; the text is appended to the message box for review. The button is hidden without a transcription model or without a microphone API. The audio attach button keeps its function and gets the music icon. - A PDF without a text layer (a scan) was rejected with a toast. Render its first ten pages to PNG with pdf.js and attach them as images, so a vision model can read them; a toast says how many pages went along. PDFs that cannot be parsed at all are still rejected. Strings for all eight locales, docs in features/chat.md, Playwright specs for both (with negative cases: no transcription model, failed transcription, unparsable PDF). Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> |
||
|
|
e715af1def |
feat(llama-cpp): add slot prompt-cache save and restore API (#12730)
* feat(llama-cpp): add slot prompt-cache save and restore API Add POST /v1/slots/save and POST /v1/slots/restore. They write one llama.cpp slot's prompt cache (KV state) to a file and load it back, so a client can warm a long system prompt in a few hundred milliseconds instead of re-running prefill. Refs #12685. The new SlotCache gRPC call posts the same slot save and restore tasks the llama-server /slots endpoint uses. It is off unless the model sets the new slot_save_path option, so existing models are not affected. The /slots monitoring endpoint is not needed here: the backend has no HTTP server. The file name comes from the caller, so it is checked twice, in Go and again in the backend. It must be one path component: no empty name, "/", "\", "..", NUL, control characters or absolute path. The resolved path, with symbolic links followed, must be a direct child of slot_save_path. The routes are admin only because they write files on the backend host. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): answer 501 when a backend lacks slot cache A backend that does not implement the SlotCache rpc returns gRPC UNIMPLEMENTED. The API passed that text through, so the client saw "unknown method SlotCache". Map the status to a typed error in core/backend and return 501 with a fixed message instead. Every other error path is unchanged. Make the SlotCache comments in backend.proto backend-neutral: slot is the backend's state slot, filename must stay inside the save directory, and an empty model_identity means the backend does not check it. Only comments change, no field or name. Add tests that the identity sent is the configured model, and for the error mapping. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): issue slot cache IDs on the server The slot save and restore API let the client pick the file name. Restore could load any file in slot_save_path, and the symlink check ran before llama.cpp opened the file. The backend now issues the name. A save returns a random 128-bit ID (32 lowercase hex characters, read from /dev/urandom) and never overwrites an existing file. Files live in a 0700 directory per model, named from the SHA-256 of the model identity, with a manifest.json that records each save. The manifest is written atomically and a corrupt one stops both save and restore. A restore takes the ID, finds it in the manifest, refuses an entry of another model, builds the path from the entry and checks with lstat that it is a plain file of the recorded size. The directory mode is the main defence against another local user, since llama.cpp still opens the path itself. Each model keeps at most 32 saves and the oldest is deleted first. A file over 8 GiB is deleted after the save, because llama.cpp does not report the size before it writes. Responses carry no host path. The HTTP bodies change: save takes {model, slot}, restore takes {model, slot, id}, and an unknown ID returns 404. In the proto, SlotCacheRequest.filename and SlotCacheResponse.filename become id. SlotCache no longer calls checkModelIdentity, which reads a field this request does not have. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): copy slot_cache files in prepare.sh grpc-server.cpp includes slot_cache.h, but prepare.sh copies the helper headers into llama.cpp/tools/grpc-server/ one by one and skipped it. The llama-cpp, turboquant and bonsai builds share this step, so they all failed with "slot_cache.h: No such file or directory". Copy slot_cache.h and slot_cache_test.cpp, which CMakeLists.txt builds when LLAMA_GRPC_BUILD_TESTS is on. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(deps): bump LocalAGI to pick up the scheduler start/stop fix LocalAGI's scheduler could hit a nil pointer in run when Start and Stop raced. That made the core/services/agentpool tests panic at random and turned tests-linux red. Move to mudler/LocalAGI@7aa0c8fea0, the merge of mudler/LocalAGI#519 ("make Start and Stop safe to call concurrently"). The new module version also adds a few indirect requirements to go.mod and go.sum. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com> |
||
|
mudler-agentandEttore Di Giacinto
|
9ad78381d8 |
[ui] Serve canvas HTML previews with their own CSP (#12715)
fix(ui): serve canvas HTML previews from a route with their own CSP External scripts in canvas previews (for example three.js from unpkg) were blocked. The preview used an iframe with srcdoc, which inherits the global Content-Security-Policy, and a meta tag can only tighten it. The UI now posts the HTML to POST /api/canvas/preview, gets a random id, and loads GET /api/canvas/preview/:id in the iframe. That response sets its own CSP that allows scripts from unpkg, jsdelivr, cdnjs, the Tailwind CDN and jQuery. The global CSP is unchanged. The iframe keeps sandbox="allow-scripts" without allow-same-origin. Previews are kept in memory for 10 minutes and are never written to disk. The GET route is public because the id is the capability and a frame navigation cannot send API key headers. Assisted-by: Claude:claude-haiku-5-5 [go test] [npm] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
39b3b31153 |
fix(backends): flag a llama-cpp backend that predates native decisions (#12733)
A decision request (/v1/systemone, /v1/decisions) sends Score with question_type "systemone". A backend built before native decisions ignores that field and runs plain scoring, so the caller saw "Score was not enabled" (500) or "candidates must be non-empty" (400). The gallery "latest" image can lag behind the server, and the upgrade check compares only tag, version and digest, so /backends/upgrades did not list the backend. Recognise those two errors for a decision request, return 501 with a message that says the backend is older than the server and must be upgraded, and mark the backend stale. The upgrade check now reports a marked backend with a `reason`, and refreshes at once. Reinstalling the backend drops the mark; if the new image is still old, the next request marks it again. Model loading is not affected. Refreshing the "latest" backend image is a separate publishing step and is not part of this change. Refs #12693 Assisted-by: Claude Code:claude-sonnet-5-5 [go vet] [go test] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
github-actions[bot]andmudler
|
fdfb171c3e |
chore: bump inference defaults from unsloth (#12732)
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
mudler-agentandEttore Di Giacinto
|
1d0a3227d7 |
[ui] Fold the No reasoning chip into the effort picker (#12716)
fix(ui): fold the "No reasoning" chip into the effort picker The chat composer had two controls for one idea: the effort picker with a "None" level, and a separate "No reasoning" chip. The chip also hid reasoning traces that a model sends despite reasoning_effort "none", which "None" did not. The chip disabled the picker while it was on. Choosing "None" now hides those traces and stays selectable, so the chip, its toasts and the /reasoning slash action are removed. A chat saved with the old reasoningOff flag loads with effort "none", so it keeps hiding reasoning. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
fc6779d41e |
feat(ui): add reasoning effort and reasoning off controls to chat (#12700)
The chat could not set reasoning_effort or turn reasoning traces off. Add an effort menu and a "No reasoning" toggle to the composer chips, and the matching /effort and /reasoning slash actions. Both are stored per chat. The effort menu defaults to the model default, which sends no reasoning_effort, so existing chats behave as before. Chosen effort goes out as the per-request reasoning_effort field. The toggle sends "none", which the server maps to enable_thinking=false, and also drops any reasoning the model still returns, so the thread shows only the answer. Assisted-by: Claude:claude-sonnet-5-5 eslint playwright Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
c7ce0c602c |
fix: stop listing the models directory on request paths and cache GGUF header reads (#12683)
* fix(http): stop listing the models directory on embeddings and bearer requests Every embeddings request read the whole models directory before the body was parsed, even when the body named its model. On a network-backed models directory this cost seconds per request. Embeddings now binds the body first and looks up a default model only when the body has none, like chat. The pre-bind default lookup and the gpt-4o constant default are gone from that route. The first-available default lookup skips the directory when the filter cannot accept an unconfigured file. Every usecase filter rejects files without a config, so the scan never added results for them. Match-all, nil and name filters keep the scan. The bearer token check is now an exact test: a configured name, or one stat of that name in the models directory. The value is a literal, not a regular expression, and a name that is not a local relative path never matches a file. Assisted-by: Claude Code:claude-haiku-5-5 * perf(config): cache GGUF header reads for model defaults Model defaults re-parsed the GGUF header on every SetDefaults call, which runs per request and at startup reload. Put a bounded LRU in front of the parser, keyed by absolute path and validated by size and modification time. The cached value is a snapshot of the fields the defaults use, not the parser output. A changed size or mtime is a miss. Missing files and parse errors are not cached, so the existing fallback is unchanged. LOCALAI_DISABLE_GUESSING, the empty-path guard, the alias early return, the panic recovery and the context-size default are all kept in front of the cache. Assisted-by: Claude:claude-haiku-5-5 * test(http): cover listing-free default lookup on every usecase route Assisted-by: Claude Code:claude-haiku-5-5 * feat(config): persist the GGUF header cache per instance The header cache kept parsed GGUF results only in memory, so every restart parsed each model file again before its defaults were known. Write the cache to a per-instance file in the data path and load it at startup. A restarted frontend with a fixed instance ID starts warm. The file is versioned and records the estimator inputs (gguf-parser version, candidate contexts, auto context size, offload count). A mismatch on any of them makes the file ignored. Each entry keeps path, size and mtime, so a changed model file is a miss on load. Writes run on one background saver, debounced, through a temp file and rename. The entry bound applies to the file as well. Without a data path the cache stays in memory only. Documented as a per-instance exception in the model configuration docs. Assisted-by: Claude:claude-haiku-5-5 * docs(embeddings): document model selection and the no-model error Assisted-by: Claude Code:claude-haiku-5-5 * refactor(config): split the header cache into a generic cache The GGUF header cache mixed two things: the mechanics (bounded LRU keyed by path and identity, per-instance file, debounced atomic save, tolerant load) and the GGUF fields. Move the mechanics into headerCache[S], driven by a headerCacheSpec[S] that names the file prefix, schema, fingerprint, parser and the snapshot encode/decode pair. The GGUF cache becomes one instance of it. File names, the identity check, LRU size, debounce, atomic write and the corruption handling are unchanged. The on-disk JSON keeps the "estimator" key so existing files still load. Assisted-by: Claude Code:claude-haiku-5-5 * feat(config): share one GGUF header cache file between frontends Every frontend used to write its own header cache file, named after its instance ID, so a header read by one frontend was never reused by another. The cache now uses one file, header-cache.json, in the data path, for all frontends and for standalone mode. The instance ID no longer changes the file name. Frontends share the file in three ways. A save first merges the file's entries in, so one frontend's write does not drop another's entries. A miss reloads the file when its modification time has changed, and a hit never reads it. Writes go to a temp file that is renamed into place, so a reader never sees a partial file. Two frontends saving at the same moment can still drop each other's last entries. That costs one re-parse, never a wrong value, because every entry is checked against its model file before use. The shared file is valid only when every frontend mounts the same data volume. The distributed-mode and model-configuration docs state this mode and its volume limit. Assisted-by: Claude:claude-haiku-5-5 * feat(config): give an instance ID its own header cache file Assisted-by: Claude Code:claude-haiku-5-5 * style(config): gofmt the distributed config struct Assisted-by: Claude Code:claude-haiku-5-5 --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-botandlocalai-org-maint-bot
|
f02d5244da |
fix(agents): warn about unowned data with auth (#12669)
Warn before the standalone pool starts root agents with user auth enabled. Count root agents, collections, skills, and configured skill repositories. Document the auth transition, bootstrap rules, and agent key behavior. Addresses the warning and documentation step of #12665. Admin visibility and data migration remain separate work. Assisted-by: Codex:GPT-6 Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> |
||
|
mudler-agentandEttore Di Giacinto
|
e28d5009d9 |
stablediffusion-ggml: run MiniMax-H3 video with references and add its gallery entries (#12668)
* feat(stablediffusion-ggml): run MiniMax-H3 video with reference conditioning Bump stable-diffusion.cpp to 2988060 and follow its tiling rename (tile_size_x/y -> tile_size_w/h, rel_size_x/y -> rel_size_w/h). GenerateVideo now passes params["ref_image"] and the request audio to the MiniMax-H3 Ref2VA inputs (ref_images, ref_audios). The audio must be a WAV file. params["ref_video"] is refused, since the proto cannot carry a frame sequence yet. stable-diffusion.cpp cannot tell the FL2VA and REF2VA DiTs apart, and a reference input on FL2VA renders a wrong video without an error. The backend now reads the video_partition option, or the model file name, and fails early on that mix. Keyframes plus references are refused too. Add gallery entries for the FL2VA and REF2VA Q4_K_M sets. They do not enable any bf16 dequantisation. Assisted-by: Claude:claude-sonnet-5-5 go vet gofmt * feat(stablediffusion-ggml): take several reference images, videos and audio clips Phase 1 passed one reference image and one reference audio clip to stable-diffusion.cpp and refused ref_video. Add repeated reference fields to GenerateVideoRequest (ref_images, ref_videos, ref_audios) so the MiniMax-H3 Ref2VA path can use as many references as upstream accepts. A reference video is a directory of frames with an fps and an optional paired audio clip. The /video endpoint stages the new inputs through the same path as start_image, end_image and audio. Frames of one reference video are written to a private directory under zero-padded names so the backend can read them in order. The input count and frame count per request are bounded. The backend merges the new fields with params.ref_image, params.ref_video and audio, which keep working. The FL2VA partition refuses any reference input, REF2VA accepts all of them, and keyframes stay exclusive with references. Reference audio is still WAV only. Assisted-by: Claude:claude-sonnet-5-5 [Bash] * feat(gallery): offer MiniMax-H3 by partition, DiT layout and format Name each MiniMax-H3 entry after its partition (fl2va or ref2va), DiT layout (unpruned or pruned) and format. Add the vllm.cpp builds that its model page lists: pruned Q8_0 (renders), pruned Q6_K, Q5_0, Q4_K, Q3_K and Q2_K (load only), and the pruned FL2VA NVFP4 file (loads only). The earlier names minimax-h3-fl2va-q4 and minimax-h3-ref2va-q4 stay as aliases. The two stable-diffusion.cpp entries become minimax-h3-fl2va-unpruned-q4-ggml and minimax-h3-ref2va-pruned-q4-ggml, and the latter now lists video input. Fix two bad sources. The stable-diffusion.cpp entries took the VAEs from a path that does not exist in the GGUF repository; they now use the Comfy-Org copies. The vllm.cpp entries paired the fp16 video VAE weights with a config from a different directory; they now use the official weights and config from FL2VA/video_vae/source/. Drop video_dequant_bf16 from every vllm.cpp entry. Expanding a quantised DiT to bf16 is the suspected cause of a host out-of-memory error, so the quantised weights stay resident. The model page records its renders with the expansion on, and each description says so. Assisted-by: Claude:claude-sonnet-5-5 [Bash] * feat(gallery): add pruned FL2VA Q4 MiniMax-H3 for sd.cpp Add minimax-h3-fl2va-pruned-q4-ggml, the 11.4 GB pruned FL2VA counterpart of the existing pruned REF2VA entry. Assisted-by: Claude Code:claude-sonnet-5-5 [rc] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
008f9fdf81 |
fix(realtime): end a client response cleanly when its commit fails
A client response.create waits for the in-flight commit before the LLM starts. Two exits left the client without a usable outcome: - When the wait was cancelled (response.cancel, a newer response or teardown) the response returned with no event at all. - When the commit produced no user item (transcription error, empty transcript, voice gate rejection) the LLM ran on a history that lacked the turn. Send response.created and response.done with status cancelled for the first case. For the second, do not run the LLM: send response.done with status failed and an input_not_committed error. The commit still reports its own cause. A response.create with no commit in flight is unchanged. Add specs for STT error, empty transcript, cancel and supersede during the wait. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
3a9a67aeb9 |
refactor(realtime): gate client responses on the commit tail
A client response.create waited on a separate mutex-guarded channel that only the last client commit recorded, so it ignored in-flight VAD commits. The commit chain already ends in session.commitTail, whose done channel closes after every earlier commit has finished. Read the tail under commitOrderMu when the response is issued and wait on it. This drops the extra mutex, the clientCommitDone field and the slotSink callback of issueCommit, and the response is now ordered against both commit producers. Fix the commitSlot.done comment: it closes when the commit function returns, not when the user item is appended. Add a spec for commit, commit, response ordering. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-bot
|
2aa8821876 |
test(realtime): sample pending response events
Poll the event count during the held transcription instead of checking\na captured value. Document the client commit and response ordering.
Assisted-by: Codex:gpt-6
Signed-off-by: nexxtmobile.de <kai@nexxtmobile.de>
(cherry picked from commit
|
||
|
kai
|
c27848c8c7 |
fix(realtime): wait for the client commit's STT before the response LLM starts
With client-driven turns (turn_detection null, e.g. a client VAD / speaker
gate) the client sends input_audio_buffer.commit and response.create
back-to-back. The response.create triggered the LLM immediately while the
commit's transcription was still in flight, so the LLM (fast) won the race
and answered an empty transcript; the response on the real transcript was
then discarded ("skipping response: turn context cancelled during
transcription").
Record the in-flight client commit's slot done channel (closed once the
commit has transcribed and appended its user item) and make the client
response.wait for it before the LLM starts, so the LLM only runs after the
STT is complete.
- issueCommit gains a slotSink callback (slot claim + issue stay under one
lock); the VAD commit path passes nil.
- The client commit / response.create handler cores are extracted to
issueClientCommit / issueClientResponse so the real path is unit-testable.
- Regression specs drive the real issue path with a held transcription: the
LLM must not start until the STT completes, and it must see the user's
transcript.
Signed-off-by: nexxtmobile.de <kai@nexxtmobile.de>
(cherry picked from commit
|
||
|
Ettore Di Giacinto
|
28c70be87e |
fix(realtime): keep sound detection for skipped turns
A VAD commit that is skipped because interrupt_response is false runs with an already cancelled response context. Sound detection used that context, so the model call failed and the turn never got its sound-detection event. Run it under the session context, like the transcription and the voice gate, so overlapping turns keep their sound events. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-bot
|
f22fd45e21 |
fix(realtime): honor interrupt_response
Keep active responses running when VAD interruption is disabled. Skip
an overlapping automatic response while preserving its transcript and
conversation history. Default interruption and explicit client actions
keep their existing behavior.
Add regression coverage for both VAD modes and document skipped turns.
Assisted-by: Codex:gpt-6
(cherry picked from commit
|
||
|
Ettore Di Giacinto
|
4ebff15e76 |
chore(motion): tidy proto, imports and usecase guess; add weights licence
Remove a stray blank line in backend.proto and group the imports in pkg/motion/validation.go. Decide the motion usecase guess from the backend capability registry instead of a hardcoded backend name, so variant builds of the backend resolve too. Declare the weights licence in the gallery entry, as stated on the model card. Assisted-by: Claude Code:claude-sonnet-5-5 |
||
|
Ettore Di Giacinto
|
958add2336 |
fix(motion): cap concurrent motion sessions per user
A model serves one motion session at a time, so a single user could hold several models for as long as the session stays active. Limit each user to four sessions and free the slot on close. The one-session-per-model rule and the global limit are unchanged. Assisted-by: Claude Code:claude-sonnet-5-5 |
||
|
Ettore Di Giacinto
|
e7687e9fe9 |
fix(http): drop the global CSRF relaxation and CORS-derived socket origin
CSRFMiddlewareWithCORS changed CSRF handling for every route and, with a wildcard allowlist, disabled the Sec-Fetch-Site check. Restore the original CSRFMiddleware: clients that authenticate with validated header credentials, or with a single-use WebSocket ticket, are already exempt, so the motion routes need no relaxation. The motion WebSocket upgrade no longer reads the CORS allowlist to decide whether a cross-origin upgrade is acceptable. It accepts one only with a valid ticket or validated header credentials; everything else gets the same-origin default. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
db929f4a08 |
test(config): assert every usecase flag is a distinct bit
FLAG_MOTION and FLAG_UPSCALE were both declared as 1 << 26 on separate branches, and a merge combined the two const blocks without a conflict. Add a spec that checks every flag from GetAllModelConfigUsecases() is a single bit that no other flag uses, so a collision fails the tests. FLAG_MOTION itself moves to 1 << 27. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Richard Palethorpe
|
3de91d48d9 |
feat(motion): add GEM-X backend and duplex streaming API
Add the GEM-X native backend with live pose and root displacement output, bounded duplex WebSocket streaming, session authorization, metrics and usage accounting. Share the embedded bidirectional RPC channel adapter across streaming backends.
Include backend build and gallery integration, API documentation, schemas and tests.
Assisted-by: Codex:gpt-6
Signed-off-by: Richard Palethorpe <io@richiejp.com>
(cherry picked from commit
|
||
|
Ettore Di Giacinto
|
29eba5b7f8 |
docs(models): describe what a queued manual import adds
A queued import goes through the gallery installer, so it persists recommended inference defaults and installs the backend when automatic backend installation is on, while the synchronous write stores the body as posted. Say so in the docs and the endpoint description, list the reference forms that are queued (URLs, hf:// and owner/repo, artifacts), and regenerate the swagger files. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |