94 Commits

Include renames

Author	SHA1	Message	Date
mudler-agentandEttore Di Giacinto	4899d28e8a	feat(chats): server-side chats with steering, sub-agents and compaction (#12918) ... * Find the Sandbox guest image in the executable, not its build info The Sandbox smoke checked that the packaged executable carries the guest image by looking for -X main.guestImage in go version -m. The package is built with -trimpath, and with -trimpath Go leaves -ldflags out of the build info, so the check failed on every gvisor run, although the image was linked in. Look for the reference in the executable instead: the value of SANDBOX_GUEST_IMAGE when it is set, any digest reference otherwise. package.sh had the same check for an executable built elsewhere; it now looks for its own reference the same way. Assisted-by: Claude:claude-opus-5-5 * Answer a network card the limit refuses with one precise reason A card that waited while the administrator's limit went down was refused on the first click with 400, and every later click was a bare 409, so a stale card looked like a conflict loop. A refused card now answers 400 egress_above_ceiling every time, the same answer sent twice is one answer (204), and another answer to an answered card is 409 already_answered. The chat view marks a card the limit cannot admit with above_ceiling. When the limit is off, or below the chat's preset, the note of the turn and the block line of the proxy now say that the server's setting blocks the network and no approval changes it, so the model reports it instead of waiting or retrying. Assisted-by: Claude:claude-sonnet-5-5 * Explain a blocked Sandbox network instead of offering dead buttons A card the limit cannot admit shows why network access is off or limited, and who can change it, with a link to Settings > Tools > Sandbox network for administrators. The buttons of a card disable while an answer is sent, and a failed answer reads the card again. The Network row says why its choices are capped. Settings opens a group from ?group=. Strings are translated in all nine locales and the chat and add-on docs describe the new behaviour. Assisted-by: Claude:claude-sonnet-5-5 * Add an always-allowed hosts list to the Sandbox network The extra registries setting only widens the package registries, so a one-off host needs the approval card in every chat. The new admin setting chats_egress_allowed_hosts lists exact hosts (optionally host:port to pin a port) that a chat reaches whenever its effective preset is registries or all. The list sits in the same decision as the presets: the ceiling still caps it, the ports setting still applies, and a private, loopback, link-local or metadata address still needs the private switch and an administrator's rule or answer. Each use is audited with the reason allowed_host. Hosts are validated like the extra registries (lower case, punycode, no scheme, path or pattern, at most 64) and an IP literal is refused unless the private switch is on. get_chat_egress and the chat egress view return the list; no MCP tool writes it, as for the extra registries. Assisted-by: Claude:claude-sonnet-5-5 * Let users delete from the Trash for good A delete only set a tombstone, so the Trash grew without end and nothing could free the space of a deleted file. The workspace stores gain a Purger: it erases the deleted nodes of some change sets, or of a whole folder, with their journal rows, the change sets that nothing is left in, and the blobs that no row refers to. Both stores implement it and pass the same contract specs. A blob is removed after a grace period, and only if it is still unused then, so an upload of the same bytes in flight keeps them. DELETE /api/chats/files/trash empties the user's Trash and DELETE /api/chats/files/trash/{rev} deletes one entry. Both are scoped to the caller's home. The Trash listing now gives files, folders and bytes per entry and for the whole Trash. The file tools of a chat cannot reach any of this. Assisted-by: Claude:claude-sonnet-5-5 * Add Empty Trash and Delete permanently to the Files page The Trash tab shows what it holds, with an Empty Trash button that is off while the Trash is empty and a Delete permanently button on each item. Both ask in the page first and say how many files and folders and how much space they remove. Texts are translated in all nine locales. Assisted-by: Claude:claude-sonnet-5-5 * Fall back to Brave Search and Bing when DuckDuckGo asks for a captcha DuckDuckGo answers some servers with HTTP 202 and a bot check (the "anomaly" page) instead of results. The parser read that page as "no results it could read", so web_search failed for every query. The keyless search is now a chain: DuckDuckGo, then the result page of Brave Search, then the result page of Bing. Each page has its own tolerant parser (title, URL, snippet; Bing's /ck/a links and DDG's uddg links are unwrapped) with saved fixtures. A bot check is a typed error (ErrChallenged), and a 403 or 429 is ErrBlocked. Both wrap ErrProvider. A Fallback with a cool-down skips a challenged or blocked provider for a few minutes, so each search does not wait for it first; when every provider is cooling down, all are asked again. Fallback.SearchAttempts and TestEach report what each provider did (ok, no_results, challenged, blocked, timeout, error, skipped). The error a model reads now lists what was tried and what each provider did, then where an administrator fixes it. safehttp keeps the status code of an answer (Response.StatusCode) and of a refusal (StatusError, still ErrStatus for errors.Is). Assisted-by: Claude:claude-opus-5-5 * Use the keyless search chain in the chats and test each provider The built-in web_search now searches with DuckDuckGo, Brave Search and Bing in turn when no SearXNG instance is set. A new runtime setting, chats_web_search_providers, sets the order (searxng, duckduckgo, brave_html, bing_html) and leaves a provider out. Empty keeps the old choice: SearXNG alone when an instance is set, so its queries do not reach the public engines unless the administrator lists them. POST /api/chats/admin/web/test asks every provider of the order and returns what each did in "providers" (ok, no_results, challenged, blocked, timeout or error, with the reason), next to the first answer. Settings > Tools has a Test search on the web_search row that shows it, and the provider list is checked on the page and by the server. The cool-down of a challenged provider is kept per frontend, like the cache and the limits; web.md says why. Docs: web.md (providers, chain, bot checks, privacy, how to fix it with SearXNG or the Web add-on) and runtime-settings.md. Strings in the nine locales. Swagger regenerated. Assisted-by: Claude:claude-opus-5-5 * Try the mirror library alone in the reinstall recovery spec The spec adds a second library with the same URL and expects an install from it to keep the retained data apart. Since two enabled libraries that list one add-on are a conflict, the reinstall from the original library then failed with library_conflict, and the install from the mirror failed for that reason instead of the trusted identity. Switch the original library off while the mirror is tried, check that the mirror is refused for its identity and not for a conflict, then switch the mirror off and the original on for the reinstall. Assisted-by: Claude:claude-opus-5-5 * Pause a chat turn at the step limit and offer Continue A chat turn ran at most 10 rounds of tool calls, the default of the harness, because nothing set a limit. A multi-step task (read some files, run commands, write a result) stopped half way, and the person saw an answer that looked final. The limit is now the runtime setting chats_max_iterations (1 to 500, default 40, Settings > Tools); a model's agent.max_iterations still wins. When the tool loop reaches it, the harness sends a paused event with the limit, and the turn ends with last_turn "paused". A paused turn counts as finished: waiting messages start the next turn and a sub-agent reports back. The status line says "Paused, after 40 steps" and has a Continue button, which sends a message that asks the model to go on. A claim that found no waiting message released the chat as "done", which hid a pause. It now keeps the status the chat had. Assisted-by: Claude:claude-opus-5-5 * Edit the always-allowed Sandbox hosts in Settings > Tools Add a list editor for chats_egress_allowed_hosts next to the extra package registries: the hosts are tags with a remove button, an input adds one on Enter, comma or Add (a pasted list adds them all), and the reason an entry is refused shows under the input, with the count out of 64. The same checks stop Apply on a bad entry, and the server still has the last word. The chat's Network panel lists the hosts read only and the row says how many an administrator always allows while the chat's network is on. The help text of both settings now says how they differ. Strings are translated in all nine locales. Assisted-by: Claude:claude-sonnet-5-5 * Document the always-allowed Sandbox hosts Describe chats_egress_allowed_hosts in the runtime settings page, the network section of the add-ons page (including the allowed_host audit reason and how the list differs from extra registries) and the chats page. Assisted-by: Claude:claude-sonnet-5-5 * Let a chat edit, pause and delete the person's triggers A chat could propose a trigger and list them, but not change one: the Triggers page and the API could. list_triggers now gives each trigger a stable ref (not its id) and shows the prompt of one trigger only when asked. update_trigger and delete_trigger store a change record in the trigger table: an edit or a delete waits for Apply or Delete on a card, a pause or a resume applies at once with a receipt. An applied change can be undone for a day; a delete restores the trigger with its id. Change records are rows in the database, so every frontend shows and answers the same card. The new routes under /api/chats/triggers/changes serve the card and are gated by the triggers feature. Assisted-by: Claude:claude-opus-5-5 * Show the trigger changes a chat asks for as cards An edit lists each changed field before and after with Apply and Dismiss, a delete shows the trigger and its last run with Delete and Keep, and an applied change is a receipt with Undo. The cards sit below the message text like the proposal card, and read their state from the server so a reload shows the same answer. Assisted-by: Claude:claude-opus-5-5 * Add a Test search button to the web tools settings POST /api/chats/admin/web/test existed, but no page called it and the docs showed only curl. The web_search row of Settings > Tools now has a Test search button. It runs one search with the applied settings and shows the provider with its first results, or the error a chat would read. Assisted-by: Claude:claude-opus-5-5 * Describe undo per tool call in the chats docs The docs said the checkpoint line of a turn carries the change set of the turn. A turn checkpoint is taken only by a chat with a VM sandbox, which the service never sets up, so that rev is always 0. What ships is one change set per file tool call, with a files_change card on its tool line. The docs now say so and how to undo a whole turn. Grouping a turn's changes into one change set in the file store is left for later: it touches the file store's revisions and the undo of shared folders. Assisted-by: Claude:claude-opus-5-5 * Mount the skill folders read-only for the built-in bash The built-in pdf skill ships scripts, but the commands saw only the home and the shared folders, so the model could read the scripts and not run them. The built-in bash now mounts the three skill layers read-only, at the names the file tools already use: shared-skills, shared-builtin-skills and shared-addon-skills. A shared folder named skills moves to "shared-skills 2", so the two never collide. A read-only mount with subfolders made every command report its folders as changes the mount refused, because the host backend lists every folder of its work tree. A folder alone is no change: both stores now refuse only files, whiteouts and opaque folders there. The Sandbox add-on mounts only the home; the docs say to copy a script there first. Assisted-by: Claude:claude-opus-5-5 * Find the add-on that runs commands by what it shadows The chats knew the add-on that runs commands from a per-process cache, filled with the Sandbox's own tool name the first time a chat listed its tools. Another add-on that replaces bash did not count, so skills that need commands showed as unavailable. The admin network view read the same cache: after a restart it said no Sandbox serves until someone chatted, each frontend of a cluster could answer differently, and a removed add-on stayed listed. The tool that runs commands is now the one that shadows bash, as the spec asks. sandboxRunTool and the cache are gone. The stand-in of an add-on that cannot work carries the built-in tools it replaces, and the admin view asks the add-on host which add-on serves bash now: the pinned one or the oldest running install, and none when the pinned one is stopped. Assisted-by: Claude:claude-opus-5-5 * Show international host names in punycode in the network checks The card-spoofing defence of the sandbox network asks for host names in their ASCII form, but CheckHost passed Unicode names through. A look-alike name reached the consent card as text that reads like the real site. CheckHost now converts an international name with the IDNA lookup profile and refuses one with no valid ASCII form. The conversion runs before the address checks, so full-width digits that spell a loopback address are refused as loopback. Assisted-by: Claude:claude-opus-5-5 * Keep the seeded Studio and Talk pins until the add-ons are installed The upgrade pins Studio and Talk for existing accounts, but both are add-ons that an administrator installs later. The sweep marked a pin whose add-on is not installed as missing and deleted it after 30 days, so an administrator who installed Studio late found the seeded pins gone. The sweep now records the first time a seeded add-on is present, in the table of seed runs or the pins file. A seeded pin keeps no clock until then, and a mark that an older sweep set is cleared. After the first install the pin follows the retention rule as before. Assisted-by: Claude:claude-opus-5-5 * Fix two small labels of the Skills and Backends pages On the Skills page, the skill of an add-on that is not chosen said "Replaced by <add-on>", the label of a built-in skill that an add-on serves. It now says "Not active: <add-on> provides it", as the add-on skills spec asks. On the Backends page, Clear filters on an empty Installed search left the library filter set, so the list stayed empty. It now clears the library filter too. Assisted-by: Claude:claude-opus-5-5 * Tell the model when no network card can appear Each turn told the model that a blocked host asks the user. No card appears when the ceiling is off or no wider than the chat's preset, in an unattended run or in a sub-agent, so the model waited for an answer that could not come. The line now says that a blocked host is refused and nothing asks the user in those cases, so the model names the host it needs in its answer. Assisted-by: Claude:claude-opus-5-5 * Translate six Backends and preview labels in every locale check, reinstallBackend, deleteBackend and filterLabel of the Backends page and the label and failed text of the file preview showed in English in the eight other locales. They now have translations, and the i18n baseline goes down to match. Also fix a comment of the home composer that still named the removed Canvas chip. Assisted-by: Claude:claude-opus-5-5 * List the admin routes that have no assistant tool, with the reason The admin oversight routes of saved agents, triggers and connectors had no MCP tool and no recorded reason, and the other deliberate skips lived only in code comments. coverage_test.go now keeps them in adminRoutesWithoutTool with a reason each, and a spec checks that no route there is also wrapped by a tool. The contributor checklist names the map. The tool catalog of the assistant docs said it matched the real one but missed most tools added since. It now lists every tool, read-only and mutating, and the actions that stay in the web interface. Assisted-by: Claude:claude-opus-5-5 * Close the doc and registry gaps of the chats and add-ons - /api/agent/* answers 404 with the chats_off reason when the chats are off, as /api/chats/* and /api/workspace/* do. - The add-ons docs get an upgrade section: Studio and Talk are add-ons, nothing is preinstalled, and users need the addons feature, which the upgrade does not grant. It says how an administrator installs them (also from a private library) and grants the feature, and carries a paragraph for the release notes. - The migration page no longer lists fork, edit, the chat settings and the token counters as removed. - Runtime settings document chats_vision_model, chats_transcription_model and the skill keys of chats_tool_providers. - The docs show POST /api/chats/files/folders, GET /api/chats/files/history and the allow route of a connector sender. - The in-tree specs get amendments where the shipped design differs. - A route test of the tagged suite still set the removed Runs field. Assisted-by: Claude:claude-opus-5-5 * Look for the kit empty state in the backends regression spec The Backends page now draws the shared EmptyState of the kit (ui-empty), not the old dk-empty block, so the regression spec of the gallery suite found no empty state after a search with no match. Check the kit's no-results form and its title. Assisted-by: Claude:claude-opus-5-5 * Ignore a features request that outlives its page in the add-on stubs stubAddonRoutes answers /api/features by fetching the real flags and turning add-ons on. A spec that ends while the page polls, like the one that checks that a viewer without the feature never reads the list, left that fetch in flight, and Playwright failed the spec with "route.fetch: Target page, context or browser has been closed" after its checks had passed. Abort the request when the fetch fails because the page is gone. Assisted-by: Claude:claude-opus-5-5 * Bring the gallery-ui case inventory up to date The suite lists the node unit cases of addons.test.mjs and the Playwright specs of addons-gallery.spec.js. The inventory missed the unit cases added since it was written, and one spec was renamed when the legacy switch came in ("hub: the Add-ons tab appears only for an account that holds the feature" is now "hub: under the legacy switch the Add-ons tab appears ..."). The list comes from a local run of both parts against the UI test server. Assisted-by: Claude:claude-opus-5-5 * Read spreadsheets, documents and slides in read_document read_document said "file.xlsx has no text layer; it may be a scan" for workbooks that some Python libraries write. Their workbook gives the sheet as an absolute part name (/xl/worksheets/sheet1.xml) and the cells as inline strings. The text extractor of the harness joined "xl/" with that name, found no sheet, and returned no text; the tool then took any empty result for a scanned PDF. The new package core/services/studio/extract has one entry point, Extract(ctx, name, data), that returns the text and the structure it read, for read_document now and for a file indexer later. Office files go through the files preview reader, which resolves both kinds of part names and reads inline strings: - a workbook comes sheet by sheet, rows as tab-separated lines, up to 2000 rows and 50 columns a sheet, with a note for a cut sheet; - a Word document gives headings, list items, paragraphs and tables; - a presentation gives each slide with its title, text and notes. PDF and EPUB keep the harness extractor. Only a PDF with no text is reported as a probable scan; an empty Office file is reported as empty. The preview reader now keeps a Word table as one block with its rows, and the preview pane draws it as a table. Assisted-by: Claude:claude-opus-5-5 * Add the missing explorer, library and tools translations Eight locales had no explorer.json, library.json or tools.json. The browser asks the server for every namespace of the language it shows, so an Italian page that used the library answered 404 for /locales/it/library.json on every load and fell back to English. The 24 files now exist, translated, with every key of English. The locale gate counted a missing file as missing keys, which the baseline allowed. It now fails on a missing namespace file whatever the baseline says, and the baseline is 24 entries shorter. Assisted-by: Claude:claude-opus-5-5 * Back off while the add-on service is unavailable A page that followed an add-on upgrade read the operation every two seconds, up to 300 times. While the add-on service restarted, each read waited for its timeout and answered 503, so the log of one session had 48 failed dispatches in a row, and the page only showed the upgrade as running. The Sandbox state of a chat's Tools menu read once and, on a 503, said nothing more. The reads of an operation now back off while the service does not answer: 2, 4, 8, then up to 30 seconds. The upgrade entry says that it waits for the add-on service and when it checks again, in Activity and in the details, which also get a Check now button. The Tools menu says that the Sandbox is starting and reads its state again with the same back-off, kept for the page, so opening another chat does not ask again before the wait is over. Assisted-by: Claude:claude-opus-5-5 * Start a chat with its switches and list them before it exists The composer of a new chat could not show the Tools and Skills menus, because the lists and the switches belong to a chat. A chat started from Home therefore always began with every tool and skill on. - POST /api/chats takes switches: the add-ons, MCP servers and connectors, built-in groups and skills that are off, and the opt-in groups that are on. They are checked as the PUT endpoints check them and stored with the new record, so the chat never exists without them; a bad entry refuses the request and creates nothing. The SQL store now writes these lists on create too. - GET /api/chats/draft/tools and /api/chats/draft/addons list what a chat on a model, with a saved agent, could use, every switch in its default state. They refuse what create refuses: an unknown or forbidden model, an agent without the agents feature, another user's agent. Document both in the chats page and regenerate the swagger. Assisted-by: Claude:claude-opus-5-5 * Give Home the composer of the chat page Home drew a smaller message box than the chat page: no files, no agent, no Tools or Skills menus, no paste to attach, no rich model list, and its text was lost on a reload. A chat started there could not be set up before its first message. Move the composer wiring out of the chat page into one place that both pages use: - useChatComposer holds the draft (text and files), the model and the saved agent of a new chat, the chat models, the switches and what attaching and pasting do. - ChatComposer draws the model list (fit, vision, memory bar), the agent picker, the Tools and Skills menus, the attach buttons, dictation, the add-on slot and the vision notice over HomeComposer. - useChatSwitches has a draft mode: before the chat exists it lists from the draft endpoints, a switch changes only the screen, and the choices go with the create request. They outlive a change of model or agent. Home now creates the chat with the model, the agent and the drafted switches, sends the first message with its files and opens the chat. When the message cannot go, the new chat is removed and the text and files stay on Home. Its text is kept in the browser across a reload. Home also gets the shortcuts row, the notices of the chat page and the line that says whether the model is loaded. Its own slash actions, widgets, examples and first-run steps stay. The empty chat page gets the same draft menus. On a phone the chips of both wrap to a second row, and the Tools and agent menus open as a sheet from the bottom, as the skills picker does. Assisted-by: Claude:claude-opus-5-5 * Move add-on settings to the add-on's own page The settings.sections slot drew every add-on's settings at the end of the global Settings page, and the Sandbox and Web sections were unstyled: the add-on UIs used class names the host does not define (btn, btn-primary) and bare inputs, the SDK offered no form primitives, its muted token named a property that does not exist, and the slot root was an inline-flex span that laid a form out in a row. An add-on's settings sections now render on the Settings tab of its details on the Add-ons page (/app/addons?detail=<id>&view=settings), narrowed to that add-on, for administrators only. An add-on page gets a host Settings link to that tab. The Settings page no longer draws the slot: an Add-ons group, shown when an add-on has settings, lists those add-ons with a link each. The old #ad-section-<addon>-<id> anchor redirects to the tab once the apps list is read. Server settings of an add-on's feature stay on the Settings page; the tab shows them under "Server settings for this add-on", with a read-only summary of the Sandbox network and links to ?group=tools#settings-egress and #settings-builtin-tools. The SDK gets a settings kit in sdk.ui (Section, Field, Select, TextInput, Switch, Choice, StatusChip, Callout, List/ListRow, Code, Actions, Table, ExternalLink), drawn with host classes and tokens, and more theme tokens. The Sandbox and Web sections are rebuilt with it: backend rows with a status chip and the raw reason collapsed in a code block, a backend choice with one line per option, and the AppArmor and user-namespace hint (with a docs link) only for the gVisor "exit status 128 ... cannot read client sync file" failure. The Settings page showed "Changed, default 30, Reset" next to a 0 for the audit retention, and the same for the Sandbox network limit, preset and ports: the server stores 0 or an empty value for a setting never set. Those fields now read that value as their default, and the retention input has its unit and bounds. Assisted-by: Claude:claude-opus-5-5 * Adapt the add-on runtime specs to the new navigation Three specs of the esm-ui suite still assumed the old shell: - the Add-ons tab that lists the pages add-ons declare is part of the legacy layout, so the navigation spec found no hub subnav; it now sets the legacy switch on the server's flags for that page; - the sidebar has no "Chat" link any more, so the unsaved-edits spec could not leave the page; it leaves through Home in the primary navigation; - the shell keeps a notification stream open, so the network never goes idle and the spec for a member without the feature waited out its timeout; it waits for the server's switches, the account and the shell instead. Assisted-by: Claude:claude-opus-5-5 * Name the legacy Add-ons tab unit case as the esm unit tests do The esm-ui inventory listed "unit: registry: the Add-ons tab lists page links only when the registry has usable add-ons", but the unit test is named "the legacy Add-ons tab lists ...", so the suite could never match its inventory. Use the name the test prints. Assisted-by: Claude:claude-opus-5-5 * Say on the Collections page when the embedding model is missing At startup the server opened a collection and its first embedding call got 404 from POST /embeddings. The collections embed through this server's own API with collections_embedding_model, the default granite-embedding-107m-multilingual, and that model was not installed. Nothing told the user: uploads and searches just failed. GET /api/collections now also returns embedding_model and embedding_model_installed (a loaded configuration of that name, or a model file of that name). When it is missing, the Collections page says "No embedding model installed", names the model, and gives an administrator a button that opens the Models page with a search for it. Assisted-by: Claude:claude-opus-5-5 * Report modified times and the upload limit in folder listings The Files page could not sort by date or show when a file changed: a folder listing had no time. It also learned the upload limit only after it had sent the whole file. The default server body limit is 15 MB, lower than the 256 MiB a workspace file may have, so a large upload failed late and without a clear reason. Each listed entry now has "modified", the time of the change set of its revision. Both workspace stores fill it in List: the JSON store from its in-memory change sets, the database store with one query per batch of revisions. The listing also has "max_file_size", the smaller of the workspace limit and the server body limit, so a client can refuse a larger file before it sends any byte. The store contract suite checks the new time against the history, and the route tests check both fields. Assisted-by: Claude:claude-opus-5-5 * Add a sortable file list with thumbnails for the Files page The building blocks of the new file browser, used by the page in a later change: - fileBrowser.js: the kind of an entry from its extension, the order (folders first, natural and case-insensitive names, then size, modified time or type), the filter of a folder, dates in words with the full date for a tooltip, the Shift range, the arrow moves of a list and of a grid, type-to-find, names that do not clash, why a name cannot be used, the name of a pasted screenshot and the view settings read back from storage. - FileList: a grid with sortable column headers or a grid of tiles with image thumbnails, a check box per row, a row menu, inline rename with the reason a name is refused, rows that can be dragged onto folders, and the keyboard: arrows, Shift, Space, Enter, F2, Delete, Ctrl+A, Escape, Backspace and typing a name. - MoveDialog: a folder picker that walks the tree and refuses a move into the moving folder itself. - ActionMenu can open at a point with no trigger (a right click), and its trigger can take another icon and class (a split button). - fileSize also writes gigabytes. Assisted-by: Claude:claude-opus-5-5 * Add an upload queue with progress, cancel and retry An upload of the Files page was one fetch per file in a loop: nothing showed while it ran, one failure stopped the rest, and leaving the page lost it. - uploadQueue.js keeps every upload outside React: queued, sending with the bytes sent and a smoothed rate, done, failed with a reason and whether Try again can help, or canceled. Two files go at once. It lives for the whole app, so opening another folder or page does not stop an upload. - filesApi.uploadWithProgress sends the same PUT with XMLHttpRequest, which reports upload progress and can be aborted. The server already streams the body into the store; nothing changes there. - useUploads reads the queue and asks before the tab closes while a file is on its way. - UploadTray shows the queue at the bottom of the page: the total and the time left, each file with its progress, speed and time left, Cancel, Try again with the reason, and a done state that fades out. A live region tells a screen reader when uploads start, fail and end. - ConflictDialog asks what to do when an upload has the name of a file in the folder: replace it, keep both, or skip it. - dropFiles.js reads dropped folders with their paths. Assisted-by: Claude:claude-opus-5-5 * Rebuild the Files page as a file browser The Files page was a list of links: the store's byte order, the same icon for every file, three icon buttons on every row, no progress on an upload, no selection, and on a phone a preview that opened below the whole list. Browsing: a breadcrumb that folds a deep path, sortable columns (name, modified, type, size) with folders first, a list or a grid of thumbnails, both kept per user in the browser, a filter of the open folder with a search of every file on Enter, type icons, human sizes, dates in words with the full date on hover, and Back and Forward between folders. Uploads go through the upload queue and its tray: the Upload button or its folder menu, drops of files and folders anywhere on the page or onto a folder row or crumb, and paste. A clash asks to replace, keep both or skip; a file over the server limit is refused before it is sent. Selection and actions: click, Ctrl and Shift, check boxes, a selection bar (Download, Move to, Move to Trash; at the bottom of the screen on a phone), a row menu and a right click, inline rename and new folder or file, drag to move, the keyboard, and a toast with Undo for a delete, a move and a rename. States: a skeleton on the first load, an error with Try again, empty states that say how files get here, and a no-match state. The preview uses the side pane: docked on a wide screen, a sheet on a phone, with the same actions as the row menu. Trash, History and the Managed by LocalAI group stay as they were. Every new string is in the nine languages. The page uses design tokens only (seven raw font sizes fewer). The specs cover sorting, the filter, the views, selection and bulk actions, the keyboard, the menu, rename checks, drag to move, drops, paste, clashes, the size limit, upload progress, cancel and retry against the real server, and the error states. Assisted-by: Claude:claude-opus-5-5 * Fix two specs that fail on the integration branch The Describe spec of the egress package checked that the no-card line does not contain "asks the user", but that line says "nothing asks the user". It now checks for the sentence that promises a card. The context meter spec gave the chat a 4k context window. The tool schemas of the chat now take 3k of it, so the turn stopped before the model request. The spec uses a 16k window. Assisted-by: Claude:claude-opus-5-5 * Translate the always-allowed hosts note of the chat network row The three egress keys of the always-allowed hosts were only in English, so the locale check failed for every chats namespace. The workspace namespaces now miss two keys fewer, so their baseline goes down too. Assisted-by: Claude:claude-opus-5-5 * Bring the contracts and host-slots case inventories up to date The contracts suite has eight new specs for the check of a tool that shadows a built-in. The host-slots suite runs the unit case of the file preview slot, which the inventory did not list. Assisted-by: Claude:claude-opus-5-5 * Add the design of the file text index The search of the chats' files reads only small UTF-8 text files, so a PDF or an Office file is found by name only. The design extracts the text when a file is written, keeps it in LocalAI's own store keyed by node and hash, checks access at query time and runs one indexer per cluster under an advisory lock. Assisted-by: Claude:claude-opus-5-5 * Give extracted documents their pages, sheets and slides Extract now says where each page, sheet and slide starts in the text, so a search hit can name its page or sheet. A PDF page starts with a "## Page N" line, as sheets and slides already start with a heading. A PDF also stops at a page limit (200 by default) and at the text limit, which it ignored before, and a panic of the reader is an error. The test fixtures move to extracttest, with a real text PDF and a workbook with shared strings, so other packages can build the same documents. Assisted-by: Claude:claude-opus-5-5 * Add a store for the text of workspace documents textindex keeps one entry per workspace node: the text read from a PDF or Office file, the hash it was read from, its pages or sheets and its status (pending, ready, failed, skipped), plus the cursor of the indexer in the workspace journal. The GORM store keeps a lower-cased copy of the text, so SQL and the Go matcher agree on every script, and searches it with FTS5 and the trigram tokenizer on SQLite or a pg_trgm index on PostgreSQL. Without them it scans the column. The JSON store keeps the metadata in memory and each text in its own file, and scans the texts. One contract suite runs on JSON, SQLite and PostgreSQL. Assisted-by: Claude:claude-opus-5-5 * Index the text of new and changed workspace documents The indexer tails the workspace journal from a cursor kept in the text store. A write of a PDF, Word, Excel or PowerPoint file marks it pending (the same hash only follows the path), a delete removes the entry and a move rewrites the paths below a folder. Then it reads the pending files two at a time, with a timeout, a size cap, a page cap and a text cap, and stores ready, failed or skipped. A panic of the reader fails the file, not the pass. The same content is read once. The first pass starts at revision 0, which indexes the files that were there before, in bounded steps that resume from the cursor. Enqueue marks an upload pending at once and Forget drops the entries of nodes a purge erased. In a cluster each pass runs under a new advisory lock, so one frontend reads. Assisted-by: Claude:claude-opus-5-5 * Search and read the text of documents in the chats' files search_files and GET /api/chats/files/search now also search the text index: after the name and text-file hits come lines of PDF, Word, Excel and PowerPoint files, as path (page 3, line 40): text, with the sheet or slide where a document has them. Each entry is shown only when the caller may read its file now and the file still has the content the text came from. Documents keep up to a third of the hits. When files below the path are still pending, the answer says so instead of "Nothing found.", and the REST result has indexing. read_file returns the text of a PDF or Office file with the usual offset and limit, from the index when it matches the file and read on the spot otherwise. An upload and a write of a chat tool queue the file at once, and emptying the Trash drops the entries of the purged files. Assisted-by: Claude:claude-opus-5-5 * Start the file text index with the files and add its settings With files on, the server opens the text store next to the workspace index (the auth database, or a folder under the data path) and starts the indexer. A chat worker searches the index but runs no indexer. chats_index_files (default on) switches the reading off and chats_index_max_text_bytes (2 MiB by default, 64 KiB to 16 MiB) caps the text kept for one document. Both apply at the next pass, and both are rows of Settings > Tools. Assisted-by: Claude:claude-opus-5-5 * Show the page, sheet or slide of a document hit on the Files page Search all files now lists hits inside PDF and Office files with their page, sheet or slide instead of a line number of the extracted text, and says how many documents are still being indexed. The strings are in all nine locales. Assisted-by: Claude:claude-opus-5-5 * Test finding an uploaded PDF by its words through the routes The PDF has a generated name and is uploaded with the REST upload. The search route first reports it as being indexed, then finds it on its page after a pass of the indexer, and so does the search_files tool of a chat. Another user finds nothing. Assisted-by: Claude:claude-opus-5-5 * Document the search inside documents The chats page says what the index reads, where the text is kept, who finds it, which search engine each database uses, the two settings, the backfill and the single-runner mode in a cluster, and how search_files and read_file answer for documents. The API instructions and the PDF skill say the same in short. Assisted-by: Claude:claude-opus-5-5 * Test the indexer on the JSON workspace store A server without a database keeps the workspace and the text in JSON files. The indexer follows the uploads, a folder move and a delete in that journal as it does in the database one. Assisted-by: Claude:claude-opus-5-5 * Read all of a long document with read_file The index keeps the first chats_index_max_text_bytes of a document, so read_file used to stop where the index stopped. It now uses the index only when it holds the whole text, and reads the document again (up to 16 MiB of text) when the index cut it. Assisted-by: Claude:claude-opus-5-5 * Start bash in the user's home The Sandbox's bash and the built-in bash started in /workspace, the folder that holds the mounts, so a model that ran a script by a relative path failed and a file it saved there was lost. A call now starts in /workspace/home, the folder of the user's files: a relative path is the same file the file tools see. The tool descriptions and the commands block of every turn say where a call starts and that only what is saved there is kept. The built-in bash says that the /workspace paths do not exist on the host and that a shared folder is next to the home. Assisted-by: Claude:claude-opus-5-5 * Name files in tool results by the path the model uses write_file, edit_file and search_files named a file of the home by its path in the home ("/scripts/build.py") while the model had written /workspace/home/scripts/build.py, and a model took the two spellings for two files. The results now give the path the model uses, and move_file says where it moved the file to. The cards and the change notes keep the path in the home. Assisted-by: Claude:claude-opus-5-5 * Name the paths a command's commit did not save The result of a bash call ended with "N file(s) changed in your files meanwhile; the sandbox's version was kept as a copy" for every path the commit refused, also for a file written outside the user's files, where no copy is kept. It now has one line a path, with the path as the guest sees it, the reason, and the path of the copy when the store kept one, so the model and the user can act on it. The host returns the paths and the copy to the Sandbox add-on relative to the user's files, as its mounts are. Assisted-by: Claude:claude-opus-5-5 * Show the files a command saved under its step A document that a script wrote with bash reached the user's files, but the chat showed nothing: only the file tools put a file on their step, and the Sandbox committed for no chat, so the chat's file list, the chips under the answer and the preview pane never saw the file. The host now commits a Sandbox call for the chat that ran it (the action resource of the request names the chat). After each bash call, of the Sandbox or of this machine, the chat reads the change sets of the chat that the call made from the shared store and puts the files on the step's card, next to the card that says who served the call (session.SetToolFiles; files with path, change set and op on the card's new files field). The step lists them as chips that open the preview pane, the chat's clients are told that its files changed, and the files reach the file list, so the answer gets its chips and the pane opens on the newest one, as for write_file. Assisted-by: Claude:claude-opus-5-5 * Keep JSON out of the answer of a trigger run The workspace skill told the model to put the widget data shape and a source marker in a trigger prompt. Some proposals then asked the run to print that JSON, and every run ended its chat with a stray line of it. The skill now says source and note live inside the data file and are never the answer, and that a prompt must not ask for JSON. The runner appends one closing instruction to the first message of every trigger run: finish with one short plain sentence. The propose_trigger and update_trigger descriptions say the prompt must stand alone and must not ask for JSON. Connector chats are talks with people and do not get the line. Assisted-by: Claude:claude-sonnet-5-5 * Read the images of a page as base64 for its frame The bridge of a page reads UTF-8 text only, so the UI could not put a png or a jpg of the page's folder into the page. A read of the page file routes with encoding=base64 now returns an image as base64 with its media type. Only the extensions of an image are read this way (png, jpg, gif, webp, avif, bmp, ico, svg); any other file is 415, so this is no way to read an arbitrary binary file. The path checks, the 1 MiB limit, the viewer's rights and, for a shared page, the consent check are the same as for a text read. Assisted-by: Claude:claude-opus-5-5 * Let a page use the files next to it A page runs from a srcdoc with no network, so fetch('news.json'), <script src="app.js">, <link rel="stylesheet" href="style.css">, <img src="logo.png"> and import './x.js' all failed without a word. Models that split a page into files, or load its data with fetch, made pages that showed nothing. Before the frame loads, the UI now reads the files that the page names with relative URLs, with the same calls as the bridge, and puts them in: classic scripts in place (deferred ones at the end of the body), style sheets as <style> with their @import and url(), images as data: URLs, and ES modules as blob: URLs behind an import map, so cycles and import() work. A page with modules gets blob: in script-src; no other part of the policy changes. Inside the frame, fetch and XMLHttpRequest read relative URLs with files.read and answer with a real Response (200, 404 for a missing file, 413, 415); any other URL rejects with "network is off in pages; put the data in a file next to the page". Every path is resolved inside the page's folder: .., %2e%2e, absolute paths, backslashes and protocol-relative URLs are refused, and the query and the fragment are dropped. The files go in only after a shared page is allowed, with the viewer's rights. The files that go in are at most 4 MiB, images 2 MiB; a script or style sheet that does not load shows a notice in the page. A change of one of these files reloads the page, as a change of index.html does. The skills, the docs and the shared pages fixture say and test the same. Assisted-by: Claude:claude-opus-5-5 * Return tool descriptions from an MCP connector test A connection test listed only the tool names. A person who adds a server cannot tell from a name such as "query" what the tool does or whether to keep the connector on. The test response now also has tool_details: each tool with the description its server gives, in the order of tools. tools is unchanged, so existing clients keep working. Assisted-by: Claude:claude-opus-5-5 * Add jq, ripgrep, sqlite3 and archive tools to the Sandbox guest The guest user cannot install packages and apt is blocked by the proxy, so any tool the model needs has to be in the image. Add jq, ripgrep, sqlite3, zip, unzip, file and tree to the final stage. A new check-tools.sh runs in the existing build step and fails the build when one of these programs, Poppler, git or curl is missing or does not start. The list is one constant, GuestTools, in the shared sandbox package. The Commands block of the prompt of an isolated chat names it. The tool description (capped at 400 characters, already full) and the turn note (cut at 400 characters) have no room for it. Tests keep the Dockerfile, the check script and the Sandbox documentation in line with the constant. Assisted-by: Claude:claude-sonnet-5-5 * Keep the helper files of a chat in its work folder A chat asked for a Word document left a 19 KB build script at the top of the user's home, next to the user's own files. Chats like it leave scripts, intermediate data, logs and drafts at the top level or in folders such as scripts/, and nothing tells them apart from what the user asked for. Each chat now has a work folder, Chats/<chat folder>/work/ in the user's home, next to its attachments. The chat takes the name of its folder at its first turn (checked against the folders and the other recent chats of the user, so two chats with one title get two names) and makes the folder only when it is used: before a bash command runs, or when a file is saved there. The prompt of every turn names the folder and says what goes in it: scripts, helper code, intermediate data, logs and drafts go there; the deliverable goes where the user said or at the top of the home; no new top-level folders for scripts; a run-once command needs no file. A sub-agent is told its parent's folder. The descriptions of bash (the Sandbox's, within its 400 characters, and the built-in one) and of write_file end with one short hint, and the built-in office, apps and workspace skills say the same in one line. The steering is soft. When write_file, move_file or a command leaves a helper file outside the work folder, the result gets one line that suggests moving it, at most once a turn of a chat. One function (workspace/workfiles) decides the kind of a file: working for anything in a chat's work folder, or for a script, a log or a temporary file by its extension or a folder named scripts, tmp or scratch; deliverable for everything else, and always for pages, the LocalAI folder, the shared folders and the files the user attached. The chat's file list (kind), the files of a command's card (files[].kind) and the card of a file tool (file_kind) carry it, so the interface only renders it. The runtime setting chats_work_folder (on by default) turns the steering off. Assisted-by: Claude:claude-opus-5-5 * Stop telling the model that the Sandbox has npm The guest image has no Node, so the network hints named a tool that does not exist. Assisted-by: Claude:claude-sonnet-5-5 * Update the Sandbox description test after dropping npm Assisted-by: Claude:claude-sonnet-5-5 * Open the links of a page after the person confirms them A page runs in a frame without allow-popups or top navigation, so a click on <a href="https://..."> did nothing, and a link to #id loaded LocalAI's own address in the frame, since a srcdoc resolves it against the parent. A news page had only dead links. The bridge of the frame now takes the clicks on links (also with a modifier, the middle button or Enter) and window.open(), and sends link.open(url) to the host. The host lets through only an http or https address without a user name or password, at most 2048 characters; javascript:, data:, file:, blob:, //host and links to other files of the page are refused with a message in the console. A bar above the page shows the site in large type, the whole address, and Open and Cancel. A host in punycode also says how it reads, since it may imitate another site. Open is a click in the app, so the new tab is not blocked and opens with noopener and noreferrer. A page asks at most 5 links at once, then one every 3 seconds. A link to #id scrolls the page. The sandbox, the policy of the frame and the consent of shared pages do not change; a shared page asks the same way. Assisted-by: Claude:claude-opus-5-5 * Let the items of a Home widget link A list widget showed each item as plain text, so a digest of articles or a list of pages could not take the person anywhere. An item now has an optional url. An http or https address without a user name opens in a new tab with noopener and noreferrer, and its tooltip names the site; a path of the app (/app/pages, /app/chats, /app/files and the other screens a person reads) opens inside LocalAI. The label is the link, with a focus ring, an underline on hover and focus, and an icon for an address outside the app. The widget-data schema checks the url, and the server adds an error in words that names the item and the rule, so a trigger run that writes a bad link learns why. The skill and the docs show the field. Assisted-by: Claude:claude-opus-5-5 * Attach any file in a chat with one button The chat had three attach buttons (image, audio, file) that all add to the same list, and the file button's dialog filtered on */*, which some file dialogs read as a type filter that greys out archives such as .zip. A chat now shows one Attach file button whose dialog has no type filter. Composers without anyFile keep the three buttons. Assisted-by: Claude:claude-opus-5-5 * Accept both Node behaviours for a broken punycode label in the link test Node 22 refuses the address, Node 24 parses it. Both mean the page never opens it unwarned. Assisted-by: Claude:claude-sonnet-5-5 * Show a chat's working files apart from what it made The Files pane, the steps and the preview treated a helper script like the document the user asked for: a 19 KB build script was listed next to the document, got a chip under the answer, and the preview opened on whichever file was newest. The interface now renders the kind the server puts on each file. The pane lists what the chat made first and folds the working files into one quiet heading under the rest, "Working files · 2", with a line that says what they are; opening it shows their rows, which preview like any file. A working file opened from a step unfolds the group. The preview that opens by itself after a turn picks the newest file that is not a working file, and the chips under the answer name only those files. A step that saved working files lists them as one line, "Also saved 2 working files", that opens to their chips, and the file of a write_file step that is a working file is a quieter chip. The files the user attached are unchanged. Assisted-by: Claude:claude-opus-5-5 * Add a search field and a segmented filter to the UI kit The Agents views each drew their own toolbar: a search box capped at 320 px on the agents list, round chip filters on the triggers history, and a bare checkbox on the tasks page. Add-ons already had the look the product wants (a search that fills the row and a segmented tray), but only as add-on classes. SearchField and Segmented are that toolbar as two small primitives with ui-* classes and token values only. Segmented buttons carry their id in data-filter, so specs can press them. The README shows how to use them. Assisted-by: Claude:claude-opus-5-5 * Draw an agent task run as a history row with its output The jobs of an agent task were table rows that opened to bare <pre> blocks: no copy, parameters printed as k=v, the traces in a raw <details>, and "by manual" in the server's words. RunItem is one run of a history list: a toggle, the state chip, what it was and what happened, when and for how long, small actions, and the details that open under it. OutputBlock draws an answer or an error in mono with a Copy button. JobRow and JobDetail use them for a job: the values it ran with as labelled chips, the answer or error, the steps, the webhook outcome and a link to the chat that holds the run. Times show as words with the full date as a tooltip. The agent task helpers gain the pure rules the page needs: the jobs of a task, the run filters (waiting and running count as in progress), the on/off filter and search of the task list, the field errors of a save, how a job started, and the API call that runs a task by name. Assisted-by: Claude:claude-opus-5-5 * Create, edit and run agent tasks in a side sheet The task form opened as bare fields above the list, with a checkbox for On and a Save button that was only disabled when a field was missing. Run with parameters was a second inline form in the same place. TaskSheet is a side sheet with three labelled groups (about, who runs it, what it does), help under each field, the parameters the prompt asks for as chips while you type, and an On switch. Save checks the required fields, says next to each one what is missing, and moves the focus to the first. RunSheet asks for each value the prompt names and shows the prompt as the task will receive it. Assisted-by: Claude:claude-opus-5-5 * Show the details of an agent task in a pane with tabs A task had no details view: its prompt, its runs and the way to run it over the API were only in the lede of the page. TaskPane is the kit's side pane with three tabs. Overview shows who runs the task, the values it asks for, the prompt and a curl call that runs it by name, with Copy. Runs lists its jobs. Settings holds the On switch, the webhook count and Delete. Run and Edit stay in the footer. The tabs move with the arrow keys, Home and End. Assisted-by: Claude:claude-opus-5-5 * Rebuild the agent tasks page from the shared list pieces The tasks page looked unstyled next to Add-ons and the agents list: two raw tables under uppercase eyebrows, a bare checkbox for All users, forms that opened above the list, three text buttons on every row, and on a phone a table that broke names one syllable per line. The page now uses the same pieces as the polished pages. The header has one line and New task. A toolbar holds the search, an All/On/Off filter and, for an administrator, an All users switch. Each task is an agent row: a tile, the name, two lines of description, who runs it and what it asks for, the last run's state and time with the run strip, an On switch, Run and a menu. A task with no parameters runs at once. The name opens the details pane; New task, Edit and Run open the sheets. The run history is grouped by day, with a state filter, a task filter and Show more. A job is deleted after an in-page question. With no tasks the page explains what one is, with an example; a failed load shows the error with Try again. On a phone the controls grow to 44 px and the filters scroll sideways. New strings, and the task strings that existed only in English, are in all nine locales; the strings the page no longer uses are gone. A new stubbed spec covers the list, filters, sheets, pane and history, and the real-server spec follows the new controls. The docs describe the page. Assisted-by: Claude:claude-opus-5-5 * Rebuild the MCP connectors page on the Add-ons patterns The page read as a form: every connector was a row of raw controls (a checkbox, a Test button, red Delete text), the add form pushed the list down, a test result vanished on reload, and the frame was narrower than the pages next to it in the sidebar. The page now uses the same blocks as Add-ons and Files: - Full page width and the shared PageHeader with a one-sentence purpose and "Add connector". - The Add-ons toolbar (search and a state filter with counts) from four connectors, and the Add-ons rows: a monogram tile, a state chip (Connected, Needs sign-in, Can't connect, Not tested, Off), one line with the tools or the reason, a meta line, the kit Switch and a menu. - Details in the shared SidePane: the switch, the last test, the tools with their descriptions and a search, the connection with the query of the address hidden, and what is saved. - Add and edit in the shared SideSheet: kind first, grouped fields with inline checks, masked secrets with show and hide, then "Save and test", the result with the tools, and Turn on. Escape and Cancel ask before they drop typed changes. - An empty state that says what a connector is for. The last test of each connector is kept in this browser with a hash of what it tested, never the address or a secret, so the page shows the last known state after a reload. Strings are in all nine locales. Assisted-by: Claude:claude-opus-5-5 * Document the MCP connectors page and its states The MCP servers section now describes the add sheet (kind, checks, masked secrets, Save and test, Turn on), the states a connector shows, where the last test is kept, the search and the filter, and the details pane. The test example shows tool_details. Assisted-by: Claude:claude-opus-5-5 * Render a preloaded page without its loading placeholder A page that was preloaded still showed the route's loading placeholder when it opened: React.lazy suspends once on its first render even when the chunk is already in memory, and React keeps a fallback on screen for at least 300 ms once it shows one. Opening a chat from Home showed a blank page for that time. The page loader now remembers the module once it has loaded and hands it to React.lazy as a thenable that resolves at once, so a preloaded page renders in the same commit. A preload of a segment now warms every page registered under it: /app/chats is the chat page, its connectors and its conversion page, and before only the last one registered was preloaded. Assisted-by: Claude:claude-opus-5-5 * Move the composer from Home into the new chat in one gesture Sending the first message on Home was a hard cut. The page stood still while the chat was created, then showed a blank placeholder, the empty "new chat" screen with its composer in the middle, an empty thread with the composer at the bottom, and only then the message, while the sidebar folded and moved the whole column. Focus was lost. Now Send fades the rest of Home and keeps the composer in place while the message goes. When the send has answered, a view transition moves the composer to the dock of the chat, lifts the text out of it into the message, moves the attached files with it and folds the sidebar in place; the model's indicator comes in after the message has landed and the empty composer has the focus. The chat page draws the message from what Home already knows (a small hand-off store: the chat record, the text and the files, with the local copy of an image for its thumbnail) and keeps the same row when the server's transcript replaces it, so nothing is drawn twice or animates again. What is sent to the server does not change. A browser without view transitions gets a FLIP slide of the composer and the message (Web Animations, transform and opacity only). Less motion gets a short cross-fade, or an instant switch. A send that fails brings Home back, shakes the composer and keeps the draft; a second Enter while one send is in flight sends nothing. The draft is cleared when Home closes, so the box never shows its placeholder on the way. Focus mode folds the sidebar in a layout effect, so the fold happens in the frame the conversation appears. One motion token is added, --dur-glide (360 ms), for a part that travels to its place on the next page. Assisted-by: Claude:claude-opus-5-5 * Draw the triggers as rows and their history like the task runs The triggers page used a table with a switch column next to the agents card list, folded its list behind Hide, filtered the history with round chips, and on a phone cut each history row at the right edge. Each trigger is now an agent row: a clock or folder tile, the name, the schedule in the server's words, the agent or model and when it runs next, the last run's state and time with the run strip, an On switch, Run now and a menu. The week sentence sits under the Run history title. The history uses the segmented filter and the same run rows and day headings as the agent tasks, with Show more after 30 runs; an open run shows its reason with Copy. On a trigger page the prompt comes before the runs, in the body face, the On switch is the kit switch, and the runs stop at 10 with Show more. Assisted-by: Claude:claude-opus-5-5 * Use the shared search and the full width on the agents list The agents list stopped its search at 320 px and used the content width, while the Triggers and Tasks views next to it use the full width, so the title and the view tabs moved when you switched views. The list now uses SearchField, which fills the toolbar row as on Add-ons, and the full page width like its sibling views. Assisted-by: Claude:claude-opus-5-5 * Polish the agent task sheets and pane after a visual review Screenshots of every state showed a few rough edges: - The name placeholder looked like a filled-in value next to the error "Give the task a name". The name field has no placeholder now, and the prompt placeholder starts with "For example:". - A task run by an agent showed the agent id until the agents were read. It says "Agent" until then. - A run opened in the details pane also opened in the history, and the two copies shared an element id. The pane keeps its own open runs and ids. - A failed run in the pane drew its meta line in red; the title carries the error colour there now. - A group title that is not a legend had a double gap under it, and the kebab menus were under 44 px on a phone. Assisted-by: Claude:claude-opus-5-5 * Cover the code files of a shared page in its consent The consent to a shared page was kept against the hash of index.html and page.json. The UI also puts the scripts, style sheets and modules next to the page into it before it runs, so the author could change app.js after a viewer allowed the page and run new code without asking again. The server now finds the same files as the UI's inliner: every script src, stylesheet link, relative CSS @import and module import of the folder, followed through the module graph. Their content hashes go into the hash of the page, and a file the page names that is missing counts too, so adding it later also asks again. Data files and images stay out, so saving data does not ask again. The files are listed as code_files. What a file names is kept per content hash in a small per-frontend cache, which only saves reads. Two more checks close the gaps around it. A frame of an older version that reads a changed file of code gets 409, so a script edited while the page is open never runs under the old consent. The UI puts no text file into a shared page that is not in code_files, so if the server and the UI ever disagree, the file fails to load instead of running outside the consent. The trust sheet names the files of code when only one of them changed. Assisted-by: Claude:claude-opus-5-5 * Let a page be a tile on Home in its compact mode Home widgets could show a list, a number or the last chats, but not one of the pages a person or the assistant built. A page can now be a tile. A page opts in with a widget block in page.json: a size (small, medium or large, the height of the tile) and an optional short title. A widget of type page in LocalAI/home.json names the page, and a folder id for a page of a shared folder. The server takes at most three page widgets and says why in words, and refuses a name that is not a page name. The tile runs the page in the same sandbox and bridge as the full view. The page reads localai.page.mode ("widget" in a tile, "page" otherwise) and localai.page.size, and <html> carries data-mode and data-size, so the page decides how its tile looks instead of being shrunk. The tile reserves the height of its size, builds its frame only once it scrolls into view, tells the page through a new visibility event when it goes out of sight or the tab is hidden, takes the frame down far off screen, and keeps at most three frames alive. It has a title bar with Open page and a refresh button, a skeleton while it loads, Retry on an error, and the link question of the full view. A page that is gone or has no widget block shows a calm line with a link to Customise Home; a shared page that the viewer has not allowed shows Allow this page, which opens the same trust sheet. Customise Home lists the pages with a widget block, with the size of their tile. The task board starter has a compact view, the skills tell the assistant how to put a page on Home, and the docs describe it. Assisted-by: Claude:claude-opus-5-5 * Give the Files upload its own size limit The Files page upload was capped by the general --upload-limit (15 MB by default), although a workspace file can be 256 MiB. Make 0 the default of --upload-limit, meaning not set. When it is not set, the general routes keep the 15 MB limit and the Files upload route skips it, so the workspace file size is its limit. When the operator sets a value, it applies to the Files upload too, lower or higher (a file never exceeds the workspace size). The max_file_size of a folder listing reports the limit in force. Assisted-by: Claude:claude-sonnet-5-5 * Refuse writes of code files from a page A page loads its own scripts, modules and styles, so a page that could overwrite app.js or style.css would run new code on its next load. The page bridge now writes data files only. The server refuses a write whose final segment has a code or document extension (js, mjs, cjs, css, html, htm, svg, xml, xhtml, wasm), is named index, or is page.json, with 403 and a clear message. The check applies to the page routes only; the Files page and the assistant's file tools keep full write access. The judgement lives in pages.IsCodeFile and is mirrored by isCodeFile in the web host, which refuses early with the error code forbidden. Both run one table of cases, covering case, double extensions, trailing dots and spaces, percent-encoding, backslashes and lookalike dots. Assisted-by: Claude:claude-sonnet-5-5 * Make the trigger tools a built-in group a chat can switch off propose_trigger, list_triggers, update_trigger and delete_trigger were added to every chat a person starts, with no switch. They are now four entries of the built-in tool registry in one group, triggers, so the Tools menu, the tools panel, the draft list and the builtins switch endpoint handle them like the web group. The group is on by default. A trigger or connector run and a sub-agent still never get them. The panel lists the group as not usable, with a reason, for those chats, for a user without the triggers feature, and not at all when the server runs no triggers. The group has no runtime setting, so the registry gets an Unlisted flag and Settings > Tools skips it. The panel now builds every grouped tool to learn whether it is usable, not only the opt-in ones. Assisted-by: Claude:claude-sonnet-5-5 * Show the Triggers group in the chat Tools menu The menu rows come from the groups the server lists, so the row needs only its label in the nine locales. A Playwright spec covers the row, its switch request, Select all and Disable all, and the disabled row with its reason. Assisted-by: Claude:claude-sonnet-5-5 * Talk about settings, not the files behind them The settings cards, the shortcuts row, Home and the change notes in a chat named the files that hold a person's settings: "Defined in LocalAI/shortcuts.json", "Layout is in LocalAI/home.json", "Saved LocalAI/persona.json". A person changes their shortcuts or their Home, not a file, and the Files page still shows the files to whoever looks. The "Stored in", "Defined in" and "Layout is in" lines are gone; the history line keeps Versions, Revert and the Invalid badge. The notes say what changed ("Saved your shortcuts.", "Created the page Task board."), the errors name the setting or the widget instead of a path, and the Source view says "The settings as text" instead of JSON and schema. A failed save now says what happened in one plain sentence and keeps the server's own words under a collapsed Details. The drafts that "Edit with the assistant" puts in the composer no longer carry the path; the built-in workspace skill already maps shortcuts and Home to their files. All nine locales are translated; only the changed keys moved, and four keys that nothing reads any more are removed. Assisted-by: Claude:claude-opus-5-5 * Drop internal names from the skills, add-ons and tasks copy Other pages used words that only make sense to someone who knows how LocalAI is built: - Skills said "a folder with a SKILL.md" and "the file workspace". It now says what a skill is for, and that skills need Files. - The add-on notes blamed "the owner", the internal name of the server that runs add-ons. They now say what happened to the action, or that the add-on service is not available; a private library names a credential "an administrator configured on this server". - The Tasks header listed an API route, and the first-run wizard spoke of a "host bash fallback". The memory line of Settings no longer mentions "the resources endpoint". All locales that have these keys are translated. The skill delete dialog, the unused skills intro and the Tasks header exist only in English and change only there. Assisted-by: Claude:claude-opus-5-5 * Fail the unit tests when UI copy names implementation details A new test reads every string of every locale and fails when one names a settings or data file (.json, .yaml, LocalAI/), an API route or HTTP method, a setting key (chats_*), an environment variable, a format word (JSON, YAML, schema) or a path shown as code. Plain words such as "schema" and "endpoint" are checked in English only, since other languages use them for other things. Operator pages may still name the model configuration, the metrics route or a startup setting that the docs name. Those keys are listed in ui-copy-allowlist.json, each with the audience and the reason. The list only shrinks: an entry whose string no longer needs it, or whose key is gone, fails too. Assisted-by: Claude:claude-opus-5-5 * Name the triggers group in the chat tools routes The routes that list and switch the built-in groups of a chat said only "a name such as web". They now name all three: web, triggers and saved_agents, with the tools of each. Assisted-by: Claude:claude-opus-5-5 * Make the built-in skills answer in plain words The skills told the model to report file names, folders and formats after a change. Add one rule near the top of localai-workspace and localai-apps: do the work with files, but say what changed, and name a file only when the user did or asks. Replace the example replies that carried paths, keep every path that says where to write, and pin the rule with a test. Assisted-by: Claude:claude-sonnet-5-5 * Apply the Go 1.26 modernizers to the latest Studio code go fix suggested slices.Contains, strings.SplitSeq and range-over-int on lines added since the first pass. ParseSearchProviders had the same hand-written membership loop, so it uses slices.Contains too. Only lines that the branch added since then are changed. Assisted-by: Claude:claude-opus-5-5 * Share the blob wait and the counting of the two workspace purges The JSON and GORM stores each had the 2-minute default for the blob grace, the same switch that reads the BlobGrace option, and the same loop that counts files, folders and bytes per revision. They now call blob.Grace and PurgeResult.Add, both with specs. The SQL and the in-memory walk that pick the nodes stay in each store. Assisted-by: Claude:claude-opus-5-5 * Read the upper layer of a sandbox in one place for both stores The JSON and GORM workspace stores had the same readUpper, place, joinRel, depth and sortedKeys, with the same upperSet types. They move to workspace/internal/upperset, which has its own specs. The commit algorithm that applies the set to the nodes stays in each store; it needs a storage interface first. Assisted-by: Claude:claude-opus-5-5 * Let chat attachments use the 1 GiB message limit The docs promise 1 GiB for a message with attachments, but the general body limit (15 MB by default) ran first, because only the Files upload skipped it. Skip it also for a multipart POST to the messages route when the operator set no --upload-limit. A JSON message keeps the general limit, and an explicit --upload-limit still wins. A message that goes over the 1 GiB limit now gets 413, not 400. Assisted-by: Claude:claude-opus-5-5 * Name the read_file tool in the skills index header A weaker model read a built-in skill with the wrong path and then tried the Sandbox, which cannot see skill folders. The header now says which tool reads a skill and that the shown path is the one to use, before it mentions where the user's own skills go. Assisted-by: Claude:claude-sonnet-5-5 * Tell the model that skill folders are not in the Sandbox The Commands block of an isolated chat now says that skills are read with read_file. The Sandbox tool description and the per-turn note are at their size caps, so the sentence goes in the system prompt. Assisted-by: Claude:claude-sonnet-5-5 * Point read_document at read_file for text files read_document reads PDF, Word, Excel, PowerPoint and EPUB files. For a Markdown or text file it said only that the file is not a document it can read. The error now names read_file. Assisted-by: Claude:claude-sonnet-5-5 * Tidy the text index stores and the indexer pool The two stores wrote the default limits 20 and 100 as numbers in five places; they are now named constants, so the stores cannot drift. The JSON store checks a missing file with errors.Is. The indexer starts its readers with wg.Go and keeps the first error inline, which drops the cmpErr helper. Assisted-by: Claude:claude-opus-5-5 * Refuse the server's public address in the Sandbox proxy web_fetch refuses the addresses of the server: its interfaces, loopback and the host of the external base URL. The Sandbox proxy runs in the add-on process and listed only the interfaces of its own machine. Under "All public sites", an always-allowed host or an Allow card, a command could reach the LocalAI API through the public name or address of the server, a NAT address or a proxy in front of it. The host now keeps one list for both. The application builds it once, gives it to web_fetch, and sends it as CIDR strings in the new self field of every egress.check allow. The proxy keeps the last list it got and refuses it on each dial, besides its own interfaces, also for an allow it cached. safehttp gets the codec and a join of deny lists, so both sides read the list the same way. Assisted-by: Claude:claude-opus-5-5 * Check the TLS server name on every port of the Sandbox proxy The proxy checked the ClientHello only on CONNECT to port 443. An always-allowed host pinned to another port, or any port added to chats_egress_ports, was a raw tunnel: a command could open TLS to another tenant of the same front through it. The docs said the server name must still match. On other ports the proxy still waits for nothing, since the protocol may be anything and the server may speak first. But when the first bytes from the guest are a TLS handshake record, it reads the whole ClientHello and passes it on only when the server name is the CONNECT host; otherwise it closes the tunnel and audits the reason sni. Assisted-by: Claude:claude-opus-5-5 * Say what sent each notification The inbox only knew whether a notification was about a run or needed attention, so a trigger that was switched off and a connector that was switched off looked the same. Each notification now carries a source: trigger, connector or system (the conversion of old data). The list endpoint returns it, and items stored before have none. Assisted-by: Claude:claude-opus-5-5 * Say where a skill is when a read of it fails When read_file, read_document or list_files fail with not found for a path that names a skill of the chat (a wrong skills folder, or just <name>/SKILL.md), the error now gives the skill's real path and says to read it with read_file. The skills come from the chat's own resolver, so every layer, add-on and per-user switch is respected, and nothing is looked up unless a read fails. Assisted-by: Claude:claude-sonnet-5-5 * Keep the last valid Sandbox network lists when a value is bad The settings endpoint refuses a bad chats_egress_allowed_hosts with the reason, and the runtime settings file keeps the value in force. A bad value that reached the configuration another way was dropped with no log line: the always-allowed hosts and the extra registries became empty, and a bad port list fell back to 80 and 443, which can widen a narrower list. Each of the three lists now keeps the last value that parsed, logs a bad value once, and uses the default only when the process never saw a valid one. The parse runs again only when the value changes, not on every egress.check. Assisted-by: Claude:claude-opus-5-5 * Use the keyless chain of the built-in search in the Web add-on The add-on replaces the built-in web_search, but its keyless provider was DuckDuckGo alone. DuckDuckGo often answers data-center addresses with a bot check, so installing the add-on without a key made search worse than the built-in, which falls back to the Brave Search and Bing result pages. The keyless entry of the add-on's order (saved name duckduckgo) is now the built-in chain from pkg/web/search: DuckDuckGo, then the Brave Search page, then the Bing page, with the same cool-down. The status tool reports the providers in their cool-down, and the Settings tab shows them with the reason. Test the providers gives one row for each page of the chain. Assisted-by: Claude:claude-opus-5-5 * Fix the GitHub hosts in the Q1 row of the egress design The Q1 row still listed raw.githubusercontent.com on the registries list. The owner decision below it and registries.json exclude it, since it serves any user content, and add release-assets.githubusercontent.com. The row now says the same. Assisted-by: Claude:claude-opus-5-5 * Keep one private-address rule for the network cards A card asked whether its host is a private IP literal by calling egress.Admits with a made-up ceiling, which re-derived the rule that egress already has. egress.PrivateLiteral is now exported (with a spec) and the card uses it. The two String methods of an allowed host write host:port with net.JoinHostPort, which brackets an IPv6 literal the same way the hand-written code did. Assisted-by: Claude:claude-opus-5-5 * Read a unique constraint violation in one place The jobs and triggers stores had the same isUniqueViolation, and the saved agents store had the same test inline before it looked for the name index. They now call dberr.IsUniqueViolation, which has specs for the SQLite and PostgreSQL error texts. The workspace store keeps its own stricter match. Assisted-by: Claude:claude-opus-5-5 * Ask before a Home widget link opens another site A list widget turned an http or https item into a plain link to a new tab. Its data file is written by trigger runs and the assistant, which read web text, so a planted link opened with one click, and only a tooltip showed the host. A link of a page asks first. An external item link now opens the confirmation bar of page links in the widget: the site, the full address, the punycode warning, and Open or Cancel. A link inside the app still navigates directly. The bar takes a label, so the widget says that it wants to open the link. Assisted-by: Claude:claude-opus-5-5 * Prune the audit of the commands run on this machine The audit of the built-in bash had no retention: the database table grew without bound in a cluster, with commands of up to 4 KiB each. The Sandbox network audit had chats_egress_audit_days and a daily prune under an advisory lock. Both audits record what the commands of the chats did, so the host command audit now uses the same retention and is pruned in the same daily pass: one frontend of a cluster, under the same lock; a standalone process alone. The GORM and file stores get Prune. No new setting. Assisted-by: Claude:claude-opus-5-5 * Save the Studio JSON stores with atomicfile.WriteJSON Seven stores (agents, connectors, jobs, MCP connections, notifications, push subscriptions and triggers) encoded their state and then called atomicfile.Write with the same four lines. atomicfile.WriteJSON does both and has specs. The load side stays in each store, since the error messages differ. Assisted-by: Claude:claude-opus-5-5 * Build the result page walkers of the keyless search on one walk hasChallenge and firstByClass each had their own recursive walk that stopped at the first match. They are now predicates passed to firstElement, which already did that walk. The order of the walk and the element tested first are the same. Assisted-by: Claude:claude-opus-5-5 * Expire network cards that a dead frontend left pending Pending network cards expired only on the turn-end path of the process that ran the turn. When that frontend died, its cards stayed pending in the chat record, counted toward the cap of three, and gave the next turn's connections cards-full blocks. The UI showed them as waiting. A card now names the turn that asked it. One rule, shared with the command card in a later change, says whether a card still has a waiter: its turn holds the chat and its wait has not run out. A card that fails it reads as expired, takes no place under the cap, and is expired in the record when the next card is asked, when the next turn starts, or when the reaper ends the dead turn. Assisted-by: Claude:claude-opus-5-5 * Reuse ChooseProvider for the expected add-on of a tool shadowers.expected repeated ChooseProvider line for line: the pinned add-on when it is installed, else the oldest install. It now calls it. commandFiles kept an order field and sorted the nodes of a map by it; it now appends each node to a list when it first sees it, which gives the same order without the sort. Assisted-by: Claude:claude-opus-5-5 * Do not print a nil error when a key or entry is missing Three checks joined "the read failed" and "the read found nothing" in one branch, so the second case wrote "<nil>" into the message: the VAPID keys of push notifications, the sealing key of connectors and a rendered catalog entry. Each case now has its own message. The message of a failed read does not change. Assisted-by: Claude:claude-opus-5-5 * Check the close errors and rename the tool variables golangci-lint failed on the new studio code. Five deferred Close calls and one Fprintf into a hash dropped their error, and two locals named t made forbidigo read `t.Run` as a testing call. Discard the errors explicitly and name the locals tool. Assisted-by: Claude:claude-opus-5-5 * Make the file text index forget purged files and say its limits Three gaps of the text index of documents: - A purge dropped the index entries of what it erased in a goroutine that only logged its errors, and the purge had already erased the journal rows. A crash or a store error left the text in the database for good. Each pass of the indexer now also checks a batch of entries (200 every 30 s, round all of them) and deletes those whose node is gone, so what a failed Forget leaves is removed. - A document search stopped at 20000 files or 64 MiB of text without a word, and the "still being indexed" count stopped at 50. The search now sets Limited (search_files says "results limited", the REST answer has "limited", the Files page shows a note), and the count is the number of pending entries below the path. - read_file extracted a long document again for every page, up to 50 MiB and 60 s each time. The first read keeps the whole text in the index store by content hash (a new documents table, or files in the JSON store), and the next pages come from it. The text goes away with the last entry of that content, on delete and on purge. Search reads the text without its folded copy, which only the database needs. Assisted-by: Claude:claude-opus-5-5 * Stop the PDF reader when the extraction gives up The PDF and EPUB reader runs in a goroutine, and the call returns after 60 seconds when it is slow. The goroutine went on reading every page, since its context ended only with the caller's. It now gets a context that ends when the call returns, so it stops at the next page. What the call returns does not change. Assisted-by: Claude:claude-opus-5-5 * Compare the code files of a shared page as the store does The guard that refuses new code to a frame of an older version compared the name it reads with the names of the page's code exactly. The workspace folds case, so an old frame that read Index.html or JS/App.js got the new code with no 409. Compare names in the store's form (workspace.FoldName), as the write check already does, and test the case variants. Assisted-by: Claude:claude-opus-5-5 * Expire the command card of a turn that is over The card of the built-in bash was cleared only by a defer in the call that asked. When that frontend died, the card stayed in the chat record. The chat hid it only while no turn ran, so the next turn showed it again, and an answer to it reached no one but still got 204. The card now names its turn and when it was asked, and follows the same rule as the network cards: it waits only while that turn holds the chat and its five minutes have not run out. GET /api/chats/{id} shows only such a card. The next turn and the reaper clear a card left behind, and the waiting call clears only its own card, so it never removes a newer one. An answer to a card with nobody waiting is 410 consent_expired, and the chat shows the card as not answered, with no buttons. The store gets UpdateHostConsent, an atomic change of the card. Assisted-by: Claude:claude-opus-5-5 * Answer a trigger change once across frontends ApplyChange, UndoChange and DismissChange checked the outcome of the change record under a mutex of one process, then wrote the store. Two frontends could both see a pending change and both apply it, or both undo it. Add Store.SetOutcome, a conditional update of the outcome (only from the outcome it names), to the GORM and JSON stores and to the store suite. The three answers now claim the record with it before they write, give the outcome back when the change itself fails, and answer ErrProposalAnswered (409) when another request won. A test runs the two answers on two frontends at the moment between the read and the write, for an edit, its undo, and a delete. Assisted-by: Claude:claude-opus-5-5 * Add the shadowers spec to the tables-tools inventory The tool adapter gained a spec that lists the add-ons that replace a built-in in any state. The tables-tools suite runs every spec of that package, so verify.sh found one case more than the inventory holds. Assisted-by: Claude:claude-opus-5-5 * Offer Clear filters when the backend Catalog shows nothing The Installed view had a Clear filters button in its no-results state; the Catalog view did not, although the spec asks for it in both. Move the button to one place and show it in the Catalog when a search, a type filter or a library filter leaves no row. Add Playwright specs for both views, which the earlier Installed fix lacked. Assisted-by: Claude:claude-opus-5-5 * Give the chat settings without an assistant tool their reason chats_max_iterations, chats_work_folder, chats_index_files and chats_index_max_text_bytes had no MCP tool and no entry in adminRoutesWithoutTool, and the older chats_saved_agents, chats_suggestions, chats_vision_model and chats_transcription_model were unlisted too. Add each with the reason it stays a person's choice in Settings: they apply to every user's chats, and the step limit is one a model must not raise for itself. Assisted-by: Claude:claude-opus-5-5 * Remove the stray allowed-hosts keys from the chats inbox locales allowedHosts, allowedHostsOff and allowedHostsNote sat twice in eight locales: under egress, where English has them and the UI reads them, and again under inbox, where nothing reads them. In de and ko the two copies had drifted apart. Delete the inbox copies and keep the egress ones. Assisted-by: Claude:claude-opus-5-5 * Bring three design specs in line with what shipped - web tools and shadowing: the status still said phase 2 waits; both phases shipped. Note that the Web add-on's keyless entry is now the built-in keyless chain. - Home front door: Home now creates the chat and sends the first message itself, with a short-lived hand-off to the chat page, and its composer takes attachments. Both were described the old way or as out of scope. - chat footer: the meter bar is 96 px with the tokens left, and a phone keeps the bar and the tokens left, not the percentage only. Assisted-by: Claude:claude-opus-5-5 * Store the command card's answer and read it back every two seconds The answer to the command card of the built-in bash went only as a bus message. When the message was lost, the call waited the full five minutes and ended as not answered, although the person had clicked. The network card stores its answer first and re-reads the card every two seconds. The command card now does the same, through one shared pair of helpers: listenConsent subscribes before the card is stored, and waitConsent wakes on the bus message or every two seconds and reads the record, which holds the truth. The network card uses the same helpers. HostExecDecide writes the decision to the card atomically before it sends the message: the same answer twice is 204, the other answer 409 already_answered, and an answered card is no longer shown as waiting. Assisted-by: Claude:claude-opus-5-5 * Load skills by name with a load_skill tool A weaker model failed to read a skill at the path the index showed. The chat now has a built-in load_skill tool that takes a skill's name and returns the instructions of its SKILL.md, without the frontmatter. The tool reads the same resolved skill set as the index: the nearest skill of each name, the chat's switches, the add-ons the chat can use now and the grants of its agent. It reads through the user's rights. When the skill's folder holds other files, the answer starts with the base directory and names up to 20 of them for read_file; a chat without read_file gets the instructions only. A skill that needs commands, in a chat that cannot run them, is refused with the reason. An unknown name is an error that gives near names first and then every name the chat has. Every chat with at least one skill gets the tool, and a sub-agent gets it with its parent's tools. It has no switch and Settings > Tools does not list it. The index of skills now lists names and descriptions only and tells the model to call load_skill. The Sandbox sentence and the error for a wrong skill path now point at load_skill too. Assisted-by: Claude:claude-opus-5-5 * Tell the built-in skills to load other skills by name localai-apps told the model to read the localai-workspace skill, and writing-skills said a description is what the agent sees before it opens the file. Both now speak of loading a skill with load_skill. localai-workspace keeps the path of its schemas folder and says to read it with read_file, since those files are not a skill. A test checks that no built-in skill sends the model to another skill by "read" or by the path of its SKILL.md. Assisted-by: Claude:claude-opus-5-5 * Document load_skill and the skills index The chats page now says that the prompt lists skills by name and description, that load_skill returns a skill's instructions and names the files of its folder, what it refuses and who gets it. It no longer says the model reads a skill by its path. The add-ons page says how a chat loads an add-on's skill, and the saved agents page says that the skills grant limits load_skill too. Assisted-by: Claude:claude-opus-5-5 * Add the proposal action spec to the tables-acceptance inventory The spec that an editor cannot confirm a schema change as a write ran in the suite but was never in the inventory, so the evidence manifest check failed. CI did not reach that check before, because the burst spec failed first. Assisted-by: Claude:claude-opus-5-5 * Give a chat a goal that it works toward by itself A chat can now hold one goal. While it is active, every turn's prompt names it, the model gets goal_done and goal_blocked, and when a turn ends by itself the frontend that ran it starts the next one with "Continue toward the goal". The person's waiting messages still go first, and a turn they start does not count as an automatic one. The goal pauses when the person presses Stop, when a turn fails or is cut short, and when the automatic turns or the model steps reach their limits. Resume gives it fresh budgets. A sub-agent, and a chat that a trigger, a connector, a job or a page started, has no goal and no goal tools. The goal lives with the chat record in the store (UpdateGoal on the JSON and GORM stores, with contract tests and a column migration), so every frontend reads the same one. The next turn goes through the same turn claim as any other, so only one frontend continues it; a reaped turn, or a JSON store that reloads after a crash, leaves the goal paused instead of continued twice. The transcript keeps goal lines (set, replaced, continue, paused, resumed, done, blocked, cleared) as their own role, so an automatic message never reads as typed by the person. Assisted-by: Claude:claude-opus-5-5 * Add a store for the loops of the chats A loop repeats a prompt in one chat, at an interval or when the model asks for the next run. The new loops package keeps the loops, the daily budget of runs of each owner, and which chats confirmed a loop once. JSON and GORM stores share one contract suite. Add checks the limits per chat and per owner in the same step as the insert, and TakeRun never hands out the last run of a day twice, also across frontends. A user message that a loop sent now carries a loop mark (loop ID, run number, self-paced or not). Both chat stores keep it. The model reads a short line before the prompt that says the message repeats by itself, and, for a self-paced loop, how to ask for the next run. Assisted-by: Claude:claude-opus-5-5 * Add the file preview slot specs to the host-slots inventory The file preview slot came with two Go specs in the host-slots package. verify.sh found them in the run but not in the inventory. Assisted-by: Claude:claude-opus-5-5 * Check the Settings page and the add-ons with the UI copy guard The guard read the locale files only. The Settings page keeps its English labels and descriptions in settingsSchema.js, and several add-ons carry their own English bundle, so neither was checked. The guard now also reads the label, description and note of every setting and the label and hint of every group, and the English bundles of the add-ons (an en.js module or an English JSON file). The allow-list takes settings:<key>.<what> and addon:<id>:<key> entries, each with its reason; the operator ones (the SearXNG JSON format, the fine-tuning file types, the API route of a voice, the schema of the Databases add-on) are listed. What it found in the voice library is fixed in plain words: "model YAML is not modified" and "Existing model YAML is not changed" now say that the models' settings are not changed, in the voice add-on, the Studio add-on and the host locales that have the strings (en, fr, pt-BR). The two host allow-list entries for them go. Assisted-by: Claude:claude-opus-5-5 * Add the update digest specs to the updates inventory Two checker specs pin that an update names its candidate digest and that a stale cached notice stays readable. The updates suite runs the whole checker package, so the inventory needs both. Assisted-by: Claude:claude-opus-5-5 * Draw the Add-ons search and state filter with the kit pieces The Add-ons page had its own copy of the search box and the segmented filter (ad-search, ad-seg), the same shapes and styles as the kit's SearchField and Segmented that the Agents pages use. Use the kit pieces and drop the copy; the narrow-screen rules now target the kit classes inside the toolbar. The search keeps its test id. Assisted-by: Claude:claude-opus-5-5 * Add the limits of a chat goal to the runtime settings chats_goal_max_turns (1 to 200, default 20) is the most turns a chat with a goal starts by itself, and chats_goal_max_steps (1 to 10000, default 400) the most model steps of all the turns of one goal, before the goal pauses. Both are read when a turn ends, so they apply without a restart. They sit in Settings > Tools next to the step limit of one turn, and the settings page documents them. Assisted-by: Claude:claude-opus-5-5 * Add the goal routes of a chat to the API GET, PUT and DELETE /api/chats/{id}/goal read, set and clear a chat's goal, and POST /api/chats/{id}/goal/resume resumes a paused or blocked one. They belong to the chat's owner like the other chat routes and take the chats feature. A chat view carries the goal with the limits that apply now, the transcript gives goal lines with goal_event and goal_turn, and the event stream has a goal event so that other tabs read the chat again. The routes are in the route feature registry, the swagger file and the chats entry of /api/instructions. They manage the person's own chat, not the server, so they have no MCP tool, as the other chat routes. The chats page documents goals, their states and limits, and how several frontends share them. Assisted-by: Claude:claude-opus-5-5 * Redesign the notification inbox The inbox panel was a plain list: every row had the same weight, a failed run looked like a finished one, the panel ran off the screen on a phone, and a failed load showed nothing. The panel now has a header with the unread count, a menu for Clear read and Clear all, Mark all read, and an All / Unread / Failed filter. Items are grouped under Today, Yesterday and Earlier. Each item shows what sent it (trigger, connector or LocalAI) with a tinted icon, the title, two lines of text or the reason of a failure, and when it happened, with the exact time on hover. A click marks it read and opens its chat; a connector that was switched off opens the connectors page. Mark read and Delete show on hover or focus, and stay in view on a touch screen. The panel has a loading skeleton, an error with Try again, an empty state that says what shows up here, and a caught-up state. A new notification slides in, is highlighted for a few seconds and is announced once. The badge stops at 99+. Focus moves into the panel when it opens; the arrow keys, Home, End and Delete work on the items, and Esc returns focus to the button. On a phone the panel is a sheet over the bottom of the screen with 44 px targets. Assisted-by: Claude:claude-opus-5-5 * Explain the reads of the text index and the provider requests to gosec gosec flagged three new lines. The text index reads files from the directory the host gave it, with names it listed there. A search provider sends its request to the endpoint fixed at construction, never to a URL from a tool call. Mark the lines with the reason. Assisted-by: Claude:claude-opus-5-5 * Keep the add-on details open when Escape closes a dialog Escape in the restore dialog of the recovery panel also closed the details pane around it. The pane handles Escape in React, and the dialog listens on the document, so the pane ran first and saw no preventDefault. The pane now registers as a modal layer and closes only when it is the top layer, as the side panes do. Assisted-by: Claude:claude-opus-5-5 * Add skill and agent messages, compact and clear to the chats API A message can now carry a skill of the chat or one of the user's saved agents. The server checks the name against the chat's resolved skill set (switches, the agent's grants, shadowing, add-ons) or the saved agents the chat may call, and refuses an unknown one with a 400 that lists what the chat has. A skill message gives the model the text of the skill's SKILL.md before the user's words, for that turn only; an agent message asks the model to start the agent with spawn_agent. The transcript keeps the words and the command (Entry.Invoked), not the instructions. A regenerate or an edit runs the command again. Such a message does not wait for a running turn: it answers 409. POST /api/chats/{id}/compact runs the summary the loop makes when the context fills up, through a new optional session.Compactor that the loop harness implements with its own history compaction. The compacted state is saved for the next turn and a "compacted" line with the sizes goes in the transcript. POST /api/chats/{id}/clear writes a "cleared" line and drops the saved state. The model then sees nothing above the line: the rebuilt state and the transcript summary start after the last such line, so a fork from above it keeps what the model saw there. Export, import and the Markdown export carry both lines. Both calls take the turn claim, so no frontend starts a turn on the chat meanwhile; messages sent during them start a turn after. No MCP tool: they act on one chat of the caller and are not admin calls. Assisted-by: Claude:claude-opus-5-5 * Give each network card spec its own settings A check that a card spec starts can still wait on its card when the next spec begins. It read the settings through a closure over a variable that the next BeforeEach replaced, which the race detector reported. Each service now reads the settings holder it was built with, under a lock, and the specs change the settings through it. Assisted-by: Claude:claude-opus-5-5 * Show the effective audit retention in the settings snapshot The settings snapshot returned 0 for chats_egress_audit_days while the setting was unset. Any page that saves the whole snapshot back, such as the Traces settings strip, then posted 0 and the server rejected it with "Invalid chats_egress_audit_days", so saving tracing settings failed on a fresh install. Report the retention that applies (30 days by default), the same way chats_max_iterations does. Assisted-by: Claude:claude-sonnet-5-5 * Run the Sandbox smoke test in a short scratch folder The smoke test step was reached for the first time and both isolated backends refused to start: their state folder under the checkout's cache made the socket paths 126 and 132 bytes, over the 107 a unix socket allows. Point SANDBOX_SMOKE_WORK at /tmp/sandbox-smoke, which leaves room for the sockets. Assisted-by: Claude:claude-opus-5-5 * Use paths from the home folder in the Sandbox command specs bash now starts in the user's home folder, so the end-to-end specs that read and wrote home/<name> found nothing. Use names relative to the home folder, as the Sandbox specs already do. Assisted-by: Claude:claude-opus-5-5 * Show the goal of a chat on the chat page A bar above the composer shows the goal with its state: Pause (which stops the running turn and so pauses the goal) while it works, Resume when it paused or needs the person, and Clear. A second line says why it paused, what the model needs, or what it did. The status line shows "Goal · turn 3 of 20" while the goal is active, and the Continue button of a paused turn stays away then, since the next turn starts by itself. The transcript shows the goal lines as quiet rows: set, replaced, continuing (with its turn), paused with the reason in plain words, resumed, reached with the summary, blocked and cleared. A goal event, also from another tab, reads the chat again, and a hidden live region says each change to a screen reader once. The strings are in all nine locales. Unit tests cover the helpers and the thread rows; a Playwright spec covers the bar, the progress, the lines, Pause, Resume, Clear and the event. Assisted-by: Claude:claude-opus-5-5 * Run the loops of the chats The loop host fires the loops that are due. A run sends the prompt to the chat as a new message only when the chat is idle: a busy chat skips the run, and an interval loop moves to its next slot instead of piling missed runs up. A self-paced loop gets two tools while it is active: schedule_next (clamped to the minimum interval and one day) and loop_done. A run in which the model calls neither ends the loop with a note. A loop ends after its run limit, pauses after three failed runs in a row, and waits for the next UTC day when the owner's daily budget of runs is used up. One frontend fires the loops, the holder of a new advisory lock. A run is claimed in the store first, and the turn claim keeps one turn per chat, so a short overlap of two leaders starts no run twice. A new leader finishes the runs the old one watched. Deleting a chat removes its loops; a due loop whose chat is gone removes itself. Five runtime settings bound the loops: chats_loop_min_interval, chats_loop_max_runs, chats_loop_max_per_chat, chats_loop_max_per_user and chats_loop_daily_runs. Assisted-by: Claude:claude-opus-5-5 * Add the loops API of the chats GET, POST and DELETE /api/chats/{id}/loops list, start and stop the loops of a chat; PUT and DELETE /api/chats/{id}/loops/{loopId} pause, resume, confirm or stop one. GET /api/chats/{id} carries the loops, the transcript marks the user lines a loop sent with loop {id, run}, and a loops event tells every open tab that the list changed. The routes are gated by the chats feature and documented in the swagger spec and the API instructions. They act on the caller's own chats only, so they get no assistant tool. The chats page gets a loop section, with when to use a loop and when a trigger, and the cluster behaviour; the runtime settings page lists the five loop settings. Assisted-by: Claude:claude-opus-5-5 * Show the loops of a chat above the message box Each loop of the open chat shows as one line over the message box, such as "Loop · every 5 min · run 4 · next at 09:20", with Pause or Resume and Stop. A loop that paused itself or ended says why; an ended one has Dismiss. The first loop of a chat shows a card instead: "Run this every 5 minutes until you stop it?" with Start and Cancel. The server keeps the pending loop, so a reload shows the card again. A message that a loop sent carries a small "Loop run 4" label. The page reads the loops again on the loops event, so every open tab follows. The answer to /loop list is ready for the slash command. The strings are in all nine locales. Assisted-by: Claude:claude-opus-5-5 * Build the chat slash menu on one command registry The slash commands of Home and of a chat now come from one registry, src/utils/slashCommands.js. A command has a kind (navigate, insert or server), the surfaces that offer it, a needs check and, optionally, an argument provider: args(query, ctx) returns the items the menu lists when the box holds "/cmd <query>". Every existing command and test id stays; the registry adds: - one command per skill of the chat or the draft (/review), marked built-in, add-on, shared or yours, and /skill <name>, which also reaches a skill named like a command; - /agent <name> <task>, which lists the saved agents, or offers to turn them on for the chat when they are off; - /model and /models with a model name, which switch at once; - /compact and /clear, which show their progress in the status line and leave a marker in the thread; the context meter drops; - /attach and /attach clear, /help (a card that lists the commands) and /about (a card with the model, its state, the tools, skills, where commands run and the context size). Tab completes a command or an argument, Enter runs it, and the first row whose name starts with what was typed is selected first. A message typed as a skill or agent command is sent with its marker, not as text; the transcript shows the words and a chip. Home offers the same set without the commands that need a thread. The words are in all nine locales. The registry has unit tests, and a Playwright spec covers the menu, the completion, the marker, the chip, compact, clear and the cards with the routes stubbed. The docs page gains a slash commands section. Assisted-by: Claude:claude-opus-5-5 * Add the /goal command to the chat /goal shows a card with the goal, its state, the automatic turns and steps it used out of the limits, and when it was set. /goal <text> sets a new goal at once, replacing the one the chat had without a question, and the goal bar says what it replaced. /goal clear and /goal resume act on the goal; typing "/goal " lists resume and clear for the goal the chat has. None of these is sent as a message. In a new chat, /goal <text> creates the chat first. The command sits in the Context group of the shared command registry, on the chat surface only, and not in a chat that a trigger or another run started. A skill named goal is reached through /skill. Unit tests cover the parser and the menu items; the Playwright spec covers the card, setting, replacing, resuming and clearing from the box. Assisted-by: Claude:claude-opus-5-5 * Add /loop to the chat /loop [interval] <prompt> starts a loop in the chat; /loop list shows the loops of the chat with the code that /loop stop takes, and /loop stop [id] stops one or all. The command is in the slash registry: after "/loop " the menu offers list and stop, and after "stop " the loops of the chat. A loop prompt that invokes a skill (/review the diff, or /skill review the diff) runs as the person's skill command would: the model gets the skill's text, and the transcript shows the words, the skill's chip and the loop label. The server checks the skill when the loop starts and at each run; any other command is refused with the reason. Assisted-by: Claude:claude-opus-5-5 * List the loop settings on the Settings page The five limits of the loops of the chats show under Tools with the other chat settings: the shortest interval, the runs per loop, the loops per chat and per user, and the loop runs per day. All apply at once. Assisted-by: Claude:claude-opus-5-5 * Clear a sent Home draft when the page goes away too Home clears the draft of a sent message when it closes, so the text stays visible while the composer moves into the new chat. While the chat page still loads, a reload or a closed tab could come first, and the sent text came back on Home. Leaving the page now clears it too. Assisted-by: Claude:claude-opus-5-5 * Check the close error of a previewed file golangci-lint (errcheck) flagged the deferred Close of the file the preview endpoint reads. Discard its error explicitly, as the other studio file handlers do. Assisted-by: Claude:claude-opus-5-5 * Explain the document read of the text index to gosec gosec reported the read of a stored document text as a file inclusion through a variable. The path is the configured index directory joined with a hash that documentPath refuses when it holds a path separator or a dot, so it cannot leave the documents folder. Assisted-by: Claude:claude-opus-5-5 * Build the darwin/arm64 add-on packages on macos-15 GitHub retires the macos-14 runner image on 2026-11-02 and runs scheduled brownouts until then: every darwin/arm64 package job failed at once with the brownout notice. macos-15 is the Apple Silicon image that replaces it. Assisted-by: Claude:claude-opus-5-5 * Expect the chat's own load_skill tool in the agent prompt spec The spec that keeps the agent engine's text out of the prompt also required that no load_skill tool or mention reach the model. Chats now offer their own load_skill tool and the skills index tells the model to call it, so the spec failed. Keep the checks on the engine's own skill and command text, and check that the chat's tool is offered. Assisted-by: Claude:claude-opus-5-5 * Read the user's scripted tool call in the mock model The chat prompt now names the chat's work folder, and the folder name comes from the chat title, which holds a cut copy of the user message. The mock model read the first ADDON_TOOL_CALL marker, the cut one in the system prompt, and called the tool with broken arguments over and over. Read the last marker, the one of the user message. Assisted-by: Claude:claude-opus-5-5 * Keep the phone targets of the inbox at 44 pixels in every build On a phone the inbox close button and the item actions must be 44 pixels wide. Their rule had the same weight as the kit's .dk-btn min-width, so the order of the stylesheets decided. The single-chunk coverage build puts the kit later and the close button shrank to 28 pixels. Scope both rules to their container so they win in any order. Assisted-by: Claude:claude-opus-5-5 * Let the about-you spec accept the sidebar's own seen date When the user has pages, the sidebar writes the date of its New label under seen.new in user.json. In the whole suite other specs make pages first, so the saved record held that date as well and the exact comparison failed. Check that the uploaded seen record is kept. Assisted-by: Claude:claude-opus-5-5 * Give the typed-while-working chat spec time for two turns The spec waits up to 30 seconds for each step, but two turns of the mock model with its tool steps take about 28 seconds on their own, so the default 30 second test limit cut it under load. Set its limit to 90 seconds. Assisted-by: Claude:claude-opus-5-5 * Scope the foundation spec's state query to the sheet header The details pane now repeats the observed state under Health, so a role query for "State: Off" inside the add-on sheet matched two elements and failed in strict mode. The administrator-manages spec, and the two specs after it, did not run. Query the chip in the header (.ad-pane__meta), which is the state the administrator sees next to the add-on's name, through one helper used for all three assertions. Assisted-by: Claude:claude-sonnet-5-5 * Wait for the page to read each apps answer in the registry polling spec The spec moved the page clock as soon as the stand-in saw a request. The registry sets its next timer only after it reads the answer, so on a slow runner the clock could pass the due time of a timer that did not exist yet, and the next read never came. With the CPU throttled ten times, the old spec failed in 3 of 3 runs. The spec now counts the answers the page has read and lets queued work finish before it moves the clock. It pauses the clock before it leaves the page, and it tries the pause again when a slow machine passed the target time. The assertions do not change: one read at once, one read per interval and no faster, and no read after the disable. Assisted-by: Claude:claude-opus-5-5 * Close the egress test listener only after it accepts The dial helper of the egress proxy specs closed its listener when it returned, while the goroutine that accepts the connection could still be waiting to run. On a loaded runner the listener sometimes closed first. The kernel then reset the connection that was still in the accept queue, the spec's write consumed the reset error, which the spec ignores, and the next read saw EOF. The spec failed with "unexpected EOF" before the proxy got the request. Wait for Accept to return before the listener closes, then start the proxy handler on the accepted connection. Assisted-by: Claude:claude-opus-5-5 * Run host commands in a shell that reads no startup file The host backend promises a fixed environment, but bash -lc reads /etc/profile and the user's dotfiles. On macOS path_helper rewrote PATH there, so the command did not get the fixed PATH. Turn the login shell into 'bash --noprofile --norc -c' in the host backend only, and make the test assert the exact fixed PATH. Assisted-by: Claude:claude-sonnet-5-5 * Answer the command card in the platform turns and 400 a bad answer The tool-list spec of the platform suite timed out: without accounts the one person is an administrator, so the chat has the built-in bash in ask mode. The tool list is sorted and bash is first, and the mock model calls the first tool, so every turn waited five minutes for a yes on the command card. The turn helper now refuses each card it sees, as a person who clicks deny, and the turn goes on to its step limit. bash stays in the checked list. An administrator who posts a consent with no card id or an unknown decision got 403 "only an administrator can allow commands". That is a bad request, not a refusal: it now answers 400, as the swagger already says, and the route check for administrators passes. Assisted-by: Claude:claude-opus-5-5 * Let a chat switch off its shell, file, media, collection and sub-agent tools The Tools menu of a chat switched only three groups of built-in tools: web, saved_agents and triggers. A person could not keep a chat away from the shell, the files, the helper models, the collections or the sub-agents. Add five groups to the registry of built-in tools, all on until the chat switches them off: - shell: bash, whoever serves it. The Sandbox add-on's bash and the built-in bash on this machine both go, and the add-on list reports commands switched_off. The row stays listed when nothing can run commands, not usable, with the reason. - files: the file tools and read_document. - media: describe_image and transcribe_audio. - collections: list_collections and search_collection, usable only for a user with the collections feature. - subagents: spawn_agent. list_agents and /agent follow it. The registry gets two flags. External marks a tool that its own code builds for the turn (with the wrappers it needs); the registry lists it in its group and Hidden tells the builder what the chat switched off. Covers makes a group's switch also hide the add-on tool that replaces the built-in one. PUT .../builtins, the panel, the draft list and the fallbacks use the existing group code. A sub-agent follows the switches of its parent chat. The turn prompt is built from the final tool list: no file tool names, no skill writing hint, no work folder and no rules pointer when the chat cannot use files, no describe_image when the helper is off, and a commands block that says the person switched commands off. Assisted-by: Claude:claude-opus-5-5 * Queue the edited chat message after the first step starts The spec queued its message as soon as the chat showed a busy state. The turn is busy before its first model request starts, and that request takes every message that waits. With the chat's tools and skills, the first request now starts a few hundred milliseconds after the turn, so the message went with it, and the edit got 409 "already delivered" (5 of 6 local runs). Wait until the answer streams, as the Stop spec does: the request is running then, and the queued message waits for the next step. Assisted-by: Claude:claude-opus-5-5 * Give the reload chat spec a turn that outlasts the reload The spec reloaded the page during a 3 second turn and expected the page to show the turn still running. A reload of the coverage build on a busy runner takes 4 to 5 seconds (measured with a 6x CPU throttle), so the turn had ended before the page could show it. With the 30 second turn the page always showed it running, so the resumed state works and only the window was short. Add MOCK_LONG_STREAM_MEDIUM to the mock model, a turn of about 10 seconds, and use it in the spec with a 60 second limit. Assisted-by: Claude:claude-opus-5-5 * Wait for the chat page after the Home hand-off in the specs Send on Home changes the address inside a view transition, and the chat page mounts about 35 ms later. Specs that waited only for the address acted on Home in between: - The title spec pressed Ctrl+K at once, because the server names a chat after its first message as it arrives, so the title poll ends at once too. The key reached the page before the chat page listened, and the menu did not open (3 of 3 presses at 0 ms failed, all at 300 ms and later passed). - The Back spec went back before the chat page was drawn. Home was never closed, so its draft of the sent message stayed in the box. startChat now waits for the chat page, the title spec for the title in the chat header, and the Back spec for the chat page before Back. Assisted-by: Claude:claude-opus-5-5 * Accept a resolved user.json conflict in the quiet page specs The sidebar notes in the seen record of user.json the first day it labels Pages as new, and saves it from the version it read. All specs share the test server's one user, so tabs of other workers save the same record at the same time: one gets 409 and useSeen reads the file again and saves once more. Three tabs that open /app/files at once give 200, 409, 200. The quiet check of the shell pages counted the 409 as a problem. Ignore a 409 on PUT /api/chats/wellknown/user in that check. Assisted-by: Claude:claude-opus-5-5 * Check only the spec's own items in the Trash after the reload After Empty Trash and a reload, the spec expected the whole Trash to be empty. The Trash belongs to the test server's one user, and specs of other workers move files to it meanwhile, so CI saw "In the Trash: 3 files, 356 B." after the reload. The checks right after Empty Trash stay. After the reload, check that the server's Trash, the Trash tab and Files do not hold the spec's items, once each list has loaded. Assisted-by: Claude:claude-opus-5-5 * Show the new tool groups in the chat Tools menu The Tools menu lists the groups of built-in tools that the server sends. The five new ones need words and an order, and the shell row needs to say who runs the commands. Name the groups in all nine locales (Run commands, Your files, Images and audio, Collections, Sub-agents) and order the rows by what they reach: the shell and the files first, the triggers last. The shell row carries a chip with the provider (Sandbox or This machine) and a line under it that says the commands run isolated, or a warning when they run on this machine. When nothing can run commands the row is disabled with the words of the commands notice. A switched-off sub-agents group also turns /agent off in the composer. The specs cover the rows, the off and on requests, the bulk actions, the provider chip, the disabled reasons, the draft on Home and a real server round trip. Assisted-by: Claude:claude-opus-5-5 * Document the groups of built-in tools a chat can switch off The Tools menu section described only the web, triggers and saved agents groups. List every group with its tools, say which groups the server may leave out and why a row can be not usable, how Run commands covers both providers, what the model is told when a group is off, and that a sub-agent follows its parent's switches. Assisted-by: Claude:claude-opus-5-5 * Fix the host command consent spec for the 400 contract The consent route answers 400 for a body without a card id or with a decision other than once or deny, whoever asks. The spec still expected 403 for always from an administrator. Cover the malformed answers for an administrator and a user, each in their own chat, keep another user's chat at 404, and say it in the chats guide. Assisted-by: Claude:claude-sonnet-5-5 * Find the web tool group by name in the route spec The tools panel lists the shell group before the web group, so the spec that read the first entry compared the wrong group. Look the group up by name, which does not depend on the registry order. Assisted-by: Claude:claude-sonnet-5-5 * Give Skills its own sidebar item Skills work in every chat, but the sidebar filed the page under Agents and only agent skill chips linked to it. Add a Skills item right after Memory, show it in the legacy list too, and stop lighting up Agents on the skills pages. Assisted-by: Claude:claude-sonnet-5-5 * Add Manage skills to the skills picker and a /skills command The picker now ends with a Manage skills link, and /skills opens the Skills page from the composer on Home and in a chat. The picker keeps the height of its status line, so a refused change does not move the rows. Words are added in all nine locales. Assisted-by: Claude:claude-sonnet-5-5 * Say on the Skills page that every chat can use skills The page subtitle named only agents as users of skills. Say that all chats use them, translate it, and document where to manage skills: the sidebar item, the picker link and /skills. Assisted-by: Claude:claude-sonnet-5-5 * Keep Home on screen while its widgets load their strings A Home widget reads the pages strings, and i18next loads a namespace the first time a component asks for it, by suspending. Home had no Suspense boundary of its own, so the route's boundary hid all of Home (display: none) and showed the route fallback until the strings came. A widget shows only after home.json is read, so this happened after the composer was already drawn. Focus given to the composer in that window went nowhere, and a key typed then was lost. The slash menu spec failed in CI this way when another spec had written widgets to the shared home.json: the box was empty and the body had the focus. With widgets in home.json the composer was hidden right after it appeared on 5 of 5 loads, and the spec failed 1 or 2 times in 5. The examples and the widgets now have their own boundary with no fallback, so only they wait. A new spec holds the pages strings while it types in the composer; it fails without the change (the composer never takes the focus) and passes with it. Assisted-by: Claude:claude-opus-5-5 * Send the stubbed file change after the Files page read the folder The change feed spec answered the first stream request with the change at once. The app shell (the appearance sync) opens the shared stream when the app starts, before the Files page chunk has loaded, so on a slow runner the change reached a tab where nobody watched the folder yet and was dropped. The page then never read the folder again, and the spec failed on all three CI attempts. A CPU throttle of 8 makes it fail 3 of 3 times locally. On a real server the page's own first read would already show such a change. The stub now holds the change until the page has read the folder; the page subscribes in the same commit as that read, so the change arrives with a subscriber in place. The resume check is kept. Assisted-by: Claude:claude-opus-5-5 * Give the oversized 3D input as a sparse file, not a buffer The spec built a 32 MiB buffer and passed it to setInputFiles. Playwright sends a buffer to the browser over the protocol: that step alone took 5.5 s on an idle machine, and on the loaded CI runner the test hit the 30 s timeout on all three attempts. It now writes a sparse file of the same size to the test output folder and passes its path, which the local browser opens as is. The page still checks the size before it reads the file, so the toast and the no-request assertion are unchanged. The step takes milliseconds. Assisted-by: Claude:claude-opus-5-5 * Check the tool line, not the tool name, in the tool call journey The spec looked for /list_files/ in the transcript. The user's own message holds that name, so the check could pass before any tool ran. Once the model's call streamed, a second element matched, and the strict locator failed (2 matches on the first attempt, 5 on the retry). It now waits for the activity line that the tool run adds ("Listed /"), which only shows after the tool ran. Assisted-by: Claude:claude-opus-5-5 * Measure the proposal card and its neighbours in one frame The layout spec read the boxes of the message text, the card and the message actions with three boundingBox calls. They are viewport positions taken in three frames, and the transcript scrolls to its end while the card fills in. On the phone the actions were read after a scroll and came out 156 px above the bottom of the card. The three boxes now come from one page.evaluate, so they are read in the same frame. The order assertions are unchanged. Assisted-by: Claude:claude-opus-5-5 * Read the page fixture time when the page is served The task board fixture of the discovery and Pages specs set its change time once, when the file loaded. A worker loads it once and keeps running specs, so a spec that ran two minutes later read "Changed 2 minutes ago by the assistant" where it expects "just now". The change time is now a getter, read when the fixture is serialized into a response. Assisted-by: Claude:claude-opus-5-5 * Check a background process without /proc in the host test The alive helper read /proc/<pid>/stat, which macOS does not have, so it reported every process as dead and the spec failed there with "did not survive the suspend". Use signal 0 for existence and ps (or /proc where present) to tell a zombie from a live process. Assisted-by: Claude:claude-sonnet-5-5 * Keep the composer switch menus at their own height The base rule of the dropdown menu caps it at 320px and lets it scroll. The switch menus set their own height with a rule of the same weight in a lazy sheet, so which one won depended on the order the sheets loaded in. When the base rule won, the picker showed a few rows, and a click on a row at the edge of the list scrolled the list by a few pixels. Double the class on the switch menu rules so they win in any order, and add a test that loads the base rule last. Assisted-by: Claude:claude-sonnet-5-5 * Show what applies in the chat Network switch The Network row highlighted the choice stored with the chat, so a chat that asked for All public sites under an administrator limit of Off showed All highlighted although it was disabled and not in effect. Highlight the effective choice, the lower of the chat's choice and the limit, which the server already computes. When the chat asks for more than the limit, say so in one line instead of highlighting it. Add a line that the limit is set by the administrator, and tell the host network line that limits need the Sandbox add-on. Assisted-by: Claude:claude-sonnet-5-5 * Design the host components that add-ons reference The host UI names the Sandbox and the Web add-ons to place their server settings, and the chat's Network row links to the global Settings page. Describe a closed registry of host components that an add-on references by ID in its manifest, entitled by the existing host operation grants and tool shadows, so the controls live on the add-on's own page and a second add-on can reuse them with no host change. Assisted-by: Claude:claude-opus-5-5 * Let an add-on ask for host controls in its manifest An add-on had no way to say that a host-owned control belongs on its page, so the host UI placed those controls by add-on name. Add ui.controls, a list of IDs from a closed registry of host controls: network-policy (the network of the commands an add-on runs) and tool-providers (the rows of the built-in tools it replaces). The host decides which controls it draws: EntitledControls keeps a control only when the add-on holds its host operation grant and replaces its built-in tool. The field is optional, so every existing manifest stays valid. Assisted-by: Claude:claude-opus-5-5 * List the host controls an add-on is entitled to The apps list now carries ui.controls with the host controls the manifest declares and the host setting entitles the add-on to: the manager passes each add-on's host operation grants to the list. GET /api/chats/{id}/egress gains addon, the add-on that runs the chat's commands when it has the network-policy control, so the chat can draw its Network row from that add-on and link to its settings. applies and applies_reason keep their meaning: the policy holds with or without the control. The model and the 400 error no longer send an administrator to Settings > Tools for the limit. Assisted-by: Claude:claude-opus-5-5 * Draw the host controls on the add-on's own page The host UI placed the Sandbox network and the tool rows by add-on name (SERVER_SETTINGS), showed a read-only summary on the add-on's Settings tab, and linked the chat's Network row to the global Settings page. Add a registry of host controls keyed by the IDs of the contract. The Settings tab of an add-on draws, below its own sections, the controls the apps list gives it, in one form with the pending bar of the Settings page: network-policy reuses the network section and its setting rows, tool-providers the rows of the built-in tools the add-on replaces. The chat draws its Network row only when the server names an add-on with network-policy, and the administrator link opens that add-on's tab. The row itself is unchanged. The Settings page keeps the settings of features that exist without an add-on, points to the add-on where one serves them, and no longer lists the network settings. Its old address opens the add-on's tab. Assisted-by: Claude:claude-opus-5-5 * Ask for the host controls in the Sandbox and Web manifests The Sandbox asks for network-policy and tool-providers, so the network of its commands and the bash row are drawn on its own Settings tab and its chats get the Network row. The Web add-on asks for tool-providers only: it holds no egress grant. The manifest tests check that the default grants entitle each add-on to what it asks for. The proxy's refusal now sends the model to the Sandbox Settings tab instead of Settings > Tools. Assisted-by: Claude:claude-opus-5-5 * Keep the button of the network pointer row at its size The row used the wide layout of the tool rows, so its Open settings button stretched across the page. Use the plain setting row, which puts the button on the right like the other controls. Assisted-by: Claude:claude-opus-5-5 * Fix the add-on contract inventory and the apps Swagger fixture The contracts case inventory missed the six host control specs, so verify.sh rejected the Ginkgo report. List them. The Swagger spec test never saw `controls` in the emitted UIView JSON because its fixture left the omitempty field empty. Set it there. Assisted-by: Claude:claude-sonnet-5-5 * Add the host controls spec to the apps inventory The apps suite ran one more spec than scripts/addons/suites.json listed, so verify.sh rejected the apps report as an inventory mismatch. List the new spec in the report inventory and in the suite's aggregate case list. Every other Ginkgo suite was re-run and matches its inventory. Assisted-by: Claude:claude-sonnet-5-5 --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-11-01 12:08:23 +01:00
mudler-agentandEttore Di Giacinto	7c74d26555	feat(addons): run the add-on owner as its own workload (#12867) ... * feat(addons): generate and load a static carrier key set A split deployment needs the owner and the frontends to share certificates and a signing key that outlive a process. The loopback carrier makes fresh keys at every start, so it cannot serve that. GenerateCarrierKeys writes a CA certificate, an owner leaf and a frontend leaf, and the frontend signing key pair. It drops the CA key, so nobody can issue a third leaf, and it refuses to overwrite any file. LoadCarrierConfig reads only the files of one role and refuses key files that group or other users can access. The owner never forwards, but NewCarrier still requires a signing key, so the owner config gets a throwaway key in memory instead of the frontends' private key. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): add util addons-carrier-keys An operator who runs the add-on owner and its frontends as separate workloads needs a way to make the shared key set. The command wraps GenerateCarrierKeys: --out names the directory and --owner-host, which can repeat, sets the names the owner certificate is valid for. It prints which files belong on the owner and which on the frontends. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): add the owner and frontend roles to local-ai run Every local-ai run process is its own add-on owner, so a distributed deployment cannot scale its frontends: replicas that share a data volume fight over the owner lock. --addons-role selects combined (today's behaviour, the default), owner or frontend. The owner and frontend roles also take --addons-owner-address and --addons-carrier-dir, and the owner takes --addons-owner-listen. They use the fixed carrier identities and the key set from the carrier key directory. The frontend role has no add-on root, so it never creates the add-ons folder. A bad role setup stops startup with a message that names the flag. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): keep the frontend role away from owner state A frontend holds no add-on state, but it still wrote the library sources file of the data path, which is the owner's file. The frontend role now skips the library option and reads only the galleries named by the flags, and logs a warning when it has no account database, because its pins are then kept per frontend. The add-on manager rejects a relative root even for a forwarding frontend, so the frontend config names the absolute path where an owner would keep its state. Nothing creates or opens it, and a spec checks that the data path stays empty. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(addons): run the owner and frontends as separate applications The split roles are only useful if an owner and its frontends work as separate processes that share nothing but the carrier keys and the owner address. A spec now builds them as separate Application values with their own data paths and keys from GenerateCarrierKeys. It covers a frontend that lists apps and reads add-on status through the owner, two frontends on one key set, a frontend that starts while the owner is down and answers unavailable, a second owner refused on the same root, and an owner that restarts on the same root and address and still serves the installed add-on to the same frontend. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(addons): describe running the owner as its own workload An operator who runs several frontend replicas had no documented way to stop each one from being its own add-on owner. The add-ons page now has a section for the owner and frontend roles: one owner replica with its own volume and a recreate-style rollout, a stable owner address, key generation and per-role mounts, a Compose example, what a restart and an owner outage look like, the models-path limit, and the frontend limits. The singleton section names the roles and the distributed-state mode, and the distributed mode page links to the new section. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): gate the addons feature in /api/features One function, Application.AddonsFeature, decides whether the add-on side is on. A combined or owner process answers true when add-ons are enabled and the owner manager exists. A frontend answers true when the owner answered the carrier's audience call within 2 s, and keeps that answer for 10 s so /api/features does not dial the owner per request. /api/features reports the result as "addons", and fine_tuning and quantization take the same value, because both are add-ons. The frontend cache is per-instance state. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): answer 503 on every add-on route when add-ons are off With --addons-enabled=false the real add-on, fine-tuning and quantization routes and services are not registered. One catch-all per prefix answers the typed unavailable response of an absent owner, for any path under it, so a client cannot tell which routes exist. The response is the one the gateway already writes (gateway.WriteUnavailable over typedError), not a second format. With add-ons on, the same routes sit behind RequireAddonsFeature, which reads Application.AddonsFeature, so a frontend that cannot reach the owner gives the same answer. The per-user feature middleware still runs after the gate. No endpoint is added, so the swagger, /api/instructions and RouteFeatureRegistry surfaces do not change. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): build no carrier for a frontend with add-ons off addonConfigFor already returned the zero config before it read a role flag when add-ons are disabled. Say so on the way out: when a role, owner address, listen address or carrier directory flag is set, log one info line that the flags are ignored. A frontend with --addons-enabled=false needs no owner address and no key directory, loads no keys and builds no carrier. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): hide add-on entries when the addons feature is off The UI reads features.addons from /api/features. When it is false: the sidebar shows no Add-ons entry and no pinned apps, the 404 page lists no add-on destination, the shell does not call /api/addons/pins or /api/addons/builtin, and /app/addons, /app/addons/*, /app/fine-tune and /app/quantize lead back to /app. The Build hub tabs for add-ons, fine-tuning and quantization were already keyed on features and now follow the gate. An absent key (an older server) counts as on. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(addons): describe the addons feature gate The --addons-enabled row now says the UI hides the add-on entries and every add-on, fine-tuning and quantization route answers 503. A new subsection describes the addons key of /api/features, the 10 s frontend cache as per-instance state, and that a frontend with add-ons off needs no owner address or key directory. The fine-tuning and quantization pages say they are off with --addons-enabled=false. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): give a frontend no add-on state root OwnerConfig.validate required an absolute root for every role, so a frontend named a path under the data directory that nothing was allowed to open. Only the owner has a state root now: validate checks it when the instance is the owner, and the frontend role leaves Root empty. Nothing on a frontend reads or creates it. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): make the carrier signing key optional for the owner NewCarrier required an Ed25519 signing key of every instance, so the owner role loaded a throwaway key in memory that it never used. The owner never signs an outbound request, so it now needs none: NewCarrier accepts an empty key when the instance is the owner and still requires a whole key from a frontend, and a malformed key is refused for both. LoadCarrierConfig no longer generates one. The owner still verifies frontend signatures with frontend-sign.pub. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Keep the owner address out of the add-on identity file The owner address is runtime configuration, so storing it and comparing the whole file made a moved owner refuse its own root. The identity file now holds the owner ID and storage ID only. An older file that still has an address loads, and the address is ignored. A different owner ID or storage ID is still refused. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(addons): state that an existing root loads under the owner role The owner identity no longer stores its address, so a root written by a single node loads under the owner role when the owner and storage identity match. Say so, and keep the caveat about absolute paths in the journal. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): refuse library-source routes on a frontend A frontend must keep no owner state, but the library-source routes, the ApplyLibraries call and the startup official-libraries step still opened and wrote addons/sources.json in its data path. Mark the frontend role in the add-on config, also when add-ons are off. On a frontend, LibrarySources returns the unavailable error, ApplyLibraries does nothing and the startup step skips the store. The six /api/library/sources routes answer 503 with the body that the add-on feature gate writes, and an X-LocalAI-Reason header names the cause. Reads are refused too: the carrier has no verb for them. Owner and combined roles are unchanged. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(addons): put the reason in the unavailable body WriteUnavailable wrote the plain text "addon request failed", and the library-source refusal put its reason in an X-LocalAI-Reason header. Write the LocalAI error envelope instead, with a message, a type and the code "unavailable", and keep the 503 status. The message names the cause: add-ons are off on this instance, the add-on owner is not reachable, or library sources are managed on the owner. Remove the header. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): serve library calls on the add-on owner Add the carrier verbs a frontend needs to work on the library through the owner: library-list, library-add, library-update, library-enable, library-disable, library-remove, library-backend-resolve and library-backend-releases. A call carries the operation in one LibraryCall field of the wire request, so an old owner rejects it as an invalid binding. Every verb but the three reads is a mutation, and the reply is an envelope of data or a typed error, so a frontend gives the status the owner's own routes give. The owner serves them through gateway.LibraryAdmin, implemented in the application on the same LocalAdmin as the owner's HTTP routes. It runs the owner's administrator check before any store call. Add two peer-authenticated reads that take no actor and leave the audience reply unchanged: GET /addons/capabilities (protocol, calls and the owner's OS and architecture) and GET /addons/library/snapshot (the gallery-relevant source list and its version). Split the backend install: Installer.Resolve is the owner's half (resolve, pick the meta backend for the asker's capability, pin by digest) and returns a Plan; the node that installs runs the collision check and the gallery install itself. Move the source administration behind library.Admin and add library.ReconcileInfos for a source list read from the owner. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): forward library routes from a frontend to the owner A frontend refused the library-source routes and did not serve the backend library routes. Forward them instead, and keep the owner the only writer of the sources. The library-source routes work on library.Admin: the local store on a combined or owner process, the carrier on a frontend. The frontend checks the calling account (a legacy API key is refused with 401), and the owner checks the administrator right again before any store call. With add-ons off, a frontend answers these routes with a 503. A frontend asks the owner for its source list every 60 seconds through the peer-authenticated snapshot call and rebuilds its model and backend gallery lists, and the official-libraries switch, only when the version changed. If the owner does not answer, it keeps the last lists and marks them stale. Changing official_libraries on a frontend is refused with 409. The backend library install is split. The owner resolves the entry and pins the image; the frontend runs the gallery install on its own node, with its own size limit and strict mode. A frontend refuses library calls to an owner that does not serve them, or that runs on another OS or architecture. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(addons): document library operations on a frontend Replace the frontend limits that said a frontend refuses the library routes with what happens now: the library sources and the backend library go through the owner, the gallery lists follow the owner every 60 seconds, official_libraries is set on the owner, and the owner and the frontends must run on one OS and architecture. Add specs for the split owner: a frontend lists sources through the owner, a source added on a frontend reaches the owner and then the other frontend after a refresh, a write with the owner down answers 503 with the error envelope, official_libraries answers 409 on a frontend and a change on the owner reaches the frontend's gallery lists, the lists are kept and marked stale while the owner is down, a platform mismatch and a legacy API key are refused, and a frontend installs a backend into its own path from the owner's plan. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): check the shared models path with a probe Exports write into the models path of the owner, and frontends load models from their own path. A wrong mount fails silently: the export succeeds and the model never appears. The owner writes a probe file with a random nonce into its models path at start. A frontend fetches the nonce over the carrier and checks at start, then every 60 seconds, that the same file is visible at the same absolute path. On a mismatch it logs an error, reports model_exports false and blocks its export and import routes. The owner refuses an export or import while its own probe is missing, and keeps running if it cannot write the probe. An unreachable or older owner gives "unknown" and leaves exports on. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(api): describe the library refusal and forwarding The library-source and backend-library routes forward to the add-on owner on a frontend and answer 503 when the owner cannot serve them. Add the 503 to their swagger annotations, regenerate the swagger files, and say in /api/instructions that a frontend forwards these routes. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): let frontends run the backend update check through the owner In named-user mode a frontend has no user to act as, so the background backend update check failed there. Add one read-only system call for the release lookup. The owner accepts it only on the peer-authenticated route from a pinned frontend, only for backend-releases, and without an actor. Calls through /addons/forward cannot carry the mark, and every user-facing call still goes through the administrator check. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(mcp): expose the backend update check as a tool Add list_backend_library_updates, a read-only tool for GET /backends/library/updates. The in-process client reads the library installer, and the HTTP client calls the route. The six library source routes stay out of the catalog: the assistant has no per-call actor and no approval card for library changes. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(api): add export status routes for fine-tuning and quantization The add-on UIs could not know that model exports are blocked until a user hit the refusal of the export route. Add GET /api/fine-tuning/status and GET /api/quantization/status. Each returns {"model_exports": bool}, from the same check that drives the model_exports key of /api/features. The routes sit behind the add-ons gate, so they answer 503 when add-ons are off. They are listed in the swagger spec, the instructions registry, the route-feature registry and the feature docs. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(addons): disable exports when the models probe fails The fine-tuning and quantization pages now read the new status route and turn off the export form, the checkpoint export button and the import button when model_exports is false. A short reason shows next to the control. A failed or slow answer leaves the controls on, as the server does for an unknown check. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-10-24 14:29:39 +02:00
Plamen K. Kosseff	7b9ef6a0f8	feat(backends): --disable-backend-install disables backend downloads. (#12754) ... feat(backends): --disable-backend-install refuses every backend acquisition A hardening switch for deployments that must run exactly the backend set they were tested with (production, air-gapped, immutable images, package-managed hosts). With LOCALAI_DISABLE_BACKEND_INSTALL=1 every install and upgrade path refuses — gallery, OCI/URL/path, node-scoped, auto-upgrade, boot-time external backends — HTTP routes answer 403 up front, and a model whose backend is missing is refused before any download: 409 on the API, an error on preload and import. Whether a backend is present is asked of the execution target: the local backend directory on a single node, the workers' listings on a distributed frontend, which installs nothing itself. Backends already present, including system-path ones, keep working. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: Plamen K. Kosseff <p.kosseff@gmail.com>	2026-10-22 17:58:53 +02:00
Stefan Walcz	b56b903c92	feat(config): let model aliases carry an opt-in profile block (system prompt, MCP, client tools) (#12661) ... * feat(config): let model aliases carry an opt-in profile block An alias can now act as a profile of its target through an explicit `profile:` block carrying system_prompt, mcp (remote/stdio servers, plus the new mcp.auto) and the new client_tools. ResolveAlias applies only that block on top of a copy of the target, while Name, Model, Backend and the persisted revision stay the target's. The loader key is therefore unchanged and every profile runs on the target's backend process; today a persona needs a separate model config, which loads the weights a second time. The block is opt-in so existing aliases never change: top-level fields on an alias stay ignored, and an alias without a profile block resolves to exactly the target. A profile block on a non-alias config is rejected. A profile with its own MCP servers caches its sessions under the alias name (MCPSessionKey), because the cache slot keyed on the target's name is invalidated whenever its config hash changes and two server sets would tear each other's sessions down. mcp.auto enables all configured servers on /v1/chat/completions when the request does not name any in metadata.mcp_servers, which plain OpenAI clients cannot send. client_tools: drop discards client tools and tool_choice before the request is merged (chat completions, responses, Anthropic messages), so only the model's own MCP tools are offered. Both keys work on any model and inside an alias's profile block. Refs #12548 Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(openai): only auto-enable model MCP when MCP is enabled and permitted mcp.auto turns a plain /v1/chat/completions request into a server-side MCP request. It now applies the same controls as the dedicated MCP routes before any server is selected: the instance switch (LOCALAI_DISABLE_MCP) and, with auth on, the caller's "mcp" feature, checked with auth.HasFeatureAccess against the auth database that the chat route now passes to ChatEndpoint. A denied request is answered as a plain chat, without contacting a server or adding tools, rather than rejected: the client did not ask for MCP, it only reached a model that offers it. Endpoint specs drive ChatEndpoint against a counting MCP fixture server: MCP disabled and a user with chat but without mcp see no tools and no connection; MCP enabled and a user with mcp get the fixture tool. The user specs need the auth build tag (SQLite auth database). Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * fix(mcp): resolve profile aliases in the MCP discovery endpoints GET /v1/mcp/servers/:model, /v1/mcp/prompts/:model and /v1/mcp/resources/:model looked the model up with GetModelConfig, which returns the alias stub. A profile alias carries its servers in its profile block, so the stub has none of its own and the endpoints reported an empty list for an alias whose chat requests do use tools. Resolve the alias the way the request middleware does before reading the MCP config, and open the sessions under MCPSessionKey so discovery shares the profile's session cache slot instead of tearing down the target's. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> * feat(config): opt-in evidence check for profile aliases A profile with server-side MCP tools (profile.mcp, mcp.auto) runs its tools on the server and returns only the answer text. On the next turn the client sends the history back without those tool calls and results, and the model then tends to skip the tool and answer from nothing. In a measurement on a follow-up question, 0 of 12 turns called the tool and the answers carried invented numbers (e.g. "0" where the true value was 9); with the tool trail kept in the history the tool was called 12 of 12 times. A prompt instruction does not reliably prevent this, so the check sits at the output. New profile keys: profile: evidence_check: numbers: true # enables the check note: "..." # optional, defaults to an English note Rule, per request and stateless: when the check is on, no MCP tool call succeeded on the server during this request (a call that returned an error delivers no evidence and does not count), and the final answer contains a number (\b\d+([.,]\d+)?\b) that occurs in none of the request's input (the profile's system prompt and every message as received, before compression), the note is appended after a blank line. "3,5" and "3.5" count as the same number. In a streamed response the note is one extra content delta before the finish chunk. Choices that hand tool calls to the client are not checked. The setting is only read from a profile block, so plain models and aliases behave as before; a note without numbers: true is rejected at validation. Tests (Ginkgo): the number rule (invented number, number from the input, no numbers, comma/dot decimals, system prompt as evidence, tool ran), the overlay into the resolved config with plain alias, target and a top-level key as counter-checks, validation, and ChatEndpoint specs on a stubbed backend for both paths: note on an invented number, none for a number from the request, none for a profile without the check, none after a successful MCP tool call against an in-process MCP server, and the note again after a failed tool call (arguments that are not JSON). Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de> --------- Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>	2026-10-19 23:42:40 +02:00
mudler-agentandEttore Di Giacinto	c7ce0c602c	fix: stop listing the models directory on request paths and cache GGUF header reads (#12683) ... * fix(http): stop listing the models directory on embeddings and bearer requests Every embeddings request read the whole models directory before the body was parsed, even when the body named its model. On a network-backed models directory this cost seconds per request. Embeddings now binds the body first and looks up a default model only when the body has none, like chat. The pre-bind default lookup and the gpt-4o constant default are gone from that route. The first-available default lookup skips the directory when the filter cannot accept an unconfigured file. Every usecase filter rejects files without a config, so the scan never added results for them. Match-all, nil and name filters keep the scan. The bearer token check is now an exact test: a configured name, or one stat of that name in the models directory. The value is a literal, not a regular expression, and a name that is not a local relative path never matches a file. Assisted-by: Claude Code:claude-haiku-5-5 * perf(config): cache GGUF header reads for model defaults Model defaults re-parsed the GGUF header on every SetDefaults call, which runs per request and at startup reload. Put a bounded LRU in front of the parser, keyed by absolute path and validated by size and modification time. The cached value is a snapshot of the fields the defaults use, not the parser output. A changed size or mtime is a miss. Missing files and parse errors are not cached, so the existing fallback is unchanged. LOCALAI_DISABLE_GUESSING, the empty-path guard, the alias early return, the panic recovery and the context-size default are all kept in front of the cache. Assisted-by: Claude:claude-haiku-5-5 * test(http): cover listing-free default lookup on every usecase route Assisted-by: Claude Code:claude-haiku-5-5 * feat(config): persist the GGUF header cache per instance The header cache kept parsed GGUF results only in memory, so every restart parsed each model file again before its defaults were known. Write the cache to a per-instance file in the data path and load it at startup. A restarted frontend with a fixed instance ID starts warm. The file is versioned and records the estimator inputs (gguf-parser version, candidate contexts, auto context size, offload count). A mismatch on any of them makes the file ignored. Each entry keeps path, size and mtime, so a changed model file is a miss on load. Writes run on one background saver, debounced, through a temp file and rename. The entry bound applies to the file as well. Without a data path the cache stays in memory only. Documented as a per-instance exception in the model configuration docs. Assisted-by: Claude:claude-haiku-5-5 * docs(embeddings): document model selection and the no-model error Assisted-by: Claude Code:claude-haiku-5-5 * refactor(config): split the header cache into a generic cache The GGUF header cache mixed two things: the mechanics (bounded LRU keyed by path and identity, per-instance file, debounced atomic save, tolerant load) and the GGUF fields. Move the mechanics into headerCache[S], driven by a headerCacheSpec[S] that names the file prefix, schema, fingerprint, parser and the snapshot encode/decode pair. The GGUF cache becomes one instance of it. File names, the identity check, LRU size, debounce, atomic write and the corruption handling are unchanged. The on-disk JSON keeps the "estimator" key so existing files still load. Assisted-by: Claude Code:claude-haiku-5-5 * feat(config): share one GGUF header cache file between frontends Every frontend used to write its own header cache file, named after its instance ID, so a header read by one frontend was never reused by another. The cache now uses one file, header-cache.json, in the data path, for all frontends and for standalone mode. The instance ID no longer changes the file name. Frontends share the file in three ways. A save first merges the file's entries in, so one frontend's write does not drop another's entries. A miss reloads the file when its modification time has changed, and a hit never reads it. Writes go to a temp file that is renamed into place, so a reader never sees a partial file. Two frontends saving at the same moment can still drop each other's last entries. That costs one re-parse, never a wrong value, because every entry is checked against its model file before use. The shared file is valid only when every frontend mounts the same data volume. The distributed-mode and model-configuration docs state this mode and its volume limit. Assisted-by: Claude:claude-haiku-5-5 * feat(config): give an instance ID its own header cache file Assisted-by: Claude Code:claude-haiku-5-5 * style(config): gofmt the distributed config struct Assisted-by: Claude Code:claude-haiku-5-5 --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-10-14 01:06:14 +02:00
Ettore Di Giacinto	db929f4a08	test(config): assert every usecase flag is a distinct bit ... FLAG_MOTION and FLAG_UPSCALE were both declared as 1 << 26 on separate branches, and a merge combined the two const blocks without a conflict. Add a spec that checks every flag from GetAllModelConfigUsecases() is a single bit that no other flag uses, so a collision fails the tests. FLAG_MOTION itself moves to 1 << 27. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-10-13 08:27:24 +00:00
Richard Palethorpe	3de91d48d9	feat(motion): add GEM-X backend and duplex streaming API ... Add the GEM-X native backend with live pose and root displacement output, bounded duplex WebSocket streaming, session authorization, metrics and usage accounting. Share the embedded bidirectional RPC channel adapter across streaming backends. Include backend build and gallery integration, API documentation, schemas and tests. Assisted-by: Codex:gpt-6 Signed-off-by: Richard Palethorpe <io@richiejp.com> (cherry picked from commit 016dc3d71a)	2026-10-13 08:27:24 +00:00
mudler-agentandEttore Di Giacinto	99d6b35cb5	perf(http): skip discovery for configured requests (#12628) ... Exact configured requests need no loose-file enumeration. Reuse the synchronized config lookup without adding a cache. Misses retain regex discovery and raw-weight fallback with ALWAYS_INCLUDE. Keep general listing and bearer discovery unchanged. Preserve request validation, alias resolution, disabled-target checks, and model access. Record diagnostics only for work performed. Assisted-by: nib:gpt-6-astra Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-10-11 17:38:27 +02:00
mudler-agentandEttore Di Giacinto	c3976a675b	fix(chat): defer model fallback until after binding (#12621) ... * fix(chat): defer model fallback until after binding Explicit chat models need neither legacy Bearer discovery nor default listing. Defer those fallbacks until binding finds no model, while keeping body validation and fresh binding inside each failover attempt. Expose the existing request-phase UUID through a response header and a nil-safe accessor. Disabled timing creates neither a header nor an ID. Document the contract and cover fallback, authentication, and retries. Assisted-by: OpenAI:undisclosed Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(http): check diagnostic router cleanup Assert that deferred router cleanup succeeds instead of ignoring its error. This fixes the errcheck failure in the chat authentication test. Assisted-by: OpenAI:undisclosed Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-10-11 00:26:40 +02:00
mudler-agentandEttore Di Giacinto	0e0e91f95c	feat(diagnostics): add opt-in profiling (#12610) ... * feat(diagnostics): add private opt-in profiler Keep profiling on a dedicated loopback listener rather than the public API. Validate settings and bind synchronously so startup reports configuration and socket failures. Own sampling and shutdown for the run process. Cover private handlers, blocked cmdline access, sampling cleanup, and serving failures with focused Ginkgo tests. Assisted-by: nib:gpt-6-astra * feat(diagnostics): add sanitized phase timing Carry generated request and reload identities through private contexts. Keep timing disabled without allocations and emit only validated fields at Info level when enabled. Cover identity isolation, nested durations, inheritance, and sanitized logging with focused race-tested Ginkgo specs. Assisted-by: nib:gpt-6-astra * fix(diagnostics): retain request observations Reuse an existing request observation across recorders to avoid splitting correlation. Keep the original context, identity, and sink without new allocations. Preserve nil-recorder behavior and fresh reload identities. Verify the regression fails before the fix and passes with the full Diagnostics race suite afterward. Assisted-by: nib:gpt-6-astra * feat(cli): own opt-in diagnostics lifecycle Expose profiling and phase timing flags without enabling them for debug. Bind the private profiler before application initialization and propagate serving failures through the run lifecycle. Bound signal cleanup and preserve preload-only backend startup. Cover flag parsing, preload isolation, cancellation, and once-only shutdown. Document initialization and shutdown cancellation limitations. Assisted-by: nib:gpt-6-astra * feat(config): measure filter and reload phases Separate request lock waits from filtering and held work. Aggregate reload filesystem and parsing durations without logging inside the config lock. Preserve caller option execution under the lock after metadata collection. Extract only the internal diagnostics option before waiting for the lock. Keep direct callers uninstrumented unless they supply diagnostics options. Verify focused Diagnostics examples with the race detector, including option ordering, gated contention, errors, panic propagation, and parity. Assisted-by: nib:gpt-6-astra * fix(config): separate early reload observation Opaque callbacks cannot select timing before their execution point. Carry the startup recorder explicitly through loader construction. Apply late options once under the lock and honor composed last writes. Document early and late phase boundaries and observation identities. Report malformed gallery syntax as an internal parse failure without changing strict or non-strict loader returns. Assisted-by: nib:gpt-6-astra * fix(config): preserve per-model option callbacks The resolved snapshot suppresses existing callbacks during model defaults, even when diagnostics are disabled. Pass original options through the existing defaults pathway without adding diagnostics-driven replay. Test stateful options with two models in enabled and disabled modes. Preserve three calls and thread defaults of two and three. Assisted-by: nib:gpt-6-astra * feat(model): observe discovery phases Carry request diagnostics through config and loose-file discovery to separate directory enumeration from in-memory filtering and fallback. Keep existing APIs as background-context wrappers and preserve discovery policies, ordering, duplicates, errors, and filesystem operation counts. Add focused parity, privacy, phase, and disabled-allocation tests. The focused diagnostics race gate passes for model and galleryop. Assisted-by: nib:gpt-6-astra * feat(http): observe extraction phases Generate diagnostic identity before request discovery, independently of client correlation headers. End each extraction segment before inference so downstream latency does not inflate discovery measurements. Carry observations through discovery helpers and rebuilt request contexts without changing cancellation or model selection. Cover disabled parity, error outcomes, privacy, concurrent identity isolation and route contexts. Assisted-by: nib:gpt-6-astra * fix(http): retain diagnostics across failover Failover restores its entry request before retrying. Initialize the observation before that capture so direct extraction retains one ID. Keep extraction spans per attempt and end them before downstream work. Exercise two real targets without default middleware, including replay, shared identity, phase boundaries, and timing-off response parity. Assisted-by: nib:gpt-6-astra * feat(model): observe initialization and routing Mark loader entry before cache and admission checks. Measure only the ModelRouter callback, without attributing transport or worker latency. Test inherited identity and sanitized events with fake backends. Exercise the actual Echo router to keep profiles private across debug and timing options. Cover capture shutdown, forced close, and lifecycle failures. Assisted-by: nib:gpt-6-astra * docs(diagnostics): guide bounded investigations Document private profiling, sampling ownership, and phase boundaries so operators can interpret captures without assuming request attribution. Add Ginkgo drift checks for flags, defaults, phases, links, and capture warnings. These checks do not certify the operator prose. The focused documentation race test passes after four expected failures. The consolidated Diagnostics race gate passes across all eight packages. This gate does not validate the full suite, auth, or production UI. Assisted-by: nib:gpt-6-astra * refactor(cli): restore inline run lifecycle Remove diagnostic startup hooks so application startup stays readable in Run. Keep the profiler as an optional sidecar with bounded cleanup and an error watcher, without a general lifecycle runner. Replace hook simulations with real CLI startup checks and component capture tests. Document that application cleanup can exceed the private listener budget. The eight-package Diagnostics race gate passes. A cleanup mutation fails the real CLI listener-release regression as expected. Assisted-by: nib:gpt-6-astra * fix(cli): retain concurrent startup errors Cancellation must not hide a public API bind failure. Join the public serve result, cancellation cause, and HTTP shutdown errors inline. Ignore ErrServerClosed only after intentional shutdown. Exercise real startup with an occupied API port and a failed private listener. Isolate the Linux socket fixture in a subprocess and wait for cancellation acknowledgment without timing sleeps. Assisted-by: nib:gpt-6-astra * refactor(http): make request context primary Move context preparation into the request extractor. Diagnostics remain an optional addition, not the entry point for request processing. Preserve context initialization before failover captures the request. Assisted-by: nib:gpt-6-astra --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-10-10 20:27:28 +02:00
localai-org-maint-botandlocalai-org-maint-bot	c525ad16e8	fix(audio): record transcription usage and traces (#12546) ... * fix(audio): record transcription usage and traces Count successful transcription requests by model, including streams. Keep token counts at zero because transcription exposes no token usage. Capture multipart API trace metadata without reading uploaded audio. Do not record failed transcription or client writes as successful usage. Assisted-by: Codex:gpt-6-astra * fix(audio): preserve aliases in streaming usage Streaming transcription records the resolved target as its usage model. Pass the requested name so JSON and SSE requests share the alias bucket. Assisted-by: Codex:gpt-6-astra * test(http): check multipart reader close errors Assert successful reader cleanup to satisfy the errcheck CI gate. Assisted-by: Codex:gpt-6 --------- Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-10-07 16:57:23 +02:00
mudler-agentandEttore Di Giacinto	99043b442c	feat(router): route with native decision models (#12449) ... * fix(schema): preserve SystemOne image inputs Assisted-by: OpenAI * test(schema): follow Ginkgo conventions for decision inputs Assisted-by: OpenAI * feat(llama-cpp): dispatch native decisions through Score Upgrade the stock dependency and reconcile Score/TTS patches. Reuse native decision parsing, tasks, formatting and response-reader cleanup; preserve ordinary scoring admission and guard older dependencies. Assisted-by: OpenAI * refactor(systemone): share request and model validation Assisted-by: OpenAI:gpt-5 * fix(systemone): preserve HTTP wire-byte validation limit Keep structural validation separate from the serialized internal request bound so HTML escaping cannot reject valid HTTP payloads. Assisted-by: OpenAI:gpt-5 * feat(systemone): bound images and account native decisions Preserve public wire limits independently from router serialization. Reject unsupported NER images, map native request/capability errors, and stamp explicit usage once. Advertise decisions for stock llama-cpp. Assisted-by: OpenAI * fix(systemone): record usage on registered native route Exercise real registration and billing with a mock native backend. Reject empty native responses, malformed image URLs, trailing JSON, and wire overflow including whitespace. Assisted-by: OpenAI * feat(router): add lazy native decision transport Bind named models through internal ModelSystemOne calls with shared validation and bounded abandoned operations. Remove request and echoed-error contents from decision traces. Assisted-by: OpenAI:gpt-5 * feat(router): classify overlapping policies with native decisions Ask independent noul questions, validate probabilities and preserve first-superset routing. Wire the central factory with config-sensitive invalidation and cancellation-safe resolution. Document native framing and bounded operation limits. Assisted-by: OpenAI:gpt-5 * feat(gallery): add pinned Julia-1 native decision model Add a separate text-only llama-cpp Q8 entry with pinned Apache-2.0 source provenance and checksum. Installed using the gallery installer and exercised choice, score and noul on CPU. Assisted-by: OpenAI * test(router): verify native decisions through central factory Add an opt-in real-model Ginkgo integration covering the native Go loader and C++ transport, token usage, independent overlapping labels, and candidate selection. Document owned-server execution and the intentionally non-quality threshold. Assisted-by: Codex:gpt-5 * fix(llama-cpp): align upstream pin and preserve decision signatures Advance to bed0a856 without losing the automated upstream bump. Detect full-request fill_task support at compile time and forward every question for Nimble framing while retaining the earlier native signature. Preserve reconciled SCORE/TTS patches; add standalone compatibility coverage. Assisted-by: Codex:gpt-5 * feat(gallery): add native decision family defaults Pin Laya, Kev-4B, lev, OpenJev and Nimble artifacts. Verify Laya/Kev/lev gallery installs and CPU contracts on both native pins; clearly mark OpenJev/Nimble runtime validation pending and their noncommercial licenses. Assisted-by: OpenAI * docs(decisions): clarify integrated Nimble prerequisite Record the exact combined backend pin while retaining pending OpenJev and Nimble installation/runtime validation status. Assisted-by: Codex:gpt-5 * fix(gallery): indent native decision model sequences Match repository yamllint indentation for Laya, Kev, lev and OpenJev list fields. Parsed gallery data is unchanged; reproduce CI gallery lint failure before the whitespace-only fix and pass the same command afterward. Assisted-by: Codex:gpt-5 * docs(decisions): record OpenJev and Nimble CPU validation Record gallery installation, checksum/metadata verification and multiquestion native smoke results on bed0a856. Retain noncommercial and text-only limitations without accuracy or deterministic-output claims. Assisted-by: OpenAI * fix(ui): expose native Decisions router classifiers Select classifier models using metadata-driven capability routing, retain tuned thresholds, and validate native decision selections before saving. Cover both native backends and create/save/reopen in the real React editor. Assisted-by: Codex:gpt-5 * fix(router): exclude aliases from native decision discovery Check the originally named config before advertising native Decisions eligibility. Retain target capability inheritance for ordinary generation aliases. Exercise the actual capabilities endpoint with native models on both backends, aliases, and disabled models. Assisted-by: Codex:gpt-5 * feat(systemone): share bounded multimodal input validation Preserve text wire limits while admitting bounded PNG/JPEG decision input. Share collection and header validation across internal and public callers and keep the native runner response budget independent. Assisted-by: OpenAI:API-assistant * fix(systemone): bound admission lifetimes and validate complete images Retain shared admission leases through actual work completion, including abandoned internal operations. Decode bounded image pixels, cap public native responses before usage stamping, and preserve oversized malformed text status precedence. Assisted-by: OpenAI:API-assistant * fix(router): classify images before media fetching Preserve ordered structured probes for native decisions. Defer OpenAI media preparation until routing selects the served model, so rejected decision URLs cannot trigger downloads before shared validation. Guard direct image collection with context-aware shared admission. Keep text classifiers and embedding caches from discarding image input. Retain fail-closed classifier configuration and runtime fallback policy. Add middleware, typed-content, admission, cancellation and cache tests. Assisted-by: OpenAI:API-assistant * fix(router): bound extraction before serialization Check probe budgets before copying text or marshaling message state. Count JSON escaping so oversized internal inputs fail before allocation. Preserve typed Anthropic blocks through selected-model conversion and fallback. Keep retry coverage in Ginkgo without global test registration. Assisted-by: OpenAI * fix(router): bound supported probe serialization Arbitrary structs can bypass the probe budget through pointer marshalers, string tags, and promoted fields. Accept concrete chat schema types and plain JSON values instead of emulating arbitrary struct serialization. Budget escaped direct prompts before marshaling so raw length cannot hide serialized expansion. Preserve runtime fallback and reject oversized input before invoking the decision runner. Add Ginkgo allocation, boundary, and marshaler invocation regressions. Six-package tests, three-package race tests, and full-T2 delta lint pass. Assisted-by: OpenAI:GPT-5 golangci-lint * feat(decisions): enable bounded OpenJev images Validate native decision images before permissive media parsing and pixel allocation. Require both decision image support and a vision projector; missing or audio-only projectors cannot silently become text decisions. Pin the OpenJev Q8 projector and document its license and disk footprint. Add native safety tests, canonical limit parity, gallery and load-option checks, and a reproducible CPU direct-RPC contrasting-image smoke. Assisted-by: OpenAI:GPT-5 * fix(decisions): reject incomplete image streams stb accepts corrupt PNG Adler checksums and truncated JPEG scans. Use bounded zlib validation and strict libjpeg decoding before parsing. Keep dimension and aggregate pixel checks ahead of decoder allocations. Wire decoder dependencies into native builds and runtime packaging. Add regressions for appended EOI and embedded marker bypasses. Assisted-by: OpenAI:GPT-5 * fix(ci): gate native decision image validation Run the decoder security tests outside the stdlib-only native suite. Fetch vendor headers at the backend pin and provision decoder dependencies. Gate Go limit parity and production CMake wiring without model downloads. Assisted-by: OpenAI:GPT-5 * test(decisions): cover multimodal public API paths Exercise shared image contracts through the registered HTTP routes and external mock backend. Add opt-in cached gallery installation and real OpenJev image decisions through SystemOne and both routing APIs. Assisted-by: Codex:gpt-5 * test(decisions): assert isolation and cache bypass Observe external RPC calls and compare complete classifier history. Winner-only and cache-miss checks could hide dropped history or cache use. Give real inference its own application and model directory so shared backend mappings and loaded processes cannot affect mixed suite order. Assisted-by: OpenAI:ChatGPT * test(decisions): isolate fixture globals Disable optional global services in the isolated HTTP fixture and register cleanup before setup assertions. Verify meter provider identity survives fixture creation and destruction. Snapshot observed usage before assertions so failures cannot retain the mutex. Require a successful usage stamp before checking error responses. Assisted-by: Codex:gpt-5 golangci-lint * fix(application): honor optional telemetry controls Skip failover gauge registration when metrics are disabled. Register against the application meter rather than looking up the global provider. Allow embedders to retain the bounded routing log without billing stats. Keep the existing default when stats are disabled. The isolated HTTP fixture uses this option without losing its native router assertions. Assisted-by: Codex:gpt-5 golangci-lint --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-10-04 09:34:21 +02:00
mudler-agentandEttore Di Giacinto	9eb5a9e61d	feat(audio): remember speakers from diarization (#12414) ... * feat(schema): validate portable speaker profiles Add the versioned profile schema for explicit speaker enrollment. Validate compatibility against separately supplied loaded-encoder metadata. Reject unusable speakers, invalid vectors, and inconsistent clean spans. This slice does not change HTTP routes, backend integration, or the UI. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(parakeet): export profiles with transcripts Export opt-in speaker profiles and trusted encoder metadata. Replay registrations by ID so duplicate display names keep independent vectors. Use one profile-capable diarization for slots, names, and clean spans. Assign timestamped ASR words to those slots without a second diarization. Preserve legacy opt-out and no-ASR behavior, and propagate failures. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(audio): enroll portable speaker profiles Gate profile exports with voice-recognition permission and validate registration against metadata from the loaded encoder. Preserve audio enrollment and independent registrations with duplicate display names. Exclude diarization and registration exchanges before API trace capture so persisted traces cannot retain profile vectors or JSON audio. Defer candidate dimensions to trusted loaded metadata. Sort candidates by registration ID so incompatible profiles cannot suppress legacy voices through registry iteration order. Keep portable identity checks closed when trusted metadata is unavailable. Test persisted traces, explicit slot zero, and selection through offline and live transport. Document privacy and the ephemeral registry lifecycle. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): remember speakers from diarization Add a Studio page for diarization and opt-in speaker profiles. Preview clean intervals from the original recording before explicit registration. Join profiles by raw speaker labels, preserve duplicate names, and relabel turns only after a successful save. Discard stale results when the model or recording changes. Share registration metadata with voice management without storing vectors or recordings from this flow. Document permissions and the global, ephemeral registry. Cover enrollment, permissions, previews, and asynchronous races with mocked Playwright tests. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: clarify HTTP speaker enrollment support Replace the stale enrollment limitation with the current HTTP workflow. Distinguish native transport from explicit registration and link its docs. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(parakeet): pin merged speaker profile support Use the merged commit from mudler/parakeet.cpp#80. Its tree matches the previously accepted native pin. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add diarization enrollment setup example Connect the existing gallery modes to the speaker enrollment workflow. Show installation, private profile export, explicit raw-slot registration, and later recognition without another export. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(blog): explain diarization speaker profiles Put the diarization walkthrough on the LocalAI website in the feature PR. Cover the three gallery modes, explicit enrollment, and privacy limits. Link setup instructions and keep availability conditional on feature support. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(blog): focus diarization on everyday use Explain what users can do with recordings before the setup steps. Replace the technical walkthrough with a short Studio guide and link readers to the existing reference for model names and developer use. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(blog): lead with speaker capabilities Present speaker recognition through everyday uses and a short UI flow. Keep technical reference details in the existing documentation. Assisted-by: OpenAI:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(diarization): satisfy Go lint checks Avoid copying protobuf message state when extending backend status, check the multipart reader close result, and document the focused testing.T lint exemptions. Assisted-by: nib:gpt-5.6-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-10-02 08:14:00 +02:00
Ettore Di Giacinto	d8571a8ee9	fix(failover): spill 429 admission rejections to the next target ... #12113 changed admission control to reject with 429 instead of 503. failoverWriter only held back responses with status >= 500, so a 429 rejection reached the client and the chain never spilled to its next target. Hold 429 as well. An admission rejection is still flagged and spills without tripping the target. Any other 429 is not retryable, so it is released to the client unchanged. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5-5 [Claude Code]	2026-09-28 15:34:29 +00:00
Ettore Di Giacinto	9c156656bd	Merge PR #12285: feat(failover): serve a model name from a chain of local and remote targets ... Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-28 15:32:30 +00:00
Stefan Walcz	950c271710	[router] fix: re-seed the knn corpus index when the vector store comes back empty (#12267) ... fix(router): re-seed the knn corpus index when the vector store comes back empty The corpus manager records a store as synced by file fingerprint and embedding fingerprint. The local-store backend behind it is an in-memory gRPC process the model loader may evict (active-backend cap, memory pressure) or the idle watchdog may kill, and relaunch on the next request — empty. The file is unchanged, so EnsureLoaded returned early and the router went blind: every probe fell back with similarity 0 while corpus/stats kept reporting the full count. Measured on a production router (LOCALAI_MAX_ACTIVE_BACKENDS=6, four resident models + two router stores): loading any further backend evicted a store, and the idle watchdog killed both after 15 minutes; /stores/find returned 0 hits against a 100-line corpus file whose stored vectors matched fresh embeddings with cosine 1.000. Two parts, because the knn classifier is built once and cached (GetOrBuildClassifier), so the sync at build time is otherwise the only one for the process lifetime: - corpus.Manager remembers one vector it inserted (probe) and, on the synced path, asks the live index for it. A miss means the index was relaunched — fall through and re-seed from the file (no re-embedding). - The router middleware wraps the knn classifier's store so every lookup runs EnsureLoaded first; the loader gets the raw store, so its probe never re-enters the wrapper. A sync error fails the lookup closed, like the build-time load. Specs: corpus package (relaunched empty store is re-seeded under an unchanged file), middleware (relaunched index behind the cached classifier is re-seeded instead of falling back; the spec is red without the wrapper). The test fake now answers Search for inserted vectors. Folds in the maintainer's follow-up (router-corpus-reseed-after-store-relaunch): reviewed and accepted. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>	2026-09-28 00:52:47 +02:00
3 people	52a6d62bbc	fix: return correct HTTP status codes for saturation and no-nodes-available (#12113) ... * feat: return 429 when backends are saturated When backends are at capacity (per-model max_concurrent or the process-wide --max-concurrent-backend-requests ceiling), the response was 503. The OpenAI SDK, litellm, and most agent harnesses key on 429 for rate-limit backoff and treat 503 as a hard error. Both saturation paths now return 429 with the existing Retry-After header and type: "rate_limit_error" in the JSON body. The per-model admission middleware keeps admission_rejected as the code field so existing alerts that match on it still fire. Non-saturation 503s are unchanged: model cold-loading (with progress body), model-load failure cooldown, PII detector fail-closed, and classifier unavailable. These mean "not ready" rather than "busy". Assisted-by: AGENT:regolo/glm5.2 [TOOL] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: return 503 when scheduler has no available nodes When the scheduler cannot find any healthy node to serve a model — all nodes are full and eviction cannot free a slot, or a node_selector excludes every candidate — the error fell through to 500. A 500 tells clients something is broken when the condition is transient and retryable. The router now wraps these errors with a new ErrNoAvailableNodes sentinel. The HTTP error handler maps it to 503 via applyNoAvailableNodes, following the same pattern as applyBackendAdmission (429). Unrelated scheduler errors (DB timeouts, registry lookups) still return 500. Three return sites are wrapped: - resolveSelectorCandidates: selector matches zero healthy nodes - scheduleNewModel eviction-busy: all models have in-flight requests - scheduleNewModel eviction-failed: eviction itself errored The existing scheduleAndLoad wrapper ("no available nodes: %w") preserves the sentinel through the chain via errors.Is, as does ModelRouterAdapter. Assisted-by: AGENT:regolo/glm5.2 [TOOL] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(http): use Ginkgo for admission tests Replace forbidden testing.T calls with Ginkgo and Gomega so the lint check accepts the admission handler tests. Assisted-by: Codex:GPT-6 forbidigo * fix(middleware): show the recorded status for admission rejections The admission audit row now records 429, but the Middleware page still printed a hard-coded 503. Read the status from the event, and update the two package comments that still said 503. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5-5 [Claude Code] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-09-28 00:11:51 +02:00
Ettore Di Giacinto	00bd5ee121	fix(failover): skip a target that has been edited into a chain ... A chain is checked for nested chains when it is saved, but not when one of its targets is later edited into a chain. Requests then served the inner chain's config as the target, which has no backend and triggers backend auto-detection. Mark such a target missing so no plan picks it, and skip it without a trip in the HTTP path when it is pinned. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-5-5 [Claude Code]	2026-09-27 20:19:09 +00:00
Ettore Di Giacinto	5ed67be0bf	fix(failover): skip a target that answers HTTP 501 instead of failing the chain ... The non-OpenAI endpoints (depth, detection, face_*, voice_*, images, video, 3d) map a backend's gRPC Unimplemented to an echo 501 without the gRPC status. The retry loop did not see a capability gap, and IsRetryable is false for 501, so the client got 501 and the next target was never tried. Treat a returned or written 501 as a capability gap. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 07:42:21 +00:00
Ettore Di Giacinto	882f51fd62	fix(failover): trip a rate-limited or exhausted target instead of skipping it ... Treating ResourceExhausted as a capability gap skipped the target without counting a failure, so a target that stays rate limited or out of memory kept its traffic. It is now an ordinary retryable failure: the request moves to the next target and the exhausted one trips. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 07:42:21 +00:00
Ettore Di Giacinto	6df1767133	fix(localai-proxy): keep rerank, stream errors and chat intact through the proxy ... Rerank no longer sends top_n 0, which the upstream rejects. A mid-stream upstream error frame now fails the call instead of ending it as a short success. Temperature 0 is forwarded. An upstream 429 becomes ResourceExhausted, which failover skips like Unimplemented. A localai-proxy config sends its own name upstream when upstream_model is unset, and a chat proxy defaults to the tokenizer template so chat reaches the upstream as messages. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 07:42:21 +00:00
Ettore Di Giacinto	509cca35e0	feat(grpc): serve Rerank from Go backends and skip Unimplemented targets ... Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 07:42:21 +00:00
Ettore Di Giacinto	2ecbaab010	fix(failover): send remote targets their own upstream model ... A chain request reached a cloud-proxy target with the client's model, the chain name, whenever the target set no upstream_model: passthrough forwards the body's model and translate falls back to it. The upstream answered 404, which neither retries nor trips, while the liveness probe, which checks the target's own name, kept passing. PrepareTarget now sets the upstream model of a remote target to proxy.upstream_model or the target name, the same name the probe uses. The request pipeline and realtime chain stages both call it. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 07:42:20 +00:00
Ettore Di Giacinto	b67b75dcdd	fix(failover): answer HasChains from a flag set at sync ... The request path called HasChains on every request, and with no chains it scanned the config source each time: the loader's lock plus a copy and sort of every config, forever, on every installation without chains. Sync now keeps an atomic flag and HasChains reads only that. A chain added since the last sync is still served because Plan syncs on a miss; only in-request retry waits for the next tick (at most 1s). Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 07:42:20 +00:00
Ettore Di Giacinto	f28d07b09d	fix(failover): spill capacity and disabled targets without tripping them ... An admission rejection or a disabled target moves the request to the next target through Attempt.Skip, which records no failure. A 4xx response no longer counts as a success. Requests skip body recording when no chain is configured, and stop it once the model is known not to be a chain. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 07:42:20 +00:00
Ettore Di Giacinto	471bd8bf96	feat(failover): resolve chains per request and retry on the next target ... The retry wraps SetModelAndConfig, so each attempt binds the request again from a replayed body. A 5xx of a chain request is held back until the handler returns, and a streamed response is never retried. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>	2026-09-27 07:42:20 +00:00
3 people	636c43baa5	fix(openai): return HTTP errors for pre-stream failures and report per-slot context (#12204) ... * fix(capabilities): report the per-request context with split KV slots With parallel slots and kv_unified:false, llama.cpp gives each slot n_ctx/n_parallel, padded up to a multiple of 256. /models/capabilities still reported the full n_ctx. A client that budgets a request against context_size then overflows at a fraction of it. EffectiveRequestContextSize returns the per-slot size in that case and the full context otherwise. With the unified KV cache, the grpc-server default, one request may use all of n_ctx. The capabilities endpoint and the router's prompt trimmer now use it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(openai): return an HTTP error when a stream fails before any chunk A streamed chat request set the SSE headers, then waited for the backend. When the backend failed before the first token, LocalAI sent a 200 with a `data: {"error":...}` chunk and [DONE]. Clients that do not parse error chunks saw an empty reply. cogito's LocalAI client was one of them: nib users got "streaming decision produced no content" instead of the context overflow that caused it. Nothing has been written at that point, so the handler now returns the error as a normal HTTP response. A failure after the first chunk keeps the in-stream error chunk. A prompt that exceeds the context is now a 400 on both paths, as in the OpenAI API and llama-server, and no longer a 500. The message is kept whole, because clients read the token counts from it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): check the error from closing the response body golangci-lint's errcheck flags the unchecked resp.Body.Close in the new pre-stream error helper. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-22 17:19:58 +02:00
Richard Palethorpe	7034d353b1	feat(kimodo): Add observability hooks (#12184) ... feat(kimodocpp): add API and backend request observability Capture animation requests, phase timings, output metadata, and failures in traces. Record correct API error statuses and cover completed, running, failed, and disabled tracing. Assisted-by: Codex:GPT-6 Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-09-22 07:06:32 +00:00
Richard Palethorpe	495165266f	feat(kimodocpp): track usage through generic backend metadata (#12162) ... Report input tokens and frame-step output units in response metadata and record them through the existing usage accounting pipeline. Preserve the accounting rule and model-specific dimensions as JSON without extending the gRPC schema for each modality. Expose animation usage only under metadata.usage, validate counts before recording, and document the response contract and loaded-model location. Add coverage for transport, defaults, failures, persistence, and recording requests once with statistics enabled or disabled. Assisted-by: Codex:GPT-6 Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-09-20 19:39:06 +01:00
lei_lei	f28e8b24e6	fix(ollama): accept :latest tag on model lookup (#11732) ... /api/tags appends :latest to untagged names, but chat and the other model endpoints looked the tagged name up as-is and 404'd. Signed-off-by: lei_lei <96427312+leilei3167@users.noreply.github.com>	2026-08-26 08:57:36 +02:00
Ettore Di Giacinto	df1a40f9c0	fix(distributed): hash the config as persisted, not as defaulted ... The revision was computed after SetDefaults, which folds in things that are not persisted configuration: the GGUF guess, the hardware defaults, and app-level options such as threads. The GGUF guess is the damaging one. It parses the model file to fill in values like context size, and when that parse fails it falls back to a different default. Whether a multi-gigabyte file on network storage parses at a given moment is not a property of the configuration, so one unchanged YAML produced two different revisions depending on when it was read. The controller rejected every request carrying the other one, and the model stayed unroutable until the stored value happened to match again. This is why it never reproduced against a model directory with no weights in it: the guess is skipped there and both values agree. The app-level defaults are the same class of bug with a slower fuse: changing threads in the settings UI changed every model's revision and made every model unroutable. The revision is now stamped when the file is parsed, before any defaults are applied, so it is a function of the file alone. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [golangci-lint]	2026-08-24 11:45:32 +00:00
Ettore Di Giacinto	f3fabe8c5c	fix(distributed): order derived usecases deterministically ... syncKnownUsecasesFromString rebuilds KnownUsecaseStrings by ranging GetAllModelConfigUsecases, which is a map. Go randomizes that order per call, and the field is part of the serialized config, so one unchanged YAML hashed to a different config revision on every load. A model that derives a single usecase hid the problem. One that derives several, such as a chat model with an mmproj, alternated between as many revisions as there are orderings. The router treats a revision it did not establish as a config change, so requests failed with "stale model config revision" until the stored value happened to match again. Sorting the list makes the revision a function of the file alone. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [golangci-lint]	2026-08-23 17:20:20 +00:00
Ettore Di Giacinto	04735cd1f6	fix(distributed): stamp config revision at load time ... The request middleware merges the caller's prediction parameters into its copy of the model config. core/backend.ModelOptions then hashed that copy, so the revision identified the request body rather than the persisted configuration. EstablishModelConfigRevision stores the first revision it sees and requires an exact match afterwards. The first request after a restart therefore pinned the model to its own temperature, top_p and stop values, and every later request that sent different ones failed with "stale model config revision". No config edit was involved. The loader now stamps the revision when it materializes a config, before any request override reaches it, and ModelOptions reads that stamp. Model administration keeps hashing the same persisted config, so both paths agree on one revision per configuration. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [golangci-lint]	2026-08-23 14:35:44 +00:00
localai-org-maint-botandlocalai-org-maint-bot	0761bd02c7	feat(chat): add end-to-end context compression (#11556) ... * feat(config): add context compression policy Define the opt-in model configuration contract before the chat middleware consumes it. Document each policy field so later request handling does not invent a second schema.\n\nRefs #9534\n\nAssisted-by: Codex:gpt-5 * fix(config): register compression fields The model editor metadata gate rejects new config fields without descriptions and suitable controls. Register the compression policy so operators can edit its six fields safely. Assisted-by: Codex:gpt-5 [monitoring-prs] * feat(chat): compress long contexts Long conversations currently fail once they reach the model context window. The opt-in policy now summarizes complete older turns before primary inference and preserves the newest tool chains. Both OpenAI and MCP chat routes share the same transformation. Usage metadata and metrics expose each compression event. Refs #9534 Assisted-by: Codex:gpt-5 --------- Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-08-18 11:31:03 +00:00
Richard Palethorpe	d10374f849	feat(router): make KNN a first-class classifier with a persisted, curated corpus (#10652) ... * feat(router): make KNN a first-class classifier with a persisted, curated corpus Add `classifier: knn` — similarity-weighted voting over labelled example prompts. Unlike score/colbert it needs no classifier model: label knowledge lives in a corpus seeded and curated through the admin API, so routing decisions are deterministic, auditable, and grounded in graded experience rather than a model's opinion. Epistemic gate: corpus entries below knn.similarity_threshold cannot vote; when none clears it the classifier activates no labels and the router uses the fallback — a prompt unlike all labelled experience is treated as undecidable, not guessed. Decisions record nearest_similarity (also on fallback rows) so admins can see how far the nearest labelled experience was; the Routing tab explains out-of-corpus fallbacks and shows per-label corpus counts. Persistence: one JSONL file per router under <data path>/router-corpus (text, labels, vector, embedder fingerprint). The file is the source of truth; the local-store index is rebuilt from it at classifier build time and stays a pure in-memory index. Entries recorded under a different embedding model re-embed on load. Also corrects the docs' false claim that local-store collections persist — the embedding cache never survived restarts (and still doesn't); the corpus does. Corpus input is API-only by design (entries may contain example user content): POST /api/router/{name}/corpus seeds (labels validated against declared policies, embedded server-side, indexed immediately), GET .../corpus/stats inspects — label counts only, entry texts are never returned by any surface — DELETE .../corpus wipes. Admin-gated like the sibling router endpoints, and exposed as MCP tools (seed_router_corpus / get_router_corpus_stats / clear_router_corpus) in both the httpapi and inproc clients with coverage-test route mappings. Plumbing: VectorStore gains SearchK (top-K was hardcoded to 1); local-store gets InsertBatch/Delete as optional fast paths; RouterConfig gains a knn block (embedding_model, k, similarity_threshold, vote_threshold, store_name) with meta-registry fields; the classifier dropdown now offers knn and the previously-missing colbert; embedding_cache is ignored (with a warning) for knn — it IS an embedding-KNN lookup; the stale /api/instructions intelligent-routing entry is rewritten (it described a classifier that no longer exists); swagger regenerated. Tests: KNN vote/gate specs with hand-computed vote shares, corpus manager suite (restart reload without re-embedding, fingerprint re-embed, dedupe, hostile store names), middleware specs (corpus routing, gate fallback, config validation, cache-wrap refusal), corpus endpoint specs pinning the texts-never-returned contract, MCP catalog + route-mapping gates, and a Playwright spec for corpus stats and the out-of-corpus decision detail. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(router): name consulted corpus neighbours in knn decisions Every knn decision (decision log rows and the /api/router/decide response) now carries neighbors: the K retrieved corpus entries by descending similarity - including ones below the epistemic gate, which is what makes fallback decisions diagnosable - each as {id, similarity, labels}. The id is the entry's content hash (first 8 bytes of the SHA-256 of its text, hex): stable across reseeds and re-embeds, and text-free, so an external platform that seeded the corpus can recompute text->id on its own copy and bucket decisions by corpus region (per- region reliability accounting) without corpus text ever leaving the server. A corrupt index payload surfaces as an id-less neighbour at a real similarity instead of disappearing. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * refactor(router): deduplicate knn plumbing and cut corpus hot-path waste Post-review cleanup of the knn-first-class-router branch; no behaviour changes on the API surface. Reuse/altitude: - RouterKNNConfig.ResolvedStoreName is now the single source of the router-corpus-<name> default (was hand-derived in four files). - corpus.ResolveKNNRouter + corpus.Seed carry the shared model resolution and seed validation; the REST endpoints and the assistant MCP client are thin transport adapters over them, with sentinel errors mapped to HTTP statuses at the echo boundary. - middleware.NewClassifierDeps assembles the classifier dependency set once for all five entry points (OpenAI, Anthropic, realtime, decide, corpus) instead of five hand-copied literals. - router.AllClassifiers feeds both the status endpoint and the unknown-classifier error, ending the classifier-list drift. - Per-classifier requirements moved out of validateRouterPolicies into their buildClassifier arms; the knn arm owns its embedding_cache opt-out instead of a name-check in the shared wrap tail. - adminOnly replaces four inline copies of the admin gate in the middleware routes. - localVectorStore.Search delegates to SearchK (identical traces). Efficiency: - Manager.Add embeds outside the manager mutex and appends to the JSONL file (O(new) instead of O(corpus) rewrite); a torn tail from a crash mid-append is tolerated on read and repaired on next write. - Stats memoises per store keyed on the file's stat fingerprint and no longer takes the manager mutex, so the 5s status poll stops parsing vector-laden JSONL and stops blocking behind seeds. - KNN Classify decodes each neighbour payload once (was twice) and builds refs and votes in a single pass with one fallback return. - Corpus file writes fsync before rename/close. - The corpus manager is built eagerly in newApplication (sync.Once dropped); test helper dead branch removed. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(router): bind knn corpus vectors to an embedder fingerprint and fail closed on mismatch Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * chore(mcp): align corpus tool prompts and the mutating-tool safety list Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(proto,backend): report embedding shape from the llama-cpp backend Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(embeddings): Go-side pooling — mean/last/decayed_mean with half-life Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(embeddings): accept chat messages[] and per-request pooling on /v1/embeddings Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * chore(middleware): name the failing fields when post-merge validation 400s An intermittent post-merge validation failure surfaced as an opaque 400 during integration (pooling scheme mismatch that no client had sent). Log the model, the request's pooling override, and the merged config's pooling fields at the failure point so the next occurrence identifies whether the request or the stored config carried the bad value. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(embeddings): scheme override must not inherit the config's half-life A model config defaulting to decayed_mean pooling carries pooling_half_life_tokens; a request overriding the scheme to mean/last without its own half-life inherited that value, and post-merge validation rejected the pair the server itself had assembled. Zero the inherited half-life when the overridden scheme is not decayed_mean; a request that explicitly pairs a half-life with a non-decayed scheme still 400s. Assisted-by: Claude:claude-fable-5 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix embedding pooling validation and router bounds Declare backend embedding layouts and reject incompatible pooling modes. Reset local-store dimensions after a full clear, validate KNN thresholds, and add real backend and store integration coverage. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * ci: run local-store integration tests Build and install the local-store backend in the Linux test job, then run the existing store integration suite so new specs are discovered automatically. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-08-18 09:37:43 +02:00
localai-org-maint-botandlocalai-org-maint-bot	9f62401fca	feat(traces): show in-flight API requests (#11368) ... Register JSON API exchanges before their handlers run so the traces dashboard can surface active work. Replace the live entry with the completed persisted record under the same ID, and clean it up if a handler panics. Assisted-by: Codex:gpt-5 Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-08-05 09:37:05 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	58ea2f5d79	feat(ui): give Operate and Studio a front door, and fix two layout regressions (#11305) ... * feat(ui): give Operate a front door and fold six rail groups into four Opening Operate ran firstVisiblePath() and landed on Backends, because Backends happens to be written first in operateConsole.groups. The section that should answer "is anything wrong" opened on a package manager, and nothing was reported until you visited it. Adds /app/operate. Its one irreplaceable block is "Needs attention", which is empty when nothing is wrong and says so in a line rather than rendering a reassuring green panel. It collects stale backends, failed operations and unhealthy nodes. Everything else on the page is a summary you could already assemble by visiting four others. The rail regroups from six headings to four: Inference and Activity were both "the runtime right now", Access and System were both administration. No destination is removed and no gate changes, so isConsoleItemVisible and consolePaths are untouched. Overview leads the first group, which is what makes firstVisiblePath() return it without knowing it exists. Rail entries now carry a signal beside the label. This does not replace the sidebar badge and is not built as if it does: the badge stays on the always-visible sidebar entry for the reason recorded in Sidebar.jsx, that the rail exists only on Operate routes and can be collapsed. The signals are orientation while inside Operate, so they are aria-hidden and nothing urgent depends on them alone. OperateSummaryContext polls once for the whole console, following OperationsContext, which exists because per-consumer setInterval against one endpoint was the defect it fixed. It is mounted by ConsoleLayout for the Operate console only, so "poll only while in Operate" needs no route check. Built on usePolling, so it pauses on a hidden tab. Operations are read from OperationsContext rather than polled a second time, and each source degrades to no-signal on its own so one dead endpoint cannot blank the rest. It reads the cached GET /api/backends/upgrades and never the POST that forces a real registry check. Traces and Usage get no signal yet: /api/traces returns the list, so a count would mean fetching every trace to render one number. A counts endpoint is the honest fix and is scoped separately. Full e2e suite green (369 passed, 4 skipped), including a render-smoke entry for the new route. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): open Studio on what this machine can actually make Studio was a tab strip over six generators that opened on Images, which was never a decision, only the first entry in BASE_TABS. Nothing said which modalities this installation could run, so the way to learn that video had no model was to pick the tab and find an empty select. Adds an overview tab and makes it the fallback. Explicit tabs still win, so existing deep links keep working; anything unrecognised or gated now lands on the overview rather than Images. Each tab carries a dot: filled when an installed model advertises that modality, hollow when nothing serves it. That is the feature in one detail, turning the strip from navigation into a report of what the machine can do before anything is clicked. The dot is aria-hidden because the overview states the same facts in words and the dots change as models load. Two kinds of unavailable, which had to stop looking alike: - switched off, via a permission: no tab and no lane, unchanged - available with no model: a lane, and a route to installing one Studio now owns one MODALITIES table so the tab strip and the overview cannot disagree about what exists, and calls useModels() once, unfiltered, grouping in the browser. useModels(capability) fetches the whole list and filters locally, so a hook per modality would have been six identical requests to /api/models/capabilities on every mount. There is a test for that. Recent outputs read every localStorage store through a new readAllMediaHistory(), which avoids mounting five hooks that carry save timers the overview has no use for. 3D is read separately through use3DHistory rather than folded in: its entries are GLB blobs in IndexedDB, so they cannot come from the same synchronous read. Typical cost is the median of this machine's own history, not a guess, and renders as a dash when there is nothing to go on. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): stop the stat cards and the console rail breaking on small screens Two unrelated causes behind one report that /app/manage looks wrong when the window is narrow. The stat cards were being laid out by the wrong rule at every width. Two different components both claimed `.stat-grid`: the dashboard card strip that holds .stat-card children, and the detail-pane StatGrid the split views introduced further down App.css. Being later, the second won every shared property, so the cards got its 120px columns and its 1px hairline gap in place of their own 180px columns and spacing-md. Four cards were packed onto a row that fits two, labels wrapped to three lines and clipped, and the icon crowded the value. Renamed the strip to `.stat-cards`, after the children it actually holds, which also removes the mismatch of a `.stat-grid` container full of `.stat-card`s. The split-view component keeps `.stat-grid` and its BEM parts. The expanded console rail had no bounded height. Thirteen destinations stacked in one column is taller than a phone, so opening the menu pushed the page's own heading past the fold: the menu replaced the page rather than annotating it. Capped at 55vh with internal scrolling below 768px, so the content behind stays reachable. Both are asserted on behaviour rather than markup: no stat-card label may be clipped, the card gap must not be the detail pane's hairline, and expanding the rail must leave the page heading on screen. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): retemper the palette to localai.io and add the lane primitive The token half of the style transfer, plus the shared list idiom the two overviews had each grown their own copy of. theme.css moves from Nord to the website's palette, variable names preserved so every consumer moves with it: ground #13171f -> #0d1117, accent frost cyan #88c0d0 -> action blue #4f8cff, success sage -> mint #56d6a4, warning -> the amber #f1b95d the site spends only on the thing asking for a decision. Eyebrows go mint. Dividers become an opaque #29384a hairline rather than alpha over a varying surface, which is what makes stacked surfaces read crisply on the site. Light is derived, not inverted. The site ships one theme and never had to answer this, but the app does: blue darkens to #2f62d8, mint to #0d8b60 and amber to #8a5d0b, all clearing 4.5:1 on a cool paper ground, where the dark-mode values sit near 2:1. Same three roles, different values. Three files restate the palette because CSS variables cannot reach them: cmTheme.js (the whole CodeMirror theme), VoiceVisualizer and WaveformPlayer (canvas). Left alone they would have quietly kept the app half-Nord. The `.lane` primitive replaces the near-identical row CSS that OperateOverview and StudioOverview had each written: a full-bleed row on a hairline that insets on hover, with no card and no shadow. Callers supply only the column template. Both pages now use it, along with `.lane-head` for section rhythm and a `.page-pad` container for top-level pages outside a console shell — without which Studio sat flush against the sidebar with its eyebrow clipped. Studio's tab strip wraps rather than running off the edge at narrow widths. Full e2e suite: 386 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): put Home's resident models on lanes and give the footer one line Home's status line was three chips saying a thing was true. It now reports figures: how many models are resident, how many nodes are healthy, what share of memory is in use, set in tabular monospace so the digits line up. A chip answers whether; a figure answers how much, which is what someone opening the page at a glance is after. Resident models move from status chips to lanes, with the id set in a new `.lane__name--id` because an id is something you might type or paste and the UI face makes it read as a label. /api/system-information carries only the id, so there is deliberately no backend or memory column: inventing one would mean a server change this does not make. The footer was three centred rows and cost the bottom sixth of every page for chrome. It is one line now, version left and links right, wrapping to centred when the viewport is too narrow to hold both. Every link it had, it keeps. Full e2e suite: 392 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): correct three contrast failures and stop a guaranteed-404 poll A contrast audit of the new palette found three values below WCAG AA, one of which the previous commit message claimed was fine: - White on the #4f8cff button is 3.22:1, which is large-text only. The website does exactly this, but a button label in an app is not large text, so the label goes to dark ink at 5.88:1. Light mode keeps white, which is 5.44:1 on its darker blue. - Light-mode success was 4.08:1 on paper, not the 4.5 claimed. Darkened to #0a734f, 5.56:1. - Nord red was already 4.28:1 on raised surfaces, a pre-existing miss carried over unexamined. Lifted to #c96f78, 5.02:1. Lanes gain the two states they were missing: a 44px target on coarse pointers, matching what EntityRail already does so the two list idioms feel the same under a thumb, and a reduced-motion variant that keeps the background feedback while dropping the hover inset, which is a position change. The Operate summary no longer asks for /api/nodes on a single-node install. The cluster API answers 503 when distributed mode is off, so it was a guaranteed miss every fifteen seconds; it is now gated on useDistributedMode, the same condition the rail already uses for the Nodes entry. Full e2e suite: 392 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): restore the gap between overview blocks, and stop claiming zero nodes Two defects a design review surfaced. `.lane-head:first-child { margin-top: 0 }` was meant to stop the first block on a page carrying a top margin. But every <section> makes its lane-head a first child, so the reset applied to all of them and the gap between blocks vanished: "Sections" sat flush against the attention row above it. The header supplies its own bottom margin, so a uniform top margin is correct everywhere. The Cluster summary read "0 nodes" on a single-node install, which looks like a fault when the cluster API is simply switched off. It now says "Single node". Full e2e suite: 392 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): open dark by default, and stop clipping the collapsed sidebar footer Dark is the identity rather than a preference: localai.io ships one theme and it is this one, so an install should look like LocalAI before anyone has chosen anything. The OS setting no longer selects light on first load. The toggle still does, and a stored choice wins forever after, which the tests assert both ways. The collapsed sidebar footer stacked its controls but kept the expanded row's inline padding, so their edges were clipped against the 51px rail. Full e2e suite: 394 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(api): count traces server-side and give the Operate overview real totals The overview's headline block had no source. /api/traces returns the trace list, so "37 errors in 24h" meant fetching every buffered exchange to count it in the browser — waste that grows with the buffer, to produce three integers. Adds GET /api/traces/summary: totals, failures, p95 and a bucketed series for sparklines, over a window that defaults to 24 hours and is capped at a week. Deliberate calls, each with a spec: - A 4xx is the caller getting it wrong, not the installation being unhealthy, so only 5xx and transport errors count as failures. - p95 is a nearest-rank percentile rather than the slowest request, which is what a max would report and what makes latency panels lie. - Buckets are oldest-first so a sparkline reads left to right, and the slice is never nil: nil serialises as null and breaks .map() on the other side, which is a silent runtime error rather than an empty chart. - Exchanges outside the window are not counted at all. The route is registered before /api/traces/:id so "summary" is not captured as a trace ID. On the client, Traces and Usage gain the rail signals they were shipped without, the Observability section summary now states counts instead of listing its destinations, and an installation that has served nothing says so rather than showing three zeroes dressed as telemetry. Sparkline is a bare stroke with an emphasised endpoint and no axes: the figure above it already states the value, so its only job is the shape. Go: 185 middleware specs pass. Full e2e suite: 396 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): stop the memory chart calling a trade-off an error The VRAM-by-context chart rendered any build over the limit in error red, and escalated the verdict to the error tone as soon as two context sizes crossed it. But an over-limit build still installs — #11288 keeps a test on exactly that — so red overstates what is happening. A model that fits at 32k and not 64k is a trade-off, not a fault. Over-limit bars and the limit line now use the warning tone, which is the constraint colour used everywhere else in this branch: know what you are doing, not you may not. The error tone is reserved for "fits nowhere", where the model genuinely cannot run on this host. Full e2e suite: 397 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): give the new surfaces orchestrated motion Uses the reveal system already in the codebase rather than adding a library: pageReveal, .reveal-stagger and staggerStyle() were built for exactly this, and anime.js would be ~17KB duplicating four lines of CSS for list reveals. The overview's headline figures, attention rows and section lanes stagger in, as do Studio's modality lanes and recent outputs, so a page assembles in the order it is read instead of appearing all at once. Two additions beyond stagger. Rail signals transition on opacity when a poll lands, so a number changing reads as an update rather than a jump cut, and it stays on the compositor so it cannot reflow the rail. The attention block animates its left edge in — the one thing on the page that should announce itself, and on the border rather than the text so nothing moves under a reader. Both are dropped entirely under prefers-reduced-motion, alongside the lane hover inset already handled. Full e2e suite: 397 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): put the generators on a hairline field stack and record the request The workbench treatment from the mocks, applied where it costs least: both changes land on shared surfaces, so all six generators get them at once rather than drifting apart page by page. The control column stops being a shadowed card of boxed groups and becomes a hairline field stack — the panel is the page's left half, not an object floating on it — with uppercase micro-labels matching the eyebrow treatment used elsewhere. Because .media-controls is shared, Images, Video, 3D, Speech, Sound and Audio FX all move together. RequestPanel shows the request the form actually built, with a copy-as-curl. LocalAI is API-first and Studio is the best place in the app to teach its own endpoints: the form stops being a black box, and a result worth keeping can be reproduced from a shell without reverse-engineering which fields the page sent. It records what was sent rather than what the form currently holds, and renders nothing until a request has been made — a panel describing a request nobody made is a tutorial, not a record. Wired into Images and Speech. Full e2e suite: 401 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): make Chat a transcript instead of a bubble thread Rounded, filled, asymmetric bubbles fight a system built on hairlines, and they carry the speaker in shape and side rather than in words. The assistant side had already given up its bubble; this finishes the job. Both roles now run full width down one column, separated by a rule, each with a mono role label. The user turn keeps a left edge in the action tone so the two are still told apart at a glance, without a fill or a corner radius. The avatars go: the accent and the label carry the speaker, so the glyph was decoration once neither side had a bubble. Saying who is speaking in words rather than in geometry is also what survives being read aloud, printed, or looked at by someone who cannot pick the sides apart by colour. Full e2e suite: 404 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * feat(ui): dress the API reference in LocalAI's palette The Swagger page was the last surface still shipping in someone else's colours, which is conspicuous now that everything it links from is dark. Swagger UI has no theming hook, so rather than fork it we serve our own index ahead of the library's wildcard and restate the palette over its stylesheet. The library's own bundle and assets are still what load, so a swagger-ui upgrade cannot silently break the page — this is a skin, not a fork. Two things needed real care. Swagger tints the entire operation row per method via .opblock.opblock-post and friends, so the palette had to match that specificity rather than reach for !important; the method now lives on one edge instead of washing across the row, because a page where every row is a status colour has no status colour left. And the filled method chip put white on pale green, which was the least readable thing on the page — it is an outlined mono chip now, carrying the method in its border and text. Palette values are copied from theme.css rather than referenced: this page is served by Go and never sees the app's CSS. The comment says so, and says to keep them in step. Go: routes and middleware suites pass. Full e2e suite: 405 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] * fix(ui): make tall split-view pages reachable, repair the Agents header, scale titles Three things found by actually using the app rather than measuring it. **Host was unusable.** The shell above a split view is overflow:hidden so the document cannot grow, which left anything taller than the viewport simply unreachable — and Host stacks a resources card, four stat cards and a tab bar above its split, so the bottom of the pane fell off at every window height with nothing to scroll. Every sweep I ran for this was horizontal, which is why it kept coming back clean. The page now scrolls inside the pinned shell. The pane keeps its own scroller: letting it grow instead pushes the document taller and stretches the rail to match, which is the regression e2e/discover-height.spec.js exists to catch, and which the first version of this fix duly caused. **The Agents header controls were unstyled** — "Create Agent" was rendering with the browser's default chrome. The markup had been mangled at some point: six unrelated classes merged into one string on the link, and the label and button left with none at all and empty icons. Repaired, with the inline flex replaced by a shared .header-actions class. **Page titles take the editorial scale from the site**: larger, tracked at -0.04em, on a line height near 1, so a two-word title reads as a statement rather than a label. The typeface is unchanged — DESIGN.md keeps the existing type system — so the whole difference is scale, tracking and leading, which is where the site gets its voice from. This was the biggest reason the running app still did not look like the mocks. Full e2e suite: 404 passed, 4 skipped. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-08-03 00:16:08 +02:00
localai-org-maint-botandlocalai-org-maint-bot	1189c6825b	feat(tracing): persist bounded trace histories (#11203) ... * feat(tracing): persist bounded trace histories Retain API and backend traces below the data path, restore them at initialization, and serialize clears with asynchronous consumers. Assisted-by: Codex:gpt-5 * fix(tracing): satisfy persistence security checks Document why persisted filenames cannot escape the trace directory and explicitly ignore the best-effort temporary-file cleanup result. Assisted-by: Codex:gpt-5 [golangci-lint] --------- Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>	2026-07-30 12:20:01 +02:00
Richard Palethorpe	49ef40a187	feat(classifier/VAD): support voice control on low power devices (#10804) ... * feat(llama-cpp): route Score through the slot loop Score previously bypassed the slot loop with a direct llama_decode: a conflict guard aborted the whole process if scoring raced generation, the config validator had to reject score alongside chat/completion/embeddings, and every candidate re-decoded the full shared prompt. Add SERVER_TASK_TYPE_SCORE to the (patched) upstream server so score tasks are scheduled like any other slot work: generation and scoring serialize naturally, the shared prompt is decoded once per call, and the slot's prompt cache carries the conversation prefix across calls. Context checkpoints at the score boundary and at the cache-divergence point keep SWA/hybrid/recurrent models (e.g. LFM2.5) from re-prefilling the whole prompt per candidate: warm-turn scoring on a 6-option set drops from ~8s to ~0.5s on a desktop CPU. The conflict guard and the validation split are removed; declaring score with generation usecases on one config is now supported and shares the slot cache. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(realtime): classifier wire types and pipeline config Wire types and YAML config for realtime classifier mode: sessions carry a localai_classifier extension (options with canned replies/tool calls, softmax threshold, normalization, history trimming, fallback modes, and a deterministic wake-word address gate), mirrored by pipeline.classifier in the model YAML and surfaced in the config-meta registry. The localai.classifier.result server event reports the full score distribution per turn. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(realtime): classifier response flow Classifier-mode responses: instead of autoregressive generation, each user turn is prefill-scored against the option list (router.ScoreClassifier prompt/candidate shapes over the Score primitive) and the winning option's canned reply and tool call are emitted through the existing response machinery. Below-threshold turns take the configured fallback (none / canned reply / generate); empty transcripts and unaddressed turns (wake word not mentioned) skip scoring entirely. The scoring probe defaults to the latest user message only — small scorers echo canned replies from prior turns back as the top option otherwise. Built for hardware that can afford prompt processing but not decode: with slot-based Score the option list stays KV-cached across turns, so a turn costs roughly one forward pass over the new words. session_update_error events now carry the validation cause instead of a generic message. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(realtime): bound the VAD tick's scan window and buffer retention The VAD tick loop re-scanned the entire input buffer every 300ms and only trimmed it on zero-segment ticks or commits. Audio that keeps producing segments without a committing pause (steady noise a mic pipeline lets through, music, continuous speech) grew the buffer toward the 100MB cap with each tick rescanning all of it — O(n^2), measured at ~3.3ms of silero per buffered second: past ~90s retained, ticks run back to back and pin ~4 cores until the stream stops. Silero's recurrent state only carries a few hundred ms of context, so rescanning old audio buys nothing. Clip the slice handed to the VAD to the largest silence the commit test can need to measure (server_vad silence window or the semantic eagerness fallback) plus a warm-up margin, and rebase the returned segment times so every downstream consumer keeps whole-buffer coordinates. An open turn whose clipped window is all silence now commits (the silence outran the window) instead of being discarded as no-speech. Independently, retain at most 90s of raw buffer, rebasing the live-feed and EOU cursors on trim — this also bounds the previously unbounded VAD-error path. Turn boundaries are otherwise unchanged: no forced commits, no new coordinator states. pipeline.turn_detection.vad_window_sec can widen the scan window; values below the automatic floor are ignored. The tick body is extracted into vadTick so specs can drive turn detection synchronously (same shape as classifySoundWindow); the babble reproduction that pinned 4 cores now plateaus under 10% of one core. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(backend): let per-model threads override the global default ModelOptions overrode a set per-model threads value with the app-level --threads whenever the latter was non-zero — and WithThreads defaults it to the physical core count, so it always was. The YAML threads: knob has been dead config: a tiny VAD model could never opt down from the global pool size. SetDefaults already fills an unset per-model value from the app config, which is the intended precedence; resolve threads through a helper that honors it (explicit threads: 0 still means unset). Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * chore(gallery): single-thread the silero VAD Silero is a ~2MB recurrent model with no exploitable graph parallelism: measured per-call latency is identical at 1 and 10 ORT threads, while every extra pool thread just spin-waits between the realtime loop's frequent tiny inferences. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * docs(realtime): classifier mode, VAD scan window, threads precedence Document the realtime classifier mode (options, threshold guidance, wake-word address gate, empty-transcript handling), the VAD scan window and 90s buffer retention (pipeline.turn_detection.vad_window_sec), the per-model threads precedence, and the M3 classifier note in the realtime state-machine design doc. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * perf(llama-cpp): score all candidates in one batched decode One scoring call is now a single SERVER_TASK_TYPE_SCORE task: the slot decodes the shared prefix (prompt + longest common candidate token prefix) once, then forks one sequence per candidate off it (metadata-only for the unified KV cache, copy-on-write for recurrent state) and decodes every candidate's unique tail in one llama_decode. Previously each candidate was its own task that restored the boundary checkpoint and re-decoded its full tail sequentially, paying per-candidate task and decode overhead. The context reserves SERVER_SCORE_FORK_SEQS extra sequence ids (and recurrent-state cells) beyond the parallel slots via the new common_params::n_seq_score_forks. Forking requires the unified KV cache (already this backend's default) since per-sequence streams would shrink n_ctx_seq; an explicit kv_unified:false disables forking and Score calls that need it fail cleanly. Candidates beyond the fork/output budget decode in successive chunks. Wire contract and scores are unchanged: per-token logprobs are stitched from the shared region and the forked tails. Verified bitwise deterministic call-to-call and independent of candidate order (no cross-fork leakage via equal-length candidate swap); ranking matches the per-candidate implementation on the drone battery (winner softmax 0.99996 vs 0.99997), and >16-candidate chunking, prefix-of-another and empty candidates all pass. Measured on a desktop CPU: warm /api/score calls 0.52s -> 0.23s; warm realtime classifier turns 196-303ms. The 9-candidate drone turn decodes ~17 unique tail tokens in one batch instead of nine sequential ~220ms checkpoint-restore tasks. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(realtime): gate scoring capacity by model usecase Reserve llama.cpp scoring slots only for models that explicitly declare the score usecase, while allowing score to coexist with chat and completion. Reject incompatible unified-KV settings and classifier activation on models without scoring capacity. Propagate application defaults when resolving realtime and preload pipeline stages so unset thread counts are resolved consistently without overriding explicit model settings. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(ci): honor APT mirrors in the prebuilt llama-cpp compile step The builder-prebuilt path installs gcc-14 with apt directly and ignored the APT_MIRROR/APT_PORTS_MIRROR build args the from-source path already honors, so an ubuntu mirror outage broke every arm64 backend build. Pass the args into the stage and run apt-mirror.sh (already in the build context via COPY . /LocalAI) before the apt step. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(realtime): classifier argument slots via constrained completion Hybrid classify-then-complete: a classifier option's canned tool call can declare typed argument slots (number | enum | string, with defaults and prompt hints) referenced as "{{name}}" in the arguments template. When the option wins, the slots are filled by a short grammar-constrained completion that continues the exact scoring prompt — rendered by the same cached ScoreClassifier, so the llama.cpp prompt cache is already warm — with the chosen route JSON re-opened at the first slot field. A GBNF grammar pins the field skeleton and frees only the values; temperature 0, a couple dozen tokens at most (~300ms on a desktop CPU for two slots). Slot declarations and hints ride the option descriptions in the shared system prompt, informing scoring and the fill alike at no per-turn token cost. The localai.classifier.result event carries the final arguments and a fill_latency_ms. On inference failure the slots' defaults apply; a slot without a default fails the response (or falls through with fallback.mode: generate). Slot filling requires completion alongside score in the scoring model's known_usecases. Verified end-to-end on the Pi drone demo: "fly forward three meters" in distance mode classifies forward and infers {"distance": 3, "units": "meters"} in ~310ms, and the drone flies exactly 3 units. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(realtime): splice filled slot values into classifier replies A classifier option's spoken reply can now reference its tool's argument slots ("Going forward {{distance}} {{units}}."): the values inferred by the slot-fill completion — or the recovery defaults — are spliced into the reply as plain text before it is emitted, so what the assistant says confirms what it actually inferred. Placeholders without a value stay literal, and options without slots are untouched. FillToolArguments now returns the raw slot values alongside the spliced arguments JSON to make the reply templating possible. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(realtime): harden classifier slot completion Reserve context for constrained slot filling, size completions from their encoded output, and encode enum grammar literals as valid JSON. Reject empty enum values and cover the failure modes with regression tests. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(realtime): prewarm the classifier scoring prompt on registration Swapping a session's classifier option list (a voice-switched command mode, for instance) made the next turns pay a full re-prefill of the new option-list prompt — measured 2.4s vs 0.3s warm on a desktop CPU, and worse: on hybrid-memory models like LFM2.5, whose state cannot be partially rewound (llama.cpp can only restore checkpoints), *every* probe change re-prefilled from scratch whenever the last checkpoint missed the probe boundary, so even same-list turns intermittently cost full prefills. Registering an option list (pipeline seed or session.update) now fires a best-effort background prewarm: two throwaway scores with distinct probes. The first prefills the new option-list prompt; the second, diverging exactly where per-turn probe text starts, plants the backend's rewind point (KV checkpoint) at the stable-prefix boundary that every real turn reuses. The prewarm hides behind the canned mode-switch reply — by the time it finishes speaking, the cache is warm. Idempotent per option set, detached from the registering request's lifetime. Measured on the drone demo (LFM2.5-1.2B, desktop CPU): first turn after a mode switch 2374ms -> 340ms; intermittent same-list full prefills (1.3-2.1s) all -> under 0.5s. For clients that swap lists frequently, options: [parallel:2] on the scoring model additionally keeps one slot per list via prefix-similarity routing (+26MB RSS, unified KV). Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * perf(llama-cpp): checkpoint scoring at the caller-declared stable prefix Hybrid-memory models (LFM2.5 shortconv, Qwen3.5 deltanet — where new small models are headed) cannot rewind their state, so any prompt-cache reuse that needs a rewind falls back to a full re-prefill. For classifier scoring that meant every probe change re-processed the whole option-list prompt: the server's checkpoints were placed reactively (at wherever the previous task happened to diverge), so a checkpoint past the next divergence was erased rather than restored — measured as intermittent 2-10s turns on prompts with a 95%+ common prefix. The classifier now computes the probe-invariant prompt prefix once (the byte-wise common prefix of two synthetic probe renders) and declares its length with every Score request; the server maps it to a token boundary and forces a KV checkpoint exactly there on each score prefill. That checkpoint sits at or before every future divergence under the same option list, so it always survives and always restores — repeat scoring costs probe+candidates regardless of how the probe changes. Also: - prewarm reruns on every option-list registration instead of memoizing per list: with boundary checkpoints a redundant rewarm costs two probe-sized decodes, while skipping one after a slot eviction (three lists sharing fewer slots evict in LRU cascades) silently moves a full re-prefill onto the user's next turn - new llama.cpp backend option rs_seq:N exposes bounded recurrent-state rollback outside speculative decoding; measured impractical for deltanet-scale states (65GB for 64 snapshots on Qwen3.5-4B) but cheap insurance for small-state models - docs: the multi-list recipe (parallel:N + sps:0.5 — the default slot similarity threshold funnels distinct lists onto one slot) Measured on the drone demo (LFM2.5-1.2B scorer, desktop CPU), steady state: every turn 285-421ms including mode switches, vs 2.4s post-switch and intermittent 1.3-2.9s re-prefills before. Assisted-by: Claude:claude-fable-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(realtime): align classifier cache guidance Document the single-score prewarm behavior and clean the vendored score patch formatting. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(llama-cpp): guard score task for fork backends TurboQuant and Bonsai reuse the primary gRPC server against llama.cpp forks that do not carry LocalAI's slot-based Score patches. Compile the Score integration only for the patched primary backend and return UNIMPLEMENTED from fork builds instead of referencing absent task types and common_params fields. Assisted-by: Codex:gpt-5 [gh] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(dev): generate gRPC code before commit lint The coverage phase regenerates ignored protobuf bindings, but lint runs first and can fail against missing or stale output. Generate the pinned bindings before lint so the gate always type-checks the current schema. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-07-29 12:50:22 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto	ff299df453	perf(http): gzip responses, cache hashed assets, bound the trace endpoints (#11056) ... Three measured HTTP-layer regressions on a live deployment, fixed together because they all shape the bytes on the wire. 1. No compression. The server sent no Content-Encoding regardless of what the client asked for, confirmed with curl straight at 127.0.0.1:8080 so it was not an ingress artefact. Adds gzip middleware, on by default and configurable via LOCALAI_DISABLE_HTTP_COMPRESSION and LOCALAI_HTTP_COMPRESSION_MIN_LENGTH (default 1024 bytes so tiny bodies are not wastefully wrapped). Streaming routes are skipped explicitly: an SSE Accept header, a WebSocket upgrade, and the completion / SSE / log-tail path prefixes, because whether a completion request streams is decided by the request body, which the middleware runs too early to see. Already-compressed formats (woff2, png, mp4, ...) are skipped too; gzip made those marginally larger. Measured over the embedded React build: JS+CSS 2815 KB raw to 808 KB gzipped (3.48x). 2. No cache headers on content-hashed assets. Vite hashes the filenames, so a given /assets/ URL can never change content, yet they shipped with no Cache-Control, ETag or Last-Modified, and the browser re-fetched the whole bundle on every navigation with no conditional request available. /assets/* now carries public, max-age=31536000, immutable. index.html stays no-cache so a deploy is picked up, and the unhashed locale JSONs get a short TTL rather than the immutable one. 3. Unbounded trace endpoints. /api/traces returned 21,033,606 bytes in 4.65s and /api/backend-traces 3,471,682 bytes in 1.50s, and the admin UI polls both every few seconds. The ring buffer holds up to 1024 entries, each embedding full input_text payloads. Both list endpoints now take limit / offset / full, default to 50 entries, and strip the heavy fields (request and response bodies plus headers for API traces, body and data for backend traces) unless full=true. Every trace gets a process-lifetime ID and GET /api/traces/{id} and /api/backend-traces/{id} serve the full record, which is what the UI fetches when a row is expanded. The list body stays a JSON array; paging metadata rides in X-Total-Count, X-Trace-Offset and X-Trace-Limit. Reproducing the live shape in a test, the polled payload goes from 21,131,097 bytes to 7,201 bytes. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-22 22:51:25 +02:00
localai-org-maint-botandEttore Di Giacinto	4be6e22b5f	feat(webui): surface user, client IP and user agent in API traces (#10886, #10887) (#10907) ... The Operate → Traces "API Traces" panel already recorded who made each request (user_id/user_name) but never showed it, and did not capture the caller's network identity at all. Operators asked to see the requesting user (#10886) and the client IP + user agent (#10887) so a trace can be attributed to who/what issued it. Backend: add ClientIP and UserAgent to APIExchange and populate them from echo's c.RealIP() (honours X-Forwarded-For / X-Real-IP behind a trusted proxy) and the request's User-Agent header. Both are omitempty and the /api/traces swagger response is map[string]any, so this is additive. UI: add a sortable "User" column to the API traces table and a metadata block (User / Client IP / User Agent) at the top of the expanded row detail. Fields render only when present, so older buffered traces and unauthenticated/local requests degrade cleanly. Adds an e2e spec covering the new column value and the expanded metadata. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-17 23:47:31 +02:00
LocalAI [bot]andEttore Di Giacinto	fe4f425fb5	fix: correct scheme/host on self-referential URLs behind an HTTPS reverse proxy (#10482) (#10504) ... * fix(http): harden BaseURL proxy scheme/host detection Split comma-separated X-Forwarded-Proto and honor the RFC 7239 Forwarded header so generated links use https behind common reverse-proxy setups. Refs #10482 Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(http): honor explicit external base URL in BaseURL When _external_base_url is set in the request context it dictates the origin (scheme+host+port); the proxy path prefix is still appended. Refs #10482 Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(config): generalize LOCALAI_BASE_URL to ExternalBaseURL LOCALAI_BASE_URL now sets a single instance-wide external base URL used for OAuth callbacks and all self-referential links. A Pre middleware stamps it into the request context for middleware.BaseURL. Refs #10482 Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: document LOCALAI_BASE_URL and reverse-proxy headers Refs #10482 Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(http): cover parseForwarded edge cases; clarify base-url flag group Adds direct unit coverage for quoted/malformed/multi-element Forwarded headers and regroups the external base URL flag away from auth-only. Refs #10482 Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-06-25 08:10:59 +02:00
LocalAI [bot]andEttore Di Giacinto	9565db5f94	feat(models): model aliases - redirect a model name to another configured model (#10414) ... * feat(config): add model alias field and self-validation Add ModelConfig.Alias (yaml: alias), IsAlias(), and an alias short-circuit at the top of Validate() that rejects self-reference and forbids setting backend/parameters.model on a pure-redirect alias. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(config): resolve and validate model alias targets in the loader Assisted-by: Claude:opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(middleware): resolve model aliases and stamp requested/served identity Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(modeladmin): reject alias configs with invalid targets on create/edit Validate alias targets at create/swap entry points (ImportModelEndpoint, EditYAML, PatchConfig) so a dangling, chained, or disabled alias target is rejected at save time rather than surfacing as a runtime error. Assisted-by: Claude:opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(api): add GET /api/aliases to list model aliases Adds an admin-gated read-only endpoint that lists every model alias config as {name, target} pairs, backed by the loader's existing GetAllModelsConfigs(). Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(mcp): add set_alias and list_aliases tools Expose model-alias management over the LocalAI Assistant MCP surface: list_aliases (read-only, GET /api/aliases) and set_alias (mutating). SetAlias is swap-first: PATCH /api/models/config-json/:name swaps an existing alias's target (validated, non-destructive) and a 404 falls back to POST /models/import to create a fresh {name, alias} config. The inproc client mirrors this via ConfigService.PatchConfig + a create path modeled on ImportModelEndpoint. Deletion reuses delete_model. Assisted-by: Claude:claude-opus-4 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * style(mcp): replace em dashes in alias tool comments Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(config-meta): expose alias as a model-select field Add an 'alias' section to DefaultSections() and an 'alias' field override in DefaultRegistry() so the schema-driven React editor renders the new top-level ModelConfig.Alias field as a model picker in its own section. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ui): add alias template card and Manage alias badge Add an 'Alias / Routing' template to the create-flow gallery that seeds a minimal name + alias config, and a read-only 'alias -> target' badge on the Manage Models tab. The capabilities row payload does not carry the alias field, so the badge resolves targets from GET /api/aliases looked up by name. Assisted-by: Claude:claude-opus-4 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: document model aliases Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(swagger): regenerate for GET /api/aliases Adds the /api/aliases path and AliasInfo schema generated from the ListAliasesEndpoint annotation. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(localai): check os.RemoveAll error in aliases_test Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix: correct alias conversion docs and advertise /api/aliases in instructions Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(mcp): write alias config 0600 to satisfy gosec G306 The inproc createAlias path wrote the alias YAML with 0644, which gosec flags as a new G306 finding on the PR. The LocalAI process is the sole reader/writer of model configs, so 0600 is correct and keeps the scan clean. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-06-20 22:38:42 +02:00
Richard Palethorpe	3fa7b2955c	feat(pii): NER tier engine — privacy-filter.cpp backend + NER-centric PII filter (#10360) ... Squashed feat/pii-ner-tier-engine rebased onto master (was 45 commits; see backup/pii-ner-tier-engine-prerebase). Net change: - privacy-filter.cpp: standalone GGML engine for the openai-privacy-filter PII/NER token classifier, wired as a LocalAI gRPC backend (CPU/CUDA/Vulkan). TokenClassify moves off the patched llama.cpp path onto this backend. - PII filter reworked to be NER-centric (encoder/NER detection tier scanning whole conversations as one document), with a recreated bounded restricted- regex secret-matching pattern detector tier alongside it (per-model pii_detection.builtins / .patterns + core/services/routing/piipattern). - Detection labelled by source (ner vs pattern); backend trace / confidence / debug observability; analyze/redact exposed as a synchronous API. - Instance-wide default detector policy + per-usecase default-on; request filtering extended to completions, embeddings, edits & Ollama. - React UI: NER-centric PII editor, detector-models table, pattern/builtins editor, middleware default-policy UI. - Gallery: privacy-filter-multilingual token-classify model + NER install filter; token_classify known_usecase; batch sized to context for NER models. privacy-filter backend registered in the backend gallery (cpu/vulkan/cuda-13 meta + image entries with a capabilities map) matching its CI matrix jobs, and an /import-model auto-detect importer (PrivacyFilterImporter, narrow privacy-filter GGUF detection) replacing the prior pref-only registration. Reconciled against master's independent evolution: - Dropped master's PIIPatternOverrides feature (global-pattern runtime overrides + /api/pii/patterns API + runtime_settings.json persistence). The per-model NER + pattern-detector design supersedes it; it was built on the global redactor pattern set this branch replaced. - Reverted the llama.cpp Score carry-patch (0006-server-task-type-score): removed the patch and restored master's grpc-server.cpp Score RPC (direct llama_decode, slot-loop bypass) and LLAMA_VERSION pin, plus master's model_config validation forbidding score + chat/completion/embeddings on llama-cpp. token_classify is unaffected (it runs on the privacy-filter backend, not llama-cpp). Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-06-18 11:45:22 +01:00
LocalAI [bot]andEttore Di Giacinto	1ab61a0875	feat: generic chat_template_kwargs (model config + per-request metadata) (#10359) ... * feat(config): add chat_template_kwargs model field + resolver Adds the ChatTemplateKwargs model-config map and RequestMetadata carrier, plus ResolveChatTemplateKwargs which layers the config map under coerced request metadata. Foundation for generic jinja chat-template kwargs (issue #10329). Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(backend): forward resolved chat_template_kwargs blob to backends gRPCPredictOpts now merges per-request client metadata over the server-derived enable_thinking/reasoning_effort (reaching all backends via the standalone keys) and serialises the resolved chat_template_kwargs map into a JSON blob for llama.cpp, written last so a client cannot clobber it. Issue #10329. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(http): wire request metadata to config.RequestMetadata The OpenAI request metadata field was parsed but unused; stamp it onto the per-request ModelConfig so gRPCPredictOpts forwards it as chat_template_kwargs overrides. Issue #10329. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): generic chat_template_kwargs merge (drop per-key blocks) Replace the per-key enable_thinking/reasoning_effort handling in both the streaming and non-streaming chat paths with a single block that parses the chat_template_kwargs JSON blob resolved by the Go layer and merges every key into body_json. New jinja template levers (e.g. preserve_thinking) now need no C++ change. Issue #10329. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: document custom chat_template_kwargs (model + per-request) Issue #10329. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(backend): pin reasoning_effort as a string in the chat_template_kwargs blob Issue #10329. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(http): e2e guard pinning chat_template_kwargs forwarded to gRPC Adds an ECHO_PREDICT_METADATA marker to the mock-backend that echoes the received PredictOptions.Metadata, and an app_test.go spec that drives a real /v1/chat/completions request (model chat_template_kwargs + per-request metadata override) and asserts the exact metadata + chat_template_kwargs blob the REST layer forwards to gRPC. Locks the REST->gRPC contract against regressions. Issue #10329. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(config): grandfather chat_template_kwargs in registry coverage chat_template_kwargs is a free-form map[string]any (like engine_args, already on the list), not a scalar the config UI registry can surface, so it is exempt from the registry-entry requirement. Fixes the TestAllFieldsHaveRegistryEntries failure introduced by the new field. Issue #10329. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-06-16 12:16:34 +02:00
Richard Palethorpe	085fc53bbc	fix(router): production-ready request router + auto-size batch for embedding/rerank (#10104) ... * fix(router): score classifier production-readiness Conversation trimming runs through the classifier model's chat template and trims by exact token count, sized to the model's n_batch which is now scaled to context so long probes can't crash the backend. Missing chat_message templates are a hard error at router build time. Router- facing factories (Embedder/Scorer/Reranker/TokenCounter) re-resolve ModelConfig per call so a model installed post-startup doesn't bind a stub Backend="" config and silently fall into the loader's auto- iterate path. New 'vector_store' backend trace recorded inside localVectorStore on every Search/Insert — including the backend-load-failure path that previously vanished into an xlog.Warn — with outcome tagging (hit/miss/empty_store/backend_load_error/find_error/insert_error/ok). Companion cleanup drops misleading similarity:0 and input_tokens_count:0 from non-hit and text-mode traces. Gallery local-store-development aliases to 'local-store' so the master image satisfies pkg/model.LocalStoreBackend lookups from the embedding cache. Misc: llama-cpp TokenizeString reads the correct 'prompt' JSON key (the original bug); ModelTokenize nil-guard; non-fatal mitm proxy startup; PII 'route_local' renamed to 'allow' with docs/UI in sync; model-editor footer no longer eats the edit area on small screens; several config-editor template/dropdown/section fixes. Tests: e2e router specs (casual/code-hint + long-conversation trim), vector_store trace specs, lazy-factory specs, gallery dev-alias resolution, Playwright trace badge + scroll regression. Assisted-by: Claude:claude-opus-4-7 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(backend): auto-size batch to context for embedding and rerank models Embedding and rerank models pool over the whole input in a single physical batch (n_ubatch). With batch left at the 512 default, the backend rejects longer inputs with "input is too large to process", silently capping a large-context embedder (e.g. 8k/32k) at 512 tokens. Size n_batch to the context for these single-pass usecases, mirroring the existing FLAG_SCORE behaviour; an explicit batch: still wins. Extracts EffectiveContextSize/EffectiveBatchSize from grpcModelOpts so the effective decode window has one home for other callers to reuse. Adds an e2e-aio regression test that embeds a >512-token input. The AIO embedding model is switched to nomic-embed-text-v1.5 (2048 context) because the previous granite model was capped at 512 tokens and could not exercise the larger batch. Assisted-by: claude-code:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(gallery): raise arch-router scoring output cap via parallel:64 Scoring decodes the whole prompt+candidate in a single llama_decode and reads one logit row per candidate token. The vendored llama.cpp server caps causal output rows at n_parallel, so the default of 1 aborts with GGML_ASSERT(n_outputs_max <= cparams.n_outputs_max) on multi-token route labels. Set options: [parallel:64] on both arch-router quant entries to lift the cap; kv_unified (the grpc-server default) keeps the full context per sequence, so this does not split the KV cache. Assisted-by: claude-code:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-06-12 16:21:15 +02:00
LocalAI [bot]andEttore Di Giacinto	e837921c2c	feat: forward reasoning_effort to the backend so jinja models honor it (#10184) ... * feat: forward reasoning_effort to the backend so jinja models honor it reasoning_effort was only mapped to the binary enable_thinking toggle and otherwise reached Go-side templates — it was never sent to the backend. So jinja-templated models whose chat template keys on reasoning_effort (gpt-oss Harmony, LFM2.5) could not be driven by it: LFM2.5 ignores enable_thinking and kept emitting <think>. Forward the effective reasoning_effort to the backend as a chat_template_kwarg (mirroring enable_thinking) in grpc-server.cpp, and put it in PredictOptions metadata (gRPCPredictOpts). Add a config-level default: ModelConfig.reasoning_effort and Pipeline.reasoning_effort, resolved by ModelConfig.ApplyReasoningEffort (request value overrides config default, none->disable / level->enable, an operator's reasoning.disable wins). request.go now uses that helper. Assisted-by: Claude:claude-opus-4-8 go test, golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(realtime): set the pipeline LLM's reasoning_effort Apply Pipeline.ReasoningEffort to the pipeline's LLM config when the realtime model is built (per-session copy, overrides the LLM's own reasoning_effort), and surface the resolved effort on the template input so Go-templated models get it too. jinja models receive it via the backend metadata. This lets a realtime pipeline disable thinking on models that only honor reasoning_effort (e.g. LFM2.5), which enable_thinking can't. Assisted-by: Claude:claude-opus-4-8 go test, golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-06-05 13:45:43 +00:00
Richard Palethorpe	718223f33b	feat(localvqe/audio): v1.3 release and add spectrograms to audio transform UI (#10113) ... * chore(localvqe): update backend to v1.3, add v1.2/v1.3 gallery models Bump the LocalVQE backend pin 72bfb4c6 -> b0f0378a, which adds the v1.2 (1.3 M) and v1.3 (4.8 M) GGUF SHA-256s to the upstream released-models allowlist (and the arch_version=3 loader) so both load without LOCALVQE_ALLOW_UNHASHED. Add gallery entries for localvqe-v1.2-1.3m and localvqe-v1.3-4.8m (SHA-256 verified against the downloaded weights) and update the audio-transform docs to make v1.3 the current default while noting the compact v1.1/v1.2 alternatives. Assisted-by: Claude:claude-opus-4-8 Claude-Code Signed-off-by: Richard Palethorpe <io@richiejp.com> * chore(flake): add ffmpeg-headless to the dev shell pkg/utils/ffmpeg_test.go shells out to the `ffmpeg` CLI, and the pre-commit gate runs those tests via `make test-coverage`. Without ffmpeg in the dev shell the gate fails with "executable file not found in $PATH". The headless build provides the CLI without GUI/X deps. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(localvqe): parse WAV by walking RIFF sub-chunks Walk the RIFF chunk list instead of assuming the canonical 44-byte header layout. Real inputs (browser-recorded clips, ffmpeg output with an 18/40-byte extensible `fmt ` chunk or trailing LIST/INFO metadata) would otherwise splice header/metadata bytes into the PCM stream as an audible impulse. Honour the `data` chunk size and validate that both `fmt ` and `data` chunks are present. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * fix(security-headers): allow blob: in connect-src for waveform fetch The waveform renderer XHRs/fetches a freshly-created blob: object URL (e.g. an uploaded or enhanced clip before it has a server URL). XHR/fetch of blob: is governed by connect-src, not media-src, so it was blocked by the CSP. Add blob: to connect-src. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * feat(react-ui): add input/output spectrogram view to AudioTransform The transform page only showed time-domain amplitude waveforms, so you could see how loud a clip was but not which frequencies the model touched. Add a time x frequency spectrogram heatmap and render the input and output spectrums side by side, so it's visible which bands the enhancement attenuates (bright input bands that go dark in the output). Computed client-side via a Hann-windowed STFT over both clips (a small dependency-free radix-2 FFT), defaulting to the LocalVQE 512/256 frame geometry. This shows the net input->output spectral change; the model's internal gain mask is not exposed by the backend. - src/utils/fft.js radix-2 FFT - src/hooks/useSpectrogram.js decode + STFT -> normalised dB magnitude grid - src/components/audio/Spectrogram.jsx canvas heatmap (magma colormap) - AudioTransform.jsx dual-spectrogram panel + CSS - e2e spec + UI coverage baseline bump (38.29 -> 39.0; measured ~39.4-40.2) Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> * test(react-ui): make UI coverage deterministic, tighten the gate UI e2e line coverage swung ~1pp run-to-run (39.1% <-> 40.2%), which forced a loose 0.8pp tolerance on the monotonic gate — a band wide enough to let a real ~300-line regression through silently. The swing was a bug, not inherent jitter: the 'Create Agent navigates' spec ended on the URL assertion, so AgentCreate.jsx's ~400 lines were collected only when its render happened to beat the coverage teardown. Wait for the page to actually render (assert its heading) so those lines are covered every run. With the race gone, repeated runs land within ~0.013pp of each other, so: - tighten UI_COVERAGE_TOLERANCE 0.8 -> 0.1 (noise floor, not a drift band) - set the baseline to the real, reliably-achieved value (39.0 -> 39.86) Localised by running the V8-coverage suite repeatedly and diffing per-file line coverage; AgentCreate.jsx was the sole ~1pp flipper. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Richard Palethorpe <io@richiejp.com> --------- Signed-off-by: Richard Palethorpe <io@richiejp.com>	2026-05-31 23:56:46 +02:00
LocalAI [bot]andEttore Di Giacinto	4a2cc64d07	feat(reasoning): honor per-request reasoning_effort on chat completions (#10082) ... The OpenAI `reasoning_effort` field only reached the prompt template; it never toggled the backend's thinking. Map it onto ReasoningConfig.DisableReasoning (which becomes the enable_thinking gRPC metadata) in the request merge, so reasoning_effort="none" disables reasoning per request: the use case from #10072 (run a single Qwen3-style model and turn reasoning off for low-latency tasks while keeping it on for others). Effort levels (minimal/low/medium/high) enable thinking unless the model config explicitly disabled it (reasoning.disable: true wins and is never re-enabled by a request); "none" always disables. Closes #10072 Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-05-29 22:09:07 +00:00
LocalAI [bot]andEttore Di Giacinto	06e777b75e	feat(distributed): gated X-LocalAI-Node response header (middleware + wrapper) (#9976) ... * feat(distributed): add per-request node ID context holder Introduce pkg/distributedhdr, a leaf package carrying a per-request *atomic.Value holder for the picked worker node ID from the SmartRouter (core/services/nodes) up to the HTTP response writer wrapper (core/http/middleware). Avoids the import cycle that a shared key in either consumer would create. Exposes NewHolder, WithHolder, Holder, Stamp, Load, Inherit. The holder is atomic.Value so cross-goroutine publish from the router to the response writer wrapper is race-clean. Assisted-by: Claude:claude-opus-4-7[1m] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(distributed): add ExposeNodeHeader middleware + response writer wrapper New ApplicationConfig.ExposeNodeHeader bool + --expose-node-header CLI flag / LOCALAI_EXPOSE_NODE_HEADER env var (default off; the node ID reveals internal topology and is opt-in). The middleware creates a per-request *atomic.Value holder, attaches it to c.Request().Context() via distributedhdr.WithHolder, and wraps c.Response().Writer with a custom http.ResponseWriter that sets the X-LocalAI-Node header on first Write / WriteHeader / Flush by reading the holder. Implements http.Flusher, http.Hijacker, Unwrap so it composes cleanly with Echo and http.NewResponseController. request.go propagates the holder onto derived contexts via distributedhdr.Inherit so the holder survives the correlation-ID context replacement. Unit + race-clean concurrency + integration specs. Assisted-by: Claude:claude-opus-4-7[1m] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(distributed): stamp node ID in router and wire middleware to inference routes ModelRouterAdapter.Route stamps the picked node ID into the per-request holder via distributedhdr.Stamp(ctx, result.Node.ID) right after replica selection. Wire ExposeNodeHeader middleware to: - OpenAI chat/completion/embeddings + audio transcriptions/speech + image generations/inpainting - Anthropic /v1/messages - Ollama /api/chat, /api/generate, /api/embed, /api/embeddings - Jina /v1/rerank - LocalAI /v1/vad The middleware's wrapper reads the holder on first byte and sets the X-LocalAI-Node response header before delegating to the underlying writer. Per-request scope means no race under concurrent multi-replica routing. Assisted-by: Claude:claude-opus-4-7[1m] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): thread request context through backend Load + cover ctx propagation Five non-OpenAI backend helpers were silently using app.Context instead of the request context for the gRPC backend call: transcription, TTS, image generation, rerank, VAD. Effect: distributedhdr.Stamp in the router callback was a silent no-op for these paths, AND client cancellation didn't propagate to in-flight inference. Thread c.Request().Context() (or the equivalent input.Context after the request middleware has installed the correlation-ID derived context) through each helper and into ModelOptions via model.WithContext(ctx). ImageGeneration's signature gains a leading ctx parameter; in-tree callers (openai image, openai inpainting, openai inpainting_test) are updated to match. ModelEmbedding gains a leading ctx parameter for the same reason; the openai and ollama embedding handlers pass the request context through. chat_stream_workers.go defers the initial role=assistant chunk emission until the first token callback so the wrapper's lazy X-LocalAI-Node lookup against the loader runs AFTER ml.Load has stamped the per-modelID node ID; semantically identical for clients (role still arrives before any text). Regression test core/backend/ctx_propagation_test.go pins ctx propagation for all five helpers. Docs updated to enumerate the full endpoint coverage of the --expose-node-header flag. Assisted-by: Claude:claude-opus-4-7[1m] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-05-25 10:51:48 +02:00