mirror of
https://github.com/mudler/LocalAI.git
synced 2026-11-03 11:04:53 -05:00
16acc8926bb8a27b67dc4d419dbceaeb4110f7af
8847
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 mudler-agentandEttore Di Giacinto
|
16acc8926b |
feat(addons): simplify Office and discover Tables (#13029)
* feat(tables): list accessible databases Let agents discover existing databases through the host gateway. Keep listing actor-scoped and separate from executable tools. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(tables): verify database listing authority Exercise model discovery through the real owner and carrier. Check shared access, revocation, denial and resource limits. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(tables): bundle database search guidance Teach agents to discover existing databases before querying records. Package and verify the skill with the Tables add-on. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(tables): cover packaged chat skill gates Load the real receipt-verified Tables instructions through the temporary Studio workspace. Check the index and load refusal after runtime, chat switch and saved-agent grant changes. This fixture starts at the receipt and broker boundary; packaging tests cover artifact construction and integrity separately. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(office): parse Markdown and CSV inputs Convert bounded text inputs into document and workbook structures. Render supported Markdown formatting without fetching external content. Preserve spreadsheet fields as strings rather than inferring formulas. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(office): preserve list markers, breaks and PDF page margins Cover actual DOCX breaks and tabs, explicit zero list starts, empty list rejection, and PDF margin/pagination regressions. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(office): guard Markdown parsing work Check cancellation and nesting during parser callbacks. Validate table dimensions before padded table allocation. Keep parser state isolated to each conversion. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(office): recognize tables before applying limits Leave mismatched table-like paragraphs as ordinary Markdown. Keep oversized real tables bounded before allocation. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(office): simplify document creation inputs Accept Markdown for DOCX and PDF, and CSV for single-sheet XLSX files. Replace nested creation schemas to reduce model tool grammar complexity. Keep existing read, edit and PowerPoint behavior and internal office IDs. Present the add-on as Simple Office and document the compatibility change. Keep same-style Markdown fragments editable in generated DOCX files. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ci): align add-on fixtures and test inventories Tables fixtures must carry the skill their manifest declares. Preserve its bytes in package tests and mark only binaries executable in UI fixtures. Assert that installation keeps the shipped skill intact. Register the new Tables tests and the connector case from master in exact suite inventories. Include the new database read in tool counts. Run Office regressions through Ginkgo and check reader close errors. Keep all document assertions and the existing CI gates intact. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: nib:gpt-6.1-sol * fix(ci): restore Office and packaging checks Keep the generated Office label consistent with Simple Office. Install Poppler for the PDF layout regressions on Linux and macOS. Separate successful runtime setup from the 300 ms deadline under test. Keep the timeout, cancellation, stop and admission assertions intact. Refresh the packaging inventory from all 209 executed cases, retaining repeated platform checks and the Office and Tables skill checks. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-botandmudler
|
11fad911c4 |
chore(website): refresh the counters (#13028)
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
mudler-agentandEttore Di Giacinto
|
a6589e4e60 |
fix(connectors): preserve disabled add-on discovery (#13030)
fix(connectors): preserve discovery when add-ons are disabled Return healthy empty connector discovery for explicitly disabled add-ons while keeping missing configured gateways and unreachable nonowners unavailable. Cover offline Telegram persistence through the real connector transport and synchronize the exact lifecycle/apps test inventories. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
9206959564 |
fix(connectors): route calls through the owner (#13026)
* fix(connectors): route calls through the owner Route connector discovery and calls through the private carrier so runtime-free frontends use the owner runtime. The owner validates each call scope and loads stored credentials rather than trusting caller tokens. Limit prospective token checks to identity requests. Cover pinned peer admission, scope refusals, stored credentials, and bounded decoding through the actual carrier. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(connectors): preserve host authority denials Reject connector creation and token replacement when the owner denies access. Keep transport and platform outages permissive during token verification. Cover both cases through the real carrier. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(connectors): distinguish authority outages Preserve ErrUnavailable when the connector authority cannot serve calls. Only explicit authority denials should reject otherwise complete saves. Cover create and token replacement during an authority outage, including persisted bindings and subsequent calls through the owner. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(connectors): cover owner routing contracts Exercise owner routing through the authenticated carrier, including polling, cancellation, payload limits, stored tokens, and transport errors. Synchronize the test fixtures for concurrent calls and race checks. Document multi-frontend readiness and token verification behavior so operators can distinguish empty discovery from owner outages. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(connectors): cover discovery failure fences Assert failed discovery leaves persisted bindings unchanged and skips runtime calls during connector creation. Distinguish healthy empty discovery from lost ownership and runtime fencing after a real persistence failure. Verify discovery recovers when a new owner restores the runtime. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(connectors): explicitly ignore response close errors Use the existing deferred close pattern for carrier responses and the owner-routing regression test to satisfy errcheck. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
5b13a606f9 |
fix(agents): allow aliases in the model picker (#13024)
Reuse the chat picker for agent creation and editing so agents can select chat-capable aliases and retain their alias names. Resolve capability flags through the existing one-hop alias resolver. Keep alias metadata and ID-based permissions unchanged, and give broken or chained aliases no usable capabilities. Cover alias persistence, default selection, filtering and permissions. Document that agents save the alias name rather than its target name. Assisted-by: nib:unknown Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
e03f427ed5 |
feat(workspace): use filesystem storage without S3 (#13012)
* fix(workspace): coordinate shared filesystem blobs Sync filesystem publication and isolate materialized tool files from immutable blob inodes. Reject symlink paths within the blob tree. Opt shared PostgreSQL participants into revision-lock coordination for ingest, reference publication, and purge checks and deletion. Preserve legacy ingest order when the explicit option is disabled. Verify process publication and interruption, injected failures, and shared-schema PostgreSQL races across independent connection pools. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(workspace): bound shared filesystem blob cleanup Honor immediate caller cancellation and give delayed PostgreSQL cleanup an independent 30-second budget for revision-lock acquisition and reference checks. Preserve blobs when cleanup cannot acquire the lock. Cover cancellation, delayed cleanup, and pool release. Convert publication tests to Ginkgo/Gomega while retaining independent-process child bootstrapping. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(workspace): opt into shared filesystem blobs Allow Files participants to share immutable blobs without S3. Require PostgreSQL metadata and reject invalid explicit roots at startup. Keep default storage and model staging unchanged. Document the volume, permissions and coordinated backend-switching contract. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(workspace): clean up failed shared startup Cancel the owned startup context and close the auth pool when explicit shared Files startup fails. Preserve the caller context and legacy startup behavior. Cover early validation and late Files failure with PostgreSQL-backed cleanup assertions. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(workspace): select blob storage automatically Use configured S3 for every workspace participant, including owners without distributed services. Otherwise retain DataPath/chats/blobs through the direct filesystem adapter, without another path setting. Derive PostgreSQL publication and purge coordination from the blob backend so non-distributed owners also protect shared filesystem bytes. Keep shared mount and metadata identity an explicit operator contract. Remove the opt-in probe and document storage selection without migration. Verify automatic selection with red/green regressions, S3 constructor spies, focused startup tests and blob race tests. Exercise shared files and PostgreSQL coordination against an owned PostgreSQL 16 fixture. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(workspace): clean up configured S3 startup failures Own the startup child context for configured S3 as well as PostgreSQL metadata. Cancel background work, shut down opened resources and close the auth pool on fatal initialization errors without canceling the caller context or changing legacy JSON/SQLite without S3. Regress missing-bucket failures using temporary SQLite and no-auth owners through the production cleanup helper. Verify pool closure, owned-context cancellation and resource shutdown. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(workspace): resolve filesystem CI diagnostics Mark best-effort cleanup errors explicitly and inspect the config type without copying its atomic fields. Document why the validated blob paths do not permit file inclusion. Keep publication, synchronization and PostgreSQL coordination unchanged. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler-agentandEttore Di Giacinto
|
b39d37f559 |
feat(agents): limit saved agents to one folder (#13009)
* Store the folder and its mode on a saved agent
A saved agent gets FolderID, the workspace node of its folder, and
FolderMode, read or write. The folder is kept by node so that a rename
or a move is followed. A mode without a node is an agent whose folder
is not chosen yet; Validate accepts it, and refuses a node without a
mode.
The GORM store adds two columns with defaults, so AutoMigrate fills
the rows of existing agents as agents without a folder. The JSON store
reads a file without the keys the same way. Both stores pass the new
contract case.
Assisted-by: Claude:claude-opus-5-5
* Keep on a chat whether its saved agent had a folder
A chat whose agent is deleted runs today without the agent's grants.
When that agent had a folder, nothing is left to say which folder the
chat may read, so the chat must be refused instead of getting the whole
home. ChatRecord.ScopedAgent records that the agent had a folder.
CreateChat writes the flag and SetScopedAgent changes it. UpdateChat
leaves it alone, like the tool switches, so the loop's stale copy of
the record cannot clear it. Both chat stores read chats stored before
the flag as not scoped; the GORM column has a default, so AutoMigrate
is the migration.
Assisted-by: Claude:claude-opus-5-5
* Add the design and plan for agent folder scope
A saved agent works on the whole home folder of its user. The spec
lets the user give an agent one folder and a mode, read only or read
and write, and confines every chat that runs it to that folder.
The workspace store enforces the folder through a fence on the actor,
so a caller that forgets it gets less access, not more. The spec maps
every caller that builds a user actor today, including the host bash,
the sandbox runner and the add-on workspace operations, and how each
is moved to the chat's access.
The plan splits the work into ten lanes; the store fence and the agent
record come first.
Assisted-by: Claude:claude-opus-5-5
* Resolve the folder of a saved agent by node
A person chooses an agent's folder by its path in their home; the
agent keeps the node, so a rename or a move is followed. The new
FolderResolver turns the path into the node and back, always as the
user. A path is refused when it is not a live folder, when it is
LocalAI/ or Chats/ or inside them, or when it names a parent. The
home itself ("/") is allowed.
The workspace stores cannot find a node by its ID yet. PathOf reads
the newest change sets of the home first, checks the path it finds
with Stat, and else lists the folders the user can read. A store that
implements NodeByID is used instead. Nothing is cached, so every
frontend answers the same.
SaveAgent keeps the stored folder when it replaces an agent, so a
caller that knows nothing of folders cannot widen one.
SaveAgentWithFolder sets or clears it. ImportAgent looks the folder of
a file up in the importer's home, and saves a folder that is not there
as unset: the mode without a node, which does not start.
Assisted-by: Claude:claude-opus-5-5
* Add a fence to the workspace actor
A chat whose saved agent has a folder must reach only that folder and
the chat's own folders. The store needs to know which folders those
are on every call, so the actor carries them.
Actor.Fenced returns a copy that reaches only the given folders, by
node ID, each at most at its mode. The fence is an unexported field:
a copy of the actor keeps it, and only Fenced can set it. Fencing a
fenced actor adds a layer, and a node is reached only when every
layer allows it, so a fence never grows.
The helpers in export.go let both store implementations apply the
same rules: the mode a fence allows on a node, its roots, and the
limit of MaxFenceRoots roots per call.
Assisted-by: Claude:claude-opus-5-5
* Show and choose the folder of an agent in the API
The agent routes carry folder. A write takes {path, mode}: the server
turns the path into the folder's node, and refuses with 400 a path
that is not a folder of the user's home, or is LocalAI or Chats. A
write without folder keeps the agent's folder, so a client that does
not know the field cannot widen the agent; {path: ""} clears it. A
read answers the folder's path now, its name, the mode and a state:
ok, missing or unset. The node never leaves the server.
An agent with a folder exports as version 2, with the folder's path
and mode. A server that reads only version 1 refuses that file, so a
scoped agent is never imported elsewhere as one with all the files.
An agent without a folder still exports as version 1. The import
reads both versions and looks the folder up in the importer's home;
a folder that is not there saves the agent as unset, with a warning.
The Swagger spec, the API instructions and the agents docs page show
the field and the new file version.
Assisted-by: Claude:claude-opus-5-5
* Enforce the actor fence in both workspace stores
The store is the one place every file call of a chat goes through, so
it applies the fence: a caller that forgets the agent's folder gets
"not found", never the whole workspace.
- The mode of a fenced actor on a node is the lower of its grants and
the fence. A root in the Trash counts for nothing, and is back after
Undo, because the fence holds the node ID.
- A fenced actor never takes the administrator shortcut, so it cannot
grant, revoke or list grants, and ReadablePrefixes gives only fence
roots or folders inside them.
- A root cannot be deleted, moved or renamed by the fenced actor, by
any path: Move, Delete, Undo, an upper layer, or a delete of a folder
above a nested root.
- A fence with too many roots or a bad root is ErrInvalid.
The shared suite in wstest runs the new specs against the JSON store
and the GORM store (SQLite and PostgreSQL).
Assisted-by: Claude:claude-opus-5-5
* Map the model's paths through the chat's place
A chat's file tools spelled every path as a path in the user's home.
A chat whose agent has a folder sees that folder at /workspace/home and
its own chat folder at /workspace/chat, so the mapping must come from
the mounts of the chat's view, not from the principal.
session.Place maps a path the model writes to a workspace path: "/x",
"x", "home/x" and "/workspace/home/x" are x in the primary mount,
"/workspace/<name>/x" is x in the mount of that exact name, and the
skill layers keep their names. It refuses "..", NUL, control characters
and invalid UTF-8 wherever they point, and a /workspace path that is no
mount. Encoded and look-alike separators stay part of a name. Place is
only a mapping: the store's fence enforces the access.
session.Access holds a turn's actor, view and place. It has no exported
fields and one constructor, so a tool cannot be built from a principal
string. Owner gives the user's own actor without the fence.
Assisted-by: Claude:claude-opus-5-5
* Build a chat's view from its folders and describe it
BuildView took the home of the principal as the folder at
/workspace/home. A chat whose agent has a folder needs that folder
there, at the agent's mode, and its own chat folder beside it.
BuildView now takes the primary folder and the chat folder. The chat
folder is mounted at /workspace/chat, read and write; the name chat is
reserved like the skill layer names. The store caps each mount at the
actor's mode, and a fenced actor gets the shared folders read only.
Every caller passes the home and no chat folder, so an unscoped chat
keeps today's view.
LayoutPrompt takes a ScopedPrompt. A scoped chat hears only of its
folders: the agent's folder by its name and mode, with a line that it
cannot see the user's other files, the chat folder and where its
working files go, and the shared folders. It gets no hint to build a
page, because pages live outside its folder. An unscoped chat keeps
today's text.
Assisted-by: Claude:claude-opus-5-5
* Find a workspace node by its ID
A caller that keeps a node ID, such as the folder of a saved agent,
had no way to ask the store where that node is now. It had to search
the journal and then list folders, which costs one query per folder
in a large home.
Store.NodeByID returns a live node at its current path, so a rename
or a move of the node or of a folder above it is followed. It answers
ErrNotFound when the node or a folder above it is deleted, and when
the actor cannot read it; a fenced actor finds only nodes inside its
fence. Both stores implement it and the shared suite checks it. The
remote store of the Sandbox add-on refuses it, like its other calls.
Assisted-by: Claude:claude-opus-5-5
* Name the work folder of a scoped chat at /workspace/chat
A chat whose agent has a folder keeps its chat folder in the home, but
sees it as a mount of its own at /workspace/chat. Its work folder is
then /workspace/chat/work, not a path under /workspace/home.
workfiles reads /workspace/chat as the chat folder: a helper file under
its work folder is working, a script beside the attachments is a stray
helper, and a folder named chat in the agent's folder is an ordinary
folder. ScopedPrompt steers the model to that work folder and sends the
deliverables to the agent's folder, or to the chat folder when the
agent may only read. It says nothing of pages or of the settings
folder, which are outside the agent's folder. The chat picks the text
by its view.
Assisted-by: Claude:claude-opus-5-5
* Look up the folder of an agent by node
The folder resolver searched the journal of the home and then listed
folders to find where an agent's folder is now. The stores can now
find a node by its ID, so the resolver asks them directly and the
search is gone.
Assisted-by: Claude:claude-opus-5-5
* Resolve the file and media tools through the chat's access
The file tools mapped paths to the user's home and acted as the chat's
actor, and the media tools did the same through files.HomePath. A chat
whose agent has a folder must reach only its folders, and search must
start in the agent's folder, so the count of documents still being
indexed cannot count files elsewhere in the home.
files.Service.Tools and media.Service.Tools take a session.Access. Every
path goes through its place and every call acts as its actor. A write
to a read-only folder is refused before the store sees it. A search
with no path starts at the primary folder, and in a scoped chat also
looks in the chat folder; the pending count is taken below that root.
files.HomePath is removed.
UnscopedAccess builds today's access of a chat (the home and the
shared folders), and the studio service takes the turn's access from
one function, turnAccess, which gives that access until the chat
access resolver replaces it.
The new specs run the paths that try to leave the folder (parents,
absolute and doubled paths, encoded and look-alike separators, NUL,
other cases) through every file tool and the media tools, and check
that nothing outside the folder is read, named or changed, and that
search counts no pending file outside it.
Assisted-by: Claude:claude-opus-5-5
* Resolve the access of a chat at every turn
A chat whose saved agent has a folder must reach only that folder. Up
to now every chat got the user's actor, which reaches the whole home.
The chat's access is now resolved from the stores at every turn
start. With no agent folder it is the actor and the view of today.
With a folder the actor is fenced to the agent's folder at its mode,
the chat's own folder read and write, and the shared folders and the
skill layers read only; the chat's folder is made before the fence,
because a fenced actor cannot make it. A sub-agent takes the folder
of its parent's agent, and its own agent's folder inside it.
A folder that is deleted, unreadable or not chosen refuses the chat
with ErrAgentFolderMissing; it never falls back to the whole home.
A chat records whether its agent had a folder, so a chat whose agent
was deleted is refused too. CreateAs refuses such an agent at once.
session.Access carries the actor and the view, has no exported fields
and one constructor. Open takes it, the turn tools are built from it,
and an open chat compares the fence as well, so an edit of the agent
restarts the harness at the next turn on any frontend. The person's
uploads and renames use the owner's actor, without the fence. The
files a command changed are read as the chat can read them.
Nothing is kept between turns, so frontends cannot disagree.
Assisted-by: Claude:claude-opus-5-5
* List a scoped chat's files by its place in the prompt
The turn prompt lists the chat's files with the paths the model uses.
For a chat whose agent has a folder those paths come from its mounts,
and a file the chat touched outside its folders, for example before
the agent got a folder, must not be named.
Manager.FilesIn is Files with a place. For a scoped place, each file
is named by its mount (/workspace/home for the agent's folder,
/workspace/chat for the chat folder) and a file outside every mount is
left out. Other places give what Files gives. The prompt's file list
uses the turn's place; the chats API keeps the user-wide list.
Assisted-by: Claude:claude-opus-5-5
* Answer 409 when an agent's folder is missing
A chat with a saved agent whose folder is missing or not chosen does
not start. The chat endpoints now say so with 409 and the type
agent_folder_missing, both when the chat is created and when a
message is sent to an open chat, instead of a server error.
The agent docs say what a chat with a folder reaches, when a change
of the folder applies, and that nothing is kept between turns.
Assisted-by: Claude:claude-opus-5-5
* Point a scoped chat at rules and work folders in its own folders
The rules nudge walked from the chat folder up to the home root and
named AGENTS.md files there, and the stray-helper note named the work
folder under /workspace/home. A chat whose agent has a folder must hear
only of its own folders.
The nudge takes the turn's access: it reads paths through the place,
starts at the primary folder (the agent's folder) and stops there, so
it never points above the folder or into the chat folder mount. The
helper note names /workspace/chat/work/ in a scoped chat, and a
sub-agent of a scoped parent gets the scoped work folder text.
Assisted-by: Claude:claude-opus-5-5
* Find workspace actors built outside the chat access
An actor built from a principal has no fence and reaches the whole
home. A spec now parses the studio service, the command runners, the
add-on host operations and the application wiring, and fails on a
workspace.Actor literal, a files.UserActor call or an Actor(principal)
call in a function that is not on the allowlist. Each allowed entry
says why: a host read of the user's settings, a user surface, or a
caller that a later change moves to the chat's access. An entry that
builds no actor any more fails the spec too, so the list stays exact.
Assisted-by: Claude:claude-opus-5-5
* Build the expected view of a chat with the view's folders
The spec of a chat without an agent folder compared its view with
BuildView called with the old arguments, so the test package did not
compile after the merge of the view builder. It now passes the home
as the primary folder and no chat folder, which is the view of an
unscoped chat.
Assisted-by: Claude:claude-opus-5-5
* Give every file tool of a turn the chat's resolved access
The resolver of a chat's access built the view of a scoped chat
itself, with no shared folders, while every other chat got its view
from the session manager. And the file tools, the media tools, the
chat's file list in the prompt, the rules nudge and the work folder
lines each asked for an access of their own, the view of the whole
home. A chat whose agent has a folder therefore had a fenced actor
but home-relative paths: it reached its folder only through paths
such as /Projects/Acme/a.txt, and its prompt named the home.
The session manager now builds every chat's view: UserAccess mounts
the home, and the new ScopedAccess mounts the agent's folder at home,
capped at the fence's mode, the chat folder at chat and the shared
folders read only. It refuses an actor without a fence, and a folder
outside the fence is an error, never a view of the home. The
resolver calls it with the innermost folder, so a sub-agent of a
saved agent still works in its own folder.
The access that the resolver builds once at the start of the turn
now goes down to every consumer. A sub-agent of a saved agent acts
for its parent chat with the same access (Access.ForChat), and a
generic sub-agent gets its tools from the parent's access. The seam
that gave the unscoped access, and the files service's own view
builder, are removed, so a new consumer cannot build a view by
mistake. A turn whose access has no folder gets no tools rather than
the home.
A scoped chat now sees its folder at /workspace/home and its chat
folder at /workspace/chat in its tools, its search, its prompt and
its work folder lines. A chat whose agent has no folder is unchanged.
Assisted-by: Claude:claude-opus-5-5
* Find chat views built outside the chat access
The turn's access is resolved once and handed down, but nothing
stopped new code from building a view of its own, which shows a
scoped chat the whole home. A second scan now parses the same
sources as the actor scan and fails on a call of BuildView,
SkillLayerMounts, NewAccess, UserAccess or ScopedAccess, or on a
store's View, in a function that is not on its allowlist. The
resolver is the one allowed caller; the host bash, the sandbox
runner and the add-on host operations are listed until they take
the chat's access. A stale entry fails the scan too.
Assisted-by: Claude:claude-opus-5-5
* Keep a sub-agent inside its parent's folder
A saved agent started from a scoped chat could name a folder
outside the chat's own, and the child was made before its first
turn failed with a message about a missing folder. spawn_agent now
checks the agent against the chat's agent from the stores before it
makes the child: no folder is fine, the same folder or one inside
it is fine, anything else fails with "this agent works in another
folder than this chat". The child never has more reach than its
parent, in folder or in mode.
The ScopedAgent flag of a sub-agent was reset to what its own
agent had at each turn. A child of a scoped chat with an agent
without a folder lost the mark, so deleting the parent's agent
would have given that child the whole home. The flag now follows
both folders.
New specs cover a generic child, a saved-agent child outside,
inside and at the parent's folder with the lower mode, a child
without a folder, and the work folder wording of a child of a
read-only or read-write parent.
Assisted-by: Claude:claude-sonnet-5-5
* Show agent folders in list_agents and limit them by scope
list_agents gave a scoped chat every saved agent, including ones
that spawn_agent would refuse, and told the model nothing about
where an agent works. Each entry now carries the name of the
agent's folder and its mode, or folder_missing when the folder is
deleted or not chosen. The path is left out, since it would show
the folders above the chat's own. A chat whose agent has a folder
lists only the agents that fit under that folder, by the same check
spawn_agent makes.
The docs for saved agents as sub-agents and for the agent folder
describe both.
Assisted-by: Claude:claude-sonnet-5-5
* Keep the triggers of a scoped agent to its folder
A chat of an agent with a folder reaches only that folder, but its
triggers still worked on the whole home. A file trigger could watch
any folder, the rules of a run were read from the home root down
with the user's actor, the first message named home paths, and a run
whose agent's folder was gone fell through to a failed chat start
after the host had already read the home's rules. The trigger tools
of a scoped chat could also list, change or propose triggers that
reach the whole home.
The agent's folder now bounds every step:
- Saving a file trigger (create, update, edit, confirm) refuses a
folder outside the agent's folder, and a trigger of an agent whose
folder is missing or not chosen.
- Each run checks the agent's folder before it reads anything. A
missing or unset folder skips the run with "the agent's folder is
missing" and switches the trigger off; a watched folder that is not
inside the agent's folder does the same with its own reason. No
chat is made and the owner is told.
- The folder check and the rules are read with an actor fenced to
the agent's folder, read only, so no AGENTS.md above it is read.
The rules of a schedule run start at the agent's folder. The first
message names folders and the file as the chat sees them, under
/workspace/home, and the rules nudge of the run's chat counts those
rules as given.
- In a scoped chat the trigger tools read folders as paths of the
chat's place, refuse one outside the agent's folder, and see only
the triggers of that agent.
The trigger watcher takes the service's administrator actor, made
once next to the change feed's, and the trigger functions leave the
actor scan's allowlist.
Assisted-by: Claude:claude-opus-5-5
* Send connector replies through the chat's access
The files a connector reply sends were listed from every change made
in the chat's name and read with the user's actor, so a reply of an
agent limited to one folder could send a file from anywhere in the
home that some host-side write had attributed to the chat. A message
to an agent whose folder was missing made or reopened a chat, failed
to start it, and answered with a generic error.
A reply now resolves the chat's access once, lists the turn's files
through its place (a file outside the chat's folders is left out)
and reads each one with its fenced actor. A file the access cannot
reach is skipped without being named. Before it makes a chat, a
message checks the bound agent's folder; when it is missing or not
chosen, the sender gets one reply, "This agent cannot work right
now: its folder is missing.", the refusal is logged and the binding
stays on. The same reply covers a folder that goes away between the
check and the turn. readChatFile leaves the actor scan's allowlist.
Assisted-by: Claude:claude-opus-5-5
* Name a read-only refusal of a commit by its guest path
The store names an upper entry that a read-only mount refused by
its path in the view (/home/x), not by its store path. The note
that a command's output ends with took every path but an
outside-view one for a store path, so it told the model that
"/home/x was not saved", a path the guest does not have. A path
that no mount holds but that starts with a mount name is now taken
as a path of the view and gets the /workspace prefix.
Assisted-by: Claude:claude-opus-5-5
* Take the actor and view of command runners from the chat's access
The host bash and the sandbox runner made their own actor and view:
the host bash a user actor from the principal and the view of the
whole home, the sandbox runner a plain actor by default and the view
from its own mount requests. Neither knew the chat's agent folder,
so a command of a scoped chat could reach the whole home.
The runner now has no default actor or view. Options.Actor takes a
context and returns an error, and Options.View returns the view
itself; both are asked at every call. A view of another actor than
the chat's is refused, and a sandbox whose actor or mounts changed
is made again, so an agent's new folder or mode applies from the
next command. A failing resolver fails the call, also with a sandbox
running.
The server wires both hooks to the chat's resolved access: the new
Service.CommandActor and Service.CommandView, the latter refusing an
actor that is no longer the chat's. Access.SandboxView adds the skill
layers to the turn's view, at the names the file tools use. The host
bash refuses a fenced actor, at the tool and again at each call, as a
command on this machine reads what the server can read. The Sandbox
add-on passes a label actor and its home view: the host binds each
workspace operation to the call's chat.
The lane 5 entries leave the actor and view scan allowlists.
Assisted-by: Claude:claude-opus-5-5
* Offer no host bash to a chat whose agent works in one folder
A command of the built-in bash runs on this machine and reads every
file LocalAI can read. No workspace rule confines it, so a chat whose
agent works in one folder cannot have it, whatever chats_host_exec
says. The tool's build now refuses a fenced actor, so the chat gets
no bash, its prompt says nothing runs commands, and the shell group
of the Tools menu gives the reason: the agent works in one folder
and runs commands only in the Sandbox add-on. Sub-agents of such a
chat have the fenced actor too.
The docs of saved agents and of the commands on this machine state
the rule and the Sandbox layout of a scoped chat.
Assisted-by: Claude:claude-opus-5-5
* Tell a scoped chat where its Sandbox commands start and what they keep
The commands block of the prompt told every chat with the Sandbox
that a command starts in /workspace/home, the user's home, and that
what it saves there or in a writable shared folder is kept. In a
chat whose agent works in one folder, /workspace/home is that folder,
the chat folder is /workspace/chat and the shared folders are read
only. The block now says so, and with a read-only folder it names
only /workspace/chat as kept.
Assisted-by: Claude:claude-opus-5-5
* Name a scoped chat's attachments where its commands see them
The model learns of an attached file by its guest path, which was
always /workspace/home/Chats/<chat>/<file>. In a chat whose agent
works in one folder, /workspace/home is that folder: the file and
media tools still found the file through the place's chat alias, but
a command sees the chat folder only at /workspace/chat, so the path
named nothing for it.
The text the model gets now names a file of the chat folder at
/workspace/chat/<file> in a scoped chat, through the chat's place:
when a message is sent, queued or injected, in the summary of the
chat, and in the model state rebuilt after an edit, a regenerate, a
fork or an import. What is stored keeps the home spelling, which
SplitMessage reads back, and an unscoped chat is unchanged.
Assisted-by: Claude:claude-opus-5-5
* Remove the duplicated allowlist entries of the actor scan
Assisted-by: Claude:claude-opus-5-5
* Read and write the folder of an agent in the agent helpers
The agent API now answers an agent's folder with its path, mode and
state, and a write keeps the stored folder unless it names one. The
page helpers learn that shape: the folder field of a save is sent only
when the person changed it, an empty path clears it, a missing or
unset folder that the person did not replace is kept, and the mode is
read unless write was chosen. Small helpers say whether a listing
entry can be chosen (not a folder LocalAI manages, not a shared
folder), which folders agents use for the Files page marker, and
whether a chat error is the missing-folder conflict.
Assisted-by: Claude:claude-opus-5-5
* Move the folder walk of the Move dialog into a folder picker
The agent editor needs the same walk through the person's folders as
the Move dialog, so the breadcrumb and the folder list move into
FolderPicker, with their styles, and the Move dialog draws it. The
picker can show a folder as one that cannot be chosen, with the
reason next to it, and a folder that cannot be read now offers to try
again; a folder still loading says so instead of an empty list.
Assisted-by: Claude:claude-opus-5-5
* Choose the folder of an agent in the agent form
The agent form gets a Files section after the instructions: All my
files, or One folder chosen in the folder picker, and whether the
agent's chats can read the folder or read and change it. The picker
shows the folders LocalAI manages as ones that cannot be chosen, and
Home itself can be chosen. A folder that was deleted, or an imported
agent whose folder was not found, is said in place with an offer to
choose another; the rest of the form still saves and keeps it.
A save sends the folder only when the person changed it, so editing
another field never touches it. Picking One folder and choosing
nothing stops the save, as it would otherwise save an agent on all
the files. The Export section now downloads the file the server
writes, which names the folder, and says so.
Assisted-by: Claude:claude-opus-5-5
* Show the folder of an agent on the agents list and its page
An agent with a folder shows it in the list's line and on its page,
with its access in words and a lock when its chats can only read it.
A folder that is gone, or an agent with no folder chosen yet, shows
a warning that opens the form; the agent's page also says so above
its facts and offers to choose another folder.
Assisted-by: Claude:claude-opus-5-5
* Mark the folders that agents work in on the Files page
A folder that a saved agent works in gets a small marker with the
agent's name, or the number of agents, and a tooltip that names each
one with its access. Inside such a folder, a line over the list says
the same. The page reads the agent list it can already fetch, so the
files API is unchanged, and the page still shows all the files.
Assisted-by: Claude:claude-opus-5-5
* Show a chat's folder and say when its agent's folder is missing
A chat with an agent that has a folder shows the folder next to the
agent in the message box, with a lock for read only, and the Tools
menu says which files the chat's tools reach. When the folder is
gone, the chip says so and opens the agent's form.
A chat whose agent's folder is missing does not start, and the server
answers with a conflict. The message box now says that in plain words,
keeps the message, and offers Edit agent, instead of the server's
text.
Assisted-by: Claude:claude-opus-5-5
* Test the folder of an agent across the web interface
The agent stubs now list folders by path, read a written folder back
as the server does, export a version 2 file for an agent with a
folder, import one whose folder is not found as unset, and refuse to
start a chat whose agent's folder is missing.
The specs choose a folder and see it on the list, switch the access,
leave an unchanged folder out of the save, clear it, keep and replace
a missing folder, keep Chats and LocalAI out of the picker, retry a
folder that cannot be read, see the Files marker, the chat's chip and
Tools line, the missing-folder chat error with Edit agent, and the
import warning. A render smoke opens the picker on the real server,
and one spec checks that the copy names no internal path.
Assisted-by: Claude:claude-opus-5-5
* Describe the agent's folder in the web interface
The agent docs said what the folder field does in the API. They now
also say where the web interface sets it, how the agents pages, the
Files page and a chat show it, and what a chat whose folder is
missing offers.
Assisted-by: Claude:claude-opus-5-5
* Record the chat an add-on call was made for
An add-on's workspace operations run as the user of the live request,
with no idea which chat made it. Only an action carries a chat, and
only as a key in its resource ID; a read tool such as the Office
add-on's read-docx carries none. So a chat that should reach only one
folder could reach the whole home through an add-on.
The in-flight record of a request now keeps, next to the envelope, a
binding to the chat the request was made for: the user's principal
and the chat ID. It is host-side only and never a field of the
envelope, which installed add-ons decode with unknown fields refused.
A context carries the binding from the chat turn to the place that
records the request. The broker gives each host operation the chat
and the scope of its live request. An operation ID that requests of
two chats share is not live, so neither chat is trusted.
Assisted-by: Claude:claude-opus-5-5
* Keep the chat binding of a runtime call on the host
The runtime client records each call's envelope for the host operation
channel. It now records the chat binding too when the call's context
carries one, so the add-on's host operations know which chat made the
request. The envelope sent to the add-on is the same bytes with or
without a binding: a spec checks that the body of a bound call equals
the body of an unbound one and still decodes with unknown fields
refused.
Assisted-by: Claude:claude-opus-5-5
* Carry the chat binding in a forwarded add-on request
A frontend that is not the add-on owner forwards an add-on call in a
signed request. The chat the call was made for now rides in that
request as a signed field, chat, and the owner puts it back on the
context of the call it serves, where the runtime client records it.
A changed, added or removed binding fails the signature check, and a
signed but incomplete one is refused. A request that no chat made has
no chat field and signs as before.
An owner that does not know the field refuses a forwarded call that a
chat made, so the owner must be upgraded before the frontends.
Assisted-by: Claude:claude-opus-5-5
* Bind every add-on tool call of a chat to the chat
The chats consumer ran an add-on tool with the conversation ID only,
which the owner used to resolve a selection and, for an action, as the
key of the add-on's state. Nothing told the host which chat a read
tool or a change came from.
The tool adapter now has ExecuteChat, which takes a ChatCall (the
chat's user and ID, from the chat turn's own record) and binds the
call to the chat on the host side for every tool kind: a read with no
resource, an action, and a change made at once. The chats adapter
refuses a call that names a conversation without that binding, so a
chat's call cannot be made unbound by mistake. Other consumers, such
as the OpenAI-compatible route, keep calling without a binding.
Assisted-by: Claude:claude-opus-5-5
* Run add-on workspace operations with the chat's access
The workspace operations of add-ons (materialize, commit, changes,
export and import) worked on the whole home of the request's user. A
chat whose agent has a folder could therefore read and write any file
through the Office or Sandbox add-on.
They now resolve their access from the chat binding of the live
request, through a new Access hook that the chats service answers
(Service.AddonAccess). A chat's call gets the chat's access, as its
file tools do: in a chat whose agent has a folder, the add-on's "/" is
that folder, /workspace/chat names the chat folder, the agent's mode
applies, and paths resolve through the chat's place, so a parent
reference is refused and an encoded separator is part of a name. The
mounts go through Access.SubView, which builds a view only inside the
access, for its fenced actor. When an add-on mounts its "/" at home,
the host adds the chat folder at chat, so the Sandbox sees the chat's
layout without a change. Changes list only what the access may read,
and a commit entry outside the view is a conflict.
A call whose chat cannot be resolved is refused and never runs
user-wide. An action request without a binding is resolved by its
chat key, as before, and refused when it names no chat. A call that
no chat made (an add-on page, the OpenAI-compatible route) still works
on the whole home, through Manager.PersonAccess.
The four lane 6 entries leave the actor scan and the view scan.
PersonAccess joins the view scan's matcher, with AddonAccess as its
one allowed caller. An end-to-end spec runs the real Office and
Sandbox add-ons in a scoped chat.
Assisted-by: Claude:claude-opus-5-5
* Build the chat access of the file index route test
files.Service.Tools takes a session.Access since the chat access
change, so the route test of the text index passed a bare actor and
its package did not compile under the auth tag.
The test now builds the access as the chats service does for a chat
whose agent has no folder: the user's home as the primary folder of
the view, then session.NewAccess. The assertions are unchanged.
Assisted-by: Claude:claude-opus-5-5
* Test the folder of an agent through every surface of a chat
The specs of the folder feature each test one surface. This suite
runs one chat of an agent limited to a folder through all of them at
once, on an indexed SQL workspace: the file and media tools, the search
and its pending count, a command of the Sandbox in a fake sandbox, a
read call and an action call of an add-on through the real workspace
operations bound to the chat, a generic and a saved sub-agent, the
trigger tools, a file trigger run, a connector reply, a renamed folder,
a mode change, an edit that narrows the folder (also seen by a second
frontend) and a deleted folder.
The files outside the folder hold a token that the scripted model never
writes. After each spec the suite collects every text a person or a
model can see: tool results and errors, prompts, messages, transcripts
with their cards, live events, chat file lists, tool panels,
notifications, connector replies and files, the add-on's data directory
and the connector log. The token must be in none of them, and the files
outside must be unchanged.
A control spec runs the same probes from an agent without a folder and
checks that they do find the token, so the probes and the collection
cannot pass by seeing nothing.
Assisted-by: Claude:claude-opus-5-5
* Complete the docs of the folder of an agent
Each change of the folder feature documented its own part. This
makes the pages one account of it:
- Saved agents: the missing and unset states in one bullet, what the
folder does not limit (MCP servers, web tools, collections, pages),
that the Files page stays user-wide, when an edit applies, the
downgrade caveat, and the folder as a stateless row of the cluster
table. The Sandbox of a scoped chat mounts the agent's folder and the
chat folder only, as the add-on asks for them; the page said it also
had the shared folders and skills.
- Chats: the chat folder at /workspace/chat for the model, sub-agents
of a scoped chat, and commands only in the Sandbox.
- Add-ons: the Sandbox and Office sections say what / is in a chat whose
agent has a folder.
- First agent: the optional folder step and the "Folder missing" case.
Assisted-by: Claude:claude-opus-5-5
* Draft the PR description and release note of agent folders
No release notes file exists for the next release yet, so the draft
text lives next to the spec and the plan. It carries the release note
paragraph with the downgrade caveat and the upgrade order of a cluster.
Assisted-by: Claude:claude-opus-5-5
* List the chat binding specs in the per-report add-on inventories
The chat binding change added five specs to the gateway and runtime
packages and named them in the case lists of the gateway, artifacts and
apps suites. These three suites also keep one case list per Ginkgo
report, and the verifier compares each report with its own list. The
new names were missing there, so the three suites failed with an
inventory mismatch.
A dry run of every Ginkgo suite's commands now lists the same It names
as each report's inventory, in both directions.
Assisted-by: Claude:claude-opus-5-5
* Hide stale paths from a fenced actor's history
A folder moved into a fence makes its parent identities readable.
Their historical paths can still name locations outside the fence.
Hide each historical side when its readable parent's current path
differs. Reuse the generic title when filtering hides a side. Keep
the journal and unfenced history unchanged so Undo uses identities.
Assisted-by: nib:gpt-6.1-sol
* List the chat binding cases in Tables tools
The Tables tools runner includes every adapter spec in its output.
List the four chat binding cases so the exact inventory accepts them.
Assisted-by: nib:gpt-6.1-sol
* Keep shared agent folders unset in exports
A followed folder outside the home cannot round-trip through a home path.
Export its mode with an empty path so import warns and stays fail-closed.
Responses keep public shared paths without exposing other internal paths.
Distinguish deleted scoped agents from unscoped agents in the guidance.
Assisted-by: nib:gpt-6.1-sol
* Refuse scoped chats over the folder limit
Keep every required shared root in each intersecting fence. Refuse
creation and turn access when the complete set exceeds 32 roots.
Reserve a chat root before its folder exists and fence listings empty
on refusal. Propagate shared enumeration failures instead of omitting
roots.
Map capacity refusals to a fixed 409 response and give connectors a
safe capacity message. Check boundaries, retries, background access,
and HTTP handlers with focused regressions.
Assisted-by: nib:gpt-6.1-sol
* Verify unset imports refuse chat startup
Assert the folder-specific refusal without a harness start or saved chat.
Clarify public response paths and unset exports in the API metadata.
Assisted-by: nib:gpt-6.1-sol
* Keep the folder retry test under user control
A background refresh consumes the one-shot successful response before
this test observes its load error. Isolate the mocked Files page from
the live workspace stream and agents list. Keep reads failing until
the error assertions pass, then allow the actual Retry click to recover.
Check that Retry reads again and removes the error, without changing
the existing assertions or their timeouts.
Assisted-by: nib:gpt-6.1-sol
* Test and document folder capacity refusals
Use the existing Ginkgo suites for capacity regressions. Exercise a
successful turn, refusal after shares change, and a retry through a
second service on the same stores. Check that refusal leaves no claim
or pending message.
Verify that connectors send one fixed refusal and that triggers stay
enabled after capacity failures. Document the 32-root budget and the
all-required-roots-or-refuse contract.
Assisted-by: nib:gpt-6.1-sol
* Control crashes in the restart budget test
The crash fixture exits after 100 ms, before startup necessarily finishes.
A busy runner can fail the readiness handshake instead of testing restarts.
Start a normal fixture and kill each published incarnation explicitly.
Keep the exact restart count and persistent failed-state assertions.
Assisted-by: nib:gpt-6.1-sol
* Check capacity test cleanup errors
Assert service and workspace cleanup through Ginkgo. Keep service cleanup before workspace cleanup and report close failures instead of discarding them.
Assisted-by: nib:gpt-6.1-sol
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
|
||
|
mudler-agentandEttore Di Giacinto
|
9bd5d0f915 |
fix(ci): publish images after large mixed diffs (#13011)
Quiet grep exits on the first image-visible path. Under pipefail, echo can then fail with SIGPIPE and the gate incorrectly skips images. Consume the full path list without changing the exclusion rules. Test the actual workflow gate with disposable Git history, including large mixed diffs, and run the regression in the lightweight lint job. Assisted-by: nib:gpt-6.1-sol Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-botandmudler
|
968c6651ae |
chore: ⬆️ Update leejet/stable-diffusion.cpp to c6dfa9def12fd0cb36ac7149ba30a75792bedbf0 (#13002)
⬆️ Update leejet/stable-diffusion.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
f49d9993b0 |
chore: ⬆️ Update ggml-org/llama.cpp to ddf48f85c933c1f438212a02eb21e7b001d2d37d (#13000)
⬆️ Update ggml-org/llama.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
32d7774e47 |
chore: ⬆️ Update mudler/vllm.cpp to fb5bfcd70ddd2717cb22908527cd503f8b007143 (#13001)
⬆️ Update mudler/vllm.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
7ded63023a |
chore: ⬆️ Update CrispStrobe/CrispASR to 1dd0bf6fb9e7a8bf5849cb7a14605b2234e6ee93 (#13003)
⬆️ Update CrispStrobe/CrispASR Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
4383a8955c |
chore: ⬆️ Update 0xShug0/audio.cpp to ad14f0c4dbd5b12b7711bb7b46abb8d94f78f067 (#13004)
⬆️ Update 0xShug0/audio.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
1775c67848 |
chore(model-gallery): ⬆️ update checksum (#13005)
⬆️ Checksum updates in the model library Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
mudler-agentandEttore Di Giacinto
|
4899d28e8a |
feat(chats): server-side chats with steering, sub-agents and compaction (#12918)
* Find the Sandbox guest image in the executable, not its build info
The Sandbox smoke checked that the packaged executable carries the
guest image by looking for -X main.guestImage in go version -m. The
package is built with -trimpath, and with -trimpath Go leaves -ldflags
out of the build info, so the check failed on every gvisor run,
although the image was linked in.
Look for the reference in the executable instead: the value of
SANDBOX_GUEST_IMAGE when it is set, any digest reference otherwise.
package.sh had the same check for an executable built elsewhere; it
now looks for its own reference the same way.
Assisted-by: Claude:claude-opus-5-5
* Answer a network card the limit refuses with one precise reason
A card that waited while the administrator's limit went down was
refused on the first click with 400, and every later click was a bare
409, so a stale card looked like a conflict loop. A refused card now
answers 400 egress_above_ceiling every time, the same answer sent twice
is one answer (204), and another answer to an answered card is 409
already_answered. The chat view marks a card the limit cannot admit
with above_ceiling.
When the limit is off, or below the chat's preset, the note of the turn
and the block line of the proxy now say that the server's setting
blocks the network and no approval changes it, so the model reports it
instead of waiting or retrying.
Assisted-by: Claude:claude-sonnet-5-5
* Explain a blocked Sandbox network instead of offering dead buttons
A card the limit cannot admit shows why network access is off or
limited, and who can change it, with a link to Settings > Tools >
Sandbox network for administrators. The buttons of a card disable while
an answer is sent, and a failed answer reads the card again. The
Network row says why its choices are capped. Settings opens a group
from ?group=. Strings are translated in all nine locales and the chat
and add-on docs describe the new behaviour.
Assisted-by: Claude:claude-sonnet-5-5
* Add an always-allowed hosts list to the Sandbox network
The extra registries setting only widens the package registries, so a
one-off host needs the approval card in every chat. The new admin
setting chats_egress_allowed_hosts lists exact hosts (optionally
host:port to pin a port) that a chat reaches whenever its effective
preset is registries or all.
The list sits in the same decision as the presets: the ceiling still
caps it, the ports setting still applies, and a private, loopback,
link-local or metadata address still needs the private switch and an
administrator's rule or answer. Each use is audited with the reason
allowed_host. Hosts are validated like the extra registries (lower
case, punycode, no scheme, path or pattern, at most 64) and an IP
literal is refused unless the private switch is on.
get_chat_egress and the chat egress view return the list; no MCP tool
writes it, as for the extra registries.
Assisted-by: Claude:claude-sonnet-5-5
* Let users delete from the Trash for good
A delete only set a tombstone, so the Trash grew without end and
nothing could free the space of a deleted file.
The workspace stores gain a Purger: it erases the deleted nodes of
some change sets, or of a whole folder, with their journal rows, the
change sets that nothing is left in, and the blobs that no row refers
to. Both stores implement it and pass the same contract specs. A blob
is removed after a grace period, and only if it is still unused then,
so an upload of the same bytes in flight keeps them.
DELETE /api/chats/files/trash empties the user's Trash and
DELETE /api/chats/files/trash/{rev} deletes one entry. Both are scoped
to the caller's home. The Trash listing now gives files, folders and
bytes per entry and for the whole Trash. The file tools of a chat
cannot reach any of this.
Assisted-by: Claude:claude-sonnet-5-5
* Add Empty Trash and Delete permanently to the Files page
The Trash tab shows what it holds, with an Empty Trash button that is
off while the Trash is empty and a Delete permanently button on each
item. Both ask in the page first and say how many files and folders and
how much space they remove. Texts are translated in all nine locales.
Assisted-by: Claude:claude-sonnet-5-5
* Fall back to Brave Search and Bing when DuckDuckGo asks for a captcha
DuckDuckGo answers some servers with HTTP 202 and a bot check (the
"anomaly" page) instead of results. The parser read that page as "no
results it could read", so web_search failed for every query.
The keyless search is now a chain: DuckDuckGo, then the result page of
Brave Search, then the result page of Bing. Each page has its own
tolerant parser (title, URL, snippet; Bing's /ck/a links and DDG's uddg
links are unwrapped) with saved fixtures.
A bot check is a typed error (ErrChallenged), and a 403 or 429 is
ErrBlocked. Both wrap ErrProvider. A Fallback with a cool-down skips a
challenged or blocked provider for a few minutes, so each search does
not wait for it first; when every provider is cooling down, all are
asked again. Fallback.SearchAttempts and TestEach report what each
provider did (ok, no_results, challenged, blocked, timeout, error,
skipped).
The error a model reads now lists what was tried and what each provider
did, then where an administrator fixes it.
safehttp keeps the status code of an answer (Response.StatusCode) and
of a refusal (StatusError, still ErrStatus for errors.Is).
Assisted-by: Claude:claude-opus-5-5
* Use the keyless search chain in the chats and test each provider
The built-in web_search now searches with DuckDuckGo, Brave Search and
Bing in turn when no SearXNG instance is set. A new runtime setting,
chats_web_search_providers, sets the order (searxng, duckduckgo,
brave_html, bing_html) and leaves a provider out. Empty keeps the old
choice: SearXNG alone when an instance is set, so its queries do not
reach the public engines unless the administrator lists them.
POST /api/chats/admin/web/test asks every provider of the order and
returns what each did in "providers" (ok, no_results, challenged,
blocked, timeout or error, with the reason), next to the first answer.
Settings > Tools has a Test search on the web_search row that shows it,
and the provider list is checked on the page and by the server.
The cool-down of a challenged provider is kept per frontend, like the
cache and the limits; web.md says why.
Docs: web.md (providers, chain, bot checks, privacy, how to fix it with
SearXNG or the Web add-on) and runtime-settings.md. Strings in the
nine locales. Swagger regenerated.
Assisted-by: Claude:claude-opus-5-5
* Try the mirror library alone in the reinstall recovery spec
The spec adds a second library with the same URL and expects an
install from it to keep the retained data apart. Since two enabled
libraries that list one add-on are a conflict, the reinstall from the
original library then failed with library_conflict, and the install
from the mirror failed for that reason instead of the trusted identity.
Switch the original library off while the mirror is tried, check that
the mirror is refused for its identity and not for a conflict, then
switch the mirror off and the original on for the reinstall.
Assisted-by: Claude:claude-opus-5-5
* Pause a chat turn at the step limit and offer Continue
A chat turn ran at most 10 rounds of tool calls, the default of the
harness, because nothing set a limit. A multi-step task (read some
files, run commands, write a result) stopped half way, and the person
saw an answer that looked final.
The limit is now the runtime setting chats_max_iterations (1 to 500,
default 40, Settings > Tools); a model's agent.max_iterations still
wins. When the tool loop reaches it, the harness sends a paused event
with the limit, and the turn ends with last_turn "paused". A paused
turn counts as finished: waiting messages start the next turn and a
sub-agent reports back. The status line says "Paused, after 40 steps"
and has a Continue button, which sends a message that asks the model
to go on.
A claim that found no waiting message released the chat as "done",
which hid a pause. It now keeps the status the chat had.
Assisted-by: Claude:claude-opus-5-5
* Edit the always-allowed Sandbox hosts in Settings > Tools
Add a list editor for chats_egress_allowed_hosts next to the extra
package registries: the hosts are tags with a remove button, an input
adds one on Enter, comma or Add (a pasted list adds them all), and the
reason an entry is refused shows under the input, with the count out
of 64. The same checks stop Apply on a bad entry, and the server still
has the last word.
The chat's Network panel lists the hosts read only and the row says
how many an administrator always allows while the chat's network is on.
The help text of both settings now says how they differ. Strings are
translated in all nine locales.
Assisted-by: Claude:claude-sonnet-5-5
* Document the always-allowed Sandbox hosts
Describe chats_egress_allowed_hosts in the runtime settings page, the
network section of the add-ons page (including the allowed_host audit
reason and how the list differs from extra registries) and the chats
page.
Assisted-by: Claude:claude-sonnet-5-5
* Let a chat edit, pause and delete the person's triggers
A chat could propose a trigger and list them, but not change one: the
Triggers page and the API could. list_triggers now gives each trigger a
stable ref (not its id) and shows the prompt of one trigger only when
asked. update_trigger and delete_trigger store a change record in the
trigger table: an edit or a delete waits for Apply or Delete on a card,
a pause or a resume applies at once with a receipt. An applied change
can be undone for a day; a delete restores the trigger with its id.
Change records are rows in the database, so every frontend shows and
answers the same card. The new routes under /api/chats/triggers/changes
serve the card and are gated by the triggers feature.
Assisted-by: Claude:claude-opus-5-5
* Show the trigger changes a chat asks for as cards
An edit lists each changed field before and after with Apply and
Dismiss, a delete shows the trigger and its last run with Delete and
Keep, and an applied change is a receipt with Undo. The cards sit below
the message text like the proposal card, and read their state from the
server so a reload shows the same answer.
Assisted-by: Claude:claude-opus-5-5
* Add a Test search button to the web tools settings
POST /api/chats/admin/web/test existed, but no page called it and the
docs showed only curl. The web_search row of Settings > Tools now has
a Test search button. It runs one search with the applied settings and
shows the provider with its first results, or the error a chat would
read.
Assisted-by: Claude:claude-opus-5-5
* Describe undo per tool call in the chats docs
The docs said the checkpoint line of a turn carries the change set of
the turn. A turn checkpoint is taken only by a chat with a VM sandbox,
which the service never sets up, so that rev is always 0. What ships
is one change set per file tool call, with a files_change card on its
tool line. The docs now say so and how to undo a whole turn.
Grouping a turn's changes into one change set in the file store is
left for later: it touches the file store's revisions and the undo of
shared folders.
Assisted-by: Claude:claude-opus-5-5
* Mount the skill folders read-only for the built-in bash
The built-in pdf skill ships scripts, but the commands saw only the
home and the shared folders, so the model could read the scripts and
not run them. The built-in bash now mounts the three skill layers
read-only, at the names the file tools already use: shared-skills,
shared-builtin-skills and shared-addon-skills. A shared folder named
skills moves to "shared-skills 2", so the two never collide.
A read-only mount with subfolders made every command report its
folders as changes the mount refused, because the host backend lists
every folder of its work tree. A folder alone is no change: both
stores now refuse only files, whiteouts and opaque folders there.
The Sandbox add-on mounts only the home; the docs say to copy a
script there first.
Assisted-by: Claude:claude-opus-5-5
* Find the add-on that runs commands by what it shadows
The chats knew the add-on that runs commands from a per-process cache,
filled with the Sandbox's own tool name the first time a chat listed
its tools. Another add-on that replaces bash did not count, so skills
that need commands showed as unavailable. The admin network view read
the same cache: after a restart it said no Sandbox serves until
someone chatted, each frontend of a cluster could answer differently,
and a removed add-on stayed listed.
The tool that runs commands is now the one that shadows bash, as the
spec asks. sandboxRunTool and the cache are gone. The stand-in of an
add-on that cannot work carries the built-in tools it replaces, and
the admin view asks the add-on host which add-on serves bash now: the
pinned one or the oldest running install, and none when the pinned one
is stopped.
Assisted-by: Claude:claude-opus-5-5
* Show international host names in punycode in the network checks
The card-spoofing defence of the sandbox network asks for host names
in their ASCII form, but CheckHost passed Unicode names through. A
look-alike name reached the consent card as text that reads like the
real site.
CheckHost now converts an international name with the IDNA lookup
profile and refuses one with no valid ASCII form. The conversion runs
before the address checks, so full-width digits that spell a loopback
address are refused as loopback.
Assisted-by: Claude:claude-opus-5-5
* Keep the seeded Studio and Talk pins until the add-ons are installed
The upgrade pins Studio and Talk for existing accounts, but both are
add-ons that an administrator installs later. The sweep marked a pin
whose add-on is not installed as missing and deleted it after 30
days, so an administrator who installed Studio late found the seeded
pins gone.
The sweep now records the first time a seeded add-on is present, in
the table of seed runs or the pins file. A seeded pin keeps no clock
until then, and a mark that an older sweep set is cleared. After the
first install the pin follows the retention rule as before.
Assisted-by: Claude:claude-opus-5-5
* Fix two small labels of the Skills and Backends pages
On the Skills page, the skill of an add-on that is not chosen said
"Replaced by <add-on>", the label of a built-in skill that an add-on
serves. It now says "Not active: <add-on> provides it", as the add-on
skills spec asks.
On the Backends page, Clear filters on an empty Installed search left
the library filter set, so the list stayed empty. It now clears the
library filter too.
Assisted-by: Claude:claude-opus-5-5
* Tell the model when no network card can appear
Each turn told the model that a blocked host asks the user. No card
appears when the ceiling is off or no wider than the chat's preset, in
an unattended run or in a sub-agent, so the model waited for an answer
that could not come.
The line now says that a blocked host is refused and nothing asks the
user in those cases, so the model names the host it needs in its
answer.
Assisted-by: Claude:claude-opus-5-5
* Translate six Backends and preview labels in every locale
check, reinstallBackend, deleteBackend and filterLabel of the Backends
page and the label and failed text of the file preview showed in
English in the eight other locales. They now have translations, and
the i18n baseline goes down to match.
Also fix a comment of the home composer that still named the removed
Canvas chip.
Assisted-by: Claude:claude-opus-5-5
* List the admin routes that have no assistant tool, with the reason
The admin oversight routes of saved agents, triggers and connectors
had no MCP tool and no recorded reason, and the other deliberate skips
lived only in code comments. coverage_test.go now keeps them in
adminRoutesWithoutTool with a reason each, and a spec checks that no
route there is also wrapped by a tool. The contributor checklist names
the map.
The tool catalog of the assistant docs said it matched the real one
but missed most tools added since. It now lists every tool, read-only
and mutating, and the actions that stay in the web interface.
Assisted-by: Claude:claude-opus-5-5
* Close the doc and registry gaps of the chats and add-ons
- /api/agent/* answers 404 with the chats_off reason when the chats
are off, as /api/chats/* and /api/workspace/* do.
- The add-ons docs get an upgrade section: Studio and Talk are
add-ons, nothing is preinstalled, and users need the addons feature,
which the upgrade does not grant. It says how an administrator
installs them (also from a private library) and grants the feature,
and carries a paragraph for the release notes.
- The migration page no longer lists fork, edit, the chat settings and
the token counters as removed.
- Runtime settings document chats_vision_model,
chats_transcription_model and the skill keys of
chats_tool_providers.
- The docs show POST /api/chats/files/folders,
GET /api/chats/files/history and the allow route of a connector
sender.
- The in-tree specs get amendments where the shipped design differs.
- A route test of the tagged suite still set the removed Runs field.
Assisted-by: Claude:claude-opus-5-5
* Look for the kit empty state in the backends regression spec
The Backends page now draws the shared EmptyState of the kit
(ui-empty), not the old dk-empty block, so the regression spec of the
gallery suite found no empty state after a search with no match.
Check the kit's no-results form and its title.
Assisted-by: Claude:claude-opus-5-5
* Ignore a features request that outlives its page in the add-on stubs
stubAddonRoutes answers /api/features by fetching the real flags and
turning add-ons on. A spec that ends while the page polls, like the
one that checks that a viewer without the feature never reads the
list, left that fetch in flight, and Playwright failed the spec with
"route.fetch: Target page, context or browser has been closed" after
its checks had passed. Abort the request when the fetch fails because
the page is gone.
Assisted-by: Claude:claude-opus-5-5
* Bring the gallery-ui case inventory up to date
The suite lists the node unit cases of addons.test.mjs and the
Playwright specs of addons-gallery.spec.js. The inventory missed the
unit cases added since it was written, and one spec was renamed when
the legacy switch came in ("hub: the Add-ons tab appears only for an
account that holds the feature" is now "hub: under the legacy switch
the Add-ons tab appears ..."). The list comes from a local run of
both parts against the UI test server.
Assisted-by: Claude:claude-opus-5-5
* Read spreadsheets, documents and slides in read_document
read_document said "file.xlsx has no text layer; it may be a scan" for
workbooks that some Python libraries write. Their workbook gives the
sheet as an absolute part name (/xl/worksheets/sheet1.xml) and the
cells as inline strings. The text extractor of the harness joined
"xl/" with that name, found no sheet, and returned no text; the tool
then took any empty result for a scanned PDF.
The new package core/services/studio/extract has one entry point,
Extract(ctx, name, data), that returns the text and the structure it
read, for read_document now and for a file indexer later. Office files
go through the files preview reader, which resolves both kinds of part
names and reads inline strings:
- a workbook comes sheet by sheet, rows as tab-separated lines, up to
2000 rows and 50 columns a sheet, with a note for a cut sheet;
- a Word document gives headings, list items, paragraphs and tables;
- a presentation gives each slide with its title, text and notes.
PDF and EPUB keep the harness extractor. Only a PDF with no text is
reported as a probable scan; an empty Office file is reported as empty.
The preview reader now keeps a Word table as one block with its rows,
and the preview pane draws it as a table.
Assisted-by: Claude:claude-opus-5-5
* Add the missing explorer, library and tools translations
Eight locales had no explorer.json, library.json or tools.json. The
browser asks the server for every namespace of the language it shows,
so an Italian page that used the library answered 404 for
/locales/it/library.json on every load and fell back to English.
The 24 files now exist, translated, with every key of English. The
locale gate counted a missing file as missing keys, which the baseline
allowed. It now fails on a missing namespace file whatever the
baseline says, and the baseline is 24 entries shorter.
Assisted-by: Claude:claude-opus-5-5
* Back off while the add-on service is unavailable
A page that followed an add-on upgrade read the operation every two
seconds, up to 300 times. While the add-on service restarted, each read
waited for its timeout and answered 503, so the log of one session had
48 failed dispatches in a row, and the page only showed the upgrade as
running. The Sandbox state of a chat's Tools menu read once and, on a
503, said nothing more.
The reads of an operation now back off while the service does not
answer: 2, 4, 8, then up to 30 seconds. The upgrade entry says that it
waits for the add-on service and when it checks again, in Activity and
in the details, which also get a Check now button. The Tools menu says
that the Sandbox is starting and reads its state again with the same
back-off, kept for the page, so opening another chat does not ask
again before the wait is over.
Assisted-by: Claude:claude-opus-5-5
* Start a chat with its switches and list them before it exists
The composer of a new chat could not show the Tools and Skills menus,
because the lists and the switches belong to a chat. A chat started
from Home therefore always began with every tool and skill on.
- POST /api/chats takes switches: the add-ons, MCP servers and
connectors, built-in groups and skills that are off, and the opt-in
groups that are on. They are checked as the PUT endpoints check them
and stored with the new record, so the chat never exists without
them; a bad entry refuses the request and creates nothing. The SQL
store now writes these lists on create too.
- GET /api/chats/draft/tools and /api/chats/draft/addons list what a
chat on a model, with a saved agent, could use, every switch in its
default state. They refuse what create refuses: an unknown or
forbidden model, an agent without the agents feature, another
user's agent.
Document both in the chats page and regenerate the swagger.
Assisted-by: Claude:claude-opus-5-5
* Give Home the composer of the chat page
Home drew a smaller message box than the chat page: no files, no
agent, no Tools or Skills menus, no paste to attach, no rich model
list, and its text was lost on a reload. A chat started there could
not be set up before its first message.
Move the composer wiring out of the chat page into one place that
both pages use:
- useChatComposer holds the draft (text and files), the model and the
saved agent of a new chat, the chat models, the switches and what
attaching and pasting do.
- ChatComposer draws the model list (fit, vision, memory bar), the
agent picker, the Tools and Skills menus, the attach buttons,
dictation, the add-on slot and the vision notice over HomeComposer.
- useChatSwitches has a draft mode: before the chat exists it lists
from the draft endpoints, a switch changes only the screen, and the
choices go with the create request. They outlive a change of model
or agent.
Home now creates the chat with the model, the agent and the drafted
switches, sends the first message with its files and opens the chat.
When the message cannot go, the new chat is removed and the text and
files stay on Home. Its text is kept in the browser across a reload.
Home also gets the shortcuts row, the notices of the chat page and the
line that says whether the model is loaded. Its own slash actions,
widgets, examples and first-run steps stay.
The empty chat page gets the same draft menus. On a phone the chips
of both wrap to a second row, and the Tools and agent menus open as a
sheet from the bottom, as the skills picker does.
Assisted-by: Claude:claude-opus-5-5
* Move add-on settings to the add-on's own page
The settings.sections slot drew every add-on's settings at the end of the
global Settings page, and the Sandbox and Web sections were unstyled: the
add-on UIs used class names the host does not define (btn, btn-primary)
and bare inputs, the SDK offered no form primitives, its muted token
named a property that does not exist, and the slot root was an
inline-flex span that laid a form out in a row.
An add-on's settings sections now render on the Settings tab of its
details on the Add-ons page (/app/addons?detail=<id>&view=settings),
narrowed to that add-on, for administrators only. An add-on page gets a
host Settings link to that tab. The Settings page no longer draws the
slot: an Add-ons group, shown when an add-on has settings, lists those
add-ons with a link each. The old #ad-section-<addon>-<id> anchor
redirects to the tab once the apps list is read. Server settings of an
add-on's feature stay on the Settings page; the tab shows them under
"Server settings for this add-on", with a read-only summary of the
Sandbox network and links to ?group=tools#settings-egress and
#settings-builtin-tools.
The SDK gets a settings kit in sdk.ui (Section, Field, Select,
TextInput, Switch, Choice, StatusChip, Callout, List/ListRow, Code,
Actions, Table, ExternalLink), drawn with host classes and tokens, and
more theme tokens. The Sandbox and Web sections are rebuilt with it:
backend rows with a status chip and the raw reason collapsed in a code
block, a backend choice with one line per option, and the AppArmor and
user-namespace hint (with a docs link) only for the gVisor
"exit status 128 ... cannot read client sync file" failure.
The Settings page showed "Changed, default 30, Reset" next to a 0 for the
audit retention, and the same for the Sandbox network limit, preset and
ports: the server stores 0 or an empty value for a setting never set.
Those fields now read that value as their default, and the retention
input has its unit and bounds.
Assisted-by: Claude:claude-opus-5-5
* Adapt the add-on runtime specs to the new navigation
Three specs of the esm-ui suite still assumed the old shell:
- the Add-ons tab that lists the pages add-ons declare is part of the
legacy layout, so the navigation spec found no hub subnav; it now
sets the legacy switch on the server's flags for that page;
- the sidebar has no "Chat" link any more, so the unsaved-edits spec
could not leave the page; it leaves through Home in the primary
navigation;
- the shell keeps a notification stream open, so the network never
goes idle and the spec for a member without the feature waited out
its timeout; it waits for the server's switches, the account and
the shell instead.
Assisted-by: Claude:claude-opus-5-5
* Name the legacy Add-ons tab unit case as the esm unit tests do
The esm-ui inventory listed "unit: registry: the Add-ons tab lists
page links only when the registry has usable add-ons", but the unit
test is named "the legacy Add-ons tab lists ...", so the suite could
never match its inventory. Use the name the test prints.
Assisted-by: Claude:claude-opus-5-5
* Say on the Collections page when the embedding model is missing
At startup the server opened a collection and its first embedding call
got 404 from POST /embeddings. The collections embed through this
server's own API with collections_embedding_model, the default
granite-embedding-107m-multilingual, and that model was not installed.
Nothing told the user: uploads and searches just failed.
GET /api/collections now also returns embedding_model and
embedding_model_installed (a loaded configuration of that name, or a
model file of that name). When it is missing, the Collections page
says "No embedding model installed", names the model, and gives an
administrator a button that opens the Models page with a search for
it.
Assisted-by: Claude:claude-opus-5-5
* Report modified times and the upload limit in folder listings
The Files page could not sort by date or show when a file changed: a
folder listing had no time. It also learned the upload limit only after
it had sent the whole file. The default server body limit is 15 MB,
lower than the 256 MiB a workspace file may have, so a large upload
failed late and without a clear reason.
Each listed entry now has "modified", the time of the change set of
its revision. Both workspace stores fill it in List: the JSON store
from its in-memory change sets, the database store with one query per
batch of revisions. The listing also has "max_file_size", the smaller
of the workspace limit and the server body limit, so a client can
refuse a larger file before it sends any byte.
The store contract suite checks the new time against the history, and
the route tests check both fields.
Assisted-by: Claude:claude-opus-5-5
* Add a sortable file list with thumbnails for the Files page
The building blocks of the new file browser, used by the page in a
later change:
- fileBrowser.js: the kind of an entry from its extension, the order
(folders first, natural and case-insensitive names, then size,
modified time or type), the filter of a folder, dates in words with
the full date for a tooltip, the Shift range, the arrow moves of a
list and of a grid, type-to-find, names that do not clash, why a
name cannot be used, the name of a pasted screenshot and the view
settings read back from storage.
- FileList: a grid with sortable column headers or a grid of tiles
with image thumbnails, a check box per row, a row menu, inline
rename with the reason a name is refused, rows that can be dragged
onto folders, and the keyboard: arrows, Shift, Space, Enter, F2,
Delete, Ctrl+A, Escape, Backspace and typing a name.
- MoveDialog: a folder picker that walks the tree and refuses a move
into the moving folder itself.
- ActionMenu can open at a point with no trigger (a right click), and
its trigger can take another icon and class (a split button).
- fileSize also writes gigabytes.
Assisted-by: Claude:claude-opus-5-5
* Add an upload queue with progress, cancel and retry
An upload of the Files page was one fetch per file in a loop: nothing
showed while it ran, one failure stopped the rest, and leaving the
page lost it.
- uploadQueue.js keeps every upload outside React: queued, sending
with the bytes sent and a smoothed rate, done, failed with a reason
and whether Try again can help, or canceled. Two files go at once.
It lives for the whole app, so opening another folder or page does
not stop an upload.
- filesApi.uploadWithProgress sends the same PUT with XMLHttpRequest,
which reports upload progress and can be aborted. The server already
streams the body into the store; nothing changes there.
- useUploads reads the queue and asks before the tab closes while a
file is on its way.
- UploadTray shows the queue at the bottom of the page: the total and
the time left, each file with its progress, speed and time left,
Cancel, Try again with the reason, and a done state that fades out.
A live region tells a screen reader when uploads start, fail and
end.
- ConflictDialog asks what to do when an upload has the name of a file
in the folder: replace it, keep both, or skip it.
- dropFiles.js reads dropped folders with their paths.
Assisted-by: Claude:claude-opus-5-5
* Rebuild the Files page as a file browser
The Files page was a list of links: the store's byte order, the same
icon for every file, three icon buttons on every row, no progress on
an upload, no selection, and on a phone a preview that opened below
the whole list.
Browsing: a breadcrumb that folds a deep path, sortable columns (name,
modified, type, size) with folders first, a list or a grid of
thumbnails, both kept per user in the browser, a filter of the open
folder with a search of every file on Enter, type icons, human sizes,
dates in words with the full date on hover, and Back and Forward
between folders.
Uploads go through the upload queue and its tray: the Upload button
or its folder menu, drops of files and folders anywhere on the page or
onto a folder row or crumb, and paste. A clash asks to replace, keep
both or skip; a file over the server limit is refused before it is
sent.
Selection and actions: click, Ctrl and Shift, check boxes, a selection
bar (Download, Move to, Move to Trash; at the bottom of the screen on
a phone), a row menu and a right click, inline rename and new
folder or file, drag to move, the keyboard, and a toast with Undo for
a delete, a move and a rename.
States: a skeleton on the first load, an error with Try again, empty
states that say how files get here, and a no-match state. The preview
uses the side pane: docked on a wide screen, a sheet on a phone, with
the same actions as the row menu. Trash, History and the Managed by
LocalAI group stay as they were.
Every new string is in the nine languages. The page uses design
tokens only (seven raw font sizes fewer). The specs cover sorting,
the filter, the views, selection and bulk actions, the keyboard, the
menu, rename checks, drag to move, drops, paste, clashes, the size
limit, upload progress, cancel and retry against the real server, and
the error states.
Assisted-by: Claude:claude-opus-5-5
* Fix two specs that fail on the integration branch
The Describe spec of the egress package checked that the no-card line
does not contain "asks the user", but that line says "nothing asks the
user". It now checks for the sentence that promises a card.
The context meter spec gave the chat a 4k context window. The tool
schemas of the chat now take 3k of it, so the turn stopped before the
model request. The spec uses a 16k window.
Assisted-by: Claude:claude-opus-5-5
* Translate the always-allowed hosts note of the chat network row
The three egress keys of the always-allowed hosts were only in English,
so the locale check failed for every chats namespace. The workspace
namespaces now miss two keys fewer, so their baseline goes down too.
Assisted-by: Claude:claude-opus-5-5
* Bring the contracts and host-slots case inventories up to date
The contracts suite has eight new specs for the check of a tool that
shadows a built-in. The host-slots suite runs the unit case of the file
preview slot, which the inventory did not list.
Assisted-by: Claude:claude-opus-5-5
* Add the design of the file text index
The search of the chats' files reads only small UTF-8 text files, so a
PDF or an Office file is found by name only. The design extracts the
text when a file is written, keeps it in LocalAI's own store keyed by
node and hash, checks access at query time and runs one indexer per
cluster under an advisory lock.
Assisted-by: Claude:claude-opus-5-5
* Give extracted documents their pages, sheets and slides
Extract now says where each page, sheet and slide starts in the text,
so a search hit can name its page or sheet. A PDF page starts with a
"## Page N" line, as sheets and slides already start with a heading.
A PDF also stops at a page limit (200 by default) and at the text
limit, which it ignored before, and a panic of the reader is an error.
The test fixtures move to extracttest, with a real text PDF and a
workbook with shared strings, so other packages can build the same
documents.
Assisted-by: Claude:claude-opus-5-5
* Add a store for the text of workspace documents
textindex keeps one entry per workspace node: the text read from a
PDF or Office file, the hash it was read from, its pages or sheets and
its status (pending, ready, failed, skipped), plus the cursor of the
indexer in the workspace journal.
The GORM store keeps a lower-cased copy of the text, so SQL and the
Go matcher agree on every script, and searches it with FTS5 and the
trigram tokenizer on SQLite or a pg_trgm index on PostgreSQL. Without
them it scans the column. The JSON store keeps the metadata in memory
and each text in its own file, and scans the texts. One contract
suite runs on JSON, SQLite and PostgreSQL.
Assisted-by: Claude:claude-opus-5-5
* Index the text of new and changed workspace documents
The indexer tails the workspace journal from a cursor kept in the text
store. A write of a PDF, Word, Excel or PowerPoint file marks it
pending (the same hash only follows the path), a delete removes the
entry and a move rewrites the paths below a folder. Then it reads the
pending files two at a time, with a timeout, a size cap, a page cap and
a text cap, and stores ready, failed or skipped. A panic of the reader
fails the file, not the pass. The same content is read once.
The first pass starts at revision 0, which indexes the files that were
there before, in bounded steps that resume from the cursor. Enqueue
marks an upload pending at once and Forget drops the entries of nodes a
purge erased. In a cluster each pass runs under a new advisory lock,
so one frontend reads.
Assisted-by: Claude:claude-opus-5-5
* Search and read the text of documents in the chats' files
search_files and GET /api/chats/files/search now also search the text
index: after the name and text-file hits come lines of PDF, Word, Excel
and PowerPoint files, as path (page 3, line 40): text, with the sheet or
slide where a document has them. Each entry is shown only when the
caller may read its file now and the file still has the content the
text came from. Documents keep up to a third of the hits. When files
below the path are still pending, the answer says so instead of
"Nothing found.", and the REST result has indexing.
read_file returns the text of a PDF or Office file with the usual offset
and limit, from the index when it matches the file and read on the
spot otherwise. An upload and a write of a chat tool queue the file at
once, and emptying the Trash drops the entries of the purged files.
Assisted-by: Claude:claude-opus-5-5
* Start the file text index with the files and add its settings
With files on, the server opens the text store next to the workspace
index (the auth database, or a folder under the data path) and starts
the indexer. A chat worker searches the index but runs no indexer.
chats_index_files (default on) switches the reading off and
chats_index_max_text_bytes (2 MiB by default, 64 KiB to 16 MiB) caps
the text kept for one document. Both apply at the next pass, and both
are rows of Settings > Tools.
Assisted-by: Claude:claude-opus-5-5
* Show the page, sheet or slide of a document hit on the Files page
Search all files now lists hits inside PDF and Office files with their
page, sheet or slide instead of a line number of the extracted text,
and says how many documents are still being indexed. The strings are
in all nine locales.
Assisted-by: Claude:claude-opus-5-5
* Test finding an uploaded PDF by its words through the routes
The PDF has a generated name and is uploaded with the REST upload. The
search route first reports it as being indexed, then finds it on its
page after a pass of the indexer, and so does the search_files tool of
a chat. Another user finds nothing.
Assisted-by: Claude:claude-opus-5-5
* Document the search inside documents
The chats page says what the index reads, where the text is kept, who
finds it, which search engine each database uses, the two settings,
the backfill and the single-runner mode in a cluster, and how
search_files and read_file answer for documents. The API instructions
and the PDF skill say the same in short.
Assisted-by: Claude:claude-opus-5-5
* Test the indexer on the JSON workspace store
A server without a database keeps the workspace and the text in JSON
files. The indexer follows the uploads, a folder move and a delete in
that journal as it does in the database one.
Assisted-by: Claude:claude-opus-5-5
* Read all of a long document with read_file
The index keeps the first chats_index_max_text_bytes of a document, so
read_file used to stop where the index stopped. It now uses the index
only when it holds the whole text, and reads the document again (up to
16 MiB of text) when the index cut it.
Assisted-by: Claude:claude-opus-5-5
* Start bash in the user's home
The Sandbox's bash and the built-in bash started in /workspace, the
folder that holds the mounts, so a model that ran a script by a
relative path failed and a file it saved there was lost. A call now
starts in /workspace/home, the folder of the user's files: a relative
path is the same file the file tools see. The tool descriptions and
the commands block of every turn say where a call starts and that
only what is saved there is kept. The built-in bash says that the
/workspace paths do not exist on the host and that a shared folder is
next to the home.
Assisted-by: Claude:claude-opus-5-5
* Name files in tool results by the path the model uses
write_file, edit_file and search_files named a file of the home by its
path in the home ("/scripts/build.py") while the model had written
/workspace/home/scripts/build.py, and a model took the two spellings
for two files. The results now give the path the model uses, and
move_file says where it moved the file to. The cards and the change
notes keep the path in the home.
Assisted-by: Claude:claude-opus-5-5
* Name the paths a command's commit did not save
The result of a bash call ended with "N file(s) changed in your files
meanwhile; the sandbox's version was kept as a copy" for every path the
commit refused, also for a file written outside the user's files,
where no copy is kept. It now has one line a path, with the path as
the guest sees it, the reason, and the path of the copy when the store
kept one, so the model and the user can act on it. The host returns
the paths and the copy to the Sandbox add-on relative to the user's
files, as its mounts are.
Assisted-by: Claude:claude-opus-5-5
* Show the files a command saved under its step
A document that a script wrote with bash reached the user's files, but
the chat showed nothing: only the file tools put a file on their step,
and the Sandbox committed for no chat, so the chat's file list, the
chips under the answer and the preview pane never saw the file.
The host now commits a Sandbox call for the chat that ran it (the
action resource of the request names the chat). After each bash call,
of the Sandbox or of this machine, the chat reads the change sets of
the chat that the call made from the shared store and puts the files
on the step's card, next to the card that says who served the call
(session.SetToolFiles; files with path, change set and op on the
card's new files field). The step lists them as chips that open the
preview pane, the chat's clients are told that its files changed, and
the files reach the file list, so the answer gets its chips and the
pane opens on the newest one, as for write_file.
Assisted-by: Claude:claude-opus-5-5
* Keep JSON out of the answer of a trigger run
The workspace skill told the model to put the widget data shape and a
source marker in a trigger prompt. Some proposals then asked the run to
print that JSON, and every run ended its chat with a stray line of it.
The skill now says source and note live inside the data file and are
never the answer, and that a prompt must not ask for JSON. The runner
appends one closing instruction to the first message of every trigger
run: finish with one short plain sentence. The propose_trigger and
update_trigger descriptions say the prompt must stand alone and must
not ask for JSON. Connector chats are talks with people and do not get
the line.
Assisted-by: Claude:claude-sonnet-5-5
* Read the images of a page as base64 for its frame
The bridge of a page reads UTF-8 text only, so the UI could not put a
png or a jpg of the page's folder into the page. A read of the page
file routes with encoding=base64 now returns an image as base64 with
its media type. Only the extensions of an image are read this way
(png, jpg, gif, webp, avif, bmp, ico, svg); any other file is 415, so
this is no way to read an arbitrary binary file. The path checks, the
1 MiB limit, the viewer's rights and, for a shared page, the consent
check are the same as for a text read.
Assisted-by: Claude:claude-opus-5-5
* Let a page use the files next to it
A page runs from a srcdoc with no network, so fetch('news.json'),
<script src="app.js">, <link rel="stylesheet" href="style.css">,
<img src="logo.png"> and import './x.js' all failed without a word.
Models that split a page into files, or load its data with fetch,
made pages that showed nothing.
Before the frame loads, the UI now reads the files that the page names
with relative URLs, with the same calls as the bridge, and puts them in:
classic scripts in place (deferred ones at the end of the body), style
sheets as <style> with their @import and url(), images as data: URLs,
and ES modules as blob: URLs behind an import map, so cycles and
import() work. A page with modules gets blob: in script-src; no other
part of the policy changes. Inside the frame, fetch and XMLHttpRequest
read relative URLs with files.read and answer with a real Response (200,
404 for a missing file, 413, 415); any other URL rejects with "network
is off in pages; put the data in a file next to the page".
Every path is resolved inside the page's folder: .., %2e%2e, absolute
paths, backslashes and protocol-relative URLs are refused, and the
query and the fragment are dropped. The files go in only after a shared
page is allowed, with the viewer's rights. The files that go in are at
most 4 MiB, images 2 MiB; a script or style sheet that does not load
shows a notice in the page. A change of one of these files reloads the
page, as a change of index.html does.
The skills, the docs and the shared pages fixture say and test the same.
Assisted-by: Claude:claude-opus-5-5
* Return tool descriptions from an MCP connector test
A connection test listed only the tool names. A person who adds a
server cannot tell from a name such as "query" what the tool does or
whether to keep the connector on.
The test response now also has tool_details: each tool with the
description its server gives, in the order of tools. tools is
unchanged, so existing clients keep working.
Assisted-by: Claude:claude-opus-5-5
* Add jq, ripgrep, sqlite3 and archive tools to the Sandbox guest
The guest user cannot install packages and apt is blocked by the proxy,
so any tool the model needs has to be in the image. Add jq, ripgrep,
sqlite3, zip, unzip, file and tree to the final stage.
A new check-tools.sh runs in the existing build step and fails the build
when one of these programs, Poppler, git or curl is missing or does not
start.
The list is one constant, GuestTools, in the shared sandbox package. The
Commands block of the prompt of an isolated chat names it. The tool
description (capped at 400 characters, already full) and the turn note
(cut at 400 characters) have no room for it. Tests keep the Dockerfile,
the check script and the Sandbox documentation in line with the constant.
Assisted-by: Claude:claude-sonnet-5-5
* Keep the helper files of a chat in its work folder
A chat asked for a Word document left a 19 KB build script at the top of
the user's home, next to the user's own files. Chats like it leave
scripts, intermediate data, logs and drafts at the top level or in
folders such as scripts/, and nothing tells them apart from what the
user asked for.
Each chat now has a work folder, Chats/<chat folder>/work/ in the
user's home, next to its attachments. The chat takes the name of its
folder at its first turn (checked against the folders and the other
recent chats of the user, so two chats with one title get two names)
and makes the folder only when it is used: before a bash command runs,
or when a file is saved there. The prompt of every turn names the
folder and says what goes in it: scripts, helper code, intermediate
data, logs and drafts go there; the deliverable goes where the user
said or at the top of the home; no new top-level folders for scripts;
a run-once command needs no file. A sub-agent is told its parent's
folder. The descriptions of bash (the Sandbox's, within its 400
characters, and the built-in one) and of write_file end with one short
hint, and the built-in office, apps and workspace skills say the same
in one line.
The steering is soft. When write_file, move_file or a command leaves a
helper file outside the work folder, the result gets one line that
suggests moving it, at most once a turn of a chat.
One function (workspace/workfiles) decides the kind of a file:
working for anything in a chat's work folder, or for a script, a log
or a temporary file by its extension or a folder named scripts, tmp or
scratch; deliverable for everything else, and always for pages, the
LocalAI folder, the shared folders and the files the user attached.
The chat's file list (kind), the files of a command's card
(files[].kind) and the card of a file tool (file_kind) carry it, so the
interface only renders it.
The runtime setting chats_work_folder (on by default) turns the
steering off.
Assisted-by: Claude:claude-opus-5-5
* Stop telling the model that the Sandbox has npm
The guest image has no Node, so the network hints named a tool that
does not exist.
Assisted-by: Claude:claude-sonnet-5-5
* Update the Sandbox description test after dropping npm
Assisted-by: Claude:claude-sonnet-5-5
* Open the links of a page after the person confirms them
A page runs in a frame without allow-popups or top navigation, so a
click on <a href="https://..."> did nothing, and a link to #id loaded
LocalAI's own address in the frame, since a srcdoc resolves it against
the parent. A news page had only dead links.
The bridge of the frame now takes the clicks on links (also with a
modifier, the middle button or Enter) and window.open(), and sends
link.open(url) to the host. The host lets through only an http or https
address without a user name or password, at most 2048 characters;
javascript:, data:, file:, blob:, //host and links to other files of
the page are refused with a message in the console. A bar above the
page shows the site in large type, the whole address, and Open and
Cancel. A host in punycode also says how it reads, since it may imitate
another site. Open is a click in the app, so the new tab is not blocked
and opens with noopener and noreferrer. A page asks at most 5 links at
once, then one every 3 seconds. A link to #id scrolls the page.
The sandbox, the policy of the frame and the consent of shared pages do
not change; a shared page asks the same way.
Assisted-by: Claude:claude-opus-5-5
* Let the items of a Home widget link
A list widget showed each item as plain text, so a digest of articles
or a list of pages could not take the person anywhere. An item now has
an optional url. An http or https address without a user name opens in
a new tab with noopener and noreferrer, and its tooltip names the site;
a path of the app (/app/pages, /app/chats, /app/files and the other
screens a person reads) opens inside LocalAI. The label is the link,
with a focus ring, an underline on hover and focus, and an icon for an
address outside the app.
The widget-data schema checks the url, and the server adds an error in
words that names the item and the rule, so a trigger run that writes a
bad link learns why. The skill and the docs show the field.
Assisted-by: Claude:claude-opus-5-5
* Attach any file in a chat with one button
The chat had three attach buttons (image, audio, file) that all add to
the same list, and the file button's dialog filtered on */*, which
some file dialogs read as a type filter that greys out archives such as
.zip. A chat now shows one Attach file button whose dialog has no type
filter. Composers without anyFile keep the three buttons.
Assisted-by: Claude:claude-opus-5-5
* Accept both Node behaviours for a broken punycode label in the link test
Node 22 refuses the address, Node 24 parses it. Both mean the page never
opens it unwarned.
Assisted-by: Claude:claude-sonnet-5-5
* Show a chat's working files apart from what it made
The Files pane, the steps and the preview treated a helper script like
the document the user asked for: a 19 KB build script was listed next
to the document, got a chip under the answer, and the preview opened on
whichever file was newest.
The interface now renders the kind the server puts on each file. The
pane lists what the chat made first and folds the working files into
one quiet heading under the rest, "Working files · 2", with a line that
says what they are; opening it shows their rows, which preview like any
file. A working file opened from a step unfolds the group. The preview
that opens by itself after a turn picks the newest file that is not a
working file, and the chips under the answer name only those files. A
step that saved working files lists them as one line, "Also saved 2
working files", that opens to their chips, and the file of a write_file
step that is a working file is a quieter chip. The files the user
attached are unchanged.
Assisted-by: Claude:claude-opus-5-5
* Add a search field and a segmented filter to the UI kit
The Agents views each drew their own toolbar: a search box capped at
320 px on the agents list, round chip filters on the triggers history,
and a bare checkbox on the tasks page. Add-ons already had the look the
product wants (a search that fills the row and a segmented tray), but
only as add-on classes.
SearchField and Segmented are that toolbar as two small primitives with
ui-* classes and token values only. Segmented buttons carry their id in
data-filter, so specs can press them. The README shows how to use them.
Assisted-by: Claude:claude-opus-5-5
* Draw an agent task run as a history row with its output
The jobs of an agent task were table rows that opened to bare <pre>
blocks: no copy, parameters printed as k=v, the traces in a raw
<details>, and "by manual" in the server's words.
RunItem is one run of a history list: a toggle, the state chip, what it
was and what happened, when and for how long, small actions, and the
details that open under it. OutputBlock draws an answer or an error in
mono with a Copy button. JobRow and JobDetail use them for a job: the
values it ran with as labelled chips, the answer or error, the steps,
the webhook outcome and a link to the chat that holds the run. Times
show as words with the full date as a tooltip.
The agent task helpers gain the pure rules the page needs: the jobs of
a task, the run filters (waiting and running count as in progress), the
on/off filter and search of the task list, the field errors of a save,
how a job started, and the API call that runs a task by name.
Assisted-by: Claude:claude-opus-5-5
* Create, edit and run agent tasks in a side sheet
The task form opened as bare fields above the list, with a checkbox for
On and a Save button that was only disabled when a field was missing.
Run with parameters was a second inline form in the same place.
TaskSheet is a side sheet with three labelled groups (about, who runs
it, what it does), help under each field, the parameters the prompt
asks for as chips while you type, and an On switch. Save checks the
required fields, says next to each one what is missing, and moves the
focus to the first. RunSheet asks for each value the prompt names and
shows the prompt as the task will receive it.
Assisted-by: Claude:claude-opus-5-5
* Show the details of an agent task in a pane with tabs
A task had no details view: its prompt, its runs and the way to run it
over the API were only in the lede of the page.
TaskPane is the kit's side pane with three tabs. Overview shows who
runs the task, the values it asks for, the prompt and a curl call that
runs it by name, with Copy. Runs lists its jobs. Settings holds the On
switch, the webhook count and Delete. Run and Edit stay in the footer.
The tabs move with the arrow keys, Home and End.
Assisted-by: Claude:claude-opus-5-5
* Rebuild the agent tasks page from the shared list pieces
The tasks page looked unstyled next to Add-ons and the agents list: two
raw tables under uppercase eyebrows, a bare checkbox for All users,
forms that opened above the list, three text buttons on every row, and
on a phone a table that broke names one syllable per line.
The page now uses the same pieces as the polished pages. The header has
one line and New task. A toolbar holds the search, an All/On/Off filter
and, for an administrator, an All users switch. Each task is an agent
row: a tile, the name, two lines of description, who runs it and what
it asks for, the last run's state and time with the run strip, an On
switch, Run and a menu. A task with no parameters runs at once. The
name opens the details pane; New task, Edit and Run open the sheets.
The run history is grouped by day, with a state filter, a task filter
and Show more. A job is deleted after an in-page question. With no
tasks the page explains what one is, with an example; a failed load
shows the error with Try again. On a phone the controls grow to 44 px
and the filters scroll sideways.
New strings, and the task strings that existed only in English, are in
all nine locales; the strings the page no longer uses are gone. A new
stubbed spec covers the list, filters, sheets, pane and history, and the
real-server spec follows the new controls. The docs describe the page.
Assisted-by: Claude:claude-opus-5-5
* Rebuild the MCP connectors page on the Add-ons patterns
The page read as a form: every connector was a row of raw controls (a
checkbox, a Test button, red Delete text), the add form pushed the list
down, a test result vanished on reload, and the frame was narrower than
the pages next to it in the sidebar.
The page now uses the same blocks as Add-ons and Files:
- Full page width and the shared PageHeader with a one-sentence purpose
and "Add connector".
- The Add-ons toolbar (search and a state filter with counts) from four
connectors, and the Add-ons rows: a monogram tile, a state chip
(Connected, Needs sign-in, Can't connect, Not tested, Off), one line
with the tools or the reason, a meta line, the kit Switch and a menu.
- Details in the shared SidePane: the switch, the last test, the tools
with their descriptions and a search, the connection with the query of
the address hidden, and what is saved.
- Add and edit in the shared SideSheet: kind first, grouped fields with
inline checks, masked secrets with show and hide, then "Save and
test", the result with the tools, and Turn on. Escape and Cancel ask
before they drop typed changes.
- An empty state that says what a connector is for.
The last test of each connector is kept in this browser with a hash of
what it tested, never the address or a secret, so the page shows the
last known state after a reload. Strings are in all nine locales.
Assisted-by: Claude:claude-opus-5-5
* Document the MCP connectors page and its states
The MCP servers section now describes the add sheet (kind, checks,
masked secrets, Save and test, Turn on), the states a connector shows,
where the last test is kept, the search and the filter, and the details
pane. The test example shows tool_details.
Assisted-by: Claude:claude-opus-5-5
* Render a preloaded page without its loading placeholder
A page that was preloaded still showed the route's loading placeholder
when it opened: React.lazy suspends once on its first render even when
the chunk is already in memory, and React keeps a fallback on screen
for at least 300 ms once it shows one. Opening a chat from Home showed
a blank page for that time.
The page loader now remembers the module once it has loaded and hands
it to React.lazy as a thenable that resolves at once, so a preloaded
page renders in the same commit. A preload of a segment now warms
every page registered under it: /app/chats is the chat page, its
connectors and its conversion page, and before only the last one
registered was preloaded.
Assisted-by: Claude:claude-opus-5-5
* Move the composer from Home into the new chat in one gesture
Sending the first message on Home was a hard cut. The page stood still
while the chat was created, then showed a blank placeholder, the empty
"new chat" screen with its composer in the middle, an empty thread
with the composer at the bottom, and only then the message, while the
sidebar folded and moved the whole column. Focus was lost.
Now Send fades the rest of Home and keeps the composer in place while
the message goes. When the send has answered, a view transition moves
the composer to the dock of the chat, lifts the text out of it into
the message, moves the attached files with it and folds the sidebar in
place; the model's indicator comes in after the message has landed and
the empty composer has the focus. The chat page draws the message from
what Home already knows (a small hand-off store: the chat record, the
text and the files, with the local copy of an image for its thumbnail)
and keeps the same row when the server's transcript replaces it, so
nothing is drawn twice or animates again. What is sent to the server
does not change.
A browser without view transitions gets a FLIP slide of the composer
and the message (Web Animations, transform and opacity only). Less
motion gets a short cross-fade, or an instant switch. A send that fails
brings Home back, shakes the composer and keeps the draft; a second
Enter while one send is in flight sends nothing. The draft is cleared
when Home closes, so the box never shows its placeholder on the way.
Focus mode folds the sidebar in a layout effect, so the fold happens in
the frame the conversation appears. One motion token is added,
--dur-glide (360 ms), for a part that travels to its place on the next
page.
Assisted-by: Claude:claude-opus-5-5
* Draw the triggers as rows and their history like the task runs
The triggers page used a table with a switch column next to the agents
card list, folded its list behind Hide, filtered the history with round
chips, and on a phone cut each history row at the right edge.
Each trigger is now an agent row: a clock or folder tile, the name, the
schedule in the server's words, the agent or model and when it runs
next, the last run's state and time with the run strip, an On switch,
Run now and a menu. The week sentence sits under the Run history title.
The history uses the segmented filter and the same run rows and day
headings as the agent tasks, with Show more after 30 runs; an open run
shows its reason with Copy. On a trigger page the prompt comes before
the runs, in the body face, the On switch is the kit switch, and the
runs stop at 10 with Show more.
Assisted-by: Claude:claude-opus-5-5
* Use the shared search and the full width on the agents list
The agents list stopped its search at 320 px and used the content width,
while the Triggers and Tasks views next to it use the full width, so the
title and the view tabs moved when you switched views.
The list now uses SearchField, which fills the toolbar row as on
Add-ons, and the full page width like its sibling views.
Assisted-by: Claude:claude-opus-5-5
* Polish the agent task sheets and pane after a visual review
Screenshots of every state showed a few rough edges:
- The name placeholder looked like a filled-in value next to the error
"Give the task a name". The name field has no placeholder now, and
the prompt placeholder starts with "For example:".
- A task run by an agent showed the agent id until the agents were
read. It says "Agent" until then.
- A run opened in the details pane also opened in the history, and the
two copies shared an element id. The pane keeps its own open runs and
ids.
- A failed run in the pane drew its meta line in red; the title carries
the error colour there now.
- A group title that is not a legend had a double gap under it, and the
kebab menus were under 44 px on a phone.
Assisted-by: Claude:claude-opus-5-5
* Cover the code files of a shared page in its consent
The consent to a shared page was kept against the hash of index.html
and page.json. The UI also puts the scripts, style sheets and modules
next to the page into it before it runs, so the author could change
app.js after a viewer allowed the page and run new code without asking
again.
The server now finds the same files as the UI's inliner: every script
src, stylesheet link, relative CSS @import and module import of the
folder, followed through the module graph. Their content hashes go into
the hash of the page, and a file the page names that is missing counts
too, so adding it later also asks again. Data files and images stay
out, so saving data does not ask again. The files are listed as
code_files. What a file names is kept per content hash in a small
per-frontend cache, which only saves reads.
Two more checks close the gaps around it. A frame of an older version
that reads a changed file of code gets 409, so a script edited while
the page is open never runs under the old consent. The UI puts no text
file into a shared page that is not in code_files, so if the server and
the UI ever disagree, the file fails to load instead of running outside
the consent. The trust sheet names the files of code when only one of
them changed.
Assisted-by: Claude:claude-opus-5-5
* Let a page be a tile on Home in its compact mode
Home widgets could show a list, a number or the last chats, but not one
of the pages a person or the assistant built. A page can now be a tile.
A page opts in with a widget block in page.json: a size (small, medium
or large, the height of the tile) and an optional short title. A
widget of type page in LocalAI/home.json names the page, and a folder
id for a page of a shared folder. The server takes at most three page
widgets and says why in words, and refuses a name that is not a page
name.
The tile runs the page in the same sandbox and bridge as the full
view. The page reads localai.page.mode ("widget" in a tile, "page"
otherwise) and localai.page.size, and <html> carries data-mode and
data-size, so the page decides how its tile looks instead of being
shrunk. The tile reserves the height of its size, builds its frame only
once it scrolls into view, tells the page through a new visibility
event when it goes out of sight or the tab is hidden, takes the frame
down far off screen, and keeps at most three frames alive. It has a
title bar with Open page and a refresh button, a skeleton while it
loads, Retry on an error, and the link question of the full view. A
page that is gone or has no widget block shows a calm line with a link
to Customise Home; a shared page that the viewer has not allowed shows
Allow this page, which opens the same trust sheet.
Customise Home lists the pages with a widget block, with the size of
their tile. The task board starter has a compact view, the skills tell
the assistant how to put a page on Home, and the docs describe it.
Assisted-by: Claude:claude-opus-5-5
* Give the Files upload its own size limit
The Files page upload was capped by the general --upload-limit (15 MB
by default), although a workspace file can be 256 MiB.
Make 0 the default of --upload-limit, meaning not set. When it is not
set, the general routes keep the 15 MB limit and the Files upload route
skips it, so the workspace file size is its limit. When the operator
sets a value, it applies to the Files upload too, lower or higher (a
file never exceeds the workspace size). The max_file_size of a folder
listing reports the limit in force.
Assisted-by: Claude:claude-sonnet-5-5
* Refuse writes of code files from a page
A page loads its own scripts, modules and styles, so a page that could
overwrite app.js or style.css would run new code on its next load. The
page bridge now writes data files only.
The server refuses a write whose final segment has a code or document
extension (js, mjs, cjs, css, html, htm, svg, xml, xhtml, wasm), is named
index, or is page.json, with 403 and a clear message. The check applies
to the page routes only; the Files page and the assistant's file tools
keep full write access. The judgement lives in pages.IsCodeFile and is
mirrored by isCodeFile in the web host, which refuses early with the
error code forbidden. Both run one table of cases, covering case, double
extensions, trailing dots and spaces, percent-encoding, backslashes and
lookalike dots.
Assisted-by: Claude:claude-sonnet-5-5
* Make the trigger tools a built-in group a chat can switch off
propose_trigger, list_triggers, update_trigger and delete_trigger were
added to every chat a person starts, with no switch. They are now four
entries of the built-in tool registry in one group, triggers, so the
Tools menu, the tools panel, the draft list and the builtins switch
endpoint handle them like the web group. The group is on by default.
A trigger or connector run and a sub-agent still never get them. The
panel lists the group as not usable, with a reason, for those chats,
for a user without the triggers feature, and not at all when the server
runs no triggers. The group has no runtime setting, so the registry
gets an Unlisted flag and Settings > Tools skips it. The panel now
builds every grouped tool to learn whether it is usable, not only the
opt-in ones.
Assisted-by: Claude:claude-sonnet-5-5
* Show the Triggers group in the chat Tools menu
The menu rows come from the groups the server lists, so the row needs
only its label in the nine locales. A Playwright spec covers the row,
its switch request, Select all and Disable all, and the disabled row
with its reason.
Assisted-by: Claude:claude-sonnet-5-5
* Talk about settings, not the files behind them
The settings cards, the shortcuts row, Home and the change notes in a
chat named the files that hold a person's settings: "Defined in
LocalAI/shortcuts.json", "Layout is in LocalAI/home.json", "Saved
LocalAI/persona.json". A person changes their shortcuts or their Home,
not a file, and the Files page still shows the files to whoever looks.
The "Stored in", "Defined in" and "Layout is in" lines are gone; the
history line keeps Versions, Revert and the Invalid badge. The notes
say what changed ("Saved your shortcuts.", "Created the page Task
board."), the errors name the setting or the widget instead of a path,
and the Source view says "The settings as text" instead of JSON and
schema. A failed save now says what happened in one plain sentence and
keeps the server's own words under a collapsed Details. The drafts that
"Edit with the assistant" puts in the composer no longer carry the
path; the built-in workspace skill already maps shortcuts and Home to
their files.
All nine locales are translated; only the changed keys moved, and four
keys that nothing reads any more are removed.
Assisted-by: Claude:claude-opus-5-5
* Drop internal names from the skills, add-ons and tasks copy
Other pages used words that only make sense to someone who knows how
LocalAI is built:
- Skills said "a folder with a SKILL.md" and "the file workspace". It
now says what a skill is for, and that skills need Files.
- The add-on notes blamed "the owner", the internal name of the server
that runs add-ons. They now say what happened to the action, or that
the add-on service is not available; a private library names a
credential "an administrator configured on this server".
- The Tasks header listed an API route, and the first-run wizard spoke
of a "host bash fallback". The memory line of Settings no longer
mentions "the resources endpoint".
All locales that have these keys are translated. The skill delete
dialog, the unused skills intro and the Tasks header exist only in
English and change only there.
Assisted-by: Claude:claude-opus-5-5
* Fail the unit tests when UI copy names implementation details
A new test reads every string of every locale and fails when one names
a settings or data file (.json, .yaml, LocalAI/), an API route or HTTP
method, a setting key (chats_*), an environment variable, a format
word (JSON, YAML, schema) or a path shown as code. Plain words such as
"schema" and "endpoint" are checked in English only, since other
languages use them for other things.
Operator pages may still name the model configuration, the metrics
route or a startup setting that the docs name. Those keys are listed in
ui-copy-allowlist.json, each with the audience and the reason. The list
only shrinks: an entry whose string no longer needs it, or whose key is
gone, fails too.
Assisted-by: Claude:claude-opus-5-5
* Name the triggers group in the chat tools routes
The routes that list and switch the built-in groups of a chat said only
"a name such as web". They now name all three: web, triggers and
saved_agents, with the tools of each.
Assisted-by: Claude:claude-opus-5-5
* Make the built-in skills answer in plain words
The skills told the model to report file names, folders and formats
after a change. Add one rule near the top of localai-workspace and
localai-apps: do the work with files, but say what changed, and name a
file only when the user did or asks. Replace the example replies that
carried paths, keep every path that says where to write, and pin the
rule with a test.
Assisted-by: Claude:claude-sonnet-5-5
* Apply the Go 1.26 modernizers to the latest Studio code
go fix suggested slices.Contains, strings.SplitSeq and range-over-int
on lines added since the first pass. ParseSearchProviders had the same
hand-written membership loop, so it uses slices.Contains too. Only
lines that the branch added since then are changed.
Assisted-by: Claude:claude-opus-5-5
* Share the blob wait and the counting of the two workspace purges
The JSON and GORM stores each had the 2-minute default for the blob
grace, the same switch that reads the BlobGrace option, and the same
loop that counts files, folders and bytes per revision. They now call
blob.Grace and PurgeResult.Add, both with specs. The SQL and the
in-memory walk that pick the nodes stay in each store.
Assisted-by: Claude:claude-opus-5-5
* Read the upper layer of a sandbox in one place for both stores
The JSON and GORM workspace stores had the same readUpper, place,
joinRel, depth and sortedKeys, with the same upperSet types. They move
to workspace/internal/upperset, which has its own specs. The commit
algorithm that applies the set to the nodes stays in each store; it
needs a storage interface first.
Assisted-by: Claude:claude-opus-5-5
* Let chat attachments use the 1 GiB message limit
The docs promise 1 GiB for a message with attachments, but the general
body limit (15 MB by default) ran first, because only the Files upload
skipped it. Skip it also for a multipart POST to the messages route
when the operator set no --upload-limit. A JSON message keeps the
general limit, and an explicit --upload-limit still wins. A message
that goes over the 1 GiB limit now gets 413, not 400.
Assisted-by: Claude:claude-opus-5-5
* Name the read_file tool in the skills index header
A weaker model read a built-in skill with the wrong path and then tried
the Sandbox, which cannot see skill folders. The header now says which
tool reads a skill and that the shown path is the one to use, before it
mentions where the user's own skills go.
Assisted-by: Claude:claude-sonnet-5-5
* Tell the model that skill folders are not in the Sandbox
The Commands block of an isolated chat now says that skills are read
with read_file. The Sandbox tool description and the per-turn note are
at their size caps, so the sentence goes in the system prompt.
Assisted-by: Claude:claude-sonnet-5-5
* Point read_document at read_file for text files
read_document reads PDF, Word, Excel, PowerPoint and EPUB files. For a
Markdown or text file it said only that the file is not a document it
can read. The error now names read_file.
Assisted-by: Claude:claude-sonnet-5-5
* Tidy the text index stores and the indexer pool
The two stores wrote the default limits 20 and 100 as numbers in five
places; they are now named constants, so the stores cannot drift. The
JSON store checks a missing file with errors.Is. The indexer starts its
readers with wg.Go and keeps the first error inline, which drops the
cmpErr helper.
Assisted-by: Claude:claude-opus-5-5
* Refuse the server's public address in the Sandbox proxy
web_fetch refuses the addresses of the server: its interfaces,
loopback and the host of the external base URL. The Sandbox proxy runs
in the add-on process and listed only the interfaces of its own
machine. Under "All public sites", an always-allowed host or an Allow
card, a command could reach the LocalAI API through the public name or
address of the server, a NAT address or a proxy in front of it.
The host now keeps one list for both. The application builds it once,
gives it to web_fetch, and sends it as CIDR strings in the new self
field of every egress.check allow. The proxy keeps the last list it
got and refuses it on each dial, besides its own interfaces, also for
an allow it cached. safehttp gets the codec and a join of deny lists,
so both sides read the list the same way.
Assisted-by: Claude:claude-opus-5-5
* Check the TLS server name on every port of the Sandbox proxy
The proxy checked the ClientHello only on CONNECT to port 443. An
always-allowed host pinned to another port, or any port added to
chats_egress_ports, was a raw tunnel: a command could open TLS to
another tenant of the same front through it. The docs said the server
name must still match.
On other ports the proxy still waits for nothing, since the protocol
may be anything and the server may speak first. But when the first
bytes from the guest are a TLS handshake record, it reads the whole
ClientHello and passes it on only when the server name is the CONNECT
host; otherwise it closes the tunnel and audits the reason sni.
Assisted-by: Claude:claude-opus-5-5
* Say what sent each notification
The inbox only knew whether a notification was about a run or needed
attention, so a trigger that was switched off and a connector that was
switched off looked the same. Each notification now carries a source:
trigger, connector or system (the conversion of old data). The list
endpoint returns it, and items stored before have none.
Assisted-by: Claude:claude-opus-5-5
* Say where a skill is when a read of it fails
When read_file, read_document or list_files fail with not found for a
path that names a skill of the chat (a wrong skills folder, or just
<name>/SKILL.md), the error now gives the skill's real path and says to
read it with read_file. The skills come from the chat's own resolver, so
every layer, add-on and per-user switch is respected, and nothing is
looked up unless a read fails.
Assisted-by: Claude:claude-sonnet-5-5
* Keep the last valid Sandbox network lists when a value is bad
The settings endpoint refuses a bad chats_egress_allowed_hosts with the
reason, and the runtime settings file keeps the value in force. A bad
value that reached the configuration another way was dropped with no
log line: the always-allowed hosts and the extra registries became
empty, and a bad port list fell back to 80 and 443, which can widen a
narrower list.
Each of the three lists now keeps the last value that parsed, logs a
bad value once, and uses the default only when the process never saw
a valid one. The parse runs again only when the value changes, not on
every egress.check.
Assisted-by: Claude:claude-opus-5-5
* Use the keyless chain of the built-in search in the Web add-on
The add-on replaces the built-in web_search, but its keyless provider
was DuckDuckGo alone. DuckDuckGo often answers data-center addresses
with a bot check, so installing the add-on without a key made search
worse than the built-in, which falls back to the Brave Search and Bing
result pages.
The keyless entry of the add-on's order (saved name duckduckgo) is now
the built-in chain from pkg/web/search: DuckDuckGo, then the Brave
Search page, then the Bing page, with the same cool-down. The status
tool reports the providers in their cool-down, and the Settings tab
shows them with the reason. Test the providers gives one row for each
page of the chain.
Assisted-by: Claude:claude-opus-5-5
* Fix the GitHub hosts in the Q1 row of the egress design
The Q1 row still listed raw.githubusercontent.com on the registries
list. The owner decision below it and registries.json exclude it,
since it serves any user content, and add
release-assets.githubusercontent.com. The row now says the same.
Assisted-by: Claude:claude-opus-5-5
* Keep one private-address rule for the network cards
A card asked whether its host is a private IP literal by calling
egress.Admits with a made-up ceiling, which re-derived the rule that
egress already has. egress.PrivateLiteral is now exported (with a spec)
and the card uses it. The two String methods of an allowed host write
host:port with net.JoinHostPort, which brackets an IPv6 literal the
same way the hand-written code did.
Assisted-by: Claude:claude-opus-5-5
* Read a unique constraint violation in one place
The jobs and triggers stores had the same isUniqueViolation, and the
saved agents store had the same test inline before it looked for the
name index. They now call dberr.IsUniqueViolation, which has specs for
the SQLite and PostgreSQL error texts. The workspace store keeps its
own stricter match.
Assisted-by: Claude:claude-opus-5-5
* Ask before a Home widget link opens another site
A list widget turned an http or https item into a plain link to a new
tab. Its data file is written by trigger runs and the assistant, which
read web text, so a planted link opened with one click, and only a
tooltip showed the host. A link of a page asks first.
An external item link now opens the confirmation bar of page links in
the widget: the site, the full address, the punycode warning, and Open
or Cancel. A link inside the app still navigates directly. The bar
takes a label, so the widget says that it wants to open the link.
Assisted-by: Claude:claude-opus-5-5
* Prune the audit of the commands run on this machine
The audit of the built-in bash had no retention: the database table
grew without bound in a cluster, with commands of up to 4 KiB each.
The Sandbox network audit had chats_egress_audit_days and a daily
prune under an advisory lock.
Both audits record what the commands of the chats did, so the host
command audit now uses the same retention and is pruned in the same
daily pass: one frontend of a cluster, under the same lock; a
standalone process alone. The GORM and file stores get Prune. No new
setting.
Assisted-by: Claude:claude-opus-5-5
* Save the Studio JSON stores with atomicfile.WriteJSON
Seven stores (agents, connectors, jobs, MCP connections, notifications,
push subscriptions and triggers) encoded their state and then called
atomicfile.Write with the same four lines. atomicfile.WriteJSON does
both and has specs. The load side stays in each store, since the error
messages differ.
Assisted-by: Claude:claude-opus-5-5
* Build the result page walkers of the keyless search on one walk
hasChallenge and firstByClass each had their own recursive walk that
stopped at the first match. They are now predicates passed to
firstElement, which already did that walk. The order of the walk and
the element tested first are the same.
Assisted-by: Claude:claude-opus-5-5
* Expire network cards that a dead frontend left pending
Pending network cards expired only on the turn-end path of the process
that ran the turn. When that frontend died, its cards stayed pending
in the chat record, counted toward the cap of three, and gave the next
turn's connections cards-full blocks. The UI showed them as waiting.
A card now names the turn that asked it. One rule, shared with the
command card in a later change, says whether a card still has a
waiter: its turn holds the chat and its wait has not run out. A card
that fails it reads as expired, takes no place under the cap, and is
expired in the record when the next card is asked, when the next turn
starts, or when the reaper ends the dead turn.
Assisted-by: Claude:claude-opus-5-5
* Reuse ChooseProvider for the expected add-on of a tool
shadowers.expected repeated ChooseProvider line for line: the pinned
add-on when it is installed, else the oldest install. It now calls it.
commandFiles kept an order field and sorted the nodes of a map by it;
it now appends each node to a list when it first sees it, which gives
the same order without the sort.
Assisted-by: Claude:claude-opus-5-5
* Do not print a nil error when a key or entry is missing
Three checks joined "the read failed" and "the read found nothing" in
one branch, so the second case wrote "<nil>" into the message: the
VAPID keys of push notifications, the sealing key of connectors and a
rendered catalog entry. Each case now has its own message. The message
of a failed read does not change.
Assisted-by: Claude:claude-opus-5-5
* Check the close errors and rename the tool variables
golangci-lint failed on the new studio code. Five deferred Close calls
and one Fprintf into a hash dropped their error, and two locals named t
made forbidigo read `t.Run` as a testing call. Discard the errors
explicitly and name the locals tool.
Assisted-by: Claude:claude-opus-5-5
* Make the file text index forget purged files and say its limits
Three gaps of the text index of documents:
- A purge dropped the index entries of what it erased in a goroutine
that only logged its errors, and the purge had already erased the
journal rows. A crash or a store error left the text in the database
for good. Each pass of the indexer now also checks a batch of entries
(200 every 30 s, round all of them) and deletes those whose node is
gone, so what a failed Forget leaves is removed.
- A document search stopped at 20000 files or 64 MiB of text without a
word, and the "still being indexed" count stopped at 50. The search
now sets Limited (search_files says "results limited", the REST
answer has "limited", the Files page shows a note), and the count is
the number of pending entries below the path.
- read_file extracted a long document again for every page, up to 50
MiB and 60 s each time. The first read keeps the whole text in the
index store by content hash (a new documents table, or files in the
JSON store), and the next pages come from it. The text goes away with
the last entry of that content, on delete and on purge.
Search reads the text without its folded copy, which only the database
needs.
Assisted-by: Claude:claude-opus-5-5
* Stop the PDF reader when the extraction gives up
The PDF and EPUB reader runs in a goroutine, and the call returns after
60 seconds when it is slow. The goroutine went on reading every page,
since its context ended only with the caller's. It now gets a context
that ends when the call returns, so it stops at the next page. What
the call returns does not change.
Assisted-by: Claude:claude-opus-5-5
* Compare the code files of a shared page as the store does
The guard that refuses new code to a frame of an older version compared
the name it reads with the names of the page's code exactly. The
workspace folds case, so an old frame that read Index.html or JS/App.js
got the new code with no 409. Compare names in the store's form
(workspace.FoldName), as the write check already does, and test the
case variants.
Assisted-by: Claude:claude-opus-5-5
* Expire the command card of a turn that is over
The card of the built-in bash was cleared only by a defer in the call
that asked. When that frontend died, the card stayed in the chat
record. The chat hid it only while no turn ran, so the next turn
showed it again, and an answer to it reached no one but still got 204.
The card now names its turn and when it was asked, and follows the
same rule as the network cards: it waits only while that turn holds
the chat and its five minutes have not run out. GET /api/chats/{id}
shows only such a card. The next turn and the reaper clear a card left
behind, and the waiting call clears only its own card, so it never
removes a newer one. An answer to a card with nobody waiting is
410 consent_expired, and the chat shows the card as not answered, with
no buttons. The store gets UpdateHostConsent, an atomic change of the
card.
Assisted-by: Claude:claude-opus-5-5
* Answer a trigger change once across frontends
ApplyChange, UndoChange and DismissChange checked the outcome of the
change record under a mutex of one process, then wrote the store. Two
frontends could both see a pending change and both apply it, or both
undo it.
Add Store.SetOutcome, a conditional update of the outcome (only from
the outcome it names), to the GORM and JSON stores and to the store
suite. The three answers now claim the record with it before they
write, give the outcome back when the change itself fails, and answer
ErrProposalAnswered (409) when another request won. A test runs the
two answers on two frontends at the moment between the read and the
write, for an edit, its undo, and a delete.
Assisted-by: Claude:claude-opus-5-5
* Add the shadowers spec to the tables-tools inventory
The tool adapter gained a spec that lists the add-ons that replace a
built-in in any state. The tables-tools suite runs every spec of that
package, so verify.sh found one case more than the inventory holds.
Assisted-by: Claude:claude-opus-5-5
* Offer Clear filters when the backend Catalog shows nothing
The Installed view had a Clear filters button in its no-results state;
the Catalog view did not, although the spec asks for it in both. Move
the button to one place and show it in the Catalog when a search, a
type filter or a library filter leaves no row. Add Playwright specs for
both views, which the earlier Installed fix lacked.
Assisted-by: Claude:claude-opus-5-5
* Give the chat settings without an assistant tool their reason
chats_max_iterations, chats_work_folder, chats_index_files and
chats_index_max_text_bytes had no MCP tool and no entry in
adminRoutesWithoutTool, and the older chats_saved_agents,
chats_suggestions, chats_vision_model and chats_transcription_model
were unlisted too. Add each with the reason it stays a person's choice
in Settings: they apply to every user's chats, and the step limit is
one a model must not raise for itself.
Assisted-by: Claude:claude-opus-5-5
* Remove the stray allowed-hosts keys from the chats inbox locales
allowedHosts, allowedHostsOff and allowedHostsNote sat twice in eight
locales: under egress, where English has them and the UI reads them,
and again under inbox, where nothing reads them. In de and ko the two
copies had drifted apart. Delete the inbox copies and keep the egress
ones.
Assisted-by: Claude:claude-opus-5-5
* Bring three design specs in line with what shipped
- web tools and shadowing: the status still said phase 2 waits; both
phases shipped. Note that the Web add-on's keyless entry is now the
built-in keyless chain.
- Home front door: Home now creates the chat and sends the first
message itself, with a short-lived hand-off to the chat page, and
its composer takes attachments. Both were described the old way or
as out of scope.
- chat footer: the meter bar is 96 px with the tokens left, and a phone
keeps the bar and the tokens left, not the percentage only.
Assisted-by: Claude:claude-opus-5-5
* Store the command card's answer and read it back every two seconds
The answer to the command card of the built-in bash went only as a
bus message. When the message was lost, the call waited the full five
minutes and ended as not answered, although the person had clicked.
The network card stores its answer first and re-reads the card every
two seconds.
The command card now does the same, through one shared pair of
helpers: listenConsent subscribes before the card is stored, and
waitConsent wakes on the bus message or every two seconds and reads
the record, which holds the truth. The network card uses the same
helpers. HostExecDecide writes the decision to the card atomically
before it sends the message: the same answer twice is 204, the other
answer 409 already_answered, and an answered card is no longer shown
as waiting.
Assisted-by: Claude:claude-opus-5-5
* Load skills by name with a load_skill tool
A weaker model failed to read a skill at the path the index showed. The
chat now has a built-in load_skill tool that takes a skill's name and
returns the instructions of its SKILL.md, without the frontmatter.
The tool reads the same resolved skill set as the index: the nearest
skill of each name, the chat's switches, the add-ons the chat can use
now and the grants of its agent. It reads through the user's rights.
When the skill's folder holds other files, the answer starts with the
base directory and names up to 20 of them for read_file; a chat without
read_file gets the instructions only. A skill that needs commands, in a
chat that cannot run them, is refused with the reason. An unknown name
is an error that gives near names first and then every name the chat
has.
Every chat with at least one skill gets the tool, and a sub-agent gets
it with its parent's tools. It has no switch and Settings > Tools does
not list it.
The index of skills now lists names and descriptions only and tells the
model to call load_skill. The Sandbox sentence and the error for a
wrong skill path now point at load_skill too.
Assisted-by: Claude:claude-opus-5-5
* Tell the built-in skills to load other skills by name
localai-apps told the model to read the localai-workspace skill, and
writing-skills said a description is what the agent sees before it
opens the file. Both now speak of loading a skill with load_skill.
localai-workspace keeps the path of its schemas folder and says to read
it with read_file, since those files are not a skill.
A test checks that no built-in skill sends the model to another skill by
"read" or by the path of its SKILL.md.
Assisted-by: Claude:claude-opus-5-5
* Document load_skill and the skills index
The chats page now says that the prompt lists skills by name and
description, that load_skill returns a skill's instructions and names
the files of its folder, what it refuses and who gets it. It no longer
says the model reads a skill by its path. The add-ons page says how a
chat loads an add-on's skill, and the saved agents page says that the
skills grant limits load_skill too.
Assisted-by: Claude:claude-opus-5-5
* Add the proposal action spec to the tables-acceptance inventory
The spec that an editor cannot confirm a schema change as a write ran in
the suite but was never in the inventory, so the evidence manifest check
failed. CI did not reach that check before, because the burst spec
failed first.
Assisted-by: Claude:claude-opus-5-5
* Give a chat a goal that it works toward by itself
A chat can now hold one goal. While it is active, every turn's prompt
names it, the model gets goal_done and goal_blocked, and when a turn
ends by itself the frontend that ran it starts the next one with
"Continue toward the goal". The person's waiting messages still go
first, and a turn they start does not count as an automatic one.
The goal pauses when the person presses Stop, when a turn fails or is
cut short, and when the automatic turns or the model steps reach their
limits. Resume gives it fresh budgets. A sub-agent, and a chat that a
trigger, a connector, a job or a page started, has no goal and no goal
tools.
The goal lives with the chat record in the store (UpdateGoal on the
JSON and GORM stores, with contract tests and a column migration), so
every frontend reads the same one. The next turn goes through the same
turn claim as any other, so only one frontend continues it; a reaped
turn, or a JSON store that reloads after a crash, leaves the goal
paused instead of continued twice. The transcript keeps goal lines
(set, replaced, continue, paused, resumed, done, blocked, cleared) as
their own role, so an automatic message never reads as typed by the
person.
Assisted-by: Claude:claude-opus-5-5
* Add a store for the loops of the chats
A loop repeats a prompt in one chat, at an interval or when the model
asks for the next run. The new loops package keeps the loops, the
daily budget of runs of each owner, and which chats confirmed a loop
once. JSON and GORM stores share one contract suite. Add checks the
limits per chat and per owner in the same step as the insert, and
TakeRun never hands out the last run of a day twice, also across
frontends.
A user message that a loop sent now carries a loop mark (loop ID, run
number, self-paced or not). Both chat stores keep it. The model reads
a short line before the prompt that says the message repeats by
itself, and, for a self-paced loop, how to ask for the next run.
Assisted-by: Claude:claude-opus-5-5
* Add the file preview slot specs to the host-slots inventory
The file preview slot came with two Go specs in the host-slots package.
verify.sh found them in the run but not in the inventory.
Assisted-by: Claude:claude-opus-5-5
* Check the Settings page and the add-ons with the UI copy guard
The guard read the locale files only. The Settings page keeps its
English labels and descriptions in settingsSchema.js, and several
add-ons carry their own English bundle, so neither was checked.
The guard now also reads the label, description and note of every
setting and the label and hint of every group, and the English
bundles of the add-ons (an en.js module or an English JSON file).
The allow-list takes settings:<key>.<what> and addon:<id>:<key>
entries, each with its reason; the operator ones (the SearXNG JSON
format, the fine-tuning file types, the API route of a voice, the
schema of the Databases add-on) are listed.
What it found in the voice library is fixed in plain words: "model
YAML is not modified" and "Existing model YAML is not changed" now say
that the models' settings are not changed, in the voice add-on, the
Studio add-on and the host locales that have the strings (en, fr,
pt-BR). The two host allow-list entries for them go.
Assisted-by: Claude:claude-opus-5-5
* Add the update digest specs to the updates inventory
Two checker specs pin that an update names its candidate digest and
that a stale cached notice stays readable. The updates suite runs the
whole checker package, so the inventory needs both.
Assisted-by: Claude:claude-opus-5-5
* Draw the Add-ons search and state filter with the kit pieces
The Add-ons page had its own copy of the search box and the segmented
filter (ad-search, ad-seg), the same shapes and styles as the kit's
SearchField and Segmented that the Agents pages use. Use the kit
pieces and drop the copy; the narrow-screen rules now target the kit
classes inside the toolbar. The search keeps its test id.
Assisted-by: Claude:claude-opus-5-5
* Add the limits of a chat goal to the runtime settings
chats_goal_max_turns (1 to 200, default 20) is the most turns a chat
with a goal starts by itself, and chats_goal_max_steps (1 to 10000,
default 400) the most model steps of all the turns of one goal, before
the goal pauses. Both are read when a turn ends, so they apply without
a restart. They sit in Settings > Tools next to the step limit of one
turn, and the settings page documents them.
Assisted-by: Claude:claude-opus-5-5
* Add the goal routes of a chat to the API
GET, PUT and DELETE /api/chats/{id}/goal read, set and clear a chat's
goal, and POST /api/chats/{id}/goal/resume resumes a paused or blocked
one. They belong to the chat's owner like the other chat routes and
take the chats feature. A chat view carries the goal with the limits
that apply now, the transcript gives goal lines with goal_event and
goal_turn, and the event stream has a goal event so that other tabs
read the chat again.
The routes are in the route feature registry, the swagger file and the
chats entry of /api/instructions. They manage the person's own chat,
not the server, so they have no MCP tool, as the other chat routes. The
chats page documents goals, their states and limits, and how several
frontends share them.
Assisted-by: Claude:claude-opus-5-5
* Redesign the notification inbox
The inbox panel was a plain list: every row had the same weight, a
failed run looked like a finished one, the panel ran off the screen on
a phone, and a failed load showed nothing.
The panel now has a header with the unread count, a menu for Clear read
and Clear all, Mark all read, and an All / Unread / Failed filter. Items
are grouped under Today, Yesterday and Earlier. Each item shows what
sent it (trigger, connector or LocalAI) with a tinted icon, the title,
two lines of text or the reason of a failure, and when it happened, with
the exact time on hover. A click marks it read and opens its chat; a
connector that was switched off opens the connectors page. Mark read and
Delete show on hover or focus, and stay in view on a touch screen.
The panel has a loading skeleton, an error with Try again, an empty
state that says what shows up here, and a caught-up state. A new
notification slides in, is highlighted for a few seconds and is
announced once. The badge stops at 99+. Focus moves into the panel when
it opens; the arrow keys, Home, End and Delete work on the items, and
Esc returns focus to the button. On a phone the panel is a sheet over
the bottom of the screen with 44 px targets.
Assisted-by: Claude:claude-opus-5-5
* Explain the reads of the text index and the provider requests to gosec
gosec flagged three new lines. The text index reads files from the
directory the host gave it, with names it listed there. A search
provider sends its request to the endpoint fixed at construction, never
to a URL from a tool call. Mark the lines with the reason.
Assisted-by: Claude:claude-opus-5-5
* Keep the add-on details open when Escape closes a dialog
Escape in the restore dialog of the recovery panel also closed the
details pane around it. The pane handles Escape in React, and the
dialog listens on the document, so the pane ran first and saw no
preventDefault. The pane now registers as a modal layer and closes only
when it is the top layer, as the side panes do.
Assisted-by: Claude:claude-opus-5-5
* Add skill and agent messages, compact and clear to the chats API
A message can now carry a skill of the chat or one of the user's saved
agents. The server checks the name against the chat's resolved skill
set (switches, the agent's grants, shadowing, add-ons) or the saved
agents the chat may call, and refuses an unknown one with a 400 that
lists what the chat has. A skill message gives the model the text of
the skill's SKILL.md before the user's words, for that turn only; an
agent message asks the model to start the agent with spawn_agent. The
transcript keeps the words and the command (Entry.Invoked), not the
instructions. A regenerate or an edit runs the command again. Such a
message does not wait for a running turn: it answers 409.
POST /api/chats/{id}/compact runs the summary the loop makes when the
context fills up, through a new optional session.Compactor that the
loop harness implements with its own history compaction. The compacted
state is saved for the next turn and a "compacted" line with the sizes
goes in the transcript.
POST /api/chats/{id}/clear writes a "cleared" line and drops the saved
state. The model then sees nothing above the line: the rebuilt state
and the transcript summary start after the last such line, so a fork
from above it keeps what the model saw there. Export, import and the
Markdown export carry both lines.
Both calls take the turn claim, so no frontend starts a turn on the
chat meanwhile; messages sent during them start a turn after. No MCP
tool: they act on one chat of the caller and are not admin calls.
Assisted-by: Claude:claude-opus-5-5
* Give each network card spec its own settings
A check that a card spec starts can still wait on its card when the
next spec begins. It read the settings through a closure over a
variable that the next BeforeEach replaced, which the race detector
reported. Each service now reads the settings holder it was built with,
under a lock, and the specs change the settings through it.
Assisted-by: Claude:claude-opus-5-5
* Show the effective audit retention in the settings snapshot
The settings snapshot returned 0 for chats_egress_audit_days while the
setting was unset. Any page that saves the whole snapshot back, such as
the Traces settings strip, then posted 0 and the server rejected it with
"Invalid chats_egress_audit_days", so saving tracing settings failed on a
fresh install.
Report the retention that applies (30 days by default), the same way
chats_max_iterations does.
Assisted-by: Claude:claude-sonnet-5-5
* Run the Sandbox smoke test in a short scratch folder
The smoke test step was reached for the first time and both isolated
backends refused to start: their state folder under the checkout's
cache made the socket paths 126 and 132 bytes, over the 107 a unix
socket allows. Point SANDBOX_SMOKE_WORK at /tmp/sandbox-smoke, which
leaves room for the sockets.
Assisted-by: Claude:claude-opus-5-5
* Use paths from the home folder in the Sandbox command specs
bash now starts in the user's home folder, so the end-to-end specs that
read and wrote home/<name> found nothing. Use names relative to the
home folder, as the Sandbox specs already do.
Assisted-by: Claude:claude-opus-5-5
* Show the goal of a chat on the chat page
A bar above the composer shows the goal with its state: Pause (which
stops the running turn and so pauses the goal) while it works, Resume
when it paused or needs the person, and Clear. A second line says why
it paused, what the model needs, or what it did. The status line shows
"Goal · turn 3 of 20" while the goal is active, and the Continue button
of a paused turn stays away then, since the next turn starts by itself.
The transcript shows the goal lines as quiet rows: set, replaced,
continuing (with its turn), paused with the reason in plain words,
resumed, reached with the summary, blocked and cleared. A goal event,
also from another tab, reads the chat again, and a hidden live region
says each change to a screen reader once.
The strings are in all nine locales. Unit tests cover the helpers and
the thread rows; a Playwright spec covers the bar, the progress, the
lines, Pause, Resume, Clear and the event.
Assisted-by: Claude:claude-opus-5-5
* Run the loops of the chats
The loop host fires the loops that are due. A run sends the prompt to
the chat as a new message only when the chat is idle: a busy chat
skips the run, and an interval loop moves to its next slot instead of
piling missed runs up. A self-paced loop gets two tools while it is
active: schedule_next (clamped to the minimum interval and one day)
and loop_done. A run in which the model calls neither ends the loop
with a note. A loop ends after its run limit, pauses after three
failed runs in a row, and waits for the next UTC day when the owner's
daily budget of runs is used up.
One frontend fires the loops, the holder of a new advisory lock. A run
is claimed in the store first, and the turn claim keeps one turn per
chat, so a short overlap of two leaders starts no run twice. A new
leader finishes the runs the old one watched. Deleting a chat removes
its loops; a due loop whose chat is gone removes itself.
Five runtime settings bound the loops: chats_loop_min_interval,
chats_loop_max_runs, chats_loop_max_per_chat, chats_loop_max_per_user
and chats_loop_daily_runs.
Assisted-by: Claude:claude-opus-5-5
* Add the loops API of the chats
GET, POST and DELETE /api/chats/{id}/loops list, start and stop the
loops of a chat; PUT and DELETE /api/chats/{id}/loops/{loopId} pause,
resume, confirm or stop one. GET /api/chats/{id} carries the loops,
the transcript marks the user lines a loop sent with loop {id, run},
and a loops event tells every open tab that the list changed.
The routes are gated by the chats feature and documented in the
swagger spec and the API instructions. They act on the caller's own
chats only, so they get no assistant tool. The chats page gets a loop
section, with when to use a loop and when a trigger, and the cluster
behaviour; the runtime settings page lists the five loop settings.
Assisted-by: Claude:claude-opus-5-5
* Show the loops of a chat above the message box
Each loop of the open chat shows as one line over the message box,
such as "Loop · every 5 min · run 4 · next at 09:20", with Pause or
Resume and Stop. A loop that paused itself or ended says why; an
ended one has Dismiss. The first loop of a chat shows a card instead:
"Run this every 5 minutes until you stop it?" with Start and Cancel.
The server keeps the pending loop, so a reload shows the card again.
A message that a loop sent carries a small "Loop run 4" label. The
page reads the loops again on the loops event, so every open tab
follows. The answer to /loop list is ready for the slash command. The
strings are in all nine locales.
Assisted-by: Claude:claude-opus-5-5
* Build the chat slash menu on one command registry
The slash commands of Home and of a chat now come from one registry,
src/utils/slashCommands.js. A command has a kind (navigate, insert or
server), the surfaces that offer it, a needs check and, optionally, an
argument provider: args(query, ctx) returns the items the menu lists
when the box holds "/cmd <query>". Every existing command and test id
stays; the registry adds:
- one command per skill of the chat or the draft (/review), marked
built-in, add-on, shared or yours, and /skill <name>, which also
reaches a skill named like a command;
- /agent <name> <task>, which lists the saved agents, or offers to turn
them on for the chat when they are off;
- /model and /models with a model name, which switch at once;
- /compact and /clear, which show their progress in the status line
and leave a marker in the thread; the context meter drops;
- /attach and /attach clear, /help (a card that lists the commands)
and /about (a card with the model, its state, the tools, skills,
where commands run and the context size).
Tab completes a command or an argument, Enter runs it, and the first
row whose name starts with what was typed is selected first. A message
typed as a skill or agent command is sent with its marker, not as text;
the transcript shows the words and a chip. Home offers the same set
without the commands that need a thread.
The words are in all nine locales. The registry has unit tests, and a
Playwright spec covers the menu, the completion, the marker, the chip,
compact, clear and the cards with the routes stubbed. The docs page
gains a slash commands section.
Assisted-by: Claude:claude-opus-5-5
* Add the /goal command to the chat
/goal shows a card with the goal, its state, the automatic turns and
steps it used out of the limits, and when it was set. /goal <text>
sets a new goal at once, replacing the one the chat had without a
question, and the goal bar says what it replaced. /goal clear and
/goal resume act on the goal; typing "/goal " lists resume and clear
for the goal the chat has. None of these is sent as a message. In a
new chat, /goal <text> creates the chat first.
The command sits in the Context group of the shared command registry,
on the chat surface only, and not in a chat that a trigger or another
run started. A skill named goal is reached through /skill. Unit tests
cover the parser and the menu items; the Playwright spec covers the
card, setting, replacing, resuming and clearing from the box.
Assisted-by: Claude:claude-opus-5-5
* Add /loop to the chat
/loop [interval] <prompt> starts a loop in the chat; /loop list shows
the loops of the chat with the code that /loop stop takes, and
/loop stop [id] stops one or all. The command is in the slash
registry: after "/loop " the menu offers list and stop, and after
"stop " the loops of the chat.
A loop prompt that invokes a skill (/review the diff, or
/skill review the diff) runs as the person's skill command would: the
model gets the skill's text, and the transcript shows the words, the
skill's chip and the loop label. The server checks the skill when the
loop starts and at each run; any other command is refused with the
reason.
Assisted-by: Claude:claude-opus-5-5
* List the loop settings on the Settings page
The five limits of the loops of the chats show under Tools with the
other chat settings: the shortest interval, the runs per loop, the
loops per chat and per user, and the loop runs per day. All apply at
once.
Assisted-by: Claude:claude-opus-5-5
* Clear a sent Home draft when the page goes away too
Home clears the draft of a sent message when it closes, so the text
stays visible while the composer moves into the new chat. While the
chat page still loads, a reload or a closed tab could come first, and
the sent text came back on Home. Leaving the page now clears it too.
Assisted-by: Claude:claude-opus-5-5
* Check the close error of a previewed file
golangci-lint (errcheck) flagged the deferred Close of the file the
preview endpoint reads. Discard its error explicitly, as the other
studio file handlers do.
Assisted-by: Claude:claude-opus-5-5
* Explain the document read of the text index to gosec
gosec reported the read of a stored document text as a file inclusion
through a variable. The path is the configured index directory joined
with a hash that documentPath refuses when it holds a path separator
or a dot, so it cannot leave the documents folder.
Assisted-by: Claude:claude-opus-5-5
* Build the darwin/arm64 add-on packages on macos-15
GitHub retires the macos-14 runner image on 2026-11-02 and runs
scheduled brownouts until then: every darwin/arm64 package job failed
at once with the brownout notice. macos-15 is the Apple Silicon image
that replaces it.
Assisted-by: Claude:claude-opus-5-5
* Expect the chat's own load_skill tool in the agent prompt spec
The spec that keeps the agent engine's text out of the prompt also
required that no load_skill tool or mention reach the model. Chats now
offer their own load_skill tool and the skills index tells the model to
call it, so the spec failed. Keep the checks on the engine's own skill
and command text, and check that the chat's tool is offered.
Assisted-by: Claude:claude-opus-5-5
* Read the user's scripted tool call in the mock model
The chat prompt now names the chat's work folder, and the folder name
comes from the chat title, which holds a cut copy of the user message.
The mock model read the first ADDON_TOOL_CALL marker, the cut one in the
system prompt, and called the tool with broken arguments over and over.
Read the last marker, the one of the user message.
Assisted-by: Claude:claude-opus-5-5
* Keep the phone targets of the inbox at 44 pixels in every build
On a phone the inbox close button and the item actions must be 44
pixels wide. Their rule had the same weight as the kit's .dk-btn
min-width, so the order of the stylesheets decided. The single-chunk
coverage build puts the kit later and the close button shrank to 28
pixels. Scope both rules to their container so they win in any order.
Assisted-by: Claude:claude-opus-5-5
* Let the about-you spec accept the sidebar's own seen date
When the user has pages, the sidebar writes the date of its New label
under seen.new in user.json. In the whole suite other specs make pages
first, so the saved record held that date as well and the exact
comparison failed. Check that the uploaded seen record is kept.
Assisted-by: Claude:claude-opus-5-5
* Give the typed-while-working chat spec time for two turns
The spec waits up to 30 seconds for each step, but two turns of the
mock model with its tool steps take about 28 seconds on their own, so
the default 30 second test limit cut it under load. Set its limit to
90 seconds.
Assisted-by: Claude:claude-opus-5-5
* Scope the foundation spec's state query to the sheet header
The details pane now repeats the observed state under Health, so a
role query for "State: Off" inside the add-on sheet matched two
elements and failed in strict mode. The administrator-manages spec,
and the two specs after it, did not run.
Query the chip in the header (.ad-pane__meta), which is the state the
administrator sees next to the add-on's name, through one helper used
for all three assertions.
Assisted-by: Claude:claude-sonnet-5-5
* Wait for the page to read each apps answer in the registry polling spec
The spec moved the page clock as soon as the stand-in saw a request.
The registry sets its next timer only after it reads the answer, so on
a slow runner the clock could pass the due time of a timer that did not
exist yet, and the next read never came. With the CPU throttled ten
times, the old spec failed in 3 of 3 runs.
The spec now counts the answers the page has read and lets queued work
finish before it moves the clock. It pauses the clock before it leaves
the page, and it tries the pause again when a slow machine passed the
target time. The assertions do not change: one read at once, one read
per interval and no faster, and no read after the disable.
Assisted-by: Claude:claude-opus-5-5
* Close the egress test listener only after it accepts
The dial helper of the egress proxy specs closed its listener when it
returned, while the goroutine that accepts the connection could still
be waiting to run. On a loaded runner the listener sometimes closed
first. The kernel then reset the connection that was still in the
accept queue, the spec's write consumed the reset error, which the
spec ignores, and the next read saw EOF. The spec failed with
"unexpected EOF" before the proxy got the request.
Wait for Accept to return before the listener closes, then start the
proxy handler on the accepted connection.
Assisted-by: Claude:claude-opus-5-5
* Run host commands in a shell that reads no startup file
The host backend promises a fixed environment, but bash -lc reads
/etc/profile and the user's dotfiles. On macOS path_helper rewrote PATH
there, so the command did not get the fixed PATH. Turn the login shell
into 'bash --noprofile --norc -c' in the host backend only, and make the
test assert the exact fixed PATH.
Assisted-by: Claude:claude-sonnet-5-5
* Answer the command card in the platform turns and 400 a bad answer
The tool-list spec of the platform suite timed out: without accounts
the one person is an administrator, so the chat has the built-in bash
in ask mode. The tool list is sorted and bash is first, and the mock
model calls the first tool, so every turn waited five minutes for a
yes on the command card. The turn helper now refuses each card it
sees, as a person who clicks deny, and the turn goes on to its step
limit. bash stays in the checked list.
An administrator who posts a consent with no card id or an unknown
decision got 403 "only an administrator can allow commands". That is
a bad request, not a refusal: it now answers 400, as the swagger
already says, and the route check for administrators passes.
Assisted-by: Claude:claude-opus-5-5
* Let a chat switch off its shell, file, media, collection and sub-agent tools
The Tools menu of a chat switched only three groups of built-in tools:
web, saved_agents and triggers. A person could not keep a chat away
from the shell, the files, the helper models, the collections or the
sub-agents.
Add five groups to the registry of built-in tools, all on until the
chat switches them off:
- shell: bash, whoever serves it. The Sandbox add-on's bash and the
built-in bash on this machine both go, and the add-on list reports
commands switched_off. The row stays listed when nothing can run
commands, not usable, with the reason.
- files: the file tools and read_document.
- media: describe_image and transcribe_audio.
- collections: list_collections and search_collection, usable only
for a user with the collections feature.
- subagents: spawn_agent. list_agents and /agent follow it.
The registry gets two flags. External marks a tool that its own code
builds for the turn (with the wrappers it needs); the registry lists it
in its group and Hidden tells the builder what the chat switched off.
Covers makes a group's switch also hide the add-on tool that replaces
the built-in one. PUT .../builtins, the panel, the draft list and the
fallbacks use the existing group code.
A sub-agent follows the switches of its parent chat. The turn prompt
is built from the final tool list: no file tool names, no skill
writing hint, no work folder and no rules pointer when the chat cannot
use files, no describe_image when the helper is off, and a commands
block that says the person switched commands off.
Assisted-by: Claude:claude-opus-5-5
* Queue the edited chat message after the first step starts
The spec queued its message as soon as the chat showed a busy state.
The turn is busy before its first model request starts, and that
request takes every message that waits. With the chat's tools and
skills, the first request now starts a few hundred milliseconds after
the turn, so the message went with it, and the edit got 409 "already
delivered" (5 of 6 local runs).
Wait until the answer streams, as the Stop spec does: the request is
running then, and the queued message waits for the next step.
Assisted-by: Claude:claude-opus-5-5
* Give the reload chat spec a turn that outlasts the reload
The spec reloaded the page during a 3 second turn and expected the
page to show the turn still running. A reload of the coverage build
on a busy runner takes 4 to 5 seconds (measured with a 6x CPU
throttle), so the turn had ended before the page could show it. With
the 30 second turn the page always showed it running, so the resumed
state works and only the window was short.
Add MOCK_LONG_STREAM_MEDIUM to the mock model, a turn of about 10
seconds, and use it in the spec with a 60 second limit.
Assisted-by: Claude:claude-opus-5-5
* Wait for the chat page after the Home hand-off in the specs
Send on Home changes the address inside a view transition, and the
chat page mounts about 35 ms later. Specs that waited only for the
address acted on Home in between:
- The title spec pressed Ctrl+K at once, because the server names a
chat after its first message as it arrives, so the title poll ends
at once too. The key reached the page before the chat page listened,
and the menu did not open (3 of 3 presses at 0 ms failed, all at
300 ms and later passed).
- The Back spec went back before the chat page was drawn. Home was
never closed, so its draft of the sent message stayed in the box.
startChat now waits for the chat page, the title spec for the title
in the chat header, and the Back spec for the chat page before Back.
Assisted-by: Claude:claude-opus-5-5
* Accept a resolved user.json conflict in the quiet page specs
The sidebar notes in the seen record of user.json the first day it
labels Pages as new, and saves it from the version it read. All specs
share the test server's one user, so tabs of other workers save the
same record at the same time: one gets 409 and useSeen reads the file
again and saves once more. Three tabs that open /app/files at once
give 200, 409, 200. The quiet check of the shell pages counted the
409 as a problem.
Ignore a 409 on PUT /api/chats/wellknown/user in that check.
Assisted-by: Claude:claude-opus-5-5
* Check only the spec's own items in the Trash after the reload
After Empty Trash and a reload, the spec expected the whole Trash to
be empty. The Trash belongs to the test server's one user, and specs
of other workers move files to it meanwhile, so CI saw "In the Trash:
3 files, 356 B." after the reload.
The checks right after Empty Trash stay. After the reload, check that
the server's Trash, the Trash tab and Files do not hold the spec's
items, once each list has loaded.
Assisted-by: Claude:claude-opus-5-5
* Show the new tool groups in the chat Tools menu
The Tools menu lists the groups of built-in tools that the server
sends. The five new ones need words and an order, and the shell row
needs to say who runs the commands.
Name the groups in all nine locales (Run commands, Your files, Images
and audio, Collections, Sub-agents) and order the rows by what they
reach: the shell and the files first, the triggers last. The shell row
carries a chip with the provider (Sandbox or This machine) and a line
under it that says the commands run isolated, or a warning when they
run on this machine. When nothing can run commands the row is disabled
with the words of the commands notice. A switched-off sub-agents group
also turns /agent off in the composer.
The specs cover the rows, the off and on requests, the bulk actions,
the provider chip, the disabled reasons, the draft on Home and a real
server round trip.
Assisted-by: Claude:claude-opus-5-5
* Document the groups of built-in tools a chat can switch off
The Tools menu section described only the web, triggers and saved
agents groups. List every group with its tools, say which groups the
server may leave out and why a row can be not usable, how Run commands
covers both providers, what the model is told when a group is off, and
that a sub-agent follows its parent's switches.
Assisted-by: Claude:claude-opus-5-5
* Fix the host command consent spec for the 400 contract
The consent route answers 400 for a body without a card id or with a
decision other than once or deny, whoever asks. The spec still expected
403 for always from an administrator. Cover the malformed answers for
an administrator and a user, each in their own chat, keep another
user's chat at 404, and say it in the chats guide.
Assisted-by: Claude:claude-sonnet-5-5
* Find the web tool group by name in the route spec
The tools panel lists the shell group before the web group, so the
spec that read the first entry compared the wrong group. Look the
group up by name, which does not depend on the registry order.
Assisted-by: Claude:claude-sonnet-5-5
* Give Skills its own sidebar item
Skills work in every chat, but the sidebar filed the page under Agents
and only agent skill chips linked to it. Add a Skills item right after
Memory, show it in the legacy list too, and stop lighting up Agents on
the skills pages.
Assisted-by: Claude:claude-sonnet-5-5
* Add Manage skills to the skills picker and a /skills command
The picker now ends with a Manage skills link, and /skills opens the
Skills page from the composer on Home and in a chat. The picker keeps
the height of its status line, so a refused change does not move the
rows. Words are added in all nine locales.
Assisted-by: Claude:claude-sonnet-5-5
* Say on the Skills page that every chat can use skills
The page subtitle named only agents as users of skills. Say that all
chats use them, translate it, and document where to manage skills: the
sidebar item, the picker link and /skills.
Assisted-by: Claude:claude-sonnet-5-5
* Keep Home on screen while its widgets load their strings
A Home widget reads the pages strings, and i18next loads a namespace
the first time a component asks for it, by suspending. Home had no
Suspense boundary of its own, so the route's boundary hid all of Home
(display: none) and showed the route fallback until the strings came.
A widget shows only after home.json is read, so this happened after
the composer was already drawn. Focus given to the composer in that
window went nowhere, and a key typed then was lost. The slash menu
spec failed in CI this way when another spec had written widgets to
the shared home.json: the box was empty and the body had the focus.
With widgets in home.json the composer was hidden right after it
appeared on 5 of 5 loads, and the spec failed 1 or 2 times in 5.
The examples and the widgets now have their own boundary with no
fallback, so only they wait. A new spec holds the pages strings while
it types in the composer; it fails without the change (the composer
never takes the focus) and passes with it.
Assisted-by: Claude:claude-opus-5-5
* Send the stubbed file change after the Files page read the folder
The change feed spec answered the first stream request with the change
at once. The app shell (the appearance sync) opens the shared stream
when the app starts, before the Files page chunk has loaded, so on a
slow runner the change reached a tab where nobody watched the folder
yet and was dropped. The page then never read the folder again, and
the spec failed on all three CI attempts. A CPU throttle of 8 makes it
fail 3 of 3 times locally.
On a real server the page's own first read would already show such a
change. The stub now holds the change until the page has read the
folder; the page subscribes in the same commit as that read, so the
change arrives with a subscriber in place. The resume check is kept.
Assisted-by: Claude:claude-opus-5-5
* Give the oversized 3D input as a sparse file, not a buffer
The spec built a 32 MiB buffer and passed it to setInputFiles.
Playwright sends a buffer to the browser over the protocol: that step
alone took 5.5 s on an idle machine, and on the loaded CI runner the
test hit the 30 s timeout on all three attempts.
It now writes a sparse file of the same size to the test output folder
and passes its path, which the local browser opens as is. The page
still checks the size before it reads the file, so the toast and the
no-request assertion are unchanged. The step takes milliseconds.
Assisted-by: Claude:claude-opus-5-5
* Check the tool line, not the tool name, in the tool call journey
The spec looked for /list_files/ in the transcript. The user's own
message holds that name, so the check could pass before any tool ran.
Once the model's call streamed, a second element matched, and the
strict locator failed (2 matches on the first attempt, 5 on the
retry).
It now waits for the activity line that the tool run adds ("Listed
/"), which only shows after the tool ran.
Assisted-by: Claude:claude-opus-5-5
* Measure the proposal card and its neighbours in one frame
The layout spec read the boxes of the message text, the card and the
message actions with three boundingBox calls. They are viewport
positions taken in three frames, and the transcript scrolls to its end
while the card fills in. On the phone the actions were read after a
scroll and came out 156 px above the bottom of the card.
The three boxes now come from one page.evaluate, so they are read in
the same frame. The order assertions are unchanged.
Assisted-by: Claude:claude-opus-5-5
* Read the page fixture time when the page is served
The task board fixture of the discovery and Pages specs set its
change time once, when the file loaded. A worker loads it once and
keeps running specs, so a spec that ran two minutes later read
"Changed 2 minutes ago by the assistant" where it expects "just now".
The change time is now a getter, read when the fixture is serialized
into a response.
Assisted-by: Claude:claude-opus-5-5
* Check a background process without /proc in the host test
The alive helper read /proc/<pid>/stat, which macOS does not have, so it
reported every process as dead and the spec failed there with "did not
survive the suspend". Use signal 0 for existence and ps (or /proc where
present) to tell a zombie from a live process.
Assisted-by: Claude:claude-sonnet-5-5
* Keep the composer switch menus at their own height
The base rule of the dropdown menu caps it at 320px and lets it scroll.
The switch menus set their own height with a rule of the same weight in a
lazy sheet, so which one won depended on the order the sheets loaded in.
When the base rule won, the picker showed a few rows, and a click on a row
at the edge of the list scrolled the list by a few pixels.
Double the class on the switch menu rules so they win in any order, and
add a test that loads the base rule last.
Assisted-by: Claude:claude-sonnet-5-5
* Show what applies in the chat Network switch
The Network row highlighted the choice stored with the chat, so a
chat that asked for All public sites under an administrator limit of
Off showed All highlighted although it was disabled and not in effect.
Highlight the effective choice, the lower of the chat's choice and the
limit, which the server already computes. When the chat asks for more
than the limit, say so in one line instead of highlighting it. Add a
line that the limit is set by the administrator, and tell the host
network line that limits need the Sandbox add-on.
Assisted-by: Claude:claude-sonnet-5-5
* Design the host components that add-ons reference
The host UI names the Sandbox and the Web add-ons to place their server
settings, and the chat's Network row links to the global Settings page.
Describe a closed registry of host components that an add-on references
by ID in its manifest, entitled by the existing host operation grants
and tool shadows, so the controls live on the add-on's own page and a
second add-on can reuse them with no host change.
Assisted-by: Claude:claude-opus-5-5
* Let an add-on ask for host controls in its manifest
An add-on had no way to say that a host-owned control belongs on its
page, so the host UI placed those controls by add-on name. Add
ui.controls, a list of IDs from a closed registry of host controls:
network-policy (the network of the commands an add-on runs) and
tool-providers (the rows of the built-in tools it replaces).
The host decides which controls it draws: EntitledControls keeps a
control only when the add-on holds its host operation grant and
replaces its built-in tool. The field is optional, so every existing
manifest stays valid.
Assisted-by: Claude:claude-opus-5-5
* List the host controls an add-on is entitled to
The apps list now carries ui.controls with the host controls the
manifest declares and the host setting entitles the add-on to: the
manager passes each add-on's host operation grants to the list.
GET /api/chats/{id}/egress gains addon, the add-on that runs the chat's
commands when it has the network-policy control, so the chat can draw
its Network row from that add-on and link to its settings. applies and
applies_reason keep their meaning: the policy holds with or without the
control. The model and the 400 error no longer send an administrator to
Settings > Tools for the limit.
Assisted-by: Claude:claude-opus-5-5
* Draw the host controls on the add-on's own page
The host UI placed the Sandbox network and the tool rows by add-on name
(SERVER_SETTINGS), showed a read-only summary on the add-on's Settings
tab, and linked the chat's Network row to the global Settings page.
Add a registry of host controls keyed by the IDs of the contract. The
Settings tab of an add-on draws, below its own sections, the controls
the apps list gives it, in one form with the pending bar of the
Settings page: network-policy reuses the network section and its
setting rows, tool-providers the rows of the built-in tools the add-on
replaces. The chat draws its Network row only when the server names an
add-on with network-policy, and the administrator link opens that
add-on's tab. The row itself is unchanged.
The Settings page keeps the settings of features that exist without an
add-on, points to the add-on where one serves them, and no longer
lists the network settings. Its old address opens the add-on's tab.
Assisted-by: Claude:claude-opus-5-5
* Ask for the host controls in the Sandbox and Web manifests
The Sandbox asks for network-policy and tool-providers, so the network
of its commands and the bash row are drawn on its own Settings tab and
its chats get the Network row. The Web add-on asks for tool-providers
only: it holds no egress grant. The manifest tests check that the
default grants entitle each add-on to what it asks for.
The proxy's refusal now sends the model to the Sandbox Settings tab
instead of Settings > Tools.
Assisted-by: Claude:claude-opus-5-5
* Keep the button of the network pointer row at its size
The row used the wide layout of the tool rows, so its Open settings
button stretched across the page. Use the plain setting row, which
puts the button on the right like the other controls.
Assisted-by: Claude:claude-opus-5-5
* Fix the add-on contract inventory and the apps Swagger fixture
The contracts case inventory missed the six host control specs, so
verify.sh rejected the Ginkgo report. List them.
The Swagger spec test never saw `controls` in the emitted UIView JSON
because its fixture left the omitempty field empty. Set it there.
Assisted-by: Claude:claude-sonnet-5-5
* Add the host controls spec to the apps inventory
The apps suite ran one more spec than scripts/addons/suites.json
listed, so verify.sh rejected the apps report as an inventory
mismatch. List the new spec in the report inventory and in the
suite's aggregate case list. Every other Ginkgo suite was re-run
and matches its inventory.
Assisted-by: Claude:claude-sonnet-5-5
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
|
||
|
mudler-agentandEttore Di Giacinto
|
832dff3c74 |
fix(model-gallery): point pinned LFM3-Audio entries at main (#12994)
The checksum bot rewrote the sha256 of the LFM3-Audio mmproj file to the checksum of the newest upstream version. The uri of every LFM3-Audio entry stayed pinned to an older commit, so the checksum and the pinned file disagreed and installs failed the checksum check. A commit-pinned entry cannot follow the bot's checksum updates. Point the uri of all files of these entries at main, so the uri and the bot's checksum agree and the bot keeps them current. The two weight files are identical in both revisions; only the Q8_0 projector changed. The entry test now expects the main uri and the new projector checksum. Assisted-by: Claude:claude-sonnet-5-5 Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-botandmudler
|
bec9638242 |
chore: ⬆️ Update ggml-org/llama.cpp to 5047748f114564fcf271e6a648f60e2d2eae6264 (#12949)
⬆️ Update ggml-org/llama.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
9386a3a7e8 |
chore: ⬆️ Update ggml-org/whisper.cpp to 482279b8a18855ba741d7b88c30fcb3874691f59 (#12950)
⬆️ Update ggml-org/whisper.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
dependabot[bot]
|
954a34295a |
chore(deps): bump torch from 2.14.0+xpu to 2.15.0+xpu in /backend/python/common/template (#12960)
chore(deps): bump torch in /backend/python/common/template Bumps torch from 2.14.0+xpu to 2.15.0+xpu. --- updated-dependencies: - dependency-name: torch dependency-version: 2.15.0+xpu dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
1faf756f4b |
chore(deps): bump grpcio from 1.84.0 to 1.85.0 in /backend/python/common/template (#12961)
chore(deps): bump grpcio in /backend/python/common/template Bumps [grpcio](https://github.com/grpc/grpc) from 1.84.0 to 1.85.0. - [Release notes](https://github.com/grpc/grpc/releases) - [Commits](https://github.com/grpc/grpc/compare/v1.84.0...v1.85.0) --- updated-dependencies: - dependency-name: grpcio dependency-version: 1.85.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
43a9c162a5 |
chore(deps): bump docs/themes/hugo-theme-relearn from 657eb0c to 53bcfee (#12962)
chore(deps): bump docs/themes/hugo-theme-relearn Bumps [docs/themes/hugo-theme-relearn](https://github.com/McShelby/hugo-theme-relearn) from `657eb0c` to `53bcfee`. - [Release notes](https://github.com/McShelby/hugo-theme-relearn/releases) - [Commits](https://github.com/McShelby/hugo-theme-relearn/compare/657eb0c78b6b304decd359529cc1ad815f57767c...53bcfee6cae7631003a0ba06a1c9281351269dc8) --- updated-dependencies: - dependency-name: docs/themes/hugo-theme-relearn dependency-version: 53bcfee6cae7631003a0ba06a1c9281351269dc8 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
244246c38c |
chore: ⬆️ Update CrispStrobe/CrispASR to 3345aab0a87241e830780ca2d7d6a729276fa5a7 (#12986)
⬆️ Update CrispStrobe/CrispASR Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
ec75014929 |
chore(model-gallery): ⬆️ update checksum (#12989)
⬆️ Checksum updates in the model library Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
f11931e178 |
chore: ⬆️ Update 0xShug0/audio.cpp to 350670db7ff64864e5c66c794e6c4b0d3dc98d2f (#12987)
⬆️ Update 0xShug0/audio.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
cfafec79ca |
chore: ⬆️ Update antirez/ds4 to c97964b8aa77034a8e17afb10ca7a686c40ab511 (#12988)
⬆️ Update antirez/ds4 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
f8ab72d267 |
chore: ⬆️ Update mudler/parakeet.cpp to 2eac2e77d9c7ef8899f7a86a46e693ac5ac1a5cd (#12985)
⬆️ Update mudler/parakeet.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
ccd0657cb0 |
chore: ⬆️ Update mudler/vllm.cpp to 8d1eb0daac798d28e08e4b53af5ece6119d16816 (#12983)
⬆️ Update mudler/vllm.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
63a4fb83ac |
chore: ⬆️ Update ikawrakow/ik_llama.cpp to 13cffc006b82e03e4d89cbb8c7327f0255ae3d18 (#12984)
⬆️ Update ikawrakow/ik_llama.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
4f329aa40d |
chore: ⬆️ Update NVIDIA/NeMo-Speech.cpp to 20999cc921d2b14aa58708f61756a8b2e787ba20 (#12982)
⬆️ Update NVIDIA/NeMo-Speech.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
ce85d29054 |
chore: ⬆️ Update localai-org/privacy-filter.cpp to 3b87bf8793f440a20fe8b2dcd00f6f27e3bfd69d (#12975)
⬆️ Update localai-org/privacy-filter.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
cd419a8442 |
chore: ⬆️ Update PrismML-Eng/llama.cpp to 9d9efeadbdd1d77458d98717a96fbfa3a016e159 (#12972)
⬆️ Update PrismML-Eng/llama.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
44d29bf9a3 |
chore: ⬆️ Update ikawrakow/ik_llama.cpp to 06d729c5aff5aad77c408ad97efb2048475414c4 (#12976)
⬆️ Update ikawrakow/ik_llama.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
ilsawaandEttore Di Giacinto
|
4c6ce725ee |
[llama-cpp] keep an existing image data URL (#12921)
fix(llama-cpp): keep an existing image data URL append_media_parts always prepended data:image/jpeg;base64,. A part that already started with data: became a doubled prefix and the image failed to load. Leave an existing data: URL unchanged. Assisted-by: Grok:grok-4.7 Signed-off-by: Igor Savitskiy <ils2000@mail.ru> Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
3f7c3f33f2 |
chore: ⬆️ Update 0xShug0/audio.cpp to aeb7bcf1a44c2ebe3308e9f206a9225a69740a92 (#12951)
⬆️ Update 0xShug0/audio.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
b01d331034 |
chore: ⬆️ Update leejet/stable-diffusion.cpp to e239e4b0fd3de49834cec67031a44796ca465abc (#12945)
⬆️ Update leejet/stable-diffusion.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
90bdaad4a5 |
chore: ⬆️ Update NVIDIA/NeMo-Speech.cpp to c540e21167017d2e34444f2fc2d70649065dc500 (#12953)
⬆️ Update NVIDIA/NeMo-Speech.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
799f2dae01 |
chore: ⬆️ Update mudler/vllm.cpp to 3c69df47a4fc450ea507b29f2c591b60458c0866 (#12971)
⬆️ Update mudler/vllm.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
e8f9b2fc52 |
chore: ⬆️ Update CrispStrobe/CrispASR to ecc5a8e7c97aca0835dc4e2864873a68bfc1ab67 (#12974)
⬆️ Update CrispStrobe/CrispASR Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandlocalai-org-maint-bot
|
1ac2ecb2df |
fix(ui): use token usage for chat throughput (#12923)
Streaming chunks can contain multiple tokens. Use reported completion totals for the final end-to-end average and label live rates as estimates. Count each client tool turn once and include reasoning in the total. Assisted-by: Codex:gpt-6 Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> |
||
|
dependabot[bot]
|
f3f9a8601b |
chore(deps): bump grpcio from 1.84.0 to 1.85.0 in /backend/python/rerankers (#12963)
chore(deps): bump grpcio in /backend/python/rerankers Bumps [grpcio](https://github.com/grpc/grpc) from 1.84.0 to 1.85.0. - [Release notes](https://github.com/grpc/grpc/releases) - [Commits](https://github.com/grpc/grpc/compare/v1.84.0...v1.85.0) --- updated-dependencies: - dependency-name: grpcio dependency-version: 1.85.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
0e3e2a6ce1 |
chore(model-gallery): ⬆️ update checksum (#12977)
⬆️ Checksum updates in the model library Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
4fcf230f9e |
chore: ⬆️ Update mudler/parakeet.cpp to 4fbf4d92b49a44bc4a24bc373ca12cf252919ca6 (#12970)
⬆️ Update mudler/parakeet.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
bc09b4ab6a |
chore: ⬆️ Update localai-org/kimodo.cpp to 75ed52d2d3ec767a9bac79d11060b65d99f61a89 (#12973)
⬆️ Update localai-org/kimodo.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
mudler-agentandEttore Di Giacinto
|
2cadf319a0 |
feat(realtime): LFM3-Audio speech-to-speech on llama.cpp, remove liquid-audio (#12958)
* chore(backends): remove the liquid-audio Python backend The backend targets liquid-audio, which loads LFM2 checkpoints only. Remove its directory, the gallery file and the LFM2.5-Audio model entries, the library and generated backend gallery entries, the backend matrix and CI job, the Makefile targets and the golden test fixtures that listed it. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(config): drop liquid-audio from capabilities, importers and UI With the liquid-audio backend gone, remove its capability entry, its importer, its artifact-gate entry and the realtime_audio heuristic that only matched that backend. realtime_audio is now set only through known_usecases. Update the Talk and fine-tuning pages, the docs and the swagger text that named the backend. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(llama-cpp): bump llama.cpp to e486a4b5 Move the llama.cpp pin to e486a4b559617dc88bf79978996f6c670b6bbc9d, the master head of 2026-10-28. It contains the LFM3-Audio support (ggml-org/llama.cpp#31257) and the realtime server split (ggml-org/llama.cpp#31269). The score task patch is regenerated against the new pin. It applies with git apply and with patch -p1, with no offset and no fuzz. Its hunks in server-task.h and server-context.cpp were rebased by hand next to SERVER_TASK_TYPE_REALTIME and the control dispatch. The C++ backend was built on CPU for this change: the llama, mtmd and server-context libraries and the grpc-server binary link. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): compile server-realtime.cpp into grpc-server server-context.cpp includes server-realtime.h, and its realtime slot type is defined in server-realtime.cpp. Upstream compiles that file into the server library. The gRPC server is one translation unit, so it must include the file too, or the link fails. The include is guarded by __has_include. Forks that predate the realtime split do not ship the file, so they keep building as before. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(llama-cpp): probe realtime and TTS voice support in prepare.sh prepare.sh writes llama_compat.h from probes on the checkout. Add two: LOCALAI_HAS_REALTIME (SERVER_TASK_TYPE_REALTIME in server-task.h) and LOCALAI_HAS_TTS_VOICE (set_voice in mtmd-helper.h). The probes let the gRPC server choose the realtime and voice code paths per checkout, so forks without them keep compiling. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): advertise speech recognition and speech-to-speech llama-cpp now registers the AudioTranscription and AudioToAudioStream RPCs, with the transcript and realtime_audio usecases. The default usecase stays chat, so a plain GGUF is still a chat model. The backend implements the RPCs in a follow-up change; until then the calls return UNIMPLEMENTED. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(config): add pipeline.any_to_any for speech-to-speech models A realtime_audio model that serves the whole speech loop itself sets pipeline.any_to_any. The flag is off by default, so chained pipelines and existing realtime_audio configs are unchanged. The realtime_audio heuristic stays explicit: a llama-cpp model is never a Talk-page model unless known_usecases says so. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(config): read the audio encoder and speech-to-speech projector flags Add HasAudioEncoderProjector for clip.has_audio_encoder and HasSpeechToSpeechProjector, which requires the LFM3 audio encoder family together with the gen-audio decoder. Qwen3-TTS sets both encoder flags too, so the flags alone cannot tell it apart from LFM3. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(importer): import LFM3-Audio projectors as speech-to-speech An mmproj that carries the LFM3 audio encoder and the gen-audio decoder now imports with known_usecases realtime_audio, chat, transcript and tts, pipeline.any_to_any: true, the parallel:2 option and the tokenizer template. Other gen-audio projectors keep the TTS-only result, Qwen3-TTS included. The probe runs against a local HTTP server in the tests. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): named-voice TTS and the LFM3 ASR prompt Models without a speaker reference (LFM3-Audio) take a named voice through their "tts" chat template instead of a reference file. The request is validated with needs_reference=false, the prompt is rendered with common_chat_get_tts_prompt(), and the voice is set on the gen-audio input. Qwen3-TTS Base keeps the reference-file rule. The branch is guarded by LOCALAI_HAS_TTS_VOICE, so forks without named voices keep the old behaviour. Transcription takes its prompt from common_chat_get_asr_prompt(). LFM3 gets a system turn that names the language and an empty user turn. Qwen3-ASR's preset is unchanged: fixed user text, no system turn, and a caller prompt is still appended after a newline. The preset call is guarded by a new LOCALAI_HAS_ASR_PRESET probe. The pure parts sit in tts_request_options.h and transcription_prompt.h, with standalone tests run by backend/cpp/run-unit-tests.sh. Add the test-extra-backend-llama-cpp-lfm3 target. It runs the tts, ttsstream and transcription specs against LFM3-Audio GGUF weights. It is a manual run and is not wired into CI. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): mock AudioToAudioStream and pending realtime S2S specs The mock backend serves AudioToAudioStream deterministically: session created, commit with RMS, a tool call on the first turn with tools, then transcript pieces and paced 24 kHz tone frames, with response.cancel ending a reply as cancelled. The realtime S2S specs cover the mock and real-weights variants. They skip until the /v1/realtime S2S path lands, so the mock suite stays green in CI. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): speech-to-speech AudioToAudioStream on the realtime slot Implement AudioToAudioStream on upstream's realtime slot, so a model that answers after the audio (LFM3-Audio) can hold a conversation over the existing duplex RPC. The proto schema does not change: tools travel in the config, tool calls leave as function_call_arguments.done and tool results arrive as a function_call_output item. realtime_session.h keeps what needs no model: the turn text derived from the chat template (with a tool turn), the conversation state, a filter that keeps <tool_call> blocks out of the transcript and drops audio produced inside them, and the translation of slot results into events. grpc-server.cpp wires it to the slot with control tasks and parses calls with the parser the chat template selects. The unit test uses a fake renderer. A ctest-only check renders the real LFM3 template and compares the slot text with the full render. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add the LFM3-Audio-2B entries Two llama-cpp entries for LiquidAI/LFM3-Audio-2B-GGUF, the Q4_K_M default (lfm3-audio-2b) and the Q8_0 build (lfm3-audio-2b-q8), both with the Q8_0 audio projector. Each declares pipeline.any_to_any and the realtime_audio, chat, transcript and tts usecases the importer writes for an LFM3 projector, with parallel:2 and the tokenizer template. The Q4 entry lists the Q8 one as a variant. File sizes and sha256 come from the Hugging Face API for the pinned repo revision. Speaker reference files and the VAD model are not part of the entries: VAD stays optional. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: describe LFM3-Audio speech-to-speech and its gallery entries Adds a speech-to-speech section to the Realtime API page (any_to_any configuration, VAD versus manual commit, tools, transcripts, barge-in, limits), a pipeline.any_to_any row, a llama.cpp speech-to-speech row in the compatibility table, and LFM3-Audio mentions on the TTS and ASR pages. The feature is marked experimental: the behaviour follows the backend session contract and has not been run with real weights. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(realtime): run any-to-any models on AudioToAudioStream A realtime_audio model with pipeline.any_to_any now opens one duplex stream to the backend and keeps it for the session, replacing the "any-to-any models not implemented" branch. The chained pipeline is unchanged. The endpoint forwards input audio (with a pre-roll from speech start when a VAD model is set, otherwise every chunk and a client commit), sends response.create, and turns the backend's events into the usual realtime events: audio and transcript deltas, tool calls through the existing function_call items, and a single response.done. A function call output is forwarded before the next response. Barge-in sends response.cancel ahead of the new speech and stops forwarding audio at once. The user's speech is transcribed on the side and reported on the user item; a failure there does not touch the response. Server VAD is on only when pipeline.vad is set. Classifier mode, compaction, streaming.llm, voice gating, sound detection and semantic_vad are rejected for these models. The mock backend now matches the llama.cpp contract (session.updated, the cancel error when idle) and its VAD reports the loud stretches of the audio, so the S2S e2e specs run a VAD turn. The specs are enabled. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): add the real-weights realtime S2S variant and the gRPC s2s spec The realtime S2S specs now check reply audio on real models: duration, loudness, clipping, and keywords found in the model transcript or in a separate ASR of the output audio. run-realtime-lfm3.sh stages the llama-cpp and sherpa-onnx backends, the LFM3-Audio files and the sherpa VAD, ASR and TTS models, then runs the real variant. It is manual and never part of CI: `make test-realtime` still filters out real-models. The backend suite gains an s2s capability (spoken turn, tool call, tool result, spoken answer over AudioToAudioStream) and optional TTS duration and RMS bounds plus a transcript keyword check, all enabled for test-extra-backend-llama-cpp-lfm3. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): expect the idle-cancel error from the mock S2S The mock now answers a response.cancel with no reply running with response_cancel_not_active, as the llama.cpp backend does. The cancel spec still treated that cancel as a no-op and read the error as extra data after the stream ended. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(talk): show any-to-any models as one speech-to-speech model /api/pipeline-models lists a pipeline.any_to_any model with any_to_any set and its optional VAD. A realtime_audio model that is not any-to-any is listed only when it has the four pipeline slots, since the session takes the chained path for it. The Talk page shows a speech-to-speech model as one model: no four-slot view, the VAD slot only when a VAD model is set, and the female and male voices offered. Without a VAD model the server does not detect turns, so the page offers Done speaking, which sends input_audio_buffer.commit and then response.create. Text-only replies stream through response.output_text.delta and .done. self_contained stays in the response for clients built before any_to_any. Assisted-by: Claude:claude-haiku-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): refuse a llama-cpp realtime model without pipeline.any_to_any llama.cpp has no VAD, transcription or TTS RPCs, so filling the empty pipeline slots with the model name failed later with UNIMPLEMENTED. Fail at session start and name the missing setting instead. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): hold manual-commit S2S audio until the commit In manual-commit mode the endpoint streamed every appended chunk to the backend, so input_audio_buffer.clear only emptied the Go buffer while the model kept the audio, and idle audio filled its context between turns. Keep the audio in the input buffer and send it, behind pending text and tool-result items, when the client commits or asks for a response. Frames and controls now go through one ordered write queue, so no gRPC send happens under the audio buffer lock. A tool result created while a VAD turn is open is refused with a clear error. When the stream ends outside a response the session reports it and closes, and a bare EOF from Send reports the status that Recv holds. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): drop the S2S readiness gate and stale comments The gate was always true. Some comments referred to work items that readers of the repository cannot see. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(realtime): add the liquid-audio migration note and the clear semantics Say that the Python liquid-audio backend is gone, which gallery entries replace it and that LFM2.5-Audio has no replacement. Describe what input_audio_buffer.clear drops for speech-to-speech sessions now that manual-commit audio is held back until the commit. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(talk): do not promise turn detection when no VAD model is set The idle note said the server notices when you stop talking. A speech-to-speech model without a VAD model needs Done speaking, so the note now says that. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): make the mock S2S backend keep the llama-cpp session limits The mock accepted what the real backend refuses and hid two endpoint bugs. It now treats output_modalities from the config as a fixed session mode and picks the default per response from the tools set at that time. A fixed text plus audio mode with tools reads the tool call out loud. After the first commit or message, session.update of instructions, voice or tools is an error that does not belong to a response, committing an empty buffer is an error, and response.create while a reply runs is refused. Errors of a refused response.create carry response_scoped in their meta. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): leave the S2S output mode to the backend unless the client sets one The stream config sent text plus audio when the session had no tools yet. The llama-cpp backend keeps a mode from the config as a fixed session mode, and tools added later by session.update never cleared it. The model then read tool calls out loud. Send output_modalities only when the client chose them. The backend picks the default of each response from the tools it has when the response starts. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(llama-cpp): mark the realtime errors that answer a response.create The error event now carries response_scoped: true when it ends the request for a response (a refused response.create, a failed commit made by it, or a stream failure). Other errors, such as a refused session.update or a commit of an empty buffer, are not about a response and one that runs keeps generating. The client needs the difference to know whether a response.done is still coming. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): end an S2S response only on the errors that belong to it Every backend error ended the response in flight, although the backend also reports errors about other things (a refused session.update, a commit of an empty buffer) and goes on generating. The rest of the reply was lost and the next response.create met either a refusal or the leftover events of the old reply. End the response only on an error marked response_scoped, forward the others to the client and keep waiting for response.done. Drop the events buffered by an earlier response when a new one starts, and skip the commit of a turn that forwarded no audio. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): run the S2S input transcription detached from the commit slot The commit waited for the side transcription of the user audio while it held the commit slot, and response.create waits for that slot. The ASR latency was added before every reply, and on a backend with few slots a busy ASR task could stall the reply. The model already has the audio, so run the transcription detached under the session context and release the slot at once. The transcript is still reported on the user item. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(realtime): do not pin the place of the input transcript in the S2S order The transcript of the user audio is reported on the side and may land before or after the first response events. Check the order of the response events without it. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): stop an S2S reader leak and a cancel that waits on the write queue The reader waited on a channel it closes itself when the events buffer was full, so it outlived the session after a turn ended on an error or timed out. It now also stops when the session closes. Stopping a response sent the cancel through the write queue and waited for it before the grace timer started. With backpressure on the stream the wait never ended and nothing read the events meanwhile. The timer now starts first and the cancel is only queued. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(realtime): let a chained pipeline run without a VAD model A pipeline that names no pipeline.vad failed to build, because the VAD slot was always loaded, and a new session still started with server_vad. The slot is now optional for chained pipelines as it already is for any-to-any models: the session starts with turn detection off and the client commits each turn. Turning server_vad on for such a session is refused, and a switch to a pipeline without a VAD drops turn detection. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add LFM3-Audio VAD and chained-pipeline entries Add lfm3-audio-2b-vad and -q8-vad (any-to-any with silero-vad-ggml as the turn detector), and lfm3-audio-2b-pipeline, -pipeline-vad and the -q8 forms (the model chained through its own transcription, chat and speech slots, with and without the VAD). The gallery has no dependencies between models, so the VAD entries state that silero-vad-ggml must be installed first. Drop the variants list from lfm3-audio-2b: it let the installer pick the larger Q8 build on any host with enough memory, so a plain install of the Q4 entry did not get Q4. All entries are now independent. Tests cover the entry shapes, the single-build install and their acceptance by the realtime endpoint. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): run the realtime specs for any-to-any and chained shapes The realtime specs covered one any-to-any model with a VAD. They now run for any-to-any and for a chained pipeline, each with and without a VAD, on the mock backend in CI. VAD-less shapes also check that turn detection starts off and that server_vad is refused. The manual real-weights script writes one config per shape that mirrors the gallery entries and runs the four shapes in turn (LFM3_MATRIX picks a subset). Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(realtime): document the LFM3-Audio VAD and chained-pipeline entries List the eight LFM3-Audio gallery entries, explain that silero-vad-ggml must be installed before the -vad entries, describe the chained pipeline on one model and the behaviour of a pipeline without a VAD, and describe the shape matrix of the e2e specs. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(realtime): wait for the detached input transcription in the S2S specs The input transcription runs in its own goroutine, so the transcription completed and failed events can arrive after the response ends. Poll for them instead of reading the event list right after the response. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): release the S2S stream before running server-side tool calls A response held the stream slot until it returned, and it ran the tool calls from inside that scope. A server-side tool starts a follow-up response on the same stream, which then waited for the slot its parent still held, so the client never got response.done. Run the tool calls once the slot is free; the event order is unchanged. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): end an S2S session whose stream dies, and stop queuing after close A stream that died during a response left the session open: later requests failed one by one and VAD frames kept being forwarded. The response now ends the session through the transport, and enqueue drops requests once the session is closed so the write queue cannot grow. The writer is now the only caller of Send and of CloseSend. close() used to call CloseSend while the writer could be inside Send, which gRPC does not allow. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): close the open S2S turn when the VAD stops Switching turn detection to manual in the middle of a VAD turn left the turn forwarding, and the next client commit sent the whole buffer a second time. Stopping the VAD now commits the forwarded audio on the backend and drops it from the input buffer. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): keep a tool result pending while an S2S user turn is open The check that refuses a tool result during a user turn ran when the item arrived, but the item was forwarded later, and the result was marked as sent before the backend took it. If the user spoke in between, the backend refused it and it was never retried. The result is now claimed and queued under the audio lock only when no turn is open, and otherwise stays pending for the next commit or response. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(talk): empty the input buffer for models without a VAD Without voice detection the microphone fills the server's input buffer for the whole session, so Done speaking committed idle silence and the echo of the previous reply. Clear the buffer when the session starts and when a reply ends. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(importer): keep an existing parallel option on LFM3-Audio imports The speech-to-speech import added parallel:2 even when the options already carried another parallel value. Use the same check as the hardware defaults, which also covers n_parallel. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(talk): list chained pipelines that have no VAD /api/pipeline-models required a VAD for a chained pipeline, so the VAD-less LFM3-Audio pipeline entries never showed up on the Talk page. Any pipeline with transcription, LLM and speech slots now qualifies. The Talk page treats every model without a VAD as manual-turn, and the session sheet says so for chained pipelines too. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(llama-cpp): bump llama.cpp to 0ac1a7fd Move the llama.cpp pin to 0ac1a7fd35c127a18c0cbc6c57d12e256a6a385b, the master head of 2026-10-29. It contains tool calls in the realtime server (ggml-org/llama.cpp#31287). The score task patch is regenerated so that it applies with no offset and no fuzz. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(functions): keep a null tool argument null A tagged parameter whose text was "null" came out as an empty string: unmarshalling JSON null into a string succeeds and leaves it empty, so the value was taken for a string. Check the first byte of the JSON value instead. A parameter with the text "null" now becomes JSON null, as in llama.cpp. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(functions): add an lfm3 XML format preset The LFM3 chat template writes a call as <tool_call>, <function=NAME>, one <parameter=KEY> block per argument, then the closing tags. The qwen3-coder format has the same shape and auto-detection already found it, but xml_format_preset: lfm3 was not a known name. Add the preset, with tests for one call, a call without arguments, two calls, string and non-string values (numbers, booleans, null, arrays, objects), a call between text, and unfinished input. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(realtime): parse tool calls with the chat template, as upstream Since ggml-org/llama.cpp#31287 the realtime server handles tools with the parser and grammar of the chat template. Do the same in the speech-to-speech stream instead of a hard-coded tag filter and a parse of the whole text at the end. - Each response gets a common_chat_session with the tools of the session and parallel calls. Its grammar and preserved tokens go into the sampling of the response, so the slot keeps the call tokens in the text and constrains the call. - The text of the response is fed to the session piece by piece (task_result_state::update_chat_msg). Content deltas become the transcript; tool-call deltas are collected into one function_call_arguments.done per call at the end. Audio after the start of a call is dropped. - A tool_choice of auto, none or required is accepted in the params of the config, in session.update and in response.create, with the response overriding the session. - The tool turn is derived from a conversation in which the assistant called a tool, like upstream, not from a plain reply. - response.done carries raw_text, everything the model wrote with the call markup, for a caller that parses it another way. If the parser throws, the rest of the response is content and the raw text is the way out. ToolCallFilter is no longer used and goes in the next commit. The unit tests cover the splitter contract (content, call deltas, the final piece, whole characters only). realtime_chat_check feeds LFM3-format text through the session in pieces of 1, 7 and all bytes and checks the call, the transcript and the grammar. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(realtime): drop the tool-call text filter The parser of the chat template now splits content from tool calls, so ToolCallFilter, which looked for the literal <tool_call> tags, has no caller. Remove it with its tests. Every case it covered (a call between text, split tags, several calls, an unfinished call, text after a call) is handled by the parser, and realtime_chat_check runs the LFM3 format through it in pieces of different sizes. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(realtime): fall back to the raw text for S2S tool calls For an any-to-any model, the calls the backend parsed still win. When there are none and the request had tools, parse the raw text that response.done now carries the way the chained pipeline parses the text of its LLM (CleanupLLMResult, ParseFunctionCall, ParseTextContent), so a call format the backend parser does not know can still be served with function.xml_format_preset or the other function settings of the model. The content around the calls replaces the streamed transcript only if the config captures it, as in the chained path. Also pass tool_choice to the backend: the session mode with the stream config and with session.update, and the mode of a response.create override with that event. A named function cannot be forced, so it is not sent. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): carry raw_text in the mock S2S response.done Match the llama-cpp session: with tools set, response.done has raw_text, the call markup of the LFM3 template for a tool turn and the transcript for any other reply. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(functions): parse tool blocks that have text between them The iterative XML parser stopped at the first scope block that was preceded by text, so "<tool_call>A</tool_call> then <tool_call>B" gave one call. Skip text between blocks once a call was found; text before the first block still ends the parse as before. Blocks that follow each other, or only whitespace, already worked. Tests cover three calls back to back and with text between them, with the lfm3 preset and with auto-detection. They also pin the current reading of argument values: the parser has no tool schema, so a value such as 42 or true is read as JSON even when the schema types it as a string. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): read LFM3-Audio tool calls with the lfm3 preset Set function.xml_format_preset: lfm3 on the eight LFM3-Audio-2B entries and in the config the importer writes for a speech-to-speech projector, so the server parses the call format of the LFM3 chat template when the backend returns no call (and in the chained pipeline entries, which use the regular chat path). gallery/index.yaml is regenerated from the library. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(realtime): describe how S2S tool calls are read Replace the description of the hard-coded tag filter with the parser and grammar of the chat template, tool_choice, and the raw-text fallback in the server. Document function.xml_format_preset, and set it in the real-weights e2e model configs like the gallery entries do. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(functions): keep earlier calls when a later block does not parse After a valid call, text that holds the opening tag again (a prose mention, a markdown example, a malformed second block) made the parser treat it as a new block, fail, and return no calls at all. Remember the state after each complete block and fall back to it when a later one fails: the calls found so far are kept, parsing stops there and the remaining text stays in the content. Text skipped between two valid blocks now goes to the content instead of being dropped, so StripToolCallMarkup keeps it. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): do not drop audio after a tool call started In mixed mode the speech for text written before a call can still be pending when the call starts, so dropping every audio frame after the first call delta cut the speech while its transcript was sent. The llama.cpp server never drops audio when it reads a result; do the same. If a model speaks the call markup aloud, that is model behaviour to observe on real weights, not something to filter here. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): flush held-back text when a response is cancelled The parser got is_partial=true on a cancelled response, so text it held back (a possible start of a tag) never reached the transcript. The llama.cpp server parses a cancelled response with is_partial=false; do the same. A cancelled response still makes no tool call. The path where the parser throws already sends the text as content and needs no change. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(realtime): stop the fake S2S stream from sending after Close The barge-in spec failed now and then with a data race and "send on closed channel": its script pushes audio from a goroutine, and the session closes the stream while that goroutine still runs. A push now does nothing once the stream is closed, and Close waits for a push in flight. The production close order is right: the writer closes the stream only after it stops sending. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): align the S2S tool-call fallback with the backend Four gaps in how the Go side reads the tool calls of an any-to-any reply: - With tool_choice none (the response value wins over the session value) the backend builds no tool parser, yet the raw-text fallback still turned a call the model wrote into a function_call item. Skip it. - Without a capture regex the fallback never cleaned the transcript, so the stored item and transcript.done kept the raw call markup. When the call is XML, use the text left after the markup is cut out; a reply that is only a call now has an empty transcript. - A named function as tool_choice mapped to nothing, so the backend kept the previous mode. Refuse it with an error in session.update and response.create. - A response-level tools list changed what the fallback looked for, but the backend only has the tools of the session. Use those in both. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs(realtime): say how S2S tool_choice and response tools behave Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(e2e): use the real llama-cpp backend in the real realtime run The suite always registered the mock backend under the name llama-cpp, so a run with REALTIME_S2S_MODEL set never reached the llama-cpp backend found in REALTIME_BACKENDS_PATH. Register the mock only when that run is off. The real run converts the input audio with ffmpeg. Check for it in the suite and in run-realtime-lfm3.sh before any download or backend start, with a message that says what to install, and list it as a prerequisite in the script header and the Makefile target. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(realtime): cover the tools path of a client session.update to the S2S backend A tool call that never came back from an any-to-any model raised the question whether the tools of the session reach the backend at all. Add a spec that sends the session.update of the real-weights e2e (flat Responses API tool, no session type) and checks that the backend gets the tool in the chat-completions form. Log the number of tools sent when the stream opens and when a session.update carries them, so a run log shows it without reading the wire. The "No tools received" line of the llama-cpp backend does not show it: the server prints it for the input transcription request, which has no tools by design. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): interrupt the chained pipeline reply while it is in flight With unary stages, the default of the lfm3 pipeline gallery entries, the chained pipeline synthesizes the whole reply and sends it as one audio delta right before response.done. The barge-in spec waited for that delta and then rendered the interruption with the TTS, so the first response was already done when the speech started and ended completed instead of cancelled. Real runs on a GPU showed it with both quantizations. For a pipeline, start the interruption at response.created, which comes before the LLM and TTS stages. Render the interruption audio before the first utterance is sent, so nothing sits between the reply and the interruption. The any-to-any shape still waits for the first audio delta, which that backend streams while it generates. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): accept a number word and its homophone in the spoken-answer check The check that the reply audio says what the transcript says needs a shared word of 4+ letters. A short answer failed it: the model wrote "Four" and the ASR heard "for". Compare numbers in one spoken form, so digits, number words and their homophones match each other. A homophone only counts when the other text has the number itself, so "for" on both sides still shares nothing and the check does not pass on a stop word. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(llama-cpp): ask LFM3-Audio for function calls when a realtime session has tools The default system prompt of a realtime session was "Respond with interleaved text and audio." whatever the tools. Under that prompt LFM3-Audio answers in words ("I'll check the weather ...") and never writes the call. The model card, and the llama.cpp realtime example client, use "Respond with function calls, and interleaved text and audio." when the session has tools, and the model then calls the tool. Pick the default by the tools of the session, and treat the plain default that the LocalAI server sends back as no choice: it follows the tools too, also when they arrive with a session.update before the first audio. A prompt the client wrote is kept as it is. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): transcribe the input of an any-to-any turn between responses The input transcript of a speech-to-speech turn ran at the same time as the reply, on the same model. llama.cpp serves both from one context: the realtime slot wants embeddings on, the chat slot of the transcript wants them off, and each flip makes it reserve its compute buffers again. In a GB10 run that was two reserves of about 175 ms per step, the reply fell to 2.6 tokens per second and the transcript to 380 ms per token, until the transcript was done. When pipeline.transcription is the model of the stream, run the transcript after the response, or before one that is not started yet, and keep responses out while it runs. A separate transcription model still runs in parallel. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): send the input transcript of an any-to-any turn before its reply The previous change moved the same-model transcript after the response. A client that reads until response.done then never saw conversation.item.input_audio_transcription.completed: the transcript was made (the log shows it) but sent after the end of the window. Run it in line with the commit instead. A VAD turn transcribes and then responds. A client response.create already waits for the commit slot, so it starts once the transcript is out. The transcript event now comes before any output of the response of its turn, for a manual commit and for a VAD turn, and never overlaps the response on the llama.cpp context. A transcription model of its own still runs in parallel. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(realtime): transcribe the input of an any-to-any model only when asked The stream answers from the audio, so the transcript of the input is for the client only. On the model of the stream it is costly: llama.cpp serves the transcript request and the realtime stream from one context and rebuilds its compute buffers on every step while both are active. On a GB10 that was 26 to 29 reserves of about 175 ms per turn, against 0 to 1 on the llama.cpp server, and the reply ran at a few tokens per second. An any-to-any model no longer fills pipeline.transcription with its own name. Without it the session does not transcribe the input; with another model it does, in parallel with the reply; with the model itself named it does, before the reply. The reply transcript always comes from the stream. The gallery entries set none, so they get the fast path. The mock configs and run-realtime-lfm3.sh name a transcription model, so the real-weights spec still covers the input transcript. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(realtime): end a response as cancelled when a barge-in lands during the LLM call The chained pipeline cancels the response context on a barge-in. When that happens while the model is generating, the backend call fails with the context error, and the response ended as failed: an error event and no response.done. A client that waits for the terminal of the interrupted response then waits forever. On a GB10 the q4 model answered before the barge-in arrived, so only the slower q8 model showed it: the spec read events for 30 s and timed out. When the context is cancelled, a failed Predict call or a failed result is a cancel, as the TTS step and the streamed LLM path already treat it. The response ends with response.done{cancelled} and no error. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): wait for the input transcript that comes from a model of its own An any-to-any model now gets its input transcript from a separate ASR that runs next to the reply. The reply is often over first, so a spec that stops reading at response.done saw no transcript in the VAD shape. Keep reading, up to 30 s, until the transcript arrives. The contract is that it comes, not when. Add a guard spec for what the input transcription is handed: the whole turn, in a WAV that says the rate it was recorded at (the session rate for a client commit, 16 kHz for a VAD turn). Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): check the errors of Close in the realtime specs golangci-lint (errcheck) flagged the unchecked Close of the websocket connections, the response body and the fake stream. Use the `defer func() { _ = x.Close() }()` form the other e2e specs use. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(e2e): write the mock S2S specs with Ginkgo and drop the env read forbidigo only allows Ginkgo/Gomega specs and no os.Getenv outside the sanctioned places. Convert the mock S2S tests to Describe/It with a suite bootstrap (the package had none), with the same 11 cases. The mock no longer reads MOCK_S2S_FRAMES: only these specs set it, so the reply length is a package variable they change per spec and restore afterwards. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
localai-org-maint-botandmudler
|
b343621bb7 |
chore: ⬆️ Update CrispStrobe/CrispASR to 6f7e812e5a7e05ab56300be8cf96abdd6a25d4a0 (#12933)
⬆️ Update CrispStrobe/CrispASR Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
6ff23294cc |
chore: ⬆️ Update mudler/vllm.cpp to 3284412c0c45976aaf661115928fcf1c64b83cab (#12952)
⬆️ Update mudler/vllm.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
080524ae51 |
chore: ⬆️ Update ikawrakow/ik_llama.cpp to 1c5ea124098890184b297023cd9fc9655451b1ec (#12954)
⬆️ Update ikawrakow/ik_llama.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
9209bcc074 |
chore(model-gallery): ⬆️ update checksum (#12955)
⬆️ Checksum updates in the model library Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
85617924fd |
chore: ⬆️ Update 0xShug0/audio.cpp to 1b8c8122c072293c435ade8dc991405286ec8856 (#12930)
⬆️ Update 0xShug0/audio.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-botandmudler
|
c99f5f82e2 |
chore: ⬆️ Update ggml-org/llama.cpp to 338fce6511c744893cc0d3d958b275f40552c7a4 (#12944)
⬆️ Update ggml-org/llama.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |