ASR backends like Nemotron-3.5 append end-of-utterance language tags to
transcripts ("<de-DE>", "<en-US>", ...). The committed transcript feeds
the LLM conversation (the user item Transcript field), but the per-utterance
tag is unreliable for short/ambiguous utterances (a German "Hallo." tagged
<en-US>) and the LLM treats it as a language signal, switching the reply
language against the session instructions.
commitUtteranceWithTranscript now strips the EOU tags from the transcript
that goes to appendUserItem/generateResponse (the LLM context). The emitted
input_audio_transcription event keeps the raw transcript (debugging /
client display).
Regression tests: stripEOULanguageTags unit specs + a client-commit spec
asserting the emitted event keeps the raw tag while the LLM user message is
tag-free.
Signed-off-by: nexxtmobile.de <git@nexxtmobile.de>
(cherry picked from commit dff488044c)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Anthropic sends tool_choice as an object: {"type": "auto" | "any" |
"none"} or {"type": "tool", "name": ...}. The endpoint only read the
"tool" object, so {"type": "none"} still offered the tools and
{"type": "any"} did not require a call. The object was also forwarded
to the backend unchanged, and the llama.cpp backend turns any object
tool_choice into "required", so even {"type": "auto"} forced a tool
call there. Read the object form, and forward the OpenAI equivalent.
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: jaeminbird <jaeminbird@gmail.com>
(cherry picked from commit 5bd07865a3)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
/v1/messages always answered stop_reason "end_turn" for text replies,
even when generation stopped at max_tokens. Clients that continue a
truncated turn (the Anthropic SDKs, Claude Code) treated the cut-off
reply as complete. Use the same check the OpenAI endpoint uses for
finish_reason "length", in both the streaming and non-streaming paths.
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: jaeminbird <jaeminbird@gmail.com>
(cherry picked from commit d905ab5fdc)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The finetune.start and quantize.start operations defaulted to the
backend names "trl" and "llama-cpp-quantization", and the backend
lister had one method per engine family. Both were special cases that
a new engine could not use.
The lister now answers for a gallery tag. A start call that names no
backend gets the first installed backend of the tag in gallery order,
and a named backend must carry the tag. With no engine installed the
call fails with a 400 that says so. The shipped gallery order keeps
trl and llama-cpp-quantization as the defaults, and a test pins it.
The change is additive for add-ons: no manifest field or operation
changes.
Assisted-by: Claude:claude-sonnet-5-5 go gofmt
The GitHub add-on made "GitHub" a provider name, so the test flagged three
strings that name the service itself: the OAuth sign-in button, the footer
link to the project repository and the "GitHub releases" package source in
the egress registries hint. None of them names the add-on, so add narrow,
reasoned allow-list entries for exactly those keys.
Assisted-by: Claude:claude-sonnet-5-5 node npm
The previous commit was made on a stale base and so reverted work that
landed on master before it. This commit puts every file back to the
state of its parent's parent. The permission enforcement of the
previous commit stays.
Assisted-by: Claude:claude-sonnet-5-5 go
Operators could not tell which client sends requests that are refused
with 401. Add --log-client (LOCALAI_LOG_CLIENT, off by default). When
set, the HTTP request log line also carries duration, remote, user_agent
and, when present, request_id.
The address is the TCP peer. It is never read from X-Forwarded-For,
X-Real-Ip or Forwarded, because LocalAI has no trusted-proxy setting and
any client can forge those headers. Behind a reverse proxy the value is
the proxy address. Authorization headers, cookies, query strings and
bodies are never logged. User agent and request id are cut to a fixed
length and have control characters replaced.
Assisted-by: Claude:claude-sonnet-5-5 go gofmt
The capabilities, role grants and pairing that a connector binding
stores now decide what a sender can make the agent do.
Authorize is one pure function: the capability must be in the owner's
ceiling for the binding and the sender must be on the allowlist. A
sensitive capability then needs a paired sender; roles never count for
it. Any other capability needs a pairing or a role that the owner
granted and that the kind lists. The default is no.
Each message gets a grant for its sender. A message that joins a
running turn narrows it to what both senders may do. The tool builder
offers a gated tool only for a held capability, and the tool checks
again when the model calls it, so a call that gets through is refused
with the reason. Today this covers propose_change (memory and profile
kinds), revert_change, the trigger tools, and files sent to the bot. A
chat worker gets the grant with the start request.
The capability gate that the trigger features ask is now answered by
Authorize, and the application sets it. Other features can add a
global switch with AddCapabilitySwitch.
Every use records an allow or deny row (time, binding, sender IDs,
roles, capability, result; no text or codes). GET
/api/chats/connectors/{id}/access returns the effective permissions
per kind of sender and the latest rows, for an add-on settings page.
A binding saved before this change has no capabilities, so its senders
lose these features until the owner enables them.
Assisted-by: Claude:claude-sonnet-5-5 go swag
The agent form offered a different set of choices than the chat: grants
as checkboxes for add-ons, connectors and skills, a free-text list of MCP
servers, and nothing for the built-in tool groups, so a person could not
tell which tools an agent can use.
The form now draws the lists of the Tools menu with the same components
(SwitchPanel, SkillsPanel) and the same rows (useToolGroups), fed by
useAgentTools: the draft lists of a new chat on the agent's model, with
the agent's grants and switches applied. A switch changes the form, and
the agent is saved with it:
- built-in groups: builtins_off and builtins_on (an opt-in group such
as saved agents starts on only when listed);
- add-ons, connectors, skills: the existing grant lists, so what is
switched on is what is granted. A granted entry this account cannot
see stays in the list, marked, so it can be removed.
The network choice and the folder and mode choice stay where they were.
The MCP servers of the model stay a free-text list in their own fold,
since a model that is not chosen yet has none to list.
Assisted-by: Claude:claude-sonnet-5-5 eslint playwright
The Tools menu built its rows inside the composer, drew only some of
them with a description, and showed a group it had no words for ("changes",
the memory and profile edits) by its raw ID.
The rows now come from useToolGroups, and every row has the same three
parts: a title, a description of what the model can do with the switch on
(and, once it is off, what it loses), and a state line (who serves the
tools, whether commands are isolated, why a switch cannot be used). The
words come from the server, which declares a title and a description for
each group and each add-on; a translation by group ID wins when the
locale has one, so a group added later needs no change here. An add-on
row shows the title its manifest declares instead of its ID.
SwitchMenu is split into the chip and SwitchPanel, and SkillsMenu into
the chip and SkillsPanel, so another page can draw the same lists inline.
New strings are in every locale.
Assisted-by: Claude:claude-sonnet-5-5 eslint playwright
The Tools menu of a chat listed a ninth built-in group, "changes", by its
raw ID with no text, and only some rows said what a switch did. The
registry now carries a title, a description of what the model can do when
the group is on, and what happens when it is off, for every group. Groups
that nobody described fall back to the label and description of their
first tool, so a new tool group needs no change in the interface.
An add-on row can show the title and description its manifest declares:
the broker may implement the optional AddonLister, and the lists carry
both fields next to the ID.
A saved agent gets builtins_off and builtins_on. builtins_off is a
ceiling: a chat of the agent shows the group off with a reason and cannot
switch it on, and a sub-agent follows it. builtins_on lists the opt-in
groups the agent's chats start with. Both are additive columns, a write
that does not name them keeps the stored ones, and groups the server does
not offer are dropped so an imported file still loads. An agent with
switches exports as version 3, so an older server refuses the file
instead of giving the agent every tool; a single-folder agent without
switches still exports as version 2.
Assisted-by: Claude:claude-sonnet-5-5 go gofmt swag
Add a manifest credentials declaration, host-enforced per-binding
resource lists, Request.Credential and ResourceKey set by the host, and
a typed ErrRateLimited that reaches the model as rate_limited.
Assisted-by: Claude:claude-sonnet-5-5 [go]
The Files section of the agent editor is now a list of rows. Each row
has a folder picker, a read or write switch and a remove button. An
Add folder button stops at eight. A folder inside another one shows the
mode of the outer folder, because that is the mode it works at. The
agent chips, the Tools menu line and the Files page marks read the whole
list. The strings are in all nine locales.
Assisted-by: Claude:claude-sonnet-5-5 eslint playwright
POST and PUT on /api/chats/agents take folders, a list of path and mode
rows, up to eight. The old folder field still works and is read as a
list with one entry. Responses carry folders, the effective mode of each
row and a notice with the count of shared folders a chat does not mount.
A folder inside another folder of the same agent takes the mode of the
outer one, because the file store gives a node the highest mode among
the roots above it and cannot hold a read only folder inside a write
folder. The editor data shows that mode.
An agent with one folder still exports as version 2. An agent with
several exports as version 3 with folders. A server that reads version 2
refuses version 3. Import keeps a folder that is not found as a row
that is not chosen.
Assisted-by: Claude:claude-sonnet-5-5 go gofmt
A chat that does not exist yet has no execution status to read, so its
isolation is unknown by construction. Three places treated that as a
failure:
- Switching an add-on in a draft reset the commands state and never
refilled it, so the "isolation is unknown" notice stayed. The draft
now keeps the state its list gave it.
- The skills list of a draft hid the document skills of the selected
commands provider, because they need isolated commands. A draft list
now offers them; the turn of the real chat still gates them on the
status it reads. The draft add-on switch also reads the skills again.
- The shell row of the Tools menu printed the generic unknown text for
the draft state. It now says nothing about isolation there.
In a real chat, switching an add-on or a tool group now reads the
chat's network view again, so the network controls appear once the
provider serves commands.
Assisted-by: Claude:claude-sonnet-5-5 go gofmt playwright
A connector chat can now propose a trigger, the person answers it in the
chat, and the run reports to the chat.
The trigger tools propose_trigger and list_triggers are offered in a
connector chat when a capability gate allows triggers.propose for the
binding. The gate is a small interface on the service with a deny-all
default, so nothing changes until the application sets one. Change and
delete stay in Studio.
A proposal from a chat stores the binding, chat and thread, a four
character code and a 24 hour expiry. The host writes the proposal and the
code in that chat. "confirm CODE" or "cancel CODE", as a whole message in
the same chat and thread, is handled before any model turn and needs
proposals.confirm. A binding may make 3 proposals an hour. Proposals that
lapse are marked expired, and the proposal card shows that state.
A trigger carries a delivery: the Inbox (default) or the chat of the
proposal, with the modes always, on_change and on_problem. The host writes
the last message of the run, cut and with broadcast mentions broken; a
message of exactly [SILENT] writes nothing; three failures in a row raise
one alert per 6 hours. A failed delivery falls back to the Inbox. Host
messages of a binding go through an outbox of 20 a minute that queues the
rest. While triggers.run is off, runs of such a trigger are skipped, not
replayed, and the record is unchanged. The audit records each step with a
hash and no text.
Assisted-by: Claude:claude-sonnet-5-5 go test ginkgo eslint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A connector kind can now declare the roles it gives to a sender and the
capabilities it can grant to a role. The owner of a binding enables a
ceiling of capabilities and maps roles to them. The host stores these
values and returns them with the binding and the kind list, so an add-on
settings page can show them.
Capability names are a closed list owned by the host. Sensitive names
may be listed by a kind but are never granted to a role; only a paired
sender can use them.
Pairing proves that the owner controls a platform account. The owner
asks for a one-use code (valid ten minutes) and sends it as a direct
message to the bot from an allowlisted account. The host checks the code
before any model turn and never logs it. A binding holds three paired
senders, a pairing ends after 90 days, and five wrong codes in an hour
lock the sender out for an hour.
Contract: connector kinds gain optional roles and permissions, and a
message gains optional roles. They need manifest api 2, which the host
already accepts; protocol is unchanged and manifests that declare
nothing new behave as before. The host drops roles a kind does not
declare.
Enforcement of the capabilities by feature is not part of this change.
Assisted-by: Claude:claude-sonnet-5-5 go swag
The entry gets a badge with an accessible label when automated chats
wait for the person, which changes the link name. The exact name match
then found nothing whenever other specs left such chats on the shared
test server.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The connector page no longer shows the switch or sends thread_context.
The host reads the thread of a mention whenever the kind can give it.
The Slack page spec checks that the switch is absent and not sent.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The thread_context binding setting is removed. When the kind declares
the history ability and the bot is mentioned in a thread, the host
reads the thread once for a new chat, as before. The other guards stay:
the ability check, the bound, the untrusted block, and a failure never
blocks the answer.
A binding stored by an earlier build still loads. The JSON store
ignores the unknown field. The GORM row no longer maps the
thread_context_off column; AutoMigrate does not drop it and it accepts
NULL, so reads and writes keep working. No data is migrated or removed.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The Settings page states the ceiling in effect from this field.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The owner policy key connector_files_ceiling_bytes gets a field on the
Settings page, next to the add-on upgrade setting. It states the ceiling
in effect and that larger files need the add-on to support streaming.
The host settings client accepts the new key as a number.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The section has a field for the largest file the bot reads and one for
the largest it sends. A value can only lower the limit of the add-on.
Each field shows the limit in effect and the layer that sets it, and the
page says that larger files need the add-on to support streaming while
the inline cap is the limit. The fixed 1 MB in the intro is gone.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The connector list shows, for each binding and each kind, the limit in
effect in both directions, the layer that sets it, the ability the kind
declares and the most an owner may set. A binding accepts
file_receive_max_bytes and file_send_max_bytes; a value above the limit
of the kind is a 400.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Chats that a trigger, a connector, a job or a page started no longer fill
the sidebar, Jump back in or the default All chats list. All chats gets a
second tab, From add-ons and automations, that groups them by origin kind
and by origin, using the name of the binding or the trigger (a kind this
release does not know falls under a generic group). The All chats entry
in the sidebar shows how many of them wait for the person. Ctrl+K still
finds every chat and marks these with a badge, and a transcript of such a
chat says what started it, with a link to its page. The browser remembers
the tab.
The lists ask the server for origin=person or origin=automated, so a
burst of bot chats cannot push a person's chats off a page.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
GET /api/chats takes an optional origin parameter: person (chats a
person started), automated (every other chat) or one origin kind such as
trigger or connector. The limit counts the chats kept, so a flood of
chats that a connector or a trigger started cannot push the chats of a
person off a page, and the other way round. Without the parameter the
list is unchanged.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A chat that is not created yet has no add-on bound to it, so its
execution status cannot be read. The host now says so with the reason
draft, and the Home composer shows no commands warning for it.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A kind can declare options with a few values and a default (manifest
api 2). The host checks the owner's choice against the declaration,
stores it on the binding, shows it in the REST API and sends the value in
effect with every poll. The add-on decides what the option means.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A binding reads the earlier messages of a thread by default. The new
thread_context option turns that off, and the kind list says which kinds
can give a thread's messages. It is stored like send_files: the zero
value keeps the default.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The Network row of the chat Tools menu, the consent and blocked cards and
the network-policy block on an add-on's Settings tab take the network
title and the preset titles from the API (network, as the add-on
declares it) and no longer say "Sandbox". The setting labels are
neutral. A second add-on with another ID and title gets the same row,
card and block.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The chat's network view, the administrator view and the line the model
reads about the network of its commands now take their name from the
capabilities.network an add-on declares, or from the title of its
commands tool when it declares none. The sentences no longer say
"Sandbox" or "the sandbox". The API gains a network object with the
title and the preset titles next to the existing addon field.
The text of the built-in bash and of the host-exec setting names an
add-on that provides bash instead of one add-on.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The agent task form, the scheduling rule sheet and the add-on model
selector now draw the shared picker: a button that opens a listbox, with
a warm or cold tag inside each option name. Four specs still used the
old select, combobox and exact option names.
Task and rule specs open the picker button and choose from the list. The
rule spec types a custom name in the filter and takes it with Enter. The
diarization and 3D animation specs match options by their model name
prefix, as the other specs already do.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Read addons/*/manifest.json at run time and check every string of every
locale and every Settings label for the name of an add-on that shadows a
built-in tool or skill, serves as a connector or declares the execution
provider. The allow-list keeps a reason per key and fails when an entry no
longer matches.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The locale files and the Settings labels said "Sandbox" for the add-on that
runs commands. The strings now say "commands" or "an add-on that isolates
commands", in every shipped locale. The name of the serving add-on in the
network section and in a transcript step comes from its declared title in
the add-on list.
Remove the strings that no component reads (the Sandbox notice texts, the
Tools menu sandbox lines and the Home wizard step). Add the seven
execution-status reasons to the locales that lacked them and lower the
i18n baseline, which master was over by seven keys.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The prompt, the Tools menu reasons, the network error texts, the CLI help,
the API instructions and the assistant tool descriptions said "Sandbox" for
the add-on that runs commands. Core must not know that name.
The reason a chat has no commands now uses the title the provider declares
("the Zeta add-on is off for this chat") or, when none is installed, says
"the add-on that runs commands". Text that has no provider in hand speaks of
the network of commands or of an add-on that isolates them.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Union of both case lists in the add-on suite inventory; the swagger
files are regenerated from the merged code.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Playwright specs with a stubbed API: confirm, edit then confirm, dismiss,
undo, expired, stale, a failed write, unknown kinds and sources, the inbox
count and row, the Proposals page, and the Memory tab with read, edit,
delete and clear all. The account spec lists the new tab.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The tab lists the context files that the host puts in the prompt (the
user's index and topics, each agent's and conversation's notes, and the
agent profiles) with their sizes against the caps. The owner can read,
edit, delete and clear them; an index over its cap cannot be saved.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A propose_change tool line now draws a card in the thread, for any kind
and any source: the kind, file and scope, the source, the time left and a
line diff against the file as it is now. The owner can Confirm, edit the
text then confirm, or Dismiss. A confirmed change shows a receipt with
Undo. Expired and stale proposals say nothing was saved and offer to ask
the assistant again; a failed write leaves the proposal waiting.
The inbox button counts the waiting changes and its panel links to a new
Proposals page that lists them with the same card.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Pure helpers with node tests: a line diff with context, the state of a
proposal (including a pending one whose time ran out), the source and
scope wording inputs, and the size of a context index against its caps.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Document the optional body of the confirm route, the context-files route,
the card the thread draws for any kind and source, and the Memory tab. The
context-file listing is a person's own action, so the actor scan allows it.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The apps API spec checks that the generated docs match the emitted
fields, so it needs a descriptor in its full fixture.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The proposal card lets the owner correct the text before confirming.
POST /api/chats/changes/{id}/confirm now accepts an optional JSON body
{"content": "..."}. The text has the same size limit and text check as a
proposal, the file and the audit hash hold what was written, and a
refused text leaves the proposal waiting.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The Memory settings page needs to know which index and topic files the
host loads into the prompt. GET /api/chats/context-files walks the
registered context sources, expands the agent and channel folders that
exist, adds the agent profiles and reports each file with its size, the
line count of the index, and the caps. It only reads; the page edits and
deletes through the Files API.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The Memory page reads the context-file sources that master added.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
List, read, confirm, dismiss and undo the changes a model proposed, and
read their audit. The routes sit under /api/chats with the other chat
routes and need the chats feature. A person reaches only their own
proposals; another person's id is a 404.
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Aniruddh Krovvidi