Commit Graph
551 Commits
Author SHA1 Message Date
Ettore Di Giacinto 9e6a7a077c fix(realtime): leave speech-to-speech client commits at the session rate
The speech-to-speech path labels its transcription WAV with the session
rate for client commits, so resampling first handed it 16 kHz audio under
a 24 kHz header. Resample only in the chained pipeline, and say so in the
docs.

Assisted-by: Claude:claude-sonnet-5-5 go-test
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
(cherry picked from commit 82cc411af54cf884e4a44a3e96292a323ba5611c)
2026-11-03 19:23:38 +00:00
nexxtmobile.de fbfb2ccadf fix(realtime): resample client commits to 16 kHz before transcription
The committed-utterance WAV is written with a fixed 16 kHz header. The VAD
commit path stores audio that is already resampled to 16 kHz (vadCommit),
but the client-commit path (input_audio_buffer.commit) passed the raw
buffer at the session input rate. For a 24 kHz OpenAI Realtime session the
backend decoded 24 kHz audio as 16 kHz (1.5x speed/pitch): garbage or empty
transcripts, then input_not_committed on the follow-up response.

commitUtterance now resamples from session.InputSampleRate to
localSampleRate, so both paths hand 16 kHz audio to the 16 kHz header.
This replaces the earlier approach of this PR (header at the session rate),
which mislabelled the already-resampled VAD audio (review feedback).

Tests go through the real paths with a 24 kHz session: commitUtterance and
a server_vad commit via vadTick; both must produce a 16 kHz WAV of the
resampled length. The client-commit spec fails without the fix.

Fixes #12747

Signed-off-by: nexxtmobile.de <git@nexxtmobile.de>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 05ba891da6)
(cherry picked from commit 56e2b6d0cc5cbf1944ef50735f7b499d998eb25c)
2026-11-03 19:23:38 +00:00
xianshujun 45d7a967c7 fix(openai): keep answers in content when thinking is disabled
qwen3's chat template wraps <think> in {% if enable_thinking %}, but the
prefill detectors scan the raw template without evaluating the condition.
With streaming and no tools, the synthetic prefill made the Go-side extractor
classify the whole answer as reasoning whenever the client sent
metadata.enable_thinking=false.

Skip prefill detection when the request disables thinking. Real
<think>...</think> blocks that a model emits anyway are still extracted from
the content itself.

Fixes #12939

Assisted-by: ZCode:deepseek-flash
Signed-off-by: xianshujun <xianshupang@gmail.com>
(cherry picked from commit cee7ed4b32)
(cherry picked from commit 29618866d07a05624934bf9f595c95a68c938461)
2026-11-03 19:21:12 +00:00
Stefan Walcz 2247835401 feat(config): mark links a profile answer names without retrieving them
A model can cite a plausible deep link it never opened, written from
memory. The client cannot tell it from a link a search tool returned,
because server-side MCP tool results never reach it.

evidence_check.urls compares every http(s) link of the final answer,
bare or inside a Markdown link, with the URLs of the request (system
prompt, system, user and tool messages) and of the request's successful
MCP tool results. Links are compared without scheme, fragment, "www."
and trailing slash; a link counts as retrieved when it equals an evidence
URL or is a shorter form of one at a path or query boundary. Unbacked
links (at most five) are listed in an appended note; urls_note replaces
the default note, with {urls} as the placeholder for the list.

Assistant messages are not evidence, so an invented link is marked again
when the client sends it back on the next turn.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit 523e8cd3c4)
2026-11-03 19:18:43 +00:00
Ettore Di Giacinto f4a32aacc9 test(openai): pass the compressor argument in the require_tool spec
ChatEndpoint gained a compressor parameter after the spec was written, so
the openai test package no longer compiled.

Assisted-by: Claude:claude-sonnet-5-5 go golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 19:18:43 +00:00
Stefan Walcz 052e02c75b fix(chat): keep evidence.go out of the tool consumer matrix
The comment in sourceURLs named ExecuteMCPToolCall, which the tool
consumer matrix test (core/services/addons/tools) reads as a marker of a
file that discovers or executes tools. evidence.go only reads tool
output, so reword the comment instead of classifying the file.

Docs-Not-Needed: comment-only change, no behaviour change
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit 1043d3cbc4)
2026-11-03 19:18:35 +00:00
localai-org-maint-bot 041f2337df fix(chat): preserve complete tool source URLs
Keep balanced parentheses in source URLs and decode MCP text arrays before
extracting links. This preserves Wikipedia paths and escaped query
separators instead of appending truncated citations.

Assisted-by: Codex:gpt-6
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit 9b81cb20ab)
2026-11-03 19:18:35 +00:00
Stefan Walcz 23012a7c4a feat(config): let a profile alias require a tool call on the first turn
A system prompt that tells a model to look things up first does not stop
it from answering from its own knowledge, for example when the question
comes with a picture.

profile.require_tool makes the first model call of a chat request that
offers the profile's server-side MCP tools use tool_choice "required".
Every follow-up call of the server-side tool loop is made with the
client's tool_choice again: a "required" that stayed in place would make
the model call a tool after every tool result and never answer until the
iteration limit.

With a tokenizer template the backend enforces "required" with a grammar
built from the chat template's tool-call format. Where LocalAI builds the
tool-call grammar itself, the first call's grammar leaves out the
no-action function instead.

The first call is not forced when no server-side tool was offered, when
the client set tool_choice or function_call itself, or when the request
does not end with a user message, so a request returning a client tool's
result cannot bounce tool calls between client and server.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit 2801938aac)
2026-11-03 19:18:35 +00:00
Stefan Walcz 864c176cd2 feat(config): append tool result URLs to uncited profile answers
A research profile asks the model in its system prompt to cite its
sources, but a model can answer from a search result without naming a
single link, and the client never sees the server-side tool results that
held them.

evidence_check.sources appends one line with the distinct http(s) URLs
returned by the request's successful MCP tool calls (at most five, in
order of appearance) when the final answer names no URL itself. An answer
that already links something is left unchanged, and nothing is added when
no tool result holds a URL, so a link is never made up. The label is
configurable through evidence_check.sources_label (default "Sources") for
profiles that answer in another language.

It runs in the existing evidence gate, on the same streaming and
non-streaming paths as the number note.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit 4ee6ff1395)
2026-11-03 19:18:35 +00:00
Ettore Di Giacinto 6b181cd20b feat(studio): enforce the action policy of unattended runs
A trigger run or an agent job has nobody to approve a step, and so far
the approval mode auto_readonly and the act.* grants were only recorded.
The prompt was the only thing that kept such a run from acting.

Every tool call of an unattended run now goes through one check at the
call. A tool that only reads runs. Any other tool needs the grant of its
class, and the check fails closed:

- act.files for write_file, edit_file, move_file, delete_file and the
  built-in bash;
- act.addons, or act.addons:<id>, for add-on tools whose declared kind is
  not read, plus act.spend when the add-on gates the tool with a
  permission;
- act.spend for MCP tools.

A trigger holds its new acts list within the grants of its agent. A
trigger that declared read only runs, or that a binding in auto_readonly
mode made without asking, is refused every non-read tool whatever it
holds. A binding that cannot be read counts as auto_readonly. A job
holds the act grants of its agent. Sub-agents follow their parent, and a
saved agent cannot be started from an unattended run.

A refused call answers with a plain sentence the model can pass on, not
an error, and writes one audit log line. Triggers set acts through the
REST API; a model never can. A trigger whose plan names a file it writes
keeps act.files.

Assisted-by: Claude:claude-sonnet-5-5 go ginkgo swag
2026-11-03 19:12:29 +00:00
Ettore Di Giacinto cf1041ca96 feat(ui): list agent skills on the Skills page
The Skills page asks for the skills of the saved agents with
agents=true and shows them with the agent's name. A skill of the
agent layer says who proposed it, can be removed (to the Trash) and
is not offered to other agents. The list of a plain chat or of the
agent form never carries them.

Assisted-by: Claude:claude-sonnet-5-5 go node
2026-11-03 19:10:39 +00:00
Ettore Di Giacinto dcd180ced8 feat(studio): add the agent skills layer
A saved agent keeps its own skills in LocalAI/agents/<agent>/skills.
Only a chat that runs that agent sees them. They sit between the folder
and personal layers, so an agent's version of a skill wins for that
agent, and they count as granted without being listed in the agent's
skills.

An agent writes one by a skill change proposal, as before; the new
layer only reads them. A proposal for an agent skill cannot take the
name of a core skill or of a document slot (read from slots.json).
When writing-skills loads in a chat of a saved agent it starts with
"Do not write files. Call propose_change with kind skill.", and the
skills.write grant makes it visible to the agent.

The Skills list gains the agent layer, with the agent's ID and name,
and a skill can be read or removed (to the Trash) with the agent
query parameter.

Assisted-by: Claude:claude-sonnet-5-5 go swag
2026-11-03 19:10:39 +00:00
Ettore Di Giacinto afb06fd42f docs(addons): describe manifest api 2 only and update the test inventory
The add-on guide names one manifest api, one way to move a file and no
migration path, and drops the section on arguments of older tool
versions. The Swagger files are generated again, which also brings in
fields that the last generation missed, and the test of the apps view
now emits every field it documents. The inventory of the add-on suites
lists the specs by their new names.

Assisted-by: Claude:claude-sonnet-5-5 go swag
2026-11-03 16:57:13 +00:00
Ettore Di Giacinto c5f636d1b9 refactor(addons): move connector file bytes by handle only
A connector add-on no longer carries file bytes in the connector
envelope. The ops media and send_file need a transfer handle, and the
gateway refuses a call to them without one, or for a kind that declares
no files.

- The manifest field files.transfer, the inline file of a request and
  of an answer (ConnectorFile) and the 1 MiB cap
  (MaxConnectorFileBytes) are removed. The 2 MiB limit of a connector
  call between hosts drops to 1 MiB, because it holds text and no file.
- A kind that declares no files moves no file: the host answers a sender
  who attaches one with a plain sentence and does not offer attach_file.
  A direction left out of a declared files block still gets the host
  ceiling.
- The inline layer is gone from the file limit, from the API
  (inline_bytes) and from the Files section of the connector page.
- The Telegram and Slack add-ons keep only the streaming paths. The
  tests moved to the handle path, and the studio test rig gives every
  connector add-on a table of handles.

Assisted-by: Claude:claude-sonnet-5-5 go test node
2026-11-03 16:57:04 +00:00
Ettore Di Giacinto 2a491cb2cf refactor(addons): remove the shims for older add-on data
No release carried add-ons, so nothing reads older add-on data or
manifests. Delete the code that did.

- Connectors: the kind and the add-on of the Telegram connectors that
  core once carried are gone (LegacyAddon, KindTelegram, the stored
  binding normalization, the platform titles). A binding needs a kind,
  and its add-on is the one that serves the kind. The conversion of the
  old agent pool lists every connector as dropped instead of making a
  Telegram binding, and its notes say so.
- The library index reads only the id field of an entry, not addonId.
- A host control that has a capability block (the network policy) is
  attached by that block alone, not by a tool that replaces bash.
- The Office create tools no longer answer for the argument names of an
  earlier schema, and the identity file note about old addresses is
  gone.
- The known debt of the provider-name guard drops the entries of the
  conversion and of the resource listing shim, and its ceiling goes
  from 13 to 5.

Assisted-by: Claude:claude-sonnet-5-5 go test
2026-11-03 16:56:49 +00:00
Ettore Di Giacinto 1834d4fd72 refactor(addons): make manifest api 2 the only version
No release ever shipped add-ons, so there is no installed base on api 1.
The manifest must now declare api {"min": 2, "max": 2}; a manifest with
api 1, no api or a wider range is refused with a message that names the
range.

Every field that was gated on api 2 is now part of the contract, so the
Min*API constants and their gates are gone, and so are HostAPIMin and
the apiMin host field. Three older-manifest shims go with them: the
resource listing the host implied for one add-on, the legacyArgs hint
for arguments an older version of a tool took, and the tests that
checked the gates. All shipped manifests and the test fixtures declare
api 2.

Assisted-by: Claude:claude-sonnet-5-5 go test
2026-11-03 16:15:07 +00:00
Ettore Di Giacinto 7efb20bd1f feat(studio): read and send large connector files by handle
For a kind that declares files.transfer, the connector manager gives the
add-on a handle and not the bytes, in both directions.

Inbound, the add-on streams the file to the host. The chat upload then
reads it from the blob store, so a 6 MB PDF reaches the chat. Outbound,
the manager checks the access of the chat to the file, gives the
process that runs the add-on only the hash and the size of the content,
and the add-on pulls the chunks. The file limit of such a kind has no
inline cap: it is the smallest of the host ceiling, the ability the kind
declares and the limit the owner set for the bot, and the sentence names
the number and the layer. A kind without transfer keeps the 1 MiB inline
cap, and its sentence says so.

Checks: the files of one message may add up to three times the limit;
the media type that the add-on states must be among the types the kind
declares; an outbound file must have a type that the kind declares and
must not start with an HTML page. Every transfer writes one audit line
with direction, binding, size, result, layer and time, and no name or
content. The binding page no longer says "larger files need streaming"
for a kind that streams.

The specs run the real Telegram and Slack add-ons, the real gateway and
the real file operations over scripted platform APIs, with a 6 MB file in
each direction.

Not in this change: a daily byte counter per binding, and a virus scan
(the table has a hook that accepts everything).

Assisted-by: Claude:claude-sonnet-5-5 go test
2026-11-03 15:54:45 +00:00
Ettore Di Giacinto 15e2321d50 feat(connectors): edit the permissions of a binding on the shared connector page
Bindings saved before permissions were enforced have no capabilities, so
they deny files from senders, triggers, memory writes and talk-back, and
the owner could only fix that through the REST API.

The shared connector page now has a section for it, so every connector
add-on gets it: a switch and a description for each capability with the
sensitive ones marked, the role matrix (never a sensitive capability, never
above the ceiling), pairing with a code shown once, the approved channels
for talk-back, and a read-only view of the access check with the latest
decisions. A binding with no capability shows a notice that says why the
features are off, with an "Allow the usual" button that turns on the
non-sensitive everyday set.

The kinds endpoint returns the closed list of capabilities with a title,
a description and the sensitive and usual flags. The field is additive.

Assisted-by: Claude:claude-sonnet-5-5 go node playwright swag
2026-11-03 15:36:21 +00:00
Ettore Di Giacinto eaea737b53 feat(connectors): let the owner choose how chat triggers are approved
A connector binding gets an approval setting: ask, auto or auto_readonly,
with four limits for the two automatic modes (live triggers, minutes
between runs of a recurring trigger, runs a day, lifetime in days). The
setting is generic, so every connector add-on can show it, and the
shared connector page of the add-ons now draws it. A binding that sets
nothing asks, as before.

This change stores, validates and exposes the setting. The next change
acts on it.

Assisted-by: Claude:claude-sonnet-5-5 go swag node
2026-11-03 14:59:30 +00:00
Ettore Di Giacinto 24c837e453 feat(studio): grant built-in capabilities to a saved agent
A saved agent now has a Builtins list: the capabilities its owner gave
it (memory.write, profile.write, skills.write, pages.write,
dashboard.write, triggers.propose, triggers.run, talkback.channels,
talkback.new_channel, files.upload, act.files, act.addons, act.send,
act.spend and the opt-in group saved_agents). The list is empty by
default, and nothing turns a grant on for a template, an import or a
write that omits the field.

A chat of the agent holds the grants the user may use at the turn
(agent scope). A tool exists when the grant is held, the chat did not
switch its group off and the run can hold rights: a sub-agent or a run
that started by itself never does. propose_change and revert_change
now exist only for a chat that holds the grant of at least one kind,
and the schema lists only those kinds.

propose_change gains the kinds skill, page and dashboard. A skill is a
SKILL.md for the personal or the agent layer, and a skill with
locked: true is refused, also as a shadow of a locked personal skill.
A page is a set of files checked against the Pages rules and written
as one change set. The dashboard is a home.json checked against its
schema. All three are re-checked when the owner confirms or edits.

GET /api/chats/agents/builtin-grants lists each grant with its title,
description and sensitive flag. The agent record, the write body and
the export file carry builtins; an old file imports as before and an
unknown name is dropped in an import and refused in a write.

Assisted-by: Claude:claude-sonnet-5-5 go gofmt swag
2026-11-03 14:56:42 +00:00
Ettore Di Giacinto e790271c63 feat(addons): let the GitHub add-on write, under a per-connection write mode
A connection now has a write mode, chosen on the add-on page: off (the
default), propose (each change waits for the person to confirm) or auto
(a chat of the person makes it at once). The host enforces it for every
write tool of an add-on that holds a credential, before the add-on is
called, and holds the call to the repository list of the connection.

Host:
- Binding.WriteMode, stored in both stores and in the connector API
  (write_mode). Absent means off, so existing bindings stay read only.
- A credentialed add-on may declare write tools. They name the resource
  with the credential slot argument, never resourceId. A tool can set
  approval "always", a ceiling that keeps a proposal even in auto.
- The proposal broker fetches the credential for the preview and again
  for the apply, so it is never stored. A mode lowered between the
  proposal and the confirmation refuses the apply. A proposal that
  awaits a person cannot be confirmed by a chat turn.
- New capability name tools.write for the roles a connector kind
  declares.

Add-on, tools (kind write): comment-issue, comment-pr, create-issue,
edit-issue (title, close or reopen), add-labels, review-comment-pr.
They preview plain facts, refuse a stale target, cap and screen the
text for secrets, and keep a receipt journal plus a hidden marker in the
text, so a change whose answer was lost is not sent twice.

Assisted-by: Claude:claude-sonnet-5-5 go gofmt ginkgo node
2026-11-03 14:53:20 +00:00
Ettore Di Giacinto b8626132ae feat(connectors): let the agent write to listed channels with send_message
A connector chat whose sender holds talkback.channels gets the tool
send_message. It writes in the conversation, or as a new top-level
message in a channel that the owner listed on the binding (id, label,
approved; at most 10). The tool names no platform.

An unlisted channel is refused and the error lists the ones that are.
A listed channel that the owner did not approve needs the sensitive
talkback.new_channel for each message, so only a paired sender can use
it. The agent cannot add or approve a channel, so text it read in a page
or a file cannot widen where it writes.

Messages go through the outbox (rate limit and queue), with the size cap
and the mention breaking. Each attempt adds an audit row with channel,
kind, capability, length and a hash, never the text.

A trigger can report to an approved channel with delivery target
channel. The binding, the capability and the listing are checked again
when the run ends; any failure sends the notice to the Inbox.
LOCALAI_DISABLE_CONNECTOR_TALKBACK turns both off.

Assisted-by: Claude:claude-sonnet-5-5 go swag
2026-11-03 14:42:21 +00:00
Ettore Di Giacinto 11f7e2800a refactor(realtime): drop an external project reference from a comment
Assisted-by: Claude:claude-sonnet-5-5 git go
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 14:40:50 +00:00
nexxtmobile.de 60d2294338 fix(realtime): strip EOU language tags from the LLM context
ASR backends like Nemotron-3.5 append end-of-utterance language tags to
transcripts ("<de-DE>", "<en-US>", ...). The committed transcript feeds
the LLM conversation (the user item Transcript field), but the per-utterance
tag is unreliable for short/ambiguous utterances (a German "Hallo." tagged
<en-US>) and the LLM treats it as a language signal, switching the reply
language against the session instructions.

commitUtteranceWithTranscript now strips the EOU tags from the transcript
that goes to appendUserItem/generateResponse (the LLM context). The emitted
input_audio_transcription event keeps the raw transcript (debugging /
client display).

Regression tests: stripEOULanguageTags unit specs + a client-commit spec
asserting the emitted event keeps the raw tag while the LLM user message is
tag-free.

Signed-off-by: nexxtmobile.de <git@nexxtmobile.de>
(cherry picked from commit dff488044c)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 14:40:50 +00:00
jaeminbird d9b2f9c900 fix(anthropic): honor object tool_choice on /v1/messages
Anthropic sends tool_choice as an object: {"type": "auto" | "any" |
"none"} or {"type": "tool", "name": ...}. The endpoint only read the
"tool" object, so {"type": "none"} still offered the tools and
{"type": "any"} did not require a call. The object was also forwarded
to the backend unchanged, and the llama.cpp backend turns any object
tool_choice into "required", so even {"type": "auto"} forced a tool
call there. Read the object form, and forward the OpenAI equivalent.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: jaeminbird <jaeminbird@gmail.com>
(cherry picked from commit 5bd07865a3)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 14:40:50 +00:00
jaeminbird fb07f676a9 fix(anthropic): report max_tokens when the token budget is exhausted
/v1/messages always answered stop_reason "end_turn" for text replies,
even when generation stopped at max_tokens. Clients that continue a
truncated turn (the Anthropic SDKs, Claude Code) treated the cut-off
reply as complete. Use the same check the OpenAI endpoint uses for
finish_reason "length", in both the streaming and non-streaming paths.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: jaeminbird <jaeminbird@gmail.com>
(cherry picked from commit d905ab5fdc)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 14:40:50 +00:00
Ettore Di Giacinto 474e3eefbe refactor(addons): pick engine job backends by gallery tag
The finetune.start and quantize.start operations defaulted to the
backend names "trl" and "llama-cpp-quantization", and the backend
lister had one method per engine family. Both were special cases that
a new engine could not use.

The lister now answers for a gallery tag. A start call that names no
backend gets the first installed backend of the tag in gallery order,
and a named backend must carry the tag. With no engine installed the
call fails with a 400 that says so. The shipped gallery order keeps
trl and llama-cpp-quantization as the defaults, and a test pins it.

The change is additive for add-ons: no manifest field or operation
changes.

Assisted-by: Claude:claude-sonnet-5-5 go gofmt
2026-11-03 14:39:13 +00:00
Ettore Di Giacinto b15905e90f fix: restore the files that the previous commit removed by mistake
The previous commit was made on a stale base and so reverted work that
landed on master before it. This commit puts every file back to the
state of its parent's parent. The permission enforcement of the
previous commit stays.

Assisted-by: Claude:claude-sonnet-5-5 go
2026-11-03 14:33:05 +00:00
Ettore Di Giacinto 9e6d6520bd feat(connectors): enforce add-on permissions on each connector turn
The capabilities, role grants and pairing that a connector binding
stores now decide what a sender can make the agent do.

Authorize is one pure function: the capability must be in the owner's
ceiling for the binding and the sender must be on the allowlist. A
sensitive capability then needs a paired sender; roles never count for
it. Any other capability needs a pairing or a role that the owner
granted and that the kind lists. The default is no.

Each message gets a grant for its sender. A message that joins a
running turn narrows it to what both senders may do. The tool builder
offers a gated tool only for a held capability, and the tool checks
again when the model calls it, so a call that gets through is refused
with the reason. Today this covers propose_change (memory and profile
kinds), revert_change, the trigger tools, and files sent to the bot. A
chat worker gets the grant with the start request.

The capability gate that the trigger features ask is now answered by
Authorize, and the application sets it. Other features can add a
global switch with AddCapabilitySwitch.

Every use records an allow or deny row (time, binding, sender IDs,
roles, capability, result; no text or codes). GET
/api/chats/connectors/{id}/access returns the effective permissions
per kind of sender and the latest rows, for an add-on settings page.

A binding saved before this change has no capabilities, so its senders
lose these features until the owner enables them.

Assisted-by: Claude:claude-sonnet-5-5 go swag
2026-11-03 14:32:01 +00:00
Ettore Di Giacinto 15740acbb9 feat(studio): describe every built-in tool switch and let a saved agent set them
The Tools menu of a chat listed a ninth built-in group, "changes", by its
raw ID with no text, and only some rows said what a switch did. The
registry now carries a title, a description of what the model can do when
the group is on, and what happens when it is off, for every group. Groups
that nobody described fall back to the label and description of their
first tool, so a new tool group needs no change in the interface.

An add-on row can show the title and description its manifest declares:
the broker may implement the optional AddonLister, and the lists carry
both fields next to the ID.

A saved agent gets builtins_off and builtins_on. builtins_off is a
ceiling: a chat of the agent shows the group off with a reason and cannot
switch it on, and a sub-agent follows it. builtins_on lists the opt-in
groups the agent's chats start with. Both are additive columns, a write
that does not name them keeps the stored ones, and groups the server does
not offer are dropped so an imported file still loads. An agent with
switches exports as version 3, so an older server refuses the file
instead of giving the agent every tool; a single-folder agent without
switches still exports as version 2.

Assisted-by: Claude:claude-sonnet-5-5 go gofmt swag
2026-11-03 14:30:07 +00:00
Ettore Di Giacinto e0da582988 feat(addons): credential slot, resource allowlist and rate-limit error
Add a manifest credentials declaration, host-enforced per-binding
resource lists, Request.Credential and ResourceKey set by the host, and
a typed ErrRateLimited that reaches the model as rate_limited.

Assisted-by: Claude:claude-sonnet-5-5 [go]
2026-11-03 14:20:12 +00:00
Ettore Di Giacinto db596e1696 feat(studio): accept a list of agent folders in the API and the agent file
POST and PUT on /api/chats/agents take folders, a list of path and mode
rows, up to eight. The old folder field still works and is read as a
list with one entry. Responses carry folders, the effective mode of each
row and a notice with the count of shared folders a chat does not mount.

A folder inside another folder of the same agent takes the mode of the
outer one, because the file store gives a node the highest mode among
the roots above it and cannot hold a read only folder inside a write
folder. The editor data shows that mode.

An agent with one folder still exports as version 2. An agent with
several exports as version 3 with folders. A server that reads version 2
refuses version 3. Import keeps a folder that is not found as a row
that is not chosen.

Assisted-by: Claude:claude-sonnet-5-5 go gofmt
2026-11-03 14:02:02 +00:00
Ettore Di Giacinto ab78cd0301 feat(studio): propose triggers from a connector chat and report back there
A connector chat can now propose a trigger, the person answers it in the
chat, and the run reports to the chat.

The trigger tools propose_trigger and list_triggers are offered in a
connector chat when a capability gate allows triggers.propose for the
binding. The gate is a small interface on the service with a deny-all
default, so nothing changes until the application sets one. Change and
delete stay in Studio.

A proposal from a chat stores the binding, chat and thread, a four
character code and a 24 hour expiry. The host writes the proposal and the
code in that chat. "confirm CODE" or "cancel CODE", as a whole message in
the same chat and thread, is handled before any model turn and needs
proposals.confirm. A binding may make 3 proposals an hour. Proposals that
lapse are marked expired, and the proposal card shows that state.

A trigger carries a delivery: the Inbox (default) or the chat of the
proposal, with the modes always, on_change and on_problem. The host writes
the last message of the run, cut and with broadcast mentions broken; a
message of exactly [SILENT] writes nothing; three failures in a row raise
one alert per 6 hours. A failed delivery falls back to the Inbox. Host
messages of a binding go through an outbox of 20 a minute that queues the
rest. While triggers.run is off, runs of such a trigger are skipped, not
replayed, and the record is unchanged. The audit records each step with a
hash and no text.

Assisted-by: Claude:claude-sonnet-5-5 go test ginkgo eslint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 14:00:31 +00:00
Ettore Di Giacinto 1f481b0747 feat(connectors): add-on roles and permissions, and sender pairing
A connector kind can now declare the roles it gives to a sender and the
capabilities it can grant to a role. The owner of a binding enables a
ceiling of capabilities and maps roles to them. The host stores these
values and returns them with the binding and the kind list, so an add-on
settings page can show them.

Capability names are a closed list owned by the host. Sensitive names
may be listed by a kind but are never granted to a role; only a paired
sender can use them.

Pairing proves that the owner controls a platform account. The owner
asks for a one-use code (valid ten minutes) and sends it as a direct
message to the bot from an allowlisted account. The host checks the code
before any model turn and never logs it. A binding holds three paired
senders, a pairing ends after 90 days, and five wrong codes in an hour
lock the sender out for an hour.

Contract: connector kinds gain optional roles and permissions, and a
message gains optional roles. They need manifest api 2, which the host
already accepts; protocol is unchanged and manifests that declare
nothing new behave as before. The host drops roles a kind does not
declare.

Enforcement of the capabilities by feature is not part of this change.

Assisted-by: Claude:claude-sonnet-5-5 go swag
2026-11-03 13:52:56 +00:00
Ettore Di Giacinto e0173d8096 docs(api): drop thread_context from the connector create description
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 11:57:44 +00:00
Ettore Di Giacinto c8cb68b434 refactor(studio): always read a thread when the bot is mentioned in it
The thread_context binding setting is removed. When the kind declares
the history ability and the bot is mentioned in a thread, the host
reads the thread once for a new chat, as before. The other guards stay:
the ability check, the bound, the untrusted block, and a failure never
blocks the answer.

A binding stored by an earlier build still loads. The JSON store
ignores the unknown field. The GORM row no longer maps the
thread_context_off column; AutoMigrate does not drop it and it accepts
NULL, so reads and writes keep working. No data is migrated or removed.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 11:54:33 +00:00
Ettore Di Giacinto 11d1f972ed feat(chats): report the host ceiling with the file limits
The Settings page states the ceiling in effect from this field.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 11:47:51 +00:00
Ettore Di Giacinto 86a93259ce feat(chats): show and set the file limits of a connector binding
The connector list shows, for each binding and each kind, the limit in
effect in both directions, the layer that sets it, the ability the kind
declares and the most an owner may set. A binding accepts
file_receive_max_bytes and file_send_max_bytes; a value above the limit
of the kind is a 400.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 11:47:50 +00:00
Ettore Di Giacinto d676d629a0 feat(studio): filter the chat list by origin
GET /api/chats takes an optional origin parameter: person (chats a
person started), automated (every other chat) or one origin kind such as
trigger or connector. The limit counts the chats kept, so a flood of
chats that a connector or a trigger started cannot push the chats of a
person off a page, and the other way round. Without the parameter the
list is unchanged.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 11:41:13 +00:00
Ettore Di Giacinto 0e50c3f9ca feat(addons): let a connector kind declare per-binding options
A kind can declare options with a few values and a default (manifest
api 2). The host checks the owner's choice against the declaration,
stores it on the binding, shows it in the REST API and sends the value in
effect with every poll. The add-on decides what the option means.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 11:30:11 +00:00
Ettore Di Giacinto f20b7250ad feat(studio): add the thread_context switch to connector bindings
A binding reads the earlier messages of a thread by default. The new
thread_context option turns that off, and the kind list says which kinds
can give a thread's messages. It is stored like send_files: the zero
value keeps the default.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 11:30:11 +00:00
Ettore Di Giacinto fc6abfca8c refactor(studio): name the network from the add-on's declaration
The chat's network view, the administrator view and the line the model
reads about the network of its commands now take their name from the
capabilities.network an add-on declares, or from the title of its
commands tool when it declares none. The sentences no longer say
"Sandbox" or "the sandbox". The API gains a network object with the
title and the preset titles next to the existing addon field.

The text of the built-in bash and of the host-exec setting names an
add-on that provides bash instead of one add-on.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 10:58:46 +00:00
Ettore Di Giacinto c8b757f4a3 refactor(chats): say "the platform" in the connector test instruction
Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 10:47:28 +00:00
Ettore Di Giacinto 61c54dc833 refactor(studio): word the command provider text without naming an add-on
The prompt, the Tools menu reasons, the network error texts, the CLI help,
the API instructions and the assistant tool descriptions said "Sandbox" for
the add-on that runs commands. Core must not know that name.

The reason a chat has no commands now uses the title the provider declares
("the Zeta add-on is off for this chat") or, when none is installed, says
"the add-on that runs commands". Text that has no provider in hand speaks of
the network of commands or of an add-on that isolates them.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 10:43:37 +00:00
Ettore Di Giacinto 0cd265e86e merge: bring master into the proposal UI branch
Union of both case lists in the add-on suite inventory; the swagger
files are regenerated from the merged code.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 10:21:41 +00:00
Ettore Di Giacinto c1876803a3 docs(chats): describe the proposal card, the Proposals page and the Memory tab
Document the optional body of the confirm route, the context-files route,
the card the thread draws for any kind and source, and the Memory tab. The
context-file listing is a person's own action, so the actor scan allows it.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 10:15:03 +00:00
Ettore Di Giacinto 9301fcf8b8 test(addons): cover the execution descriptor in the apps API shape
The apps API spec checks that the generated docs match the emitted
fields, so it needs a descriptor in its full fixture.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 10:07:46 +00:00
Ettore Di Giacinto 9b22862a2d feat(studio): confirm a change with the owner's own text
The proposal card lets the owner correct the text before confirming.
POST /api/chats/changes/{id}/confirm now accepts an optional JSON body
{"content": "..."}. The text has the same size limit and text check as a
proposal, the file and the audit hash hold what was written, and a
refused text leaves the proposal waiting.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 09:47:19 +00:00
Ettore Di Giacinto 2c154c892f feat(studio): list the context files of a user
The Memory settings page needs to know which index and topic files the
host loads into the prompt. GET /api/chats/context-files walks the
registered context sources, expands the agent and channel folders that
exist, adds the agent profiles and reports each file with its size, the
line count of the index, and the caps. It only reads; the page edits and
deletes through the Files API.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 09:45:27 +00:00
Ettore Di Giacinto acba42ae7b feat(studio): add REST for proposed changes
List, read, confirm, dismiss and undo the changes a model proposed, and
read their audit. The routes sit under /api/chats with the other chat
routes and need the chats feature. A person reaches only their own
proposals; another person's id is a 404.

Assisted-by: Claude Code:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-11-03 09:33:44 +00:00