feat(chats): server-side chats with steering, sub-agents and compaction (#12918)

* Find the Sandbox guest image in the executable, not its build info

The Sandbox smoke checked that the packaged executable carries the
guest image by looking for -X main.guestImage in go version -m. The
package is built with -trimpath, and with -trimpath Go leaves -ldflags
out of the build info, so the check failed on every gvisor run,
although the image was linked in.

Look for the reference in the executable instead: the value of
SANDBOX_GUEST_IMAGE when it is set, any digest reference otherwise.
package.sh had the same check for an executable built elsewhere; it
now looks for its own reference the same way.

Assisted-by: Claude:claude-opus-5-5

* Answer a network card the limit refuses with one precise reason

A card that waited while the administrator's limit went down was
refused on the first click with 400, and every later click was a bare
409, so a stale card looked like a conflict loop. A refused card now
answers 400 egress_above_ceiling every time, the same answer sent twice
is one answer (204), and another answer to an answered card is 409
already_answered. The chat view marks a card the limit cannot admit
with above_ceiling.

When the limit is off, or below the chat's preset, the note of the turn
and the block line of the proxy now say that the server's setting
blocks the network and no approval changes it, so the model reports it
instead of waiting or retrying.

Assisted-by: Claude:claude-sonnet-5-5

* Explain a blocked Sandbox network instead of offering dead buttons

A card the limit cannot admit shows why network access is off or
limited, and who can change it, with a link to Settings > Tools >
Sandbox network for administrators. The buttons of a card disable while
an answer is sent, and a failed answer reads the card again. The
Network row says why its choices are capped. Settings opens a group
from ?group=. Strings are translated in all nine locales and the chat
and add-on docs describe the new behaviour.

Assisted-by: Claude:claude-sonnet-5-5

* Add an always-allowed hosts list to the Sandbox network

The extra registries setting only widens the package registries, so a
one-off host needs the approval card in every chat. The new admin
setting chats_egress_allowed_hosts lists exact hosts (optionally
host:port to pin a port) that a chat reaches whenever its effective
preset is registries or all.

The list sits in the same decision as the presets: the ceiling still
caps it, the ports setting still applies, and a private, loopback,
link-local or metadata address still needs the private switch and an
administrator's rule or answer. Each use is audited with the reason
allowed_host. Hosts are validated like the extra registries (lower
case, punycode, no scheme, path or pattern, at most 64) and an IP
literal is refused unless the private switch is on.

get_chat_egress and the chat egress view return the list; no MCP tool
writes it, as for the extra registries.

Assisted-by: Claude:claude-sonnet-5-5

* Let users delete from the Trash for good

A delete only set a tombstone, so the Trash grew without end and
nothing could free the space of a deleted file.

The workspace stores gain a Purger: it erases the deleted nodes of
some change sets, or of a whole folder, with their journal rows, the
change sets that nothing is left in, and the blobs that no row refers
to. Both stores implement it and pass the same contract specs. A blob
is removed after a grace period, and only if it is still unused then,
so an upload of the same bytes in flight keeps them.

DELETE /api/chats/files/trash empties the user's Trash and
DELETE /api/chats/files/trash/{rev} deletes one entry. Both are scoped
to the caller's home. The Trash listing now gives files, folders and
bytes per entry and for the whole Trash. The file tools of a chat
cannot reach any of this.

Assisted-by: Claude:claude-sonnet-5-5

* Add Empty Trash and Delete permanently to the Files page

The Trash tab shows what it holds, with an Empty Trash button that is
off while the Trash is empty and a Delete permanently button on each
item. Both ask in the page first and say how many files and folders and
how much space they remove. Texts are translated in all nine locales.

Assisted-by: Claude:claude-sonnet-5-5

* Fall back to Brave Search and Bing when DuckDuckGo asks for a captcha

DuckDuckGo answers some servers with HTTP 202 and a bot check (the
"anomaly" page) instead of results. The parser read that page as "no
results it could read", so web_search failed for every query.

The keyless search is now a chain: DuckDuckGo, then the result page of
Brave Search, then the result page of Bing. Each page has its own
tolerant parser (title, URL, snippet; Bing's /ck/a links and DDG's uddg
links are unwrapped) with saved fixtures.

A bot check is a typed error (ErrChallenged), and a 403 or 429 is
ErrBlocked. Both wrap ErrProvider. A Fallback with a cool-down skips a
challenged or blocked provider for a few minutes, so each search does
not wait for it first; when every provider is cooling down, all are
asked again. Fallback.SearchAttempts and TestEach report what each
provider did (ok, no_results, challenged, blocked, timeout, error,
skipped).

The error a model reads now lists what was tried and what each provider
did, then where an administrator fixes it.

safehttp keeps the status code of an answer (Response.StatusCode) and
of a refusal (StatusError, still ErrStatus for errors.Is).

Assisted-by: Claude:claude-opus-5-5

* Use the keyless search chain in the chats and test each provider

The built-in web_search now searches with DuckDuckGo, Brave Search and
Bing in turn when no SearXNG instance is set. A new runtime setting,
chats_web_search_providers, sets the order (searxng, duckduckgo,
brave_html, bing_html) and leaves a provider out. Empty keeps the old
choice: SearXNG alone when an instance is set, so its queries do not
reach the public engines unless the administrator lists them.

POST /api/chats/admin/web/test asks every provider of the order and
returns what each did in "providers" (ok, no_results, challenged,
blocked, timeout or error, with the reason), next to the first answer.
Settings > Tools has a Test search on the web_search row that shows it,
and the provider list is checked on the page and by the server.

The cool-down of a challenged provider is kept per frontend, like the
cache and the limits; web.md says why.

Docs: web.md (providers, chain, bot checks, privacy, how to fix it with
SearXNG or the Web add-on) and runtime-settings.md. Strings in the
nine locales. Swagger regenerated.

Assisted-by: Claude:claude-opus-5-5

* Try the mirror library alone in the reinstall recovery spec

The spec adds a second library with the same URL and expects an
install from it to keep the retained data apart. Since two enabled
libraries that list one add-on are a conflict, the reinstall from the
original library then failed with library_conflict, and the install
from the mirror failed for that reason instead of the trusted identity.

Switch the original library off while the mirror is tried, check that
the mirror is refused for its identity and not for a conflict, then
switch the mirror off and the original on for the reinstall.

Assisted-by: Claude:claude-opus-5-5

* Pause a chat turn at the step limit and offer Continue

A chat turn ran at most 10 rounds of tool calls, the default of the
harness, because nothing set a limit. A multi-step task (read some
files, run commands, write a result) stopped half way, and the person
saw an answer that looked final.

The limit is now the runtime setting chats_max_iterations (1 to 500,
default 40, Settings > Tools); a model's agent.max_iterations still
wins. When the tool loop reaches it, the harness sends a paused event
with the limit, and the turn ends with last_turn "paused". A paused
turn counts as finished: waiting messages start the next turn and a
sub-agent reports back. The status line says "Paused, after 40 steps"
and has a Continue button, which sends a message that asks the model
to go on.

A claim that found no waiting message released the chat as "done",
which hid a pause. It now keeps the status the chat had.

Assisted-by: Claude:claude-opus-5-5

* Edit the always-allowed Sandbox hosts in Settings > Tools

Add a list editor for chats_egress_allowed_hosts next to the extra
package registries: the hosts are tags with a remove button, an input
adds one on Enter, comma or Add (a pasted list adds them all), and the
reason an entry is refused shows under the input, with the count out
of 64. The same checks stop Apply on a bad entry, and the server still
has the last word.

The chat's Network panel lists the hosts read only and the row says
how many an administrator always allows while the chat's network is on.
The help text of both settings now says how they differ. Strings are
translated in all nine locales.

Assisted-by: Claude:claude-sonnet-5-5

* Document the always-allowed Sandbox hosts

Describe chats_egress_allowed_hosts in the runtime settings page, the
network section of the add-ons page (including the allowed_host audit
reason and how the list differs from extra registries) and the chats
page.

Assisted-by: Claude:claude-sonnet-5-5

* Let a chat edit, pause and delete the person's triggers

A chat could propose a trigger and list them, but not change one: the
Triggers page and the API could. list_triggers now gives each trigger a
stable ref (not its id) and shows the prompt of one trigger only when
asked. update_trigger and delete_trigger store a change record in the
trigger table: an edit or a delete waits for Apply or Delete on a card,
a pause or a resume applies at once with a receipt. An applied change
can be undone for a day; a delete restores the trigger with its id.

Change records are rows in the database, so every frontend shows and
answers the same card. The new routes under /api/chats/triggers/changes
serve the card and are gated by the triggers feature.

Assisted-by: Claude:claude-opus-5-5

* Show the trigger changes a chat asks for as cards

An edit lists each changed field before and after with Apply and
Dismiss, a delete shows the trigger and its last run with Delete and
Keep, and an applied change is a receipt with Undo. The cards sit below
the message text like the proposal card, and read their state from the
server so a reload shows the same answer.

Assisted-by: Claude:claude-opus-5-5

* Add a Test search button to the web tools settings

POST /api/chats/admin/web/test existed, but no page called it and the
docs showed only curl. The web_search row of Settings > Tools now has
a Test search button. It runs one search with the applied settings and
shows the provider with its first results, or the error a chat would
read.

Assisted-by: Claude:claude-opus-5-5

* Describe undo per tool call in the chats docs

The docs said the checkpoint line of a turn carries the change set of
the turn. A turn checkpoint is taken only by a chat with a VM sandbox,
which the service never sets up, so that rev is always 0. What ships
is one change set per file tool call, with a files_change card on its
tool line. The docs now say so and how to undo a whole turn.

Grouping a turn's changes into one change set in the file store is
left for later: it touches the file store's revisions and the undo of
shared folders.

Assisted-by: Claude:claude-opus-5-5

* Mount the skill folders read-only for the built-in bash

The built-in pdf skill ships scripts, but the commands saw only the
home and the shared folders, so the model could read the scripts and
not run them. The built-in bash now mounts the three skill layers
read-only, at the names the file tools already use: shared-skills,
shared-builtin-skills and shared-addon-skills. A shared folder named
skills moves to "shared-skills 2", so the two never collide.

A read-only mount with subfolders made every command report its
folders as changes the mount refused, because the host backend lists
every folder of its work tree. A folder alone is no change: both
stores now refuse only files, whiteouts and opaque folders there.

The Sandbox add-on mounts only the home; the docs say to copy a
script there first.

Assisted-by: Claude:claude-opus-5-5

* Find the add-on that runs commands by what it shadows

The chats knew the add-on that runs commands from a per-process cache,
filled with the Sandbox's own tool name the first time a chat listed
its tools. Another add-on that replaces bash did not count, so skills
that need commands showed as unavailable. The admin network view read
the same cache: after a restart it said no Sandbox serves until
someone chatted, each frontend of a cluster could answer differently,
and a removed add-on stayed listed.

The tool that runs commands is now the one that shadows bash, as the
spec asks. sandboxRunTool and the cache are gone. The stand-in of an
add-on that cannot work carries the built-in tools it replaces, and
the admin view asks the add-on host which add-on serves bash now: the
pinned one or the oldest running install, and none when the pinned one
is stopped.

Assisted-by: Claude:claude-opus-5-5

* Show international host names in punycode in the network checks

The card-spoofing defence of the sandbox network asks for host names
in their ASCII form, but CheckHost passed Unicode names through. A
look-alike name reached the consent card as text that reads like the
real site.

CheckHost now converts an international name with the IDNA lookup
profile and refuses one with no valid ASCII form. The conversion runs
before the address checks, so full-width digits that spell a loopback
address are refused as loopback.

Assisted-by: Claude:claude-opus-5-5

* Keep the seeded Studio and Talk pins until the add-ons are installed

The upgrade pins Studio and Talk for existing accounts, but both are
add-ons that an administrator installs later. The sweep marked a pin
whose add-on is not installed as missing and deleted it after 30
days, so an administrator who installed Studio late found the seeded
pins gone.

The sweep now records the first time a seeded add-on is present, in
the table of seed runs or the pins file. A seeded pin keeps no clock
until then, and a mark that an older sweep set is cleared. After the
first install the pin follows the retention rule as before.

Assisted-by: Claude:claude-opus-5-5

* Fix two small labels of the Skills and Backends pages

On the Skills page, the skill of an add-on that is not chosen said
"Replaced by <add-on>", the label of a built-in skill that an add-on
serves. It now says "Not active: <add-on> provides it", as the add-on
skills spec asks.

On the Backends page, Clear filters on an empty Installed search left
the library filter set, so the list stayed empty. It now clears the
library filter too.

Assisted-by: Claude:claude-opus-5-5

* Tell the model when no network card can appear

Each turn told the model that a blocked host asks the user. No card
appears when the ceiling is off or no wider than the chat's preset, in
an unattended run or in a sub-agent, so the model waited for an answer
that could not come.

The line now says that a blocked host is refused and nothing asks the
user in those cases, so the model names the host it needs in its
answer.

Assisted-by: Claude:claude-opus-5-5

* Translate six Backends and preview labels in every locale

check, reinstallBackend, deleteBackend and filterLabel of the Backends
page and the label and failed text of the file preview showed in
English in the eight other locales. They now have translations, and
the i18n baseline goes down to match.

Also fix a comment of the home composer that still named the removed
Canvas chip.

Assisted-by: Claude:claude-opus-5-5

* List the admin routes that have no assistant tool, with the reason

The admin oversight routes of saved agents, triggers and connectors
had no MCP tool and no recorded reason, and the other deliberate skips
lived only in code comments. coverage_test.go now keeps them in
adminRoutesWithoutTool with a reason each, and a spec checks that no
route there is also wrapped by a tool. The contributor checklist names
the map.

The tool catalog of the assistant docs said it matched the real one
but missed most tools added since. It now lists every tool, read-only
and mutating, and the actions that stay in the web interface.

Assisted-by: Claude:claude-opus-5-5

* Close the doc and registry gaps of the chats and add-ons

- /api/agent/* answers 404 with the chats_off reason when the chats
  are off, as /api/chats/* and /api/workspace/* do.
- The add-ons docs get an upgrade section: Studio and Talk are
  add-ons, nothing is preinstalled, and users need the addons feature,
  which the upgrade does not grant. It says how an administrator
  installs them (also from a private library) and grants the feature,
  and carries a paragraph for the release notes.
- The migration page no longer lists fork, edit, the chat settings and
  the token counters as removed.
- Runtime settings document chats_vision_model,
  chats_transcription_model and the skill keys of
  chats_tool_providers.
- The docs show POST /api/chats/files/folders,
  GET /api/chats/files/history and the allow route of a connector
  sender.
- The in-tree specs get amendments where the shipped design differs.
- A route test of the tagged suite still set the removed Runs field.

Assisted-by: Claude:claude-opus-5-5

* Look for the kit empty state in the backends regression spec

The Backends page now draws the shared EmptyState of the kit
(ui-empty), not the old dk-empty block, so the regression spec of the
gallery suite found no empty state after a search with no match.
Check the kit's no-results form and its title.

Assisted-by: Claude:claude-opus-5-5

* Ignore a features request that outlives its page in the add-on stubs

stubAddonRoutes answers /api/features by fetching the real flags and
turning add-ons on. A spec that ends while the page polls, like the
one that checks that a viewer without the feature never reads the
list, left that fetch in flight, and Playwright failed the spec with
"route.fetch: Target page, context or browser has been closed" after
its checks had passed. Abort the request when the fetch fails because
the page is gone.

Assisted-by: Claude:claude-opus-5-5

* Bring the gallery-ui case inventory up to date

The suite lists the node unit cases of addons.test.mjs and the
Playwright specs of addons-gallery.spec.js. The inventory missed the
unit cases added since it was written, and one spec was renamed when
the legacy switch came in ("hub: the Add-ons tab appears only for an
account that holds the feature" is now "hub: under the legacy switch
the Add-ons tab appears ..."). The list comes from a local run of
both parts against the UI test server.

Assisted-by: Claude:claude-opus-5-5

* Read spreadsheets, documents and slides in read_document

read_document said "file.xlsx has no text layer; it may be a scan" for
workbooks that some Python libraries write. Their workbook gives the
sheet as an absolute part name (/xl/worksheets/sheet1.xml) and the
cells as inline strings. The text extractor of the harness joined
"xl/" with that name, found no sheet, and returned no text; the tool
then took any empty result for a scanned PDF.

The new package core/services/studio/extract has one entry point,
Extract(ctx, name, data), that returns the text and the structure it
read, for read_document now and for a file indexer later. Office files
go through the files preview reader, which resolves both kinds of part
names and reads inline strings:

- a workbook comes sheet by sheet, rows as tab-separated lines, up to
  2000 rows and 50 columns a sheet, with a note for a cut sheet;
- a Word document gives headings, list items, paragraphs and tables;
- a presentation gives each slide with its title, text and notes.

PDF and EPUB keep the harness extractor. Only a PDF with no text is
reported as a probable scan; an empty Office file is reported as empty.
The preview reader now keeps a Word table as one block with its rows,
and the preview pane draws it as a table.

Assisted-by: Claude:claude-opus-5-5

* Add the missing explorer, library and tools translations

Eight locales had no explorer.json, library.json or tools.json. The
browser asks the server for every namespace of the language it shows,
so an Italian page that used the library answered 404 for
/locales/it/library.json on every load and fell back to English.

The 24 files now exist, translated, with every key of English. The
locale gate counted a missing file as missing keys, which the baseline
allowed. It now fails on a missing namespace file whatever the
baseline says, and the baseline is 24 entries shorter.

Assisted-by: Claude:claude-opus-5-5

* Back off while the add-on service is unavailable

A page that followed an add-on upgrade read the operation every two
seconds, up to 300 times. While the add-on service restarted, each read
waited for its timeout and answered 503, so the log of one session had
48 failed dispatches in a row, and the page only showed the upgrade as
running. The Sandbox state of a chat's Tools menu read once and, on a
503, said nothing more.

The reads of an operation now back off while the service does not
answer: 2, 4, 8, then up to 30 seconds. The upgrade entry says that it
waits for the add-on service and when it checks again, in Activity and
in the details, which also get a Check now button. The Tools menu says
that the Sandbox is starting and reads its state again with the same
back-off, kept for the page, so opening another chat does not ask
again before the wait is over.

Assisted-by: Claude:claude-opus-5-5

* Start a chat with its switches and list them before it exists

The composer of a new chat could not show the Tools and Skills menus,
because the lists and the switches belong to a chat. A chat started
from Home therefore always began with every tool and skill on.

- POST /api/chats takes switches: the add-ons, MCP servers and
  connectors, built-in groups and skills that are off, and the opt-in
  groups that are on. They are checked as the PUT endpoints check them
  and stored with the new record, so the chat never exists without
  them; a bad entry refuses the request and creates nothing. The SQL
  store now writes these lists on create too.
- GET /api/chats/draft/tools and /api/chats/draft/addons list what a
  chat on a model, with a saved agent, could use, every switch in its
  default state. They refuse what create refuses: an unknown or
  forbidden model, an agent without the agents feature, another
  user's agent.

Document both in the chats page and regenerate the swagger.

Assisted-by: Claude:claude-opus-5-5

* Give Home the composer of the chat page

Home drew a smaller message box than the chat page: no files, no
agent, no Tools or Skills menus, no paste to attach, no rich model
list, and its text was lost on a reload. A chat started there could
not be set up before its first message.

Move the composer wiring out of the chat page into one place that
both pages use:

- useChatComposer holds the draft (text and files), the model and the
  saved agent of a new chat, the chat models, the switches and what
  attaching and pasting do.
- ChatComposer draws the model list (fit, vision, memory bar), the
  agent picker, the Tools and Skills menus, the attach buttons,
  dictation, the add-on slot and the vision notice over HomeComposer.
- useChatSwitches has a draft mode: before the chat exists it lists
  from the draft endpoints, a switch changes only the screen, and the
  choices go with the create request. They outlive a change of model
  or agent.

Home now creates the chat with the model, the agent and the drafted
switches, sends the first message with its files and opens the chat.
When the message cannot go, the new chat is removed and the text and
files stay on Home. Its text is kept in the browser across a reload.
Home also gets the shortcuts row, the notices of the chat page and the
line that says whether the model is loaded. Its own slash actions,
widgets, examples and first-run steps stay.

The empty chat page gets the same draft menus. On a phone the chips
of both wrap to a second row, and the Tools and agent menus open as a
sheet from the bottom, as the skills picker does.

Assisted-by: Claude:claude-opus-5-5

* Move add-on settings to the add-on's own page

The settings.sections slot drew every add-on's settings at the end of the
global Settings page, and the Sandbox and Web sections were unstyled: the
add-on UIs used class names the host does not define (btn, btn-primary)
and bare inputs, the SDK offered no form primitives, its muted token
named a property that does not exist, and the slot root was an
inline-flex span that laid a form out in a row.

An add-on's settings sections now render on the Settings tab of its
details on the Add-ons page (/app/addons?detail=<id>&view=settings),
narrowed to that add-on, for administrators only. An add-on page gets a
host Settings link to that tab. The Settings page no longer draws the
slot: an Add-ons group, shown when an add-on has settings, lists those
add-ons with a link each. The old #ad-section-<addon>-<id> anchor
redirects to the tab once the apps list is read. Server settings of an
add-on's feature stay on the Settings page; the tab shows them under
"Server settings for this add-on", with a read-only summary of the
Sandbox network and links to ?group=tools#settings-egress and
#settings-builtin-tools.

The SDK gets a settings kit in sdk.ui (Section, Field, Select,
TextInput, Switch, Choice, StatusChip, Callout, List/ListRow, Code,
Actions, Table, ExternalLink), drawn with host classes and tokens, and
more theme tokens. The Sandbox and Web sections are rebuilt with it:
backend rows with a status chip and the raw reason collapsed in a code
block, a backend choice with one line per option, and the AppArmor and
user-namespace hint (with a docs link) only for the gVisor
"exit status 128 ... cannot read client sync file" failure.

The Settings page showed "Changed, default 30, Reset" next to a 0 for the
audit retention, and the same for the Sandbox network limit, preset and
ports: the server stores 0 or an empty value for a setting never set.
Those fields now read that value as their default, and the retention
input has its unit and bounds.

Assisted-by: Claude:claude-opus-5-5

* Adapt the add-on runtime specs to the new navigation

Three specs of the esm-ui suite still assumed the old shell:

- the Add-ons tab that lists the pages add-ons declare is part of the
  legacy layout, so the navigation spec found no hub subnav; it now
  sets the legacy switch on the server's flags for that page;
- the sidebar has no "Chat" link any more, so the unsaved-edits spec
  could not leave the page; it leaves through Home in the primary
  navigation;
- the shell keeps a notification stream open, so the network never
  goes idle and the spec for a member without the feature waited out
  its timeout; it waits for the server's switches, the account and
  the shell instead.

Assisted-by: Claude:claude-opus-5-5

* Name the legacy Add-ons tab unit case as the esm unit tests do

The esm-ui inventory listed "unit: registry: the Add-ons tab lists
page links only when the registry has usable add-ons", but the unit
test is named "the legacy Add-ons tab lists ...", so the suite could
never match its inventory. Use the name the test prints.

Assisted-by: Claude:claude-opus-5-5

* Say on the Collections page when the embedding model is missing

At startup the server opened a collection and its first embedding call
got 404 from POST /embeddings. The collections embed through this
server's own API with collections_embedding_model, the default
granite-embedding-107m-multilingual, and that model was not installed.
Nothing told the user: uploads and searches just failed.

GET /api/collections now also returns embedding_model and
embedding_model_installed (a loaded configuration of that name, or a
model file of that name). When it is missing, the Collections page
says "No embedding model installed", names the model, and gives an
administrator a button that opens the Models page with a search for
it.

Assisted-by: Claude:claude-opus-5-5

* Report modified times and the upload limit in folder listings

The Files page could not sort by date or show when a file changed: a
folder listing had no time. It also learned the upload limit only after
it had sent the whole file. The default server body limit is 15 MB,
lower than the 256 MiB a workspace file may have, so a large upload
failed late and without a clear reason.

Each listed entry now has "modified", the time of the change set of
its revision. Both workspace stores fill it in List: the JSON store
from its in-memory change sets, the database store with one query per
batch of revisions. The listing also has "max_file_size", the smaller
of the workspace limit and the server body limit, so a client can
refuse a larger file before it sends any byte.

The store contract suite checks the new time against the history, and
the route tests check both fields.

Assisted-by: Claude:claude-opus-5-5

* Add a sortable file list with thumbnails for the Files page

The building blocks of the new file browser, used by the page in a
later change:

- fileBrowser.js: the kind of an entry from its extension, the order
  (folders first, natural and case-insensitive names, then size,
  modified time or type), the filter of a folder, dates in words with
  the full date for a tooltip, the Shift range, the arrow moves of a
  list and of a grid, type-to-find, names that do not clash, why a
  name cannot be used, the name of a pasted screenshot and the view
  settings read back from storage.
- FileList: a grid with sortable column headers or a grid of tiles
  with image thumbnails, a check box per row, a row menu, inline
  rename with the reason a name is refused, rows that can be dragged
  onto folders, and the keyboard: arrows, Shift, Space, Enter, F2,
  Delete, Ctrl+A, Escape, Backspace and typing a name.
- MoveDialog: a folder picker that walks the tree and refuses a move
  into the moving folder itself.
- ActionMenu can open at a point with no trigger (a right click), and
  its trigger can take another icon and class (a split button).
- fileSize also writes gigabytes.

Assisted-by: Claude:claude-opus-5-5

* Add an upload queue with progress, cancel and retry

An upload of the Files page was one fetch per file in a loop: nothing
showed while it ran, one failure stopped the rest, and leaving the
page lost it.

- uploadQueue.js keeps every upload outside React: queued, sending
  with the bytes sent and a smoothed rate, done, failed with a reason
  and whether Try again can help, or canceled. Two files go at once.
  It lives for the whole app, so opening another folder or page does
  not stop an upload.
- filesApi.uploadWithProgress sends the same PUT with XMLHttpRequest,
  which reports upload progress and can be aborted. The server already
  streams the body into the store; nothing changes there.
- useUploads reads the queue and asks before the tab closes while a
  file is on its way.
- UploadTray shows the queue at the bottom of the page: the total and
  the time left, each file with its progress, speed and time left,
  Cancel, Try again with the reason, and a done state that fades out.
  A live region tells a screen reader when uploads start, fail and
  end.
- ConflictDialog asks what to do when an upload has the name of a file
  in the folder: replace it, keep both, or skip it.
- dropFiles.js reads dropped folders with their paths.

Assisted-by: Claude:claude-opus-5-5

* Rebuild the Files page as a file browser

The Files page was a list of links: the store's byte order, the same
icon for every file, three icon buttons on every row, no progress on
an upload, no selection, and on a phone a preview that opened below
the whole list.

Browsing: a breadcrumb that folds a deep path, sortable columns (name,
modified, type, size) with folders first, a list or a grid of
thumbnails, both kept per user in the browser, a filter of the open
folder with a search of every file on Enter, type icons, human sizes,
dates in words with the full date on hover, and Back and Forward
between folders.

Uploads go through the upload queue and its tray: the Upload button
or its folder menu, drops of files and folders anywhere on the page or
onto a folder row or crumb, and paste. A clash asks to replace, keep
both or skip; a file over the server limit is refused before it is
sent.

Selection and actions: click, Ctrl and Shift, check boxes, a selection
bar (Download, Move to, Move to Trash; at the bottom of the screen on
a phone), a row menu and a right click, inline rename and new
folder or file, drag to move, the keyboard, and a toast with Undo for
a delete, a move and a rename.

States: a skeleton on the first load, an error with Try again, empty
states that say how files get here, and a no-match state. The preview
uses the side pane: docked on a wide screen, a sheet on a phone, with
the same actions as the row menu. Trash, History and the Managed by
LocalAI group stay as they were.

Every new string is in the nine languages. The page uses design
tokens only (seven raw font sizes fewer). The specs cover sorting,
the filter, the views, selection and bulk actions, the keyboard, the
menu, rename checks, drag to move, drops, paste, clashes, the size
limit, upload progress, cancel and retry against the real server, and
the error states.

Assisted-by: Claude:claude-opus-5-5

* Fix two specs that fail on the integration branch

The Describe spec of the egress package checked that the no-card line
does not contain "asks the user", but that line says "nothing asks the
user". It now checks for the sentence that promises a card.

The context meter spec gave the chat a 4k context window. The tool
schemas of the chat now take 3k of it, so the turn stopped before the
model request. The spec uses a 16k window.

Assisted-by: Claude:claude-opus-5-5

* Translate the always-allowed hosts note of the chat network row

The three egress keys of the always-allowed hosts were only in English,
so the locale check failed for every chats namespace. The workspace
namespaces now miss two keys fewer, so their baseline goes down too.

Assisted-by: Claude:claude-opus-5-5

* Bring the contracts and host-slots case inventories up to date

The contracts suite has eight new specs for the check of a tool that
shadows a built-in. The host-slots suite runs the unit case of the file
preview slot, which the inventory did not list.

Assisted-by: Claude:claude-opus-5-5

* Add the design of the file text index

The search of the chats' files reads only small UTF-8 text files, so a
PDF or an Office file is found by name only. The design extracts the
text when a file is written, keeps it in LocalAI's own store keyed by
node and hash, checks access at query time and runs one indexer per
cluster under an advisory lock.

Assisted-by: Claude:claude-opus-5-5

* Give extracted documents their pages, sheets and slides

Extract now says where each page, sheet and slide starts in the text,
so a search hit can name its page or sheet. A PDF page starts with a
"## Page N" line, as sheets and slides already start with a heading.
A PDF also stops at a page limit (200 by default) and at the text
limit, which it ignored before, and a panic of the reader is an error.

The test fixtures move to extracttest, with a real text PDF and a
workbook with shared strings, so other packages can build the same
documents.

Assisted-by: Claude:claude-opus-5-5

* Add a store for the text of workspace documents

textindex keeps one entry per workspace node: the text read from a
PDF or Office file, the hash it was read from, its pages or sheets and
its status (pending, ready, failed, skipped), plus the cursor of the
indexer in the workspace journal.

The GORM store keeps a lower-cased copy of the text, so SQL and the
Go matcher agree on every script, and searches it with FTS5 and the
trigram tokenizer on SQLite or a pg_trgm index on PostgreSQL. Without
them it scans the column. The JSON store keeps the metadata in memory
and each text in its own file, and scans the texts. One contract
suite runs on JSON, SQLite and PostgreSQL.

Assisted-by: Claude:claude-opus-5-5

* Index the text of new and changed workspace documents

The indexer tails the workspace journal from a cursor kept in the text
store. A write of a PDF, Word, Excel or PowerPoint file marks it
pending (the same hash only follows the path), a delete removes the
entry and a move rewrites the paths below a folder. Then it reads the
pending files two at a time, with a timeout, a size cap, a page cap and
a text cap, and stores ready, failed or skipped. A panic of the reader
fails the file, not the pass. The same content is read once.

The first pass starts at revision 0, which indexes the files that were
there before, in bounded steps that resume from the cursor. Enqueue
marks an upload pending at once and Forget drops the entries of nodes a
purge erased. In a cluster each pass runs under a new advisory lock,
so one frontend reads.

Assisted-by: Claude:claude-opus-5-5

* Search and read the text of documents in the chats' files

search_files and GET /api/chats/files/search now also search the text
index: after the name and text-file hits come lines of PDF, Word, Excel
and PowerPoint files, as path (page 3, line 40): text, with the sheet or
slide where a document has them. Each entry is shown only when the
caller may read its file now and the file still has the content the
text came from. Documents keep up to a third of the hits. When files
below the path are still pending, the answer says so instead of
"Nothing found.", and the REST result has indexing.

read_file returns the text of a PDF or Office file with the usual offset
and limit, from the index when it matches the file and read on the
spot otherwise. An upload and a write of a chat tool queue the file at
once, and emptying the Trash drops the entries of the purged files.

Assisted-by: Claude:claude-opus-5-5

* Start the file text index with the files and add its settings

With files on, the server opens the text store next to the workspace
index (the auth database, or a folder under the data path) and starts
the indexer. A chat worker searches the index but runs no indexer.

chats_index_files (default on) switches the reading off and
chats_index_max_text_bytes (2 MiB by default, 64 KiB to 16 MiB) caps
the text kept for one document. Both apply at the next pass, and both
are rows of Settings > Tools.

Assisted-by: Claude:claude-opus-5-5

* Show the page, sheet or slide of a document hit on the Files page

Search all files now lists hits inside PDF and Office files with their
page, sheet or slide instead of a line number of the extracted text,
and says how many documents are still being indexed. The strings are
in all nine locales.

Assisted-by: Claude:claude-opus-5-5

* Test finding an uploaded PDF by its words through the routes

The PDF has a generated name and is uploaded with the REST upload. The
search route first reports it as being indexed, then finds it on its
page after a pass of the indexer, and so does the search_files tool of
a chat. Another user finds nothing.

Assisted-by: Claude:claude-opus-5-5

* Document the search inside documents

The chats page says what the index reads, where the text is kept, who
finds it, which search engine each database uses, the two settings,
the backfill and the single-runner mode in a cluster, and how
search_files and read_file answer for documents. The API instructions
and the PDF skill say the same in short.

Assisted-by: Claude:claude-opus-5-5

* Test the indexer on the JSON workspace store

A server without a database keeps the workspace and the text in JSON
files. The indexer follows the uploads, a folder move and a delete in
that journal as it does in the database one.

Assisted-by: Claude:claude-opus-5-5

* Read all of a long document with read_file

The index keeps the first chats_index_max_text_bytes of a document, so
read_file used to stop where the index stopped. It now uses the index
only when it holds the whole text, and reads the document again (up to
16 MiB of text) when the index cut it.

Assisted-by: Claude:claude-opus-5-5

* Start bash in the user's home

The Sandbox's bash and the built-in bash started in /workspace, the
folder that holds the mounts, so a model that ran a script by a
relative path failed and a file it saved there was lost. A call now
starts in /workspace/home, the folder of the user's files: a relative
path is the same file the file tools see. The tool descriptions and
the commands block of every turn say where a call starts and that
only what is saved there is kept. The built-in bash says that the
/workspace paths do not exist on the host and that a shared folder is
next to the home.

Assisted-by: Claude:claude-opus-5-5

* Name files in tool results by the path the model uses

write_file, edit_file and search_files named a file of the home by its
path in the home ("/scripts/build.py") while the model had written
/workspace/home/scripts/build.py, and a model took the two spellings
for two files. The results now give the path the model uses, and
move_file says where it moved the file to. The cards and the change
notes keep the path in the home.

Assisted-by: Claude:claude-opus-5-5

* Name the paths a command's commit did not save

The result of a bash call ended with "N file(s) changed in your files
meanwhile; the sandbox's version was kept as a copy" for every path the
commit refused, also for a file written outside the user's files,
where no copy is kept. It now has one line a path, with the path as
the guest sees it, the reason, and the path of the copy when the store
kept one, so the model and the user can act on it. The host returns
the paths and the copy to the Sandbox add-on relative to the user's
files, as its mounts are.

Assisted-by: Claude:claude-opus-5-5

* Show the files a command saved under its step

A document that a script wrote with bash reached the user's files, but
the chat showed nothing: only the file tools put a file on their step,
and the Sandbox committed for no chat, so the chat's file list, the
chips under the answer and the preview pane never saw the file.

The host now commits a Sandbox call for the chat that ran it (the
action resource of the request names the chat). After each bash call,
of the Sandbox or of this machine, the chat reads the change sets of
the chat that the call made from the shared store and puts the files
on the step's card, next to the card that says who served the call
(session.SetToolFiles; files with path, change set and op on the
card's new files field). The step lists them as chips that open the
preview pane, the chat's clients are told that its files changed, and
the files reach the file list, so the answer gets its chips and the
pane opens on the newest one, as for write_file.

Assisted-by: Claude:claude-opus-5-5

* Keep JSON out of the answer of a trigger run

The workspace skill told the model to put the widget data shape and a
source marker in a trigger prompt. Some proposals then asked the run to
print that JSON, and every run ended its chat with a stray line of it.

The skill now says source and note live inside the data file and are
never the answer, and that a prompt must not ask for JSON. The runner
appends one closing instruction to the first message of every trigger
run: finish with one short plain sentence. The propose_trigger and
update_trigger descriptions say the prompt must stand alone and must
not ask for JSON. Connector chats are talks with people and do not get
the line.

Assisted-by: Claude:claude-sonnet-5-5

* Read the images of a page as base64 for its frame

The bridge of a page reads UTF-8 text only, so the UI could not put a
png or a jpg of the page's folder into the page. A read of the page
file routes with encoding=base64 now returns an image as base64 with
its media type. Only the extensions of an image are read this way
(png, jpg, gif, webp, avif, bmp, ico, svg); any other file is 415, so
this is no way to read an arbitrary binary file. The path checks, the
1 MiB limit, the viewer's rights and, for a shared page, the consent
check are the same as for a text read.

Assisted-by: Claude:claude-opus-5-5

* Let a page use the files next to it

A page runs from a srcdoc with no network, so fetch('news.json'),
<script src="app.js">, <link rel="stylesheet" href="style.css">,
<img src="logo.png"> and import './x.js' all failed without a word.
Models that split a page into files, or load its data with fetch,
made pages that showed nothing.

Before the frame loads, the UI now reads the files that the page names
with relative URLs, with the same calls as the bridge, and puts them in:
classic scripts in place (deferred ones at the end of the body), style
sheets as <style> with their @import and url(), images as data: URLs,
and ES modules as blob: URLs behind an import map, so cycles and
import() work. A page with modules gets blob: in script-src; no other
part of the policy changes. Inside the frame, fetch and XMLHttpRequest
read relative URLs with files.read and answer with a real Response (200,
404 for a missing file, 413, 415); any other URL rejects with "network
is off in pages; put the data in a file next to the page".

Every path is resolved inside the page's folder: .., %2e%2e, absolute
paths, backslashes and protocol-relative URLs are refused, and the
query and the fragment are dropped. The files go in only after a shared
page is allowed, with the viewer's rights. The files that go in are at
most 4 MiB, images 2 MiB; a script or style sheet that does not load
shows a notice in the page. A change of one of these files reloads the
page, as a change of index.html does.

The skills, the docs and the shared pages fixture say and test the same.

Assisted-by: Claude:claude-opus-5-5

* Return tool descriptions from an MCP connector test

A connection test listed only the tool names. A person who adds a
server cannot tell from a name such as "query" what the tool does or
whether to keep the connector on.

The test response now also has tool_details: each tool with the
description its server gives, in the order of tools. tools is
unchanged, so existing clients keep working.

Assisted-by: Claude:claude-opus-5-5

* Add jq, ripgrep, sqlite3 and archive tools to the Sandbox guest

The guest user cannot install packages and apt is blocked by the proxy,
so any tool the model needs has to be in the image. Add jq, ripgrep,
sqlite3, zip, unzip, file and tree to the final stage.

A new check-tools.sh runs in the existing build step and fails the build
when one of these programs, Poppler, git or curl is missing or does not
start.

The list is one constant, GuestTools, in the shared sandbox package. The
Commands block of the prompt of an isolated chat names it. The tool
description (capped at 400 characters, already full) and the turn note
(cut at 400 characters) have no room for it. Tests keep the Dockerfile,
the check script and the Sandbox documentation in line with the constant.

Assisted-by: Claude:claude-sonnet-5-5

* Keep the helper files of a chat in its work folder

A chat asked for a Word document left a 19 KB build script at the top of
the user's home, next to the user's own files. Chats like it leave
scripts, intermediate data, logs and drafts at the top level or in
folders such as scripts/, and nothing tells them apart from what the
user asked for.

Each chat now has a work folder, Chats/<chat folder>/work/ in the
user's home, next to its attachments. The chat takes the name of its
folder at its first turn (checked against the folders and the other
recent chats of the user, so two chats with one title get two names)
and makes the folder only when it is used: before a bash command runs,
or when a file is saved there. The prompt of every turn names the
folder and says what goes in it: scripts, helper code, intermediate
data, logs and drafts go there; the deliverable goes where the user
said or at the top of the home; no new top-level folders for scripts;
a run-once command needs no file. A sub-agent is told its parent's
folder. The descriptions of bash (the Sandbox's, within its 400
characters, and the built-in one) and of write_file end with one short
hint, and the built-in office, apps and workspace skills say the same
in one line.

The steering is soft. When write_file, move_file or a command leaves a
helper file outside the work folder, the result gets one line that
suggests moving it, at most once a turn of a chat.

One function (workspace/workfiles) decides the kind of a file:
working for anything in a chat's work folder, or for a script, a log
or a temporary file by its extension or a folder named scripts, tmp or
scratch; deliverable for everything else, and always for pages, the
LocalAI folder, the shared folders and the files the user attached.
The chat's file list (kind), the files of a command's card
(files[].kind) and the card of a file tool (file_kind) carry it, so the
interface only renders it.

The runtime setting chats_work_folder (on by default) turns the
steering off.

Assisted-by: Claude:claude-opus-5-5

* Stop telling the model that the Sandbox has npm

The guest image has no Node, so the network hints named a tool that
does not exist.

Assisted-by: Claude:claude-sonnet-5-5

* Update the Sandbox description test after dropping npm

Assisted-by: Claude:claude-sonnet-5-5

* Open the links of a page after the person confirms them

A page runs in a frame without allow-popups or top navigation, so a
click on <a href="https://..."> did nothing, and a link to #id loaded
LocalAI's own address in the frame, since a srcdoc resolves it against
the parent. A news page had only dead links.

The bridge of the frame now takes the clicks on links (also with a
modifier, the middle button or Enter) and window.open(), and sends
link.open(url) to the host. The host lets through only an http or https
address without a user name or password, at most 2048 characters;
javascript:, data:, file:, blob:, //host and links to other files of
the page are refused with a message in the console. A bar above the
page shows the site in large type, the whole address, and Open and
Cancel. A host in punycode also says how it reads, since it may imitate
another site. Open is a click in the app, so the new tab is not blocked
and opens with noopener and noreferrer. A page asks at most 5 links at
once, then one every 3 seconds. A link to #id scrolls the page.

The sandbox, the policy of the frame and the consent of shared pages do
not change; a shared page asks the same way.

Assisted-by: Claude:claude-opus-5-5

* Let the items of a Home widget link

A list widget showed each item as plain text, so a digest of articles
or a list of pages could not take the person anywhere. An item now has
an optional url. An http or https address without a user name opens in
a new tab with noopener and noreferrer, and its tooltip names the site;
a path of the app (/app/pages, /app/chats, /app/files and the other
screens a person reads) opens inside LocalAI. The label is the link,
with a focus ring, an underline on hover and focus, and an icon for an
address outside the app.

The widget-data schema checks the url, and the server adds an error in
words that names the item and the rule, so a trigger run that writes a
bad link learns why. The skill and the docs show the field.

Assisted-by: Claude:claude-opus-5-5

* Attach any file in a chat with one button

The chat had three attach buttons (image, audio, file) that all add to
the same list, and the file button's dialog filtered on */*, which
some file dialogs read as a type filter that greys out archives such as
.zip. A chat now shows one Attach file button whose dialog has no type
filter. Composers without anyFile keep the three buttons.

Assisted-by: Claude:claude-opus-5-5

* Accept both Node behaviours for a broken punycode label in the link test

Node 22 refuses the address, Node 24 parses it. Both mean the page never
opens it unwarned.

Assisted-by: Claude:claude-sonnet-5-5

* Show a chat's working files apart from what it made

The Files pane, the steps and the preview treated a helper script like
the document the user asked for: a 19 KB build script was listed next
to the document, got a chip under the answer, and the preview opened on
whichever file was newest.

The interface now renders the kind the server puts on each file. The
pane lists what the chat made first and folds the working files into
one quiet heading under the rest, "Working files · 2", with a line that
says what they are; opening it shows their rows, which preview like any
file. A working file opened from a step unfolds the group. The preview
that opens by itself after a turn picks the newest file that is not a
working file, and the chips under the answer name only those files. A
step that saved working files lists them as one line, "Also saved 2
working files", that opens to their chips, and the file of a write_file
step that is a working file is a quieter chip. The files the user
attached are unchanged.

Assisted-by: Claude:claude-opus-5-5

* Add a search field and a segmented filter to the UI kit

The Agents views each drew their own toolbar: a search box capped at
320 px on the agents list, round chip filters on the triggers history,
and a bare checkbox on the tasks page. Add-ons already had the look the
product wants (a search that fills the row and a segmented tray), but
only as add-on classes.

SearchField and Segmented are that toolbar as two small primitives with
ui-* classes and token values only. Segmented buttons carry their id in
data-filter, so specs can press them. The README shows how to use them.

Assisted-by: Claude:claude-opus-5-5

* Draw an agent task run as a history row with its output

The jobs of an agent task were table rows that opened to bare <pre>
blocks: no copy, parameters printed as k=v, the traces in a raw
<details>, and "by manual" in the server's words.

RunItem is one run of a history list: a toggle, the state chip, what it
was and what happened, when and for how long, small actions, and the
details that open under it. OutputBlock draws an answer or an error in
mono with a Copy button. JobRow and JobDetail use them for a job: the
values it ran with as labelled chips, the answer or error, the steps,
the webhook outcome and a link to the chat that holds the run. Times
show as words with the full date as a tooltip.

The agent task helpers gain the pure rules the page needs: the jobs of
a task, the run filters (waiting and running count as in progress), the
on/off filter and search of the task list, the field errors of a save,
how a job started, and the API call that runs a task by name.

Assisted-by: Claude:claude-opus-5-5

* Create, edit and run agent tasks in a side sheet

The task form opened as bare fields above the list, with a checkbox for
On and a Save button that was only disabled when a field was missing.
Run with parameters was a second inline form in the same place.

TaskSheet is a side sheet with three labelled groups (about, who runs
it, what it does), help under each field, the parameters the prompt
asks for as chips while you type, and an On switch. Save checks the
required fields, says next to each one what is missing, and moves the
focus to the first. RunSheet asks for each value the prompt names and
shows the prompt as the task will receive it.

Assisted-by: Claude:claude-opus-5-5

* Show the details of an agent task in a pane with tabs

A task had no details view: its prompt, its runs and the way to run it
over the API were only in the lede of the page.

TaskPane is the kit's side pane with three tabs. Overview shows who
runs the task, the values it asks for, the prompt and a curl call that
runs it by name, with Copy. Runs lists its jobs. Settings holds the On
switch, the webhook count and Delete. Run and Edit stay in the footer.
The tabs move with the arrow keys, Home and End.

Assisted-by: Claude:claude-opus-5-5

* Rebuild the agent tasks page from the shared list pieces

The tasks page looked unstyled next to Add-ons and the agents list: two
raw tables under uppercase eyebrows, a bare checkbox for All users,
forms that opened above the list, three text buttons on every row, and
on a phone a table that broke names one syllable per line.

The page now uses the same pieces as the polished pages. The header has
one line and New task. A toolbar holds the search, an All/On/Off filter
and, for an administrator, an All users switch. Each task is an agent
row: a tile, the name, two lines of description, who runs it and what
it asks for, the last run's state and time with the run strip, an On
switch, Run and a menu. A task with no parameters runs at once. The
name opens the details pane; New task, Edit and Run open the sheets.
The run history is grouped by day, with a state filter, a task filter
and Show more. A job is deleted after an in-page question. With no
tasks the page explains what one is, with an example; a failed load
shows the error with Try again. On a phone the controls grow to 44 px
and the filters scroll sideways.

New strings, and the task strings that existed only in English, are in
all nine locales; the strings the page no longer uses are gone. A new
stubbed spec covers the list, filters, sheets, pane and history, and the
real-server spec follows the new controls. The docs describe the page.

Assisted-by: Claude:claude-opus-5-5

* Rebuild the MCP connectors page on the Add-ons patterns

The page read as a form: every connector was a row of raw controls (a
checkbox, a Test button, red Delete text), the add form pushed the list
down, a test result vanished on reload, and the frame was narrower than
the pages next to it in the sidebar.

The page now uses the same blocks as Add-ons and Files:

- Full page width and the shared PageHeader with a one-sentence purpose
  and "Add connector".
- The Add-ons toolbar (search and a state filter with counts) from four
  connectors, and the Add-ons rows: a monogram tile, a state chip
  (Connected, Needs sign-in, Can't connect, Not tested, Off), one line
  with the tools or the reason, a meta line, the kit Switch and a menu.
- Details in the shared SidePane: the switch, the last test, the tools
  with their descriptions and a search, the connection with the query of
  the address hidden, and what is saved.
- Add and edit in the shared SideSheet: kind first, grouped fields with
  inline checks, masked secrets with show and hide, then "Save and
  test", the result with the tools, and Turn on. Escape and Cancel ask
  before they drop typed changes.
- An empty state that says what a connector is for.

The last test of each connector is kept in this browser with a hash of
what it tested, never the address or a secret, so the page shows the
last known state after a reload. Strings are in all nine locales.

Assisted-by: Claude:claude-opus-5-5

* Document the MCP connectors page and its states

The MCP servers section now describes the add sheet (kind, checks,
masked secrets, Save and test, Turn on), the states a connector shows,
where the last test is kept, the search and the filter, and the details
pane. The test example shows tool_details.

Assisted-by: Claude:claude-opus-5-5

* Render a preloaded page without its loading placeholder

A page that was preloaded still showed the route's loading placeholder
when it opened: React.lazy suspends once on its first render even when
the chunk is already in memory, and React keeps a fallback on screen
for at least 300 ms once it shows one. Opening a chat from Home showed
a blank page for that time.

The page loader now remembers the module once it has loaded and hands
it to React.lazy as a thenable that resolves at once, so a preloaded
page renders in the same commit. A preload of a segment now warms
every page registered under it: /app/chats is the chat page, its
connectors and its conversion page, and before only the last one
registered was preloaded.

Assisted-by: Claude:claude-opus-5-5

* Move the composer from Home into the new chat in one gesture

Sending the first message on Home was a hard cut. The page stood still
while the chat was created, then showed a blank placeholder, the empty
"new chat" screen with its composer in the middle, an empty thread
with the composer at the bottom, and only then the message, while the
sidebar folded and moved the whole column. Focus was lost.

Now Send fades the rest of Home and keeps the composer in place while
the message goes. When the send has answered, a view transition moves
the composer to the dock of the chat, lifts the text out of it into
the message, moves the attached files with it and folds the sidebar in
place; the model's indicator comes in after the message has landed and
the empty composer has the focus. The chat page draws the message from
what Home already knows (a small hand-off store: the chat record, the
text and the files, with the local copy of an image for its thumbnail)
and keeps the same row when the server's transcript replaces it, so
nothing is drawn twice or animates again. What is sent to the server
does not change.

A browser without view transitions gets a FLIP slide of the composer
and the message (Web Animations, transform and opacity only). Less
motion gets a short cross-fade, or an instant switch. A send that fails
brings Home back, shakes the composer and keeps the draft; a second
Enter while one send is in flight sends nothing. The draft is cleared
when Home closes, so the box never shows its placeholder on the way.

Focus mode folds the sidebar in a layout effect, so the fold happens in
the frame the conversation appears. One motion token is added,
--dur-glide (360 ms), for a part that travels to its place on the next
page.

Assisted-by: Claude:claude-opus-5-5

* Draw the triggers as rows and their history like the task runs

The triggers page used a table with a switch column next to the agents
card list, folded its list behind Hide, filtered the history with round
chips, and on a phone cut each history row at the right edge.

Each trigger is now an agent row: a clock or folder tile, the name, the
schedule in the server's words, the agent or model and when it runs
next, the last run's state and time with the run strip, an On switch,
Run now and a menu. The week sentence sits under the Run history title.
The history uses the segmented filter and the same run rows and day
headings as the agent tasks, with Show more after 30 runs; an open run
shows its reason with Copy. On a trigger page the prompt comes before
the runs, in the body face, the On switch is the kit switch, and the
runs stop at 10 with Show more.

Assisted-by: Claude:claude-opus-5-5

* Use the shared search and the full width on the agents list

The agents list stopped its search at 320 px and used the content width,
while the Triggers and Tasks views next to it use the full width, so the
title and the view tabs moved when you switched views.

The list now uses SearchField, which fills the toolbar row as on
Add-ons, and the full page width like its sibling views.

Assisted-by: Claude:claude-opus-5-5

* Polish the agent task sheets and pane after a visual review

Screenshots of every state showed a few rough edges:

- The name placeholder looked like a filled-in value next to the error
  "Give the task a name". The name field has no placeholder now, and
  the prompt placeholder starts with "For example:".
- A task run by an agent showed the agent id until the agents were
  read. It says "Agent" until then.
- A run opened in the details pane also opened in the history, and the
  two copies shared an element id. The pane keeps its own open runs and
  ids.
- A failed run in the pane drew its meta line in red; the title carries
  the error colour there now.
- A group title that is not a legend had a double gap under it, and the
  kebab menus were under 44 px on a phone.

Assisted-by: Claude:claude-opus-5-5

* Cover the code files of a shared page in its consent

The consent to a shared page was kept against the hash of index.html
and page.json. The UI also puts the scripts, style sheets and modules
next to the page into it before it runs, so the author could change
app.js after a viewer allowed the page and run new code without asking
again.

The server now finds the same files as the UI's inliner: every script
src, stylesheet link, relative CSS @import and module import of the
folder, followed through the module graph. Their content hashes go into
the hash of the page, and a file the page names that is missing counts
too, so adding it later also asks again. Data files and images stay
out, so saving data does not ask again. The files are listed as
code_files. What a file names is kept per content hash in a small
per-frontend cache, which only saves reads.

Two more checks close the gaps around it. A frame of an older version
that reads a changed file of code gets 409, so a script edited while
the page is open never runs under the old consent. The UI puts no text
file into a shared page that is not in code_files, so if the server and
the UI ever disagree, the file fails to load instead of running outside
the consent. The trust sheet names the files of code when only one of
them changed.

Assisted-by: Claude:claude-opus-5-5

* Let a page be a tile on Home in its compact mode

Home widgets could show a list, a number or the last chats, but not one
of the pages a person or the assistant built. A page can now be a tile.

A page opts in with a widget block in page.json: a size (small, medium
or large, the height of the tile) and an optional short title. A
widget of type page in LocalAI/home.json names the page, and a folder
id for a page of a shared folder. The server takes at most three page
widgets and says why in words, and refuses a name that is not a page
name.

The tile runs the page in the same sandbox and bridge as the full
view. The page reads localai.page.mode ("widget" in a tile, "page"
otherwise) and localai.page.size, and <html> carries data-mode and
data-size, so the page decides how its tile looks instead of being
shrunk. The tile reserves the height of its size, builds its frame only
once it scrolls into view, tells the page through a new visibility
event when it goes out of sight or the tab is hidden, takes the frame
down far off screen, and keeps at most three frames alive. It has a
title bar with Open page and a refresh button, a skeleton while it
loads, Retry on an error, and the link question of the full view. A
page that is gone or has no widget block shows a calm line with a link
to Customise Home; a shared page that the viewer has not allowed shows
Allow this page, which opens the same trust sheet.

Customise Home lists the pages with a widget block, with the size of
their tile. The task board starter has a compact view, the skills tell
the assistant how to put a page on Home, and the docs describe it.

Assisted-by: Claude:claude-opus-5-5

* Give the Files upload its own size limit

The Files page upload was capped by the general --upload-limit (15 MB
by default), although a workspace file can be 256 MiB.

Make 0 the default of --upload-limit, meaning not set. When it is not
set, the general routes keep the 15 MB limit and the Files upload route
skips it, so the workspace file size is its limit. When the operator
sets a value, it applies to the Files upload too, lower or higher (a
file never exceeds the workspace size). The max_file_size of a folder
listing reports the limit in force.

Assisted-by: Claude:claude-sonnet-5-5

* Refuse writes of code files from a page

A page loads its own scripts, modules and styles, so a page that could
overwrite app.js or style.css would run new code on its next load. The
page bridge now writes data files only.

The server refuses a write whose final segment has a code or document
extension (js, mjs, cjs, css, html, htm, svg, xml, xhtml, wasm), is named
index, or is page.json, with 403 and a clear message. The check applies
to the page routes only; the Files page and the assistant's file tools
keep full write access. The judgement lives in pages.IsCodeFile and is
mirrored by isCodeFile in the web host, which refuses early with the
error code forbidden. Both run one table of cases, covering case, double
extensions, trailing dots and spaces, percent-encoding, backslashes and
lookalike dots.

Assisted-by: Claude:claude-sonnet-5-5

* Make the trigger tools a built-in group a chat can switch off

propose_trigger, list_triggers, update_trigger and delete_trigger were
added to every chat a person starts, with no switch. They are now four
entries of the built-in tool registry in one group, triggers, so the
Tools menu, the tools panel, the draft list and the builtins switch
endpoint handle them like the web group. The group is on by default.

A trigger or connector run and a sub-agent still never get them. The
panel lists the group as not usable, with a reason, for those chats,
for a user without the triggers feature, and not at all when the server
runs no triggers. The group has no runtime setting, so the registry
gets an Unlisted flag and Settings > Tools skips it. The panel now
builds every grouped tool to learn whether it is usable, not only the
opt-in ones.

Assisted-by: Claude:claude-sonnet-5-5

* Show the Triggers group in the chat Tools menu

The menu rows come from the groups the server lists, so the row needs
only its label in the nine locales. A Playwright spec covers the row,
its switch request, Select all and Disable all, and the disabled row
with its reason.

Assisted-by: Claude:claude-sonnet-5-5

* Talk about settings, not the files behind them

The settings cards, the shortcuts row, Home and the change notes in a
chat named the files that hold a person's settings: "Defined in
LocalAI/shortcuts.json", "Layout is in LocalAI/home.json", "Saved
LocalAI/persona.json". A person changes their shortcuts or their Home,
not a file, and the Files page still shows the files to whoever looks.

The "Stored in", "Defined in" and "Layout is in" lines are gone; the
history line keeps Versions, Revert and the Invalid badge. The notes
say what changed ("Saved your shortcuts.", "Created the page Task
board."), the errors name the setting or the widget instead of a path,
and the Source view says "The settings as text" instead of JSON and
schema. A failed save now says what happened in one plain sentence and
keeps the server's own words under a collapsed Details. The drafts that
"Edit with the assistant" puts in the composer no longer carry the
path; the built-in workspace skill already maps shortcuts and Home to
their files.

All nine locales are translated; only the changed keys moved, and four
keys that nothing reads any more are removed.

Assisted-by: Claude:claude-opus-5-5

* Drop internal names from the skills, add-ons and tasks copy

Other pages used words that only make sense to someone who knows how
LocalAI is built:

- Skills said "a folder with a SKILL.md" and "the file workspace". It
  now says what a skill is for, and that skills need Files.
- The add-on notes blamed "the owner", the internal name of the server
  that runs add-ons. They now say what happened to the action, or that
  the add-on service is not available; a private library names a
  credential "an administrator configured on this server".
- The Tasks header listed an API route, and the first-run wizard spoke
  of a "host bash fallback". The memory line of Settings no longer
  mentions "the resources endpoint".

All locales that have these keys are translated. The skill delete
dialog, the unused skills intro and the Tasks header exist only in
English and change only there.

Assisted-by: Claude:claude-opus-5-5

* Fail the unit tests when UI copy names implementation details

A new test reads every string of every locale and fails when one names
a settings or data file (.json, .yaml, LocalAI/), an API route or HTTP
method, a setting key (chats_*), an environment variable, a format
word (JSON, YAML, schema) or a path shown as code. Plain words such as
"schema" and "endpoint" are checked in English only, since other
languages use them for other things.

Operator pages may still name the model configuration, the metrics
route or a startup setting that the docs name. Those keys are listed in
ui-copy-allowlist.json, each with the audience and the reason. The list
only shrinks: an entry whose string no longer needs it, or whose key is
gone, fails too.

Assisted-by: Claude:claude-opus-5-5

* Name the triggers group in the chat tools routes

The routes that list and switch the built-in groups of a chat said only
"a name such as web". They now name all three: web, triggers and
saved_agents, with the tools of each.

Assisted-by: Claude:claude-opus-5-5

* Make the built-in skills answer in plain words

The skills told the model to report file names, folders and formats
after a change. Add one rule near the top of localai-workspace and
localai-apps: do the work with files, but say what changed, and name a
file only when the user did or asks. Replace the example replies that
carried paths, keep every path that says where to write, and pin the
rule with a test.

Assisted-by: Claude:claude-sonnet-5-5

* Apply the Go 1.26 modernizers to the latest Studio code

go fix suggested slices.Contains, strings.SplitSeq and range-over-int
on lines added since the first pass. ParseSearchProviders had the same
hand-written membership loop, so it uses slices.Contains too. Only
lines that the branch added since then are changed.

Assisted-by: Claude:claude-opus-5-5

* Share the blob wait and the counting of the two workspace purges

The JSON and GORM stores each had the 2-minute default for the blob
grace, the same switch that reads the BlobGrace option, and the same
loop that counts files, folders and bytes per revision. They now call
blob.Grace and PurgeResult.Add, both with specs. The SQL and the
in-memory walk that pick the nodes stay in each store.

Assisted-by: Claude:claude-opus-5-5

* Read the upper layer of a sandbox in one place for both stores

The JSON and GORM workspace stores had the same readUpper, place,
joinRel, depth and sortedKeys, with the same upperSet types. They move
to workspace/internal/upperset, which has its own specs. The commit
algorithm that applies the set to the nodes stays in each store; it
needs a storage interface first.

Assisted-by: Claude:claude-opus-5-5

* Let chat attachments use the 1 GiB message limit

The docs promise 1 GiB for a message with attachments, but the general
body limit (15 MB by default) ran first, because only the Files upload
skipped it. Skip it also for a multipart POST to the messages route
when the operator set no --upload-limit. A JSON message keeps the
general limit, and an explicit --upload-limit still wins. A message
that goes over the 1 GiB limit now gets 413, not 400.

Assisted-by: Claude:claude-opus-5-5

* Name the read_file tool in the skills index header

A weaker model read a built-in skill with the wrong path and then tried
the Sandbox, which cannot see skill folders. The header now says which
tool reads a skill and that the shown path is the one to use, before it
mentions where the user's own skills go.

Assisted-by: Claude:claude-sonnet-5-5

* Tell the model that skill folders are not in the Sandbox

The Commands block of an isolated chat now says that skills are read
with read_file. The Sandbox tool description and the per-turn note are
at their size caps, so the sentence goes in the system prompt.

Assisted-by: Claude:claude-sonnet-5-5

* Point read_document at read_file for text files

read_document reads PDF, Word, Excel, PowerPoint and EPUB files. For a
Markdown or text file it said only that the file is not a document it
can read. The error now names read_file.

Assisted-by: Claude:claude-sonnet-5-5

* Tidy the text index stores and the indexer pool

The two stores wrote the default limits 20 and 100 as numbers in five
places; they are now named constants, so the stores cannot drift. The
JSON store checks a missing file with errors.Is. The indexer starts its
readers with wg.Go and keeps the first error inline, which drops the
cmpErr helper.

Assisted-by: Claude:claude-opus-5-5

* Refuse the server's public address in the Sandbox proxy

web_fetch refuses the addresses of the server: its interfaces,
loopback and the host of the external base URL. The Sandbox proxy runs
in the add-on process and listed only the interfaces of its own
machine. Under "All public sites", an always-allowed host or an Allow
card, a command could reach the LocalAI API through the public name or
address of the server, a NAT address or a proxy in front of it.

The host now keeps one list for both. The application builds it once,
gives it to web_fetch, and sends it as CIDR strings in the new self
field of every egress.check allow. The proxy keeps the last list it
got and refuses it on each dial, besides its own interfaces, also for
an allow it cached. safehttp gets the codec and a join of deny lists,
so both sides read the list the same way.

Assisted-by: Claude:claude-opus-5-5

* Check the TLS server name on every port of the Sandbox proxy

The proxy checked the ClientHello only on CONNECT to port 443. An
always-allowed host pinned to another port, or any port added to
chats_egress_ports, was a raw tunnel: a command could open TLS to
another tenant of the same front through it. The docs said the server
name must still match.

On other ports the proxy still waits for nothing, since the protocol
may be anything and the server may speak first. But when the first
bytes from the guest are a TLS handshake record, it reads the whole
ClientHello and passes it on only when the server name is the CONNECT
host; otherwise it closes the tunnel and audits the reason sni.

Assisted-by: Claude:claude-opus-5-5

* Say what sent each notification

The inbox only knew whether a notification was about a run or needed
attention, so a trigger that was switched off and a connector that was
switched off looked the same. Each notification now carries a source:
trigger, connector or system (the conversion of old data). The list
endpoint returns it, and items stored before have none.

Assisted-by: Claude:claude-opus-5-5

* Say where a skill is when a read of it fails

When read_file, read_document or list_files fail with not found for a
path that names a skill of the chat (a wrong skills folder, or just
<name>/SKILL.md), the error now gives the skill's real path and says to
read it with read_file. The skills come from the chat's own resolver, so
every layer, add-on and per-user switch is respected, and nothing is
looked up unless a read fails.

Assisted-by: Claude:claude-sonnet-5-5

* Keep the last valid Sandbox network lists when a value is bad

The settings endpoint refuses a bad chats_egress_allowed_hosts with the
reason, and the runtime settings file keeps the value in force. A bad
value that reached the configuration another way was dropped with no
log line: the always-allowed hosts and the extra registries became
empty, and a bad port list fell back to 80 and 443, which can widen a
narrower list.

Each of the three lists now keeps the last value that parsed, logs a
bad value once, and uses the default only when the process never saw
a valid one. The parse runs again only when the value changes, not on
every egress.check.

Assisted-by: Claude:claude-opus-5-5

* Use the keyless chain of the built-in search in the Web add-on

The add-on replaces the built-in web_search, but its keyless provider
was DuckDuckGo alone. DuckDuckGo often answers data-center addresses
with a bot check, so installing the add-on without a key made search
worse than the built-in, which falls back to the Brave Search and Bing
result pages.

The keyless entry of the add-on's order (saved name duckduckgo) is now
the built-in chain from pkg/web/search: DuckDuckGo, then the Brave
Search page, then the Bing page, with the same cool-down. The status
tool reports the providers in their cool-down, and the Settings tab
shows them with the reason. Test the providers gives one row for each
page of the chain.

Assisted-by: Claude:claude-opus-5-5

* Fix the GitHub hosts in the Q1 row of the egress design

The Q1 row still listed raw.githubusercontent.com on the registries
list. The owner decision below it and registries.json exclude it,
since it serves any user content, and add
release-assets.githubusercontent.com. The row now says the same.

Assisted-by: Claude:claude-opus-5-5

* Keep one private-address rule for the network cards

A card asked whether its host is a private IP literal by calling
egress.Admits with a made-up ceiling, which re-derived the rule that
egress already has. egress.PrivateLiteral is now exported (with a spec)
and the card uses it. The two String methods of an allowed host write
host:port with net.JoinHostPort, which brackets an IPv6 literal the
same way the hand-written code did.

Assisted-by: Claude:claude-opus-5-5

* Read a unique constraint violation in one place

The jobs and triggers stores had the same isUniqueViolation, and the
saved agents store had the same test inline before it looked for the
name index. They now call dberr.IsUniqueViolation, which has specs for
the SQLite and PostgreSQL error texts. The workspace store keeps its
own stricter match.

Assisted-by: Claude:claude-opus-5-5

* Ask before a Home widget link opens another site

A list widget turned an http or https item into a plain link to a new
tab. Its data file is written by trigger runs and the assistant, which
read web text, so a planted link opened with one click, and only a
tooltip showed the host. A link of a page asks first.

An external item link now opens the confirmation bar of page links in
the widget: the site, the full address, the punycode warning, and Open
or Cancel. A link inside the app still navigates directly. The bar
takes a label, so the widget says that it wants to open the link.

Assisted-by: Claude:claude-opus-5-5

* Prune the audit of the commands run on this machine

The audit of the built-in bash had no retention: the database table
grew without bound in a cluster, with commands of up to 4 KiB each.
The Sandbox network audit had chats_egress_audit_days and a daily
prune under an advisory lock.

Both audits record what the commands of the chats did, so the host
command audit now uses the same retention and is pruned in the same
daily pass: one frontend of a cluster, under the same lock; a
standalone process alone. The GORM and file stores get Prune. No new
setting.

Assisted-by: Claude:claude-opus-5-5

* Save the Studio JSON stores with atomicfile.WriteJSON

Seven stores (agents, connectors, jobs, MCP connections, notifications,
push subscriptions and triggers) encoded their state and then called
atomicfile.Write with the same four lines. atomicfile.WriteJSON does
both and has specs. The load side stays in each store, since the error
messages differ.

Assisted-by: Claude:claude-opus-5-5

* Build the result page walkers of the keyless search on one walk

hasChallenge and firstByClass each had their own recursive walk that
stopped at the first match. They are now predicates passed to
firstElement, which already did that walk. The order of the walk and
the element tested first are the same.

Assisted-by: Claude:claude-opus-5-5

* Expire network cards that a dead frontend left pending

Pending network cards expired only on the turn-end path of the process
that ran the turn. When that frontend died, its cards stayed pending
in the chat record, counted toward the cap of three, and gave the next
turn's connections cards-full blocks. The UI showed them as waiting.

A card now names the turn that asked it. One rule, shared with the
command card in a later change, says whether a card still has a
waiter: its turn holds the chat and its wait has not run out. A card
that fails it reads as expired, takes no place under the cap, and is
expired in the record when the next card is asked, when the next turn
starts, or when the reaper ends the dead turn.

Assisted-by: Claude:claude-opus-5-5

* Reuse ChooseProvider for the expected add-on of a tool

shadowers.expected repeated ChooseProvider line for line: the pinned
add-on when it is installed, else the oldest install. It now calls it.
commandFiles kept an order field and sorted the nodes of a map by it;
it now appends each node to a list when it first sees it, which gives
the same order without the sort.

Assisted-by: Claude:claude-opus-5-5

* Do not print a nil error when a key or entry is missing

Three checks joined "the read failed" and "the read found nothing" in
one branch, so the second case wrote "<nil>" into the message: the
VAPID keys of push notifications, the sealing key of connectors and a
rendered catalog entry. Each case now has its own message. The message
of a failed read does not change.

Assisted-by: Claude:claude-opus-5-5

* Check the close errors and rename the tool variables

golangci-lint failed on the new studio code. Five deferred Close calls
and one Fprintf into a hash dropped their error, and two locals named t
made forbidigo read `t.Run` as a testing call. Discard the errors
explicitly and name the locals tool.

Assisted-by: Claude:claude-opus-5-5

* Make the file text index forget purged files and say its limits

Three gaps of the text index of documents:

- A purge dropped the index entries of what it erased in a goroutine
  that only logged its errors, and the purge had already erased the
  journal rows. A crash or a store error left the text in the database
  for good. Each pass of the indexer now also checks a batch of entries
  (200 every 30 s, round all of them) and deletes those whose node is
  gone, so what a failed Forget leaves is removed.

- A document search stopped at 20000 files or 64 MiB of text without a
  word, and the "still being indexed" count stopped at 50. The search
  now sets Limited (search_files says "results limited", the REST
  answer has "limited", the Files page shows a note), and the count is
  the number of pending entries below the path.

- read_file extracted a long document again for every page, up to 50
  MiB and 60 s each time. The first read keeps the whole text in the
  index store by content hash (a new documents table, or files in the
  JSON store), and the next pages come from it. The text goes away with
  the last entry of that content, on delete and on purge.

Search reads the text without its folded copy, which only the database
needs.

Assisted-by: Claude:claude-opus-5-5

* Stop the PDF reader when the extraction gives up

The PDF and EPUB reader runs in a goroutine, and the call returns after
60 seconds when it is slow. The goroutine went on reading every page,
since its context ended only with the caller's. It now gets a context
that ends when the call returns, so it stops at the next page. What
the call returns does not change.

Assisted-by: Claude:claude-opus-5-5

* Compare the code files of a shared page as the store does

The guard that refuses new code to a frame of an older version compared
the name it reads with the names of the page's code exactly. The
workspace folds case, so an old frame that read Index.html or JS/App.js
got the new code with no 409. Compare names in the store's form
(workspace.FoldName), as the write check already does, and test the
case variants.

Assisted-by: Claude:claude-opus-5-5

* Expire the command card of a turn that is over

The card of the built-in bash was cleared only by a defer in the call
that asked. When that frontend died, the card stayed in the chat
record. The chat hid it only while no turn ran, so the next turn
showed it again, and an answer to it reached no one but still got 204.

The card now names its turn and when it was asked, and follows the
same rule as the network cards: it waits only while that turn holds
the chat and its five minutes have not run out. GET /api/chats/{id}
shows only such a card. The next turn and the reaper clear a card left
behind, and the waiting call clears only its own card, so it never
removes a newer one. An answer to a card with nobody waiting is
410 consent_expired, and the chat shows the card as not answered, with
no buttons. The store gets UpdateHostConsent, an atomic change of the
card.

Assisted-by: Claude:claude-opus-5-5

* Answer a trigger change once across frontends

ApplyChange, UndoChange and DismissChange checked the outcome of the
change record under a mutex of one process, then wrote the store. Two
frontends could both see a pending change and both apply it, or both
undo it.

Add Store.SetOutcome, a conditional update of the outcome (only from
the outcome it names), to the GORM and JSON stores and to the store
suite. The three answers now claim the record with it before they
write, give the outcome back when the change itself fails, and answer
ErrProposalAnswered (409) when another request won. A test runs the
two answers on two frontends at the moment between the read and the
write, for an edit, its undo, and a delete.

Assisted-by: Claude:claude-opus-5-5

* Add the shadowers spec to the tables-tools inventory

The tool adapter gained a spec that lists the add-ons that replace a
built-in in any state. The tables-tools suite runs every spec of that
package, so verify.sh found one case more than the inventory holds.

Assisted-by: Claude:claude-opus-5-5

* Offer Clear filters when the backend Catalog shows nothing

The Installed view had a Clear filters button in its no-results state;
the Catalog view did not, although the spec asks for it in both. Move
the button to one place and show it in the Catalog when a search, a
type filter or a library filter leaves no row. Add Playwright specs for
both views, which the earlier Installed fix lacked.

Assisted-by: Claude:claude-opus-5-5

* Give the chat settings without an assistant tool their reason

chats_max_iterations, chats_work_folder, chats_index_files and
chats_index_max_text_bytes had no MCP tool and no entry in
adminRoutesWithoutTool, and the older chats_saved_agents,
chats_suggestions, chats_vision_model and chats_transcription_model
were unlisted too. Add each with the reason it stays a person's choice
in Settings: they apply to every user's chats, and the step limit is
one a model must not raise for itself.

Assisted-by: Claude:claude-opus-5-5

* Remove the stray allowed-hosts keys from the chats inbox locales

allowedHosts, allowedHostsOff and allowedHostsNote sat twice in eight
locales: under egress, where English has them and the UI reads them,
and again under inbox, where nothing reads them. In de and ko the two
copies had drifted apart. Delete the inbox copies and keep the egress
ones.

Assisted-by: Claude:claude-opus-5-5

* Bring three design specs in line with what shipped

- web tools and shadowing: the status still said phase 2 waits; both
  phases shipped. Note that the Web add-on's keyless entry is now the
  built-in keyless chain.
- Home front door: Home now creates the chat and sends the first
  message itself, with a short-lived hand-off to the chat page, and
  its composer takes attachments. Both were described the old way or
  as out of scope.
- chat footer: the meter bar is 96 px with the tokens left, and a phone
  keeps the bar and the tokens left, not the percentage only.

Assisted-by: Claude:claude-opus-5-5

* Store the command card's answer and read it back every two seconds

The answer to the command card of the built-in bash went only as a
bus message. When the message was lost, the call waited the full five
minutes and ended as not answered, although the person had clicked.
The network card stores its answer first and re-reads the card every
two seconds.

The command card now does the same, through one shared pair of
helpers: listenConsent subscribes before the card is stored, and
waitConsent wakes on the bus message or every two seconds and reads
the record, which holds the truth. The network card uses the same
helpers. HostExecDecide writes the decision to the card atomically
before it sends the message: the same answer twice is 204, the other
answer 409 already_answered, and an answered card is no longer shown
as waiting.

Assisted-by: Claude:claude-opus-5-5

* Load skills by name with a load_skill tool

A weaker model failed to read a skill at the path the index showed. The
chat now has a built-in load_skill tool that takes a skill's name and
returns the instructions of its SKILL.md, without the frontmatter.

The tool reads the same resolved skill set as the index: the nearest
skill of each name, the chat's switches, the add-ons the chat can use
now and the grants of its agent. It reads through the user's rights.
When the skill's folder holds other files, the answer starts with the
base directory and names up to 20 of them for read_file; a chat without
read_file gets the instructions only. A skill that needs commands, in a
chat that cannot run them, is refused with the reason. An unknown name
is an error that gives near names first and then every name the chat
has.

Every chat with at least one skill gets the tool, and a sub-agent gets
it with its parent's tools. It has no switch and Settings > Tools does
not list it.

The index of skills now lists names and descriptions only and tells the
model to call load_skill. The Sandbox sentence and the error for a
wrong skill path now point at load_skill too.

Assisted-by: Claude:claude-opus-5-5

* Tell the built-in skills to load other skills by name

localai-apps told the model to read the localai-workspace skill, and
writing-skills said a description is what the agent sees before it
opens the file. Both now speak of loading a skill with load_skill.
localai-workspace keeps the path of its schemas folder and says to read
it with read_file, since those files are not a skill.

A test checks that no built-in skill sends the model to another skill by
"read" or by the path of its SKILL.md.

Assisted-by: Claude:claude-opus-5-5

* Document load_skill and the skills index

The chats page now says that the prompt lists skills by name and
description, that load_skill returns a skill's instructions and names
the files of its folder, what it refuses and who gets it. It no longer
says the model reads a skill by its path. The add-ons page says how a
chat loads an add-on's skill, and the saved agents page says that the
skills grant limits load_skill too.

Assisted-by: Claude:claude-opus-5-5

* Add the proposal action spec to the tables-acceptance inventory

The spec that an editor cannot confirm a schema change as a write ran in
the suite but was never in the inventory, so the evidence manifest check
failed. CI did not reach that check before, because the burst spec
failed first.

Assisted-by: Claude:claude-opus-5-5

* Give a chat a goal that it works toward by itself

A chat can now hold one goal. While it is active, every turn's prompt
names it, the model gets goal_done and goal_blocked, and when a turn
ends by itself the frontend that ran it starts the next one with
"Continue toward the goal". The person's waiting messages still go
first, and a turn they start does not count as an automatic one.

The goal pauses when the person presses Stop, when a turn fails or is
cut short, and when the automatic turns or the model steps reach their
limits. Resume gives it fresh budgets. A sub-agent, and a chat that a
trigger, a connector, a job or a page started, has no goal and no goal
tools.

The goal lives with the chat record in the store (UpdateGoal on the
JSON and GORM stores, with contract tests and a column migration), so
every frontend reads the same one. The next turn goes through the same
turn claim as any other, so only one frontend continues it; a reaped
turn, or a JSON store that reloads after a crash, leaves the goal
paused instead of continued twice. The transcript keeps goal lines
(set, replaced, continue, paused, resumed, done, blocked, cleared) as
their own role, so an automatic message never reads as typed by the
person.

Assisted-by: Claude:claude-opus-5-5

* Add a store for the loops of the chats

A loop repeats a prompt in one chat, at an interval or when the model
asks for the next run. The new loops package keeps the loops, the
daily budget of runs of each owner, and which chats confirmed a loop
once. JSON and GORM stores share one contract suite. Add checks the
limits per chat and per owner in the same step as the insert, and
TakeRun never hands out the last run of a day twice, also across
frontends.

A user message that a loop sent now carries a loop mark (loop ID, run
number, self-paced or not). Both chat stores keep it. The model reads
a short line before the prompt that says the message repeats by
itself, and, for a self-paced loop, how to ask for the next run.

Assisted-by: Claude:claude-opus-5-5

* Add the file preview slot specs to the host-slots inventory

The file preview slot came with two Go specs in the host-slots package.
verify.sh found them in the run but not in the inventory.

Assisted-by: Claude:claude-opus-5-5

* Check the Settings page and the add-ons with the UI copy guard

The guard read the locale files only. The Settings page keeps its
English labels and descriptions in settingsSchema.js, and several
add-ons carry their own English bundle, so neither was checked.

The guard now also reads the label, description and note of every
setting and the label and hint of every group, and the English
bundles of the add-ons (an en.js module or an English JSON file).
The allow-list takes settings:<key>.<what> and addon:<id>:<key>
entries, each with its reason; the operator ones (the SearXNG JSON
format, the fine-tuning file types, the API route of a voice, the
schema of the Databases add-on) are listed.

What it found in the voice library is fixed in plain words: "model
YAML is not modified" and "Existing model YAML is not changed" now say
that the models' settings are not changed, in the voice add-on, the
Studio add-on and the host locales that have the strings (en, fr,
pt-BR). The two host allow-list entries for them go.

Assisted-by: Claude:claude-opus-5-5

* Add the update digest specs to the updates inventory

Two checker specs pin that an update names its candidate digest and
that a stale cached notice stays readable. The updates suite runs the
whole checker package, so the inventory needs both.

Assisted-by: Claude:claude-opus-5-5

* Draw the Add-ons search and state filter with the kit pieces

The Add-ons page had its own copy of the search box and the segmented
filter (ad-search, ad-seg), the same shapes and styles as the kit's
SearchField and Segmented that the Agents pages use. Use the kit
pieces and drop the copy; the narrow-screen rules now target the kit
classes inside the toolbar. The search keeps its test id.

Assisted-by: Claude:claude-opus-5-5

* Add the limits of a chat goal to the runtime settings

chats_goal_max_turns (1 to 200, default 20) is the most turns a chat
with a goal starts by itself, and chats_goal_max_steps (1 to 10000,
default 400) the most model steps of all the turns of one goal, before
the goal pauses. Both are read when a turn ends, so they apply without
a restart. They sit in Settings > Tools next to the step limit of one
turn, and the settings page documents them.

Assisted-by: Claude:claude-opus-5-5

* Add the goal routes of a chat to the API

GET, PUT and DELETE /api/chats/{id}/goal read, set and clear a chat's
goal, and POST /api/chats/{id}/goal/resume resumes a paused or blocked
one. They belong to the chat's owner like the other chat routes and
take the chats feature. A chat view carries the goal with the limits
that apply now, the transcript gives goal lines with goal_event and
goal_turn, and the event stream has a goal event so that other tabs
read the chat again.

The routes are in the route feature registry, the swagger file and the
chats entry of /api/instructions. They manage the person's own chat,
not the server, so they have no MCP tool, as the other chat routes. The
chats page documents goals, their states and limits, and how several
frontends share them.

Assisted-by: Claude:claude-opus-5-5

* Redesign the notification inbox

The inbox panel was a plain list: every row had the same weight, a
failed run looked like a finished one, the panel ran off the screen on
a phone, and a failed load showed nothing.

The panel now has a header with the unread count, a menu for Clear read
and Clear all, Mark all read, and an All / Unread / Failed filter. Items
are grouped under Today, Yesterday and Earlier. Each item shows what
sent it (trigger, connector or LocalAI) with a tinted icon, the title,
two lines of text or the reason of a failure, and when it happened, with
the exact time on hover. A click marks it read and opens its chat; a
connector that was switched off opens the connectors page. Mark read and
Delete show on hover or focus, and stay in view on a touch screen.

The panel has a loading skeleton, an error with Try again, an empty
state that says what shows up here, and a caught-up state. A new
notification slides in, is highlighted for a few seconds and is
announced once. The badge stops at 99+. Focus moves into the panel when
it opens; the arrow keys, Home, End and Delete work on the items, and
Esc returns focus to the button. On a phone the panel is a sheet over
the bottom of the screen with 44 px targets.

Assisted-by: Claude:claude-opus-5-5

* Explain the reads of the text index and the provider requests to gosec

gosec flagged three new lines. The text index reads files from the
directory the host gave it, with names it listed there. A search
provider sends its request to the endpoint fixed at construction, never
to a URL from a tool call. Mark the lines with the reason.

Assisted-by: Claude:claude-opus-5-5

* Keep the add-on details open when Escape closes a dialog

Escape in the restore dialog of the recovery panel also closed the
details pane around it. The pane handles Escape in React, and the
dialog listens on the document, so the pane ran first and saw no
preventDefault. The pane now registers as a modal layer and closes only
when it is the top layer, as the side panes do.

Assisted-by: Claude:claude-opus-5-5

* Add skill and agent messages, compact and clear to the chats API

A message can now carry a skill of the chat or one of the user's saved
agents. The server checks the name against the chat's resolved skill
set (switches, the agent's grants, shadowing, add-ons) or the saved
agents the chat may call, and refuses an unknown one with a 400 that
lists what the chat has. A skill message gives the model the text of
the skill's SKILL.md before the user's words, for that turn only; an
agent message asks the model to start the agent with spawn_agent. The
transcript keeps the words and the command (Entry.Invoked), not the
instructions. A regenerate or an edit runs the command again. Such a
message does not wait for a running turn: it answers 409.

POST /api/chats/{id}/compact runs the summary the loop makes when the
context fills up, through a new optional session.Compactor that the
loop harness implements with its own history compaction. The compacted
state is saved for the next turn and a "compacted" line with the sizes
goes in the transcript.

POST /api/chats/{id}/clear writes a "cleared" line and drops the saved
state. The model then sees nothing above the line: the rebuilt state
and the transcript summary start after the last such line, so a fork
from above it keeps what the model saw there. Export, import and the
Markdown export carry both lines.

Both calls take the turn claim, so no frontend starts a turn on the
chat meanwhile; messages sent during them start a turn after. No MCP
tool: they act on one chat of the caller and are not admin calls.

Assisted-by: Claude:claude-opus-5-5

* Give each network card spec its own settings

A check that a card spec starts can still wait on its card when the
next spec begins. It read the settings through a closure over a
variable that the next BeforeEach replaced, which the race detector
reported. Each service now reads the settings holder it was built with,
under a lock, and the specs change the settings through it.

Assisted-by: Claude:claude-opus-5-5

* Show the effective audit retention in the settings snapshot

The settings snapshot returned 0 for chats_egress_audit_days while the
setting was unset. Any page that saves the whole snapshot back, such as
the Traces settings strip, then posted 0 and the server rejected it with
"Invalid chats_egress_audit_days", so saving tracing settings failed on a
fresh install.

Report the retention that applies (30 days by default), the same way
chats_max_iterations does.

Assisted-by: Claude:claude-sonnet-5-5

* Run the Sandbox smoke test in a short scratch folder

The smoke test step was reached for the first time and both isolated
backends refused to start: their state folder under the checkout's
cache made the socket paths 126 and 132 bytes, over the 107 a unix
socket allows. Point SANDBOX_SMOKE_WORK at /tmp/sandbox-smoke, which
leaves room for the sockets.

Assisted-by: Claude:claude-opus-5-5

* Use paths from the home folder in the Sandbox command specs

bash now starts in the user's home folder, so the end-to-end specs that
read and wrote home/<name> found nothing. Use names relative to the
home folder, as the Sandbox specs already do.

Assisted-by: Claude:claude-opus-5-5

* Show the goal of a chat on the chat page

A bar above the composer shows the goal with its state: Pause (which
stops the running turn and so pauses the goal) while it works, Resume
when it paused or needs the person, and Clear. A second line says why
it paused, what the model needs, or what it did. The status line shows
"Goal · turn 3 of 20" while the goal is active, and the Continue button
of a paused turn stays away then, since the next turn starts by itself.

The transcript shows the goal lines as quiet rows: set, replaced,
continuing (with its turn), paused with the reason in plain words,
resumed, reached with the summary, blocked and cleared. A goal event,
also from another tab, reads the chat again, and a hidden live region
says each change to a screen reader once.

The strings are in all nine locales. Unit tests cover the helpers and
the thread rows; a Playwright spec covers the bar, the progress, the
lines, Pause, Resume, Clear and the event.

Assisted-by: Claude:claude-opus-5-5

* Run the loops of the chats

The loop host fires the loops that are due. A run sends the prompt to
the chat as a new message only when the chat is idle: a busy chat
skips the run, and an interval loop moves to its next slot instead of
piling missed runs up. A self-paced loop gets two tools while it is
active: schedule_next (clamped to the minimum interval and one day)
and loop_done. A run in which the model calls neither ends the loop
with a note. A loop ends after its run limit, pauses after three
failed runs in a row, and waits for the next UTC day when the owner's
daily budget of runs is used up.

One frontend fires the loops, the holder of a new advisory lock. A run
is claimed in the store first, and the turn claim keeps one turn per
chat, so a short overlap of two leaders starts no run twice. A new
leader finishes the runs the old one watched. Deleting a chat removes
its loops; a due loop whose chat is gone removes itself.

Five runtime settings bound the loops: chats_loop_min_interval,
chats_loop_max_runs, chats_loop_max_per_chat, chats_loop_max_per_user
and chats_loop_daily_runs.

Assisted-by: Claude:claude-opus-5-5

* Add the loops API of the chats

GET, POST and DELETE /api/chats/{id}/loops list, start and stop the
loops of a chat; PUT and DELETE /api/chats/{id}/loops/{loopId} pause,
resume, confirm or stop one. GET /api/chats/{id} carries the loops,
the transcript marks the user lines a loop sent with loop {id, run},
and a loops event tells every open tab that the list changed.

The routes are gated by the chats feature and documented in the
swagger spec and the API instructions. They act on the caller's own
chats only, so they get no assistant tool. The chats page gets a loop
section, with when to use a loop and when a trigger, and the cluster
behaviour; the runtime settings page lists the five loop settings.

Assisted-by: Claude:claude-opus-5-5

* Show the loops of a chat above the message box

Each loop of the open chat shows as one line over the message box,
such as "Loop · every 5 min · run 4 · next at 09:20", with Pause or
Resume and Stop. A loop that paused itself or ended says why; an
ended one has Dismiss. The first loop of a chat shows a card instead:
"Run this every 5 minutes until you stop it?" with Start and Cancel.
The server keeps the pending loop, so a reload shows the card again.

A message that a loop sent carries a small "Loop run 4" label. The
page reads the loops again on the loops event, so every open tab
follows. The answer to /loop list is ready for the slash command. The
strings are in all nine locales.

Assisted-by: Claude:claude-opus-5-5

* Build the chat slash menu on one command registry

The slash commands of Home and of a chat now come from one registry,
src/utils/slashCommands.js. A command has a kind (navigate, insert or
server), the surfaces that offer it, a needs check and, optionally, an
argument provider: args(query, ctx) returns the items the menu lists
when the box holds "/cmd <query>". Every existing command and test id
stays; the registry adds:

- one command per skill of the chat or the draft (/review), marked
  built-in, add-on, shared or yours, and /skill <name>, which also
  reaches a skill named like a command;
- /agent <name> <task>, which lists the saved agents, or offers to turn
  them on for the chat when they are off;
- /model and /models with a model name, which switch at once;
- /compact and /clear, which show their progress in the status line
  and leave a marker in the thread; the context meter drops;
- /attach and /attach clear, /help (a card that lists the commands)
  and /about (a card with the model, its state, the tools, skills,
  where commands run and the context size).

Tab completes a command or an argument, Enter runs it, and the first
row whose name starts with what was typed is selected first. A message
typed as a skill or agent command is sent with its marker, not as text;
the transcript shows the words and a chip. Home offers the same set
without the commands that need a thread.

The words are in all nine locales. The registry has unit tests, and a
Playwright spec covers the menu, the completion, the marker, the chip,
compact, clear and the cards with the routes stubbed. The docs page
gains a slash commands section.

Assisted-by: Claude:claude-opus-5-5

* Add the /goal command to the chat

/goal shows a card with the goal, its state, the automatic turns and
steps it used out of the limits, and when it was set. /goal <text>
sets a new goal at once, replacing the one the chat had without a
question, and the goal bar says what it replaced. /goal clear and
/goal resume act on the goal; typing "/goal " lists resume and clear
for the goal the chat has. None of these is sent as a message. In a
new chat, /goal <text> creates the chat first.

The command sits in the Context group of the shared command registry,
on the chat surface only, and not in a chat that a trigger or another
run started. A skill named goal is reached through /skill. Unit tests
cover the parser and the menu items; the Playwright spec covers the
card, setting, replacing, resuming and clearing from the box.

Assisted-by: Claude:claude-opus-5-5

* Add /loop to the chat

/loop [interval] <prompt> starts a loop in the chat; /loop list shows
the loops of the chat with the code that /loop stop takes, and
/loop stop [id] stops one or all. The command is in the slash
registry: after "/loop " the menu offers list and stop, and after
"stop " the loops of the chat.

A loop prompt that invokes a skill (/review the diff, or
/skill review the diff) runs as the person's skill command would: the
model gets the skill's text, and the transcript shows the words, the
skill's chip and the loop label. The server checks the skill when the
loop starts and at each run; any other command is refused with the
reason.

Assisted-by: Claude:claude-opus-5-5

* List the loop settings on the Settings page

The five limits of the loops of the chats show under Tools with the
other chat settings: the shortest interval, the runs per loop, the
loops per chat and per user, and the loop runs per day. All apply at
once.

Assisted-by: Claude:claude-opus-5-5

* Clear a sent Home draft when the page goes away too

Home clears the draft of a sent message when it closes, so the text
stays visible while the composer moves into the new chat. While the
chat page still loads, a reload or a closed tab could come first, and
the sent text came back on Home. Leaving the page now clears it too.

Assisted-by: Claude:claude-opus-5-5

* Check the close error of a previewed file

golangci-lint (errcheck) flagged the deferred Close of the file the
preview endpoint reads. Discard its error explicitly, as the other
studio file handlers do.

Assisted-by: Claude:claude-opus-5-5

* Explain the document read of the text index to gosec

gosec reported the read of a stored document text as a file inclusion
through a variable. The path is the configured index directory joined
with a hash that documentPath refuses when it holds a path separator
or a dot, so it cannot leave the documents folder.

Assisted-by: Claude:claude-opus-5-5

* Build the darwin/arm64 add-on packages on macos-15

GitHub retires the macos-14 runner image on 2026-11-02 and runs
scheduled brownouts until then: every darwin/arm64 package job failed
at once with the brownout notice. macos-15 is the Apple Silicon image
that replaces it.

Assisted-by: Claude:claude-opus-5-5

* Expect the chat's own load_skill tool in the agent prompt spec

The spec that keeps the agent engine's text out of the prompt also
required that no load_skill tool or mention reach the model. Chats now
offer their own load_skill tool and the skills index tells the model to
call it, so the spec failed. Keep the checks on the engine's own skill
and command text, and check that the chat's tool is offered.

Assisted-by: Claude:claude-opus-5-5

* Read the user's scripted tool call in the mock model

The chat prompt now names the chat's work folder, and the folder name
comes from the chat title, which holds a cut copy of the user message.
The mock model read the first ADDON_TOOL_CALL marker, the cut one in the
system prompt, and called the tool with broken arguments over and over.
Read the last marker, the one of the user message.

Assisted-by: Claude:claude-opus-5-5

* Keep the phone targets of the inbox at 44 pixels in every build

On a phone the inbox close button and the item actions must be 44
pixels wide. Their rule had the same weight as the kit's .dk-btn
min-width, so the order of the stylesheets decided. The single-chunk
coverage build puts the kit later and the close button shrank to 28
pixels. Scope both rules to their container so they win in any order.

Assisted-by: Claude:claude-opus-5-5

* Let the about-you spec accept the sidebar's own seen date

When the user has pages, the sidebar writes the date of its New label
under seen.new in user.json. In the whole suite other specs make pages
first, so the saved record held that date as well and the exact
comparison failed. Check that the uploaded seen record is kept.

Assisted-by: Claude:claude-opus-5-5

* Give the typed-while-working chat spec time for two turns

The spec waits up to 30 seconds for each step, but two turns of the
mock model with its tool steps take about 28 seconds on their own, so
the default 30 second test limit cut it under load. Set its limit to
90 seconds.

Assisted-by: Claude:claude-opus-5-5

* Scope the foundation spec's state query to the sheet header

The details pane now repeats the observed state under Health, so a
role query for "State: Off" inside the add-on sheet matched two
elements and failed in strict mode. The administrator-manages spec,
and the two specs after it, did not run.

Query the chip in the header (.ad-pane__meta), which is the state the
administrator sees next to the add-on's name, through one helper used
for all three assertions.

Assisted-by: Claude:claude-sonnet-5-5

* Wait for the page to read each apps answer in the registry polling spec

The spec moved the page clock as soon as the stand-in saw a request.
The registry sets its next timer only after it reads the answer, so on
a slow runner the clock could pass the due time of a timer that did not
exist yet, and the next read never came. With the CPU throttled ten
times, the old spec failed in 3 of 3 runs.

The spec now counts the answers the page has read and lets queued work
finish before it moves the clock. It pauses the clock before it leaves
the page, and it tries the pause again when a slow machine passed the
target time. The assertions do not change: one read at once, one read
per interval and no faster, and no read after the disable.

Assisted-by: Claude:claude-opus-5-5

* Close the egress test listener only after it accepts

The dial helper of the egress proxy specs closed its listener when it
returned, while the goroutine that accepts the connection could still
be waiting to run. On a loaded runner the listener sometimes closed
first. The kernel then reset the connection that was still in the
accept queue, the spec's write consumed the reset error, which the
spec ignores, and the next read saw EOF. The spec failed with
"unexpected EOF" before the proxy got the request.

Wait for Accept to return before the listener closes, then start the
proxy handler on the accepted connection.

Assisted-by: Claude:claude-opus-5-5

* Run host commands in a shell that reads no startup file

The host backend promises a fixed environment, but bash -lc reads
/etc/profile and the user's dotfiles. On macOS path_helper rewrote PATH
there, so the command did not get the fixed PATH. Turn the login shell
into 'bash --noprofile --norc -c' in the host backend only, and make the
test assert the exact fixed PATH.

Assisted-by: Claude:claude-sonnet-5-5

* Answer the command card in the platform turns and 400 a bad answer

The tool-list spec of the platform suite timed out: without accounts
the one person is an administrator, so the chat has the built-in bash
in ask mode. The tool list is sorted and bash is first, and the mock
model calls the first tool, so every turn waited five minutes for a
yes on the command card. The turn helper now refuses each card it
sees, as a person who clicks deny, and the turn goes on to its step
limit. bash stays in the checked list.

An administrator who posts a consent with no card id or an unknown
decision got 403 "only an administrator can allow commands". That is
a bad request, not a refusal: it now answers 400, as the swagger
already says, and the route check for administrators passes.

Assisted-by: Claude:claude-opus-5-5

* Let a chat switch off its shell, file, media, collection and sub-agent tools

The Tools menu of a chat switched only three groups of built-in tools:
web, saved_agents and triggers. A person could not keep a chat away
from the shell, the files, the helper models, the collections or the
sub-agents.

Add five groups to the registry of built-in tools, all on until the
chat switches them off:

- shell: bash, whoever serves it. The Sandbox add-on's bash and the
  built-in bash on this machine both go, and the add-on list reports
  commands switched_off. The row stays listed when nothing can run
  commands, not usable, with the reason.
- files: the file tools and read_document.
- media: describe_image and transcribe_audio.
- collections: list_collections and search_collection, usable only
  for a user with the collections feature.
- subagents: spawn_agent. list_agents and /agent follow it.

The registry gets two flags. External marks a tool that its own code
builds for the turn (with the wrappers it needs); the registry lists it
in its group and Hidden tells the builder what the chat switched off.
Covers makes a group's switch also hide the add-on tool that replaces
the built-in one. PUT .../builtins, the panel, the draft list and the
fallbacks use the existing group code.

A sub-agent follows the switches of its parent chat. The turn prompt
is built from the final tool list: no file tool names, no skill
writing hint, no work folder and no rules pointer when the chat cannot
use files, no describe_image when the helper is off, and a commands
block that says the person switched commands off.

Assisted-by: Claude:claude-opus-5-5

* Queue the edited chat message after the first step starts

The spec queued its message as soon as the chat showed a busy state.
The turn is busy before its first model request starts, and that
request takes every message that waits. With the chat's tools and
skills, the first request now starts a few hundred milliseconds after
the turn, so the message went with it, and the edit got 409 "already
delivered" (5 of 6 local runs).

Wait until the answer streams, as the Stop spec does: the request is
running then, and the queued message waits for the next step.

Assisted-by: Claude:claude-opus-5-5

* Give the reload chat spec a turn that outlasts the reload

The spec reloaded the page during a 3 second turn and expected the
page to show the turn still running. A reload of the coverage build
on a busy runner takes 4 to 5 seconds (measured with a 6x CPU
throttle), so the turn had ended before the page could show it. With
the 30 second turn the page always showed it running, so the resumed
state works and only the window was short.

Add MOCK_LONG_STREAM_MEDIUM to the mock model, a turn of about 10
seconds, and use it in the spec with a 60 second limit.

Assisted-by: Claude:claude-opus-5-5

* Wait for the chat page after the Home hand-off in the specs

Send on Home changes the address inside a view transition, and the
chat page mounts about 35 ms later. Specs that waited only for the
address acted on Home in between:

- The title spec pressed Ctrl+K at once, because the server names a
  chat after its first message as it arrives, so the title poll ends
  at once too. The key reached the page before the chat page listened,
  and the menu did not open (3 of 3 presses at 0 ms failed, all at
  300 ms and later passed).
- The Back spec went back before the chat page was drawn. Home was
  never closed, so its draft of the sent message stayed in the box.

startChat now waits for the chat page, the title spec for the title
in the chat header, and the Back spec for the chat page before Back.

Assisted-by: Claude:claude-opus-5-5

* Accept a resolved user.json conflict in the quiet page specs

The sidebar notes in the seen record of user.json the first day it
labels Pages as new, and saves it from the version it read. All specs
share the test server's one user, so tabs of other workers save the
same record at the same time: one gets 409 and useSeen reads the file
again and saves once more. Three tabs that open /app/files at once
give 200, 409, 200. The quiet check of the shell pages counted the
409 as a problem.

Ignore a 409 on PUT /api/chats/wellknown/user in that check.

Assisted-by: Claude:claude-opus-5-5

* Check only the spec's own items in the Trash after the reload

After Empty Trash and a reload, the spec expected the whole Trash to
be empty. The Trash belongs to the test server's one user, and specs
of other workers move files to it meanwhile, so CI saw "In the Trash:
3 files, 356 B." after the reload.

The checks right after Empty Trash stay. After the reload, check that
the server's Trash, the Trash tab and Files do not hold the spec's
items, once each list has loaded.

Assisted-by: Claude:claude-opus-5-5

* Show the new tool groups in the chat Tools menu

The Tools menu lists the groups of built-in tools that the server
sends. The five new ones need words and an order, and the shell row
needs to say who runs the commands.

Name the groups in all nine locales (Run commands, Your files, Images
and audio, Collections, Sub-agents) and order the rows by what they
reach: the shell and the files first, the triggers last. The shell row
carries a chip with the provider (Sandbox or This machine) and a line
under it that says the commands run isolated, or a warning when they
run on this machine. When nothing can run commands the row is disabled
with the words of the commands notice. A switched-off sub-agents group
also turns /agent off in the composer.

The specs cover the rows, the off and on requests, the bulk actions,
the provider chip, the disabled reasons, the draft on Home and a real
server round trip.

Assisted-by: Claude:claude-opus-5-5

* Document the groups of built-in tools a chat can switch off

The Tools menu section described only the web, triggers and saved
agents groups. List every group with its tools, say which groups the
server may leave out and why a row can be not usable, how Run commands
covers both providers, what the model is told when a group is off, and
that a sub-agent follows its parent's switches.

Assisted-by: Claude:claude-opus-5-5

* Fix the host command consent spec for the 400 contract

The consent route answers 400 for a body without a card id or with a
decision other than once or deny, whoever asks. The spec still expected
403 for always from an administrator. Cover the malformed answers for
an administrator and a user, each in their own chat, keep another
user's chat at 404, and say it in the chats guide.

Assisted-by: Claude:claude-sonnet-5-5

* Find the web tool group by name in the route spec

The tools panel lists the shell group before the web group, so the
spec that read the first entry compared the wrong group. Look the
group up by name, which does not depend on the registry order.

Assisted-by: Claude:claude-sonnet-5-5

* Give Skills its own sidebar item

Skills work in every chat, but the sidebar filed the page under Agents
and only agent skill chips linked to it. Add a Skills item right after
Memory, show it in the legacy list too, and stop lighting up Agents on
the skills pages.

Assisted-by: Claude:claude-sonnet-5-5

* Add Manage skills to the skills picker and a /skills command

The picker now ends with a Manage skills link, and /skills opens the
Skills page from the composer on Home and in a chat. The picker keeps
the height of its status line, so a refused change does not move the
rows. Words are added in all nine locales.

Assisted-by: Claude:claude-sonnet-5-5

* Say on the Skills page that every chat can use skills

The page subtitle named only agents as users of skills. Say that all
chats use them, translate it, and document where to manage skills: the
sidebar item, the picker link and /skills.

Assisted-by: Claude:claude-sonnet-5-5

* Keep Home on screen while its widgets load their strings

A Home widget reads the pages strings, and i18next loads a namespace
the first time a component asks for it, by suspending. Home had no
Suspense boundary of its own, so the route's boundary hid all of Home
(display: none) and showed the route fallback until the strings came.

A widget shows only after home.json is read, so this happened after
the composer was already drawn. Focus given to the composer in that
window went nowhere, and a key typed then was lost. The slash menu
spec failed in CI this way when another spec had written widgets to
the shared home.json: the box was empty and the body had the focus.
With widgets in home.json the composer was hidden right after it
appeared on 5 of 5 loads, and the spec failed 1 or 2 times in 5.

The examples and the widgets now have their own boundary with no
fallback, so only they wait. A new spec holds the pages strings while
it types in the composer; it fails without the change (the composer
never takes the focus) and passes with it.

Assisted-by: Claude:claude-opus-5-5

* Send the stubbed file change after the Files page read the folder

The change feed spec answered the first stream request with the change
at once. The app shell (the appearance sync) opens the shared stream
when the app starts, before the Files page chunk has loaded, so on a
slow runner the change reached a tab where nobody watched the folder
yet and was dropped. The page then never read the folder again, and
the spec failed on all three CI attempts. A CPU throttle of 8 makes it
fail 3 of 3 times locally.

On a real server the page's own first read would already show such a
change. The stub now holds the change until the page has read the
folder; the page subscribes in the same commit as that read, so the
change arrives with a subscriber in place. The resume check is kept.

Assisted-by: Claude:claude-opus-5-5

* Give the oversized 3D input as a sparse file, not a buffer

The spec built a 32 MiB buffer and passed it to setInputFiles.
Playwright sends a buffer to the browser over the protocol: that step
alone took 5.5 s on an idle machine, and on the loaded CI runner the
test hit the 30 s timeout on all three attempts.

It now writes a sparse file of the same size to the test output folder
and passes its path, which the local browser opens as is. The page
still checks the size before it reads the file, so the toast and the
no-request assertion are unchanged. The step takes milliseconds.

Assisted-by: Claude:claude-opus-5-5

* Check the tool line, not the tool name, in the tool call journey

The spec looked for /list_files/ in the transcript. The user's own
message holds that name, so the check could pass before any tool ran.
Once the model's call streamed, a second element matched, and the
strict locator failed (2 matches on the first attempt, 5 on the
retry).

It now waits for the activity line that the tool run adds ("Listed
/"), which only shows after the tool ran.

Assisted-by: Claude:claude-opus-5-5

* Measure the proposal card and its neighbours in one frame

The layout spec read the boxes of the message text, the card and the
message actions with three boundingBox calls. They are viewport
positions taken in three frames, and the transcript scrolls to its end
while the card fills in. On the phone the actions were read after a
scroll and came out 156 px above the bottom of the card.

The three boxes now come from one page.evaluate, so they are read in
the same frame. The order assertions are unchanged.

Assisted-by: Claude:claude-opus-5-5

* Read the page fixture time when the page is served

The task board fixture of the discovery and Pages specs set its
change time once, when the file loaded. A worker loads it once and
keeps running specs, so a spec that ran two minutes later read
"Changed 2 minutes ago by the assistant" where it expects "just now".

The change time is now a getter, read when the fixture is serialized
into a response.

Assisted-by: Claude:claude-opus-5-5

* Check a background process without /proc in the host test

The alive helper read /proc/<pid>/stat, which macOS does not have, so it
reported every process as dead and the spec failed there with "did not
survive the suspend". Use signal 0 for existence and ps (or /proc where
present) to tell a zombie from a live process.

Assisted-by: Claude:claude-sonnet-5-5

* Keep the composer switch menus at their own height

The base rule of the dropdown menu caps it at 320px and lets it scroll.
The switch menus set their own height with a rule of the same weight in a
lazy sheet, so which one won depended on the order the sheets loaded in.
When the base rule won, the picker showed a few rows, and a click on a row
at the edge of the list scrolled the list by a few pixels.

Double the class on the switch menu rules so they win in any order, and
add a test that loads the base rule last.

Assisted-by: Claude:claude-sonnet-5-5

* Show what applies in the chat Network switch

The Network row highlighted the choice stored with the chat, so a
chat that asked for All public sites under an administrator limit of
Off showed All highlighted although it was disabled and not in effect.

Highlight the effective choice, the lower of the chat's choice and the
limit, which the server already computes. When the chat asks for more
than the limit, say so in one line instead of highlighting it. Add a
line that the limit is set by the administrator, and tell the host
network line that limits need the Sandbox add-on.

Assisted-by: Claude:claude-sonnet-5-5

* Design the host components that add-ons reference

The host UI names the Sandbox and the Web add-ons to place their server
settings, and the chat's Network row links to the global Settings page.
Describe a closed registry of host components that an add-on references
by ID in its manifest, entitled by the existing host operation grants
and tool shadows, so the controls live on the add-on's own page and a
second add-on can reuse them with no host change.

Assisted-by: Claude:claude-opus-5-5

* Let an add-on ask for host controls in its manifest

An add-on had no way to say that a host-owned control belongs on its
page, so the host UI placed those controls by add-on name. Add
ui.controls, a list of IDs from a closed registry of host controls:
network-policy (the network of the commands an add-on runs) and
tool-providers (the rows of the built-in tools it replaces).

The host decides which controls it draws: EntitledControls keeps a
control only when the add-on holds its host operation grant and
replaces its built-in tool. The field is optional, so every existing
manifest stays valid.

Assisted-by: Claude:claude-opus-5-5

* List the host controls an add-on is entitled to

The apps list now carries ui.controls with the host controls the
manifest declares and the host setting entitles the add-on to: the
manager passes each add-on's host operation grants to the list.

GET /api/chats/{id}/egress gains addon, the add-on that runs the chat's
commands when it has the network-policy control, so the chat can draw
its Network row from that add-on and link to its settings. applies and
applies_reason keep their meaning: the policy holds with or without the
control. The model and the 400 error no longer send an administrator to
Settings > Tools for the limit.

Assisted-by: Claude:claude-opus-5-5

* Draw the host controls on the add-on's own page

The host UI placed the Sandbox network and the tool rows by add-on name
(SERVER_SETTINGS), showed a read-only summary on the add-on's Settings
tab, and linked the chat's Network row to the global Settings page.

Add a registry of host controls keyed by the IDs of the contract. The
Settings tab of an add-on draws, below its own sections, the controls
the apps list gives it, in one form with the pending bar of the
Settings page: network-policy reuses the network section and its
setting rows, tool-providers the rows of the built-in tools the add-on
replaces. The chat draws its Network row only when the server names an
add-on with network-policy, and the administrator link opens that
add-on's tab. The row itself is unchanged.

The Settings page keeps the settings of features that exist without an
add-on, points to the add-on where one serves them, and no longer
lists the network settings. Its old address opens the add-on's tab.

Assisted-by: Claude:claude-opus-5-5

* Ask for the host controls in the Sandbox and Web manifests

The Sandbox asks for network-policy and tool-providers, so the network
of its commands and the bash row are drawn on its own Settings tab and
its chats get the Network row. The Web add-on asks for tool-providers
only: it holds no egress grant. The manifest tests check that the
default grants entitle each add-on to what it asks for.

The proxy's refusal now sends the model to the Sandbox Settings tab
instead of Settings > Tools.

Assisted-by: Claude:claude-opus-5-5

* Keep the button of the network pointer row at its size

The row used the wide layout of the tool rows, so its Open settings
button stretched across the page. Use the plain setting row, which
puts the button on the right like the other controls.

Assisted-by: Claude:claude-opus-5-5

* Fix the add-on contract inventory and the apps Swagger fixture

The contracts case inventory missed the six host control specs, so
verify.sh rejected the Ginkgo report. List them.

The Swagger spec test never saw `controls` in the emitted UIView JSON
because its fixture left the omitempty field empty. Set it there.

Assisted-by: Claude:claude-sonnet-5-5

* Add the host controls spec to the apps inventory

The apps suite ran one more spec than scripts/addons/suites.json
listed, so verify.sh rejected the apps report as an inventory
mismatch. List the new spec in the report inventory and in the
suite's aggregate case list. Every other Ginkgo suite was re-run
and matches its inventory.

Assisted-by: Claude:claude-sonnet-5-5

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

.agents/api-endpoints-and-auth.md

@@ -41,7 +41,7 @@ Add routes in the appropriate file under `core/http/routes/`. The file you use d
 |------|----------|
 | `routes/openai.go` | OpenAI-compatible API endpoints (`/v1/...`) |
 | `routes/localai.go` | LocalAI-specific endpoints (`/api/...`, `/models/...`, `/backends/...`) |
-| `routes/agents.go` | Agent pool endpoints (`/api/agents/...`) |
+| `routes/studio.go` | Chats endpoints (`/api/chats/...`, including `/api/chats/admin/...` behind the admin middleware) and the agent tasks and jobs (`/api/agent/...`, feature `mcp_jobs`) |
 | `routes/auth.go` | Auth endpoints (`/api/auth/...`) |
 | `routes/ui_api.go` | UI backend API endpoints |
 
@@ -187,10 +187,10 @@ func MyHandler(c echo.Context) error {
 
 Middleware can be composed at different levels. Here are the patterns used in the codebase:
 
-### Group-level middleware (agents pattern)
+### Group-level middleware (collections pattern)
 ```go
 // All routes in the group share the middleware
-g := e.Group("/api/agents", poolReadyMw, agentsMw)
+g := e.Group("/api/collections", readyMw, collectionsMw)
 g.GET("", listHandler)
 g.POST("", createHandler)
 ```

.agents/building-and-testing.md

@@ -45,3 +45,13 @@ Rules (both gates):
 - **Don't weaken the gate:** never hand-lower a baseline or widen a tolerance to turn a red gate green. The ratchet only moves up.
 - If a change drops coverage, **add tests** (sort `coverage-summary.json` by line% ascending to find untested code) rather than editing the baseline. When coverage legitimately rises, commit the regenerated baseline (`make test-coverage-baseline` / `test-ui-coverage-baseline`).
 - The Go gate is **strict — no tolerance**; `covermode=atomic` keeps it deterministic. The UI gate keeps a small tolerance only because its e2e coverage isn't.
+
+## Other CI gates
+
+- **Add-on Go tests:** `TEST_PATHS` and `COVERAGE_ROOTS` include `./addons/...` and `./tests/addons/actione2e` on Linux (the Sandbox add-on does not build on macOS). The other `tests/addons/*` suites need fixtures that `scripts/addons/verify.sh` prepares and run in `.github/workflows/addons-tests.yml`. See `docs/content/development/addon-testing.md`.
+- **UI unit checks (`make test-ui-unit`, `tests-ui-unit.yml`):** eslint (errors only), the inline-style ratchet, the host UI unit tests, the unit tests of every `addons/*/ui` package and the locale ratchet.
+- **Locale ratchet (`make test-i18n`):** every locale must have every namespace and key of English. `core/http/react-ui/i18n-baseline.json` records the gaps that exist today as a count per locale and namespace; the count may only fall. A namespace file that a locale lacks fails the gate whatever the baseline says (the browser would get a 404 for it on every page load). After you add keys to a locale, run `node scripts/i18n-check.mjs --write` in `core/http/react-ui` and commit the lower baseline; `--report` lists every missing key.
+- **PostgreSQL stores:** `test.yml` starts a PostgreSQL service and sets `LOCALAI_TEST_POSTGRES_URL`; `make test-postgres` fails when any PostgreSQL store test skips.
+- **Shipped bundle:** specs tagged `@production-chunks` (stale chunk recovery, `e2e/production-bundle.spec.js`) run on the production code-split build through `make test-ui-stale-chunk`; the coverage build inlines dynamic imports and skips them.
+- **Distributed:** `tests-e2e-distributed.yml` runs `make test-e2e-distributed` (PostgreSQL and NATS in containers) and `make test-distributed-fakebus` (the in-process bus, race detector) nightly and on changes to the distributed code.
+- **Auth modes:** `core/http/route_authmodes_test.go` runs every `/api/chats` and `/api/addons` route as the local owner, with a shared API key, and with accounts (session, per-user API key, revoked feature, non-administrator).

.agents/distributed-seams.md

@@ -10,11 +10,12 @@ connections to a worker. Only the carrier files import `nats.go`
 | Seam | Interface | Lives in | Today |
 |---|---|---|---|
 | Fan-out | `messaging.Broadcaster` | `core/services/messaging` | NATS client |
-| Queues | `messaging.WorkQueue` (producer), `messaging.WorkConsumer` (worker), keyed by `messaging.WorkKind` | `core/services/messaging` (`workqueue.go`) | `NewNATSWorkQueue`, `NewNATSWorkConsumer` |
 | Control verbs, frontend half | `nodes.NodeCommandSender`, `nodes.FileStager` | `core/services/nodes` | `RemoteUnloaderAdapter`, `S3NATSFileStager` over NATS request/reply; `HTTPFileStager` over HTTP |
 | Control verbs, worker half | unexported `controlServer` (`handle`, `handleWithProgress`) | `core/services/worker` | `natsControlServer` |
 | Agent RPC, frontend half | `AgentControl` (package `mcp`) | `core/http/endpoints/mcp` | `nodes.NATSAgentControl` |
 | Agent RPC, worker half | unexported `agentRPCServer` | `core/cli/agent_worker.go` | `nodes.NATSAgentRPCServer` |
+| Chat start, frontend half | `studio.ChatWorkers` | `core/services/studio` | `nodes.NATSChatStarter` |
+| Chat start, worker half | `studio.Service.ServeStart`, served by `nodes.ChatStartHandler` | `core/application/studio_chatworker.go` | `nodes.NATSChatStartServer` |
 | Dial to a worker | `nodes.BackendClientFactory`, `nodes.ModelProber`, `nodes.WorkerNetDialerFor` | `core/services/nodes` | direct dial |
 
 Request and reply payloads of the control verbs live in
@@ -23,23 +24,36 @@ carrier share one wire format without importing each other.
 
 ## Queues
 
-`WorkKind` names the work (`WorkTask`, `WorkMCPCI`, `WorkAgentRun`).
-`natsRoute` in `workqueue_nats.go` is the one place a kind becomes a subject and
-a queue group. A nil error from `Enqueue` means the carrier accepted the
-payload, not that a consumer exists. The NATS carrier ignores the ctx of
-`Enqueue`.
+There are none. The agent-run, MCP CI and job queues were removed with the old
+agents and jobs; chat runs and triggers use the database and advisory locks
+(`.agents/distributed-state.md`). A new queue is a new interface in
+`core/services/messaging`, with its own subject root added to
+`broadcastRoots` in `subject_rules.go`.
 
-`Consume(ctx, kind, maxInFlight, h)` keeps two concurrency models on purpose.
-With `maxInFlight` 1 the handler runs inline on the NATS delivery goroutine and
-a panic is not recovered. Any other value spawns a recovered goroutine per
-delivery; when bounded, the slot is taken on the delivery goroutine. 0 and
-negative values are unbounded. `Unsubscribe` stops delivery, then waits for
-running handlers, so a handler must not call it. The agent worker asks for
-MCP CI with 1 (`startMCPCIConsumer`) and for agent runs with the dispatcher's
-`maxConcurrent` (0 from the CLI); specs pin both.
-`WithAgentRunRoute` lets an agent worker move the agent-run subject and group.
-An empty subject keeps `agent.execute`; an empty queue is kept and makes a
-plain subscription, as an explicitly empty `LOCALAI_AGENT_QUEUE` always did.
+The chat start is not a queue: it is a request/reply in a queue group, like the
+agent RPC. Nothing is buffered, and the reply says whether the turn started.
+
+## Chat start
+
+`studio.ChatWorkers.StartChat` hands the start of an unattended run (trigger,
+job, connector message) to one chat worker on `chats.start`, queue group
+`chat-workers`. The payloads are `workerctl.ChatStartRequest` and
+`ChatStartReply`. Only the deadline of ctx bounds the wait, as for the agent
+RPC. Without one the start's budget does: `LOCALAI_CHAT_START_TIMEOUT`
+(`nodes.WithChatStartTimeout`, default `nodes.DefaultChatStartTimeout`). The
+worker gives the handler the same budget.
+
+- `ErrNoRoute` (wrapped): no worker listens. `Service.SendBackground` runs the
+  turn on the frontend.
+- A timeout or a transport error: the worker may have started the turn. The
+  start fails and nothing runs on the frontend, so a message cannot run twice.
+- A reply with `Error` set: the worker's refusal. The start fails with
+  `studio.ErrWorkerRefused`.
+
+The worker half runs the handler inline on the delivery goroutine. A start
+only claims the turn and hands the message to the loop, so it is short; the
+turn runs in the background. Events and stop of the turn use the existing
+`chats.events.*` and `chats.control.*` subjects.
 
 ## Control verbs
 
@@ -150,8 +164,20 @@ other seams against a real server, also through Docker.
   `ForgetNode`.
 - `clientFor` returns no error; a carrier with no dialer for a node needs that
   path.
-- The agent worker still uses NATS directly for its connection and for agent
-  events (`agents.NewEventBridge`).
+- The agent worker still uses NATS directly for its connection.
+- A chat worker (`local-ai run` with `LOCALAI_CHAT_WORKER`) connects as a
+  frontend does, with the frontend's NATS credentials. It registers itself in
+  the node registry through the database (an approved `agent` node with
+  `node.role=chat-worker`, see `registerChatWorkerNode`), so the node list
+  shows it and the scheduler skips it. A drain does not reach it, and it gets
+  no minted JWT: that needs `chats.start` (sub, queue) and the
+  `chats.events.*` / `chats.control.*` subjects in `pkg/natsauth`.
+- A chat worker refuses a start that would take it past
+  `LOCALAI_CHAT_WORKER_MAX_TURNS` running turns (`studio.ErrWorkerFull`), and
+  the run fails. A full worker cannot pass a start on to another one: NATS
+  picks the worker of the queue group, not the frontend.
+- A start that carries files runs on the frontend: the uploads are not sent
+  over the bus.
 - `ReplicaReconcilerOptions` has no `ClientFactory` field. Without a
   `Prober`, `NewReplicaReconciler` builds its own `tokenClientFactory`, so a
   carrier that dials through another factory must add the field.
@@ -163,16 +189,5 @@ other seams against a real server, also through Docker.
 - `natsControlServer.subject` refuses a verb it has no subject for, so
   registration fails. A verb that only one carrier serves needs a per-carrier
   opt-out where the verbs are registered.
-- `WorkHandler` returns only an error. A carrier whose stream handler must send
-  a terminal reply derives it from the `jobs.<id>.result` event the handler
-  publishes on `events` (`handleMCPCIJob` does this).
-- Not additive: the agent-run consumer (`NATSDispatcher.runDelivery`) ignores
-  the per-delivery `events` publisher and publishes through the process-wide
-  `EventBridge`, which is built on the NATS client. A second carrier must
-  change `NATSDispatcher` and `EventBridge`, for example by binding
-  `handleJob`'s publishes to `events` through a bridge view that shares the
-  cancel registry.
 - `controlHandler`'s `undecodable` return is the recorded exception described
   under Control verbs.
-- Agent cancel has no production sender (`EventBridge.CancelExecution` has no
-  caller), so it is not part of the agent RPC seam.

.agents/distributed-state.md

@@ -87,6 +87,13 @@ Examples:
 - Sticky: the failover prober (`core/application/failover_distributed.go`),
   via `HeldLock`.
 
+A single runner may hand its work to another process and stay the runner. The
+chat hosts (triggers, connectors, jobs) keep their `HeldLock` on a frontend
+and, with chat workers on, send each turn to a worker (`studio.SendBackground`,
+`.agents/distributed-seams.md`). The turn claim in the database, not the lock,
+keeps one turn per chat, so the frontend and the worker use the same claim,
+heartbeat and reaper.
+
 ### 3. Stateless per request
 
 Nothing to share: the feature re-derives everything it needs from the

.agents/localai-assistant-mcp.md

@@ -35,6 +35,7 @@ If you ship a REST endpoint without (2) and (3), conversational admins won't see
    - `pkg/mcp/localaitools/server_test.go` adds the tool name to `expectedFullCatalog` and `expectedReadOnlyCatalog` (if read-only).
    - Tool dispatch is added to `TestEachToolDispatchesToClient`.
    - `pkg/mcp/localaitools/httpapi/client_test.go` covers the new HTTP path.
+- [ ] An admin route or setting that deliberately gets no tool goes in `adminRoutesWithoutTool` in `pkg/mcp/localaitools/coverage_test.go`, with the reason. The PR description repeats the reason.
 
 ## Adding a new skill recipe (no new tool)
 

.agents/preparing-a-release.md

@@ -11,6 +11,21 @@ A release is not finished when the tag is pushed. The GitHub release, the blog p
 
 A release without a post and without clips is incomplete, in the same way a user-facing code change without a docs update is incomplete.
 
+## Add-ons and the add-on catalog
+
+The first-party add-ons (Tables, Voice, Face, Fine-tuning, Quantization, Studio, Talk, Office, Sandbox) are separate OCI packages built by `.github/workflows/addons-package.yml`, not part of the LocalAI binary. A release ships them with the tag. A release build of LocalAI reads the add-on index attached to its own release and installs the packages that tag built; it must never fall back to the mutable development builds signed by master.
+
+What happens, and what to check:
+
+1. **Tag push (`v<semver>`).** The workflow packages every add-on for every platform enabled in `addons/<id>/package/platforms.yaml`, pushes `<id>-<version>` to the add-on registry, signs it with the workflow identity of the tag, and the `catalog` job renders `addons-index.yaml` (one pinned entry for each add-on and platform) as the artifact `addons-release-index`. No alias moves. Check that every `publish and sign` leg and the `render the release catalog` job are green, and that the entry count is the number of add-ons times their enabled platforms.
+2. **Sandbox guest image.** The `sandbox-image` job builds and signs the guest image; the Sandbox package is stamped with its digest (`SANDBOX_GUEST_IMAGE`). A publishing run fails when the digest is missing. Check the digest in the job summary and that the Sandbox page of a test install shows a guest image.
+3. **Release published (stable, not a draft).** The `promote` job moves `<id>-latest`, and the `attach the catalog to the release` job downloads `addons-release-index` from the tag run and uploads it to the release as `addons-index.yaml`. Release builds read that URL: `https://github.com/mudler/LocalAI/releases/download/v<version>/addons-index.yaml`. A release without this asset has no installable add-ons. The artifact is kept for 90 days, so publish the release within that time of the tag, or re-run the tag workflow.
+4. **Platforms.** A platform is enabled only after its native check passes on a runner of that platform. `scripts/addons/catalog.sh --check` fails the PR when `gallery/addons/index.yaml` does not list every enabled platform.
+5. **Smoke test.** On a clean data directory, start the release binary, open Add-ons, Find add-ons, and install Studio. It must resolve to the `<id>-<version>` tag and verify against the identity of the tag. Repeat on macOS and on linux/arm64 when the release covers them.
+6. **Sources.** The release binary and the container image read the model and backend libraries (see the cutover steps in `release.yaml` and the `LOCALAI_*_SOURCE` build arguments in the `Dockerfile`).
+
+The development entries in `gallery/addons/index.yaml` are regenerated with `sh scripts/addons/catalog.sh --channel development --repository quay.io/go-skynet/local-ai-addons --out gallery/addons/index.yaml`; they do not need a change per release.
+
 ## Clip conventions
 
 - MP4, H.264, no audio track unless the feature is about audio. Keep them short (10 to 30 seconds) and loopable.

.agents/testing-mcp-apps.md

@@ -1,117 +1,76 @@
 # Testing MCP Apps (Interactive Tool UIs)
 
-MCP Apps is an extension to MCP where tools declare interactive HTML UIs via `_meta.ui.resourceUri`. When the LLM calls such a tool, the UI renders the app in a sandboxed iframe inline in the chat. The app communicates bidirectionally with the host via `postMessage` (JSON-RPC) and can call server tools, send messages, and update model context.
+MCP Apps is an extension to MCP where tools declare interactive HTML UIs via `_meta.ui.resourceUri`. When the model calls such a tool, the chat shows the app in a sandboxed iframe under the tool line. The app talks to the host with `postMessage` (JSON-RPC) and can call tools of its own server.
 
 Spec: https://modelcontextprotocol.io/extensions/apps/overview
 
-## Quick Start: Run a Test MCP App Server
+## How it works in the chats page
 
-The `@modelcontextprotocol/server-basic-react` npm package is a ready-to-use test server that exposes a `get-time` tool with an interactive React clock UI. It requires Node >= 20, so run it in Docker:
+The MCP connection is on the **server**, not in the browser. There is no browser MCP client, no `AppBridge` and no CORS proxy in this path.
+
+1. A turn calls a tool that has `_meta.ui.resourceUri`. The tool wrapper (`core/services/studio/mcpapps.go`, `withApp`) runs the tool, keeps the full MCP result, and sets a card on the call with `session.SetToolCard`. The card holds the app reference: server (`connector:<id>` or the model's server name), tool, resource URI, visibility, input and result.
+2. The card is stored with the tool entry of the transcript. `GET /api/chats/{id}/messages` returns it as `app` (with `input` and `result`). The live `tool_end` event carries the reference only (the bus drops the payload) and `seq`, the number of that transcript entry. The entry is stored before the event is sent, in the shared store, so any frontend can serve it.
+   The page shows the app live: on `tool_end` it reads that one entry with `GET /api/chats/{id}/messages?after=<seq-1>&limit=1` and renders the app in the tool card at once. The live card and the transcript line have the same key (`tool-<seq>`, `src/utils/chatLive.js`) in one flat list, so the frame is not remounted when the turn ends and the transcript is read again. If the entry cannot be read, the card shows the result as text and the transcript read at the turn end brings the app.
+3. `ChatAppFrame.jsx` reads the page with `GET /api/chats/{id}/mcp-apps/resource?server=&uri=` and renders it in `<iframe sandbox="allow-scripts">` with a CSP meta tag put before the page. `src/utils/chatApps.js` (`createHost`) is the host side of the JSON-RPC bridge. A tool call of the page goes to `POST /api/chats/{id}/mcp-apps/call`.
+4. The server checks on every request that the chat may use the server now (chat owner, not switched off in the tools panel, agent grant, connector owned and enabled), that the resource is declared by a tool of that server, and that the tool's `_meta.ui.visibility` includes `"app"`.
+
+A tool whose visibility lacks `"model"` is not offered to the model.
+
+## Tests
+
+- Go, the rules of the routes: `core/services/studio/mcpapps_test.go` (a real MCP server over HTTP with a clock app, a connector and a model server) and `core/http/routes/studio_mcpapps_test.go` (status codes, auth, no secret in an answer).
+- Go, the parts: `pkg/mcpapps`, `core/http/endpoints/mcp/apps_test.go`, `core/services/studio/mcpconns/apps_test.go`, `core/services/studio/session/cardapp_test.go`.
+- UI units: `node tests/unit/run.mjs chatapps.test.mjs` (CSP, srcdoc, the host bridge, no DOM) and `chatlive.test.mjs` (live tool calls, stable keys, the merge of the live turn into the transcript).
+- Playwright, API stubbed: `e2e/chats-apps.spec.js` (sandbox attribute, bridge round trip, isolation probes, suggestions and links, foreign `postMessage`, fallback, phone width) and, with the event stream faked, the live cases: the app shows when `tool_end` arrives and the frame is not reloaded when the turn ends or the stream reconnects, and a card that cannot be read falls back to text.
+
+Playwright runs one process at a time, against the server on port 8089. Build the UI and the test server first:
+
+```bash
+cd core/http/react-ui && npx vite build
+cd ../../.. && go build -tags auth -o tests/e2e-ui/ui-test-server ./tests/e2e-ui
+cd core/http/react-ui && PLAYWRIGHT_CHROMIUM_PATH=/usr/bin/google-chrome PW_WORKERS=1 npx playwright test e2e/chats-apps.spec.js --project=chromium
+```
+
+## Try it by hand
+
+The `@modelcontextprotocol/server-basic-react` package is a ready-made test server with a `get-time` tool and a React clock UI. It needs Node >= 20, so run it in Docker:
 
 ```bash
 docker run -d --name mcp-app-test -p 3001:3001 node:22-slim \
   sh -c 'npx -y @modelcontextprotocol/server-basic-react'
-```
-
-Wait ~10 seconds for it to start, then verify:
-
-```bash
-# Check it's running
-docker logs mcp-app-test
-# Expected: "MCP server listening on http://localhost:3001/mcp"
-
-# Verify MCP protocol works
 curl -s -X POST http://localhost:3001/mcp \
-  -H 'Content-Type: application/json' \
-  -H 'Accept: application/json, text/event-stream' \
-  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"test","version":"1.0.0"}}}'
-
-# List tools — should show get-time with _meta.ui.resourceUri
-curl -s -X POST http://localhost:3001/mcp \
-  -H 'Content-Type: application/json' \
-  -H 'Accept: application/json, text/event-stream' \
+  -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
   -d '{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}'
+# get-time has "_meta": {"ui": {"resourceUri": "ui://get-time/mcp-app.html"}}
 ```
 
-The `tools/list` response should contain:
-```json
-{
-  "name": "get-time",
-  "_meta": {
-    "ui": { "resourceUri": "ui://get-time/mcp-app.html" }
-  }
-}
-```
+1. Run LocalAI with chats on and the `mcp` feature for your user. Connectors refuse private addresses by default; for a server on localhost an administrator sets `mcp_connectors_allow_private`.
+2. Open **Chats**, **Connectors**, **MCP server**, and add `http://localhost:3001/mcp`. Use **Test connection**, then enable it.
+3. Start a chat and send "What time is it?".
+4. The clock app appears under the `get-time` tool line as soon as the tool call ends, before the answer is complete.
 
-## Testing in LocalAI's UI
+## What to verify
 
-1. Make sure LocalAI is running (e.g. `http://localhost:8080`)
-2. Build the React UI: `cd core/http/react-ui && npm install && npm run build`
-3. Open the Chat page in your browser
-4. Click **"Client MCP"** in the chat header
-5. Add a new client MCP server:
-   - **URL**: `http://localhost:3001/mcp`
-   - **Use CORS proxy**: enabled (default) — required because the browser can't hit `localhost:3001` directly due to CORS; LocalAI's proxy at `/api/cors-proxy` handles it
-6. The server should connect and discover the `get-time` tool
-7. Select a model and send: **"What time is it?"**
-8. The LLM should call the `get-time` tool
-9. The tool result should render the interactive React clock app in an iframe as a standalone chat message (not inside the collapsed activity group)
+- [ ] The tool is offered to the model; a tool with `visibility: ["app"]` is not.
+- [ ] The app shows under its tool line as soon as the call ends (during the turn), is not reloaded or duplicated when the turn ends, and shows again after a reload of the page.
+- [ ] The iframe has `sandbox="allow-scripts"` and nothing else, and the page cannot read the host (no cookies, no storage, no parent DOM, no network).
+- [ ] A click in the app that calls a tool reaches the server (`POST .../mcp-apps/call`) and the result comes back.
+- [ ] `ui/message` shows a suggestion that you send or dismiss; `ui/open-link` shows an Open link that works after a click and refuses `javascript:` and `data:`.
+- [ ] Switching the connector off in the tools panel makes the page and its calls answer `403`.
+- [ ] A page that cannot load shows the result as text.
+- [ ] Non-UI tools keep working with text results.
 
-## What to Verify
+Benign console output: `Source map error: ... about:srcdoc` (devtools cannot find source maps for srcdoc iframes).
 
-- [ ] Tool appears in the connected tools list (not filtered — `get-time` is callable by the LLM)
-- [ ] The iframe renders as a standalone chat message with a puzzle-piece icon
-- [ ] The app loads and is interactive (clock UI, buttons work)
-- [ ] No "Reconnect to MCP server" overlay (connection is live)
-- [ ] Console logs show bidirectional communication:
-  - `tools/call` messages from app to host (app calling server tools)
-  - `ui/message` notifications (app sending messages)
-- [ ] After the app renders, the LLM continues and produces a text response with the time
-- [ ] Non-UI tools continue to work normally (text-only results)
-- [ ] Page reload shows the HTML statically with a reconnect overlay until you reconnect
+## Limits to know
 
-## Console Log Patterns
+- MCP servers of the model (not connectors) serve apps only when the frontend holds their sessions. In distributed mode they run on workers and the routes answer `409 mcp_app_unsupported`. Connectors work in both modes.
+- Browser permissions an app asks for (camera, microphone) are not granted.
+- Out of scope: OAuth, apps from add-on tools (add-ons have their own UI slots), apps in triggers or connectors (no UI there).
 
-Healthy bidirectional communication looks like:
+## Other test servers
 
-```
-Parsed message { jsonrpc: "2.0", id: N, result: {...} }     // Bridge init
-get-time result: { content: [...] }                          // Tool result received
-Calling get-time tool...                                     // App calls tool
-Sending message { method: "tools/call", ... }                // App -> host -> server
-Parsed message { jsonrpc: "2.0", id: N, result: {...} }     // Server response
-Sending message text to Host: ...                            // App sends message
-Sending message { method: "ui/message", ... }                // Message notification
-Message accepted                                             // Host acknowledged
-```
-
-Benign warnings to ignore:
-- `Source map error: ... about:srcdoc` — browser devtools can't find source maps for srcdoc iframes
-- `Ignoring message from unknown source` — duplicate postMessage from iframe navigation
-- `notifications/cancelled` — app cleaning up previous requests
-
-## Architecture Notes
-
-- **No server-side changes needed** — the MCP App protocol runs entirely in the browser
-- `PostMessageTransport` wraps `window.postMessage` between host and `srcdoc` iframe
-- `AppBridge` (from `@modelcontextprotocol/ext-apps`) auto-forwards `tools/call`, `resources/read`, `resources/list` from the app to the MCP server via the host's `Client`
-- The iframe uses `sandbox="allow-scripts allow-forms"` (no `allow-same-origin`) — opaque origin, no access to host cookies/DOM/localStorage
-- App-only tools (`_meta.ui.visibility: "app-only"`) are filtered from the LLM's tool list but remain callable by the app iframe
-
-## Key Files
-
-- `core/http/react-ui/src/components/MCPAppFrame.jsx` — iframe + AppBridge component
-- `core/http/react-ui/src/hooks/useMCPClient.js` — MCP client hook with app UI helpers (`hasAppUI`, `getAppResource`, `getClientForTool`, `getToolDefinition`)
-- `core/http/react-ui/src/hooks/useChat.js` — agentic loop, attaches `appUI` to tool_result messages
-- `core/http/react-ui/src/pages/Chat.jsx` — renders MCPAppFrame as standalone chat messages
-
-## Other Test Servers
-
-The `@modelcontextprotocol/ext-apps` repo has many example servers:
-- `@modelcontextprotocol/server-basic-react` — simple clock (React)
-- More examples at https://github.com/modelcontextprotocol/ext-apps/tree/main/examples
-
-All examples support both stdio and HTTP transport. Run without `--stdio` for HTTP mode on port 3001.
+The `@modelcontextprotocol/ext-apps` repo has many example servers: https://github.com/modelcontextprotocol/ext-apps/tree/main/examples . They run over HTTP on port 3001 without `--stdio`.
 
 ## Cleanup
 

.github/workflows/addons-package.yml

@@ -2,7 +2,8 @@
 name: 'add-on packages'
 
 # Builds, checks, signs and publishes the first-party add-on OCI packages
-# (Tables, Voice, Face, Fine-tuning and Quantization). This workflow is
+# (Tables, Voice, Face, Fine-tuning, Quantization, Studio, Talk, Telegram, Slack, Office and
+# Sandbox). This workflow is
 # independent of the backend image workflows: it has its own registry input,
 # its own signer identity and its own jobs.
 #
@@ -27,6 +28,23 @@ name: 'add-on packages'
 #
 # Aliases move only in the promote job, after every platform leg passed, the
 # index was pushed by digest, signed and verified.
+#
+# Catalog. gallery/addons/index.yaml holds the development entries (mutable, one
+# for each enabled platform) and is generated by scripts/addons/catalog.sh; the
+# validate job fails when the committed file is stale. A v<semver> tag push
+# also renders the release index, with entries pinned by digest and by the
+# immutable tag and signed by the run of the tag, and keeps it as the artifact
+# addons-release-index. When the release is published, the release-index job
+# attaches it to the GitHub release as addons-index.yaml. A release build of
+# LocalAI reads that file (core/services/addons/catalog/official.go), so it
+# installs the packages its own release built, not the moving development ones.
+#
+# Sandbox guest image. The sandbox-image job (sandbox-images.yml) builds and
+# signs the guest image and exposes its reference by digest as the output
+# "image" (<repository>@sha256:<digest>). The package job passes it to the
+# Sandbox package as SANDBOX_GUEST_IMAGE; a publishing event fails without it.
+# An event that publishes nothing pushes no image, so its Sandbox leg is a
+# local-check build.
 
 on:
   push:
@@ -38,7 +56,13 @@ on:
       - 'scripts/addons/**'
       - 'gallery/addons/**'
       - 'tests/addons/packaging/**'
+      - 'core/services/addons/**'
+      - 'core/http/react-ui/package.json'
+      - 'core/http/react-ui/package-lock.json'
+      - 'go.mod'
+      - 'go.sum'
       - '.github/workflows/addons-package.yml'
+      - '.github/workflows/sandbox-images.yml'
   pull_request:
     paths:
       - 'addons/**'
@@ -46,7 +70,13 @@ on:
       - 'scripts/addons/**'
       - 'gallery/addons/**'
       - 'tests/addons/packaging/**'
+      - 'core/services/addons/**'
+      - 'core/http/react-ui/package.json'
+      - 'core/http/react-ui/package-lock.json'
+      - 'go.mod'
+      - 'go.sum'
       - '.github/workflows/addons-package.yml'
+      - '.github/workflows/sandbox-images.yml'
   release:
     types: [published]
 
@@ -63,7 +93,7 @@ jobs:
     strategy:
       fail-fast: true
       matrix:
-        addon: [tables, voice, face, fine-tuning, quantization, studio, talk]
+        addon: [tables, voice, face, fine-tuning, quantization, studio, talk, telegram, slack, office, sandbox, web]
     steps:
       - uses: actions/checkout@v7
         with:
@@ -103,6 +133,22 @@ jobs:
       - name: Run the packaging checks
         if: matrix.addon == 'tables'
         run: sh scripts/addons/package_test.sh
+      - name: Check that the committed catalog is the generated one
+        # One entry for each enabled platform of each add-on.
+        if: matrix.addon == 'tables'
+        run: sh scripts/addons/catalog.sh --channel development --check
+
+  sandbox-image:
+    name: sandbox guest image
+    # Builds the guest image of the Sandbox add-on. The reusable workflow
+    # decides from the event whether it pushes and signs, as this workflow
+    # does, and exposes the reference of the image by digest as "digest".
+    uses: ./.github/workflows/sandbox-images.yml
+    secrets: inherit
+    permissions:
+      contents: read
+      packages: write
+      id-token: write
 
   plan:
     name: plan the event
@@ -166,7 +212,7 @@ jobs:
           # event it fails the plan. A missing or malformed platform file, or
           # any other error, fails the plan on every event.
           include='[]'
-          for id in tables voice face fine-tuning quantization studio talk; do
+          for id in tables voice face fine-tuning quantization studio talk telegram slack office sandbox web; do
             cfg="addons/$id/package/platforms.yaml"
             err="matrix-$id.err"
             if rows=$(go run ./scripts/addons/cmd/addonpkg matrix --config "$cfg" 2>"$err"); then
@@ -198,7 +244,7 @@ jobs:
 
   package:
     name: package ${{ matrix.addon }} ${{ matrix.platform }}
-    needs: [plan, validate]
+    needs: [plan, validate, sandbox-image]
     if: needs.plan.outputs.has_platforms == 'true'
     runs-on: ${{ matrix.runner }}
     timeout-minutes: 60
@@ -223,14 +269,43 @@ jobs:
         run: |
           npm ci --prefix core/http/react-ui
           node "addons/$ADDON/ui/build.mjs"
+      - name: Fetch the runtime files
+        env:
+          ADDON: ${{ matrix.addon }}
+          PLATFORM: ${{ matrix.platform }}
+        run: |
+          # An add-on that ships files besides its executable describes where
+          # they come from; the script checks their checksums.
+          if [ -f "addons/$ADDON/package/fetch-runtime.sh" ]; then
+            sh "addons/$ADDON/package/fetch-runtime.sh" "$PLATFORM" "$RUNNER_TEMP/runtime"
+          fi
       - name: Package this platform
         env:
           ADDON: ${{ matrix.addon }}
           VERSION: ${{ needs.plan.outputs.version }}
           PLATFORM: ${{ matrix.platform }}
           SHA: ${{ github.sha }}
+          PUBLISH: ${{ needs.plan.outputs.publish }}
+          # The guest image of the Sandbox add-on, by digest. scripts/addons/
+          # package.sh stamps it into the executable.
+          SANDBOX_GUEST_IMAGE: ${{ matrix.addon == 'sandbox' && needs.sandbox-image.outputs.image || '' }}
         run: |
-          sh scripts/addons/package.sh --config "addons/$ADDON/package/platforms.yaml" --layout "layout" --version "$VERSION" --platform "$PLATFORM" --revision "$SHA"
+          set -- --config "addons/$ADDON/package/platforms.yaml" --layout "layout" --version "$VERSION" --platform "$PLATFORM" --revision "$SHA"
+          if [ "$ADDON" = sandbox ] && [ "$PUBLISH" = true ]; then
+            # Fail closed: a published Sandbox package without a guest image
+            # can run the host backend only.
+            case "$SANDBOX_GUEST_IMAGE" in
+              *@sha256:????????????????????????????????????????????????????????????????) ;;
+              *) echo "SANDBOX_GUEST_IMAGE must be <repository>@sha256:<digest>; the sandbox-image job gave '$SANDBOX_GUEST_IMAGE'" >&2; exit 1 ;;
+            esac
+          elif [ "$ADDON" = sandbox ] && [ -z "$SANDBOX_GUEST_IMAGE" ]; then
+            # An event that publishes nothing pushes no guest image, so there
+            # is no digest to stamp. Build a local-check leg: package.sh then
+            # uses a stand-in reference and the receipt is not publishable.
+            set -- "$@" --local-check
+          fi
+          if [ -d "$RUNNER_TEMP/runtime" ]; then set -- "$@" --runtime-dir "$RUNNER_TEMP/runtime"; fi
+          sh scripts/addons/package.sh "$@"
       - name: Native runtime check
         env:
           ADDON: ${{ matrix.addon }}
@@ -332,6 +407,14 @@ jobs:
               --certificate-oidc-issuer "$ISSUER" \
               "${ADDON_OCI_REPOSITORY}@${d}" > /dev/null
           done
+      - name: Keep the merged layout for the catalog job
+        # The release index is rendered from the layout that was pushed.
+        uses: actions/upload-artifact@v7
+        with:
+          name: merged-${{ matrix.addon }}
+          path: merged
+          if-no-files-found: error
+          retention-days: 7
       - name: Keep the digest for the promote job
         # A matrix job cannot hand one output to the next job, so the digest of
         # each add-on travels in an artifact of its own.
@@ -403,3 +486,79 @@ jobs:
               --digest "$DIGEST" --version "$VERSION" --revision "$SHA" \
               --ref "$REF" --event "$EVENT" --git-dir .git
           done
+
+  catalog:
+    name: render the release catalog
+    # Only the push of a v<semver> tag: the entries are pinned to the version
+    # that run published and signed. A release event rebuilds the packages and
+    # adopts the published ones, so its own layout is not the pinned content.
+    needs: [plan, publish]
+    if: needs.plan.outputs.publish == 'true' && github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
+    runs-on: ubuntu-24.04
+    timeout-minutes: 20
+    permissions:
+      contents: read
+    env:
+      ADDON_OCI_REPOSITORY: ${{ vars.ADDON_OCI_REPOSITORY }}
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+      - uses: actions/setup-go@v6
+        with:
+          go-version-file: go.mod
+      - uses: actions/download-artifact@v8
+        with:
+          pattern: merged-*
+          path: downloaded
+      - name: Render the entries of every add-on
+        env:
+          VERSION: ${{ needs.plan.outputs.version }}
+          REF: ${{ github.ref }}
+          OWNER_REPO: ${{ github.repository }}
+        run: |
+          set -eu
+          mkdir roots
+          for d in downloaded/merged-*; do
+            mv "$d" "roots/${d##*/merged-}"
+          done
+          sh scripts/addons/catalog.sh --channel release --repository "$ADDON_OCI_REPOSITORY" \
+            --version "$VERSION" --signer-ref "$REF" --owner-repo "$OWNER_REPO" \
+            --merged-root roots --out addons-index.yaml
+          grep -c '^- id:' addons-index.yaml
+      - uses: actions/upload-artifact@v7
+        with:
+          name: addons-release-index
+          path: addons-index.yaml
+          if-no-files-found: error
+          # The longest the service keeps an artifact. The release must be
+          # published within this time of the tag.
+          retention-days: 90
+
+  release-index:
+    name: attach the catalog to the release
+    # The release is public from here on, so the asset can be attached whether
+    # or not the release was a draft when the tag was pushed.
+    needs: [plan]
+    if: github.event_name == 'release' && !github.event.release.draft
+    runs-on: ubuntu-24.04
+    timeout-minutes: 10
+    permissions:
+      contents: write
+      actions: read
+    env:
+      GH_TOKEN: ${{ github.token }}
+      GH_REPO: ${{ github.repository }}
+    steps:
+      - name: Find the tag run and attach its release index
+        env:
+          TAG: ${{ github.event.release.tag_name }}
+          SHA: ${{ github.sha }}
+        run: |
+          set -eu
+          run=$(gh run list --workflow addons-package.yml --event push --branch "$TAG" --status success \
+            --json databaseId,headSha --jq "[.[] | select(.headSha == \"$SHA\")][0].databaseId // empty")
+          [ -n "$run" ] || { echo "no successful add-on package run for the tag $TAG at $SHA" >&2; exit 1; }
+          gh run download "$run" --name addons-release-index --dir index
+          test -s index/addons-index.yaml
+          gh release upload "$TAG" index/addons-index.yaml --clobber

.github/workflows/addons-tests.yml

@@ -0,0 +1,130 @@
+---
+name: 'Add-on Tests'
+
+# Runs the suites of scripts/addons/suites.json: the add-on contract, gateway,
+# policy, recovery, upgrade and host-slot suites, the add-on library sources,
+# the Tables suites and the browser specs of the add-on pages. verify.sh checks
+# each suite against its exact case inventory and fails on a skipped, flaky or
+# missing case.
+#
+# The Go tests under addons/ and the tests/addons/actione2e suite run in the
+# main test workflow (make test, make test-coverage-check). The packaging suite
+# runs in addons-package.yml. The UI unit tests of every add-on run in
+# tests-ui-unit.yml.
+on:
+  pull_request:
+    paths:
+      - 'addons/**'
+      - 'tests/addons/**'
+      - 'tests/e2e-ui/**'
+      - 'tests/e2e/mock-backend/**'
+      - 'pkg/addons/**'
+      - 'core/services/addons/**'
+      - 'core/http/endpoints/addons/**'
+      - 'core/http/auth/addons*.go'
+      - 'core/application/**'
+      - 'core/library/**'
+      - 'core/http/react-ui/**'
+      - 'scripts/addons/**'
+      - 'go.mod'
+      - 'go.sum'
+      - '.github/workflows/addons-tests.yml'
+  push:
+    branches:
+      - master
+
+concurrency:
+  group: ci-addons-tests-${{ github.event.pull_request.number || github.sha }}-${{ github.repository }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+permissions:
+  contents: read
+
+jobs:
+  suites:
+    name: ${{ matrix.group }}
+    runs-on: ubuntu-24.04
+    timeout-minutes: 60
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          # Go suites over the owner, the carrier, the gateway and the policy.
+          - group: contracts and gateway
+            suites: prerequisites contracts catalog packages lifecycle runtime gateway policy proposals artifacts apps
+          # Browser suites over the host pages of the add-ons.
+          - group: host pages
+            suites: gallery-ui esm-ui host-slots library-sources updates recovery
+          # Two frontends and an owner, with real authentication.
+          - group: foundation
+            suites: foundation-acceptance
+          # The Tables add-on: stores, tools and administration.
+          - group: tables
+            suites: tables-databases tables-schema tables-rows tables-tools tables-admin
+          # The Tables add-on: pages, CSV export and the end-to-end acceptance.
+          - group: tables pages
+            suites: tables-ui tables-csv tables-acceptance
+    steps:
+      - name: Clone
+        uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+      - name: Setup Go
+        uses: actions/setup-go@v6
+        with:
+          # prepare.sh accepts Go 1.26.x only and runs with GOTOOLCHAIN=local.
+          go-version: '1.26.x'
+          cache: false
+      - name: Setup Node.js
+        uses: actions/setup-node@v7
+        with:
+          # prepare.sh accepts Node 22.12 or later within Node 22.
+          node-version: '22'
+      - name: Install system dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y build-essential libopus-dev unzip curl
+      - name: Install protoc
+        # prepare.sh requires exactly libprotoc 26.1.
+        run: |
+          curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip
+          sudo unzip -j -d /usr/local/bin protoc.zip bin/protoc
+          rm protoc.zip
+          protoc --version
+      - name: Prepare the toolchain
+        # Downloads the Go modules, installs the pinned protobuf plugins, runs
+        # npm ci and installs the Playwright Chromium. Nothing later reaches
+        # the network.
+        run: sh scripts/addons/prepare.sh --prepare
+      - name: Install the browser's system libraries
+        run: sudo env "PATH=$PATH" core/http/react-ui/node_modules/.bin/playwright install-deps chromium
+      - name: Build the UI, the mock backend and the UI test server
+        run: sh scripts/addons/prepare.sh --integration
+      - name: Run the suites
+        env:
+          SUITES: ${{ matrix.suites }}
+        run: |
+          failed=""
+          for suite in $SUITES; do
+            echo "::group::$suite"
+            if ! sh scripts/addons/verify.sh "$suite"; then
+              failed="$failed $suite"
+              echo "::error::suite $suite failed"
+            fi
+            echo "::endgroup::"
+          done
+          if [ -n "$failed" ]; then
+            echo "failed suites:$failed" >&2
+            exit 1
+          fi
+      - name: Upload the suite logs
+        if: ${{ failure() }}
+        uses: actions/upload-artifact@v7
+        with:
+          name: addons-suite-logs-${{ strategy.job-index }}
+          path: |
+            .cache/addons/logs/
+            core/http/react-ui/playwright-report/
+            core/http/react-ui/test-results/
+          if-no-files-found: ignore
+          retention-days: 7

.github/workflows/sandbox-images.yml

@@ -0,0 +1,120 @@
+---
+name: 'sandbox guest image'
+
+# Builds the guest image of the Sandbox add-on for linux/amd64 and linux/arm64
+# (addons/sandbox/images/guest/Dockerfile) and pushes it to the project
+# registry, the one the other image jobs use (quay.io/go-skynet). The release
+# of the add-on stamps the digest into the executable:
+#
+#   SANDBOX_GUEST_IMAGE=<repository>@sha256:<digest> sh scripts/addons/package.sh ...
+#
+# This workflow outputs that value as "image" (and the bare "digest"), so a
+# caller workflow can run it with `uses:` and pass the output to the package
+# legs of the sandbox add-on.
+#
+# Event policy:
+#   pull_request          build both architectures, push nothing, no secrets
+#   push to master        build and push, tag sha-<short sha>
+#   workflow_dispatch     build and push, tag sha-<short sha>
+#   workflow_call         as the event of the caller: build and push, tag
+#                         sha-<short sha>, except for a pull request and for a
+#                         push to a branch other than master, which push nothing
+#
+# The repository is quay.io/go-skynet/sandbox-guest unless the repository
+# variable SANDBOX_GUEST_REPOSITORY names another one. The repository must
+# allow anonymous pulls: the add-on pulls the image on the host that runs it.
+
+on:
+  workflow_call:
+    outputs:
+      image:
+        description: 'Reference of the guest image by digest: <repository>@sha256:<digest>'
+        value: ${{ jobs.build.outputs.image }}
+      digest:
+        description: 'Digest of the guest image index'
+        value: ${{ jobs.build.outputs.digest }}
+  workflow_dispatch:
+  push:
+    branches: [master]
+    paths:
+      - 'addons/sandbox/images/guest/**'
+      - '.github/workflows/sandbox-images.yml'
+  pull_request:
+    paths:
+      - 'addons/sandbox/images/guest/**'
+      - '.github/workflows/sandbox-images.yml'
+
+permissions:
+  contents: read
+
+# The group names the workflow that runs: for a workflow_call it is the
+# caller, so the run of this file and the run that add-on packages calls for
+# the same pull request do not cancel each other.
+concurrency:
+  group: sandbox-images-${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  build:
+    name: build and push the guest image
+    runs-on: ubuntu-24.04
+    timeout-minutes: 120
+    outputs:
+      image: ${{ steps.ref.outputs.image }}
+      digest: ${{ steps.build.outputs.digest }}
+    env:
+      REPOSITORY: ${{ vars.SANDBOX_GUEST_REPOSITORY || 'quay.io/go-skynet/sandbox-guest' }}
+      # A caller's push to a branch other than master publishes nothing
+      # (scripts/addons/channel.sh), so it pushes no guest image either.
+      PUSH: ${{ github.repository == 'mudler/LocalAI' && github.event_name != 'pull_request' && !(github.event_name == 'push' && startsWith(github.ref, 'refs/heads/') && github.ref != 'refs/heads/master') }}
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+      - name: Free disk space
+        uses: ./.github/actions/free-disk-space
+      # The kernel builds on the build machine and cross-compiles. Only the
+      # userland stage of the arm64 image needs emulation.
+      - uses: docker/setup-qemu-action@master
+        with:
+          platforms: arm64
+      - uses: docker/setup-buildx-action@master
+      - name: Registry login
+        if: env.PUSH == 'true'
+        uses: docker/login-action@v4
+        with:
+          registry: quay.io
+          username: ${{ secrets.LOCALAI_REGISTRY_USERNAME }}
+          password: ${{ secrets.LOCALAI_REGISTRY_PASSWORD }}
+      - name: Build
+        id: build
+        uses: docker/build-push-action@v7
+        with:
+          context: .
+          file: ./addons/sandbox/images/guest/Dockerfile
+          platforms: linux/amd64,linux/arm64
+          # No attestation manifests: the add-on reads the index and picks the
+          # manifest of its own architecture.
+          provenance: false
+          sbom: false
+          cache-from: type=registry,ref=quay.io/go-skynet/ci-cache:cache-sandbox-guest
+          cache-to: ${{ env.PUSH == 'true' && 'type=registry,ref=quay.io/go-skynet/ci-cache:cache-sandbox-guest,mode=max,ignore-error=true' || '' }}
+          tags: ${{ env.REPOSITORY }}:sha-${{ github.sha }}
+          push: ${{ env.PUSH == 'true' }}
+      - name: Name the image by digest
+        id: ref
+        if: env.PUSH == 'true'
+        env:
+          DIGEST: ${{ steps.build.outputs.digest }}
+        run: |
+          set -eu
+          case "$DIGEST" in
+          sha256:????????????????????????????????????????????????????????????????) ;;
+          *) echo "the build returned no image digest: '$DIGEST'" >&2; exit 1 ;;
+          esac
+          echo "image=${REPOSITORY}@${DIGEST}" >> "$GITHUB_OUTPUT"
+          {
+            echo '### Sandbox guest image'
+            echo
+            echo "SANDBOX_GUEST_IMAGE=${REPOSITORY}@${DIGEST}"
+          } >> "$GITHUB_STEP_SUMMARY"

.github/workflows/sandbox-tests.yml

@@ -0,0 +1,224 @@
+---
+name: 'sandbox tests'
+
+# Runs what the other workflows leave out for the Sandbox add-on:
+#   unit      the Go tests of addons/sandbox, the real-add-on end to end suite
+#             (tests/addons/actione2e), the UI unit tests of the add-on and the
+#             check of the privilege drop in entrypoint.sh
+#   gvisor    the gVisor integration suite, on amd64 and arm64, with the pinned
+#             runsc, its AppArmor profile loaded and the guest image built on
+#             the runner; then the release smoke test with the host required to
+#             run commands in gVisor
+#   kvm       the KVM integration suite, on a runner that has /dev/kvm. It
+#             skips cleanly, and says so, when the runner has none.
+#
+# The integration suites need SANDBOX_TEST_RUNSC and SANDBOX_TEST_IMAGE (gVisor)
+# or SANDBOX_TEST_IMAGE (KVM). When these are set, a backend that cannot start
+# fails the test: the suites must not pass by skipping.
+
+on:
+  workflow_dispatch:
+  push:
+    branches: [master]
+    paths:
+      - 'addons/sandbox/**'
+      - 'tests/addons/actione2e/**'
+      - 'scripts/addons/sandbox_smoke.sh'
+      - 'scripts/build/entrypoint-privileges_test.sh'
+      - 'entrypoint.sh'
+      - '.github/workflows/sandbox-tests.yml'
+  pull_request:
+    paths:
+      - 'addons/sandbox/**'
+      - 'tests/addons/actione2e/**'
+      - 'scripts/addons/sandbox_smoke.sh'
+      - 'scripts/build/entrypoint-privileges_test.sh'
+      - 'entrypoint.sh'
+      - '.github/workflows/sandbox-tests.yml'
+
+permissions:
+  contents: read
+
+concurrency:
+  group: sandbox-tests-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  unit:
+    name: unit and end to end
+    runs-on: ubuntu-24.04
+    timeout-minutes: 40
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+      - uses: actions/setup-go@v6
+        with:
+          go-version-file: go.mod
+      - uses: actions/setup-node@v7
+        with:
+          node-version: '22'
+      - name: Generate the gRPC stubs the host packages import
+        run: |
+          curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip
+          sudo unzip -j -d /usr/local/bin protoc.zip bin/protoc
+          rm protoc.zip
+          go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
+          go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
+          PATH="$PATH:$HOME/go/bin" make protogen-go
+      - name: Go tests of the add-on
+        run: go test -tags auth -count=1 ./addons/sandbox/...
+      - name: End to end with the real add-on process and the real supervisor
+        run: go test -tags auth -count=1 ./tests/addons/actione2e/...
+      - name: UI unit tests
+        run: |
+          npm ci --prefix core/http/react-ui
+          node addons/sandbox/ui/test/run.mjs
+      - name: Privilege drop of the image entrypoint
+        run: bash scripts/build/entrypoint-privileges_test.sh
+
+  gvisor:
+    name: gvisor ${{ matrix.arch }}
+    runs-on: ${{ matrix.runner }}
+    timeout-minutes: 90
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - arch: amd64
+            runner: ubuntu-24.04
+          - arch: arm64
+            runner: ubuntu-24.04-arm
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+      - uses: actions/setup-go@v6
+        with:
+          go-version-file: go.mod
+      - uses: actions/setup-node@v7
+        with:
+          node-version: '22'
+      - name: Generate the gRPC stubs the host packages import
+        run: |
+          case "$(uname -m)" in aarch64) pb=aarch_64 ;; *) pb=x86_64 ;; esac
+          curl -L -s "https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-$pb.zip" -o protoc.zip
+          sudo unzip -j -d /usr/local/bin protoc.zip bin/protoc
+          rm protoc.zip
+          go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
+          go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
+          PATH="$PATH:$HOME/go/bin" make protogen-go
+      - name: Fetch the pinned runtime files
+        run: sh addons/sandbox/package/fetch-runtime.sh "linux/${{ matrix.arch }}" "$RUNNER_TEMP/runtime"
+      - name: Install runsc where its AppArmor profile names it
+        run: |
+          set -eu
+          mkdir -p "$HOME/gvisor"
+          cp "$RUNNER_TEMP/runtime/runsc" "$HOME/gvisor/runsc"
+          cp -r "$RUNNER_TEMP/runtime/gvisor-bin" "$HOME/gvisor/gvisor-bin"
+          chmod -R a+rx "$HOME/gvisor"
+          # The administrator step of the documentation: print the profile of
+          # this runsc and load it.
+          "$RUNNER_TEMP/runtime/gvisor-profile" "$HOME/gvisor/runsc" | sudo tee /etc/apparmor.d/localai-runsc
+          sudo apparmor_parser -r /etc/apparmor.d/localai-runsc
+      - uses: docker/setup-buildx-action@master
+      - name: Build the guest image as an OCI layout
+        id: image
+        run: |
+          set -eu
+          docker buildx build --provenance=false -f addons/sandbox/images/guest/Dockerfile \
+            --platform "linux/${{ matrix.arch }}" --output "type=oci,dest=$RUNNER_TEMP/guest.tar" .
+          mkdir -p "$RUNNER_TEMP/guest-layout"
+          tar -xf "$RUNNER_TEMP/guest.tar" -C "$RUNNER_TEMP/guest-layout"
+          digest=$(jq -r '.manifests[0].digest' "$RUNNER_TEMP/guest-layout/index.json")
+          case "$digest" in sha256:*) ;; *) echo "no manifest digest in the layout: $digest" >&2; exit 1 ;; esac
+          echo "ref=oci-layout:$RUNNER_TEMP/guest-layout@$digest" >> "$GITHUB_OUTPUT"
+      - name: gVisor integration suite
+        env:
+          SANDBOX_TEST_IMAGE: ${{ steps.image.outputs.ref }}
+        run: |
+          export SANDBOX_TEST_RUNSC="$HOME/gvisor/runsc"
+          go test -tags auth -count=1 -timeout 45m ./addons/sandbox/internal/sandbox/gvisor/
+      - name: Install the host UI build tools
+        # The smoke test packages the add-on, and the package build bundles
+        # the add-on UI with the bundler the host UI installs.
+        run: npm ci --prefix core/http/react-ui
+      - name: Release smoke test
+        env:
+          SANDBOX_GUEST_IMAGE: ${{ steps.image.outputs.ref }}
+          # The backends keep unix sockets under the test's data folder, and
+          # a socket path has at most 107 bytes: the default scratch folder in
+          # the checkout leaves too little room for it.
+          SANDBOX_SMOKE_WORK: /tmp/sandbox-smoke
+        run: |
+          set -eu
+          # The add-on copies runsc to its data folder, whose path changes with
+          # each run, so the profile of the fixed path does not apply to it.
+          # The runner allows the user namespaces instead.
+          sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
+          sh scripts/addons/sandbox_smoke.sh --runtime-dir "$RUNNER_TEMP/runtime" --require isolated
+
+  kvm:
+    name: kvm amd64
+    runs-on: ubuntu-24.04
+    timeout-minutes: 90
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+      - name: Look for /dev/kvm
+        id: kvm
+        run: |
+          set -u
+          echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules >/dev/null
+          sudo udevadm control --reload-rules || true
+          sudo udevadm trigger --name-match=kvm || true
+          if [ -w /dev/kvm ]; then
+            echo "available=true" >> "$GITHUB_OUTPUT"
+          else
+            echo "available=false" >> "$GITHUB_OUTPUT"
+            echo "::notice::This runner has no usable /dev/kvm; the KVM suite is skipped."
+          fi
+      - uses: actions/setup-go@v6
+        if: steps.kvm.outputs.available == 'true'
+        with:
+          go-version-file: go.mod
+      - name: Generate the gRPC stubs the host packages import
+        if: steps.kvm.outputs.available == 'true'
+        run: |
+          curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip
+          sudo unzip -j -d /usr/local/bin protoc.zip bin/protoc
+          rm protoc.zip
+          go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
+          go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
+          PATH="$PATH:$HOME/go/bin" make protogen-go
+      - name: Install the pinned host programs
+        if: steps.kvm.outputs.available == 'true'
+        run: |
+          set -eu
+          sudo apt-get update
+          sudo apt-get install -y --no-install-recommends bubblewrap e2fsprogs zstd
+          sh addons/sandbox/package/fetch-runtime.sh linux/amd64 "$RUNNER_TEMP/runtime"
+          # The backend looks in the system folders; the add-on itself finds
+          # the copies in its package.
+          sudo install -m 0755 "$RUNNER_TEMP/runtime/cloud-hypervisor" "$RUNNER_TEMP/runtime/virtiofsd" /usr/local/bin/
+          sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
+      - uses: docker/setup-buildx-action@master
+        if: steps.kvm.outputs.available == 'true'
+      - name: Build the guest image as an OCI layout
+        id: image
+        if: steps.kvm.outputs.available == 'true'
+        run: |
+          set -eu
+          docker buildx build --provenance=false -f addons/sandbox/images/guest/Dockerfile \
+            --platform linux/amd64 --output "type=oci,dest=$RUNNER_TEMP/guest.tar" .
+          mkdir -p "$RUNNER_TEMP/guest-layout"
+          tar -xf "$RUNNER_TEMP/guest.tar" -C "$RUNNER_TEMP/guest-layout"
+          digest=$(jq -r '.manifests[0].digest' "$RUNNER_TEMP/guest-layout/index.json")
+          case "$digest" in sha256:*) ;; *) echo "no manifest digest in the layout: $digest" >&2; exit 1 ;; esac
+          echo "ref=oci-layout:$RUNNER_TEMP/guest-layout@$digest" >> "$GITHUB_OUTPUT"
+      - name: KVM integration suite
+        if: steps.kvm.outputs.available == 'true'
+        env:
+          SANDBOX_TEST_IMAGE: ${{ steps.image.outputs.ref }}
+        run: go test -tags auth -count=1 -timeout 45m ./addons/sandbox/internal/sandbox/chv/

.github/workflows/test.yml

@@ -19,6 +19,27 @@ jobs:
     strategy:
       matrix:
         go-version: ['1.26.x']
+    # The PostgreSQL stores of the chats, agents, connectors, notifications,
+    # workspace and triggers skip without a database, so a regression that only
+    # shows on a cluster passed unseen. The coverage run below sees the URL,
+    # and a separate step fails when any of those tests skipped. The tests
+    # wipe their own tables, so the database is throwaway.
+    services:
+      postgres:
+        image: postgres:16-alpine
+        env:
+          POSTGRES_USER: localai
+          POSTGRES_PASSWORD: localai
+          POSTGRES_DB: localai_test
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd "pg_isready -U localai -d localai_test"
+          --health-interval 5s
+          --health-timeout 5s
+          --health-retries 10
+    env:
+      LOCALAI_TEST_POSTGRES_URL: postgres://localai:localai@localhost:5432/localai_test?sslmode=disable
     steps:
       - name: Clone
         uses: actions/checkout@v7
@@ -65,12 +86,21 @@ jobs:
       - name: Test (with coverage gate)
         run: |
           PATH="$PATH:/root/go/bin" make --jobs 5 --output-sync=target test-coverage-check
+      # The coverage run above set the URL, but a skipped test is a passed test
+      # to it. This runs the PostgreSQL stores alone and fails on any skip.
+      - name: Test the PostgreSQL stores (no skips)
+        run: PATH="$PATH:$HOME/go/bin" make test-postgres
       # tests/integration is outside the coverage roots because its store specs
       # need a live backend. test-stores builds and installs local-store before
       # running the complete suite, so new local-store specs are collected
       # automatically without adding another workflow entry.
       - name: Test local-store integration
         run: PATH="$PATH:$HOME/go/bin" make test-stores
+      # Runs the real local-ai binary (MCP connectors and apps, collections,
+      # process flags, route permissions). The suite builds the binary, the
+      # mock backend and an MCP fixture itself and needs no Docker.
+      - name: Test platform e2e
+        run: PATH="$PATH:$HOME/go/bin" make test-e2e-platform
       - name: Upload coverage report
         if: ${{ always() }}
         uses: actions/upload-artifact@v4

.github/workflows/tests-e2e-distributed.yml

@@ -0,0 +1,113 @@
+---
+name: 'Distributed E2E Tests'
+
+# The distributed architecture against real PostgreSQL and NATS (including the
+# NATS JWT specs), started by testcontainers, plus the multi-frontend paths
+# against the in-process bus under the race detector. Nothing else runs the
+# dedicated `make test-e2e-distributed` target: `make test-e2e` reaches these
+# specs only because it recurses, and only after it has built the e2e image.
+on:
+  pull_request:
+    paths:
+      - 'tests/e2e/distributed/**'
+      - 'core/services/nodes/**'
+      - 'core/services/messaging/**'
+      - 'core/services/testutil/**'
+      - 'core/services/studio/**'
+      - 'core/services/collections/**'
+      - 'core/services/syncstate/**'
+      - 'core/services/advisorylock/**'
+      - 'core/services/galleryop/**'
+      - 'core/services/modeladmin/**'
+      - 'core/services/failover/**'
+      - 'core/services/storage/**'
+      - 'core/application/distributed*.go'
+      - 'core/cli/**'
+      - 'pkg/natsauth/**'
+      - 'scripts/nats-auth-setup.sh'
+      - 'Makefile'
+      - '.github/workflows/tests-e2e-distributed.yml'
+  push:
+    branches:
+      - master
+  schedule:
+    # Nightly, so a regression that the path filter missed shows within a day.
+    - cron: '17 3 * * *'
+  workflow_dispatch:
+
+concurrency:
+  group: ci-tests-e2e-distributed-${{ github.event.pull_request.number || github.sha }}-${{ github.repository }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+permissions:
+  contents: read
+
+jobs:
+  fakebus:
+    # No containers: the in-process bus carries the messages between two
+    # frontends, a worker and the cluster state, with the race detector on.
+    name: multi-frontend over the in-process bus
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - name: Clone
+        uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+      - name: Setup Go
+        uses: actions/setup-go@v5
+        with:
+          go-version: '1.26.x'
+          cache: false
+      - name: Proto Dependencies
+        run: |
+          curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip && \
+          sudo unzip -j -d /usr/local/bin protoc.zip bin/protoc && \
+          rm protoc.zip
+          go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
+          go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
+          PATH="$PATH:$HOME/go/bin" make protogen-go
+      - name: System Dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y build-essential libopus-dev
+      - name: Stub the React UI for go:embed
+        # core/http embeds react-ui/dist. These tests do not serve the UI.
+        run: |
+          mkdir -p core/http/react-ui/dist
+          touch core/http/react-ui/dist/index.html
+      - name: Test the multi-frontend paths under the race detector
+        run: PATH="$PATH:$HOME/go/bin" make test-distributed-fakebus
+
+  distributed:
+    name: PostgreSQL and NATS in containers
+    runs-on: ubuntu-latest
+    timeout-minutes: 60
+    steps:
+      - name: Clone
+        uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+      - name: Setup Go
+        uses: actions/setup-go@v5
+        with:
+          go-version: '1.26.x'
+          cache: false
+      - name: Proto Dependencies
+        run: |
+          curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip && \
+          sudo unzip -j -d /usr/local/bin protoc.zip bin/protoc && \
+          rm protoc.zip
+          go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
+          go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
+      - name: System Dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y build-essential libopus-dev
+      - name: Stub the React UI for go:embed
+        run: |
+          mkdir -p core/http/react-ui/dist
+          touch core/http/react-ui/dist/index.html
+      - name: Run the distributed e2e suite
+        # protogen-go is a prerequisite of the target.
+        run: PATH="$PATH:$HOME/go/bin" make test-e2e-distributed

.github/workflows/tests-ui-e2e.yml

@@ -3,10 +3,22 @@ name: 'UI E2E Tests'
 
 on:
   pull_request:
+    # The suite builds the add-on UI bundles from addons/<id>/ui (global-setup)
+    # and drives the real chats service, the add-on host and the studio
+    # services through the test server, so a change to any of them can break a
+    # spec without touching core/http.
     paths:
       - 'core/http/**'
+      - 'core/application/**'
+      - 'core/services/studio/**'
+      - 'core/services/addons/**'
+      - 'pkg/addons/**'
+      - 'addons/**'
       - 'tests/e2e-ui/**'
       - 'tests/e2e/mock-backend/**'
+      - 'tests/addons/fixture/**'
+      - 'scripts/ensure-playwright-browser.sh'
+      - '.github/workflows/tests-ui-e2e.yml'
   push:
     branches:
       - master
@@ -52,7 +64,11 @@ jobs:
         run: |
           sudo apt-get update
           sudo apt-get install -y build-essential libopus-dev
-      - name: Run stale chunk recovery tests
+      # Runs the specs tagged @production-chunks on the production bundle the
+      # binary embeds (code-split, CSS per chunk): the stale chunk recovery
+      # specs and the production bundle walk of the main pages. The coverage
+      # run below uses a different bundle and skips them.
+      - name: Run production bundle tests
         run: PATH="$PATH:$HOME/go/bin" make test-ui-stale-chunk
       # Builds an instrumented UI bundle, runs the Playwright specs, and fails
       # if line coverage regressed beyond the jitter tolerance (the gate is

.github/workflows/tests-ui-unit.yml

@@ -0,0 +1,46 @@
+---
+name: 'UI Unit Tests'
+
+# The checks that need only Node: eslint (errors), the inline-style ratchet,
+# the host UI unit tests, the UI unit tests of every add-on and the locale
+# completeness ratchet. They take seconds, so they run on every change to the
+# UI or to an add-on UI package, ahead of the browser suites.
+on:
+  pull_request:
+    paths:
+      - 'core/http/react-ui/**'
+      - 'addons/*/ui/**'
+      - 'addons/internal/**'
+      - 'Makefile'
+      - '.github/workflows/tests-ui-unit.yml'
+  push:
+    branches:
+      - master
+
+concurrency:
+  group: ci-tests-ui-unit-${{ github.event.pull_request.number || github.sha }}-${{ github.repository }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+permissions:
+  contents: read
+
+jobs:
+  tests-ui-unit:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - name: Clone
+        uses: actions/checkout@v7
+        with:
+          persist-credentials: false
+      - name: Setup Node.js
+        uses: actions/setup-node@v7
+        with:
+          node-version: '22'
+      # The add-on UI tests render with the host's React and bundle with the
+      # host's bundler (addons/internal/uibuild), so they read the host's
+      # node_modules.
+      - name: Install the host UI dependencies
+        run: npm ci --no-audit --no-fund --prefix core/http/react-ui
+      - name: Run the UI unit checks
+        run: make test-ui-unit

.gitignore

@@ -141,3 +141,9 @@ formal-verification/out/
 /backend/go/gemxcpp/package/
 /backend/go/gemxcpp/gemxcpp
 addons/internal/uibuild/test/*/dist
+
+# Generated by the real-binary chats suite (tests/e2e-chats).
+/tests/e2e-chats/data/
+/tests/e2e-chats/cache/
+/tests/e2e-chats/models/
+/tests/e2e-chats/backends/

Dockerfile

@@ -21,7 +21,7 @@ RUN --mount=type=bind,source=.docker/apt-mirror.sh,target=/usr/local/sbin/apt-mi
     apt-get install -y --no-install-recommends \
         ca-certificates curl wget espeak-ng libgomp1 \
         ffmpeg libopenblas0 libopenblas-dev libopus0 sox \
-        hwdata && \
+        hwdata e2fsprogs && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/*
 
@@ -297,6 +297,14 @@ ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
 ENV NVIDIA_REQUIRE_CUDA="cuda>=${CUDA_MAJOR_VERSION}.0"
 ENV NVIDIA_VISIBLE_DEVICES=all
 ENV LD_FLAGS=${LD_FLAGS}
+# The default model and backend source of the binary. The Makefile picks the
+# gallery on a commit that carries a tag, so a tagged image would read other
+# sources than a release binary, which the release workflow cuts over to the
+# libraries. Pin the libraries here so both read the same sources.
+ARG LOCALAI_MODEL_SOURCE=library
+ARG LOCALAI_BACKEND_SOURCE=library
+ENV LOCALAI_MODEL_SOURCE=${LOCALAI_MODEL_SOURCE}
+ENV LOCALAI_BACKEND_SOURCE=${LOCALAI_BACKEND_SOURCE}
 
 RUN echo "GO_TAGS: $GO_TAGS" && echo "TARGETARCH: $TARGETARCH"
 
@@ -420,6 +428,18 @@ RUN --mount=from=builder,src=/build/,dst=/mnt/build \
 # Make sure the models directory exists
 RUN mkdir -p /models /backends /data
 
+# The unprivileged user that entrypoint.sh drops to. The image still starts as
+# root so that the entrypoint can hand a volume created by an older image over
+# to this user; the Sandbox add-on refuses to run as root.
+RUN groupadd --gid 10001 localai && \
+    useradd --uid 10001 --gid 10001 --home-dir /home/localai --create-home --shell /usr/sbin/nologin localai && \
+    chown localai:localai /models /backends /data
+
+# The default staging directory sits next to the binary, which is the root
+# directory here, and the unprivileged user cannot write there. Stage the
+# in-flight downloads in the data volume instead of the RAM-backed /tmp.
+ENV LOCALAI_DOWNLOAD_STAGING_PATH=/data/downloading
+
 # Define the health check command.
 #
 # --start-period is the knob for slow starts, not --timeout/--retries. Since

Makefile

@@ -89,7 +89,22 @@ else
 	GORELEASER=$(shell which goreleaser)
 endif
 
-TEST_PATHS?=./api/... ./pkg/... ./core/... ./backend/go/cloud-proxy/... ./backend/go/localai-proxy/... ./backend/go/local-store/... ./backend/go/valkey-store/... ./backend/go/qdrant-store/...
+## ./addons/... holds the Go tests of every add-on (sandbox, office, tables,
+## fine-tuning, quantization and the shared packages). The Sandbox add-on is
+## Linux only and does not compile elsewhere, so the add-on tests run on Linux
+## only. tests/addons/actione2e is the one
+## tests/addons suite that builds everything it needs itself; the other
+## tests/addons suites need the fixtures and executables that
+## scripts/addons/verify.sh prepares, so the addons-tests workflow runs them
+## through verify.sh.
+ifeq ($(OS),Linux)
+ADDON_TEST_PATHS:=./addons/... ./tests/addons/actione2e/...
+ADDON_COVERAGE_ROOTS:=./addons ./tests/addons/actione2e
+else
+ADDON_TEST_PATHS:=
+ADDON_COVERAGE_ROOTS:=
+endif
+TEST_PATHS?=./api/... ./pkg/... ./core/... $(ADDON_TEST_PATHS) ./backend/go/cloud-proxy/... ./backend/go/localai-proxy/... ./backend/go/local-store/... ./backend/go/valkey-store/... ./backend/go/qdrant-store/...
 
 ## Coverage output and the committed baseline that CI compares against.
 ## The gate is strict: total coverage must never decrease (no tolerance).
@@ -101,8 +116,8 @@ COVERAGE_BASELINE?=coverage-baseline.txt
 ## Coverage is collected one recursive root at a time and merged (see
 ## scripts/run-coverage.sh): passing several recursive roots to a single
 ## ginkgo invocation only keeps one root's coverprofile. Mirrors TEST_PATHS
-## minus ./api (which doesn't exist).
-COVERAGE_ROOTS?=./pkg ./core
+## minus ./api (which doesn't exist) and the backend/go modules.
+COVERAGE_ROOTS?=./pkg ./core $(ADDON_COVERAGE_ROOTS)
 ## Build tags for the coverage build. `auth` is required to compile the real
 ## auth implementation and its ~150 `//go:build auth` tests (otherwise they're
 ## invisible and the gate scores auth against a stub). `debug` matches `test`.
@@ -123,7 +138,7 @@ COVERAGE_E2E_LABELS?=!real-models
 COVERAGE_EXCLUDE_RE?=grpc/proto/.*[.]pb[.]go
 
 
-.PHONY: all test test-coverage test-coverage-baseline test-coverage-check test-backend-cpp test-build-scripts test-ui test-ui-stale-chunk test-ui-coverage-baseline test-ui-coverage-check build vendor lint lint-all
+.PHONY: all test test-coverage test-coverage-baseline test-coverage-check test-backend-cpp test-build-scripts test-ui test-ui-stale-chunk test-ui-unit test-i18n test-postgres test-distributed-fakebus test-addons-suites test-ui-coverage-baseline test-ui-coverage-check build vendor lint lint-all
 
 all: help
 
@@ -362,6 +377,14 @@ test-e2e-distributed: protogen-go
 	@echo 'Running distributed e2e tests (label Distributed, incl. NatsJWT)'
 	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --label-filter='Distributed && !VLLMMultinode' --flake-attempts $(TEST_FLAKES) -v -r ./tests/e2e/distributed
 
+# Platform e2e: MCP connectors and apps, collections, the process flags and the
+# permission of the new routes, against the real local-ai binary. The suite builds
+# the binary (with the auth tag), the mock backend and an MCP fixture server
+# itself. No Docker is needed. LOCALAI_E2E_BINARY uses a binary you built.
+test-e2e-platform: protogen-go
+	@echo 'Running platform e2e tests'
+	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --label-filter='platform' --flake-attempts $(TEST_FLAKES) -v -r ./tests/e2e-platform
+
 # vLLM multi-node DP smoke (CPU). Builds local-ai:tests and the
 # cpu-vllm backend from the current working tree, then drives a
 # head + headless follower via testcontainers-go and asserts a chat
@@ -398,6 +421,14 @@ prepare-e2e:
 run-e2e-image:
 	docker run -p 5390:8080 -e MODELS_PATH=/models -e THREADS=1 -e DEBUG=true -d --rm -v $(TEST_DIR):/models --name e2e-tests-$(RANDOM) localai-tests
 
+# Runs the real local-ai binary with the packaged first-party add-ons. It needs
+# the generated gRPC code and the built UI (see .agents/building-and-testing.md)
+# and network access for the gVisor download of the Sandbox package; without
+# it set LOCALAI_E2E_RUNSC_DIR or the Sandbox package is skipped.
+.PHONY: test-addons-shipped
+test-addons-shipped:
+	$(GOCMD) test -tags auth -count=1 -timeout 40m -v ./tests/addons/shipped/
+
 test-e2e: build-mock-backend build-cloud-proxy-backend build-localai-proxy-backend prepare-e2e run-e2e-image
 	@echo 'Running e2e tests'
 	BUILD_TYPE=$(BUILD_TYPE) \
@@ -463,6 +494,14 @@ test-opus-docker:
 	docker run --rm localai-opus-test \
 	  bash -c 'cd /LocalAI && go run github.com/onsi/ginkgo/v2/ginkgo --flake-attempts $(TEST_FLAKES) -v -r ./backend/go/opus/...'
 
+## Drives the real local-ai binary over HTTP against a scripted upstream
+## (tests/e2e-chats). It builds its own binaries; set E2E_LOCALAI_BIN,
+## E2E_MOCK_BACKEND and E2E_CLOUD_PROXY to reuse built ones, and
+## E2E_CHATS_PORT to change the port (default 8211).
+test-e2e-chats: protogen-go react-ui
+	@echo 'Running chats real-binary e2e tests'
+	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --flake-attempts $(TEST_FLAKES) -v ./tests/e2e-chats
+
 test-realtime: build-mock-backend
 	@echo 'Running realtime e2e tests (mock backend)'
 	$(GOCMD) run github.com/onsi/ginkgo/v2/ginkgo --label-filter="Realtime && !real-models" --flake-attempts $(TEST_FLAKES) -v -r ./tests/e2e
@@ -1612,7 +1651,7 @@ clean-localai-proxy-backend:
 ########################################################
 
 build-ui-test-server: build-mock-backend react-ui protogen-go
-	$(GOCMD) build -o tests/e2e-ui/ui-test-server ./tests/e2e-ui
+	$(GOCMD) build -tags auth -o tests/e2e-ui/ui-test-server ./tests/e2e-ui
 
 test-ui-e2e: build-ui-test-server
 	cd core/http/react-ui && npm install && npx playwright install --with-deps chromium && npx playwright test
@@ -1632,14 +1671,17 @@ PLAYWRIGHT_WORKERS_FLAG = $(if $(UI_TEST_WORKERS),--workers=$(UI_TEST_WORKERS),)
 ## downloading it as `test-ui-e2e` does.
 test-ui: build-mock-backend protogen-go
 	cd core/http/react-ui && bun install && bun run build
-	$(GOCMD) build -o tests/e2e-ui/ui-test-server ./tests/e2e-ui
+	$(GOCMD) build -tags auth -o tests/e2e-ui/ui-test-server ./tests/e2e-ui
 	cd core/http/react-ui && sh $(CURDIR)/scripts/ensure-playwright-browser.sh && bunx playwright test $(PLAYWRIGHT_WORKERS_FLAG)
 
-## The stale-chunk specs need the production code-split bundle. The V8 coverage
-## bundle below inlines dynamic imports to keep every page in its denominator.
+## The specs tagged @production-chunks need the production code-split bundle
+## the binary embeds: the stale-chunk recovery specs and the production bundle
+## walk (e2e/production-bundle.spec.js). The V8 coverage bundle below inlines
+## dynamic imports to keep every page in its denominator, so it skips them and
+## this target is the only run that tests the bundle that ships.
 test-ui-stale-chunk: build-mock-backend protogen-go
 	cd core/http/react-ui && bun install && bun run build
-	$(GOCMD) build -o tests/e2e-ui/ui-test-server ./tests/e2e-ui
+	$(GOCMD) build -tags auth -o tests/e2e-ui/ui-test-server ./tests/e2e-ui
 	cd core/http/react-ui && sh $(CURDIR)/scripts/ensure-playwright-browser.sh && bunx playwright test --grep @production-chunks --workers=1
 
 ## React UI code coverage from the Playwright e2e suite. Builds a
@@ -1654,7 +1696,7 @@ test-ui-stale-chunk: build-mock-backend protogen-go
 test-ui-coverage: build-mock-backend protogen-go
 	trap 'rm -rf "$(CURDIR)/core/http/react-ui/dist"' EXIT; \
 	( cd core/http/react-ui && bun install && bun run build:coverage-v8 ) && \
-	$(GOCMD) build -o tests/e2e-ui/ui-test-server ./tests/e2e-ui && \
+	$(GOCMD) build -tags auth -o tests/e2e-ui/ui-test-server ./tests/e2e-ui && \
 	( cd core/http/react-ui && rm -rf .nyc_output coverage && \
 	    sh $(CURDIR)/scripts/ensure-playwright-browser.sh && \
 	    PW_V8_COVERAGE=1 bunx playwright test --grep-invert @production-chunks $(PLAYWRIGHT_WORKERS_FLAG) && bun run coverage:report )
@@ -1668,6 +1710,64 @@ test-ui-coverage-baseline: test-ui-coverage
 test-ui-coverage-check: test-ui-coverage
 	sh $(CURDIR)/scripts/ui-coverage-check.sh core/http/react-ui/coverage/coverage-summary.json core/http/react-ui/coverage-baseline.txt
 
+## Node-only UI checks that need no server and no browser: eslint (errors
+## only, warnings are tracked by the editor), the inline-style ratchet, the
+## host UI unit tests, the UI unit tests of every add-on and the locale
+## completeness ratchet. Run `npm ci` in core/http/react-ui first.
+test-ui-unit: test-i18n
+	cd core/http/react-ui && npm run lint -- --quiet
+	cd core/http/react-ui && npm run lint:inline-styles
+	cd core/http/react-ui && npm run test:unit
+	@set -e; for dir in addons/*/ui; do \
+		[ -f "$$dir/test/run.mjs" ] || continue; \
+		echo "== $$dir"; \
+		(cd "$$dir" && node test/run.mjs > /dev/null); \
+	done
+
+## Every locale must have every namespace and key of English, except the gaps
+## recorded in core/http/react-ui/i18n-baseline.json, which may only shrink.
+test-i18n:
+	cd core/http/react-ui && node scripts/i18n-check.mjs
+
+## The PostgreSQL stores of the chats, agents, connectors, notifications,
+## workspace and triggers. They skip without a database, which would let a
+## cluster-only regression pass, so this target fails when the URL is unset or
+## when any of them skipped. The stores are Ginkgo specs whose names contain
+## "PostgreSQL"; the focus selects them. The packages are listed one by one
+## because -ginkgo.focus is an error for a test binary that has no Ginkgo.
+POSTGRES_STORE_PKGS = ./core/services/studio/agents ./core/services/studio/connectors \
+	./core/services/studio/notify ./core/services/studio/store ./core/services/studio/triggers \
+	./core/services/studio/workspace/gormstore
+test-postgres:
+	@test -n "$$LOCALAI_TEST_POSTGRES_URL" || { echo 'LOCALAI_TEST_POSTGRES_URL is not set; point it at a database the tests may wipe' >&2; exit 1; }
+	@out=$$($(GOCMD) test -count=1 -v -tags "$(COVERAGE_TAGS)" $(POSTGRES_STORE_PKGS) -ginkgo.focus 'PostgreSQL' -ginkgo.v -ginkgo.no-color 2>&1) || { printf '%s\n' "$$out"; exit 1; }; \
+	printf '%s\n' "$$out" | grep -E '^(Ran [0-9]+ of|ok |FAIL|\s*\[SKIPPED\])' ; \
+	if printf '%s\n' "$$out" | grep -q -- '\[SKIPPED\]'; then echo 'a PostgreSQL test skipped' >&2; exit 1; fi; \
+	test "$$(printf '%s\n' "$$out" | grep -c -- '^Ran [1-9]')" -ge 6 || { echo 'fewer PostgreSQL stores ran than expected' >&2; exit 1; }
+
+## The multi-frontend paths run against the in-process bus (testutil.FakeBus)
+## under the race detector: two frontends, a worker and the cluster state, with
+## no containers. The container-based suite is test-e2e-distributed. The
+## galleryop and nodes packages are left out: some of their specs start
+## containers, and galleryop has a data race in a test (cancellable_phase_test.go
+## reassigns the gate channel the worker reads).
+test-distributed-fakebus:
+	$(GOCMD) test -race -count=1 -tags "$(COVERAGE_TAGS)" \
+		./core/services/testutil/... ./core/services/messaging/... ./core/services/studio/... \
+		./core/services/collections/... ./core/services/syncstate/... \
+		./core/services/modeladmin/... ./core/services/failover/...
+
+## The add-on suites of scripts/addons/suites.json that need the integration
+## build (UI server, mock backend, browser). ADDON_SUITES picks the ones to run;
+## `packaging` is left out because addons-package.yml runs it.
+## Prepare first with: sh scripts/addons/prepare.sh --prepare, then --integration
+ADDON_SUITES?=prerequisites contracts catalog packages lifecycle runtime gateway policy proposals artifacts apps gallery-ui esm-ui host-slots library-sources foundation-acceptance recovery updates tables-databases tables-schema tables-rows tables-tools tables-admin tables-ui tables-csv tables-acceptance
+test-addons-suites:
+	@set -e; for suite in $(ADDON_SUITES); do \
+		echo "== $$suite"; \
+		sh scripts/addons/verify.sh $$suite; \
+	done
+
 test-ui-e2e-docker:
 	docker build -t localai-ui-e2e -f tests/e2e-ui/Dockerfile .
 	docker run --rm localai-ui-e2e

README.md

@@ -48,7 +48,8 @@
 - **Any model, any modality**: LLMs, vision, voice, image, and video behind one API
 - **Any hardware**: NVIDIA, AMD, Intel, Apple Silicon, Vulkan, or CPU-only
 - **Multi-user ready**: API key auth, user quotas, role-based access
-- **Built-in AI agents**: autonomous agents with tool use, RAG, MCP, and skills
+- **Chats with saved agents**: conversations the server keeps, with agents that use tools, MCP, skills and files, triggers, and a Telegram connector
+- **Add-ons**: Studio, Talk, Voice, Face, Office, Sandbox and more install from a signed catalog, and run beside the server
 - **Privacy-first**: your data never leaves your infrastructure
 
 ![A small LocalAI core with backends (llama.cpp, vLLM, MLX, whisper.cpp, stable-diffusion, kokoro, parakeet.cpp...) plugged in as separate on-demand images](docs/static/images/diagrams/composable-core.png)
@@ -71,10 +72,6 @@ Click to see more!
 
 https://github.com/user-attachments/assets/228fa9ad-81a3-4d43-bfb9-31557e14a36c
 
-#### Agents
-
-https://github.com/user-attachments/assets/6270b331-e21d-4087-a540-6290006b381a
-
 #### Usage metrics per user
 
 https://github.com/user-attachments/assets/cbb03379-23b4-4e3d-bd26-d152f057007f
@@ -224,7 +221,7 @@ For older news and full release notes, see [GitHub Releases](https://github.com/
 - [P2P Inferencing](https://localai.io/features/distribute/)
 - [Distributed Mode](https://localai.io/features/distributed-mode/) — Horizontal scaling with PostgreSQL + NATS
 - [Model Context Protocol (MCP)](https://localai.io/docs/features/mcp/)
-- [Built-in Agents](https://localai.io/features/agents/) — Autonomous AI agents with tool use, RAG, skills, SSE streaming, and [Agent Hub](https://agenthub.localai.io)
+- [Chats](https://localai.io/features/chats/) — Conversations the server keeps, with saved agents, triggers, skills, files and a Telegram connector
 - [Backend Gallery](https://localai.io/backends/) — Install/remove backends on the fly via OCI images
 - Voice Activity Detection (Silero-VAD)
 - Integrated WebUI

addons/face/package/platforms.yaml

@@ -30,7 +30,7 @@ platforms:
     native_check: executable-start
   - platform: darwin/arm64
     enabled: true
-    runner: macos-14
+    runner: macos-15
     native_check: executable-start
   - platform: darwin/amd64
     enabled: true

addons/fine-tuning/internal/tools/journal.go

@@ -3,12 +3,13 @@ package tools
 
 import (
 	"encoding/json"
-	"errors"
 	"os"
 	"path/filepath"
 	"sync"
 
 	"github.com/mudler/LocalAI/pkg/addons/contract"
+
+	"github.com/mudler/LocalAI/addons/internal/atomicfile"
 )
 
 // maxReceipts bounds the receipts the journal keeps. A repeat of an older
@@ -29,6 +30,12 @@ type job struct {
 	// Revision counts the changes a tool made to the job. It is the data
 	// revision of the resource, so a stale approval is noticed.
 	Revision uint64 `json:"revision"`
+	// StartOp is the operation that started the job: the change set a person
+	// can undo. StartedNS is when, in Unix nanoseconds, for the retention.
+	// UndoneBy is the operation that undid it.
+	StartOp   string `json:"startOp,omitempty"`
+	StartedNS int64  `json:"startedNs,omitempty"`
+	UndoneBy  string `json:"undoneBy,omitempty"`
 }
 
 type journalFile struct {
@@ -114,29 +121,5 @@ func writeAtomic(path string, v any) error {
 	if err != nil {
 		return err
 	}
-	tmp, err := os.CreateTemp(filepath.Dir(path), "journal-*.tmp")
-	if err != nil {
-		return err
-	}
-	name := tmp.Name()
-	fail := func(e error) error { _ = tmp.Close(); _ = os.Remove(name); return e }
-	if _, err := tmp.Write(b); err != nil {
-		return fail(err)
-	}
-	if err := tmp.Sync(); err != nil {
-		return fail(err)
-	}
-	if err := tmp.Close(); err != nil {
-		_ = os.Remove(name)
-		return err
-	}
-	if err := os.Chmod(name, 0o600); err != nil {
-		_ = os.Remove(name)
-		return errors.Join(err)
-	}
-	if err := os.Rename(name, path); err != nil {
-		_ = os.Remove(name)
-		return err
-	}
-	return nil
+	return atomicfile.Write(path, b, 0o600)
 }

addons/fine-tuning/internal/tools/tools.go

@@ -7,7 +7,6 @@
 package tools
 
 import (
-	"bytes"
 	"context"
 	"encoding/json"
 	"errors"
@@ -19,6 +18,7 @@ import (
 	"unicode/utf8"
 
 	"github.com/mudler/LocalAI/addons/internal/hostclient"
+	"github.com/mudler/LocalAI/addons/internal/jobundo"
 	"github.com/mudler/LocalAI/addons/internal/protocol"
 	"github.com/mudler/LocalAI/pkg/addons/contract"
 )
@@ -135,56 +135,6 @@ type exportArgs struct {
 	Name         string `json:"name"`
 }
 
-// decode reads the arguments into a closed struct: an unknown field or
-// trailing data is a bad request.
-func decode(raw json.RawMessage, v any) error {
-	if len(raw) == 0 {
-		raw = json.RawMessage(`{}`)
-	}
-	dec := json.NewDecoder(bytes.NewReader(raw))
-	dec.DisallowUnknownFields()
-	if err := dec.Decode(v); err != nil {
-		return protocol.ErrBadRequest
-	}
-	if _, err := dec.Token(); err != io.EOF {
-		return protocol.ErrBadRequest
-	}
-	return nil
-}
-
-func toolOf(env contract.Envelope) (string, error) {
-	var a struct {
-		Tool string `json:"tool"`
-	}
-	if json.Unmarshal(env.Request.Arguments, &a) != nil || a.Tool == "" {
-		return "", protocol.ErrBadRequest
-	}
-	return a.Tool, nil
-}
-
-func payload(v any) json.RawMessage { b, _ := json.Marshal(v); return b }
-
-// mapErr turns the refusal of a host operation into the protocol error that
-// tells the owner what happened. A failure that does not say the request was
-// wrong stays as it is, and the owner reads it as uncertain.
-func mapErr(err error) error {
-	var he *contract.HostError
-	if !errors.As(err, &he) {
-		return err
-	}
-	switch he.Code {
-	case contract.HostBadRequest, contract.HostTooLarge:
-		return fmt.Errorf("%w: %s", protocol.ErrBadRequest, he.Message)
-	case contract.HostNotFound:
-		return fmt.Errorf("%w: %s", protocol.ErrNotFound, he.Message)
-	case contract.HostConflict:
-		return fmt.Errorf("%w: %s", protocol.ErrConflict, he.Message)
-	case contract.HostFeature, contract.HostScope:
-		return fmt.Errorf("%w: %s", protocol.ErrForbidden, he.Message)
-	}
-	return err
-}
-
 // jobOf is the job of the resource. A dataset resource is not a job.
 func (r *runtime) jobOf(env contract.Envelope) (job, error) {
 	j, ok := r.entryOf(env)
@@ -202,29 +152,31 @@ func (r *runtime) entryOf(env contract.Envelope) (job, bool) {
 }
 
 func (r *runtime) Read(ctx context.Context, env contract.Envelope) (contract.Result, error) {
-	tool, err := toolOf(env)
+	tool, err := protocol.ToolOf(env)
 	if err != nil {
 		return contract.Result{}, err
 	}
 	op := env.Request.OperationID
 	switch tool {
+	case contract.ToolChangesGet:
+		return r.readChange(ctx, env)
 	case ToolResourceExists:
 		j, ok := r.j.job(env.Request.ResourceID)
 		if !ok {
-			return contract.Result{Payload: payload(map[string]bool{"exists": false})}, nil
+			return contract.Result{Payload: protocol.Payload(map[string]bool{"exists": false})}, nil
 		}
 		if j.Dataset {
 			// The reference lives in the host and the journal; there is no
 			// engine job to ask.
-			return contract.Result{Payload: payload(map[string]bool{"exists": true})}, nil
+			return contract.Result{Payload: protocol.Payload(map[string]bool{"exists": true})}, nil
 		}
 		err := r.eng.Do(ctx, op, "finetune.get", map[string]string{"jobId": j.JobID}, nil)
 		var he *contract.HostError
 		switch {
 		case err == nil:
-			return contract.Result{Payload: payload(map[string]bool{"exists": true})}, nil
+			return contract.Result{Payload: protocol.Payload(map[string]bool{"exists": true})}, nil
 		case errors.As(err, &he) && he.Code == contract.HostNotFound:
-			return contract.Result{Payload: payload(map[string]bool{"exists": false})}, nil
+			return contract.Result{Payload: protocol.Payload(map[string]bool{"exists": false})}, nil
 		}
 		// The engine could not answer. The host settles a creation only on an
 		// explicit answer, so this is an error and never "does not exist".
@@ -235,12 +187,12 @@ func (r *runtime) Read(ctx context.Context, env contract.Envelope) (contract.Res
 		var a struct {
 			Tool string `json:"tool"`
 		}
-		if err := decode(env.Request.Arguments, &a); err != nil {
+		if err := protocol.DecodeArgs(env.Request.Arguments, &a); err != nil {
 			return contract.Result{}, err
 		}
 		var got json.RawMessage
 		if err := r.eng.Do(ctx, op, "finetune.list", nil, &got); err != nil {
-			return contract.Result{}, mapErr(err)
+			return contract.Result{}, protocol.FromHostError(err)
 		}
 		return contract.Result{Payload: got}, nil
 	case ToolStatus:
@@ -249,20 +201,20 @@ func (r *runtime) Read(ctx context.Context, env contract.Envelope) (contract.Res
 			return contract.Result{}, protocol.ErrNotFound
 		}
 		if j.Dataset {
-			return contract.Result{Payload: payload(map[string]any{
+			return contract.Result{Payload: protocol.Payload(map[string]any{
 				"name": j.Name, "revision": contract.Revision{Schema: schema, Data: j.Revision}, "kind": "dataset", "datasetRef": j.Ref,
 			})}, nil
 		}
 		var got map[string]any
 		if err := r.eng.Do(ctx, op, "finetune.get", map[string]string{"jobId": j.JobID}, &got); err != nil {
-			return contract.Result{}, mapErr(err)
+			return contract.Result{}, protocol.FromHostError(err)
 		}
 		// The engine job ID stays in the add-on. The name and the revision
 		// are what the host reads to label and track the resource.
 		delete(got, "id")
 		got["name"] = j.Name
 		got["revision"] = contract.Revision{Schema: schema, Data: j.Revision}
-		return contract.Result{Payload: payload(got)}, nil
+		return contract.Result{Payload: protocol.Payload(got)}, nil
 	case ToolCheckpoints:
 		j, err := r.jobOf(env)
 		if err != nil {
@@ -270,7 +222,7 @@ func (r *runtime) Read(ctx context.Context, env contract.Envelope) (contract.Res
 		}
 		var got json.RawMessage
 		if err := r.eng.Do(ctx, op, "finetune.checkpoints", map[string]string{"jobId": j.JobID}, &got); err != nil {
-			return contract.Result{}, mapErr(err)
+			return contract.Result{}, protocol.FromHostError(err)
 		}
 		return contract.Result{Payload: got}, nil
 	}
@@ -296,7 +248,7 @@ func checkExport(a exportArgs) error {
 }
 
 func (r *runtime) Preview(_ context.Context, env contract.Envelope) (contract.Preview, error) {
-	tool, err := toolOf(env)
+	tool, err := protocol.ToolOf(env)
 	if err != nil {
 		return contract.Preview{}, err
 	}
@@ -304,7 +256,7 @@ func (r *runtime) Preview(_ context.Context, env contract.Envelope) (contract.Pr
 	switch tool {
 	case ToolStart:
 		var a startArgs
-		if err := decode(env.Request.Arguments, &a); err != nil {
+		if err := protocol.DecodeArgs(env.Request.Arguments, &a); err != nil {
 			return out, err
 		}
 		if err := r.checkStart(a); err != nil {
@@ -318,11 +270,11 @@ func (r *runtime) Preview(_ context.Context, env contract.Envelope) (contract.Pr
 			}
 		}
 		out.Revision = contract.Revision{Schema: schema, Data: 0}
-		out.Summary = payload(map[string]any{"count": 1, "sample": map[string]string{"action": "start fine-tuning job", "name": nameOf(a), "model": a.Model}})
+		out.Summary = protocol.Payload(map[string]any{"count": 1, "sample": map[string]string{"action": "start fine-tuning job", "name": nameOf(a), "model": a.Model}})
 		return out, nil
 	case ToolDataset:
 		var a datasetArgs
-		if err := decode(env.Request.Arguments, &a); err != nil {
+		if err := protocol.DecodeArgs(env.Request.Arguments, &a); err != nil {
 			return out, err
 		}
 		if err := checkDataset(a); err != nil {
@@ -335,7 +287,7 @@ func (r *runtime) Preview(_ context.Context, env contract.Envelope) (contract.Pr
 		}
 		out.Revision = contract.Revision{Schema: schema, Data: 0}
 		// The card shows the name and the size, never the content.
-		out.Summary = payload(map[string]any{"count": 1, "sample": map[string]any{"action": "send dataset", "name": a.Name, "bytes": len(a.Content)}})
+		out.Summary = protocol.Payload(map[string]any{"count": 1, "sample": map[string]any{"action": "send dataset", "name": a.Name, "bytes": len(a.Content)}})
 		return out, nil
 	case ToolStop, ToolExport:
 		j, err := r.jobOf(env)
@@ -345,13 +297,13 @@ func (r *runtime) Preview(_ context.Context, env contract.Envelope) (contract.Pr
 		sample := map[string]any{"name": j.Name}
 		if tool == ToolStop {
 			var a stopArgs
-			if err := decode(env.Request.Arguments, &a); err != nil {
+			if err := protocol.DecodeArgs(env.Request.Arguments, &a); err != nil {
 				return out, err
 			}
 			sample["action"], sample["saveCheckpoint"] = "stop fine-tuning job", a.SaveCheckpoint
 		} else {
 			var a exportArgs
-			if err := decode(env.Request.Arguments, &a); err != nil {
+			if err := protocol.DecodeArgs(env.Request.Arguments, &a); err != nil {
 				return out, err
 			}
 			if err := checkExport(a); err != nil {
@@ -360,7 +312,7 @@ func (r *runtime) Preview(_ context.Context, env contract.Envelope) (contract.Pr
 			sample["action"], sample["exportFormat"] = "export fine-tuned model", a.ExportFormat
 		}
 		out.Revision = contract.Revision{Schema: schema, Data: j.Revision}
-		out.Summary = payload(map[string]any{"count": 1, "sample": sample})
+		out.Summary = protocol.Payload(map[string]any{"count": 1, "sample": sample})
 		return out, nil
 	}
 	return out, protocol.ErrBadRequest
@@ -388,10 +340,13 @@ func nameOf(a startArgs) string {
 }
 
 func (r *runtime) Apply(ctx context.Context, env contract.Envelope, rev contract.Revision) (contract.Receipt, error) {
-	tool, err := toolOf(env)
+	tool, err := protocol.ToolOf(env)
 	if err != nil {
 		return contract.Receipt{}, err
 	}
+	if tool == contract.ToolChangesUndo {
+		return r.applyUndo(ctx, env, rev)
+	}
 	// An operation that already committed is answered from its receipt: the
 	// engine ran once and runs no more.
 	if rec, ok := r.j.receipt(env.Request.OperationID); ok {
@@ -404,7 +359,7 @@ func (r *runtime) Apply(ctx context.Context, env contract.Envelope, rev contract
 		return r.applyDataset(ctx, env)
 	case ToolStop:
 		var a stopArgs
-		if err := decode(env.Request.Arguments, &a); err != nil {
+		if err := protocol.DecodeArgs(env.Request.Arguments, &a); err != nil {
 			return contract.Receipt{}, err
 		}
 		return r.mutate(ctx, env, rev, "finetune.stop", func(id string) any {
@@ -412,7 +367,7 @@ func (r *runtime) Apply(ctx context.Context, env contract.Envelope, rev contract
 		})
 	case ToolExport:
 		var a exportArgs
-		if err := decode(env.Request.Arguments, &a); err != nil {
+		if err := protocol.DecodeArgs(env.Request.Arguments, &a); err != nil {
 			return contract.Receipt{}, err
 		}
 		if err := checkExport(a); err != nil {
@@ -434,7 +389,7 @@ func (r *runtime) Apply(ctx context.Context, env contract.Envelope, rev contract
 
 func (r *runtime) applyStart(ctx context.Context, env contract.Envelope) (contract.Receipt, error) {
 	var a startArgs
-	if err := decode(env.Request.Arguments, &a); err != nil {
+	if err := protocol.DecodeArgs(env.Request.Arguments, &a); err != nil {
 		return contract.Receipt{}, err
 	}
 	if err := r.checkStart(a); err != nil {
@@ -458,19 +413,20 @@ func (r *runtime) applyStart(ctx context.Context, env contract.Envelope) (contra
 		NumEpochs: a.NumEpochs, BatchSize: a.BatchSize, MaxSteps: a.MaxSteps, AdapterRank: a.AdapterRank, AdapterAlpha: a.AdapterAlpha,
 	}, &started)
 	if err != nil {
-		return contract.Receipt{}, mapErr(err)
+		return contract.Receipt{}, protocol.FromHostError(err)
 	}
 	if started.ID == "" {
 		return contract.Receipt{}, errors.New("the engine started a job and returned no ID")
 	}
 	rec := contract.Receipt{
 		OperationID: op, State: contract.ReceiptCommitted, Revision: contract.Revision{Schema: schema, Data: 1},
-		Result: contract.Result{Payload: payload(map[string]string{"status": started.Status, "message": started.Message})},
+		Result: contract.Result{Payload: jobundo.Merge(protocol.Payload(map[string]string{"status": started.Status, "message": started.Message}),
+			contract.UndoNote{ID: op, Available: true, Summary: contract.CleanUndoText(fmt.Sprintf("Stop the fine-tuning job %s while it runs", nameOf(a)))})},
 	}
 	// If this write fails the job runs without a mapping. The error is
 	// uncertain, so the host asks again with the same operation ID and the
 	// engine answers it with the same job.
-	if err := r.j.commit(res, &job{JobID: started.ID, Name: nameOf(a), Revision: 1}, rec); err != nil {
+	if err := r.j.commit(res, &job{JobID: started.ID, Name: nameOf(a), Revision: 1, StartOp: op, StartedNS: r.now().UnixNano()}, rec); err != nil {
 		return contract.Receipt{}, err
 	}
 	return rec, nil
@@ -482,7 +438,7 @@ func (r *runtime) applyStart(ctx context.Context, env contract.Envelope) (contra
 // of the resource, which only the owner of the resource can read.
 func (r *runtime) applyDataset(ctx context.Context, env contract.Envelope) (contract.Receipt, error) {
 	var a datasetArgs
-	if err := decode(env.Request.Arguments, &a); err != nil {
+	if err := protocol.DecodeArgs(env.Request.Arguments, &a); err != nil {
 		return contract.Receipt{}, err
 	}
 	if err := checkDataset(a); err != nil {
@@ -499,14 +455,14 @@ func (r *runtime) applyDataset(ctx context.Context, env contract.Envelope) (cont
 		DatasetRef string `json:"datasetRef"`
 	}
 	if err := r.eng.Do(ctx, op, "finetune.dataset", map[string]string{"name": a.Name, "content": a.Content}, &stored); err != nil {
-		return contract.Receipt{}, mapErr(err)
+		return contract.Receipt{}, protocol.FromHostError(err)
 	}
 	if stored.DatasetRef == "" {
 		return contract.Receipt{}, errors.New("the engine stored a dataset and returned no reference")
 	}
 	rec := contract.Receipt{
 		OperationID: op, State: contract.ReceiptCommitted, Revision: contract.Revision{Schema: schema, Data: 1},
-		Result: contract.Result{Payload: payload(map[string]string{"datasetRef": stored.DatasetRef})},
+		Result: contract.Result{Payload: jobundo.Merge(protocol.Payload(map[string]string{"datasetRef": stored.DatasetRef}), contract.UndoNote{Reason: jobundo.ReasonDataset})},
 	}
 	if err := r.j.commit(res, &job{Name: a.Name, Dataset: true, Ref: stored.DatasetRef, Revision: 1}, rec); err != nil {
 		return contract.Receipt{}, err
@@ -524,11 +480,11 @@ func (r *runtime) mutate(ctx context.Context, env contract.Envelope, rev contrac
 	id := env.Request.OperationID
 	if rev.Schema != schema || rev.Data != j.Revision {
 		return contract.Receipt{OperationID: id, State: contract.ReceiptRejected,
-			Result: contract.Result{Payload: payload(map[string]any{"error": "stale", "revision": contract.Revision{Schema: schema, Data: j.Revision}})}}, nil
+			Result: contract.Result{Payload: protocol.Payload(map[string]any{"error": "stale", "revision": contract.Revision{Schema: schema, Data: j.Revision}})}}, nil
 	}
 	var out json.RawMessage
 	if err := r.eng.Do(ctx, id, op, args(j.JobID), &out); err != nil {
-		return contract.Receipt{}, mapErr(err)
+		return contract.Receipt{}, protocol.FromHostError(err)
 	}
 	next := j
 	next.Revision++
@@ -536,8 +492,13 @@ func (r *runtime) mutate(ctx context.Context, env contract.Envelope, rev contrac
 	if len(out) > 0 && json.Valid(out) {
 		rec.Result.Payload = out
 	} else {
-		rec.Result.Payload = payload(map[string]any{})
+		rec.Result.Payload = protocol.Payload(map[string]any{})
 	}
+	reason := jobundo.ReasonStop
+	if op == "finetune.export" {
+		reason = jobundo.ReasonExport
+	}
+	rec.Result.Payload = jobundo.Merge(rec.Result.Payload, contract.UndoNote{Reason: reason})
 	if err := r.j.commit(env.Request.ResourceID, &next, rec); err != nil {
 		return contract.Receipt{}, err
 	}

addons/fine-tuning/internal/tools/undo.go

@@ -0,0 +1,54 @@
+// SPDX-License-Identifier: MIT
+package tools
+
+import (
+	"context"
+	"time"
+
+	"github.com/mudler/LocalAI/addons/internal/jobundo"
+	"github.com/mudler/LocalAI/addons/internal/protocol"
+	"github.com/mudler/LocalAI/pkg/addons/contract"
+)
+
+// The undo of a chat's change (see contract.UndoAction). The start of a job is
+// the one change that has a change set: undoing it stops the job while it
+// runs. Both calls are the host's alone: they are not in the manifest's tools,
+// so no model or agent can name them. The shared steps are in jobundo.
+
+// startedAt is when the job was started, from the journal.
+func (j job) startedAt() time.Time {
+	if j.StartedNS == 0 {
+		return time.Time{}
+	}
+	return time.Unix(0, j.StartedNS)
+}
+
+func (r *runtime) undoer() *jobundo.Undoer[job] {
+	return &jobundo.Undoer[job]{
+		Label: "fine-tuning", Schema: schema, GetOp: "finetune.get", StopOp: "finetune.stop",
+		StopArgs: func(jobID string) any { return map[string]any{"jobId": jobID, "saveCheckpoint": false} },
+		Engine:   r.eng, Now: r.now,
+		Lookup: r.entryOf,
+		Entry: func(j job, known bool) jobundo.Entry {
+			return jobundo.Entry{Known: known && !j.Dataset, Name: j.Name, JobID: j.JobID, Revision: j.Revision, StartOp: j.StartOp, Started: j.startedAt(), UndoneBy: j.UndoneBy}
+		},
+		Receipt: r.j.receipt,
+		Commit: func(resource string, j job, undoneBy string, rec contract.Receipt) error {
+			next := j
+			next.Revision++
+			next.UndoneBy = undoneBy
+			return r.j.commit(resource, &next, rec)
+		},
+		MapErr: protocol.FromHostError, Payload: protocol.Payload,
+	}
+}
+
+// readChange answers what the change set of a job start says now.
+func (r *runtime) readChange(ctx context.Context, env contract.Envelope) (contract.Result, error) {
+	return r.undoer().Read(ctx, env)
+}
+
+// applyUndo stops the job the change started, if it still runs.
+func (r *runtime) applyUndo(ctx context.Context, env contract.Envelope, rev contract.Revision) (contract.Receipt, error) {
+	return r.undoer().Apply(ctx, env, rev)
+}

addons/fine-tuning/internal/tools/undo_test.go

@@ -0,0 +1,138 @@
+// SPDX-License-Identifier: MIT
+package tools
+
+import (
+	"encoding/json"
+	"errors"
+
+	"github.com/mudler/LocalAI/addons/internal/protocol"
+	"github.com/mudler/LocalAI/pkg/addons/contract"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// undoEnv is an envelope of the host's undo calls.
+func undoEnv(tool, op, res, change, scope, action string) contract.Envelope {
+	e := envelope(tool, op, res, `{"change":"`+change+`"}`)
+	e.Request.Action, e.Authority.Scope = action, scope
+	return e
+}
+
+func changeOf(rt contract.Runtime, res, change string) contract.ChangeInfo {
+	GinkgoHelper()
+	r, err := rt.Read(ctx, undoEnv("changes-get", "get-"+change, res, change, "read", "read"))
+	Expect(err).NotTo(HaveOccurred())
+	var info contract.ChangeInfo
+	Expect(json.Unmarshal(r.Payload, &info)).To(Succeed())
+	return info
+}
+
+var _ = Describe("undo of a job start", func() {
+	It("says in the receipt of the start that it can be undone, and that a stop, an export and a dataset cannot", func() {
+		eng := &fakeEngine{result: map[string]any{"finetune.start": map[string]any{"id": "job-1", "status": "queued"}, "finetune.stop": map[string]any{}, "finetune.export": map[string]any{}, "finetune.dataset": map[string]any{"datasetRef": "ref-1"}}}
+		rt := New(eng, GinkgoT().TempDir())
+		rec, err := rt.Apply(ctx, envelope("fine-tune-start", "create-res1", "res1", startJSON), contract.Revision{Schema: 1})
+		Expect(err).NotTo(HaveOccurred())
+		n, ok := contract.ParseUndoNote(rec.Result.Payload)
+		Expect(ok).To(BeTrue())
+		Expect(n.Available).To(BeTrue())
+		Expect(n.ID).To(Equal("create-res1"))
+		Expect(n.Summary).To(ContainSubstring("my run"))
+		stop, err := rt.Apply(ctx, envelope("fine-tune-stop", "stop-1", "res1", ""), contract.Revision{Schema: 1, Data: 1})
+		Expect(err).NotTo(HaveOccurred())
+		n, ok = contract.ParseUndoNote(stop.Result.Payload)
+		Expect(ok).To(BeTrue())
+		Expect(n.Available).To(BeFalse())
+		Expect(n.Reason).To(ContainSubstring("cannot be undone"))
+		ds, err := rt.Apply(ctx, envelope("fine-tune-dataset", "create-ds", "ds1", `{"name":"d.jsonl","content":"{}"}`), contract.Revision{Schema: 1})
+		Expect(err).NotTo(HaveOccurred())
+		n, _ = contract.ParseUndoNote(ds.Result.Payload)
+		Expect(n.Available).To(BeFalse())
+		Expect(n.Reason).To(ContainSubstring("dataset"))
+	})
+
+	It("stops a running job once, and answers the same operation ID with the same receipt", func() {
+		rt, eng := started()
+		eng.result["finetune.get"] = map[string]any{"status": "training"}
+		info := changeOf(rt, "res1", "create-res1")
+		Expect(info.State).To(Equal(contract.ChangeAvailable))
+		Expect(info.Class).To(Equal(contract.ChangeClassSchema))
+		Expect(info.Summary).To(ContainSubstring("Stop"))
+		undo := undoEnv("changes-undo", "undo-1", "res1", "create-res1", "apply", "undo")
+		rec, err := rt.Apply(ctx, undo, info.Current)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(rec.State).To(Equal(contract.ReceiptCommitted))
+		Expect(eng.count("finetune.stop")).To(Equal(1))
+		Expect(eng.last("finetune.stop").args).To(HaveKeyWithValue("jobId", "job-1"))
+		again, err := rt.Apply(ctx, undo, info.Current)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(again).To(Equal(rec))
+		Expect(eng.count("finetune.stop")).To(Equal(1), "the engine ran once")
+		Expect(changeOf(rt, "res1", "create-res1").State).To(Equal(contract.ChangeUndone))
+		other, err := rt.Apply(ctx, undoEnv("changes-undo", "undo-2", "res1", "create-res1", "apply", "undo"), info.Current)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(other.State).To(Equal(contract.ReceiptRejected))
+		Expect(string(other.Result.Payload)).To(ContainSubstring(contract.UndoAlreadyUndone))
+		_, err = rt.Apply(ctx, undoEnv("changes-undo", "undo-1", "res1", "other-change", "apply", "undo"), info.Current)
+		Expect(errors.Is(err, protocol.ErrConflict)).To(BeTrue(), "the same ID for another change: %v", err)
+	})
+
+	It("says a finished job cannot be undone, and does not stop it", func() {
+		rt, eng := started()
+		for _, st := range []string{"completed", "failed", "stopped"} {
+			eng.result["finetune.get"] = map[string]any{"status": st}
+			info := changeOf(rt, "res1", "create-res1")
+			Expect(info.State).To(Equal(contract.ChangeFinal), st)
+			Expect(info.Detail).To(ContainSubstring("finished"))
+		}
+		rec, err := rt.Apply(ctx, undoEnv("changes-undo", "undo-1", "res1", "create-res1", "apply", "undo"), contract.Revision{Schema: 1, Data: 1})
+		Expect(err).NotTo(HaveOccurred())
+		Expect(rec.State).To(Equal(contract.ReceiptRejected))
+		Expect(string(rec.Result.Payload)).To(ContainSubstring(contract.UndoCannot))
+		Expect(eng.count("finetune.stop")).To(Equal(0))
+	})
+
+	It("reports a job that finished between the read and the stop as cannot_undo", func() {
+		rt, eng := started()
+		eng.result["finetune.get"] = map[string]any{"status": "training"}
+		eng.result["finetune.stop"] = &contract.HostError{Code: contract.HostConflict, Message: "the job is not running"}
+		rec, err := rt.Apply(ctx, undoEnv("changes-undo", "undo-1", "res1", "create-res1", "apply", "undo"), contract.Revision{Schema: 1, Data: 1})
+		Expect(err).NotTo(HaveOccurred())
+		Expect(rec.State).To(Equal(contract.ReceiptRejected))
+		Expect(string(rec.Result.Payload)).To(ContainSubstring(contract.UndoCannot))
+	})
+
+	It("refuses after a later change to the job, after a week, and for a change that is not the start", func() {
+		rt, eng := started()
+		eng.result["finetune.get"] = map[string]any{"status": "training"}
+		Expect(changeOf(rt, "res1", "nothing").State).To(Equal(contract.ChangeUnknown))
+		Expect(changeOf(rt, "nores", "create-res1").State).To(Equal(contract.ChangeUnknown))
+		eng.result["finetune.export"] = map[string]any{}
+		_, err := rt.Apply(ctx, envelope("fine-tune-export", "export-1", "res1", ""), contract.Revision{Schema: 1, Data: 1})
+		Expect(err).NotTo(HaveOccurred())
+		Expect(changeOf(rt, "res1", "create-res1").State).To(Equal(contract.ChangeFinal))
+
+		rt2, eng2 := started()
+		eng2.result["finetune.get"] = map[string]any{"status": "training"}
+		r := rt2.(*runtime)
+		j, _ := r.j.job("res1")
+		j.StartedNS = 1
+		Expect(r.j.commit("res1", &j, contract.Receipt{OperationID: "bump-1", State: contract.ReceiptCommitted})).To(Succeed())
+		Expect(changeOf(rt2, "res1", "create-res1").State).To(Equal(contract.ChangeExpired))
+	})
+
+	It("answers only the host: the undo calls need their own action and scope", func() {
+		rt, eng := started()
+		eng.result["finetune.get"] = map[string]any{"status": "training"}
+		for _, c := range []struct{ scope, action string }{{"apply", "write"}, {"apply", "schema"}, {"preview", "undo"}, {"read", "undo"}} {
+			_, err := rt.Apply(ctx, undoEnv("changes-undo", "undo-x", "res1", "create-res1", c.scope, c.action), contract.Revision{Schema: 1, Data: 1})
+			Expect(errors.Is(err, protocol.ErrForbidden)).To(BeTrue(), "%v %v", c, err)
+		}
+		_, err := rt.Read(ctx, undoEnv("changes-get", "g", "res1", "create-res1", "apply", "read"))
+		Expect(errors.Is(err, protocol.ErrForbidden)).To(BeTrue())
+		Expect(eng.count("finetune.stop")).To(Equal(0))
+		for _, tool := range Tools {
+			Expect(tool).NotTo(HavePrefix("changes-"), "the undo calls are never model tools")
+		}
+	})
+})

addons/fine-tuning/manifest.json

@@ -46,7 +46,7 @@
       {
         "name": "fine-tune-start",
         "kind": "create",
-        "description": "Propose a new fine-tuning job. A person confirms it in the approval card; nothing starts before that. datasetSource is a Hugging Face ID or a reference from fine-tune-dataset. A model can only control jobs it started.",
+        "description": "Start a new fine-tuning job. In a chat it runs at once and the person can undo it; from an API key it is only a proposal that a person confirms. datasetSource is a Hugging Face ID or a reference from fine-tune-dataset. A model can only control jobs it started.",
         "inputSchema": {
           "type": "object",
           "properties": {
@@ -144,7 +144,7 @@
       {
         "name": "fine-tune-dataset",
         "kind": "create",
-        "description": "Propose sending a small text dataset (up to 900 KiB, a .jsonl, .json, .csv or .txt file) to the server. A person confirms it. The reference it returns is the datasetSource of fine-tune-start, works for that person only and is lost when the server restarts. A model can only control jobs it started.",
+        "description": "Send a small text dataset (up to 900 KiB, a .jsonl, .json, .csv or .txt file) to the server. In a chat it runs at once and the person can undo it; from an API key it is only a proposal that a person confirms. The reference it returns is the datasetSource of fine-tune-start, works for that person only and is lost when the server restarts. A model can only control jobs it started.",
         "inputSchema": {
           "type": "object",
           "properties": {
@@ -205,7 +205,7 @@
       {
         "name": "fine-tune-stop",
         "kind": "write",
-        "description": "Propose stopping a fine-tuning job that a tool started. A person confirms it. A model can only control jobs it started.",
+        "description": "Stop a fine-tuning job that a tool started. Stopping cannot be undone. In a chat it runs at once and the person can undo it; from an API key it is only a proposal that a person confirms. A model can only control jobs it started.",
         "inputSchema": {
           "type": "object",
           "properties": {
@@ -227,7 +227,7 @@
       {
         "name": "fine-tune-export",
         "kind": "write",
-        "description": "Propose registering the result of a fine-tuning job that a tool started as a model. Administrators only. A model can only control jobs it started.",
+        "description": "Register the result of a fine-tuning job that a tool started as a model. Administrators only. Exporting cannot be undone. In a chat it runs at once and the person can undo it; from an API key it is only a proposal that a person confirms. A model can only control jobs it started.",
         "inputSchema": {
           "type": "object",
           "properties": {
@@ -258,6 +258,7 @@
         }
       }
     ],
+    "undo": true,
     "resources": {
       "noun": "fine-tuning job",
       "create": "fine-tune-start",

addons/fine-tuning/package/platforms.yaml

@@ -30,7 +30,7 @@ platforms:
     native_check: executable-start
   - platform: darwin/arm64
     enabled: true
-    runner: macos-14
+    runner: macos-15
     native_check: executable-start
   - platform: darwin/amd64
     enabled: true

addons/internal/atomicfile/atomicfile.go

@@ -0,0 +1,62 @@
+// SPDX-License-Identifier: MIT
+
+// Package atomicfile replaces a file of an add-on so that a reader sees the old
+// content or the new content, never a mix, and so that the new content
+// survives a power loss once the call returns.
+//
+// It uses the standard library only, so an add-on that has a short list of
+// allowed imports can use it. The server has its own copy in pkg/atomicfile.
+package atomicfile
+
+import (
+	"bytes"
+	"fmt"
+	"io"
+	"os"
+	"path/filepath"
+)
+
+// Write replaces the file at path with data and perm.
+func Write(path string, data []byte, perm os.FileMode) error {
+	return WriteFrom(path, bytes.NewReader(data), perm)
+}
+
+// WriteFrom replaces the file at path with what r reads and perm. It writes a
+// temporary file in the same folder, syncs it, renames it over path and syncs
+// the folder. On error the old file is untouched and the temporary file is
+// gone, except when only the sync of the folder failed: the new content is in
+// place then, but a crash may still lose it.
+func WriteFrom(path string, r io.Reader, perm os.FileMode) error {
+	dir := filepath.Dir(path)
+	f, err := os.CreateTemp(dir, "."+filepath.Base(path)+".tmp-*")
+	if err != nil {
+		return err
+	}
+	// Removing the temporary name is a no-op once it was renamed.
+	defer func() { _ = os.Remove(f.Name()) }()
+	if _, err = io.Copy(f, r); err == nil {
+		// Without the sync a power loss after the rename can leave an empty file.
+		err = f.Sync()
+	}
+	if err == nil {
+		err = f.Chmod(perm)
+	}
+	if cerr := f.Close(); err == nil {
+		err = cerr
+	}
+	if err != nil {
+		return err
+	}
+	if err := os.Rename(f.Name(), path); err != nil {
+		return err
+	}
+	d, err := os.Open(dir) // #nosec G304 -- the folder of the file being written
+	if err != nil {
+		return fmt.Errorf("atomicfile: sync %s: %w", dir, err)
+	}
+	defer func() { _ = d.Close() }()
+	if err := d.Sync(); err != nil {
+		return fmt.Errorf("atomicfile: sync %s: %w", dir, err)
+	}
+	return nil
+}

addons/internal/atomicfile/atomicfile_test.go

@@ -0,0 +1,75 @@
+// SPDX-License-Identifier: MIT
+package atomicfile
+
+import (
+	"errors"
+	"io"
+	"os"
+	"path/filepath"
+	"strings"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// failingReader gives some bytes and then fails, as a source that breaks off
+// does.
+type failingReader struct{ done bool }
+
+func (r *failingReader) Read(p []byte) (int, error) {
+	if r.done {
+		return 0, errors.New("the source broke off")
+	}
+	r.done = true
+	return copy(p, "half"), nil
+}
+
+var _ = Describe("atomicfile", func() {
+	var dir, path string
+
+	BeforeEach(func() {
+		dir = GinkgoT().TempDir()
+		path = filepath.Join(dir, "state.json")
+	})
+
+	entries := func() []string {
+		es, err := os.ReadDir(dir)
+		Expect(err).NotTo(HaveOccurred())
+		var names []string
+		for _, e := range es {
+			names = append(names, e.Name())
+		}
+		return names
+	}
+
+	It("writes a new file with its mode and leaves no temporary file", func() {
+		Expect(Write(path, []byte("one"), 0o640)).To(Succeed())
+		Expect(os.ReadFile(path)).To(Equal([]byte("one")))
+		fi, err := os.Stat(path)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(fi.Mode().Perm()).To(Equal(os.FileMode(0o640)))
+		Expect(entries()).To(Equal([]string{"state.json"}))
+	})
+
+	It("replaces the content of a file", func() {
+		Expect(Write(path, []byte("one"), 0o600)).To(Succeed())
+		Expect(WriteFrom(path, strings.NewReader("two"), 0o755)).To(Succeed())
+		Expect(os.ReadFile(path)).To(Equal([]byte("two")))
+		fi, err := os.Stat(path)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(fi.Mode().Perm()).To(Equal(os.FileMode(0o755)))
+	})
+
+	It("keeps the old file when the source fails", func() {
+		Expect(Write(path, []byte("one"), 0o600)).To(Succeed())
+		var r io.Reader = &failingReader{}
+		Expect(WriteFrom(path, r, 0o600)).To(MatchError("the source broke off"))
+		Expect(os.ReadFile(path)).To(Equal([]byte("one")))
+		Expect(entries()).To(Equal([]string{"state.json"}))
+	})
+
+	It("fails when the folder does not exist", func() {
+		Expect(Write(filepath.Join(dir, "missing", "state.json"), []byte("one"), 0o600)).NotTo(Succeed())
+		Expect(entries()).To(BeEmpty())
+	})
+})

addons/internal/atomicfile/suite_test.go

@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: MIT
+package atomicfile
+
+import (
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestAtomicfile(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "Atomicfile suite")
+}

addons/internal/connector-ui/src/AnswerFields.jsx

@@ -0,0 +1,98 @@
+/* eslint-disable no-unused-vars -- components used only inside JSX look unused to this config, which has no eslint-plugin-react */
+// The fields of a binding that say how the bot answers: what a sender sees
+// while the agent works, which chat a message belongs to, and whether the
+// files of a turn go out. The words are the same on every platform, so they
+// live here as defaults; an add-on can still change one in its namespace.
+import { Field, Hint, Select, Switch, TextInput } from './controls'
+import { CONVERSATIONS, MAX_WINDOW_MINUTES, REPLY_MODES } from './connector'
+
+export const ANSWER_TEXT = {
+  'answer.title': 'How it answers',
+  'answer.replyLabel': 'While the agent works',
+  'answer.reply.off': 'Typing, then the answer',
+  'answer.reply.progress': 'Progress: the tools it uses',
+  'answer.reply.stream': 'Live: the answer as it is written',
+  'answer.replyHelp': 'Progress and Live show one working message, and the answer replaces it. On a platform with drafts, Live shows a private chat a draft instead, and the answer follows it. A long answer goes on in new messages.',
+  'answer.conversationLabel': 'Conversations',
+  'answer.conversation.sender': 'One chat per person',
+  'answer.conversation.chat': 'One chat per group or channel',
+  'answer.conversation.window': 'Per person, new after a quiet time',
+  'answer.conversation.thread': 'New chat per message; replies go on',
+  'answer.window': 'Quiet time, in minutes',
+  'answer.windowRange': 'Use a whole number from 1 to {{max}}.',
+  'answer.files': 'Send the files the agent makes',
+  'aside.title': 'How it works',
+  'aside.step1': 'Give the connection a name and its token. The token is sealed and never shown again.',
+  'aside.step2': 'Pick the agent that answers. It needs a model.',
+  'aside.step3': 'Choose who may write to it, and how it answers.',
+  'aside.testTitle': 'Test the connection',
+  'aside.testText': 'When you save a new token, LocalAI checks it and names the account it belongs to. Use Test on a saved row to check it again.',
+  'add': 'Add',
+  'more': 'More',
+  'less': 'Less',
+  'answer.filesHelp': 'After the answer, up to 5 files of at most 1 MB each. Programs and scripts are not sent.',
+  'answer.summary': 'Answers: {{reply}} · {{conversation}}',
+  'answer.short.off': 'typing',
+  'answer.short.progress': 'progress',
+  'answer.short.stream': 'live answer',
+  'answer.short.sender': 'a chat per person',
+  'answer.short.chat': 'a chat per group',
+  'answer.short.window': 'new chat after {{count}} min',
+  'answer.short.thread': 'a chat per message',
+}
+
+// text reads a string of the add-on's namespace, with the default above.
+export function answerT(t) {
+  return (key, opts = {}) => t(key, { defaultValue: ANSWER_TEXT[key], ...opts })
+}
+
+// answerSummary says in a few words how a binding answers, for its row.
+export function answerSummary(c, t) {
+  const s = answerT(t)
+  return s('answer.summary', {
+    reply: s(`answer.short.${c.reply_mode || 'progress'}`),
+    conversation: s(`answer.short.${c.conversation || 'sender'}`, { count: c.window_minutes }),
+  })
+}
+
+export default function AnswerFields({ form, setForm, t }) {
+  const s = answerT(t)
+  const m = Number(form.window_minutes)
+  const windowBad = form.conversation === 'window' && (!Number.isInteger(m) || m < 1 || m > MAX_WINDOW_MINUTES)
+  return (
+    <fieldset className="cn-group" data-testid="connector-answer">
+      <legend>{s('answer.title')}</legend>
+      <Field label={s('answer.replyLabel')} hint={s('answer.replyHelp')}>
+        {(p) => (
+          <Select {...p} data-testid="connector-reply" value={form.reply_mode || 'progress'}
+            onChange={(e) => setForm({ ...form, reply_mode: e.target.value })}>
+            {REPLY_MODES.map((x) => <option key={x} value={x}>{s(`answer.reply.${x}`)}</option>)}
+          </Select>
+        )}
+      </Field>
+      <Field label={s('answer.conversationLabel')}>
+        {(p) => (
+          <Select {...p} data-testid="connector-conversation" value={form.conversation || 'sender'}
+            onChange={(e) => setForm({ ...form, conversation: e.target.value })}>
+            {CONVERSATIONS.map((x) => <option key={x} value={x}>{s(`answer.conversation.${x}`)}</option>)}
+          </Select>
+        )}
+      </Field>
+      {form.conversation === 'window' && (
+        <Field label={s('answer.window')} className="cn-answer__window"
+          error={windowBad ? s('answer.windowRange', { max: MAX_WINDOW_MINUTES }) : undefined}>
+          {(p) => (
+            <TextInput {...p} type="number" inputMode="numeric" min={1} max={MAX_WINDOW_MINUTES} required data-testid="connector-window"
+              value={form.window_minutes ?? ''} onChange={(e) => setForm({ ...form, window_minutes: e.target.value === '' ? '' : Number(e.target.value) })} />
+          )}
+        </Field>
+      )}
+      <div className="cn-switchfield">
+        <Switch testId="connector-files" checked={form.send_files !== false} onChange={(on) => setForm({ ...form, send_files: on })}>
+          {s('answer.files')}
+        </Switch>
+        <Hint>{s('answer.filesHelp')}</Hint>
+      </div>
+    </fieldset>
+  )
+}

addons/internal/connector-ui/src/ConnectorPage.jsx

@@ -0,0 +1,256 @@
+/* eslint-disable no-unused-vars -- components used only inside JSX look unused to this config, which has no eslint-plugin-react */
+// The page of a connector add-on (Telegram, Slack): the bindings of one kind
+// of bot to the saved agents. The bindings, the sealed token and the sender
+// log live in core; this page edits them through the core API. An add-on
+// gives its own words (an i18n namespace), the fields of its token, and its
+// allowlists; the rest is the same for every platform.
+import { useCallback, useEffect, useState } from 'react'
+import { useTranslation } from 'react-i18next'
+import ConfirmDialog from '../../host-compat/src/components/ConfirmDialog'
+import Icon from '../../host-compat/src/components/Icon'
+import AnswerFields, { answerSummary, answerT } from './AnswerFields'
+import { Check, Field, Notice, RadioCards, Select, Switch, TextInput } from './controls'
+import NameList from './NameList'
+import { agentsApi, connectorsApi } from './api'
+import { canSave, connectorPayload, emptyConnector, listEntries, replaceList, senderLine } from './connector'
+import './connector.css'
+
+function Senders({ id, onAllowed, t }) {
+  const [list, setList] = useState(null)
+  const [error, setError] = useState('')
+  const load = useCallback(() => {
+    connectorsApi.senders(id).then((r) => setList(r.senders || [])).catch((e) => setError(e?.message || ''))
+  }, [id])
+  useEffect(() => { load() }, [load])
+  if (error) return <Notice tone="error" role="alert">{error}</Notice>
+  if (list === null) return null
+  if (list.length === 0) return <p className="cn-muted">{t('noSenders')}</p>
+  return (
+    <ul className="cn-runs" aria-label={t('senders')} data-testid="sender-log">
+      {list.map((s) => (
+        <li key={s.id}>
+          <span className={`cn-status cn-status--${s.allowed ? 'done' : 'skipped'}`}>{s.allowed ? t('accepted') : t('ignored')}</span>
+          <span>{senderLine(s)}</span>
+          {!s.allowed && (
+            <button type="button" className="dk-btn dk-btn--secondary dk-btn--sm"
+              onClick={async () => {
+                setError('')
+                try { await connectorsApi.allow(id, s.id); load(); onAllowed() } catch (e) { setError(e?.message || '') }
+              }}>{t('allowButton')}</button>
+          )}
+        </li>
+      ))}
+    </ul>
+  )
+}
+
+// The default fields of a token: one password field.
+function SingleToken({ form, setForm, t }) {
+  return (
+    <Field label={t('token')} hint={t('tokenHelp')}>
+      {(p) => (
+        <TextInput {...p} mono type="password" autoComplete="off" value={form.token || ''} required={!form.id}
+          placeholder={form.id ? t('tokenSaved') : ''} onChange={(e) => setForm({ ...form, token: e.target.value })} />
+      )}
+    </Field>
+  )
+}
+
+const ONE_LIST = [{ key: 'allow', label: 'allow', hint: 'allowHint', testId: 'connector-allow' }]
+
+/**
+ * @param {object} props
+ * @param {string} props.kind          the kind of the bindings, such as telegram
+ * @param {string} props.ns            the i18n namespace of the add-on
+ * @param {string} props.testId        the test id of the page
+ * @param {Function} [props.TokenFields] the fields of the token ({ form, setForm, t })
+ * @param {Function} [props.tokenOf]   the secret that the fields make, '' when not typed
+ * @param {Array} [props.lists]        the allowlists: { key, label, hint, testId, match }
+ * @param {Function} [props.Help]      a block under the intro ({ t })
+ * @param {string} [props.notifyMode]  the inputMode of the notification field
+ */
+export default function ConnectorPage({ kind, ns, testId, TokenFields = SingleToken, tokenOf = (f) => f.token || '', lists = ONE_LIST, Help, notifyMode = 'text' }) {
+  const { t } = useTranslation(ns)
+  const [items, setItems] = useState([])
+  const [agents, setAgents] = useState([])
+  const [off, setOff] = useState('')
+  const [error, setError] = useState('')
+  const [note, setNote] = useState('')
+  const [form, setForm] = useState(null)
+  const [confirmed, setConfirmed] = useState(false)
+  const [busy, setBusy] = useState(false)
+  const [toDelete, setToDelete] = useState(null)
+  const [openSenders, setOpenSenders] = useState('')
+
+  const fail = useCallback((e) => {
+    if (e?.status === 409 && e.body?.error?.type === 'connectors_off') { setOff(e.body.error.message); return }
+    setError(e?.message || '')
+  }, [])
+
+  const load = useCallback(async () => {
+    try {
+      const res = await connectorsApi.list()
+      setItems((res.connectors || []).filter((c) => c.kind === kind))
+      setOff('')
+    } catch (e) { fail(e) }
+  }, [fail, kind])
+
+  useEffect(() => { load() }, [load])
+  useEffect(() => { agentsApi.list().then((r) => setAgents(r.agents || [])).catch(() => {}) }, [])
+
+  const agentName = (id) => agents.find((a) => a.id === id)?.name || id
+  const open = (c) => { setForm(c); setConfirmed(Boolean(c.accept_any)); setNote('') }
+
+  // The server refuses a token that the platform refuses. After a save with
+  // a new token, the page names the bot the token belongs to.
+  const save = async (e) => {
+    e.preventDefault()
+    setBusy(true)
+    setError('')
+    setNote('')
+    try {
+      const token = tokenOf(form).trim()
+      const body = connectorPayload(form, kind, token)
+      const saved = form.id ? await connectorsApi.update(form.id, body) : await connectorsApi.create(body)
+      setForm(null)
+      await load()
+      if (token && saved?.id) {
+        try {
+          const r = await connectorsApi.test(saved.id)
+          setNote(t('testOk', { bot: r.bot_username }))
+        } catch (err) { setError(t('checkFailed', { reason: err?.message || '' })) }
+      }
+    } catch (err) { fail(err) } finally { setBusy(false) }
+  }
+
+  const act = async (fn) => {
+    setError('')
+    try { await fn(); await load() } catch (e) { fail(e) }
+  }
+
+  if (off) {
+    return (
+      <div className="cn-page" data-testid={testId}>
+        <h1 className="cn-title">{t('title')}</h1>
+        <Notice tone="info" role="note">{off}</Notice>
+      </div>
+    )
+  }
+
+  return (
+    <div className="cn-page" data-testid={testId}>
+      <header className="cn-head">
+        <h1 className="cn-title">{t('title')}</h1>
+        <button type="button" className="dk-btn dk-btn--primary" onClick={() => open(emptyConnector())}>
+          <Icon name="plus" /> {t('new')}
+        </button>
+      </header>
+      <p className="cn-muted">{t('intro')}</p>
+      {Help && <Help t={t} />}
+
+      {error && (
+        <Notice tone="error" role="alert">
+          <span>{error}</span>
+          <button type="button" className="cn-link" aria-label={t('dismiss', { defaultValue: 'Dismiss' })} onClick={() => setError('')}>×</button>
+        </Notice>
+      )}
+      {note && <Notice tone="ok" role="status">{note}</Notice>}
+
+      {form && (
+        <div className="cn-formgrid">
+          <form className="cn-form" onSubmit={save} aria-label={form.id ? t('edit') : t('new')}>
+            <Field label={t('name')}>
+              {(p) => <TextInput {...p} value={form.name} required maxLength={80} onChange={(e) => setForm({ ...form, name: e.target.value })} />}
+            </Field>
+            <TokenFields form={form} setForm={setForm} t={t} />
+            <Field label={t('agent')}>
+              {(p) => (
+                <Select {...p} aria-label={t('agent')} value={form.agent_id} required onChange={(e) => setForm({ ...form, agent_id: e.target.value })}>
+                  <option value="">{t('agentNone')}</option>
+                  {agents.map((a) => (
+                    <option key={a.id} value={a.id} disabled={!a.model}>{a.model ? a.name : t('agentNoModel', { name: a.name })}</option>
+                  ))}
+                </Select>
+              )}
+            </Field>
+            <fieldset className="cn-group">
+              <legend>{t('who')}</legend>
+              <RadioCards name="who" legend={t('who')} value={form.accept_any} onChange={(any) => { setForm({ ...form, accept_any: any }); if (!any) setConfirmed(false) }}
+                options={[{ value: false, label: t('onlyListed') }, { value: true, label: t('anyone') }]} />
+              {form.accept_any && (
+                <Notice tone="warn" role="alert" testId="anyone-warning">
+                  <p className="cn-notice__text">{t('anyoneWarning')}</p>
+                  <Check checked={confirmed} onChange={setConfirmed}>{t('anyoneConfirm')}</Check>
+                </Notice>
+              )}
+              {!form.accept_any && lists.map((l, i) => (
+                <NameList key={l.key} label={t(l.label)} hint={t(l.hint)} value={listEntries(form.allow, lists, i)} addLabel={answerT(t)('add')}
+                  onChange={(v) => setForm({ ...form, allow: replaceList(form.allow, lists, i, v) })} testId={l.testId} />
+              ))}
+            </fieldset>
+            <Field label={t('notifyChat')} hint={t('notifyHelp')} optional>
+              {(p) => <TextInput {...p} mono inputMode={notifyMode} value={form.notify_chat} onChange={(e) => setForm({ ...form, notify_chat: e.target.value })} />}
+            </Field>
+            <AnswerFields form={form} setForm={setForm} t={t} />
+            <div className="cn-actions">
+              <button type="submit" className="dk-btn dk-btn--primary" aria-busy={busy ? 'true' : undefined} disabled={busy || !canSave(form, confirmed, Boolean(tokenOf(form).trim()))}>{t('save')}</button>
+              <button type="button" className="dk-btn dk-btn--secondary" onClick={() => setForm(null)}>{t('cancel')}</button>
+            </div>
+          </form>
+          <aside className="cn-aside" aria-label={answerT(t)('aside.title')}>
+            <h2 className="cn-aside__title">{answerT(t)('aside.title')}</h2>
+            <ol className="cn-aside__steps">
+              <li>{answerT(t)('aside.step1')}</li>
+              <li>{answerT(t)('aside.step2')}</li>
+              <li>{answerT(t)('aside.step3')}</li>
+            </ol>
+            <h2 className="cn-aside__title">{answerT(t)('aside.testTitle')}</h2>
+            <p className="cn-muted">{answerT(t)('aside.testText')}</p>
+          </aside>
+        </div>
+      )}
+
+      <ul className="cn-list">
+        {items.map((c) => (
+          <li key={c.id} className="cn-row" data-testid="connector-row">
+            <div className="cn-row__text">
+              <span className="cn-row__name">{c.name}</span>
+              <span className="cn-muted">
+                {t('agentOf', { name: agentName(c.agent_id) })}
+                {' · '}
+                {t('accepts', { who: c.accept_any ? t('modes.anyone') : (c.allow || []).length === 0 ? t('modes.nobody') : t('modes.listed', { count: c.allow.length }) })}
+              </span>
+              <span className="cn-muted" data-testid="connector-answer-summary">{answerSummary(c, t)}</span>
+              {c.active === false && <span className="cn-muted" data-testid="connector-inactive">{t('inactive', { reason: c.reason })}</span>}
+              {openSenders === c.id && <Senders id={c.id} onAllowed={load} t={t} />}
+            </div>
+            <span className="cn-row__actions">
+              <Switch checked={c.enabled} label={t('onFor', { name: c.name })}
+                onChange={(on) => {
+                  setItems((list) => list.map((x) => (x.id === c.id ? { ...x, enabled: on } : x)))
+                  act(() => (on ? connectorsApi.enable(c.id) : connectorsApi.disable(c.id)))
+                }} />
+              <button type="button" className="dk-btn dk-btn--secondary dk-btn--sm"
+                onClick={async () => { setError(''); try { const r = await connectorsApi.test(c.id); setNote(t('testOk', { bot: r.bot_username })) } catch (e) { fail(e) } }}>
+                {t('test')}
+              </button>
+              <button type="button" className="dk-btn dk-btn--secondary dk-btn--sm" onClick={() => setOpenSenders(openSenders === c.id ? '' : c.id)}>{t('senders')}</button>
+              <button type="button" className="dk-btn dk-btn--secondary dk-btn--sm" onClick={() => open({ ...emptyConnector(), ...c, token: '' })}>{t('editButton')}</button>
+              <button type="button" className="dk-btn dk-btn--secondary dk-btn--sm" onClick={() => setToDelete(c)}>{t('delete')}</button>
+            </span>
+          </li>
+        ))}
+      </ul>
+      {items.length === 0 && !form && <p className="cn-muted">{t('empty')}</p>}
+      <ConfirmDialog
+        open={Boolean(toDelete)}
+        title={t('confirmDelete.title', { name: toDelete?.name || '' })}
+        message={t('confirmDelete.message')}
+        confirmLabel={t('confirmDelete.confirm')}
+        danger
+        onConfirm={() => { const c = toDelete; setToDelete(null); act(() => connectorsApi.remove(c.id)) }}
+        onCancel={() => setToDelete(null)}
+      />
+    </div>
+  )
+}

addons/internal/connector-ui/src/NameList.jsx

@@ -0,0 +1,43 @@
+/* eslint-disable no-unused-vars -- components used only inside JSX look unused to this config, which has no eslint-plugin-react */
+import { useState } from 'react'
+import Icon from '../../host-compat/src/components/Icon'
+import { Field } from './controls'
+
+// A list of senders as chips, with a field to add one: Enter or a comma adds
+// the entry. Entries are trimmed and kept once; the server checks their shape.
+export default function NameList({ label, hint, value, onChange, testId, addLabel = 'Add' }) {
+  const [text, setText] = useState('')
+  const add = () => {
+    const v = text.trim()
+    if (v && !value.includes(v)) onChange([...value, v])
+    setText('')
+  }
+  return (
+    <div data-testid={testId}>
+      <Field label={label}>
+        {(p) => (
+          <>
+            {value.length > 0 && (
+              <ul className="cn-chips" aria-label={label}>
+                {value.map((v) => (
+                  <li key={v} className="dk-chip dk-chip--sm">
+                    <span>{v}</span>
+                    <button type="button" className="dk-chip-x" aria-label={`${v} ×`} onClick={() => onChange(value.filter((x) => x !== v))}>
+                      <Icon name="close" />
+                    </button>
+                  </li>
+                ))}
+              </ul>
+            )}
+            <div className="cn-add">
+              <input {...p} className="dk-input" value={text} aria-label={`${label} +`} placeholder={hint}
+                onChange={(e) => setText(e.target.value)}
+                onKeyDown={(e) => { if (e.key === 'Enter' || e.key === ',') { e.preventDefault(); add() } }} />
+              <button type="button" className="dk-btn dk-btn--secondary" onClick={add} disabled={!text.trim()}>{addLabel}</button>
+            </div>
+          </>
+        )}
+      </Field>
+    </div>
+  )
+}

addons/internal/connector-ui/src/api.js

@@ -0,0 +1,40 @@
+// The core calls of a connector page, with the signed-in person's own session
+// (sdk.core.fetch). The bindings, the sealed token and the sender log live in
+// core; the add-on only carries the messages.
+import { getSdk } from '../../host-compat/src/runtime.js'
+
+const enc = encodeURIComponent
+
+async function call(path, init = {}) {
+  const response = await getSdk().core.fetch(path, { headers: { 'Content-Type': 'application/json' }, ...init })
+  if (!response.ok) {
+    let message = `HTTP ${response.status}`
+    let body = null
+    try {
+      body = await response.json()
+      message = body?.error?.message || message
+    } catch (_e) { /* the body was not JSON */ }
+    const err = new Error(message)
+    err.status = response.status
+    err.body = body
+    throw err
+  }
+  if (response.status === 204) return null
+  return response.json()
+}
+
+const send = (method, path, body) => call(path, { method, body: body === undefined ? undefined : JSON.stringify(body) })
+
+export const agentsApi = { list: () => call('/api/chats/agents') }
+
+export const connectorsApi = {
+  list: () => call('/api/chats/connectors'),
+  create: (body) => send('POST', '/api/chats/connectors', body),
+  update: (id, body) => send('PUT', `/api/chats/connectors/${enc(id)}`, body),
+  remove: (id) => send('DELETE', `/api/chats/connectors/${enc(id)}`),
+  enable: (id) => send('POST', `/api/chats/connectors/${enc(id)}/enable`, {}),
+  disable: (id) => send('POST', `/api/chats/connectors/${enc(id)}/disable`, {}),
+  test: (id) => send('POST', `/api/chats/connectors/${enc(id)}/test`, {}),
+  senders: (id) => call(`/api/chats/connectors/${enc(id)}/senders`),
+  allow: (id, sender) => send('POST', `/api/chats/connectors/${enc(id)}/senders/${enc(sender)}/allow`, {}),
+}

addons/internal/connector-ui/src/connector.css

@@ -0,0 +1,151 @@
+/* The page of a connector add-on. Flat lists and hairlines, like the chats
+   pages of the host. Controls come from the kit (dk-*); this sheet only lays
+   them out, and uses tokens only. */
+
+.cn-page {
+  flex: 1;
+  min-width: 0;
+  display: flex;
+  flex-direction: column;
+  gap: var(--dk-space-4);
+  padding: var(--dk-space-5) var(--dk-space-6);
+  overflow-y: auto;
+}
+
+.cn-head { display: flex; align-items: center; justify-content: space-between; gap: var(--dk-space-3); }
+
+.cn-title { margin: 0; font-size: var(--fs-title); line-height: var(--lh-title); font-weight: var(--font-weight-semibold); }
+
+.cn-muted { margin: 0; color: var(--color-text-muted); font-size: var(--fs-small); line-height: var(--lh-small); }
+
+.cn-list { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; }
+
+.cn-row {
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+  gap: var(--dk-space-3);
+  padding: var(--dk-space-3);
+  border-bottom: 1px solid var(--color-border-subtle);
+}
+
+.cn-row__text { display: flex; flex-direction: column; gap: 2px; min-width: 0; }
+
+.cn-row__name { font-weight: var(--font-weight-semibold); }
+
+.cn-row__actions { display: flex; align-items: center; gap: var(--dk-space-2); flex: none; }
+
+/* Fields on the left, a sticky aside on the right from 1024 px. */
+.cn-formgrid { display: grid; grid-template-columns: minmax(0, 1fr); gap: var(--dk-space-5); align-items: start; }
+
+.cn-form {
+  display: flex;
+  flex-direction: column;
+  gap: var(--dk-space-4);
+  max-width: 640px;
+  padding: var(--dk-space-5);
+  border: 1px solid var(--color-border-subtle);
+  border-radius: var(--r-lg);
+  background: var(--dk-card);
+}
+
+.cn-aside { display: flex; flex-direction: column; gap: var(--dk-space-2); }
+.cn-aside__title { margin: 0; font-size: var(--fs-small); line-height: var(--lh-small); font-weight: var(--font-weight-semibold); color: var(--color-text-primary); }
+.cn-aside__steps { margin: 0 0 var(--dk-space-3); padding-left: var(--dk-space-5); display: flex; flex-direction: column; gap: var(--dk-space-1); color: var(--color-text-muted); font-size: var(--fs-small); line-height: var(--lh-small); }
+
+@media (min-width: 1024px) {
+  .cn-formgrid { grid-template-columns: minmax(0, 640px) minmax(240px, 320px); }
+  .cn-aside { position: sticky; top: var(--dk-space-4); padding-left: var(--dk-space-5); border-left: 1px solid var(--color-border-subtle); }
+}
+
+/* Sections of the form: a quiet heading and a hairline, not a bordered box. */
+fieldset.cn-group {
+  display: flex;
+  flex-direction: column;
+  gap: var(--dk-space-3);
+  min-width: 0;
+  margin: 0;
+  padding: var(--dk-space-4) 0 0;
+  border: 0;
+  border-top: 1px solid var(--color-border-subtle);
+}
+.cn-group legend { float: left; width: 100%; margin: 0; padding: 0; font-size: var(--fs-small); line-height: var(--lh-small); font-weight: var(--font-weight-semibold); color: var(--color-text-primary); }
+.cn-group legend + * { clear: both; }
+
+.cn-actions { display: flex; gap: var(--dk-space-2); }
+
+.cn-link { background: none; border: 0; padding: 0 var(--dk-space-1); color: inherit; cursor: pointer; font: inherit; }
+.ui-callout__text .cn-link { margin-left: var(--dk-space-2); }
+.ui-callout__text .cn-notice__text { margin: 0 0 var(--dk-space-2); }
+
+@media (max-width: 900px) {
+  .cn-page { padding: var(--dk-space-4); }
+  .cn-row { flex-wrap: wrap; align-items: flex-start; }
+  .cn-row__actions { flex-wrap: wrap; }
+  .cn-form { padding: var(--dk-space-4); }
+}
+
+/* Help text: two lines, then a "More" control. */
+.cn-hint { display: flex; flex-direction: column; align-items: flex-start; gap: 2px; }
+.cn-hint__text:not(.is-open) {
+  display: -webkit-box;
+  -webkit-box-orient: vertical;
+  -webkit-line-clamp: 2;
+  line-clamp: 2;
+  overflow: hidden;
+}
+.cn-hint__more { padding: 0; border: 0; background: none; cursor: pointer; font: inherit; font-size: var(--fs-meta); color: var(--dk-accent-text); }
+.cn-hint__more:hover { text-decoration: underline; text-underline-offset: 3px; }
+
+/* Allowlists: chips and an input with an Add button, all 36 px. */
+.cn-chips { list-style: none; margin: 0; padding: 0; display: flex; flex-wrap: wrap; gap: var(--dk-space-1); }
+.cn-add { display: flex; gap: var(--dk-space-2); }
+.cn-add .dk-input { flex: 1; min-width: 0; }
+
+/* Radio cards: one real radio per card. */
+.cn-cards { display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: var(--dk-space-2); }
+.cn-card {
+  display: flex;
+  align-items: center;
+  gap: var(--dk-space-3);
+  min-height: var(--ctl-h);
+  padding: var(--dk-space-2) var(--dk-space-3);
+  border: 1px solid var(--dk-control-edge);
+  border-radius: var(--r-md);
+  background: var(--dk-card);
+  cursor: pointer;
+  font-size: var(--fs-body);
+  transition: border-color var(--dur-fast) var(--ease-out), background-color var(--dur-fast) var(--ease-out);
+}
+.cn-card:hover { border-color: color-mix(in srgb, var(--dk-control-edge) 70%, var(--dk-text)); }
+.cn-card.is-on { border-color: var(--dk-accent); background: var(--dk-accent-wash); }
+.cn-card:has(:focus-visible) { outline: 2px solid var(--dk-accent); outline-offset: 2px; }
+
+/* A switch with its label and help. */
+.cn-switchrow { display: flex; align-items: center; justify-content: space-between; gap: var(--dk-space-3); min-height: var(--ctl-h); font-size: var(--fs-body); }
+.cn-switchfield { display: flex; flex-direction: column; gap: var(--dk-space-1); }
+
+.cn-status { font-size: var(--fs-meta); padding: 0 var(--dk-space-2); border-radius: var(--r-sm); background: var(--color-bg-secondary); color: var(--color-text-secondary); }
+.cn-status--done { color: var(--color-success, var(--color-text-secondary)); }
+.cn-status--failed { color: var(--color-error, var(--color-text-secondary)); }
+.cn-status--skipped { color: var(--color-warning, var(--color-text-secondary)); }
+
+.cn-runs { margin: var(--dk-space-2) 0 0; padding: 0; list-style: none; display: flex; flex-direction: column; gap: var(--dk-space-1); font-size: var(--fs-small); }
+.cn-runs li { display: flex; flex-wrap: wrap; align-items: center; gap: var(--dk-space-2); }
+
+.cn-help { max-width: 640px; }
+.cn-help summary { cursor: pointer; color: var(--color-text-secondary); font-size: var(--fs-small); }
+.cn-help ol { margin: var(--dk-space-2) 0; padding-left: var(--dk-space-5); display: flex; flex-direction: column; gap: var(--dk-space-1); }
+.cn-code {
+  margin: var(--dk-space-2) 0 0;
+  padding: var(--dk-space-3);
+  max-height: 320px;
+  overflow: auto;
+  border-radius: var(--r-md);
+  background: var(--color-bg-secondary);
+  font-family: var(--font-mono, monospace);
+  font-size: var(--fs-meta);
+  white-space: pre;
+}
+
+.cn-answer__window .dk-input { max-width: 10rem; }

addons/internal/connector-ui/src/connector.js

@@ -0,0 +1,77 @@
+// Helpers of the connector pages of the add-ons (Telegram, Slack). No React,
+// so a node test drives them.
+
+// The modes a binding answers with. The server has the same defaults.
+export const REPLY_MODES = ['off', 'progress', 'stream']
+export const CONVERSATIONS = ['sender', 'chat', 'window', 'thread']
+export const MAX_WINDOW_MINUTES = 7 * 24 * 60
+
+export function emptyConnector() {
+  return {
+    id: '', name: '', token: '', agent_id: '', enabled: true, accept_any: false, allow: [], notify_chat: '',
+    reply_mode: 'progress', conversation: 'sender', window_minutes: 60, send_files: true,
+  }
+}
+
+// The body of a create or replace request for a binding of kind. token is the
+// secret the add-on's fields make; it goes only when it was typed: an update
+// without one keeps the stored token.
+export function connectorPayload(form, kind, token = form.token) {
+  const body = {
+    kind,
+    name: (form.name || '').trim(),
+    agent_id: form.agent_id || '',
+    enabled: Boolean(form.enabled),
+    accept_any: Boolean(form.accept_any),
+    allow: form.accept_any ? [] : (form.allow || []).map(x => x.trim()).filter((x, i, a) => x && a.indexOf(x) === i),
+    notify_chat: (form.notify_chat || '').trim(),
+    reply_mode: form.reply_mode || 'progress',
+    conversation: form.conversation || 'sender',
+    send_files: form.send_files !== false,
+  }
+  if (body.conversation === 'window') body.window_minutes = Number(form.window_minutes) || 0
+  const secret = (token || '').trim()
+  if (secret) body.token = secret
+  return body
+}
+
+// Who the binding accepts, in a few words.
+export function modeText(c) {
+  if (c.accept_any) return 'anyone'
+  const n = (c.allow || []).length
+  return n === 0 ? 'nobody yet' : `${n} listed sender${n === 1 ? '' : 's'}`
+}
+
+// A sender of the log as a short line: name, how many messages, and the id.
+export function senderLine(s) {
+  return `${s.name || s.id} · ${s.count} message${s.count === 1 ? '' : 's'} · ${s.id}`
+}
+
+// "Accept any message" is only saved after the person confirms the warning.
+// hasToken says whether the add-on's fields hold a complete secret.
+export function canSave(form, confirmed, hasToken = Boolean((form.token || '').trim())) {
+  if (!(form.name || '').trim() || !form.agent_id) return false
+  if (!form.id && !hasToken) return false
+  if (form.accept_any && !confirmed) return false
+  if (form.conversation === 'window') {
+    const m = Number(form.window_minutes)
+    if (!Number.isInteger(m) || m < 1 || m > MAX_WINDOW_MINUTES) return false
+  }
+  return true
+}
+
+// The entries of the allowlist that one of several lists shows. Each list
+// has a match function; the last list takes what no other list matches.
+export function listEntries(allow, lists, index) {
+  const own = (x) => {
+    const first = lists.findIndex((l) => l.match && l.match(x))
+    return first === -1 ? index === lists.length - 1 : first === index
+  }
+  return (allow || []).filter(own)
+}
+
+// The allowlist after list index changed to values.
+export function replaceList(allow, lists, index, values) {
+  const others = lists.flatMap((_, i) => (i === index ? [] : listEntries(allow, lists, i)))
+  return [...others, ...values]
+}

addons/internal/connector-ui/src/controls.jsx

@@ -0,0 +1,125 @@
+/* eslint-disable no-unused-vars -- components used only inside JSX look unused to this config, which has no eslint-plugin-react */
+// The form controls of the connector pages. They draw the host's kit classes
+// (dk-input, dk-select, dk-switch, dk-radio, dk-chip, dk-field), which the host
+// style sheet already loads, so a form of an add-on looks like a form of a core
+// page. The markup is plain: no browser default control is left.
+import { useId, useLayoutEffect, useRef, useState } from 'react'
+import Icon from '../../host-compat/src/components/Icon'
+
+// Label, control, hint and error of one field. The control comes from a render
+// function so that the label, the hint and the error are tied to it:
+//
+//   <Field label="Name" hint="...">{(p) => <input className="dk-input" {...p} />}</Field>
+export function Field({ label, hint, error, optional, className = '', children }) {
+  const id = useId()
+  const hintId = hint ? `${id}-hint` : undefined
+  const errorId = error ? `${id}-error` : undefined
+  const describedBy = [hintId, errorId].filter(Boolean).join(' ') || undefined
+  return (
+    <div className={`dk-field ${className}`.trim()}>
+      <label className="dk-label" htmlFor={id} data-optional={optional ? '' : undefined}>{label}</label>
+      {children({ id, 'aria-describedby': describedBy, 'aria-invalid': error ? 'true' : undefined })}
+      {hint && <Hint id={hintId}>{hint}</Hint>}
+      {error && <FieldError id={errorId}>{error}</FieldError>}
+    </div>
+  )
+}
+
+export function FieldError({ id, children, ...rest }) {
+  return (
+    <p id={id} className="dk-field-error" {...rest}>
+      <Icon name="alert-circle" />
+      <span>{children}</span>
+    </p>
+  )
+}
+
+// Help text of at most two lines. When the text is longer, a "More" control
+// opens the rest, so a long paragraph never pushes the next field down.
+export function Hint({ id, children, more = 'More', less = 'Less' }) {
+  const ref = useRef(null)
+  const [open, setOpen] = useState(false)
+  const [clipped, setClipped] = useState(false)
+  useLayoutEffect(() => {
+    const el = ref.current
+    if (el && !open) setClipped(el.scrollHeight > el.clientHeight + 1)
+  }, [children, open])
+  return (
+    <div className="cn-hint">
+      <p id={id} ref={ref} className={`dk-hint cn-hint__text${open ? ' is-open' : ''}`}>{children}</p>
+      {(clipped || open) && (
+        <button type="button" className="cn-hint__more" aria-expanded={open} onClick={() => setOpen(!open)}>
+          {open ? less : more}
+        </button>
+      )}
+    </div>
+  )
+}
+
+// A text input of the kit. `mono` is for tokens and identifiers.
+export function TextInput({ mono, className = '', ...rest }) {
+  return <input className={`dk-input${mono ? ' dk-input--mono' : ''} ${className}`.trim()} {...rest} />
+}
+
+// A select of the kit: the chevron is drawn by the wrapper, not by the browser.
+export function Select({ children, className = '', ...rest }) {
+  return (
+    <span className="dk-select-wrap">
+      <select className={`dk-select ${className}`.trim()} {...rest}>{children}</select>
+    </span>
+  )
+}
+
+// A switch for an on/off setting. `label` names it for assistive technology;
+// when `children` is set, it is drawn as the visible label next to the switch.
+export function Switch({ checked, onChange, label, children, testId, disabled }) {
+  const id = useId()
+  const button = (
+    <button type="button" role="switch" className="dk-switch" aria-checked={checked ? 'true' : 'false'}
+      aria-label={children ? undefined : label} aria-labelledby={children ? id : undefined}
+      data-testid={testId} disabled={disabled} onClick={() => onChange(!checked)} />
+  )
+  if (!children) return button
+  return (
+    <div className="cn-switchrow">
+      <span id={id}>{children}</span>
+      {button}
+    </div>
+  )
+}
+
+// One choice out of a few, as cards with a real radio inside: the arrow keys,
+// the group name and the form all work as for a native radio.
+export function RadioCards({ name, legend, value, options, onChange }) {
+  return (
+    <div className="cn-cards" role="radiogroup" aria-label={legend}>
+      {options.map((o) => (
+        <label key={String(o.value)} className={`cn-card${value === o.value ? ' is-on' : ''}`}>
+          <input type="radio" className="dk-radio" name={name} checked={value === o.value} onChange={() => onChange(o.value)} />
+          <span>{o.label}</span>
+        </label>
+      ))}
+    </div>
+  )
+}
+
+// A checkbox with its label, in the kit's style.
+export function Check({ checked, onChange, children, testId }) {
+  return (
+    <label className="dk-choice">
+      <input type="checkbox" className="dk-check" data-testid={testId} checked={checked} onChange={(e) => onChange(e.target.checked)} />
+      <span>{children}</span>
+    </label>
+  )
+}
+
+// An inline notice with a tone (the host's callout classes).
+export function Notice({ tone = 'info', role, testId, children, className = '' }) {
+  const icon = { info: 'info', ok: 'check-circle', warn: 'warning', error: 'alert-circle' }[tone] || 'info'
+  return (
+    <div className={`ui-callout ui-callout--${tone} ${className}`.trim()} role={role} data-testid={testId}>
+      <Icon name={icon} className="ui-callout__icon" />
+      <div className="ui-callout__text">{children}</div>
+    </div>
+  )
+}

addons/internal/connector/route.go

@@ -0,0 +1,271 @@
+// SPDX-License-Identifier: MIT
+
+// Package connector is the part that every connector add-on shares: the
+// dispatch of the ops of the route "/connector", the typed errors of the
+// contract, the wait that a platform asks for, and the cutting of long texts.
+// An add-on implements Platform with the API of its platform and serves the
+// route with Serve.
+package connector
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"time"
+
+	"github.com/mudler/LocalAI/pkg/addons/contract"
+)
+
+// Errors that a Platform returns. Serve maps them to the codes of the contract.
+var (
+	// ErrUnauthorized means the platform refused the token.
+	ErrUnauthorized = errors.New("the platform refused the token")
+	// ErrConflict means another process reads the updates of the same bot.
+	ErrConflict = errors.New("another process reads the updates of this bot")
+	// ErrTooLarge means a file of a message is larger than the add-on may
+	// hand over (contract.MaxConnectorFileBytes) or than the platform lets a
+	// bot download.
+	ErrTooLarge = errors.New("the file is too large")
+)
+
+// RetryAfter means the platform asks the caller to wait.
+type RetryAfter struct{ Seconds int }
+
+func (r *RetryAfter) Error() string {
+	return fmt.Sprintf("the platform asks to wait %d s", r.Seconds)
+}
+
+// Bot is the identity of the bot of a token.
+type Bot struct {
+	ID       string
+	Username string
+}
+
+// Platform is the API of one messaging platform. The host passes the token in
+// every request; a Platform keeps none.
+type Platform interface {
+	// Kind is the kind of binding that the platform serves.
+	Kind() string
+	// MaxRunes is the longest text of one message.
+	MaxRunes() int
+	// Poll returns the updates from offset on, waiting up to wait seconds.
+	Poll(ctx context.Context, token string, offset int64, wait int) ([]contract.ConnectorMessage, error)
+	// Send sends req.Text, which fits one message, to req.Chat and req.Thread
+	// and returns the ID of the message.
+	Send(ctx context.Context, req contract.ConnectorRequest) (string, error)
+	// Edit replaces the text of the message req.MessageID with req.Text.
+	Edit(ctx context.Context, req contract.ConnectorRequest) error
+	// Typing shows that the bot works in req.Chat and req.Thread.
+	Typing(ctx context.Context, req contract.ConnectorRequest) error
+	// Me returns the bot of the token, and so checks the token.
+	Me(ctx context.Context, token string) (Bot, error)
+}
+
+// Formatter is a Platform that writes Markdown in its own markup. Serve
+// formats a text before it cuts it.
+type Formatter interface {
+	Format(markdown string) string
+}
+
+// MediaReader is a Platform that reads the files of messages for the op media.
+type MediaReader interface {
+	Media(ctx context.Context, token, ref string) (contract.ConnectorFile, error)
+}
+
+// FileSender is a Platform that sends files for the op send_file. It returns
+// the ID of the message that holds the file.
+type FileSender interface {
+	SendFile(ctx context.Context, req contract.ConnectorRequest) (string, error)
+}
+
+// RichSender is a Platform that sends an answer in its own markup and falls
+// back to plain text itself when the platform refuses the markup. The op send
+// uses SendRich and EditRich for each part of req.Text, which is Markdown; the
+// op edit, which shows a preview, still uses Edit.
+type RichSender interface {
+	SendRich(ctx context.Context, req contract.ConnectorRequest) (string, error)
+	EditRich(ctx context.Context, req contract.ConnectorRequest) error
+}
+
+// Drafter is a Platform that shows a draft for the op draft: a preview of an
+// answer that is not a message yet. req.DraftID is not zero.
+type Drafter interface {
+	Draft(ctx context.Context, req contract.ConnectorRequest) error
+}
+
+// Splitter is a Platform that cuts its texts itself; others use Split.
+type Splitter interface {
+	Split(text string) []string
+}
+
+// Serve answers one call of the host. A failure of the platform is an answer
+// with a typed error, not an error of the route: the host decides what it
+// means. A request for another kind, without a token or with an unknown op is
+// refused.
+func Serve(ctx context.Context, p Platform, req contract.ConnectorRequest) (contract.ConnectorResponse, error) {
+	if req.Kind != p.Kind() || req.Token == "" {
+		return contract.ConnectorResponse{}, fmt.Errorf("%w: unknown kind or no token", contract.ErrRefused)
+	}
+	var out contract.ConnectorResponse
+	var err error
+	switch req.Op {
+	case contract.ConnectorOpPoll:
+		out, err = poll(ctx, p, req)
+	case contract.ConnectorOpSend:
+		out.MessageID, err = send(ctx, p, req)
+	case contract.ConnectorOpEdit:
+		parts := parts(p, req.Text)
+		if len(parts) == 0 {
+			return contract.ConnectorResponse{}, fmt.Errorf("%w: no text", contract.ErrRefused)
+		}
+		// An edit replaces one message: the text keeps what fits. It is not
+		// retried after a wait: an edit shows progress, and the host would
+		// rather skip one than hold the turn's answer behind it.
+		req.Text = parts[0]
+		err = p.Edit(ctx, req)
+		out.MessageID = req.MessageID
+	case contract.ConnectorOpDraft:
+		d, ok := p.(Drafter)
+		parts := parts(p, req.Text)
+		if !ok || req.DraftID == 0 || len(parts) == 0 {
+			return contract.ConnectorResponse{}, fmt.Errorf("%w: no draft, no draft ID or no text", contract.ErrRefused)
+		}
+		// Like an edit, a draft is a preview: it is not retried after a wait.
+		req.Text = parts[0]
+		err = d.Draft(ctx, req)
+	case contract.ConnectorOpTyping:
+		err = p.Typing(ctx, req)
+	case contract.ConnectorOpMe:
+		var bot Bot
+		if bot, err = p.Me(ctx, req.Token); err == nil {
+			out.BotID, out.BotUsername = bot.ID, bot.Username
+		}
+	case contract.ConnectorOpSendFile:
+		fs, ok := p.(FileSender)
+		if !ok || !validFile(req.File) {
+			return contract.ConnectorResponse{}, fmt.Errorf("%w: no file, or one this platform cannot send", contract.ErrRefused)
+		}
+		err = Retry(ctx, func() error {
+			id, err := fs.SendFile(ctx, req)
+			out.MessageID = id
+			return err
+		})
+	case contract.ConnectorOpMedia:
+		m, ok := p.(MediaReader)
+		if !ok || req.Media == "" {
+			return contract.ConnectorResponse{}, fmt.Errorf("%w: no media", contract.ErrRefused)
+		}
+		var f contract.ConnectorFile
+		if f, err = m.Media(ctx, req.Token, req.Media); err == nil {
+			out.File = &f
+		}
+	default:
+		return contract.ConnectorResponse{}, fmt.Errorf("%w: unknown op", contract.ErrRefused)
+	}
+	if err != nil {
+		return contract.ConnectorResponse{Error: Typed(err)}, nil
+	}
+	return out, nil
+}
+
+func poll(ctx context.Context, p Platform, req contract.ConnectorRequest) (contract.ConnectorResponse, error) {
+	wait := min(max(req.TimeoutSeconds, 0), contract.MaxPollSeconds)
+	ups, err := p.Poll(ctx, req.Token, req.Offset, wait)
+	if err != nil {
+		return contract.ConnectorResponse{}, err
+	}
+	next := req.Offset
+	for _, u := range ups {
+		next = max(next, u.Cursor+1)
+	}
+	return contract.ConnectorResponse{Updates: ups, NextOffset: next}, nil
+}
+
+// parts formats a text for the platform and cuts it to its limit.
+func parts(p Platform, text string) []string {
+	if f, ok := p.(Formatter); ok {
+		text = f.Format(text)
+	}
+	if s, ok := p.(Splitter); ok {
+		return s.Split(text)
+	}
+	return Split(text, p.MaxRunes())
+}
+
+// validFile tells whether the op send_file may carry f.
+func validFile(f *contract.ConnectorFile) bool {
+	return f != nil && len(f.Data) > 0 && len(f.Data) <= contract.MaxConnectorFileBytes && f.Name == contract.SafeFileName(f.Name)
+}
+
+// send sends the parts of a text in order and returns the ID of the last one.
+// When the request names a message, the first part replaces it; a message
+// that cannot be edited (deleted, too old) gets the part as a new message.
+func send(ctx context.Context, p Platform, req contract.ConnectorRequest) (string, error) {
+	sendPart, editPart := p.Send, p.Edit
+	if r, ok := p.(RichSender); ok {
+		sendPart, editPart = r.SendRich, r.EditRich
+	}
+	last := ""
+	replace := req.MessageID
+	req.MessageID = ""
+	for _, part := range parts(p, req.Text) {
+		r := req
+		r.Text = part
+		if replace != "" {
+			e := r
+			e.MessageID = replace
+			replace = ""
+			if err := Retry(ctx, func() error { return editPart(ctx, e) }); err == nil {
+				last = e.MessageID
+				continue
+			} else if ctx.Err() != nil {
+				return "", ctx.Err()
+			}
+		}
+		err := Retry(ctx, func() error {
+			id, err := sendPart(ctx, r)
+			if err == nil {
+				last = id
+			}
+			return err
+		})
+		if err != nil {
+			return "", err
+		}
+	}
+	return last, nil
+}
+
+// Retry runs call, and runs it once more after the wait the platform asks
+// for. A second wait is the caller's: the error goes back.
+func Retry(ctx context.Context, call func() error) error {
+	err := call()
+	var ra *RetryAfter
+	if !errors.As(err, &ra) {
+		return err
+	}
+	timer := time.NewTimer(time.Duration(ra.Seconds) * time.Second)
+	defer timer.Stop()
+	select {
+	case <-ctx.Done():
+		return ctx.Err()
+	case <-timer.C:
+	}
+	return call()
+}
+
+// Typed maps an error of a Platform to the typed error of the contract.
+func Typed(err error) *contract.ConnectorError {
+	var ra *RetryAfter
+	switch {
+	case errors.Is(err, ErrUnauthorized):
+		return &contract.ConnectorError{Code: contract.ConnectorUnauthorized, Message: err.Error()}
+	case errors.Is(err, ErrConflict):
+		return &contract.ConnectorError{Code: contract.ConnectorConflict, Message: err.Error()}
+	case errors.Is(err, ErrTooLarge):
+		return &contract.ConnectorError{Code: contract.ConnectorTooLarge, Message: err.Error()}
+	case errors.As(err, &ra):
+		return &contract.ConnectorError{Code: contract.ConnectorRateLimited, Message: err.Error(), RetryAfter: ra.Seconds}
+	}
+	return &contract.ConnectorError{Code: contract.ConnectorFailed, Message: err.Error()}
+}

addons/internal/connector/route_test.go

@@ -0,0 +1,290 @@
+// SPDX-License-Identifier: MIT
+package connector_test
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"strings"
+	"time"
+
+	"github.com/mudler/LocalAI/addons/internal/connector"
+	"github.com/mudler/LocalAI/pkg/addons/contract"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// script is a Platform whose answers a test sets.
+type script struct {
+	sent    []contract.ConnectorRequest
+	edited  []contract.ConnectorRequest
+	typed   int
+	ups     []contract.ConnectorMessage
+	err     error
+	sendErr []error
+	editErr error
+}
+
+// filing also sends files.
+type filing struct {
+	script
+	files []contract.ConnectorFile
+}
+
+func (f *filing) SendFile(_ context.Context, req contract.ConnectorRequest) (string, error) {
+	f.files = append(f.files, *req.File)
+	return "f1", f.err
+}
+
+func (*script) Kind() string  { return "chat" }
+func (*script) MaxRunes() int { return 10 }
+func (s *script) Poll(_ context.Context, _ string, _ int64, wait int) ([]contract.ConnectorMessage, error) {
+	if wait > contract.MaxPollSeconds {
+		return nil, errors.New("the wait was not capped")
+	}
+	return s.ups, s.err
+}
+func (s *script) Send(_ context.Context, req contract.ConnectorRequest) (string, error) {
+	if len(s.sendErr) > 0 {
+		err := s.sendErr[0]
+		s.sendErr = s.sendErr[1:]
+		if err != nil {
+			return "", err
+		}
+	}
+	s.sent = append(s.sent, req)
+	return "m" + req.Text[:1], nil
+}
+func (s *script) Edit(_ context.Context, req contract.ConnectorRequest) error {
+	s.edited = append(s.edited, req)
+	if s.editErr != nil {
+		return s.editErr
+	}
+	return s.err
+}
+func (s *script) Typing(context.Context, contract.ConnectorRequest) error { s.typed++; return s.err }
+func (s *script) Me(context.Context, string) (connector.Bot, error) {
+	return connector.Bot{ID: "1", Username: "bot"}, s.err
+}
+
+// shouting formats and reads media.
+type shouting struct{ script }
+
+func (*shouting) Format(md string) string { return strings.ToUpper(md) }
+func (*shouting) Media(_ context.Context, _, ref string) (contract.ConnectorFile, error) {
+	return contract.ConnectorFile{Name: ref, Data: []byte("x")}, nil
+}
+
+// rich sends answers in its markup and shows drafts.
+type rich struct {
+	script
+	richSent, richEdited, drafts []contract.ConnectorRequest
+}
+
+func (r *rich) SendRich(_ context.Context, req contract.ConnectorRequest) (string, error) {
+	r.richSent = append(r.richSent, req)
+	return "r" + req.Text[:1], nil
+}
+func (r *rich) EditRich(_ context.Context, req contract.ConnectorRequest) error {
+	r.richEdited = append(r.richEdited, req)
+	return nil
+}
+func (r *rich) Draft(_ context.Context, req contract.ConnectorRequest) error {
+	r.drafts = append(r.drafts, req)
+	return r.err
+}
+
+var _ = Describe("Serve", func() {
+	ctx := context.Background()
+	req := func(op string) contract.ConnectorRequest {
+		r := contract.ConnectorRequest{Op: op, Kind: "chat", Token: "t", Chat: "c", Text: "alpha beta gamma"}
+		if op == "edit" {
+			r.MessageID = "9"
+		}
+		return r
+	}
+
+	It("polls with a capped wait and names the next offset", func() {
+		s := &script{ups: []contract.ConnectorMessage{{Cursor: 4}, {Cursor: 7, Skip: true}}}
+		r := req("poll")
+		r.Offset, r.TimeoutSeconds = 2, 999
+		out, err := connector.Serve(ctx, s, r)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(out.NextOffset).To(Equal(int64(8)))
+	})
+
+	It("sends the parts of a text in order and edits with what fits", func() {
+		s := &script{}
+		out, err := connector.Serve(ctx, s, req("send"))
+		Expect(err).NotTo(HaveOccurred())
+		Expect(s.sent).To(HaveLen(2))
+		Expect(out.MessageID).To(Equal("mb"))
+		out, err = connector.Serve(ctx, s, req("edit"))
+		Expect(err).NotTo(HaveOccurred())
+		Expect(s.edited[0].Text).To(Equal("alpha"))
+		Expect(out.MessageID).To(Equal("9"))
+		r := req("edit")
+		r.Text = " "
+		_, err = connector.Serve(ctx, s, r)
+		Expect(err).To(MatchError(contract.ErrRefused))
+	})
+
+	It("formats before it cuts, and reads media only on a platform that can", func() {
+		s := &shouting{}
+		_, err := connector.Serve(ctx, s, req("send"))
+		Expect(err).NotTo(HaveOccurred())
+		Expect(s.sent[0].Text).To(Equal("ALPHA"))
+		r := req("media")
+		r.Media = "F1"
+		out, err := connector.Serve(ctx, s, r)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(out.File.Name).To(Equal("F1"))
+		_, err = connector.Serve(ctx, &script{}, r)
+		Expect(err).To(MatchError(contract.ErrRefused))
+	})
+
+	It("answers who the bot is and types", func() {
+		s := &script{}
+		out, err := connector.Serve(ctx, s, req("me"))
+		Expect(err).NotTo(HaveOccurred())
+		Expect(out.BotUsername).To(Equal("bot"))
+		_, err = connector.Serve(ctx, s, req("typing"))
+		Expect(err).NotTo(HaveOccurred())
+		Expect(s.typed).To(Equal(1))
+	})
+
+	DescribeTable("types the errors of a platform",
+		func(err error, code string) {
+			out, rerr := connector.Serve(ctx, &script{err: err}, req("me"))
+			Expect(rerr).NotTo(HaveOccurred())
+			Expect(out.Error.Code).To(Equal(code))
+		},
+		Entry("a refused token", connector.ErrUnauthorized, contract.ConnectorUnauthorized),
+		Entry("another reader", connector.ErrConflict, contract.ConnectorConflict),
+		Entry("a file too large", fmt.Errorf("telegram: %w", connector.ErrTooLarge), contract.ConnectorTooLarge),
+		Entry("a wait", &connector.RetryAfter{Seconds: 3}, contract.ConnectorRateLimited),
+		Entry("anything else", errors.New("boom"), contract.ConnectorFailed),
+	)
+
+	It("waits once when the platform asks, and gives up on a second wait or a cancel", func() {
+		s := &script{sendErr: []error{&connector.RetryAfter{Seconds: 1}, nil}}
+		r := req("send")
+		r.Text = "short"
+		start := time.Now()
+		out, err := connector.Serve(ctx, s, r)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(out.Error).To(BeNil())
+		Expect(time.Since(start)).To(BeNumerically(">=", 900*time.Millisecond))
+		s = &script{sendErr: []error{&connector.RetryAfter{Seconds: 1}, &connector.RetryAfter{Seconds: 2}}}
+		out, _ = connector.Serve(ctx, s, r)
+		Expect(out.Error.RetryAfter).To(Equal(2))
+		cctx, cancel := context.WithCancel(ctx)
+		cancel()
+		Expect(connector.Retry(cctx, func() error { return &connector.RetryAfter{Seconds: 5} })).To(MatchError(context.Canceled))
+	})
+
+	It("refuses another kind, no token and an unknown op", func() {
+		for _, r := range []contract.ConnectorRequest{{Op: "poll", Kind: "x", Token: "t"}, {Op: "poll", Kind: "chat"}, {Op: "nope", Kind: "chat", Token: "t"}} {
+			_, err := connector.Serve(ctx, &script{}, r)
+			Expect(err).To(MatchError(contract.ErrRefused))
+		}
+	})
+
+	It("replaces the working message with the first part of a send and sends the rest", func() {
+		s := &script{}
+		r := req("send")
+		r.MessageID = "9"
+		out, err := connector.Serve(ctx, s, r)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(s.edited).To(HaveLen(1))
+		Expect(s.edited[0].MessageID).To(Equal("9"))
+		Expect(s.edited[0].Text).To(Equal("alpha"))
+		Expect(s.sent).To(HaveLen(1))
+		Expect(s.sent[0].Text).To(Equal("beta gamma"))
+		Expect(s.sent[0].MessageID).To(BeEmpty())
+		Expect(out.MessageID).To(Equal("mb"))
+	})
+
+	It("sends the first part as a new message when the working message cannot be edited", func() {
+		s := &script{editErr: errors.New("message to edit not found")}
+		r := req("send")
+		r.MessageID = "9"
+		out, err := connector.Serve(ctx, s, r)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(out.Error).To(BeNil())
+		Expect(s.sent).To(HaveLen(2))
+		Expect(s.sent[0].Text).To(Equal("alpha"))
+	})
+
+	It("answers an edit that the platform rate-limits at once, without waiting", func() {
+		s := &script{editErr: &connector.RetryAfter{Seconds: 5}}
+		start := time.Now()
+		out, err := connector.Serve(ctx, s, req("edit"))
+		Expect(err).NotTo(HaveOccurred())
+		Expect(out.Error.Code).To(Equal(contract.ConnectorRateLimited))
+		Expect(time.Since(start)).To(BeNumerically("<", time.Second))
+	})
+
+	It("sends an answer through the markup of a rich platform, and a preview edit as plain text", func() {
+		p := &rich{}
+		r := req("send")
+		r.MessageID = "9"
+		out, err := connector.Serve(ctx, p, r)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(out.MessageID).To(Equal("rb"))
+		Expect(p.richEdited).To(HaveLen(1))
+		Expect(p.richEdited[0].MessageID).To(Equal("9"))
+		Expect(p.richSent).To(HaveLen(1))
+		Expect(p.sent).To(BeEmpty(), "no part of an answer goes as plain text")
+		_, err = connector.Serve(ctx, p, req("edit"))
+		Expect(err).NotTo(HaveOccurred())
+		Expect(p.edited).To(HaveLen(1), "an edit is a preview")
+		Expect(p.richEdited).To(HaveLen(1))
+	})
+
+	It("shows a draft only on a platform that can, with a draft ID and what fits", func() {
+		p := &rich{}
+		r := req("draft")
+		r.DraftID = 7
+		_, err := connector.Serve(ctx, p, r)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(p.drafts).To(HaveLen(1))
+		Expect(p.drafts[0].Text).To(Equal("alpha"))
+		Expect(p.drafts[0].DraftID).To(Equal(int64(7)))
+		_, err = connector.Serve(ctx, &script{}, r)
+		Expect(err).To(MatchError(contract.ErrRefused), "a platform without drafts")
+		r.DraftID = 0
+		_, err = connector.Serve(ctx, p, r)
+		Expect(err).To(MatchError(contract.ErrRefused), "a draft without an ID")
+		r.DraftID, p.err = 7, &connector.RetryAfter{Seconds: 5}
+		start := time.Now()
+		out, err := connector.Serve(ctx, p, r)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(out.Error.Code).To(Equal(contract.ConnectorRateLimited), "a draft is not retried after a wait")
+		Expect(time.Since(start)).To(BeNumerically("<", time.Second))
+	})
+
+	It("sends a file only on a platform that can, with a safe name and a size in bounds", func() {
+		f := &filing{}
+		r := req("send_file")
+		r.File = &contract.ConnectorFile{Name: "chart.png", MIME: "image/png", Data: []byte("png")}
+		out, err := connector.Serve(ctx, f, r)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(out.MessageID).To(Equal("f1"))
+		Expect(f.files).To(HaveLen(1))
+		Expect(f.files[0].Name).To(Equal("chart.png"))
+		_, err = connector.Serve(ctx, &script{}, r)
+		Expect(err).To(MatchError(contract.ErrRefused))
+		for _, bad := range []*contract.ConnectorFile{
+			nil,
+			{Name: "a.txt"},
+			{Name: "../a.txt", Data: []byte("x")},
+			{Name: "a.txt", Data: make([]byte, contract.MaxConnectorFileBytes+1)},
+		} {
+			r.File = bad
+			_, err = connector.Serve(ctx, f, r)
+			Expect(err).To(MatchError(contract.ErrRefused))
+		}
+		Expect(f.files).To(HaveLen(1))
+	})
+})

addons/internal/connector/split.go

@@ -0,0 +1,106 @@
+// SPDX-License-Identifier: MIT
+package connector
+
+import (
+	"strings"
+	"unicode/utf8"
+)
+
+// Split cuts text into parts of at most max characters. It cuts at a line
+// break when there is one in the last half of the part, else at a space, else at
+// max, and it never cuts a character in two.
+func Split(text string, max int) []string {
+	r := []rune(strings.TrimSpace(text))
+	if len(r) == 0 {
+		return nil
+	}
+	var out []string
+	for len(r) > max {
+		n := cut(r, max)
+		out = append(out, strings.TrimRight(string(r[:n]), " \n"))
+		r = []rune(strings.TrimLeft(string(r[n:]), " \n"))
+	}
+	if len(r) > 0 {
+		out = append(out, string(r))
+	}
+	return out
+}
+
+// cut is where Split cuts r, which is longer than max: after a line break in
+// the last half of the part, else after a space there, else at max.
+func cut(r []rune, max int) int {
+	for _, sep := range []rune{'\n', ' '} {
+		for i := max; i > max/2; i-- {
+			if r[i-1] == sep {
+				return i
+			}
+		}
+	}
+	return max
+}
+
+// fence is the line that opens and closes a code block in Markdown and in
+// the markup of the platforms that borrow it.
+const fence = "```"
+
+// SplitFenced is Split for text with code blocks. A part that ends inside a
+// code block gets the block closed, and the next part opens it again with the
+// language of the block, so each message shows its code as code. Inside a
+// block a cut keeps the indentation of the next line. Parts stay within max
+// characters.
+func SplitFenced(text string, max int) []string {
+	// Room for the fence that closes a part.
+	closing := len(fence) + 1
+	if max <= 4*closing || !strings.Contains(text, fence) {
+		return Split(text, max)
+	}
+	r := []rune(strings.TrimSpace(text))
+	var out []string
+	open, lang := false, ""
+	for len(r) > 0 {
+		head := ""
+		if open {
+			head = fence + lang + "\n"
+		}
+		limit := max - utf8.RuneCountInString(head)
+		n := len(r)
+		if n > limit {
+			n = cut(r, limit-closing)
+		}
+		part, rest := string(r[:n]), r[n:]
+		open, lang = fenceState(open, lang, part, max/4)
+		trim := " \n"
+		if open {
+			// A space at a cut inside a block is code: keep it.
+			trim = "\n"
+		}
+		part = head + strings.TrimRight(part, trim)
+		if open && len(rest) > 0 {
+			part += "\n" + fence
+		}
+		out = append(out, part)
+		r = []rune(strings.TrimLeft(string(rest), trim))
+	}
+	return out
+}
+
+// fenceState follows the fences of part from the state open, with the
+// language lang of the open block. A language longer than maxLang is not
+// carried, so the fence that opens the block again stays short.
+func fenceState(open bool, lang, part string, maxLang int) (bool, string) {
+	for line := range strings.SplitSeq(part, "\n") {
+		t := strings.TrimSpace(line)
+		if !strings.HasPrefix(t, fence) {
+			continue
+		}
+		if open {
+			open, lang = false, ""
+			continue
+		}
+		open, lang = true, strings.TrimSpace(strings.TrimPrefix(t, fence))
+		if utf8.RuneCountInString(lang) > maxLang || strings.Contains(lang, "`") {
+			lang = ""
+		}
+	}
+	return open, lang
+}

addons/internal/connector/split_test.go

@@ -0,0 +1,67 @@
+// SPDX-License-Identifier: MIT
+package connector_test
+
+import (
+	"strings"
+	"unicode/utf8"
+
+	"github.com/mudler/LocalAI/addons/internal/connector"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Split", func() {
+	It("keeps parts under the limit and whole words", func() {
+		Expect(connector.Split("  ", 10)).To(BeNil())
+		Expect(connector.Split("short", 10)).To(Equal([]string{"short"}))
+		for _, p := range connector.Split(strings.Repeat("alpha beta\n", 50), 40) {
+			Expect(len([]rune(p))).To(BeNumerically("<=", 40))
+			Expect(p).NotTo(HavePrefix("lpha"))
+			Expect(p).NotTo(HaveSuffix("alph"))
+		}
+	})
+	It("cuts on a character boundary when there is no break", func() {
+		got := connector.Split(strings.Repeat("é", 25), 10)
+		Expect(got).To(HaveLen(3))
+		Expect([]int{len([]rune(got[0])), len([]rune(got[2]))}).To(Equal([]int{10, 5}))
+	})
+})
+
+var _ = Describe("SplitFenced", func() {
+	It("closes a code block at the end of a part and opens it again in the next", func() {
+		code := "```go\n" + strings.Repeat("x := 1\n", 40) + "```\nafter"
+		parts := connector.SplitFenced(code, 100)
+		Expect(len(parts)).To(BeNumerically(">", 2))
+		for i, p := range parts {
+			Expect(len([]rune(p))).To(BeNumerically("<=", 100))
+			Expect(strings.Count(p, "```")%2).To(Equal(0), "part %d has an open block: %q", i, p)
+		}
+		Expect(parts[len(parts)-1]).To(HaveSuffix("after"))
+	})
+	It("opens a block again with its language and keeps the indentation of the code", func() {
+		code := "before\n```go\n" + strings.Repeat("\tx := 1\n", 30) + "```\nafter"
+		parts := connector.SplitFenced(code, 60)
+		Expect(len(parts)).To(BeNumerically(">", 2))
+		for i, p := range parts[1 : len(parts)-1] {
+			Expect(p).To(HavePrefix("```go\n\tx := 1"), "part %d", i+1)
+		}
+		// Taking out the fences each cut added gives back the text.
+		joined := strings.Join(parts, "\n")
+		joined = strings.ReplaceAll(joined, "\n```\n```go\n", "\n")
+		Expect(joined).To(Equal(code))
+	})
+	It("never cuts a character in two", func() {
+		code := "```\n" + strings.Repeat("🙂", 70) + "\n```"
+		parts := connector.SplitFenced(code, 20)
+		for _, p := range parts {
+			Expect(utf8.ValidString(p)).To(BeTrue())
+			Expect(utf8.RuneCountInString(p)).To(BeNumerically("<=", 20))
+		}
+		Expect(strings.Count(strings.Join(parts, ""), "🙂")).To(Equal(70))
+	})
+	It("leaves text without code as Split does, and a tiny limit to Split", func() {
+		text := strings.Repeat("word ", 60)
+		Expect(connector.SplitFenced(text, 100)).To(Equal(connector.Split(text, 100)))
+		Expect(connector.SplitFenced("```\nabc\n```", 10)).To(Equal(connector.Split("```\nabc\n```", 10)))
+	})
+})

addons/internal/connector/suite_test.go

@@ -0,0 +1,14 @@
+// SPDX-License-Identifier: MIT
+package connector_test
+
+import (
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestConnector(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "Connector add-on kit")
+}

addons/internal/jobundo/jobundo.go

@@ -0,0 +1,113 @@
+// SPDX-License-Identifier: MIT
+
+// Package jobundo is what the job add-ons (fine-tuning and quantization) share
+// of the undo of a change a chat made at once. The only change of theirs that
+// can be rolled back is the start of a job: undoing it stops the job while it
+// still runs. A job that finished cannot be un-run, and the engine has no
+// operation that removes its output, so the answer is then that it cannot be
+// undone. A stop, an export, an import and a dataset have no undo.
+package jobundo
+
+import (
+	"encoding/json"
+	"time"
+
+	"github.com/mudler/LocalAI/addons/internal/protocol"
+	"github.com/mudler/LocalAI/pkg/addons/contract"
+)
+
+// Retention is how long the start of a job can be undone.
+const Retention = 7 * 24 * time.Hour
+
+// Reasons a change keeps no change set, in the receipts of the add-ons.
+const (
+	ReasonStop    = "cannot be undone: a stopped job cannot be started again; start a new one"
+	ReasonExport  = "cannot be undone: the exported model stays; remove it from the models page"
+	ReasonImport  = "cannot be undone: the imported model stays; remove it from the models page"
+	ReasonDataset = "cannot be undone: the stored dataset stays; the host offers no way to remove it from a tool"
+)
+
+// Merge adds the undo note to a receipt payload. A payload that is not a JSON
+// object is kept under "result".
+func Merge(payload json.RawMessage, n contract.UndoNote) json.RawMessage {
+	m := map[string]any{}
+	if len(payload) > 0 && json.Unmarshal(payload, &m) != nil {
+		m = map[string]any{"result": payload}
+	}
+	m["undo"] = n
+	b, _ := json.Marshal(m)
+	return b
+}
+
+// Finished reports an engine job status that is last.
+func Finished(status string) bool {
+	return status == "completed" || status == "failed" || status == "stopped"
+}
+
+// StatusOf reads the status of an engine job answer.
+func StatusOf(raw json.RawMessage) string {
+	var v struct {
+		Status string `json:"status"`
+	}
+	_ = json.Unmarshal(raw, &v)
+	return v.Status
+}
+
+// Authority is the add-on's own check of an undo call: the action and the
+// scope go together, and the host named an actor. A caller that reaches the
+// transport with another action cannot undo anything.
+func Authority(env contract.Envelope, scope, action string) error {
+	if env.Request.Action != action || env.Authority.Scope != scope || env.Authority.Actor.UserID == "" {
+		return protocol.ErrForbidden
+	}
+	return nil
+}
+
+// Reject is the rejected receipt of an undo the add-on refuses, with a typed
+// code from the contract.
+func Reject(op, code, message string) contract.Receipt {
+	b, _ := json.Marshal(map[string]any{"error": code, "message": message})
+	return contract.Receipt{OperationID: op, State: contract.ReceiptRejected, Result: contract.Result{Payload: b}}
+}
+
+// Change is the state of the start of one job, as the add-on's journal knows
+// it. The engine's own status is read separately.
+type Change struct {
+	// Known says the journal has this change set: the start of this job.
+	Known    bool
+	Undone   bool
+	Started  time.Time
+	Revision uint64
+}
+
+// Classify says what the change set is now, without the engine: the answer is
+// final unless it is available, and then the engine status decides.
+func Classify(c Change, now time.Time) (state, detail string) {
+	switch {
+	case !c.Known:
+		return contract.ChangeUnknown, ""
+	case c.Undone:
+		return contract.ChangeUndone, ""
+	case !c.Started.IsZero() && c.Started.Add(Retention).Before(now):
+		return contract.ChangeExpired, ""
+	case c.Revision != 1:
+		return contract.ChangeFinal, "a later change was made to the job"
+	}
+	return contract.ChangeAvailable, ""
+}
+
+// CodeOf is the rejection code of a state that is not available.
+func CodeOf(state string) string {
+	switch state {
+	case contract.ChangeUndone:
+		return contract.UndoAlreadyUndone
+	case contract.ChangeExpired:
+		return contract.UndoExpired
+	case contract.ChangeFinal, contract.ChangeStale:
+		if state == contract.ChangeStale {
+			return contract.UndoStale
+		}
+		return contract.UndoCannot
+	}
+	return contract.UndoNotFound
+}

addons/internal/jobundo/undo.go

@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: MIT
+package jobundo
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"time"
+
+	"github.com/mudler/LocalAI/addons/internal/protocol"
+	"github.com/mudler/LocalAI/pkg/addons/contract"
+)
+
+// Entry is what an undo reads of a journal entry.
+type Entry struct {
+	// Known says the journal has the entry and that it stands for the start of
+	// a job (an entry for something else, such as an inline dataset, is not).
+	Known    bool
+	Name     string
+	JobID    string
+	Revision uint64
+	// StartOp is the operation that started the job: the change set a person
+	// can undo.
+	StartOp  string
+	Started  time.Time
+	UndoneBy string
+}
+
+// Undoer is the undo of one add-on, over its journal entry type J. The two
+// add-ons that start jobs differ only in the names of their engine operations,
+// the label in the summary and their journal, which Undoer takes as fields.
+type Undoer[J any] struct {
+	// Label names the kind of job in the summary: "fine-tuning".
+	Label  string
+	Schema uint64
+	// GetOp and StopOp are the engine operations that read and stop a job.
+	GetOp, StopOp string
+	// StopArgs gives the arguments of StopOp for an engine job.
+	StopArgs func(jobID string) any
+	Engine   interface {
+		Do(ctx context.Context, operationID, op string, args, out any) error
+	}
+	Now func() time.Time
+	// Lookup finds the journal entry the request names, and says whether there
+	// is one. Entry reads the undo fields of it.
+	Lookup func(env contract.Envelope) (J, bool)
+	Entry  func(j J, known bool) Entry
+	// Receipt finds the receipt of an operation that was done before.
+	Receipt func(op string) (contract.Receipt, bool)
+	// Commit stores the receipt and the job as undone by op, in one write.
+	Commit func(resourceID string, j J, undoneBy string, rec contract.Receipt) error
+	// MapErr and Payload are the add-on's own helpers.
+	MapErr  func(error) error
+	Payload func(any) json.RawMessage
+}
+
+type changeArgs struct {
+	Tool   string `json:"tool"`
+	Change string `json:"change"`
+}
+
+func (u *Undoer[J]) state(ctx context.Context, op, change string, e Entry) (contract.ChangeInfo, error) {
+	info := contract.ChangeInfo{ID: change, Class: contract.ChangeClassSchema, Current: contract.Revision{Schema: u.Schema, Data: e.Revision}}
+	c := Change{Known: e.Known && e.StartOp != "" && e.StartOp == change, Undone: e.UndoneBy != "", Started: e.Started, Revision: e.Revision}
+	info.State, info.Detail = Classify(c, u.Now())
+	if info.State != contract.ChangeAvailable {
+		return info, nil
+	}
+	info.Summary = fmt.Sprintf("Stop the %s job %s while it runs", u.Label, e.Name)
+	// Only the engine knows whether the job still runs.
+	var got json.RawMessage
+	if err := u.Engine.Do(ctx, op, u.GetOp, map[string]string{"jobId": e.JobID}, &got); err != nil {
+		var he *contract.HostError
+		if errors.As(err, &he) && he.Code == contract.HostNotFound {
+			info.State, info.Detail = contract.ChangeFinal, "the job is gone"
+			return info, nil
+		}
+		return info, u.MapErr(err)
+	}
+	if st := StatusOf(got); Finished(st) {
+		info.State, info.Detail = contract.ChangeFinal, fmt.Sprintf("the job already finished (%s)", st)
+	}
+	return info, nil
+}
+
+func parseChange(env contract.Envelope) (changeArgs, error) {
+	var a changeArgs
+	if json.Unmarshal(env.Request.Arguments, &a) != nil || !contract.ValidChangeID(a.Change) {
+		return a, protocol.ErrBadRequest
+	}
+	return a, nil
+}
+
+// Read answers what the change set of a job start says now.
+func (u *Undoer[J]) Read(ctx context.Context, env contract.Envelope) (contract.Result, error) {
+	if err := Authority(env, "read", "read"); err != nil {
+		return contract.Result{}, err
+	}
+	a, err := parseChange(env)
+	if err != nil {
+		return contract.Result{}, err
+	}
+	j, known := u.Lookup(env)
+	info, err := u.state(ctx, env.Request.OperationID, a.Change, u.Entry(j, known))
+	if err != nil {
+		return contract.Result{}, err
+	}
+	return contract.Result{Payload: u.Payload(info)}, nil
+}
+
+// Apply stops the job the change started, if it still runs.
+func (u *Undoer[J]) Apply(ctx context.Context, env contract.Envelope, rev contract.Revision) (contract.Receipt, error) {
+	if err := Authority(env, "apply", contract.UndoAction); err != nil {
+		return contract.Receipt{}, err
+	}
+	a, err := parseChange(env)
+	if err != nil {
+		return contract.Receipt{}, err
+	}
+	op := env.Request.OperationID
+	// The same operation ID is the same undo of the same change, once.
+	if rec, ok := u.Receipt(op); ok {
+		var p struct {
+			Change string `json:"change"`
+		}
+		if json.Unmarshal(rec.Result.Payload, &p) != nil || p.Change != a.Change {
+			return contract.Receipt{}, protocol.ErrConflict
+		}
+		return rec, nil
+	}
+	j, known := u.Lookup(env)
+	e := u.Entry(j, known)
+	info, err := u.state(ctx, op, a.Change, e)
+	if err != nil {
+		return contract.Receipt{}, err
+	}
+	if info.State != contract.ChangeAvailable {
+		return Reject(op, CodeOf(info.State), info.Detail), nil
+	}
+	if rev != (contract.Revision{}) && rev != info.Current {
+		return Reject(op, contract.UndoStale, "the job changed while the undo was prepared"), nil
+	}
+	var out json.RawMessage
+	if err := u.Engine.Do(ctx, op, u.StopOp, u.StopArgs(e.JobID), &out); err != nil {
+		var he *contract.HostError
+		if errors.As(err, &he) && (he.Code == contract.HostBadRequest || he.Code == contract.HostNotFound || he.Code == contract.HostConflict) {
+			// The job finished between the read and the stop.
+			return Reject(op, contract.UndoCannot, "the job already finished"), nil
+		}
+		return contract.Receipt{}, u.MapErr(err)
+	}
+	rec := contract.Receipt{OperationID: op, State: contract.ReceiptCommitted, Revision: contract.Revision{Schema: u.Schema, Data: e.Revision + 1},
+		Result: contract.Result{Payload: u.Payload(map[string]any{"kind": "undo", "change": a.Change, "stopRequested": true})}}
+	if err := u.Commit(env.Request.ResourceID, j, op, rec); err != nil {
+		return contract.Receipt{}, err
+	}
+	return rec, nil
+}

addons/internal/pdfread/pdfread.go

@@ -0,0 +1,291 @@
+// Package pdfread reads PDF files: page count, page text, whether a page has
+// text at all, and page rendering to PNG.
+//
+// It runs PDFium (github.com/klippa-app/go-pdfium) through its WebAssembly
+// backend (wazero). The package needs no cgo and cross-builds for darwin and
+// windows. The platform's production extraction uses the same backend.
+//
+// Each open document holds one PDFium instance from a shared pool. The pool
+// keeps at most four instances, so at most four documents can be open at the
+// same time. Close every document when you are done with it.
+//
+// The package never writes temporary files. Every public function turns a
+// panic into an error.
+package pdfread
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"image/png"
+	"io"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/klippa-app/go-pdfium"
+	pdfium_errors "github.com/klippa-app/go-pdfium/errors"
+	"github.com/klippa-app/go-pdfium/references"
+	"github.com/klippa-app/go-pdfium/requests"
+	"github.com/klippa-app/go-pdfium/webassembly"
+)
+
+// MaxBytes is the largest input Open and OpenWithPassword accept. It bounds
+// the memory one document can take.
+const MaxBytes = 256 << 20
+
+// Render DPI limits. Values outside the range are clamped.
+const (
+	MinDPI = 36
+	MaxDPI = 300
+)
+
+// ErrTooLarge is returned by Open when the input is over MaxBytes.
+var ErrTooLarge = errors.New("pdfread: input larger than 256 MiB")
+
+// ErrNotPDF is returned by Open when the input does not look like a PDF.
+var ErrNotPDF = errors.New("pdfread: input is not a PDF")
+
+// ErrPasswordRequired is returned when the document is encrypted and no
+// password was given.
+var ErrPasswordRequired = errors.New("pdfread: document is encrypted and needs a password")
+
+// ErrWrongPassword is returned when the document is encrypted and the given
+// password is rejected.
+var ErrWrongPassword = errors.New("pdfread: wrong password for encrypted document")
+
+// ErrClosed is returned by methods of a Doc after Close.
+var ErrClosed = errors.New("pdfread: document is closed")
+
+// ErrPageRange is returned for a page number outside 1..Pages().
+var ErrPageRange = errors.New("pdfread: page out of range")
+
+const (
+	pdfMagic     = "%PDF-"
+	magicWindow  = 1024 // the spec allows junk before the header
+	openTimeout  = 30 * time.Second
+	poolMaxTotal = 4
+)
+
+var (
+	poolOnce sync.Once
+	pool     pdfium.Pool
+	poolErr  error
+)
+
+func initPool() {
+	poolOnce.Do(func() {
+		pool, poolErr = webassembly.Init(webassembly.Config{
+			MinIdle:  0,
+			MaxIdle:  1,
+			MaxTotal: poolMaxTotal,
+			// The guest's stdout and stderr are discarded so PDFium
+			// warnings do not reach the host's terminal.
+			Stdout: io.Discard,
+			Stderr: io.Discard,
+		})
+	})
+}
+
+// Doc is an open PDF document. It is safe for concurrent use. Close it when
+// done.
+type Doc struct {
+	mu     sync.Mutex
+	inst   pdfium.Pdfium
+	handle references.FPDF_DOCUMENT
+	pages  int
+	closed bool
+}
+
+// Open parses a PDF held in data. It is OpenWithPassword with an empty
+// password, so an encrypted document returns ErrPasswordRequired.
+func Open(data []byte) (*Doc, error) {
+	return OpenWithPassword(data, "")
+}
+
+// OpenWithPassword parses a PDF held in data, decrypting it with password when
+// it is encrypted. With an empty password, an encrypted document returns
+// ErrPasswordRequired; a non-empty password that is rejected returns
+// ErrWrongPassword. A password given for an unencrypted document is ignored.
+// The bytes are copied into the PDFium instance, so the caller may reuse data
+// after the call returns.
+func OpenWithPassword(data []byte, password string) (doc *Doc, err error) {
+	defer recoverErr(&err)
+
+	if len(data) > MaxBytes {
+		return nil, ErrTooLarge
+	}
+	head := data
+	if len(head) > magicWindow {
+		head = head[:magicWindow]
+	}
+	if !bytes.Contains(head, []byte(pdfMagic)) {
+		return nil, ErrNotPDF
+	}
+
+	initPool()
+	if poolErr != nil {
+		return nil, fmt.Errorf("pdfread: pdfium init: %w", poolErr)
+	}
+
+	inst, err := pool.GetInstance(openTimeout)
+	if err != nil {
+		return nil, fmt.Errorf("pdfread: pdfium instance: %w", err)
+	}
+
+	buf := data
+	req := &requests.OpenDocument{File: &buf}
+	if password != "" {
+		req.Password = &password
+	}
+	resp, err := inst.OpenDocument(req)
+	if err != nil {
+		_ = inst.Close()
+		if errors.Is(err, pdfium_errors.ErrPassword) {
+			if password == "" {
+				return nil, ErrPasswordRequired
+			}
+			return nil, ErrWrongPassword
+		}
+		return nil, fmt.Errorf("pdfread: open: %w", err)
+	}
+
+	count, err := inst.FPDF_GetPageCount(&requests.FPDF_GetPageCount{Document: resp.Document})
+	if err != nil {
+		_, _ = inst.FPDF_CloseDocument(&requests.FPDF_CloseDocument{Document: resp.Document})
+		_ = inst.Close()
+		return nil, fmt.Errorf("pdfread: page count: %w", err)
+	}
+
+	return &Doc{
+		inst:   inst,
+		handle: resp.Document,
+		pages:  count.PageCount,
+	}, nil
+}
+
+// Pages returns the number of pages. It returns 0 after Close.
+func (d *Doc) Pages() int {
+	d.mu.Lock()
+	defer d.mu.Unlock()
+	if d.closed {
+		return 0
+	}
+	return d.pages
+}
+
+// Text returns the text of one page. Pages are numbered from 1.
+func (d *Doc) Text(page int) (text string, err error) {
+	defer recoverErr(&err)
+
+	d.mu.Lock()
+	defer d.mu.Unlock()
+	if err := d.checkPage(page); err != nil {
+		return "", err
+	}
+
+	resp, err := d.inst.GetPageText(&requests.GetPageText{
+		Page: d.pageRef(page),
+	})
+	if err != nil {
+		return "", fmt.Errorf("pdfread: text of page %d: %w", page, err)
+	}
+	return resp.Text, nil
+}
+
+// HasText reports whether a page has extractable text. A scanned page, which
+// holds only images, returns false. The caller can send such a page to vision
+// or OCR.
+func (d *Doc) HasText(page int) (ok bool, err error) {
+	text, err := d.Text(page)
+	if err != nil {
+		return false, err
+	}
+	return strings.TrimSpace(text) != "", nil
+}
+
+// Render draws one page as a PNG image. Pages are numbered from 1. The dpi is
+// clamped to MinDPI..MaxDPI.
+func (d *Doc) Render(page int, dpi int) (out []byte, err error) {
+	defer recoverErr(&err)
+
+	if dpi < MinDPI {
+		dpi = MinDPI
+	}
+	if dpi > MaxDPI {
+		dpi = MaxDPI
+	}
+
+	d.mu.Lock()
+	defer d.mu.Unlock()
+	if err := d.checkPage(page); err != nil {
+		return nil, err
+	}
+
+	resp, err := d.inst.RenderPageInDPI(&requests.RenderPageInDPI{
+		DPI:  dpi,
+		Page: d.pageRef(page),
+	})
+	if err != nil {
+		return nil, fmt.Errorf("pdfread: render page %d: %w", page, err)
+	}
+	// The image lives in WebAssembly memory until CleanupFunc runs, so encode
+	// it before cleanup.
+	defer func() {
+		if resp.CleanupFunc != nil {
+			resp.CleanupFunc()
+		}
+	}()
+	if resp.Result.Image == nil {
+		return nil, fmt.Errorf("pdfread: render page %d: no image", page)
+	}
+
+	var buf bytes.Buffer
+	if err := png.Encode(&buf, resp.Result.Image); err != nil {
+		return nil, fmt.Errorf("pdfread: encode page %d: %w", page, err)
+	}
+	return buf.Bytes(), nil
+}
+
+// Close releases the document and its PDFium instance. Calling Close more
+// than once is safe.
+func (d *Doc) Close() (err error) {
+	defer recoverErr(&err)
+
+	d.mu.Lock()
+	defer d.mu.Unlock()
+	if d.closed {
+		return nil
+	}
+	d.closed = true
+	_, _ = d.inst.FPDF_CloseDocument(&requests.FPDF_CloseDocument{Document: d.handle})
+	return d.inst.Close()
+}
+
+// checkPage requires d.mu to be held.
+func (d *Doc) checkPage(page int) error {
+	if d.closed {
+		return ErrClosed
+	}
+	if page < 1 || page > d.pages {
+		return fmt.Errorf("%w: %d of %d", ErrPageRange, page, d.pages)
+	}
+	return nil
+}
+
+// pageRef converts a 1-based page number to PDFium's 0-based index.
+func (d *Doc) pageRef(page int) requests.Page {
+	return requests.Page{
+		ByIndex: &requests.PageByIndex{
+			Document: d.handle,
+			Index:    page - 1,
+		},
+	}
+}
+
+// recoverErr turns a panic into an error on the named return value.
+func recoverErr(err *error) {
+	if r := recover(); r != nil {
+		*err = fmt.Errorf("pdfread: internal error: %v", r)
+	}
+}

addons/internal/pdfread/pdfread_suite_test.go

@@ -1,4 +1,4 @@
-package agentpool_test
+package pdfread
 
 import (
 	"testing"
@@ -7,7 +7,7 @@ import (
 	. "github.com/onsi/gomega"
 )
 
-func TestServices(t *testing.T) {
+func TestPDFRead(t *testing.T) {
 	RegisterFailHandler(Fail)
-	RunSpecs(t, "LocalAI services test")
+	RunSpecs(t, "PDF Read Suite")
 }

addons/internal/pdfread/pdfread_test.go

@@ -0,0 +1,332 @@
+package pdfread
+
+import (
+	"bytes"
+	"crypto/md5"
+	"crypto/rc4"
+	"errors"
+	"fmt"
+	"image/png"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// assemble writes a PDF from the bodies of objects 1..n. trailer holds the
+// trailer entries after /Size, for example "/Root 1 0 R".
+func assemble(objs [][]byte, trailer string) []byte {
+	var buf bytes.Buffer
+	buf.WriteString("%PDF-1.4\n%\xe2\xe3\xcf\xd3\n")
+	offsets := make([]int, len(objs))
+	for i, obj := range objs {
+		offsets[i] = buf.Len()
+		fmt.Fprintf(&buf, "%d 0 obj\n", i+1)
+		buf.Write(obj)
+		buf.WriteString("\nendobj\n")
+	}
+
+	xref := buf.Len()
+	fmt.Fprintf(&buf, "xref\n0 %d\n0000000000 65535 f \n", len(objs)+1)
+	for _, off := range offsets {
+		fmt.Fprintf(&buf, "%010d 00000 n \n", off)
+	}
+	fmt.Fprintf(&buf, "trailer\n<< /Size %d %s >>\nstartxref\n%d\n%%%%EOF\n",
+		len(objs)+1, trailer, xref)
+	return buf.Bytes()
+}
+
+// pageObjects returns objects 1 to 4: the catalog, the page tree, a 300 by
+// 144 point page whose content is object 5, and the Helvetica font as /F1.
+func pageObjects() [][]byte {
+	return [][]byte{
+		[]byte("<< /Type /Catalog /Pages 2 0 R >>"),
+		[]byte("<< /Type /Pages /Kids [3 0 R] /Count 1 >>"),
+		[]byte("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 144] " +
+			"/Resources << /Font << /F1 4 0 R >> >> /Contents 5 0 R >>"),
+		[]byte("<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>"),
+	}
+}
+
+// streamObject is a stream object holding data as it is written.
+func streamObject(data []byte) []byte {
+	var b bytes.Buffer
+	fmt.Fprintf(&b, "<< /Length %d >>\nstream\n", len(data))
+	b.Write(data)
+	b.WriteString("\nendstream")
+	return b.Bytes()
+}
+
+// buildPDF returns a one-page PDF whose page content stream is content. It is
+// built by hand so the tests need no PDF writer.
+func buildPDF(content string) []byte {
+	objs := append(pageObjects(), streamObject([]byte(content)))
+	return assemble(objs, "/Root 1 0 R")
+}
+
+// rc4XOR encrypts or decrypts src with RC4 under key.
+func rc4XOR(key, src []byte) []byte {
+	c, err := rc4.NewCipher(key)
+	if err != nil {
+		panic(err)
+	}
+	out := make([]byte, len(src))
+	c.XORKeyStream(out, src)
+	return out
+}
+
+// pdfPadding is the password padding string of PDF 1.7, section 7.6.3.3.
+const pdfPadding = "\x28\xbf\x4e\x5e\x4e\x75\x8a\x41\x64\x00\x4e\x56\xff\xfa\x01\x08" +
+	"\x2e\x2e\x00\xb6\xd0\x68\x3e\x80\x2f\x0c\xa9\xfe\x64\x53\x69\x7a"
+
+func padPassword(password string) []byte {
+	out := make([]byte, 32)
+	n := copy(out, password)
+	copy(out[n:], pdfPadding)
+	return out
+}
+
+// encryptedPDF returns the page of helloPDF encrypted with the RC4 40-bit
+// standard security handler (revision 2). The user password is userPW and the
+// owner password is ownerPW. Object 5, the content stream, is encrypted. The
+// key derivation follows PDF 1.7, algorithms 1 to 4.
+func encryptedPDF(content, userPW, ownerPW string) []byte {
+	id := []byte("0123456789abcdef")
+
+	// Algorithm 3: the owner key encrypts the user password into /O.
+	ownerSum := md5.Sum(padPassword(ownerPW))
+	o := rc4XOR(ownerSum[:5], padPassword(userPW))
+
+	// Algorithm 2: the file key comes from the user password, /O, /P and the
+	// first file identifier.
+	h := md5.New()
+	h.Write(padPassword(userPW))
+	h.Write(o)
+	h.Write([]byte{0xfc, 0xff, 0xff, 0xff}) // /P -4, all permissions
+	h.Write(id)
+	fileKey := h.Sum(nil)[:5]
+
+	// Algorithm 4: /U is the padding string encrypted with the file key.
+	u := rc4XOR(fileKey, []byte(pdfPadding))
+
+	// Algorithm 1: each object is encrypted with its own key.
+	num := 5
+	oh := md5.New()
+	oh.Write(fileKey)
+	oh.Write([]byte{byte(num), byte(num >> 8), byte(num >> 16), 0, 0})
+	objKey := oh.Sum(nil)[:10]
+	encrypted := rc4XOR(objKey, []byte(content))
+
+	objs := append(pageObjects(), streamObject(encrypted))
+	objs = append(objs, []byte(fmt.Sprintf(
+		"<< /Filter /Standard /V 1 /R 2 /Length 40 /O <%x> /U <%x> /P -4 >>", o, u)))
+	return assemble(objs, fmt.Sprintf("/Root 1 0 R /Encrypt 6 0 R /ID [<%x> <%x>]", id, id))
+}
+
+func helloPDF() []byte {
+	return buildPDF("BT /F1 24 Tf 20 60 Td (Hello PDF) Tj ET")
+}
+
+func blankPDF() []byte {
+	return buildPDF("")
+}
+
+func mustOpen(data []byte) *Doc {
+	GinkgoHelper()
+	doc, err := Open(data)
+	Expect(err).ToNot(HaveOccurred(), "Open")
+	DeferCleanup(func() { _ = doc.Close() })
+	return doc
+}
+
+const (
+	testUserPassword  = "secret"
+	testOwnerPassword = "owner-secret"
+)
+
+const helloContent = "BT /F1 24 Tf 20 60 Td (Hello PDF) Tj ET"
+
+var _ = Describe("pdfread", func() {
+	It("opens a document and reads its pages and text", func() {
+		doc := mustOpen(helloPDF())
+
+		Expect(doc.Pages()).To(Equal(1))
+		text, err := doc.Text(1)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(text).To(ContainSubstring("Hello PDF"))
+	})
+
+	It("reports whether a page has text", func() {
+		doc := mustOpen(helloPDF())
+		ok, err := doc.HasText(1)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(ok).To(BeTrue(), "HasText(1) on a page with text")
+
+		blank := mustOpen(blankPDF())
+		ok, err = blank.HasText(1)
+		Expect(err).ToNot(HaveOccurred(), "HasText(1) on blank page")
+		Expect(ok).To(BeFalse(), "HasText(1) on a page with no text")
+	})
+
+	It("renders a page as a PNG", func() {
+		doc := mustOpen(helloPDF())
+
+		// At 72 dpi a 300 by 144 point page is 300 by 144 pixels.
+		data, err := doc.Render(1, 72)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(len(data)).To(BeNumerically(">=", 8))
+		Expect(data[:8]).To(Equal([]byte("\x89PNG\r\n\x1a\n")), "Render output is not a PNG")
+		cfg, err := png.DecodeConfig(bytes.NewReader(data))
+		Expect(err).ToNot(HaveOccurred(), "decode PNG")
+		Expect(cfg.Width).To(Equal(300))
+		Expect(cfg.Height).To(Equal(144))
+		Expect(len(data)).To(BeNumerically(">=", 200), "PNG too small for a rendered page")
+	})
+
+	It("clamps the render DPI", func() {
+		doc := mustOpen(helloPDF())
+
+		cases := []struct {
+			dpi       int
+			wantWidth int
+		}{
+			{dpi: 1, wantWidth: 150},     // clamped to 36 dpi: 300 * 36 / 72
+			{dpi: 1000, wantWidth: 1250}, // clamped to 300 dpi: 300 * 300 / 72
+		}
+		for _, c := range cases {
+			data, err := doc.Render(1, c.dpi)
+			Expect(err).ToNot(HaveOccurred(), "Render(1, %d)", c.dpi)
+			cfg, err := png.DecodeConfig(bytes.NewReader(data))
+			Expect(err).ToNot(HaveOccurred(), "decode PNG at dpi %d", c.dpi)
+			Expect(cfg.Width).To(Equal(c.wantWidth), "dpi %d", c.dpi)
+		}
+	})
+
+	It("returns ErrPageRange for pages out of range", func() {
+		doc := mustOpen(helloPDF())
+
+		for _, page := range []int{0, -1, 2} {
+			_, err := doc.Text(page)
+			Expect(err).To(MatchError(ErrPageRange), "Text(%d)", page)
+			_, err = doc.HasText(page)
+			Expect(err).To(MatchError(ErrPageRange), "HasText(%d)", page)
+			_, err = doc.Render(page, 72)
+			Expect(err).To(MatchError(ErrPageRange), "Render(%d)", page)
+		}
+	})
+
+	It("rejects input that is not a PDF", func() {
+		cases := map[string][]byte{
+			"nil":    nil,
+			"empty":  {},
+			"text":   []byte("this is not a pdf file at all"),
+			"binary": bytes.Repeat([]byte{0x00, 0xff, 0x7f}, 100),
+		}
+		for name, data := range cases {
+			doc, err := Open(data)
+			if doc != nil {
+				_ = doc.Close()
+			}
+			Expect(err).To(MatchError(ErrNotPDF), "%s", name)
+		}
+	})
+
+	It("rejects input that is too large", func() {
+		Expect(MaxBytes).To(Equal(256<<20), "MaxBytes is 256 MiB")
+		data := make([]byte, MaxBytes+1)
+		copy(data, pdfMagic)
+		_, err := Open(data)
+		Expect(err).To(MatchError(ErrTooLarge), "Open(256 MiB + 1)")
+		_, err = OpenWithPassword(data, "secret")
+		Expect(err).To(MatchError(ErrTooLarge), "OpenWithPassword(256 MiB + 1)")
+	})
+
+	It("opens an encrypted document with the right password", func() {
+		data := encryptedPDF(helloContent, testUserPassword, testOwnerPassword)
+		doc, err := OpenWithPassword(data, testUserPassword)
+		Expect(err).ToNot(HaveOccurred(), "OpenWithPassword(encrypted, right password)")
+		DeferCleanup(func() { _ = doc.Close() })
+
+		Expect(doc.Pages()).To(Equal(1))
+		text, err := doc.Text(1)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(text).To(ContainSubstring("Hello PDF"))
+	})
+
+	It("requires a password for an encrypted document", func() {
+		data := encryptedPDF(helloContent, testUserPassword, testOwnerPassword)
+		for name, open := range map[string]func() (*Doc, error){
+			"Open":                   func() (*Doc, error) { return Open(data) },
+			"OpenWithPassword(\"\")": func() (*Doc, error) { return OpenWithPassword(data, "") },
+		} {
+			doc, err := open()
+			if doc != nil {
+				_ = doc.Close()
+			}
+			Expect(doc).To(BeNil(), "%s: got a document for an encrypted file without a password", name)
+			Expect(err).To(MatchError(ErrPasswordRequired), "%s", name)
+		}
+	})
+
+	It("rejects a wrong password", func() {
+		data := encryptedPDF(helloContent, testUserPassword, testOwnerPassword)
+		doc, err := OpenWithPassword(data, "not-the-password")
+		if doc != nil {
+			_ = doc.Close()
+		}
+		Expect(doc).To(BeNil(), "got a document for a wrong password")
+		Expect(err).To(MatchError(ErrWrongPassword))
+		Expect(errors.Is(err, ErrPasswordRequired)).To(BeFalse(), "a wrong password must not report ErrPasswordRequired")
+	})
+
+	It("opens a plain document with any password", func() {
+		doc, err := OpenWithPassword(helloPDF(), "")
+		Expect(err).ToNot(HaveOccurred(), "OpenWithPassword(plain, empty)")
+		DeferCleanup(func() { _ = doc.Close() })
+		Expect(doc.Pages()).To(Equal(1))
+
+		doc2, err := OpenWithPassword(helloPDF(), "unused")
+		Expect(err).ToNot(HaveOccurred(), "OpenWithPassword(plain, password)")
+		DeferCleanup(func() { _ = doc2.Close() })
+		text, err := doc2.Text(1)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(text).To(ContainSubstring("Hello PDF"))
+	})
+
+	It("returns ErrClosed after Close", func() {
+		doc, err := Open(helloPDF())
+		Expect(err).ToNot(HaveOccurred())
+		Expect(doc.Close()).To(Succeed())
+		Expect(doc.Close()).To(Succeed(), "second Close")
+		_, err = doc.Text(1)
+		Expect(err).To(MatchError(ErrClosed), "Text after Close")
+		_, err = doc.Render(1, 72)
+		Expect(err).To(MatchError(ErrClosed), "Render after Close")
+		Expect(doc.Pages()).To(Equal(0), "Pages after Close")
+	})
+
+	// Opens every prefix of a valid PDF. Truncated input must produce an error
+	// or a usable document, never a panic.
+	It("does not panic on truncated prefixes", func() {
+		good := helloPDF()
+		for i := 0; i <= len(good); i++ {
+			doc, err := Open(good[:i])
+			if err != nil {
+				continue
+			}
+			_, _ = doc.Text(1)
+			_, _ = doc.Render(1, 36)
+			_ = doc.Close()
+		}
+	})
+
+	It("does not panic on garbage after the header", func() {
+		junk := append([]byte("%PDF-1.7\n"), bytes.Repeat([]byte("0 obj << /Type garbage >> endobj\x00\xff"), 50)...)
+		doc, err := Open(junk)
+		if err != nil {
+			return
+		}
+		defer func() { _ = doc.Close() }()
+		_, _ = doc.Text(1)
+		_, _ = doc.Render(1, 72)
+	})
+})

addons/internal/protocol/act_test.go

@@ -0,0 +1,101 @@
+// SPDX-License-Identifier: MIT
+package protocol_test
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"time"
+
+	"github.com/mudler/LocalAI/addons/internal/idle"
+	. "github.com/mudler/LocalAI/addons/internal/protocol"
+	"github.com/mudler/LocalAI/pkg/addons/contract"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Handler and action tools", func() {
+	startup := func() Startup {
+		return Startup{
+			Registration: Registration{AddonID: "sandbox", Generation: 1, Incarnation: "inc", Protocol: 1, Capabilities: contract.Capabilities{Actions: true}},
+			Credential:   "secret", Limits: contract.DefaultLimits(),
+		}
+	}
+	post := func(h http.Handler, path, scope, action string) *httptest.ResponseRecorder {
+		env := contract.Envelope{
+			Request: contract.Request{AddonID: "sandbox", OperationID: "op1", Generation: 1, Action: action, Arguments: json.RawMessage(`{"command":"echo"}`)},
+			Authority: contract.Authority{Actor: contract.Actor{UserID: "alice", CredentialID: "k", Mode: "named-user"}, Scope: scope,
+				ExpiresAt: time.Now().Add(time.Minute)},
+		}
+		b, _ := json.Marshal(env)
+		req := httptest.NewRequest(http.MethodPost, path, bytes.NewReader(b))
+		req.Header.Set("Authorization", "Bearer secret")
+		req.Header.Set("X-LocalAI-Incarnation", "inc")
+		req.Header.Set("Content-Type", "application/json")
+		rec := httptest.NewRecorder()
+		h.ServeHTTP(rec, req)
+		return rec
+	}
+
+	It("serves /act for a runtime that implements ActRuntime, under the act scope only", func() {
+		h := Handler(startup(), actingRuntime{})
+		rec := post(h, "/act", "act", "act")
+		Expect(rec.Code).To(Equal(http.StatusOK), rec.Body.String())
+		Expect(rec.Body.String()).To(ContainSubstring("alice"))
+		Expect(post(h, "/act", "read", "act").Code).To(BeNumerically(">=", 400))
+		Expect(post(h, "/act", "act", "read").Code).To(BeNumerically(">=", 400))
+	})
+	It("has no /act for a runtime that does not act", func() {
+		h := Handler(startup(), idle.Runtime{})
+		Expect(post(h, "/act", "act", "act").Code).To(Equal(http.StatusNotFound))
+	})
+})
+
+var _ = Describe("Handler and refusals", func() {
+	read := func(err error) *httptest.ResponseRecorder {
+		st := Startup{Registration: Registration{AddonID: "sandbox", Generation: 1, Incarnation: "inc", Protocol: 1}, Credential: "secret", Limits: contract.DefaultLimits()}
+		env := contract.Envelope{
+			Request:   contract.Request{AddonID: "sandbox", OperationID: "op1", Generation: 1, Action: "read", Arguments: json.RawMessage(`{}`)},
+			Authority: contract.Authority{Actor: contract.Actor{UserID: "alice", CredentialID: "k", Mode: "named-user"}, Scope: "read", ExpiresAt: time.Now().Add(time.Minute)},
+		}
+		b, _ := json.Marshal(env)
+		req := httptest.NewRequest(http.MethodPost, "/read", bytes.NewReader(b))
+		req.Header.Set("Authorization", "Bearer secret")
+		req.Header.Set("X-LocalAI-Incarnation", "inc")
+		req.Header.Set("Content-Type", "application/json")
+		rec := httptest.NewRecorder()
+		Handler(st, failingRuntime{err: err}).ServeHTTP(rec, req)
+		return rec
+	}
+	It("sends the reason of a bad request on, so the owner can tell the person what to fix", func() {
+		rec := read(fmt.Errorf("%w: %w: quantization engine llama-cpp-quantization is not installed", ErrBadRequest, ErrBadRequest))
+		Expect(rec.Code).To(Equal(http.StatusBadRequest))
+		Expect(rec.Body.String()).To(ContainSubstring("quantization engine llama-cpp-quantization is not installed"))
+	})
+	It("keeps the plain answer for a bad request without a reason", func() {
+		rec := read(ErrBadRequest)
+		Expect(rec.Code).To(Equal(http.StatusBadRequest))
+		Expect(strings.TrimSpace(rec.Body.String())).To(Equal("invalid request"))
+	})
+})
+
+type failingRuntime struct {
+	idle.Runtime
+	err error
+}
+
+func (f failingRuntime) Read(context.Context, contract.Envelope) (contract.Result, error) {
+	return contract.Result{}, f.err
+}
+
+// actingRuntime answers an act call with the user it ran for.
+type actingRuntime struct{ idle.Runtime }
+
+func (actingRuntime) Act(_ context.Context, e contract.Envelope) (contract.Result, error) {
+	b, _ := json.Marshal(map[string]string{"ranFor": e.Authority.Actor.UserID})
+	return contract.Result{Payload: b}, nil
+}

addons/internal/protocol/args.go

@@ -0,0 +1,66 @@
+// SPDX-License-Identifier: MIT
+package protocol
+
+import (
+	"bytes"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+
+	"github.com/mudler/LocalAI/pkg/addons/contract"
+)
+
+// Payload encodes v as the payload of a result. It is for values the add-on
+// builds itself, which always encode, so it drops the error.
+func Payload(v any) json.RawMessage { b, _ := json.Marshal(v); return b }
+
+// ToolOf returns the tool the host named in the arguments of the request. A
+// request with no tool is a bad request.
+func ToolOf(env contract.Envelope) (string, error) {
+	var a struct {
+		Tool string `json:"tool"`
+	}
+	if json.Unmarshal(env.Request.Arguments, &a) != nil || a.Tool == "" {
+		return "", ErrBadRequest
+	}
+	return a.Tool, nil
+}
+
+// DecodeArgs reads the arguments into a closed struct: an unknown field or
+// trailing data is ErrBadRequest. No arguments decode as an empty object.
+func DecodeArgs(raw json.RawMessage, v any) error {
+	if len(raw) == 0 {
+		raw = json.RawMessage(`{}`)
+	}
+	dec := json.NewDecoder(bytes.NewReader(raw))
+	dec.DisallowUnknownFields()
+	if err := dec.Decode(v); err != nil {
+		return ErrBadRequest
+	}
+	if _, err := dec.Token(); err != io.EOF {
+		return ErrBadRequest
+	}
+	return nil
+}
+
+// FromHostError turns the refusal of a host operation into the protocol error
+// that tells the owner what happened. A failure that does not say the request
+// was wrong stays as it is, and the owner reads it as uncertain.
+func FromHostError(err error) error {
+	var he *contract.HostError
+	if !errors.As(err, &he) {
+		return err
+	}
+	switch he.Code {
+	case contract.HostBadRequest, contract.HostTooLarge:
+		return fmt.Errorf("%w: %s", ErrBadRequest, he.Message)
+	case contract.HostNotFound:
+		return fmt.Errorf("%w: %s", ErrNotFound, he.Message)
+	case contract.HostConflict:
+		return fmt.Errorf("%w: %s", ErrConflict, he.Message)
+	case contract.HostFeature, contract.HostScope:
+		return fmt.Errorf("%w: %s", ErrForbidden, he.Message)
+	}
+	return err
+}

addons/internal/protocol/args_test.go

@@ -0,0 +1,85 @@
+// SPDX-License-Identifier: MIT
+package protocol_test
+
+import (
+	"encoding/json"
+	"errors"
+	"fmt"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+
+	. "github.com/mudler/LocalAI/addons/internal/protocol"
+	"github.com/mudler/LocalAI/pkg/addons/contract"
+)
+
+var _ = Describe("tool argument helpers", func() {
+	envWith := func(args string) contract.Envelope {
+		var env contract.Envelope
+		env.Request.Arguments = json.RawMessage(args)
+		return env
+	}
+
+	It("encodes a payload", func() {
+		Expect(string(Payload(map[string]any{"ok": true}))).To(Equal(`{"ok":true}`))
+	})
+
+	It("reads the tool of a request", func() {
+		tool, err := ToolOf(envWith(`{"tool":"x-list","name":"a"}`))
+		Expect(err).NotTo(HaveOccurred())
+		Expect(tool).To(Equal("x-list"))
+	})
+
+	DescribeTable("refuses a request with no tool",
+		func(args string) {
+			_, err := ToolOf(envWith(args))
+			Expect(err).To(MatchError(ErrBadRequest))
+		},
+		Entry("empty tool", `{"tool":""}`),
+		Entry("no tool", `{}`),
+		Entry("not an object", `[]`),
+		Entry("no arguments", ``),
+	)
+
+	It("decodes arguments into a closed struct", func() {
+		var a struct {
+			Name string `json:"name"`
+		}
+		Expect(DecodeArgs(json.RawMessage(`{"name":"a"}`), &a)).To(Succeed())
+		Expect(a.Name).To(Equal("a"))
+		Expect(DecodeArgs(nil, &a)).To(Succeed())
+	})
+
+	DescribeTable("refuses arguments that do not match",
+		func(args string) {
+			var a struct {
+				Name string `json:"name"`
+			}
+			Expect(DecodeArgs(json.RawMessage(args), &a)).To(MatchError(ErrBadRequest))
+		},
+		Entry("unknown field", `{"name":"a","other":1}`),
+		Entry("trailing value", `{"name":"a"} {}`),
+		Entry("wrong type", `{"name":1}`),
+	)
+
+	DescribeTable("maps a host refusal to a protocol error",
+		func(code string, want error) {
+			err := FromHostError(fmt.Errorf("op: %w", &contract.HostError{Code: code, Message: "why"}))
+			Expect(err).To(MatchError(want))
+			Expect(err.Error()).To(HaveSuffix(": why"))
+		},
+		Entry("bad request", contract.HostBadRequest, ErrBadRequest),
+		Entry("too large", contract.HostTooLarge, ErrBadRequest),
+		Entry("not found", contract.HostNotFound, ErrNotFound),
+		Entry("conflict", contract.HostConflict, ErrConflict),
+		Entry("feature", contract.HostFeature, ErrForbidden),
+		Entry("scope", contract.HostScope, ErrForbidden),
+	)
+
+	It("keeps any other failure as it is", func() {
+		other := errors.New("boom")
+		Expect(FromHostError(other)).To(BeIdenticalTo(other))
+		unavailable := &contract.HostError{Code: contract.HostUnavailable, Message: "later"}
+		Expect(FromHostError(unavailable)).To(BeIdenticalTo(error(unavailable)))
+	})
+})

addons/internal/protocol/connector_test.go

@@ -0,0 +1,57 @@
+// SPDX-License-Identifier: MIT
+package protocol_test
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+
+	"github.com/mudler/LocalAI/addons/internal/idle"
+	. "github.com/mudler/LocalAI/addons/internal/protocol"
+	"github.com/mudler/LocalAI/pkg/addons/contract"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("Handler and the connector route", func() {
+	startup := Startup{
+		Registration: Registration{AddonID: "telegram", Generation: 1, Incarnation: "inc", Protocol: 1},
+		Credential:   "secret", Limits: contract.DefaultLimits(),
+	}
+	post := func(h http.Handler, bearer string, body any) *httptest.ResponseRecorder {
+		b, _ := json.Marshal(body)
+		req := httptest.NewRequest(http.MethodPost, "/connector", bytes.NewReader(b))
+		req.Header.Set("Authorization", "Bearer "+bearer)
+		req.Header.Set("X-LocalAI-Incarnation", "inc")
+		req.Header.Set("Content-Type", "application/json")
+		rec := httptest.NewRecorder()
+		h.ServeHTTP(rec, req)
+		return rec
+	}
+
+	It("serves /connector for a runtime that implements ConnectorRuntime, behind the incarnation credential", func() {
+		h := Handler(startup, echoConnector{})
+		rec := post(h, "secret", contract.ConnectorRequest{Op: "me", Kind: "telegram", Token: "t"})
+		Expect(rec.Code).To(Equal(http.StatusOK), rec.Body.String())
+		Expect(rec.Body.String()).To(ContainSubstring(`"botUsername":"me"`))
+		Expect(post(h, "wrong", contract.ConnectorRequest{Op: "me"}).Code).To(Equal(http.StatusUnauthorized))
+	})
+	It("answers a refusal of the add-on as a bad request", func() {
+		rec := post(Handler(startup, echoConnector{}), "secret", contract.ConnectorRequest{Op: "nope", Kind: "telegram", Token: "t"})
+		Expect(rec.Code).To(Equal(http.StatusBadRequest))
+	})
+	It("has no /connector for a runtime that carries no connector", func() {
+		Expect(post(Handler(startup, idle.Runtime{}), "secret", contract.ConnectorRequest{Op: "me"}).Code).To(Equal(http.StatusNotFound))
+	})
+})
+
+type echoConnector struct{ idle.Runtime }
+
+func (echoConnector) Connector(_ context.Context, r contract.ConnectorRequest) (contract.ConnectorResponse, error) {
+	if r.Op != "me" {
+		return contract.ConnectorResponse{}, contract.ErrRefused
+	}
+	return contract.ConnectorResponse{BotUsername: "me"}, nil
+}

addons/internal/protocol/protocol.go

@@ -15,9 +15,15 @@ import (
 	"net"
 	"net/http"
 	"os"
+	"os/signal"
 	"path/filepath"
+	"strings"
+	"sync"
+	"syscall"
 	"time"
 
+	"github.com/mudler/xlog"
+
 	"github.com/mudler/LocalAI/pkg/addons/contract"
 )
 
@@ -63,6 +69,26 @@ var (
 	ErrForbidden  = errors.New("forbidden")
 )
 
+// badRequestText is the body of a 400 answer: the reason a runtime put after
+// ErrBadRequest ("invalid request: <reason>"), or the plain class. The owner
+// keeps a short printable part of it, so a runtime writes a fixed sentence
+// here and never an echo of the request.
+func badRequestText(err error) string {
+	// A host refusal carries the prefix too, so a runtime that wraps it adds a second.
+	reason, stripped := err.Error(), false
+	for {
+		rest, ok := strings.CutPrefix(reason, ErrBadRequest.Error()+": ")
+		if !ok {
+			break
+		}
+		reason, stripped = rest, true
+	}
+	if !stripped || reason == "" {
+		return ErrBadRequest.Error()
+	}
+	return reason
+}
+
 // ReadStartup decodes the startup object from the inherited descriptor and
 // returns whatever the decoder buffered beyond it, for the parent-death watch.
 func ReadStartup(parent io.Reader) (Startup, io.Reader, error) {
@@ -74,9 +100,62 @@ func ReadStartup(parent io.Reader) (Startup, io.Reader, error) {
 	return s, io.MultiReader(d.Buffered(), parent), nil
 }
 
-// ExitWithParent ends the process when the owner closes the startup channel.
-func ExitWithParent(rest io.Reader) {
-	go func() { _, _ = io.Copy(io.Discard, rest); os.Exit(0) }()
+// StopWithParent returns a context that ends when the owner closes the startup
+// channel (which also happens when the owner dies) or the process gets SIGTERM
+// or SIGINT. An executable that holds resources serves with it and returns from
+// Run when it ends, so its deferred cleanup runs; os.Exit would skip it. The
+// caller calls the cancel function when it is done.
+func StopWithParent(parent context.Context, rest io.Reader) (context.Context, context.CancelFunc) {
+	ctx, stop := signal.NotifyContext(parent, syscall.SIGTERM, syscall.SIGINT)
+	exitOnParentLoss(rest, stop)
+	return ctx, stop
+}
+
+// exitOnParentLoss calls stop when the owner closes the startup channel, which
+// also happens when the owner dies.
+func exitOnParentLoss(rest io.Reader, stop func()) {
+	go func() { _, _ = io.Copy(io.Discard, rest); stop() }()
+}
+
+// Closer is implemented by a runtime that owns resources outside its process,
+// such as sandboxes that a helper started, and must release them before the
+// add-on exits. Run calls it when the owner closes the startup channel or the
+// process gets SIGTERM or SIGINT. The context carries the shutdown budget.
+// The owner kills a process that outlives it, and SIGKILL cannot be caught, so
+// a Closer must not be the only protection against a leak.
+type Closer interface {
+	Close(ctx context.Context) error
+}
+
+// shutdownBudget bounds Closer.Close. The owner waits longer for the exit.
+var shutdownBudget = 8 * time.Second
+
+// newShutdown returns a function that closes rt (when it is a Closer) within
+// shutdownBudget and then calls exit(0). It runs the close once; a second call
+// waits for nothing and does not exit twice. Close runs in its own goroutine,
+// so a runtime that ignores its context cannot keep the process alive.
+func newShutdown(rt contract.Runtime, exit func(int)) func() {
+	var once sync.Once
+	return func() {
+		once.Do(func() {
+			if c, ok := rt.(Closer); ok {
+				ctx, cancel := context.WithTimeout(context.Background(), shutdownBudget)
+				defer cancel()
+				done := make(chan struct{})
+				go func() {
+					defer close(done)
+					if err := c.Close(ctx); err != nil {
+						xlog.Warn("add-on shutdown", "error", err)
+					}
+				}()
+				select {
+				case <-done:
+				case <-ctx.Done():
+				}
+			}
+			exit(0)
+		})
+	}
 }
 
 // Handler authenticates every request with the incarnation credential and
@@ -118,7 +197,9 @@ func Handler(startup Startup, rt contract.Runtime) http.Handler {
 				w.Header().Set("Content-Type", "application/json")
 				_ = json.NewEncoder(w).Encode(v)
 			case errors.Is(err, ErrBadRequest):
-				http.Error(w, "invalid request", http.StatusBadRequest)
+				http.Error(w, badRequestText(err), http.StatusBadRequest)
+			case errors.Is(err, contract.ErrRefused):
+				http.Error(w, ErrBadRequest.Error(), http.StatusBadRequest)
 			case errors.Is(err, ErrNotFound):
 				http.Error(w, "not found", http.StatusNotFound)
 			case errors.Is(err, ErrForbidden):
@@ -139,6 +220,33 @@ func Handler(startup Startup, rt contract.Runtime) http.Handler {
 				out, err := rt.Read(ctx, v)
 				reply(out, err)
 			}
+		case "/act":
+			// Only a runtime that serves action tools has the route.
+			ar, ok := rt.(contract.ActRuntime)
+			if !ok {
+				http.NotFound(w, r)
+				return
+			}
+			var v contract.Envelope
+			if read(&v) && valid(v, "act") {
+				out, err := ar.Act(ctx, v)
+				reply(out, err)
+			}
+		case "/connector":
+			// Only a runtime that declared a connector has the route. The
+			// host calls it with no user behind the call, so it takes no
+			// envelope: the bearer credential and the incarnation header
+			// above are its authority.
+			cr, ok := rt.(contract.ConnectorRuntime)
+			if !ok {
+				http.NotFound(w, r)
+				return
+			}
+			var v contract.ConnectorRequest
+			if read(&v) {
+				out, err := cr.Connector(ctx, v)
+				reply(out, err)
+			}
 		case "/preview":
 			var v contract.Envelope
 			if read(&v) && valid(v, "preview") {
@@ -199,12 +307,31 @@ func Run(args []string, factory func(Startup) contract.Runtime) int {
 	if err != nil || !filepath.IsAbs(startup.DataDir) {
 		return 3
 	}
-	ExitWithParent(rest)
+	// The watch starts before the factory, which may be slow. A stop that
+	// arrives meanwhile waits for the runtime, but not longer than the budget.
+	var rt contract.Runtime
+	built := make(chan struct{})
+	var once sync.Once
+	stop := func() {
+		once.Do(func() {
+			select {
+			case <-built:
+			case <-time.After(shutdownBudget):
+			}
+			newShutdown(rt, os.Exit)()
+		})
+	}
+	exitOnParentLoss(rest, stop)
+	sig := make(chan os.Signal, 1)
+	signal.Notify(sig, syscall.SIGTERM, syscall.SIGINT)
+	go func() { <-sig; stop() }()
 	listener, err := net.FileListener(os.NewFile(3, "listener"))
 	if err != nil {
 		return 6
 	}
-	if err := Serve(context.Background(), listener, startup, factory(startup)); err != nil {
+	rt = factory(startup)
+	close(built)
+	if err := Serve(context.Background(), listener, startup, rt); err != nil {
 		return 8
 	}
 	return 0

addons/internal/protocol/run_test.go

@@ -2,6 +2,11 @@
 package protocol
 
 import (
+	"context"
+	"io"
+	"sync/atomic"
+	"time"
+
 	"github.com/mudler/LocalAI/pkg/addons/contract"
 	. "github.com/onsi/ginkgo/v2"
 	. "github.com/onsi/gomega"
@@ -17,3 +22,72 @@ var _ = Describe("Run", func() {
 		Expect(built).To(BeFalse(), "the factory ran before the protocol flag was checked")
 	})
 })
+
+// closingRuntime is a runtime that has work to do before the process exits.
+type closingRuntime struct {
+	contract.Runtime
+	closeFn func(context.Context) error
+}
+
+func (c closingRuntime) Close(ctx context.Context) error { return c.closeFn(ctx) }
+
+var _ = Describe("the shutdown of an add-on", func() {
+	It("closes the runtime with a bounded context before it exits, once", func() {
+		var calls, exits atomic.Int32
+		var bounded atomic.Bool
+		rt := closingRuntime{closeFn: func(ctx context.Context) error {
+			calls.Add(1)
+			_, ok := ctx.Deadline()
+			bounded.Store(ok)
+			return nil
+		}}
+		code := make(chan int, 2)
+		stop := newShutdown(rt, func(c int) { exits.Add(1); code <- c })
+		stop()
+		stop()
+		Expect(<-code).To(Equal(0))
+		Expect(calls.Load()).To(Equal(int32(1)))
+		Expect(bounded.Load()).To(BeTrue())
+		Expect(exits.Load()).To(Equal(int32(1)))
+	})
+
+	It("exits when the owner closes the startup channel", func() {
+		r, w := io.Pipe()
+		done := make(chan struct{})
+		var closed atomic.Bool
+		rt := closingRuntime{closeFn: func(context.Context) error { closed.Store(true); return nil }}
+		exitOnParentLoss(r, newShutdown(rt, func(int) { close(done) }))
+		Expect(w.Close()).To(Succeed())
+		Eventually(done).Should(BeClosed())
+		Expect(closed.Load()).To(BeTrue())
+	})
+
+	It("still exits when the runtime does not return in time", func() {
+		old := shutdownBudget
+		shutdownBudget = 100 * time.Millisecond
+		DeferCleanup(func() { shutdownBudget = old })
+		release := make(chan struct{})
+		DeferCleanup(func() { close(release) })
+		rt := closingRuntime{closeFn: func(context.Context) error { <-release; return nil }}
+		done := make(chan struct{})
+		go newShutdown(rt, func(int) { close(done) })()
+		Eventually(done, 2*time.Second).Should(BeClosed())
+	})
+
+	It("exits at once for a runtime with nothing to close", func() {
+		done := make(chan struct{})
+		go newShutdown(nil, func(int) { close(done) })()
+		Eventually(done).Should(BeClosed())
+	})
+})
+
+var _ = Describe("StopWithParent", func() {
+	It("ends the context when the owner closes the startup channel, so Run can return", func() {
+		r, w := io.Pipe()
+		ctx, stop := StopWithParent(context.Background(), r)
+		DeferCleanup(stop)
+		Consistently(ctx.Done(), 50*time.Millisecond).ShouldNot(BeClosed())
+		Expect(w.Close()).To(Succeed())
+		Eventually(ctx.Done()).Should(BeClosed())
+	})
+})

addons/internal/tools-ui/src/tools.css

@@ -80,8 +80,8 @@
 .bt-kv__row { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 1fr) auto; gap: var(--dk-space-2); width: 100%; align-items: center; }
 .bt-banner { display: flex; align-items: center; gap: var(--dk-space-3); padding: var(--dk-space-3) var(--dk-space-4); border-radius: var(--dk-radius-2); background: var(--dk-accent-wash); color: var(--dk-text); font-size: var(--dk-text-13); flex-wrap: wrap; }
 .bt-banner code { overflow-wrap: anywhere; }
-.bt-rewards, .bt-liquid { display: flex; flex-direction: column; gap: var(--dk-space-3); padding-top: var(--dk-space-3); border-top: 1px solid var(--dk-edge); }
-.bt-rewards .dk-hint, .bt-liquid .dk-hint { margin: 0; }
+.bt-rewards { display: flex; flex-direction: column; gap: var(--dk-space-3); padding-top: var(--dk-space-3); border-top: 1px solid var(--dk-edge); }
+.bt-rewards .dk-hint { margin: 0; }
 .bt-reward { display: flex; flex-direction: column; gap: var(--dk-space-2); padding: var(--dk-space-3); border: 1px solid var(--dk-edge); border-radius: var(--dk-radius-2); background: var(--dk-card); min-width: 0; }
 .bt-reward[data-on="true"] { background: var(--dk-accent-wash); border-color: transparent; }
 .bt-reward__name { display: block; font-size: var(--dk-text-13); font-weight: var(--dk-weight-semibold); }

addons/internal/tools-ui/src/tools.js

@@ -50,17 +50,6 @@ export function isActive(job, stages) {
   return !!job && stages.includes(job.status)
 }
 
-// The newest job that is running, else the newest job when it failed. Used by the
-// Build landing to put one line above the list.
-export function jobToSurface(jobs, stages) {
-  const list = Array.isArray(jobs) ? jobs : []
-  const byNewest = [...list].sort((a, b) => Date.parse(b.created_at || 0) - Date.parse(a.created_at || 0))
-  const running = byNewest.find(j => stages.includes(j.status))
-  if (running) return { job: running, kind: 'running' }
-  if (byNewest[0]?.status === 'failed') return { job: byNewest[0], kind: 'failed' }
-  return null
-}
-
 // The phase a log line belongs to: info, warn or err.
 export function logLevel(event) {
   const text = `${event?.message || ''}`
@@ -119,58 +108,6 @@ export function looksLikeMemoryFailure(message) {
 
 const names = (list) => (Array.isArray(list) ? list.map(b => b?.name || b).filter(Boolean) : [])
 
-// What one tool needs from this machine, read from real data. `state` is
-// 'ready' or 'needs-backend'; `needs` lists lines for the card, each with a tone
-// ('ok', 'warn' or 'info') and a translation key.
-//
-//   facts      machineFacts(resources), or null
-//   backends   { fineTune: [...] | null, quantize: [...] | null }; null when the
-//              list could not be read
-export function toolStatus(tool, { facts, backends }) {
-  if (tool === 'import') {
-    const needs = [{ tone: 'info', key: 'import.noSetup' }]
-    if (facts?.diskFree != null) needs.push({ tone: 'ok', key: 'import.disk', vals: { free: facts.diskFree } })
-    return { state: 'ready', needs }
-  }
-  if (tool === 'quantize') {
-    const list = names(backends?.quantize)
-    const needs = []
-    if (backends?.quantize == null) {
-      needs.push({ tone: 'info', key: 'quantize.backendUnknown' })
-      return { state: 'ready', needs }
-    }
-    if (list.length === 0) {
-      needs.push({ tone: 'warn', key: 'quantize.backendMissing' })
-      return { state: 'needs-backend', needs }
-    }
-    needs.push({ tone: 'ok', key: 'quantize.backend', vals: { name: list[0] } })
-    needs.push({ tone: 'info', key: 'quantize.cpu' })
-    return { state: 'ready', needs }
-  }
-  if (tool === 'fine-tune') {
-    const list = names(backends?.fineTune)
-    const needs = []
-    let state = 'ready'
-    if (backends?.fineTune == null) {
-      needs.push({ tone: 'info', key: 'fineTune.backendUnknown' })
-    } else if (list.length === 0) {
-      needs.push({ tone: 'warn', key: 'fineTune.backendMissing' })
-      state = 'needs-backend'
-    } else {
-      needs.push({ tone: 'ok', key: 'fineTune.backend', vals: { name: list[0] } })
-    }
-    if (facts?.cluster) {
-      needs.push({ tone: 'info', key: 'fineTune.cluster', vals: { total: facts.cluster.total, node: facts.cluster.node } })
-    } else if (facts?.hasGpu) {
-      needs.push({ tone: 'ok', key: 'fineTune.gpu', vals: { name: facts.gpuName, free: facts.vramFree, total: facts.vramTotal } })
-    } else if (facts) {
-      needs.push({ tone: 'warn', key: 'fineTune.noGpu' })
-    }
-    return { state, needs }
-  }
-  return { state: 'ready', needs: [] }
-}
-
 // ---- Pre-flight checks -------------------------------------------------------------
 
 // A check: id, tone ('ok', 'warn', 'fail' or 'info'), key and vals for the text,

addons/internal/tools-ui/test/tools.test.mjs

@@ -2,9 +2,9 @@ import assert from 'node:assert/strict'
 import test from 'node:test'
 
 import {
-  appendLog, blocked, detectSource, fineTuneChecks, fitFor, importChecks, importRequest, jobToSurface,
+  appendLog, blocked, detectSource, fineTuneChecks, fitFor, importChecks, importRequest,
   joinCommands, logLines, logLevel, LOG_LIMIT, looksLikeMemoryFailure, machineFacts, quantizeChecks,
-  requestText, shortToken, sortNetworks, toolStatus, warned, workerCount,
+  requestText, shortToken, sortNetworks, warned, workerCount,
 } from '../src/tools.js'
 
 const GB = 1024 ** 3
@@ -38,34 +38,6 @@ test('machineFacts names several GPUs and reads the cluster block', () => {
   assert.deepEqual(f.cluster, { node: 'w1', total: 80 * GB, isGpu: true, nodes: 3 })
 })
 
-test('toolStatus: quantize and fine-tune say what is missing and why', () => {
-  const facts = machineFacts(cpuHost)
-  assert.equal(toolStatus('quantize', { facts, backends: { quantize: [] } }).state, 'needs-backend')
-  const q = toolStatus('quantize', { facts, backends: { quantize: [{ name: 'llama-cpp-quantization' }] } })
-  assert.equal(q.state, 'ready')
-  assert.equal(q.needs[0].vals.name, 'llama-cpp-quantization')
-  const none = toolStatus('fine-tune', { facts, backends: { fineTune: [{ name: 'trl' }] } })
-  assert.equal(none.state, 'ready')
-  assert.ok(none.needs.some(n => n.key === 'fineTune.noGpu' && n.tone === 'warn'))
-  const gpu = toolStatus('fine-tune', { facts: machineFacts(gpuHost), backends: { fineTune: [{ name: 'trl' }] } })
-  assert.ok(gpu.needs.some(n => n.key === 'fineTune.gpu' && n.vals.free === 9 * GB))
-  assert.equal(toolStatus('fine-tune', { facts: null, backends: { fineTune: null } }).needs[0].key, 'fineTune.backendUnknown')
-  assert.ok(toolStatus('import', { facts, backends: {} }).needs.some(n => n.key === 'import.disk'))
-})
-
-test('jobToSurface prefers a running job, then a newest failure', () => {
-  const stages = ['queued', 'training']
-  const jobs = [
-    { id: 'a', status: 'completed', created_at: '2026-10-01T10:00:00Z' },
-    { id: 'b', status: 'training', created_at: '2026-10-02T10:00:00Z' },
-    { id: 'c', status: 'failed', created_at: '2026-10-03T10:00:00Z' },
-  ]
-  assert.deepEqual(jobToSurface(jobs, stages), { job: jobs[1], kind: 'running' })
-  assert.deepEqual(jobToSurface([jobs[0], jobs[2]], stages), { job: jobs[2], kind: 'failed' })
-  assert.equal(jobToSurface([jobs[2], { id: 'd', status: 'completed', created_at: '2026-10-04T10:00:00Z' }], stages), null)
-  assert.equal(jobToSurface(null, stages), null)
-})
-
 test('log lines come from what the event holds and nothing else', () => {
   const at = new Date(2026, 9, 8, 10, 2, 11)
   const first = logLines({ status: 'training', message: 'Loading model', current_step: 0, total_steps: 900 }, null, at)

addons/internal/uibuild/kitstub.mjs

@@ -0,0 +1,31 @@
+// Stand-ins for the host's settings kit (sdk.ui.Section, Field, Choice, ...)
+// in the node tests of first-party add-on UIs. The host draws the real ones
+// (core/http/react-ui/src/addons/hostComponents.jsx); these keep the same
+// props and draw plain markup with the test ids, so a test can check what an
+// add-on asks the host to draw without the host's build.
+export const KIT = ['Section', 'Field', 'Select', 'TextInput', 'Switch', 'Choice', 'Callout', 'StatusChip', 'List', 'ListRow', 'Code', 'Actions', 'Table', 'ExternalLink']
+
+export function kitStubs(React) {
+  const h = React.createElement
+  const tid = (p) => (p.testId ? { 'data-testid': p.testId } : {})
+  return {
+    Section: (p) => h('section', { 'data-kit': 'Section', ...tid(p) }, p.title ? h('h3', null, p.title) : null, p.description ? h('p', null, p.description) : null, p.actions || null, p.children),
+    Field: (p) => h('label', { 'data-kit': 'Field', ...tid(p) }, h('span', null, p.label), p.children, p.unit ? h('span', null, p.unit) : null, p.help ? h('span', { 'data-kit-help': '' }, p.help) : null, p.error ? h('span', { role: 'alert' }, p.error) : null),
+    Select: (p) => h('select', { 'data-kit': 'Select', value: p.value, disabled: p.disabled, onChange: () => {}, ...tid(p) }, (p.options || []).map(o => h('option', { key: o.value, value: o.value }, o.label))),
+    TextInput: (p) => h('input', { 'data-kit': 'TextInput', type: p.type === 'password' ? 'password' : 'text', value: p.value ?? '', disabled: p.disabled, onChange: () => {}, ...tid(p) }),
+    Switch: (p) => h('button', { 'data-kit': 'Switch', role: 'switch', 'aria-checked': !!p.checked, 'aria-label': p.label, disabled: p.disabled, ...tid(p) }),
+    Choice: (p) => h('fieldset', { 'data-kit': 'Choice', disabled: p.disabled, ...tid(p) }, p.legend ? h('legend', null, p.legend) : null,
+      (p.options || []).map(o => h('label', { key: o.value, 'data-value': o.value, 'data-selected': p.value === o.value ? '' : undefined },
+        h('input', { type: 'radio', checked: p.value === o.value, onChange: () => {}, disabled: o.disabled }), o.label, o.chip || null, o.description ? h('span', null, o.description) : null))),
+    Callout: (p) => h('div', { 'data-kit': 'Callout', 'data-tone': p.tone || 'info', role: p.role, ...tid(p) }, p.title ? h('strong', null, p.title) : null, p.children, p.actions || null),
+    StatusChip: (p) => h('span', { 'data-kit': 'StatusChip', 'data-tone': p.tone || 'idle', ...tid(p) }, p.children),
+    List: (p) => h('ul', { 'data-kit': 'List', 'aria-label': p.label, ...tid(p) }, p.children),
+    ListRow: (p) => h('li', { 'data-kit': 'ListRow', ...tid(p) }, h('strong', null, p.title), p.status || null, p.meta ? h('span', null, p.meta) : null, p.children,
+      p.details ? h('details', null, h('summary', null, p.detailsLabel || 'Details'), p.details) : null),
+    Code: (p) => h('pre', { 'data-kit': 'Code', ...tid(p) }, p.children),
+    Actions: (p) => h('div', { 'data-kit': 'Actions' }, p.children),
+    Table: (p) => h('table', { 'data-kit': 'Table', ...tid(p) }, h('tbody', null, (p.rows || []).map((r, i) => h('tr', { key: r.key ?? i }, (p.columns || []).map(c => h('td', { key: c.key }, r[c.key])))))),
+    ExternalLink: (p) => h('a', { 'data-kit': 'ExternalLink', href: p.href, ...tid(p) }, p.children),
+    Button: (p) => h('button', { 'data-kit': 'Button', 'data-variant': p.variant, disabled: p.disabled || p.busy === true, ...tid(p) }, p.children),
+  }
+}

addons/office/cmd/office/main.go

@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: MIT
+
+// Command office is the executable of the Office add-on. It reads and edits
+// Word, Excel, PowerPoint and PDF files for the chats; the files themselves
+// stay with the host, which hands them over through the workspace operations.
+package main
+
+import (
+	"os"
+
+	"github.com/mudler/LocalAI/addons/internal/hostclient"
+	"github.com/mudler/LocalAI/addons/internal/protocol"
+	"github.com/mudler/LocalAI/addons/office/internal/tools"
+	"github.com/mudler/LocalAI/pkg/addons/contract"
+)
+
+func main() {
+	os.Exit(protocol.Run(os.Args, func(s protocol.Startup) contract.Runtime {
+		return tools.New(hostclient.New(s), s.DataDir)
+	}))
+}

addons/office/internal/deck/chart.go

@@ -0,0 +1,172 @@
+package deck
+
+import (
+	"math"
+	"strconv"
+	"strings"
+)
+
+// Chart geometry: a 300x170 view box with padding left 30, right 10, top 18
+// and bottom 26. The TypeScript twin is src/studio/chart.ts; keep the two in
+// lockstep.
+const (
+	chartW  = 300.0
+	chartH  = 170.0
+	chartPL = 30.0
+	chartPR = 10.0
+	chartPT = 18.0
+	chartPB = 26.0
+)
+
+// niceMax rounds m up to 1, 2, 5 or 10 times a power of ten. It returns 1 for
+// values that are not positive and finite.
+func niceMax(m float64) float64 {
+	if !(m > 0) || math.IsInf(m, 0) {
+		return 1
+	}
+	p := math.Pow(10, math.Floor(math.Log10(m)))
+	n := m / p
+	switch {
+	case n <= 1:
+		return p
+	case n <= 2:
+		return 2 * p
+	case n <= 5:
+		return 5 * p
+	}
+	return 10 * p
+}
+
+var xmlEscaper = strings.NewReplacer("&", "&amp;", "<", "&lt;", ">", "&gt;", `"`, "&quot;", "'", "&#39;")
+
+func esc(s string) string { return xmlEscaper.Replace(s) }
+
+// num prints a coordinate rounded to two decimals.
+func num(v float64) string {
+	return strconv.FormatFloat(math.Round(v*100)/100, 'f', -1, 64)
+}
+
+func clampY(v float64) float64 { return min(max(v, 0), chartH) }
+
+// ChartSVG draws a bar or line chart as an inline SVG string. It uses
+// currentColor for ink and var(--sa) for the accent, so the slide style
+// decides the colours. Negative values hang from the zero baseline. The
+// output only uses elements and attributes that the presenter's SVG
+// sanitizer keeps.
+func ChartSVG(c StudioChart) string {
+	iw := chartW - chartPL - chartPR
+	ih := chartH - chartPT - chartPB
+
+	vals := make([]float64, len(c.Data))
+	hiV, loV := 0.0, 0.0
+	for i, p := range c.Data {
+		v := p.V
+		if math.IsNaN(v) || math.IsInf(v, 0) {
+			v = 0
+		}
+		vals[i] = v
+		hiV, loV = max(hiV, v), min(loV, v)
+	}
+	hi, lo := 0.0, 0.0
+	if hiV > 0 {
+		hi = niceMax(hiV)
+	}
+	if loV < 0 {
+		lo = -niceMax(-loV)
+	}
+	if hi == lo {
+		hi = 1
+	}
+	y := func(v float64) float64 { return chartPT + ih - (v-lo)/(hi-lo)*ih }
+	base := y(0)
+
+	var s strings.Builder
+	s.WriteString(`<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 300 170">`)
+	if c.Label != "" {
+		s.WriteString("<title>" + esc(c.Label) + "</title>")
+	}
+
+	grid := []float64{0, hi / 2, hi}
+	switch {
+	case hi == 0:
+		grid = []float64{0, lo / 2, lo}
+	case lo < 0:
+		grid = []float64{0, lo, hi}
+	}
+	for _, v := range grid {
+		op := ".15"
+		if v == 0 {
+			op = ".4"
+		}
+		gy := num(y(v))
+		s.WriteString(`<line x1="` + num(chartPL) + `" x2="` + num(chartW-chartPR) + `" y1="` + gy + `" y2="` + gy +
+			`" stroke="currentColor" stroke-opacity="` + op + `"/>`)
+		s.WriteString(`<text x="` + num(chartPL-5) + `" y="` + num(clampY(y(v)+3)) +
+			`" font-size="9" text-anchor="end" fill="currentColor" fill-opacity=".65">` +
+			strconv.FormatFloat(math.Round(v*10)/10, 'f', -1, 64) + `</text>`)
+	}
+	if len(vals) == 0 {
+		return s.String() + "</svg>"
+	}
+
+	step := iw / float64(len(vals))
+	keyLabel := func(x float64, k string) string {
+		return `<text x="` + num(x) + `" y="` + num(chartH-8) +
+			`" font-size="9.5" text-anchor="middle" fill="currentColor" fill-opacity=".75">` + esc(k) + `</text>`
+	}
+	valueText := func(v float64) string { return strconv.FormatFloat(v, 'f', -1, 64) }
+
+	if c.Kind == ChartLine {
+		type pt struct{ x, y float64 }
+		pts := make([]pt, len(vals))
+		var line strings.Builder
+		for i, v := range vals {
+			pts[i] = pt{chartPL + step*(float64(i)+.5), y(v)}
+			if i > 0 {
+				line.WriteString("L")
+			}
+			line.WriteString(num(pts[i].x) + " " + num(pts[i].y))
+		}
+		last := pts[len(pts)-1]
+		s.WriteString(`<path d="M` + line.String() + `L` + num(last.x) + ` ` + num(base) + `L` + num(pts[0].x) + ` ` + num(base) +
+			`Z" fill="var(--sa)" fill-opacity=".14"/>`)
+		s.WriteString(`<path d="M` + line.String() +
+			`" fill="none" stroke="var(--sa)" stroke-width="2.6" stroke-linejoin="round" stroke-linecap="round"/>`)
+		for i, p := range pts {
+			r := "3.4"
+			if i == len(pts)-1 {
+				r = "5"
+			}
+			ly := p.y - 9
+			if vals[i] < 0 {
+				ly = p.y + 15
+			}
+			s.WriteString(`<circle cx="` + num(p.x) + `" cy="` + num(p.y) + `" r="` + r + `" fill="var(--sa)"/>`)
+			s.WriteString(`<text x="` + num(p.x) + `" y="` + num(clampY(ly)) +
+				`" font-size="10" font-weight="700" text-anchor="middle" fill="currentColor">` + valueText(vals[i]) + `</text>`)
+			s.WriteString(keyLabel(p.x, c.Data[i].K))
+		}
+		return s.String() + "</svg>"
+	}
+
+	unit := ""
+	if c.Unit != "" && !strings.Contains(c.Unit, " ") && !strings.HasPrefix(c.Unit, "/") {
+		unit = esc(c.Unit)
+	}
+	bw := min(52, step*.6)
+	for i, v := range vals {
+		x := chartPL + step*float64(i) + (step-bw)/2
+		yv := y(v)
+		top, height := min(yv, base), math.Abs(yv-base)
+		ly := yv - 5
+		if v < 0 {
+			ly = yv + 12
+		}
+		s.WriteString(`<rect x="` + num(x) + `" y="` + num(top) + `" width="` + num(bw) + `" height="` + num(height) +
+			`" rx="4" fill="var(--sa)" fill-opacity=".92"/>`)
+		s.WriteString(`<text x="` + num(x+bw/2) + `" y="` + num(clampY(ly)) +
+			`" font-size="10.5" font-weight="700" text-anchor="middle" fill="currentColor">` + valueText(v) + unit + `</text>`)
+		s.WriteString(keyLabel(x+bw/2, c.Data[i].K))
+	}
+	return s.String() + "</svg>"
+}

addons/office/internal/deck/chart_test.go

@@ -0,0 +1,165 @@
+package deck
+
+import (
+	"regexp"
+	"strconv"
+	"strings"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("chart SVG", func() {
+	It("rounds a maximum up to a nice axis value", func() {
+		cases := map[float64]float64{49: 50, 4.4: 5, 1: 1, 10: 10, 11: 20, 0.3: 0.5, 0: 1, -3: 1}
+		for in, want := range cases {
+			Expect(niceMax(in)).To(Equal(want), "niceMax(%v)", in)
+		}
+	})
+
+	It("draws a bar chart", func() {
+		svg := ChartSVG(StudioChart{
+			Kind:  ChartBar,
+			Label: "Cost <by> office & team",
+			Unit:  "%",
+			Data:  []ChartPoint{{K: "Rome", V: 12}, {K: "Milan & <Turin>", V: 49}, {K: "Naples", V: 30}},
+		})
+		Expect(svg).To(HavePrefix("<svg"))
+		Expect(svg).To(HaveSuffix("</svg>"))
+		Expect(svg).To(ContainSubstring(`viewBox="0 0 300 170"`))
+		Expect(strings.Count(svg, "<rect")).To(Equal(3), "bars")
+		Expect(strings.Count(svg, "<line")).To(Equal(3), "grid lines")
+		Expect(svg).To(ContainSubstring("Milan &amp; &lt;Turin&gt;"), "labels not escaped")
+		Expect(svg).NotTo(ContainSubstring("<Turin>"), "labels not escaped")
+		Expect(svg).To(ContainSubstring("Cost &lt;by&gt; office &amp; team"), "chart label not escaped")
+		Expect(svg).To(ContainSubstring(">49%<"), "expected value with unit")
+		Expect(svg).To(ContainSubstring(">50<"), "expected nice max ticks 25 and 50")
+		Expect(svg).To(ContainSubstring(">25<"), "expected nice max ticks 25 and 50")
+		Expect(svg).To(ContainSubstring(`fill="var(--sa)"`))
+		Expect(svg).To(ContainSubstring(`rx="4"`))
+		assertSafeSVG(svg)
+		assertInsideViewBox(svg)
+	})
+
+	It("hides a unit that has a space or a slash", func() {
+		for _, u := range []string{"k EUR", "/5"} {
+			svg := ChartSVG(StudioChart{Kind: ChartBar, Unit: u, Data: []ChartPoint{{K: "a", V: 3}}})
+			Expect(svg).NotTo(ContainSubstring(">3"+u), "unit %q should not be printed", u)
+			Expect(svg).NotTo(ContainSubstring(">3k"), "unit %q should not be printed", u)
+		}
+	})
+
+	It("draws a line chart", func() {
+		svg := ChartSVG(StudioChart{
+			Kind: ChartLine,
+			Data: []ChartPoint{{K: "W1", V: 4}, {K: "W2", V: 4.4}, {K: "W3", V: 3}},
+		})
+		Expect(strings.Count(svg, "<path")).To(Equal(2), "area + line paths")
+		Expect(strings.Count(svg, "<circle")).To(Equal(3), "dots")
+		Expect(strings.Count(svg, `r="5"`)).To(Equal(1), "last dot r 5")
+		Expect(strings.Count(svg, `r="3.4"`)).To(Equal(2), "other dots r 3.4")
+		Expect(svg).To(ContainSubstring(`fill-opacity=".14"`))
+		Expect(svg).To(ContainSubstring(`stroke-width="2.6"`))
+		Expect(svg).NotTo(ContainSubstring("<rect"), "line chart must not draw bars")
+		Expect(svg).To(ContainSubstring(">5<"), "ticks of niceMax(4.4)=5")
+		Expect(svg).To(ContainSubstring(">2.5<"), "ticks of niceMax(4.4)=5")
+		assertSafeSVG(svg)
+		assertInsideViewBox(svg)
+	})
+
+	DescribeTable("negative values stay safe and inside the viewBox",
+		func(data []ChartPoint, kind ChartKind) {
+			svg := ChartSVG(StudioChart{Kind: kind, Label: "x", Data: data})
+			assertSafeSVG(svg)
+			assertInsideViewBox(svg)
+			if kind == ChartBar {
+				Expect(strings.Count(svg, "<rect")).To(Equal(len(data)), "bars: %s", svg)
+			}
+		},
+		Entry("mixed bar", []ChartPoint{{K: "a", V: 12}, {K: "b", V: -30}, {K: "c", V: 49}, {K: "d", V: 0}}, ChartBar),
+		Entry("mixed line", []ChartPoint{{K: "a", V: 12}, {K: "b", V: -30}, {K: "c", V: 49}, {K: "d", V: 0}}, ChartLine),
+		Entry("all-negative bar", []ChartPoint{{K: "a", V: -4}, {K: "b", V: -12}, {K: "c", V: -0.5}}, ChartBar),
+		Entry("all-negative line", []ChartPoint{{K: "a", V: -4}, {K: "b", V: -12}, {K: "c", V: -0.5}}, ChartLine),
+		Entry("all-zero bar", []ChartPoint{{K: "a", V: 0}, {K: "b", V: 0}}, ChartBar),
+		Entry("all-zero line", []ChartPoint{{K: "a", V: 0}, {K: "b", V: 0}}, ChartLine),
+	)
+
+	It("hangs negative bars from the baseline", func() {
+		svg := ChartSVG(StudioChart{Kind: ChartBar, Data: []ChartPoint{{K: "a", V: 10}, {K: "b", V: -10}}})
+		rects := regexp.MustCompile(`<rect x="[^"]+" y="([^"]+)" width="[^"]+" height="([^"]+)"`).FindAllStringSubmatch(svg, -1)
+		Expect(rects).To(HaveLen(2), svg)
+		posY, _ := strconv.ParseFloat(rects[0][1], 64)
+		posH, _ := strconv.ParseFloat(rects[0][2], 64)
+		negY, _ := strconv.ParseFloat(rects[1][1], 64)
+		Expect(posY+posH).To(Equal(negY), "positive bar must end where negative bar starts (baseline)")
+	})
+
+	It("is safe for an empty chart", func() {
+		assertInsideViewBox(ChartSVG(StudioChart{Kind: ChartBar}))
+		assertInsideViewBox(ChartSVG(StudioChart{Kind: ChartLine}))
+	})
+})
+
+var (
+	attrNumRe  = regexp.MustCompile(`\s(x|y|x1|x2|y1|y2|cx|cy|width|height|r)="([^"]*)"`)
+	pathDRe    = regexp.MustCompile(`\sd="([^"]*)"`)
+	pathNumsRe = regexp.MustCompile(`-?[0-9.]+(?:e[-+]?[0-9]+)?`)
+	tagRe      = regexp.MustCompile(`<([a-zA-Z]+)`)
+	attrNameRe = regexp.MustCompile(`\s([a-zA-Z:-]+)="`)
+)
+
+func assertInsideViewBox(svg string) {
+	GinkgoHelper()
+	for _, m := range attrNumRe.FindAllStringSubmatch(svg, -1) {
+		v, err := strconv.ParseFloat(m[2], 64)
+		Expect(err).NotTo(HaveOccurred(), "attribute %s=%q is not a number in %s", m[1], m[2], svg)
+		switch m[1] {
+		case "width", "height", "r":
+			Expect(v).To(BeNumerically(">=", 0), "negative %s in %s", m[1], svg)
+		case "y", "y1", "y2", "cy":
+			Expect(v).To(BeNumerically(">=", 0), "%s outside the viewBox in %s", m[1], svg)
+			Expect(v).To(BeNumerically("<=", 170), "%s outside the viewBox in %s", m[1], svg)
+		case "x", "x1", "x2", "cx":
+			Expect(v).To(BeNumerically(">=", 0), "%s outside the viewBox in %s", m[1], svg)
+			Expect(v).To(BeNumerically("<=", 300), "%s outside the viewBox in %s", m[1], svg)
+		}
+	}
+	for _, m := range pathDRe.FindAllStringSubmatch(svg, -1) {
+		nums := pathNumsRe.FindAllString(m[1], -1)
+		for i, s := range nums {
+			v, err := strconv.ParseFloat(s, 64)
+			Expect(err).NotTo(HaveOccurred(), "bad path number %q", s)
+			limit := 300.0
+			if i%2 == 1 {
+				limit = 170
+			}
+			Expect(v).To(BeNumerically(">=", 0), "path coordinate outside the viewBox in %s", m[1])
+			Expect(v).To(BeNumerically("<=", limit), "path coordinate outside the viewBox in %s", m[1])
+		}
+	}
+	Expect(svg).NotTo(ContainSubstring("NaN"), "non-finite number")
+	Expect(svg).NotTo(ContainSubstring("Inf"), "non-finite number")
+}
+
+// assertSafeSVG mirrors the allowlists of the presenter's sanitizeSvg so the
+// chart survives it unchanged.
+func assertSafeSVG(svg string) {
+	GinkgoHelper()
+	tags := map[string]bool{"svg": true, "g": true, "path": true, "rect": true, "circle": true, "line": true, "text": true, "title": true}
+	attrs := map[string]bool{
+		"xmlns": true, "viewBox": true, "width": true, "height": true, "x": true, "y": true,
+		"x1": true, "y1": true, "x2": true, "y2": true, "cx": true, "cy": true, "r": true,
+		"rx": true, "d": true, "fill": true, "stroke": true, "stroke-width": true,
+		"stroke-linecap": true, "stroke-linejoin": true, "fill-opacity": true,
+		"stroke-opacity": true, "text-anchor": true, "font-size": true, "font-weight": true,
+	}
+	for _, m := range tagRe.FindAllStringSubmatch(svg, -1) {
+		Expect(tags).To(HaveKey(m[1]), "tag %q is not allowed: %s", m[1], svg)
+	}
+	for _, m := range attrNameRe.FindAllStringSubmatch(svg, -1) {
+		Expect(attrs).To(HaveKey(m[1]), "attribute %q is not allowed: %s", m[1], svg)
+	}
+	Expect(svg).NotTo(ContainSubstring("href"), "unsafe content")
+	Expect(svg).NotTo(ContainSubstring("url("), "unsafe content")
+	Expect(svg).NotTo(ContainSubstring("<script"), "unsafe content")
+}

addons/office/internal/deck/deck.go

@@ -0,0 +1,163 @@
+// Package deck defines the constrained slide-deck JSON contract shared by
+// the slides generation path and (later) shareable canvases. The shape and
+// the validation rules are the same as those of the deck validator in the
+// web client, so that a deck accepted by one language is accepted by the
+// other. Keep the two in lockstep.
+package deck
+
+import (
+	"encoding/json"
+	"fmt"
+	"regexp"
+)
+
+// Deck is the top-level presentation object.
+type Deck struct {
+	Version    int     `json:"version"`
+	Theme      string  `json:"theme"`
+	Accent     string  `json:"accent,omitempty"`
+	Font       string  `json:"font,omitempty"`
+	Background string  `json:"background,omitempty"`
+	Title      string  `json:"title,omitempty"`
+	Slides     []Slide `json:"slides"`
+}
+
+// Slide is a single slide; Blocks are kept loosely typed to mirror the TS
+// discriminated union without an explosion of Go types.
+type Slide struct {
+	Layout string           `json:"layout"`
+	Notes  string           `json:"notes,omitempty"`
+	Steps  bool             `json:"steps,omitempty"`
+	Blocks []map[string]any `json:"blocks"`
+}
+
+var (
+	themes  = map[string]bool{"midnight": true, "paper": true, "mono": true}
+	bgs     = map[string]bool{"solid": true, "gradient": true, "grid": true}
+	layouts = map[string]bool{
+		"title": true, "section": true, "single": true, "two-col": true,
+		"media-left": true, "media-right": true, "media-full": true,
+		"quote": true, "stat": true, "code": true, "agenda": true, "end": true,
+	}
+	// required non-empty string fields per block type.
+	blockStrings = map[string][]string{
+		"kicker": {"text"}, "heading": {"text"}, "subtitle": {"text"},
+		"paragraph": {"text"}, "quote": {"text"}, "stat": {"value"},
+		"code": {"code"}, "image": {"ref"}, "video": {"ref"},
+		"embed": {"url"}, "svg": {"svg"},
+	}
+	// required array-of-strings fields per block type (empty arrays allowed,
+	// mirroring the TS validator which only rejects non-string elements).
+	blockArrays = map[string][]string{"bullets": {"items"}}
+	hexRe       = regexp.MustCompile(`^#(?:[0-9a-fA-F]{3}|[0-9a-fA-F]{6})$`)
+	// Media refs (image/video ref, video poster) must be INTERNAL so a compiled
+	// artifact fetches nothing external; embed urls must be a real http(s) site.
+	// Mirrors INTERNAL_MEDIA_REF / HTTP_URL in deck.ts.
+	internalMediaRe = regexp.MustCompile(`^(?:/_/image/|/api/documents/)`)
+	httpURLRe       = regexp.MustCompile(`(?i)^https?://`)
+)
+
+const (
+	mediaRefMsg = "image ref must be an internal image reference"
+	embedURLMsg = "embed url must be an http(s) URL"
+)
+
+// Validate returns human-readable errors; an empty slice means the deck is
+// valid. It is deliberately permissive about unknown OPTIONAL fields but
+// strict about enums, required fields, and value shapes. The rules mirror
+// validateDeck() in deck.ts.
+func Validate(raw []byte) []string {
+	var d map[string]any
+	if err := json.Unmarshal(raw, &d); err != nil {
+		return []string{fmt.Sprintf("invalid JSON: %v", err)}
+	}
+	var errs []string
+	if v, _ := d["version"].(float64); int(v) != 1 {
+		errs = append(errs, "version must be 1")
+	}
+	if th, _ := d["theme"].(string); !themes[th] {
+		errs = append(errs, "theme must be one of midnight, paper, mono")
+	}
+	if acc, present := d["accent"]; present {
+		s, ok := acc.(string)
+		if !ok || !hexRe.MatchString(s) {
+			errs = append(errs, "accent must be a hex color")
+		}
+	}
+	if bg, present := d["background"]; present {
+		s, ok := bg.(string)
+		if !ok || !bgs[s] {
+			errs = append(errs, "background must be one of solid, gradient, grid")
+		}
+	}
+	slides, ok := d["slides"].([]any)
+	if !ok || len(slides) == 0 {
+		return append(errs, "slides must be a non-empty array")
+	}
+	for i, s := range slides {
+		sm, ok := s.(map[string]any)
+		if !ok {
+			errs = append(errs, fmt.Sprintf("slide %d: must be an object", i))
+			continue
+		}
+		if lay, _ := sm["layout"].(string); !layouts[lay] {
+			errs = append(errs, fmt.Sprintf("slide %d: layout must be one of title, section, single, two-col, media-left, media-right, media-full, quote, stat, code, agenda, end", i))
+		}
+		blocks, ok := sm["blocks"].([]any)
+		if !ok {
+			errs = append(errs, fmt.Sprintf("slide %d: blocks must be an array", i))
+			continue
+		}
+		for j, b := range blocks {
+			bm, ok := b.(map[string]any)
+			if !ok {
+				errs = append(errs, fmt.Sprintf("slide %d block %d: must be an object", i, j))
+				continue
+			}
+			bt, _ := bm["type"].(string)
+			_, known := blockStrings[bt]
+			_, knownArr := blockArrays[bt]
+			if !known && !knownArr {
+				errs = append(errs, fmt.Sprintf("slide %d block %d: unknown block type %q", i, j, bt))
+				continue
+			}
+			for _, k := range blockStrings[bt] {
+				if v, _ := bm[k].(string); v == "" {
+					errs = append(errs, fmt.Sprintf("slide %d block %d (%s): %q is required", i, j, bt, k))
+				}
+			}
+			for _, k := range blockArrays[bt] {
+				arr, ok := bm[k].([]any)
+				if !ok {
+					errs = append(errs, fmt.Sprintf("slide %d block %d (%s): %q must be an array of strings", i, j, bt, k))
+					continue
+				}
+				for _, x := range arr {
+					if _, isStr := x.(string); !isStr {
+						errs = append(errs, fmt.Sprintf("slide %d block %d (%s): %q must be an array of strings", i, j, bt, k))
+						break
+					}
+				}
+			}
+			// Scheme allowlist for media/embed URLs (mirrors deck.ts). Only
+			// non-empty strings are checked; empty/missing required fields keep
+			// their "required" error above.
+			if bt == "image" || bt == "video" {
+				if ref, _ := bm["ref"].(string); ref != "" && !internalMediaRe.MatchString(ref) {
+					errs = append(errs, fmt.Sprintf("slide %d block %d (%s): %s", i, j, bt, mediaRefMsg))
+				}
+			}
+			if bt == "video" {
+				if poster, _ := bm["poster"].(string); poster != "" && !internalMediaRe.MatchString(poster) {
+					errs = append(errs, fmt.Sprintf("slide %d block %d (video): %s", i, j, mediaRefMsg))
+				}
+			}
+			if bt == "embed" {
+				if url, _ := bm["url"].(string); url != "" && !httpURLRe.MatchString(url) {
+					errs = append(errs, fmt.Sprintf("slide %d block %d (embed): %s", i, j, embedURLMsg))
+				}
+			}
+		}
+	}
+	return errs
+}

addons/office/internal/deck/deck_suite_test.go

@@ -1,4 +1,4 @@
-package chathistory_test
+package deck_test
 
 import (
 	"testing"
@@ -7,7 +7,7 @@ import (
 	. "github.com/onsi/gomega"
 )
 
-func TestChatHistory(t *testing.T) {
+func TestDeck(t *testing.T) {
 	RegisterFailHandler(Fail)
-	RunSpecs(t, "ChatHistory test suite")
+	RunSpecs(t, "Deck Suite")
 }

addons/office/internal/deck/deck_test.go

@@ -0,0 +1,132 @@
+package deck_test
+
+import (
+	"github.com/mudler/LocalAI/addons/office/internal/deck"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+const validDeck = `{"version":1,"theme":"midnight","slides":[{"layout":"title","blocks":[{"type":"heading","text":"Hi"}]}]}`
+
+var _ = Describe("deck validation", func() {
+	It("accepts a good deck", func() {
+		Expect(deck.Validate([]byte(validDeck))).To(BeEmpty())
+	})
+
+	It("rejects a bad theme", func() {
+		Expect(deck.Validate([]byte(`{"version":1,"theme":"neon","slides":[{"layout":"title","blocks":[]}]}`))).NotTo(BeEmpty(), "expected theme error")
+	})
+
+	It("rejects a bad layout and block", func() {
+		errs := deck.Validate([]byte(`{"version":1,"theme":"mono","slides":[{"layout":"carousel","blocks":[{"type":"table"}]}]}`))
+		Expect(len(errs)).To(BeNumerically(">=", 2), "expected layout+block errors, got %v", errs)
+	})
+
+	It("rejects a bad accent and malformed JSON", func() {
+		Expect(deck.Validate([]byte(`{"version":1,"theme":"mono","accent":"red","slides":[{"layout":"end","blocks":[]}]}`))).NotTo(BeEmpty(), "expected accent error")
+		Expect(deck.Validate([]byte(`{ not json`))).NotTo(BeEmpty(), "expected json parse error")
+	})
+
+	Describe("parity with the deck validator of the web client", func() {
+		// A bullets block with a missing "items" is flagged, as in the web client.
+		It("rejects a bullets block with missing items", func() {
+			errs := deck.Validate([]byte(`{"version":1,"theme":"midnight","slides":[{"layout":"single","blocks":[{"type":"bullets"}]}]}`))
+			Expect(errs).To(ContainElement(ContainSubstring("items")))
+		})
+
+		// TS validateDeck treats an EMPTY bullets array as valid (some() over [] is false).
+		// Go must match so a deck valid in one language is valid in the other.
+		It("accepts an empty bullets array", func() {
+			Expect(deck.Validate([]byte(`{"version":1,"theme":"midnight","slides":[{"layout":"single","blocks":[{"type":"bullets","items":[]}]}]}`))).To(BeEmpty(), "expected empty bullets to be valid (TS parity)")
+		})
+
+		// TS rejects non-string bullet items. Go must too.
+		It("rejects non-string bullet items", func() {
+			Expect(deck.Validate([]byte(`{"version":1,"theme":"midnight","slides":[{"layout":"single","blocks":[{"type":"bullets","items":[1,2]}]}]}`))).NotTo(BeEmpty(), "expected non-string bullet items to be rejected (TS parity)")
+		})
+
+		// TS accepts an empty blocks array on a slide (forEach over [] does nothing).
+		It("accepts an empty blocks array", func() {
+			Expect(deck.Validate([]byte(`{"version":1,"theme":"paper","slides":[{"layout":"end","blocks":[]}]}`))).To(BeEmpty())
+		})
+
+		// TS accepts a 3-digit hex accent (#RGB) per the HEX regex.
+		It("accepts a short hex accent", func() {
+			Expect(deck.Validate([]byte(`{"version":1,"theme":"mono","accent":"#abc","slides":[{"layout":"title","blocks":[{"type":"heading","text":"Hi"}]}]}`))).To(BeEmpty())
+		})
+
+		// TS flags version !== 1.
+		It("rejects a wrong version", func() {
+			Expect(deck.Validate([]byte(`{"version":2,"theme":"mono","slides":[{"layout":"end","blocks":[]}]}`))).NotTo(BeEmpty(), "expected version error")
+		})
+
+		// TS rejects a PRESENT accent that is not a string (typeof d.accent !== "string").
+		// Go must too: {"accent":42} passes the old ok-guarded check but TS rejects it.
+		It("rejects a non-string accent", func() {
+			errs := deck.Validate([]byte(`{"version":1,"theme":"midnight","accent":42,"slides":[{"layout":"title","blocks":[]}]}`))
+			Expect(errs).To(ContainElement(ContainSubstring("accent must be a hex color")))
+		})
+
+		// TS rejects a PRESENT background that is not a valid enum string, including a
+		// present-but-non-string value (BACKGROUNDS.includes(5) is false).
+		It("rejects a non-string background", func() {
+			errs := deck.Validate([]byte(`{"version":1,"theme":"midnight","background":5,"slides":[{"layout":"title","blocks":[]}]}`))
+			Expect(errs).To(ContainElement(ContainSubstring("background must be one of solid, gradient, grid")))
+		})
+
+		// Positive parity case: valid string accent + background still passes.
+		It("accepts a valid accent and background", func() {
+			Expect(deck.Validate([]byte(`{"version":1,"theme":"midnight","accent":"#abc","background":"grid","slides":[{"layout":"title","blocks":[]}]}`))).To(BeEmpty())
+		})
+	})
+
+	// Media/embed scheme allowlist parity (deck.ts validateDeck).
+	Describe("media and embed allowlist", func() {
+		It("rejects an external image ref", func() {
+			errs := deck.Validate([]byte(`{"version":1,"theme":"midnight","slides":[{"layout":"single","blocks":[{"type":"image","ref":"https://evil/x.png"}]}]}`))
+			Expect(errs).To(ContainElement(ContainSubstring("image ref must be an internal image reference")))
+		})
+
+		It("accepts internal image refs", func() {
+			for _, ref := range []string{"/_/image/x", "/api/documents/1/file"} {
+				errs := deck.Validate([]byte(`{"version":1,"theme":"midnight","slides":[{"layout":"single","blocks":[{"type":"image","ref":"` + ref + `"}]}]}`))
+				Expect(errs).To(BeEmpty(), "internal image ref %q", ref)
+			}
+		})
+
+		It("rejects an external video ref and poster", func() {
+			errs := deck.Validate([]byte(`{"version":1,"theme":"midnight","slides":[{"layout":"single","blocks":[{"type":"video","ref":"https://evil/x.mp4"}]}]}`))
+			Expect(errs).To(ContainElement(ContainSubstring("image ref must be an internal image reference")), "external video ref")
+			errs = deck.Validate([]byte(`{"version":1,"theme":"midnight","slides":[{"layout":"single","blocks":[{"type":"video","ref":"/_/image/x","poster":"https://evil/p.png"}]}]}`))
+			Expect(errs).To(ContainElement(ContainSubstring("image ref must be an internal image reference")), "external video poster")
+		})
+
+		It("accepts an internal video ref and poster", func() {
+			Expect(deck.Validate([]byte(`{"version":1,"theme":"midnight","slides":[{"layout":"single","blocks":[{"type":"video","ref":"/_/image/x","poster":"/api/documents/1/file"}]}]}`))).To(BeEmpty())
+		})
+
+		It("rejects a non-http embed URL", func() {
+			for _, url := range []string{"data:text/html,x", "javascript:alert(1)"} {
+				errs := deck.Validate([]byte(`{"version":1,"theme":"midnight","slides":[{"layout":"single","blocks":[{"type":"embed","url":"` + url + `"}]}]}`))
+				Expect(errs).To(ContainElement(ContainSubstring("embed url must be an http(s) URL")), "embed url %q", url)
+			}
+		})
+
+		It("accepts an http embed URL", func() {
+			Expect(deck.Validate([]byte(`{"version":1,"theme":"midnight","slides":[{"layout":"single","blocks":[{"type":"embed","url":"https://ok.example/"}]}]}`))).To(BeEmpty())
+		})
+	})
+
+	It("counts the slides of an outline", func() {
+		cases := map[string]int{
+			"Slide 1: Intro\n- a\nSlide 2: Next\n- b":            2,
+			"## Slide 1: Intro\n**Slide 2:** Next\n> slide 3: x": 3,
+			"Il deck è stato generato con successo.":             0,
+			"": 0,
+			"see the slide 2: inline, not a line start": 0,
+		}
+		for in, want := range cases {
+			Expect(deck.CountOutlineSlides(in)).To(Equal(want), "CountOutlineSlides(%q)", in)
+		}
+	})
+})

addons/office/internal/deck/outline.go

@@ -0,0 +1,14 @@
+package deck
+
+import "regexp"
+
+// slideLine matches a `Slide N:` outline line. Leading markdown noise is
+// tolerated: models routinely emit "## Slide 1:" or "**Slide 1:**". The web
+// client uses the same pattern to find the slides of an outline.
+var slideLine = regexp.MustCompile(`(?im)^[\s*#>-]*slide\s+\d+\s*:`)
+
+// CountOutlineSlides counts `Slide N:` lines. 0 means the text isn't shaped
+// like a slide outline (e.g. a one-line "the deck is ready" sign-off).
+func CountOutlineSlides(text string) int {
+	return len(slideLine.FindAllStringIndex(text, -1))
+}

addons/office/internal/deck/pptx/application_test.go

@@ -0,0 +1,44 @@
+package pptx
+
+import (
+	"archive/zip"
+	"bytes"
+	"io"
+
+	"github.com/mudler/LocalAI/addons/office/internal/deck"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// appProps writes a one-slide deck and returns docProps/app.xml from it.
+func appProps(application string) string {
+	GinkgoHelper()
+	slides := []deck.StudioSlide{{Layout: "title", Title: "Hello"}}
+	var buf bytes.Buffer
+	Expect(Write(&buf, slides, Options{Title: "Hello", Application: application}, nil)).To(Succeed())
+	zr, err := zip.NewReader(bytes.NewReader(buf.Bytes()), int64(buf.Len()))
+	Expect(err).NotTo(HaveOccurred())
+	for _, f := range zr.File {
+		if f.Name != "docProps/app.xml" {
+			continue
+		}
+		rc, err := f.Open()
+		Expect(err).NotTo(HaveOccurred())
+		defer func() { _ = rc.Close() }()
+		b, err := io.ReadAll(rc)
+		Expect(err).NotTo(HaveOccurred())
+		return string(b)
+	}
+	Fail("docProps/app.xml missing")
+	return ""
+}
+
+var _ = Describe("application name", func() {
+	It("is the caller's name", func() {
+		Expect(appProps("Example App")).To(ContainSubstring("<Application>Example App</Application>"), "app.xml does not carry the caller's name")
+	})
+
+	It("is omitted when empty", func() {
+		Expect(appProps("")).NotTo(ContainSubstring("<Application>"), "app.xml names an application without a caller value")
+	})
+})

addons/office/internal/deck/pptx/package.go

@@ -0,0 +1,311 @@
+package pptx
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/mudler/LocalAI/addons/office/internal/ooxml"
+)
+
+// XML namespaces and relationship types.
+const (
+	xmlHeader = `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>` + "\n"
+	nsA       = `http://schemas.openxmlformats.org/drawingml/2006/main`
+	nsR       = `http://schemas.openxmlformats.org/officeDocument/2006/relationships`
+	nsP       = `http://schemas.openxmlformats.org/presentationml/2006/main`
+	nsDecl    = `xmlns:a="` + nsA + `" xmlns:r="` + nsR + `" xmlns:p="` + nsP + `"`
+
+	relBase        = `http://schemas.openxmlformats.org/officeDocument/2006/relationships/`
+	relOfficeDoc   = relBase + "officeDocument"
+	relCoreProps   = `http://schemas.openxmlformats.org/package/2006/relationships/metadata/core-properties`
+	relExtProps    = relBase + "extended-properties"
+	relSlideMaster = relBase + "slideMaster"
+	relSlideLayout = relBase + "slideLayout"
+	relTheme       = relBase + "theme"
+	relSlide       = relBase + "slide"
+	relNotesMaster = relBase + "notesMaster"
+	relNotesSlide  = relBase + "notesSlide"
+	relImage       = relBase + "image"
+	relPresProps   = relBase + "presProps"
+	relViewProps   = relBase + "viewProps"
+	relTableStyles = relBase + "tableStyles"
+
+	ctBase        = `application/vnd.openxmlformats-officedocument.`
+	ctPresMain    = ctBase + "presentationml.presentation.main+xml"
+	ctSlideMaster = ctBase + "presentationml.slideMaster+xml"
+	ctSlideLayout = ctBase + "presentationml.slideLayout+xml"
+	ctSlide       = ctBase + "presentationml.slide+xml"
+	ctNotesMaster = ctBase + "presentationml.notesMaster+xml"
+	ctNotesSlide  = ctBase + "presentationml.notesSlide+xml"
+	ctTheme       = ctBase + "theme+xml"
+	ctPresProps   = ctBase + "presentationml.presProps+xml"
+	ctViewProps   = ctBase + "presentationml.viewProps+xml"
+	ctTableStyles = ctBase + "presentationml.tableStyles+xml"
+	ctCoreProps   = `application/vnd.openxmlformats-package.core-properties+xml`
+	ctExtProps    = ctBase + "extended-properties+xml"
+)
+
+// groupHeader opens every shape tree.
+const groupHeader = `<p:nvGrpSpPr><p:cNvPr id="1" name=""/><p:cNvGrpSpPr/><p:nvPr/></p:nvGrpSpPr>` +
+	`<p:grpSpPr><a:xfrm><a:off x="0" y="0"/><a:ext cx="0" cy="0"/><a:chOff x="0" y="0"/><a:chExt cx="0" cy="0"/></a:xfrm></p:grpSpPr>`
+
+const clrMapAttrs = `bg1="lt1" tx1="dk1" bg2="lt2" tx2="dk2" accent1="accent1" accent2="accent2" accent3="accent3" ` +
+	`accent4="accent4" accent5="accent5" accent6="accent6" hlink="hlink" folHlink="folHlink"`
+
+func relsXML(rels []rel) []byte {
+	var b strings.Builder
+	b.WriteString(xmlHeader)
+	b.WriteString(`<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">`)
+	for _, r := range rels {
+		fmt.Fprintf(&b, `<Relationship Id="%s" Type="%s" Target="%s"/>`, r.id, r.typ, ooxml.Escape(r.target))
+	}
+	b.WriteString(`</Relationships>`)
+	return []byte(b.String())
+}
+
+// packageParts lays out every part of the package, [Content_Types].xml first.
+func packageParts(opts Options, pal palette, slides []builtSlide, media []media) []part {
+	hasNotes := false
+	for _, s := range slides {
+		if s.notes != "" {
+			hasNotes = true
+		}
+	}
+
+	var ct strings.Builder
+	ct.WriteString(xmlHeader)
+	ct.WriteString(`<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">`)
+	ct.WriteString(`<Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/>`)
+	ct.WriteString(`<Default Extension="xml" ContentType="application/xml"/>`)
+	ct.WriteString(`<Default Extension="png" ContentType="image/png"/>`)
+	ct.WriteString(`<Default Extension="jpeg" ContentType="image/jpeg"/>`)
+	override := func(name, typ string) {
+		fmt.Fprintf(&ct, `<Override PartName="/%s" ContentType="%s"/>`, name, typ)
+	}
+
+	var parts []part
+	add := func(name string, data []byte, typ string) {
+		parts = append(parts, part{name: name, data: data})
+		if typ != "" {
+			override(name, typ)
+		}
+	}
+
+	add("_rels/.rels", relsXML([]rel{
+		{"rId1", relOfficeDoc, "ppt/presentation.xml"},
+		{"rId2", relCoreProps, "docProps/core.xml"},
+		{"rId3", relExtProps, "docProps/app.xml"},
+	}), "")
+	notesCount := 0
+	for _, s := range slides {
+		if s.notes != "" {
+			notesCount++
+		}
+	}
+	add("docProps/core.xml", coreXML(opts.Title), ctCoreProps)
+	add("docProps/app.xml", appXML(len(slides), notesCount, opts.Application), ctExtProps)
+
+	presRels := []rel{
+		{"rId1", relSlideMaster, "slideMasters/slideMaster1.xml"},
+		{"rId2", relTheme, "theme/theme1.xml"},
+		{"rId3", relPresProps, "presProps.xml"},
+		{"rId4", relViewProps, "viewProps.xml"},
+		{"rId5", relTableStyles, "tableStyles.xml"},
+	}
+	notesMasterRID := ""
+	if hasNotes {
+		notesMasterRID = "rId6"
+		presRels = append(presRels, rel{notesMasterRID, relNotesMaster, "notesMasters/notesMaster1.xml"})
+	}
+	slideRIDs := make([]string, len(slides))
+	for i := range slides {
+		slideRIDs[i] = fmt.Sprintf("rId%d", len(presRels)+1)
+		presRels = append(presRels, rel{slideRIDs[i], relSlide, fmt.Sprintf("slides/slide%d.xml", i+1)})
+	}
+	add("ppt/presentation.xml", presentationXML(slideRIDs, notesMasterRID), ctPresMain)
+	add("ppt/_rels/presentation.xml.rels", relsXML(presRels), "")
+	add("ppt/presProps.xml", []byte(xmlHeader+`<p:presentationPr `+nsDecl+`/>`), ctPresProps)
+	add("ppt/viewProps.xml", []byte(xmlHeader+`<p:viewPr `+nsDecl+`/>`), ctViewProps)
+	add("ppt/tableStyles.xml", []byte(xmlHeader+`<a:tblStyleLst xmlns:a="`+nsA+`" def="{5C22544A-7EE6-4342-B048-85BDC9FD1C3A}"/>`), ctTableStyles)
+
+	add("ppt/slideMasters/slideMaster1.xml", slideMasterXML(pal), ctSlideMaster)
+	add("ppt/slideMasters/_rels/slideMaster1.xml.rels", relsXML([]rel{
+		{"rId1", relSlideLayout, "../slideLayouts/slideLayout1.xml"},
+		{"rId2", relTheme, "../theme/theme1.xml"},
+		{"rId3", relSlideLayout, "../slideLayouts/slideLayout2.xml"},
+	}), "")
+	add("ppt/slideLayouts/slideLayout1.xml", []byte(xmlHeader+`<p:sldLayout `+nsDecl+` type="blank" preserve="1">`+
+		`<p:cSld name="Blank"><p:spTree>`+groupHeader+`</p:spTree></p:cSld>`+
+		`<p:clrMapOvr><a:masterClrMapping/></p:clrMapOvr></p:sldLayout>`), ctSlideLayout)
+	add("ppt/slideLayouts/_rels/slideLayout1.xml.rels", relsXML([]rel{
+		{"rId1", relSlideMaster, "../slideMasters/slideMaster1.xml"},
+	}), "")
+	add("ppt/slideLayouts/slideLayout2.xml", titleLayoutXML(), ctSlideLayout)
+	add("ppt/slideLayouts/_rels/slideLayout2.xml.rels", relsXML([]rel{
+		{"rId1", relSlideMaster, "../slideMasters/slideMaster1.xml"},
+	}), "")
+	add("ppt/theme/theme1.xml", themeXML("Studio", pal), ctTheme)
+	if hasNotes {
+		add("ppt/theme/theme2.xml", themeXML("Notes", paletteFor("plain")), ctTheme)
+		add("ppt/notesMasters/notesMaster1.xml", notesMasterXML(), ctNotesMaster)
+		add("ppt/notesMasters/_rels/notesMaster1.xml.rels", relsXML([]rel{
+			{"rId1", relTheme, "../theme/theme2.xml"},
+		}), "")
+	}
+
+	notesN := 0
+	for i, s := range slides {
+		n := i + 1
+		layout := "../slideLayouts/slideLayout1.xml"
+		if s.titled {
+			layout = "../slideLayouts/slideLayout2.xml"
+		}
+		rels := append([]rel{{"rId1", relSlideLayout, layout}}, s.rels...)
+		if s.notes != "" {
+			notesN++
+			rels = append(rels, rel{fmt.Sprintf("rId%d", len(rels)+1), relNotesSlide, fmt.Sprintf("../notesSlides/notesSlide%d.xml", notesN)})
+			add(fmt.Sprintf("ppt/notesSlides/notesSlide%d.xml", notesN), notesSlideXML(s.notes), ctNotesSlide)
+			add(fmt.Sprintf("ppt/notesSlides/_rels/notesSlide%d.xml.rels", notesN), relsXML([]rel{
+				{"rId1", relNotesMaster, "../notesMasters/notesMaster1.xml"},
+				{"rId2", relSlide, fmt.Sprintf("../slides/slide%d.xml", n)},
+			}), "")
+		}
+		add(fmt.Sprintf("ppt/slides/slide%d.xml", n), []byte(s.xml), ctSlide)
+		add(fmt.Sprintf("ppt/slides/_rels/slide%d.xml.rels", n), relsXML(rels), "")
+	}
+	for _, m := range media {
+		add("ppt/media/"+m.name, m.data, "")
+	}
+
+	ct.WriteString(`</Types>`)
+	return append([]part{{name: "[Content_Types].xml", data: []byte(ct.String())}}, parts...)
+}
+
+func coreXML(title string) []byte {
+	return []byte(xmlHeader + `<cp:coreProperties xmlns:cp="http://schemas.openxmlformats.org/package/2006/metadata/core-properties" ` +
+		`xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dcterms="http://purl.org/dc/terms/" ` +
+		`xmlns:dcmitype="http://purl.org/dc/dcmitype/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">` +
+		`<dc:title>` + ooxml.Escape(title) + `</dc:title></cp:coreProperties>`)
+}
+
+func appXML(slides, notes int, application string) []byte {
+	var app string
+	if application != "" {
+		app = "<Application>" + ooxml.Escape(application) + "</Application>"
+	}
+	return fmt.Appendf(nil, xmlHeader+`<Properties xmlns="http://schemas.openxmlformats.org/officeDocument/2006/extended-properties" `+
+		`xmlns:vt="http://schemas.openxmlformats.org/officeDocument/2006/docPropsVTypes">`+
+		`%s<PresentationFormat>Widescreen</PresentationFormat>`+
+		`<Slides>%d</Slides><Notes>%d</Notes></Properties>`, app, slides, notes)
+}
+
+func presentationXML(slideRIDs []string, notesMasterRID string) []byte {
+	var b strings.Builder
+	b.WriteString(xmlHeader)
+	b.WriteString(`<p:presentation ` + nsDecl + ` saveSubsetFonts="1">`)
+	b.WriteString(`<p:sldMasterIdLst><p:sldMasterId id="2147483648" r:id="rId1"/></p:sldMasterIdLst>`)
+	if notesMasterRID != "" {
+		b.WriteString(`<p:notesMasterIdLst><p:notesMasterId r:id="` + notesMasterRID + `"/></p:notesMasterIdLst>`)
+	}
+	if len(slideRIDs) > 0 {
+		b.WriteString(`<p:sldIdLst>`)
+		for i, rid := range slideRIDs {
+			fmt.Fprintf(&b, `<p:sldId id="%d" r:id="%s"/>`, firstSlideID+i, rid)
+		}
+		b.WriteString(`</p:sldIdLst>`)
+	}
+	fmt.Fprintf(&b, `<p:sldSz cx="%d" cy="%d"/><p:notesSz cx="6858000" cy="9144000"/>`, slideW, slideH)
+	b.WriteString(`<p:defaultTextStyle><a:defPPr><a:defRPr lang="en-US"/></a:defPPr></p:defaultTextStyle>`)
+	b.WriteString(`</p:presentation>`)
+	return []byte(b.String())
+}
+
+// titleLayoutXML is the Title and Content layout: a title placeholder and an
+// object placeholder (idx 1) for the body. Both have their place on the slide
+// so that a slide added from the layout in an editor is laid out like ours.
+func titleLayoutXML() []byte {
+	return []byte(xmlHeader + `<p:sldLayout ` + nsDecl + ` type="obj" preserve="1">` +
+		`<p:cSld name="Title and Content"><p:spTree>` + groupHeader +
+		`<p:sp><p:nvSpPr><p:cNvPr id="2" name="Title 1"/><p:cNvSpPr><a:spLocks noGrp="1"/></p:cNvSpPr><p:nvPr>` + phTitle + `</p:nvPr></p:nvSpPr>` +
+		`<p:spPr>` + xfrm(padSide, padTop, frameW, 12, "") + `</p:spPr>` +
+		`<p:txBody><a:bodyPr anchor="b"/><a:lstStyle/><a:p><a:r><a:rPr lang="en-US"/><a:t>Click to edit the title</a:t></a:r><a:endParaRPr lang="en-US"/></a:p></p:txBody></p:sp>` +
+		`<p:sp><p:nvSpPr><p:cNvPr id="3" name="Content Placeholder 2"/><p:cNvSpPr><a:spLocks noGrp="1"/></p:cNvSpPr><p:nvPr>` + phBody + `</p:nvPr></p:nvSpPr>` +
+		`<p:spPr>` + xfrm(padSide, padTop+14, frameW, frameBot-padTop-14, "") + `</p:spPr>` +
+		`<p:txBody><a:bodyPr/><a:lstStyle/><a:p><a:pPr lvl="0"/><a:r><a:rPr lang="en-US"/><a:t>Click to edit the text</a:t></a:r><a:endParaRPr lang="en-US"/></a:p></p:txBody></p:sp>` +
+		`</p:spTree></p:cSld><p:clrMapOvr><a:masterClrMapping/></p:clrMapOvr></p:sldLayout>`)
+}
+
+func slideMasterXML(pal palette) []byte {
+	return []byte(xmlHeader + `<p:sldMaster ` + nsDecl + `>` +
+		`<p:cSld><p:bg><p:bgPr><a:solidFill><a:srgbClr val="` + pal.bg + `"/></a:solidFill><a:effectLst/></p:bgPr></p:bg>` +
+		`<p:spTree>` + groupHeader + `</p:spTree></p:cSld>` +
+		`<p:clrMap ` + clrMapAttrs + `/>` +
+		`<p:sldLayoutIdLst><p:sldLayoutId id="2147483649" r:id="rId1"/><p:sldLayoutId id="2147483650" r:id="rId3"/></p:sldLayoutIdLst>` +
+		`<p:txStyles>` +
+		`<p:titleStyle><a:lvl1pPr><a:defRPr sz="4400"><a:solidFill><a:schemeClr val="tx1"/></a:solidFill><a:latin typeface="+mj-lt"/></a:defRPr></a:lvl1pPr></p:titleStyle>` +
+		`<p:bodyStyle><a:lvl1pPr><a:defRPr sz="2400"><a:solidFill><a:schemeClr val="tx1"/></a:solidFill><a:latin typeface="+mn-lt"/></a:defRPr></a:lvl1pPr></p:bodyStyle>` +
+		`<p:otherStyle><a:lvl1pPr><a:defRPr sz="1800"><a:solidFill><a:schemeClr val="tx1"/></a:solidFill><a:latin typeface="+mn-lt"/></a:defRPr></a:lvl1pPr></p:otherStyle>` +
+		`</p:txStyles></p:sldMaster>`)
+}
+
+func themeXML(name string, pal palette) []byte {
+	clr := func(tag, val string) string { return `<a:` + tag + `><a:srgbClr val="` + val + `"/></a:` + tag + `>` }
+	fill := `<a:solidFill><a:schemeClr val="phClr"/></a:solidFill>`
+	line := `<a:ln w="9525"><a:solidFill><a:schemeClr val="phClr"/></a:solidFill></a:ln>`
+	effect := `<a:effectStyle><a:effectLst/></a:effectStyle>`
+	font := func(tag string) string {
+		return `<a:` + tag + `><a:latin typeface="` + ooxml.Escape(pal.font) + `"/><a:ea typeface=""/><a:cs typeface=""/></a:` + tag + `>`
+	}
+	return []byte(xmlHeader + `<a:theme xmlns:a="` + nsA + `" name="` + name + `"><a:themeElements>` +
+		`<a:clrScheme name="` + name + `">` +
+		clr("dk1", pal.ink) + clr("lt1", pal.bg) + clr("dk2", pal.muted) + clr("lt2", pal.wash) +
+		clr("accent1", pal.accent) + clr("accent2", pal.muted) + clr("accent3", pal.wash) +
+		clr("accent4", pal.ink) + clr("accent5", pal.accent) + clr("accent6", pal.muted) +
+		clr("hlink", pal.accent) + clr("folHlink", pal.muted) +
+		`</a:clrScheme>` +
+		`<a:fontScheme name="` + name + `">` + font("majorFont") + font("minorFont") + `</a:fontScheme>` +
+		`<a:fmtScheme name="` + name + `">` +
+		`<a:fillStyleLst>` + fill + fill + fill + `</a:fillStyleLst>` +
+		`<a:lnStyleLst>` + line + line + line + `</a:lnStyleLst>` +
+		`<a:effectStyleLst>` + effect + effect + effect + `</a:effectStyleLst>` +
+		`<a:bgFillStyleLst>` + fill + fill + fill + `</a:bgFillStyleLst>` +
+		`</a:fmtScheme></a:themeElements><a:objectDefaults/><a:extraClrSchemeLst/></a:theme>`)
+}
+
+func notesMasterXML() []byte {
+	return []byte(xmlHeader + `<p:notesMaster ` + nsDecl + `>` +
+		`<p:cSld><p:bg><p:bgRef idx="1001"><a:schemeClr val="bg1"/></p:bgRef></p:bg><p:spTree>` + groupHeader +
+		`<p:sp><p:nvSpPr><p:cNvPr id="2" name="Slide Image Placeholder 1"/><p:cNvSpPr><a:spLocks noGrp="1" noRot="1" noChangeAspect="1"/></p:cNvSpPr>` +
+		`<p:nvPr><p:ph type="sldImg" idx="2"/></p:nvPr></p:nvSpPr>` +
+		`<p:spPr><a:xfrm><a:off x="685800" y="1143000"/><a:ext cx="5486400" cy="3086100"/></a:xfrm>` +
+		`<a:prstGeom prst="rect"><a:avLst/></a:prstGeom><a:noFill/><a:ln w="12700"><a:solidFill><a:prstClr val="black"/></a:solidFill></a:ln></p:spPr></p:sp>` +
+		`<p:sp><p:nvSpPr><p:cNvPr id="3" name="Notes Placeholder 2"/><p:cNvSpPr><a:spLocks noGrp="1"/></p:cNvSpPr>` +
+		`<p:nvPr><p:ph type="body" sz="quarter" idx="3"/></p:nvPr></p:nvSpPr>` +
+		`<p:spPr><a:xfrm><a:off x="685800" y="4400550"/><a:ext cx="5486400" cy="3600450"/></a:xfrm><a:prstGeom prst="rect"><a:avLst/></a:prstGeom></p:spPr>` +
+		`<p:txBody><a:bodyPr vert="horz" lIns="91440" tIns="45720" rIns="91440" bIns="45720" rtlCol="0"/><a:lstStyle/>` +
+		`<a:p><a:pPr lvl="0"/><a:r><a:rPr lang="en-US"/><a:t>Notes</a:t></a:r></a:p></p:txBody></p:sp>` +
+		`</p:spTree></p:cSld>` +
+		`<p:clrMap ` + clrMapAttrs + `/>` +
+		`<p:notesStyle><a:lvl1pPr marL="0" algn="l" defTabSz="914400" rtl="0" eaLnBrk="1" latinLnBrk="0" hangingPunct="1">` +
+		`<a:defRPr sz="1200" kern="1200"><a:solidFill><a:schemeClr val="tx1"/></a:solidFill><a:latin typeface="+mn-lt"/></a:defRPr></a:lvl1pPr></p:notesStyle>` +
+		`</p:notesMaster>`)
+}
+
+func notesSlideXML(notes string) []byte {
+	var paras strings.Builder
+	for line := range strings.SplitSeq(notes, "\n") {
+		line = strings.TrimRight(line, "\r")
+		if line == "" {
+			paras.WriteString(`<a:p><a:endParaRPr lang="en-US"/></a:p>`)
+			continue
+		}
+		paras.WriteString(`<a:p><a:r><a:rPr lang="en-US" dirty="0"/><a:t>` + ooxml.Escape(line) + `</a:t></a:r></a:p>`)
+	}
+	return []byte(xmlHeader + `<p:notes ` + nsDecl + `><p:cSld><p:spTree>` + groupHeader +
+		`<p:sp><p:nvSpPr><p:cNvPr id="2" name="Slide Image Placeholder 1"/><p:cNvSpPr><a:spLocks noGrp="1" noRot="1" noChangeAspect="1"/></p:cNvSpPr>` +
+		`<p:nvPr><p:ph type="sldImg"/></p:nvPr></p:nvSpPr><p:spPr/></p:sp>` +
+		`<p:sp><p:nvSpPr><p:cNvPr id="3" name="Notes Placeholder 2"/><p:cNvSpPr><a:spLocks noGrp="1"/></p:cNvSpPr>` +
+		`<p:nvPr><p:ph type="body" idx="1"/></p:nvPr></p:nvSpPr><p:spPr/>` +
+		`<p:txBody><a:bodyPr/><a:lstStyle/>` + paras.String() + `</p:txBody></p:sp>` +
+		`</p:spTree></p:cSld><p:clrMapOvr><a:masterClrMapping/></p:clrMapOvr></p:notes>`)
+}

addons/office/internal/deck/pptx/pptx.go

@@ -0,0 +1,134 @@
+// Package pptx writes studio slides as a PowerPoint (.pptx) file.
+//
+// The output is a plain OOXML package built with archive/zip and text
+// templates: one slide master, two layouts (Title and Content, and Blank), one
+// theme, and one slide per finished studio slide. A slide with a title uses
+// the Title and Content layout: its title is the title placeholder and its
+// bullets or subtitle are the body placeholder. A slide without a title uses
+// Blank. Every text is a real, editable text box. Charts
+// are drawn with native shapes (bars as rectangles, lines as connectors), so
+// the receiver can still edit them. Images come from an ImageLoader; an
+// image that cannot be loaded leaves a captioned placeholder.
+package pptx
+
+import (
+	"archive/zip"
+	"fmt"
+	"io"
+	"strings"
+
+	"github.com/mudler/LocalAI/addons/office/internal/deck"
+)
+
+// SourceKind is how well a line of the sources slide is supported.
+type SourceKind string
+
+const (
+	SourceOK       SourceKind = "ok"
+	SourceWeak     SourceKind = "weak"
+	SourceNoSource SourceKind = "nosrc"
+	SourceOwn      SourceKind = "own"
+)
+
+// SourceLine is one line of the closing sources slide.
+type SourceLine struct {
+	Label  string
+	Detail string
+	Kind   SourceKind
+}
+
+// Options controls the export.
+type Options struct {
+	Title string
+	// Application is written to docProps/app.xml as the producing application.
+	// Empty means the element is left out. The library has no default name.
+	Application  string
+	Style        deck.StudioStyle
+	Notes        bool
+	SourcesSlide bool
+	Sources      []SourceLine
+}
+
+// ImageLoader returns the bytes and the media type of an image reference.
+// Only JPEG and PNG images are embedded.
+type ImageLoader func(ref string) ([]byte, string, error)
+
+// Slide size, 16:9, in EMU.
+const (
+	slideW = 12192000
+	slideH = 6858000
+)
+
+// firstSlideID is the lowest id PowerPoint accepts in sldIdLst.
+const firstSlideID = 256
+
+// part is one file of the package.
+type part struct {
+	name string
+	data []byte
+}
+
+// builtSlide is one slide with its relationships and optional notes. titled
+// is true when the slide uses the Title and Content layout.
+type builtSlide struct {
+	xml    string
+	rels   []rel
+	notes  string
+	titled bool
+}
+
+// rel is one relationship of a part.
+type rel struct {
+	id     string
+	typ    string
+	target string
+}
+
+// media is one embedded image.
+type media struct {
+	name string // e.g. image1.png
+	ext  string // png or jpeg
+	data []byte
+}
+
+// Write writes the finished slides (status ok) as a .pptx package to w.
+// Queued, working and failed slides are skipped. load may be nil, in which
+// case every image becomes a placeholder.
+func Write(w io.Writer, slides []deck.StudioSlide, opts Options, load ImageLoader) error {
+	pal := paletteFor(opts.Style)
+	var media []media
+	var built []builtSlide
+	for _, s := range slides {
+		if s.Status != deck.SlideOK {
+			continue
+		}
+		b := newSlideBuilder(pal, load, &media)
+		b.studioSlide(s)
+		bs := b.finish()
+		if opts.Notes {
+			bs.notes = strings.TrimSpace(s.Notes)
+		}
+		built = append(built, bs)
+	}
+	if opts.SourcesSlide {
+		b := newSlideBuilder(pal, load, &media)
+		b.sourcesSlide(opts.Sources)
+		built = append(built, b.finish())
+	}
+
+	parts := packageParts(opts, pal, built, media)
+	zw := zip.NewWriter(w)
+	for _, p := range parts {
+		f, err := zw.Create(p.name)
+		if err != nil {
+			return fmt.Errorf("pptx: create %s: %w", p.name, err)
+		}
+		if _, err := f.Write(p.data); err != nil {
+			return fmt.Errorf("pptx: write %s: %w", p.name, err)
+		}
+	}
+	if err := zw.Close(); err != nil {
+		return fmt.Errorf("pptx: close: %w", err)
+	}
+	return nil
+}

addons/office/internal/deck/pptx/pptx_suite_test.go

@@ -0,0 +1,13 @@
+package pptx
+
+import (
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestPptx(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "Pptx Suite")
+}

addons/office/internal/deck/pptx/pptx_test.go

@@ -0,0 +1,297 @@
+package pptx
+
+import (
+	"archive/zip"
+	"bytes"
+	"encoding/xml"
+	"errors"
+	"image"
+	"image/color"
+	"image/png"
+	"io"
+	"regexp"
+	"slices"
+	"strconv"
+	"strings"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+
+	"github.com/mudler/LocalAI/addons/office/internal/deck"
+	"github.com/mudler/LocalAI/addons/office/internal/officecheck"
+	"github.com/mudler/LocalAI/addons/office/internal/pptxedit"
+)
+
+func tinyPNG() []byte {
+	GinkgoHelper()
+	img := image.NewRGBA(image.Rect(0, 0, 4, 3))
+	img.Set(1, 1, color.RGBA{R: 255, A: 255})
+	var buf bytes.Buffer
+	Expect(png.Encode(&buf, img)).To(Succeed())
+	return buf.Bytes()
+}
+
+// unzipParts writes the deck and returns its parts by name. Every XML part
+// must parse.
+func unzipParts(slides []deck.StudioSlide, opts Options, load ImageLoader) map[string][]byte {
+	GinkgoHelper()
+	var buf bytes.Buffer
+	Expect(Write(&buf, slides, opts, load)).To(Succeed(), "Write")
+	zr, err := zip.NewReader(bytes.NewReader(buf.Bytes()), int64(buf.Len()))
+	Expect(err).NotTo(HaveOccurred(), "not a zip")
+	parts := map[string][]byte{}
+	for _, f := range zr.File {
+		rc, err := f.Open()
+		Expect(err).NotTo(HaveOccurred())
+		b, err := io.ReadAll(rc)
+		_ = rc.Close()
+		Expect(err).NotTo(HaveOccurred())
+		Expect(parts).NotTo(HaveKey(f.Name), "duplicate part")
+		parts[f.Name] = b
+		if strings.HasSuffix(f.Name, ".xml") || strings.HasSuffix(f.Name, ".rels") {
+			dec := xml.NewDecoder(bytes.NewReader(b))
+			for {
+				_, err := dec.Token()
+				if errors.Is(err, io.EOF) {
+					break
+				}
+				Expect(err).NotTo(HaveOccurred(), "part %s does not parse:\n%s", f.Name, b)
+			}
+		}
+	}
+	Expect(zr.File[0].Name).To(Equal("[Content_Types].xml"), "first part")
+	return parts
+}
+
+func countPrefix(parts map[string][]byte, re *regexp.Regexp) int {
+	n := 0
+	for name := range parts {
+		if re.MatchString(name) {
+			n++
+		}
+	}
+	return n
+}
+
+var (
+	slidePartRe = regexp.MustCompile(`^ppt/slides/slide\d+\.xml$`)
+	notesPartRe = regexp.MustCompile(`^ppt/notesSlides/notesSlide\d+\.xml$`)
+	mediaPartRe = regexp.MustCompile(`^ppt/media/image\d+\.(png|jpeg)$`)
+)
+
+func sampleSlides() []deck.StudioSlide {
+	return []deck.StudioSlide{
+		{ID: "s1", Title: "Q3 review", Sub: "Partners & <friends>", Layout: deck.LayoutTitle, Status: deck.SlideOK, Notes: "Open with the headline.\nThen pause."},
+		{ID: "s2", Title: "What \"we\" heard", Layout: deck.LayoutText, Status: deck.SlideOK, Bullets: []deck.StudioBullet{
+			{Text: "Revenue grew 12% <fast> & steady", Evidence: []string{"e1"}},
+			{Text: "Teams like it", Flag: deck.FlagNoSource},
+			{Text: "Churn is low", Flag: deck.FlagWeak},
+		}},
+		{ID: "s3", Title: "Queued", Layout: deck.LayoutText, Status: deck.SlideQueued},
+		{ID: "s4", Title: "Broken", Layout: deck.LayoutText, Status: deck.SlideFailed},
+		{ID: "s5", Title: "Big one", Layout: deck.LayoutNumber, Status: deck.SlideOK, Big: &deck.BigNumber{Num: "42%", Label: "of users came back"}},
+		{ID: "s6", Title: "By office", Layout: deck.LayoutChart, Status: deck.SlideOK, Chart: &deck.StudioChart{
+			Kind: deck.ChartBar, Label: "Cost by office", Unit: "%", Data: []deck.ChartPoint{{K: "Rome", V: 12}, {K: "Milan", V: 49}},
+		}},
+		{ID: "s7", Title: "Trend", Layout: deck.LayoutChart, Status: deck.SlideOK, Chart: &deck.StudioChart{
+			Kind: deck.ChartLine, Label: "Users", Data: []deck.ChartPoint{{K: "Jan", V: 3}, {K: "Feb", V: 9}, {K: "Mar", V: 5}},
+		}},
+		{ID: "s8", Title: "Whiteboard", Layout: deck.LayoutImage, Status: deck.SlideOK, Notes: "Point at the board.",
+			Image: &deck.StudioImage{Ref: "/api/documents/a/file", Label: "Board photo", Weak: true}},
+		{ID: "s9", Title: "Missing", Layout: deck.LayoutImage, Status: deck.SlideOK,
+			Image: &deck.StudioImage{Ref: "/api/documents/b/file", Label: "Lost frame"}},
+	}
+}
+
+// shapeWith returns the p:sp element that holds the placeholder element ph.
+func shapeWith(xmlText, ph string) string {
+	GinkgoHelper()
+	i := strings.Index(xmlText, ph)
+	Expect(i).To(BeNumerically(">=", 0), "no %s in %s", ph, xmlText)
+	start := strings.LastIndex(xmlText[:i], "<p:sp>")
+	end := strings.Index(xmlText[i:], "</p:sp>")
+	Expect(start).To(BeNumerically(">=", 0), "%s is not inside a p:sp", ph)
+	Expect(end).To(BeNumerically(">=", 0), "%s is not inside a p:sp", ph)
+	return xmlText[start : i+end]
+}
+
+func contains(lines []string, s string) bool {
+	return slices.Contains(lines, s)
+}
+
+var _ = Describe("Write", func() {
+	It("writes the parts of a deck", func() {
+		pic := tinyPNG()
+		var asked []string
+		load := func(ref string) ([]byte, string, error) {
+			asked = append(asked, ref)
+			if ref == "/api/documents/a/file" {
+				return pic, "image/png", nil
+			}
+			return nil, "", errors.New("gone")
+		}
+		for _, style := range []deck.StudioStyle{deck.StylePaper, deck.StyleNight, deck.StyleStudio, deck.StylePlain} {
+			parts := unzipParts(sampleSlides(), Options{Title: "Q3 & <more>", Style: style}, load)
+			for _, name := range []string{
+				"[Content_Types].xml", "_rels/.rels", "docProps/app.xml", "docProps/core.xml",
+				"ppt/presentation.xml", "ppt/_rels/presentation.xml.rels",
+				"ppt/slideMasters/slideMaster1.xml", "ppt/slideMasters/_rels/slideMaster1.xml.rels",
+				"ppt/slideLayouts/slideLayout1.xml", "ppt/slideLayouts/_rels/slideLayout1.xml.rels",
+				"ppt/slideLayouts/slideLayout2.xml", "ppt/slideLayouts/_rels/slideLayout2.xml.rels",
+				"ppt/theme/theme1.xml",
+			} {
+				Expect(parts).To(HaveKey(name), "%s: missing part", style)
+			}
+			Expect(countPrefix(parts, slidePartRe)).To(Equal(7), "%s: slides (ok slides only)", style)
+			Expect(countPrefix(parts, notesPartRe)).To(Equal(0), "%s: notes slides without Notes", style)
+			Expect(parts).NotTo(HaveKey("ppt/notesMasters/notesMaster1.xml"), "%s: notes master without Notes", style)
+			Expect(countPrefix(parts, mediaPartRe)).To(Equal(1), "%s: media parts", style)
+			pres := string(parts["ppt/presentation.xml"])
+			Expect(pres).To(ContainSubstring(`<p:sldSz cx="12192000" cy="6858000"`), "%s: not 16:9", style)
+			Expect(pres).To(ContainSubstring(`<p:sldId id="256"`), "%s: slide ids must start at 256", style)
+			ct := string(parts["[Content_Types].xml"])
+			for i := 1; i <= 7; i++ {
+				Expect(ct).To(ContainSubstring(`PartName="/ppt/slides/slide`+strconv.Itoa(i)+`.xml"`), "%s: content type for slide %d missing", style, i)
+			}
+			Expect(ct).To(ContainSubstring(`Extension="png"`), "%s: png content type missing", style)
+			Expect(string(parts["docProps/core.xml"])).To(ContainSubstring("Q3 &amp; &lt;more&gt;"), "%s: title not escaped in core.xml", style)
+		}
+		parts := unzipParts(sampleSlides(), Options{Title: "Q3", Style: deck.StylePaper}, load)
+		s1 := string(parts["ppt/slides/slide1.xml"])
+		Expect(s1).To(ContainSubstring("Q3 review"))
+		Expect(s1).To(ContainSubstring("Partners &amp; &lt;friends&gt;"))
+		s2 := string(parts["ppt/slides/slide2.xml"])
+		for _, want := range []string{"What &#34;we&#34; heard", "Revenue grew 12% &lt;fast&gt; &amp; steady", "Teams like it", `<a:buChar char="•"/>`, "FF6B3D"} {
+			Expect(s2).To(ContainSubstring(want), "text slide")
+		}
+		s3 := string(parts["ppt/slides/slide3.xml"])
+		Expect(s3).To(ContainSubstring("Big one"), "number slide should be third after skipping queued and failed")
+		Expect(s3).To(ContainSubstring("42%"))
+		Expect(s3).To(ContainSubstring("of users came back"))
+		bar := string(parts["ppt/slides/slide4.xml"])
+		Expect(bar).To(ContainSubstring("Rome"))
+		Expect(bar).To(ContainSubstring("49%"))
+		Expect(strings.Count(bar, `prst="rect"`)).To(BeNumerically(">=", 2), "bar chart shapes")
+		line := string(parts["ppt/slides/slide5.xml"])
+		Expect(strings.Count(line, "<p:cxnSp>")).To(Equal(2), "line chart connectors")
+		Expect(line).To(ContainSubstring("Feb"))
+		img := string(parts["ppt/slides/slide6.xml"])
+		Expect(img).To(ContainSubstring("<p:pic>"))
+		Expect(img).To(ContainSubstring("Board photo (low confidence)"))
+		rels := string(parts["ppt/slides/_rels/slide6.xml.rels"])
+		Expect(rels).To(ContainSubstring("../media/image1.png"), "image rels")
+		ph := string(parts["ppt/slides/slide7.xml"])
+		Expect(ph).NotTo(ContainSubstring("<p:pic>"), "placeholder slide")
+		Expect(ph).To(ContainSubstring("Lost frame"))
+		Expect(ph).To(ContainSubstring(`prstDash val="dash"`))
+		Expect(asked).NotTo(BeEmpty(), "loader never called")
+	})
+
+	It("writes notes and a sources slide", func() {
+		sources := []SourceLine{
+			{Label: "Survey p.1", Detail: "Slide 2", Kind: "ok"},
+			{Label: "Interview 0:42", Detail: "Slide 2", Kind: "weak"},
+			{Label: "Teams like it", Detail: "Slide 2", Kind: "nosrc"},
+			{Label: "We lead the market", Detail: "Slide 3", Kind: "own"},
+		}
+		parts := unzipParts(sampleSlides(), Options{Title: "Q3", Style: deck.StyleNight, Notes: true, SourcesSlide: true, Sources: sources}, nil)
+		Expect(countPrefix(parts, slidePartRe)).To(Equal(8), "slides (7 ok + sources)")
+		Expect(parts).To(HaveKey("ppt/notesMasters/notesMaster1.xml"), "notes master missing")
+		Expect(countPrefix(parts, notesPartRe)).To(Equal(2), "notes slides (slides with notes)")
+		n1 := string(parts["ppt/notesSlides/notesSlide1.xml"])
+		Expect(n1).To(ContainSubstring("Open with the headline."))
+		Expect(n1).To(ContainSubstring("Then pause."))
+		Expect(string(parts["ppt/slides/_rels/slide1.xml.rels"])).To(ContainSubstring("../notesSlides/notesSlide1.xml"), "slide 1 does not link its notes")
+		Expect(string(parts["ppt/notesSlides/_rels/notesSlide1.xml.rels"])).To(ContainSubstring("../slides/slide1.xml"), "notes do not link back to slide 1")
+		Expect(string(parts["ppt/presentation.xml"])).To(ContainSubstring("<p:notesMasterIdLst>"), "presentation does not list the notes master")
+		ct := string(parts["[Content_Types].xml"])
+		for _, want := range []string{"/ppt/notesMasters/notesMaster1.xml", "/ppt/notesSlides/notesSlide2.xml", "/ppt/theme/theme2.xml"} {
+			Expect(ct).To(ContainSubstring(want), "content types")
+		}
+		last := string(parts["ppt/slides/slide8.xml"])
+		for _, want := range []string{"Sources", "Survey p.1", "Interview 0:42", "(low confidence)", "Teams like it", "(no source)", "We lead the market"} {
+			Expect(last).To(ContainSubstring(want), "sources slide")
+		}
+		// A nil loader leaves placeholders for both images.
+		Expect(countPrefix(parts, mediaPartRe)).To(Equal(0), "media parts with a nil loader")
+	})
+
+	It("rejects bad image bytes", func() {
+		slides := []deck.StudioSlide{{ID: "x", Title: "Pic", Layout: deck.LayoutImage, Status: deck.SlideOK,
+			Image: &deck.StudioImage{Ref: "/api/documents/a/file", Label: "Not a picture"}}}
+		load := func(string) ([]byte, string, error) { return []byte("<svg/>"), "image/png", nil }
+		parts := unzipParts(slides, Options{Title: "T", Style: deck.StylePlain}, load)
+		Expect(countPrefix(parts, mediaPartRe)).To(Equal(0), "bad bytes embedded")
+		Expect(string(parts["ppt/slides/slide1.xml"])).To(ContainSubstring("Not a picture"), "placeholder caption missing")
+	})
+
+	It("writes an empty deck", func() {
+		parts := unzipParts(nil, Options{Title: "", Style: ""}, nil)
+		Expect(countPrefix(parts, slidePartRe)).To(Equal(0), "slides for an empty deck")
+		Expect(string(parts["ppt/presentation.xml"])).To(ContainSubstring("<p:sldSz"), "presentation incomplete")
+	})
+
+	It("writes title and body placeholders", func() {
+		slides := []deck.StudioSlide{
+			{ID: "a", Title: "Titled slide", Layout: deck.LayoutText, Status: deck.SlideOK, Bullets: []deck.StudioBullet{
+				{Text: "First point"}, {Text: "Second point"},
+			}},
+			{ID: "b", Title: "", Layout: deck.LayoutText, Status: deck.SlideOK, Bullets: []deck.StudioBullet{{Text: "Untitled point"}}},
+		}
+		parts := unzipParts(slides, Options{Title: "T", Style: deck.StylePaper}, nil)
+
+		layout2 := string(parts["ppt/slideLayouts/slideLayout2.xml"])
+		for _, want := range []string{`type="obj"`, `name="Title and Content"`, phTitle, phBody} {
+			Expect(layout2).To(ContainSubstring(want), "Title and Content layout")
+		}
+		Expect(string(parts["ppt/slideLayouts/slideLayout1.xml"])).To(ContainSubstring(`name="Blank"`), "Blank layout lost")
+		Expect(strings.Count(string(parts["ppt/slideMasters/slideMaster1.xml"]), "<p:sldLayoutId ")).To(Equal(2), "layouts listed by the master")
+		Expect(string(parts["ppt/slideMasters/_rels/slideMaster1.xml.rels"])).To(ContainSubstring("../slideLayouts/slideLayout2.xml"), "master does not link the Title and Content layout")
+
+		s1 := string(parts["ppt/slides/slide1.xml"])
+		Expect(string(parts["ppt/slides/_rels/slide1.xml.rels"])).To(ContainSubstring("../slideLayouts/slideLayout2.xml"), "titled slide does not use the Title and Content layout")
+		Expect(shapeWith(s1, phTitle)).To(ContainSubstring("Titled slide"), "title placeholder does not hold the title")
+		body := shapeWith(s1, phBody)
+		Expect(body).To(ContainSubstring("First point"))
+		Expect(body).To(ContainSubstring("Second point"))
+		Expect(body).To(ContainSubstring(`<a:buChar char="•"/>`))
+		Expect(body).NotTo(ContainSubstring("txBox"), "a placeholder must not be a text box")
+
+		s2 := string(parts["ppt/slides/slide2.xml"])
+		Expect(string(parts["ppt/slides/_rels/slide2.xml.rels"])).To(ContainSubstring("../slideLayouts/slideLayout1.xml"), "untitled slide does not use the Blank layout")
+		Expect(s2).NotTo(ContainSubstring("<p:ph"), "untitled slide has placeholders")
+		Expect(s2).To(ContainSubstring("Untitled point"), "untitled slide lost its bullets")
+	})
+
+	It("gives the sources slide placeholders", func() {
+		parts := unzipParts(sampleSlides(), Options{Title: "Q3", Style: deck.StylePlain, SourcesSlide: true,
+			Sources: []SourceLine{{Label: "Survey p.1", Kind: SourceOK}}}, nil)
+		last := string(parts["ppt/slides/slide8.xml"])
+		Expect(shapeWith(last, phTitle)).To(ContainSubstring("Sources"), "sources title placeholder")
+		Expect(shapeWith(last, phBody)).To(ContainSubstring("Survey p.1"), "sources body placeholder")
+	})
+
+	// The written deck opens with pptxedit: the title can be set, and a slide
+	// can be added from the Title and Content layout.
+	It("writes a deck that pptxedit can edit", func() {
+		var buf bytes.Buffer
+		Expect(Write(&buf, sampleSlides(), Options{Title: "Q3", Style: deck.StyleNight}, nil)).To(Succeed())
+		d, err := pptxedit.Open(buf.Bytes())
+		Expect(err).NotTo(HaveOccurred(), "pptxedit.Open")
+		slides := d.Slides()
+		Expect(slides).To(HaveLen(7))
+		Expect(slides[1].Title).To(Equal(`What "we" heard`))
+		Expect(d.SetTitle(2, "Edited title")).To(Succeed(), "SetTitle")
+		Expect(d.AddSlide("Title and Content", "Added", []string{"New point"})).To(Succeed(), "AddSlide")
+		out, err := d.Bytes()
+		Expect(err).NotTo(HaveOccurred())
+		Expect(officecheck.Check(out)).To(BeEmpty(), "officecheck problems")
+		re, err := pptxedit.Open(out)
+		Expect(err).NotTo(HaveOccurred())
+		got := re.Slides()
+		Expect(got).To(HaveLen(8))
+		Expect(got[1].Title).To(Equal("Edited title"))
+		Expect(got[7].Title).To(Equal("Added"))
+		Expect(contains(got[7].Text, "New point")).To(BeTrue(), "added slide text %q", got[7].Text)
+	})
+})

addons/office/internal/deck/pptx/slide.go

@@ -0,0 +1,616 @@
+package pptx
+
+import (
+	"bytes"
+	"fmt"
+	"image"
+	_ "image/jpeg" // register the JPEG decoder for DecodeConfig
+	_ "image/png"  // register the PNG decoder for DecodeConfig
+	"math"
+	"strconv"
+	"strings"
+	"unicode/utf8"
+
+	"github.com/mudler/LocalAI/addons/office/internal/deck"
+	"github.com/mudler/LocalAI/addons/office/internal/ooxml"
+)
+
+// palette is the content colours of one deck style (the studio's
+// .sl[data-deck=…] rules): background, ink, accent, muted text, wash.
+type palette struct {
+	style  deck.StudioStyle
+	bg     string
+	ink    string
+	accent string
+	muted  string
+	wash   string
+	font   string
+}
+
+var palettes = map[deck.StudioStyle]palette{
+	deck.StylePaper:  {deck.StylePaper, "FFFFFF", "141C2B", "FF6B3D", "5B687D", "FFE9DF", "Bricolage Grotesque"},
+	deck.StyleNight:  {deck.StyleNight, "0F1626", "F2F5FA", "FFB26B", "A9B4C8", "1D2A44", "Bricolage Grotesque"},
+	deck.StyleStudio: {deck.StyleStudio, "FFF0C4", "2A1D00", "B23610", "6A5420", "FFDC85", "Onest"},
+	deck.StylePlain:  {deck.StylePlain, "EEF3FA", "0B1220", "1B4A98", "44505F", "D6E4FA", "Onest"},
+}
+
+// paletteFor returns the palette of style, paper when unknown.
+func paletteFor(style deck.StudioStyle) palette {
+	if p, ok := palettes[style]; ok {
+		return p
+	}
+	return palettes[deck.StylePaper]
+}
+
+// Claim marks, as in the studio editor.
+const (
+	nosrcColor = "E5524B"
+	weakColor  = "D19A1E"
+	white      = "FFFFFF"
+	capFill    = "0A0F19"
+)
+
+// maxImageBytes caps one embedded image.
+const maxImageBytes = 10 << 20
+
+// cqw is one percent of the slide width, the unit of the studio's slide CSS.
+const cqw = slideW / 100
+
+// emu converts container-width percent to EMU.
+func emu(v float64) int64 { return int64(math.Round(v * cqw)) }
+
+// fontSize converts container-width percent to hundredths of a point (the
+// slide is 960 pt wide).
+func fontSize(v float64) int { return int(math.Round(v * 960)) }
+
+// Slide frame, in cqw: padding 6.2 top, 7 sides, 5 bottom.
+const (
+	padTop    = 6.2
+	padSide   = 7.0
+	padBottom = 5.0
+	frameW    = 100 - 2*padSide
+	frameH    = 56.25
+	frameBot  = frameH - padBottom
+	colGap    = 3.5
+	leftColW  = (frameW - colGap) * 1.05 / 2.05
+	rightColX = padSide + leftColW + colGap
+	rightColW = frameW - leftColW - colGap
+)
+
+// oneLine folds line breaks into spaces.
+func oneLine(s string) string {
+	return strings.Join(strings.Fields(s), " ")
+}
+
+// slideBuilder writes the shape tree of one slide.
+type slideBuilder struct {
+	pal    palette
+	load   ImageLoader
+	media  *[]media
+	id     int
+	sp     strings.Builder
+	rels   []rel
+	titled bool // the slide has a title placeholder and a body placeholder
+}
+
+// Placeholder shape types of the Title and Content layout. Only the title and
+// the one body shape of a titled slide carry them.
+const (
+	phTitle = `<p:ph type="title"/>`
+	phBody  = `<p:ph idx="1"/>`
+)
+
+func newSlideBuilder(pal palette, load ImageLoader, m *[]media) *slideBuilder {
+	return &slideBuilder{pal: pal, load: load, media: m, id: 1}
+}
+
+func (b *slideBuilder) nextID() int {
+	b.id++
+	return b.id
+}
+
+func (b *slideBuilder) finish() builtSlide {
+	x := xmlHeader + `<p:sld ` + nsDecl + `><p:cSld><p:spTree>` + groupHeader + b.sp.String() +
+		`</p:spTree></p:cSld><p:clrMapOvr><a:masterClrMapping/></p:clrMapOvr></p:sld>`
+	return builtSlide{xml: x, rels: b.rels, titled: b.titled}
+}
+
+func xfrm(x, y, w, h float64, extra string) string {
+	return fmt.Sprintf(`<a:xfrm%s><a:off x="%d" y="%d"/><a:ext cx="%d" cy="%d"/></a:xfrm>`,
+		extra, emu(x), emu(y), max(emu(w), 0), max(emu(h), 0))
+}
+
+func solid(color string) string {
+	return `<a:solidFill><a:srgbClr val="` + color + `"/></a:solidFill>`
+}
+
+// shape adds a filled preset shape without text.
+func (b *slideBuilder) shape(name, prst string, x, y, w, h float64, fill string) {
+	id := b.nextID()
+	fmt.Fprintf(&b.sp, `<p:sp><p:nvSpPr><p:cNvPr id="%d" name="%s %d"/><p:cNvSpPr/><p:nvPr/></p:nvSpPr>`+
+		`<p:spPr>%s<a:prstGeom prst="%s"><a:avLst/></a:prstGeom>%s<a:ln><a:noFill/></a:ln></p:spPr></p:sp>`,
+		id, name, id, xfrm(x, y, w, h, ""), prst, fill)
+}
+
+// run is one piece of text with its look.
+type run struct {
+	text  string
+	size  float64 // cqw
+	color string
+	bold  bool
+	under string // "", "wavy" or "dotted"
+	uclr  string
+}
+
+func (r run) xml(font string) string {
+	attrs := fmt.Sprintf(` lang="en-US" sz="%d"`, fontSize(r.size))
+	if r.bold {
+		attrs += ` b="1"`
+	}
+	if r.under != "" {
+		attrs += ` u="` + r.under + `"`
+	}
+	inner := solid(r.color)
+	if r.under != "" && r.uclr != "" {
+		inner += `<a:uFill>` + solid(r.uclr) + `</a:uFill>`
+	}
+	inner += `<a:latin typeface="` + ooxml.Escape(font) + `"/>`
+	return `<a:r><a:rPr` + attrs + ` dirty="0">` + inner + `</a:rPr><a:t>` + ooxml.Escape(r.text) + `</a:t></a:r>`
+}
+
+// para is one paragraph.
+type para struct {
+	runs   []run
+	bullet bool
+	algn   string
+	before float64 // space before, cqw
+}
+
+// textBox adds an editable text box.
+type textBoxOpts struct {
+	name   string
+	anchor string // t, ctr, b
+	fill   string // spPr fill, empty for none
+	inset  float64
+	geom   string
+	line   string
+	ph     string // placeholder element (phTitle or phBody), "" for a text box
+}
+
+func (b *slideBuilder) textBox(x, y, w, h float64, o textBoxOpts, paras []para) {
+	id := b.nextID()
+	anchor := o.anchor
+	if anchor == "" {
+		anchor = "t"
+	}
+	geom := o.geom
+	if geom == "" {
+		geom = "rect"
+	}
+	fill := o.fill
+	if fill == "" {
+		fill = `<a:noFill/>`
+	}
+	line := o.line
+	if line == "" {
+		line = `<a:ln><a:noFill/></a:ln>`
+	}
+	ins := emu(o.inset)
+	nv := `<p:cNvSpPr txBox="1"/><p:nvPr/>`
+	if o.ph != "" {
+		nv = `<p:cNvSpPr><a:spLocks noGrp="1"/></p:cNvSpPr><p:nvPr>` + o.ph + `</p:nvPr>`
+	}
+	fmt.Fprintf(&b.sp, `<p:sp><p:nvSpPr><p:cNvPr id="%d" name="%s %d"/>%s</p:nvSpPr>`+
+		`<p:spPr>%s<a:prstGeom prst="%s"><a:avLst/></a:prstGeom>%s%s</p:spPr>`+
+		`<p:txBody><a:bodyPr wrap="square" lIns="%d" tIns="%d" rIns="%d" bIns="%d" anchor="%s" rtlCol="0"><a:normAutofit/></a:bodyPr><a:lstStyle/>`,
+		id, o.name, id, nv, xfrm(x, y, w, h, ""), geom, fill, line, ins, ins, ins, ins, anchor)
+	for _, p := range paras {
+		b.sp.WriteString(b.paraXML(p))
+	}
+	b.sp.WriteString(`</p:txBody></p:sp>`)
+}
+
+func (b *slideBuilder) paraXML(p para) string {
+	var s strings.Builder
+	s.WriteString(`<a:p><a:pPr`)
+	size := 2.5
+	if len(p.runs) > 0 {
+		size = p.runs[0].size
+	}
+	if p.bullet {
+		fmt.Fprintf(&s, ` marL="%d" indent="%d"`, emu(3), -emu(3))
+	} else {
+		s.WriteString(` marL="0" indent="0"`)
+	}
+	if p.algn != "" {
+		s.WriteString(` algn="` + p.algn + `"`)
+	}
+	s.WriteString(`>`)
+	if p.before > 0 {
+		fmt.Fprintf(&s, `<a:spcBef><a:spcPts val="%d"/></a:spcBef>`, fontSize(p.before))
+	}
+	if p.bullet {
+		s.WriteString(`<a:buClr><a:srgbClr val="` + b.pal.accent + `"/></a:buClr><a:buSzPct val="100000"/><a:buFont typeface="Arial"/><a:buChar char="•"/>`)
+	} else {
+		s.WriteString(`<a:buNone/>`)
+	}
+	s.WriteString(`</a:pPr>`)
+	for _, r := range p.runs {
+		if r.text == "" {
+			continue
+		}
+		s.WriteString(r.xml(b.pal.font))
+	}
+	fmt.Fprintf(&s, `<a:endParaRPr lang="en-US" sz="%d" dirty="0"/></a:p>`, fontSize(size))
+	return s.String()
+}
+
+// decor draws the style's fixed decoration: the left edge of studio and the
+// wash circle of night.
+func (b *slideBuilder) decor() {
+	switch b.pal.style {
+	case deck.StyleStudio:
+		b.shape("Edge", "rect", 0, 0, 2.2, frameH, solid(b.pal.accent))
+	case deck.StyleNight:
+		b.shape("Circle", "ellipse", 100+8-34, frameH+14-34, 34, 34,
+			`<a:solidFill><a:srgbClr val="`+b.pal.wash+`"><a:alpha val="60000"/></a:srgbClr></a:solidFill>`)
+	}
+}
+
+// titleLines estimates how many lines a title takes at size s in width w.
+func titleLines(text string, s, w float64) int {
+	perLine := max(int(w/(s*0.52)), 1)
+	return min(max((utf8.RuneCountInString(text)+perLine-1)/perLine, 1), 3)
+}
+
+// title draws the slide title and, for paper, the accent bar under it. It
+// returns the y (cqw) where the content below may start. A non-empty title
+// makes the slide titled: the title shape becomes the title placeholder.
+func (b *slideBuilder) title(text string, y, size float64) float64 {
+	text = oneLine(text)
+	b.titled = text != ""
+	h := float64(titleLines(text, size, frameW)) * size * 1.12
+	b.textBox(padSide, y, frameW, h, textBoxOpts{name: "Title", anchor: "b", ph: b.titlePh()},
+		[]para{{runs: []run{{text: text, size: size, color: b.pal.ink, bold: true}}}})
+	y += h
+	if b.pal.style == deck.StylePaper {
+		b.shape("Accent bar", "roundRect", padSide, y+1.6, 7, 0.9, solid(b.pal.accent))
+		y += 1.6 + 0.9
+	}
+	return y + 2.2
+}
+
+// titlePh is the placeholder of the title shape: none on an untitled slide.
+func (b *slideBuilder) titlePh() string {
+	if b.titled {
+		return phTitle
+	}
+	return ""
+}
+
+// bodyPh is the placeholder of the body shape of a slide: none on an
+// untitled slide.
+func (b *slideBuilder) bodyPh() string {
+	if b.titled {
+		return phBody
+	}
+	return ""
+}
+
+func (b *slideBuilder) bulletParas(bullets []deck.StudioBullet, size float64) []para {
+	paras := make([]para, 0, len(bullets))
+	for i, bl := range bullets {
+		r := run{text: oneLine(bl.Text), size: size, color: b.pal.ink}
+		switch bl.Flag {
+		case deck.FlagNoSource:
+			r.under, r.uclr = "wavy", nosrcColor
+		case deck.FlagWeak:
+			r.under, r.uclr = "dotted", weakColor
+		}
+		p := para{runs: []run{r}, bullet: true}
+		if i > 0 {
+			p.before = 1.7
+		}
+		paras = append(paras, p)
+	}
+	return paras
+}
+
+func (b *slideBuilder) bullets(bullets []deck.StudioBullet, x, y, w, h float64) {
+	if len(bullets) == 0 || h <= 0 {
+		return
+	}
+	b.textBox(x, y, w, h, textBoxOpts{name: "Bullets", ph: b.bodyPh()}, b.bulletParas(bullets, 2.55))
+}
+
+// studioSlide draws one finished studio slide.
+func (b *slideBuilder) studioSlide(s deck.StudioSlide) {
+	b.decor()
+	switch {
+	case s.Layout == deck.LayoutTitle:
+		t := oneLine(s.Title)
+		lines := titleLines(t, 6.6, frameW)
+		th := float64(lines) * 6.6 * 1.12
+		total := th
+		if s.Sub != "" {
+			total += 2.6 + 2.9*1.3
+		}
+		if b.pal.style == deck.StylePaper {
+			total += 2.5
+		}
+		y := max((frameH-total)/2, padTop)
+		y = b.title(t, y, 6.6) - 2.2 + 2.6
+		if s.Sub != "" {
+			b.textBox(padSide, y, frameW, 2.9*1.3*2, textBoxOpts{name: "Subtitle", ph: b.bodyPh()},
+				[]para{{runs: []run{{text: oneLine(s.Sub), size: 2.9, color: b.pal.muted}}}})
+		}
+	case s.Layout == deck.LayoutNumber && s.Big != nil && s.Big.Num != "":
+		y := b.title(s.Title, padTop, 4.5) - 2.2 + 1.6
+		b.textBox(padSide, y, frameW, 13*1.0, textBoxOpts{name: "Number"},
+			[]para{{runs: []run{{text: oneLine(s.Big.Num), size: 13, color: b.pal.accent, bold: true}}}})
+		y += 13 + 1.6
+		if s.Big.Label != "" {
+			b.textBox(padSide, y, frameW, 2.8*1.3, textBoxOpts{name: "Number label"},
+				[]para{{runs: []run{{text: oneLine(s.Big.Label), size: 2.8, color: b.pal.muted}}}})
+			y += 2.8*1.3 + 1.6
+		}
+		b.bullets(s.Bullets, padSide, y, frameW, frameBot-y)
+	case s.Layout == deck.LayoutChart && s.Chart != nil:
+		y := b.title(s.Title, padTop, 4.5)
+		b.bullets(s.Bullets, padSide, y, leftColW, frameBot-y)
+		x, w := rightColX, rightColW
+		if len(s.Bullets) == 0 {
+			x, w = padSide, frameW
+		}
+		b.chart(*s.Chart, x, y, w, frameBot-y)
+	case s.Layout == deck.LayoutImage && s.Image != nil:
+		y := b.title(s.Title, padTop, 4.5)
+		b.bullets(s.Bullets, padSide, y, leftColW, frameBot-y)
+		x, w := rightColX, rightColW
+		if len(s.Bullets) == 0 {
+			x, w = padSide, frameW
+		}
+		b.picture(*s.Image, x, y, w, frameBot-y)
+	default:
+		y := b.title(s.Title, padTop, 4.5)
+		b.bullets(s.Bullets, padSide, y, frameW, frameBot-y)
+	}
+}
+
+// formatValue prints a chart value with its unit.
+func formatValue(v float64, unit string) string {
+	return strconv.FormatFloat(v, 'f', -1, 64) + unit
+}
+
+// niceMax rounds m up to 1, 2, 5 or 10 times a power of ten.
+func niceMax(m float64) float64 {
+	if !(m > 0) || math.IsInf(m, 0) {
+		return 1
+	}
+	p := math.Pow(10, math.Floor(math.Log10(m)))
+	switch n := m / p; {
+	case n <= 1:
+		return p
+	case n <= 2:
+		return 2 * p
+	case n <= 5:
+		return 5 * p
+	}
+	return 10 * p
+}
+
+// chart draws a bar chart as rectangles or a line chart as connectors, with
+// value and category labels, inside the box x, y, w, h (cqw).
+func (b *slideBuilder) chart(c deck.StudioChart, x, y, w, h float64) {
+	if len(c.Data) == 0 || h <= 0 {
+		return
+	}
+	const labelH = 2.4
+	const labelSize = 1.6
+	if c.Label != "" {
+		b.textBox(x, y, w, labelH, textBoxOpts{name: "Chart label"},
+			[]para{{runs: []run{{text: oneLine(c.Label), size: 1.8, color: b.pal.muted, bold: true}}}})
+		y += labelH + 0.6
+		h -= labelH + 0.6
+	}
+	// Plot area: room for value labels on top and category labels below.
+	plotY, plotH := y+labelH, h-2*labelH
+	if plotH <= 0 {
+		return
+	}
+	hiV, loV := 0.0, 0.0
+	vals := make([]float64, len(c.Data))
+	for i, p := range c.Data {
+		v := p.V
+		if math.IsNaN(v) || math.IsInf(v, 0) {
+			v = 0
+		}
+		vals[i] = v
+		hiV, loV = max(hiV, v), min(loV, v)
+	}
+	hi, lo := 0.0, 0.0
+	if hiV > 0 {
+		hi = niceMax(hiV)
+	}
+	if loV < 0 {
+		lo = -niceMax(-loV)
+	}
+	if hi == lo {
+		hi = 1
+	}
+	vy := func(v float64) float64 { return plotY + plotH - (v-lo)/(hi-lo)*plotH }
+	zero := vy(0)
+	// Baseline.
+	b.shape("Baseline", "rect", x, zero-0.05, w, 0.1,
+		`<a:solidFill><a:srgbClr val="`+b.pal.ink+`"><a:alpha val="30000"/></a:srgbClr></a:solidFill>`)
+
+	n := float64(len(c.Data))
+	slot := w / n
+	label := func(name, text string, cx, top float64, color string) {
+		b.textBox(cx-slot/2, top, slot, labelH, textBoxOpts{name: name, anchor: "ctr"},
+			[]para{{runs: []run{{text: text, size: labelSize, color: color}}, algn: "ctr"}})
+	}
+	if c.Kind == deck.ChartLine {
+		pts := make([][2]float64, len(vals))
+		for i, v := range vals {
+			pts[i] = [2]float64{x + slot*(float64(i)+0.5), vy(v)}
+		}
+		for i := 1; i < len(pts); i++ {
+			b.connector(pts[i-1], pts[i])
+		}
+		for i, p := range pts {
+			b.shape("Point", "ellipse", p[0]-0.6, p[1]-0.6, 1.2, 1.2, solid(b.pal.accent))
+			label("Value", formatValue(vals[i], c.Unit), p[0], p[1]-labelH-0.6, b.pal.ink)
+			label("Category", oneLine(c.Data[i].K), p[0], plotY+plotH+0.3, b.pal.muted)
+		}
+		return
+	}
+	barW := slot * 0.62
+	for i, v := range vals {
+		cx := x + slot*(float64(i)+0.5)
+		top, bottom := min(vy(v), zero), max(vy(v), zero)
+		b.shape("Bar", "rect", cx-barW/2, top, barW, max(bottom-top, 0.1), solid(b.pal.accent))
+		vt := top - labelH
+		if v < 0 {
+			vt = bottom
+		}
+		label("Value", formatValue(v, c.Unit), cx, vt, b.pal.ink)
+		label("Category", oneLine(c.Data[i].K), cx, plotY+plotH+0.3, b.pal.muted)
+	}
+}
+
+// connector draws a straight line from p to q.
+func (b *slideBuilder) connector(p, q [2]float64) {
+	id := b.nextID()
+	x, y := min(p[0], q[0]), min(p[1], q[1])
+	w, h := math.Abs(q[0]-p[0]), math.Abs(q[1]-p[1])
+	flip := ""
+	// A line that rises left to right (or falls right to left) runs from
+	// bottom-left to top-right of its box.
+	if (q[0] >= p[0]) != (q[1] >= p[1]) {
+		flip = ` flipV="1"`
+	}
+	fmt.Fprintf(&b.sp, `<p:cxnSp><p:nvCxnSpPr><p:cNvPr id="%d" name="Line %d"/><p:cNvCxnSpPr/><p:nvPr/></p:nvCxnSpPr>`+
+		`<p:spPr>%s<a:prstGeom prst="line"><a:avLst/></a:prstGeom>`+
+		`<a:ln w="%d" cap="rnd">%s<a:round/></a:ln></p:spPr></p:cxnSp>`,
+		id, id, xfrm(x, y, w, h, flip), emu(0.45), solid(b.pal.accent))
+}
+
+// loadImage loads and checks an image; it returns the extension and size.
+func (b *slideBuilder) loadImage(ref string) ([]byte, string, image.Config, bool) {
+	if b.load == nil {
+		return nil, "", image.Config{}, false
+	}
+	data, _, err := b.load(ref)
+	if err != nil || len(data) == 0 || len(data) > maxImageBytes {
+		return nil, "", image.Config{}, false
+	}
+	cfg, format, err := image.DecodeConfig(bytes.NewReader(data))
+	if err != nil || cfg.Width <= 0 || cfg.Height <= 0 {
+		return nil, "", image.Config{}, false
+	}
+	switch format {
+	case "png":
+		return data, "png", cfg, true
+	case "jpeg":
+		return data, "jpeg", cfg, true
+	}
+	return nil, "", image.Config{}, false
+}
+
+// picture embeds the image, cropped to fill the box, with its caption. An
+// image that cannot be loaded leaves a dashed placeholder with the caption.
+func (b *slideBuilder) picture(img deck.StudioImage, x, y, w, h float64) {
+	if h <= 0 {
+		return
+	}
+	caption := strings.TrimSpace(oneLine(img.Label))
+	if img.Weak {
+		caption = strings.TrimSpace(caption + " (low confidence)")
+	}
+	data, ext, cfg, ok := b.loadImage(img.Ref)
+	if !ok {
+		text := "Image not available"
+		if caption != "" {
+			text = caption + " (image not available)"
+		}
+		b.textBox(x, y, w, h, textBoxOpts{
+			name: "Image placeholder", anchor: "ctr", inset: 2, geom: "roundRect",
+			fill: `<a:noFill/>`,
+			line: `<a:ln w="` + strconv.FormatInt(emu(0.35), 10) + `"><a:solidFill><a:srgbClr val="` + b.pal.ink +
+				`"><a:alpha val="35000"/></a:srgbClr></a:solidFill><a:prstDash val="dash"/></a:ln>`,
+		}, []para{{runs: []run{{text: text, size: 2.1, color: b.pal.muted}}, algn: "ctr"}})
+		return
+	}
+	name := fmt.Sprintf("image%d.%s", len(*b.media)+1, ext)
+	*b.media = append(*b.media, media{name: name, ext: ext, data: data})
+	rid := fmt.Sprintf("rId%d", len(b.rels)+2) // rId1 is the layout
+	b.rels = append(b.rels, rel{rid, relImage, "../media/" + name})
+
+	// Cover: crop the side that overflows the box.
+	box := w / h
+	ar := float64(cfg.Width) / float64(cfg.Height)
+	var l, t int
+	if ar > box {
+		l = int(math.Round((1 - box/ar) / 2 * 100000))
+	} else {
+		t = int(math.Round((1 - ar/box) / 2 * 100000))
+	}
+	id := b.nextID()
+	fmt.Fprintf(&b.sp, `<p:pic><p:nvPicPr><p:cNvPr id="%d" name="Picture %d" descr="%s"/><p:cNvPicPr><a:picLocks noChangeAspect="1"/></p:cNvPicPr><p:nvPr/></p:nvPicPr>`+
+		`<p:blipFill><a:blip r:embed="%s"/><a:srcRect l="%d" t="%d" r="%d" b="%d"/><a:stretch><a:fillRect/></a:stretch></p:blipFill>`+
+		`<p:spPr>%s<a:prstGeom prst="roundRect"><a:avLst/></a:prstGeom>%s</p:spPr></p:pic>`,
+		id, id, ooxml.Escape(caption), rid, l, t, l, t, xfrm(x, y, w, h, ""), solid(b.pal.wash))
+	if caption != "" {
+		cw := min(float64(utf8.RuneCountInString(caption))*1.5*0.55+2, w-2.4)
+		b.textBox(x+1.2, y+h-1.2-3.1, cw, 3.1, textBoxOpts{
+			name: "Caption", anchor: "ctr", inset: 0.7, geom: "roundRect",
+			fill: `<a:solidFill><a:srgbClr val="` + capFill + `"><a:alpha val="72000"/></a:srgbClr></a:solidFill>`,
+		}, []para{{runs: []run{{text: caption, size: 1.5, color: white}}}})
+	}
+}
+
+// sourceSuffix marks a line of the sources slide.
+var sourceSuffix = map[SourceKind]string{
+	SourceWeak:     " (low confidence)",
+	SourceNoSource: " (no source)",
+	SourceOwn:      " (your own claim)",
+}
+
+// sourcesSlide lists every source line, weak and unsourced claims included.
+func (b *slideBuilder) sourcesSlide(lines []SourceLine) {
+	b.decor()
+	y := b.title("Sources", padTop, 4.5)
+	if len(lines) == 0 {
+		b.textBox(padSide, y, frameW, 3, textBoxOpts{name: "Sources", ph: b.bodyPh()},
+			[]para{{runs: []run{{text: "No sources were cited.", size: 2, color: b.pal.muted}}}})
+		return
+	}
+	size := 2.0
+	if len(lines) > 10 {
+		size = max(2.0*10/float64(len(lines)), 0.9)
+	}
+	paras := make([]para, 0, len(lines))
+	for i, l := range lines {
+		runs := []run{{text: oneLine(l.Label), size: size, color: b.pal.ink, bold: true}}
+		if d := oneLine(l.Detail); d != "" {
+			runs = append(runs, run{text: ": " + d, size: size, color: b.pal.muted})
+		}
+		if sfx, ok := sourceSuffix[l.Kind]; ok {
+			c := b.pal.muted
+			switch l.Kind {
+			case SourceWeak:
+				c = weakColor
+			case SourceNoSource:
+				c = nosrcColor
+			}
+			runs = append(runs, run{text: sfx, size: size, color: c, bold: true})
+		}
+		p := para{runs: runs, bullet: true}
+		if i > 0 {
+			p.before = size * 0.5
+		}
+		paras = append(paras, p)
+	}
+	b.textBox(padSide, y, frameW, frameBot-y, textBoxOpts{name: "Sources", ph: b.bodyPh()}, paras)
+}

addons/office/internal/deck/studio.go

@@ -0,0 +1,464 @@
+package deck
+
+import (
+	"encoding/json"
+	"errors"
+	"fmt"
+	"math"
+	"strings"
+	"unicode/utf8"
+)
+
+// The Slides studio keeps a richer slide model than the presenter deck: each
+// claim carries its evidence and an honesty flag, and each slide carries its
+// build status. StudioToDeck converts a studio deck into the presenter Deck
+// so canvases, sharing and publishing keep working unchanged. The types
+// match the slide model of the web client.
+
+// StudioStyle is the visual style of a studio deck.
+type StudioStyle string
+
+const (
+	StylePaper  StudioStyle = "paper"
+	StyleNight  StudioStyle = "night"
+	StyleStudio StudioStyle = "studio"
+	StylePlain  StudioStyle = "plain"
+)
+
+// Valid reports whether s is a known style.
+func (s StudioStyle) Valid() bool {
+	switch s {
+	case StylePaper, StyleNight, StyleStudio, StylePlain:
+		return true
+	}
+	return false
+}
+
+// StudioLayout is the layout of one studio slide.
+type StudioLayout string
+
+const (
+	LayoutTitle  StudioLayout = "title"
+	LayoutText   StudioLayout = "text"
+	LayoutImage  StudioLayout = "image"
+	LayoutChart  StudioLayout = "chart"
+	LayoutNumber StudioLayout = "number"
+)
+
+// Valid reports whether l is a known layout.
+func (l StudioLayout) Valid() bool {
+	switch l {
+	case LayoutTitle, LayoutText, LayoutImage, LayoutChart, LayoutNumber:
+		return true
+	}
+	return false
+}
+
+// ClaimFlag marks a claim that needs the user's attention.
+type ClaimFlag string
+
+const (
+	FlagNone     ClaimFlag = ""
+	FlagNoSource ClaimFlag = "nosrc"
+	FlagWeak     ClaimFlag = "weak"
+	FlagOwn      ClaimFlag = "own"
+)
+
+func (f ClaimFlag) valid() bool {
+	switch f {
+	case FlagNone, FlagNoSource, FlagWeak, FlagOwn:
+		return true
+	}
+	return false
+}
+
+// SlideStatus is the build status of one studio slide.
+type SlideStatus string
+
+const (
+	SlideQueued  SlideStatus = "queued"
+	SlideWorking SlideStatus = "working"
+	SlideOK      SlideStatus = "ok"
+	SlideFailed  SlideStatus = "failed"
+)
+
+func (s SlideStatus) valid() bool {
+	switch s {
+	case SlideQueued, SlideWorking, SlideOK, SlideFailed:
+		return true
+	}
+	return false
+}
+
+// StudioBullet is one claim on a slide with the evidence ids that support it.
+type StudioBullet struct {
+	Text     string    `json:"text"`
+	Evidence []string  `json:"evidence"`
+	Flag     ClaimFlag `json:"flag,omitempty"`
+}
+
+// StudioImage is an image taken from a source document. Ref is an internal
+// media reference such as /api/documents/<id>/file.
+type StudioImage struct {
+	Ref      string `json:"ref"`
+	Label    string `json:"label"`
+	Evidence string `json:"evidence"`
+	Weak     bool   `json:"weak,omitempty"`
+}
+
+// ChartKind is the kind of a studio chart.
+type ChartKind string
+
+const (
+	ChartBar  ChartKind = "bar"
+	ChartLine ChartKind = "line"
+)
+
+// ChartPoint is one category and its value.
+type ChartPoint struct {
+	K string  `json:"k"`
+	V float64 `json:"v"`
+}
+
+// StudioChart is a small chart built from numbers found in the sources.
+type StudioChart struct {
+	Kind     ChartKind    `json:"kind"`
+	Label    string       `json:"label"`
+	Unit     string       `json:"unit,omitempty"`
+	Data     []ChartPoint `json:"data"`
+	Evidence string       `json:"evidence"`
+}
+
+// BigNumber is the headline number of a number slide.
+type BigNumber struct {
+	Num      string `json:"num"`
+	Label    string `json:"label"`
+	Evidence string `json:"evidence"`
+}
+
+// StudioSlide is one slide of a studio deck. Sources are evidence ids;
+// SourceHashes maps a document id to the content hash the slide was built from.
+type StudioSlide struct {
+	ID           string            `json:"id"`
+	Title        string            `json:"title"`
+	Layout       StudioLayout      `json:"layout"`
+	Sub          string            `json:"sub,omitempty"`
+	Bullets      []StudioBullet    `json:"bullets"`
+	Image        *StudioImage      `json:"image,omitempty"`
+	Chart        *StudioChart      `json:"chart,omitempty"`
+	Big          *BigNumber        `json:"big,omitempty"`
+	Notes        string            `json:"notes,omitempty"`
+	Sources      []string          `json:"sources"`
+	Status       SlideStatus       `json:"status"`
+	Error        string            `json:"error,omitempty"`
+	SourceHashes map[string]string `json:"sourceHashes,omitempty"`
+	// ErrorKey is the reason a slide failed, as a key of the interface
+	// texts (it starts with SlideErrorKeyPrefix), so the viewer reads it in
+	// their own language. Error keeps the English as a fallback.
+	ErrorKey string `json:"errorKey,omitempty"`
+}
+
+// SlideErrorKeyPrefix starts every StudioSlide.ErrorKey.
+const SlideErrorKeyPrefix = "studio.server.slideError."
+
+// OutlineItem is one planned slide of the outline.
+type OutlineItem struct {
+	ID       string   `json:"id"`
+	Title    string   `json:"title"`
+	Summary  string   `json:"summary"`
+	Evidence []string `json:"evidence"`
+	Merged   int      `json:"merged,omitempty"`
+}
+
+// DeckLength is the requested size of a deck.
+type DeckLength string
+
+const (
+	LengthShort    DeckLength = "short"
+	LengthStandard DeckLength = "standard"
+	LengthLong     DeckLength = "long"
+)
+
+// Slides returns the number of slides for the length, or 0 when unknown.
+func (l DeckLength) Slides() int {
+	switch l {
+	case LengthShort:
+		return 5
+	case LengthStandard:
+		return 7
+	case LengthLong:
+		return 8
+	}
+	return 0
+}
+
+// Tone is the writing tone of a deck.
+type Tone string
+
+const (
+	TonePlain  Tone = "plain"
+	ToneWarm   Tone = "warm"
+	ToneFormal Tone = "formal"
+	TonePunchy Tone = "punchy"
+)
+
+func (t Tone) valid() bool {
+	switch t {
+	case TonePlain, ToneWarm, ToneFormal, TonePunchy:
+		return true
+	}
+	return false
+}
+
+// Brief is what the user asks for. Sources are the selected document ids.
+type Brief struct {
+	Title    string      `json:"title"`
+	Goal     string      `json:"goal"`
+	Audience string      `json:"audience"`
+	Length   DeckLength  `json:"length"`
+	Tone     Tone        `json:"tone"`
+	Style    StudioStyle `json:"style"`
+	Sources  []string    `json:"sources"`
+}
+
+// Text limits, in runes.
+const (
+	maxBriefTitle  = 200
+	maxBriefGoal   = 2000
+	maxAudience    = 500
+	maxBriefSrcs   = 50
+	maxSlideTitle  = 200
+	maxSub         = 300
+	maxBullets     = 8
+	maxBulletText  = 500
+	maxNotes       = 5000
+	maxLabel       = 200
+	maxChartPoints = 12
+	maxChartKey    = 60
+	maxUnit        = 20
+	maxNumber      = 40
+	maxError       = 1000
+)
+
+func tooLong(s string, n int) bool { return utf8.RuneCountInString(s) > n }
+
+// Validate returns human-readable errors; an empty slice means the brief is valid.
+func (b Brief) Validate() []string {
+	var errs []string
+	switch {
+	case strings.TrimSpace(b.Title) == "":
+		errs = append(errs, "title is required")
+	case tooLong(b.Title, maxBriefTitle):
+		errs = append(errs, fmt.Sprintf("title must be at most %d characters", maxBriefTitle))
+	}
+	if tooLong(b.Goal, maxBriefGoal) {
+		errs = append(errs, fmt.Sprintf("goal must be at most %d characters", maxBriefGoal))
+	}
+	if tooLong(b.Audience, maxAudience) {
+		errs = append(errs, fmt.Sprintf("audience must be at most %d characters", maxAudience))
+	}
+	if b.Length.Slides() == 0 {
+		errs = append(errs, "length must be one of short, standard, long")
+	}
+	if !b.Tone.valid() {
+		errs = append(errs, "tone must be one of plain, warm, formal, punchy")
+	}
+	if !b.Style.Valid() {
+		errs = append(errs, "style must be one of paper, night, studio, plain")
+	}
+	if len(b.Sources) > maxBriefSrcs {
+		errs = append(errs, fmt.Sprintf("at most %d sources can be selected", maxBriefSrcs))
+	}
+	return errs
+}
+
+// ValidateStudioSlides returns human-readable errors; an empty slice means
+// the slides are valid.
+func ValidateStudioSlides(slides []StudioSlide) []string {
+	var errs []string
+	seen := make(map[string]bool, len(slides))
+	for i, s := range slides {
+		add := func(format string, a ...any) {
+			errs = append(errs, fmt.Sprintf("slide %d: ", i)+fmt.Sprintf(format, a...))
+		}
+		switch {
+		case s.ID == "":
+			add("id is required")
+		case seen[s.ID]:
+			add("id %q is used twice", s.ID)
+		}
+		seen[s.ID] = true
+		if !s.Layout.Valid() {
+			add("layout must be one of title, text, image, chart, number")
+		}
+		if !s.Status.valid() {
+			add("status must be one of queued, working, ok, failed")
+		}
+		if tooLong(s.Title, maxSlideTitle) {
+			add("title must be at most %d characters", maxSlideTitle)
+		}
+		if tooLong(s.Sub, maxSub) {
+			add("subtitle must be at most %d characters", maxSub)
+		}
+		if tooLong(s.Notes, maxNotes) {
+			add("notes must be at most %d characters", maxNotes)
+		}
+		if tooLong(s.Error, maxError) {
+			add("error must be at most %d characters", maxError)
+		}
+		if s.ErrorKey != "" && !strings.HasPrefix(s.ErrorKey, SlideErrorKeyPrefix) {
+			add("errorKey must start with %q", SlideErrorKeyPrefix)
+		}
+		if len(s.Bullets) > maxBullets {
+			add("at most %d bullets are allowed", maxBullets)
+		}
+		for j, b := range s.Bullets {
+			if tooLong(b.Text, maxBulletText) {
+				add("bullet %d must be at most %d characters", j, maxBulletText)
+			}
+			if !b.Flag.valid() {
+				add("bullet %d: flag must be one of nosrc, weak, own", j)
+			}
+		}
+		if s.Image != nil {
+			if !internalMediaRe.MatchString(s.Image.Ref) {
+				add("%s", mediaRefMsg)
+			}
+			if tooLong(s.Image.Label, maxLabel) {
+				add("image label must be at most %d characters", maxLabel)
+			}
+		}
+		if s.Chart != nil {
+			for _, e := range validateChart(*s.Chart) {
+				add("%s", e)
+			}
+		}
+		if s.Big != nil {
+			if tooLong(s.Big.Num, maxNumber) {
+				add("number must be at most %d characters", maxNumber)
+			}
+			if tooLong(s.Big.Label, maxLabel) {
+				add("number label must be at most %d characters", maxLabel)
+			}
+		}
+	}
+	return errs
+}
+
+func validateChart(c StudioChart) []string {
+	var errs []string
+	if c.Kind != ChartBar && c.Kind != ChartLine {
+		errs = append(errs, "chart kind must be bar or line")
+	}
+	if tooLong(c.Label, maxLabel) {
+		errs = append(errs, fmt.Sprintf("chart label must be at most %d characters", maxLabel))
+	}
+	if tooLong(c.Unit, maxUnit) {
+		errs = append(errs, fmt.Sprintf("chart unit must be at most %d characters", maxUnit))
+	}
+	if len(c.Data) == 0 || len(c.Data) > maxChartPoints {
+		errs = append(errs, fmt.Sprintf("chart must have 1 to %d points", maxChartPoints))
+	}
+	for j, p := range c.Data {
+		if math.IsNaN(p.V) || math.IsInf(p.V, 0) {
+			errs = append(errs, fmt.Sprintf("chart point %d must be a finite number", j))
+		}
+		if tooLong(p.K, maxChartKey) {
+			errs = append(errs, fmt.Sprintf("chart point %d: name must be at most %d characters", j, maxChartKey))
+		}
+	}
+	return errs
+}
+
+// styleThemes maps a studio style to the presenter theme and accent.
+var styleThemes = map[StudioStyle][2]string{
+	StylePaper:  {"paper", "#FF6B3D"},
+	StyleNight:  {"midnight", "#FFB26B"},
+	StyleStudio: {"paper", "#B23610"},
+	StylePlain:  {"mono", "#1B4A98"},
+}
+
+// StudioToDeck converts finished studio slides into a presenter Deck. Slides
+// whose status is not ok are skipped. Claim flags stay in the studio.
+func StudioToDeck(title string, style StudioStyle, slides []StudioSlide) Deck {
+	th, ok := styleThemes[style]
+	if !ok {
+		th = styleThemes[StylePaper]
+	}
+	d := Deck{Version: 1, Theme: th[0], Accent: th[1], Title: title, Slides: []Slide{}}
+	for _, s := range slides {
+		if s.Status != SlideOK {
+			continue
+		}
+		d.Slides = append(d.Slides, studioSlideToSlide(s))
+	}
+	return d
+}
+
+func studioSlideToSlide(s StudioSlide) Slide {
+	blocks := []map[string]any{}
+	if s.Title != "" {
+		blocks = append(blocks, map[string]any{"type": "heading", "text": s.Title})
+	}
+	bullets := func() {
+		if len(s.Bullets) == 0 {
+			return
+		}
+		items := make([]string, 0, len(s.Bullets))
+		for _, b := range s.Bullets {
+			items = append(items, b.Text)
+		}
+		blocks = append(blocks, map[string]any{"type": "bullets", "items": items})
+	}
+	withCaption := func(b map[string]any, caption string) map[string]any {
+		if caption != "" {
+			b["caption"] = caption
+		}
+		return b
+	}
+	layout := "single"
+	switch {
+	case s.Layout == LayoutTitle:
+		layout = "title"
+		if s.Sub != "" {
+			blocks = append(blocks, map[string]any{"type": "subtitle", "text": s.Sub})
+		}
+	case s.Layout == LayoutImage && s.Image != nil:
+		layout = "media-right"
+		bullets()
+		caption := s.Image.Label
+		if s.Image.Weak {
+			caption += " (low confidence)"
+		}
+		blocks = append(blocks, withCaption(map[string]any{"type": "image", "ref": s.Image.Ref}, strings.TrimSpace(caption)))
+	case s.Layout == LayoutChart && s.Chart != nil:
+		layout = "media-right"
+		bullets()
+		blocks = append(blocks, withCaption(map[string]any{"type": "svg", "svg": ChartSVG(*s.Chart)}, s.Chart.Label))
+	case s.Layout == LayoutNumber && s.Big != nil && s.Big.Num != "":
+		layout = "stat"
+		stat := map[string]any{"type": "stat", "value": s.Big.Num}
+		if s.Big.Label != "" {
+			stat["label"] = s.Big.Label
+		}
+		blocks = append(blocks, stat)
+		bullets()
+	default:
+		bullets()
+	}
+	return Slide{Layout: layout, Notes: s.Notes, Blocks: blocks}
+}
+
+// StudioToDeckJSON converts with StudioToDeck and returns the deck JSON. It
+// returns an error when the result does not pass Validate, for example when
+// no slide is finished yet.
+func StudioToDeckJSON(title string, style StudioStyle, slides []StudioSlide) ([]byte, error) {
+	raw, err := json.Marshal(StudioToDeck(title, style, slides))
+	if err != nil {
+		return nil, err
+	}
+	if errs := Validate(raw); len(errs) != 0 {
+		return nil, errors.New("invalid deck: " + strings.Join(errs, "; "))
+	}
+	return raw, nil
+}

addons/office/internal/deck/studio_test.go

@@ -0,0 +1,200 @@
+package deck_test
+
+import (
+	"encoding/json"
+	"math"
+	"strings"
+
+	"github.com/mudler/LocalAI/addons/office/internal/deck"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func okSlide(id string, layout deck.StudioLayout) deck.StudioSlide {
+	return deck.StudioSlide{
+		ID:      id,
+		Title:   "Slide " + id,
+		Layout:  layout,
+		Bullets: []deck.StudioBullet{{Text: "First point", Evidence: []string{"e1"}}, {Text: "Unsourced", Flag: deck.FlagNoSource}},
+		Sources: []string{"e1"},
+		Status:  deck.SlideOK,
+	}
+}
+
+func allLayouts() []deck.StudioSlide {
+	title := okSlide("s1", deck.LayoutTitle)
+	title.Sub = "A subtitle"
+	text := okSlide("s2", deck.LayoutText)
+	text.Notes = "Say this."
+	img := okSlide("s3", deck.LayoutImage)
+	img.Image = &deck.StudioImage{Ref: "/api/documents/abc/keyframes/2/image", Label: "Whiteboard", Evidence: "e2", Weak: true}
+	chart := okSlide("s4", deck.LayoutChart)
+	chart.Chart = &deck.StudioChart{Kind: deck.ChartBar, Label: "Cost by office", Data: []deck.ChartPoint{{K: "Rome", V: 12}, {K: "Milan", V: 28}}, Evidence: "e3"}
+	num := okSlide("s5", deck.LayoutNumber)
+	num.Big = &deck.BigNumber{Num: "49k EUR", Label: "to approve", Evidence: "e4"}
+	return []deck.StudioSlide{title, text, img, chart, num}
+}
+
+func decodeDeck(raw []byte) deck.Deck {
+	GinkgoHelper()
+	var d deck.Deck
+	Expect(json.Unmarshal(raw, &d)).To(Succeed())
+	return d
+}
+
+func blockTypes(s deck.Slide) []string {
+	var out []string
+	for _, b := range s.Blocks {
+		out = append(out, b["type"].(string))
+	}
+	return out
+}
+
+var _ = Describe("studio deck", func() {
+	It("converts every layout to a valid presenter deck", func() {
+		raw, err := deck.StudioToDeckJSON("Pilot", deck.StylePaper, allLayouts())
+		Expect(err).NotTo(HaveOccurred())
+		Expect(deck.Validate(raw)).To(BeEmpty(), "deck invalid")
+		d := decodeDeck(raw)
+		Expect(d.Theme).To(Equal("paper"))
+		Expect(d.Accent).To(Equal("#FF6B3D"))
+		Expect(d.Title).To(Equal("Pilot"))
+		Expect(d.Version).To(Equal(1))
+		want := []string{"title", "single", "media-right", "media-right", "stat"}
+		for i, s := range d.Slides {
+			Expect(s.Layout).To(Equal(want[i]), "slide %d layout", i)
+		}
+		Expect(strings.Join(blockTypes(d.Slides[0]), ",")).To(Equal("heading,subtitle"), "title blocks")
+		Expect(strings.Join(blockTypes(d.Slides[1]), ",")).To(Equal("heading,bullets"), "text blocks")
+		Expect(d.Slides[1].Notes).To(Equal("Say this."), "notes not copied")
+		Expect(strings.Join(blockTypes(d.Slides[2]), ",")).To(Equal("heading,bullets,image"), "image blocks")
+		Expect(d.Slides[2].Blocks[2]["caption"]).To(Equal("Whiteboard (low confidence)"), "image caption")
+		Expect(strings.Join(blockTypes(d.Slides[3]), ",")).To(Equal("heading,bullets,svg"), "chart blocks")
+		Expect(d.Slides[3].Blocks[2]["caption"]).To(Equal("Cost by office"), "chart caption")
+		svg, _ := d.Slides[3].Blocks[2]["svg"].(string)
+		Expect(svg).To(HavePrefix("<svg"), "chart svg missing")
+		Expect(strings.Join(blockTypes(d.Slides[4]), ",")).To(Equal("heading,stat,bullets"), "number blocks")
+		st := d.Slides[4].Blocks[1]
+		Expect(st["value"]).To(Equal("49k EUR"))
+		Expect(st["label"]).To(Equal("to approve"))
+		Expect(string(raw)).NotTo(ContainSubstring("nosrc"), "flags must not reach the presenter deck")
+		Expect(string(raw)).NotTo(ContainSubstring("flag"), "flags must not reach the presenter deck")
+	})
+
+	It("maps studio styles to themes and accents", func() {
+		cases := map[deck.StudioStyle][2]string{
+			deck.StylePaper:  {"paper", "#FF6B3D"},
+			deck.StyleNight:  {"midnight", "#FFB26B"},
+			deck.StyleStudio: {"paper", "#B23610"},
+			deck.StylePlain:  {"mono", "#1B4A98"},
+		}
+		for style, want := range cases {
+			d := deck.StudioToDeck("t", style, allLayouts())
+			Expect([]string{d.Theme, d.Accent}).To(Equal(want[:]), "style %s", style)
+		}
+	})
+
+	It("skips unfinished slides", func() {
+		slides := allLayouts()
+		slides[0].Status = deck.SlideFailed
+		slides[1].Status = deck.SlideQueued
+		slides[2].Status = deck.SlideWorking
+		d := deck.StudioToDeck("t", deck.StyleNight, slides)
+		Expect(d.Slides).To(HaveLen(2))
+	})
+
+	It("fails when no slide is finished", func() {
+		s := okSlide("s1", deck.LayoutText)
+		s.Status = deck.SlideFailed
+		_, err := deck.StudioToDeckJSON("t", deck.StylePaper, []deck.StudioSlide{s})
+		Expect(err).To(HaveOccurred(), "expected an error for a deck with no finished slides")
+	})
+
+	It("falls back to a single layout for empty bullets and missing payloads", func() {
+		text := okSlide("s1", deck.LayoutText)
+		text.Bullets = nil
+		img := okSlide("s2", deck.LayoutImage)
+		chart := okSlide("s3", deck.LayoutChart)
+		num := okSlide("s4", deck.LayoutNumber)
+		raw, err := deck.StudioToDeckJSON("t", deck.StylePlain, []deck.StudioSlide{text, img, chart, num})
+		Expect(err).NotTo(HaveOccurred())
+		d := decodeDeck(raw)
+		for i, s := range d.Slides {
+			Expect(s.Layout).To(Equal("single"), "slide %d layout", i)
+		}
+	})
+
+	It("validates a brief", func() {
+		good := deck.Brief{Title: "Pilot", Goal: "Get a go", Audience: "Partners", Length: deck.LengthStandard, Tone: deck.TonePlain, Style: deck.StylePaper, Sources: []string{"d1"}}
+		Expect(good.Validate()).To(BeEmpty())
+		bad := good
+		bad.Title = ""
+		bad.Length = "huge"
+		bad.Tone = "angry"
+		bad.Style = "neon"
+		bad.Goal = strings.Repeat("g", 2001)
+		bad.Sources = make([]string, 51)
+		Expect(bad.Validate()).To(HaveLen(6))
+		long := good
+		long.Title = strings.Repeat("é", 201)
+		Expect(long.Validate()).To(HaveLen(1), "expected title length error")
+	})
+
+	It("maps deck lengths to slide counts", func() {
+		Expect(deck.LengthShort.Slides()).To(Equal(5))
+		Expect(deck.LengthStandard.Slides()).To(Equal(7))
+		Expect(deck.LengthLong.Slides()).To(Equal(8))
+	})
+
+	It("validates the enums", func() {
+		Expect(deck.StyleStudio.Valid()).To(BeTrue())
+		Expect(deck.StudioStyle("x").Valid()).To(BeFalse())
+		Expect(deck.LayoutNumber.Valid()).To(BeTrue())
+		Expect(deck.StudioLayout("x").Valid()).To(BeFalse())
+	})
+
+	Describe("ValidateStudioSlides", func() {
+		It("accepts valid slides, including negative chart values", func() {
+			Expect(deck.ValidateStudioSlides(allLayouts())).To(BeEmpty())
+
+			mixed := allLayouts()
+			mixed[3].Chart.Data = []deck.ChartPoint{{K: "a", V: -3}, {K: "b", V: 4}}
+			Expect(deck.ValidateStudioSlides(mixed)).To(BeEmpty(), "negative chart values should be accepted")
+		})
+
+		DescribeTable("rejects invalid slides",
+			func(mutate func(s []deck.StudioSlide)) {
+				s := allLayouts()
+				mutate(s)
+				Expect(deck.ValidateStudioSlides(s)).NotTo(BeEmpty(), "expected a validation error")
+			},
+			Entry("empty id", func(s []deck.StudioSlide) { s[0].ID = "" }),
+			Entry("duplicate id", func(s []deck.StudioSlide) { s[1].ID = s[0].ID }),
+			Entry("bad layout", func(s []deck.StudioSlide) { s[0].Layout = "grid" }),
+			Entry("bad status", func(s []deck.StudioSlide) { s[0].Status = "done" }),
+			Entry("bad flag", func(s []deck.StudioSlide) { s[0].Bullets[0].Flag = "maybe" }),
+			Entry("too many", func(s []deck.StudioSlide) { s[1].Bullets = make([]deck.StudioBullet, 9) }),
+			Entry("long title", func(s []deck.StudioSlide) { s[0].Title = strings.Repeat("x", 201) }),
+			Entry("long bullet", func(s []deck.StudioSlide) { s[0].Bullets[0].Text = strings.Repeat("x", 501) }),
+			Entry("long notes", func(s []deck.StudioSlide) { s[0].Notes = strings.Repeat("x", 5001) }),
+			Entry("no chart data", func(s []deck.StudioSlide) { s[3].Chart.Data = nil }),
+			Entry("chart 13 pts", func(s []deck.StudioSlide) { s[3].Chart.Data = make([]deck.ChartPoint, 13) }),
+			Entry("chart NaN", func(s []deck.StudioSlide) { s[3].Chart.Data[0].V = math.NaN() }),
+			Entry("chart Inf", func(s []deck.StudioSlide) { s[3].Chart.Data[0].V = math.Inf(-1) }),
+			Entry("chart kind", func(s []deck.StudioSlide) { s[3].Chart.Kind = "pie" }),
+			Entry("external image", func(s []deck.StudioSlide) { s[2].Image.Ref = "https://example.com/x.png" }),
+			Entry("long number", func(s []deck.StudioSlide) { s[4].Big.Num = strings.Repeat("9", 41) }),
+		)
+	})
+
+	It("uses the expected JSON tags", func() {
+		raw, err := json.Marshal(deck.StudioSlide{ID: "a", Layout: deck.LayoutText, Status: deck.SlideOK, SourceHashes: map[string]string{"d": "h"}})
+		Expect(err).NotTo(HaveOccurred())
+		for _, k := range []string{`"id"`, `"title"`, `"layout"`, `"bullets"`, `"sources"`, `"status"`, `"sourceHashes"`} {
+			Expect(string(raw)).To(ContainSubstring(k))
+		}
+		for _, k := range []string{`"sub"`, `"image"`, `"chart"`, `"big"`, `"notes"`, `"error"`} {
+			Expect(string(raw)).NotTo(ContainSubstring(k), "should be omitted")
+		}
+	})
+})

addons/office/internal/docx/docx.go

@@ -0,0 +1,233 @@
+// Package docx writes a Word document (.docx, OOXML) with the standard
+// library only. It builds the package with archive/zip and plain text, in the
+// same way as xlsx and deck/pptx. It has no reader.
+//
+// A Document is an ordered list of blocks: Heading, Paragraph, Bullets and
+// Table. Bullets use real Word numbering (word/numbering.xml), so Word shows
+// native lists. Text is plain strings. Page size is A4 with 2.5 cm margins.
+package docx
+
+import (
+	"archive/zip"
+	"errors"
+	"fmt"
+	"io"
+	"strings"
+)
+
+// Defaults used when a Style field is left at its zero value.
+const (
+	DefaultFont         = "Calibri"
+	DefaultBodySize     = 22 // half-points, so 11 pt
+	DefaultHeadingColor = "000000"
+)
+
+// Heading levels the writer supports.
+const (
+	minHeadingLevel = 1
+	maxHeadingLevel = 3
+)
+
+// Block is one element of the document body. The block types of this package
+// are Heading, Paragraph, Bullets and Table, as values (not pointers).
+type Block interface {
+	validate() error
+}
+
+// Heading is a section title. Level is 1 to 3.
+type Heading struct {
+	Level int
+	Text  string
+}
+
+// Run is a piece of paragraph text with its own emphasis.
+type Run struct {
+	Text   string
+	Bold   bool
+	Italic bool
+}
+
+// Paragraph is a body paragraph. Set either Text (plain, no emphasis) or
+// Runs (text with bold and italic pieces), not both. A paragraph with
+// neither is an empty line.
+type Paragraph struct {
+	Text string
+	Runs []Run
+}
+
+// Bullets is a bulleted list. It needs at least one item and no item may be
+// empty.
+type Bullets struct {
+	Items []string
+}
+
+// Table is a table with a header row and body rows. Every row has as many
+// cells as Header. Widths are column widths in twips (1/20 of a point). When
+// Widths is set it has one entry per column and no entry is negative. A zero
+// width gives that column an automatic share. An empty paragraph follows each
+// table, because Word needs a paragraph after a table at the end of the body.
+type Table struct {
+	Header []string
+	Rows   [][]string
+	Widths []int
+}
+
+// Style is the house look. Colours are six digit hex RGB values such as
+// "1F2937", with no "#". Zero values mean the defaults:
+//
+//   - Font: DefaultFont ("Calibri")
+//   - BodySize: DefaultBodySize (22 half-points, 11 pt)
+//   - HeadingColor: DefaultHeadingColor ("000000", black)
+//   - TableHeaderFill: no fill
+type Style struct {
+	Font            string
+	BodySize        int
+	HeadingColor    string
+	TableHeaderFill string
+}
+
+// Document is a whole Word file.
+type Document struct {
+	// Title and Author are written to docProps/core.xml.
+	Title  string
+	Author string
+	// Application is written to docProps/app.xml as the producing application.
+	// Empty means the element is left out. The library has no default name.
+	Application string
+	Style       Style
+	Blocks      []Block
+}
+
+func (h Heading) validate() error {
+	if h.Level < minHeadingLevel || h.Level > maxHeadingLevel {
+		return fmt.Errorf("heading level %d is not between %d and %d", h.Level, minHeadingLevel, maxHeadingLevel)
+	}
+	if strings.TrimSpace(h.Text) == "" {
+		return errors.New("heading text is empty")
+	}
+	return nil
+}
+
+func (p Paragraph) validate() error {
+	if p.Text != "" && len(p.Runs) > 0 {
+		return errors.New("paragraph sets both Text and Runs")
+	}
+	return nil
+}
+
+func (b Bullets) validate() error {
+	if len(b.Items) == 0 {
+		return errors.New("bullet list has no item")
+	}
+	for i, it := range b.Items {
+		if strings.TrimSpace(it) == "" {
+			return fmt.Errorf("bullet item %d is empty", i+1)
+		}
+	}
+	return nil
+}
+
+func (t Table) validate() error {
+	if len(t.Header) == 0 {
+		return errors.New("table has no header")
+	}
+	for i, r := range t.Rows {
+		if len(r) != len(t.Header) {
+			return fmt.Errorf("table row %d has %d cells, header has %d", i+1, len(r), len(t.Header))
+		}
+	}
+	if t.Widths != nil {
+		if len(t.Widths) != len(t.Header) {
+			return fmt.Errorf("table has %d widths, header has %d columns", len(t.Widths), len(t.Header))
+		}
+		for i, w := range t.Widths {
+			if w < 0 {
+				return fmt.Errorf("table column %d width is negative", i+1)
+			}
+		}
+	}
+	return nil
+}
+
+func validHex(s string) bool {
+	if len(s) != 6 {
+		return false
+	}
+	for _, c := range s {
+		if !(c >= '0' && c <= '9' || c >= 'a' && c <= 'f' || c >= 'A' && c <= 'F') {
+			return false
+		}
+	}
+	return true
+}
+
+// resolve fills the defaults and validates the style.
+func (s Style) resolve() (Style, error) {
+	if s.Font == "" {
+		s.Font = DefaultFont
+	}
+	if s.BodySize < 0 {
+		return s, errors.New("body size is negative")
+	}
+	if s.BodySize == 0 {
+		s.BodySize = DefaultBodySize
+	}
+	if s.HeadingColor == "" {
+		s.HeadingColor = DefaultHeadingColor
+	}
+	if !validHex(s.HeadingColor) {
+		return s, fmt.Errorf("heading colour %q must be six hex digits", s.HeadingColor)
+	}
+	if s.TableHeaderFill != "" && !validHex(s.TableHeaderFill) {
+		return s, fmt.Errorf("table header fill %q must be six hex digits", s.TableHeaderFill)
+	}
+	return s, nil
+}
+
+type builder struct {
+	doc        Document
+	style      Style
+	hasBullets bool
+}
+
+func build(d Document) (*builder, error) {
+	st, err := d.Style.resolve()
+	if err != nil {
+		return nil, fmt.Errorf("docx: %w", err)
+	}
+	b := &builder{doc: d, style: st}
+	for i, blk := range d.Blocks {
+		if blk == nil {
+			return nil, fmt.Errorf("docx: block %d is nil", i+1)
+		}
+		if err := blk.validate(); err != nil {
+			return nil, fmt.Errorf("docx: block %d: %w", i+1, err)
+		}
+		if _, ok := blk.(Bullets); ok {
+			b.hasBullets = true
+		}
+	}
+	return b, nil
+}
+
+// Write validates d and writes it to w as a .docx package.
+func Write(w io.Writer, d Document) error {
+	b, err := build(d)
+	if err != nil {
+		return err
+	}
+	zw := zip.NewWriter(w)
+	for _, p := range b.parts() {
+		f, err := zw.Create(p.name)
+		if err != nil {
+			return fmt.Errorf("docx: create %s: %w", p.name, err)
+		}
+		if _, err := f.Write(p.data); err != nil {
+			return fmt.Errorf("docx: write %s: %w", p.name, err)
+		}
+	}
+	if err := zw.Close(); err != nil {
+		return fmt.Errorf("docx: close: %w", err)
+	}
+	return nil
+}

addons/office/internal/docx/docx_suite_test.go

@@ -0,0 +1,13 @@
+package docx
+
+import (
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestDocx(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "Office docx writer suite")
+}

addons/office/internal/docx/docx_test.go

@@ -0,0 +1,432 @@
+package docx
+
+import (
+	"archive/zip"
+	"bytes"
+	"encoding/xml"
+	"io"
+	"path"
+	"strings"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// attrVal reads the w:val attribute of an element.
+type attrVal struct {
+	V string `xml:"val,attr"`
+}
+
+type xRun struct {
+	Bold   *struct{} `xml:"rPr>b"`
+	Italic *struct{} `xml:"rPr>i"`
+	Text   string    `xml:"t"`
+}
+
+type xPara struct {
+	Style attrVal `xml:"pPr>pStyle"`
+	NumID attrVal `xml:"pPr>numPr>numId"`
+	Runs  []xRun  `xml:"r"`
+}
+
+type xCell struct {
+	Width struct {
+		W string `xml:"w,attr"`
+	} `xml:"tcPr>tcW"`
+	Fill struct {
+		F string `xml:"fill,attr"`
+	} `xml:"tcPr>shd"`
+	Paras []xPara `xml:"p"`
+}
+
+type xTable struct {
+	Grid []struct {
+		W string `xml:"w,attr"`
+	} `xml:"tblGrid>gridCol"`
+	Rows []struct {
+		Cells []xCell `xml:"tc"`
+	} `xml:"tr"`
+}
+
+// element is one child of w:body, in document order.
+type element struct {
+	kind  string // "p", "tbl" or "sectPr"
+	para  xPara
+	table xTable
+}
+
+func writeDoc(d Document) map[string][]byte {
+	GinkgoHelper()
+	var buf bytes.Buffer
+	Expect(Write(&buf, d)).To(Succeed())
+	zr, err := zip.NewReader(bytes.NewReader(buf.Bytes()), int64(buf.Len()))
+	Expect(err).ToNot(HaveOccurred(), "not a zip")
+	files := map[string][]byte{}
+	for _, f := range zr.File {
+		rc, err := f.Open()
+		Expect(err).ToNot(HaveOccurred())
+		data, err := io.ReadAll(rc)
+		_ = rc.Close()
+		Expect(err).ToNot(HaveOccurred())
+		files[f.Name] = data
+	}
+	return files
+}
+
+func body(files map[string][]byte) []element {
+	GinkgoHelper()
+	dec := xml.NewDecoder(bytes.NewReader(files["word/document.xml"]))
+	var out []element
+	depth := 0
+	inBody := false
+	for {
+		tok, err := dec.Token()
+		if err == io.EOF {
+			break
+		}
+		Expect(err).ToNot(HaveOccurred(), "document.xml")
+		se, ok := tok.(xml.StartElement)
+		if !ok {
+			if _, end := tok.(xml.EndElement); end {
+				depth--
+			}
+			continue
+		}
+		depth++
+		if se.Name.Local == "body" {
+			inBody = true
+			continue
+		}
+		if !inBody || depth != 3 {
+			continue
+		}
+		e := element{kind: se.Name.Local}
+		switch se.Name.Local {
+		case "p":
+			Expect(dec.DecodeElement(&e.para, &se)).To(Succeed())
+		case "tbl":
+			Expect(dec.DecodeElement(&e.table, &se)).To(Succeed())
+		default:
+			Expect(dec.Skip()).To(Succeed())
+		}
+		depth--
+		out = append(out, e)
+	}
+	return out
+}
+
+func paraText(p xPara) string {
+	var s string
+	for _, r := range p.Runs {
+		s += r.Text
+	}
+	return s
+}
+
+func sampleDoc() Document {
+	return Document{
+		Title:  "Quarterly report",
+		Author: "A. Writer",
+		Style:  Style{Font: "Georgia", BodySize: 24, HeadingColor: "1f2937", TableHeaderFill: "dbeafe"},
+		Blocks: []Block{
+			Heading{Level: 1, Text: "Overview"},
+			Paragraph{Text: "Plain text."},
+			Heading{Level: 2, Text: "Details"},
+			Paragraph{Runs: []Run{{Text: "bold", Bold: true}, {Text: " and "}, {Text: "italic", Italic: true}}},
+			Bullets{Items: []string{"one", "two", "three"}},
+			Heading{Level: 3, Text: "Numbers"},
+			Table{
+				Header: []string{"Name", "Count"},
+				Rows:   [][]string{{"a", "1"}, {"b", "2"}},
+				Widths: []int{3000, 1500},
+			},
+		},
+	}
+}
+
+var _ = Describe("docx writer", func() {
+	It("writes well-formed parts with consistent content types and relationships", func() {
+		files := writeDoc(sampleDoc())
+		for _, name := range []string{
+			"[Content_Types].xml", "_rels/.rels", "word/document.xml", "word/styles.xml",
+			"word/numbering.xml", "word/_rels/document.xml.rels", "docProps/core.xml", "docProps/app.xml",
+		} {
+			data, ok := files[name]
+			Expect(ok).To(BeTrue(), "missing part %s", name)
+			Expect(xml.Unmarshal(data, new(struct{}))).To(Succeed(), "%s is not well-formed XML", name)
+		}
+
+		var ct struct {
+			Overrides []struct {
+				PartName    string `xml:"PartName,attr"`
+				ContentType string `xml:"ContentType,attr"`
+			} `xml:"Override"`
+		}
+		Expect(xml.Unmarshal(files["[Content_Types].xml"], &ct)).To(Succeed())
+		types := map[string]string{}
+		for _, o := range ct.Overrides {
+			types[o.PartName] = o.ContentType
+			Expect(files).To(HaveKey(strings.TrimPrefix(o.PartName, "/")), "content type override for missing part %s", o.PartName)
+		}
+		for _, name := range []string{"word/document.xml", "word/styles.xml", "word/numbering.xml", "docProps/core.xml", "docProps/app.xml"} {
+			Expect(types["/"+name]).ToNot(BeEmpty(), "[Content_Types].xml does not list %s", name)
+		}
+		Expect(types["/word/document.xml"]).To(ContainSubstring("wordprocessingml.document.main+xml"))
+
+		checkRels := func(relsName, baseDir string) {
+			var rels struct {
+				Rels []struct {
+					ID     string `xml:"Id,attr"`
+					Target string `xml:"Target,attr"`
+				} `xml:"Relationship"`
+			}
+			Expect(xml.Unmarshal(files[relsName], &rels)).To(Succeed())
+			Expect(rels.Rels).ToNot(BeEmpty(), "%s has no relationship", relsName)
+			ids := map[string]bool{}
+			for _, r := range rels.Rels {
+				Expect(ids[r.ID]).To(BeFalse(), "%s: duplicate id %s", relsName, r.ID)
+				ids[r.ID] = true
+				Expect(files).To(HaveKey(path.Join(baseDir, r.Target)), "%s: target %s does not exist", relsName, r.Target)
+			}
+		}
+		checkRels("_rels/.rels", "")
+		checkRels("word/_rels/document.xml.rels", "word")
+	})
+
+	It("writes no numbering part without a bullet list", func() {
+		files := writeDoc(Document{Blocks: []Block{Paragraph{Text: "x"}}})
+		Expect(files).ToNot(HaveKey("word/numbering.xml"))
+		Expect(files["[Content_Types].xml"]).ToNot(ContainSubstring("numbering"))
+		Expect(files["word/_rels/document.xml.rels"]).ToNot(ContainSubstring("numbering"))
+	})
+
+	It("writes the blocks in document order", func() {
+		els := body(writeDoc(sampleDoc()))
+		type want struct{ kind, style, text string }
+		wants := []want{
+			{"p", "Heading1", "Overview"},
+			{"p", "", "Plain text."},
+			{"p", "Heading2", "Details"},
+			{"p", "", "bold and italic"},
+			{"p", "ListBullet", "one"},
+			{"p", "ListBullet", "two"},
+			{"p", "ListBullet", "three"},
+			{"p", "Heading3", "Numbers"},
+			{"tbl", "", ""},
+			{"p", "", ""}, // spacer after the table
+			{"sectPr", "", ""},
+		}
+		Expect(els).To(HaveLen(len(wants)))
+		for i, w := range wants {
+			e := els[i]
+			Expect(e.kind).To(Equal(w.kind), "element %d", i)
+			if e.kind == "p" {
+				Expect(e.para.Style.V).To(Equal(w.style), "element %d style", i)
+				Expect(paraText(e.para)).To(Equal(w.text), "element %d text", i)
+			}
+		}
+	})
+
+	It("marks bold and italic runs", func() {
+		els := body(writeDoc(sampleDoc()))
+		runs := els[3].para.Runs
+		Expect(runs).To(HaveLen(3))
+		Expect(runs[0].Bold).ToNot(BeNil(), "run 0 should be bold only")
+		Expect(runs[0].Italic).To(BeNil(), "run 0 should be bold only")
+		Expect(runs[1].Bold).To(BeNil(), "run 1 should be plain")
+		Expect(runs[1].Italic).To(BeNil(), "run 1 should be plain")
+		Expect(runs[2].Bold).To(BeNil(), "run 2 should be italic only")
+		Expect(runs[2].Italic).ToNot(BeNil(), "run 2 should be italic only")
+	})
+
+	It("ties bullets to a bullet numbering definition", func() {
+		files := writeDoc(sampleDoc())
+		var num struct {
+			Abstract []struct {
+				ID     string  `xml:"abstractNumId,attr"`
+				Format attrVal `xml:"lvl>numFmt"`
+			} `xml:"abstractNum"`
+			Nums []struct {
+				ID       string  `xml:"numId,attr"`
+				Abstract attrVal `xml:"abstractNumId"`
+			} `xml:"num"`
+		}
+		Expect(xml.Unmarshal(files["word/numbering.xml"], &num)).To(Succeed())
+		Expect(num.Nums).To(HaveLen(1))
+		Expect(num.Abstract).To(HaveLen(1))
+		Expect(num.Abstract[0].Format.V).To(Equal("bullet"))
+		Expect(num.Nums[0].Abstract.V).To(Equal(num.Abstract[0].ID), "num does not refer to the abstract definition")
+		count := 0
+		for _, e := range body(files) {
+			if e.kind == "p" && e.para.Style.V == "ListBullet" {
+				count++
+				Expect(e.para.NumID.V).To(Equal(num.Nums[0].ID))
+			}
+		}
+		Expect(count).To(Equal(3))
+	})
+
+	It("writes a table with header emphasis, fill, grid and widths", func() {
+		els := body(writeDoc(sampleDoc()))
+		tbl := els[8].table
+		Expect(tbl.Rows).To(HaveLen(3))
+		var got [][]string
+		for _, r := range tbl.Rows {
+			var row []string
+			for _, c := range r.Cells {
+				row = append(row, paraText(c.Paras[0]))
+			}
+			got = append(got, row)
+		}
+		want := [][]string{{"Name", "Count"}, {"a", "1"}, {"b", "2"}}
+		Expect(got).To(Equal(want))
+		Expect(tbl.Rows[0].Cells[0].Paras[0].Runs[0].Bold).ToNot(BeNil(), "header cell is not bold")
+		Expect(tbl.Rows[1].Cells[0].Paras[0].Runs[0].Bold).To(BeNil(), "body cell is bold")
+		Expect(tbl.Rows[0].Cells[0].Fill.F).To(Equal("DBEAFE"))
+		Expect(tbl.Rows[0].Cells[1].Fill.F).To(Equal("DBEAFE"))
+		Expect(tbl.Rows[1].Cells[0].Fill.F).To(BeEmpty(), "body cell has a fill")
+		Expect(tbl.Grid).To(HaveLen(2))
+		Expect(tbl.Grid[0].W).To(Equal("3000"))
+		Expect(tbl.Grid[1].W).To(Equal("1500"))
+		Expect(tbl.Rows[1].Cells[1].Width.W).To(Equal("1500"))
+	})
+
+	It("gives table columns an equal share of the width by default", func() {
+		d := Document{Blocks: []Block{Table{Header: []string{"a", "b", "c"}}}}
+		tbl := body(writeDoc(d))[0].table
+		Expect(tbl.Grid).To(HaveLen(3))
+		for _, g := range tbl.Grid {
+			Expect(g.W).To(Equal(tbl.Grid[0].W), "grid widths are not an equal share: %+v", tbl.Grid)
+			Expect(g.W).ToNot(Equal("0"))
+			Expect(g.W).ToNot(BeEmpty())
+		}
+	})
+
+	It("writes the style values into styles.xml", func() {
+		files := writeDoc(sampleDoc())
+		var st struct {
+			Font struct {
+				A string `xml:"ascii,attr"`
+			} `xml:"docDefaults>rPrDefault>rPr>rFonts"`
+			Size   attrVal `xml:"docDefaults>rPrDefault>rPr>sz"`
+			Styles []struct {
+				ID    string  `xml:"styleId,attr"`
+				Color attrVal `xml:"rPr>color"`
+			} `xml:"style"`
+		}
+		Expect(xml.Unmarshal(files["word/styles.xml"], &st)).To(Succeed())
+		Expect(st.Font.A).To(Equal("Georgia"))
+		Expect(st.Size.V).To(Equal("24"))
+		colours := map[string]string{}
+		for _, s := range st.Styles {
+			colours[s.ID] = s.Color.V
+		}
+		for _, id := range []string{"Heading1", "Heading2", "Heading3"} {
+			Expect(colours[id]).To(Equal("1F2937"), id)
+		}
+		for _, id := range []string{"Normal", "ListBullet", "TableGrid"} {
+			Expect(colours).To(HaveKey(id), "styles.xml has no %s style", id)
+		}
+		Expect(files["word/document.xml"]).To(ContainSubstring(`w:fill="DBEAFE"`), "document.xml has no header fill")
+	})
+
+	It("uses default style values when none are set", func() {
+		files := writeDoc(Document{Blocks: []Block{
+			Heading{Level: 1, Text: "h"},
+			Table{Header: []string{"a"}},
+		}})
+		s := string(files["word/styles.xml"])
+		for _, want := range []string{`w:ascii="Calibri"`, `<w:sz w:val="22"/>`, `<w:color w:val="000000"/>`} {
+			Expect(s).To(ContainSubstring(want), "styles.xml lacks %s", want)
+		}
+		Expect(files["word/document.xml"]).ToNot(ContainSubstring("w:fill="), "header fill written although none was set")
+	})
+
+	It("names an application only when the caller sets one", func() {
+		files := writeDoc(Document{})
+		Expect(files["docProps/app.xml"]).ToNot(ContainSubstring("Application"))
+		files = writeDoc(Document{Application: `Tool <1> & "co"`})
+		var app struct {
+			Application string `xml:"Application"`
+		}
+		Expect(xml.Unmarshal(files["docProps/app.xml"], &app)).To(Succeed())
+		Expect(app.Application).To(Equal(`Tool <1> & "co"`))
+	})
+
+	It("writes the title and author into the core properties", func() {
+		files := writeDoc(Document{Title: `A & B`, Author: `"Me" <x>`})
+		var core struct {
+			Title   string `xml:"title"`
+			Creator string `xml:"creator"`
+		}
+		Expect(xml.Unmarshal(files["docProps/core.xml"], &core)).To(Succeed())
+		Expect(core.Title).To(Equal(`A & B`))
+		Expect(core.Creator).To(Equal(`"Me" <x>`))
+	})
+
+	DescribeTable("rejecting an invalid document",
+		func(doc Document) {
+			var buf bytes.Buffer
+			err := Write(&buf, doc)
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(HavePrefix("docx: "), "error lacks the package prefix")
+			Expect(buf.Len()).To(BeZero(), "output written although validation failed")
+		},
+		Entry("empty heading", Document{Blocks: []Block{Heading{Level: 1, Text: ""}}}),
+		Entry("blank heading", Document{Blocks: []Block{Heading{Level: 1, Text: "  "}}}),
+		Entry("level 0", Document{Blocks: []Block{Heading{Level: 0, Text: "x"}}}),
+		Entry("level 4", Document{Blocks: []Block{Heading{Level: 4, Text: "x"}}}),
+		Entry("bad heading colour", Document{Style: Style{HeadingColor: "12345"}, Blocks: nil}),
+		Entry("colour with hash", Document{Style: Style{HeadingColor: "#112233"}}),
+		Entry("non-hex colour", Document{Style: Style{HeadingColor: "GG0000"}}),
+		Entry("bad fill", Document{Style: Style{TableHeaderFill: "xyz"}}),
+		Entry("negative body size", Document{Style: Style{BodySize: -2}}),
+		Entry("ragged row", Document{Blocks: []Block{Table{Header: []string{"a", "b"}, Rows: [][]string{{"1", "2"}, {"3"}}}}}),
+		Entry("no header", Document{Blocks: []Block{Table{Rows: [][]string{{"1"}}}}}),
+		Entry("negative width", Document{Blocks: []Block{Table{Header: []string{"a", "b"}, Widths: []int{100, -1}}}}),
+		Entry("width count", Document{Blocks: []Block{Table{Header: []string{"a", "b"}, Widths: []int{100}}}}),
+		Entry("empty bullets", Document{Blocks: []Block{Bullets{}}}),
+		Entry("empty bullet item", Document{Blocks: []Block{Bullets{Items: []string{"a", ""}}}}),
+		Entry("text and runs", Document{Blocks: []Block{Paragraph{Text: "a", Runs: []Run{{Text: "b"}}}}}),
+		Entry("nil block", Document{Blocks: []Block{nil}}),
+	)
+
+	It("escapes special characters everywhere they are written", func() {
+		nasty := `a < b && c > "d" 'e'`
+		d := Document{
+			Title: nasty,
+			Style: Style{Font: `Font "X" & <Y>`},
+			Blocks: []Block{
+				Heading{Level: 1, Text: nasty},
+				Paragraph{Text: nasty},
+				Paragraph{Runs: []Run{{Text: nasty, Bold: true}}},
+				Bullets{Items: []string{nasty}},
+				Table{Header: []string{nasty}, Rows: [][]string{{nasty}}},
+			},
+		}
+		files := writeDoc(d)
+		for name, data := range files {
+			Expect(xml.Unmarshal(data, new(struct{}))).To(Succeed(), name)
+		}
+		els := body(files)
+		for i := range 4 {
+			Expect(paraText(els[i].para)).To(Equal(nasty), "element %d text", i)
+		}
+		tbl := els[4].table
+		Expect(paraText(tbl.Rows[0].Cells[0].Paras[0])).To(Equal(nasty), "table cell text did not round-trip")
+		Expect(paraText(tbl.Rows[1].Cells[0].Paras[0])).To(Equal(nasty), "table cell text did not round-trip")
+		var st struct {
+			Font struct {
+				A string `xml:"ascii,attr"`
+			} `xml:"docDefaults>rPrDefault>rPr>rFonts"`
+		}
+		Expect(xml.Unmarshal(files["word/styles.xml"], &st)).To(Succeed())
+		Expect(st.Font.A).To(Equal(`Font "X" & <Y>`))
+	})
+
+	It("writes only the section properties for an empty document", func() {
+		els := body(writeDoc(Document{}))
+		Expect(els).To(HaveLen(1))
+		Expect(els[0].kind).To(Equal("sectPr"))
+	})
+})

addons/office/internal/docx/package.go

@@ -0,0 +1,254 @@
+package docx
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/mudler/LocalAI/addons/office/internal/ooxml"
+)
+
+const (
+	xmlHeader = `<?xml version="1.0" encoding="UTF-8" standalone="yes"?>` + "\n"
+	nsW       = `http://schemas.openxmlformats.org/wordprocessingml/2006/main`
+	nsR       = `http://schemas.openxmlformats.org/officeDocument/2006/relationships`
+
+	relBase      = `http://schemas.openxmlformats.org/officeDocument/2006/relationships/`
+	relOfficeDoc = relBase + "officeDocument"
+	relCoreProps = `http://schemas.openxmlformats.org/package/2006/relationships/metadata/core-properties`
+	relExtProps  = relBase + "extended-properties"
+	relStyles    = relBase + "styles"
+	relNumbering = relBase + "numbering"
+
+	ctBase      = `application/vnd.openxmlformats-officedocument.`
+	ctDocument  = ctBase + "wordprocessingml.document.main+xml"
+	ctStyles    = ctBase + "wordprocessingml.styles+xml"
+	ctNumbering = ctBase + "wordprocessingml.numbering+xml"
+	ctCoreProps = `application/vnd.openxmlformats-package.core-properties+xml`
+	ctExtProps  = ctBase + "extended-properties+xml"
+)
+
+// Page layout in twips: A4 with 2.5 cm margins. The text width is the page
+// width minus both margins.
+const (
+	pageWidth  = 11906
+	pageHeight = 16838
+	margin     = 1417
+	textWidth  = pageWidth - 2*margin
+)
+
+// bulletNumID is the w:numId of the bullet list in numbering.xml.
+const bulletNumID = 1
+
+type part struct {
+	name string
+	data []byte
+}
+
+type rel struct{ id, typ, target string }
+
+func relsXML(rels []rel) []byte {
+	var b strings.Builder
+	b.WriteString(xmlHeader)
+	b.WriteString(`<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">`)
+	for _, r := range rels {
+		fmt.Fprintf(&b, `<Relationship Id="%s" Type="%s" Target="%s"/>`, r.id, r.typ, ooxml.Escape(r.target))
+	}
+	b.WriteString(`</Relationships>`)
+	return []byte(b.String())
+}
+
+// parts lays out every part of the package, [Content_Types].xml first.
+func (b *builder) parts() []part {
+	var ct strings.Builder
+	ct.WriteString(xmlHeader)
+	ct.WriteString(`<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">`)
+	ct.WriteString(`<Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/>`)
+	ct.WriteString(`<Default Extension="xml" ContentType="application/xml"/>`)
+
+	var parts []part
+	add := func(name string, data []byte, typ string) {
+		parts = append(parts, part{name: name, data: data})
+		if typ != "" {
+			fmt.Fprintf(&ct, `<Override PartName="/%s" ContentType="%s"/>`, name, typ)
+		}
+	}
+
+	add("_rels/.rels", relsXML([]rel{
+		{"rId1", relOfficeDoc, "word/document.xml"},
+		{"rId2", relCoreProps, "docProps/core.xml"},
+		{"rId3", relExtProps, "docProps/app.xml"},
+	}), "")
+	add("docProps/core.xml", coreXML(b.doc.Title, b.doc.Author), ctCoreProps)
+	add("docProps/app.xml", appXML(b.doc.Application), ctExtProps)
+
+	rels := []rel{{"rId1", relStyles, "styles.xml"}}
+	if b.hasBullets {
+		rels = append(rels, rel{"rId2", relNumbering, "numbering.xml"})
+	}
+	add("word/document.xml", b.documentXML(), ctDocument)
+	add("word/_rels/document.xml.rels", relsXML(rels), "")
+	add("word/styles.xml", b.stylesXML(), ctStyles)
+	if b.hasBullets {
+		add("word/numbering.xml", numberingXML(), ctNumbering)
+	}
+
+	ct.WriteString(`</Types>`)
+	return append([]part{{name: "[Content_Types].xml", data: []byte(ct.String())}}, parts...)
+}
+
+func coreXML(title, author string) []byte {
+	var s strings.Builder
+	s.WriteString(xmlHeader + `<cp:coreProperties xmlns:cp="http://schemas.openxmlformats.org/package/2006/metadata/core-properties" ` +
+		`xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dcterms="http://purl.org/dc/terms/" ` +
+		`xmlns:dcmitype="http://purl.org/dc/dcmitype/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">`)
+	s.WriteString(`<dc:title>` + ooxml.Escape(title) + `</dc:title>`)
+	s.WriteString(`<dc:creator>` + ooxml.Escape(author) + `</dc:creator>`)
+	s.WriteString(`</cp:coreProperties>`)
+	return []byte(s.String())
+}
+
+func appXML(application string) []byte {
+	var app string
+	if application != "" {
+		app = "<Application>" + ooxml.Escape(application) + "</Application>"
+	}
+	return []byte(xmlHeader + `<Properties xmlns="http://schemas.openxmlformats.org/officeDocument/2006/extended-properties" ` +
+		`xmlns:vt="http://schemas.openxmlformats.org/officeDocument/2006/docPropsVTypes">` + app + `</Properties>`)
+}
+
+func numberingXML() []byte {
+	return []byte(xmlHeader + `<w:numbering xmlns:w="` + nsW + `">` +
+		`<w:abstractNum w:abstractNumId="0"><w:multiLevelType w:val="hybridMultilevel"/>` +
+		`<w:lvl w:ilvl="0"><w:start w:val="1"/><w:numFmt w:val="bullet"/><w:lvlText w:val="&#8226;"/><w:lvlJc w:val="left"/>` +
+		`<w:pPr><w:ind w:left="720" w:hanging="360"/></w:pPr></w:lvl>` +
+		`</w:abstractNum>` +
+		fmt.Sprintf(`<w:num w:numId="%d"><w:abstractNumId w:val="0"/></w:num>`, bulletNumID) +
+		`</w:numbering>`)
+}
+
+// Heading sizes in half-points.
+var headingSize = map[int]int{1: 32, 2: 28, 3: 26}
+
+func (b *builder) stylesXML() []byte {
+	st := b.style
+	font := ooxml.Escape(st.Font)
+	var s strings.Builder
+	s.WriteString(xmlHeader)
+	s.WriteString(`<w:styles xmlns:w="` + nsW + `">`)
+	fmt.Fprintf(&s, `<w:docDefaults><w:rPrDefault><w:rPr><w:rFonts w:ascii="%[1]s" w:hAnsi="%[1]s" w:cs="%[1]s" w:eastAsia="%[1]s"/>`+
+		`<w:sz w:val="%[2]d"/><w:szCs w:val="%[2]d"/></w:rPr></w:rPrDefault>`+
+		`<w:pPrDefault><w:pPr><w:spacing w:after="120" w:line="264" w:lineRule="auto"/></w:pPr></w:pPrDefault></w:docDefaults>`,
+		font, st.BodySize)
+	s.WriteString(`<w:style w:type="paragraph" w:default="1" w:styleId="Normal"><w:name w:val="Normal"/><w:qFormat/></w:style>`)
+	for lvl := 1; lvl <= maxHeadingLevel; lvl++ {
+		fmt.Fprintf(&s, `<w:style w:type="paragraph" w:styleId="Heading%[1]d"><w:name w:val="heading %[1]d"/>`+
+			`<w:basedOn w:val="Normal"/><w:next w:val="Normal"/><w:qFormat/>`+
+			`<w:pPr><w:keepNext/><w:spacing w:before="240" w:after="120"/><w:outlineLvl w:val="%[2]d"/></w:pPr>`+
+			`<w:rPr><w:b/><w:color w:val="%[3]s"/><w:sz w:val="%[4]d"/><w:szCs w:val="%[4]d"/></w:rPr></w:style>`,
+			lvl, lvl-1, strings.ToUpper(st.HeadingColor), headingSize[lvl])
+	}
+	fmt.Fprintf(&s, `<w:style w:type="paragraph" w:styleId="ListBullet"><w:name w:val="List Bullet"/><w:basedOn w:val="Normal"/><w:qFormat/>`+
+		`<w:pPr><w:numPr><w:numId w:val="%d"/></w:numPr><w:spacing w:after="60"/><w:contextualSpacing/></w:pPr></w:style>`, bulletNumID)
+	s.WriteString(`<w:style w:type="table" w:default="1" w:styleId="TableNormal"><w:name w:val="Normal Table"/>` +
+		`<w:tblPr><w:tblInd w:w="0" w:type="dxa"/><w:tblCellMar><w:top w:w="0" w:type="dxa"/><w:left w:w="108" w:type="dxa"/>` +
+		`<w:bottom w:w="0" w:type="dxa"/><w:right w:w="108" w:type="dxa"/></w:tblCellMar></w:tblPr></w:style>`)
+	s.WriteString(`<w:style w:type="table" w:styleId="TableGrid"><w:name w:val="Table Grid"/><w:basedOn w:val="TableNormal"/>` +
+		`<w:tblPr><w:tblBorders>`)
+	for _, side := range []string{"top", "left", "bottom", "right", "insideH", "insideV"} {
+		fmt.Fprintf(&s, `<w:%s w:val="single" w:sz="4" w:space="0" w:color="auto"/>`, side)
+	}
+	s.WriteString(`</w:tblBorders></w:tblPr></w:style>`)
+	s.WriteString(`</w:styles>`)
+	return []byte(s.String())
+}
+
+func (b *builder) documentXML() []byte {
+	var x strings.Builder
+	x.WriteString(xmlHeader)
+	x.WriteString(`<w:document xmlns:w="` + nsW + `" xmlns:r="` + nsR + `"><w:body>`)
+	for _, blk := range b.doc.Blocks {
+		switch v := blk.(type) {
+		case Heading:
+			fmt.Fprintf(&x, `<w:p><w:pPr><w:pStyle w:val="Heading%d"/></w:pPr>%s</w:p>`, v.Level, runXML(Run{Text: v.Text}))
+		case Paragraph:
+			x.WriteString(`<w:p>`)
+			if v.Text != "" {
+				x.WriteString(runXML(Run{Text: v.Text}))
+			}
+			for _, r := range v.Runs {
+				x.WriteString(runXML(r))
+			}
+			x.WriteString(`</w:p>`)
+		case Bullets:
+			for _, it := range v.Items {
+				fmt.Fprintf(&x, `<w:p><w:pPr><w:pStyle w:val="ListBullet"/><w:numPr><w:ilvl w:val="0"/><w:numId w:val="%d"/></w:numPr></w:pPr>%s</w:p>`,
+					bulletNumID, runXML(Run{Text: it}))
+			}
+		case Table:
+			b.tableXML(&x, v)
+			x.WriteString(`<w:p/>`)
+		}
+	}
+	fmt.Fprintf(&x, `<w:sectPr><w:pgSz w:w="%d" w:h="%d"/><w:pgMar w:top="%d" w:right="%d" w:bottom="%d" w:left="%d" w:header="708" w:footer="708" w:gutter="0"/></w:sectPr>`,
+		pageWidth, pageHeight, margin, margin, margin, margin)
+	x.WriteString(`</w:body></w:document>`)
+	return []byte(x.String())
+}
+
+func runXML(r Run) string {
+	var rpr string
+	if r.Bold {
+		rpr += `<w:b/>`
+	}
+	if r.Italic {
+		rpr += `<w:i/>`
+	}
+	if rpr != "" {
+		rpr = `<w:rPr>` + rpr + `</w:rPr>`
+	}
+	return `<w:r>` + rpr + `<w:t xml:space="preserve">` + ooxml.Escape(r.Text) + `</w:t></w:r>`
+}
+
+// columnWidths returns the grid widths: the given widths, or an equal split of
+// the text width. A zero width in a partly set list takes an equal share of
+// the text width too.
+func columnWidths(t Table) []int {
+	n := len(t.Header)
+	w := make([]int, n)
+	for i := range w {
+		if t.Widths != nil && t.Widths[i] > 0 {
+			w[i] = t.Widths[i]
+		} else {
+			w[i] = textWidth / n
+		}
+	}
+	return w
+}
+
+func (b *builder) tableXML(x *strings.Builder, t Table) {
+	widths := columnWidths(t)
+	x.WriteString(`<w:tbl><w:tblPr><w:tblStyle w:val="TableGrid"/><w:tblW w:w="0" w:type="auto"/><w:tblLook w:val="04A0" w:firstRow="1"/></w:tblPr><w:tblGrid>`)
+	for _, w := range widths {
+		fmt.Fprintf(x, `<w:gridCol w:w="%d"/>`, w)
+	}
+	x.WriteString(`</w:tblGrid>`)
+	row := func(cells []string, header bool) {
+		x.WriteString(`<w:tr>`)
+		if header {
+			x.WriteString(`<w:trPr><w:tblHeader/></w:trPr>`)
+		}
+		for i, c := range cells {
+			fmt.Fprintf(x, `<w:tc><w:tcPr><w:tcW w:w="%d" w:type="dxa"/>`, widths[i])
+			if header && b.style.TableHeaderFill != "" {
+				fmt.Fprintf(x, `<w:shd w:val="clear" w:color="auto" w:fill="%s"/>`, strings.ToUpper(b.style.TableHeaderFill))
+			}
+			x.WriteString(`</w:tcPr><w:p>` + runXML(Run{Text: c, Bold: header}) + `</w:p></w:tc>`)
+		}
+		x.WriteString(`</w:tr>`)
+	}
+	row(t.Header, true)
+	for _, r := range t.Rows {
+		row(r, false)
+	}
+	x.WriteString(`</w:tbl>`)
+}

addons/office/internal/docxedit/docxedit.go

@@ -0,0 +1,654 @@
+// Package docxedit edits an existing Word document (.docx, OOXML) with the
+// standard library only. It is the edit side of the docx writer, the
+// officeread reader and the officecheck checker.
+//
+// Open parses the package and Bytes writes it back. An edit rewrites only the
+// bytes it touches inside word/document.xml. Every other part of the package
+// is copied without change, and the zip entries keep their order.
+//
+// Scope of this version. Only top-level body paragraphs are addressable: a
+// w:p that is a direct child of w:body. Paragraphs inside tables, text boxes,
+// content controls, headers, footers, footnotes and comments are not listed
+// and not edited. Images, tables, comments and other parts are never changed.
+// Tracked changes are not interpreted: the text of inserted runs counts as
+// text and deleted text (w:delText) does not.
+package docxedit
+
+import (
+	"archive/zip"
+	"bytes"
+	"encoding/xml"
+	"errors"
+	"fmt"
+	"io"
+	"path"
+	"sort"
+	"strings"
+
+	"github.com/mudler/LocalAI/addons/office/internal/ooxml"
+)
+
+const (
+	nsW       = `http://schemas.openxmlformats.org/wordprocessingml/2006/main`
+	relOffice = `http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument`
+	relStyles = `http://schemas.openxmlformats.org/officeDocument/2006/relationships/styles`
+)
+
+// Limits on what is unpacked.
+const (
+	maxEntry int64 = 64 << 20
+	maxTotal int64 = 256 << 20
+)
+
+// ParagraphInfo describes one top-level body paragraph.
+type ParagraphInfo struct {
+	// Index is the 0-based position among the top-level body paragraphs.
+	Index int
+	// Style is the w:pStyle id, empty when the paragraph has none.
+	Style string
+	// Text is the text of the paragraph: w:t text, with a tab for w:tab and a
+	// newline for w:br and w:cr.
+	Text string
+}
+
+// Document is an opened .docx file. It is not safe for concurrent use.
+type Document struct {
+	raw     []byte // the original package
+	zr      *zip.Reader
+	main    string // name of the main document part
+	body    []byte // current content of the main part
+	changed bool
+	styles  map[string]bool // paragraph style ids defined in the styles part
+}
+
+// Open parses data as a .docx file. It returns an error when data is not a
+// zip, has an unsafe or duplicate entry name, is too large, has no main
+// document part, or the main part is not well-formed WordprocessingML. It
+// never panics.
+func Open(data []byte) (doc *Document, err error) {
+	defer recoverErr(&err)
+	zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
+	if err != nil {
+		return nil, fmt.Errorf("docxedit: not a zip file: %w", err)
+	}
+	files := map[string]*zip.File{}
+	var total int64
+	for _, f := range zr.File {
+		name := f.Name
+		if strings.HasPrefix(name, "/") || strings.HasPrefix(name, "\\") || ooxml.HasDotDot(name) {
+			return nil, fmt.Errorf("docxedit: unsafe entry name %q", name)
+		}
+		if _, dup := files[name]; dup {
+			return nil, fmt.Errorf("docxedit: duplicate entry %q", name)
+		}
+		files[name] = f
+		if f.UncompressedSize64 > uint64(maxEntry) {
+			return nil, fmt.Errorf("docxedit: entry %q is larger than %d bytes", name, maxEntry)
+		}
+		total += int64(f.UncompressedSize64)
+		if total > maxTotal {
+			return nil, fmt.Errorf("docxedit: package is larger than %d bytes", maxTotal)
+		}
+	}
+	d := &Document{raw: append([]byte(nil), data...), zr: zr}
+	d.zr, err = zip.NewReader(bytes.NewReader(d.raw), int64(len(d.raw)))
+	if err != nil {
+		return nil, fmt.Errorf("docxedit: not a zip file: %w", err)
+	}
+	read := func(name string) ([]byte, bool, error) {
+		f, ok := files[name]
+		if !ok {
+			return nil, false, nil
+		}
+		for _, g := range d.zr.File {
+			if g.Name == name {
+				f = g
+				break
+			}
+		}
+		rc, err := f.Open()
+		if err != nil {
+			return nil, true, fmt.Errorf("docxedit: cannot open %q: %w", name, err)
+		}
+		defer func() { _ = rc.Close() }()
+		b, err := io.ReadAll(io.LimitReader(rc, maxEntry+1))
+		if err != nil {
+			return nil, true, fmt.Errorf("docxedit: cannot read %q: %w", name, err)
+		}
+		if int64(len(b)) > maxEntry {
+			return nil, true, fmt.Errorf("docxedit: entry %q is larger than %d bytes", name, maxEntry)
+		}
+		return b, true, nil
+	}
+
+	rels, ok, err := read("_rels/.rels")
+	if err != nil {
+		return nil, err
+	}
+	if !ok {
+		return nil, errors.New("docxedit: not a docx file: missing _rels/.rels")
+	}
+	target, err := findRel(rels, relOffice)
+	if err != nil {
+		return nil, err
+	}
+	d.main = resolve("", target)
+	if d.main == "" {
+		return nil, errors.New("docxedit: not a docx file: bad main part target")
+	}
+	d.body, ok, err = read(d.main)
+	if err != nil {
+		return nil, err
+	}
+	if !ok {
+		return nil, fmt.Errorf("docxedit: main part %q is missing", d.main)
+	}
+	if _, err := scan(d.body); err != nil {
+		return nil, err
+	}
+
+	// Paragraph styles come from the styles part of the main part, if any.
+	relName := path.Join(path.Dir(d.main), "_rels", path.Base(d.main)+".rels")
+	if mrels, ok, err := read(relName); err != nil {
+		return nil, err
+	} else if ok {
+		if st, err := findRel(mrels, relStyles); err == nil {
+			if sdata, ok, err := read(resolve(path.Dir(d.main), st)); err != nil {
+				return nil, err
+			} else if ok {
+				d.styles, err = styleIDs(sdata)
+				if err != nil {
+					return nil, err
+				}
+			}
+		}
+	}
+	return d, nil
+}
+
+// Paragraphs lists the top-level body paragraphs in body order. Paragraphs
+// inside tables are not listed.
+func (d *Document) Paragraphs() []ParagraphInfo {
+	s, err := scan(d.body)
+	if err != nil {
+		return nil
+	}
+	out := make([]ParagraphInfo, len(s.paras))
+	for i, p := range s.paras {
+		out[i] = ParagraphInfo{Index: i, Style: p.style, Text: p.text.String()}
+	}
+	return out
+}
+
+// SetText replaces the text of the top-level paragraph at index. The
+// paragraph start tag and its paragraph properties (w:pPr) are kept, and so
+// are the run properties (w:rPr) of the first text run. The text becomes one
+// run; all other content of the paragraph (other runs, hyperlinks,
+// bookmarks, fields) is dropped. It is an error when index is out of range or
+// the paragraph has no text run (no w:t), for example an empty paragraph.
+func (d *Document) SetText(index int, text string) (err error) {
+	defer recoverErr(&err)
+	s, err := scan(d.body)
+	if err != nil {
+		return err
+	}
+	p, err := s.para(index)
+	if err != nil {
+		return err
+	}
+	if !p.hasText {
+		return fmt.Errorf("docxedit: paragraph %d has no text run", index)
+	}
+	var b bytes.Buffer
+	b.Write(d.body[p.start:p.tagEnd])
+	if p.hasPPr {
+		b.Write(d.body[p.pPr[0]:p.pPr[1]])
+	}
+	b.WriteString(runXML(s.prefix, d.rPrBytes(p), text))
+	b.WriteString("</" + p.name + ">")
+	d.replace(p.start, p.end, b.Bytes())
+	return nil
+}
+
+func (d *Document) rPrBytes(p *para) []byte {
+	if p.hasRPr {
+		return d.body[p.rPr[0]:p.rPr[1]]
+	}
+	return nil
+}
+
+// InsertParagraph inserts a new top-level paragraph after the paragraph at
+// index after; -1 inserts at the start of the body. When style is not empty
+// it must be a paragraph style id defined in the styles part (word/styles.xml)
+// of the document, otherwise it is an error. An empty style gives a paragraph
+// with no w:pStyle. An empty text gives a paragraph with no run.
+func (d *Document) InsertParagraph(after int, style string, text string) (err error) {
+	defer recoverErr(&err)
+	s, err := scan(d.body)
+	if err != nil {
+		return err
+	}
+	pos := s.bodyTagEnd
+	if after != -1 {
+		p, err := s.para(after)
+		if err != nil {
+			return err
+		}
+		pos = p.end
+	}
+	if style != "" && !d.styles[style] {
+		return fmt.Errorf("docxedit: paragraph style %q is not defined in the styles part", style)
+	}
+	var b strings.Builder
+	pf := s.prefix
+	b.WriteString("<" + pf + "p>")
+	if style != "" {
+		b.WriteString("<" + pf + "pPr><" + pf + `pStyle ` + pf + `val="` + ooxml.Escape(style) + `"/></` + pf + "pPr>")
+	}
+	if text != "" {
+		b.WriteString(runXML(pf, nil, text))
+	}
+	b.WriteString("</" + pf + "p>")
+	d.replace(pos, pos, []byte(b.String()))
+	return nil
+}
+
+// DeleteParagraph removes the top-level paragraph at index. It is an error
+// when index is out of range, or when the removal would leave the body empty
+// or end it with a table (Word needs a paragraph after a final table). A
+// paragraph whose properties hold a section break (w:sectPr) takes the break
+// with it.
+func (d *Document) DeleteParagraph(index int) (err error) {
+	defer recoverErr(&err)
+	s, err := scan(d.body)
+	if err != nil {
+		return err
+	}
+	p, err := s.para(index)
+	if err != nil {
+		return err
+	}
+	last := len(s.blocks) - 1
+	if last == 0 {
+		return errors.New("docxedit: cannot delete the only block of the body")
+	}
+	if p.block == last && s.blocks[last-1] == "tbl" {
+		return fmt.Errorf("docxedit: cannot delete paragraph %d: it is the paragraph after the last table", index)
+	}
+	d.replace(p.start, p.end, nil)
+	return nil
+}
+
+// ReplaceAll replaces every occurrence of old with new in the text of the
+// top-level paragraphs and returns the number of replacements. Matching is
+// done on the text of one w:t element at a time, so old is found only when it
+// lies entirely inside one run's text. Text that Word split across several
+// runs is not matched. Paragraphs inside tables, text boxes, headers and
+// footers are not searched. Run properties are not changed. It is an error
+// when old is empty.
+func (d *Document) ReplaceAll(old, new string) (count int, err error) {
+	defer recoverErr(&err)
+	if old == "" {
+		return 0, errors.New("docxedit: the text to replace is empty")
+	}
+	s, err := scan(d.body)
+	if err != nil {
+		return 0, err
+	}
+	var edits []edit
+	for _, t := range s.ts {
+		if t.selfClose || !strings.Contains(t.text, old) {
+			continue
+		}
+		count += strings.Count(t.text, old)
+		if !t.hasSpace {
+			edits = append(edits, edit{t.bodyStart - 1, t.bodyStart - 1, []byte(` xml:space="preserve"`)})
+		}
+		edits = append(edits, edit{t.bodyStart, t.bodyEnd, []byte(ooxml.Escape(strings.ReplaceAll(t.text, old, new)))})
+	}
+	if count == 0 {
+		return 0, nil
+	}
+	d.body = applyEdits(d.body, edits)
+	d.changed = true
+	return count, nil
+}
+
+// Bytes returns the document as a .docx package. Only word/document.xml is
+// rewritten, and only when an edit changed it. Every other entry is copied as
+// it was, in the original order.
+func (d *Document) Bytes() (out []byte, err error) {
+	defer recoverErr(&err)
+	var buf bytes.Buffer
+	zw := zip.NewWriter(&buf)
+	for _, f := range d.zr.File {
+		if d.changed && f.Name == d.main {
+			h := &zip.FileHeader{Name: f.Name, Method: zip.Deflate, Modified: f.Modified}
+			w, err := zw.CreateHeader(h)
+			if err != nil {
+				return nil, fmt.Errorf("docxedit: create %s: %w", f.Name, err)
+			}
+			if _, err := w.Write(d.body); err != nil {
+				return nil, fmt.Errorf("docxedit: write %s: %w", f.Name, err)
+			}
+			continue
+		}
+		if err := zw.Copy(f); err != nil {
+			return nil, fmt.Errorf("docxedit: copy %s: %w", f.Name, err)
+		}
+	}
+	if err := zw.Close(); err != nil {
+		return nil, fmt.Errorf("docxedit: close: %w", err)
+	}
+	return buf.Bytes(), nil
+}
+
+// ---- edits ----
+
+type edit struct {
+	start, end int
+	data       []byte
+}
+
+func (d *Document) replace(start, end int, data []byte) {
+	d.body = applyEdits(d.body, []edit{{start, end, data}})
+	d.changed = true
+}
+
+func applyEdits(src []byte, edits []edit) []byte {
+	sort.SliceStable(edits, func(i, j int) bool { return edits[i].start < edits[j].start })
+	var out bytes.Buffer
+	pos := 0
+	for _, e := range edits {
+		out.Write(src[pos:e.start])
+		out.Write(e.data)
+		pos = e.end
+	}
+	out.Write(src[pos:])
+	return out.Bytes()
+}
+
+func runXML(prefix string, rPr []byte, text string) string {
+	return "<" + prefix + "r>" + string(rPr) + "<" + prefix + `t xml:space="preserve">` + ooxml.Escape(text) + "</" + prefix + "t></" + prefix + "r>"
+}
+
+func recoverErr(err *error) {
+	if r := recover(); r != nil {
+		*err = fmt.Errorf("docxedit: internal error: %v", r)
+	}
+}
+
+// ---- package helpers ----
+
+// resolve turns a relationship target into a part name. A target that starts
+// with "/" is from the package root, any other is relative to base.
+func resolve(base, target string) string {
+	var p string
+	if after, ok := strings.CutPrefix(target, "/"); ok {
+		p = path.Clean(after)
+	} else {
+		p = path.Join(base, target)
+	}
+	if p == "." || p == ".." || strings.HasPrefix(p, "../") {
+		return ""
+	}
+	return p
+}
+
+// findRel returns the target of the first relationship of the given type.
+func findRel(data []byte, typ string) (string, error) {
+	d := xml.NewDecoder(bytes.NewReader(data))
+	for {
+		tok, err := d.Token()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			return "", fmt.Errorf("docxedit: bad relationships part: %w", err)
+		}
+		se, ok := tok.(xml.StartElement)
+		if !ok || se.Name.Local != "Relationship" {
+			continue
+		}
+		var t, target, mode string
+		for _, a := range se.Attr {
+			switch a.Name.Local {
+			case "Type":
+				t = a.Value
+			case "Target":
+				target = a.Value
+			case "TargetMode":
+				mode = a.Value
+			}
+		}
+		if t == typ && mode != "External" && target != "" {
+			return target, nil
+		}
+	}
+	return "", fmt.Errorf("docxedit: no relationship of type %s", path.Base(typ))
+}
+
+// styleIDs returns the ids of the paragraph styles defined in a styles part.
+func styleIDs(data []byte) (map[string]bool, error) {
+	ids := map[string]bool{}
+	d := xml.NewDecoder(bytes.NewReader(data))
+	for {
+		tok, err := d.Token()
+		if err == io.EOF {
+			return ids, nil
+		}
+		if err != nil {
+			return nil, fmt.Errorf("docxedit: bad styles part: %w", err)
+		}
+		se, ok := tok.(xml.StartElement)
+		if !ok || se.Name.Local != "style" || se.Name.Space != nsW {
+			continue
+		}
+		var id, typ string
+		for _, a := range se.Attr {
+			switch a.Name.Local {
+			case "styleId":
+				id = a.Value
+			case "type":
+				typ = a.Value
+			}
+		}
+		if id != "" && (typ == "" || typ == "paragraph") {
+			ids[id] = true
+		}
+	}
+}
+
+// ---- scanning the main part ----
+
+type para struct {
+	start, tagEnd, end int    // byte range, and end of the start tag
+	name               string // qualified name of the start tag
+	block              int    // position among the body blocks
+	hasPPr             bool
+	pPr                [2]int
+	hasRPr             bool
+	rPr                [2]int // run properties of the first text run
+	hasText            bool
+	style              string
+	text               strings.Builder
+}
+
+// tspan is one w:t element inside a top-level paragraph.
+type tspan struct {
+	bodyStart, bodyEnd int // range of the text between the tags
+	selfClose          bool
+	hasSpace           bool // the start tag has an xml:space attribute
+	text               string
+}
+
+type scanned struct {
+	paras      []*para
+	blocks     []string // local names of the body children, without w:sectPr
+	ts         []tspan
+	bodyTagEnd int
+	prefix     string // prefix of WordprocessingML elements, with the colon
+}
+
+func (s *scanned) para(i int) (*para, error) {
+	if i < 0 || i >= len(s.paras) {
+		return nil, fmt.Errorf("docxedit: paragraph index %d is out of range (0 to %d)", i, len(s.paras)-1)
+	}
+	return s.paras[i], nil
+}
+
+// rawName returns the qualified name of the element that starts at data[0]=='<'.
+func rawName(data []byte) string {
+	i := 1
+	for i < len(data) && !strings.ContainsRune(" \t\r\n/>", rune(data[i])) {
+		i++
+	}
+	return string(data[1:i])
+}
+
+func scan(data []byte) (*scanned, error) {
+	d := xml.NewDecoder(bytes.NewReader(data))
+	s := &scanned{bodyTagEnd: -1}
+	var (
+		stack   []xml.Name
+		cur     *para
+		nested  int // paragraphs nested inside cur (text boxes)
+		inT     bool
+		tIdx    = -1
+		runRPr  [2]int
+		hasRun  bool
+		chosen  bool
+		rPrFrom int
+	)
+	isW := func(n xml.Name, local string) bool { return n.Space == nsW && n.Local == local }
+	for {
+		off0 := int(d.InputOffset())
+		tok, err := d.Token()
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			return nil, fmt.Errorf("docxedit: bad document XML: %w", err)
+		}
+		switch t := tok.(type) {
+		case xml.StartElement:
+			stack = append(stack, t.Name)
+			depth := len(stack)
+			if depth == 1 && !isW(t.Name, "document") {
+				return nil, errors.New("docxedit: the main part is not a WordprocessingML document")
+			}
+			if depth == 2 && isW(t.Name, "body") {
+				s.bodyTagEnd = int(d.InputOffset())
+				rn := rawName(data[off0:])
+				s.prefix = ""
+				if i := strings.IndexByte(rn, ':'); i >= 0 {
+					s.prefix = rn[:i+1]
+				}
+				continue
+			}
+			if depth == 3 && isW(stack[1], "body") {
+				switch {
+				case isW(t.Name, "p"):
+					cur = &para{start: off0, tagEnd: int(d.InputOffset()), name: rawName(data[off0:]), block: len(s.blocks)}
+					nested, chosen, hasRun = 0, false, false
+					s.paras = append(s.paras, cur)
+					s.blocks = append(s.blocks, "p")
+				case isW(t.Name, "sectPr"):
+				default:
+					s.blocks = append(s.blocks, t.Name.Local)
+				}
+				continue
+			}
+			if cur == nil || depth <= 3 {
+				continue
+			}
+			if isW(t.Name, "p") {
+				nested++
+				continue
+			}
+			if nested > 0 {
+				continue
+			}
+			parent := stack[depth-2]
+			switch {
+			case depth == 4 && isW(t.Name, "pPr"):
+				cur.hasPPr, cur.pPr[0] = true, off0
+			case depth == 5 && isW(t.Name, "pStyle") && isW(parent, "pPr"):
+				for _, a := range t.Attr {
+					if a.Name.Local == "val" {
+						cur.style = a.Value
+					}
+				}
+			case isW(t.Name, "r"):
+				hasRun = false
+			case isW(t.Name, "rPr") && isW(parent, "r"):
+				rPrFrom = off0
+			case isW(t.Name, "t"):
+				ts := tspan{bodyStart: int(d.InputOffset())}
+				for _, a := range t.Attr {
+					if a.Name.Local == "space" {
+						ts.hasSpace = true
+					}
+				}
+				s.ts = append(s.ts, ts)
+				tIdx, inT = len(s.ts)-1, true
+			case (isW(t.Name, "tab") || isW(t.Name, "br") || isW(t.Name, "cr")) && isW(parent, "r"):
+				if t.Name.Local == "tab" {
+					cur.text.WriteByte('\t')
+				} else {
+					cur.text.WriteByte('\n')
+				}
+			}
+		case xml.EndElement:
+			depth := len(stack)
+			if depth == 0 {
+				return nil, errors.New("docxedit: bad document XML: unbalanced tags")
+			}
+			name := stack[depth-1]
+			stack = stack[:depth-1]
+			if cur != nil && depth == 3 && isW(name, "p") {
+				cur.end = int(d.InputOffset())
+				cur = nil
+			}
+			if cur == nil || depth <= 3 {
+				continue
+			}
+			if isW(name, "p") {
+				if nested > 0 {
+					nested--
+				}
+				continue
+			}
+			if nested > 0 {
+				continue
+			}
+			switch {
+			case depth == 4 && isW(name, "pPr"):
+				cur.pPr[1] = int(d.InputOffset())
+			case isW(name, "rPr") && depth >= 5 && isW(stack[depth-2], "r"):
+				runRPr, hasRun = [2]int{rPrFrom, int(d.InputOffset())}, true
+			case isW(name, "t"):
+				ts := &s.ts[tIdx]
+				ts.bodyEnd = off0
+				ts.selfClose = ts.bodyEnd == ts.bodyStart && data[ts.bodyStart-1] == '>' && data[ts.bodyStart-2] == '/'
+				cur.hasText = true
+				if !chosen {
+					chosen = true
+					if hasRun {
+						cur.hasRPr, cur.rPr = true, runRPr
+					}
+				}
+				inT = false
+			}
+		case xml.CharData:
+			if inT && cur != nil && nested == 0 {
+				s.ts[tIdx].text += string(t)
+				cur.text.Write(t)
+			}
+		}
+	}
+	if s.bodyTagEnd < 0 {
+		return nil, errors.New("docxedit: the document has no body")
+	}
+	return s, nil
+}

addons/office/internal/docxedit/docxedit_suite_test.go

@@ -0,0 +1,13 @@
+package docxedit
+
+import (
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestDocxedit(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "Office docxedit suite")
+}

addons/office/internal/docxedit/docxedit_test.go

@@ -0,0 +1,352 @@
+package docxedit
+
+import (
+	"archive/zip"
+	"bytes"
+	"io"
+	"strings"
+
+	"github.com/mudler/LocalAI/addons/office/internal/docx"
+	"github.com/mudler/LocalAI/addons/office/internal/officecheck"
+	"github.com/mudler/LocalAI/addons/office/internal/officeread"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// Body order of the source document, top-level paragraphs only:
+//
+//	0 Heading1 "Title"
+//	1 "Hello world, hello"
+//	2 "Bold lead" (bold run) + " plain tail"
+//	3 ListBullet "alpha"
+//	4 ListBullet "beta"
+//	  (table: header Name, Note; row "hello", "hello table")
+//	5 empty paragraph that docx.Write puts after a table
+//	6 "Last hello"
+func source() []byte {
+	GinkgoHelper()
+	var buf bytes.Buffer
+	err := docx.Write(&buf, docx.Document{
+		Title: "Sample", Author: "Tester",
+		Blocks: []docx.Block{
+			docx.Heading{Level: 1, Text: "Title"},
+			docx.Paragraph{Text: "Hello world, hello"},
+			docx.Paragraph{Runs: []docx.Run{{Text: "Bold lead", Bold: true}, {Text: " plain tail"}}},
+			docx.Bullets{Items: []string{"alpha", "beta"}},
+			docx.Table{Header: []string{"Name", "Note"}, Rows: [][]string{{"hello", "hello table"}}},
+			docx.Paragraph{Text: "Last hello"},
+		},
+	})
+	Expect(err).ToNot(HaveOccurred())
+	return buf.Bytes()
+}
+
+func open() *Document {
+	GinkgoHelper()
+	d, err := Open(source())
+	Expect(err).ToNot(HaveOccurred())
+	return d
+}
+
+// finish writes the document, checks it with officecheck and returns the
+// bytes and the text read back.
+func finish(d *Document) ([]byte, string) {
+	GinkgoHelper()
+	b, err := d.Bytes()
+	Expect(err).ToNot(HaveOccurred())
+	Expect(officecheck.Check(b)).To(BeEmpty(), "officecheck problems")
+	r, err := officeread.Read(b)
+	Expect(err).ToNot(HaveOccurred())
+	return b, r.Parts[0].Text
+}
+
+func texts(d *Document) []string {
+	var out []string
+	for _, p := range d.Paragraphs() {
+		out = append(out, p.Text)
+	}
+	return out
+}
+
+func zipParts(b []byte) (names []string, data map[string][]byte) {
+	GinkgoHelper()
+	zr, err := zip.NewReader(bytes.NewReader(b), int64(len(b)))
+	Expect(err).ToNot(HaveOccurred())
+	data = map[string][]byte{}
+	for _, f := range zr.File {
+		rc, err := f.Open()
+		Expect(err).ToNot(HaveOccurred())
+		c, _ := io.ReadAll(rc)
+		_ = rc.Close()
+		names = append(names, f.Name)
+		data[f.Name] = c
+	}
+	return
+}
+
+func replacePart(b []byte, name string, content string) []byte {
+	GinkgoHelper()
+	names, data := zipParts(b)
+	var out bytes.Buffer
+	zw := zip.NewWriter(&out)
+	for _, n := range names {
+		w, _ := zw.Create(n)
+		if n == name {
+			_, _ = w.Write([]byte(content))
+		} else {
+			_, _ = w.Write(data[n])
+		}
+	}
+	_ = zw.Close()
+	return out.Bytes()
+}
+
+var _ = Describe("docxedit", func() {
+	It("opens a document and lists its top-level paragraphs", func() {
+		src := source()
+		d, err := Open(src)
+		Expect(err).ToNot(HaveOccurred())
+		ps := d.Paragraphs()
+		want := []ParagraphInfo{
+			{0, "Heading1", "Title"},
+			{1, "", "Hello world, hello"},
+			{2, "", "Bold lead plain tail"},
+			{3, "ListBullet", "alpha"},
+			{4, "ListBullet", "beta"},
+			{5, "", ""},
+			{6, "", "Last hello"},
+		}
+		Expect(ps).To(Equal(want))
+		r, err := officeread.Read(src)
+		Expect(err).ToNot(HaveOccurred())
+		for _, s := range []string{"# Title", "Hello world, hello", "- alpha", "hello\thello table", "Last hello"} {
+			Expect(r.Parts[0].Text).To(ContainSubstring(s), "source text lacks %q", s)
+		}
+	})
+
+	It("keeps every part when nothing is edited", func() {
+		src := source()
+		d, _ := Open(src)
+		b, _ := finish(d)
+		n1, d1 := zipParts(src)
+		n2, d2 := zipParts(b)
+		Expect(n2).To(Equal(n1), "entry order changed")
+		for _, n := range n1 {
+			Expect(bytes.Equal(d1[n], d2[n])).To(BeTrue(), "part %s changed", n)
+		}
+	})
+
+	It("replaces paragraph text and keeps style, run properties and list properties", func() {
+		d := open()
+		Expect(d.SetText(0, "New <title> & more")).To(Succeed())
+		Expect(d.SetText(2, "Rewritten")).To(Succeed())
+		b, text := finish(d)
+		Expect(text).To(ContainSubstring("# New <title> & more"), "heading not replaced (style lost?)")
+		Expect(text).To(ContainSubstring("Rewritten"))
+		Expect(text).ToNot(ContainSubstring("Bold lead"))
+		Expect(text).ToNot(ContainSubstring("plain tail"))
+		Expect(text).To(ContainSubstring("Hello world, hello"), "other paragraphs changed")
+		Expect(text).To(ContainSubstring("- alpha"), "other paragraphs changed")
+		d2, err := Open(b)
+		Expect(err).ToNot(HaveOccurred())
+		ps := d2.Paragraphs()
+		Expect(ps[0].Style).To(Equal("Heading1"))
+		Expect(ps[2].Text).To(Equal("Rewritten"))
+		// The first run's run properties (bold) are kept.
+		_, data := zipParts(b)
+		Expect(string(data["word/document.xml"])).To(ContainSubstring(`<w:r><w:rPr><w:b/></w:rPr><w:t xml:space="preserve">Rewritten</w:t></w:r>`), "run properties lost")
+		// List numbering properties are kept.
+		d3 := open()
+		Expect(d3.SetText(3, "first item")).To(Succeed())
+		_, text = finish(d3)
+		Expect(text).To(ContainSubstring("- first item"), "list paragraph lost its list properties")
+	})
+
+	It("inserts paragraphs with a style at the start, the middle and the end", func() {
+		d := open()
+		Expect(d.InsertParagraph(1, "Heading2", "Inserted <h2>")).To(Succeed())
+		Expect(d.InsertParagraph(-1, "", "At the start")).To(Succeed())
+		Expect(d.InsertParagraph(len(d.Paragraphs())-1, "ListBullet", "At the end")).To(Succeed())
+		b, text := finish(d)
+		lines := strings.Split(text, "\n")
+		Expect(lines[:4]).To(Equal([]string{"At the start", "# Title", "Hello world, hello", "## Inserted <h2>"}), "unexpected order:\n%s", text)
+		Expect(lines[len(lines)-1]).To(Equal("- At the end"), "last line wrong:\n%s", text)
+		d2, _ := Open(b)
+		Expect(d2.Paragraphs()[3].Style).To(Equal("Heading2"), "style not set")
+	})
+
+	It("refuses to insert a paragraph with an undefined style", func() {
+		d := open()
+		err := d.InsertParagraph(0, "NoSuchStyle", "x")
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("NoSuchStyle"))
+		// A table style id is not a paragraph style.
+		Expect(d.InsertParagraph(0, "TableGrid", "x")).ToNot(Succeed(), "table style accepted as paragraph style")
+		Expect(d.Paragraphs()).To(HaveLen(7), "a failed insert changed the document")
+	})
+
+	It("deletes paragraphs", func() {
+		d := open()
+		Expect(d.DeleteParagraph(1)).To(Succeed())
+		Expect(d.DeleteParagraph(2)).To(Succeed()) // was "alpha"
+		_, text := finish(d)
+		Expect(text).ToNot(ContainSubstring("Hello world"), "paragraphs still there")
+		Expect(text).ToNot(ContainSubstring("alpha"), "paragraphs still there")
+		Expect(text).To(ContainSubstring("- beta"), "other content lost")
+		Expect(text).To(ContainSubstring("hello\thello table"), "other content lost")
+	})
+
+	It("keeps a paragraph after the last table", func() {
+		// Paragraph 5 sits between the table and paragraph 6, so it can go.
+		d := open()
+		Expect(d.DeleteParagraph(5)).To(Succeed())
+		finish(d)
+		// Without paragraph 6 the table would end the body.
+		d = open()
+		Expect(d.DeleteParagraph(6)).To(Succeed())
+		Expect(d.DeleteParagraph(5)).ToNot(Succeed(), "deleted the paragraph after the last table")
+		finish(d)
+	})
+
+	It("replaces text in paragraphs but not in table cells", func() {
+		d := open()
+		n, err := d.ReplaceAll("hello", "a<b & c")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(n).To(Equal(2), "table cells are not searched")
+		_, text := finish(d)
+		Expect(text).To(ContainSubstring("Hello world, a<b & c"), "paragraphs not replaced")
+		Expect(text).To(ContainSubstring("Last a<b & c"), "paragraphs not replaced")
+		Expect(text).To(ContainSubstring("hello\thello table"), "table cell changed")
+		Expect(text).To(ContainSubstring("Bold lead plain tail"), "untouched paragraph changed")
+	})
+
+	It("does not match text across runs and rejects empty search text", func() {
+		d := open()
+		n, err := d.ReplaceAll("lead plain", "x") // spans two runs
+		Expect(err).ToNot(HaveOccurred())
+		Expect(n).To(BeZero())
+		n, err = d.ReplaceAll("lead", "head") // inside the bold run
+		Expect(err).ToNot(HaveOccurred())
+		Expect(n).To(Equal(1))
+		_, text := finish(d)
+		Expect(text).To(ContainSubstring("Bold head plain tail"))
+		_, err = d.ReplaceAll("", "x")
+		Expect(err).To(HaveOccurred(), "empty search text accepted")
+	})
+
+	It("keeps edge spaces of replaced text", func() {
+		// A w:t without xml:space gets it when the new text has edge spaces.
+		src := source()
+		_, data := zipParts(src)
+		doc := strings.Replace(string(data["word/document.xml"]), `<w:t xml:space="preserve">Last hello</w:t>`, `<w:t>Last hello</w:t>`, 1)
+		d, err := Open(replacePart(src, "word/document.xml", doc))
+		Expect(err).ToNot(HaveOccurred())
+		n, err := d.ReplaceAll("hello", " hi ")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(n).To(BeNumerically(">=", 1))
+		b, _ := finish(d)
+		d2, _ := Open(b)
+		ps := d2.Paragraphs()
+		Expect(ps[6].Text).To(Equal("Last  hi "))
+	})
+
+	It("changes only the document part and only the edited paragraph", func() {
+		src := source()
+		d, _ := Open(src)
+		Expect(d.SetText(1, "changed")).To(Succeed())
+		b, _ := finish(d)
+		n1, d1 := zipParts(src)
+		n2, d2 := zipParts(b)
+		Expect(n2).To(Equal(n1), "entry order changed")
+		for _, n := range n1 {
+			same := bytes.Equal(d1[n], d2[n])
+			if n == "word/document.xml" {
+				Expect(same).To(BeFalse(), "document.xml did not change")
+			} else {
+				Expect(same).To(BeTrue(), "part %s changed", n)
+			}
+		}
+		// Bytes outside the edited paragraph are untouched.
+		old, cur := string(d1["word/document.xml"]), string(d2["word/document.xml"])
+		i := strings.Index(old, "<w:p><w:r><w:t xml:space=\"preserve\">Hello")
+		Expect(i).To(BeNumerically(">=", 0), "prefix of document.xml changed")
+		Expect(cur).To(HavePrefix(old[:i]), "prefix of document.xml changed")
+		j := strings.Index(old, "<w:p><w:r><w:rPr><w:b/>")
+		Expect(cur).To(HaveSuffix(old[j:]), "suffix of document.xml changed")
+	})
+
+	It("composes several edits", func() {
+		d := open()
+		Expect(d.InsertParagraph(0, "Heading2", "Section")).To(Succeed())
+		Expect(d.SetText(2, "Second")).To(Succeed())
+		Expect(d.DeleteParagraph(3)).To(Succeed())
+		_, err := d.ReplaceAll("Last", "Final")
+		Expect(err).ToNot(HaveOccurred())
+		_, text := finish(d)
+		Expect(text).To(HavePrefix("# Title\n## Section\nSecond\n"), text)
+		Expect(text).To(ContainSubstring("Final hello"), text)
+		Expect(text).ToNot(ContainSubstring("Bold lead"), "deleted paragraph still there")
+	})
+
+	It("rejects out-of-range edits and edits of an empty paragraph without changing the document", func() {
+		d := open()
+		for _, i := range []int{-1, 7, 100} {
+			Expect(d.SetText(i, "x")).ToNot(Succeed(), "SetText(%d)", i)
+			Expect(d.DeleteParagraph(i)).ToNot(Succeed(), "DeleteParagraph(%d)", i)
+		}
+		for _, i := range []int{-2, 7} {
+			Expect(d.InsertParagraph(i, "", "x")).ToNot(Succeed(), "InsertParagraph(%d)", i)
+		}
+		// Paragraph 5 is empty: it has no text run.
+		err := d.SetText(5, "x")
+		Expect(err).To(MatchError(ContainSubstring("no text run")))
+		Expect(d.Paragraphs()).To(HaveLen(7), "failed edits changed the document")
+		finish(d)
+	})
+
+	It("fails to open a package that is damaged or incomplete", func() {
+		src := source()
+		bad := [][]byte{
+			nil,
+			[]byte("not a zip"),
+			replacePart(src, "_rels/.rels", `<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"></Relationships>`),
+			replacePart(src, "_rels/.rels", `<Relationships`),
+			replacePart(src, "word/document.xml", `<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"><w:body><w:p>`),
+			replacePart(src, "word/document.xml", `<html/>`),
+			replacePart(src, "word/document.xml", `<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"/>`),
+			replacePart(src, "word/styles.xml", `<w:styles`),
+		}
+		// A zip with no main part.
+		var buf bytes.Buffer
+		zw := zip.NewWriter(&buf)
+		w, _ := zw.Create("_rels/.rels")
+		_, data := zipParts(src)
+		_, _ = w.Write(data["_rels/.rels"])
+		_ = zw.Close()
+		bad = append(bad, buf.Bytes())
+		for i, b := range bad {
+			d, err := Open(b)
+			Expect(err).To(HaveOccurred(), "case %d: Open succeeded", i)
+			Expect(d).To(BeNil(), "case %d: Open succeeded", i)
+		}
+	})
+
+	It("does not list the paragraphs inside tables", func() {
+		d := open()
+		for _, s := range texts(d) {
+			Expect(s).ToNot(ContainSubstring("table"), "table text listed: %q", s)
+			Expect(s).ToNot(Equal("Name"), "table text listed: %q", s)
+		}
+	})
+
+	It("accepts only the default style when the document has no styles part", func() {
+		src := source()
+		_, data := zipParts(src)
+		// Drop the relationship to the styles part: no style is defined.
+		rels := strings.Replace(string(data["word/_rels/document.xml.rels"]), `Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/styles"`, `Type="urn:other"`, 1)
+		d, err := Open(replacePart(src, "word/_rels/document.xml.rels", rels))
+		Expect(err).ToNot(HaveOccurred())
+		Expect(d.InsertParagraph(0, "Heading1", "x")).ToNot(Succeed(), "style accepted without a styles part")
+		Expect(d.InsertParagraph(0, "", "x")).To(Succeed())
+	})
+})

addons/office/internal/officecheck/officecheck.go

@@ -0,0 +1,618 @@
+// Package officecheck inspects a generated .xlsx, .docx or .pptx file and
+// reports structural problems without rendering it: a broken zip, missing
+// content types, dangling relationships, malformed XML, missing required
+// parts and a few format rules (cell references, style indexes, list
+// numbering, slide lists). It uses only the standard library.
+package officecheck
+
+import (
+	"archive/zip"
+	"bytes"
+	"encoding/xml"
+	"fmt"
+	"io"
+	"path"
+	"regexp"
+	"sort"
+	"strconv"
+	"strings"
+	"unicode/utf8"
+
+	"github.com/mudler/LocalAI/addons/office/internal/ooxml"
+)
+
+// Problem is one structural problem. Part is the zip entry where it was
+// found, or empty when it concerns the whole file.
+type Problem struct {
+	Part    string
+	Message string
+}
+
+func (p Problem) String() string {
+	if p.Part == "" {
+		return p.Message
+	}
+	return p.Part + ": " + p.Message
+}
+
+const (
+	contentTypesPart = "[Content_Types].xml"
+	rootRels         = "_rels/.rels"
+	maxPartSize      = 256 << 20
+
+	nsRels = "http://schemas.openxmlformats.org/officeDocument/2006/relationships"
+
+	maxCol = 16384
+	maxRow = 1048576
+)
+
+type format int
+
+const (
+	fmtNone format = iota
+	fmtXLSX
+	fmtDOCX
+	fmtPPTX
+)
+
+type checker struct {
+	problems []Problem
+	parts    map[string][]byte
+	names    []string // sorted entry names
+}
+
+func (c *checker) add(part, format string, args ...any) {
+	c.problems = append(c.problems, Problem{Part: part, Message: fmt.Sprintf(format, args...)})
+}
+
+// Check inspects data as an .xlsx, .docx or .pptx file and returns every
+// structural problem found. It returns an empty slice for a clean file and
+// never panics on malformed input.
+func Check(data []byte) (problems []Problem) {
+	c := &checker{parts: map[string][]byte{}}
+	defer func() {
+		if r := recover(); r != nil {
+			c.add("", "internal error while checking: %v", r)
+		}
+		problems = c.problems
+		if problems == nil {
+			problems = []Problem{}
+		}
+	}()
+
+	if !c.readZip(data) {
+		return
+	}
+	f := c.checkContentTypes()
+	c.checkRels()
+	c.checkXML()
+	switch f {
+	case fmtXLSX:
+		c.checkXLSX()
+	case fmtDOCX:
+		c.checkDOCX()
+	case fmtPPTX:
+		c.checkPPTX()
+	}
+	return
+}
+
+// readZip loads every entry. It reports false when nothing can be checked.
+func (c *checker) readZip(data []byte) bool {
+	zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
+	if err != nil {
+		c.add("", "not a valid zip file: %v", err)
+		return false
+	}
+	for _, f := range zr.File {
+		name := f.Name
+		if strings.HasPrefix(name, "/") || strings.HasPrefix(name, "\\") || ooxml.HasDotDot(name) {
+			c.add(name, "entry name is absolute or contains ..")
+		}
+		if strings.HasSuffix(name, "/") {
+			continue
+		}
+		if _, dup := c.parts[name]; dup {
+			c.add(name, "duplicate zip entry")
+			continue
+		}
+		rc, err := f.Open()
+		if err != nil {
+			c.add(name, "cannot open zip entry: %v", err)
+			continue
+		}
+		b, err := io.ReadAll(io.LimitReader(rc, maxPartSize+1))
+		_ = rc.Close()
+		switch {
+		case err != nil:
+			c.add(name, "cannot read zip entry: %v", err)
+		case len(b) > maxPartSize:
+			c.add(name, "entry is larger than %d bytes", maxPartSize)
+		default:
+			c.parts[name] = b
+		}
+	}
+	for n := range c.parts {
+		c.names = append(c.names, n)
+	}
+	sort.Strings(c.names)
+	return true
+}
+
+// attr returns the value of the attribute with the given local name.
+func attr(se xml.StartElement, local string) (string, bool) {
+	for _, a := range se.Attr {
+		if a.Name.Local == local {
+			return a.Value, true
+		}
+	}
+	return "", false
+}
+
+// walk reads data token by token and calls fn for each token. It returns the
+// parse error, or nil at a clean end of input.
+func walk(data []byte, fn func(xml.Token)) error {
+	d := xml.NewDecoder(bytes.NewReader(data))
+	for {
+		t, err := d.Token()
+		if err == io.EOF {
+			return nil
+		}
+		if err != nil {
+			return err
+		}
+		fn(t)
+	}
+}
+
+// checkContentTypes parses [Content_Types].xml, checks that every part has a
+// type and detects the format from the main part's type.
+func (c *checker) checkContentTypes() format {
+	raw, ok := c.parts[contentTypesPart]
+	if !ok {
+		c.add(contentTypesPart, "missing")
+		return fmtNone
+	}
+	defaults := map[string]string{}
+	overrides := map[string]string{}
+	err := walk(raw, func(t xml.Token) {
+		se, ok := t.(xml.StartElement)
+		if !ok {
+			return
+		}
+		ct, _ := attr(se, "ContentType")
+		switch se.Name.Local {
+		case "Default":
+			if ext, ok := attr(se, "Extension"); ok {
+				defaults[strings.ToLower(ext)] = ct
+			}
+		case "Override":
+			if pn, ok := attr(se, "PartName"); ok {
+				overrides[strings.TrimPrefix(pn, "/")] = ct
+			}
+		}
+	})
+	if err != nil {
+		c.add(contentTypesPart, "not well-formed XML: %v", err)
+		return fmtNone
+	}
+	for _, n := range c.names {
+		if n == contentTypesPart || strings.HasPrefix(n, "_rels/") || strings.Contains(n, "/_rels/") {
+			continue
+		}
+		if _, ok := overrides[n]; ok {
+			continue
+		}
+		ext := strings.ToLower(strings.TrimPrefix(path.Ext(n), "."))
+		if _, ok := defaults[ext]; ok && ext != "" {
+			continue
+		}
+		c.add(n, "no content type (no Override and no Default for its extension)")
+	}
+	for pn := range overrides {
+		if _, ok := c.parts[pn]; !ok {
+			c.add(contentTypesPart, "Override for %s, which is not in the package", pn)
+		}
+	}
+	var f format
+	for _, ct := range overrides {
+		switch {
+		case strings.Contains(ct, "spreadsheetml.sheet.main"):
+			f = fmtXLSX
+		case strings.Contains(ct, "wordprocessingml.document.main"):
+			f = fmtDOCX
+		case strings.Contains(ct, "presentationml.presentation.main"):
+			f = fmtPPTX
+		}
+	}
+	if f == fmtNone {
+		c.add(contentTypesPart, "format not recognised: no workbook, document or presentation main part")
+	}
+	return f
+}
+
+type relationship struct {
+	id, typ, target string
+	external        bool
+}
+
+// parseRels reads a .rels part. It reports false when the part is missing or
+// malformed (the malformed case is reported by checkXML).
+func (c *checker) parseRels(name string) ([]relationship, bool) {
+	raw, ok := c.parts[name]
+	if !ok {
+		return nil, false
+	}
+	var rels []relationship
+	err := walk(raw, func(t xml.Token) {
+		se, ok := t.(xml.StartElement)
+		if !ok || se.Name.Local != "Relationship" {
+			return
+		}
+		var r relationship
+		r.id, _ = attr(se, "Id")
+		r.typ, _ = attr(se, "Type")
+		r.target, _ = attr(se, "Target")
+		mode, _ := attr(se, "TargetMode")
+		r.external = strings.EqualFold(mode, "External") || strings.HasPrefix(strings.ToLower(r.target), "http")
+		rels = append(rels, r)
+	})
+	return rels, err == nil
+}
+
+// relsOwner returns the part that owns a .rels part, and its folder.
+func relsOwner(rels string) (owner, base string) {
+	dir, file := path.Split(rels) // dir ends in "_rels/"
+	dir = strings.TrimSuffix(dir, "_rels/")
+	if file == ".rels" {
+		return "", dir
+	}
+	return dir + strings.TrimSuffix(file, ".rels"), strings.TrimSuffix(dir, "/")
+}
+
+// resolve turns a relationship target into a part name. It reports false
+// when the target climbs out of the package root.
+func resolve(base, target string) (string, bool) {
+	if i := strings.IndexAny(target, "#?"); i >= 0 {
+		target = target[:i]
+	}
+	if strings.HasPrefix(target, "/") {
+		target = target[1:]
+	} else {
+		target = path.Join(base, target)
+	}
+	target = path.Clean(target)
+	if target == ".." || strings.HasPrefix(target, "../") || target == "." {
+		return "", false
+	}
+	return target, true
+}
+
+func (c *checker) checkRels() {
+	for _, n := range c.names {
+		if !strings.HasSuffix(n, ".rels") {
+			continue
+		}
+		rels, ok := c.parseRels(n)
+		if !ok {
+			continue
+		}
+		owner, base := relsOwner(n)
+		if owner != "" {
+			if _, ok := c.parts[owner]; !ok {
+				c.add(n, "relationships for %s, which is not in the package", owner)
+			}
+		}
+		for _, r := range rels {
+			if r.external {
+				continue
+			}
+			target, ok := resolve(base, r.target)
+			if !ok {
+				c.add(n, "relationship %s target %q leaves the package", r.id, r.target)
+				continue
+			}
+			if _, ok := c.parts[target]; !ok {
+				c.add(n, "relationship %s target %q resolves to %s, which is missing", r.id, r.target, target)
+			}
+		}
+	}
+}
+
+func (c *checker) checkXML() {
+	for _, n := range c.names {
+		if !strings.HasSuffix(n, ".xml") && !strings.HasSuffix(n, ".rels") {
+			continue
+		}
+		if err := walk(c.parts[n], func(xml.Token) {}); err != nil {
+			c.add(n, "not well-formed XML: %v", err)
+		}
+	}
+}
+
+// requireMain checks that _rels/.rels points an officeDocument relationship
+// at main.
+func (c *checker) requireMain(main string) {
+	rels, ok := c.parseRels(rootRels)
+	if !ok {
+		if _, present := c.parts[rootRels]; !present {
+			c.add(rootRels, "missing")
+		}
+		return
+	}
+	for _, r := range rels {
+		if !strings.HasSuffix(r.typ, "/officeDocument") || r.external {
+			continue
+		}
+		if t, ok := resolve("", r.target); ok && t == main {
+			return
+		}
+	}
+	c.add(rootRels, "no officeDocument relationship points at %s", main)
+}
+
+func (c *checker) require(name string) bool {
+	if _, ok := c.parts[name]; !ok {
+		c.add(name, "required part is missing")
+		return false
+	}
+	return true
+}
+
+// partsMatching returns the sorted part names in dir that match
+// prefixNNN.xml.
+func (c *checker) partsMatching(dir, prefix string) []string {
+	var out []string
+	for _, n := range c.names {
+		if path.Dir(n) != dir {
+			continue
+		}
+		b := path.Base(n)
+		if strings.HasPrefix(b, prefix) && strings.HasSuffix(b, ".xml") {
+			out = append(out, n)
+		}
+	}
+	return out
+}
+
+// ---- xlsx ----
+
+var cellRefRe = regexp.MustCompile(`^([A-Z]+)([0-9]+)$`)
+
+func validCellRef(ref string) bool {
+	m := cellRefRe.FindStringSubmatch(ref)
+	if m == nil || len(m[1]) > 3 {
+		return false
+	}
+	col := 0
+	for _, ch := range m[1] {
+		col = col*26 + int(ch-'A') + 1
+	}
+	if col > maxCol {
+		return false
+	}
+	if m[2][0] == '0' {
+		return false
+	}
+	row, err := strconv.Atoi(m[2])
+	return err == nil && row >= 1 && row <= maxRow
+}
+
+func (c *checker) checkXLSX() {
+	c.requireMain("xl/workbook.xml")
+	hasWB := c.require("xl/workbook.xml")
+	sheets := c.partsMatching("xl/worksheets", "sheet")
+	if len(sheets) == 0 {
+		c.add("xl/worksheets", "no xl/worksheets/sheet*.xml part")
+	}
+	hasStyles := c.require("xl/styles.xml")
+
+	if hasWB {
+		c.checkSheetNames()
+	}
+	xfCount := -1
+	if hasStyles {
+		xfCount = countCellXfs(c.parts["xl/styles.xml"])
+	}
+	for _, s := range sheets {
+		c.checkSheet(s, xfCount)
+	}
+}
+
+func (c *checker) checkSheetNames() {
+	_ = walk(c.parts["xl/workbook.xml"], func(t xml.Token) {
+		se, ok := t.(xml.StartElement)
+		if !ok || se.Name.Local != "sheet" {
+			return
+		}
+		name, _ := attr(se, "name")
+		if n := utf8.RuneCountInString(name); n > 31 {
+			c.add("xl/workbook.xml", "sheet name %q is %d characters, the limit is 31", name, n)
+		}
+		if strings.ContainsAny(name, `[]:*?/\`) {
+			c.add("xl/workbook.xml", "sheet name %q contains one of []:*?/\\", name)
+		}
+		if name == "" {
+			c.add("xl/workbook.xml", "sheet name is empty")
+		}
+	})
+}
+
+// countCellXfs returns the number of xf entries in cellXfs, or -1 when
+// styles.xml does not parse or has no cellXfs.
+func countCellXfs(data []byte) int {
+	depth, in, n := 0, -1, -1
+	err := walk(data, func(t xml.Token) {
+		switch t := t.(type) {
+		case xml.StartElement:
+			depth++
+			if t.Name.Local == "cellXfs" && in < 0 {
+				in, n = depth, 0
+			} else if in >= 0 && depth == in+1 && t.Name.Local == "xf" {
+				n++
+			}
+		case xml.EndElement:
+			if depth == in {
+				in = -2
+			}
+			depth--
+		}
+	})
+	if err != nil {
+		return -1
+	}
+	return n
+}
+
+func (c *checker) checkSheet(name string, xfCount int) {
+	seen := map[string]bool{}
+	inF := false
+	var fText strings.Builder
+	var fShared bool
+	err := walk(c.parts[name], func(t xml.Token) {
+		switch t := t.(type) {
+		case xml.StartElement:
+			switch t.Name.Local {
+			case "c":
+				if ref, ok := attr(t, "r"); ok {
+					if !validCellRef(ref) {
+						c.add(name, "cell reference %q is not a valid A1 reference", ref)
+					} else if seen[ref] {
+						c.add(name, "duplicate cell %s", ref)
+					}
+					seen[ref] = true
+				}
+				if s, ok := attr(t, "s"); ok && xfCount >= 0 {
+					ref, _ := attr(t, "r")
+					idx, err := strconv.Atoi(s)
+					if err != nil || idx < 0 || idx >= xfCount {
+						c.add(name, "cell %s style index %q is outside the %d cellXfs in styles.xml", ref, s, xfCount)
+					}
+				}
+			case "f":
+				inF = true
+				fText.Reset()
+				_, hasSI := attr(t, "si")
+				ft, _ := attr(t, "t")
+				fShared = ft == "shared" && hasSI
+			}
+		case xml.CharData:
+			if inF {
+				fText.Write(t)
+			}
+		case xml.EndElement:
+			if t.Name.Local == "f" && inF {
+				inF = false
+				if strings.TrimSpace(fText.String()) == "" && !fShared {
+					c.add(name, "formula element <f> is empty")
+				}
+			}
+		}
+	})
+	_ = err // malformed XML is reported by checkXML
+}
+
+// ---- docx ----
+
+func (c *checker) checkDOCX() {
+	c.requireMain("word/document.xml")
+	if !c.require("word/document.xml") {
+		return
+	}
+	body := 0
+	used := map[string]bool{}
+	_ = walk(c.parts["word/document.xml"], func(t xml.Token) {
+		se, ok := t.(xml.StartElement)
+		if !ok {
+			return
+		}
+		switch se.Name.Local {
+		case "p", "tbl":
+			body++
+		case "numId":
+			if v, ok := attr(se, "val"); ok && v != "0" {
+				used[v] = true
+			}
+		}
+	})
+	if body == 0 {
+		c.add("word/document.xml", "body has no paragraph or table")
+	}
+	num, ok := c.parts["word/numbering.xml"]
+	if !ok {
+		return
+	}
+	defined := map[string]bool{}
+	_ = walk(num, func(t xml.Token) {
+		if se, ok := t.(xml.StartElement); ok && se.Name.Local == "num" {
+			if v, ok := attr(se, "numId"); ok {
+				defined[v] = true
+			}
+		}
+	})
+	ids := make([]string, 0, len(used))
+	for id := range used {
+		if !defined[id] {
+			ids = append(ids, id)
+		}
+	}
+	sort.Strings(ids)
+	for _, id := range ids {
+		c.add("word/document.xml", "numId %s is not defined in word/numbering.xml", id)
+	}
+}
+
+// ---- pptx ----
+
+func (c *checker) checkPPTX() {
+	c.requireMain("ppt/presentation.xml")
+	hasPres := c.require("ppt/presentation.xml")
+	slides := c.partsMatching("ppt/slides", "slide")
+	if len(slides) == 0 {
+		c.add("ppt/slides", "no ppt/slides/slide*.xml part")
+	}
+	if !hasPres {
+		return
+	}
+	var rids []string
+	_ = walk(c.parts["ppt/presentation.xml"], func(t xml.Token) {
+		se, ok := t.(xml.StartElement)
+		if !ok || se.Name.Local != "sldId" {
+			return
+		}
+		rid := ""
+		for _, a := range se.Attr {
+			if a.Name.Local == "id" && (a.Name.Space == nsRels || a.Name.Space == "r") {
+				rid = a.Value
+			}
+		}
+		rids = append(rids, rid)
+	})
+	if len(rids) != len(slides) {
+		c.add("ppt/presentation.xml", "%d sldId entries but %d slide parts", len(rids), len(slides))
+	}
+	rels, ok := c.parseRels("ppt/_rels/presentation.xml.rels")
+	if !ok {
+		if len(rids) > 0 {
+			c.add("ppt/_rels/presentation.xml.rels", "missing or unreadable, needed by the slide list")
+		}
+		return
+	}
+	byID := map[string]relationship{}
+	for _, r := range rels {
+		byID[r.id] = r
+	}
+	for _, rid := range rids {
+		r, ok := byID[rid]
+		if !ok {
+			c.add("ppt/presentation.xml", "sldId r:id %q has no relationship", rid)
+			continue
+		}
+		target, ok := resolve("ppt", r.target)
+		if !ok {
+			c.add("ppt/presentation.xml", "sldId r:id %q target %q leaves the package", rid, r.target)
+		} else if _, ok := c.parts[target]; !ok {
+			c.add("ppt/presentation.xml", "sldId r:id %q points at %s, which is missing", rid, target)
+		}
+	}
+}

addons/office/internal/officecheck/officecheck_suite_test.go

@@ -0,0 +1,13 @@
+package officecheck
+
+import (
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestOfficecheck(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "Officecheck Suite")
+}

addons/office/internal/officecheck/officecheck_test.go

@@ -0,0 +1,291 @@
+package officecheck
+
+import (
+	"archive/zip"
+	"bytes"
+	"fmt"
+	"strings"
+
+	"github.com/mudler/LocalAI/addons/office/internal/deck"
+	"github.com/mudler/LocalAI/addons/office/internal/deck/pptx"
+	"github.com/mudler/LocalAI/addons/office/internal/docx"
+	"github.com/mudler/LocalAI/addons/office/internal/xlsx"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func goodXLSX() []byte {
+	GinkgoHelper()
+	var buf bytes.Buffer
+	err := xlsx.Write(&buf, xlsx.Workbook{
+		Title:       "Budget",
+		HeaderStyle: xlsx.Style{Bold: true, FillColor: "DDDDDD"},
+		Sheets: []xlsx.Sheet{{
+			Name:      "Q1",
+			HeaderRow: 1,
+			Columns:   []xlsx.Column{{Width: 20}, {NumberFormat: "0.00"}},
+			Cells: []xlsx.Cell{
+				{Ref: "A1", Value: xlsx.Str("Item")},
+				{Ref: "B1", Value: xlsx.Str("Cost")},
+				{Ref: "A2", Value: xlsx.Str("Rent")},
+				{Ref: "B2", Value: xlsx.Num(1200.5)},
+				{Ref: "B3", Formula: "SUM(B2:B2)", Value: xlsx.Num(1200.5)},
+			},
+		}, {Name: "Notes", Cells: []xlsx.Cell{{Ref: "A1", Value: xlsx.Boolean(true)}}}},
+	})
+	Expect(err).NotTo(HaveOccurred())
+	return buf.Bytes()
+}
+
+func goodDOCX() []byte {
+	GinkgoHelper()
+	var buf bytes.Buffer
+	err := docx.Write(&buf, docx.Document{
+		Title: "Report",
+		Blocks: []docx.Block{
+			docx.Heading{Level: 1, Text: "Summary"},
+			docx.Paragraph{Text: "Hello"},
+			docx.Bullets{Items: []string{"one", "two"}},
+			docx.Table{Header: []string{"A", "B"}, Rows: [][]string{{"1", "2"}}},
+		},
+	})
+	Expect(err).NotTo(HaveOccurred())
+	return buf.Bytes()
+}
+
+func goodPPTX() []byte {
+	GinkgoHelper()
+	slides := []deck.StudioSlide{
+		{ID: "s1", Title: "Intro", Layout: deck.LayoutTitle, Status: deck.SlideOK, Notes: "say hi"},
+		{ID: "s2", Title: "Points", Layout: deck.LayoutText, Status: deck.SlideOK, Bullets: []deck.StudioBullet{{Text: "One"}, {Text: "Two"}}},
+	}
+	var buf bytes.Buffer
+	Expect(pptx.Write(&buf, slides, pptx.Options{Title: "Deck", Notes: true}, nil)).To(Succeed())
+	return buf.Bytes()
+}
+
+// rewrite returns a copy of zipped with one entry changed. fn gets the entry
+// content and returns the new content; a nil result drops the entry.
+func rewrite(zipped []byte, name string, fn func([]byte) []byte) []byte {
+	GinkgoHelper()
+	zr, err := zip.NewReader(bytes.NewReader(zipped), int64(len(zipped)))
+	Expect(err).NotTo(HaveOccurred())
+	var out bytes.Buffer
+	zw := zip.NewWriter(&out)
+	found := false
+	for _, f := range zr.File {
+		rc, err := f.Open()
+		Expect(err).NotTo(HaveOccurred())
+		var b bytes.Buffer
+		_, err = b.ReadFrom(rc)
+		Expect(err).NotTo(HaveOccurred())
+		_ = rc.Close()
+		data := b.Bytes()
+		if f.Name == name {
+			found = true
+			if data = fn(data); data == nil {
+				continue
+			}
+		}
+		w, err := zw.Create(f.Name)
+		Expect(err).NotTo(HaveOccurred())
+		_, _ = w.Write(data)
+	}
+	Expect(found).To(BeTrue(), "entry %s not in the package", name)
+	Expect(zw.Close()).To(Succeed())
+	return out.Bytes()
+}
+
+// replace returns an fn that swaps old for new once and fails when old is absent.
+func replace(old, new string) func([]byte) []byte {
+	return func(b []byte) []byte {
+		Expect(string(b)).To(ContainSubstring(old), "not found in part")
+		return []byte(strings.Replace(string(b), old, new, 1))
+	}
+}
+
+func drop([]byte) []byte { return nil }
+
+func wantProblem(data []byte, part, substr string) {
+	GinkgoHelper()
+	ps := Check(data)
+	for _, p := range ps {
+		if p.Part == part && strings.Contains(p.Message, substr) {
+			return
+		}
+	}
+	Fail(fmt.Sprintf("want a problem in %q containing %q, got %v", part, substr, ps))
+}
+
+func renameEntry(zipped []byte, from, to string) []byte {
+	GinkgoHelper()
+	zr, _ := zip.NewReader(bytes.NewReader(zipped), int64(len(zipped)))
+	var out bytes.Buffer
+	zw := zip.NewWriter(&out)
+	for _, f := range zr.File {
+		rc, _ := f.Open()
+		var b bytes.Buffer
+		_, _ = b.ReadFrom(rc)
+		_ = rc.Close()
+		name := f.Name
+		if name == from {
+			name = to
+		}
+		w, _ := zw.Create(name)
+		_, _ = w.Write(b.Bytes())
+	}
+	_ = zw.Close()
+	return out.Bytes()
+}
+
+var _ = Describe("Check", func() {
+	It("finds no problem in files from the writers", func() {
+		for name, data := range map[string][]byte{"xlsx": goodXLSX(), "docx": goodDOCX(), "pptx": goodPPTX()} {
+			Expect(Check(data)).To(BeEmpty(), name)
+		}
+	})
+
+	It("reports input that is not a zip", func() {
+		for _, in := range [][]byte{nil, []byte("hello"), goodDOCX()[:100]} {
+			ps := Check(in)
+			Expect(ps).To(HaveLen(1))
+			Expect(ps[0].Message).To(ContainSubstring("zip"))
+		}
+	})
+
+	It("reports an unknown format", func() {
+		data := rewrite(goodDOCX(), "[Content_Types].xml", replace("wordprocessingml.document.main+xml", "text/plain"))
+		wantProblem(data, "[Content_Types].xml", "format not recognised")
+	})
+
+	It("reports bad zip entry names", func() {
+		var buf bytes.Buffer
+		zw := zip.NewWriter(&buf)
+		w, _ := zw.Create("../evil.xml")
+		_, _ = w.Write([]byte("<a/>"))
+		_ = zw.Close()
+		wantProblem(buf.Bytes(), "../evil.xml", "absolute or contains ..")
+
+		// Also in an otherwise good package.
+		good := goodDOCX()
+		zr, _ := zip.NewReader(bytes.NewReader(good), int64(len(good)))
+		var out bytes.Buffer
+		zw = zip.NewWriter(&out)
+		for _, f := range zr.File {
+			rc, _ := f.Open()
+			var b bytes.Buffer
+			_, _ = b.ReadFrom(rc)
+			_ = rc.Close()
+			w, _ := zw.Create(f.Name)
+			_, _ = w.Write(b.Bytes())
+		}
+		w, _ = zw.Create("word/../../x.xml")
+		_, _ = w.Write([]byte("<a/>"))
+		_ = zw.Close()
+		wantProblem(out.Bytes(), "word/../../x.xml", "absolute or contains ..")
+	})
+
+	It("reports a missing content type", func() {
+		data := rewrite(goodDOCX(), "[Content_Types].xml", replace(`<Override PartName="/word/document.xml"`, `<Override PartName="/word/other.xml"`))
+		wantProblem(data, "[Content_Types].xml", "word/other.xml")
+
+		// A part with an extension that has no Default.
+		data = rewrite(goodXLSX(), "[Content_Types].xml", replace(`<Override PartName="/xl/sharedStrings.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.sharedStrings+xml"/>`, ""))
+		// sharedStrings.xml still has the Default xml type, so it is fine.
+		Expect(Check(data)).To(BeEmpty(), "a part covered by a Default type must pass")
+		data = renameEntry(goodDOCX(), "word/styles.xml", "word/styles.bin")
+		wantProblem(data, "word/styles.bin", "no content type")
+	})
+
+	It("reports a broken relationship target", func() {
+		data := rewrite(goodXLSX(), "xl/_rels/workbook.xml.rels", replace(`Target="styles.xml"`, `Target="nostyles.xml"`))
+		wantProblem(data, "xl/_rels/workbook.xml.rels", "xl/nostyles.xml")
+
+		data = rewrite(goodDOCX(), "_rels/.rels", replace(`Target="word/document.xml"`, `Target="word/missing.xml"`))
+		wantProblem(data, "_rels/.rels", "word/missing.xml")
+		wantProblem(data, "_rels/.rels", "no officeDocument relationship points at word/document.xml")
+
+		data = rewrite(goodDOCX(), "_rels/.rels", replace(`Target="word/document.xml"`, `Target="../../escape.xml"`))
+		wantProblem(data, "_rels/.rels", "leaves the package")
+	})
+
+	It("resolves relationships relative to the owner folder", func() {
+		// "../docProps/core.xml" from xl/ is valid; the checker must not report it.
+		data := rewrite(goodXLSX(), "xl/_rels/workbook.xml.rels", replace(`Target="styles.xml"`, `Target="../xl/styles.xml"`))
+		Expect(Check(data)).To(BeEmpty())
+		// An external target is not looked up.
+		data = rewrite(goodXLSX(), "xl/_rels/workbook.xml.rels", replace(`Target="styles.xml"`, `Target="https://example.com/x" TargetMode="External"`))
+		wantProblem(rewrite(data, "xl/styles.xml", drop), "xl/styles.xml", "required part is missing")
+	})
+
+	It("reports malformed XML", func() {
+		data := rewrite(goodDOCX(), "word/styles.xml", func(b []byte) []byte { return append(b[:len(b)-5:len(b)-5], []byte("<<")...) })
+		wantProblem(data, "word/styles.xml", "not well-formed")
+		data = rewrite(goodDOCX(), "[Content_Types].xml", func(b []byte) []byte { return []byte("<Types>") })
+		wantProblem(data, "[Content_Types].xml", "not well-formed")
+		data = rewrite(goodDOCX(), "_rels/.rels", func(b []byte) []byte { return []byte("not xml <") })
+		wantProblem(data, "_rels/.rels", "not well-formed")
+	})
+
+	It("reports missing required parts", func() {
+		wantProblem(rewrite(goodPPTX(), "ppt/presentation.xml", drop), "ppt/presentation.xml", "required part is missing")
+		wantProblem(rewrite(goodDOCX(), "word/document.xml", drop), "word/document.xml", "required part is missing")
+		wantProblem(rewrite(goodXLSX(), "xl/workbook.xml", drop), "xl/workbook.xml", "required part is missing")
+		wantProblem(rewrite(goodXLSX(), "xl/styles.xml", drop), "xl/styles.xml", "required part is missing")
+		data := rewrite(goodXLSX(), "xl/worksheets/sheet1.xml", drop)
+		data = rewrite(data, "xl/worksheets/sheet2.xml", drop)
+		wantProblem(data, "xl/worksheets", "no xl/worksheets/sheet*.xml")
+	})
+
+	It("checks xlsx cells", func() {
+		const sheet = "xl/worksheets/sheet1.xml"
+		wantProblem(rewrite(goodXLSX(), sheet, replace(`<f>SUM(B2:B2)</f>`, `<f></f>`)), sheet, "<f> is empty")
+		wantProblem(rewrite(goodXLSX(), sheet, replace(`<c r="B2"`, `<c r="B2" s="99"`)), sheet, "style index")
+		wantProblem(rewrite(goodXLSX(), sheet, replace(`<c r="B2"`, `<c r="a0"`)), sheet, `"a0"`)
+		wantProblem(rewrite(goodXLSX(), sheet, replace(`<c r="B2"`, `<c r="XFE1"`)), sheet, `"XFE1"`)
+		wantProblem(rewrite(goodXLSX(), sheet, replace(`<c r="B2"`, `<c r="A1048577"`)), sheet, `"A1048577"`)
+		wantProblem(rewrite(goodXLSX(), sheet, replace(`<c r="B2"`, `<c r="A2"`)), sheet, "duplicate cell A2")
+		for _, ok := range []string{"XFD1", "A1048576"} {
+			Expect(Check(rewrite(goodXLSX(), sheet, replace(`<c r="B2"`, `<c r="`+ok+`"`)))).To(BeEmpty(), "%s is valid", ok)
+		}
+	})
+
+	It("checks xlsx sheet names", func() {
+		long := strings.Repeat("x", 32)
+		wantProblem(rewrite(goodXLSX(), "xl/workbook.xml", replace(`name="Q1"`, `name="`+long+`"`)), "xl/workbook.xml", "limit is 31")
+		wantProblem(rewrite(goodXLSX(), "xl/workbook.xml", replace(`name="Q1"`, `name="a/b"`)), "xl/workbook.xml", "contains one of")
+		Expect(Check(rewrite(goodXLSX(), "xl/workbook.xml", replace(`name="Q1"`, `name="`+strings.Repeat("é", 31)+`"`)))).To(BeEmpty(), "31 runes is valid")
+	})
+
+	It("checks the docx body and numbering", func() {
+		const doc = "word/document.xml"
+		wantProblem(rewrite(goodDOCX(), doc, replace(`<w:numId w:val="1"/>`, `<w:numId w:val="77"/>`)), doc, "numId 77")
+		empty := func([]byte) []byte {
+			return []byte(`<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"><w:body/></w:document>`)
+		}
+		wantProblem(rewrite(goodDOCX(), doc, empty), doc, "no paragraph or table")
+	})
+
+	It("checks the pptx slide list", func() {
+		const pres = "ppt/presentation.xml"
+		wantProblem(rewrite(goodPPTX(), "ppt/slides/slide2.xml", drop), pres, "2 sldId entries but 1 slide parts")
+		wantProblem(rewrite(goodPPTX(), pres, replace(`<p:sldId id="257" r:id="rId`, `<p:sldId id="257" r:id="rIdNope`)), pres, "has no relationship")
+		data := rewrite(goodPPTX(), "ppt/_rels/presentation.xml.rels", replace(`slides/slide2.xml`, `slides/slide9.xml`))
+		wantProblem(data, pres, "ppt/slides/slide9.xml")
+	})
+
+	It("never panics", func() {
+		for _, data := range [][]byte{goodXLSX(), goodDOCX(), goodPPTX()} {
+			for _, n := range []int{0, 1, 30, 200, len(data) / 2, len(data) - 1} {
+				Check(data[:n])
+			}
+		}
+		// Every part replaced by junk, one at a time.
+		data := goodXLSX()
+		zr, _ := zip.NewReader(bytes.NewReader(data), int64(len(data)))
+		for _, f := range zr.File {
+			junk := rewrite(data, f.Name, func([]byte) []byte { return []byte("\x00<\xff") })
+			Check(junk)
+		}
+	})
+})

addons/office/internal/officeread/docx.go

@@ -0,0 +1,138 @@
+package officeread
+
+import (
+	"encoding/xml"
+	"strings"
+)
+
+func readDOCX(p parts, main string) ([]Section, error) {
+	var (
+		lines    []string
+		tables   [][][]string
+		para     strings.Builder
+		style    string
+		numbered bool
+		pDepth   int
+		tblDepth int
+		inT      bool
+		rows     [][]string
+		row      []string
+		cell     []string
+	)
+	err := walkPart(p, main, func(t xml.Token) {
+		switch t := t.(type) {
+		case xml.StartElement:
+			if !strings.Contains(t.Name.Space, "wordprocessingml") {
+				return
+			}
+			switch t.Name.Local {
+			case "tbl":
+				tblDepth++
+				if tblDepth == 1 {
+					rows = nil
+				}
+			case "tr":
+				if tblDepth == 1 {
+					row = nil
+				}
+			case "tc":
+				if tblDepth == 1 {
+					cell = nil
+				}
+			case "p":
+				pDepth++
+				if pDepth == 1 {
+					para.Reset()
+					style, numbered = "", false
+				}
+			case "pStyle":
+				if pDepth == 1 {
+					style, _ = attrNS(t, "val")
+				}
+			case "numPr":
+				numbered = pDepth == 1
+			case "numId":
+				if v, _ := attrNS(t, "val"); v == "0" {
+					numbered = false
+				}
+			case "t":
+				inT = true
+			case "tab":
+				para.WriteByte('\t')
+			case "br", "cr":
+				para.WriteByte('\n')
+			}
+		case xml.EndElement:
+			if !strings.Contains(t.Name.Space, "wordprocessingml") {
+				return
+			}
+			switch t.Name.Local {
+			case "t":
+				inT = false
+			case "p":
+				pDepth--
+				if pDepth != 0 {
+					return
+				}
+				text := para.String()
+				if tblDepth > 0 {
+					cell = append(cell, text)
+					return
+				}
+				lines = append(lines, paraPrefix(style, numbered)+text)
+			case "tc":
+				if tblDepth == 1 {
+					row = append(row, strings.Join(cell, " "))
+				}
+			case "tr":
+				if tblDepth == 1 {
+					rows = append(rows, row)
+				}
+			case "tbl":
+				tblDepth--
+				if tblDepth == 0 {
+					tables = append(tables, rows)
+					for _, r := range rows {
+						lines = append(lines, strings.Join(r, "\t"))
+					}
+				}
+			}
+		case xml.CharData:
+			if inT {
+				para.Write(t)
+			}
+		}
+	})
+	if err != nil {
+		return nil, err
+	}
+	text := strings.TrimRight(strings.Join(lines, "\n"), "\n")
+	return []Section{{Title: "Document", Text: text, Tables: tables}}, nil
+}
+
+// attrNS returns an attribute by local name, in any namespace.
+func attrNS(se xml.StartElement, local string) (string, bool) {
+	for _, a := range se.Attr {
+		if a.Name.Local == local {
+			return a.Value, true
+		}
+	}
+	return "", false
+}
+
+func paraPrefix(style string, numbered bool) string {
+	switch strings.ToLower(strings.ReplaceAll(style, " ", "")) {
+	case "heading1":
+		return "# "
+	case "heading2":
+		return "## "
+	case "heading3":
+		return "### "
+	case "listbullet":
+		return "- "
+	}
+	if numbered {
+		return "- "
+	}
+	return ""
+}

addons/office/internal/officeread/officeread.go

@@ -0,0 +1,246 @@
+// Package officeread reads an .xlsx, .docx or .pptx file and returns its
+// content as plain structured text. It is the read side of the xlsx, docx
+// and deck/pptx writers. It uses only the standard library.
+//
+// The format comes from the content types part. An xlsx file gives one
+// Section per sheet, a docx file gives one Section with the body in order,
+// and a pptx file gives one Section per slide in presentation order. Formula
+// cells give their cached value, never the formula. Nothing is rendered and
+// nothing is executed: the input is untrusted, so every entry name is
+// checked and the size of what is unpacked is limited.
+package officeread
+
+import (
+	"archive/zip"
+	"bytes"
+	"encoding/xml"
+	"errors"
+	"fmt"
+	"io"
+	"path"
+	"strings"
+
+	"github.com/mudler/LocalAI/addons/office/internal/ooxml"
+)
+
+// Document is the content of an office file.
+type Document struct {
+	// Format is "xlsx", "docx" or "pptx".
+	Format string
+	Parts  []Section
+}
+
+// Section is one sheet, one slide or the body of a document.
+type Section struct {
+	// Title is the sheet name, "Slide N" or "Document".
+	Title string
+	// Text is the plain text of the part.
+	Text string
+	// Tables holds the grid of an xlsx sheet, or the tables of a docx body.
+	// Each table is a list of rows, each row a list of cell texts.
+	Tables [][][]string
+}
+
+// ErrTooLarge is wrapped by the error of a package with an entry over 64 MiB
+// or over 256 MiB in total.
+var ErrTooLarge = errors.New("officeread: package too large")
+
+// Limits on what is unpacked. Tests lower them.
+var (
+	maxEntry int64 = 64 << 20
+	maxTotal int64 = 256 << 20
+)
+
+// maxGridCells bounds the grid built for one sheet, so a cell reference far
+// out in a sparse sheet cannot use all the memory.
+const maxGridCells = 4 << 20
+
+const (
+	contentTypesPart = "[Content_Types].xml"
+	nsRels           = "/relationships"
+)
+
+// Read parses data as an .xlsx, .docx or .pptx file. It returns an error
+// for a file that is not a zip, is not a recognised format, lacks a required
+// part, has a part that is not well-formed XML, has an unsafe entry name or
+// is too large. It never panics.
+func Read(data []byte) (doc Document, err error) {
+	defer func() {
+		if r := recover(); r != nil {
+			doc, err = Document{}, fmt.Errorf("officeread: internal error: %v", r)
+		}
+	}()
+	p, err := unzip(data)
+	if err != nil {
+		return Document{}, err
+	}
+	format, main, err := detect(p)
+	if err != nil {
+		return Document{}, err
+	}
+	switch format {
+	case "xlsx":
+		doc.Parts, err = readXLSX(p, main)
+	case "docx":
+		doc.Parts, err = readDOCX(p, main)
+	default:
+		doc.Parts, err = readPPTX(p, main)
+	}
+	if err != nil {
+		return Document{}, err
+	}
+	doc.Format = format
+	return doc, nil
+}
+
+type parts map[string][]byte
+
+func unzip(data []byte) (parts, error) {
+	zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
+	if err != nil {
+		return nil, fmt.Errorf("officeread: not a zip file: %w", err)
+	}
+	out := parts{}
+	var total int64
+	for _, f := range zr.File {
+		name := f.Name
+		if strings.HasPrefix(name, "/") || strings.HasPrefix(name, "\\") || ooxml.HasDotDot(name) {
+			return nil, fmt.Errorf("officeread: unsafe entry name %q", name)
+		}
+		if strings.HasSuffix(name, "/") {
+			continue
+		}
+		if _, dup := out[name]; dup {
+			return nil, fmt.Errorf("officeread: duplicate entry %q", name)
+		}
+		rc, err := f.Open()
+		if err != nil {
+			return nil, fmt.Errorf("officeread: cannot open %q: %w", name, err)
+		}
+		b, err := io.ReadAll(io.LimitReader(rc, maxEntry+1))
+		_ = rc.Close()
+		if err != nil {
+			return nil, fmt.Errorf("officeread: cannot read %q: %w", name, err)
+		}
+		if int64(len(b)) > maxEntry {
+			return nil, fmt.Errorf("officeread: entry %q is larger than %d bytes: %w", name, maxEntry, ErrTooLarge)
+		}
+		total += int64(len(b))
+		if total > maxTotal {
+			return nil, fmt.Errorf("officeread: package is larger than %d bytes: %w", maxTotal, ErrTooLarge)
+		}
+		out[name] = b
+	}
+	return out, nil
+}
+
+// detect finds the main part from the content types.
+func detect(p parts) (format, main string, err error) {
+	raw, ok := p[contentTypesPart]
+	if !ok {
+		return "", "", errors.New("officeread: " + contentTypesPart + " is missing")
+	}
+	err = walk(raw, func(t xml.Token) {
+		se, ok := t.(xml.StartElement)
+		if !ok || se.Name.Local != "Override" {
+			return
+		}
+		ct, _ := attr(se, "ContentType")
+		pn, _ := attr(se, "PartName")
+		switch {
+		case strings.Contains(ct, "spreadsheetml.sheet.main"):
+			format, main = "xlsx", strings.TrimPrefix(pn, "/")
+		case strings.Contains(ct, "wordprocessingml.document.main"):
+			format, main = "docx", strings.TrimPrefix(pn, "/")
+		case strings.Contains(ct, "presentationml.presentation.main"):
+			format, main = "pptx", strings.TrimPrefix(pn, "/")
+		}
+	})
+	if err != nil {
+		return "", "", fmt.Errorf("officeread: %s: %w", contentTypesPart, err)
+	}
+	if format == "" {
+		return "", "", errors.New("officeread: format not recognised: no workbook, document or presentation part")
+	}
+	if _, ok := p[main]; !ok {
+		return "", "", fmt.Errorf("officeread: required part %s is missing", main)
+	}
+	return format, main, nil
+}
+
+// walk passes every token of raw to fn and reports XML that is not well formed.
+func walk(raw []byte, fn func(xml.Token)) error {
+	dec := xml.NewDecoder(bytes.NewReader(raw))
+	for {
+		t, err := dec.Token()
+		if err == io.EOF {
+			return nil
+		}
+		if err != nil {
+			return fmt.Errorf("not well-formed XML: %w", err)
+		}
+		fn(t)
+	}
+}
+
+// walkPart is walk for a named part, with the name in the error.
+func walkPart(p parts, name string, fn func(xml.Token)) error {
+	raw, ok := p[name]
+	if !ok {
+		return fmt.Errorf("officeread: required part %s is missing", name)
+	}
+	if err := walk(raw, fn); err != nil {
+		return fmt.Errorf("officeread: %s: %w", name, err)
+	}
+	return nil
+}
+
+func attr(se xml.StartElement, local string) (string, bool) {
+	for _, a := range se.Attr {
+		if a.Name.Local == local && a.Name.Space == "" {
+			return a.Value, true
+		}
+	}
+	return "", false
+}
+
+// relID returns the r:id attribute, which lives in the relationships namespace.
+func relID(se xml.StartElement) string {
+	for _, a := range se.Attr {
+		if a.Name.Local == "id" && strings.HasSuffix(a.Name.Space, nsRels) {
+			return a.Value
+		}
+	}
+	return ""
+}
+
+// rel is one relationship with its target resolved to a package part name.
+type rel struct{ typ, target string }
+
+// relsOf reads the relationships of a part. A part without a rels file has none.
+func relsOf(p parts, part string) (map[string]rel, error) {
+	name := path.Join(path.Dir(part), "_rels", path.Base(part)+".rels")
+	out := map[string]rel{}
+	if _, ok := p[name]; !ok {
+		return out, nil
+	}
+	err := walkPart(p, name, func(t xml.Token) {
+		se, ok := t.(xml.StartElement)
+		if !ok || se.Name.Local != "Relationship" {
+			return
+		}
+		if mode, _ := attr(se, "TargetMode"); strings.EqualFold(mode, "External") {
+			return
+		}
+		id, _ := attr(se, "Id")
+		typ, _ := attr(se, "Type")
+		target, _ := attr(se, "Target")
+		if after, ok0 := strings.CutPrefix(target, "/"); ok0 {
+			target = path.Clean(after)
+		} else {
+			target = path.Join(path.Dir(part), target)
+		}
+		out[id] = rel{typ: typ, target: target}
+	})
+	return out, err
+}

addons/office/internal/officeread/officeread_suite_test.go

@@ -0,0 +1,13 @@
+package officeread
+
+import (
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestOfficeRead(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "Office Read Suite")
+}

addons/office/internal/officeread/officeread_test.go

@@ -0,0 +1,305 @@
+package officeread
+
+import (
+	"archive/zip"
+	"bytes"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+
+	"github.com/mudler/LocalAI/addons/office/internal/deck"
+	"github.com/mudler/LocalAI/addons/office/internal/deck/pptx"
+	"github.com/mudler/LocalAI/addons/office/internal/docx"
+	"github.com/mudler/LocalAI/addons/office/internal/xlsx"
+)
+
+func sampleXLSX() []byte {
+	GinkgoHelper()
+	var buf bytes.Buffer
+	err := xlsx.Write(&buf, xlsx.Workbook{Sheets: []xlsx.Sheet{
+		{Name: "Sales", Cells: []xlsx.Cell{
+			{Ref: "A1", Value: xlsx.Str("Item")}, {Ref: "B1", Value: xlsx.Str("Qty")},
+			{Ref: "A2", Value: xlsx.Str("Apples & pears")}, {Ref: "B2", Value: xlsx.Num(3.5)},
+			{Ref: "A3", Value: xlsx.Boolean(true)}, {Ref: "B3", Value: xlsx.Boolean(false)},
+			{Ref: "A4", Value: xlsx.Str("Double")}, {Ref: "B4", Formula: "=B2*2", Value: xlsx.Num(7)},
+			{Ref: "D6", Value: xlsx.Str("far")},
+		}},
+		{Name: "Notes", Cells: []xlsx.Cell{{Ref: "A1", Value: xlsx.Str("second sheet")}}},
+	}})
+	Expect(err).ToNot(HaveOccurred())
+	return buf.Bytes()
+}
+
+func samplePPTX() []byte {
+	GinkgoHelper()
+	slides := []deck.StudioSlide{
+		{ID: "s1", Title: "Q3 review", Sub: "Partners", Layout: deck.LayoutTitle, Status: deck.SlideOK},
+		{ID: "s2", Title: "What we heard", Layout: deck.LayoutText, Status: deck.SlideOK,
+			Notes:   "Say this first.\nThen pause.",
+			Bullets: []deck.StudioBullet{{Text: "Revenue grew"}, {Text: "Churn is low"}}},
+	}
+	var buf bytes.Buffer
+	Expect(pptx.Write(&buf, slides, pptx.Options{Title: "Q3", Notes: true}, nil)).To(Succeed())
+	return buf.Bytes()
+}
+
+// rewrite returns a copy of a zip with fn applied to each entry. fn returns
+// the new content, or nil to drop the entry.
+func rewrite(data []byte, fn func(name string, b []byte) []byte) []byte {
+	GinkgoHelper()
+	zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
+	Expect(err).ToNot(HaveOccurred())
+	var out bytes.Buffer
+	zw := zip.NewWriter(&out)
+	for _, f := range zr.File {
+		rc, _ := f.Open()
+		var b bytes.Buffer
+		_, _ = b.ReadFrom(rc)
+		_ = rc.Close()
+		nb := fn(f.Name, b.Bytes())
+		if nb == nil {
+			continue
+		}
+		w, _ := zw.Create(f.Name)
+		_, _ = w.Write(nb)
+	}
+	_ = zw.Close()
+	return out.Bytes()
+}
+
+func addEntry(data []byte, name, content string) []byte {
+	GinkgoHelper()
+	zr, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
+	Expect(err).ToNot(HaveOccurred())
+	var out bytes.Buffer
+	zw := zip.NewWriter(&out)
+	for _, f := range zr.File {
+		rc, _ := f.Open()
+		var b bytes.Buffer
+		_, _ = b.ReadFrom(rc)
+		_ = rc.Close()
+		w, _ := zw.Create(f.Name)
+		_, _ = w.Write(b.Bytes())
+	}
+	w, _ := zw.Create(name)
+	_, _ = w.Write([]byte(content))
+	_ = zw.Close()
+	return out.Bytes()
+}
+
+func makeZip(entries map[string]string) []byte {
+	GinkgoHelper()
+	var out bytes.Buffer
+	zw := zip.NewWriter(&out)
+	for n, c := range entries {
+		w, err := zw.Create(n)
+		Expect(err).ToNot(HaveOccurred())
+		_, _ = w.Write([]byte(c))
+	}
+	_ = zw.Close()
+	return out.Bytes()
+}
+
+var _ = Describe("officeread", func() {
+	It("reads an XLSX round trip", func() {
+		doc, err := Read(sampleXLSX())
+		Expect(err).ToNot(HaveOccurred())
+		Expect(doc.Format).To(Equal("xlsx"))
+		Expect(doc.Parts).To(HaveLen(2))
+		s := doc.Parts[0]
+		Expect(s.Title).To(Equal("Sales"))
+		Expect(doc.Parts[1].Title).To(Equal("Notes"))
+		want := [][]string{
+			{"Item", "Qty", "", ""},
+			{"Apples & pears", "3.5", "", ""},
+			{"TRUE", "FALSE", "", ""},
+			{"Double", "7", "", ""},
+			{"", "", "", ""},
+			{"", "", "", "far"},
+		}
+		Expect(s.Tables).To(HaveLen(1))
+		Expect(s.Tables[0]).To(Equal(want))
+		wantText := "Item\tQty\nApples & pears\t3.5\nTRUE\tFALSE\nDouble\t7\n\n\t\t\tfar"
+		Expect(s.Text).To(Equal(wantText))
+		Expect(s.Text).ToNot(ContainSubstring("B2*2"), "formula text leaked")
+		Expect(s.Text).ToNot(ContainSubstring("="), "formula text leaked")
+		Expect(doc.Parts[1].Text).To(Equal("second sheet"))
+	})
+
+	It("reads a DOCX round trip", func() {
+		var buf bytes.Buffer
+		err := docx.Write(&buf, docx.Document{Title: "T", Blocks: []docx.Block{
+			docx.Heading{Level: 1, Text: "Plan"},
+			docx.Paragraph{Text: "Intro line."},
+			docx.Heading{Level: 2, Text: "Steps"},
+			docx.Bullets{Items: []string{"one", "two"}},
+			docx.Paragraph{Runs: []docx.Run{{Text: "bold ", Bold: true}, {Text: "plain"}}},
+			docx.Heading{Level: 3, Text: "Data"},
+			docx.Table{Header: []string{"Name", "Score"}, Rows: [][]string{{"a", "1"}, {"b", "2"}}},
+			docx.Paragraph{Text: "After."},
+		}})
+		Expect(err).ToNot(HaveOccurred())
+		doc, err := Read(buf.Bytes())
+		Expect(err).ToNot(HaveOccurred())
+		Expect(doc.Format).To(Equal("docx"))
+		Expect(doc.Parts).To(HaveLen(1))
+		Expect(doc.Parts[0].Title).To(Equal("Document"))
+		want := "# Plan\nIntro line.\n## Steps\n- one\n- two\nbold plain\n### Data\nName\tScore\na\t1\nb\t2\n\nAfter."
+		Expect(doc.Parts[0].Text).To(Equal(want))
+		wantTable := [][][]string{{{"Name", "Score"}, {"a", "1"}, {"b", "2"}}}
+		Expect(doc.Parts[0].Tables).To(Equal(wantTable))
+	})
+
+	It("reads a PPTX round trip", func() {
+		doc, err := Read(samplePPTX())
+		Expect(err).ToNot(HaveOccurred())
+		Expect(doc.Format).To(Equal("pptx"))
+		Expect(doc.Parts).To(HaveLen(2))
+		Expect(doc.Parts[0].Title).To(Equal("Slide 1"))
+		Expect(doc.Parts[1].Title).To(Equal("Slide 2"))
+		for _, want := range []string{"Q3 review", "Partners"} {
+			Expect(doc.Parts[0].Text).To(ContainSubstring(want), "slide 1")
+		}
+		Expect(doc.Parts[0].Text).ToNot(ContainSubstring("Notes:"), "slide 1 has no notes")
+		s2 := doc.Parts[1].Text
+		for _, want := range []string{"What we heard", "Revenue grew\n", "Churn is low", "\nNotes:\nSay this first.\nThen pause."} {
+			Expect(s2).To(ContainSubstring(want), "slide 2")
+		}
+		Expect(s2).To(HaveSuffix("Notes:\nSay this first.\nThen pause."), "notes must come last")
+	})
+
+	It("reads XLSX shared strings and inline strings", func() {
+		sheet := `<?xml version="1.0"?><worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"><sheetData>` +
+			`<row r="1"><c r="A1" t="s"><v>1</v></c><c r="B1" t="s"><v>0</v></c><c r="C1" t="inlineStr"><is><t>in</t><r><t>line</t></r></is></c></row>` +
+			`<row r="2"><c r="A2" t="str"><f>A1&amp;B1</f><v>cached</v></c><c r="B2" t="b"><v>1</v></c><c r="C2"><v>42</v></c></row>` +
+			`</sheetData></worksheet>`
+		sst := `<?xml version="1.0"?><sst xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main">` +
+			`<si><t>zero</t></si><si><r><t>on</t></r><r><t xml:space="preserve">e </t></r><rPh><t>ignored</t></rPh></si></sst>`
+		data := rewrite(sampleXLSX(), func(name string, b []byte) []byte {
+			switch name {
+			case "xl/worksheets/sheet1.xml":
+				return []byte(sheet)
+			case "xl/sharedStrings.xml":
+				return []byte(sst)
+			}
+			return b
+		})
+		doc, err := Read(data)
+		Expect(err).ToNot(HaveOccurred())
+		want := [][]string{{"on" + "e ", "zero", "inline"}, {"cached", "TRUE", "42"}}
+		Expect(doc.Parts[0].Tables[0]).To(Equal(want))
+	})
+
+	It("returns errors for bad input", func() {
+		good := sampleXLSX()
+		cases := map[string][]byte{
+			"not a zip":        []byte("hello"),
+			"empty":            nil,
+			"no content types": makeZip(map[string]string{"a.txt": "x"}),
+			"unrecognised format": makeZip(map[string]string{
+				contentTypesPart: `<Types xmlns="x"><Override PartName="/a.xml" ContentType="text/plain"/></Types>`,
+				"a.xml":          "<a/>",
+			}),
+			"missing workbook": rewrite(good, func(n string, b []byte) []byte {
+				if n == "xl/workbook.xml" {
+					return nil
+				}
+				return b
+			}),
+			"malformed sheet": rewrite(good, func(n string, b []byte) []byte {
+				if n == "xl/worksheets/sheet1.xml" {
+					return []byte("<worksheet><sheetData><row>")
+				}
+				return b
+			}),
+			"malformed content types": rewrite(good, func(n string, b []byte) []byte {
+				if n == contentTypesPart {
+					return []byte("<Types><x></Types>")
+				}
+				return b
+			}),
+			"dotdot entry":   addEntry(good, "../evil.xml", "<a/>"),
+			"absolute entry": addEntry(good, "/evil.xml", "<a/>"),
+		}
+		for name, data := range cases {
+			_, err := Read(data)
+			Expect(err).To(HaveOccurred(), "%s", name)
+		}
+		for _, f := range []struct{ name, part string }{{"docx", "word/document.xml"}, {"pptx", "ppt/presentation.xml"}} {
+			var data []byte
+			if f.name == "docx" {
+				var buf bytes.Buffer
+				Expect(docx.Write(&buf, docx.Document{Blocks: []docx.Block{docx.Paragraph{Text: "x"}}})).To(Succeed())
+				data = buf.Bytes()
+			} else {
+				data = samplePPTX()
+			}
+			data = rewrite(data, func(n string, b []byte) []byte {
+				if n == f.part {
+					return nil
+				}
+				return b
+			})
+			_, err := Read(data)
+			Expect(err).To(HaveOccurred(), "missing %s", f.part)
+		}
+	})
+
+	It("never panics on truncated input", func() {
+		var d bytes.Buffer
+		Expect(docx.Write(&d, docx.Document{Blocks: []docx.Block{docx.Heading{Level: 1, Text: "h"}, docx.Bullets{Items: []string{"a"}}}})).To(Succeed())
+		for name, good := range map[string][]byte{"xlsx": sampleXLSX(), "docx": d.Bytes(), "pptx": samplePPTX()} {
+			step := len(good)/300 + 1
+			for n := 0; n < len(good); n += step {
+				_, err := Read(good[:n])
+				Expect(err).To(HaveOccurred(), "%s truncated to %d bytes", name, n)
+			}
+			_, err := Read(good)
+			Expect(err).ToNot(HaveOccurred(), "%s", name)
+		}
+	})
+
+	It("enforces the size limits", func() {
+		oldE, oldT := maxEntry, maxTotal
+		DeferCleanup(func() { maxEntry, maxTotal = oldE, oldT })
+		data := sampleXLSX()
+		maxEntry = 100
+		_, err := Read(data)
+		Expect(err).To(MatchError(ContainSubstring("larger")), "entry limit")
+		Expect(err).To(MatchError(ErrTooLarge), "entry limit")
+		maxEntry, maxTotal = oldE, 500
+		_, err = Read(data)
+		Expect(err).To(MatchError(ContainSubstring("larger")), "total limit")
+		Expect(err).To(MatchError(ErrTooLarge), "total limit")
+	})
+
+	It("rejects a sparse sheet that is too large", func() {
+		sheet := `<worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"><sheetData>` +
+			`<row r="1048576"><c r="XFD1048576"><v>1</v></c></row></sheetData></worksheet>`
+		data := rewrite(sampleXLSX(), func(n string, b []byte) []byte {
+			if n == "xl/worksheets/sheet1.xml" {
+				return []byte(sheet)
+			}
+			return b
+		})
+		_, err := Read(data)
+		Expect(err).To(HaveOccurred(), "huge sparse sheet")
+	})
+
+	// A formula cell that was written without a cached result has no value in
+	// the file. The reader shows the formula so a model can check what it wrote.
+	It("shows a formula that has no cached value", func() {
+		var buf bytes.Buffer
+		err := xlsx.Write(&buf, xlsx.Workbook{Sheets: []xlsx.Sheet{{Name: "S", Cells: []xlsx.Cell{
+			{Ref: "A1", Value: xlsx.Num(2)}, {Ref: "A2", Value: xlsx.Num(3)},
+			{Ref: "A3", Formula: "=SUM(A1:A2)"},
+			{Ref: "B3", Formula: "A1*2", Value: xlsx.Num(4)},
+		}}}})
+		Expect(err).ToNot(HaveOccurred())
+		doc, err := Read(buf.Bytes())
+		Expect(err).ToNot(HaveOccurred())
+		grid := doc.Parts[0].Tables[0]
+		Expect(grid[2][0]).To(Equal("=SUM(A1:A2)"), "A3 is the formula")
+		Expect(grid[2][1]).To(Equal("4"), "B3 is the cached result")
+	})
+})

addons/office/internal/officeread/pptx.go

@@ -0,0 +1,110 @@
+package officeread
+
+import (
+	"encoding/xml"
+	"fmt"
+	"strings"
+)
+
+func readPPTX(p parts, main string) ([]Section, error) {
+	var order []string
+	err := walkPart(p, main, func(t xml.Token) {
+		if se, ok := t.(xml.StartElement); ok && se.Name.Local == "sldId" {
+			order = append(order, relID(se))
+		}
+	})
+	if err != nil {
+		return nil, err
+	}
+	rels, err := relsOf(p, main)
+	if err != nil {
+		return nil, err
+	}
+	var out []Section
+	for i, rid := range order {
+		r, ok := rels[rid]
+		if !ok {
+			return nil, fmt.Errorf("officeread: slide %d has no relationship %q", i+1, rid)
+		}
+		lines, err := slideLines(p, r.target)
+		if err != nil {
+			return nil, err
+		}
+		slideRels, err := relsOf(p, r.target)
+		if err != nil {
+			return nil, err
+		}
+		for _, sr := range slideRels {
+			if !strings.HasSuffix(sr.typ, "/notesSlide") {
+				continue
+			}
+			notes, err := slideLines(p, sr.target)
+			if err != nil {
+				return nil, err
+			}
+			if len(notes) > 0 {
+				lines = append(lines, "Notes:")
+				lines = append(lines, notes...)
+			}
+			break
+		}
+		out = append(out, Section{Title: fmt.Sprintf("Slide %d", i+1), Text: strings.Join(lines, "\n")})
+	}
+	return out, nil
+}
+
+// slideLines returns the non-empty paragraphs of a slide or notes part. Text
+// in slide number, date, footer, header and slide image placeholders is left out.
+func slideLines(p parts, name string) ([]string, error) {
+	var (
+		lines []string
+		para  strings.Builder
+		inP   bool
+		inT   bool
+		skip  bool
+	)
+	err := walkPart(p, name, func(t xml.Token) {
+		switch t := t.(type) {
+		case xml.StartElement:
+			switch t.Name.Local {
+			case "sp":
+				skip = false
+			case "ph":
+				switch v, _ := attr(t, "type"); v {
+				case "sldNum", "dt", "ftr", "hdr", "sldImg":
+					skip = true
+				}
+			case "p":
+				if t.Name.Space != "" && strings.HasSuffix(t.Name.Space, "drawingml/2006/main") {
+					inP = true
+					para.Reset()
+				}
+			case "t":
+				inT = inP
+			case "br":
+				if inP {
+					para.WriteByte('\n')
+				}
+			}
+		case xml.EndElement:
+			switch t.Name.Local {
+			case "t":
+				inT = false
+			case "p":
+				if inP && strings.HasSuffix(t.Name.Space, "drawingml/2006/main") {
+					inP = false
+					if s := para.String(); s != "" && !skip {
+						lines = append(lines, s)
+					}
+				}
+			case "sp":
+				skip = false
+			}
+		case xml.CharData:
+			if inT {
+				para.Write(t)
+			}
+		}
+	})
+	return lines, err
+}

addons/office/internal/officeread/xlsx.go

@@ -0,0 +1,245 @@
+package officeread
+
+import (
+	"encoding/xml"
+	"fmt"
+	"strconv"
+	"strings"
+)
+
+type sheetRef struct{ name, rid string }
+
+func readXLSX(p parts, main string) ([]Section, error) {
+	var refs []sheetRef
+	err := walkPart(p, main, func(t xml.Token) {
+		if se, ok := t.(xml.StartElement); ok && se.Name.Local == "sheet" {
+			name, _ := attr(se, "name")
+			refs = append(refs, sheetRef{name, relID(se)})
+		}
+	})
+	if err != nil {
+		return nil, err
+	}
+	rels, err := relsOf(p, main)
+	if err != nil {
+		return nil, err
+	}
+	shared, err := sharedStrings(p, rels)
+	if err != nil {
+		return nil, err
+	}
+	var out []Section
+	for _, ref := range refs {
+		r, ok := rels[ref.rid]
+		if !ok {
+			return nil, fmt.Errorf("officeread: sheet %q has no relationship %q", ref.name, ref.rid)
+		}
+		grid, err := readSheet(p, r.target, shared)
+		if err != nil {
+			return nil, err
+		}
+		out = append(out, Section{Title: ref.name, Text: gridText(grid), Tables: [][][]string{grid}})
+	}
+	return out, nil
+}
+
+// sharedStrings reads the optional shared strings table.
+func sharedStrings(p parts, rels map[string]rel) ([]string, error) {
+	name := "xl/sharedStrings.xml"
+	for _, r := range rels {
+		if strings.HasSuffix(r.typ, "/sharedStrings") {
+			name = r.target
+		}
+	}
+	if _, ok := p[name]; !ok {
+		return nil, nil
+	}
+	var out []string
+	var cur strings.Builder
+	inSI, inT, phonetic := false, false, 0
+	err := walkPart(p, name, func(t xml.Token) {
+		switch t := t.(type) {
+		case xml.StartElement:
+			switch t.Name.Local {
+			case "si":
+				inSI = true
+				cur.Reset()
+			case "rPh":
+				phonetic++
+			case "t":
+				inT = inSI && phonetic == 0
+			}
+		case xml.EndElement:
+			switch t.Name.Local {
+			case "si":
+				out = append(out, cur.String())
+				inSI = false
+			case "rPh":
+				phonetic--
+			case "t":
+				inT = false
+			}
+		case xml.CharData:
+			if inT {
+				cur.Write(t)
+			}
+		}
+	})
+	return out, err
+}
+
+// readSheet builds the grid of one worksheet from the cell references.
+func readSheet(p parts, name string, shared []string) ([][]string, error) {
+	type pos struct{ row, col int }
+	cells := map[pos]string{}
+	maxRow, maxCol := 0, 0
+	var (
+		cur         pos
+		typ         string
+		val         strings.Builder
+		inCell, inV bool
+		inF         bool
+		formula     strings.Builder
+		inIS, inT   bool
+		nextRow     int
+		nextCol     int
+		sheetErr    error
+	)
+	err := walkPart(p, name, func(t xml.Token) {
+		switch t := t.(type) {
+		case xml.StartElement:
+			switch t.Name.Local {
+			case "row":
+				nextRow++
+				if r, ok := attr(t, "r"); ok {
+					if n, err := strconv.Atoi(r); err == nil && n > 0 {
+						nextRow = n
+					}
+				}
+				nextCol = 0
+			case "c":
+				inCell = true
+				typ, _ = attr(t, "t")
+				val.Reset()
+				formula.Reset()
+				nextCol++
+				cur = pos{nextRow, nextCol}
+				if r, ok := attr(t, "r"); ok {
+					if row, col, ok := splitRef(r); ok {
+						cur = pos{row, col}
+					} else if sheetErr == nil {
+						sheetErr = fmt.Errorf("officeread: %s: bad cell reference %q", name, r)
+					}
+				}
+				nextRow, nextCol = cur.row, cur.col
+			case "v":
+				inV = inCell
+			case "f":
+				inF = inCell
+			case "is":
+				inIS = inCell
+			case "t":
+				inT = inIS
+			}
+		case xml.EndElement:
+			switch t.Name.Local {
+			case "v":
+				inV = false
+			case "f":
+				inF = false
+			case "is":
+				inIS = false
+			case "t":
+				inT = false
+			case "c":
+				if inCell {
+					text := cellText(typ, val.String(), shared)
+					if text == "" && formula.Len() > 0 {
+						text = "=" + strings.TrimPrefix(formula.String(), "=")
+					}
+					cells[cur] = text
+					maxRow, maxCol = max(maxRow, cur.row), max(maxCol, cur.col)
+				}
+				inCell = false
+			}
+		case xml.CharData:
+			if inV || inT {
+				val.Write(t)
+			}
+			if inF {
+				formula.Write(t)
+			}
+		}
+	})
+	if err != nil {
+		return nil, err
+	}
+	if sheetErr != nil {
+		return nil, sheetErr
+	}
+	if maxRow*maxCol > maxGridCells {
+		return nil, fmt.Errorf("officeread: %s: sheet is too large (%d rows, %d columns)", name, maxRow, maxCol)
+	}
+	grid := make([][]string, maxRow)
+	for r := range grid {
+		grid[r] = make([]string, maxCol)
+	}
+	for k, v := range cells {
+		grid[k.row-1][k.col-1] = v
+	}
+	return grid, nil
+}
+
+func cellText(typ, v string, shared []string) string {
+	switch typ {
+	case "s":
+		i, err := strconv.Atoi(strings.TrimSpace(v))
+		if err != nil || i < 0 || i >= len(shared) {
+			return ""
+		}
+		return shared[i]
+	case "b":
+		if strings.TrimSpace(v) == "1" {
+			return "TRUE"
+		}
+		return "FALSE"
+	}
+	return v
+}
+
+// splitRef turns "B3" into row 3 and column 2.
+func splitRef(ref string) (row, col int, ok bool) {
+	i := 0
+	for ; i < len(ref); i++ {
+		c := ref[i]
+		if c >= 'a' && c <= 'z' {
+			c -= 'a' - 'A'
+		}
+		if c < 'A' || c > 'Z' {
+			break
+		}
+		col = col*26 + int(c-'A') + 1
+		if col > 16384 {
+			return 0, 0, false
+		}
+	}
+	n, err := strconv.Atoi(ref[i:])
+	if i == 0 || err != nil || n < 1 || n > 1048576 {
+		return 0, 0, false
+	}
+	return n, col, true
+}
+
+// gridText joins the cells with tabs, one row per line, without trailing
+// empty cells.
+func gridText(grid [][]string) string {
+	lines := make([]string, len(grid))
+	for i, row := range grid {
+		end := len(row)
+		for end > 0 && row[end-1] == "" {
+			end--
+		}
+		lines[i] = strings.Join(row[:end], "\t")
+	}
+	return strings.TrimRight(strings.Join(lines, "\n"), "\n")
+}

addons/office/internal/ooxml/ooxml.go

@@ -0,0 +1,25 @@
+// Package ooxml holds the small pieces that every Office Open XML writer and
+// reader of the add-on shares: text escaping and the check of a part
+// name.
+package ooxml
+
+import (
+	"bytes"
+	"encoding/xml"
+	"slices"
+	"strings"
+)
+
+// Escape returns s escaped for XML character data and attribute values.
+func Escape(s string) string {
+	var b bytes.Buffer
+	_ = xml.EscapeText(&b, []byte(s))
+	return b.String()
+}
+
+// HasDotDot reports whether a part name has a ".." segment, with "/" or "\"
+// as the separator. A package that names such a part could write or read
+// outside its folder once it is unpacked.
+func HasDotDot(name string) bool {
+	return slices.Contains(strings.FieldsFunc(name, func(r rune) bool { return r == '/' || r == '\\' }), "..")
+}

addons/office/internal/ooxml/ooxml_suite_test.go

@@ -0,0 +1,13 @@
+package ooxml_test
+
+import (
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestOOXML(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "Office OOXML helpers suite")
+}

addons/office/internal/ooxml/ooxml_test.go

@@ -0,0 +1,32 @@
+package ooxml_test
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+
+	"github.com/mudler/LocalAI/addons/office/internal/ooxml"
+)
+
+var _ = Describe("Escape", func() {
+	It("escapes the XML special characters", func() {
+		Expect(ooxml.Escape(`a<b>&"c'`)).To(Equal("a&lt;b&gt;&amp;&#34;c&#39;"))
+	})
+
+	It("keeps plain text as it is", func() {
+		Expect(ooxml.Escape("plain text")).To(Equal("plain text"))
+	})
+})
+
+var _ = Describe("HasDotDot", func() {
+	DescribeTable("finds a parent segment with either separator",
+		func(name string, want bool) {
+			Expect(ooxml.HasDotDot(name)).To(Equal(want))
+		},
+		Entry("plain part", "word/document.xml", false),
+		Entry("dots in a name", "word/a..b.xml", false),
+		Entry("slash parent", "word/../x.xml", true),
+		Entry("backslash parent", `word\..\x.xml`, true),
+		Entry("leading parent", "../x.xml", true),
+		Entry("only parent", "..", true),
+	)
+})

addons/office/internal/pdf/create.go

@@ -0,0 +1,437 @@
+package pdf
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"strings"
+
+	"github.com/go-pdf/fpdf"
+)
+
+// Heading levels the writer supports.
+const (
+	minHeadingLevel = 1
+	maxHeadingLevel = 3
+)
+
+// Layout in millimetres (A4, portrait). Body text is 11 pt, table text 9 pt.
+const (
+	pageMargin   = 20.0
+	pageHeight   = 297.0
+	bodySize     = 11.0
+	bodyLine     = 5.0
+	tableSize    = 9.0
+	tableLine    = 4.2
+	tablePad     = 1.5
+	bulletIndent = 6.0
+	coreFamily   = "Helvetica"
+	userFamily   = "pdfuser"
+)
+
+// headingSize and headingLine give the font size (pt) and line height (mm)
+// of each heading level.
+var (
+	headingSize = [...]float64{0, 20, 16, 13}
+	headingLine = [...]float64{0, 9, 7.5, 6.5}
+)
+
+// Block is one element of a page. The block types are Heading, Paragraph,
+// Bullets and Table, as values (not pointers). The method set is private,
+// so no other type can be a Block.
+type Block interface {
+	validate() error
+	texts() []string
+	render(l *layout)
+}
+
+// Heading is a section title. Level is 1 to 3.
+type Heading struct {
+	Level int
+	Text  string
+}
+
+// Paragraph is a body paragraph. It wraps to the page width. An empty Text
+// is an empty line.
+type Paragraph struct {
+	Text string
+}
+
+// Bullets is a bulleted list. It needs at least one item and no item may be
+// empty.
+type Bullets struct {
+	Items []string
+}
+
+// Table is a grid with a header row and body rows. Every row has as many
+// cells as Header. Columns share the text width equally. Cell text wraps,
+// and the row grows to fit it.
+type Table struct {
+	Header []string
+	Rows   [][]string
+}
+
+// Page is one page of a Spec. Each Page starts on a new PDF page. Text that
+// does not fit goes on further pages, so a Page can take more than one.
+type Page struct {
+	Blocks []Block
+}
+
+// Spec is a whole PDF document.
+type Spec struct {
+	// Title and Author are written to the document information.
+	Title  string
+	Author string
+	Pages  []Page
+	// FontTTF is an optional Unicode TrueType font. With it, any text is
+	// allowed. Without it, the built-in Helvetica font is used and text
+	// outside Latin-1 (U+0000 to U+00FF) is an error.
+	FontTTF []byte
+}
+
+func (h Heading) validate() error {
+	if h.Level < minHeadingLevel || h.Level > maxHeadingLevel {
+		return fmt.Errorf("heading level %d is not between %d and %d", h.Level, minHeadingLevel, maxHeadingLevel)
+	}
+	if strings.TrimSpace(h.Text) == "" {
+		return errors.New("heading text is empty")
+	}
+	return nil
+}
+
+func (p Paragraph) validate() error { return nil }
+
+func (b Bullets) validate() error {
+	if len(b.Items) == 0 {
+		return errors.New("bullet list has no item")
+	}
+	for i, it := range b.Items {
+		if strings.TrimSpace(it) == "" {
+			return fmt.Errorf("bullet item %d is empty", i+1)
+		}
+	}
+	return nil
+}
+
+func (t Table) validate() error {
+	if len(t.Header) == 0 {
+		return errors.New("table has no header")
+	}
+	for i, r := range t.Rows {
+		if len(r) != len(t.Header) {
+			return fmt.Errorf("table row %d has %d cells, header has %d", i+1, len(r), len(t.Header))
+		}
+	}
+	return nil
+}
+
+func (h Heading) texts() []string   { return []string{h.Text} }
+func (p Paragraph) texts() []string { return []string{p.Text} }
+func (b Bullets) texts() []string   { return b.Items }
+
+func (t Table) texts() []string {
+	out := append([]string{}, t.Header...)
+	for _, r := range t.Rows {
+		out = append(out, r...)
+	}
+	return out
+}
+
+// validate checks the spec, its limits and, without FontTTF, the characters.
+func (s Spec) validate() error {
+	if len(s.Pages) == 0 {
+		return errors.New("spec has no pages")
+	}
+	if len(s.Pages) > MaxPages {
+		return fmt.Errorf("spec has %d pages, the limit is %d", len(s.Pages), MaxPages)
+	}
+	blocks := 0
+	for i, p := range s.Pages {
+		blocks += len(p.Blocks)
+		for j, b := range p.Blocks {
+			if b == nil {
+				return fmt.Errorf("page %d block %d is nil", i+1, j+1)
+			}
+			if err := b.validate(); err != nil {
+				return fmt.Errorf("page %d block %d: %w", i+1, j+1, err)
+			}
+		}
+	}
+	if blocks > MaxBlocks {
+		return fmt.Errorf("spec has %d blocks, the limit is %d", blocks, MaxBlocks)
+	}
+	if len(s.FontTTF) > MaxInputBytes {
+		return fmt.Errorf("font is %d bytes, the limit is %d", len(s.FontTTF), MaxInputBytes)
+	}
+	if s.FontTTF != nil {
+		if !isSFNT(s.FontTTF) {
+			return errors.New("font is not a TrueType or OpenType file")
+		}
+		return nil
+	}
+	for i, p := range s.Pages {
+		for _, b := range p.Blocks {
+			for _, t := range b.texts() {
+				if err := checkLatin1(t); err != nil {
+					return fmt.Errorf("page %d: %w", i+1, err)
+				}
+			}
+		}
+	}
+	return nil
+}
+
+// isSFNT reports whether b starts with a TrueType or OpenType signature.
+func isSFNT(b []byte) bool {
+	if len(b) < 4 {
+		return false
+	}
+	switch string(b[:4]) {
+	case "\x00\x01\x00\x00", "true", "OTTO":
+		return true
+	}
+	return false
+}
+
+// checkLatin1 refuses any rune above U+00FF and the control characters, which
+// the built-in font cannot draw. Tab and newline are allowed. It does not echo
+// the text.
+func checkLatin1(s string) error {
+	for _, r := range s {
+		switch {
+		case r == '\n' || r == '\t':
+		case r < 0x20 || (r >= 0x7F && r <= 0x9F):
+			return fmt.Errorf("character U+%04X is a control character", r)
+		case r > 0xFF:
+			return fmt.Errorf("character U+%04X is outside Latin-1: set Spec.FontTTF to a Unicode font", r)
+		}
+	}
+	return nil
+}
+
+// latin1Bytes turns Latin-1 text into the single-byte form fpdf expects for
+// the built-in fonts (cp1252). For Latin-1 the code point is the byte. The
+// text was checked by checkLatin1 first.
+func latin1Bytes(s string) string {
+	b := make([]byte, 0, len(s))
+	for _, r := range s {
+		b = append(b, byte(r)) // #nosec G115 -- checkLatin1 accepted the text, so every rune is below 256
+	}
+	return string(b)
+}
+
+// Create writes a new PDF from spec. The output is checked with Check
+// before it is returned.
+func Create(spec Spec) ([]byte, error) {
+	if err := spec.validate(); err != nil {
+		return nil, fmt.Errorf("pdf: create: %w", err)
+	}
+	var out []byte
+	err := guard("create", func() error {
+		b, err := render(spec)
+		if err != nil {
+			return err
+		}
+		out = b
+		return nil
+	})
+	if err != nil {
+		return nil, err
+	}
+	if err := Check(out); err != nil {
+		return nil, fmt.Errorf("pdf: create: output failed the check: %w", err)
+	}
+	return out, nil
+}
+
+// layout holds the drawing state of one document.
+type layout struct {
+	pdf    *fpdf.Fpdf
+	family string
+	ttf    bool
+	left   float64
+	width  float64
+	err    error
+}
+
+// text returns s in the encoding fpdf expects for the current font: UTF-8
+// for the TrueType font, single-byte Latin-1 for the built-in font.
+func (l *layout) text(s string) string {
+	if l.ttf {
+		return s
+	}
+	return latin1Bytes(s)
+}
+
+// bullet is the glyph that starts a list item, in the current encoding.
+func (l *layout) bullet() string {
+	if l.ttf {
+		return "\u2022"
+	}
+	return "\x95" // bullet in cp1252, the built-in font encoding
+}
+
+// font selects the family and size. Bold is used only with the built-in font,
+// because a TrueType font is embedded in one style.
+func (l *layout) font(size float64, bold bool) {
+	style := ""
+	if bold && !l.ttf {
+		style = "B"
+	}
+	l.pdf.SetFont(l.family, style, size)
+}
+
+// room is the height of the text area of one page.
+func (l *layout) room() float64 {
+	_, ph := l.pdf.GetPageSize()
+	return ph - 2*pageMargin
+}
+
+// fail records the first layout error. Later blocks are skipped.
+func (l *layout) fail(msg string) {
+	if l.err == nil {
+		l.err = errors.New(msg)
+	}
+}
+
+// ensure starts a new page when h millimetres do not fit above the bottom
+// margin. It reports whether it did.
+func (l *layout) ensure(h float64) bool {
+	_, ph := l.pdf.GetPageSize()
+	if l.pdf.GetY()+h > ph-pageMargin {
+		l.pdf.AddPage()
+		l.pdf.SetX(l.left)
+		return true
+	}
+	return false
+}
+
+func render(spec Spec) ([]byte, error) {
+	pdf := fpdf.New("P", "mm", "A4", "")
+	pdf.SetTitle(spec.Title, true)
+	pdf.SetAuthor(spec.Author, true)
+	pdf.SetMargins(pageMargin, pageMargin, pageMargin)
+	pdf.SetAutoPageBreak(true, pageMargin)
+
+	l := &layout{pdf: pdf, family: coreFamily, left: pageMargin, width: 210 - 2*pageMargin}
+	if spec.FontTTF != nil {
+		l.family = userFamily
+		l.ttf = true
+		pdf.AddUTF8FontFromBytes(userFamily, "", spec.FontTTF)
+		if err := pdf.Error(); err != nil {
+			return nil, fmt.Errorf("pdf: create: font: %w", err)
+		}
+	}
+
+	for _, p := range spec.Pages {
+		pdf.AddPage()
+		pdf.SetX(l.left)
+		for _, b := range p.Blocks {
+			if l.err == nil {
+				b.render(l)
+			}
+		}
+	}
+	if l.err != nil {
+		return nil, fmt.Errorf("pdf: create: %w", l.err)
+	}
+	if err := pdf.Error(); err != nil {
+		return nil, fmt.Errorf("pdf: create: %w", err)
+	}
+	var buf bytes.Buffer
+	if err := pdf.Output(&buf); err != nil {
+		return nil, fmt.Errorf("pdf: create: write: %w", err)
+	}
+	return buf.Bytes(), nil
+}
+
+func (h Heading) render(l *layout) {
+	size, line := headingSize[h.Level], headingLine[h.Level]
+	l.pdf.Ln(2)
+	l.ensure(2 * line)
+	l.pdf.SetX(l.left)
+	l.font(size, true)
+	l.pdf.MultiCell(0, line, l.text(h.Text), "", "L", false)
+	l.pdf.Ln(2)
+}
+
+func (p Paragraph) render(l *layout) {
+	l.ensure(bodyLine)
+	l.pdf.SetX(l.left)
+	l.font(bodySize, false)
+	if p.Text == "" {
+		l.pdf.Ln(bodyLine)
+		return
+	}
+	l.pdf.MultiCell(0, bodyLine, l.text(p.Text), "", "L", false)
+	l.pdf.Ln(2)
+}
+
+func (b Bullets) render(l *layout) {
+	l.font(bodySize, false)
+	for _, item := range b.Items {
+		l.ensure(bodyLine)
+		y := l.pdf.GetY()
+		l.pdf.SetXY(l.left+1, y)
+		l.pdf.CellFormat(4, bodyLine, l.bullet(), "", 0, "L", false, 0, "")
+		l.pdf.SetXY(l.left+bulletIndent, y)
+		l.pdf.MultiCell(l.width-bulletIndent, bodyLine, l.text(item), "", "L", false)
+	}
+	l.pdf.SetX(l.left)
+	l.pdf.Ln(2)
+}
+
+func (t Table) render(l *layout) {
+	cw := l.width / float64(len(t.Header))
+	hh := l.rowHeight(t.Header, cw, true)
+	if hh > l.room() {
+		l.fail("table header is taller than a page")
+		return
+	}
+	l.ensure(hh)
+	l.drawRow(t.Header, cw, hh, true)
+	for _, row := range t.Rows {
+		h := l.rowHeight(row, cw, false)
+		if h+hh > l.room() {
+			l.fail("table row does not fit on a page together with the header")
+			return
+		}
+		if l.ensure(h) {
+			// The row starts a new page, so the header repeats there.
+			l.drawRow(t.Header, cw, hh, true)
+		}
+		l.drawRow(row, cw, h, false)
+	}
+	l.pdf.SetX(l.left)
+	l.pdf.Ln(3)
+}
+
+// rowHeight returns the height of a row, which fits its tallest wrapped cell.
+func (l *layout) rowHeight(cells []string, cw float64, header bool) float64 {
+	l.font(tableSize, header)
+	n := 1
+	for _, c := range cells {
+		if k := len(l.pdf.SplitLines([]byte(l.text(c)), cw-2*tablePad)); k > n {
+			n = k
+		}
+	}
+	return float64(n)*tableLine + 2*tablePad
+}
+
+// drawRow draws one row of h millimetres at the current y, then moves below it.
+func (l *layout) drawRow(cells []string, cw, h float64, header bool) {
+	y := l.pdf.GetY()
+	l.font(tableSize, header)
+	border := "D"
+	if header {
+		border = "DF"
+		l.pdf.SetFillColor(230, 230, 230)
+	}
+	for i, c := range cells {
+		x := l.left + float64(i)*cw
+		l.pdf.Rect(x, y, cw, h, border)
+		l.pdf.SetXY(x+tablePad, y+tablePad)
+		l.pdf.MultiCell(cw-2*tablePad, tableLine, l.text(c), "", "L", false)
+	}
+	l.pdf.SetXY(l.left, y+h)
+}

addons/office/internal/pdf/edit.go

@@ -0,0 +1,111 @@
+package pdf
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"io"
+	"strconv"
+
+	"github.com/pdfcpu/pdfcpu/pkg/api"
+	"github.com/pdfcpu/pdfcpu/pkg/pdfcpu/types"
+)
+
+// watermarkDesc is the pdfcpu description of a diagonal stamp: lower left
+// to upper right, 48 pt, 30 % opacity, drawn on top of the page.
+const watermarkDesc = "diagonal:1, points:48, opacity:0.3, scalefactor:1.0 rel"
+
+// Merge joins PDFs in the given order into one PDF.
+func Merge(parts ...[]byte) ([]byte, error) {
+	if len(parts) == 0 {
+		return nil, errors.New("pdf: merge: no input")
+	}
+	for i, p := range parts {
+		if err := checkInput(p); err != nil {
+			return nil, fmt.Errorf("pdf: merge: part %d: %w", i+1, err)
+		}
+	}
+	return writeTo("merge", func(w io.Writer) error {
+		rsc := make([]io.ReadSeeker, len(parts))
+		for i, p := range parts {
+			rsc[i] = bytes.NewReader(p)
+		}
+		return api.MergeRaw(rsc, w, false, newConf())
+	})
+}
+
+// Split returns one PDF for each page, in page order. Each part is a valid
+// one-page PDF. It runs one extraction per page, so a long document takes a
+// while.
+func Split(data []byte) ([][]byte, error) {
+	if err := checkInput(data); err != nil {
+		return nil, err
+	}
+	n, err := Pages(data)
+	if err != nil {
+		return nil, err
+	}
+	parts := make([][]byte, 0, n)
+	for page := 1; page <= n; page++ {
+		part, err := writeTo("split", func(w io.Writer) error {
+			return api.Trim(bytes.NewReader(data), w, []string{strconv.Itoa(page)}, newConf())
+		})
+		if err != nil {
+			return nil, fmt.Errorf("pdf: split: page %d: %w", page, err)
+		}
+		parts = append(parts, part)
+	}
+	return parts, nil
+}
+
+// Rotate turns the given 1-based pages clockwise. degrees must be 90, 180 or
+// 270. An empty pages list means every page.
+func Rotate(data []byte, pages []int, degrees int) ([]byte, error) {
+	if err := checkInput(data); err != nil {
+		return nil, err
+	}
+	switch degrees {
+	case 90, 180, 270:
+	default:
+		return nil, fmt.Errorf("pdf: rotate: degrees %d must be 90, 180 or 270", degrees)
+	}
+	n, err := Pages(data)
+	if err != nil {
+		return nil, err
+	}
+	var sel []string
+	for _, p := range pages {
+		if err := checkPage(p, n); err != nil {
+			return nil, fmt.Errorf("pdf: rotate: %w", err)
+		}
+		sel = append(sel, strconv.Itoa(p))
+	}
+	return writeTo("rotate", func(w io.Writer) error {
+		return api.Rotate(bytes.NewReader(data), w, degrees, sel, newConf())
+	})
+}
+
+// Watermark stamps text diagonally on every page. The text must be Latin-1,
+// because the stamp uses the built-in Helvetica font.
+func Watermark(data []byte, text string) ([]byte, error) {
+	if err := checkInput(data); err != nil {
+		return nil, err
+	}
+	if text == "" {
+		return nil, errors.New("pdf: watermark: text is empty")
+	}
+	if err := checkLatin1(text); err != nil {
+		return nil, fmt.Errorf("pdf: watermark: %w", err)
+	}
+	out, err := writeTo("watermark", func(w io.Writer) error {
+		wm, err := api.TextWatermark(text, watermarkDesc, true, false, types.POINTS)
+		if err != nil {
+			return err
+		}
+		return api.AddWatermarks(bytes.NewReader(data), w, nil, wm, newConf())
+	})
+	if err != nil {
+		return nil, err
+	}
+	return out, nil
+}

addons/office/internal/pdf/forms.go

@@ -0,0 +1,132 @@
+package pdf
+
+import (
+	"bytes"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"sort"
+	"strings"
+
+	"github.com/pdfcpu/pdfcpu/pkg/api"
+	"github.com/pdfcpu/pdfcpu/pkg/pdfcpu/form"
+)
+
+// FormField is a named AcroForm text field and its current value.
+type FormField struct {
+	Name  string
+	Value string
+}
+
+// Fields lists the named AcroForm text fields, sorted by name. Checkboxes,
+// radio buttons, lists and other field kinds are left out. A PDF without a
+// form gives an empty list.
+func Fields(data []byte) ([]FormField, error) {
+	if err := checkInput(data); err != nil {
+		return nil, err
+	}
+	var out []FormField
+	err := guard("fields", func() error {
+		fields, err := api.FormFields(bytes.NewReader(data), newConf())
+		if err != nil {
+			// pdfcpu v0.15.0 has no sentinel for a missing form. A PDF
+			// without an AcroForm has no text fields, so it is not an error.
+			if strings.Contains(err.Error(), "no form available") {
+				return nil
+			}
+			return fmt.Errorf("pdf: fields: %w", err)
+		}
+		seen := map[string]bool{}
+		for _, f := range fields {
+			if f.Typ != form.FTText || f.Name == "" || seen[f.Name] {
+				continue
+			}
+			seen[f.Name] = true
+			out = append(out, FormField{Name: f.Name, Value: f.V})
+		}
+		return nil
+	})
+	if err != nil {
+		return nil, err
+	}
+	sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
+	return out, nil
+}
+
+// FillFields sets the values of the named text fields and returns the new
+// PDF. A name that is not a text field of the form is an error, and nothing
+// is returned for it.
+func FillFields(data []byte, values map[string]string) ([]byte, error) {
+	if err := checkInput(data); err != nil {
+		return nil, err
+	}
+	if len(values) == 0 {
+		return nil, errors.New("pdf: fill: no values")
+	}
+	fields, err := Fields(data)
+	if err != nil {
+		return nil, err
+	}
+	known := make(map[string]bool, len(fields))
+	for _, f := range fields {
+		known[f.Name] = true
+	}
+	names := make([]string, 0, len(values))
+	for name := range values {
+		names = append(names, name)
+	}
+	sort.Strings(names)
+	for _, name := range names {
+		if !known[name] {
+			return nil, fmt.Errorf("pdf: fill: no text field named %q", name)
+		}
+	}
+
+	return guard2("fill", func() ([]byte, error) {
+		group, err := api.ExportForm(bytes.NewReader(data), "", newConf())
+		if err != nil {
+			return nil, fmt.Errorf("pdf: fill: %w", err)
+		}
+		if len(group.Forms) == 0 {
+			return nil, errors.New("pdf: fill: no form data")
+		}
+		set := 0
+		for i := range group.Forms {
+			for _, tf := range group.Forms[i].TextFields {
+				if tf == nil {
+					continue
+				}
+				if v, ok := values[tf.Name]; ok {
+					tf.Value = v
+					set++
+				}
+			}
+		}
+		if set == 0 {
+			return nil, errors.New("pdf: fill: no field was set")
+		}
+		js, err := json.Marshal(group)
+		if err != nil {
+			return nil, fmt.Errorf("pdf: fill: encode values: %w", err)
+		}
+		var out bytes.Buffer
+		if err := api.FillForm(bytes.NewReader(data), bytes.NewReader(js), &out, newConf()); err != nil {
+			return nil, fmt.Errorf("pdf: fill: %w", err)
+		}
+		return out.Bytes(), nil
+	})
+}
+
+// guard2 is guard for a call that returns bytes.
+func guard2(op string, fn func() ([]byte, error)) ([]byte, error) {
+	var out []byte
+	err := guard(op, func() error {
+		b, err := fn()
+		out = b
+		return err
+	})
+	if err != nil {
+		return nil, err
+	}
+	return out, nil
+}

addons/office/internal/pdf/images.go

@@ -0,0 +1,731 @@
+package pdf
+
+import (
+	"bytes"
+	"compress/zlib"
+	"fmt"
+	"image"
+	"image/png"
+	"io"
+	"math"
+	"sort"
+	"strings"
+
+	"github.com/pdfcpu/pdfcpu/pkg/api"
+	"github.com/pdfcpu/pdfcpu/pkg/pdfcpu/model"
+	"github.com/pdfcpu/pdfcpu/pkg/pdfcpu/types"
+)
+
+// Limits for ExtractImages. Input over maxImageInputBytes is refused. An
+// extraction that would return more than maxImages images, or more than
+// maxImageOutput bytes of Data, is an error. An image wider or higher than
+// maxImageDim is skipped.
+const (
+	maxImageInputBytes = 256 << 20
+	maxImages          = 200
+	maxImageOutput     = 256 << 20
+	maxFormDepth       = 16
+	maxImageDim        = 65535
+)
+
+// Image is one image XObject found on a page. It is either returned in its
+// original encoding or decoded and re-encoded as PNG. When Skipped is set,
+// Data is nil and the caller should render the page instead.
+//
+// Format is one of:
+//   - "jpeg": Data is the original JPEG stream (DCTDecode, DeviceRGB or DeviceGray).
+//   - "jbig2": Data is the original JBIG2 stream, not decoded.
+//   - "jpx": Data is the original JPEG 2000 stream, not decoded.
+//   - "png": Data is a PNG that this package decoded from a Flate stream.
+//
+// HasSMask reports that the image has an /SMask. The soft mask is not
+// applied: Data holds the colour samples only.
+type Image struct {
+	Page             int
+	Name             string
+	Format           string
+	Width, Height    int
+	BitsPerComponent int
+	ColorSpace       string
+	Data             []byte
+	HasSMask         bool
+	Skipped          string
+}
+
+// ExtractImages returns the image XObjects of the given pages. Pages are
+// 1-based. An empty pages list selects every page. Resources are read
+// through each page, including the resources of nested form XObjects.
+// Results are in page order, then in resource-name order.
+//
+// The image bytes are read straight from the PDF streams, so pdfcpu's image
+// helper, which fails on image masks and mishandles CMYK JPEGs, is not used.
+// Images that cannot be returned as-is or decoded here get a Skipped reason.
+// Such images have no Data, so the caller can render that page.
+//
+// The soft mask of an image is not applied. See Image.HasSMask.
+//
+// A form XObject is walked once per page, however many times that page refers
+// to it. The set of walked forms is reset at the start of each page. A form
+// used by two requested pages therefore reports its images under both pages,
+// and each page gives the same result as when it is requested alone. The image
+// count and output size limits apply to the whole call.
+//
+// Only in-memory readers are used. Nothing is written to disk.
+func ExtractImages(data []byte, pages []int) (out []Image, err error) {
+	if len(data) == 0 {
+		return nil, errEmptyInput
+	}
+	if len(data) > maxImageInputBytes {
+		return nil, fmt.Errorf("pdf: extract images: input is %d bytes, the limit is %d", len(data), maxImageInputBytes)
+	}
+	err = guard("extract images", func() error {
+		ctx, err := api.ReadContext(bytes.NewReader(data), newConf())
+		if err != nil {
+			return fmt.Errorf("pdf: extract images: %w", err)
+		}
+		if err := ctx.EnsurePageCount(); err != nil {
+			return fmt.Errorf("pdf: extract images: %w", err)
+		}
+		sel, err := selectPages(pages, ctx.PageCount)
+		if err != nil {
+			return err
+		}
+		x := &imageExtractor{ctx: ctx}
+		for _, p := range sel {
+			if err := x.page(p); err != nil {
+				return fmt.Errorf("pdf: extract images: page %d: %w", p, err)
+			}
+		}
+		out = x.out
+		return nil
+	})
+	if err != nil {
+		return nil, err
+	}
+	return out, nil
+}
+
+// selectPages returns the sorted, de-duplicated pages to read. An empty
+// request means every page of a document with n pages.
+func selectPages(pages []int, n int) ([]int, error) {
+	if len(pages) == 0 {
+		sel := make([]int, n)
+		for i := range sel {
+			sel[i] = i + 1
+		}
+		return sel, nil
+	}
+	seen := make(map[int]bool, len(pages))
+	var sel []int
+	for _, p := range pages {
+		if err := checkPage(p, n); err != nil {
+			return nil, err
+		}
+		if !seen[p] {
+			seen[p] = true
+			sel = append(sel, p)
+		}
+	}
+	sort.Ints(sel)
+	return sel, nil
+}
+
+// imageExtractor holds the state of one ExtractImages call.
+type imageExtractor struct {
+	ctx   *model.Context
+	out   []Image
+	total int
+	// visited holds the object numbers of the forms walked on the current
+	// page. It is reset for each page. A form is walked at most once per
+	// page, so cycles end and a form reached again on that page costs nothing.
+	visited map[int]bool
+}
+
+// page walks the resources of one page, with a fresh set of walked forms.
+func (x *imageExtractor) page(p int) error {
+	x.visited = map[int]bool{}
+	_, _, attrs, err := x.ctx.PageDict(p, false)
+	if err != nil {
+		return err
+	}
+	if attrs == nil || attrs.Resources == nil {
+		return nil
+	}
+	return x.resources(attrs.Resources, p, 0)
+}
+
+// resources visits the XObjects of one resource dictionary. Images are
+// added to the result. Forms are walked recursively, each one once per call.
+func (x *imageExtractor) resources(res types.Dict, page int, depth int) error {
+	xobjs, err := x.dict(res, "XObject")
+	if err != nil || xobjs == nil {
+		return err
+	}
+	names := make([]string, 0, len(xobjs))
+	for name := range xobjs {
+		names = append(names, name)
+	}
+	sort.Strings(names)
+	for _, name := range names {
+		objNr := 0
+		if ref, ok := xobjs[name].(types.IndirectRef); ok {
+			objNr = ref.ObjectNumber.Value()
+		}
+		sd, _, err := x.ctx.DereferenceStreamDict(xobjs[name])
+		if err != nil {
+			return fmt.Errorf("xobject %s: %w", name, err)
+		}
+		if sd == nil {
+			continue
+		}
+		subtype, _ := x.name(sd.Dict, "Subtype")
+		switch subtype {
+		case "Image":
+			if err := x.addImage(page, name, sd); err != nil {
+				return err
+			}
+		case "Form":
+			if objNr != 0 {
+				if x.visited[objNr] {
+					continue // walked already, and its images were emitted then
+				}
+				x.visited[objNr] = true
+			}
+			if depth >= maxFormDepth {
+				return fmt.Errorf("forms nested more than %d levels", maxFormDepth)
+			}
+			formRes, err := x.dict(sd.Dict, "Resources")
+			if err != nil {
+				return err
+			}
+			if formRes == nil {
+				formRes = res // a form without its own resources uses the caller's
+			}
+			err = x.resources(formRes, page, depth+1)
+			if err != nil {
+				return err
+			}
+		}
+	}
+	return nil
+}
+
+// addImage decodes one image XObject and appends it to the result.
+func (x *imageExtractor) addImage(page int, name string, sd *types.StreamDict) error {
+	if len(x.out) >= maxImages {
+		return fmt.Errorf("more than %d images", maxImages)
+	}
+	img := Image{Page: page, Name: name}
+	if err := x.describe(sd, &img); err != nil {
+		return err
+	}
+	if img.Data != nil {
+		if x.total+len(img.Data) > maxImageOutput {
+			return fmt.Errorf("image data is over the %d byte limit", maxImageOutput)
+		}
+		x.total += len(img.Data)
+	}
+	x.out = append(x.out, img)
+	return nil
+}
+
+// describe fills img from the stream dictionary. Its fields are set, and
+// either Data and Format are set or Skipped is set.
+func (x *imageExtractor) describe(sd *types.StreamDict, img *Image) error {
+	d := sd.Dict
+	img.Width, _ = x.intEntry(d, "Width")
+	img.Height, _ = x.intEntry(d, "Height")
+	img.BitsPerComponent, _ = x.intEntry(d, "BitsPerComponent")
+	cs, comps, err := x.colorSpace(d)
+	if err != nil {
+		return err
+	}
+	img.ColorSpace = cs
+	if _, ok := d.Find("SMask"); ok {
+		img.HasSMask = true
+	}
+
+	if mask, err := x.boolEntry(d, "ImageMask"); err != nil {
+		return err
+	} else if mask {
+		img.Skipped = "image mask"
+		return nil
+	}
+
+	filters, parms, err := x.filters(d)
+	if err != nil {
+		return err
+	}
+	if len(filters) > 1 {
+		img.Skipped = "filter chain " + strings.Join(filters, "+")
+		return nil
+	}
+	filter := ""
+	if len(filters) == 1 {
+		filter = filters[0]
+	}
+
+	switch filter {
+	case "DCTDecode":
+		switch {
+		case cs == "DeviceCMYK":
+			img.Skipped = "CMYK JPEG needs conversion"
+			return nil
+		case cs != "DeviceRGB" && cs != "DeviceGray":
+			img.Skipped = unsupportedCS(cs)
+			return nil
+		}
+		return x.passthrough(sd, img, "jpeg")
+	case "JBIG2Decode":
+		if _, ok := parms.Find("JBIG2Globals"); ok {
+			img.Skipped = "JBIG2 needs globals"
+			return nil
+		}
+		return x.passthrough(sd, img, "jbig2")
+	case "JPXDecode":
+		return x.passthrough(sd, img, "jpx")
+	case "FlateDecode":
+		return x.flate(sd, img, parms, comps, true)
+	case "":
+		return x.flate(sd, img, parms, comps, false)
+	default:
+		img.Skipped = "unsupported filter " + filter
+		return nil
+	}
+}
+
+// passthrough returns the original stream bytes for a format that is not
+// decoded here. A Decode array other than the default would change the
+// colours, so such an image is skipped.
+func (x *imageExtractor) passthrough(sd *types.StreamDict, img *Image, format string) error {
+	ok, err := x.defaultDecode(sd.Dict)
+	if err != nil {
+		return err
+	}
+	if !ok {
+		img.Skipped = "non-default Decode array"
+		return nil
+	}
+	if len(sd.Raw) == 0 {
+		img.Skipped = "empty stream"
+		return nil
+	}
+	img.Format = format
+	img.Data = append([]byte(nil), sd.Raw...)
+	return nil
+}
+
+// flate decodes a Flate stream of DeviceRGB or DeviceGray samples with 1 or
+// 8 bits per component, applies a PNG predictor if there is one, and encodes
+// the pixels as PNG. Other kinds are skipped with a reason.
+func (x *imageExtractor) flate(sd *types.StreamDict, img *Image, parms types.Dict, comps int, compressed bool) error {
+	if img.ColorSpace != "DeviceRGB" && img.ColorSpace != "DeviceGray" {
+		img.Skipped = unsupportedCS(img.ColorSpace)
+		return nil
+	}
+	if ok, err := x.defaultDecode(sd.Dict); err != nil {
+		return err
+	} else if !ok {
+		img.Skipped = "non-default Decode array"
+		return nil
+	}
+	bpc := img.BitsPerComponent
+	if bpc != 1 && bpc != 8 {
+		img.Skipped = fmt.Sprintf("unsupported bits per component %d", bpc)
+		return nil
+	}
+	if img.Width <= 0 || img.Height <= 0 {
+		img.Skipped = "missing image size"
+		return nil
+	}
+	if img.Width > maxImageDim || img.Height > maxImageDim {
+		img.Skipped = "image dimensions too large"
+		return nil
+	}
+	// Width is at most maxImageDim, so this product cannot overflow. The
+	// total is compared by division, which cannot overflow either.
+	rowBytes := (img.Width*comps*bpc + 7) / 8
+	if rowBytes > maxImageOutput/img.Height {
+		img.Skipped = "image dimensions too large"
+		return nil
+	}
+
+	predictor, _ := x.intEntry(parms, "Predictor")
+	usePNG := false
+	switch {
+	case predictor <= 1:
+	case predictor >= 10:
+		usePNG = true
+		colors, ok := x.intEntry(parms, "Colors")
+		if !ok {
+			colors = comps
+		}
+		pbpc, ok := x.intEntry(parms, "BitsPerComponent")
+		if !ok {
+			pbpc = bpc
+		}
+		columns, ok := x.intEntry(parms, "Columns")
+		if !ok {
+			columns = img.Width
+		}
+		if colors != comps || pbpc != bpc || columns != img.Width {
+			img.Skipped = "predictor parameters do not match the image"
+			return nil
+		}
+	default:
+		img.Skipped = fmt.Sprintf("predictor %d", predictor)
+		return nil
+	}
+
+	stride := rowBytes
+	if usePNG {
+		stride++ // each row starts with its filter type byte
+	}
+	src, err := x.samples(sd, compressed, stride*img.Height)
+	if err != nil {
+		img.Skipped = "corrupt image data: " + err.Error()
+		return nil
+	}
+
+	samples := src
+	if usePNG {
+		samples, err = unfilter(src, rowBytes, img.Height, bytesPerPixel(comps, bpc))
+		if err != nil {
+			img.Skipped = "corrupt image data: " + err.Error()
+			return nil
+		}
+	}
+
+	pix, err := toImage(samples, img.Width, img.Height, comps, bpc)
+	if err != nil {
+		return err
+	}
+	var buf bytes.Buffer
+	if err := png.Encode(&buf, pix); err != nil {
+		return fmt.Errorf("encode png: %w", err)
+	}
+	img.Format = "png"
+	img.Data = buf.Bytes()
+	return nil
+}
+
+// samples returns the first n bytes of the stream, inflated when the stream
+// is Flate-compressed. It fails unless the stream holds exactly n bytes.
+// The buffer grows with the inflated output, not with n, so a small stream
+// that claims a large image costs little memory.
+func (x *imageExtractor) samples(sd *types.StreamDict, compressed bool, n int) ([]byte, error) {
+	if !compressed {
+		if len(sd.Raw) < n {
+			return nil, fmt.Errorf("stream has %d bytes, need %d", len(sd.Raw), n)
+		}
+		return sd.Raw[:n], nil
+	}
+	zr, err := zlib.NewReader(bytes.NewReader(sd.Raw))
+	if err != nil {
+		return nil, err
+	}
+	defer func() { _ = zr.Close() }()
+	// Read one byte past n, so output longer than n is seen without
+	// reading the rest of the stream.
+	var buf bytes.Buffer
+	if _, err := buf.ReadFrom(io.LimitReader(zr, int64(n)+1)); err != nil {
+		return nil, err
+	}
+	switch {
+	case buf.Len() > n:
+		return nil, fmt.Errorf("stream inflates to more than %d bytes", n)
+	case buf.Len() < n:
+		return nil, fmt.Errorf("stream inflates to %d bytes, need %d", buf.Len(), n)
+	}
+	return buf.Bytes(), nil
+}
+
+// bytesPerPixel is the byte distance the PNG predictor uses to find the
+// left neighbour. It is at least 1.
+func bytesPerPixel(comps, bpc int) int {
+	if n := comps * bpc / 8; n > 1 {
+		return n
+	}
+	return 1
+}
+
+// unfilter reverses the PNG row filters (types 0 to 4). src holds one filter
+// byte and rowBytes bytes per row.
+func unfilter(src []byte, rowBytes, height, bpp int) ([]byte, error) {
+	out := make([]byte, rowBytes*height)
+	for y := range height {
+		ft := src[y*(rowBytes+1)]
+		cur := src[y*(rowBytes+1)+1 : (y+1)*(rowBytes+1)]
+		row := out[y*rowBytes : (y+1)*rowBytes]
+		var prev []byte
+		if y > 0 {
+			prev = out[(y-1)*rowBytes : y*rowBytes]
+		}
+		for i := range rowBytes {
+			var left, up, upLeft byte
+			if i >= bpp {
+				left = row[i-bpp]
+			}
+			if prev != nil {
+				up = prev[i]
+				if i >= bpp {
+					upLeft = prev[i-bpp]
+				}
+			}
+			switch ft {
+			case 0:
+				row[i] = cur[i]
+			case 1:
+				row[i] = cur[i] + left
+			case 2:
+				row[i] = cur[i] + up
+			case 3:
+				row[i] = cur[i] + byte((int(left)+int(up))/2)
+			case 4:
+				row[i] = cur[i] + paeth(left, up, upLeft)
+			default:
+				return nil, fmt.Errorf("predictor filter type %d", ft)
+			}
+		}
+	}
+	return out, nil
+}
+
+// paeth is the PNG Paeth predictor.
+func paeth(a, b, c byte) byte {
+	p := int(a) + int(b) - int(c)
+	pa, pb, pc := abs(p-int(a)), abs(p-int(b)), abs(p-int(c))
+	switch {
+	case pa <= pb && pa <= pc:
+		return a
+	case pb <= pc:
+		return b
+	default:
+		return c
+	}
+}
+
+func abs(v int) int {
+	if v < 0 {
+		return -v
+	}
+	return v
+}
+
+// toImage packs unfiltered samples into an image. DeviceGray becomes
+// image.Gray, and DeviceRGB becomes an opaque image.RGBA. A 1-bit sample is
+// scaled to 0 or 255.
+func toImage(samples []byte, w, h, comps, bpc int) (image.Image, error) {
+	rowBytes := (w*comps*bpc + 7) / 8
+	sample := func(row []byte, i int) byte {
+		if bpc == 8 {
+			return row[i]
+		}
+		v := (row[i/8] >> (7 - uint(i%8))) & 1
+		return v * 255
+	}
+	if comps == 1 {
+		g := image.NewGray(image.Rect(0, 0, w, h))
+		for y := range h {
+			row := samples[y*rowBytes : (y+1)*rowBytes]
+			for x := range w {
+				g.Pix[y*g.Stride+x] = sample(row, x)
+			}
+		}
+		return g, nil
+	}
+	if comps != 3 {
+		return nil, fmt.Errorf("pdf: extract images: %d components", comps)
+	}
+	rgba := image.NewRGBA(image.Rect(0, 0, w, h))
+	for y := range h {
+		row := samples[y*rowBytes : (y+1)*rowBytes]
+		for x := range w {
+			o := y*rgba.Stride + 4*x
+			rgba.Pix[o] = sample(row, 3*x)
+			rgba.Pix[o+1] = sample(row, 3*x+1)
+			rgba.Pix[o+2] = sample(row, 3*x+2)
+			rgba.Pix[o+3] = 0xff
+		}
+	}
+	return rgba, nil
+}
+
+// unsupportedCS is the Skipped reason for a colour space that is not handled.
+func unsupportedCS(cs string) string {
+	if cs == "" {
+		return "missing colour space"
+	}
+	return "unsupported colour space " + cs
+}
+
+// get dereferences the value of key in d. It returns nil when the key is
+// absent.
+func (x *imageExtractor) get(d types.Dict, key string) (types.Object, error) {
+	o, found := d.Find(key)
+	if !found || o == nil {
+		return nil, nil
+	}
+	return x.ctx.Dereference(o)
+}
+
+// dict returns the dictionary value of key, or nil when it is absent or not
+// a dictionary.
+func (x *imageExtractor) dict(d types.Dict, key string) (types.Dict, error) {
+	o, err := x.get(d, key)
+	if err != nil {
+		return nil, err
+	}
+	dd, _ := o.(types.Dict)
+	return dd, nil
+}
+
+// name returns the name value of key.
+func (x *imageExtractor) name(d types.Dict, key string) (string, error) {
+	o, err := x.get(d, key)
+	if err != nil {
+		return "", err
+	}
+	n, _ := o.(types.Name)
+	return string(n), nil
+}
+
+// intEntry returns the integer value of key.
+func (x *imageExtractor) intEntry(d types.Dict, key string) (int, bool) {
+	o, err := x.get(d, key)
+	if err != nil {
+		return 0, false
+	}
+	switch v := o.(type) {
+	case types.Integer:
+		return v.Value(), true
+	case types.Float:
+		// A float that is not finite or does not fit in int32 is not a
+		// usable integer. It reads as absent, so the caller skips the image.
+		f := float64(v)
+		if math.IsNaN(f) || math.IsInf(f, 0) || f < math.MinInt32 || f > math.MaxInt32 {
+			return 0, false
+		}
+		return int(f), true
+	}
+	return 0, false
+}
+
+// boolEntry returns the boolean value of key. An absent key is false.
+func (x *imageExtractor) boolEntry(d types.Dict, key string) (bool, error) {
+	o, err := x.get(d, key)
+	if err != nil {
+		return false, err
+	}
+	b, _ := o.(types.Boolean)
+	return bool(b), nil
+}
+
+// colorSpace returns the name of the colour space and its component count.
+// The name is the first name of an array colour space, such as ICCBased. The
+// count is 0 for a colour space this package does not decode.
+func (x *imageExtractor) colorSpace(d types.Dict) (string, int, error) {
+	o, err := x.get(d, "ColorSpace")
+	if err != nil {
+		return "", 0, err
+	}
+	var cs string
+	switch v := o.(type) {
+	case types.Name:
+		cs = string(v)
+	case types.Array:
+		if len(v) > 0 {
+			first, err := x.ctx.Dereference(v[0])
+			if err != nil {
+				return "", 0, err
+			}
+			if n, ok := first.(types.Name); ok {
+				cs = string(n)
+			}
+		}
+	case nil:
+	default:
+		cs = fmt.Sprintf("%T", o)
+	}
+	switch cs {
+	case "DeviceRGB":
+		return cs, 3, nil
+	case "DeviceGray":
+		return cs, 1, nil
+	case "DeviceCMYK":
+		return cs, 4, nil
+	}
+	return cs, 0, nil
+}
+
+// filters returns the filter names of a stream and the DecodeParms of the
+// first filter. The parameters are nil when there are none.
+func (x *imageExtractor) filters(d types.Dict) ([]string, types.Dict, error) {
+	o, err := x.get(d, "Filter")
+	if err != nil {
+		return nil, nil, err
+	}
+	var names []string
+	switch v := o.(type) {
+	case types.Name:
+		names = []string{string(v)}
+	case types.Array:
+		for _, e := range v {
+			eo, err := x.ctx.Dereference(e)
+			if err != nil {
+				return nil, nil, err
+			}
+			if n, ok := eo.(types.Name); ok {
+				names = append(names, string(n))
+			}
+		}
+	}
+
+	po, err := x.get(d, "DecodeParms")
+	if err != nil {
+		return nil, nil, err
+	}
+	switch v := po.(type) {
+	case types.Dict:
+		return names, v, nil
+	case types.Array:
+		if len(v) > 0 {
+			first, err := x.ctx.Dereference(v[0])
+			if err != nil {
+				return nil, nil, err
+			}
+			if pd, ok := first.(types.Dict); ok {
+				return names, pd, nil
+			}
+		}
+	}
+	return names, nil, nil
+}
+
+// defaultDecode reports whether the image has no Decode array, or the
+// identity one ([0 1] for each component). Any other Decode array inverts or
+// remaps the samples, which this package does not apply.
+func (x *imageExtractor) defaultDecode(d types.Dict) (bool, error) {
+	o, err := x.get(d, "Decode")
+	if err != nil {
+		return false, err
+	}
+	arr, ok := o.(types.Array)
+	if !ok {
+		return true, nil
+	}
+	for i := 0; i+1 < len(arr); i += 2 {
+		lo, err := x.ctx.DereferenceNumber(arr[i])
+		if err != nil {
+			return false, nil
+		}
+		hi, err := x.ctx.DereferenceNumber(arr[i+1])
+		if err != nil {
+			return false, nil
+		}
+		if lo != 0 || hi != 1 {
+			return false, nil
+		}
+	}
+	return true, nil
+}

addons/office/internal/pdf/images_test.go

@@ -0,0 +1,562 @@
+package pdf
+
+import (
+	"bytes"
+	"compress/zlib"
+	"fmt"
+	"image"
+	"image/color"
+	"image/jpeg"
+	"image/png"
+	"strings"
+	"time"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// pdfObj is one numbered object of a hand-written PDF. dict is the text
+// inside << >>. When stream is set the object is a stream, and its /Length
+// is added here.
+type pdfObj struct {
+	dict   string
+	stream []byte
+}
+
+// buildPDF writes objs as objects 1..n with a cross-reference table and a
+// trailer whose /Root is object 1.
+func buildPDF(objs []pdfObj) []byte {
+	var b bytes.Buffer
+	b.WriteString("%PDF-1.7\n")
+	offs := make([]int, len(objs))
+	for i, o := range objs {
+		offs[i] = b.Len()
+		fmt.Fprintf(&b, "%d 0 obj\n", i+1)
+		if o.stream == nil {
+			fmt.Fprintf(&b, "<< %s >>\nendobj\n", o.dict)
+			continue
+		}
+		fmt.Fprintf(&b, "<< %s /Length %d >>\nstream\n", o.dict, len(o.stream))
+		b.Write(o.stream)
+		b.WriteString("\nendstream\nendobj\n")
+	}
+	xref := b.Len()
+	fmt.Fprintf(&b, "xref\n0 %d\n0000000000 65535 f \n", len(objs)+1)
+	for _, o := range offs {
+		fmt.Fprintf(&b, "%010d 00000 n \n", o)
+	}
+	fmt.Fprintf(&b, "trailer\n<< /Size %d /Root 1 0 R >>\nstartxref\n%d\n%%%%EOF\n", len(objs)+1, xref)
+	return b.Bytes()
+}
+
+// pagesDoc builds a document with one page per resources entry. Each entry
+// is the text of that page's /Resources. Object 1 is the catalog, object 2
+// the page tree and page i is object 3+i. The extra objects follow the pages,
+// so the first of them is object 3+len(resources).
+func pagesDoc(resources []string, extra ...pdfObj) []byte {
+	kids := ""
+	for i := range resources {
+		kids += fmt.Sprintf("%d 0 R ", 3+i)
+	}
+	objs := []pdfObj{
+		{dict: "/Type /Catalog /Pages 2 0 R"},
+		{dict: fmt.Sprintf("/Type /Pages /Kids [%s] /Count %d", kids, len(resources))},
+	}
+	for _, res := range resources {
+		objs = append(objs, pdfObj{dict: "/Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Resources " + res})
+	}
+	return buildPDF(append(objs, extra...))
+}
+
+// zlibBytes compresses b as a zlib stream, the way a Flate filter expects.
+func zlibBytes(b []byte) []byte {
+	GinkgoHelper()
+	var buf bytes.Buffer
+	w := zlib.NewWriter(&buf)
+	_, err := w.Write(b)
+	Expect(err).ToNot(HaveOccurred())
+	Expect(w.Close()).To(Succeed())
+	return buf.Bytes()
+}
+
+// flateRGB returns an 8-bit DeviceRGB Flate image object of w by h pixels
+// with the given raw samples.
+func flateRGB(w, h int, samples []byte, extraEntries string) pdfObj {
+	GinkgoHelper()
+	return pdfObj{
+		dict:   fmt.Sprintf("/Type /XObject /Subtype /Image /Width %d /Height %d /ColorSpace /DeviceRGB /BitsPerComponent 8 /Filter /FlateDecode %s", w, h, extraEntries),
+		stream: zlibBytes(samples),
+	}
+}
+
+// xobjectRes returns a /Resources dictionary with the given names mapped to
+// object numbers.
+func xobjectRes(entries ...string) string {
+	return "<< /XObject << " + strings.Join(entries, " ") + " >> >>"
+}
+
+// pngFilter applies one PNG row filter to cur. prev is the row above, or nil
+// for the first row. The filter is written from the PNG specification and
+// does not use the package's own code.
+func pngFilter(ft byte, cur, prev []byte, bpp int) []byte {
+	out := make([]byte, 1+len(cur))
+	out[0] = ft
+	for i := range cur {
+		var a, b, c int
+		if i >= bpp {
+			a = int(cur[i-bpp])
+		}
+		if prev != nil {
+			b = int(prev[i])
+			if i >= bpp {
+				c = int(prev[i-bpp])
+			}
+		}
+		var pred int
+		switch ft {
+		case 0:
+			pred = 0
+		case 1:
+			pred = a
+		case 2:
+			pred = b
+		case 3:
+			pred = (a + b) / 2
+		case 4:
+			p := a + b - c
+			pa, pb, pc := abs(p-a), abs(p-b), abs(p-c)
+			switch {
+			case pa <= pb && pa <= pc:
+				pred = a
+			case pb <= pc:
+				pred = b
+			default:
+				pred = c
+			}
+		}
+		out[1+i] = cur[i] - byte(pred)
+	}
+	return out
+}
+
+// decodePNG decodes PNG bytes and fails the spec when they are not a PNG.
+func decodePNG(data []byte) image.Image {
+	GinkgoHelper()
+	img, err := png.Decode(bytes.NewReader(data))
+	Expect(err).ToNot(HaveOccurred(), "decode png")
+	return img
+}
+
+// rgbaAt returns the colour of one pixel. Gray pixels come back with equal
+// R, G and B.
+func rgbaAt(img image.Image, x, y int) color.RGBA {
+	return color.RGBAModel.Convert(img.At(x, y)).(color.RGBA)
+}
+
+// onlyImage returns the single image of a result or fails the spec.
+func onlyImage(imgs []Image) Image {
+	GinkgoHelper()
+	Expect(imgs).To(HaveLen(1))
+	return imgs[0]
+}
+
+// mustExtract runs ExtractImages or fails the spec.
+func mustExtract(data []byte, pages []int) []Image {
+	GinkgoHelper()
+	imgs, err := ExtractImages(data, pages)
+	Expect(err).ToNot(HaveOccurred(), "ExtractImages")
+	return imgs
+}
+
+func imagesByName(imgs []Image) map[string]Image {
+	byName := map[string]Image{}
+	for _, im := range imgs {
+		byName[im.Name] = im
+	}
+	return byName
+}
+
+var _ = Describe("ExtractImages", func() {
+	It("extracts a Flate RGB image with PNG predictors exactly", func() {
+		const w, h = 4, 5
+		bpp := 3
+		// Every PNG filter type (0 to 4) is used, one per row, so each unfilter
+		// path runs. Values wrap past 255 on purpose.
+		want := make([][]byte, h)
+		for y := range want {
+			want[y] = make([]byte, w*3)
+			for x := range w {
+				want[y][3*x] = byte(x*97 + y*31)
+				want[y][3*x+1] = byte(y*83 + x*7)
+				want[y][3*x+2] = byte(250 - x*40 - y*9)
+			}
+		}
+		var filtered []byte
+		for y := range h {
+			var prev []byte
+			if y > 0 {
+				prev = want[y-1]
+			}
+			filtered = append(filtered, pngFilter(byte(y), want[y], prev, bpp)...)
+		}
+		img := pdfObj{
+			dict:   fmt.Sprintf("/Type /XObject /Subtype /Image /Width %d /Height %d /ColorSpace /DeviceRGB /BitsPerComponent 8 /Filter /FlateDecode /DecodeParms << /Predictor 15 /Colors 3 /Columns %d /BitsPerComponent 8 >>", w, h, w),
+			stream: zlibBytes(filtered),
+		}
+		data := pagesDoc([]string{xobjectRes("/Im1 4 0 R")}, img)
+
+		got := onlyImage(mustExtract(data, nil))
+		Expect(got.Format).To(Equal("png"), "unexpected image: %+v", got)
+		Expect(got.Skipped).To(BeEmpty(), "unexpected image: %+v", got)
+		Expect(got.Page).To(Equal(1))
+		Expect(got.Name).To(Equal("Im1"))
+		Expect(got.Width).To(Equal(w), "metadata: %+v", got)
+		Expect(got.Height).To(Equal(h), "metadata: %+v", got)
+		Expect(got.BitsPerComponent).To(Equal(8), "metadata: %+v", got)
+		Expect(got.ColorSpace).To(Equal("DeviceRGB"), "metadata: %+v", got)
+		px := decodePNG(got.Data)
+		for y := range h {
+			for x := range w {
+				c := rgbaAt(px, x, y)
+				wantPix := color.RGBA{want[y][3*x], want[y][3*x+1], want[y][3*x+2], 0xff}
+				Expect(c).To(Equal(wantPix), "pixel (%d,%d)", x, y)
+			}
+		}
+	})
+
+	It("extracts a one-bit gray image", func() {
+		// 5 by 2 samples. Row 0 is 1 0 1 0 1 and row 1 is 1 1 1 0 0.
+		// A 1 sample is white (255) and a 0 sample is black (0).
+		samples := []byte{0xa8, 0xe0}
+		img := pdfObj{
+			dict:   "/Type /XObject /Subtype /Image /Width 5 /Height 2 /ColorSpace /DeviceGray /BitsPerComponent 1 /Filter /FlateDecode",
+			stream: zlibBytes(samples),
+		}
+		data := pagesDoc([]string{xobjectRes("/Im1 4 0 R")}, img)
+
+		got := onlyImage(mustExtract(data, nil))
+		Expect(got.Format).To(Equal("png"), "skipped %q", got.Skipped)
+		px := decodePNG(got.Data)
+		want := [][]uint8{{255, 0, 255, 0, 255}, {255, 255, 255, 0, 0}}
+		for y := range want {
+			for x := range want[y] {
+				Expect(rgbaAt(px, x, y).R).To(Equal(want[y][x]), "pixel (%d,%d)", x, y)
+			}
+		}
+	})
+
+	It("returns a DCT JPEG unchanged", func() {
+		src := image.NewRGBA(image.Rect(0, 0, 5, 4))
+		for y := range 4 {
+			for x := range 5 {
+				src.Set(x, y, color.RGBA{uint8(x * 50), uint8(y * 60), 128, 0xff})
+			}
+		}
+		var jb bytes.Buffer
+		Expect(jpeg.Encode(&jb, src, &jpeg.Options{Quality: 90})).To(Succeed())
+		jpg := pdfObj{
+			dict:   "/Type /XObject /Subtype /Image /Width 5 /Height 4 /ColorSpace /DeviceRGB /BitsPerComponent 8 /Filter /DCTDecode",
+			stream: jb.Bytes(),
+		}
+		data := pagesDoc([]string{xobjectRes("/Im1 4 0 R")}, jpg)
+
+		got := onlyImage(mustExtract(data, nil))
+		Expect(got.Format).To(Equal("jpeg"), "unexpected image: %+v", got)
+		Expect(got.Skipped).To(BeEmpty(), "unexpected image: %+v", got)
+		Expect(bytes.Equal(got.Data, jb.Bytes())).To(BeTrue(), "Data is not the original JPEG stream")
+		_, err := jpeg.Decode(bytes.NewReader(got.Data))
+		Expect(err).ToNot(HaveOccurred(), "Data does not decode as JPEG")
+	})
+
+	It("skips a CMYK JPEG", func() {
+		jpg := pdfObj{
+			dict:   "/Type /XObject /Subtype /Image /Width 8 /Height 8 /ColorSpace /DeviceCMYK /BitsPerComponent 8 /Filter /DCTDecode",
+			stream: []byte("not decoded here"),
+		}
+		data := pagesDoc([]string{xobjectRes("/Im1 4 0 R")}, jpg)
+
+		got := onlyImage(mustExtract(data, nil))
+		Expect(got.Skipped).To(Equal("CMYK JPEG needs conversion"))
+		Expect(got.Data).To(BeNil(), "skipped image has data")
+		Expect(got.Format).To(BeEmpty(), "skipped image has a format")
+		Expect(got.ColorSpace).To(Equal("DeviceCMYK"))
+	})
+
+	It("skips an image mask and still processes the page", func() {
+		mask := pdfObj{
+			dict:   "/Type /XObject /Subtype /Image /Width 8 /Height 2 /ImageMask true /BitsPerComponent 1 /Filter /FlateDecode",
+			stream: zlibBytes([]byte{0xaa, 0x55}),
+		}
+		rgb := flateRGB(2, 1, []byte{255, 0, 0, 0, 255, 0}, "")
+		data := pagesDoc([]string{xobjectRes("/Mask 4 0 R", "/Im1 5 0 R")}, mask, rgb)
+
+		imgs := mustExtract(data, nil)
+		Expect(imgs).To(HaveLen(2))
+		// Names are read in sorted order, so /Im1 comes before /Mask.
+		byName := imagesByName(imgs)
+		m := byName["Mask"]
+		Expect(m.Skipped).To(Equal("image mask"), "mask: %+v", m)
+		Expect(m.Data).To(BeNil(), "mask: %+v", m)
+		im1 := byName["Im1"]
+		Expect(im1.Format).To(Equal("png"), "rgb image: %+v", im1)
+		Expect(im1.Skipped).To(BeEmpty(), "rgb image: %+v", im1)
+		px := decodePNG(im1.Data)
+		Expect(rgbaAt(px, 0, 0)).To(Equal(color.RGBA{255, 0, 0, 255}))
+		Expect(rgbaAt(px, 1, 0)).To(Equal(color.RGBA{0, 255, 0, 255}))
+	})
+
+	It("returns nothing for a page without images", func() {
+		data := pagesDoc([]string{"<< >>"})
+		Expect(mustExtract(data, nil)).To(BeEmpty())
+	})
+
+	It("extracts only the requested page range", func() {
+		// Three pages occupy objects 3 to 5, so the images are objects 6 to 8.
+		extra := []pdfObj{
+			flateRGB(1, 1, []byte{0, 0, 0}, ""),
+			flateRGB(1, 1, []byte{1, 0, 0}, ""),
+			flateRGB(1, 1, []byte{2, 0, 0}, ""),
+		}
+		data := pagesDoc([]string{
+			xobjectRes("/Im1 6 0 R"),
+			xobjectRes("/Im1 7 0 R"),
+			xobjectRes("/Im1 8 0 R"),
+		}, extra...)
+
+		all := mustExtract(data, nil)
+		Expect(all).To(HaveLen(3))
+		Expect(all[0].Page).To(Equal(1))
+		Expect(all[1].Page).To(Equal(2))
+		Expect(all[2].Page).To(Equal(3))
+		one := mustExtract(data, []int{2})
+		Expect(one).To(HaveLen(1))
+		Expect(one[0].Page).To(Equal(2))
+		Expect(rgbaAt(decodePNG(one[0].Data), 0, 0).R).To(Equal(uint8(1)), "page 2 image pixel")
+		_, err := ExtractImages(data, []int{4})
+		Expect(err).To(HaveOccurred(), "page outside the document")
+		_, err = ExtractImages(data, []int{0})
+		Expect(err).To(HaveOccurred(), "page 0")
+	})
+
+	It("finds an image nested in a form", func() {
+		form := pdfObj{
+			dict:   "/Type /XObject /Subtype /Form /BBox [0 0 100 100] /Resources " + xobjectRes("/Im2 5 0 R"),
+			stream: []byte(""),
+		}
+		data := pagesDoc([]string{xobjectRes("/Fm1 4 0 R")}, form, flateRGB(1, 1, []byte{1, 2, 3}, ""))
+
+		got := onlyImage(mustExtract(data, nil))
+		Expect(got.Name).To(Equal("Im2"), "nested image: %+v", got)
+		Expect(got.Format).To(Equal("png"), "nested image: %+v", got)
+	})
+
+	It("terminates on a form that refers to itself", func() {
+		// The walk must stop and still find the image.
+		form := pdfObj{
+			dict:   "/Type /XObject /Subtype /Form /BBox [0 0 100 100] /Resources " + xobjectRes("/Self 4 0 R", "/Im2 5 0 R"),
+			stream: []byte(""),
+		}
+		data := pagesDoc([]string{xobjectRes("/Fm1 4 0 R")}, form, flateRGB(1, 1, []byte{1, 2, 3}, ""))
+
+		got := onlyImage(mustExtract(data, nil))
+		Expect(got.Name).To(Equal("Im2"))
+	})
+
+	// Each image kind that is returned as is or skipped with a reason. Every case
+	// is a one-page document with the image as object 4 and, when needed, an
+	// extra object 5.
+	gray := "/Type /XObject /Subtype /Image /Width 2 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 /Filter /FlateDecode"
+	DescribeTable("image kinds",
+		func(img pdfObj, extra []pdfObj, wantFormat, wantSkipped string, wantData, wantSMask bool) {
+			data := pagesDoc([]string{xobjectRes("/Im1 4 0 R")}, append([]pdfObj{img}, extra...)...)
+			got := onlyImage(mustExtract(data, nil))
+			Expect(got.Format).To(Equal(wantFormat))
+			if wantSkipped == "" {
+				Expect(got.Skipped).To(BeEmpty())
+			} else {
+				Expect(got.Skipped).To(HavePrefix(wantSkipped))
+			}
+			Expect(got.Data != nil).To(Equal(wantData), "Data present")
+			Expect(got.HasSMask).To(Equal(wantSMask))
+			if wantData && wantFormat == "jbig2" {
+				Expect(string(got.Data)).To(Equal("jbig2 bytes"), "JBIG2 Data is the original stream")
+			}
+		},
+		Entry("JBIG2 without globals is passed through",
+			pdfObj{dict: "/Type /XObject /Subtype /Image /Width 8 /Height 8 /BitsPerComponent 1 /ColorSpace /DeviceGray /Filter /JBIG2Decode", stream: []byte("jbig2 bytes")},
+			[]pdfObj(nil), "jbig2", "", true, false),
+		Entry("JBIG2 with globals is skipped",
+			pdfObj{dict: "/Type /XObject /Subtype /Image /Width 8 /Height 8 /BitsPerComponent 1 /ColorSpace /DeviceGray /Filter /JBIG2Decode /DecodeParms << /JBIG2Globals 5 0 R >>", stream: []byte("jbig2 bytes")},
+			[]pdfObj{{dict: "/Type /XObject", stream: []byte("globals")}}, "", "JBIG2 needs globals", false, false),
+		Entry("JPX is passed through",
+			pdfObj{dict: "/Type /XObject /Subtype /Image /Width 8 /Height 8 /Filter /JPXDecode", stream: []byte("jpx bytes")},
+			[]pdfObj(nil), "jpx", "", true, false),
+		Entry("CMYK Flate is skipped with its colour space",
+			pdfObj{dict: "/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceCMYK /BitsPerComponent 8 /Filter /FlateDecode", stream: zlibBytes([]byte{1, 2, 3, 4})},
+			[]pdfObj(nil), "", "unsupported colour space DeviceCMYK", false, false),
+		Entry("indexed colour space is skipped",
+			pdfObj{dict: "/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace [/Indexed /DeviceRGB 1 <000000ff00>] /BitsPerComponent 8 /Filter /FlateDecode", stream: zlibBytes([]byte{0})},
+			[]pdfObj(nil), "", "unsupported colour space Indexed", false, false),
+		Entry("LZW filter is skipped with its name",
+			pdfObj{dict: "/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 /Filter /LZWDecode", stream: []byte("x")},
+			[]pdfObj(nil), "", "unsupported filter LZWDecode", false, false),
+		Entry("filter chain is skipped",
+			pdfObj{dict: "/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 /Filter [/ASCIIHexDecode /FlateDecode]", stream: []byte("x")},
+			[]pdfObj(nil), "", "filter chain ASCIIHexDecode+FlateDecode", false, false),
+		Entry("16 bit samples are skipped",
+			pdfObj{dict: "/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 16 /Filter /FlateDecode", stream: zlibBytes([]byte{0, 1})},
+			[]pdfObj(nil), "", "unsupported bits per component 16", false, false),
+		Entry("inverted JPEG Decode array is skipped",
+			pdfObj{dict: "/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceRGB /BitsPerComponent 8 /Decode [1 0 1 0 1 0] /Filter /DCTDecode", stream: []byte("jpeg")},
+			[]pdfObj(nil), "", "non-default Decode array", false, false),
+		Entry("corrupt Flate data is skipped, not an error",
+			pdfObj{dict: gray, stream: []byte("garbage, not zlib")},
+			[]pdfObj(nil), "", "corrupt image data", false, false),
+		Entry("SMask is reported and not applied",
+			pdfObj{dict: gray + " /SMask 5 0 R", stream: zlibBytes([]byte{10, 20})},
+			[]pdfObj{{dict: "/Type /XObject /Subtype /Image /Width 2 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 /Filter /FlateDecode", stream: zlibBytes([]byte{255, 0})}},
+			"png", "", true, true),
+	)
+
+	// 200 images are allowed. The 201st is an error.
+	It("enforces the image count cap", func() {
+		build := func(n int) []byte {
+			var names []string
+			var extra []pdfObj
+			for i := range n {
+				names = append(names, fmt.Sprintf("/I%d %d 0 R", i, 4+i))
+				extra = append(extra, flateRGB(1, 1, []byte{1, 2, 3}, ""))
+			}
+			return pagesDoc([]string{xobjectRes(names...)}, extra...)
+		}
+		Expect(mustExtract(build(maxImages), nil)).To(HaveLen(maxImages))
+		_, err := ExtractImages(build(maxImages+1), nil)
+		Expect(err).To(HaveOccurred(), "%d images", maxImages+1)
+	})
+
+	It("returns an error for input that is not a PDF", func() {
+		_, err := ExtractImages([]byte("this is not a pdf file"), nil)
+		Expect(err).To(HaveOccurred(), "bad input")
+		_, err = ExtractImages(nil, nil)
+		Expect(err).To(HaveOccurred(), "empty input")
+	})
+
+	// Cuts a PDF with several kinds of image at many lengths. Each cut must
+	// return without a panic.
+	It("does not panic on truncated prefixes", func() {
+		good := pagesDoc(
+			[]string{xobjectRes("/Im1 4 0 R", "/Im2 5 0 R")},
+			flateRGB(2, 2, []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}, "/DecodeParms << /Predictor 2 >>"),
+			pdfObj{dict: "/Type /XObject /Subtype /Image /Width 8 /Height 8 /ColorSpace /DeviceRGB /BitsPerComponent 8 /Filter /DCTDecode", stream: []byte("abcdefgh")},
+		)
+		step := max(len(good)/150, 1)
+		for n := 0; n <= len(good); n += step {
+			_, _ = ExtractImages(good[:n], nil)
+		}
+	})
+
+	It("returns an error for input over the limit", func() {
+		_, err := ExtractImages(make([]byte, maxImageInputBytes+1), nil)
+		Expect(err).To(HaveOccurred())
+	})
+
+	// The first image claims 8000 by 8000 RGB but inflates to 3 bytes. The
+	// second claims 1 pixel but inflates to 6 bytes. Both are skipped, and the
+	// page's other image is kept.
+	It("skips Flate images whose size claim does not match the data", func() {
+		short := flateRGB(8000, 8000, []byte{1, 2, 3}, "")
+		long := flateRGB(1, 1, []byte{1, 2, 3, 4, 5, 6}, "")
+		ok := flateRGB(1, 1, []byte{1, 2, 3}, "")
+		data := pagesDoc([]string{xobjectRes("/Short 4 0 R", "/Long 5 0 R", "/Ok 6 0 R")}, short, long, ok)
+
+		imgs := mustExtract(data, nil)
+		Expect(imgs).To(HaveLen(3))
+		byName := imagesByName(imgs)
+		for _, name := range []string{"Short", "Long"} {
+			im := byName[name]
+			Expect(im.Skipped).To(HavePrefix("corrupt image data"), "%s: %+v", name, im)
+			Expect(im.Data).To(BeNil(), "%s: %+v", name, im)
+		}
+		im := byName["Ok"]
+		Expect(im.Format).To(Equal("png"), "Ok: %+v", im)
+		Expect(im.Skipped).To(BeEmpty(), "Ok: %+v", im)
+	})
+
+	// Widths that overflow the row arithmetic, and a float width that is not a
+	// usable integer. Each image is skipped alone. The page's other image is
+	// still returned.
+	It("skips images with huge dimensions", func() {
+		cases := []struct {
+			width      string
+			wantReason string
+		}{
+			{"3074457345618258603", "image dimensions too large"},
+			{"9223372036854775807", "image dimensions too large"},
+			{"100000000000.5", "missing image size"},
+		}
+		for _, c := range cases {
+			big := pdfObj{
+				dict:   "/Type /XObject /Subtype /Image /Width " + c.width + " /Height 1 /ColorSpace /DeviceRGB /BitsPerComponent 8 /Filter /FlateDecode",
+				stream: zlibBytes([]byte{1, 2, 3}),
+			}
+			data := pagesDoc([]string{xobjectRes("/Big 4 0 R", "/Ok 5 0 R")}, big, flateRGB(1, 1, []byte{1, 2, 3}, ""))
+
+			imgs := mustExtract(data, nil)
+			Expect(imgs).To(HaveLen(2), "width %s", c.width)
+			byName := imagesByName(imgs)
+			b := byName["Big"]
+			Expect(b.Skipped).To(Equal(c.wantReason), "width %s: %+v", c.width, b)
+			Expect(b.Data).To(BeNil(), "width %s: %+v", c.width, b)
+			ok := byName["Ok"]
+			Expect(ok.Format).To(Equal("png"), "width %s: Ok image: %+v", c.width, ok)
+			Expect(ok.Skipped).To(BeEmpty(), "width %s: Ok image: %+v", c.width, ok)
+		}
+	})
+
+	// 12 forms, where each form refers to the next one three times. Walked as a
+	// tree, that is 3^11 visits. Each form must be walked once, so the call is
+	// fast and the image appears once.
+	It("walks a form DAG once", func() {
+		const levels = 12
+		var extra []pdfObj
+		for i := range levels {
+			res := xobjectRes("/Im 16 0 R") // the last form holds the image, object 16
+			if i < levels-1 {
+				next := fmt.Sprintf("%d 0 R", 5+i) // form i+1 is object 5+i
+				res = xobjectRes("/A "+next, "/B "+next, "/C "+next)
+			}
+			extra = append(extra, pdfObj{
+				dict:   "/Type /XObject /Subtype /Form /BBox [0 0 100 100] /Resources " + res,
+				stream: []byte(""),
+			})
+		}
+		extra = append(extra, flateRGB(1, 1, []byte{1, 2, 3}, ""))
+		data := pagesDoc([]string{xobjectRes("/Fm 4 0 R")}, extra...)
+
+		start := time.Now()
+		got := onlyImage(mustExtract(data, nil))
+		Expect(time.Since(start)).To(BeNumerically("<=", 2*time.Second), "form DAG duration")
+		Expect(got.Name).To(Equal("Im"), "image: %+v", got)
+		Expect(got.Format).To(Equal("png"), "image: %+v", got)
+	})
+
+	// Two pages share the same form (object 5), which holds one image
+	// (object 6). Each page must report the image under its own number, as it
+	// would when asked for alone.
+	It("reports a form shared by two pages on both", func() {
+		form := pdfObj{
+			dict:   "/Type /XObject /Subtype /Form /BBox [0 0 100 100] /Resources " + xobjectRes("/Im 6 0 R"),
+			stream: []byte(""),
+		}
+		data := pagesDoc([]string{xobjectRes("/Fm 5 0 R"), xobjectRes("/Fm 5 0 R")},
+			form, flateRGB(1, 1, []byte{1, 2, 3}, ""))
+
+		got := mustExtract(data, []int{1, 2})
+		Expect(got).To(HaveLen(2))
+		for i, page := range []int{1, 2} {
+			Expect(got[i].Page).To(Equal(page), "image %d: %+v", i, got[i])
+			Expect(got[i].Name).To(Equal("Im"), "image %d: %+v", i, got[i])
+			Expect(got[i].Format).To(Equal("png"), "image %d: %+v", i, got[i])
+		}
+	})
+})

addons/office/internal/pdf/lib.go

@@ -0,0 +1,23 @@
+package pdf
+
+import (
+	"io"
+
+	"github.com/pdfcpu/pdfcpu/pkg/api"
+	"github.com/pdfcpu/pdfcpu/pkg/pdfcpu/model"
+)
+
+// newConf returns the default pdfcpu configuration. It reads no file.
+func newConf() *model.Configuration {
+	return model.NewDefaultConfiguration()
+}
+
+// validate runs the pdfcpu validator on rs.
+func validate(rs io.ReadSeeker) error {
+	return api.Validate(rs, newConf())
+}
+
+// pageCount returns the number of pages of rs.
+func pageCount(rs io.ReadSeeker) (int, error) {
+	return api.PageCount(rs, newConf())
+}

addons/office/internal/pdf/pdf.go

@@ -0,0 +1,108 @@
+// Package pdf creates, merges, splits, rotates, stamps and fills PDF files.
+// Creation uses go-pdf/fpdf. Everything else uses pdfcpu, through its
+// in-memory API. No temporary file is written.
+//
+// This is a nested module (its own go.mod). Its two third-party
+// dependencies stay out of the builds of the root module.
+//
+// Page numbers are 1-based. Every public function returns an error for
+// malformed bytes, bad page numbers and bad arguments. A panic inside a
+// library call is turned into an error.
+package pdf
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"io"
+)
+
+// Size limits. Input over MaxInputBytes is refused. A Spec over MaxPages
+// pages or MaxBlocks blocks is refused.
+const (
+	MaxInputBytes = 64 << 20
+	MaxPages      = 1000
+	MaxBlocks     = 2000
+)
+
+// errEmptyInput is returned when a function gets no bytes.
+var errEmptyInput = errors.New("pdf: input is empty")
+
+// checkInput refuses empty input and input over MaxInputBytes.
+func checkInput(data []byte) error {
+	if len(data) == 0 {
+		return errEmptyInput
+	}
+	if len(data) > MaxInputBytes {
+		return fmt.Errorf("pdf: input is %d bytes, the limit is %d", len(data), MaxInputBytes)
+	}
+	return nil
+}
+
+// guard runs fn and turns a panic into an error. Every pdfcpu and fpdf call
+// goes through it.
+func guard(op string, fn func() error) (err error) {
+	defer func() {
+		if r := recover(); r != nil {
+			err = fmt.Errorf("pdf: %s: internal error: %v", op, r)
+		}
+	}()
+	return fn()
+}
+
+// Check validates the bytes with pdfcpu. It returns nil when the file is a
+// valid PDF.
+func Check(data []byte) error {
+	if err := checkInput(data); err != nil {
+		return err
+	}
+	return guard("check", func() error {
+		if err := validate(bytes.NewReader(data)); err != nil {
+			return fmt.Errorf("pdf: check: %w", err)
+		}
+		return nil
+	})
+}
+
+// Pages returns the number of pages of a PDF.
+func Pages(data []byte) (int, error) {
+	if err := checkInput(data); err != nil {
+		return 0, err
+	}
+	var n int
+	err := guard("pages", func() error {
+		count, err := pageCount(bytes.NewReader(data))
+		if err != nil {
+			return fmt.Errorf("pdf: pages: %w", err)
+		}
+		n = count
+		return nil
+	})
+	if err != nil {
+		return 0, err
+	}
+	return n, nil
+}
+
+// checkPage returns an error when a 1-based page number is outside 1..n.
+func checkPage(page, n int) error {
+	if page < 1 || page > n {
+		return fmt.Errorf("pdf: page %d is outside 1..%d", page, n)
+	}
+	return nil
+}
+
+// writeTo is a helper for the pdfcpu calls that write to an io.Writer.
+func writeTo(op string, fn func(w io.Writer) error) ([]byte, error) {
+	var out bytes.Buffer
+	err := guard(op, func() error {
+		if err := fn(&out); err != nil {
+			return fmt.Errorf("pdf: %s: %w", op, err)
+		}
+		return nil
+	})
+	if err != nil {
+		return nil, err
+	}
+	return out.Bytes(), nil
+}

addons/office/internal/pdf/pdf_suite_test.go

@@ -0,0 +1,13 @@
+package pdf
+
+import (
+	"testing"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+func TestPDF(t *testing.T) {
+	RegisterFailHandler(Fail)
+	RunSpecs(t, "Office PDF Suite")
+}

addons/office/internal/pdf/pdf_test.go

@@ -0,0 +1,418 @@
+package pdf
+
+import (
+	"bytes"
+	"fmt"
+	"io"
+	"os"
+	"strings"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+	"github.com/pdfcpu/pdfcpu/pkg/api"
+	"github.com/pdfcpu/pdfcpu/pkg/pdfcpu"
+	"github.com/pdfcpu/pdfcpu/pkg/pdfcpu/model"
+)
+
+// sampleSpec has two pages and every block kind.
+func sampleSpec() Spec {
+	return Spec{
+		Title:  "Quarterly review",
+		Author: "Test",
+		Pages: []Page{
+			{Blocks: []Block{
+				Heading{Level: 1, Text: "Quarterly report"},
+				Paragraph{Text: "The team shipped the release on time and closed the open reviews."},
+				Heading{Level: 2, Text: "Highlights"},
+				Bullets{Items: []string{"First highlight", "Second highlight"}},
+				Heading{Level: 3, Text: "Figures"},
+				Table{
+					Header: []string{"Item", "Count"},
+					Rows:   [][]string{{"Revenue", "42"}, {"Costs", "7"}},
+				},
+			}},
+			{Blocks: []Block{
+				Heading{Level: 1, Text: "Appendix"},
+				Paragraph{Text: "Second page marker."},
+			}},
+		},
+	}
+}
+
+// mustCreate creates a PDF or fails the spec.
+func mustCreate(spec Spec) []byte {
+	GinkgoHelper()
+	out, err := Create(spec)
+	Expect(err).ToNot(HaveOccurred(), "Create")
+	return out
+}
+
+// onePageSpec returns a spec of n pages, each with one short paragraph.
+func onePageSpec(n int, text string) Spec {
+	s := Spec{Title: "t"}
+	for i := range n {
+		s.Pages = append(s.Pages, Page{Blocks: []Block{Paragraph{Text: fmt.Sprintf("%s %d", text, i+1)}}})
+	}
+	return s
+}
+
+// pageContent returns the decoded content stream of one page.
+func pageContent(data []byte, page int) string {
+	GinkgoHelper()
+	ctx, err := api.ReadValidateAndOptimize(bytes.NewReader(data), model.NewDefaultConfiguration())
+	Expect(err).ToNot(HaveOccurred(), "read")
+	r, err := pdfcpu.ExtractPageContent(ctx, page)
+	Expect(err).ToNot(HaveOccurred(), "content of page %d", page)
+	b, err := io.ReadAll(r)
+	Expect(err).ToNot(HaveOccurred(), "read content")
+	return string(b)
+}
+
+// rotationOf returns the /Rotate value of one page, 0 when it is not set.
+func rotationOf(data []byte, page int) int {
+	GinkgoHelper()
+	ctx, err := api.ReadValidateAndOptimize(bytes.NewReader(data), model.NewDefaultConfiguration())
+	Expect(err).ToNot(HaveOccurred(), "read")
+	d, _, _, err := ctx.PageDict(page, false)
+	Expect(err).ToNot(HaveOccurred(), "page %d dict", page)
+	if r := d.IntEntry("Rotate"); r != nil {
+		return *r
+	}
+	return 0
+}
+
+// formPDF builds a one-page PDF with one AcroForm text field per name. It is
+// written by hand so the test does not depend on a fixture file.
+func formPDF(names ...string) []byte {
+	var fields, annots string
+	for i := range names {
+		fields += fmt.Sprintf("%d 0 R ", 5+i)
+		annots += fmt.Sprintf("%d 0 R ", 5+i)
+	}
+	objs := []string{
+		fmt.Sprintf("<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [%s] /DA (/Helv 0 Tf 0 g) /DR << /Font << /Helv 4 0 R >> >> >> >>", fields),
+		"<< /Type /Pages /Kids [3 0 R] /Count 1 >>",
+		fmt.Sprintf("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 200] /Resources << /Font << /Helv 4 0 R >> >> /Annots [%s] >>", annots),
+		"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>",
+	}
+	for i, name := range names {
+		y := 150 - 30*i
+		objs = append(objs, fmt.Sprintf("<< /Type /Annot /Subtype /Widget /FT /Tx /T (%s) /Rect [20 %d 200 %d] /F 4 /DA (/Helv 12 Tf 0 g) /P 3 0 R >>", name, y, y+20))
+	}
+	var b bytes.Buffer
+	b.WriteString("%PDF-1.7\n")
+	offs := make([]int, len(objs))
+	for i, o := range objs {
+		offs[i] = b.Len()
+		fmt.Fprintf(&b, "%d 0 obj\n%s\nendobj\n", i+1, o)
+	}
+	xref := b.Len()
+	fmt.Fprintf(&b, "xref\n0 %d\n0000000000 65535 f \n", len(objs)+1)
+	for _, o := range offs {
+		fmt.Fprintf(&b, "%010d 00000 n \n", o)
+	}
+	fmt.Fprintf(&b, "trailer\n<< /Size %d /Root 1 0 R >>\nstartxref\n%d\n%%%%EOF\n", len(objs)+1, xref)
+	return b.Bytes()
+}
+
+var _ = Describe("PDF creation and editing", func() {
+	Describe("Create", func() {
+		It("creates every block kind across two pages", func() {
+			data := mustCreate(sampleSpec())
+			Expect(Check(data)).To(Succeed())
+			n, err := Pages(data)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(n).To(Equal(2))
+			first := pageContent(data, 1)
+			for _, want := range []string{"Quarterly report", "Highlights", "First highlight", "Revenue"} {
+				Expect(first).To(ContainSubstring(want), "page 1 content")
+			}
+			Expect(pageContent(data, 2)).To(ContainSubstring("Second page marker"))
+		})
+
+		It("starts a page for each spec page", func() {
+			data := mustCreate(onePageSpec(3, "short"))
+			n, err := Pages(data)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(n).To(Equal(3))
+		})
+
+		It("wraps long text across pages", func() {
+			long := strings.Repeat("lorem ipsum dolor sit amet ", 800)
+			data := mustCreate(Spec{Pages: []Page{{Blocks: []Block{Paragraph{Text: long}}}}})
+			n, err := Pages(data)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(n).To(BeNumerically(">=", 2))
+			Expect(Check(data)).To(Succeed())
+		})
+
+		It("breaks table rows across pages", func() {
+			var rows [][]string
+			for i := range 200 {
+				rows = append(rows, []string{fmt.Sprintf("row %d", i), "value"})
+			}
+			data := mustCreate(Spec{Pages: []Page{{Blocks: []Block{
+				Table{Header: []string{"Name", "Value"}, Rows: rows},
+			}}}})
+			n, err := Pages(data)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(n).To(BeNumerically(">=", 2))
+			Expect(Check(data)).To(Succeed())
+		})
+
+		It("accepts Latin-1 text", func() {
+			data := mustCreate(Spec{Pages: []Page{{Blocks: []Block{
+				Paragraph{Text: "Caffè, città, für Ön: ¡olé! Ünïcödé"},
+			}}}})
+			Expect(Check(data)).To(Succeed())
+			// The built-in font takes single bytes (cp1252), not UTF-8.
+			c := pageContent(data, 1)
+			Expect(c).To(ContainSubstring("Caff\xe8"), "content holds the single byte of è")
+			Expect(c).ToNot(ContainSubstring("Caff\xc3\xa8"), "content holds UTF-8 bytes for è")
+		})
+
+		It("writes the bullet glyph as the cp1252 bullet byte", func() {
+			data := mustCreate(Spec{Pages: []Page{{Blocks: []Block{Bullets{Items: []string{"alpha"}}}}}})
+			Expect(pageContent(data, 1)).To(ContainSubstring("(\x95)Tj"))
+		})
+
+		It("rejects control characters", func() {
+			for _, s := range []string{"bell\a", "c1 \u0085 control"} {
+				spec := Spec{Pages: []Page{{Blocks: []Block{Paragraph{Text: s}}}}}
+				_, err := Create(spec)
+				Expect(err).To(HaveOccurred(), "%q: Create succeeded, want an error", s)
+			}
+		})
+
+		It("rejects non-Latin-1 text without a font", func() {
+			spec := Spec{Pages: []Page{{Blocks: []Block{Paragraph{Text: "price 5 €"}}}}}
+			_, err := Create(spec)
+			Expect(err).To(MatchError(ContainSubstring("U+20AC")))
+		})
+
+		It("rejects an invalid font", func() {
+			spec := Spec{FontTTF: []byte("not a font"), Pages: []Page{{Blocks: []Block{Paragraph{Text: "x"}}}}}
+			_, err := Create(spec)
+			Expect(err).To(HaveOccurred())
+		})
+
+		It("refuses a font file that is not a TrueType font", func() {
+			// A file that is not a font is refused before fpdf sees it.
+			spec := Spec{FontTTF: []byte("%PDF-1.4 not a font at all"), Pages: []Page{{Blocks: []Block{Paragraph{Text: "x"}}}}}
+			_, err := Create(spec)
+			Expect(err).To(MatchError(ContainSubstring("TrueType")))
+		})
+
+		// Runs only when OFFICE_PDF_TEST_TTF names a TrueType font file. The
+		// module ships no font.
+		It("creates a document with a Unicode font", func() {
+			path := os.Getenv("OFFICE_PDF_TEST_TTF")
+			if path == "" {
+				Skip("set OFFICE_PDF_TEST_TTF to a TrueType font to run this test")
+			}
+			ttf, err := os.ReadFile(path)
+			Expect(err).ToNot(HaveOccurred(), "read font")
+			data := mustCreate(Spec{FontTTF: ttf, Pages: []Page{{Blocks: []Block{
+				Heading{Level: 1, Text: "中文 and αβγ"},
+				Paragraph{Text: "Any text is allowed with a Unicode font."},
+			}}}})
+			n, err := Pages(data)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(n).To(Equal(1))
+		})
+
+		It("validates the spec", func() {
+			big := make([]Page, MaxPages+1)
+			for i := range big {
+				big[i] = Page{}
+			}
+			manyBlocks := make([]Block, MaxBlocks+1)
+			for i := range manyBlocks {
+				manyBlocks[i] = Paragraph{Text: "x"}
+			}
+			cases := map[string]Spec{
+				"no pages":          {},
+				"heading level 4":   {Pages: []Page{{Blocks: []Block{Heading{Level: 4, Text: "x"}}}}},
+				"heading level 0":   {Pages: []Page{{Blocks: []Block{Heading{Level: 0, Text: "x"}}}}},
+				"empty heading":     {Pages: []Page{{Blocks: []Block{Heading{Level: 1, Text: " "}}}}},
+				"empty bullets":     {Pages: []Page{{Blocks: []Block{Bullets{}}}}},
+				"empty bullet item": {Pages: []Page{{Blocks: []Block{Bullets{Items: []string{"a", ""}}}}}},
+				"table no header":   {Pages: []Page{{Blocks: []Block{Table{}}}}},
+				"table ragged row":  {Pages: []Page{{Blocks: []Block{Table{Header: []string{"a", "b"}, Rows: [][]string{{"x"}}}}}}},
+				"nil block":         {Pages: []Page{{Blocks: []Block{nil}}}},
+				"too many pages":    {Pages: big},
+				"too many blocks":   {Pages: []Page{{Blocks: manyBlocks}}},
+			}
+			for name, spec := range cases {
+				_, err := Create(spec)
+				Expect(err).To(HaveOccurred(), "%s: Create succeeded, want an error", name)
+			}
+		})
+	})
+
+	Describe("Merge", func() {
+		It("counts the pages of both parts", func() {
+			a := mustCreate(onePageSpec(2, "a"))
+			b := mustCreate(onePageSpec(3, "b"))
+			m, err := Merge(a, b)
+			Expect(err).ToNot(HaveOccurred())
+			n, err := Pages(m)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(n).To(Equal(5))
+			Expect(Check(m)).To(Succeed())
+			Expect(pageContent(m, 3)).To(ContainSubstring("b 1"), "merged page 3 is the first page of the second part")
+		})
+	})
+
+	Describe("Split", func() {
+		It("returns one page per part", func() {
+			data := mustCreate(onePageSpec(3, "part"))
+			parts, err := Split(data)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(parts).To(HaveLen(3))
+			for i, p := range parts {
+				n, err := Pages(p)
+				Expect(err).ToNot(HaveOccurred(), "part %d", i+1)
+				Expect(n).To(Equal(1), "part %d", i+1)
+				Expect(Check(p)).To(Succeed(), "part %d", i+1)
+			}
+			Expect(pageContent(parts[1], 1)).To(ContainSubstring("part 2"), "part 2 holds page 2")
+		})
+	})
+
+	Describe("Rotate", func() {
+		It("changes only the selected pages", func() {
+			data := mustCreate(onePageSpec(2, "r"))
+			out, err := Rotate(data, []int{1}, 90)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(rotationOf(out, 1)).To(Equal(90))
+			Expect(rotationOf(out, 2)).To(Equal(0))
+			all, err := Rotate(data, nil, 270)
+			Expect(err).ToNot(HaveOccurred())
+			for p := 1; p <= 2; p++ {
+				Expect(rotationOf(all, p)).To(Equal(270), "page %d rotation", p)
+			}
+		})
+
+		It("rejects bad degrees and pages", func() {
+			data := mustCreate(onePageSpec(2, "r"))
+			cases := map[string]struct {
+				pages   []int
+				degrees int
+			}{
+				"degrees 45":    {nil, 45},
+				"degrees 0":     {nil, 0},
+				"degrees -90":   {nil, -90},
+				"page 0":        {[]int{0}, 90},
+				"page past end": {[]int{3}, 90},
+				"page negative": {[]int{-1}, 90},
+			}
+			for name, c := range cases {
+				_, err := Rotate(data, c.pages, c.degrees)
+				Expect(err).To(HaveOccurred(), "%s: Rotate succeeded, want an error", name)
+			}
+		})
+	})
+
+	Describe("Watermark", func() {
+		It("stamps every page", func() {
+			data := mustCreate(onePageSpec(2, "w"))
+			out, err := Watermark(data, "DRAFT")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(Check(out)).To(Succeed())
+			Expect(len(out)).To(BeNumerically(">", len(data)), "watermarked output is larger than the input")
+		})
+
+		It("rejects empty and non-Latin-1 text", func() {
+			data := mustCreate(onePageSpec(1, "w"))
+			_, err := Watermark(data, "")
+			Expect(err).To(HaveOccurred(), "empty text")
+			_, err = Watermark(data, "€")
+			Expect(err).To(HaveOccurred(), "non-Latin-1 text")
+		})
+	})
+
+	Describe("form fields", func() {
+		It("lists the fields and fills them", func() {
+			form := formPDF("customer", "notes")
+			Expect(Check(form)).To(Succeed(), "fixture Check")
+			fields, err := Fields(form)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(fields).To(HaveLen(2))
+			Expect(fields[0].Name).To(Equal("customer"))
+			Expect(fields[1].Name).To(Equal("notes"))
+
+			out, err := FillFields(form, map[string]string{"customer": "Ada Lovelace"})
+			Expect(err).ToNot(HaveOccurred())
+			Expect(Check(out)).To(Succeed(), "Check after fill")
+			filled, err := Fields(out)
+			Expect(err).ToNot(HaveOccurred())
+			for _, f := range filled {
+				if f.Name == "customer" {
+					Expect(f.Value).To(Equal("Ada Lovelace"))
+				}
+				if f.Name == "notes" {
+					Expect(f.Value).To(BeEmpty(), "notes stays untouched")
+				}
+			}
+		})
+
+		It("rejects unknown field names and empty values", func() {
+			form := formPDF("customer")
+			_, err := FillFields(form, map[string]string{"unknown": "x"})
+			Expect(err).To(HaveOccurred(), "unknown field name")
+			_, err = FillFields(form, nil)
+			Expect(err).To(HaveOccurred(), "no values")
+		})
+
+		It("returns no fields for a PDF without a form", func() {
+			fields, err := Fields(mustCreate(onePageSpec(1, "f")))
+			Expect(err).ToNot(HaveOccurred())
+			Expect(fields).To(BeEmpty())
+		})
+	})
+
+	Describe("input checks", func() {
+		It("returns an error for input that is not a PDF", func() {
+			junk := []byte("this is not a pdf file")
+			checks := map[string]func() error{
+				"Check":      func() error { return Check(junk) },
+				"Pages":      func() error { _, err := Pages(junk); return err },
+				"Split":      func() error { _, err := Split(junk); return err },
+				"Merge":      func() error { _, err := Merge(junk); return err },
+				"Rotate":     func() error { _, err := Rotate(junk, nil, 90); return err },
+				"Watermark":  func() error { _, err := Watermark(junk, "x"); return err },
+				"Fields":     func() error { _, err := Fields(junk); return err },
+				"FillFields": func() error { _, err := FillFields(junk, map[string]string{"a": "b"}); return err },
+				"empty":      func() error { return Check(nil) },
+			}
+			for name, f := range checks {
+				Expect(f()).To(HaveOccurred(), "%s: no error for bad input", name)
+			}
+		})
+
+		It("returns an error for input over the limit", func() {
+			big := make([]byte, MaxInputBytes+1)
+			Expect(Check(big)).To(HaveOccurred())
+		})
+
+		// Cuts a good PDF at many lengths and calls every operation on each cut.
+		// A truncated file may still be readable, because pdfcpu rebuilds a
+		// broken cross-reference table, so only the absence of a panic is checked.
+		It("does not panic on truncated prefixes", func() {
+			good := mustCreate(sampleSpec())
+			step := max(len(good)/150, 1)
+			for n := 0; n <= len(good); n += step {
+				p := good[:n]
+				_ = Check(p)
+				_, _ = Pages(p)
+				_, _ = Split(p)
+				_, _ = Merge(p, good)
+				_, _ = Rotate(p, []int{1}, 90)
+				_, _ = Watermark(p, "x")
+				_, _ = Fields(p)
+				_, _ = FillFields(p, map[string]string{"x": "y"})
+			}
+		})
+	})
+})

addons/office/internal/pptxedit/edit.go

@@ -0,0 +1,246 @@
+package pptxedit
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/mudler/LocalAI/addons/office/internal/ooxml"
+)
+
+// Slides lists the slides in presentation order. A slide that cannot be
+// parsed is listed with no title and no text.
+func (d *Deck) Slides() []SlideInfo {
+	refs, err := d.slideRefs()
+	if err != nil {
+		return nil
+	}
+	out := make([]SlideInfo, len(refs))
+	for i, r := range refs {
+		out[i].Index = i + 1
+		raw, err := d.get(r.part)
+		if err != nil {
+			continue
+		}
+		root, err := parseTree(raw)
+		if err != nil {
+			continue
+		}
+		if sp := titleShape(root); sp != nil {
+			var parts []string
+			for _, p := range paragraphs(sp) {
+				parts = append(parts, paraText(p))
+			}
+			out[i].Title = strings.Join(parts, "\n")
+		}
+		out[i].Text = slideText(root)
+	}
+	return out
+}
+
+// shapes returns every p:sp element of a slide in document order.
+func shapes(root *node) []*node {
+	var out []*node
+	root.walk(func(n *node) {
+		if n.is(nsP, "sp") {
+			out = append(out, n)
+		}
+	})
+	return out
+}
+
+func shapeName(sp *node) string {
+	v, _ := sp.child(nsP, "nvSpPr").child(nsP, "cNvPr").attrLocal("name")
+	return v
+}
+
+func placeholder(sp *node) (typ, idx string, ok bool) {
+	ph := sp.child(nsP, "nvSpPr").child(nsP, "nvPr").child(nsP, "ph")
+	if ph == nil {
+		return "", "", false
+	}
+	typ, _ = ph.attrLocal("type")
+	idx, _ = ph.attrLocal("idx")
+	return typ, idx, true
+}
+
+func titleShape(root *node) *node {
+	for _, sp := range shapes(root) {
+		if t, _, ok := placeholder(sp); ok && (t == "title" || t == "ctrTitle") {
+			return sp
+		}
+	}
+	return nil
+}
+
+func paragraphs(sp *node) []*node {
+	return sp.child(nsP, "txBody").children(nsA, "p")
+}
+
+// slideText lists the non-empty paragraphs the way officeread does: text in
+// slide number, date, footer, header and slide image placeholders is left out.
+func slideText(root *node) []string {
+	var out []string
+	root.walk(func(n *node) {
+		if !n.is(nsA, "p") {
+			return
+		}
+		for a := n.parent; a != nil; a = a.parent {
+			if a.is(nsP, "sp") {
+				switch t, _, _ := placeholder(a); t {
+				case "sldNum", "dt", "ftr", "hdr", "sldImg":
+					return
+				}
+				break
+			}
+		}
+		if s := paraText(n); s != "" {
+			out = append(out, s)
+		}
+	})
+	return out
+}
+
+// textRuns renders the runs of a paragraph: one run per line, an a:br between
+// lines. rPr is the verbatim run properties to reuse, or "".
+func textRuns(pfx, text, rPr string) string {
+	text = strings.ReplaceAll(strings.ReplaceAll(text, "\r\n", "\n"), "\r", "\n")
+	var b strings.Builder
+	for i, line := range strings.Split(text, "\n") {
+		if i > 0 {
+			b.WriteString("<" + qual(pfx, "br") + "/>")
+		}
+		if line == "" {
+			continue
+		}
+		b.WriteString("<" + qual(pfx, "r") + ">" + rPr + "<" + qual(pfx, "t") + ">" + ooxml.Escape(line) + "</" + qual(pfx, "t") + "></" + qual(pfx, "r") + ">")
+	}
+	return b.String()
+}
+
+// rewriteParagraph returns the replacement for paragraph p: the same open
+// tag, paragraph properties and end properties, and one run (more with line
+// breaks) that carries the run properties of the first run of p.
+func rewriteParagraph(raw []byte, p *node, text string) string {
+	open := string(raw[p.start:p.openEnd])
+	if p.self {
+		open = strings.TrimSuffix(open, "/>") + ">"
+	}
+	pfx := prefixOf(raw, p)
+	var pPr, rPr, endPr string
+	seenRun := false
+	for _, k := range p.kids {
+		switch {
+		case k.is(nsA, "pPr") && pPr == "":
+			pPr = string(raw[k.start:k.end])
+		case k.is(nsA, "r") && !seenRun:
+			seenRun = true
+			if rp := k.child(nsA, "rPr"); rp != nil {
+				rPr = string(raw[rp.start:rp.end])
+			}
+		case k.is(nsA, "endParaRPr") && endPr == "":
+			endPr = string(raw[k.start:k.end])
+		}
+	}
+	return open + pPr + textRuns(pfx, text, rPr) + endPr + "</" + tagName(raw, p) + ">"
+}
+
+// newParagraph builds a paragraph from scratch with the a: prefix. With decl
+// the element declares the prefix itself, for use inside a part that may not.
+func newParagraph(text string, decl bool) string {
+	open := "<a:p>"
+	if decl {
+		open = `<a:p xmlns:a="` + nsA + `">`
+	}
+	runs := textRuns("a", text, `<a:rPr lang="en-US"/>`)
+	if runs == "" {
+		return open + `<a:endParaRPr lang="en-US"/></a:p>`
+	}
+	return open + runs + `</a:p>`
+}
+
+// SetTitle replaces the text of the title placeholder (type title or
+// ctrTitle) of a slide. The formatting of the first run is kept and any
+// further paragraphs of the title are removed.
+func (d *Deck) SetTitle(slide int, text string) error {
+	return d.tx(func(d *Deck) error {
+		ref, err := d.slide(slide)
+		if err != nil {
+			return err
+		}
+		raw, err := d.get(ref.part)
+		if err != nil {
+			return err
+		}
+		root, err := parseTree(raw)
+		if err != nil {
+			return fmt.Errorf("pptxedit: %s: %w", ref.part, err)
+		}
+		sp := titleShape(root)
+		if sp == nil {
+			return fmt.Errorf("pptxedit: slide %d has no title placeholder", slide)
+		}
+		var edits []edit
+		tb := sp.child(nsP, "txBody")
+		switch ps := paragraphs(sp); {
+		case tb == nil:
+			pfx := prefixOf(raw, sp)
+			edits = append(edits, edit{sp.closeStart, sp.closeStart,
+				"<" + qual(pfx, "txBody") + ` xmlns:a="` + nsA + `"><a:bodyPr/><a:lstStyle/>` + newParagraph(text, false) + "</" + qual(pfx, "txBody") + ">"})
+		case len(ps) == 0:
+			edits = append(edits, edit{tb.closeStart, tb.closeStart, newParagraph(text, true)})
+		default:
+			edits = append(edits, edit{ps[0].start, ps[0].end, rewriteParagraph(raw, ps[0], text)})
+			for _, p := range ps[1:] {
+				edits = append(edits, edit{p.start, p.end, ""})
+			}
+		}
+		out, err := splice(raw, edits)
+		if err != nil {
+			return fmt.Errorf("pptxedit: %w", err)
+		}
+		d.set(ref.part, out)
+		return nil
+	})
+}
+
+// SetParagraph replaces the text of one paragraph of the first shape with
+// this name on a slide. The paragraph number counts every a:p of the shape,
+// from 0. The properties of the paragraph and of its first run are kept and
+// the other runs are dropped. A newline in text becomes a line break.
+func (d *Deck) SetParagraph(slide int, shape string, paragraph int, text string) error {
+	return d.tx(func(d *Deck) error {
+		ref, err := d.slide(slide)
+		if err != nil {
+			return err
+		}
+		raw, err := d.get(ref.part)
+		if err != nil {
+			return err
+		}
+		root, err := parseTree(raw)
+		if err != nil {
+			return fmt.Errorf("pptxedit: %s: %w", ref.part, err)
+		}
+		var sp *node
+		for _, c := range shapes(root) {
+			if shapeName(c) == shape {
+				sp = c
+				break
+			}
+		}
+		if sp == nil {
+			return fmt.Errorf("pptxedit: slide %d has no shape named %q", slide, shape)
+		}
+		ps := paragraphs(sp)
+		if paragraph < 0 || paragraph >= len(ps) {
+			return fmt.Errorf("pptxedit: shape %q has %d paragraphs, no paragraph %d", shape, len(ps), paragraph)
+		}
+		p := ps[paragraph]
+		out, err := splice(raw, []edit{{p.start, p.end, rewriteParagraph(raw, p, text)}})
+		if err != nil {
+			return fmt.Errorf("pptxedit: %w", err)
+		}
+		d.set(ref.part, out)
+		return nil
+	})
+}