mirror/LocalAI
1
0 Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-09-13 06:45:26 -04:00
Code Issues Packages Projects Releases Wiki Activity
7,533 Commits 191 Branches 177 Tags
8b01ac2d4e8880099b17013eaed88a4773d20981
Commit Graph
61 Commits
Author SHA1 Message Date
mudler's LocalAI [bot]andEttore Di Giacinto 45cb3983ee fix(ui): unmerge the class strings that left buttons in browser chrome (#11462) 
Eight header controls across seven pages had two or three elements' classes
collapsed into one string. The wrapper or the icon ended up wearing the
button classes, and the buttons themselves were left with no class at all,
so they rendered in the browser's own chrome. Reported on Agent Jobs; the
grep found the rest.

`fas` does not draw anything by itself: it sets
`font-family: "Font Awesome 6 Free"` and weight 900 on whatever carries it,
and the `fa-*` class supplies the glyph via ::before. So

    <button className="btn btn-primary fas fa-plus">

renders its own label "New Task" in the icon font, and

    <div className="hstack btn btn-primary btn-sm fas fa-edit btn-secondary fa-arrow-left">
      <button>Edit</button>
      <button>Back</button>
    </div>

styles the flex wrapper as a button that is both primary and secondary,
points two glyphs at one ::before, and leaves both real buttons bare.

Fixed, all of them keeping the correct `<i>` child they already had:

  - AgentJobs, AgentTaskDetails (x2), AgentCreate - icon classes off the
    button.
  - AgentTaskDetails, AgentJobDetails - wrapper back to plain `hstack`, and
    the two buttons inside each get the variants the wrapper had been
    holding. Back is secondary and leads, Edit/Cancel is the emphatic one
    and trails, matching every other detail header.
  - VoiceLibrary, VoiceProfileCreate - the title `<i>` had swallowed the
    action link's classes, so "Create voice" and "Back to library" were
    unstyled anchors. Back was also drawing a "+" because it had inherited
    fa-plus while its own fa-arrow-left sat up in the title.
  - P2P - a stray fa-circle-info on the title icon.

The ninth instance was ImportModel, where this class of bug was first
found. #11461 rewrote that file and landed first, so nothing is left to fix
there.

Guarded by e2e/class-hygiene.spec.js, which reads the source rather than
walking routes: several of these pages need agent or voice data before they
render a header, so a route walk would skip exactly the pages that had the
bug. It fails on an icon-font class outside an `<i>`/`<span>`, on two glyphs
or two button variants on one element, and on a layout wrapper that is also
a button. Font Awesome modifiers (fa-spin, fa-fw, sizes) are excluded, so
the `fa-spinner fa-spin` idiom stays legal.

e2e: 428 passed.


Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] [Playwright]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-08-11 12:07:42 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 7a7fb00730 feat(ui): rebuild the import form on the restyled design language (#11461) 
The import page took the new palette in #11305 but kept its old layout, so
it stayed a 760px column with the primary action detached from the form it
submits. Two of the problems were outright bugs.

The Import button carried no className at all, so the page's single most
important control fell through to the user-agent button: system chrome,
wrong radius, no design-system focus ring. The YAML button carried
`fas fa-save fa-upload`, which sets Font Awesome as the button's own font
family (its label text inherits it) and points two glyph classes at one
::before.

On the layout: `page--narrow` is documented for "forms / single-record edit
views", and in Advanced mode this page held a URI field, a six-section
format guide, ten modality chips, nine preference fields, a key-value
repeater and a YAML editor at `calc(100vh - 400px)`. The width was the
symptom; one column was the disease.

  - `page--medium` with a work column and a format reference beside it.
    The reference answers the only question a first-time admin has and used
    to sit behind a chevron, closed by default. Below 1024px it becomes a
    disclosure rather than disappearing.
  - The source field is the hero: monospace, because it holds something you
    paste, and it carries its own Import button. That removes the hidden
    aria-hidden submit button that existed only because the real action sat
    outside the form.
  - Simple and Advanced are gone. They were ~80% the same surface, and the
    overlap cost a mode switch, a localStorage key and a three-button
    Keep/Discard/Cancel dialog whose only job was protecting state that
    switching modes would hide. One form with a collapsible options panel
    hides nothing, so none of it is needed. What genuinely differs is the
    kind of input, which is now the two tabs: a source, or YAML.
  - The size/VRAM estimate reports under the field that produced it instead
    of as a banner above the page header, and an import in flight gets the
    progress, phase and byte counts the poller already returned and the old
    status card threw away.
  - ModalityChips resolves its labels through the same `modality.*` keys as
    the dropdown it filters. It hardcoded English shorthand, so one modality
    carried two names on one screen ("Speech" on the chip, "Speech
    recognition" on the group it scrolled to) and seven locales had neither.
    Its inline styles and its pill radius move onto the design system.
  - Three inline styles go, including both conditional-padding hacks; the
    only one left is the progress bar's runtime width. Baseline 538 -> 535.

Docs updated in the same change: the WebUI section described a Simple and an
Advanced mode and told the reader to "Toggle to Advanced Mode".

e2e: 426 passed. The mode-switch suite is replaced by one covering the tabs
and the disclosure, and a new layout suite pins the width, the styled
primary action, the absence of an icon-font button, the reference column at
both widths, and the estimate's position.


Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] [Playwright]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-08-11 12:00:59 +02:00
Adiraandlocalai-org-maint-bot ab52813342 feat(modelartifacts): support bounded parallel Hugging Face file downloads (#11162) 
* feat(modelartifacts): support bounded parallel Hugging Face file downloads

Closes #11114.

Snapshot materialization fetched every file through the sequential
executor in DownloadFilesWithContext, so a repository split into many
shards spent most of its wall clock in per-file request latency rather
than moving bytes.

Add DownloadFilesWithConcurrency, an errgroup with SetLimit, and keep
DownloadFilesWithContext as a wrapper that passes a limit of 1. That
leaves the two non-artifact callers (core/gallery and the model config
loader) on exactly the path they had: tasks still run in slice order,
and the first failure still returns before any later task starts.

Only whole files run in parallel. A single file is never split, so the
.partial resume machinery and the per-file SHA check in
downloadTaskWithRetry are untouched.

Two details the parallel path forced:

- completedBytes becomes an atomic.Int64. Several AfterDownload hooks
  add to it while other files' progress callbacks read it; without this
  the race detector reports three races on the new specs.
- The caller's status callback is serialized. The sequential path gave
  it an implicit guarantee of never being entered twice at once, and it
  belongs to the caller, so the executor keeps that promise rather than
  pushing locking onto every caller. AfterDownload is deliberately not
  serialized -- it does the verify-and-promote work that parallelism
  exists to overlap.

Manifest order needed no work: each hook already writes its own
manifest.Files slot by snapshot index, so entries stay in snapshot
order whatever the completion order. A spec now pins that.

The default is 1, unchanged behaviour. A shared models volume is often
the bottleneck rather than the link, so raising it is a deployment
decision; --artifact-download-concurrency and
LOCALAI_ARTIFACT_DOWNLOAD_CONCURRENCY expose it on both `run` and
`models install`.

Not done here, per the issue: no chunk-level parallelism within a single
file, and no throughput measurements across concurrency 1/2/4/8 -- that
needs a representative sharded repo and a real link.

Assisted-by: Claude:claude-opus-5 go-test gofmt
Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com>

* feat(modelartifacts): expose download concurrency in settings

Follow-up to review feedback on #11162:

- The CLI flag and docs no longer describe the limit as Hugging Face
  specific. It applies to any artifact source, as @mudler pointed out.
- artifact_download_concurrency is now a persisted runtime setting and
  is editable from the WebUI, so it can be changed without a restart.

The manager's limit becomes an atomic.Int64 behind
SetDownloadConcurrency, because a live runtime setting can be updated
while a materialization is already in flight. Injected materializers
stay compatible through an optional setter interface, so a manager that
does not implement it is simply left alone.

Verified before taking this on: go build, go vet and go test -race all
pass for pkg/modelartifacts, pkg/downloader and core/config. The React
UI builds with vite, artifact_download_concurrency is present in the
built Settings chunk, and eslint reports the same 8 pre-existing
warnings on Settings.jsx as it does without the change.

Implementation contributed by localai-org-maint-bot on the review
thread; reviewed, verified and signed off by me.

Assisted-by: Codex:gpt-5
Assisted-by: Claude:claude-opus-5 go-test vite eslint
Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com>

---------

Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com>
Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
 2026-08-07 18:00:45 +02:00
Richard Palethorpe 5ac445e1d4 fix(react-ui): restore 3D Studio results and history (#11393) 
* fix(react-ui): restore 3D Studio results and history

Keep large conditioning-image payloads out of the rendered request panel so the generated viewer can mount reliably. Accept clipboard images and synchronize 3D history consumers so new results appear in Studio without a reload.

Cover clipboard input, bounded request rendering, result display, and cross-view history synchronization with Playwright.

Assisted-by: Codex:gpt-5 Playwright

* perf(react-ui): idle the 3D viewport when still

Limit auto-rotate rendering to 30 FPS and stop scheduling frames when rotation is disabled. Resize, view controls, and pointer input invalidate the still frame on demand.

Assisted-by: Codex:gpt-5 Playwright
 2026-08-06 17:34:46 +02:00
Nandana Dileep 147a5ee783 fix(react-ui): stop traces page crash when switching trace tabs (#11387) 
Switching from Backend Traces back to API Traces crashed the page
with "can't access property status, e.response is undefined" (#11376).
The API table briefly renders the previous tab's backend rows while the
refetch effect is still pending, and those rows carry no `response`
envelope. The status column dereferenced it unguarded. Render a neutral
placeholder instead of throwing, and cover the tab-switch scenario with
a regression spec.

Assisted-by: opencode:big-pickle

Signed-off-by: Nandana Dileep <110280757+nandanadileep@users.noreply.github.com>
 2026-08-06 09:06:08 +02:00
localai-org-maint-botandlocalai-org-maint-bot 9f62401fca feat(traces): show in-flight API requests (#11368) 
Register JSON API exchanges before their handlers run so the traces dashboard can surface active work. Replace the live entry with the completed persisted record under the same ID, and clean it up if a handler panics.

Assisted-by: Codex:gpt-5

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
 2026-08-05 09:37:05 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 1741df0bf1 fix(ui): scale the chrome audit's timeout to the number of routes it walks (#11319) 
chrome-audit.spec.js walks 25 routes in a single test, and has been the
UI E2E suite's failure on 5 of the last 6 master runs. It always dies the
same way, at the 30s per-test default:

    Test timeout of 30000ms exceeded.
    Error: page.waitForTimeout: Test timeout of 30000ms exceeded.
      19 |     await page.goto(route)
    > 20 |     await page.waitForTimeout(400)

The spec is new in 5cb0c1a8; the commit before it was green, and every
run since has been red on this file.

The failure is cumulative rather than one bad route. Across those runs
the clock runs out at line 19, 20 or 21 depending on where the loop
happens to be, and the timeout lands on waitForTimeout rather than on
goto, which is what running out of budget looks like as opposed to a
navigation that hangs. 30s over 25 routes is ~1.2s each, including a
deliberate 400ms settle, so there is very little headroom to begin with.

Give the test a budget proportional to its work: six seconds a route.
That absorbs a slow runner and still fails promptly if a route genuinely
hangs.

Verified: the spec passes on the current UI in 12.2s solo, and the full
suite passes 418 at 8 workers locally. What I could NOT do is reproduce
the CI timeout on this machine, which has 20 cores against the runner's
2 to 4; under synthetic CPU load it still finished in 13.5s. So the fix
is argued from the CI signature and the arithmetic, not from a local
repro, and the proof is this spec going green on the hosted runner.

Note test.setTimeout() has to be called inside the test body. At module
scope Playwright rejects it with "test.setTimeout() can only be called
from a test".


Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-08-03 19:01:06 +02:00
localai-org-maint-botandlocalai-org-maint-bot fd4ec083b9 feat(downloads): add resume-safe pause action (#11222) 
Give gallery operations distinct pause and cancel paths. Pause preserves partial download data so reinstalling the same model or backend resumes through HTTP Range, while cancel keeps its destructive semantics. Surface the action in the Activity UI and document the API behavior.

Assisted-by: Codex:gpt-5

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
 2026-08-03 15:25:23 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 5cb0c1a872 feat(ui): close the gap between the shipped UI and the design mocks (#11307) 
* feat(ui): give the Operate overview real numbers and traces a latency shape

First two items from a component-by-component comparison against the mocks.
The pattern that audit found: everything newly built matched, everything
pre-existing got the palette but not the layout, and an "absent rather than
empty" rule hid most of the overview exactly when someone was looking at an
idle installation.

**The headline grid is always rendered**, including at zero, with a fourth cell
for host memory. Hiding it removed the page's structure precisely when it was
most likely to be read, and "0 failed" is information — an absent panel is not.
The quiet case is now said in a line underneath instead of by showing nothing.

**The sections state counts** rather than listing their destinations: backends,
models, updates and running operations instead of the words "Usage and traces".
That needed installed backend and model counts in the summary context, which
are two more cheap reads on the poll that was already running.

**Traces rows carry latency as a bar as well as a figure**, scaled against the
slowest request currently in view and turning amber past two seconds. The table
had no latency column at all — the number was buried in the expanded detail, so
the shape of the tail was invisible while scanning. Scaling against the view
rather than an absolute ceiling is deliberate: what matters when reading a page
of traces is which of these are the outliers, and an absolute scale flattens
every row on a fast installation into nothing.

Full e2e suite: 409 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): name the engine on Home's resident models, and add jump-back-in

Third item from the mock comparison.

The mock showed each resident model with the engine serving it. /system carried
only the id, so the audit recorded this as blocked on a server field — but the
config loader is already in scope where that response is built, so it is one
lookup. SysInfoModel gains an optional `backend`, resolved from the model's
config and omitted rather than guessed when there is none (a loose file, or a
config since removed). Home renders the column blank in that case; the test
pins both halves of that.

Memory per model stays out. It is not one lookup — it would mean asking each
backend process — and inventing a number beside a real one is worse than
leaving the column off.

"Jump back in" is the block the mock had and Home did not. The quick-links row
above it is a set of first-run actions; these are the three places someone
returns to, each stated with what it currently holds rather than as a bare
label.

Go: routes suite passes. Full e2e suite: 412 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): rank the recommended models as lanes instead of equal cards

The hardware recommendations were a grid of equally-weighted cards. The list is
already sorted by fit, and a grid throws that order away: three cards side by
side say "pick one", when the page has actually formed an opinion about which
one.

They are lanes now, read top to bottom in fit order, with the leader carrying
the single amber "Best fit" label and the rest marked "Also fits". One opinion
per page — the alternatives are alternatives, not runners-up each worth their
own colour, which is how a strip of coloured badges ends up meaning nothing.

Below 720px the size and VRAM columns drop and the lane keeps the name and the
install action, which are the two things a narrow screen needs.

The existing panel spec moves off .rec-models-item onto .lane rather than being
deleted; dismissal, collapse, keyboard operation and install all still pass
unchanged, and there is a new assertion that exactly one row is called out.

Full e2e suite: 413 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* fix(ui): drop capsule chips app-wide, and un-break the empty voice library

**Pills are gone.** A capsule radius reads as a tag floating on the surface,
which fights a system whose structure is hairlines and square corners — and
with chips on Discover, Host, Activity and the biometrics pages, "some pages
have pills" was the real inconsistency rather than any one page.

Sixteen selectors move to the small radius: filter buttons, tab pills, activity
and biometrics chips, file and count badges, the jump-to-latest control, the
nav badge. Round *buttons* keep their circle — .lightbox__nav and
.home-send-btn are circles, not capsules — as do every progress track, status
dot and avatar, which are round because they are round, not because they are
tags.

**The empty voice library was unusable.** `.voice-library-empty` sets
min-height: 430px, border: 0 and background: transparent — a description of the
empty PANEL — and it had been attached to the action instead. The create button
was therefore a 430px transparent box that pushed itself out of the panel and
could not be seen. Moved onto the container it describes, which now centres its
action rather than letting it fall off the bottom. Same class-mangling shape as
the Agents header fixed earlier.

Full e2e suite: 416 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): put Host's headline figures on the shared hairline strip

Host had shadowed, clickable StatCards above a page that already has a rail, a
pane and a tab bar — a second dashboard language on one screen, and a different
one again from the figures inside its own detail pane.

The Operate overview's figure grid is generalised into a shared `.stat-strip`
and Host adopts it, so the two pages read as one system: same cell, same figure
scale, same tone vocabulary, and the same hairline grid the split-view StatGrid
already uses. The cells stay clickable and still route into the tab and filter
they describe, because a count is worth more when it is also the way to the
thing counted.

Tone is spent only where the number means something — running and updates when
non-zero — since a strip where every cell is coloured has no emphasis left.

Two bugs made on the way, both now covered:

- The first version put `<button>` elements inside a `<dl>` with `<dt>`/`<dd>`
  inside the buttons. Neither is valid, the browser re-parents both, and the
  cells collapsed. These cells are a set of controls, so a plain container of
  buttons is also the honest markup.
- Even correct, the strip rendered 2px tall: `.page--app` is a flex column
  whose split view takes flex:1, so a child with no intrinsic minimum is shrunk
  away. The old cards survived only because `.stat-card` carried
  min-height:96px. The strip now declines to shrink, with a test pinning it.

The stat-card specs are retargeted rather than deleted: they were written to
guard a class collision on a page that no longer uses cards, so they now guard
the strip's labels and its height.

Full e2e suite: 417 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): make Backends notices an edge rather than a filled card

The install and upgrade banners were tinted cards with a full border. A filled
panel makes every notice shout at the weight of an error, which is how notices
stop being read — and Backends shows one on most visits, so it was shouting
routinely.

They are now a hairline with a coloured left edge, the same treatment the
Operate overview gives rows that want a decision, so "this needs you" looks the
same wherever it appears. Counts in the notice take the monospace tabular
figures the rest of the console uses.

Also drops the last inline style on the page, and refreshes the inline-style
baseline, which has read 624 against a real count since #11288 landed. The gate
exits 0 either way, so nothing was failing — but a baseline 86 above the truth
would have let that many inline styles back in unnoticed. Now at 538, which
tightens the ratchet rather than loosening it.

The spec creates the upgrade it asserts on rather than skipping when the mock
has no notice: a test that skips is a test that proves nothing.

Full e2e suite: 418 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): finish the mock parity list, and stop hiding the recommendations

The last two items from the audit, plus a correction.

**Discover's use-case shelf is lanes.** These are a list of ways in, read in
order; a grid of equal cards asks the reader to compare them, which is not the
choice on offer.

**The request panel reaches every generator.** Video, 3D, Sound and Audio FX
join Images and Speech, so each one teaches its own endpoint rather than two of
six doing it. Audio FX records the fields that shape the request rather than
the bytes, since its payload is multipart.

**Recommendations no longer collapse themselves.** They were folded away by
default once anything was installed. That is the page's one opinion about this
host, and an opinion hidden by default is one the reader never gets. Someone
who disagrees can still collapse it and that choice is remembered — the
difference is that we no longer make it for them. Three specs asserted the old
default and now assert the new one.

The use-case heading also sat a line's width from the text it introduces, so
the two read as one paragraph. It has air under it now, and the shelf is
separated from the recommendations above it.

Two tests removed rather than kept: a generator loop whose only real assertion
was `expect(endpoint.length).toBeGreaterThan(0)`, and an earlier card-gap guard
that could only skip. A test that cannot fail is worse than no test, because it
reads as coverage.

Full e2e suite: 418 passed, 4 skipped. Inline styles at baseline.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* fix(ui): make the Host figures legible and give the strip its spacing back

Three defects introduced by the Host redesign, all found by looking at the
running app rather than by the suite.

**The figures were invisible.** "Running now" and "Updates available" rendered
pure black on the dark ground. Two causes compounding: the `--muted` tone alias
never landed, because the source rule has extra spaces before its brace and the
exact-match edit missed it silently; and a `<button>` does not inherit colour,
so with no tone rule the value fell back to the user agent's `buttontext`.
Both fixed, and a test now fails on any figure computing to pure black.

**The strip sat flush against the resources panel.** `.stat-strip` declares
`margin: 0 0 ...` and is declared later in the file than `.manage-summary`, so
the shorthand quietly won and the top margin became zero. Raised to
`.stat-strip.manage-summary` so it beats the shorthand on specificity rather
than on declaration order, which is the kind of thing that breaks again the
next time a rule moves.

**Discover's use-case heading had a doubled gap.** `.zero-pane` is a flex
column that already separates its children; adding a margin on top of the gap
stacked the two. The margin is gone and the heading keeps only its own breathing
room.

Full e2e suite: 420 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): make Studio's tabs path segments rather than a query parameter

`/app/studio?tab=images` reads like a filter applied to a page. It is
navigation: a different generator, with its own state and its own deep link. It
is now `/app/studio/images`, with the overview at `/app/studio`.

Legacy `?tab=` links are redirected once to the path form, replacing the
history entry so Back does not bounce between two spellings of the same place.
Bookmarks and older links keep working and land on the canonical URL rather
than a second version of it, which is the part worth having a test for.

The nine `?tab=` references were all in specs, none in docs, so the migration
is contained. They move to paths, and a new spec pins the redirect.

Full e2e suite: 421 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): make the hardware recommendation a section, not a dismissable card

It was a bordered card with a collapse control and a close button, sitting
inside a pane that is otherwise hairline sections. Two problems: it read as
something bolted onto the page rather than part of it, and treating it as an
interruption to be shut is the wrong frame for the one thing the page has to
say about the machine it is running on.

It is now a plain section with the same heading treatment as the shelves below
it. The collapse state, the dismissal, their storage keys and the legacy key
read for backwards compatibility all go with it, along with the installedCount
prop that existed only to pick a default collapse.

Five specs described behaviour that no longer exists and are removed rather
than adjusted — collapsing, dismissing, persistence of both, and the toggle's
keyboard handling. One new spec asserts the replacement contract: no control
with aria-expanded, no dismiss, and no card border.

Full e2e suite: 416 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* fix(ui): restore every stripped icon and every default-chrome control

You reported two broken icons. They were not two: an earlier automated edit had
stripped the `fa-*` class from twenty `<i>` elements across eleven pages, and an
`<i>` with no icon class renders nothing at all. Settings' save button, the
voice-profile back link, and eighteen others — agent row actions, task and job
buttons, import and create actions — were all drawing empty space.

Each is restored from its own context rather than a blanket icon: the agent row
gets pause/play, pen, comments, file-export and trash; the fine-tune toggle
swaps plus for xmark as it opens; the P2P documentation link gets the
external-link glyph.

The same edit left controls without their classes. Fine-tune's "Import config"
was rendering in the browser's own chrome, and `.p2p-cmd__copy` set a border
but no background, so it fell back to `buttonface` — a pale grey chip on a dark
command block. FineTune's "New job" also had its icon classes folded into the
button's className, the same mangling already fixed on the Agents header.

Rather than fix the reported two and wait for the next report, this adds a
standing audit: twenty-five routes are walked and the test fails on any visible
control rendering with user-agent chrome, or any `<i>` without an `fa-*` class.
It found the three remaining cases after the first sweep, and it is the reason
the next one cannot ship quietly.

Full e2e suite: 417 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-08-03 12:16:09 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 58ea2f5d79 feat(ui): give Operate and Studio a front door, and fix two layout regressions (#11305) 
* feat(ui): give Operate a front door and fold six rail groups into four

Opening Operate ran firstVisiblePath() and landed on Backends, because
Backends happens to be written first in operateConsole.groups. The section
that should answer "is anything wrong" opened on a package manager, and
nothing was reported until you visited it.

Adds /app/operate. Its one irreplaceable block is "Needs attention", which
is empty when nothing is wrong and says so in a line rather than rendering a
reassuring green panel. It collects stale backends, failed operations and
unhealthy nodes. Everything else on the page is a summary you could already
assemble by visiting four others.

The rail regroups from six headings to four: Inference and Activity were both
"the runtime right now", Access and System were both administration. No
destination is removed and no gate changes, so isConsoleItemVisible and
consolePaths are untouched. Overview leads the first group, which is what
makes firstVisiblePath() return it without knowing it exists.

Rail entries now carry a signal beside the label. This does not replace the
sidebar badge and is not built as if it does: the badge stays on the
always-visible sidebar entry for the reason recorded in Sidebar.jsx, that the
rail exists only on Operate routes and can be collapsed. The signals are
orientation while inside Operate, so they are aria-hidden and nothing urgent
depends on them alone.

OperateSummaryContext polls once for the whole console, following
OperationsContext, which exists because per-consumer setInterval against one
endpoint was the defect it fixed. It is mounted by ConsoleLayout for the
Operate console only, so "poll only while in Operate" needs no route check.
Built on usePolling, so it pauses on a hidden tab. Operations are read from
OperationsContext rather than polled a second time, and each source degrades
to no-signal on its own so one dead endpoint cannot blank the rest. It reads
the cached GET /api/backends/upgrades and never the POST that forces a real
registry check.

Traces and Usage get no signal yet: /api/traces returns the list, so a count
would mean fetching every trace to render one number. A counts endpoint is
the honest fix and is scoped separately.

Full e2e suite green (369 passed, 4 skipped), including a render-smoke entry
for the new route.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): open Studio on what this machine can actually make

Studio was a tab strip over six generators that opened on Images, which was
never a decision, only the first entry in BASE_TABS. Nothing said which
modalities this installation could run, so the way to learn that video had no
model was to pick the tab and find an empty select.

Adds an overview tab and makes it the fallback. Explicit tabs still win, so
existing deep links keep working; anything unrecognised or gated now lands on
the overview rather than Images.

Each tab carries a dot: filled when an installed model advertises that
modality, hollow when nothing serves it. That is the feature in one detail,
turning the strip from navigation into a report of what the machine can do
before anything is clicked. The dot is aria-hidden because the overview states
the same facts in words and the dots change as models load.

Two kinds of unavailable, which had to stop looking alike:
  - switched off, via a permission: no tab and no lane, unchanged
  - available with no model: a lane, and a route to installing one

Studio now owns one MODALITIES table so the tab strip and the overview cannot
disagree about what exists, and calls useModels() once, unfiltered, grouping in
the browser. useModels(capability) fetches the whole list and filters locally,
so a hook per modality would have been six identical requests to
/api/models/capabilities on every mount. There is a test for that.

Recent outputs read every localStorage store through a new
readAllMediaHistory(), which avoids mounting five hooks that carry save timers
the overview has no use for. 3D is read separately through use3DHistory rather
than folded in: its entries are GLB blobs in IndexedDB, so they cannot come
from the same synchronous read.

Typical cost is the median of this machine's own history, not a guess, and
renders as a dash when there is nothing to go on.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* fix(ui): stop the stat cards and the console rail breaking on small screens

Two unrelated causes behind one report that /app/manage looks wrong when the
window is narrow.

The stat cards were being laid out by the wrong rule at every width. Two
different components both claimed `.stat-grid`: the dashboard card strip that
holds .stat-card children, and the detail-pane StatGrid the split views
introduced further down App.css. Being later, the second won every shared
property, so the cards got its 120px columns and its 1px hairline gap in place
of their own 180px columns and spacing-md. Four cards were packed onto a row
that fits two, labels wrapped to three lines and clipped, and the icon crowded
the value. Renamed the strip to `.stat-cards`, after the children it actually
holds, which also removes the mismatch of a `.stat-grid` container full of
`.stat-card`s. The split-view component keeps `.stat-grid` and its BEM parts.

The expanded console rail had no bounded height. Thirteen destinations stacked
in one column is taller than a phone, so opening the menu pushed the page's own
heading past the fold: the menu replaced the page rather than annotating it.
Capped at 55vh with internal scrolling below 768px, so the content behind stays
reachable.

Both are asserted on behaviour rather than markup: no stat-card label may be
clipped, the card gap must not be the detail pane's hairline, and expanding the
rail must leave the page heading on screen.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): retemper the palette to localai.io and add the lane primitive

The token half of the style transfer, plus the shared list idiom the two
overviews had each grown their own copy of.

theme.css moves from Nord to the website's palette, variable names preserved so
every consumer moves with it: ground #13171f -> #0d1117, accent frost cyan
#88c0d0 -> action blue #4f8cff, success sage -> mint #56d6a4, warning -> the
amber #f1b95d the site spends only on the thing asking for a decision. Eyebrows
go mint. Dividers become an opaque #29384a hairline rather than alpha over a
varying surface, which is what makes stacked surfaces read crisply on the site.

Light is derived, not inverted. The site ships one theme and never had to
answer this, but the app does: blue darkens to #2f62d8, mint to #0d8b60 and
amber to #8a5d0b, all clearing 4.5:1 on a cool paper ground, where the
dark-mode values sit near 2:1. Same three roles, different values.

Three files restate the palette because CSS variables cannot reach them:
cmTheme.js (the whole CodeMirror theme), VoiceVisualizer and WaveformPlayer
(canvas). Left alone they would have quietly kept the app half-Nord.

The `.lane` primitive replaces the near-identical row CSS that OperateOverview
and StudioOverview had each written: a full-bleed row on a hairline that insets
on hover, with no card and no shadow. Callers supply only the column template.
Both pages now use it, along with `.lane-head` for section rhythm and a
`.page-pad` container for top-level pages outside a console shell — without
which Studio sat flush against the sidebar with its eyebrow clipped.

Studio's tab strip wraps rather than running off the edge at narrow widths.

Full e2e suite: 386 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): put Home's resident models on lanes and give the footer one line

Home's status line was three chips saying a thing was true. It now reports
figures: how many models are resident, how many nodes are healthy, what share
of memory is in use, set in tabular monospace so the digits line up. A chip
answers whether; a figure answers how much, which is what someone opening the
page at a glance is after.

Resident models move from status chips to lanes, with the id set in a new
`.lane__name--id` because an id is something you might type or paste and the UI
face makes it read as a label. /api/system-information carries only the id, so
there is deliberately no backend or memory column: inventing one would mean a
server change this does not make.

The footer was three centred rows and cost the bottom sixth of every page for
chrome. It is one line now, version left and links right, wrapping to centred
when the viewport is too narrow to hold both. Every link it had, it keeps.

Full e2e suite: 392 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* fix(ui): correct three contrast failures and stop a guaranteed-404 poll

A contrast audit of the new palette found three values below WCAG AA, one of
which the previous commit message claimed was fine:

- White on the #4f8cff button is 3.22:1, which is large-text only. The website
  does exactly this, but a button label in an app is not large text, so the
  label goes to dark ink at 5.88:1. Light mode keeps white, which is 5.44:1 on
  its darker blue.
- Light-mode success was 4.08:1 on paper, not the 4.5 claimed. Darkened to
  #0a734f, 5.56:1.
- Nord red was already 4.28:1 on raised surfaces, a pre-existing miss carried
  over unexamined. Lifted to #c96f78, 5.02:1.

Lanes gain the two states they were missing: a 44px target on coarse pointers,
matching what EntityRail already does so the two list idioms feel the same
under a thumb, and a reduced-motion variant that keeps the background feedback
while dropping the hover inset, which is a position change.

The Operate summary no longer asks for /api/nodes on a single-node install. The
cluster API answers 503 when distributed mode is off, so it was a guaranteed
miss every fifteen seconds; it is now gated on useDistributedMode, the same
condition the rail already uses for the Nodes entry.

Full e2e suite: 392 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* fix(ui): restore the gap between overview blocks, and stop claiming zero nodes

Two defects a design review surfaced.

`.lane-head:first-child { margin-top: 0 }` was meant to stop the first block on
a page carrying a top margin. But every <section> makes its lane-head a first
child, so the reset applied to all of them and the gap between blocks vanished:
"Sections" sat flush against the attention row above it. The header supplies its
own bottom margin, so a uniform top margin is correct everywhere.

The Cluster summary read "0 nodes" on a single-node install, which looks like a
fault when the cluster API is simply switched off. It now says "Single node".

Full e2e suite: 392 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): open dark by default, and stop clipping the collapsed sidebar footer

Dark is the identity rather than a preference: localai.io ships one theme and
it is this one, so an install should look like LocalAI before anyone has chosen
anything. The OS setting no longer selects light on first load. The toggle
still does, and a stored choice wins forever after, which the tests assert
both ways.

The collapsed sidebar footer stacked its controls but kept the expanded row's
inline padding, so their edges were clipped against the 51px rail.

Full e2e suite: 394 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(api): count traces server-side and give the Operate overview real totals

The overview's headline block had no source. /api/traces returns the trace
list, so "37 errors in 24h" meant fetching every buffered exchange to count it
in the browser — waste that grows with the buffer, to produce three integers.

Adds GET /api/traces/summary: totals, failures, p95 and a bucketed series for
sparklines, over a window that defaults to 24 hours and is capped at a week.

Deliberate calls, each with a spec:
- A 4xx is the caller getting it wrong, not the installation being unhealthy,
  so only 5xx and transport errors count as failures.
- p95 is a nearest-rank percentile rather than the slowest request, which is
  what a max would report and what makes latency panels lie.
- Buckets are oldest-first so a sparkline reads left to right, and the slice is
  never nil: nil serialises as null and breaks .map() on the other side, which
  is a silent runtime error rather than an empty chart.
- Exchanges outside the window are not counted at all.

The route is registered before /api/traces/:id so "summary" is not captured as
a trace ID.

On the client, Traces and Usage gain the rail signals they were shipped
without, the Observability section summary now states counts instead of listing
its destinations, and an installation that has served nothing says so rather
than showing three zeroes dressed as telemetry.

Sparkline is a bare stroke with an emphasised endpoint and no axes: the figure
above it already states the value, so its only job is the shape.

Go: 185 middleware specs pass. Full e2e suite: 396 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* fix(ui): stop the memory chart calling a trade-off an error

The VRAM-by-context chart rendered any build over the limit in error red, and
escalated the verdict to the error tone as soon as two context sizes crossed
it. But an over-limit build still installs — #11288 keeps a test on exactly
that — so red overstates what is happening. A model that fits at 32k and not
64k is a trade-off, not a fault.

Over-limit bars and the limit line now use the warning tone, which is the
constraint colour used everywhere else in this branch: know what you are doing,
not you may not. The error tone is reserved for "fits nowhere", where the model
genuinely cannot run on this host.

Full e2e suite: 397 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): give the new surfaces orchestrated motion

Uses the reveal system already in the codebase rather than adding a library:
pageReveal, .reveal-stagger and staggerStyle() were built for exactly this, and
anime.js would be ~17KB duplicating four lines of CSS for list reveals.

The overview's headline figures, attention rows and section lanes stagger in,
as do Studio's modality lanes and recent outputs, so a page assembles in the
order it is read instead of appearing all at once.

Two additions beyond stagger. Rail signals transition on opacity when a poll
lands, so a number changing reads as an update rather than a jump cut, and it
stays on the compositor so it cannot reflow the rail. The attention block
animates its left edge in — the one thing on the page that should announce
itself, and on the border rather than the text so nothing moves under a reader.

Both are dropped entirely under prefers-reduced-motion, alongside the lane
hover inset already handled.

Full e2e suite: 397 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): put the generators on a hairline field stack and record the request

The workbench treatment from the mocks, applied where it costs least: both
changes land on shared surfaces, so all six generators get them at once rather
than drifting apart page by page.

The control column stops being a shadowed card of boxed groups and becomes a
hairline field stack — the panel is the page's left half, not an object
floating on it — with uppercase micro-labels matching the eyebrow treatment
used elsewhere. Because .media-controls is shared, Images, Video, 3D, Speech,
Sound and Audio FX all move together.

RequestPanel shows the request the form actually built, with a copy-as-curl.
LocalAI is API-first and Studio is the best place in the app to teach its own
endpoints: the form stops being a black box, and a result worth keeping can be
reproduced from a shell without reverse-engineering which fields the page sent.
It records what was sent rather than what the form currently holds, and renders
nothing until a request has been made — a panel describing a request nobody
made is a tutorial, not a record. Wired into Images and Speech.

Full e2e suite: 401 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): make Chat a transcript instead of a bubble thread

Rounded, filled, asymmetric bubbles fight a system built on hairlines, and they
carry the speaker in shape and side rather than in words. The assistant side
had already given up its bubble; this finishes the job.

Both roles now run full width down one column, separated by a rule, each with a
mono role label. The user turn keeps a left edge in the action tone so the two
are still told apart at a glance, without a fill or a corner radius. The
avatars go: the accent and the label carry the speaker, so the glyph was
decoration once neither side had a bubble.

Saying who is speaking in words rather than in geometry is also what survives
being read aloud, printed, or looked at by someone who cannot pick the sides
apart by colour.

Full e2e suite: 404 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* feat(ui): dress the API reference in LocalAI's palette

The Swagger page was the last surface still shipping in someone else's colours,
which is conspicuous now that everything it links from is dark.

Swagger UI has no theming hook, so rather than fork it we serve our own index
ahead of the library's wildcard and restate the palette over its stylesheet.
The library's own bundle and assets are still what load, so a swagger-ui
upgrade cannot silently break the page — this is a skin, not a fork.

Two things needed real care. Swagger tints the entire operation row per method
via .opblock.opblock-post and friends, so the palette had to match that
specificity rather than reach for !important; the method now lives on one edge
instead of washing across the row, because a page where every row is a status
colour has no status colour left. And the filled method chip put white on pale
green, which was the least readable thing on the page — it is an outlined mono
chip now, carrying the method in its border and text.

Palette values are copied from theme.css rather than referenced: this page is
served by Go and never sees the app's CSS. The comment says so, and says to
keep them in step.

Go: routes and middleware suites pass. Full e2e suite: 405 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

* fix(ui): make tall split-view pages reachable, repair the Agents header, scale titles

Three things found by actually using the app rather than measuring it.

**Host was unusable.** The shell above a split view is overflow:hidden so the
document cannot grow, which left anything taller than the viewport simply
unreachable — and Host stacks a resources card, four stat cards and a tab bar
above its split, so the bottom of the pane fell off at every window height with
nothing to scroll. Every sweep I ran for this was horizontal, which is why it
kept coming back clean.

The page now scrolls inside the pinned shell. The pane keeps its own scroller:
letting it grow instead pushes the document taller and stretches the rail to
match, which is the regression e2e/discover-height.spec.js exists to catch, and
which the first version of this fix duly caused.

**The Agents header controls were unstyled** — "Create Agent" was rendering
with the browser's default chrome. The markup had been mangled at some point:
six unrelated classes merged into one string on the link, and the label and
button left with none at all and empty icons. Repaired, with the inline flex
replaced by a shared .header-actions class.

**Page titles take the editorial scale from the site**: larger, tracked at
-0.04em, on a line height near 1, so a two-word title reads as a statement
rather than a label. The typeface is unchanged — DESIGN.md keeps the existing
type system — so the whole difference is scale, tracking and leading, which is
where the site gets its voice from. This was the biggest reason the running app
still did not look like the mocks.

Full e2e suite: 404 passed, 4 skipped.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash]

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-08-03 00:16:08 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 74b7ea2829 feat(ui): replace the gallery and inventory tables with a rail and a detail pane (#11288) 
* feat(ui): rename the Install Models nav entry to Discover

"Install Models" named the action rather than the destination, and it was
the only multi-word entry in a rail of one-word ones (Home, Chat, Studio,
Talk, Build, Operate). A bare "Models" was the obvious fix but it collides
with the installed-models view under Host, which is a different page for a
different job.

"Discover" keeps the rhythm and says what the page is for. The icon moves
from a download arrow to a compass for the same reason: the page is browsed
before it is installed from.

Translated in all seven locales rather than left to fall back, so a locale
switch does not leave the entry in English.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* feat(ui): replace the gallery table with a rail and a detail pane

The eight-column table was not the real problem; the click-to-expand row
underneath it was. Variants, files and a VRAM estimate never fitted inside a
<tr>, so they were pushed into a drawer that could hold one model at a time,
could not be linked to, and had no room to say anything useful.

The gallery is now a rail to scan and a pane that answers. The pane has two
states and no third: with nothing selected it is the discovery page, and with
a model selected it is that model's detail. Selection lives in the URL, so a
model is linkable and Back steps out of the detail instead of off the page.

The rail groups by capability while browsing and flattens to results the
moment a term is typed. That is a rule rather than a toggle: once someone has
said what they are looking for, the buckets are between them and the answer,
and making the user choose would be handing them our problem.

The detail pane plots VRAM against context length with the host's own limit
drawn across it. This is new information, not a restyle. A single number
invites "so will it run?", and the honest answer is usually "yes, up to a 32k
context", which is a shape rather than a number. The estimates were already
fetched for every context size, so it costs no new request. Backends that
take no context length say so instead of being given a meaningless chart, and
a host with no GPU gets no chart at all rather than bars with nothing to
compare against.

The split-button variant menu goes with the actions column. The pane lists
every build with its backend, quantization, size, fit and a details
disclosure, each installable, which is what the dropdown was a cramped
substitute for. Its tests move onto that list; the three contracts it alone
carried (fetch-once caching, the loading state, an unfit build staying
installable) are backfilled against the pane.

RecommendedModels moves inside the pane, where it has the width to argue for
a model instead of listing one, and keeps its own dismissal and collapse.

Rail entries carry no description. Two lines is the budget and the second is
better spent on whether the thing will run; the stripped-Markdown contract
moves to the pane's lede, tooltip included.

e2e: 123 passing across models-gallery, navigation, recommended-panel,
model-artifact-operation, operations-strip and page-render-smoke. Inline
styles in Models.jsx drop from 82 to 41.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* refactor(ui): extract the split view into shared components

Discover shipped its rail, pane and detail header as private functions inside
Models.jsx. Backends and Host have the same defect and want the same shape, so
leaving them there guarantees three rails that drift.

SplitView, EntityRail, DetailHeader and StatGrid now live under
components/split/. EntityRail is deliberately data-driven: a surface maps its
own entity onto { id, name, icon, meta, stripe, groupId } and keeps its
vocabulary to itself, which is what stops the rail learning about models,
backends and loaded state all at once.

The CSS moves with it. What was .discover__rail is .entity-rail, .discover__
pane is .split-view__pane and so on, because a class named after one page is a
lie on the next two. Only what is genuinely Discover's stays behind the old
prefix: the shelves, the hero and the VRAM-by-context chart.

Two additions the shared rail needs and Discover did not: a state stripe, for
surfaces read by condition before they are read by name, and an empty label.
Discover passes neither.

No behaviour change. e2e 100 passing across models-gallery, navigation and
models-recommended-panel.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* feat(ui): put the backend gallery on the split view

Same defect as the model gallery, so the same shape: a seven-column table over
a click-to-expand row that was the only place the repository, licence, tags and
links could go.

The rail groups backends by the use case they serve, sharing Discover's
taxonomy on purpose: a backend is the runtime a use case needs, so "vision"
ought to mean the same thing one level down. It flattens on a query for the
same reason it does on Discover.

The zero state is the one real departure. A backend's fitness is not free
memory, it is the accelerator and platform it was built for, so the pane leads
with what this host is, then what is not installed yet, then whether anything
installed has gone stale. The table listed 37 runtimes and left "which of these
can even run here" entirely to the reader.

Distribution moves into the pane, which is the one thing a row could never
carry: which nodes hold a copy and which do not, with the install-on-more
control next to it rather than squeezed against a chip.

The distributed and target-node action logic is unchanged, including the guard
that keeps a hardware-specific build off the fan-out path. The split-button
popover loses its per-row anchoring because there are no rows; one pane, one
anchor.

Selection lives in ?backend=, preserving the ?target= scope rather than
clobbering it.

e2e: 139 passing across models-gallery, navigation, backends-management,
models-recommended-panel, nodes-per-node-backend-actions, page-render-smoke,
operations-strip and model-artifact-operation. The backends spec gains six
split-view tests; its three description-cell tests move onto the pane lede.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* feat(ui): put the Host inventory on the split view

The last of the three surfaces, and the one that is not a catalog. Both tabs
had the same click-to-expand row, so the shell transfers; what does not
transfer is the zero state, because there is nothing to discover in your own
inventory.

With nothing selected the pane reports what is happening: how many models are
loaded, what failed, what has an update, and which models are holding VRAM
right now. Every number was already on the page. None of them had been
assembled into one statement, so "what is going on" was a question the tabs
could not answer however long you looked at them.

The rail buckets by state rather than capability - Running, Idle, Disabled for
models; Update available, Installed for backends - which is the opposite of the
galleries and deliberately so: nobody opens Host wondering which of their
models does vision. Entries carry a state stripe for the same reason.

Load and Stop are promoted out of the kebab, because that is what an operator
came for; the rest stays behind the menu rather than diluting it. Adopted,
pinned and alias badges follow the model into the pane: they are facts about
the thing, not about its state, and the rail line is spent on state.

Deliberately NOT done: folding the two tabs into one rail, as the mock had it.
It costs five URL parameters, the manage-tab localStorage key and the
stat-card shortcuts, all of which are live deep-links today. The tabs stay as
the group selector; merging them is a follow-up with its own migration.

e2e: full suite 355 passing. New host-split-view spec; alias-template,
manage-logs-link, manage-action-menu-position and model-editor-back-nav move
off `.table` and the row kebab onto the rail and the pane.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* polish(ui): accessibility and consistency pass over the three split views

Findings from a pass over what the previous four commits actually shipped,
rather than what they were supposed to.

The rail was not a listbox. ARIA lets a listbox contain options and groups,
and nothing else, but each group's collapse control is a button that has to
sit inside the scroller with the entries it folds. It is now a labelled group
of buttons, which is the honest description; selection is announced with
aria-current and the arrow keys are unaffected.

Every entry was its own tab stop, so tabbing past a forty-entry rail to reach
the pane took forty keystrokes. Roving tabindex makes the rail one stop, and
arrowing now moves focus with the selection instead of leaving it behind on an
entry Tab can no longer reach.

The rail rounds its corners with overflow:hidden, which was clipping the focus
ring off the first and last entries entirely. Inset outlines fix it.

A 30px row is fine under a mouse and too small under a thumb, so coarse
pointers get a 44px target without costing density on a desktop.

One slot said three different things: "9 models loaded" on Discover, "12
loaded" on Backends, "3 of 9" on Host. All three lists are a page of a larger
set, so all three now say so the same way.

Also removed: an emptyLabel prop on EntityRail that nothing passed, its dead
CSS rule, and MODELS_COLSPAN and ResourceRowDesc, which died with the tables.

e2e: full suite 355 passing.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* fix(ui): correct three defects only a real gallery exposed

Running the branch against a live instance with 1,595 models and 1,017
backends, rather than against mocked fixtures, surfaced three things the e2e
suite could not.

Grouping did nothing. The rails matched on the use-case keys the filter chips
send (`chat`, `tts`, `transcript`), but those are a server-side vocabulary the
handler maps onto entries. What entries actually carry is free-form and
inconsistent: models come back tagged `llm`, `gguf`, `vision`, `coding`, and
backends `LLM`, `text-to-text`, `audio-transcription`. Nothing matched, so
every model landed in "Everything else" and the feature was decorative.

Grouping now lives in utils/entityGroups.js, shared by both galleries, matching
case-insensitively against the vocabulary the API really uses, with the entry's
backend as a fallback signal - a backend named `whisper` is a speech backend
whatever its tags say. Order is specific before general and that is
load-bearing: a vision model is tagged `llm` too, so testing text first would
swallow it.

The zero state claimed GPU memory on a machine with no GPU. The resources
endpoint reports system RAM in the same field when gpu_count is 0, so the hero
read "84.4 GB of GPU memory" next to the recommendations panel correctly
saying "No GPU detected". The number was never wrong, only its label; it now
says system memory unless a GPU is actually present.

The page title still said "Install Models" under a nav entry saying Discover.

Also: the keyboard test named the model it expected to arrive at, which made it
a hostage of the grouping table and broke the moment the buckets were fixed. It
now asserts that the selection moves and returns.

e2e: full suite 355 passing.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* fix(ui): the filters and the rail were fighting over the same job

Four things you find odd on Discover, and they turn out to be one mistake seen
from four sides.

The rail grouped the current page. The listing is paginated at nine rows, so
those bucket headers described nine entries out of 1,595, and turning a page
reshuffled the sections under the reader. The structure was never stable
because it was computed over the wrong set.

The chips were redundant for the same reason, seen from the other side. They
send tag= and filter all 1,595 server-side. The rail grouped nine of them
client-side by the same axis. Two controls for one job, and the weaker one was
the one this branch added, so it goes. Grouping stays only on Host, where the
list is complete, local, and bucketed by state rather than capability.

The search bar felt odd because it sat in a full-width band while the thing it
narrowed was a 290px rail below and to the left. The whole band now lives in
the rail column: search, backend, use cases, refinements, then the list it
narrows. One column to say what you want, one to show what you got. Nineteen
chips do not fit at that width, so they fold into a disclosure that states the
selection. A disclosure and not a popover, deliberately: picking use cases is
multi-select and interleaves with the backend select and the toggles below,
and a popover dismisses itself the moment you touch either.

The header held two counts and two buttons at arm's length from all of it. The
counts were the third statement of the same number on one screen, after the
rail's "9 of 1,247" and the pane's own headline, so they go. The buttons move
into the pane's zero state, which is the surface that answers "what do I do
here".

Also: the two first-run empty states wore .loading-center, which is
display:flex in the default row direction because it exists to centre one
spinner. With four children that put the icon, the heading, the sentence and
the buttons on a single line with no gap. They are now a proper full-height
empty state.

e2e: full suite 353 passing. Grouping tests are replaced by ones asserting the
rail stays flat; chip tests open the disclosure first; two filter-layout tests
that asserted the old three-band arrangement now assert the column.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* polish(ui): make Discover a full-height view, group the chips, name the refinements

Four things, all of them the same complaint: the page read as a document with
controls scattered on it rather than as one view.

The header is fused. A title block with its own padding, a subtitle and two
counts made the split view look like an attachment to a document that happened
to sit below it. It is now a slim bar carrying the title, the count and the two
page-level actions, and the split fills the rest of the window. Rail and pane
scroll independently, so the filters and the pane's headline stay put while a
long list moves under them.

The chips group. Nineteen in a flat row is a lot to scan even behind a
disclosure, and they already belong to the four families the rest of the UI
speaks, so they are bucketed by those. "All" sits on its own above them without
a heading, because it is a reset rather than a use case.

The refinements stop looking dumped. When the band became a column they were
three controls left where they landed; they now read as a named section with
one control per row.

The zero state suggests again. It had decayed into a "Browsing / 9 of 1,247 /
select a model" line that restated the count for the third time on one screen.
It now offers the four use cases as tiles that set the filter, which is the
shelf idea from the mock without inventing curation or paying for a second
fetch.

Two bugs found by looking at it rather than at the tests: the disclosure was
clamped to 190px, which cut it off partway through its third section so two of
the five never appeared at all; and the creation actions rendered twice, once
in the new bar and once in the pane hero a few pixels away.

e2e: full suite 353 passing. The chip-row test now holds its contract across
the per-family rows rather than a single one, and additionally asserts every
family is present and non-empty.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* fix(ui): pin the split view's height so a long detail scrolls the pane

Selecting a model with a long description grew the whole page and dragged the
rail down with it, which is the opposite of what "full height" was supposed to
buy.

The flex chain was right and the ceiling was missing. .app-layout and
.main-content are min-height:100dvh, which is a floor: flex distributes free
space but nothing caps growth, so a pane taller than the viewport expanded the
column, the document scrolled, and the rail stretched to match. height:100% on
the pane then resolved against an auto-height parent and did nothing.

The chat route already solves this by pinning .main-content to 100dvh. The
same treatment now applies to any route containing a .page--app, selected with
:has() so the shell does not have to learn which pages happen to be split
views. Below the stacking breakpoint the pin is lifted, because two stacked
halves in two short scrollers is worse than a page that scrolls.

Measured on a live instance: document height stays at the viewport across
selection (950px either side) and the pane overflows internally instead.

Adds discover-height.spec.js, which asserts the page height and the rail height
are unchanged by selection and that the pane is the thing that scrolls. The
existing specs could not have caught this: they mock short descriptions, and
the bug only appears when the pane has more content than the viewport holds.

e2e: full suite 355 passing.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* feat(ui): give Backends and Host the full-height view, and fix the Update button

Backends now matches Discover: the header fuses into a slim bar carrying the
title, the count and the page-level actions, the filters move into the rail
column where they narrow the rail and nothing else, and the split fills the
window. Its seven chips fit at rail width, so unlike Discover's nineteen they
need no disclosure. Host gets the bar and the height; its resource monitor,
summary cards and tabs stay above the split, because those are read once while
the rail and the pane are worked in.

Two things the height change surfaced.

The console layout is a flex row with align-items:flex-start, so its body sizes
to content. Right for the pages it was built for, wrong for a split view, which
needs a ceiling to scroll inside: without it the Backends rail ran past the
viewport and over the footer. Pinned with :has() so only split-view routes are
affected.

The filters vanished when nothing matched. Both galleries swapped the whole
shell for an empty state, which took the search box and the chips with it, so
the page said "try adjusting your search or filters" while offering neither.
The shell now stays and the empty state moves into the pane.

Also fixes the Update control on Host, which had no className at all and
rendered as bare text, next to a status span that had picked up btn classes and
two copies of `fas` and so rendered as a button you cannot press. They have
swapped appearances back.

e2e: full suite 355 passing. The render-smoke selector learns .view-bar__title,
since the pages it checks no longer all use PageHeader.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* fix(ui): keep the view mounted while searching, and bring rail grouping back

Searching replaced the whole view with a loader. The search box lives in the
rail column, so every debounced refetch unmounted the field being typed into
and dropped its focus with it. The list, the filters and the pane went too.

The shell now stays and the rail says it is busy: a sweep bar under its header
and the stale list dimmed, so the eye knows the answer is being replaced
without losing its place. A cold start still gets the skeleton, because there
is nothing to keep.

The condition for that is "nothing has loaded yet", not "the list is empty".
Those differ exactly when someone is editing a query that matched nothing, and
getting it wrong there would unmount the view on the keystroke after a
no-results search - the worst possible moment.

Grouping comes back on both galleries. It was removed because nine rows could
not fill five buckets, so a page turn rebuilt the rail's whole structure. That
was a symptom of the page size rather than of grouping: the rail now asks for
30 rows instead of 9 (Backends 60 instead of 21), which is enough for the
sections to read as structure and turns five times fewer pages. The order of
the sections is fixed, so what changes between pages is membership, not
arrangement.

Grouped while browsing, flat while searching, as before: once a term is typed
the buckets stand between the reader and the answer.

Also gives GalleryLoader a class and a testid instead of six inline style
declarations on a bare div, which is why nothing could select it.

e2e: full suite 359 passing, including a new spec asserting the search box
keeps its focus and its value across a refetch, and that a cold start still
shows the skeleton.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* perf(gallery): stop invalidating the VRAM estimate caches on every request

Searching or turning a page felt slow. It was not the search and not the
listing: /api/models answers in 3-9ms. It was the VRAM estimate, which the
gallery asks for once per row, and which took ~2.3s every single time however
often the same model was asked about.

pkg/vram already caches what makes that expensive - the remote content-length
probes, the GGUF metadata reads and the HF repo sizes. Those caches key on a
gallery generation counter, and AvailableGalleryModelsCached triggered a
background refresh on every call, with each refresh bumping the counter. One
page view is one listing request plus thirty estimate requests, each of which
re-read the gallery and started another refresh, so the generation moved
constantly and every cache entry was stale before it could ever be read. The
caches were dead in production.

Three changes, each doing one thing:

A refresh interval. The cached list is still served immediately; this only
decides how often re-fetching from upstream is worth starting. Five minutes,
as a package variable so tests can drive it without waiting.

A generation bump only when the gallery actually changed. An unchanged gallery
re-fetched on schedule must not throw away work that is still valid, which is
the difference between an estimate costing nothing and costing a network round
trip.

A separate "loaded" flag. The cache engaged on `cached != nil`, so a gallery
that legitimately holds nothing read as never-loaded and took the blocking path
on every call, bumping the generation each time. Found by the test for the
interval, which could not pass while this was true.

Measured against a live instance with 1,595 models:

  one estimate, repeated     2.3s  -> 2ms
  a page of 30, in parallel  10s   -> 0.04s

A first, genuinely unseen model still costs its remote probe. That is inherent;
what changed is that it is now paid once per model per gallery version rather
than once per request.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* perf(gallery): warm VRAM estimates at startup, and stop the UI waiting on them

Two halves of the same complaint: the gallery stalls on VRAM estimation.

Server side, the estimates are now warmed in the background at startup.
Estimating an entry nobody has asked about costs a remote probe of its weight
files, and the gallery needs one per row, so the first visitor was paying for
the whole page. The warm-up walks the gallery in the order the UI lists it, so
the first page is ready before anyone reaches it.

It is bounded and it never blocks: 300 entries at 4 at a time by default, on
its own goroutine, stopping with the server's context. Warming the whole
gallery would be thousands of probes on every boot, which is rude to the
upstream and slow to finish; warming nothing leaves the first page paying two
seconds a row. Anything past the limit still warms itself on first view.
LOCALAI_VRAM_WARM_LIMIT=0 turns it off for an air-gapped host,
LOCALAI_VRAM_WARM_CONCURRENCY=1 slows it for a metered link.

Client side, the page no longer waits on estimates it does not need yet. It
fired one request per row at once; a browser allows about six connections per
host, so thirty estimates took every slot and the request behind a click - the
variant list, an install - queued behind work nobody asked for. That is the
freeze: the list was already usable, and the UI was busy fetching sizes. Four
at a time leaves room for the interactive request to overtake, and a row whose
estimate is still in flight says "sizing…" rather than leaving a blank where a
number will appear.

buildEstimateInput moves to core/gallery as EstimateInput, since the handler
and the warmer both need it.

Measured against 1,595 models, from a cold boot:

  page 1, 30 estimates in parallel   10s -> 0.04s
  full warm-up (299 of 300 entries)  3m, in the background

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

* chore: untrack data/.local_user_id and ignore the runtime data dir

`local-ai run` writes its instance state under ./data when started from the
repo root, which is exactly what a contributor testing a build does. The
identity file ended up committed on this branch by a `git add -A` while
verifying the gallery changes against a live instance.

Anchored, so it matches the runtime directory at the repo root and not a
`data` directory nested inside some package.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-08-02 19:28:36 +02:00
localai-org-maint-botandlocalai-org-maint-bot 9584377a50 feat(chat): edit saved conversation messages (#11189) 
* feat(chat): edit saved conversation messages

Add inline edit, save, and cancel controls for stored user and assistant messages without triggering inference. Preserve structured message attachments and cancel edits when streaming starts.

Assisted-by: Codex:gpt-5

* test(chat): preserve seeded conversation on reload

The saved-message edit test reloads the page to verify persistence, but its init script was replacing localStorage with the original fixture on every navigation. Seed only an empty store so reloads exercise the data written by the application.

Assisted-by: Codex:gpt-5 [Codex]

---------

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
 2026-07-31 23:38:03 +02:00
Richard Palethorpeandlocalai-org-maint-bot 9058a2bb46 feat: Add 3d generation UI/API and trellis2cpp backend (#10979) 
* feat(3d): add Generate3D RPC, FLAG_3D capability, and /v1/3d/generations endpoint

Adds the plumbing for image-conditioned 3D asset generation (binary
glTF / GLB output), modeled on the video generation path:

- backend.proto: Generate3D RPC + Generate3DRequest (staged image src,
  glb dst, seed/step/cfg_scale/texture_steps, quality and background
  enums, params map for backend-specific extras)
- pkg/grpc: thread Generate3D through client, server, embed, base and
  the backend interfaces; connection-evicting and distributed-node
  wrappers (in-flight tracking + file staging) included
- core/config: FLAG_3D usecase (guessed only for the trellis2cpp
  backend), '3d' canonical usecase string mapped to the Generate3D
  method, and a '3d' output modality
- REST: POST /v1/3d/generations (+ unversioned alias) returning
  OpenAIResponse with a /generated-3d URL or b64_json; conditioning
  image accepted as URL, base64, or data URI; quality/background
  validated at the edge; .glb served as model/gltf-binary
- auth: '3d' route feature (default ON); /api/instructions entry

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(trellis2cpp): add the trellis2.cpp image-to-3D backend

Wraps localai-org/trellis2cpp (C++/GGML port of Microsoft TRELLIS.2,
pbr-textures branch) as a Go+purego backend, following the
stablediffusion-ggml pattern:

- backend/go/trellis2cpp: purego bindings to the flat C ABI (v9,
  asserted at startup), eager pipeline load with model-set validation
  (refuses non-trellis GGUFs; degrades coarse/geometry-only/textured
  exactly like the upstream demo), Generate3D via t2_generate +
  t2_bake_glb writing a binary glTF to dst. Weight-free unit tests
  cover resolution/validation/param mapping — CI never downloads the
  multi-GB GGUF set or runs inference.
- CPU SIMD variants build into per-variant directories (the shared
  libggml sonames collide across variants, unlike sd-ggml's flat
  renamed-.so scheme); run.sh picks one via /proc/cpuinfo.
- CI wiring: backend-matrix entries (cpu, cuda12/13, vulkan
  amd64+arm64, l4t, l4t-cuda13, darwin metal), index.yaml meta +
  latest/master image entries, bump_deps tracking of the pbr-textures
  branch, changed-backends.js mapping, top-level Makefile targets.
- Importer: auto-detects trellis GGUF repos/URIs (registered before
  llama-cpp so the .gguf match isn't stolen) and expands any trellis
  URI to the full 10-file component set spanning the three LocalAI-io
  HF repos.
- Gallery: trellis2-4b (full PBR + 1024 cascade) and
  trellis2-4b-geometry (512 untextured) with verified sha256s.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(ui): 3D generation page with native GLB viewer and IndexedDB history

Adds a Studio tab + /app/3d page for the new image-to-3D endpoint:

- GlbViewer ports the trellis2cpp demo's dependency-free WebGL2
  renderer (quaternion trackball, metallic-roughness PBR, ACES,
  hidden-line wireframe with a bounded index budget) and pairs it with
  a minimal GLB parser for the two forms t2_bake_glb emits — dense
  vertex-PBR (linear COLOR_0 + _METALLIC_ROUGHNESS, uploaded as
  normalized integers) and the opt-in UV-atlas textured form. Parsing
  happens before any GL so stats and errors render without WebGL2.
- use3DHistory stores past generations (params, input thumbnail, and
  the GLB blob itself) in IndexedDB with keep-newest-20 eviction —
  GLBs are multi-MB binaries localStorage can't hold — and the page
  offers a download button for the active GLB.
- Wiring: CAP_3D capability constant (FLAG_3D — the exact string
  /api/models/capabilities serves), threeDApi, router entries, Studio
  tab, vite dev proxy, en locale keys.
- e2e: render-smoke entry plus a focused spec that feeds a real
  one-triangle vertex-PBR GLB through the parser/viewer and exercises
  IndexedDB persistence, selection, deletion, and API errors.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(3d): address API correctness and UX issues

Keep 3D generation on the LocalAI-specific /3d/generations route and ensure authentication and permissions cover it.

Propagate distributed transfer failures, publish a portable ARM64 backend image, honor importer overrides, and align discovery, upload validation, and touch controls.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(3d): add previewable print remeshing

Add a single-detail CGAL Alpha Wrap workflow for existing Trellis GLBs, including PBR reprojection, API documentation, tracing, and an in-browser preview before download.

Allow the remesh route to enforce its 512 MiB upload cap independently of the smaller global default so generated high-resolution meshes can be processed.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* build(trellis2cpp): centralize remesh dependency pins

Assisted-by: Codex:GPT-5 [apply_patch] [exec_command]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(kokoros): implement Generate3D stub for new proto RPC

The Generate3D RPC added to backend.proto for the trellis2cpp backend
made tonic's generated Backend trait require generate3_d, breaking the
kokoros-grpc build. Return unimplemented like the other unsupported
modalities.

Assisted-by: Claude Code:claude-fable-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com>
 2026-07-29 16:15:04 +02:00
localai-org-maint-botandlocalai-org-maint-bot 034df6ceb1 fix(worker): report RAM alongside GPU memory (#11167) 
* fix(worker): report RAM alongside GPU memory

Assisted-by: Codex:gpt-5

* feat(ui): show worker RAM on node views

Assisted-by: Codex:gpt-5

---------

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
 2026-07-28 23:55:35 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 0f7186f214 feat(ui): replace the stacked operations bar with a one-line strip and an Activity page (#11163) 
* feat(ui): record finished gallery operations in a bounded history ring

The operations panel drops an operation the moment it succeeds, so a user
who steps away cannot tell whether an install finished, failed or was never
started. OpCache now keeps the last 50 terminal operations, recorded from
the point where an op leaves the cache.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): pin the history ring's dedupe, outcome order and start stamp

Review of the history ring found four gaps. The dedupe guard and the
bounded seen set were unreachable through the exported API and so had no
coverage; an in-package spec file now drives opHistory directly. The
outcome switch claimed an ordering was load bearing that nothing pinned,
so an errored op that never reached Processed now has a spec.

Two behaviour fixes come with it. StartedAt was the zero time for ops
recovered from the store or replicated from a peer, since neither path
stamps a start time, which would have rendered as a two-millennia
duration; it now falls back to the finish time. Reusing a cache key with
a fresh job ID orphaned the previous stamp, so Set and SetBackend now
drop it.

The comment on the outcome switch described a state the code cannot be
in: CancelOperation sets Cancelled and Processed synchronously before the
handler removes the entry, so status.Cancelled already covers the cancel
endpoint. The !Processed clause stays for the dismiss endpoint firing on
an in-flight op, and the comments now say so.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): record operations that end on a peer replica

The NATS end event is the only signal a replica gets for an install another
replica ran. Record from applyEnd too, deduped by job ID so the originating
replica does not record its own broadcast twice.

Three start-stamp defects in the same path go with it. applyEnd now drops the
stamp unconditionally, since recordTerminal only cleans up on the path where it
found a cache key and an end event can overtake the local Set. applyStart drops
the stamp of the job whose cache key it replaces, which a peer-driven retry
previously stranded. And recordTerminal reads the stamp once instead of testing
Exists and then reading, so a concurrent record for the same job can no longer
delete the stamp between the two and let the zero time overwrite the
finish-time fallback, which the Activity page would render as a two-millennia
run.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): do not guess the outcome of a peer operation with no local status

A replica that restarts mid-operation hydrates its OpCache keys from
PostgreSQL, but gallery statuses are in-memory only and come back empty. The
end broadcast then landed on recordTerminal's nil-status branch, which reads a
missing status as queued-and-removed and filed a successful install as
cancelled. That reading is right locally and wrong on the peer path, where a
missing status means the outcome was never held here.

recordTerminal now takes the source of the terminal event and records nothing
when the peer path finds no status, restoring what the replica did before the
end event started recording. The local path is unchanged.

Also move the ApplyEndForTest seam to the conventional export_test.go, and stop
the dedupe spec from claiming to guard the ring's seen set: the local delete
removes the status keys, so the broadcast that follows returns before reaching
it. An in-package spec that calls recordTerminal twice does the pinning.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(api): add GET and DELETE /api/operations/history

Admin gated like the rest of the operations API. The live /api/operations
payload is unchanged so the one second poll stays small.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): expose operation history through OperationsContext

Fetched on demand and when the live list shrinks, never on the one second
poll interval.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): detect operation departure by identity and ignore committing ops in the ETA gate

Refetching history on a shrinking live count missed a completion that
coincided with a start, which is the common case during a batch install.
Track the live job IDs instead, so any departure triggers the refetch
regardless of how the count moved.

An operation that has finished downloading stays live at
currentBytes == totalBytes for the whole commit and install phase and can
never produce an estimate, so counting it in the all-or-nothing gate blanked
every other operation's time remaining for as long as it lasted. Only
operations still moving bytes get a vote.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): let only downloading operations gate the time remaining estimate

Verifying pins an operation flat below its total for the whole sha256 pass:
the AfterDownload hook reports completedBytes plus the finished file against
a total summed over every file, then hashes synchronously without emitting
progress. Files download sequentially, so a 15 shard model enters that
window 14 times, and a byte comparison cannot see it because the counter is
genuinely below the total throughout.

Gating on phase closes resolving, verifying, committing and persisting in
one predicate, so a quiet neighbour no longer blanks every other
operation's estimate for minutes at a time. The byte clauses stay: a
producer can report downloading with bytes already at the total.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): collapse the operations bar to a single line

Four concurrent installs used to take four rows above every page. The strip
now shows one operation, failure first, with a counter linking to Activity.
The close button hides the strip and no longer cancels an install.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): keep the operations strip from widening the page and from muting a failure

A long install error made the strip report a 1600px minimum width, which sized
main-content to fit and gave every page under it a horizontal scrollbar.
Inline-size containment plus shrinkable detail and bytes cells keep it inside
the viewport.

Hiding is no longer able to swallow the hidden job's own failure, a completed
removal or staging says so instead of claiming an install, a cancelling
operation renders as cancelling, and the live region no longer covers the
per-second percentage.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): shrink main-content instead of containing the strip, and expose progress

min-width on .main-content is what actually lets a long install error shrink,
and unlike inline-size containment it has no browser support floor and no
latent collapse if the strip ever lands in a shrink-to-fit context. It matches
what .app-layout-chat .main-content already does, and it clears pre-existing
horizontal overflow on narrow viewports as a side effect.

The progress track is now a labelled progressbar, so assistive tech can read
the value on demand rather than losing it to the aria-hidden that stopped the
live region re-announcing every poll.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): add the live operation card for the Activity page

Carries the detail the one-line strip has to drop: phase, bytes, the per-node
breakdown for cluster installs, and a labelled Cancel button. Cancelling is
destructive, so it gets a labelled button rather than a glyph.

A cancelling operation drops its progress bar and its time estimate, the same
call the strip makes: a percentage still climbing under "Cancelling" reads as
the cancel not having taken.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): give the operation card a verb, live node disclosure and its per-node detail

The card carried no verb, so an install, a removal and a staging op rendered as
spinner plus name plus kind tag and were indistinguishable. It now runs the same
verb and icon chain as the one-line strip, which is what stops the page that is
meant to carry more detail from carrying less.

The auto-expand default was evaluated once at mount. An operation is listed as
soon as it is admitted but its nodes are filled in only when the fan-out starts
reporting, so a card mounted at creation latched on the empty list and stayed
collapsed. The default is a live expression now, and state holds only an
explicit choice.

Also: an optional onRetry gates a Retry button, so the page can own the install
reconstruction without the card ever showing a control with nothing behind it;
the disclosure moved above the region it controls and gained aria-controls; the
toggle is gated at more than one node so the count is never "1 nodes"; an
unmapped node status is passed through instead of being relabelled "Queued";
error text is clamped with the full string in the title; and file_name plus the
per-node progress bar are rendered again, reviving three CSS rules that had gone
dead along with the detail they styled.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): add the Activity page

Live operations, unacknowledged failures and the record of what finished, at
/app/activity in the Operate console. Cancelling an install now lives here
behind a labelled button rather than on the strip, and a failed install can be
retried: the retry dismisses the failure first so it still reaches the record,
then reissues the model, backend or node-scoped backend install.

The sidebar Operate entry carries the operation count. The console rail is only
rendered on an Operate route and can be collapsed, so a badge there could
vanish while operations were still running.

Two follow-ups from review fold in here: a failed removal or staging job no
longer reports a failed install on either the card or the strip, and the card's
error text can shrink so one unbroken token cannot widen the card.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): dismiss operations by job, and stop the Activity page contradicting itself

Dismissing resolved the job by display id, but /api/operations strips the
"node:<nodeID>:" prefix before emitting, so a local install and a node-scoped
install of one backend arrive as two jobs sharing one id. Dismissing by id
retired whichever came first. That defeated the guarantee retry was built
around: with the wrong job dismissed, the reinstall overwrote the acted-on
failure's opcache entry in place, bypassing recordTerminal, while an unrelated
failure vanished from Needs attention. dismissFailedOp, the card's dismiss
control and the strip now all pass the jobID, which is what the endpoint takes.

A filter matching nothing rendered the "nothing has ever run" empty state while
the header counted the records the filter had hidden. The empty state is now
gated on the All chip and a narrowed view gets its own message plus a way back;
the header counts the instance rather than the chip, so selecting Backends no
longer reports "Nothing running" over running model installs.

Also: the summary drops a zero clause instead of rendering "0 needs attention"
on the happy path and pluralises both counts; a record duration is floored at
"< 1s" and rejected above a day, so a zero-value start stamp cannot render a
span of millennia and a zero span cannot render "installed in" with nothing
after it; a deletion cancelled mid-flight reports the cancellation rather than
claiming it was removed; and the retry variant comment names the fix instead of
calling the gap closed.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: document the Activity page and the operations history endpoints

Adds an Activity page under Operations covering the one-line operations
strip, the /app/activity sections and filters, per-operation cancel,
retry and dismiss, the in-memory 50-entry record, and the sidebar count.
Documents GET and DELETE /api/operations/history, and fills the gap in
the admin-only endpoint list, which also omitted the pre-existing
POST /api/operations/:jobID/dismiss.

Corrects the distributed-mode install-watching section: the per-node
breakdown now lives on the Activity page rather than on the strip, which
rolls a fan-out up into a single phrase.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: correct nine details in the Activity page documentation

The operations strip never renders a file name: its detail line is the
error, the node roll-up, the target node, the phase or the queued note.
Drops the stale clause in the distributed-mode section, where the
per-node bullet is now the only place a file name is described.

Scopes the phase vocabulary to artifact-backed gallery models, since a
plain GGUF install emits no phase. Corrects the per-node list: the
toggle exists for any fan-out of two or more workers and the four-node
threshold only governs whether it starts open, while the N nodes tag
needs more than one node. Notes that a cancelled operation can sit in
the live section reading Cancelling, that cluster staging never reaches
the record, and that Clear history appears only when the record has
something in it.

Names the operations response envelope, with a JSON example, so callers
do not index a bare array, and stops describing the icon-only dismiss
control as a labelled button.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: drop the unreachable Cancelling state and scope the byte claims

An operation can only report isCancelled while it is unprocessed, but
every writer of Cancelled sets Processed in the same breath, on the peer
path as much as the local one, and the cache evicts cancelled entries
before the handler sees them. The state cannot reach the page, so the
live section is described again as running or queued operations.

Byte counts come from the artifact bridge alone, the same producer as
the phase, so a plain GGUF install, a removal and a backend install
report none. Scopes both to artifact-backed gallery models and leaves
the verb, the name and the percentage as what every operation shows. A
worker backend install reports its bytes through fields the operations
payload does not carry, so the distributed section now describes the
percentage and the node roll-up, with per-file counts pointed at the
per-node detail.

Also: staging jobs carry no error, so they never reach Needs attention
and Retry never had a staging case to exclude; an install that involves
workers is no longer called node-scoped, which this page uses for
node-targeted installs; and the record timestamps carry nanoseconds.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: state only the verb and the name as unconditional on the strip

The percentage is as conditional as the bytes were: it renders only for
a running operation that has reported progress, so a queued operation, a
failed one and a removal never carry it. A removal in particular sits at
progress zero for its whole visible life, since the delete path reports
none and its completion is filtered out. Both the strip and the card
paragraphs now lead with what always shows and list the rest as
conditions.

The Cluster chip matches on a node list that finished operations do not
carry, so a fan-out install leaves the chip once it reaches the record.
Scoped that claim to the live sections.

Two more of the same shape, found by re-reading each clause alone: the
strip also appears for a failure, which is not running, and the
four-second hold only applies when nothing replaces the operation that
just finished.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): stop reporting a cancelled install as installed, and make queued real

Three defects that all trace to one root cause: `isCancelled: true` is
unreachable from /api/operations. Every writer of Cancelled=true also sets
Processed=true, the handler skips Processed && Cancelled, and OpCache.GetStatus
evicts a cancelled op before the handler iterates it.

Cancelling the last running operation put a green "Installed model X" on the
strip for four seconds: the completion hold was guarded by
`!previous.isCancelled`, which is dead. A cancellation deletes the operation
server side, so the strip sees exactly what it sees on a completion, and
nothing in the payload separates the two. The signal now comes from the side
that issued the cancel: the operations context remembers the job IDs it
cancelled (pruned after a minute) and the strip asks before it holds anything.
A cancelled operation goes as soon as it stops; the record already reports it
as cancelled.

isQueued was set only when the gallery status was missing, but markQueued
publishes a "queued" status at admission, so a queued op has a status for its
whole queued life and the state was unreachable outside a microsecond window.
Every operation waiting behind a running install rendered as "Installing model
X" with a spinner. The queued phase is now the signal, via an exported
PhaseQueued and a nil-safe OpStatus.IsQueued() next to the writer.

With those two fixed, the Cancelling state has no way to be entered: cancelling
is instantaneous from the API's point of view. Its branches, CSS, locale key
and the isCancelled field itself are removed rather than left for a future
reader to assume they work.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): keep a removal a removal, and say what an install is doing

OpStatus.Deletion was set once, at admission, and lost on the next status
write: UpdateStatus replaces the whole status and only carried Nodes
forward. Every later writer (the worker's first write, the progress
ticks, the failure path) leaves the field at its zero value, so the flag
survived only the queued window, and both surfaces test isQueued first.

The reachable consequence is that a failed removal reported itself as a
failed install, which is exactly the shape the Activity page offers Retry
for, and Retry installs: pressing it on a removal that failed
re-downloaded the model. A running delete also rendered as "Installing
model X" with a spinner, and a successful one as "Installed model X".

Carry Deletion forward the way Nodes already is. A job is a delete or an
install for its whole life; an unset flag means "no new information", not
"this is an install". Pinned by Go specs on both the service and
/api/operations: the existing Playwright specs were green only because
they stubbed a payload the server could not emit.

Also restore the operation's own status message on the Activity card.
Phases and byte counters exist only on the managed-artifact path, so a
legacy files: gallery model and every backend install rendered a sub-row
with nothing in it but the verb. The strip stays terse on purpose.

And give the strip's name a min-width floor: overflow: hidden zeroes its
automatic minimum, so a long error squeezed the name down to "mod…" and
the identity of the thing that broke was the first thing lost.

primaryOperation is made module-private: its comment claimed the Activity
page selected the same operation, but that page shows all of them,
partitioned into failed and running, and never imported it.

Assisted-by: Claude Code:Opus 5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(activity): read the operations record from PostgreSQL

The Activity page's record of finished installs and removals was a 50-entry
in-memory ring per frontend replica. In distributed mode that is the wrong
place for it: each replica keeps its own copy, a replica added by a scale-out
or a rolling deploy starts empty and never backfills, and "Clear history"
clears only the replica that served the request, so the record reappears on
the next poll routed elsewhere.

The data is already in gallery_operations. Read it from there.

GalleryStore gains ListTerminal and ClearTerminal, sharing a lifted
terminalStatuses set with CleanOld so there is one definition of "finished".
ListTerminal orders by updated_at, when the operation reached its terminal
status, because the record reports what finished and when.

OpCache.History and ClearHistory dispatch on whether a store is wired, so the
HTTP handlers and the OpRecord JSON shape are unchanged and the page needed no
change. A failed store read falls back to the local ring rather than blanking
the page, and ClearHistory empties the ring as well so a database blip cannot
resurrect a record the admin just cleared.

The name derivation in recordTerminal is lifted into operationDisplayName and
used by both paths, so the ring and the store cannot name the same operation
differently.

Also fixes a pre-existing bug the store path made visible: the backend channel
hardcoded op_type "backend_install" even for a removal, while the model channel
derives model_install/model_delete from op.Delete. Both channels carry the same
ManagementOp, whose Delete field the backend handler already branches on, so
the backend channel now derives backend_delete the same way.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(activity): keep a cancelled operation cancelled, and report a failed clear

Review follow-up on the store-backed Activity record.

A cancelled install was recorded as a failure. The cancel handler persists
"cancelled" synchronously, then the handler goroutine unwinds with the context
error and Start hands that to updateError unconditionally, which overwrote the
row with "failed: context canceled". The page rendered a cancelled install as a
red failure card offering Retry, with a raw context error as the reason.

Fixed in GalleryStore rather than in Start, because an operation finishes once
and the paths that retire one are not mutually exclusive: UpdateStatus now
refuses to rewrite a row that already reached a terminal status. That also pins
updated_at to when the operation really finished, which is the key the record is
ordered by, and Create's upsert now freezes the same columns so a worker
dequeuing an operation the admin cancelled while it was queued cannot reopen it
as pending.

ClearHistory returned nothing, so a failed delete logged a warning while the
handler still answered 200. The admin watched the record clear and come back on
the next fetch with nothing said about why. It now returns the error, the DELETE
handler answers 500, and the store is cleared before the local ring so a failure
leaves the fallback record intact rather than faking an empty one.

Hydrate is the only reader that decides from op_type whether an operation is a
removal, and it tested for "model_delete" exactly, so the backend_delete added
in the previous commit hydrated as an install: a replica restarting during a
backend removal rendered "Installing backend X". Both discriminations now go
through IsDeleteOpType/IsBackendOpType so a fifth op_type cannot silently read
as an install in whichever consumer was missed.

Also: the backend channel now persists Cancellable as !op.Delete, matching the
model channel; IsBackend falls back to the op_type prefix, since is_backend_op
is only written by UpsertCacheKey and the rows needing the name fallback were
reporting backend operations as models; and an unrecognized terminal status is
logged rather than quietly filed as a success, which is what the comment already
claimed.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(activity): keep a reaped operation correctable by its real outcome

The terminal-status freeze added in the previous commit was too wide. It froze
"failed" alongside "completed" and "cancelled", and the stale reaper writes
"failed" onto operations that are still going to run.

The gallery worker is a single goroutine consuming both channels serially, so
an operation queued behind a large download sits in "pending" with nothing
bumping updated_at, and ReapStaleOperations gives up on it after 30 minutes.
That used to be self-healing: the worker dequeued it, Create reset the row to
"pending", and the operation reported its real outcome. With the freeze the row
stayed "failed" forever while the install ran and succeeded underneath it: a red
failure card offering Retry for a model that is installed, omitted from
ListActive so no replica hydrates it, and no longer deduped cluster-wide by
FindDuplicate.

Freeze on ("completed", "cancelled") instead. That is all the cancelled-install
fix ever needed, and it leaves a failure correctable by what actually happened.
The set is separate from terminalStatuses, which ListTerminal, ClearTerminal and
CleanOld all still want in full, because the two mean different things: a
failure can be superseded by a real outcome, a completion or a cancellation is
the real outcome.

UpdateStatus now writes the error column unconditionally, so a corrected
outcome drops the previous attempt's reason rather than being recorded as
completed while still carrying "stale operation reaped" as its error.

Also adds the route-level spec for the 500 branch of DELETE
/api/operations/history, and trims a comment that credited the persisted
cancellable column with more than it survives long enough to do.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(activity): offer Cancel in the phase that can honour it

The cancellable flag was set at both ends of an operation's life and was wrong
at both, in opposite directions.

A queued operation is cancellable whatever it is. EnqueueModelOp and
EnqueueBackendOp select on the operation context, so cancelling one that is
still waiting releases the delivery goroutine and abandonQueued retires it: the
worker never sees it, nothing is downloaded, nothing is deleted. markQueued
nevertheless wrote Cancellable: !deletion, so a queued removal reported
cancellable: false and the UI hid the Cancel button in the one window where
pressing it both works and leaves no trace. A removal queued behind a large
install was stuck there until the install finished.

A running removal is not cancellable at all. DeleteModel and DeleteBackend take
no context, and modelHandler only checks the operation context after the call
returns, so a "cancelled" verdict would land after the model was already gone.
Both handlers nevertheless wrote Cancellable: true unconditionally at entry,
ahead of the op.Delete branch, offering a Cancel button the server cannot
honour.

So the queued phase is more cancellable than the running phase, which is the
reverse of the usual shape. markQueued now reports true unconditionally, and
the handler-entry writes report !op.Delete. Both sites carry a comment saying
why, because reading either one alone suggests the other is a bug.

GalleryStore.Create keeps !op.Delete: it runs at dequeue, so its value already
describes the running phase. Its comment now says so.

Specs cover queued removal, queued install, running removal and running install
through the handlers, plus the queued-removal case through /api/operations
where the flag is consumed, plus the behaviour the whole asymmetry rests on: a
removal cancelled while queued never reaches the worker and deletes nothing.
No existing spec asserted the old values.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Write] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(activity): clamp the installer message, and add a real-binary e2e spec

Running the page against a real local-ai showed the legacy installer message
wrapping to three lines and dominating the card: it embeds an absolute file
path, so it is both long and a single unbreakable token. One line, ellipsised,
full text in the title, matching what the error string already does.

The spec that found it runs with no route stubbing at all. Every other spec
here stubs /api/operations, which is how a payload the server cannot emit
(isDeletion true on a live operation) stayed green through a full review while
the UI rendered a removal as an install. It is skipped unless
LOCALAI_REAL_BINARY is set, so CI is unaffected.

Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-07-28 19:33:48 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto ff299df453 perf(http): gzip responses, cache hashed assets, bound the trace endpoints (#11056) 
Three measured HTTP-layer regressions on a live deployment, fixed together
because they all shape the bytes on the wire.

1. No compression. The server sent no Content-Encoding regardless of what
   the client asked for, confirmed with curl straight at 127.0.0.1:8080 so
   it was not an ingress artefact. Adds gzip middleware, on by default and
   configurable via LOCALAI_DISABLE_HTTP_COMPRESSION and
   LOCALAI_HTTP_COMPRESSION_MIN_LENGTH (default 1024 bytes so tiny bodies
   are not wastefully wrapped). Streaming routes are skipped explicitly:
   an SSE Accept header, a WebSocket upgrade, and the completion / SSE /
   log-tail path prefixes, because whether a completion request streams is
   decided by the request body, which the middleware runs too early to see.
   Already-compressed formats (woff2, png, mp4, ...) are skipped too; gzip
   made those marginally larger. Measured over the embedded React build:
   JS+CSS 2815 KB raw to 808 KB gzipped (3.48x).

2. No cache headers on content-hashed assets. Vite hashes the filenames,
   so a given /assets/ URL can never change content, yet they shipped with
   no Cache-Control, ETag or Last-Modified, and the browser re-fetched the
   whole bundle on every navigation with no conditional request available.
   /assets/* now carries public, max-age=31536000, immutable. index.html
   stays no-cache so a deploy is picked up, and the unhashed locale JSONs
   get a short TTL rather than the immutable one.

3. Unbounded trace endpoints. /api/traces returned 21,033,606 bytes in
   4.65s and /api/backend-traces 3,471,682 bytes in 1.50s, and the admin
   UI polls both every few seconds. The ring buffer holds up to 1024
   entries, each embedding full input_text payloads. Both list endpoints
   now take limit / offset / full, default to 50 entries, and strip the
   heavy fields (request and response bodies plus headers for API traces,
   body and data for backend traces) unless full=true. Every trace gets a
   process-lifetime ID and GET /api/traces/{id} and
   /api/backend-traces/{id} serve the full record, which is what the UI
   fetches when a row is expanded. The list body stays a JSON array;
   paging metadata rides in X-Total-Count, X-Trace-Offset and
   X-Trace-Limit. Reproducing the live shape in a test, the polled payload
   goes from 21,131,097 bytes to 7,201 bytes.


Assisted-by: Claude:claude-opus-4-8 [Claude Code]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-07-22 22:51:25 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 83a0f16a21 feat(gallery): let one gallery entry offer several builds of the same model (#10943) 
* feat(system): expose raw detected capability for model meta resolution

Model meta gallery entries express hardware fallback through candidate
ordering rather than a capability map, so they need the undecorated
detected capability string without Capability's default/cpu fallback
chain.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* refactor(system): drop duplicate capability accessor, cover DetectedCapability

ReportedCapability was added with a body identical to the existing
DetectedCapability. Keep one accessor and move the specs onto it, since
DetectedCapability had no direct coverage of its no-fallback behavior.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): parse IEC binary size suffixes (KiB..PiB)

ParseSizeString accepted only SI suffixes, so a "20GiB" floor was rejected
outright. Model and VRAM sizes are conventionally quoted in IEC units, and
silently reading GiB as GB would understate a floor by about 7%.

Purely additive: these inputs previously returned an unknown-suffix error.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): add Candidate type for meta model entries

Candidate is one option in a meta entry's ordered variant list. It names a
concrete gallery entry and declares when that entry suits the host.

EffectiveMinVRAM resolves the VRAM floor, letting an authored min_vram win
over a nightly-inferred one. An unparseable floor errors instead of being
treated as absent: swallowing a typo would turn a constrained candidate into
an unconstrained one and select a too-large variant rather than fail loudly.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): add hardware-aware model variant resolver

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): allow gallery model entries to declare variant candidates

A gallery entry with a non-empty candidates list is a meta entry: it names
an ordered list of concrete entries and resolves to the first one the host
can satisfy, instead of describing model files directly.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): resolve meta model entries to hardware-appropriate variants at install

Meta gallery entries carry an ordered candidate list; at install time the
first candidate the host satisfies is resolved and its payload installed
under the meta's name, so the model keeps a stable name regardless of which
variant backs it. The resolution is recorded in the installed gallery
config so a reinstall honors a prior pin and operators can see the backing
variant.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): key meta pin recall on the installed name and detach resolved entries

Six review findings on the meta-entry install path.

Pin recall was keyed on the gallery entry name while applyModel writes the
record under the install name (req.Name when supplied), so a meta installed
under a custom name with a pin lost that pin on reinstall and was silently
re-resolved onto a different variant, possibly swapping its backend. Compute
the install name with applyModel's own precedence before the recall.

ResolveMetaModel returned a shallow struct copy, so the resolved entry's
Overrides aliased the gallery entry's map and the install path's in-place
mergo merge wrote the caller's request into the shared catalog. Detach
Overrides, ConfigFile, AdditionalFiles, URLs and Tags. Not exploitable today
only because this path re-unmarshals the gallery per call, which is a
property nobody should have to rely on.

Also: overlay the meta's name onto the persisted config for meta installs so
the gallery file no longer records the variant's name; move the pinned-VRAM
warning below the variant validation so a pin naming a nonexistent entry does
not warn about VRAM before failing for an unrelated reason; and stop seeding
config.URLs in the config_file branch, which duplicated every declared URL.

Add seven network-free specs driving InstallModelFromGallery with a meta
entry: variant payload wins over the meta's legacy url fallback, the
resolution record round-trips to disk, a pin is recorded and honored on
reinstall including under a custom install name, and the resolved entry does
not alias the gallery's maps.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): deep-copy meta overrides and make two specs functional

ResolveMetaModel detached the resolved entry's Overrides and ConfigFile with
maps.Clone, which only copies the top level. Gallery overrides are nested in
practice (parameters.model is near-universal) and the install path merges the
caller's request with mergo.WithOverride, which recurses into nested maps and
overwrites them in place, so the gallery entry's own inner maps were still
reachable and still got rewritten by the last caller to install.

Copy both maps all the way down instead, recursing through the container shapes
a YAML decoder produces. ConfigFile is not mutated on the install path today,
but it carries the same kind of nested payload and leaving it shallowly cloned
would invite the bug back.

Also fix two specs that passed whether or not their target fix was present:

- "does not write the caller's overrides back into the gallery entry" re-read
  the catalog from disk, which re-unmarshals fresh structs and so cannot
  observe in-memory aliasing. It now asserts against the in-memory gallery
  entry and drives the real mergo merge.
- "round-trips the resolution record to disk under the meta's name" asserted a
  name that is already correct in the config_file branch. It now drives the url
  branch via a file:// fixture, where the meta-name overlay actually applies.

Both were verified red by reverting their fix.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(gallery): lint meta model entry invariants in index.yaml

Adds Ginkgo specs that parse the shipped gallery/index.yaml and enforce
the invariants that keep meta entries safe: a legacy url fallback equal
to the final candidate's url, references only to existing non-meta
entries, a min_vram floor on every candidate but the last-resort one,
a capability drawn only from the vocabulary the system can report, and
descending VRAM floors within a capability group.

The capability check is the only compensating control for a typo there.
Candidate matching is a case-sensitive exact comparison against
SystemState.DetectedCapability(), so an unknown value never matches and
falls through silently instead of erroring. The vocabulary therefore
mirrors the raw return set of getSystemCapabilities(), which notably
excludes "cpu": that is a fallback key inside Capability(capMap) on the
meta backend path, never a reported capability. A CPU-only host reports
"default".

These pass vacuously until the pilot meta entry lands; the guard is
intentionally in place before the thing it guards.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(gallery): close coverage gaps in the meta entry lint

The ordering invariant grouped candidates by capability and asserted floors
descend within a group. A candidate with an EMPTY capability matches every
host, so it does not belong in its own group: it dominates every later
candidate whose floor is at or above its own, across capability groups.
Track a running minimum floor over the unconditional candidates instead,
which subsumes the old same-group check for the empty capability.

Every spec skipped non-meta entries, so with zero meta entries in the index
all five bodies were no-ops. Aligning GalleryModel.IsMeta() with
GalleryBackend.IsMeta(), whose semantics are deliberately opposite, would
have made all of them pass while checking nothing. Extract each invariant
into a helper over a slice of entries returning the violations it finds, and
cover those helpers with synthetic fixtures so the logic stays tested at zero
meta entries. The index-driven specs are now a thin application of already
proven logic.

Also assert the index parses non-empty, report every violation in one run
rather than aborting on the first, and parse the index once for the suite.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* ci(gallery): add nightly denormalization of meta model candidates

Fills the read-only backend, quantization and inferred_min_vram fields on
meta gallery candidates and opens a PR, modeled on the existing
checksum_checker job. Computing these needs network access, so it happens
nightly rather than at install time.

An authored min_vram is never modified: a human who measured a real load
knows more than a pre-download estimate does.

The index is rewritten via yaml.Node rather than a document round-trip. A
full round-trip reflows all ~26k lines of gallery/index.yaml, which would
bury the computed values and make the nightly PR unreviewable. The rewrite
touches only the three derived keys, so authored styling survives and a run
that computes nothing leaves the file untouched.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ci): keep the gallery denormalize diff reviewable and self-healing

The nightly denormalization job edits YAML nodes instead of round-tripping
structs so its PR stays small enough for a human to review, but the write
path undid that: yaml.Marshal re-encoded the node tree at yaml.v3's default
4-space indent and dropped the leading document marker, reflowing roughly
6000 lines around the handful of real changes. Encode through
yaml.NewEncoder at the index's authored 2-space indent and restore the
header. A write that changes three fields now changes three lines.

Stale inferred_min_vram values were also never cleared. Both skip paths
(an authored min_vram is present, or the candidate is the last resort)
returned before touching the field, so a candidate that gained a floor or
became the last resort after a reorder kept an inferred value that
EffectiveMinVRAM reported as a real constraint, failing the meta lint with
no way for the job to self-heal. Clear the field before both skips.

The workflow discarded a whole night's work on any single failure: the
program exits 1 when a candidate cannot be estimated, which aborted the job
before the PR step, so one unreachable candidate blocked every other
refresh indefinitely. Capture the status, open the PR with what was
computed, mark the PR body as partial, and fail the run afterwards so the
problem still surfaces.

Also preserve the index's existing file mode instead of forcing 0644, and
drop the redundant //go:build ignore tag, since Go already skips dot
directories and the sibling modelslist.go carries no tag.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): add nanbeige4.1-3b meta entry with hardware-resolved variants

Adds the first real meta entry to the gallery index. It resolves to the
Q8_0 build on hosts with at least 6GiB of VRAM and to the Q4_K_M build
everywhere else, installing either payload under the stable name
nanbeige4.1-3b.

The entry carries a url equal to its final candidate's url. LocalAI
releases that predate candidates support parse the index non-strictly
and drop the key silently, so without that url they would list the entry
and install nothing. A regression spec parses the index the way those
releases do and asserts every meta entry stays installable for them.

Also teaches core/schema/gallery-model.schema.json about candidates. The
schema sets additionalProperties: false at the top level, so an author
following CONTRIBUTING.md and adding the yaml-language-server comment
would otherwise get a validation error on this entry.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): make candidate entries complete, installable entries

Reworks hardware-resolved gallery variants after a design pivot. There is no
longer a separate "meta" entry kind. A gallery entry is a normal, complete
entry that may additionally carry candidates:, a list of hardware-gated
upgrades over itself, and the entry is itself the last-resort candidate.

The previous design relied on a bare url: as the fallback for LocalAI releases
that predate candidates support. That fallback is empty in practice: none of
the 80 gallery/*.yaml files carry a top-level files:, and 1216 of 1281 index
entries carry their payload in the index entry itself, so a url alone yields a
config template with nothing to download. Since every released LocalAI reads
gallery/index.yaml live from master, merging a payload-less entry would have
shown every existing user a model that installs to a broken state. Making the
entry its own base candidate removes the problem at the root: old clients drop
the candidates key and install the entry exactly as they do today.

Resolution order is now explicit pin, then capability plus VRAM over the
declared upgrades, then the entry itself. The entry ALWAYS installs: when its
own min_vram or capability is unmet the installer warns and installs it
anyway, because there is nothing below it and refusing would make the gallery
behave worse the newer the client is. A pin naming the entry's own name is
valid and is how an operator declines an upgrade.

IsMeta() becomes HasCandidates(), ResolveMetaModel becomes ResolveVariant, and
the persisted meta_name record key becomes entry_name. GalleryBackend.IsMeta()
is a separate concept and is untouched.

The lint drops the three rules the pivot makes wrong (url equality with the
final candidate, no inline payload, unconstrained final candidate) and gains
one: the entry's own floor must sit strictly below every candidate's, since a
base that outranks a candidate makes that candidate unreachable.

The pilot entry is now the existing nanbeige4.1-3b-q4, which gains a 2GiB
floor of its own and a single 6GiB upgrade to nanbeige4.1-3b-q8, replacing the
separate nanbeige4.1-3b entry added in d0d441bb4.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): select model variants by hardware fit, not authored order

Gallery entries could already carry a list of alternatives, but selection was
an authored, ordered, first-match policy: every candidate declared a
`capability` string and the VRAM floors had to descend in a hand-tuned order.
That pushed hardware knowledge onto whoever edits the gallery and made ordering
load-bearing, so a reordered list silently changed what users installed.

None of it was necessary. SystemState.IsBackendCompatible already derives
hardware support from a backend name alone: it knows MLX and metal are
Darwin-only, CUDA is NVIDIA-only, ROCm AMD-only, SYCL Intel-only. Selection can
read that instead of asking authors to restate it.

Authoring is now just a list of names:

    - name: qwen3.6-27b
      min_memory: 4GiB
      variants:
        - model: qwen3.6-27b-mlx-8bit
        - model: qwen3.6-27b-gguf-q8
          min_memory: 28GiB

and all the intelligence moved into the selector. Given a host it drops the
variants whose backend cannot run here, drops those whose known memory
requirement exceeds what the host has, and takes the LARGEST of what is left,
because a bigger footprint is a higher quality quantization of the same model.
A variant of unknown size is kept, since nothing proves it does not fit, but it
ranks last so a proven fit always beats a guess. An explicit pin still wins
outright, and if nothing survives the entry installs its own payload: the base
always installs, this never refuses.

Available memory is VRAM when a GPU was detected and system RAM otherwise, read
through xsysinfo so a cgroup limit is honored and a container gets its own
limit rather than the node's RAM.

Capability disappears entirely, from the types, the schema and the lint. VRAM
and RAM collapse into one `min_memory`, because a model's footprint is roughly
the same wherever it lives and one figure is compared against whichever applies.
The lint rules about ordering, the capability vocabulary and floor
relationships are deleted with the hazards they described; what remains is that
every variant names an entry that exists and does not itself declare variants,
plus that any memory figure actually parses.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* gallery: size model variants with a live probe, drop the nightly denormalizer

Selection needs each variant's size to decide whether it fits and to rank
largest-first. That figure was written into the index by a nightly job, which
made the gallery carry a derived value that could drift from the entry it was
derived from. Derive it at install time instead.

pkg/vram already sizes a model without downloading it, and the gallery UI
already uses it: a remote GGUF header range-fetch, then an HTTP HEAD for the
content length, then any declared size:. It caches its results, so reuse it
rather than writing a second probing path.

A probe failure must never fail an install, so an unprobeable variant is
treated as unknown: it survives the memory filter, because nothing proves it
does not fit, and it ranks last, so a known-good fit always beats a guess. If
every probe fails, selection still terminates on the base entry.

The probe is injected through ResolveEnv rather than called directly, for the
same reason the backend compatibility check is: specs pin an exact size, or an
exact failure, without reaching the network.

With that in place three things are dead weight and go:

- The nightly job and the fields it populated. Variant.Backend was redundant
  because the backend is resolved live from the referenced entry during
  selection, and Quantization was display-only that nothing read.
- min_memory on the base entry. The base always installs and its floor could
  only warn, so it could not change any outcome.
- The lint rules and schema entries for both.

min_memory on individual variants stays, as the override for when the probed
size is wrong. An authored figure now suppresses the probe entirely rather
than merely outranking it, so it costs no round trip.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): expose model variants for selection over API, CLI and MCP

A gallery entry may carry `variants:`, alternative builds of the same model.
Selection already worked at install time, but nothing could see what an entry
offered or ask for a specific build, so the feature was undrivable.

Listing: `GET /api/models` now reports `variants` and `auto_variant` for the
entries that declare variants. Each variant carries its resolved backend, its
measured size and whether it fits this host. `auto_variant` is what installing
without a choice would pick right now.

The new gallery.DescribeVariants runs the same variantOptions + SelectVariant
pass the installer runs, so the reported default cannot drift from what
installing actually does, and HostResolveEnv is extracted so both derive the
host and share pkg/vram's probe cache from one place.

Performance: an entry that declares no variants returns early without touching
the probe, so the ~1280 ordinary entries cost exactly what they cost before.

Selection: `variant` is accepted on POST /models/apply, as a query param on
POST /api/models/install/:id, on the gallery apply file/string request, as
`local-ai models install --variant`, and as a parameter on the install_model
MCP tool (both the httpapi and inproc clients). Empty means auto-select.

An unknown variant name now fails the install naming what was requested. This
closes a real hole: an entry declaring no variants short-circuits before
selection runs, so a requested variant was previously dropped silently and the
install reported success.

startup.InstallModels ends in a variadic model list, so install options could
not be appended to it; InstallModelsWithOptions is added alongside and
InstallModels delegates to it. No caller signature changed.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): drop the redundant variant min_memory field

Variant.MinMemory was an authored override for when the live probe misreads
a variant's footprint. It duplicated an existing field: probeEntryMemory
already passes the entry's declared size: into EstimateModelMultiContext,
whose cascade prefers that declared size over its own guesswork. Correcting
size: on the referenced entry fixes the figure for every consumer rather
than only for variant selection, so min_memory shadowed the right answer.

A variant is now nothing but a name. Its effective size is exactly the probe
result, and an unknown stays unknown: it survives the filter and ranks last.

EffectiveMemory loses its error return along with the field. The authored
string was the only thing that could fail to parse, so the error had no
remaining source and was propagating dead nil-checks through SelectVariant,
DescribeVariants and the pin warning.

Selection behaviour is unchanged. The specs covering probe-derived sizing,
ranking, filtering, the unknown-size path, pin recall, entry/variant
metadata split and deep-copy isolation all survive; the three install specs
that needed a definite size now declare it through the referenced entry's
own size:, which exercises the documented escape hatch directly.

gallery/index.yaml is untouched: no entry ever carried the key.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): rank the entry's own build against its variants

Variant selection pulled the declaring entry's own payload, the base, out
of the candidate set and consulted it only once every declared variant had
been rejected. Two real failures followed.

A variant whose size the probe cannot determine deliberately survives the
memory filter, because nothing proves it does not fit. As the only survivor
it then won outright on any host, however small: a 2GiB machine installed an
unmeasured variant in preference to the 4GiB build the entry itself ships,
with no warning. 241 of the 1280 current index entries carry no files and no
size, which is exactly that shape.

"Largest wins" also broke whenever the base was the largest. An author
writing a Q8 entry that offers a Q4 downgrade for small hosts, a natural
shape that nothing in the lint, schema or docs discourages, had the Q4
installed on every large host instead.

Make the base an ordinary participant. It is still exempt from both filters,
so selection always terminates on something installable, but it is now
ranked against the variants: a proven fit first and largest, then the base,
then any variant whose size nothing could measure. Both failures disappear
together. The base is probed for its size accordingly, which it was not
before, because an unsized base would lose every contest to an unmeasurable
variant.

FellBackToBase is kept but narrowed to "no declared variant survived",
rather than "the base was chosen", since the base now also wins on merit and
that is not worth warning about.

A recalled variant pin also became a permanent install failure. A pin the
caller supplies on this request must stay fatal, but one recalled from
._gallery_<name>.yaml can be invalidated by any later gallery edit, and
failing on it turned one rename into a model that could never be reinstalled
or upgraded again short of deleting a dotfile the user has never heard of.
A stale recalled pin is now dropped with a warning naming it, and selection
runs as if it had never been recorded.

Also drop the last textual reference to two abandoned designs from the
DetectedCapability comment, correct the documented variants JSON example,
which showed a memory_bytes of 0 that omitempty makes impossible, and remove
an em dash from the install skill.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): budget variant memory from RAM when a GPU reports no VRAM

Variant selection read its memory budget from VRAM whenever a GPU
capability was detected, and from system RAM only when none was. Apple
Silicon satisfies the first branch and fails the premise: arm64 macs report
the metal capability unconditionally, without probing anything, while
TotalAvailableVRAM has no discrete VRAM pool to find and returns zero. The
budget therefore came out as zero on every Mac.

Zero drops every variant carrying a known size, so the base build was
installed on all of them however much memory the machine had. The feature
was inert on the platform, and silently: falling back to the base is a
legitimate outcome, so nothing looked wrong.

Take VRAM only when it is actually a number, and fall back to RAM
otherwise. On a unified-memory host RAM is not an approximation of the
budget, it is the budget, since the GPU shares it. A discrete GPU whose
VRAM could not be read also lands on RAM, which overstates what the card
holds but understates nothing the host has; the previous zero understated
both.

An unreadable RAM figure still yields zero and still installs the base, so
a genuinely unknown host is not talked into a larger download.

This is what turned tests-apple red: "installs a fitting variant's payload
under the entry's own name" asserts on selection, and the runner resolved
to the base because its budget was zero. The added specs pin the branch
directly rather than relying on a macOS runner to notice again.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): add a model variant picker to the models gallery

PR #10943 shipped the server side: a gallery entry may declare `variants:`,
`GET /api/models` attaches `variants` and `auto_variant` to declaring
entries, and `POST /api/models/install/:id` accepts a `variant` query
parameter. Nothing in the UI consumed any of it, so the feature was not
reachable from the browser. This wires it up.

modelsApi.install takes an optional second argument and appends an encoded
`?variant=` only when one is given, so every existing call site keeps
sending exactly the request it sent before.

On the models table, an entry that declares variants gets a split button.
The primary Install still installs the auto-selected build, because auto is
the default and the point of the feature; the chevron opens a menu for a
deliberate override. It follows the Backends.jsx precedent: one shared
Popover re-anchored per row, rendering .action-menu items, which brings
Escape, outside-click and focus return along with it. An entry that
declares no variants renders exactly as it did before.

A variant that does not fit is dimmed but stays selectable, since the server
honors an explicit choice with a warning rather than refusing it.

memory_bytes is omitempty on the wire, so an absent key means the size is
unknown and never zero. A single helper guards both the menu and the detail
row, because formatBytes would otherwise render a falsy value as "0 B",
which reads as "needs nothing".

The expanded detail row gains a Variants section listing each build's
backend, size, whether it fits, which is the entry's own build, and which
one auto-selection would pick, built from the existing DetailRow helper and
.badge classes.

Eight Playwright specs cover the picker, including that plain Install sends
no variant parameter and that choosing one sends it. One pre-existing
assertion was scoped with .first(): the Variants section legitimately adds
more llama-cpp badges to the detail row, which tripped strict mode.

UI line coverage 49.42% -> 49.36% against a 40.0 baseline and 0.8pp
tolerance; branch coverage rose 72.04% -> 72.66%.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): describe model variants from a companion endpoint

Variant description probes each referenced entry's weight files over the
network: an HTTP HEAD plus a ranged GET, serial, five seconds per probe
with no aggregate deadline. Running it inline in GET /api/models made one
listing cost (entries x variants) round trips. The Manage page fetches
with items=9999, so at 200 declaring entries that is ~1000 serial probes,
minutes of a blocked handler and gigabytes of range traffic for a single
page load. Only one entry declares variants today, but the feature exists
so that many will.

Follow the precedent already set for VRAM estimates. The listing now
reports only has_variants, a length check on loaded metadata that touches
nothing, and GET /api/models/variants/:id returns the description for one
entry, mirroring estimate/:id in route shape, auth and error handling.
DescribeVariants itself is unchanged; only its caller moved.

The picker fetches lazily at the two points where a user asks to see
variants, opening the split-button menu and expanding the detail row, and
caches per entry for the page session. An entry declaring no variants
issues no request at all.

A spec counts real HTTP hits on the weight files, so it goes red if
description becomes reachable from the listing path again through any
caller.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): filter the model gallery to entries that declare variants

The gallery is heading towards showing parent entries and hiding the
individual builds they reference, so a user sees one row per model
rather than six quantizations of it.

Adoption is a single entry today, so defaulting to that would leave a
one-row gallery. This ships the migration-phase inverse instead: the
default is untouched, and a toggle narrows the list to only the entries
that declare variants. It previews the end state and changes nothing
until someone asks for it.

The filter is server-side, next to term/tag/backend/capability and above
the pagination arithmetic. The listing paginates at 9 items, so
narrowing on the client would leave totalPages and availableModels
describing the unfiltered set and hand the user empty pages. It selects
on HasVariants(), which reads already-loaded metadata, so it issues no
variant probes.

The parameter is named has_variants after the listing field it selects
on, and is compared against "true" like the other boolean query params
(all_users, save_checkpoint), so has_variants=false reads as absent.
With it omitted the response is byte-for-byte what it was before.

The control is the shared Toggle component, matching the fitsFilter
toggle already on this page: same wrapper class, same icon and label
shape, same localStorage persistence. Unlike fitsFilter it resets to
page 1 on change, which a server-side filter has to do.

Stacking the toggle with a tag or backend filter easily yields nothing
while one entry declares variants, so the empty state now names the
variants filter as the cause rather than leaving a user to conclude the
gallery is broken.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): render gallery model descriptions as Markdown

Gallery descriptions are Markdown, but the React UI dumped them raw, so a
model whose description opens with an ATX heading showed a literal
"# Qwen3.6-27B [](https://chat.qwen.ai)" in the list.

Full-description areas now render through renderMarkdown (marked +
DOMPurify), matching how Backends.jsx and the Manage detail panels already
handle the same content:

  - Models.jsx expanded detail row
  - VoiceLibrary.jsx voice detail header

The truncated one-line previews must not render block Markdown: a leading
"#" would become an <h1> and wreck the row height and rhythm. They get a new
stripMarkdown() helper instead, which reduces Markdown to a single line of
readable plain text. It is used for the cell text and for the title tooltip,
since a tooltip full of "[](url)" is no better than a cell full of it:

  - Models.jsx gallery table description cell
  - Manage.jsx model and backend resource-row descriptions

stripMarkdown walks marked's lexer output rather than running regexes over
the source, so what it strips is by construction what renderMarkdown would
have rendered, and it needs no new dependency. Output lands in JSX text
nodes, so React escapes it; no new dangerouslySetInnerHTML beyond the two
full-description sites, both of which run DOMPurify.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): strip Markdown from the backends table description cell

Commit b35d630cf fixed this for gallery models but left the Backends admin
page with the same asymmetry: its detail panel renders the description
through renderMarkdown, while the collapsed table row dumped the raw gallery
string into both the cell body and the title tooltip.

That is user-visible. 40 of the 949 entries in backend/index.yaml carry
Markdown - insightface uses inline code backticks, others use lists and
links - and backend descriptions also contain embedded newlines, so the
one-line cell showed literal syntax.

The cell now runs stripMarkdown over the description once and uses the
result for the text and the title, matching Models.jsx and the
ResourceRowDesc component in Manage.jsx. The '-' placeholder is preserved,
and now also fires when a description reduces to nothing after stripping.
The detail panel is untouched and no new dangerouslySetInnerHTML is
introduced: stripMarkdown output lands in a JSX text node, so React escapes
it.

Three Playwright specs cover it: a description with a heading, inline code
and a link renders as clean text with no literal syntax and no block
element in the cell, the title tooltip carries the same stripped text, and
a backend without a description still shows the placeholder.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* ui(models): polish the variant detail view and scope rendered Markdown

The gallery detail pane rendered every field through the same two-column
label/value row, including the description. Multi-paragraph prose in a value
cell ran eight rows tall at the top of the pane on a ~1200px measure, breaking
the grid's rhythm exactly where the eye enters. Move it into its own full-width
block above the table, capped at a 68ch measure, keeping the label.

Rendered Markdown had no scoped typography anywhere in the app, so a
description opening with `#` inherited the browser default 2em inside a 13px
surface while a `##` further down was indistinguishable from body text. Add a
reusable .markdown-body block mapping h1-h6, paragraphs, lists, links, code,
blockquotes, images and tables onto the existing type scale, and apply it to
every renderMarkdown() consumer: the models detail, the backends detail, both
Manage details and the voice library detail.

Rebalance the variants list so the name leads. Backend and size drop from
badge/secondary weight to muted metadata; the FITS badge goes entirely, since
it was true of nearly every row and so said nothing, while the variant that
does not fit keeps a warning badge and a dimmed name. AUTO-SELECTED stays
marked because it answers what a plain Install produces. Rows share the
parent's grid tracks via subgrid so name, backend, size and status line up
down the list instead of raggedly following name length.

Finally, make each variant row actionable. It looked like a list of choices
but was inert text, with per-variant install hidden behind the split-button
chevron elsewhere; each row is now a button onto the existing
handleInstall(modelId, variant) path, with hover, keyboard focus and a
disabled state while an install is in flight.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): collapse the listing to one row per model

The listing supported has_variants=true, which narrowed to entries that
DECLARE variants. With adoption at three entries that showed three rows,
which is useless; it was always a placeholder.

Replace it with the view that is actually useful: the deduplicated
gallery. Show every entry installable in its own right and nothing twice,
which means the parents plus every entry nobody references, and hide only
the builds another entry already offers as a variant, since those are
reachable through their parent.

The parameter is renamed to collapse_variants accordingly: the filter is
no longer a predicate on a row's own metadata but a view over the whole
gallery. Default stays off, so the response with the parameter absent is
unchanged.

VariantReferencedIDs never reports an entry that declares variants of its
own, so parents are always visible. That guarantees every hidden entry
has a visible entry offering it, and no chain can strand a row. Variant
resolution already refuses to install such a reference, but the listing
has to stay coherent in the presence of a gallery that has one rather
than silently swallowing entries. Self-references and dangling references
hide nothing.

The referenced set is computed over the whole gallery rather than over
what the other filters left, so an entry is hidden because a parent
offers it and never because of what the user searched for. The pass is
over metadata already in memory: it resolves nothing over the network and
triggers no variant description or size probe, so the listing's zero-probe
contract still holds.

The UI toggle keeps its behaviour (persistence, page reset, clear
filters) and becomes "One row per model", which says what the user gets.
Its localStorage key moves too, since the stored value meant a different
filter.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): show the collapsed model listing by default

The gallery listing is what a user reaches for to answer "what can I
install". Answering that with several rows for the same model, one per
build, makes the reader do the deduplication the collapsed view already
does, so the collapsed view is the one to land on.

The UI now asks for collapse_variants=true unless the toggle says
otherwise. The server default is deliberately untouched: a request with
the parameter absent still returns the full listing, because other API
clients depend on that response and collapsing it under them would be a
breaking change. Opting out omits the parameter rather than sending
false, so it asks for exactly the listing everyone else gets.

The stored preference changes vocabulary from '1'/'0' to 'on'/'off'. The
previous build wrote it from an effect that runs on mount, so a stored
'0' recorded that the page had been opened rather than that anyone chose
the expanded view, and honouring it would pin every earlier visitor to a
default they never picked. Only the new vocabulary counts as a choice;
a legacy '1' meant the collapsed view and is what the new default gives
anyway, so no earlier deliberate choice is lost.

Collapsing being the default also changes what the empty state may say
about it. An opted-into filter can be named as the cause of an empty
result; a default cannot, so the filters keep the top line and the
collapsed view drops to a hint below it, shown only once filters are
narrowing the set. For the same reason "Clear filters" now restores the
collapsed default instead of switching it off, and the toggle alone no
longer counts as a filter worth offering to clear.

The label stays "One row per model": it describes the view the user is
looking at rather than an action, so it reads the same whether it is
opted into or out of.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* gallery: group alternative builds of the same weights under variants

Sweep the gallery for entries that are alternative builds of the same
weights (different quantization, precision, or runtime format) and declare
them as variants of a single parent row, so the listing offers one row per
model instead of one row per quantization and the installer picks the
largest build that this host can actually run.

41 families over 95 entries, turning 54 entries into variants.

The parent is the bare-named entry wherever one exists, so nothing changes
about what any existing entry installs. Ranking already selects the largest
fitting build regardless of which entry is nominally the parent, so the
parent only decides the pathological case where nothing fits. For the ten
families that have no bare-named entry, the smallest build is the parent,
since that is the one that has to install when nothing fits.

Grouping was verified against the actual model filenames rather than the
entry names alone. Different parameter sizes, languages, finetunes, and
products that merely share a name prefix are left as separate rows: the
qwen3.6 APEX and pi-tune finetunes, the DFlash and MTP speculative-decoding
pairings, English-only versus multilingual Whisper, the QAT versus non-QAT
Gemma 4 weights, and the abliterated FLUX build are all distinct models.

Six parents define YAML anchors that other entries pull in with a merge key,
which would have handed their variants to every merging child. For the two
depth-anything anchors that would have made fourteen unrelated entries
advertise the base model's builds as their own. All 26 merging children
therefore carry an explicit empty variants list, which overrides the merged
key and is equivalent to the key being absent.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): rank model variants by host backend preference

Variant auto-selection filtered candidates by whether their backend can
run on the host, then ranked the survivors by size alone. The backend
never influenced the choice beyond that gate, so a Mac offered both an
MLX build and a llama.cpp build kept neither filtered and installed
whichever was larger, leaving the native accelerated runtime unused. The
same held for CUDA against CPU on NVIDIA and ROCm against Vulkan on AMD.

Rank by the host's backend preference between the fit tier and size: fit
stays a filter, preference decides among the builds the host can equally
hold, and size still separates builds on equally preferred runtimes.

The preference data stays in one declarative table in pkg/system, now
read by a prefix lookup instead of a switch, so adding a capability or
reordering one host's runtimes is a one-line edit and the gallery's
ranking code carries no per-backend branching. MLX joins the metal rule
ahead of metal itself, which is inert for the existing alias-resolution
consumer because no alias group holds a candidate named for mlx.

An unrecognised backend, an unrecognised capability and an absent
preference list all collapse to the previous size-only ordering rather
than erroring or dropping candidates.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): rank variants by engine name, not backend build tag

Variant auto-selection ranked candidates with
SystemState.BackendPreferenceTokens, but that function and the variant
ranker speak different vocabularies.

BackendPreferenceTokens returns BUILD TAGS ("cuda", "rocm", "sycl",
"vulkan", "metal", "cpu"). It exists to match installed backend build
directory names like "llama-cpp-cuda-12" during alias resolution in
ListSystemBackends. Variant ranking instead matches a gallery entry's
`backend:` value, which is an ENGINE NAME: "llama-cpp", "vllm",
"vllm-omni", "sglang", "mlx" and the rest. No engine name in
gallery/index.yaml contains "cuda", "rocm", "sycl" or "vulkan".

preferenceRank matches by substring, so on an NVIDIA host the tokens
[cuda, vulkan, cpu] matched neither "llama-cpp" nor "vllm", every
candidate scored identically and size alone decided. The NVIDIA, AMD,
Intel, darwin-x86 and vulkan rules were all inert. Only metal appeared
to work, and only because the token "mlx" happens to equal an engine
name. The mismatch does not error, it silently deletes the feature.

Separate the two vocabularies. backendBuildTagPreferenceRules keeps the
build tags and its original output for every capability, including
metal, whose "mlx" token is removed again; its alias-resolution consumer
is byte-identical to before. engineNamePreferenceRules is new, holds
engine names, and is read by the new EnginePreferenceTokens, which
HostResolveEnv wires into the renamed ResolveEnv.EnginePreference. Both
tables sit adjacent under one block comment naming each vocabulary and
each consumer, and share one lookup helper so their semantics cannot
drift.

On NVIDIA the order is vLLM, then SGLang, then llama-cpp: vLLM is the
throughput engine and a model published with a vLLM build is published
that way because that build is the one worth running. AMD and Intel get
the same order, since rocm and intel builds of both serving engines
ship. Metal prefers mlx over llama-cpp. Vulkan prefers llama-cpp, the
only LLM engine with a Vulkan build. darwin-x86 and unknown
capabilities are deliberately absent rather than guessed at, degrading
to the size-only ordering that predates preference.

preferenceRank stays generic and names no engine and no capability, so
adding a runtime remains a one-line table edit.

Specs pin the NVIDIA and metal rules through the live table and the real
HostResolveEnv wiring, so emptying the engine table or wiring the build
tag source back in both go red. A regression table asserts
BackendPreferenceTokens' original output per capability, and mirrored
locks assert neither table carries the other's vocabulary.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: record that variant selection ranks by engine before size

A gallery entry can now declare variants, and selection ranks the builds a
host can run by engine preference before size. Nothing told a contributor
adding a backend that engineNamePreferenceRules exists, so a new engine would
silently rank below every known one and lose to whatever build happened to be
larger on hosts where it should have won.

Document the step where a backend is added, warn against the sibling
backendBuildTagPreferenceRules table (build tags, not engine names: the wrong
table matches nothing, scores every candidate equally and disables the
preference without erroring), and index it from AGENTS.md.

Fix the authoring and user docs, which still claimed the largest surviving
build wins. An author grouping builds under one entry has to be able to
predict what a user gets, and size alone no longer decides it.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(cli,mcp): describe variant auto-selection as preference before size

The CLI flag help and the install_model tool schema both still said
auto-selection takes the largest build that runs. Ranking now puts engine
preference ahead of size, so on NVIDIA a vLLM build wins over a larger
llama.cpp one. An assistant reading the old schema would tell users the
wrong thing.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): prefer llama.cpp over GPU serving engines on hosts with no GPU

engineNamePreferenceRules had no row for the "default" capability, which
getSystemCapabilities() returns both when no GPU is detected and when a GPU
is present but under the 4 GiB VRAM floor. A missing row yields an empty
preference list, which preferenceRank reads as "score everything equally",
collapsing variant selection to size alone.

That would be harmless if the hardware filter dropped GPU serving engines on
such a host, but it does not. IsBackendCompatible derives support from the
engine NAME, and "vllm" and "sglang" contain none of the darwin, cuda, rocm
or sycl tokens it keys on, so they fall through to its closing "return true".
A vLLM variant therefore survives on a CPU-only box and wins whenever its
build is the larger of the two on offer: the machine installs vLLM in
preference to llama.cpp.

darwin-x86 had the identical hole. It was documented as a deliberate omission
because nothing accelerates on an Intel Mac, which is true about acceleration
and wrong about consequence: with every engine tied, download size decides.

Add rows for both putting llama-cpp first. The GPU engines are enumerated
behind it rather than left unmatched: an unmatched engine already ranks below
every listed one, so llama.cpp would win either way, but unmatched engines
also tie with each other and let size decide among them. Naming them fixes
that order. MLX is left off the darwin-x86 row on purpose so it ranks last,
since IsBackendCompatible admits darwin-tokened engines on that capability
even though MLX needs Apple silicon.

Preference orders survivors and never filters, so a model published only as a
vLLM build is still installed on a host with no GPU; there is a spec for it.

Surveyed every other value getSystemCapabilities() can return. nvidia, amd,
intel and vulkan have rows; the l4t and cuda-refined values reach the nvidia
row by prefix; "apple" and "" cannot reach the vendor fallthrough because the
darwin and no-GPU branches return earlier. default and darwin-x86 were the
only live holes.

BackendPreferenceTokens and its build-tag table are untouched, and
preferenceRank stays generic, naming no engine and no capability.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* gallery: prefer speculative-decoding builds when they fit

Rank serving features between engine preference and size, so a host that can
hold a DFlash or MTP build of a model's weights installs it instead of the
plain build. Both answer faster for the same output, so whenever one survives
the filters there is no reason to take the plain build.

Precedence is now fit, then engine, then serving feature, then size. Engine
outranks the feature deliberately: a serving feature makes the right engine
faster, it does not make a wrong engine right, so a plain vLLM build still
beats a DFlash llama.cpp build on NVIDIA. Fit outranks both, and a drafter
pairing is strictly larger than the plain build, so the existing size filter
drops it on a host too small for it before this axis is consulted.

The order lives in a third preference table in pkg/system, alongside the build
tag and engine name tables. It is the odd one of the three: not keyed by
capability, because no hardware prefers a plain build over an equivalent
faster one, and matched against whole segments of a gallery ENTRY NAME rather
than as a substring of a backend value. Nothing on a gallery entry declares a
serving feature, and tags are not a usable substitute: gemma-4-e2b-it:sglang-mtp
carries an mtp tag while ornith-1.0-9b-mtp and qwen3.6-27b-nvfp4-mtp carry
none. Entry names are author-supplied free text, unlike the closed engine
vocabulary, so a short marker can turn up inside an unrelated word and whole
segment matching is what keeps smtp-assistant from ranking as an MTP build.
The block comment over the tables now documents all three together and states
what each is matched against; the ranking code names no feature, so adding one
stays a one-line edit to the table.

29c49203b rejected these entries as serving configurations rather than
alternative builds of the same weights. The definition is now "alternative ways
to serve the same model", which includes them, so regroup 14 entries under 12
parents. Judged by the files each entry points at: the qwen3.6, qwen3.5, qwen3
and deepseek pairings are the base GGUF plus a drafter, the gemma-4 QAT MTP
entries are the same QAT weights at a different quantization plus an MTP
drafter, and the two sglang MTP entries describe themselves as the same model
served with speculative decoding. Left separate: qwen3.6-27b-mtp-pi-tune, a
finetune with its own weights, and every entry whose base model LocalAI does
not ship as its own row, which is the whole Qwopus line plus gemmable-4-12b-mtp,
mimo-7b-mtp:sglang and qwen3.5-4b-dflash.

None of the twelve parents defines a YAML anchor, so no variants key can leak
through a merge key and no empty override was needed this time. The index was
edited by line insertion only.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test: check env restore errors in capability and variant specs

errcheck flagged ten unchecked os.Setenv and os.Unsetenv returns in the
specs added while the pre-commit hook was being skipped. Restoring an env
var is exactly the place a silent failure leaks state into the next spec,
so assert on it rather than suppressing the linter.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): make the mtp tag authoritative for serving-feature ranking

Variant auto-selection ranks survivors by fit, then engine, then serving
feature, then size. The serving-feature lookup read only whole alphanumeric
segments of a variant's entry name, because tags were inconsistent: every
dflash entry carried a dflash tag, but only 7 of 20 MTP entries carried an
mtp tag.

Tag the 13 untagged MTP entries, then teach the lookup to read tags as well
as names. A tag is now the authoritative signal and is compared whole and
case-insensitively, which is safe precisely because a tag is a deliberate
declaration rather than free text: there is no word-inside-a-word failure
mode, so the segment splitting the name half needs is unnecessary there.

The name check stays as a fallback rather than being replaced. Switching to
tags only would have regressed the six already-grouped entries on the day it
shipped, and would depend on tagging discipline that does not exist yet.

The lookup still names no feature, so adding one remains a one-line edit to
servingFeaturePreferenceTokens.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): make a declared tag the sole serving-feature signal

Variant auto-selection ranks survivors by fit, then engine, then serving
feature, then size. The serving-feature lookup recognised a speculative build
by either a declared tag or a whole segment of its entry name. Drop the name
half: a tag is now the only signal.

A name is author-supplied free text and a naming convention is not a contract,
so reading a marker out of one infers a capability nobody declared. The gallery
already had the failure in it: the four NVFP4 entries name MTP-bearing weights
while setting no option that enables speculative decoding, and being live
variants they were winning the feature axis without answering any faster.

overrides.options was considered as the replacement and rejected. It carries
spec_type:draft-mtp / spec_type:draft-dflash, which is what actually turns the
feature on, but that spelling is llama.cpp's config vocabulary: ds4 spells the
same feature mtp_path and sglang spells it speculative_algorithm in a
referenced config. Keying a cross-backend ranking decision on one backend's
option syntax would rank the other backends' builds as plain. Options are the
curation-time check instead, and never reach the selection logic.

With no fallback left, tag correctness is load bearing, so audit every entry
against the rule "tagged when the entry configures that feature, in whatever
vocabulary its backend uses". Three entries configure MTP untagged and gain the
tag (hy3, glm-5.2, qwythos-9b-claude-mythos-5-1m, all spec_type:draft-mtp with
no marker in their names). Four carry the tag while configuring nothing and
lose it: qwen3.6-27b-nvfp4-mtp, qwen3.6-35b-a3b-nvfp4-mtp,
qwopus3.6-27b-coder-mtp-nvfp4 and qwopus3.6-27b-v2-mtp-nvfp4, whose only option
is use_jinja:true. The dflash side was checked independently rather than assumed
consistent: all five dflash entries declare spec_type:draft-dflash and all five
are tagged, so it needed no edits.

Four entries keep a tag that a literal spec_type-only reading would strip,
because they configure MTP through a different backend: deepseek-v4-flash-q2-mtp
via ds4's mtp_path/mtp_draft, and the three sglang entries via
speculative_algorithm in their referenced configs. Stripping those would
contradict the reason spec_type was rejected as the signal and would demote four
genuinely faster builds to plain.

The index was edited by line insertion and deletion only, never round-tripped
through a serializer. A resolved-tag diff across all 1272 named entries, taken
after merge keys are applied, shows exactly these 7 changing and no entry
gaining or losing a tag through an anchor.

The two specs that pinned the name fallback are inverted rather than deleted,
since a name silently promoting a build is the regression worth guarding. The
whole-token guard survives on the tag path, where smtp must still not match mtp.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): make deepseek-v4-flash variant targets installable

Clicking install on deepseek-v4-flash failed with "invalid gallery model".
The parent entry is fine, but all four entries it was grouped with declared
neither url: nor config_file:, and applyModel needs one of the two to have
anything to build a config from. They carry urls: (plural), the informational
HuggingFace link list, which is a different field. None of the four was ever
independently installable, so grouping them routed a previously-working
install into a broken entry.

Give each the url: the parent already resolves through. virtual.yaml is a
no-op base, and applyModel passes overrides to InstallModel separately from
the fetched config, so backend: ds4, the parameters and the ssd/mtp options
all still land exactly as authored. This is the same pattern the parent and
many other GGUF entries in the index already use.

Add the lint rule that should have caught this. checkVariantReferences only
proved a target exists and is not itself a parent, which is structural
validity: an entry can exist, declare no variants, and still be
uninstallable. checkVariantTargetsInstallable mirrors applyModel's
precondition instead, and names the parent, the target and the missing
fields, because whoever hits it is reading a gallery entry and has no reason
to know applyModel exists.

The two index-driven resolution specs live in their own Ordered container:
an Ordered container stops at its first failure, so sharing one with the lint
rules let a lint breach skip them silently.

Nine further entries gallery-wide have the same defect and are unrelated to
variants, so they are broken installs that predate this branch. They are left
alone here rather than buried in a regression fix, and widening the rule to
cover every entry is deferred with them so the gate can ratchet up in one
step instead of needing a skip list.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): install entries with no url or config_file on an empty base

applyModel had three branches: fetch a base config from url:, build one from
an inline config_file:, or fail with "invalid gallery model". An entry
declaring neither is now installed on an empty base config, with overrides:
and files: supplying everything.

This is what the ~345 entries pointing at gallery/virtual.yaml were already
getting. That stub is five lines carrying name, description and license.
description and license are overwritten from the gallery entry immediately
after the fetch, and the name never reaches disk because InstallModel prefers
the install name. Crucially applyModel passes model.Overrides to InstallModel
as a separate argument rather than merging it into the fetched config, so
nothing an author writes depends on that base existing. The fetch bought a
round trip to GitHub and nothing else.

That makes f4ef80173 the wrong fix, so it is unwound. The four url: lines it
added to the deepseek-v4-flash variants are reverted: they are a pointless
network fetch now, and the family installs without them.

Relaxing the branch would hide a real authoring mistake, so a payload rule
replaces the base-config rule. An entry with no url, no config_file, no
overrides and no files installs nothing and would leave an empty model
directory while reporting success, so it is refused by name. The caller's
request counts toward the payload, because its overrides and files are merged
into the install exactly as the entry's own are. urls: (plural) is the
informational link list and does not count, which is what the four entries
that shipped broken had and why they were still uninstallable.

checkVariantTargetsInstallable asserted every variant target declares a url:
or a config_file:, which is no longer true and would now reject correct
authoring. checkEntriesInstallSomething pins what survives instead, and covers
every entry rather than only variant targets: the hazard is a half-written
stanza and a parent can be one as easily as a target. The old rule was scoped
to targets precisely because nine unrelated entries would have failed a
gallery-wide version; those nine are valid now, so the deferred ratchet
happens here in one step. 1280 entries, zero violations.

Those nine (aurore-reveil_koto-small-7b-it, lfm2-1.2b, the six liquidai_lfm2
entries and deepseek-v4-pro-q2-ssd) become installable for free. Each carries
overrides: and files:, and one of them is driven through the real install path
in a spec.

The no-fetch spec is paired rather than bare: an assertion that nothing was
fetched proves nothing unless something could have been, so a control runs the
same fixture with a url: pointing at a base config that is not there and
asserts the install fails. Only then does the identical fixture without the
url passing mean the read was skipped.

Follow-up, deliberately not here: the ~345 entries still naming virtual.yaml
can drop their url:. That is 345 index edits with their own risk, and mixing
them in would bury this change.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* ui(models): let search bypass the variant collapse, drop the toggle

The models page collapsed the gallery to one row per model by default and
offered a toggle to see every individual build. Because the collapse composed
with the search term, a build another entry offers as a variant could not be
found by typing its name, so the toggle was the only way to reach those builds
in the UI. A user who typed a name they knew existed got "no models found",
which reads as "that model does not exist".

Collapse is for browsing; search is for finding. An explicit search term now
bypasses the collapse in the listing handler, so a name lookup returns matching
entries whether or not a parent offers them. The term is trimmed once at the
top of the handler, so whitespace is neither a search nor a bypass; previously
an untrimmed blank term also narrowed the listing to whatever contained a
space. Tag and backend deliberately do not bypass: they refine a listing the
user is still reading rather than name an entry already known to exist.

That makes the toggle redundant, so it goes, along with its i18n strings in all
six locales, its localStorage persistence, its participation in "Clear filters"
and the empty-state hint telling users to turn it off. The hint was doubly
stale: it pointed at a control that no longer exists, and it was untrue exactly
when a user has a search term, since searching now sees every build. The page
always requests the collapsed listing.

The stored preference key is left inert rather than cleaned up: nothing reads
it, so a user who had the toggle off simply gets the collapsed view.

collapse_variants stays on the API, off by default, because other clients want
either view and the UI dropping its control is no reason to remove a working
parameter.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): give the models gallery filter form a deliberate structure

The filter area had accreted controls into one undifferentiated flow. The
"Fits in GPU" toggle and the backend select were direct children of
.filter-bar, the same wrapping container as the 18 taxonomy chips, so their
position was decided by how many chips happened to wrap at the current width
rather than by any layout intent. At narrow widths they were pushed past the
right edge of that container's horizontal scroll and became unreachable
entirely.

Restructure into three bands inside the house .filter-bar-group wrapper that
components/FilterBar.jsx already uses on Backends and the System tabs:

  1. query scope: search plus the backend select
  2. taxonomy: the chip row, alone, free to wrap
  3. refinements: fits-in-GPU and context size, under a hairline rule

The backend select leads the chips rather than trailing them because picking a
backend disables the use cases that backend cannot serve, so it gates the row
below it. Fits-in-GPU and context size share a band because they are one
control group: the context size is the length the VRAM estimate is computed at,
and that estimate is what the fits filter tests against.

Chips had no visible keyboard focus indicator. The global focus ring is wrapped
in :where(), so it carries the specificity of a bare :focus-visible, ties with
.filter-btn and loses on source order, leaving focused chips showing their
resting drop shadow. Restate the ring where it outranks both resting and hover.

Also: aria-pressed on the chips, a real label association and aria-valuetext on
the context slider (it steps over an index, so it announced "2"), disabled chip
styling moved off inline styles, a prefers-reduced-motion block for the chip
transition, and the hard-coded English "Context:" moved into all seven locales.

No behaviour change: same filters, same state, same requests. Page reset on
change, localStorage persistence and "Clear filters" verified unchanged.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): let the models recommendations panel fade into the background

The "Recommended for your hardware" strip rendered at full height on every
visit regardless of how many models were already installed, costing 186px at
1600px wide (287px at 1100px, where its cards wrapped to two rows) and pushing
the first gallery row to y=554 / y=703.

Make its prominence track how much the user still needs it. The panel now
defaults to a one-line summary once anything is installed, and both the
collapse choice and the existing dismissal persist:

  collapsed = explicit user choice, if one exists
            : installedCount > 0

The preference is three-valued on purpose. A boolean cannot tell "the user
expanded it" apart from "the user has never chosen", and those need opposite
handling when the installed count later crosses zero: someone who deliberately
opened the panel on an empty instance should not have it collapse out from
under them when their first model finishes installing.

Collapsed keeps the card, icon, title and a suggestion count, so the panel is
recovered by clicking what you are already looking at rather than by hunting.
Expanded is unchanged, because for a user with nothing installed it was never
the problem. Collapsed reclaims 145px at 1600 and 420, and 246px at 1100.

Models.jsx gains a statsLoaded flag: stats initializes to installed:0, so
reading it before the fetch resolves would render expanded and collapse a frame
later, which is exactly the layout shove this removes.

The dismissal key moves to the page's localai-models-* convention; the old
localai_rec_models_dismissed is still read, never written, so an existing
dismissal is honoured rather than resurrected by the rename.

Accessibility: the disclosure is a real button whose accessible name is the
visible title alone, with state on aria-expanded and aria-controls resolving in
both states, because the grid is hidden via the hidden attribute rather than
unmounted. That also keeps the four install buttons out of the tab order while
collapsed. The app's global focus ring applies; no per-component outline is
added, per the warning in App.css. Reveal animates opacity and transform only,
never height, and both it and the chevron rotation are disabled under
prefers-reduced-motion.

Only en had a recommended block, so the other six locales were falling back to
English for the whole panel. Translated the complete block rather than adding
one orphaned key to files that would still render the title in English.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(downloader): recover from a leftover .partial on non-HTTP URIs

An interrupted download leaves a `<file>.partial` behind. The partial
handling in DownloadFileWithContext gated resume on `err == nil &&
uri.LooksLikeHTTPURL()`, so for any URI that is not literally http(s)
the branch fell through to `else if !errors.Is(err, os.ErrNotExist)`,
which with a nil err is true. The download then failed with an error
wrapping nil:

  failed to check file ".../Ternary-Bonsai-27B-Q2_g64.gguf" existence: <nil>

Every gallery file URI uses `huggingface://`, so a single interrupted
download made that model permanently uninstallable until someone
deleted the partial by hand. The `<nil>` in the message compounded it
by pointing debugging at a filesystem failure that never happened.

Restructure the handling as an explicit switch over the four real
states: partial exists and is resumable, partial exists and is not
resumable (discard and restart, as already done for an HTTP server
without range support), no partial, and a genuine stat failure. The
error branch is now only reachable with a non-nil error, names the
path that was actually stat'd, and wraps with %w.

Discarding is required for correctness and not merely convenience: the
writer opens the partial with O_APPEND, so an un-resumed download would
concatenate a fresh body onto stale bytes.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): tell the models gallery's variant rows apart, and let browsing see every build

Both variant surfaces rendered name, backend and size. For two builds of one
model that is close to no information: a variant exists precisely because the
same weights are offered another way, so the backend usually matches and the
sizes usually land within a few hundred megabytes. Comparing
ternary-bonsai-27b-pq2 against ternary-bonsai-27b-q2-g64 meant reading two names
that differ by a suffix nobody has defined anywhere in the UI.

Report the quantization and the serving features on VariantView, and derive both
server-side from the referenced entry rather than parsing names in the browser,
so every client reads the same format out of the same file the installer will
hand the backend.

Quantization comes from overrides.parameters.model first, falling back to the
file list. That order is load bearing: entries routinely ship a vision tower
alongside the language model at a different quantization, so reading the file
list first reports the mmproj's format. Matching walks `-` and `.` delimited
segments right to left; `_` deliberately does not split, because it separates the
parts INSIDE a quant token and splitting on it reports Q4 for a Q4_K_M build. A
second, looser pass takes a segment's `_`-delimited tail, which catches the
gemma-4-E2B_q4_0-it.gguf style; it runs second so a precise match can never lose
to a fuzzy one further right in the name. An entry naming no format reports
nothing, which is the honest answer for a backend served from a directory of
weights.

Features are the same tag-against-vocabulary match servingFeatureRank already
ranks on, over the same host preference list. A build can therefore never be
shown as faster than one selection did not actually reward, nor rewarded without
being shown; a spec pins that agreement rather than trusting it.

The compact dropdown gets the quantization on its meta line and the bare feature
token. The detail row, which has the room, gets the quantization as its own
monospaced column so precision lines up down the list, and the feature spelled
out, because DFLASH names nothing to a user who has not met it. The referenced
entry's description stays out of both: the detail row already renders the
parent's prose above the table, and a second block per variant would push a
three-variant list past a screen to restate what the columns now say precisely.

The collapse toggle comes back. 462583f38 dropped it once search bypassed the
collapse, on the reasoning that nothing was unreachable any more. That holds for
finding a build whose name you know and does not hold for browsing: no sequence
of actions enumerated the 68 builds the default view hides. Collapse is for
browsing and search is for finding, and the toggle was the browsing half.

It goes in the refinements band 0d4823362 established, not back among the
taxonomy chips where its position depended on how many chips happened to wrap. It
leads that band because it decides how many rows the other two refine over, and
because unlike fits-in-GPU it is unconditional: a host with no GPU still browses.

The search bypass is untouched and re-checked by a spec in the toggle's default
state, since restoring the control must not restore the dead end it replaced. The
empty-state hint returns but only without a search term, because a term bypasses
the collapse and the hint would otherwise point at a control that cannot change
the result. The stored preference reads 'on'/'off' only: an older build wrote
'1'/'0' from an effect that ran on mount, so those record that the page was
opened, not that anyone chose a view.

Also fixes a latent flake it exposed. The collapse_variants spec compared whole
response bodies byte for byte, and the listing envelope carries live host
telemetry that drifts between two calls milliseconds apart, so it was asserting
on the machine's memory pressure. It now compares everything the parameter
governs -- the entries, their serialization and the paging -- and is green 25/25
where it was failing about one run in three.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): let the models gallery show a variant's full details

The variant list in an entry's expanded detail row says how the builds
differ: name, backend, quantization, size, and the auto-selected, base
and serving-feature markers. It cannot say what any one of them is. A
variant's own description, tags, license, source links and file list are
unreachable anywhere in the UI, because while the collapse is on a
variant has no gallery row of its own.

Give each variant row an info control that reveals its entry, rendered by
the same ModelDetail a top-level row gets, so a field added to the detail
view appears here too. variantData is withheld from the nested render: a
variant may declare variants of its own, and recursing would nest a
picker inside a picker two levels deep already.

An inline disclosure rather than a modal. The control sits inside a table
row that is already expanded, inside a variant list within that; a dialog
opened from there stacks a dismissal on a dismissal for a handful of
extra fields about the entry the user is already reading, and breaks the
page's own expand idiom. The third level is carried by an inset and a
left rule instead of another card.

The entry is fetched by exact name from the listing, once, on first use.
The listing already returns every field the detail view renders, and a
search term bypasses the variant collapse server-side, so no new endpoint
is needed and neither the listing nor DescribeVariants gains any work.
Expanding a row costs nothing; a variant nobody opens costs nothing. A
name the listing no longer returns is stated, not blanked: an empty panel
reads as a rendering fault rather than as a lookup that came back empty.

The control is a sibling of the install button, not a descendant, so
asking about a build can never install it.

The variant list keeps its content-sized columns via a trailing filler
track instead of max-content sizing, so the rows are unchanged while the
panel spanning them gets the pane width its file table needs.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): let search respect the collapse instead of switching it off

The models listing collapsed to one row per model, and an explicit search term
turned that off wholesale. Searching while collapsed therefore answered with the
individual builds a parent already offers, which are exactly the rows the view
the user asked for has no place for: typing "mtp" returned
qwen3.6-27b-nvfp4-mtp, a row that is invisible the moment the box is cleared.
The bypass was the right shape of fix for the wrong half of the problem. What a
search must not do is answer "no models found" for a build the gallery does
hold; that does not require abandoning the grouping the user asked for.

So the term is now matched against every entry either way, hidden builds
included, and the collapse decides how a match is reported rather than which
matches exist. Collapsing stops being a filter that drops rows and becomes a
substitution: a match on a build another entry offers is reported as that entry,
the one installable in its own right. Nothing becomes unfindable and nothing
comes back that the requested view cannot show.

Substitution happens after search, tag and backend, so every filter is judged
against the build that really carries the name, tag or backend rather than
against a parent that merely offers it; the other order would let backend=vllm
match a parent whose own backend is something else. The price is that the
surfaced row shows the parent's own metadata while the match was on a variant,
which is what grouping means, and the alternative is claiming the gallery holds
no such build. It happens before the count and the page math, so both describe
the rows actually handed out rather than the matches that produced them.

A parent already in the result keeps its own position and absorbs its matching
variants there, which is what leaves the browsing listing ordered exactly as it
was; a parent surfaced only by a variant takes the position of the first variant
that surfaced it. Either way it appears once, however many of its builds matched
and whether or not it matched itself. Search preserves gallery order rather than
scoring, so a surfaced parent has a real position rather than an invented one.

VariantParents never reports an entry that declares variants of its own, so a
parent is never itself hidden and one hop always lands on a visible row. The
handler follows exactly one anyway: refusing the second is what makes a gallery
the linter would have rejected terminate rather than loop.

The empty-state hint pointing at the toggle goes with it for every server-side
filter. Substitution means a match is always reported as some row, so the
collapse can no longer be why a term, a chip or a backend came back empty, and
naming it there sends the user to a control that cannot change the result. It
survives for the fits filter alone, which runs in the browser after the
substitution and judges the surfaced entry's own size: there the build that fits
really can be filtered out along with a parent that does not.

Searching a build's exact name while collapsed now answers with its parent, so
the result no longer contains the string the user typed. That is intended, and
the row is the one they can act on, but it is a real rough edge: nothing on the
row explains the connection. Closing it properly means reporting which variant
matched so the UI can say so, which the listing does not do today.

ResetGalleryModelCache is added for tests. The model cache is a package global
keyed by nothing, so a background refresh one spec triggers can land in the
middle of the next and answer it with the previous spec's gallery; the extra
specs here made that fail about one run in five. It waits for the in-flight
refresh to publish before clearing, since clearing alone only narrows the
window.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-07-20 18:43:02 +02:00
localai-org-maint-botandEttore Di Giacinto 4be6e22b5f feat(webui): surface user, client IP and user agent in API traces (#10886, #10887) (#10907) 
The Operate → Traces "API Traces" panel already recorded who made each
request (user_id/user_name) but never showed it, and did not capture the
caller's network identity at all. Operators asked to see the requesting
user (#10886) and the client IP + user agent (#10887) so a trace can be
attributed to who/what issued it.

Backend: add ClientIP and UserAgent to APIExchange and populate them from
echo's c.RealIP() (honours X-Forwarded-For / X-Real-IP behind a trusted
proxy) and the request's User-Agent header. Both are omitempty and the
/api/traces swagger response is map[string]any, so this is additive.

UI: add a sortable "User" column to the API traces table and a metadata
block (User / Client IP / User Agent) at the top of the expanded row
detail. Fields render only when present, so older buffered traces and
unauthenticated/local requests degrade cleanly.

Adds an e2e spec covering the new column value and the expanded metadata.


Assisted-by: Claude:opus-4.8 [Claude Code]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-07-17 23:47:31 +02:00
LocalAI [bot]andEttore Di Giacinto bcc41219f7 feat: materialize Hugging Face model artifacts (#10825) 
* feat(config): add model artifact source contract

Assisted-by: Codex:GPT-5 [Codex]

* feat(downloader): add authenticated raw-byte progress

Assisted-by: Codex:GPT-5 [Codex]

* feat(huggingface): resolve immutable snapshot manifests

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): add artifact storage primitives

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): materialize pinned Hugging Face snapshots

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): bind managed snapshots at runtime

Assisted-by: Codex:GPT-5 [Codex]

* feat(gallery): materialize model artifacts during install

Assisted-by: Codex:GPT-5 [Codex]

* feat(gallery): declare managed Hugging Face artifacts

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): preload managed model artifacts

Assisted-by: Codex:GPT-5 [Codex]

* fix(gallery): retain shared artifact caches on delete

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): report artifact acquisition progress

Assisted-by: Codex:GPT-5 [Codex]

* refactor(backends): load managed models from ModelFile

Assisted-by: Codex:GPT-5 [Codex]

* refactor(backends): load staged speech model snapshots

Assisted-by: Codex:GPT-5 [Codex]

* refactor(backends): use staged snapshots in engine backends

Assisted-by: Codex:GPT-5 [Codex]

* test(distributed): cover staged artifact snapshots

Assisted-by: Codex:GPT-5 [Codex]

* docs: explain managed model artifacts

Assisted-by: Codex:GPT-5 [Codex]

* docs: add product design context

Assisted-by: Codex:GPT-5 [Codex]

* feat(ui): show model artifact download progress

Assisted-by: Codex:GPT-5 [Codex]

* Eagerly materialize Hugging Face artifacts

Materialize HF-backed model references as managed GGUF artifacts during load, with lazy download retained only as fallback.

Assisted-by: Codex:GPT-5 [shell]

* Refactor HF
  downloads through a shared executor

Assisted-by: Codex:GPT-5 [shell]

* drop

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-07-15 01:09:33 +02:00
Ijasandajuijas a5aa56db81 fix: preserve uploaded file content when regenerating a non-last answer (#10819) 
handleRegenerate rebuilt the outbound message from the display-only
message.files metadata ({name, type: 'file'|'image'|..., content}),
which doesn't carry the base64/textContent payload sendMessage's
file-building loop expects. As a result, regenerating any answer whose
own question had an attachment silently dropped that attachment from
the resent message. This wasn't fork-specific, but forking a chat and
then regenerating an earlier (now non-last) answer is the natural way
to hit it.

Fix by reusing the original message's already-assembled `content`
verbatim (it already has the file text / image_url / audio_url /
video_url parts embedded from the first send) instead of trying to
reconstruct it from lossy display metadata.

Fixes #10806

Assisted-by: Claude:claude-sonnet-5

Signed-off-by: ajuijas <189517297+ajuijas@users.noreply.github.com>
Co-authored-by: ajuijas <189517297+ajuijas@users.noreply.github.com>
 2026-07-14 11:24:42 +02:00
LocalAI [bot]andEttore Di Giacinto 4056283aa4 [voice] feat: add managed voice cloning profiles (#10799) 
* feat(ui): add voice library workflow

Give administrators a production-ready flow to record or upload consented reference audio, manage reusable profiles, inspect API usage, discover compatible models, and hand a saved voice directly to text-to-speech.

Assisted-by: Codex:gpt-5

* feat(voice): add managed voice cloning profiles

Make reusable reference voices manageable through the admin API instead of requiring model-directory and YAML edits. Discover compatible installed and gallery models from server-side backend capabilities, retain explicit model configuration controls, and stage saved references for supported backends.

Expose profile management through REST and MCP, document backend-specific behavior, and cover the workflow from profile creation through real Qwen3-TTS synthesis. Harden the agent-job HTTP test against completion racing cancellation.

Assisted-by: Codex:gpt-5

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-07-13 09:54:46 +02:00
LocalAI [bot]andEttore Di Giacinto b00422e45f feat(backends): add LongCat video and avatar generation (#10792) 
* feat(backends): add LongCat video and avatar generation

Assisted-by: Codex:GPT-5 [apply_patch] [exec_command] [web]

* refactor(config): declare model I/O modalities

Make model configs declare input and output modalities so capability discovery no longer branches on backend or checkpoint names. Complete the LongCat gallery and user documentation, make the SDPA patch apply to the pinned upstream revision, and stabilize the Agent Jobs race exposed by the required hook.

Assisted-by: Codex:GPT-5 [web]

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-07-12 23:58:46 +02:00
LocalAI [bot]andEttore Di Giacinto 237bce48e8 feat(ui): forking chat - retry any answer, copy, duplicate, branch (#10645) (#10654) 
* feat(ui): clone a chat into a new conversation (#10645)

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): retry any assistant answer, not just the last (#10645)

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): copy an entire chat to the clipboard (#10645)

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): branch a new chat from any assistant answer (#10645)

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): send truncated history on mid-conversation retry (#10645)

Mid-conversation retry regenerated an answer with the downstream turns
still in the model's context. handleRegenerate truncated the DOM history
via updateChatSettings (a scheduled state update), but the synchronous
sendMessage that followed read the stale, pre-truncation history from its
closure to build the outbound API payload. Thread the intended base
history explicitly through sendMessage's options.baseHistory so the
request body matches the truncated view. Backward compatible: the normal
send path (no baseHistory) is unchanged.

Also guard two minor issues in Chat.jsx: the "Branch from here" button now
renders under !isStreaming to match the retry button, and the duplicate
toast only fires when forkChat returns a chat (not on a null result).

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-07-03 00:04:44 +02:00
Richard Palethorpe 5d0c43ec6e feat(realtime): Semantic VAD EOU token (#10444) 
* feat(realtime): EOU-driven semantic_vad turn detection

Add a `semantic_vad` turn-detection mode to the realtime API that feeds
the transcription model live and decides "the user finished speaking"
from the `<EOU>` end-of-utterance token rather than from silence alone.
When EOU fires the turn commits immediately (~0.3s); otherwise it falls
back to an eagerness-scaled silence threshold (low/med/high = 8/4/2s).

Plumbing, bottom to top:

- proto: `AudioTranscriptionLive` bidirectional RPC (config-first oneof,
  mono float PCM @16k, ready-ack / Unimplemented degrade signal) plus
  `TranscriptResult.eou` for the unary retranscribe gate.
- pkg/grpc: client/server/base/embed scaffolding for the bidi stream,
  modeled on AudioTransformStream; release stream conns on terminal Recv.
- parakeet-cpp: live transcription RPC with per-C-call engine locking
  (one live stream per turn, finalize+free at commit); bump parakeet.cpp
  to ABI v5 — incremental StreamingMel (no more quadratic per-feed mel
  recompute that delayed EOU on long turns) and the <EOU>/<EOB> split;
  strip the literal <EOU>/<EOB> from offline text and set Eou.
- core/backend: LiveTranscriptionSession wrapper + pipeline
  `turn_detection:` config block (type/eagerness/retranscribe).
- realtime: semantic_vad integration — live input captions streamed as
  transcription deltas while the user speaks, EOU-immediate commit with
  eagerness fallback, optional retranscribe gate (batch re-decode must
  also end in <EOU> to confirm), clause synthesis off the LLM token
  callback, and per-turn live-transcription / model_load telemetry.
- UI: show the realtime pipeline components as a vertical list.

Docs and tests included; opt-in via the pipeline YAML or per-session
`session.update`. Non-streaming STT backends degrade to silence-only.

Assisted-by: Claude Code:claude-opus-4-8 [Read] [Edit] [Write] [Bash]
Assisted-by: Claude Code:claude-fable-5 [Read] [Edit] [Bash]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(realtime): explicit formally-verified state machines + parakeet streaming driver

The realtime API had several implicit state machines whose state was inferred
from scattered booleans, channels, and five separate mutexes, leaving
illegal/inconsistent states reachable. Make them explicit and keep the
implementation in step with a formal design; rework the parakeet streaming
backend along the same lines.

Realtime state machines (M1-M5). Each is a sealed sum-type State/Event/Effect
with a total, pure Next(state,event)->(state,[]effect) behind a single-writer
Coordinator:

  M1 conncoord    connection lifecycle: VAD toggle + once-only teardown
                  (replaces vadServerStarted + a `done` channel closed from
                  two sites).
  M2 turncoord    turn detection: collapses speechStarted and the live-stream
                  "turn open" flag into one state, so discardTurn can no longer
                  desync them and suppress the next onset.
  M3 respcoord    response coordination: serializes the dual-writer
                  start/cancel so at most one response is live; one
                  response.done per response.create.
  M4 compactcoord conversation compaction: single-flight (replaces the
                  `compacting atomic.Bool` CAS).
  M5 ttscoord     TTS pipeline: open->closing->closed, idempotent wait(),
                  rejects enqueue-after-close (was a silent drop).

The Coordinator/Sink/Next plumbing — only the sealed types and Next differed
per machine — is extracted once into core/http/endpoints/openai/coordinator as
a generic Coordinator[S,E,F]; each machine keeps its public API via type
aliases, so no sink, call-site, or test moved.

Hierarchy. session_lifecycle.fizz models M1 as the parent region with its
children (M2/M3/M4) as one statechart and asserts ChildrenDieWithParent (conn
torn => all children terminal, none start after teardown). respcoord and
compactcoord gain an absorbing Terminated state + Shutdown event; conncoord's
teardown drives the children terminal. This closes a compaction teardown gap: a
fire-and-forget compaction could outlive a torn session — compactionSink now
takes a session-scoped cancellable context + WaitGroup and joins the in-flight
summarize+evict on shutdown.

Formal verification. formal-verification/ holds one authoritative FizzBee spec
per machine plus the composition spec, each with an always-assertion and a
documented one-line edit that makes the checker fail (verified non-vacuous).
scripts/realtime-conformance.sh is fail-closed: all Go conformance suites under
-race AND a model-check of every .fizz spec; a missing FizzBee is a hard error
(only the loud REALTIME_CONFORMANCE_SKIP_FIZZBEE=1 bypasses it, never in CI).
FizzBee is pinned by sha256 and installed via scripts/install-fizzbee.sh into
.tools/ (gitignored). Wired as make test-realtime-conformance, a CI workflow,
and a pre-commit path filter. Go conformance tests are Ginkgo/Gomega (per the
repo's forbidigo lint): transition tables + fixed-seed property walks +
concurrent/-race specs, no rapid dependency. Design map:
docs/design/realtime-state-machines.md.

Parakeet streaming backend. The same treatment applied to the parakeet-cpp
streaming paths:
- AudioTranscriptionStream returns codes.Unimplemented for non-streaming models
  instead of decoding offline and emitting it as one delta + final. A client
  that asked for streaming learns the model cannot stream rather than receiving
  a batch result shaped like a stream. New grpcerrors.StreamTranscriptionUnsupported
  carries that signal; the HTTP /v1/audio/transcriptions stream path surfaces it
  as an SSE error event. Mirrors AudioTranscriptionLive, which already did this.
- utteranceBoundary (boundary.go): a single definition of the end-of-utterance
  latch, replacing three open-coded finalEou toggles. Modelled as a two-valued
  type so illegal states are unrepresentable.
- Shared decode driver (driver.go): streamFeedResult (one per-feed event) +
  feedChunk (hides the ABI v4 JSON vs text-only split) + feedSlices + flushTail.
  The feed loop is written once.
- AudioTranscriptionLive becomes a bidi adapter: it streams the per-feed
  {delta,eou,eob,words} the realtime turn detector consumes and a terminal
  FinalResult carrying only Text. Segments/duration/eou are offline-only and no
  longer produced (nor read) on the live path; liveTraceState drops the terminal
  eou and keeps the per-feed eou_events count.
- AudioTranscriptionStream + streamJSON merge into one driver-based function;
  streamSegmenter is generalized to the unified event with a text-only fallback
  that preserves the legacy (no-words) library's per-utterance segmentation.

Verified: build/vet/gofumpt clean, golangci-lint 0 issues, all coordinator and
parakeet packages under -race, the fail-closed conformance gate green, and
make test-realtime (12 e2e WS+WebRTC).

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2026-06-30 09:01:22 +02:00
Ettore Di Giacinto 4755d676a3 Revert "feat(ui): role and deployment-mode adaptive UI (landing, sidebar, top navbar)" (#10453) 
Revert "feat(ui): role and deployment-mode adaptive UI (landing, sidebar, top…"

This reverts commit 9d54a599b0.
 2026-06-22 21:59:05 +02:00
LocalAI [bot]andEttore Di Giacinto 9d54a599b0 feat(ui): role and deployment-mode adaptive UI (landing, sidebar, top navbar) (#10449) 
* feat(ui): add shared DeploymentContext (features + p2p signal)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* refactor(ui): extract launchAssistantChat shared helper

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): role/mode-aware landing redirect at /app

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): pin Cluster group and collapse Create for cluster admins

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): desktop top navbar with mode pill and admin-via-chat jump

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): admin token-usage meter in the top navbar

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): top-navbar breakpoint handoff + assistant jump from chat page

M1: the desktop .top-navbar was hidden at max-width 768px while the
.mobile-header only appears at max-width 639px, leaving 640-768px with
neither bar so admins lost the mode pill, token meter and admin-via-chat
jump. Hide the top bar at 639px instead so it covers every width the rail
sidebar is shown and hands off to the mobile-header exactly at 639px.

M2: the navbar 'Admin via chat' button wrote localStorage and called
navigate('/app/chat'), but when already on the chat page Chat does not
remount so its mount-time payload reader never fired and the click was a
no-op until reload. The payload consume logic is factored into a shared
callback; the launcher now dispatches a localai-open-assistant event that
the mounted Chat listens for to re-consume the payload. Mount behavior is
unchanged.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-06-22 21:27:43 +02:00
Richard Palethorpe 63bcbf6c12 fix(pii): post-merge review fixes + live NER e2e for the privacy-filter tier (#10401) 
* fix(pii): post-merge review fixes + live NER e2e for the privacy-filter tier

Follow-up to the NER tier engine (#10360), already on master. This carries
only the incremental review fixes and tests that postdate that merge — the
feature itself is not re-introduced.

Review fixes:
- openai_completion.go: remove the dead `elem >= 0` conjunct in applyAnyText
  (the `elem < 0` guard above already returns).
- application.go: collapse ResolvePIIPolicy's inline re-implementation of
  PIIIsEnabled to a single cfg.PIIIsEnabled() call (sole source of the
  "explicit pii.enabled wins, else cloud-proxy default" rule) and return true
  past the !enabled guard where it is provable.
- pattern.go: hoist the triple `appConfig != nil && EnableTracing` check in
  patternDetector.Detect into one local.
- grammar.go: MaxQuantifier was 4096, but Go's regexp/syntax rejects repeat
  bounds above 1000 at Parse time, so walk()'s {n,m} guard could never fire —
  dead code shadowed by the parser. Lower it to 512 so a bound in (512,1000]
  is rejected here with an actionable error; >1000 still fails closed via
  Parse. Specs pin the relationship so the guard can't silently revert.
- PatternListEditor.jsx: clamp a directly-typed negative min_len to >=0 and
  force the DOM value back when clamping (min={0} only constrained the spinner,
  so a negative reached saved config and silently disabled the length filter).

Tests:
- piipattern_test.go: MaxQuantifier guard specs (must stay live, not dead).
- model-config.spec.js: assert the min_len clamp, and that entity_actions
  collapses a duplicate group to a single row (map semantics; regression guard
  against emitting an array that drops a row on save).
- tests/e2e-backends: token_classify capability driving the TokenClassify gRPC
  RPC against the backend image, asserting byte-correct, UTF-8 rune-aligned
  spans (entity.Text == text[start:end]) at threshold 0. Verified on CPU via
  `make test-extra-backend-privacy-filter` (3/3 specs).
- Makefile: test-extra-backend-privacy-filter wrapper.
- tests/e2e: e2e_pii_ner_test.go drives /api/pii/analyze + /api/pii/redact
  (mask + block) through the full HTTP -> detector -> redactor path; gated on
  PII_NER_MODEL_GGUF so the default suite is unaffected.
- .github/workflows/tests-pii-ner-e2e.yml: path-filtered / nightly CI job
  running the container harness on CPU.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(gallery): add privacy-filter-nemotron (f16 + q8)

GGUF conversions of OpenMed/privacy-filter-nemotron — a fine-grained English
PII token-classifier (55 categories / 221 BIOES classes), fine-tuned from
openai/privacy-filter on NVIDIA's Nemotron-PII dataset. Sibling to the existing
privacy-filter-multilingual entry, trading language breadth for category depth.

- privacy-filter-nemotron: F16 reference artifact (~2.8 GB).
- privacy-filter-nemotron-q8: Q8_0 quant (~1.64 GB) for RAM-constrained / edge
  use; description notes the size/speed tradeoff and to validate on your own
  data (a single dropped span is a PII leak).

Both run on the privacy-filter backend with known_usecases [token_classify] and
a default mask policy (min_score 0.5); operators add per-category entity_actions
as needed. sha256s taken from the HF repo's LFS object ids.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2026-06-22 18:26:19 +02:00
LocalAI [bot]andEttore Di Giacinto 95b058e1c5 feat(ui): restructure Cluster Nodes view (pulse + panel roster + detail page) (#10447) 
* chore: gitignore SDD scratch directory

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* feat(nodes): add GET /api/nodes/models cluster-wide loaded-models endpoint

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* feat(ui): add nodesApi.allModels() for cluster-wide model roster

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* feat(ui): move Scheduling to its own page and nav item

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* feat(ui): replace nodes stat-card strip with cluster pulse + attention callout

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* feat(ui): node-panel roster with inline model chips and segmented filter

Replace the Nodes table with a full-width node-panel roster that shows
each backend node's running-model chips without an expand click, plus an
All/Backend/Agent segmented filter. Per-node detail (models, backends,
labels, capacity) moves to the node detail page.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* feat(ui): add deep-linkable node detail page at /app/nodes/:id

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* fix(ui): remove em-dash from CapacityEditor comment; align detail spec backend mock

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* chore(ui): nodes page cleanup, hover/chip polish, docs for restructured cluster view

Nodes.jsx dead-code sweep confirmed clean (no StatCard/table/expand
state/scheduling-form leftovers). Two App.css polish fixes: move the
node-panel hover border-color onto the bordered element so hover gives
real feedback, and add the missing .model-chip__state rule the
ModelChip component already emits. Update distributed-mode docs prose to
describe the restructured cluster view (cluster pulse, attention
callout, node-panel roster with inline model chips, All/Backend/Agent
filter, node detail page at /app/nodes/:id, Scheduling as its own page).

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* chore(ui): drop unused gpuVendorLabel export from nodeStatus

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-06-22 18:24:29 +02:00
LocalAI [bot]andEttore Di Giacinto 3e96d811b7 fix(ui): keep row action menu anchored and stop scroll snap on /app/manage (#10419) 
Opening a model row's kebab (ActionMenu) on the Manage dashboard snapped the
page scroll to the top and rendered the menu detached from its trigger, making
it impossible to operate.

Two compounding causes:

- The menu auto-focus called el.focus() without preventScroll, so the browser
  scrolled the focused element into view, yanking the page to the top.
- The position:fixed Popover was rendered inline inside the table row. The
  editorial UI overhaul added hover transforms to rows/cards, and a transformed
  ancestor re-anchors position:fixed to itself instead of the viewport, so the
  menu (positioned from the trigger's viewport rect) landed in the wrong place.

Fix: portal the Popover to document.body so position:fixed always resolves
against the viewport, position it before paint with useLayoutEffect (no {0,0}
flash), and pass preventScroll:true to both focus calls.

Adds an e2e regression test that reproduces the symptom (scroll jumped from 564
to 0 on the old code) and asserts the menu tracks its trigger.


Assisted-by: Claude:claude-opus-4-8 [Claude Code]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-06-20 23:25:29 +02:00
LocalAI [bot]andEttore Di Giacinto 9565db5f94 feat(models): model aliases - redirect a model name to another configured model (#10414) 
* feat(config): add model alias field and self-validation

Add ModelConfig.Alias (yaml: alias), IsAlias(), and an alias
short-circuit at the top of Validate() that rejects self-reference and
forbids setting backend/parameters.model on a pure-redirect alias.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(config): resolve and validate model alias targets in the loader

Assisted-by: Claude:opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(middleware): resolve model aliases and stamp requested/served identity

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(modeladmin): reject alias configs with invalid targets on create/edit

Validate alias targets at create/swap entry points (ImportModelEndpoint,
EditYAML, PatchConfig) so a dangling, chained, or disabled alias target is
rejected at save time rather than surfacing as a runtime error.

Assisted-by: Claude:opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(api): add GET /api/aliases to list model aliases

Adds an admin-gated read-only endpoint that lists every model alias
config as {name, target} pairs, backed by the loader's existing
GetAllModelsConfigs().

Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(mcp): add set_alias and list_aliases tools

Expose model-alias management over the LocalAI Assistant MCP surface:
list_aliases (read-only, GET /api/aliases) and set_alias (mutating).
SetAlias is swap-first: PATCH /api/models/config-json/:name swaps an
existing alias's target (validated, non-destructive) and a 404 falls
back to POST /models/import to create a fresh {name, alias} config. The
inproc client mirrors this via ConfigService.PatchConfig + a create path
modeled on ImportModelEndpoint. Deletion reuses delete_model.

Assisted-by: Claude:claude-opus-4 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* style(mcp): replace em dashes in alias tool comments

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(config-meta): expose alias as a model-select field

Add an 'alias' section to DefaultSections() and an 'alias' field override
in DefaultRegistry() so the schema-driven React editor renders the new
top-level ModelConfig.Alias field as a model picker in its own section.

Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): add alias template card and Manage alias badge

Add an 'Alias / Routing' template to the create-flow gallery that seeds a
minimal name + alias config, and a read-only 'alias -> target' badge on the
Manage Models tab. The capabilities row payload does not carry the alias
field, so the badge resolves targets from GET /api/aliases looked up by name.

Assisted-by: Claude:claude-opus-4 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: document model aliases

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs(swagger): regenerate for GET /api/aliases

Adds the /api/aliases path and AliasInfo schema generated from the
ListAliasesEndpoint annotation.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(localai): check os.RemoveAll error in aliases_test

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix: correct alias conversion docs and advertise /api/aliases in instructions

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(mcp): write alias config 0600 to satisfy gosec G306

The inproc createAlias path wrote the alias YAML with 0644, which gosec
flags as a new G306 finding on the PR. The LocalAI process is the sole
reader/writer of model configs, so 0600 is correct and keeps the scan clean.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-06-20 22:38:42 +02:00
LocalAI [bot]andEttore Di Giacinto f68edfc85f feat(ui): editorial UI/UX overhaul - design language, shell/nav, conversation/canvas, sub-menus (#10390) 
* feat(ui): add Fraunces variable serif + --font-serif token

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): serif display tier + section-heading typography scale

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): un-overload accent — nav rail, stronger focus ring, neutral hover

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): orchestrated page reveal + stagger motion primitives

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* refactor(ui): fix dead token refs + dedupe toggle to one primitive

Migrate all .toggle-slider consumers (Users, Chat, AgentChat) to the
canonical BEM toggle primitive and delete the legacy duplicate CSS block.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* refactor(ui): route boot fallback through the LoadingSpinner primitive

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): EmptyState primitive with serif title

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): Skeleton shimmer primitive

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): PageHeader + SectionHeading editorial primitives

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): StatusPill primitive + time-of-day greeting helper

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): Home editorial header + status line (north-star redesign)

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): Home loaded-models skeleton list, button hierarchy, EmptyState wizard

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): single focus ring (no double-ring) + neutralize stagger delay under reduced motion

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* refactor(ui): all-sans editorial headings + tint-only active nav

Per design review, pivot the heading strategy from hybrid-serif to a
refined grotesk: drop the Fraunces dependency, token, and import; page
titles, the Home greeting, and section/empty-state titles now use Geist
at semibold with the editorial fluid sizing and tight tracking. No serif
anywhere.

Active sidebar item is now a tint-only treatment (accent text + tinted
background); the left accent rail is removed and the shared base
.nav-item.active inset bar is suppressed in the sidebar (as the console
rail already does). Update the design-system e2e specs to assert the
sans display font and the tinted-background active state.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(e2e): add --host flag to ui-test-server

Allow binding the e2e/preview server to an arbitrary address (e.g.
0.0.0.0 to review the UI from another device on the LAN). Defaults to
127.0.0.1 so existing e2e behavior is unchanged.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* refactor(ui): declutter Home - discoverable + dismissable API, vertical balance

Home felt overloaded and top-heavy. Three changes from review:
- The API endpoint catalog (12 endpoints) is collapsed by default behind a
  "Browse the API" disclosure; only the base URL + copy stay visible, so the
  catalog is discoverable without dominating the page.
- The whole connect card is dismissable (x): dismissing unmounts it so the
  vertical space is recovered, and the choice is remembered (localStorage).
- .home-page now fills its column and vertically centers its content when
  there is slack, so sparse states (no models / card dismissed) read as a
  balanced launcher instead of content jammed at the top. Overflow-safe:
  tall content flows from the top and scrolls.

Adds connect.browse / connect.hide / connect.dismiss i18n keys to all locales.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): editorial PageHeader with section eyebrow + scroll-to-top on nav

PageHeader now derives its eyebrow from the route's section/console (Build /
Operate / Create) via sectionKeyForPath, so pages get a consistent, meaningful
eyebrow with no per-page wiring (override with the eyebrow prop, suppress with
eyebrow={null}). Settings adopts it as the first consumer.

Also fix a navigation scroll bug: the default layout uses the document as its
scroll container and route changes did not reset it, so navigating the console
rail from a scrolled page landed mid-view. App now scrolls to top on pathname
change.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* refactor(ui): adopt PageHeader on agent/media/import/backend pages (batch A)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]

* refactor(ui): adopt PageHeader on ops/admin/media pages (batch B)

Replace hand-rolled .page-header title blocks with the shared editorial
PageHeader component across 14 pages (Manage, Middleware, Models,
NodeBackendLogs, Nodes, P2P, SkillEdit, Skills, Sound, Traces, TTS, Usage,
Users, VideoGen). Title/subtitle move into PageHeader; header-own action
clusters (Models stats+buttons, Skills search+buttons) move into the actions
slot. Tabs, filters, stat cards, ResourceMonitor and page body stay as
siblings. Eyebrow is left to auto-derive from the route.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): home greeting asserts sans font, not the dropped serif

The greeting render-smoke still asserted Fraunces; update it to assert the
Geist sans display font (and not Fraunces), matching the all-sans direction.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): ThemeToggle i18n + animated icon, drop transition:all

The theme toggle hard-coded its English tooltip; route it through the existing
nav switchToLightMode/switchToDarkMode keys and add an aria-label. The sun/moon
icon now replays a small rotate+fade on theme change (keyed remount; honored by
the global reduced-motion block). Replace the .theme-toggle `transition: all`
with explicit properties.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): canvas drag-to-resize + slide-in, fix hooks order, typed download

Canvas was a fixed pane; make it a workbench:
- Drag the panel's left edge to resize (clamped 360px..75vw), persisted to
  localStorage, double-click to reset; hidden and full-width on narrow screens.
- Slide-in/fade on open via canvasSlideIn (honored by reduced-motion).
- Fix a rules-of-hooks bug: the `if (!current) return null` early return sat
  above useEffect, so the hook count changed when artifacts emptied. All hooks
  now run unconditionally before the guard.
- Downloads use the artifact language's real extension + MIME (a Python
  artifact saves as .py, not .txt) via extensionForLanguage.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): per-message code blocks get a language header + copy button

Chat code blocks now render inside a framed block with a header showing the
language and a copy button (delegated handler, copies the block and flips to a
check briefly). Decoration + highlighting run from a MutationObserver scoped to
the messages container, which fires reliably for streamed responses AND for
chats loaded/switched from storage - the prior render-keyed effect missed the
load path (code was left unhighlighted on reload). The observer disconnects
while mutating so it does not retrigger on its own edits.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): image attachments show a thumbnail in the composer

Staged image attachments now preview as a 28px thumbnail (from their data URL)
instead of a bare file icon; other types keep the icon. File names truncate and
the remove button gets an aria-label.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): jump-to-latest pill when scrolled up in chat

When the user scrolls away from the bottom of a conversation, a floating
"Jump to latest" pill appears (sticky, centered above the composer); clicking
it smooth-scrolls to the newest message and re-pins auto-scroll. Resets on
chat switch. Adds the chat.actions.jumpToLatest i18n key to all locales.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): canvas fullscreen toggle + keyboard tab navigation

The canvas header gains a fullscreen toggle (expands the panel to cover the
viewport; resize handle hidden while fullscreen). The artifact tab strip is now
a proper ARIA tablist with roving tabindex and Left/Right arrow-key navigation.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): image result lightbox (zoom, prev/next, download, keyboard)

Generated/history images on the Image page are now clickable, opening a
fullscreen Lightbox with a download button, prev/next navigation, an N/M
counter, and keyboard control (Esc to close, Left/Right to navigate). Adds a
reusable `Lightbox` component (usable later for Video) and the media.image
.actions.view i18n key.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): generation progress with placeholder tiles + elapsed timer

Image generation replaces the bare spinner with a GenerationProgress scaffold:
shimmer placeholder tiles matching the requested count plus a live elapsed-time
readout, so the (often slow) wait feels accountable. Reusable for the other
media generation pages.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): generation progress on Video, TTS, and Sound pages

Reuse GenerationProgress (placeholder tile + elapsed timer) in place of the
bare spinner on the remaining media generation pages, so every slow generation
gives the same accountable feedback.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): agent chat gets per-message code-copy + reliable highlighting

AgentChat now shares Chat's code-block treatment: it runs highlightAll +
enhanceCodeBlocks from a MutationObserver on its messages container (the same
proven path), so agent responses get language headers, copy buttons, and
highlighting that fires for both streamed and loaded messages - closing the
divergence with the main chat without a large refactor.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): Talk voice visualizer

Add a hero frequency-bar visualizer at the top of the Talk page so users get
ambient feedback that they are heard and that the assistant is speaking - the
audit's main Talk gap (the only prior feedback was a small status pill; the
waveform was buried in the dev diagnostics panel).

VoiceVisualizer is self-contained: it builds its own AudioContext + analysers
from the output <audio> stream (speaking) and the mic stream (listening) so it
does not touch the existing WebRTC/diagnostics graph. Bars are status-tinted
(idle/connected/listening/speaking/error) and animate with a gentle idle wave
when not connected. Live mic/output animation is exercised on a real session.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-06-18 23:03:27 +02:00
Richard Palethorpe 3fa7b2955c feat(pii): NER tier engine — privacy-filter.cpp backend + NER-centric PII filter (#10360) 
Squashed feat/pii-ner-tier-engine rebased onto master (was 45 commits; see
backup/pii-ner-tier-engine-prerebase). Net change:

- privacy-filter.cpp: standalone GGML engine for the openai-privacy-filter
  PII/NER token classifier, wired as a LocalAI gRPC backend (CPU/CUDA/Vulkan).
  TokenClassify moves off the patched llama.cpp path onto this backend.
- PII filter reworked to be NER-centric (encoder/NER detection tier scanning
  whole conversations as one document), with a recreated bounded restricted-
  regex secret-matching pattern detector tier alongside it (per-model
  pii_detection.builtins / .patterns + core/services/routing/piipattern).
- Detection labelled by source (ner vs pattern); backend trace / confidence /
  debug observability; analyze/redact exposed as a synchronous API.
- Instance-wide default detector policy + per-usecase default-on; request
  filtering extended to completions, embeddings, edits & Ollama.
- React UI: NER-centric PII editor, detector-models table, pattern/builtins
  editor, middleware default-policy UI.
- Gallery: privacy-filter-multilingual token-classify model + NER install
  filter; token_classify known_usecase; batch sized to context for NER models.
  privacy-filter backend registered in the backend gallery (cpu/vulkan/cuda-13
  meta + image entries with a capabilities map) matching its CI matrix jobs,
  and an /import-model auto-detect importer (PrivacyFilterImporter, narrow
  privacy-filter GGUF detection) replacing the prior pref-only registration.

Reconciled against master's independent evolution:

- Dropped master's PIIPatternOverrides feature (global-pattern runtime
  overrides + /api/pii/patterns API + runtime_settings.json persistence). The
  per-model NER + pattern-detector design supersedes it; it was built on the
  global redactor pattern set this branch replaced.
- Reverted the llama.cpp Score carry-patch (0006-server-task-type-score):
  removed the patch and restored master's grpc-server.cpp Score RPC (direct
  llama_decode, slot-loop bypass) and LLAMA_VERSION pin, plus master's
  model_config validation forbidding score + chat/completion/embeddings on
  llama-cpp. token_classify is unaffected (it runs on the privacy-filter
  backend, not llama-cpp).

Assisted-by: Claude:claude-opus-4-8 [Claude Code]

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2026-06-18 11:45:22 +01:00
LocalAI [bot]andEttore Di Giacinto 5ac864dbed feat(ui): console-based navigation + drop-in API endpoint section (#10377) 
* feat(ui): restructure sidebar into Create/Recognition/Build tiers

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): preserve exact sidebar gating for agent items and fine-tune/quantize

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* i18n(ui): add nav tier + console keys to all locales

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): add grouped admin console via pathless layout route

Wrap the existing admin pages in a pathless AdminConsoleLayout route so
they keep their exact flat URLs while gaining a grouped left rail
(Inference / Cluster / Observability / Access / System). Rail item gating
mirrors the sidebar (adminOnly / authOnly / feature + /api/features). The
layout forwards the App-level outlet context (addToast) to the wrapped
pages, which read it via useOutletContext().

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): fold Audio Transform into Studio as a tab

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(ui): update e2e specs for tiered nav + admin console

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): gate embedded Studio transform view on audio_transform feature

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): visual polish + console-ize Build/Recognition tiers

Generalize the one-off admin console into a reusable ConsoleLayout driven by
a shared consoleConfig (single source of truth for the rail, its gating, and
the sidebar entry that opens it — removes the prior rail/sidebar drift).

- Promote Install Models to the top menu next to Home.
- Build and Operate are now console tiers (secondary rail); Create stays inline.
- Fold Recognition (Faces/Voices) into the Build console as a group alongside
  Automation and Training so it no longer feels split off.
- Style the console rail as a panel (header, grouped dividers, rounded active
  pills) with a hover nudge; sidebar items become inset rounded pills. The rail
  slide-in plays only when entering a console, not on item-to-item sub-nav
  (which remounts the layout), so switching no longer flashes the menu. All
  token-based (light + dark), respects reduced-motion.
- Add a delayed RouteFallback loader so lazy routes no longer flash blank;
  scoped inside ConsoleLayout so the rail stays put while the body loads.
- Update e2e specs for the new structure (.console-* classes, console entries).

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): persist console layout across sub-nav + add drop-in endpoint section

- Keep the page-transition key stable within a console (derived from the
  shared console config) so the ConsoleLayout and its rail persist across
  item-to-item navigation instead of remounting — fixes the submenu flash.
  Cache /api/features across mounts and play the rail entrance animation only
  when actually entering a console.
- Add a "One endpoint, every API" section to Home: leads with LocalAI's own
  native API (images, video, realtime voice over WebRTC/WS, depth, object
  detection, rerank, audio/TTS, face & voice recognition) plus a Full API
  reference link, then the drop-in compatibility layer (OpenAI, Anthropic,
  Ollama, OpenAI Responses) with the live copyable base URL. All 7 locales.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): revert Middleware nav label rename (keep Middleware in all locales)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-06-18 00:09:17 +02:00
Dedy F. SetyawanandEttore Di Giacinto 9ba8521e7e feat(react-ui): localize models and fix 'Import' typo (#10341) 
* feat(react-ui): localize SearchableSelect component

Signed-off-by: Dedy F. Setyawan <dedyfajars@gmail.com>

* feat(react-ui): localize ModelSelector component

Signed-off-by: Dedy F. Setyawan <dedyfajars@gmail.com>

* fix(react-ui): dynamically localize back navigation caption to match page title

Signed-off-by: Dedy F. Setyawan <dedyfajars@gmail.com>

* feat(react-ui): localize back navigation state on Models page

Signed-off-by: Dedy F. Setyawan <dedyfajars@gmail.com>

* feat(react-ui): localize ModelEditor page

Signed-off-by: Dedy F. Setyawan <dedyfajars@gmail.com>

* fix(react-ui): fix Indonesian typo 'Import' to 'Impor' in importModel locale

Signed-off-by: Dedy F. Setyawan <dedyfajars@gmail.com>

---------

Signed-off-by: Dedy F. Setyawan <dedyfajars@gmail.com>
Co-authored-by: Ettore Di Giacinto <mudler@users.noreply.github.com>
 2026-06-15 18:26:27 +02:00
Richard Palethorpe 085fc53bbc fix(router): production-ready request router + auto-size batch for embedding/rerank (#10104) 
* fix(router): score classifier production-readiness

Conversation trimming runs through the classifier model's chat template
and trims by exact token count, sized to the model's n_batch which is
now scaled to context so long probes can't crash the backend. Missing
chat_message templates are a hard error at router build time. Router-
facing factories (Embedder/Scorer/Reranker/TokenCounter) re-resolve
ModelConfig per call so a model installed post-startup doesn't bind a
stub Backend="" config and silently fall into the loader's auto-
iterate path.

New 'vector_store' backend trace recorded inside localVectorStore on
every Search/Insert — including the backend-load-failure path that
previously vanished into an xlog.Warn — with outcome tagging
(hit/miss/empty_store/backend_load_error/find_error/insert_error/ok).
Companion cleanup drops misleading similarity:0 and input_tokens_count:0
from non-hit and text-mode traces.

Gallery local-store-development aliases to 'local-store' so the master
image satisfies pkg/model.LocalStoreBackend lookups from the embedding
cache.

Misc: llama-cpp TokenizeString reads the correct 'prompt' JSON key
(the original bug); ModelTokenize nil-guard; non-fatal mitm proxy
startup; PII 'route_local' renamed to 'allow' with docs/UI in sync;
model-editor footer no longer eats the edit area on small screens;
several config-editor template/dropdown/section fixes.

Tests: e2e router specs (casual/code-hint + long-conversation trim),
vector_store trace specs, lazy-factory specs, gallery dev-alias
resolution, Playwright trace badge + scroll regression.

Assisted-by: Claude:claude-opus-4-7 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(backend): auto-size batch to context for embedding and rerank models

Embedding and rerank models pool over the whole input in a single physical batch (n_ubatch). With batch left at the 512 default, the backend rejects longer inputs with "input is too large to process", silently capping a large-context embedder (e.g. 8k/32k) at 512 tokens. Size n_batch to the context for these single-pass usecases, mirroring the existing FLAG_SCORE behaviour; an explicit batch: still wins.

Extracts EffectiveContextSize/EffectiveBatchSize from grpcModelOpts so the effective decode window has one home for other callers to reuse.

Adds an e2e-aio regression test that embeds a >512-token input. The AIO embedding model is switched to nomic-embed-text-v1.5 (2048 context) because the previous granite model was capped at 512 tokens and could not exercise the larger batch.

Assisted-by: claude-code:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(gallery): raise arch-router scoring output cap via parallel:64

Scoring decodes the whole prompt+candidate in a single llama_decode and
reads one logit row per candidate token. The vendored llama.cpp server
caps causal output rows at n_parallel, so the default of 1 aborts with
GGML_ASSERT(n_outputs_max <= cparams.n_outputs_max) on multi-token route
labels. Set options: [parallel:64] on both arch-router quant entries to
lift the cap; kv_unified (the grpc-server default) keeps the full context
per sequence, so this does not split the KV cache.

Assisted-by: claude-code:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2026-06-12 16:21:15 +02:00
Richard Palethorpe 5a0013defe test(react-ui): add page render-smoke specs, reset the coverage gate (#10122) 
The UI coverage gate was tightened to 0.1pp against a fast-local
measurement (39.86% baseline); CI's slower runners measure ~0.9pp lower,
so tests-ui-e2e failed there. UI e2e coverage is diffusely
non-deterministic and tracks machine speed — a 0.1pp band can't hold
across environments.

Rather than loosen the gate, raise the floor under it: a render-smoke
spec mounts each lazy page (navigate + assert the header renders),
covering a dozen previously-untested pages and lifting coverage from
~39% to ~42.7% locally. Restore the tolerance to 0.8pp and set the
baseline conservatively (40.0), below the slow-CI floor, so the ratchet
holds without flapping.

Document the coverage policy — install the git hooks and don't bypass
them (no --no-verify, no hand-lowering the baseline or widening the
tolerance); raise coverage by adding tests instead; set the UI baseline
below the slow-CI floor — in AGENTS.md, CONTRIBUTING.md and
.agents/building-and-testing.md.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2026-06-01 14:24:36 +02:00
Richard Palethorpe 718223f33b feat(localvqe/audio): v1.3 release and add spectrograms to audio transform UI (#10113) 
* chore(localvqe): update backend to v1.3, add v1.2/v1.3 gallery models

Bump the LocalVQE backend pin 72bfb4c6 -> b0f0378a, which adds the v1.2
(1.3 M) and v1.3 (4.8 M) GGUF SHA-256s to the upstream released-models
allowlist (and the arch_version=3 loader) so both load without
LOCALVQE_ALLOW_UNHASHED.

Add gallery entries for localvqe-v1.2-1.3m and localvqe-v1.3-4.8m
(SHA-256 verified against the downloaded weights) and update the
audio-transform docs to make v1.3 the current default while noting the
compact v1.1/v1.2 alternatives.

Assisted-by: Claude:claude-opus-4-8 Claude-Code
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* chore(flake): add ffmpeg-headless to the dev shell

pkg/utils/ffmpeg_test.go shells out to the `ffmpeg` CLI, and the
pre-commit gate runs those tests via `make test-coverage`. Without
ffmpeg in the dev shell the gate fails with "executable file not found
in $PATH". The headless build provides the CLI without GUI/X deps.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(localvqe): parse WAV by walking RIFF sub-chunks

Walk the RIFF chunk list instead of assuming the canonical 44-byte
header layout. Real inputs (browser-recorded clips, ffmpeg output with
an 18/40-byte extensible `fmt ` chunk or trailing LIST/INFO metadata)
would otherwise splice header/metadata bytes into the PCM stream as an
audible impulse. Honour the `data` chunk size and validate that both
`fmt ` and `data` chunks are present.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(security-headers): allow blob: in connect-src for waveform fetch

The waveform renderer XHRs/fetches a freshly-created blob: object URL
(e.g. an uploaded or enhanced clip before it has a server URL). XHR/fetch
of blob: is governed by connect-src, not media-src, so it was blocked by
the CSP. Add blob: to connect-src.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(react-ui): add input/output spectrogram view to AudioTransform

The transform page only showed time-domain amplitude waveforms, so you
could see how loud a clip was but not which frequencies the model
touched. Add a time x frequency spectrogram heatmap and render the input
and output spectrums side by side, so it's visible which bands the
enhancement attenuates (bright input bands that go dark in the output).

Computed client-side via a Hann-windowed STFT over both clips (a small
dependency-free radix-2 FFT), defaulting to the LocalVQE 512/256 frame
geometry. This shows the net input->output spectral change; the model's
internal gain mask is not exposed by the backend.

- src/utils/fft.js            radix-2 FFT
- src/hooks/useSpectrogram.js decode + STFT -> normalised dB magnitude grid
- src/components/audio/Spectrogram.jsx  canvas heatmap (magma colormap)
- AudioTransform.jsx          dual-spectrogram panel + CSS
- e2e spec + UI coverage baseline bump (38.29 -> 39.0; measured ~39.4-40.2)

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test(react-ui): make UI coverage deterministic, tighten the gate

UI e2e line coverage swung ~1pp run-to-run (39.1% <-> 40.2%), which forced
a loose 0.8pp tolerance on the monotonic gate — a band wide enough to let
a real ~300-line regression through silently. The swing was a bug, not
inherent jitter: the 'Create Agent navigates' spec ended on the URL
assertion, so AgentCreate.jsx's ~400 lines were collected only when its
render happened to beat the coverage teardown.

Wait for the page to actually render (assert its heading) so those lines
are covered every run. With the race gone, repeated runs land within
~0.013pp of each other, so:

- tighten UI_COVERAGE_TOLERANCE 0.8 -> 0.1 (noise floor, not a drift band)
- set the baseline to the real, reliably-achieved value (39.0 -> 39.86)

Localised by running the V8-coverage suite repeatedly and diffing per-file
line coverage; AgentCreate.jsx was the sole ~1pp flipper.

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2026-05-31 23:56:46 +02:00
Ching a473a32678 test(react-ui): cover models gallery empty-state reset flow (#10019) 
Exercise the filtered empty-state path in the models gallery and verify
that the clear-filters action restores the list and resets the filter
selection.

Assisted-by: Codex:gpt-5

Signed-off-by: Ching Kao <0980124jim@gmail.com>
 2026-05-29 10:39:33 +00:00
Richard Palethorpe b81a6d01b3 perf(react-ui): code-split bundle, speed up coverage suite (#10042) 
* Curate the highlight.js build to ~29 languages (lib/core + the
  common set) instead of the full ~190-grammar default: -787 KB raw /
  -230 KB gz on the base bundle.
* Code-split every route via React.lazy with a per-layout <Suspense>
  in App.jsx so the sidebar stays mounted on navigation. Initial entry
  chunk drops from 3194 KB raw / 887 KB gz to 397 KB / 122 KB (-87%).
  Warm chunks on sidebar hover/focus/touch via a preload registry so
  the click finds the chunk already in flight or cached.
* Migrate Playwright coverage from istanbul (build-time counters) to
  native Chromium V8 coverage, with per-worker accumulation +
  conversion. Suite drops from 71s to 30s at 20 workers (~58%) at the
  non-instrumented floor.
* Keep the coverage gate bundling-invariant: the coverage build inlines
  dynamic imports so every shipped source file lands in the denominator
  (otherwise untested page chunks silently drop out and inflate the
  percentage). Production builds stay code-split.
* Add UI_TEST_WORKERS=N Makefile knob; tighten coverage tolerance to
  0.8pp now that jitter sits near istanbul's ~0.5pp again.

Assisted-by: Claude:claude-opus-4-7 [Claude Code]

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2026-05-28 13:43:15 +02:00
LocalAI [bot]andEttore Di Giacinto 373dc44992 fix(react-ui): force .check() on hidden Toggle input in fits-filter e2e (#10031) 
* fix(react-ui): force .check() on hidden Toggle input in fits-filter e2e

The polish PR (#10030) swapped the raw <input type=checkbox> for the
shared <Toggle> component, which visually hides its native input via
opacity:0;width:0;height:0. Playwright's .check() waits for visibility
before clicking and times out after 30 s, breaking two UI E2E tests:

  - enabling fits filter hides models that exceed available VRAM
  - fits filter state persists after reload

Pass { force: true } to skip the visibility check; the input is still
the real focusable checkbox and toggles state on click. The companion
.toBeChecked() assertion only reads state and works unchanged.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-7

* fix(react-ui): click visible Toggle track in fits-filter e2e

force:true skips the actionability checks but not the viewport check,
and the Toggle's hidden input has width:0;height:0 so Playwright still
reports "Element is outside of the viewport". Click the visible
.toggle__track inside the filter-bar-group__toggle wrapper instead —
that's what a real user clicks, and label-input association toggles
the wrapped checkbox naturally.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-7

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-05-27 22:41:01 +02:00
LocalAI [bot]andEttore Di Giacinto 02a0e70396 fix(react-ui): polish 'Fits in my GPU' filter to use design-system Toggle (#10030) 
* fix(react-ui): polish 'Fits in my GPU' filter to use design-system Toggle

The recently added VRAM-fit filter in the Models page used a raw
<input type="checkbox"> next to the themed range slider, breaking the
visual language of the rest of the row. Swap it for the shared
<Toggle> component (already used by Backends, Settings, Traces,
AgentCreate), adopt the filter-bar-group__toggle class to drop the
duplicated inline styles, add a fa-microchip icon to mirror the
per-row fit indicator, and add a subtle left divider so the filter
reads as separate from the context-size slider on its left.

Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(react-ui): move 'Fits in GPU' filter to filter row and unify copy

Two follow-ups on the previous polish pass:

1. Move the toggle from the context-slider row into the filter-button
   row above. The toggle is a filter on the result set, not a config
   for VRAM estimation, so it belongs with the type chips and backend
   select. The context slider stays its own thing.

2. Unify the label copy. The same locale file had "Fits in my GPU"
   for the filter and "Fits in GPU" for the per-row indicator; pick
   the shorter, possessive-free variant everywhere (en/de/es/it/zh-CN).
   Update e2e selectors to match.

Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-05-27 21:09:14 +02:00
Siddharth More c9a1a7e6a0 UI: add 'Fits in my GPU' filter on Install Models (#10017) 
* feat(ui): add GPU fit filter on models install page

* Delete docs/vram-fits-filter-backend-optionals.md

Signed-off-by: Siddharth More <siddimore@gmail.com>

---------

Signed-off-by: Siddharth More <siddimore@gmail.com>
 2026-05-27 15:17:44 +02:00
Richard Palethorpe 8d70855ea6 test: add Go + React UI coverage gates and fill test gaps (#9989) 
- Strict monotonic Go coverage gate (make test-coverage-check, 45% baseline)
  run in CI; fixes ginkgo dropping all-but-one coverprofile across multiple
  recursive roots, builds with -tags auth, and folds in the in-process
  tests/e2e suite via --coverpkg.
- React UI e2e coverage (make test-ui-coverage: vite-plugin-istanbul + nyc,
  nix-provided Chromium) plus e2e specs for 6 previously-untested pages, and a
  UI coverage gate (make test-ui-coverage-check) with a small tolerance since
  e2e line coverage jitters ~0.5pp run-to-run.
- pre-commit hook: lint + coverage on Go changes, Playwright e2e + UI coverage
  gate on react-ui changes; install with make install-hooks.
- New Go handler tests (settings, branding), hermetic base64 download test.
- fix(ui): model editor reads vram_display (snake_case), so the VRAM estimate
  renders again; covered by a regression test.

Assisted-by: Claude:claude-opus-4-7

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2026-05-26 22:06:10 +02:00
Richard Palethorpe 6a80e23733 feat(middleware): Model routing, PII filtering, Cloud model proxies (#9802) 
Add a routing middleware stack and a cloud-proxy backend.

* cloud-proxy: a Go gRPC backend that forwards OpenAI- and
  Anthropic-shaped chat requests to upstream providers, with an
  optional translate mode (OpenAI request -> Anthropic /v1/messages
  -> OpenAI response) and full tool-calling support.

* routing: admission control, content-aware model routing
  (embedding cache + classifier + rerank + Arch-Router score),
  PII detection/redaction (regex + NER) with streaming filter and
  OpenAI/Anthropic adapters, and a per-user/per-key billing recorder
  backed by GORM or in-memory storage.

* middleware: UsageMiddleware records usage via the billing recorder,
  plus admission, route-model, usage-stamp and trace middlewares.

* observability: BackendTrace ring buffer stores full request bodies
  (capped), MITM proxy emits structured trace events, and router
  classifier decisions surface at /api/router/decide.

* gallery: Arch-Router-1.5B (Q4_K_M and Q8_0).

* UI: cloud-proxy model-editor fields, classifier system-prompt and
  score-normalization config, and a Traces page rendering request
  bodies.

Assisted-by: claude-code:claude-opus-4-7 [Read] [Edit] [Bash]

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2026-05-25 09:28:27 +02:00
LocalAI [bot]andEttore Di Giacinto 1198d10b58 fix(traces): cap backend trace Data to keep admin UI responsive (#9960) 
* fix(traces): cap backend trace Data field so the admin UI stays responsive

The previous fix (#9946) capped API trace bodies but missed backend traces,
which carry the same blast radius:

  - LLM backend traces store the full chat messages JSON, full response, and
    full streaming deltas. Every agent-pool reasoning step ships the full
    RAG-augmented history (50-500 KiB per trace, often 100+ traces queued).
  - TTS / audio_transform / transcript traces embed a 30s audio snippet as
    base64, around 1.3 MiB per trace.

Both blow the /api/backend-traces JSON past tens of MiB. The admin Traces
page then keeps re-downloading and re-parsing the buffer faster than the
5s auto-refresh and stays in the loading state forever, the same symptom
the API-side fix addressed.

Apply two complementary caps, both honoring LOCALAI_TRACING_MAX_BODY_BYTES:

Option A (safety net in core/trace): RecordBackendTrace walks the Data map
recursively and replaces any string value larger than the cap with
"<truncated: N bytes>". Catches anything a future producer forgets.

Option B (head-preserving at the producer):
  - core/backend/llm.go: TruncateToBytes on messages, response, and
    chat_deltas content/reasoning_content so the leading content stays
    readable in the UI.
  - core/trace/audio_snippet.go: omit audio_wav_base64 when the encoded
    blob would exceed the cap (truncated base64 is undecodable). The
    quality metrics still ship and the UI's WaveformPlayer simply skips
    when the field is absent.

TruncateToBytes is bounded to <= maxBytes so Option A leaves the producer's
head-preserving output alone instead of replacing it with the bare marker.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-7

* fix(react-ui): expose tracing_max_body_bytes in Settings and Traces panels

The setting was already plumbed through env (LOCALAI_TRACING_MAX_BODY_BYTES),
CLI flag, and the runtime_settings.json GET/PUT schema, but neither the main
Settings page nor the inline Traces panel offered an input for it. Admins
hitting the "Traces UI stuck loading" symptom had to know to set an env var
or PUT raw JSON to /api/settings to dial the cap.

Add a "Max Body Bytes" row next to "Max Items" in both places. Same input
type, same disabled-when-tracing-off semantics, placeholder shows the 65536
default so users see what they're inheriting.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-7

* test(react-ui): disambiguate Max Items locator after adding Max Body Bytes

The Tracing settings panel now has two number inputs. The previous spec
matched 'input[type="number"]' which became ambiguous and triggered a
Playwright strict-mode violation in CI. Switch to getByPlaceholder('100')
for Max Items and add a parallel spec for the new Max Body Bytes field
using getByPlaceholder('65536').

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-7

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-05-23 14:50:40 +02:00
LocalAI [bot]andEttore Di Giacinto 11d5bd0cc3 fix(react-ui/chat): stop wiping selection on every /api/operations poll (#9904) (#9917) 
useOperations() was calling setOperations() with a fresh array on every
1s poll, even when the payload was identical. In React 19 the DOM diff
no longer short-circuits dangerouslySetInnerHTML on equal __html, so the
forced Chat re-render re-assigned innerHTML on every assistant message
once per second — wiping any text the user had selected.

Skip the state update when the serialised operations payload is
unchanged, and switch loading/error to functional setters so they also
short-circuit at the source.

Also fixes the chat copy button on plain HTTP: navigator.clipboard is
undefined in non-secure contexts (a common LXC+Docker deployment), but
the previous code called it unconditionally and showed a success toast
regardless. Routed Chat, AgentChat and CanvasPanel through a new
copyToClipboard() helper that uses navigator.clipboard when available
and falls back to a hidden-textarea + execCommand('copy') trick that
browsers still honour outside secure contexts. The fallback preserves
the user's existing selection.

Regression coverage in e2e/chat-polling-selection.spec.js: a
MutationObserver counts mutations on the assistant content node across
3s of polling (must be 0); the copy test stubs out navigator.clipboard
and asserts that execCommand('copy') is invoked.


Assisted-by: claude-opus-4-7-1m

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
 2026-05-21 12:17:51 +02:00
Richard Palethorpe 969005b2a1 feat(gallery): Speed up load times and clean gallery entries (#9211) 
* feat: Rework VRAM estimation and use known_usecases in gallery

Signed-off-by: Richard Palethorpe <io@richiejp.com>
Assisted-by: Claude:claude-opus-4-7[1m] [Claude Code]

* chore(gallery): regenerate gallery index and add known_usecases to model entries

Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2026-05-06 14:51:38 +02:00
Richard Palethorpe bb033b16a9 feat: add LocalVQE backend and audio transformations UI (#9640) 
feat(audio-transform): add LocalVQE backend, bidi gRPC RPC, Studio UI

Introduce a generic "audio transform" capability for any audio-in / audio-out
operation (echo cancellation, noise suppression, dereverberation, voice
conversion, etc.) and ship LocalVQE as the first backend implementation.

Backend protocol:
- Two new gRPC RPCs in backend.proto: unary AudioTransform for batch and
  bidirectional AudioTransformStream for low-latency frame-by-frame use.
  This is the first bidi stream in the proto; per-frame unary at LocalVQE's
  16 ms hop would be RTT-bound. Wire it through pkg/grpc/{client,server,
  embed,interface,base} with paired-channel ergonomics.

LocalVQE backend (backend/go/localvqe/):
- Go-Purego wrapper around upstream liblocalvqe.so. CMake builds the upstream
  shared lib + its libggml-cpu-*.so runtime variants directly — no MODULE
  wrapper needed because LocalVQE handles CPU feature selection internally
  via GGML_BACKEND_DL.
- Sets GGML_NTHREADS from opts.Threads (or runtime.NumCPU()-1) — without it
  LocalVQE runs single-threaded at ~1× realtime instead of the documented
  ~9.6×.
- Reference-length policy: zero-pad short refs, truncate long ones (the
  trailing portion can't have leaked into a mic that wasn't recording).
- Ginkgo test suite (9 always-on specs + 2 model-gated).

HTTP layer:
- POST /audio/transformations (alias /audio/transform): multipart batch
  endpoint, accepts audio + optional reference + params[*]=v form fields.
  Persists inputs alongside the output in GeneratedContentDir/audio so the
  React UI history can replay past (audio, reference, output) triples.
- GET /audio/transformations/stream: WebSocket bidi, 16 ms PCM frames
  (interleaved stereo mic+ref in, mono out). JSON session.update envelope
  for config; constants hoisted in core/schema/audio_transform.go.
- ffmpeg-based input normalisation to 16 kHz mono s16 WAV via the existing
  utils.AudioToWav (with passthrough fast-path), so the user can upload any
  format / rate without seeing the model's strict 16 kHz constraint.
- BackendTraceAudioTransform integration so /api/backend-traces and the
  Traces UI light up with audio_snippet base64 and timing.
- Routes registered under routes/localai.go (LocalAI extension; OpenAI has
  no /audio/transformations endpoint), traced via TraceMiddleware.

Auth + capability + importer:
- FLAG_AUDIO_TRANSFORM (model_config.go), FeatureAudioTransform (default-on,
  in APIFeatures), three RouteFeatureRegistry rows.
- localvqe added to knownPrefOnlyBackends with modality "audio-transform".
- Gallery entry localvqe-v1-1.3m (sha256-pinned, hosted on
  huggingface.co/LocalAI-io/LocalVQE).

React UI:
- New /app/transform page surfaced via a dedicated "Enhance" sidebar
  section (sibling of Tools / Biometrics) — the page is enhancement, not
  generation, so it lives outside Studio. Two AudioInput components
  (Upload + Record tabs, drag-drop, mic capture).
- Echo-test button: records mic while playing the loaded reference through
  the speakers — the mic naturally picks up speaker bleed, giving a real
  (mic, ref) pair for AEC testing without leaving the UI.
- Reusable WaveformPlayer (canvas peaks + click-to-seek + audio controls)
  and useAudioPeaks hook (shared module-scoped AudioContext to avoid
  hitting browser context limits with three players on one page); migrated
  TTS, Sound, Traces audio blocks to use it.
- Past runs saved in localStorage via useMediaHistory('audio-transform') —
  the history entry stores all three URLs so clicking re-renders the full
  triple, not just the output.

Build + e2e:
- 11 matrix entries removed from .github/workflows/backend.yml (CUDA, ROCm,
  SYCL, Metal, L4T): upstream supports only CPU + Vulkan, so we ship those
  two and let GPU-class hardware route through Vulkan in the gallery
  capabilities map.
- tests-localvqe-grpc-transform job in test-extra.yml (gated on
  detect-changes.outputs.localvqe).
- New audio_transform capability + 4 specs in tests/e2e-backends.
- Playwright spec suite in core/http/react-ui/e2e/audio-transform.spec.js
  (8 specs covering tabs, file upload, multipart shape, history, errors).

Docs:
- New docs/content/features/audio-transform.md covering the (audio,
  reference) mental model, batch + WebSocket wire formats, LocalVQE param
  keys, and a YAML config example. Cross-links from text-to-audio and
  audio-to-text feature pages.

Assisted-by: Claude:claude-opus-4-7 [Bash Read Edit Write Agent TaskCreate]

Signed-off-by: Richard Palethorpe <io@richiejp.com>
 2026-05-04 22:07:11 +02:00
Ettore Di Giacinto a271c72931 fix(react-ui/e2e): scope backendTrigger to <main> so it skips LanguageSwitcher 
The LanguageSwitcher added in the i18n PR (#9642) lives in the sidebar
and also uses aria-haspopup="listbox" — same attribute the import-form
SearchableSelect uses. The Batch D / E tests' helper resolved the trigger
with `page.locator('button[aria-haspopup="listbox"]').first()`, which now
returns the language switcher (rendered first in DOM order, in the
sidebar) instead of the backend dropdown.

After clicking the wrong button, getByRole('option', { name: 'llama-cpp' })
naturally never resolves — language options aren't backend names — and
the test times out at 30s.

Scope the locator to the <main className="main-content"> wrapper so only
buttons inside the route's main content area match. The page layout has
the Sidebar outside <main>, so this cleanly excludes it.

Assisted-by: Claude:claude-opus-4-7[1m] [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2026-05-04 08:58:25 +00:00
Ettore Di Giacinto 6b63b47f61 feat(distributed): support multiple replicas of one model on the same node (#9583) 
* feat(distributed): support multiple replicas of one model on the same node

The distributed scheduler implicitly assumed `(node_id, model_name)` was
unique, but the schema didn't enforce it and the worker keyed all gRPC
processes by model name alone. With `MinReplicas=2` against a single
worker, the reconciler "scaled up" every 30s but the registry never
advanced past 1 row — the worker re-loaded the model in-place every tick
until VRAM fragmented and the gRPC process died.

This change introduces multi-replica-per-node as a first-class concept,
with capacity-aware scheduling, a circuit breaker, and VRAM
soft-reservation. Operators can declare per-node capacity via the worker
flag `--max-replicas-per-model` (mirrored as auto-label
`node.replica-slots=N`) or override per-node from the UI.

* Schema: BackendNode gains MaxReplicasPerModel (default 1) and
  ReservedVRAM. NodeModel gains ReplicaIndex (composite with node_id +
  model_name). ModelSchedulingConfig gains UnsatisfiableUntil/Ticks for
  the reconciler circuit breaker.

* Registry: replica_index threaded through SetNodeModel, RemoveNodeModel,
  IncrementInFlight, DecrementInFlight, TouchNodeModel, GetNodeModel,
  SetNodeModelLoadInfo and the InFlightTrackingClient. New helpers:
  CountReplicasOnNode, NextFreeReplicaIndex (with ErrNoFreeSlot),
  RemoveAllNodeModelReplicas, FindNodesWithFreeSlot,
  ClusterCapacityForModel, ReserveVRAM/ReleaseVRAM (atomic UPDATE with
  ErrInsufficientVRAM), and the unsatisfiable-flag CRUD.

* Worker: processKey now `<modelID>#<replicaIndex>` so concurrent loads
  of the same model land on distinct ports. Adds CLI flag
  --max-replicas-per-model (env LOCALAI_MAX_REPLICAS_PER_MODEL, default 1)
  and emits the auto-label.

* Router: scheduleNewModel filters candidates by free slot, allocates the
  replica index, and soft-reserves VRAM before installing the backend.
  evictLRUAndFreeNode now deletes the targeted row by ID instead of all
  replicas of the model on the node — fixes a latent bug where evicting
  one replica orphaned its siblings.

* Reconciler: caps scale-up at ClusterCapacityForModel so a misconfig
  (MinReplicas > capacity) doesn't loop forever. After 3 consecutive
  ticks of capacity==0 it sets UnsatisfiableUntil for a 5m cooldown and
  emits a warning. ClearAllUnsatisfiable fires from Register,
  ApproveNode, SetNodeLabel(s), RemoveNodeLabel and
  UpdateMaxReplicasPerModel so a new node joining or label changes wake
  the reconciler immediately. scaleDownIdle removes highest-replica-index
  first to keep slots compact.

* Heartbeat resets reserved_vram to 0 — worker is the source of truth
  for actual free VRAM; the reservation is only for the in-tick race
  window between two scheduling decisions.

* Probe path (reconciler.probeLoadedModels and health.doCheckAll) now
  pass the row's replica_index to RemoveNodeModel so an unreachable
  replica doesn't orphan healthy siblings.

* Admin override: PUT /api/nodes/:id/max-replicas-per-model sets a
  sticky override (preserved across worker re-registration). DELETE
  clears the override so the worker's flag applies again on next
  register. Required because Kong defaults the worker flag to 1, so
  every worker restart would have silently reverted the UI value.

* React UI: always-visible slot badge on the node row (muted at default
  1, accented when >1); inline editor in the expanded drawer with
  pencil-to-edit, Save/Cancel, Esc/Enter, "(override)" indicator when
  the value is admin-set, and a "Reset" button to hand control back to
  the worker. Soft confirm when shrinking the cap below the count of
  loaded replicas. Scheduling rules table gets an "Unsatisfiable until
  HH:MM" status badge surfacing the cooldown.

* node.replica-slots filtered out of the labels strip on the row to
  avoid duplicating the slot badge.

23 new Ginkgo specs (registry, reconciler, inflight, health) cover:
multi-replica row independence, RemoveNodeModel of one replica
preserving siblings, NextFreeReplicaIndex slot allocation including
ErrNoFreeSlot, capacity-gated scale-up with circuit breaker tripping
and recovery on Register, scheduleDownIdle ordering, ClusterCapacity
math, ReserveVRAM admission gating, Heartbeat reset, override survival
across worker re-registration, and ResetMaxReplicasPerModel handing
control back. Plus 8 stdlib tests for the worker processKey / CLI /
auto-label.

Closes the flap reproduced on Qwen3.6-35B against the nvidia-thor
worker (single 128 GiB node, MinReplicas=2): the reconciler now caps
the scale-up at the cluster's actual capacity instead of looping.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: claude-code:opus-4-7 [Read] [Edit] [Bash] [Skill:critique] [Skill:audit] [Skill:polish] [Skill:golang-testing]

* refactor(react-ui/nodes): tighten capacity editor copy + adopt ActionMenu for row actions

* Capacity editor hint trimmed from operator-doc-style ("Sourced from
  the worker's `--max-replicas-per-model` flag. Changing it here makes it
  a sticky admin override that survives worker restarts." → "Saved
  values stick across worker restarts.") and the override-state copy
  similarly compressed. The full mechanic is no longer needed in the UI
  — the override pill carries the meaning and the docs cover the rest.

* Node row actions migrated from an inline cluster of icon buttons
  (Drain / Resume / Trash) to the kebab ActionMenu used by /manage for
  per-row model actions, so dense Nodes tables stay clean. Approve
  stays as a prominent primary button — it's a stateful admission gate,
  not a routine action, and elevating it matches how /manage surfaces
  install-time decisions outside the menu.

* The expanded drawer's Labels section now filters node.replica-slots
  out of the editable label list. The label is owned by the Capacity
  editor above; surfacing it again as an editable label invited
  confusion (the Capacity save would clobber any direct edit).

Both backend and agent workers benefit — they share the row rendering
path, so the action menu and label filter apply to both.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: claude-code:opus-4-7 [Edit] [chrome-devtools-mcp] [Skill:critique] [Skill:audit] [Skill:polish]

* fix(react-ui/nodes): suppress slot badge on agent workers

Agent workers don't load models, so the per-node replica capacity is
inapplicable to them. Showing "1× slots" on agent rows was a tiny
inconsistency from the unified rendering path — gate the badge on
node_type !== 'agent' so it only appears on backend workers.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: claude-code:opus-4-7 [Edit] [chrome-devtools-mcp]

* refactor(react-ui/nodes): distill expanded drawer + restyle scheduling form

The expanded node drawer used to stack five panels — slot badge,
filled capacity box, Loaded Models h4+empty-state, Installed Backends
h4+empty-state, Labels h4+chips+form — making routine inspections feel
like a control panel. The scheduling rule form wrapped its mode toggle
as two 50%-width filled buttons that competed visually with the actual
primary action.

* Drawer: collapse three rarely-touched config zones (Capacity,
  Backends, Labels) into one `<details>` "Manage" disclosure (closed by
  default) with small uppercase eyebrow labels for each zone instead of
  parallel h4 sub-headings. Loaded Models stays as the at-a-glance
  headline with a single-line empty hint instead of a boxed empty state.
  CapacityEditor renders flat (no filled background) — the Manage
  disclosure provides framing.

* Scheduling form: replace the chunky 50%-width button-tabs with the
  project's existing `.segmented` control (icon + label, sized to
  content). Mode hint becomes a single tied line below. Fields stack
  vertically with helper text under inputs and a hairline divider above
  the right-aligned Save / Cancel.

The empty drawer collapses from ~5 stacked sections (~280px tall) to
two lines (~80px). The scheduling form now reads as a designed dialog
instead of raw building blocks. Both surfaces now match the typographic
density and weight of the rest of the admin pages.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: claude-code:opus-4-7 [Edit] [chrome-devtools-mcp] [Skill:distill] [Skill:audit] [Skill:polish]

* feat(react-ui/nodes): replace scheduling form's model picker with searchable combobox

The native <select> made operators scroll through every gallery entry to
find a model name. The project already has SearchableModelSelect (used
in Studio/Talk/etc.) which combines free-text search with the gallery
list and accepts typed model names that aren't installed yet — useful
for pre-staging a scheduling rule before the node it'll run on has
finished bootstrapping.

Also drops the now-unused useModels import (the combobox manages the
gallery hook internally).

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: claude-code:opus-4-7 [Edit]

* refactor(react-ui/nodes): consolidate key/value chip editor + add replica preset chips

The Nodes page was rendering the same key=value chip pattern in two
places with subtly different markup: the Labels editor in the expanded
drawer and (post-distill) the Node Selector input in the scheduling
form. The form's input was also a comma-separated string that operators
were getting wrong.

* Extract <KeyValueChips> as a fully controlled chip-builder. Parent
  owns the map and decides what onAdd/onRemove does — form state for the
  scheduling form, API calls for the live drawer Labels editor. Same
  visuals everywhere; one component to change when polish needs apply.

* Replace the comma-separated Node Selector text input with KeyValueChips.
  Operators were copying syntax from docs and missing commas; the chip
  vocabulary makes the key=value structure self-documenting.

* Add <ReplicaInput>: numeric input + quick-pick preset chips for Min/Max
  replicas. Picked over a slider because replica counts are exact specs
  derived from VRAM math (operator decision, not a fuzzy estimate). The
  chips give one-click access to common values (1/2/3/4 for Min,
  0=no-limit/2/4/8 for Max) without the slider's special-value problem
  (MaxReplicas=0 is categorical, not a position on a continuum).

* Drop the now-unused labelInputs state in the Nodes page (the inline
  label editor's per-node draft state lived there and is now owned by
  KeyValueChips).

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: claude-code:opus-4-7 [Edit] [Skill:distill]

* test: fix CI fallout from multi-replica refactor (e2e/distributed + playwright)

Two breakages caught by CI that didn't surface in the local run:

* tests/e2e/distributed/*.go — multiple files used the pre-PR2 registry
  signatures for SetNodeModel / IncrementInFlight / DecrementInFlight /
  RemoveNodeModel / TouchNodeModel / GetNodeModel / SetNodeModelLoadInfo
  and one stale adapter.InstallBackend call in node_lifecycle_test.go.
  All updated to pass replicaIndex=0 — these tests don't exercise
  multi-replica behavior, they just need to compile against the new
  signatures. The chip-builder tests in core/services/nodes/ already
  cover the multi-replica logic.

* core/http/react-ui/e2e/nodes-per-node-backend-actions.spec.js — the
  drawer's distill refactor moved Backends inside a "Manage" <details>
  disclosure that's collapsed by default. The test helper expanded the
  node row but never opened Manage, so the per-node backend table was
  never in the DOM. Helper now clicks `.node-manage > summary` after
  expanding the row.

All 100 playwright tests pass locally; tests/e2e/distributed compiles
clean.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: claude-code:opus-4-7 [Edit] [Bash]

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
 2026-04-27 21:20:05 +02:00