Variant auto-selection filtered candidates by whether their backend can
run on the host, then ranked the survivors by size alone. The backend
never influenced the choice beyond that gate, so a Mac offered both an
MLX build and a llama.cpp build kept neither filtered and installed
whichever was larger, leaving the native accelerated runtime unused. The
same held for CUDA against CPU on NVIDIA and ROCm against Vulkan on AMD.
Rank by the host's backend preference between the fit tier and size: fit
stays a filter, preference decides among the builds the host can equally
hold, and size still separates builds on equally preferred runtimes.
The preference data stays in one declarative table in pkg/system, now
read by a prefix lookup instead of a switch, so adding a capability or
reordering one host's runtimes is a one-line edit and the gallery's
ranking code carries no per-backend branching. MLX joins the metal rule
ahead of metal itself, which is inert for the existing alias-resolution
consumer because no alias group holds a candidate named for mlx.
An unrecognised backend, an unrecognised capability and an absent
preference list all collapse to the previous size-only ordering rather
than erroring or dropping candidates.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Sweep the gallery for entries that are alternative builds of the same
weights (different quantization, precision, or runtime format) and declare
them as variants of a single parent row, so the listing offers one row per
model instead of one row per quantization and the installer picks the
largest build that this host can actually run.
41 families over 95 entries, turning 54 entries into variants.
The parent is the bare-named entry wherever one exists, so nothing changes
about what any existing entry installs. Ranking already selects the largest
fitting build regardless of which entry is nominally the parent, so the
parent only decides the pathological case where nothing fits. For the ten
families that have no bare-named entry, the smallest build is the parent,
since that is the one that has to install when nothing fits.
Grouping was verified against the actual model filenames rather than the
entry names alone. Different parameter sizes, languages, finetunes, and
products that merely share a name prefix are left as separate rows: the
qwen3.6 APEX and pi-tune finetunes, the DFlash and MTP speculative-decoding
pairings, English-only versus multilingual Whisper, the QAT versus non-QAT
Gemma 4 weights, and the abliterated FLUX build are all distinct models.
Six parents define YAML anchors that other entries pull in with a merge key,
which would have handed their variants to every merging child. For the two
depth-anything anchors that would have made fourteen unrelated entries
advertise the base model's builds as their own. All 26 merging children
therefore carry an explicit empty variants list, which overrides the merged
key and is equivalent to the key being absent.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The gallery listing is what a user reaches for to answer "what can I
install". Answering that with several rows for the same model, one per
build, makes the reader do the deduplication the collapsed view already
does, so the collapsed view is the one to land on.
The UI now asks for collapse_variants=true unless the toggle says
otherwise. The server default is deliberately untouched: a request with
the parameter absent still returns the full listing, because other API
clients depend on that response and collapsing it under them would be a
breaking change. Opting out omits the parameter rather than sending
false, so it asks for exactly the listing everyone else gets.
The stored preference changes vocabulary from '1'/'0' to 'on'/'off'. The
previous build wrote it from an effect that runs on mount, so a stored
'0' recorded that the page had been opened rather than that anyone chose
the expanded view, and honouring it would pin every earlier visitor to a
default they never picked. Only the new vocabulary counts as a choice;
a legacy '1' meant the collapsed view and is what the new default gives
anyway, so no earlier deliberate choice is lost.
Collapsing being the default also changes what the empty state may say
about it. An opted-into filter can be named as the cause of an empty
result; a default cannot, so the filters keep the top line and the
collapsed view drops to a hint below it, shown only once filters are
narrowing the set. For the same reason "Clear filters" now restores the
collapsed default instead of switching it off, and the toggle alone no
longer counts as a filter worth offering to clear.
The label stays "One row per model": it describes the view the user is
looking at rather than an action, so it reads the same whether it is
opted into or out of.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The listing supported has_variants=true, which narrowed to entries that
DECLARE variants. With adoption at three entries that showed three rows,
which is useless; it was always a placeholder.
Replace it with the view that is actually useful: the deduplicated
gallery. Show every entry installable in its own right and nothing twice,
which means the parents plus every entry nobody references, and hide only
the builds another entry already offers as a variant, since those are
reachable through their parent.
The parameter is renamed to collapse_variants accordingly: the filter is
no longer a predicate on a row's own metadata but a view over the whole
gallery. Default stays off, so the response with the parameter absent is
unchanged.
VariantReferencedIDs never reports an entry that declares variants of its
own, so parents are always visible. That guarantees every hidden entry
has a visible entry offering it, and no chain can strand a row. Variant
resolution already refuses to install such a reference, but the listing
has to stay coherent in the presence of a gallery that has one rather
than silently swallowing entries. Self-references and dangling references
hide nothing.
The referenced set is computed over the whole gallery rather than over
what the other filters left, so an entry is hidden because a parent
offers it and never because of what the user searched for. The pass is
over metadata already in memory: it resolves nothing over the network and
triggers no variant description or size probe, so the listing's zero-probe
contract still holds.
The UI toggle keeps its behaviour (persistence, page reset, clear
filters) and becomes "One row per model", which says what the user gets.
Its localStorage key moves too, since the stored value meant a different
filter.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The gallery detail pane rendered every field through the same two-column
label/value row, including the description. Multi-paragraph prose in a value
cell ran eight rows tall at the top of the pane on a ~1200px measure, breaking
the grid's rhythm exactly where the eye enters. Move it into its own full-width
block above the table, capped at a 68ch measure, keeping the label.
Rendered Markdown had no scoped typography anywhere in the app, so a
description opening with `#` inherited the browser default 2em inside a 13px
surface while a `##` further down was indistinguishable from body text. Add a
reusable .markdown-body block mapping h1-h6, paragraphs, lists, links, code,
blockquotes, images and tables onto the existing type scale, and apply it to
every renderMarkdown() consumer: the models detail, the backends detail, both
Manage details and the voice library detail.
Rebalance the variants list so the name leads. Backend and size drop from
badge/secondary weight to muted metadata; the FITS badge goes entirely, since
it was true of nearly every row and so said nothing, while the variant that
does not fit keeps a warning badge and a dimmed name. AUTO-SELECTED stays
marked because it answers what a plain Install produces. Rows share the
parent's grid tracks via subgrid so name, backend, size and status line up
down the list instead of raggedly following name length.
Finally, make each variant row actionable. It looked like a list of choices
but was inert text, with per-variant install hidden behind the split-button
chevron elsewhere; each row is now a button onto the existing
handleInstall(modelId, variant) path, with hover, keyboard focus and a
disabled state while an install is in flight.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Commit b35d630cf fixed this for gallery models but left the Backends admin
page with the same asymmetry: its detail panel renders the description
through renderMarkdown, while the collapsed table row dumped the raw gallery
string into both the cell body and the title tooltip.
That is user-visible. 40 of the 949 entries in backend/index.yaml carry
Markdown - insightface uses inline code backticks, others use lists and
links - and backend descriptions also contain embedded newlines, so the
one-line cell showed literal syntax.
The cell now runs stripMarkdown over the description once and uses the
result for the text and the title, matching Models.jsx and the
ResourceRowDesc component in Manage.jsx. The '-' placeholder is preserved,
and now also fires when a description reduces to nothing after stripping.
The detail panel is untouched and no new dangerouslySetInnerHTML is
introduced: stripMarkdown output lands in a JSX text node, so React escapes
it.
Three Playwright specs cover it: a description with a heading, inline code
and a link renders as clean text with no literal syntax and no block
element in the cell, the title tooltip carries the same stripped text, and
a backend without a description still shows the placeholder.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Gallery descriptions are Markdown, but the React UI dumped them raw, so a
model whose description opens with an ATX heading showed a literal
"# Qwen3.6-27B [](https://chat.qwen.ai)" in the list.
Full-description areas now render through renderMarkdown (marked +
DOMPurify), matching how Backends.jsx and the Manage detail panels already
handle the same content:
- Models.jsx expanded detail row
- VoiceLibrary.jsx voice detail header
The truncated one-line previews must not render block Markdown: a leading
"#" would become an <h1> and wreck the row height and rhythm. They get a new
stripMarkdown() helper instead, which reduces Markdown to a single line of
readable plain text. It is used for the cell text and for the title tooltip,
since a tooltip full of "[](url)" is no better than a cell full of it:
- Models.jsx gallery table description cell
- Manage.jsx model and backend resource-row descriptions
stripMarkdown walks marked's lexer output rather than running regexes over
the source, so what it strips is by construction what renderMarkdown would
have rendered, and it needs no new dependency. Output lands in JSX text
nodes, so React escapes it; no new dangerouslySetInnerHTML beyond the two
full-description sites, both of which run DOMPurify.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The gallery is heading towards showing parent entries and hiding the
individual builds they reference, so a user sees one row per model
rather than six quantizations of it.
Adoption is a single entry today, so defaulting to that would leave a
one-row gallery. This ships the migration-phase inverse instead: the
default is untouched, and a toggle narrows the list to only the entries
that declare variants. It previews the end state and changes nothing
until someone asks for it.
The filter is server-side, next to term/tag/backend/capability and above
the pagination arithmetic. The listing paginates at 9 items, so
narrowing on the client would leave totalPages and availableModels
describing the unfiltered set and hand the user empty pages. It selects
on HasVariants(), which reads already-loaded metadata, so it issues no
variant probes.
The parameter is named has_variants after the listing field it selects
on, and is compared against "true" like the other boolean query params
(all_users, save_checkpoint), so has_variants=false reads as absent.
With it omitted the response is byte-for-byte what it was before.
The control is the shared Toggle component, matching the fitsFilter
toggle already on this page: same wrapper class, same icon and label
shape, same localStorage persistence. Unlike fitsFilter it resets to
page 1 on change, which a server-side filter has to do.
Stacking the toggle with a tag or backend filter easily yields nothing
while one entry declares variants, so the empty state now names the
variants filter as the cause rather than leaving a user to conclude the
gallery is broken.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Variant description probes each referenced entry's weight files over the
network: an HTTP HEAD plus a ranged GET, serial, five seconds per probe
with no aggregate deadline. Running it inline in GET /api/models made one
listing cost (entries x variants) round trips. The Manage page fetches
with items=9999, so at 200 declaring entries that is ~1000 serial probes,
minutes of a blocked handler and gigabytes of range traffic for a single
page load. Only one entry declares variants today, but the feature exists
so that many will.
Follow the precedent already set for VRAM estimates. The listing now
reports only has_variants, a length check on loaded metadata that touches
nothing, and GET /api/models/variants/:id returns the description for one
entry, mirroring estimate/:id in route shape, auth and error handling.
DescribeVariants itself is unchanged; only its caller moved.
The picker fetches lazily at the two points where a user asks to see
variants, opening the split-button menu and expanding the detail row, and
caches per entry for the page session. An entry declaring no variants
issues no request at all.
A spec counts real HTTP hits on the weight files, so it goes red if
description becomes reachable from the listing path again through any
caller.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
PR #10943 shipped the server side: a gallery entry may declare `variants:`,
`GET /api/models` attaches `variants` and `auto_variant` to declaring
entries, and `POST /api/models/install/:id` accepts a `variant` query
parameter. Nothing in the UI consumed any of it, so the feature was not
reachable from the browser. This wires it up.
modelsApi.install takes an optional second argument and appends an encoded
`?variant=` only when one is given, so every existing call site keeps
sending exactly the request it sent before.
On the models table, an entry that declares variants gets a split button.
The primary Install still installs the auto-selected build, because auto is
the default and the point of the feature; the chevron opens a menu for a
deliberate override. It follows the Backends.jsx precedent: one shared
Popover re-anchored per row, rendering .action-menu items, which brings
Escape, outside-click and focus return along with it. An entry that
declares no variants renders exactly as it did before.
A variant that does not fit is dimmed but stays selectable, since the server
honors an explicit choice with a warning rather than refusing it.
memory_bytes is omitempty on the wire, so an absent key means the size is
unknown and never zero. A single helper guards both the menu and the detail
row, because formatBytes would otherwise render a falsy value as "0 B",
which reads as "needs nothing".
The expanded detail row gains a Variants section listing each build's
backend, size, whether it fits, which is the entry's own build, and which
one auto-selection would pick, built from the existing DetailRow helper and
.badge classes.
Eight Playwright specs cover the picker, including that plain Install sends
no variant parameter and that choosing one sends it. One pre-existing
assertion was scoped with .first(): the Variants section legitimately adds
more llama-cpp badges to the detail row, which tripped strict mode.
UI line coverage 49.42% -> 49.36% against a 40.0 baseline and 0.8pp
tolerance; branch coverage rose 72.04% -> 72.66%.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Variant selection read its memory budget from VRAM whenever a GPU
capability was detected, and from system RAM only when none was. Apple
Silicon satisfies the first branch and fails the premise: arm64 macs report
the metal capability unconditionally, without probing anything, while
TotalAvailableVRAM has no discrete VRAM pool to find and returns zero. The
budget therefore came out as zero on every Mac.
Zero drops every variant carrying a known size, so the base build was
installed on all of them however much memory the machine had. The feature
was inert on the platform, and silently: falling back to the base is a
legitimate outcome, so nothing looked wrong.
Take VRAM only when it is actually a number, and fall back to RAM
otherwise. On a unified-memory host RAM is not an approximation of the
budget, it is the budget, since the GPU shares it. A discrete GPU whose
VRAM could not be read also lands on RAM, which overstates what the card
holds but understates nothing the host has; the previous zero understated
both.
An unreadable RAM figure still yields zero and still installs the base, so
a genuinely unknown host is not talked into a larger download.
This is what turned tests-apple red: "installs a fitting variant's payload
under the entry's own name" asserts on selection, and the runner resolved
to the base because its budget was zero. The added specs pin the branch
directly rather than relying on a macOS runner to notice again.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Variant selection pulled the declaring entry's own payload, the base, out
of the candidate set and consulted it only once every declared variant had
been rejected. Two real failures followed.
A variant whose size the probe cannot determine deliberately survives the
memory filter, because nothing proves it does not fit. As the only survivor
it then won outright on any host, however small: a 2GiB machine installed an
unmeasured variant in preference to the 4GiB build the entry itself ships,
with no warning. 241 of the 1280 current index entries carry no files and no
size, which is exactly that shape.
"Largest wins" also broke whenever the base was the largest. An author
writing a Q8 entry that offers a Q4 downgrade for small hosts, a natural
shape that nothing in the lint, schema or docs discourages, had the Q4
installed on every large host instead.
Make the base an ordinary participant. It is still exempt from both filters,
so selection always terminates on something installable, but it is now
ranked against the variants: a proven fit first and largest, then the base,
then any variant whose size nothing could measure. Both failures disappear
together. The base is probed for its size accordingly, which it was not
before, because an unsized base would lose every contest to an unmeasurable
variant.
FellBackToBase is kept but narrowed to "no declared variant survived",
rather than "the base was chosen", since the base now also wins on merit and
that is not worth warning about.
A recalled variant pin also became a permanent install failure. A pin the
caller supplies on this request must stay fatal, but one recalled from
._gallery_<name>.yaml can be invalidated by any later gallery edit, and
failing on it turned one rename into a model that could never be reinstalled
or upgraded again short of deleting a dotfile the user has never heard of.
A stale recalled pin is now dropped with a warning naming it, and selection
runs as if it had never been recorded.
Also drop the last textual reference to two abandoned designs from the
DetectedCapability comment, correct the documented variants JSON example,
which showed a memory_bytes of 0 that omitempty makes impossible, and remove
an em dash from the install skill.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Variant.MinMemory was an authored override for when the live probe misreads
a variant's footprint. It duplicated an existing field: probeEntryMemory
already passes the entry's declared size: into EstimateModelMultiContext,
whose cascade prefers that declared size over its own guesswork. Correcting
size: on the referenced entry fixes the figure for every consumer rather
than only for variant selection, so min_memory shadowed the right answer.
A variant is now nothing but a name. Its effective size is exactly the probe
result, and an unknown stays unknown: it survives the filter and ranks last.
EffectiveMemory loses its error return along with the field. The authored
string was the only thing that could fail to parse, so the error had no
remaining source and was propagating dead nil-checks through SelectVariant,
DescribeVariants and the pin warning.
Selection behaviour is unchanged. The specs covering probe-derived sizing,
ranking, filtering, the unknown-size path, pin recall, entry/variant
metadata split and deep-copy isolation all survive; the three install specs
that needed a definite size now declare it through the referenced entry's
own size:, which exercises the documented escape hatch directly.
gallery/index.yaml is untouched: no entry ever carried the key.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A gallery entry may carry `variants:`, alternative builds of the same model.
Selection already worked at install time, but nothing could see what an entry
offered or ask for a specific build, so the feature was undrivable.
Listing: `GET /api/models` now reports `variants` and `auto_variant` for the
entries that declare variants. Each variant carries its resolved backend, its
measured size and whether it fits this host. `auto_variant` is what installing
without a choice would pick right now.
The new gallery.DescribeVariants runs the same variantOptions + SelectVariant
pass the installer runs, so the reported default cannot drift from what
installing actually does, and HostResolveEnv is extracted so both derive the
host and share pkg/vram's probe cache from one place.
Performance: an entry that declares no variants returns early without touching
the probe, so the ~1280 ordinary entries cost exactly what they cost before.
Selection: `variant` is accepted on POST /models/apply, as a query param on
POST /api/models/install/:id, on the gallery apply file/string request, as
`local-ai models install --variant`, and as a parameter on the install_model
MCP tool (both the httpapi and inproc clients). Empty means auto-select.
An unknown variant name now fails the install naming what was requested. This
closes a real hole: an entry declaring no variants short-circuits before
selection runs, so a requested variant was previously dropped silently and the
install reported success.
startup.InstallModels ends in a variadic model list, so install options could
not be appended to it; InstallModelsWithOptions is added alongside and
InstallModels delegates to it. No caller signature changed.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Selection needs each variant's size to decide whether it fits and to rank
largest-first. That figure was written into the index by a nightly job, which
made the gallery carry a derived value that could drift from the entry it was
derived from. Derive it at install time instead.
pkg/vram already sizes a model without downloading it, and the gallery UI
already uses it: a remote GGUF header range-fetch, then an HTTP HEAD for the
content length, then any declared size:. It caches its results, so reuse it
rather than writing a second probing path.
A probe failure must never fail an install, so an unprobeable variant is
treated as unknown: it survives the memory filter, because nothing proves it
does not fit, and it ranks last, so a known-good fit always beats a guess. If
every probe fails, selection still terminates on the base entry.
The probe is injected through ResolveEnv rather than called directly, for the
same reason the backend compatibility check is: specs pin an exact size, or an
exact failure, without reaching the network.
With that in place three things are dead weight and go:
- The nightly job and the fields it populated. Variant.Backend was redundant
because the backend is resolved live from the referenced entry during
selection, and Quantization was display-only that nothing read.
- min_memory on the base entry. The base always installs and its floor could
only warn, so it could not change any outcome.
- The lint rules and schema entries for both.
min_memory on individual variants stays, as the override for when the probed
size is wrong. An authored figure now suppresses the probe entirely rather
than merely outranking it, so it costs no round trip.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Gallery entries could already carry a list of alternatives, but selection was
an authored, ordered, first-match policy: every candidate declared a
`capability` string and the VRAM floors had to descend in a hand-tuned order.
That pushed hardware knowledge onto whoever edits the gallery and made ordering
load-bearing, so a reordered list silently changed what users installed.
None of it was necessary. SystemState.IsBackendCompatible already derives
hardware support from a backend name alone: it knows MLX and metal are
Darwin-only, CUDA is NVIDIA-only, ROCm AMD-only, SYCL Intel-only. Selection can
read that instead of asking authors to restate it.
Authoring is now just a list of names:
- name: qwen3.6-27b
min_memory: 4GiB
variants:
- model: qwen3.6-27b-mlx-8bit
- model: qwen3.6-27b-gguf-q8
min_memory: 28GiB
and all the intelligence moved into the selector. Given a host it drops the
variants whose backend cannot run here, drops those whose known memory
requirement exceeds what the host has, and takes the LARGEST of what is left,
because a bigger footprint is a higher quality quantization of the same model.
A variant of unknown size is kept, since nothing proves it does not fit, but it
ranks last so a proven fit always beats a guess. An explicit pin still wins
outright, and if nothing survives the entry installs its own payload: the base
always installs, this never refuses.
Available memory is VRAM when a GPU was detected and system RAM otherwise, read
through xsysinfo so a cgroup limit is honored and a container gets its own
limit rather than the node's RAM.
Capability disappears entirely, from the types, the schema and the lint. VRAM
and RAM collapse into one `min_memory`, because a model's footprint is roughly
the same wherever it lives and one figure is compared against whichever applies.
The lint rules about ordering, the capability vocabulary and floor
relationships are deleted with the hazards they described; what remains is that
every variant names an entry that exists and does not itself declare variants,
plus that any memory figure actually parses.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Reworks hardware-resolved gallery variants after a design pivot. There is no
longer a separate "meta" entry kind. A gallery entry is a normal, complete
entry that may additionally carry candidates:, a list of hardware-gated
upgrades over itself, and the entry is itself the last-resort candidate.
The previous design relied on a bare url: as the fallback for LocalAI releases
that predate candidates support. That fallback is empty in practice: none of
the 80 gallery/*.yaml files carry a top-level files:, and 1216 of 1281 index
entries carry their payload in the index entry itself, so a url alone yields a
config template with nothing to download. Since every released LocalAI reads
gallery/index.yaml live from master, merging a payload-less entry would have
shown every existing user a model that installs to a broken state. Making the
entry its own base candidate removes the problem at the root: old clients drop
the candidates key and install the entry exactly as they do today.
Resolution order is now explicit pin, then capability plus VRAM over the
declared upgrades, then the entry itself. The entry ALWAYS installs: when its
own min_vram or capability is unmet the installer warns and installs it
anyway, because there is nothing below it and refusing would make the gallery
behave worse the newer the client is. A pin naming the entry's own name is
valid and is how an operator declines an upgrade.
IsMeta() becomes HasCandidates(), ResolveMetaModel becomes ResolveVariant, and
the persisted meta_name record key becomes entry_name. GalleryBackend.IsMeta()
is a separate concept and is untouched.
The lint drops the three rules the pivot makes wrong (url equality with the
final candidate, no inline payload, unconstrained final candidate) and gains
one: the entry's own floor must sit strictly below every candidate's, since a
base that outranks a candidate makes that candidate unreachable.
The pilot entry is now the existing nanbeige4.1-3b-q4, which gains a 2GiB
floor of its own and a single 6GiB upgrade to nanbeige4.1-3b-q8, replacing the
separate nanbeige4.1-3b entry added in d0d441bb4.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Adds the first real meta entry to the gallery index. It resolves to the
Q8_0 build on hosts with at least 6GiB of VRAM and to the Q4_K_M build
everywhere else, installing either payload under the stable name
nanbeige4.1-3b.
The entry carries a url equal to its final candidate's url. LocalAI
releases that predate candidates support parse the index non-strictly
and drop the key silently, so without that url they would list the entry
and install nothing. A regression spec parses the index the way those
releases do and asserts every meta entry stays installable for them.
Also teaches core/schema/gallery-model.schema.json about candidates. The
schema sets additionalProperties: false at the top level, so an author
following CONTRIBUTING.md and adding the yaml-language-server comment
would otherwise get a validation error on this entry.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The nightly denormalization job edits YAML nodes instead of round-tripping
structs so its PR stays small enough for a human to review, but the write
path undid that: yaml.Marshal re-encoded the node tree at yaml.v3's default
4-space indent and dropped the leading document marker, reflowing roughly
6000 lines around the handful of real changes. Encode through
yaml.NewEncoder at the index's authored 2-space indent and restore the
header. A write that changes three fields now changes three lines.
Stale inferred_min_vram values were also never cleared. Both skip paths
(an authored min_vram is present, or the candidate is the last resort)
returned before touching the field, so a candidate that gained a floor or
became the last resort after a reorder kept an inferred value that
EffectiveMinVRAM reported as a real constraint, failing the meta lint with
no way for the job to self-heal. Clear the field before both skips.
The workflow discarded a whole night's work on any single failure: the
program exits 1 when a candidate cannot be estimated, which aborted the job
before the PR step, so one unreachable candidate blocked every other
refresh indefinitely. Capture the status, open the PR with what was
computed, mark the PR body as partial, and fail the run afterwards so the
problem still surfaces.
Also preserve the index's existing file mode instead of forcing 0644, and
drop the redundant //go:build ignore tag, since Go already skips dot
directories and the sibling modelslist.go carries no tag.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Fills the read-only backend, quantization and inferred_min_vram fields on
meta gallery candidates and opens a PR, modeled on the existing
checksum_checker job. Computing these needs network access, so it happens
nightly rather than at install time.
An authored min_vram is never modified: a human who measured a real load
knows more than a pre-download estimate does.
The index is rewritten via yaml.Node rather than a document round-trip. A
full round-trip reflows all ~26k lines of gallery/index.yaml, which would
bury the computed values and make the nightly PR unreviewable. The rewrite
touches only the three derived keys, so authored styling survives and a run
that computes nothing leaves the file untouched.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The ordering invariant grouped candidates by capability and asserted floors
descend within a group. A candidate with an EMPTY capability matches every
host, so it does not belong in its own group: it dominates every later
candidate whose floor is at or above its own, across capability groups.
Track a running minimum floor over the unconditional candidates instead,
which subsumes the old same-group check for the empty capability.
Every spec skipped non-meta entries, so with zero meta entries in the index
all five bodies were no-ops. Aligning GalleryModel.IsMeta() with
GalleryBackend.IsMeta(), whose semantics are deliberately opposite, would
have made all of them pass while checking nothing. Extract each invariant
into a helper over a slice of entries returning the violations it finds, and
cover those helpers with synthetic fixtures so the logic stays tested at zero
meta entries. The index-driven specs are now a thin application of already
proven logic.
Also assert the index parses non-empty, report every violation in one run
rather than aborting on the first, and parse the index once for the suite.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Adds Ginkgo specs that parse the shipped gallery/index.yaml and enforce
the invariants that keep meta entries safe: a legacy url fallback equal
to the final candidate's url, references only to existing non-meta
entries, a min_vram floor on every candidate but the last-resort one,
a capability drawn only from the vocabulary the system can report, and
descending VRAM floors within a capability group.
The capability check is the only compensating control for a typo there.
Candidate matching is a case-sensitive exact comparison against
SystemState.DetectedCapability(), so an unknown value never matches and
falls through silently instead of erroring. The vocabulary therefore
mirrors the raw return set of getSystemCapabilities(), which notably
excludes "cpu": that is a fallback key inside Capability(capMap) on the
meta backend path, never a reported capability. A CPU-only host reports
"default".
These pass vacuously until the pilot meta entry lands; the guard is
intentionally in place before the thing it guards.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
ResolveMetaModel detached the resolved entry's Overrides and ConfigFile with
maps.Clone, which only copies the top level. Gallery overrides are nested in
practice (parameters.model is near-universal) and the install path merges the
caller's request with mergo.WithOverride, which recurses into nested maps and
overwrites them in place, so the gallery entry's own inner maps were still
reachable and still got rewritten by the last caller to install.
Copy both maps all the way down instead, recursing through the container shapes
a YAML decoder produces. ConfigFile is not mutated on the install path today,
but it carries the same kind of nested payload and leaving it shallowly cloned
would invite the bug back.
Also fix two specs that passed whether or not their target fix was present:
- "does not write the caller's overrides back into the gallery entry" re-read
the catalog from disk, which re-unmarshals fresh structs and so cannot
observe in-memory aliasing. It now asserts against the in-memory gallery
entry and drives the real mergo merge.
- "round-trips the resolution record to disk under the meta's name" asserted a
name that is already correct in the config_file branch. It now drives the url
branch via a file:// fixture, where the meta-name overlay actually applies.
Both were verified red by reverting their fix.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Six review findings on the meta-entry install path.
Pin recall was keyed on the gallery entry name while applyModel writes the
record under the install name (req.Name when supplied), so a meta installed
under a custom name with a pin lost that pin on reinstall and was silently
re-resolved onto a different variant, possibly swapping its backend. Compute
the install name with applyModel's own precedence before the recall.
ResolveMetaModel returned a shallow struct copy, so the resolved entry's
Overrides aliased the gallery entry's map and the install path's in-place
mergo merge wrote the caller's request into the shared catalog. Detach
Overrides, ConfigFile, AdditionalFiles, URLs and Tags. Not exploitable today
only because this path re-unmarshals the gallery per call, which is a
property nobody should have to rely on.
Also: overlay the meta's name onto the persisted config for meta installs so
the gallery file no longer records the variant's name; move the pinned-VRAM
warning below the variant validation so a pin naming a nonexistent entry does
not warn about VRAM before failing for an unrelated reason; and stop seeding
config.URLs in the config_file branch, which duplicated every declared URL.
Add seven network-free specs driving InstallModelFromGallery with a meta
entry: variant payload wins over the meta's legacy url fallback, the
resolution record round-trips to disk, a pin is recorded and honored on
reinstall including under a custom install name, and the resolved entry does
not alias the gallery's maps.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Meta gallery entries carry an ordered candidate list; at install time the
first candidate the host satisfies is resolved and its payload installed
under the meta's name, so the model keeps a stable name regardless of which
variant backs it. The resolution is recorded in the installed gallery
config so a reinstall honors a prior pin and operators can see the backing
variant.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
A gallery entry with a non-empty candidates list is a meta entry: it names
an ordered list of concrete entries and resolves to the first one the host
can satisfy, instead of describing model files directly.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Candidate is one option in a meta entry's ordered variant list. It names a
concrete gallery entry and declares when that entry suits the host.
EffectiveMinVRAM resolves the VRAM floor, letting an authored min_vram win
over a nightly-inferred one. An unparseable floor errors instead of being
treated as absent: swallowing a typo would turn a constrained candidate into
an unconstrained one and select a too-large variant rather than fail loudly.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
ParseSizeString accepted only SI suffixes, so a "20GiB" floor was rejected
outright. Model and VRAM sizes are conventionally quoted in IEC units, and
silently reading GiB as GB would understate a floor by about 7%.
Purely additive: these inputs previously returned an unknown-suffix error.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
ReportedCapability was added with a body identical to the existing
DetectedCapability. Keep one accessor and move the specs onto it, since
DetectedCapability had no direct coverage of its no-fallback behavior.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Model meta gallery entries express hardware fallback through candidate
ordering rather than a capability map, so they need the undecorated
detected capability string without Capability's default/cpu fallback
chain.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The Vite build emitted path-absolute asset URLs (base: '/'). index.html
entry scripts and the favicon were rewritten to include the reverse-proxy
prefix in serveIndex, but two reference kinds are not in index.html and so
bypassed that rewrite:
- CSS `url()` font references (e.g. Font Awesome .woff2), which the browser
resolves relative to the stylesheet and which `<base href>` never affects
- lazily-imported route chunks, whose preload base came from the absolute
Vite base
Under a subpath mount (X-Forwarded-Prefix: /llm/) both were fetched from the
origin root, 404ing — missing-glyph "tofu" icons and broken lazy-loaded pages.
Switch Vite to a relative base ('./') so every generated URL resolves against
the file that references it: CSS fonts and route chunks now load from
`/llm/assets/...`, and index.html's now-relative entry refs resolve via the
`<base href>` serveIndex already injects on every response. Root deployments
are unaffected. The existing path-absolute rewrite in app.go still covers the
public `/favicon.svg`.
Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The weekly "Dependabot Updates" pip job for /backend/python/diffusers has been
failing with `private_source_authentication_failure` against the Jetson pip
index (https://pypi.jetson-ai-lab.io/jp6/cu129/), referenced by that backend's
requirements-l4t12.txt. diffusers is the only dependabot-configured pip
directory that pulls from that private index, so it is the only update job that
fails; the other backends update cleanly.
torch and transformers are deliberately pinned in this backend for
reproducibility (see backend/python/diffusers/requirements-*.txt and #9979), so
we do not want dependabot bumping them anyway. Ignoring both dependencies for
this directory stops dependabot from resolving them against the unreachable
Jetson index and keeps the weekly update job green, without removing update
coverage for the rest of the backend's dependencies.
Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The managed-artifact materializer stages a HuggingFace snapshot into a
directory (.artifacts/huggingface/<key>/snapshot/). That is the right load
target for directory-consuming backends (transformers, vLLM, diffusers, ...),
but PrimaryArtifactSpec inferred a managed artifact from ANY HuggingFace-shaped
model reference regardless of backend. A single-file backend such as llama.cpp
or whisper was therefore handed the snapshot directory instead of the weight
file and failed to load it.
The /import-model importer already guards this with a backend allow-list
(managedArtifactBackends), but the loader-side inference did not. Move the
allow-list into core/config as IsManagedArtifactBackend and apply it in
PrimaryArtifactSpec: only directory-consuming backends may have an artifact
inferred from a bare reference; every other backend stays on the legacy
download-to-file path. An explicit artifacts: block still bypasses the gate,
where single-file snapshot resolution handles the load path.
The importer now shares the same predicate, so both paths agree on which
backends auto-materialize.
Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
fix(model-artifacts): load single-file HF snapshots from the file, not the dir
The managed Hugging Face artifact materializer (#10825) always pointed
backends at the snapshot *directory*
(.artifacts/huggingface/<key>/snapshot). For a single-file model
reference such as huggingface://nomic-ai/nomic-embed-text-v1.5-GGUF/nomic-embed-text-v1.5.f16.gguf,
the GGUF lives *inside* that directory, so llama.cpp was handed a
directory and failed with "gguf_init_from_reader: failed to read magic".
This has kept the tests-aio job red on master since the feature merged
(the embeddings e2e tests could not load text-embedding-ada-002).
Record the single file of a one-file snapshot as Resolved.PrimaryFile and
have ModelFileName() resolve to snapshot/<PrimaryFile> when it is set.
Multi-file snapshots (e.g. transformers repos consumed as a directory)
keep pointing at the snapshot directory. PrimaryFile is derived from the
resolved contents and is deliberately excluded from the artifact cache
key. estimateModelSizeBytes now derives the snapshot directory from the
cache key instead of ModelFileName(), so its manifest lookup is unaffected
by the file-vs-directory resolution.
Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The artifact progress bridge mapped every PhaseVerifying event to a flat
95%. The materializer emits PhaseVerifying once per file (from each file's
AfterDownload hook) and downloads run sequentially, so the first small file
to finish pinned the bar at 95% - and, because progress is monotonic, it
stayed at 95% for the entire remaining download (e.g. a 70GB checkpoint
reporting 95% at 410MB / 69.7GB).
Track per-file verify proportionally to the running aggregate bytes, the
same way downloading does. CurrentBytes already reflects "completed files +
this file", so the percentage advances honestly. The flat 95%/99% is now
reserved for the genuinely once-per-install Committing/Persisting phases.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The Operate → Traces "API Traces" panel already recorded who made each
request (user_id/user_name) but never showed it, and did not capture the
caller's network identity at all. Operators asked to see the requesting
user (#10886) and the client IP + user agent (#10887) so a trace can be
attributed to who/what issued it.
Backend: add ClientIP and UserAgent to APIExchange and populate them from
echo's c.RealIP() (honours X-Forwarded-For / X-Real-IP behind a trusted
proxy) and the request's User-Agent header. Both are omitempty and the
/api/traces swagger response is map[string]any, so this is additive.
UI: add a sortable "User" column to the API traces table and a metadata
block (User / Client IP / User Agent) at the top of the expanded row
detail. Fields render only when present, so older buffered traces and
unauthenticated/local requests degrade cleanly.
Adds an e2e spec covering the new column value and the expanded metadata.
Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
The Operate -> Traces table rendered the request time with the time of day
only, so entries that span more than one day were ambiguous. Add a
formatDateTime helper (localized date + existing time-with-millis) and use it
for the Traces "Time" column, keeping the cell on a single line. The shared
formatTimestamp used by the log views is unchanged.
Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
