feat(gallery): let one gallery entry offer several builds of the same model (#10943)

* feat(system): expose raw detected capability for model meta resolution

Model meta gallery entries express hardware fallback through candidate
ordering rather than a capability map, so they need the undecorated
detected capability string without Capability's default/cpu fallback
chain.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* refactor(system): drop duplicate capability accessor, cover DetectedCapability

ReportedCapability was added with a body identical to the existing
DetectedCapability. Keep one accessor and move the specs onto it, since
DetectedCapability had no direct coverage of its no-fallback behavior.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): parse IEC binary size suffixes (KiB..PiB)

ParseSizeString accepted only SI suffixes, so a "20GiB" floor was rejected
outright. Model and VRAM sizes are conventionally quoted in IEC units, and
silently reading GiB as GB would understate a floor by about 7%.

Purely additive: these inputs previously returned an unknown-suffix error.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): add Candidate type for meta model entries

Candidate is one option in a meta entry's ordered variant list. It names a
concrete gallery entry and declares when that entry suits the host.

EffectiveMinVRAM resolves the VRAM floor, letting an authored min_vram win
over a nightly-inferred one. An unparseable floor errors instead of being
treated as absent: swallowing a typo would turn a constrained candidate into
an unconstrained one and select a too-large variant rather than fail loudly.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): add hardware-aware model variant resolver

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): allow gallery model entries to declare variant candidates

A gallery entry with a non-empty candidates list is a meta entry: it names
an ordered list of concrete entries and resolves to the first one the host
can satisfy, instead of describing model files directly.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): resolve meta model entries to hardware-appropriate variants at install

Meta gallery entries carry an ordered candidate list; at install time the
first candidate the host satisfies is resolved and its payload installed
under the meta's name, so the model keeps a stable name regardless of which
variant backs it. The resolution is recorded in the installed gallery
config so a reinstall honors a prior pin and operators can see the backing
variant.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): key meta pin recall on the installed name and detach resolved entries

Six review findings on the meta-entry install path.

Pin recall was keyed on the gallery entry name while applyModel writes the
record under the install name (req.Name when supplied), so a meta installed
under a custom name with a pin lost that pin on reinstall and was silently
re-resolved onto a different variant, possibly swapping its backend. Compute
the install name with applyModel's own precedence before the recall.

ResolveMetaModel returned a shallow struct copy, so the resolved entry's
Overrides aliased the gallery entry's map and the install path's in-place
mergo merge wrote the caller's request into the shared catalog. Detach
Overrides, ConfigFile, AdditionalFiles, URLs and Tags. Not exploitable today
only because this path re-unmarshals the gallery per call, which is a
property nobody should have to rely on.

Also: overlay the meta's name onto the persisted config for meta installs so
the gallery file no longer records the variant's name; move the pinned-VRAM
warning below the variant validation so a pin naming a nonexistent entry does
not warn about VRAM before failing for an unrelated reason; and stop seeding
config.URLs in the config_file branch, which duplicated every declared URL.

Add seven network-free specs driving InstallModelFromGallery with a meta
entry: variant payload wins over the meta's legacy url fallback, the
resolution record round-trips to disk, a pin is recorded and honored on
reinstall including under a custom install name, and the resolved entry does
not alias the gallery's maps.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): deep-copy meta overrides and make two specs functional

ResolveMetaModel detached the resolved entry's Overrides and ConfigFile with
maps.Clone, which only copies the top level. Gallery overrides are nested in
practice (parameters.model is near-universal) and the install path merges the
caller's request with mergo.WithOverride, which recurses into nested maps and
overwrites them in place, so the gallery entry's own inner maps were still
reachable and still got rewritten by the last caller to install.

Copy both maps all the way down instead, recursing through the container shapes
a YAML decoder produces. ConfigFile is not mutated on the install path today,
but it carries the same kind of nested payload and leaving it shallowly cloned
would invite the bug back.

Also fix two specs that passed whether or not their target fix was present:

- "does not write the caller's overrides back into the gallery entry" re-read
  the catalog from disk, which re-unmarshals fresh structs and so cannot
  observe in-memory aliasing. It now asserts against the in-memory gallery
  entry and drives the real mergo merge.
- "round-trips the resolution record to disk under the meta's name" asserted a
  name that is already correct in the config_file branch. It now drives the url
  branch via a file:// fixture, where the meta-name overlay actually applies.

Both were verified red by reverting their fix.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(gallery): lint meta model entry invariants in index.yaml

Adds Ginkgo specs that parse the shipped gallery/index.yaml and enforce
the invariants that keep meta entries safe: a legacy url fallback equal
to the final candidate's url, references only to existing non-meta
entries, a min_vram floor on every candidate but the last-resort one,
a capability drawn only from the vocabulary the system can report, and
descending VRAM floors within a capability group.

The capability check is the only compensating control for a typo there.
Candidate matching is a case-sensitive exact comparison against
SystemState.DetectedCapability(), so an unknown value never matches and
falls through silently instead of erroring. The vocabulary therefore
mirrors the raw return set of getSystemCapabilities(), which notably
excludes "cpu": that is a fallback key inside Capability(capMap) on the
meta backend path, never a reported capability. A CPU-only host reports
"default".

These pass vacuously until the pilot meta entry lands; the guard is
intentionally in place before the thing it guards.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(gallery): close coverage gaps in the meta entry lint

The ordering invariant grouped candidates by capability and asserted floors
descend within a group. A candidate with an EMPTY capability matches every
host, so it does not belong in its own group: it dominates every later
candidate whose floor is at or above its own, across capability groups.
Track a running minimum floor over the unconditional candidates instead,
which subsumes the old same-group check for the empty capability.

Every spec skipped non-meta entries, so with zero meta entries in the index
all five bodies were no-ops. Aligning GalleryModel.IsMeta() with
GalleryBackend.IsMeta(), whose semantics are deliberately opposite, would
have made all of them pass while checking nothing. Extract each invariant
into a helper over a slice of entries returning the violations it finds, and
cover those helpers with synthetic fixtures so the logic stays tested at zero
meta entries. The index-driven specs are now a thin application of already
proven logic.

Also assert the index parses non-empty, report every violation in one run
rather than aborting on the first, and parse the index once for the suite.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* ci(gallery): add nightly denormalization of meta model candidates

Fills the read-only backend, quantization and inferred_min_vram fields on
meta gallery candidates and opens a PR, modeled on the existing
checksum_checker job. Computing these needs network access, so it happens
nightly rather than at install time.

An authored min_vram is never modified: a human who measured a real load
knows more than a pre-download estimate does.

The index is rewritten via yaml.Node rather than a document round-trip. A
full round-trip reflows all ~26k lines of gallery/index.yaml, which would
bury the computed values and make the nightly PR unreviewable. The rewrite
touches only the three derived keys, so authored styling survives and a run
that computes nothing leaves the file untouched.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ci): keep the gallery denormalize diff reviewable and self-healing

The nightly denormalization job edits YAML nodes instead of round-tripping
structs so its PR stays small enough for a human to review, but the write
path undid that: yaml.Marshal re-encoded the node tree at yaml.v3's default
4-space indent and dropped the leading document marker, reflowing roughly
6000 lines around the handful of real changes. Encode through
yaml.NewEncoder at the index's authored 2-space indent and restore the
header. A write that changes three fields now changes three lines.

Stale inferred_min_vram values were also never cleared. Both skip paths
(an authored min_vram is present, or the candidate is the last resort)
returned before touching the field, so a candidate that gained a floor or
became the last resort after a reorder kept an inferred value that
EffectiveMinVRAM reported as a real constraint, failing the meta lint with
no way for the job to self-heal. Clear the field before both skips.

The workflow discarded a whole night's work on any single failure: the
program exits 1 when a candidate cannot be estimated, which aborted the job
before the PR step, so one unreachable candidate blocked every other
refresh indefinitely. Capture the status, open the PR with what was
computed, mark the PR body as partial, and fail the run afterwards so the
problem still surfaces.

Also preserve the index's existing file mode instead of forcing 0644, and
drop the redundant //go:build ignore tag, since Go already skips dot
directories and the sibling modelslist.go carries no tag.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): add nanbeige4.1-3b meta entry with hardware-resolved variants

Adds the first real meta entry to the gallery index. It resolves to the
Q8_0 build on hosts with at least 6GiB of VRAM and to the Q4_K_M build
everywhere else, installing either payload under the stable name
nanbeige4.1-3b.

The entry carries a url equal to its final candidate's url. LocalAI
releases that predate candidates support parse the index non-strictly
and drop the key silently, so without that url they would list the entry
and install nothing. A regression spec parses the index the way those
releases do and asserts every meta entry stays installable for them.

Also teaches core/schema/gallery-model.schema.json about candidates. The
schema sets additionalProperties: false at the top level, so an author
following CONTRIBUTING.md and adding the yaml-language-server comment
would otherwise get a validation error on this entry.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): make candidate entries complete, installable entries

Reworks hardware-resolved gallery variants after a design pivot. There is no
longer a separate "meta" entry kind. A gallery entry is a normal, complete
entry that may additionally carry candidates:, a list of hardware-gated
upgrades over itself, and the entry is itself the last-resort candidate.

The previous design relied on a bare url: as the fallback for LocalAI releases
that predate candidates support. That fallback is empty in practice: none of
the 80 gallery/*.yaml files carry a top-level files:, and 1216 of 1281 index
entries carry their payload in the index entry itself, so a url alone yields a
config template with nothing to download. Since every released LocalAI reads
gallery/index.yaml live from master, merging a payload-less entry would have
shown every existing user a model that installs to a broken state. Making the
entry its own base candidate removes the problem at the root: old clients drop
the candidates key and install the entry exactly as they do today.

Resolution order is now explicit pin, then capability plus VRAM over the
declared upgrades, then the entry itself. The entry ALWAYS installs: when its
own min_vram or capability is unmet the installer warns and installs it
anyway, because there is nothing below it and refusing would make the gallery
behave worse the newer the client is. A pin naming the entry's own name is
valid and is how an operator declines an upgrade.

IsMeta() becomes HasCandidates(), ResolveMetaModel becomes ResolveVariant, and
the persisted meta_name record key becomes entry_name. GalleryBackend.IsMeta()
is a separate concept and is untouched.

The lint drops the three rules the pivot makes wrong (url equality with the
final candidate, no inline payload, unconstrained final candidate) and gains
one: the entry's own floor must sit strictly below every candidate's, since a
base that outranks a candidate makes that candidate unreachable.

The pilot entry is now the existing nanbeige4.1-3b-q4, which gains a 2GiB
floor of its own and a single 6GiB upgrade to nanbeige4.1-3b-q8, replacing the
separate nanbeige4.1-3b entry added in d0d441bb4.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): select model variants by hardware fit, not authored order

Gallery entries could already carry a list of alternatives, but selection was
an authored, ordered, first-match policy: every candidate declared a
`capability` string and the VRAM floors had to descend in a hand-tuned order.
That pushed hardware knowledge onto whoever edits the gallery and made ordering
load-bearing, so a reordered list silently changed what users installed.

None of it was necessary. SystemState.IsBackendCompatible already derives
hardware support from a backend name alone: it knows MLX and metal are
Darwin-only, CUDA is NVIDIA-only, ROCm AMD-only, SYCL Intel-only. Selection can
read that instead of asking authors to restate it.

Authoring is now just a list of names:

    - name: qwen3.6-27b
      min_memory: 4GiB
      variants:
        - model: qwen3.6-27b-mlx-8bit
        - model: qwen3.6-27b-gguf-q8
          min_memory: 28GiB

and all the intelligence moved into the selector. Given a host it drops the
variants whose backend cannot run here, drops those whose known memory
requirement exceeds what the host has, and takes the LARGEST of what is left,
because a bigger footprint is a higher quality quantization of the same model.
A variant of unknown size is kept, since nothing proves it does not fit, but it
ranks last so a proven fit always beats a guess. An explicit pin still wins
outright, and if nothing survives the entry installs its own payload: the base
always installs, this never refuses.

Available memory is VRAM when a GPU was detected and system RAM otherwise, read
through xsysinfo so a cgroup limit is honored and a container gets its own
limit rather than the node's RAM.

Capability disappears entirely, from the types, the schema and the lint. VRAM
and RAM collapse into one `min_memory`, because a model's footprint is roughly
the same wherever it lives and one figure is compared against whichever applies.
The lint rules about ordering, the capability vocabulary and floor
relationships are deleted with the hazards they described; what remains is that
every variant names an entry that exists and does not itself declare variants,
plus that any memory figure actually parses.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* gallery: size model variants with a live probe, drop the nightly denormalizer

Selection needs each variant's size to decide whether it fits and to rank
largest-first. That figure was written into the index by a nightly job, which
made the gallery carry a derived value that could drift from the entry it was
derived from. Derive it at install time instead.

pkg/vram already sizes a model without downloading it, and the gallery UI
already uses it: a remote GGUF header range-fetch, then an HTTP HEAD for the
content length, then any declared size:. It caches its results, so reuse it
rather than writing a second probing path.

A probe failure must never fail an install, so an unprobeable variant is
treated as unknown: it survives the memory filter, because nothing proves it
does not fit, and it ranks last, so a known-good fit always beats a guess. If
every probe fails, selection still terminates on the base entry.

The probe is injected through ResolveEnv rather than called directly, for the
same reason the backend compatibility check is: specs pin an exact size, or an
exact failure, without reaching the network.

With that in place three things are dead weight and go:

- The nightly job and the fields it populated. Variant.Backend was redundant
  because the backend is resolved live from the referenced entry during
  selection, and Quantization was display-only that nothing read.
- min_memory on the base entry. The base always installs and its floor could
  only warn, so it could not change any outcome.
- The lint rules and schema entries for both.

min_memory on individual variants stays, as the override for when the probed
size is wrong. An authored figure now suppresses the probe entirely rather
than merely outranking it, so it costs no round trip.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): expose model variants for selection over API, CLI and MCP

A gallery entry may carry `variants:`, alternative builds of the same model.
Selection already worked at install time, but nothing could see what an entry
offered or ask for a specific build, so the feature was undrivable.

Listing: `GET /api/models` now reports `variants` and `auto_variant` for the
entries that declare variants. Each variant carries its resolved backend, its
measured size and whether it fits this host. `auto_variant` is what installing
without a choice would pick right now.

The new gallery.DescribeVariants runs the same variantOptions + SelectVariant
pass the installer runs, so the reported default cannot drift from what
installing actually does, and HostResolveEnv is extracted so both derive the
host and share pkg/vram's probe cache from one place.

Performance: an entry that declares no variants returns early without touching
the probe, so the ~1280 ordinary entries cost exactly what they cost before.

Selection: `variant` is accepted on POST /models/apply, as a query param on
POST /api/models/install/:id, on the gallery apply file/string request, as
`local-ai models install --variant`, and as a parameter on the install_model
MCP tool (both the httpapi and inproc clients). Empty means auto-select.

An unknown variant name now fails the install naming what was requested. This
closes a real hole: an entry declaring no variants short-circuits before
selection runs, so a requested variant was previously dropped silently and the
install reported success.

startup.InstallModels ends in a variadic model list, so install options could
not be appended to it; InstallModelsWithOptions is added alongside and
InstallModels delegates to it. No caller signature changed.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): drop the redundant variant min_memory field

Variant.MinMemory was an authored override for when the live probe misreads
a variant's footprint. It duplicated an existing field: probeEntryMemory
already passes the entry's declared size: into EstimateModelMultiContext,
whose cascade prefers that declared size over its own guesswork. Correcting
size: on the referenced entry fixes the figure for every consumer rather
than only for variant selection, so min_memory shadowed the right answer.

A variant is now nothing but a name. Its effective size is exactly the probe
result, and an unknown stays unknown: it survives the filter and ranks last.

EffectiveMemory loses its error return along with the field. The authored
string was the only thing that could fail to parse, so the error had no
remaining source and was propagating dead nil-checks through SelectVariant,
DescribeVariants and the pin warning.

Selection behaviour is unchanged. The specs covering probe-derived sizing,
ranking, filtering, the unknown-size path, pin recall, entry/variant
metadata split and deep-copy isolation all survive; the three install specs
that needed a definite size now declare it through the referenced entry's
own size:, which exercises the documented escape hatch directly.

gallery/index.yaml is untouched: no entry ever carried the key.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): rank the entry's own build against its variants

Variant selection pulled the declaring entry's own payload, the base, out
of the candidate set and consulted it only once every declared variant had
been rejected. Two real failures followed.

A variant whose size the probe cannot determine deliberately survives the
memory filter, because nothing proves it does not fit. As the only survivor
it then won outright on any host, however small: a 2GiB machine installed an
unmeasured variant in preference to the 4GiB build the entry itself ships,
with no warning. 241 of the 1280 current index entries carry no files and no
size, which is exactly that shape.

"Largest wins" also broke whenever the base was the largest. An author
writing a Q8 entry that offers a Q4 downgrade for small hosts, a natural
shape that nothing in the lint, schema or docs discourages, had the Q4
installed on every large host instead.

Make the base an ordinary participant. It is still exempt from both filters,
so selection always terminates on something installable, but it is now
ranked against the variants: a proven fit first and largest, then the base,
then any variant whose size nothing could measure. Both failures disappear
together. The base is probed for its size accordingly, which it was not
before, because an unsized base would lose every contest to an unmeasurable
variant.

FellBackToBase is kept but narrowed to "no declared variant survived",
rather than "the base was chosen", since the base now also wins on merit and
that is not worth warning about.

A recalled variant pin also became a permanent install failure. A pin the
caller supplies on this request must stay fatal, but one recalled from
._gallery_<name>.yaml can be invalidated by any later gallery edit, and
failing on it turned one rename into a model that could never be reinstalled
or upgraded again short of deleting a dotfile the user has never heard of.
A stale recalled pin is now dropped with a warning naming it, and selection
runs as if it had never been recorded.

Also drop the last textual reference to two abandoned designs from the
DetectedCapability comment, correct the documented variants JSON example,
which showed a memory_bytes of 0 that omitempty makes impossible, and remove
an em dash from the install skill.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): budget variant memory from RAM when a GPU reports no VRAM

Variant selection read its memory budget from VRAM whenever a GPU
capability was detected, and from system RAM only when none was. Apple
Silicon satisfies the first branch and fails the premise: arm64 macs report
the metal capability unconditionally, without probing anything, while
TotalAvailableVRAM has no discrete VRAM pool to find and returns zero. The
budget therefore came out as zero on every Mac.

Zero drops every variant carrying a known size, so the base build was
installed on all of them however much memory the machine had. The feature
was inert on the platform, and silently: falling back to the base is a
legitimate outcome, so nothing looked wrong.

Take VRAM only when it is actually a number, and fall back to RAM
otherwise. On a unified-memory host RAM is not an approximation of the
budget, it is the budget, since the GPU shares it. A discrete GPU whose
VRAM could not be read also lands on RAM, which overstates what the card
holds but understates nothing the host has; the previous zero understated
both.

An unreadable RAM figure still yields zero and still installs the base, so
a genuinely unknown host is not talked into a larger download.

This is what turned tests-apple red: "installs a fitting variant's payload
under the entry's own name" asserts on selection, and the runner resolved
to the base because its budget was zero. The added specs pin the branch
directly rather than relying on a macOS runner to notice again.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): add a model variant picker to the models gallery

PR #10943 shipped the server side: a gallery entry may declare `variants:`,
`GET /api/models` attaches `variants` and `auto_variant` to declaring
entries, and `POST /api/models/install/:id` accepts a `variant` query
parameter. Nothing in the UI consumed any of it, so the feature was not
reachable from the browser. This wires it up.

modelsApi.install takes an optional second argument and appends an encoded
`?variant=` only when one is given, so every existing call site keeps
sending exactly the request it sent before.

On the models table, an entry that declares variants gets a split button.
The primary Install still installs the auto-selected build, because auto is
the default and the point of the feature; the chevron opens a menu for a
deliberate override. It follows the Backends.jsx precedent: one shared
Popover re-anchored per row, rendering .action-menu items, which brings
Escape, outside-click and focus return along with it. An entry that
declares no variants renders exactly as it did before.

A variant that does not fit is dimmed but stays selectable, since the server
honors an explicit choice with a warning rather than refusing it.

memory_bytes is omitempty on the wire, so an absent key means the size is
unknown and never zero. A single helper guards both the menu and the detail
row, because formatBytes would otherwise render a falsy value as "0 B",
which reads as "needs nothing".

The expanded detail row gains a Variants section listing each build's
backend, size, whether it fits, which is the entry's own build, and which
one auto-selection would pick, built from the existing DetailRow helper and
.badge classes.

Eight Playwright specs cover the picker, including that plain Install sends
no variant parameter and that choosing one sends it. One pre-existing
assertion was scoped with .first(): the Variants section legitimately adds
more llama-cpp badges to the detail row, which tripped strict mode.

UI line coverage 49.42% -> 49.36% against a 40.0 baseline and 0.8pp
tolerance; branch coverage rose 72.04% -> 72.66%.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): describe model variants from a companion endpoint

Variant description probes each referenced entry's weight files over the
network: an HTTP HEAD plus a ranged GET, serial, five seconds per probe
with no aggregate deadline. Running it inline in GET /api/models made one
listing cost (entries x variants) round trips. The Manage page fetches
with items=9999, so at 200 declaring entries that is ~1000 serial probes,
minutes of a blocked handler and gigabytes of range traffic for a single
page load. Only one entry declares variants today, but the feature exists
so that many will.

Follow the precedent already set for VRAM estimates. The listing now
reports only has_variants, a length check on loaded metadata that touches
nothing, and GET /api/models/variants/:id returns the description for one
entry, mirroring estimate/:id in route shape, auth and error handling.
DescribeVariants itself is unchanged; only its caller moved.

The picker fetches lazily at the two points where a user asks to see
variants, opening the split-button menu and expanding the detail row, and
caches per entry for the page session. An entry declaring no variants
issues no request at all.

A spec counts real HTTP hits on the weight files, so it goes red if
description becomes reachable from the listing path again through any
caller.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): filter the model gallery to entries that declare variants

The gallery is heading towards showing parent entries and hiding the
individual builds they reference, so a user sees one row per model
rather than six quantizations of it.

Adoption is a single entry today, so defaulting to that would leave a
one-row gallery. This ships the migration-phase inverse instead: the
default is untouched, and a toggle narrows the list to only the entries
that declare variants. It previews the end state and changes nothing
until someone asks for it.

The filter is server-side, next to term/tag/backend/capability and above
the pagination arithmetic. The listing paginates at 9 items, so
narrowing on the client would leave totalPages and availableModels
describing the unfiltered set and hand the user empty pages. It selects
on HasVariants(), which reads already-loaded metadata, so it issues no
variant probes.

The parameter is named has_variants after the listing field it selects
on, and is compared against "true" like the other boolean query params
(all_users, save_checkpoint), so has_variants=false reads as absent.
With it omitted the response is byte-for-byte what it was before.

The control is the shared Toggle component, matching the fitsFilter
toggle already on this page: same wrapper class, same icon and label
shape, same localStorage persistence. Unlike fitsFilter it resets to
page 1 on change, which a server-side filter has to do.

Stacking the toggle with a tag or backend filter easily yields nothing
while one entry declares variants, so the empty state now names the
variants filter as the cause rather than leaving a user to conclude the
gallery is broken.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): render gallery model descriptions as Markdown

Gallery descriptions are Markdown, but the React UI dumped them raw, so a
model whose description opens with an ATX heading showed a literal
"# Qwen3.6-27B [](https://chat.qwen.ai)" in the list.

Full-description areas now render through renderMarkdown (marked +
DOMPurify), matching how Backends.jsx and the Manage detail panels already
handle the same content:

  - Models.jsx expanded detail row
  - VoiceLibrary.jsx voice detail header

The truncated one-line previews must not render block Markdown: a leading
"#" would become an <h1> and wreck the row height and rhythm. They get a new
stripMarkdown() helper instead, which reduces Markdown to a single line of
readable plain text. It is used for the cell text and for the title tooltip,
since a tooltip full of "[](url)" is no better than a cell full of it:

  - Models.jsx gallery table description cell
  - Manage.jsx model and backend resource-row descriptions

stripMarkdown walks marked's lexer output rather than running regexes over
the source, so what it strips is by construction what renderMarkdown would
have rendered, and it needs no new dependency. Output lands in JSX text
nodes, so React escapes it; no new dangerouslySetInnerHTML beyond the two
full-description sites, both of which run DOMPurify.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): strip Markdown from the backends table description cell

Commit b35d630cf fixed this for gallery models but left the Backends admin
page with the same asymmetry: its detail panel renders the description
through renderMarkdown, while the collapsed table row dumped the raw gallery
string into both the cell body and the title tooltip.

That is user-visible. 40 of the 949 entries in backend/index.yaml carry
Markdown - insightface uses inline code backticks, others use lists and
links - and backend descriptions also contain embedded newlines, so the
one-line cell showed literal syntax.

The cell now runs stripMarkdown over the description once and uses the
result for the text and the title, matching Models.jsx and the
ResourceRowDesc component in Manage.jsx. The '-' placeholder is preserved,
and now also fires when a description reduces to nothing after stripping.
The detail panel is untouched and no new dangerouslySetInnerHTML is
introduced: stripMarkdown output lands in a JSX text node, so React escapes
it.

Three Playwright specs cover it: a description with a heading, inline code
and a link renders as clean text with no literal syntax and no block
element in the cell, the title tooltip carries the same stripped text, and
a backend without a description still shows the placeholder.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* ui(models): polish the variant detail view and scope rendered Markdown

The gallery detail pane rendered every field through the same two-column
label/value row, including the description. Multi-paragraph prose in a value
cell ran eight rows tall at the top of the pane on a ~1200px measure, breaking
the grid's rhythm exactly where the eye enters. Move it into its own full-width
block above the table, capped at a 68ch measure, keeping the label.

Rendered Markdown had no scoped typography anywhere in the app, so a
description opening with `#` inherited the browser default 2em inside a 13px
surface while a `##` further down was indistinguishable from body text. Add a
reusable .markdown-body block mapping h1-h6, paragraphs, lists, links, code,
blockquotes, images and tables onto the existing type scale, and apply it to
every renderMarkdown() consumer: the models detail, the backends detail, both
Manage details and the voice library detail.

Rebalance the variants list so the name leads. Backend and size drop from
badge/secondary weight to muted metadata; the FITS badge goes entirely, since
it was true of nearly every row and so said nothing, while the variant that
does not fit keeps a warning badge and a dimmed name. AUTO-SELECTED stays
marked because it answers what a plain Install produces. Rows share the
parent's grid tracks via subgrid so name, backend, size and status line up
down the list instead of raggedly following name length.

Finally, make each variant row actionable. It looked like a list of choices
but was inert text, with per-variant install hidden behind the split-button
chevron elsewhere; each row is now a button onto the existing
handleInstall(modelId, variant) path, with hover, keyboard focus and a
disabled state while an install is in flight.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): collapse the listing to one row per model

The listing supported has_variants=true, which narrowed to entries that
DECLARE variants. With adoption at three entries that showed three rows,
which is useless; it was always a placeholder.

Replace it with the view that is actually useful: the deduplicated
gallery. Show every entry installable in its own right and nothing twice,
which means the parents plus every entry nobody references, and hide only
the builds another entry already offers as a variant, since those are
reachable through their parent.

The parameter is renamed to collapse_variants accordingly: the filter is
no longer a predicate on a row's own metadata but a view over the whole
gallery. Default stays off, so the response with the parameter absent is
unchanged.

VariantReferencedIDs never reports an entry that declares variants of its
own, so parents are always visible. That guarantees every hidden entry
has a visible entry offering it, and no chain can strand a row. Variant
resolution already refuses to install such a reference, but the listing
has to stay coherent in the presence of a gallery that has one rather
than silently swallowing entries. Self-references and dangling references
hide nothing.

The referenced set is computed over the whole gallery rather than over
what the other filters left, so an entry is hidden because a parent
offers it and never because of what the user searched for. The pass is
over metadata already in memory: it resolves nothing over the network and
triggers no variant description or size probe, so the listing's zero-probe
contract still holds.

The UI toggle keeps its behaviour (persistence, page reset, clear
filters) and becomes "One row per model", which says what the user gets.
Its localStorage key moves too, since the stored value meant a different
filter.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): show the collapsed model listing by default

The gallery listing is what a user reaches for to answer "what can I
install". Answering that with several rows for the same model, one per
build, makes the reader do the deduplication the collapsed view already
does, so the collapsed view is the one to land on.

The UI now asks for collapse_variants=true unless the toggle says
otherwise. The server default is deliberately untouched: a request with
the parameter absent still returns the full listing, because other API
clients depend on that response and collapsing it under them would be a
breaking change. Opting out omits the parameter rather than sending
false, so it asks for exactly the listing everyone else gets.

The stored preference changes vocabulary from '1'/'0' to 'on'/'off'. The
previous build wrote it from an effect that runs on mount, so a stored
'0' recorded that the page had been opened rather than that anyone chose
the expanded view, and honouring it would pin every earlier visitor to a
default they never picked. Only the new vocabulary counts as a choice;
a legacy '1' meant the collapsed view and is what the new default gives
anyway, so no earlier deliberate choice is lost.

Collapsing being the default also changes what the empty state may say
about it. An opted-into filter can be named as the cause of an empty
result; a default cannot, so the filters keep the top line and the
collapsed view drops to a hint below it, shown only once filters are
narrowing the set. For the same reason "Clear filters" now restores the
collapsed default instead of switching it off, and the toggle alone no
longer counts as a filter worth offering to clear.

The label stays "One row per model": it describes the view the user is
looking at rather than an action, so it reads the same whether it is
opted into or out of.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* gallery: group alternative builds of the same weights under variants

Sweep the gallery for entries that are alternative builds of the same
weights (different quantization, precision, or runtime format) and declare
them as variants of a single parent row, so the listing offers one row per
model instead of one row per quantization and the installer picks the
largest build that this host can actually run.

41 families over 95 entries, turning 54 entries into variants.

The parent is the bare-named entry wherever one exists, so nothing changes
about what any existing entry installs. Ranking already selects the largest
fitting build regardless of which entry is nominally the parent, so the
parent only decides the pathological case where nothing fits. For the ten
families that have no bare-named entry, the smallest build is the parent,
since that is the one that has to install when nothing fits.

Grouping was verified against the actual model filenames rather than the
entry names alone. Different parameter sizes, languages, finetunes, and
products that merely share a name prefix are left as separate rows: the
qwen3.6 APEX and pi-tune finetunes, the DFlash and MTP speculative-decoding
pairings, English-only versus multilingual Whisper, the QAT versus non-QAT
Gemma 4 weights, and the abliterated FLUX build are all distinct models.

Six parents define YAML anchors that other entries pull in with a merge key,
which would have handed their variants to every merging child. For the two
depth-anything anchors that would have made fourteen unrelated entries
advertise the base model's builds as their own. All 26 merging children
therefore carry an explicit empty variants list, which overrides the merged
key and is equivalent to the key being absent.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): rank model variants by host backend preference

Variant auto-selection filtered candidates by whether their backend can
run on the host, then ranked the survivors by size alone. The backend
never influenced the choice beyond that gate, so a Mac offered both an
MLX build and a llama.cpp build kept neither filtered and installed
whichever was larger, leaving the native accelerated runtime unused. The
same held for CUDA against CPU on NVIDIA and ROCm against Vulkan on AMD.

Rank by the host's backend preference between the fit tier and size: fit
stays a filter, preference decides among the builds the host can equally
hold, and size still separates builds on equally preferred runtimes.

The preference data stays in one declarative table in pkg/system, now
read by a prefix lookup instead of a switch, so adding a capability or
reordering one host's runtimes is a one-line edit and the gallery's
ranking code carries no per-backend branching. MLX joins the metal rule
ahead of metal itself, which is inert for the existing alias-resolution
consumer because no alias group holds a candidate named for mlx.

An unrecognised backend, an unrecognised capability and an absent
preference list all collapse to the previous size-only ordering rather
than erroring or dropping candidates.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): rank variants by engine name, not backend build tag

Variant auto-selection ranked candidates with
SystemState.BackendPreferenceTokens, but that function and the variant
ranker speak different vocabularies.

BackendPreferenceTokens returns BUILD TAGS ("cuda", "rocm", "sycl",
"vulkan", "metal", "cpu"). It exists to match installed backend build
directory names like "llama-cpp-cuda-12" during alias resolution in
ListSystemBackends. Variant ranking instead matches a gallery entry's
`backend:` value, which is an ENGINE NAME: "llama-cpp", "vllm",
"vllm-omni", "sglang", "mlx" and the rest. No engine name in
gallery/index.yaml contains "cuda", "rocm", "sycl" or "vulkan".

preferenceRank matches by substring, so on an NVIDIA host the tokens
[cuda, vulkan, cpu] matched neither "llama-cpp" nor "vllm", every
candidate scored identically and size alone decided. The NVIDIA, AMD,
Intel, darwin-x86 and vulkan rules were all inert. Only metal appeared
to work, and only because the token "mlx" happens to equal an engine
name. The mismatch does not error, it silently deletes the feature.

Separate the two vocabularies. backendBuildTagPreferenceRules keeps the
build tags and its original output for every capability, including
metal, whose "mlx" token is removed again; its alias-resolution consumer
is byte-identical to before. engineNamePreferenceRules is new, holds
engine names, and is read by the new EnginePreferenceTokens, which
HostResolveEnv wires into the renamed ResolveEnv.EnginePreference. Both
tables sit adjacent under one block comment naming each vocabulary and
each consumer, and share one lookup helper so their semantics cannot
drift.

On NVIDIA the order is vLLM, then SGLang, then llama-cpp: vLLM is the
throughput engine and a model published with a vLLM build is published
that way because that build is the one worth running. AMD and Intel get
the same order, since rocm and intel builds of both serving engines
ship. Metal prefers mlx over llama-cpp. Vulkan prefers llama-cpp, the
only LLM engine with a Vulkan build. darwin-x86 and unknown
capabilities are deliberately absent rather than guessed at, degrading
to the size-only ordering that predates preference.

preferenceRank stays generic and names no engine and no capability, so
adding a runtime remains a one-line table edit.

Specs pin the NVIDIA and metal rules through the live table and the real
HostResolveEnv wiring, so emptying the engine table or wiring the build
tag source back in both go red. A regression table asserts
BackendPreferenceTokens' original output per capability, and mirrored
locks assert neither table carries the other's vocabulary.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs: record that variant selection ranks by engine before size

A gallery entry can now declare variants, and selection ranks the builds a
host can run by engine preference before size. Nothing told a contributor
adding a backend that engineNamePreferenceRules exists, so a new engine would
silently rank below every known one and lose to whatever build happened to be
larger on hosts where it should have won.

Document the step where a backend is added, warn against the sibling
backendBuildTagPreferenceRules table (build tags, not engine names: the wrong
table matches nothing, scores every candidate equally and disables the
preference without erroring), and index it from AGENTS.md.

Fix the authoring and user docs, which still claimed the largest surviving
build wins. An author grouping builds under one entry has to be able to
predict what a user gets, and size alone no longer decides it.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(cli,mcp): describe variant auto-selection as preference before size

The CLI flag help and the install_model tool schema both still said
auto-selection takes the largest build that runs. Ranking now puts engine
preference ahead of size, so on NVIDIA a vLLM build wins over a larger
llama.cpp one. An assistant reading the old schema would tell users the
wrong thing.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): prefer llama.cpp over GPU serving engines on hosts with no GPU

engineNamePreferenceRules had no row for the "default" capability, which
getSystemCapabilities() returns both when no GPU is detected and when a GPU
is present but under the 4 GiB VRAM floor. A missing row yields an empty
preference list, which preferenceRank reads as "score everything equally",
collapsing variant selection to size alone.

That would be harmless if the hardware filter dropped GPU serving engines on
such a host, but it does not. IsBackendCompatible derives support from the
engine NAME, and "vllm" and "sglang" contain none of the darwin, cuda, rocm
or sycl tokens it keys on, so they fall through to its closing "return true".
A vLLM variant therefore survives on a CPU-only box and wins whenever its
build is the larger of the two on offer: the machine installs vLLM in
preference to llama.cpp.

darwin-x86 had the identical hole. It was documented as a deliberate omission
because nothing accelerates on an Intel Mac, which is true about acceleration
and wrong about consequence: with every engine tied, download size decides.

Add rows for both putting llama-cpp first. The GPU engines are enumerated
behind it rather than left unmatched: an unmatched engine already ranks below
every listed one, so llama.cpp would win either way, but unmatched engines
also tie with each other and let size decide among them. Naming them fixes
that order. MLX is left off the darwin-x86 row on purpose so it ranks last,
since IsBackendCompatible admits darwin-tokened engines on that capability
even though MLX needs Apple silicon.

Preference orders survivors and never filters, so a model published only as a
vLLM build is still installed on a host with no GPU; there is a spec for it.

Surveyed every other value getSystemCapabilities() can return. nvidia, amd,
intel and vulkan have rows; the l4t and cuda-refined values reach the nvidia
row by prefix; "apple" and "" cannot reach the vendor fallthrough because the
darwin and no-GPU branches return earlier. default and darwin-x86 were the
only live holes.

BackendPreferenceTokens and its build-tag table are untouched, and
preferenceRank stays generic, naming no engine and no capability.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* gallery: prefer speculative-decoding builds when they fit

Rank serving features between engine preference and size, so a host that can
hold a DFlash or MTP build of a model's weights installs it instead of the
plain build. Both answer faster for the same output, so whenever one survives
the filters there is no reason to take the plain build.

Precedence is now fit, then engine, then serving feature, then size. Engine
outranks the feature deliberately: a serving feature makes the right engine
faster, it does not make a wrong engine right, so a plain vLLM build still
beats a DFlash llama.cpp build on NVIDIA. Fit outranks both, and a drafter
pairing is strictly larger than the plain build, so the existing size filter
drops it on a host too small for it before this axis is consulted.

The order lives in a third preference table in pkg/system, alongside the build
tag and engine name tables. It is the odd one of the three: not keyed by
capability, because no hardware prefers a plain build over an equivalent
faster one, and matched against whole segments of a gallery ENTRY NAME rather
than as a substring of a backend value. Nothing on a gallery entry declares a
serving feature, and tags are not a usable substitute: gemma-4-e2b-it:sglang-mtp
carries an mtp tag while ornith-1.0-9b-mtp and qwen3.6-27b-nvfp4-mtp carry
none. Entry names are author-supplied free text, unlike the closed engine
vocabulary, so a short marker can turn up inside an unrelated word and whole
segment matching is what keeps smtp-assistant from ranking as an MTP build.
The block comment over the tables now documents all three together and states
what each is matched against; the ranking code names no feature, so adding one
stays a one-line edit to the table.

29c49203b rejected these entries as serving configurations rather than
alternative builds of the same weights. The definition is now "alternative ways
to serve the same model", which includes them, so regroup 14 entries under 12
parents. Judged by the files each entry points at: the qwen3.6, qwen3.5, qwen3
and deepseek pairings are the base GGUF plus a drafter, the gemma-4 QAT MTP
entries are the same QAT weights at a different quantization plus an MTP
drafter, and the two sglang MTP entries describe themselves as the same model
served with speculative decoding. Left separate: qwen3.6-27b-mtp-pi-tune, a
finetune with its own weights, and every entry whose base model LocalAI does
not ship as its own row, which is the whole Qwopus line plus gemmable-4-12b-mtp,
mimo-7b-mtp:sglang and qwen3.5-4b-dflash.

None of the twelve parents defines a YAML anchor, so no variants key can leak
through a merge key and no empty override was needed this time. The index was
edited by line insertion only.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test: check env restore errors in capability and variant specs

errcheck flagged ten unchecked os.Setenv and os.Unsetenv returns in the
specs added while the pre-commit hook was being skipped. Restoring an env
var is exactly the place a silent failure leaks state into the next spec,
so assert on it rather than suppressing the linter.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): make the mtp tag authoritative for serving-feature ranking

Variant auto-selection ranks survivors by fit, then engine, then serving
feature, then size. The serving-feature lookup read only whole alphanumeric
segments of a variant's entry name, because tags were inconsistent: every
dflash entry carried a dflash tag, but only 7 of 20 MTP entries carried an
mtp tag.

Tag the 13 untagged MTP entries, then teach the lookup to read tags as well
as names. A tag is now the authoritative signal and is compared whole and
case-insensitively, which is safe precisely because a tag is a deliberate
declaration rather than free text: there is no word-inside-a-word failure
mode, so the segment splitting the name half needs is unnecessary there.

The name check stays as a fallback rather than being replaced. Switching to
tags only would have regressed the six already-grouped entries on the day it
shipped, and would depend on tagging discipline that does not exist yet.

The lookup still names no feature, so adding one remains a one-line edit to
servingFeaturePreferenceTokens.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): make a declared tag the sole serving-feature signal

Variant auto-selection ranks survivors by fit, then engine, then serving
feature, then size. The serving-feature lookup recognised a speculative build
by either a declared tag or a whole segment of its entry name. Drop the name
half: a tag is now the only signal.

A name is author-supplied free text and a naming convention is not a contract,
so reading a marker out of one infers a capability nobody declared. The gallery
already had the failure in it: the four NVFP4 entries name MTP-bearing weights
while setting no option that enables speculative decoding, and being live
variants they were winning the feature axis without answering any faster.

overrides.options was considered as the replacement and rejected. It carries
spec_type:draft-mtp / spec_type:draft-dflash, which is what actually turns the
feature on, but that spelling is llama.cpp's config vocabulary: ds4 spells the
same feature mtp_path and sglang spells it speculative_algorithm in a
referenced config. Keying a cross-backend ranking decision on one backend's
option syntax would rank the other backends' builds as plain. Options are the
curation-time check instead, and never reach the selection logic.

With no fallback left, tag correctness is load bearing, so audit every entry
against the rule "tagged when the entry configures that feature, in whatever
vocabulary its backend uses". Three entries configure MTP untagged and gain the
tag (hy3, glm-5.2, qwythos-9b-claude-mythos-5-1m, all spec_type:draft-mtp with
no marker in their names). Four carry the tag while configuring nothing and
lose it: qwen3.6-27b-nvfp4-mtp, qwen3.6-35b-a3b-nvfp4-mtp,
qwopus3.6-27b-coder-mtp-nvfp4 and qwopus3.6-27b-v2-mtp-nvfp4, whose only option
is use_jinja:true. The dflash side was checked independently rather than assumed
consistent: all five dflash entries declare spec_type:draft-dflash and all five
are tagged, so it needed no edits.

Four entries keep a tag that a literal spec_type-only reading would strip,
because they configure MTP through a different backend: deepseek-v4-flash-q2-mtp
via ds4's mtp_path/mtp_draft, and the three sglang entries via
speculative_algorithm in their referenced configs. Stripping those would
contradict the reason spec_type was rejected as the signal and would demote four
genuinely faster builds to plain.

The index was edited by line insertion and deletion only, never round-tripped
through a serializer. A resolved-tag diff across all 1272 named entries, taken
after merge keys are applied, shows exactly these 7 changing and no entry
gaining or losing a tag through an anchor.

The two specs that pinned the name fallback are inverted rather than deleted,
since a name silently promoting a build is the regression worth guarding. The
whole-token guard survives on the tag path, where smtp must still not match mtp.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): make deepseek-v4-flash variant targets installable

Clicking install on deepseek-v4-flash failed with "invalid gallery model".
The parent entry is fine, but all four entries it was grouped with declared
neither url: nor config_file:, and applyModel needs one of the two to have
anything to build a config from. They carry urls: (plural), the informational
HuggingFace link list, which is a different field. None of the four was ever
independently installable, so grouping them routed a previously-working
install into a broken entry.

Give each the url: the parent already resolves through. virtual.yaml is a
no-op base, and applyModel passes overrides to InstallModel separately from
the fetched config, so backend: ds4, the parameters and the ssd/mtp options
all still land exactly as authored. This is the same pattern the parent and
many other GGUF entries in the index already use.

Add the lint rule that should have caught this. checkVariantReferences only
proved a target exists and is not itself a parent, which is structural
validity: an entry can exist, declare no variants, and still be
uninstallable. checkVariantTargetsInstallable mirrors applyModel's
precondition instead, and names the parent, the target and the missing
fields, because whoever hits it is reading a gallery entry and has no reason
to know applyModel exists.

The two index-driven resolution specs live in their own Ordered container:
an Ordered container stops at its first failure, so sharing one with the lint
rules let a lint breach skip them silently.

Nine further entries gallery-wide have the same defect and are unrelated to
variants, so they are broken installs that predate this branch. They are left
alone here rather than buried in a regression fix, and widening the rule to
cover every entry is deferred with them so the gate can ratchet up in one
step instead of needing a skip list.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): install entries with no url or config_file on an empty base

applyModel had three branches: fetch a base config from url:, build one from
an inline config_file:, or fail with "invalid gallery model". An entry
declaring neither is now installed on an empty base config, with overrides:
and files: supplying everything.

This is what the ~345 entries pointing at gallery/virtual.yaml were already
getting. That stub is five lines carrying name, description and license.
description and license are overwritten from the gallery entry immediately
after the fetch, and the name never reaches disk because InstallModel prefers
the install name. Crucially applyModel passes model.Overrides to InstallModel
as a separate argument rather than merging it into the fetched config, so
nothing an author writes depends on that base existing. The fetch bought a
round trip to GitHub and nothing else.

That makes f4ef80173 the wrong fix, so it is unwound. The four url: lines it
added to the deepseek-v4-flash variants are reverted: they are a pointless
network fetch now, and the family installs without them.

Relaxing the branch would hide a real authoring mistake, so a payload rule
replaces the base-config rule. An entry with no url, no config_file, no
overrides and no files installs nothing and would leave an empty model
directory while reporting success, so it is refused by name. The caller's
request counts toward the payload, because its overrides and files are merged
into the install exactly as the entry's own are. urls: (plural) is the
informational link list and does not count, which is what the four entries
that shipped broken had and why they were still uninstallable.

checkVariantTargetsInstallable asserted every variant target declares a url:
or a config_file:, which is no longer true and would now reject correct
authoring. checkEntriesInstallSomething pins what survives instead, and covers
every entry rather than only variant targets: the hazard is a half-written
stanza and a parent can be one as easily as a target. The old rule was scoped
to targets precisely because nine unrelated entries would have failed a
gallery-wide version; those nine are valid now, so the deferred ratchet
happens here in one step. 1280 entries, zero violations.

Those nine (aurore-reveil_koto-small-7b-it, lfm2-1.2b, the six liquidai_lfm2
entries and deepseek-v4-pro-q2-ssd) become installable for free. Each carries
overrides: and files:, and one of them is driven through the real install path
in a spec.

The no-fetch spec is paired rather than bare: an assertion that nothing was
fetched proves nothing unless something could have been, so a control runs the
same fixture with a url: pointing at a base config that is not there and
asserts the install fails. Only then does the identical fixture without the
url passing mean the read was skipped.

Follow-up, deliberately not here: the ~345 entries still naming virtual.yaml
can drop their url:. That is 345 index edits with their own risk, and mixing
them in would bury this change.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* ui(models): let search bypass the variant collapse, drop the toggle

The models page collapsed the gallery to one row per model by default and
offered a toggle to see every individual build. Because the collapse composed
with the search term, a build another entry offers as a variant could not be
found by typing its name, so the toggle was the only way to reach those builds
in the UI. A user who typed a name they knew existed got "no models found",
which reads as "that model does not exist".

Collapse is for browsing; search is for finding. An explicit search term now
bypasses the collapse in the listing handler, so a name lookup returns matching
entries whether or not a parent offers them. The term is trimmed once at the
top of the handler, so whitespace is neither a search nor a bypass; previously
an untrimmed blank term also narrowed the listing to whatever contained a
space. Tag and backend deliberately do not bypass: they refine a listing the
user is still reading rather than name an entry already known to exist.

That makes the toggle redundant, so it goes, along with its i18n strings in all
six locales, its localStorage persistence, its participation in "Clear filters"
and the empty-state hint telling users to turn it off. The hint was doubly
stale: it pointed at a control that no longer exists, and it was untrue exactly
when a user has a search term, since searching now sees every build. The page
always requests the collapsed listing.

The stored preference key is left inert rather than cleaned up: nothing reads
it, so a user who had the toggle off simply gets the collapsed view.

collapse_variants stays on the API, off by default, because other clients want
either view and the UI dropping its control is no reason to remove a working
parameter.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): give the models gallery filter form a deliberate structure

The filter area had accreted controls into one undifferentiated flow. The
"Fits in GPU" toggle and the backend select were direct children of
.filter-bar, the same wrapping container as the 18 taxonomy chips, so their
position was decided by how many chips happened to wrap at the current width
rather than by any layout intent. At narrow widths they were pushed past the
right edge of that container's horizontal scroll and became unreachable
entirely.

Restructure into three bands inside the house .filter-bar-group wrapper that
components/FilterBar.jsx already uses on Backends and the System tabs:

  1. query scope: search plus the backend select
  2. taxonomy: the chip row, alone, free to wrap
  3. refinements: fits-in-GPU and context size, under a hairline rule

The backend select leads the chips rather than trailing them because picking a
backend disables the use cases that backend cannot serve, so it gates the row
below it. Fits-in-GPU and context size share a band because they are one
control group: the context size is the length the VRAM estimate is computed at,
and that estimate is what the fits filter tests against.

Chips had no visible keyboard focus indicator. The global focus ring is wrapped
in :where(), so it carries the specificity of a bare :focus-visible, ties with
.filter-btn and loses on source order, leaving focused chips showing their
resting drop shadow. Restate the ring where it outranks both resting and hover.

Also: aria-pressed on the chips, a real label association and aria-valuetext on
the context slider (it steps over an index, so it announced "2"), disabled chip
styling moved off inline styles, a prefers-reduced-motion block for the chip
transition, and the hard-coded English "Context:" moved into all seven locales.

No behaviour change: same filters, same state, same requests. Page reset on
change, localStorage persistence and "Clear filters" verified unchanged.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): let the models recommendations panel fade into the background

The "Recommended for your hardware" strip rendered at full height on every
visit regardless of how many models were already installed, costing 186px at
1600px wide (287px at 1100px, where its cards wrapped to two rows) and pushing
the first gallery row to y=554 / y=703.

Make its prominence track how much the user still needs it. The panel now
defaults to a one-line summary once anything is installed, and both the
collapse choice and the existing dismissal persist:

  collapsed = explicit user choice, if one exists
            : installedCount > 0

The preference is three-valued on purpose. A boolean cannot tell "the user
expanded it" apart from "the user has never chosen", and those need opposite
handling when the installed count later crosses zero: someone who deliberately
opened the panel on an empty instance should not have it collapse out from
under them when their first model finishes installing.

Collapsed keeps the card, icon, title and a suggestion count, so the panel is
recovered by clicking what you are already looking at rather than by hunting.
Expanded is unchanged, because for a user with nothing installed it was never
the problem. Collapsed reclaims 145px at 1600 and 420, and 246px at 1100.

Models.jsx gains a statsLoaded flag: stats initializes to installed:0, so
reading it before the fetch resolves would render expanded and collapse a frame
later, which is exactly the layout shove this removes.

The dismissal key moves to the page's localai-models-* convention; the old
localai_rec_models_dismissed is still read, never written, so an existing
dismissal is honoured rather than resurrected by the rename.

Accessibility: the disclosure is a real button whose accessible name is the
visible title alone, with state on aria-expanded and aria-controls resolving in
both states, because the grid is hidden via the hidden attribute rather than
unmounted. That also keeps the four install buttons out of the tab order while
collapsed. The app's global focus ring applies; no per-component outline is
added, per the warning in App.css. Reveal animates opacity and transform only,
never height, and both it and the chevron rotation are disabled under
prefers-reduced-motion.

Only en had a recommended block, so the other six locales were falling back to
English for the whole panel. Translated the complete block rather than adding
one orphaned key to files that would still render the title in English.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(downloader): recover from a leftover .partial on non-HTTP URIs

An interrupted download leaves a `<file>.partial` behind. The partial
handling in DownloadFileWithContext gated resume on `err == nil &&
uri.LooksLikeHTTPURL()`, so for any URI that is not literally http(s)
the branch fell through to `else if !errors.Is(err, os.ErrNotExist)`,
which with a nil err is true. The download then failed with an error
wrapping nil:

  failed to check file ".../Ternary-Bonsai-27B-Q2_g64.gguf" existence: <nil>

Every gallery file URI uses `huggingface://`, so a single interrupted
download made that model permanently uninstallable until someone
deleted the partial by hand. The `<nil>` in the message compounded it
by pointing debugging at a filesystem failure that never happened.

Restructure the handling as an explicit switch over the four real
states: partial exists and is resumable, partial exists and is not
resumable (discard and restart, as already done for an HTTP server
without range support), no partial, and a genuine stat failure. The
error branch is now only reachable with a non-nil error, names the
path that was actually stat'd, and wraps with %w.

Discarding is required for correctness and not merely convenience: the
writer opens the partial with O_APPEND, so an un-resumed download would
concatenate a fresh body onto stale bytes.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): tell the models gallery's variant rows apart, and let browsing see every build

Both variant surfaces rendered name, backend and size. For two builds of one
model that is close to no information: a variant exists precisely because the
same weights are offered another way, so the backend usually matches and the
sizes usually land within a few hundred megabytes. Comparing
ternary-bonsai-27b-pq2 against ternary-bonsai-27b-q2-g64 meant reading two names
that differ by a suffix nobody has defined anywhere in the UI.

Report the quantization and the serving features on VariantView, and derive both
server-side from the referenced entry rather than parsing names in the browser,
so every client reads the same format out of the same file the installer will
hand the backend.

Quantization comes from overrides.parameters.model first, falling back to the
file list. That order is load bearing: entries routinely ship a vision tower
alongside the language model at a different quantization, so reading the file
list first reports the mmproj's format. Matching walks `-` and `.` delimited
segments right to left; `_` deliberately does not split, because it separates the
parts INSIDE a quant token and splitting on it reports Q4 for a Q4_K_M build. A
second, looser pass takes a segment's `_`-delimited tail, which catches the
gemma-4-E2B_q4_0-it.gguf style; it runs second so a precise match can never lose
to a fuzzy one further right in the name. An entry naming no format reports
nothing, which is the honest answer for a backend served from a directory of
weights.

Features are the same tag-against-vocabulary match servingFeatureRank already
ranks on, over the same host preference list. A build can therefore never be
shown as faster than one selection did not actually reward, nor rewarded without
being shown; a spec pins that agreement rather than trusting it.

The compact dropdown gets the quantization on its meta line and the bare feature
token. The detail row, which has the room, gets the quantization as its own
monospaced column so precision lines up down the list, and the feature spelled
out, because DFLASH names nothing to a user who has not met it. The referenced
entry's description stays out of both: the detail row already renders the
parent's prose above the table, and a second block per variant would push a
three-variant list past a screen to restate what the columns now say precisely.

The collapse toggle comes back. 462583f38 dropped it once search bypassed the
collapse, on the reasoning that nothing was unreachable any more. That holds for
finding a build whose name you know and does not hold for browsing: no sequence
of actions enumerated the 68 builds the default view hides. Collapse is for
browsing and search is for finding, and the toggle was the browsing half.

It goes in the refinements band 0d4823362 established, not back among the
taxonomy chips where its position depended on how many chips happened to wrap. It
leads that band because it decides how many rows the other two refine over, and
because unlike fits-in-GPU it is unconditional: a host with no GPU still browses.

The search bypass is untouched and re-checked by a spec in the toggle's default
state, since restoring the control must not restore the dead end it replaced. The
empty-state hint returns but only without a search term, because a term bypasses
the collapse and the hint would otherwise point at a control that cannot change
the result. The stored preference reads 'on'/'off' only: an older build wrote
'1'/'0' from an effect that ran on mount, so those record that the page was
opened, not that anyone chose a view.

Also fixes a latent flake it exposed. The collapse_variants spec compared whole
response bodies byte for byte, and the listing envelope carries live host
telemetry that drifts between two calls milliseconds apart, so it was asserting
on the machine's memory pressure. It now compares everything the parameter
governs -- the entries, their serialization and the paging -- and is green 25/25
where it was failing about one run in three.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): let the models gallery show a variant's full details

The variant list in an entry's expanded detail row says how the builds
differ: name, backend, quantization, size, and the auto-selected, base
and serving-feature markers. It cannot say what any one of them is. A
variant's own description, tags, license, source links and file list are
unreachable anywhere in the UI, because while the collapse is on a
variant has no gallery row of its own.

Give each variant row an info control that reveals its entry, rendered by
the same ModelDetail a top-level row gets, so a field added to the detail
view appears here too. variantData is withheld from the nested render: a
variant may declare variants of its own, and recursing would nest a
picker inside a picker two levels deep already.

An inline disclosure rather than a modal. The control sits inside a table
row that is already expanded, inside a variant list within that; a dialog
opened from there stacks a dismissal on a dismissal for a handful of
extra fields about the entry the user is already reading, and breaks the
page's own expand idiom. The third level is carried by an inset and a
left rule instead of another card.

The entry is fetched by exact name from the listing, once, on first use.
The listing already returns every field the detail view renders, and a
search term bypasses the variant collapse server-side, so no new endpoint
is needed and neither the listing nor DescribeVariants gains any work.
Expanding a row costs nothing; a variant nobody opens costs nothing. A
name the listing no longer returns is stated, not blanked: an empty panel
reads as a rendering fault rather than as a lookup that came back empty.

The control is a sibling of the install button, not a descendant, so
asking about a build can never install it.

The variant list keeps its content-sized columns via a trailing filler
track instead of max-content sizing, so the rows are unchanged while the
panel spanning them gets the pane width its file table needs.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(gallery): let search respect the collapse instead of switching it off

The models listing collapsed to one row per model, and an explicit search term
turned that off wholesale. Searching while collapsed therefore answered with the
individual builds a parent already offers, which are exactly the rows the view
the user asked for has no place for: typing "mtp" returned
qwen3.6-27b-nvfp4-mtp, a row that is invisible the moment the box is cleared.
The bypass was the right shape of fix for the wrong half of the problem. What a
search must not do is answer "no models found" for a build the gallery does
hold; that does not require abandoning the grouping the user asked for.

So the term is now matched against every entry either way, hidden builds
included, and the collapse decides how a match is reported rather than which
matches exist. Collapsing stops being a filter that drops rows and becomes a
substitution: a match on a build another entry offers is reported as that entry,
the one installable in its own right. Nothing becomes unfindable and nothing
comes back that the requested view cannot show.

Substitution happens after search, tag and backend, so every filter is judged
against the build that really carries the name, tag or backend rather than
against a parent that merely offers it; the other order would let backend=vllm
match a parent whose own backend is something else. The price is that the
surfaced row shows the parent's own metadata while the match was on a variant,
which is what grouping means, and the alternative is claiming the gallery holds
no such build. It happens before the count and the page math, so both describe
the rows actually handed out rather than the matches that produced them.

A parent already in the result keeps its own position and absorbs its matching
variants there, which is what leaves the browsing listing ordered exactly as it
was; a parent surfaced only by a variant takes the position of the first variant
that surfaced it. Either way it appears once, however many of its builds matched
and whether or not it matched itself. Search preserves gallery order rather than
scoring, so a surfaced parent has a real position rather than an invented one.

VariantParents never reports an entry that declares variants of its own, so a
parent is never itself hidden and one hop always lands on a visible row. The
handler follows exactly one anyway: refusing the second is what makes a gallery
the linter would have rejected terminate rather than loop.

The empty-state hint pointing at the toggle goes with it for every server-side
filter. Substitution means a match is always reported as some row, so the
collapse can no longer be why a term, a chip or a backend came back empty, and
naming it there sends the user to a control that cannot change the result. It
survives for the fits filter alone, which runs in the browser after the
substitution and judges the surfaced entry's own size: there the build that fits
really can be filtered out along with a parent that does not.

Searching a build's exact name while collapsed now answers with its parent, so
the result no longer contains the string the user typed. That is intended, and
the row is the one they can act on, but it is a real rough edge: nothing on the
row explains the connection. Closing it properly means reporting which variant
matched so the UI can say so, which the listing does not do today.

ResetGalleryModelCache is added for tests. The model cache is a package global
keyed by nothing, so a background refresh one spec triggers can land in the
middle of the next and answer it with the previous spec's gallery; the extra
specs here made that fail about one run in five. It waits for the in-flight
refresh to publish before clearing, since clearing alone only narrows the
window.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

.agents/adding-backends.md

@@ -218,6 +218,69 @@ docker-build-backends: ... docker-build-<backend-name>
 - If the backend is in `backend/python/<backend-name>/` but uses `.` as context in the workflow file, use `.` context
 - Check similar backends to determine the correct context
 
+## Engine preference for gallery model variants
+
+A gallery entry can declare `variants`, alternative builds of the same weights,
+and LocalAI picks one per host: it drops builds whose backend cannot run here or
+that do not fit memory, then ranks the survivors by **engine preference
+first, serving feature second, size third** (`SelectVariant` in
+`core/gallery/resolve_variant.go`).
+
+Ask whether your backend should outrank another one on some hardware. If it
+should, add it to `engineNamePreferenceRules` in `pkg/system/capabilities.go`,
+best engine first for that capability:
+
+```go
+ 	{Nvidia, []string{engineVLLM, engineSGLang, engineLlamaCpp}},
++	{Nvidia, []string{engineVLLM, engineSGLang, engineMyEngine, engineLlamaCpp}},
+```
+
+That is the ENGINE NAME table, matched as a substring of a gallery entry's
+`backend:` value. Two sibling tables in the same file speak different
+vocabularies and are matched against different things:
+
+| Table | Vocabulary | Matched against | Consumer |
+|-------|-----------|-----------------|----------|
+| `backendBuildTagPreferenceRules` | build tags (`cuda`, `rocm`, `metal`) | installed build directory names, as a substring | alias resolution in `ListSystemBackends` |
+| `engineNamePreferenceRules` | engine names (`vllm`, `llama-cpp`, `mlx`) | a gallery entry's `backend:`, as a substring | gallery variant ranking |
+| `servingFeaturePreferenceTokens` | serving features (`dflash`, `mtp`) | a gallery entry's `tags:`, compared whole and case-insensitively, and nothing else | gallery variant ranking, one rank below the engine |
+
+**Putting a token in the wrong table matches nothing and does not error**: every
+candidate scores equal and the next sort key decides, so the preference silently
+stops existing. The block comment above all three tables spells the contract out.
+
+The serving feature table is the odd one: it is not keyed by capability, because
+no hardware prefers a plain build over an equivalent faster build of the same
+weights. It reads a declared tag and nothing else. The entry name was the
+original signal and is gone: a naming convention is not a contract, and names
+are author-supplied free text where a short marker like `mtp` turns up inside
+unrelated words or on weights whose entry enables nothing.
+`overrides.options` was rejected for the mirror-image reason: `spec_type:` is
+llama.cpp's config vocabulary, whereas a cross-backend ranking decision must
+work the same for `ds4`'s `mtp_path:` and `sglang`'s `speculative_algorithm:`.
+
+**If your backend can serve the same weights faster** (speculative decoding,
+multi-token prediction), say so in the docs for its gallery entries so curators
+tag them: the tagging rule and the per-backend evidence table live in
+[adding-gallery-models.md](adding-gallery-models.md). A backend never needs to
+appear in the token table itself; it ranks builds, not engines.
+
+Leaving your backend out is a valid choice when no ordering can be justified for
+it. It then ranks below every known engine and selection falls back to size,
+which is the behaviour that predates preference.
+
+**Leaving a whole capability out is not.** A missing row gives that host an
+empty preference list, so size alone decides among everything that survives the
+filters, and the filter will not save you: `IsBackendCompatible` derives hardware
+support from the engine NAME, so `vllm` and `sglang` carry no darwin, cuda, rocm
+or sycl token and are never dropped on a host with no GPU. That is why `default`
+(no usable accelerator, including a GPU under the 4 GiB VRAM floor) and
+`darwin-x86` both have rows putting `llama-cpp` first. Every capability
+`getSystemCapabilities()` can return needs a row unless every engine really is
+equally at home there. When you add one, enumerate the engines you are demoting
+rather than relying on them falling through unmatched: unmatched engines all tie
+with each other, so size decides among them.
+
 ## Documenting the backend (README + docs)
 
 A backend is not "added" until it is discoverable. Update the user-facing docs:
@@ -254,6 +317,7 @@ After adding a new backend, verify:
 - [ ] No Makefile syntax errors (check with linter)
 - [ ] Follows the same pattern as similar backends (e.g., if it's a transcription backend, follow `faster-whisper` pattern)
 - [ ] **`Load` validates its input and refuses models it can't serve.** When a model config has no explicit `backend:`, the model loader greedily probes *every* installed backend with the model's name and binds to the first `Load` that succeeds — an accept-anything `Load` will capture arbitrary LLMs (issue #9287). Backends that load a real artefact get this for free (the load fails); backends with no artefact must gate on the name: `opus` accepts only its own name (or none), `local-store` requires the `store.NamespacePrefix` namespace marker sent by `core/backend/stores.go`.
+- [ ] **Gallery variant ranking considered**: if this backend should be preferred over another on some hardware, it is listed in `engineNamePreferenceRules` (NOT `backendBuildTagPreferenceRules`, NOT `servingFeaturePreferenceTokens`) in `pkg/system/capabilities.go`. A missing entry silently ranks it last and lets the next sort key decide.
 - [ ] Documented: added to the category list in `docs/content/features/backends.md` (and any new endpoint/realtime capability documented under `docs/content/`)
 - [ ] If it is an in-house native C/C++/GGML engine, added to the maintained-engines table in the top-level `README.md`
 

.agents/adding-gallery-models.md

@@ -91,6 +91,108 @@ To add a variant (e.g., different quantization), use YAML merge:
       uri: huggingface://<gguf-org>/<gguf-repo>/<filename>-Q8_0.gguf
 ```
 
+## Offering several builds of one model (`variants`)
+
+When the same model is published in more than one quantization, or is also
+servable by another engine, add each build as its own ordinary gallery entry and
+then point one of them at the others with `variants`:
+
+```yaml
+- !!merge <<: *chatml
+  name: "nanbeige4.1-3b-q4"
+  # ... the usual urls / overrides / files for the Q4 build ...
+  variants:
+    - model: nanbeige4.1-3b-q8
+```
+
+Rules:
+
+- The declaring entry is a **complete, normal entry**. It keeps its own
+  `files`/`overrides` and stays installable on every host and by every older
+  LocalAI release, which simply ignore `variants`.
+- A variant references another gallery entry **by name**. That entry must exist
+  and must not declare `variants` of its own.
+- **A referenced entry keeps its own gallery row by default.** It is hidden only
+  in the collapsed listing (`collapse_variants=true`, which the web UI requests
+  by default), where the declaring entry stands in for it. Searching there still
+  matches the referenced entry and answers with the entry declaring it, so
+  referencing an entry never makes it unfindable; turning the collapse off
+  returns it under its own name.
+- **Order carries no meaning.** Do not try to encode a preference; write the
+  list in whatever order reads best.
+- **A variant may be smaller than the declaring entry.** Offering a downgrade
+  for small hosts is a normal shape: the declaring entry's own build competes
+  like every other candidate, so a large host keeps the large build.
+- **Do not describe hardware.** At install time LocalAI drops variants whose
+  backend cannot run on the host, then drops those that do not fit available
+  memory. The declaring entry's own build is exempt from both filters, so
+  selection always terminates on something installable. Sizes are measured live
+  from the weights and cached, so nothing has to be written down.
+- **Engine preference outranks size.** Among the builds that survive the
+  filters, the host's preferred engine wins first and only then does the larger
+  footprint win. On NVIDIA a vLLM build beats a larger llama.cpp one; on Apple
+  silicon an MLX build beats a larger GGUF one; on a host with no preference for
+  either engine the larger build wins, since a bigger footprint is a higher
+  quality quantization of the same weights. Predict what a user gets by asking
+  which engine the host prefers before asking which build is biggest. The
+  per-capability order lives in `engineNamePreferenceRules`
+  (`pkg/system/capabilities.go`); see
+  [adding-backends.md](adding-backends.md) for how a backend gets into it.
+- **Serving feature preference sits between engine and size.** Among builds on
+  an equally preferred engine, one that speculates or predicts several tokens
+  per step beats the plain build of the same weights, because it answers faster
+  for the same output: a `dflash` build beats an `mtp` one, and either beats a
+  plain build. The order lives in `servingFeaturePreferenceTokens`
+  (`pkg/system/capabilities.go`) and is matched against the entry's `tags:` and
+  **nothing else**: not the entry name, not `overrides.options`. See
+  [the tagging rule](#the-dflash--mtp-tagging-rule) below. Engine deliberately
+  outranks it: a serving feature makes the right engine faster, it does not make
+  a wrong engine right. Fit still outranks both, so a drafter pairing (strictly
+  larger than the plain build, since it ships a drafter alongside it) is dropped
+  on a host too small for it before this order is ever consulted.
+- A variant is nothing but a name; there is no per-variant memory field. When
+  the measured size for a build is wrong, correct it on the referenced entry by
+  setting that entry's own `size:` (e.g. `size: "20GiB"`). The estimator prefers
+  a declared size over its own guesswork, so the fix applies everywhere the size
+  is shown or compared rather than only to variant selection.
+
+Users can override the automatic choice with `variant` on `POST /models/apply`,
+`local-ai models install --variant`, or the `install_model` MCP tool. See
+`docs/content/features/model-gallery.md`.
+
+The gallery lint specs live in `core/gallery`, so run that suite after adding a
+`variants` list.
+
+### The `dflash` / `mtp` tagging rule
+
+**Tag an entry `dflash` or `mtp` when the entry actually configures that
+feature. Variant ranking reads the tag and nothing else.**
+
+Decide by looking at what the entry configures, in whatever vocabulary its
+backend uses:
+
+| Backend | Configures the feature when it declares |
+|---------|------------------------------------------|
+| `llama-cpp` | `overrides.options` contains `spec_type:draft-dflash` or `spec_type:draft-mtp` |
+| `ds4` | `overrides.options` contains `mtp_path:` / `mtp_draft:` |
+| `sglang` | the referenced `gallery/*.yaml` sets `speculative_algorithm:` |
+
+That check is curation-time only. `spec_type` is llama.cpp's config vocabulary,
+and a cross-backend ranking decision must not depend on one backend's option
+syntax, which is precisely why the ranker reads the tag instead of the options.
+
+Two mistakes the rule exists to prevent:
+
+- **Weights that carry the heads are not an entry that enables them.** The
+  NVFP4 GGUF entries ship MTP-bearing weights but set only `use_jinja:true`, so
+  they enable no speculative decoding and must NOT be tagged. Tagging them wins
+  them the feature axis without being any faster.
+- **A name is not a declaration.** An entry whose name spells `-mtp` while
+  configuring nothing gets no tag, and an entry that configures the feature is
+  tagged even when its name says nothing (`hy3`, `glm-5.2`). Ranking never reads
+  the name, so an untagged build that does enable the feature is simply ranked
+  as plain rather than promoted on a marker nobody meant.
+
 ## Available template configs
 
 Look at existing `.yaml` files in `gallery/` to find the right prompt template for your model architecture:

AGENTS.md

@@ -44,4 +44,5 @@ LocalAI follows the Linux kernel project's [guidelines for AI coding assistants]
 - **Admin endpoints → MCP tool**: every admin endpoint that an admin would manage conversationally (install/list/edit/toggle/upgrade) MUST also be exposed as an MCP tool in `pkg/mcp/localaitools/`. The LocalAI Assistant chat modality and the standalone `local-ai mcp-server` consume that package; drift between REST and MCP is a real risk. Read [.agents/localai-assistant-mcp.md](.agents/localai-assistant-mcp.md) — the `TestToolHTTPRouteMappingComplete` test fails until you wire the new tool and update the route map.
 - **Build**: Inspect `Makefile` and `.github/workflows/` — ask the user before running long builds
 - **Backend OS coverage**: a new backend must target every OS it can build for, not just Linux. `.github/backend-matrix.yml` has two matrices — `include:` (Linux) and `includeDarwin:` (macOS / Apple Silicon). Most C/C++/GGML and many Python backends build on Darwin too — wire the `includeDarwin` entry + `backend/index.yaml` `metal:` entries, or say in the PR why an OS is unsupported. See the darwin checklist in [.agents/adding-backends.md](.agents/adding-backends.md).
+- **Gallery variant ranking**: a gallery entry can declare `variants` (alternative builds of the same weights), and LocalAI ranks the ones a host can run by engine preference first, size second. A new backend that should be preferred on some hardware must be listed in `engineNamePreferenceRules` in `pkg/system/capabilities.go`; the sibling `backendBuildTagPreferenceRules` speaks build tags rather than engine names, and using the wrong table matches nothing without erroring. See [.agents/adding-backends.md](.agents/adding-backends.md).
 - **UI**: The active UI is the React app in `core/http/react-ui/`. The older Alpine.js/HTML UI in `core/http/static/` is pending deprecation — all new UI work goes in the React UI

core/cli/models.go

@@ -39,6 +39,7 @@ type ModelsInstall struct {
 	RequireBackendIntegrity  bool     `env:"LOCALAI_REQUIRE_BACKEND_INTEGRITY,REQUIRE_BACKEND_INTEGRITY" help:"If true, reject backend installs without a configured signature verification policy (OCI URIs) or SHA256 (tarball/HTTP URIs)." group:"hardening" default:"false"`
 	AutoloadBackendGalleries bool     `env:"LOCALAI_AUTOLOAD_BACKEND_GALLERIES" help:"If true, automatically loads backend galleries" group:"backends" default:"true"`
 	ModelArgs                []string `arg:"" optional:"" name:"models" help:"Model configuration URLs to load"`
+	Variant                  string   `name:"variant" help:"Install a specific variant of a gallery entry that declares them, by the variant's model name. Leave unset to let LocalAI auto-select: builds this hardware cannot run or cannot fit are dropped, the engine this hardware prefers wins, and size decides among equals." group:"models"`
 
 	ModelsCMDFlags `embed:""`
 }
@@ -147,7 +148,11 @@ func (mi *ModelsInstall) Run(ctx *cliContext.Context) error {
 		}
 
 		modelLoader := model.NewModelLoader(systemState)
-		err = startup.InstallModels(context.Background(), galleryService, galleries, backendGalleries, systemState, modelLoader, !mi.DisablePredownloadScan, mi.AutoloadBackendGalleries, mi.RequireBackendIntegrity, progressCallback, modelName)
+		var installOptions []gallery.InstallOption
+		if mi.Variant != "" {
+			installOptions = append(installOptions, gallery.WithVariant(mi.Variant))
+		}
+		err = startup.InstallModelsWithOptions(context.Background(), galleryService, galleries, backendGalleries, systemState, modelLoader, !mi.DisablePredownloadScan, mi.AutoloadBackendGalleries, mi.RequireBackendIntegrity, progressCallback, installOptions, modelName)
 		if err != nil {
 			return err
 		}

core/gallery/available_memory_internal_test.go

@@ -0,0 +1,73 @@
+package gallery
+
+import (
+	"os"
+
+	"github.com/mudler/LocalAI/pkg/system"
+	"github.com/mudler/LocalAI/pkg/xsysinfo"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("availableModelMemory", func() {
+	// hostRAM is the figure the fallback resolves to. A host that cannot report
+	// its own RAM cannot exercise the fallback at all, so those specs skip
+	// rather than assert on a number that is legitimately unavailable.
+	hostRAM := func() uint64 {
+		ram, err := xsysinfo.GetSystemRAMInfo()
+		if err != nil || ram == nil || ram.Total == 0 {
+			Skip("this host does not report system RAM; the RAM fallback cannot be exercised here")
+		}
+		return ram.Total
+	}
+
+	// stateWithCapability forces DetectedCapability for one spec. The state is
+	// built fresh so nothing is served from the capability cache.
+	stateWithCapability := func(capability string, vram uint64) *system.SystemState {
+		previous, had := os.LookupEnv("LOCALAI_FORCE_META_BACKEND_CAPABILITY")
+		Expect(os.Setenv("LOCALAI_FORCE_META_BACKEND_CAPABILITY", capability)).To(Succeed())
+		DeferCleanup(func() {
+			if had {
+				Expect(os.Setenv("LOCALAI_FORCE_META_BACKEND_CAPABILITY", previous)).To(Succeed())
+				return
+			}
+			Expect(os.Unsetenv("LOCALAI_FORCE_META_BACKEND_CAPABILITY")).To(Succeed())
+		})
+
+		s := &system.SystemState{}
+		s.VRAM = vram
+		Expect(s.DetectedCapability()).To(Equal(capability), "the capability override did not take")
+		return s
+	}
+
+	// A unified-memory host reports a GPU capability but no discrete VRAM pool.
+	// Apple Silicon is the case that matters: metal is reported unconditionally
+	// on arm64 macs while TotalAvailableVRAM finds nothing to add up. Reading
+	// that zero as the budget drops every sized variant and strands the host on
+	// the base build however much memory it has, which is what turned the macOS
+	// runner red.
+	It("falls back to system RAM when a detected GPU reports no VRAM", func() {
+		ram := hostRAM()
+
+		got := availableModelMemory(stateWithCapability("metal", 0))
+
+		Expect(got).ToNot(BeZero(), "a unified-memory host was given a zero budget; every sized variant would be dropped")
+		Expect(got).To(Equal(ram))
+	})
+
+	// A discrete GPU reports real VRAM, and that stays the budget: the model
+	// lives on the card, so system RAM is not what bounds it.
+	It("prefers VRAM when the GPU reports it", func() {
+		const vram = uint64(24) << 30
+
+		Expect(availableModelMemory(stateWithCapability("nvidia-cuda-12", vram))).To(Equal(vram))
+	})
+
+	// With no GPU at all the model lives in system RAM.
+	It("uses system RAM when no GPU is detected", func() {
+		ram := hostRAM()
+
+		Expect(availableModelMemory(stateWithCapability("default", 0))).To(Equal(ram))
+	})
+})

core/gallery/collapse_variants.go

@@ -0,0 +1,109 @@
+package gallery
+
+import (
+	"os"
+	"strings"
+)
+
+// VariantReferencedIDs reports the IDs of entries that another entry already
+// offers as one of its variants, and which therefore need no row of their own
+// in a deduplicated listing: they are reachable by installing the entry that
+// references them.
+//
+// The returned keys are GalleryModel.ID() values, so an entry is identified by
+// gallery and name rather than by name alone. Two galleries may legitimately
+// ship an entry of the same name, and only the one actually referenced should
+// disappear.
+//
+// This is the key set of VariantParents; see there for the resolution rules.
+func VariantReferencedIDs(models []*GalleryModel) map[string]struct{} {
+	parents := VariantParents(models)
+	referenced := make(map[string]struct{}, len(parents))
+	for id := range parents {
+		referenced[id] = struct{}{}
+	}
+	return referenced
+}
+
+// VariantParents maps the ID of every entry another entry already offers as one
+// of its variants to the entry that offers it. It is what a collapsed listing
+// needs in order to answer a match on a hidden build with the row the user can
+// actually act on: the parent.
+//
+// An entry that declares variants of its own is never reported, even when
+// something else references it. That guard is what keeps a chain from hiding a
+// row the user cannot otherwise reach: parents are always visible, so every
+// hidden entry is guaranteed to have a visible entry that offers it, and no
+// parent this returns is itself a key. Such a reference is an authoring error
+// anyway, and variant resolution already refuses to install it, but the listing
+// must stay coherent in the presence of a gallery that has one rather than
+// silently swallowing entries.
+//
+// When two entries reference the same build, the first in gallery order wins,
+// so the listing is deterministic for a gallery the linter would reject.
+//
+// This is a pure pass over metadata already in memory. It resolves nothing over
+// the network and probes no weight files: whether an entry is referenced is
+// answerable from the declared names alone.
+func VariantParents(models []*GalleryModel) map[string]*GalleryModel {
+	// FindGalleryElement resolves a reference by scanning every entry, which
+	// would make this quadratic over the whole gallery. The index below is the
+	// same lookup precomputed once, so the matching rules have to agree with
+	// it: case-insensitive, path separators folded to "__", and a reference
+	// carrying an "@" addressed against "gallery@name" instead of the bare name.
+	byName := make(map[string]*GalleryModel, len(models))
+	byQualified := make(map[string]*GalleryModel, len(models))
+	for _, m := range models {
+		if m == nil {
+			continue
+		}
+		// First match wins, mirroring FindGalleryElement's scan order, so a
+		// name colliding across galleries resolves the same way here as it
+		// does at install time.
+		name := variantLookupKey(m.Name)
+		if _, seen := byName[name]; !seen {
+			byName[name] = m
+		}
+		qualified := variantLookupKey(m.ID())
+		if _, seen := byQualified[qualified]; !seen {
+			byQualified[qualified] = m
+		}
+	}
+
+	referenced := map[string]*GalleryModel{}
+	for _, m := range models {
+		if m == nil {
+			continue
+		}
+		for _, v := range m.Variants {
+			key := variantLookupKey(v.Model)
+			target, ok := byQualified[key]
+			if !strings.Contains(key, "@") {
+				target, ok = byName[key]
+			}
+			// A dangling reference names nothing to hide. Reporting it is the
+			// linter's job, not the listing's.
+			if !ok || target == nil {
+				continue
+			}
+			if target.HasVariants() {
+				continue
+			}
+			// An entry naming itself would otherwise erase itself from the
+			// gallery entirely.
+			if target.ID() == m.ID() {
+				continue
+			}
+			if _, claimed := referenced[target.ID()]; claimed {
+				continue
+			}
+			referenced[target.ID()] = m
+		}
+	}
+
+	return referenced
+}
+
+func variantLookupKey(name string) string {
+	return strings.ToLower(strings.ReplaceAll(name, string(os.PathSeparator), "__"))
+}

core/gallery/collapse_variants_test.go

@@ -0,0 +1,157 @@
+package gallery_test
+
+import (
+	"github.com/mudler/LocalAI/core/config"
+	. "github.com/mudler/LocalAI/core/gallery"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// VariantReferencedIDs answers the only question the deduplicated listing asks:
+// which entries does another entry already offer, and so need no row of their
+// own. Everything else is shown.
+var _ = Describe("VariantReferencedIDs", func() {
+	entry := func(gal, name string, variants ...string) *GalleryModel {
+		m := &GalleryModel{}
+		m.Name = name
+		m.Gallery = config.Gallery{Name: gal}
+		for _, v := range variants {
+			m.Variants = append(m.Variants, Variant{Model: v})
+		}
+		return m
+	}
+
+	It("reports the builds a parent references and nothing else", func() {
+		parent := entry("test", "parent", "build-a", "build-b")
+		models := []*GalleryModel{
+			parent,
+			entry("test", "build-a"),
+			entry("test", "build-b"),
+			entry("test", "standalone"),
+		}
+
+		Expect(VariantReferencedIDs(models)).To(HaveLen(2))
+		Expect(VariantReferencedIDs(models)).To(HaveKey("test@build-a"))
+		Expect(VariantReferencedIDs(models)).To(HaveKey("test@build-b"))
+		// The parent is installable in its own right, and a standalone entry
+		// is nobody's variant.
+		Expect(VariantReferencedIDs(models)).NotTo(HaveKey("test@parent"))
+		Expect(VariantReferencedIDs(models)).NotTo(HaveKey("test@standalone"))
+	})
+
+	It("never hides an entry that declares variants of its own", func() {
+		// A parent referencing another parent is an authoring error that
+		// variant resolution refuses to install. The listing still has to stay
+		// coherent: if the chain hid the middle entry, the build it offers
+		// would be unreachable, since nothing visible would mention it.
+		top := entry("test", "top", "middle")
+		middle := entry("test", "middle", "leaf")
+		models := []*GalleryModel{top, middle, entry("test", "leaf")}
+
+		referenced := VariantReferencedIDs(models)
+		Expect(referenced).NotTo(HaveKey("test@middle"),
+			"hiding a parent would strand the builds only it offers")
+		Expect(referenced).To(HaveKey("test@leaf"))
+	})
+
+	It("ignores an entry that references itself", func() {
+		// Self-reference would otherwise erase the entry from the gallery.
+		models := []*GalleryModel{entry("test", "loop", "loop", "build")}
+		models = append(models, entry("test", "build"))
+
+		Expect(VariantReferencedIDs(models)).NotTo(HaveKey("test@loop"))
+		Expect(VariantReferencedIDs(models)).To(HaveKey("test@build"))
+	})
+
+	It("ignores a reference that names no existing entry", func() {
+		// A dangling reference hides nothing; reporting it is the linter's job.
+		models := []*GalleryModel{entry("test", "parent", "does-not-exist")}
+		Expect(VariantReferencedIDs(models)).To(BeEmpty())
+	})
+
+	It("resolves references the way install-time lookup does", func() {
+		// Matching has to agree with FindGalleryElement, otherwise a reference
+		// that installs fine would fail to hide its target. Case-insensitive,
+		// and an "@" addresses gallery and name together.
+		models := []*GalleryModel{
+			entry("test", "parent", "BUILD-A", "other@build-b"),
+			entry("test", "build-a"),
+			entry("other", "build-b"),
+			entry("test", "build-b"),
+		}
+
+		referenced := VariantReferencedIDs(models)
+		Expect(referenced).To(HaveKey("test@build-a"))
+		Expect(referenced).To(HaveKey("other@build-b"))
+		// The qualified reference names the other gallery's entry, so the
+		// same-named local entry keeps its row.
+		Expect(referenced).NotTo(HaveKey("test@build-b"))
+	})
+
+	It("returns an empty set for a gallery declaring no variants at all", func() {
+		models := []*GalleryModel{entry("test", "a"), entry("test", "b")}
+		Expect(VariantReferencedIDs(models)).To(BeEmpty())
+	})
+
+	// The collapsed listing needs more than "is this hidden": to report a match
+	// on a hidden build it has to name the row the user can act on.
+	Context("VariantParents", func() {
+		It("names the entry that offers each hidden build", func() {
+			parent := entry("test", "parent", "build-a", "build-b")
+			models := []*GalleryModel{
+				parent,
+				entry("test", "build-a"),
+				entry("test", "build-b"),
+				entry("test", "standalone"),
+			}
+
+			parents := VariantParents(models)
+			Expect(parents).To(HaveLen(2))
+			Expect(parents["test@build-a"]).To(BeIdenticalTo(parent))
+			Expect(parents["test@build-b"]).To(BeIdenticalTo(parent))
+			Expect(parents).NotTo(HaveKey("test@standalone"))
+		})
+
+		It("never names a parent that is itself hidden", func() {
+			// What makes substitution safe in one hop: whatever a hidden build
+			// resolves to is guaranteed to be a row the listing still shows, so
+			// no chain can substitute a user into an entry that is not there.
+			models := []*GalleryModel{
+				entry("test", "top", "middle"),
+				entry("test", "middle", "leaf"),
+				entry("test", "leaf"),
+			}
+
+			parents := VariantParents(models)
+			for hidden, parent := range parents {
+				Expect(parents).NotTo(HaveKey(parent.ID()),
+					"the parent of "+hidden+" is itself hidden, so substituting lands on a row nobody can see")
+			}
+		})
+
+		It("gives a build claimed twice a single parent, the first in order", func() {
+			// A gallery the linter rejects, but the listing must still be
+			// deterministic rather than ordered by map iteration.
+			first := entry("test", "first", "shared")
+			second := entry("test", "second", "shared")
+			models := []*GalleryModel{first, second, entry("test", "shared")}
+
+			Expect(VariantParents(models)["test@shared"]).To(BeIdenticalTo(first))
+		})
+
+		It("agrees with VariantReferencedIDs on which entries are hidden", func() {
+			models := []*GalleryModel{
+				entry("test", "parent", "build-a"),
+				entry("test", "build-a"),
+				entry("test", "standalone"),
+			}
+
+			parents := VariantParents(models)
+			referenced := VariantReferencedIDs(models)
+			Expect(parents).To(HaveLen(len(referenced)))
+			for id := range referenced {
+				Expect(parents).To(HaveKey(id))
+			}
+		})
+	})
+})

core/gallery/describe_variants.go

@@ -0,0 +1,120 @@
+package gallery
+
+// VariantView is one selectable build of an entry, as a client sees it.
+//
+// It is the read-only mirror of what selection decides at install time, so a
+// picker can show the user the same facts the installer acts on rather than a
+// second, independently-derived opinion that could disagree with it.
+type VariantView struct {
+	// Model is the name to send back as the install request's `variant`.
+	Model string `json:"model"`
+	// Backend is the engine the referenced entry resolves to. A client renders
+	// it, and it is also the reason Fits may be false on a host whose memory
+	// would be ample.
+	Backend string `json:"backend"`
+	// MemoryBytes is the measured footprint. It is omitted from the JSON
+	// entirely when the size could not be determined, rather than serialized as
+	// a zero that a client would have to know to read as unknown. An absent key
+	// never means "needs nothing".
+	MemoryBytes uint64 `json:"memory_bytes,omitempty"`
+	// Fits reports whether auto-selection would consider this variant on this
+	// host: its backend can run here and its known footprint is within budget.
+	// An unknown footprint counts as fitting, exactly as selection treats it.
+	Fits bool `json:"fits"`
+	// IsBase marks the entry's own payload. It is always installable and is
+	// what auto-selection falls back to, which is why it is listed alongside
+	// the declared variants rather than hidden.
+	IsBase bool `json:"is_base"`
+	// Quantization is the weight format the referenced entry installs, e.g.
+	// "Q2_G64", "PQ2_0", "F16". It is the fact that actually separates two
+	// builds of one model: name, backend and probed size routinely agree
+	// between variants that differ entirely in precision.
+	//
+	// Derived server-side from the referenced entry's model filename rather
+	// than parsed by clients, so every client reads the same format out of the
+	// same file the installer will point the backend at, and a naming
+	// convention change is one edit here rather than one per client.
+	//
+	// Omitted when the entry names no recognisable format. That is a normal
+	// outcome for a backend served from a directory of weights, and clients
+	// must render its absence rather than an empty or invented value.
+	Quantization string `json:"quantization,omitempty"`
+	// Features are the serving features this build declares, best first, e.g.
+	// ["dflash"]. They mean the same weights are served faster, which is a
+	// reason to prefer a variant that neither its size nor its precision
+	// conveys.
+	//
+	// This is the SAME tag-against-vocabulary match servingFeatureRank ranks
+	// on, over the same host preference list, so what a client shows as a speed
+	// advantage cannot disagree with what selection actually rewarded. A tag
+	// outside that vocabulary is not a serving feature and is not reported
+	// here: the entry's own tag list already carries it.
+	Features []string `json:"features,omitempty"`
+}
+
+// EntryVariants is the variant surface of a single gallery entry.
+type EntryVariants struct {
+	Variants []VariantView `json:"variants"`
+	// AutoSelected is the variant auto-selection would install on this host
+	// right now. Clients show it as the default choice.
+	AutoSelected string `json:"auto_selected"`
+}
+
+// DescribeVariants reports an entry's selectable builds and which one
+// auto-selection would currently pick.
+//
+// It runs the SAME variantOptions + SelectVariant pass the installer runs, so
+// the reported auto-selection cannot drift from what installing would actually
+// do. The env carries the same probe seam too, so the size shown here and the
+// size the installer compares against come from one cache and one round trip.
+//
+// An entry that declares no variants returns nil, nil WITHOUT touching the
+// probe. That is load-bearing: the gallery listing walks entries by the
+// thousand and the overwhelming majority declare nothing, so the no-variant
+// case must cost nothing at all.
+func DescribeVariants(models []*GalleryModel, entry *GalleryModel, env ResolveEnv) (*EntryVariants, error) {
+	if entry == nil || !entry.HasVariants() {
+		return nil, nil
+	}
+
+	options, err := variantOptions(models, entry, env)
+	if err != nil {
+		return nil, err
+	}
+
+	selection, err := SelectVariant(options, env, "")
+	if err != nil {
+		return nil, err
+	}
+
+	views := make([]VariantView, 0, len(options))
+	for _, o := range options {
+		memory, known := o.EffectiveMemory()
+		view := VariantView{
+			Model:   o.Variant.Model,
+			Backend: o.Backend,
+			// The base is exempt from every filter and always installs, so
+			// reporting it as anything but fitting would misdescribe it.
+			Fits:   o.IsBase || (env.backendRuns(o.Backend) && (!known || memory <= env.AvailableMemory)),
+			IsBase: o.IsBase,
+			// o.Tags is already the REFERENCED entry's tag list, which is the
+			// only correct source: the feature belongs to the build that would
+			// be installed, not to the family the parent describes.
+			Features: env.servingFeaturesOf(o),
+		}
+		if known {
+			view.MemoryBytes = memory
+		}
+		// The base option's payload is the entry itself; every other option
+		// names a gallery entry variantOptions has already proved resolvable,
+		// so this second lookup is an in-memory map hit and cannot fail.
+		source := entry
+		if !o.IsBase {
+			source = FindGalleryElement(models, o.Variant.Model)
+		}
+		view.Quantization = quantizationOfEntry(source)
+		views = append(views, view)
+	}
+
+	return &EntryVariants{Variants: views, AutoSelected: selection.Option.Variant.Model}, nil
+}

core/gallery/describe_variants_test.go

@@ -0,0 +1,369 @@
+package gallery_test
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+
+	"github.com/mudler/LocalAI/core/gallery"
+)
+
+var _ = Describe("DescribeVariants", func() {
+	gib := func(n uint64) uint64 { return n * 1024 * 1024 * 1024 }
+
+	newModel := func(name, backend string) *gallery.GalleryModel {
+		m := &gallery.GalleryModel{}
+		m.Name = name
+		m.Backend = backend
+		return m
+	}
+
+	var models []*gallery.GalleryModel
+	var base *gallery.GalleryModel
+	// probed records every entry the probe was asked about, so a spec can
+	// assert on the ABSENCE of a round trip and not merely on the result.
+	var probed []string
+
+	// probing builds an env whose sizes are injected rather than measured, so
+	// these specs pin exact footprints without reaching the network.
+	probing := func(available uint64, sizes map[string]uint64) gallery.ResolveEnv {
+		return gallery.ResolveEnv{
+			AvailableMemory:   available,
+			BackendCompatible: func(string) bool { return true },
+			ProbeMemory: func(target *gallery.GalleryModel) uint64 {
+				probed = append(probed, target.Name)
+				return sizes[target.Name]
+			},
+		}
+	}
+
+	byName := func(view *gallery.EntryVariants, name string) gallery.VariantView {
+		GinkgoHelper()
+		for _, v := range view.Variants {
+			if v.Model == name {
+				return v
+			}
+		}
+		Fail("no variant named " + name + " in the reported view")
+		return gallery.VariantView{}
+	}
+
+	BeforeEach(func() {
+		probed = nil
+		big := newModel("qwen3-8b-vllm-awq", "vllm")
+		small := newModel("qwen3-8b-gguf-q8", "llama-cpp")
+		base = newModel("qwen3-8b-gguf-q4", "llama-cpp")
+		base.Variants = []gallery.Variant{
+			{Model: "qwen3-8b-vllm-awq"},
+			{Model: "qwen3-8b-gguf-q8"},
+		}
+		models = []*gallery.GalleryModel{big, small, base}
+	})
+
+	Describe("an entry that declares no variants", func() {
+		It("reports nothing and issues no probe at all", func() {
+			// This is the performance contract of the gallery listing: the
+			// listing walks over a thousand entries and virtually none of them
+			// declare variants, so the ordinary entry must cost zero round
+			// trips. Asserting on `probed` rather than on timing makes that
+			// contract testable.
+			plain := newModel("plain-entry", "llama-cpp")
+			models = append(models, plain)
+
+			view, err := gallery.DescribeVariants(models, plain, probing(gib(24), map[string]uint64{
+				"plain-entry": gib(4),
+			}))
+
+			Expect(err).ToNot(HaveOccurred())
+			Expect(view).To(BeNil())
+			Expect(probed).To(BeEmpty())
+		})
+
+		It("reports nothing for a nil entry rather than panicking", func() {
+			view, err := gallery.DescribeVariants(models, nil, probing(gib(24), nil))
+			Expect(err).ToNot(HaveOccurred())
+			Expect(view).To(BeNil())
+			Expect(probed).To(BeEmpty())
+		})
+	})
+
+	Describe("an entry that declares variants", func() {
+		It("reports every declared variant plus the entry's own build", func() {
+			view, err := gallery.DescribeVariants(models, base, probing(gib(24), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(20),
+				"qwen3-8b-gguf-q8":  gib(9),
+			}))
+			Expect(err).ToNot(HaveOccurred())
+
+			names := []string{}
+			for _, v := range view.Variants {
+				names = append(names, v.Model)
+			}
+			// The base is listed so a picker can offer "decline the upgrade",
+			// which is a real choice an operator makes.
+			Expect(names).To(ConsistOf("qwen3-8b-vllm-awq", "qwen3-8b-gguf-q8", "qwen3-8b-gguf-q4"))
+			Expect(byName(view, "qwen3-8b-gguf-q4").IsBase).To(BeTrue())
+			Expect(byName(view, "qwen3-8b-vllm-awq").IsBase).To(BeFalse())
+		})
+
+		It("reports the backend of the referenced entry, not of the declaring one", func() {
+			// A picker renders this, and it is also the reason a variant can be
+			// unavailable on a host with ample memory.
+			view, err := gallery.DescribeVariants(models, base, probing(gib(24), nil))
+			Expect(err).ToNot(HaveOccurred())
+			Expect(byName(view, "qwen3-8b-vllm-awq").Backend).To(Equal("vllm"))
+			Expect(byName(view, "qwen3-8b-gguf-q8").Backend).To(Equal("llama-cpp"))
+		})
+
+		It("reports the probed size of each variant", func() {
+			view, err := gallery.DescribeVariants(models, base, probing(gib(24), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(20),
+				"qwen3-8b-gguf-q8":  gib(9),
+			}))
+			Expect(err).ToNot(HaveOccurred())
+			Expect(byName(view, "qwen3-8b-vllm-awq").MemoryBytes).To(Equal(gib(20)))
+			Expect(byName(view, "qwen3-8b-gguf-q8").MemoryBytes).To(Equal(gib(9)))
+		})
+
+		It("reports a size it could not determine as unknown rather than as free", func() {
+			// Zero on the wire means unknown. Rendering it as a zero-byte model
+			// would advertise the largest download on offer as costless.
+			view, err := gallery.DescribeVariants(models, base, probing(gib(24), map[string]uint64{}))
+			Expect(err).ToNot(HaveOccurred())
+			Expect(byName(view, "qwen3-8b-vllm-awq").MemoryBytes).To(Equal(uint64(0)))
+			Expect(byName(view, "qwen3-8b-vllm-awq").Fits).To(BeTrue())
+		})
+
+		It("marks a variant too large for this host as not fitting", func() {
+			view, err := gallery.DescribeVariants(models, base, probing(gib(10), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(20),
+				"qwen3-8b-gguf-q8":  gib(9),
+			}))
+			Expect(err).ToNot(HaveOccurred())
+			Expect(byName(view, "qwen3-8b-vllm-awq").Fits).To(BeFalse())
+			Expect(byName(view, "qwen3-8b-gguf-q8").Fits).To(BeTrue())
+		})
+
+		It("marks a variant whose backend cannot run here as not fitting, however much memory there is", func() {
+			env := probing(gib(1024), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(20),
+				"qwen3-8b-gguf-q8":  gib(9),
+			})
+			env.BackendCompatible = func(backend string) bool { return backend != "vllm" }
+
+			view, err := gallery.DescribeVariants(models, base, env)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(byName(view, "qwen3-8b-vllm-awq").Fits).To(BeFalse())
+			Expect(byName(view, "qwen3-8b-gguf-q8").Fits).To(BeTrue())
+		})
+
+		It("always reports the entry's own build as fitting", func() {
+			// The base is exempt from every filter and always installs, so
+			// showing it as unavailable would misdescribe it. Neither a hostile
+			// backend gate nor a zero memory budget may change that.
+			env := probing(0, map[string]uint64{})
+			env.BackendCompatible = func(string) bool { return false }
+
+			view, err := gallery.DescribeVariants(models, base, env)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(byName(view, "qwen3-8b-gguf-q4").Fits).To(BeTrue())
+		})
+
+		It("surfaces a variant that references an entry no gallery declares", func() {
+			base.Variants = []gallery.Variant{{Model: "does-not-exist"}}
+			_, err := gallery.DescribeVariants(models, base, probing(gib(24), nil))
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring("does-not-exist"))
+		})
+	})
+
+	Describe("the reported auto-selection", func() {
+		// These are the specs that keep a picker honest: what the listing shows
+		// as the default must be what installing with no variant actually does.
+		// They assert against ResolveVariant rather than against a hardcoded
+		// name, so a change to the selection rules cannot make the two drift
+		// without failing here.
+		agreesWithInstall := func(env gallery.ResolveEnv) {
+			GinkgoHelper()
+			view, err := gallery.DescribeVariants(models, base, env)
+			Expect(err).ToNot(HaveOccurred())
+
+			_, chosen, err := gallery.ResolveVariant(models, base, env, "")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(view.AutoSelected).To(Equal(chosen.Model))
+		}
+
+		It("matches what installing would pick when everything fits", func() {
+			agreesWithInstall(probing(gib(64), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(20),
+				"qwen3-8b-gguf-q8":  gib(9),
+			}))
+		})
+
+		It("matches what installing would pick when only the smaller variant fits", func() {
+			agreesWithInstall(probing(gib(12), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(20),
+				"qwen3-8b-gguf-q8":  gib(9),
+			}))
+		})
+
+		It("matches what installing would pick when nothing fits and the base wins", func() {
+			agreesWithInstall(probing(gib(2), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(20),
+				"qwen3-8b-gguf-q8":  gib(9),
+			}))
+		})
+
+		It("names the largest fitting variant, not the first authored", func() {
+			// Pinned separately from agreesWithInstall so that a mutation making
+			// BOTH functions pick the wrong variant still fails a spec.
+			view, err := gallery.DescribeVariants(models, base, probing(gib(64), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(20),
+				"qwen3-8b-gguf-q8":  gib(9),
+			}))
+			Expect(err).ToNot(HaveOccurred())
+			Expect(view.AutoSelected).To(Equal("qwen3-8b-vllm-awq"))
+		})
+
+		It("matches what installing would pick when the host prefers an engine", func() {
+			// The picker and the installer both have to apply engine
+			// preference, or a Mac would be shown the GGUF build and handed the
+			// MLX one. Asserting the agreement AND the name pins both halves.
+			mlx := newModel("qwen3-8b-mlx-4bit", "mlx")
+			models = append(models, mlx)
+			base.Variants = append(base.Variants, gallery.Variant{Model: "qwen3-8b-mlx-4bit"})
+
+			env := probing(gib(64), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(20),
+				"qwen3-8b-gguf-q8":  gib(9),
+				"qwen3-8b-mlx-4bit": gib(5),
+			})
+			// Engine names as SystemState.EnginePreferenceTokens reports them for
+			// metal. Build tags would match no gallery `backend:` value here.
+			env.EnginePreference = []string{"mlx", "llama-cpp"}
+
+			agreesWithInstall(env)
+
+			view, err := gallery.DescribeVariants(models, base, env)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(view.AutoSelected).To(Equal("qwen3-8b-mlx-4bit"))
+		})
+
+		It("matches what installing would pick when the host prefers vLLM", func() {
+			// The NVIDIA rule the user asked for, checked through the listing so
+			// the picker and the installer cannot drift on it. The GGUF build is
+			// deliberately the LARGER one, so only preference can produce this
+			// answer: size alone would name the llama.cpp build.
+			env := probing(gib(64), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(9),
+				"qwen3-8b-gguf-q8":  gib(20),
+			})
+			env.EnginePreference = []string{"vllm", "sglang", "llama-cpp"}
+
+			agreesWithInstall(env)
+
+			view, err := gallery.DescribeVariants(models, base, env)
+			Expect(err).ToNot(HaveOccurred())
+			Expect(view.AutoSelected).To(Equal("qwen3-8b-vllm-awq"))
+		})
+
+		It("names the entry itself when no variant fits", func() {
+			view, err := gallery.DescribeVariants(models, base, probing(gib(2), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(20),
+				"qwen3-8b-gguf-q8":  gib(9),
+			}))
+			Expect(err).ToNot(HaveOccurred())
+			Expect(view.AutoSelected).To(Equal("qwen3-8b-gguf-q4"))
+		})
+	})
+
+	Describe("the facts that tell two builds of one model apart", func() {
+		// servedAs points an entry at a weight file the way the gallery does,
+		// so the reported quantization comes from the same field the installer
+		// hands the backend.
+		servedAs := func(m *gallery.GalleryModel, filename string) {
+			m.Overrides = map[string]any{
+				"parameters": map[string]any{"model": filename},
+			}
+		}
+
+		It("reports each variant's quantization from the entry it references", func() {
+			// The whole point of the field: these three rows share a backend
+			// and sit within a gigabyte of each other, so quantization is the
+			// only thing a user can actually choose on.
+			servedAs(models[0], "models/Qwen3-8B-AWQ-4bit.safetensors")
+			models[0].Backend = "llama-cpp"
+			servedAs(models[1], "models/Qwen3-8B-Q8_0.gguf")
+			servedAs(base, "models/Qwen3-8B-Q4_K_M.gguf")
+
+			view, err := gallery.DescribeVariants(models, base, probing(gib(64), map[string]uint64{}))
+
+			Expect(err).ToNot(HaveOccurred())
+			Expect(byName(view, "qwen3-8b-vllm-awq").Quantization).To(Equal("4BIT"))
+			Expect(byName(view, "qwen3-8b-gguf-q8").Quantization).To(Equal("Q8_0"))
+			// The base is described from the entry's own payload, not skipped:
+			// it is a selectable build like any other.
+			Expect(byName(view, "qwen3-8b-gguf-q4").Quantization).To(Equal("Q4_K_M"))
+		})
+
+		It("leaves the quantization unset when the referenced entry names none", func() {
+			// A vLLM build served from a directory of safetensors declares no
+			// format. The field must be absent rather than empty-but-present,
+			// so a client renders "unknown" instead of a blank cell.
+			servedAs(models[0], "models/Qwen3-8B/")
+			servedAs(base, "models/Qwen3-8B-Q4_K_M.gguf")
+
+			view, err := gallery.DescribeVariants(models, base, probing(gib(64), map[string]uint64{}))
+
+			Expect(err).ToNot(HaveOccurred())
+			Expect(byName(view, "qwen3-8b-vllm-awq").Quantization).To(BeEmpty())
+			Expect(byName(view, "qwen3-8b-gguf-q4").Quantization).To(Equal("Q4_K_M"))
+		})
+
+		It("names the serving features a build declares, best first", func() {
+			models[0].Tags = []string{"llm", "MTP", "gguf", "dflash"}
+
+			env := probing(gib(64), map[string]uint64{})
+			env.ServingFeaturePreference = []string{"dflash", "mtp"}
+
+			view, err := gallery.DescribeVariants(models, base, env)
+
+			Expect(err).ToNot(HaveOccurred())
+			// Preference order, not the author's order, and case-folded the way
+			// the ranker folds it: "MTP" and "mtp" declare one feature.
+			Expect(byName(view, "qwen3-8b-vllm-awq").Features).To(Equal([]string{"dflash", "mtp"}))
+			// A tag outside the vocabulary is not a serving feature.
+			Expect(byName(view, "qwen3-8b-gguf-q8").Features).To(BeEmpty())
+		})
+
+		It("agrees with the ranking: the build shown as faster is the one selection rewards", func() {
+			// The contract that keeps the badge honest. Both builds fit and are
+			// runnable, so only the serving feature can decide, and the variant
+			// carrying the reported feature must be the one auto-selected.
+			models[1].Tags = []string{"dflash"}
+
+			env := probing(gib(64), map[string]uint64{
+				"qwen3-8b-vllm-awq": gib(9),
+				"qwen3-8b-gguf-q8":  gib(9),
+			})
+			env.ServingFeaturePreference = []string{"dflash", "mtp"}
+
+			view, err := gallery.DescribeVariants(models, base, env)
+
+			Expect(err).ToNot(HaveOccurred())
+			Expect(byName(view, "qwen3-8b-gguf-q8").Features).To(Equal([]string{"dflash"}))
+			Expect(view.AutoSelected).To(Equal("qwen3-8b-gguf-q8"))
+		})
+
+		It("reports no features on a host with no serving feature vocabulary", func() {
+			// An env with no preference list ranks every build equally on this
+			// axis, so claiming a speed advantage would describe an advantage
+			// nothing acted on.
+			models[0].Tags = []string{"dflash"}
+
+			view, err := gallery.DescribeVariants(models, base, probing(gib(64), map[string]uint64{}))
+
+			Expect(err).ToNot(HaveOccurred())
+			Expect(byName(view, "qwen3-8b-vllm-awq").Features).To(BeEmpty())
+		})
+	})
+})

core/gallery/empty_base_install_test.go

@@ -0,0 +1,243 @@
+package gallery_test
+
+import (
+	"context"
+	"fmt"
+	"os"
+	"path/filepath"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+	"gopkg.in/yaml.v3"
+
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/gallery"
+	"github.com/mudler/LocalAI/pkg/system"
+)
+
+// An entry declaring neither url: nor config_file: is installed on an empty
+// base config, with overrides: and files: supplying everything. These specs
+// cover that path, the payload rule that still rejects an entry with nothing to
+// install, and the two older paths, which must be untouched.
+//
+// Nothing here reaches the network. The whole point of the empty-base path is
+// that it fetches nothing, and a spec that quietly went to GitHub would be
+// asserting the opposite of the change.
+var _ = Describe("InstallModelFromGallery with an empty base config", func() {
+	var tempdir string
+	var galleries []config.Gallery
+	var systemState *system.SystemState
+	// The gallery listing is cached on the name and URL pair, so every spec
+	// needs a gallery of its own or it reads the previous spec's catalog.
+	galleryRevision := 0
+
+	newGallery := func(entries ...gallery.GalleryModel) {
+		out, err := yaml.Marshal(entries)
+		Expect(err).ToNot(HaveOccurred())
+		name := fmt.Sprintf("empty-base-%d", galleryRevision)
+		galleryRevision++
+		galleryPath := filepath.Join(tempdir, name+".yaml")
+		Expect(os.WriteFile(galleryPath, out, 0600)).To(Succeed())
+		galleries = []config.Gallery{{Name: name, URL: "file://" + galleryPath}}
+	}
+
+	install := func(name string, req gallery.GalleryModel, options ...gallery.InstallOption) error {
+		return gallery.InstallModelFromGallery(
+			context.TODO(), galleries, []config.Gallery{}, systemState, nil,
+			name, req, func(string, string, string, float64) {}, false, false, false, options...)
+	}
+
+	installedConfig := func(name string) map[string]any {
+		dat, err := os.ReadFile(filepath.Join(tempdir, name+".yaml"))
+		Expect(err).ToNot(HaveOccurred())
+		content := map[string]any{}
+		Expect(yaml.Unmarshal(dat, &content)).To(Succeed())
+		return content
+	}
+
+	// localWeights lets a spec carry a files: list without leaving the
+	// filesystem. The downloader treats an already-present destination with no
+	// declared sha256 as fetched and skips it, so seeding the file is what keeps
+	// these specs off the network. The URI is still authored, because the
+	// installer walks the list either way and a missing one would not exercise
+	// the same code.
+	localWeights := func(name string) gallery.File {
+		Expect(os.WriteFile(filepath.Join(tempdir, name), []byte("weights for "+name), 0600)).To(Succeed())
+		return gallery.File{Filename: name, URI: "https://example.com/" + name}
+	}
+
+	BeforeEach(func() {
+		var err error
+		tempdir, err = os.MkdirTemp("", "empty-base-install")
+		Expect(err).ToNot(HaveOccurred())
+		DeferCleanup(func() { Expect(os.RemoveAll(tempdir)).To(Succeed()) })
+
+		systemState, err = system.GetSystemState(system.WithModelPath(tempdir))
+		Expect(err).ToNot(HaveOccurred())
+	})
+
+	It("installs an entry that declares only overrides and files", func() {
+		e := gallery.GalleryModel{Overrides: map[string]any{
+			"backend":    "ds4",
+			"parameters": map[string]any{"model": "weights.gguf"},
+		}}
+		e.Name = "overrides-only"
+		e.Description = "an entry with no base config"
+		e.AdditionalFiles = []gallery.File{localWeights("weights.gguf")}
+		newGallery(e)
+
+		Expect(install("overrides-only", gallery.GalleryModel{})).To(Succeed())
+
+		cfg := installedConfig("overrides-only")
+		Expect(cfg["backend"]).To(Equal("ds4"))
+		Expect(cfg["parameters"]).To(HaveKeyWithValue("model", "weights.gguf"))
+		// The name comes from the install, never from a base config, which is
+		// what the several hundred virtual.yaml entries were getting wrong: they
+		// inherited the stub's name.
+		Expect(cfg["name"]).To(Equal("overrides-only"))
+		Expect(filepath.Join(tempdir, "weights.gguf")).To(BeAnExistingFile())
+	})
+
+	It("installs an entry that declares only files", func() {
+		e := gallery.GalleryModel{}
+		e.Name = "files-only"
+		e.AdditionalFiles = []gallery.File{localWeights("plain.gguf")}
+		newGallery(e)
+
+		Expect(install("files-only", gallery.GalleryModel{})).To(Succeed())
+		Expect(filepath.Join(tempdir, "plain.gguf")).To(BeAnExistingFile())
+	})
+
+	// Half the point of the change is that the empty-base path costs no fetch,
+	// and an assertion that nothing was fetched is worthless unless something
+	// could have been. So the control runs the same install with a url: pointing
+	// at a base config that is not there: it must fail, proving a url IS read on
+	// this path, and only then does the same fixture without the url passing
+	// mean the read was skipped rather than silently succeeding.
+	Describe("the base config fetch", func() {
+		var missing string
+
+		BeforeEach(func() {
+			missing = "file://" + filepath.Join(tempdir, "no-such-base.yaml")
+			Expect(filepath.Join(tempdir, "no-such-base.yaml")).ToNot(BeAnExistingFile())
+		})
+
+		It("is attempted when the entry declares a url", func() {
+			e := gallery.GalleryModel{Overrides: map[string]any{"backend": "ds4"}}
+			e.Name = "with-url"
+			e.URL = missing
+			newGallery(e)
+
+			// If this ever starts passing, the control has stopped controlling
+			// and the spec below proves nothing.
+			Expect(install("with-url", gallery.GalleryModel{})).To(HaveOccurred())
+		})
+
+		It("is skipped entirely when the entry declares none", func() {
+			e := gallery.GalleryModel{Overrides: map[string]any{"backend": "ds4"}}
+			e.Name = "without-url"
+			newGallery(e)
+
+			Expect(install("without-url", gallery.GalleryModel{})).To(Succeed())
+			Expect(installedConfig("without-url")["backend"]).To(Equal("ds4"))
+		})
+	})
+
+	Describe("an entry with nothing to install", func() {
+		// No url, no config_file, no overrides and no files. Accepting this
+		// would write an empty model directory and report success, which is the
+		// authoring mistake the relaxation would otherwise hide.
+		emptyEntry := func() gallery.GalleryModel {
+			e := gallery.GalleryModel{}
+			e.Name = "hollow"
+			// urls: is the informational link list. It reads like a payload and
+			// is not one.
+			e.URLs = []string{"https://huggingface.co/example/hollow"}
+			return e
+		}
+
+		It("is refused rather than installed empty", func() {
+			newGallery(emptyEntry())
+
+			err := install("hollow", gallery.GalleryModel{})
+			Expect(err).To(HaveOccurred())
+			Expect(err.Error()).To(ContainSubstring("installs nothing"))
+			Expect(err.Error()).To(ContainSubstring("hollow"))
+			Expect(filepath.Join(tempdir, "hollow.yaml")).ToNot(BeAnExistingFile())
+		})
+
+		It("is accepted when the caller's request supplies the payload", func() {
+			// The request's overrides are merged into the install exactly as the
+			// entry's own are, so a caller who brings them really has asked for
+			// something installable.
+			newGallery(emptyEntry())
+
+			req := gallery.GalleryModel{Overrides: map[string]any{"backend": "llama-cpp"}}
+			Expect(install("hollow", req)).To(Succeed())
+			Expect(installedConfig("hollow")["backend"]).To(Equal("llama-cpp"))
+		})
+	})
+
+	// The two older paths are meant to be untouched by the relaxation.
+	Describe("the pre-existing paths", func() {
+		It("still installs an entry described by an inline config_file", func() {
+			e := gallery.GalleryModel{ConfigFile: map[string]any{"backend": "llama-cpp"}}
+			e.Name = "inline"
+			newGallery(e)
+
+			Expect(install("inline", gallery.GalleryModel{})).To(Succeed())
+			Expect(installedConfig("inline")["backend"]).To(Equal("llama-cpp"))
+		})
+
+		It("still installs an entry described by a url", func() {
+			payload, err := yaml.Marshal(gallery.ModelConfig{
+				Name:       "fetched",
+				ConfigFile: "backend: vllm\n",
+			})
+			Expect(err).ToNot(HaveOccurred())
+			payloadPath := filepath.Join(tempdir, "payload.yaml")
+			Expect(os.WriteFile(payloadPath, payload, 0600)).To(Succeed())
+
+			e := gallery.GalleryModel{}
+			e.Name = "from-url"
+			e.URL = "file://" + payloadPath
+			newGallery(e)
+
+			Expect(install("from-url", gallery.GalleryModel{})).To(Succeed())
+			Expect(installedConfig("from-url")["backend"]).To(Equal("vllm"))
+		})
+	})
+
+	// One of the entries that shipped uninstallable, driven through the real
+	// install path rather than only checked as text. Its files: are swapped for
+	// a local one because the real ones are gigabytes on HuggingFace; its
+	// overrides: are the catalog's own, so this proves the authored payload
+	// lands.
+	It("installs a previously broken index entry once the empty base is allowed", func() {
+		entries, err := loadGalleryIndex()
+		Expect(err).ToNot(HaveOccurred())
+
+		var e gallery.GalleryModel
+		for _, candidate := range entries {
+			if candidate.Name == "liquidai_lfm2-1.2b-rag" {
+				e = candidate
+				break
+			}
+		}
+		Expect(e.Name).To(Equal("liquidai_lfm2-1.2b-rag"))
+		// The defect: no base config of any kind, which used to be fatal.
+		Expect(e.URL).To(BeEmpty())
+		Expect(e.ConfigFile).To(BeEmpty())
+		Expect(e.Overrides).ToNot(BeEmpty())
+
+		e.AdditionalFiles = []gallery.File{localWeights("LiquidAI_LFM2-1.2B-RAG-Q4_K_M.gguf")}
+		newGallery(e)
+
+		Expect(install(e.Name, gallery.GalleryModel{})).To(Succeed())
+		cfg := installedConfig(e.Name)
+		Expect(cfg["name"]).To(Equal(e.Name))
+		// The catalog's own overrides, verbatim, laid over the empty base.
+		Expect(cfg["parameters"]).To(Equal(e.Overrides["parameters"]))
+		Expect(cfg["known_usecases"]).To(Equal(e.Overrides["known_usecases"]))
+	})
+})

core/gallery/gallery.go

@@ -334,6 +334,27 @@ var (
 // invalidate entries when the gallery data changes.
 func GalleryGeneration() uint64 { return galleryGeneration.Load() }
 
+// ResetGalleryModelCache drops the cached model list, once any background
+// refresh already in flight has finished writing to it.
+//
+// It exists for tests. The cache is a package global keyed by nothing, which is
+// right for a process serving one gallery configuration and wrong for a suite
+// where each spec stands up its own: a refresh one spec triggered can land in
+// the middle of the next and answer it with the previous spec's entries, so
+// whichever assertion happens to straddle it fails at random.
+//
+// Waiting for the in-flight refresh rather than only clearing is the point. The
+// refresh publishes its result after this call would otherwise have returned,
+// so clearing without waiting just narrows the window.
+func ResetGalleryModelCache() {
+	for refreshing.Load() {
+		time.Sleep(time.Millisecond)
+	}
+	availableModelsMu.Lock()
+	availableModelsCache = nil
+	availableModelsMu.Unlock()
+}
+
 // AvailableGalleryModelsCached returns gallery models from an in-memory cache.
 // Local-only fields (installed status) are refreshed on every call. A background
 // goroutine is triggered to re-fetch the full model list (including network

core/gallery/model_artifacts.go

@@ -19,6 +19,7 @@ type ArtifactMaterializer interface {
 
 type installOptions struct {
 	materializer ArtifactMaterializer
+	variant      string
 }
 
 type InstallOption func(*installOptions)
@@ -31,6 +32,16 @@ func WithArtifactMaterializer(materializer ArtifactMaterializer) InstallOption {
 	}
 }
 
+// WithVariant pins a gallery entry to a specific variant by name, bypassing
+// hardware-based selection. The entry's own name is a valid pin, since the
+// entry is itself the last resort. Ignored for entries that declare no
+// variants.
+func WithVariant(variant string) InstallOption {
+	return func(options *installOptions) {
+		options.variant = variant
+	}
+}
+
 func applyInstallOptions(options ...InstallOption) installOptions {
 	result := installOptions{materializer: modelartifacts.NewDefaultManager()}
 	for _, option := range options {

core/gallery/models.go

@@ -9,6 +9,7 @@ import (
 	"path/filepath"
 	"slices"
 	"strings"
+	"time"
 
 	"dario.cat/mergo"
 	lconfig "github.com/mudler/LocalAI/core/config"
@@ -17,6 +18,8 @@ import (
 	"github.com/mudler/LocalAI/pkg/modelartifacts"
 	"github.com/mudler/LocalAI/pkg/system"
 	"github.com/mudler/LocalAI/pkg/utils"
+	"github.com/mudler/LocalAI/pkg/vram"
+	"github.com/mudler/LocalAI/pkg/xsysinfo"
 
 	"github.com/mudler/xlog"
 	"gopkg.in/yaml.v3"
@@ -60,6 +63,13 @@ type ModelConfig struct {
 	ConfigFile      string           `yaml:"config_file"`
 	Files           []File           `yaml:"files"`
 	PromptTemplates []PromptTemplate `yaml:"prompt_templates"`
+
+	// The fields below record how an entry carrying variants was resolved,
+	// so a reinstall or upgrade can honor the same pin and so operators can
+	// see which variant a stable model name is actually backed by.
+	EntryName       string `yaml:"entry_name,omitempty"`
+	ResolvedVariant string `yaml:"resolved_variant,omitempty"`
+	PinnedVariant   string `yaml:"pinned_variant,omitempty"`
 }
 
 type File struct {
@@ -73,6 +83,292 @@ type PromptTemplate struct {
 	Content string `yaml:"content"`
 }
 
+// variantOptions turns an entry's declared variants into selectable options,
+// resolving each referenced gallery entry so the selector gets the backend it
+// filters on, and appends the entry itself as the base option.
+//
+// The base is appended rather than special-cased downstream so there is exactly
+// one selection path, and so an operator can pin the entry's own name to
+// decline an upgrade their hardware would otherwise take.
+func variantOptions(models []*GalleryModel, entry *GalleryModel, env ResolveEnv) ([]VariantOption, error) {
+	options := make([]VariantOption, 0, len(entry.Variants)+1)
+	for _, v := range entry.Variants {
+		// Every referenced entry is looked up here rather than lazily after
+		// selection, because the backend that decides whether a variant is even
+		// eligible lives on that entry, not on the variant.
+		target := FindGalleryElement(models, v.Model)
+		if target == nil {
+			return nil, fmt.Errorf("model %q references variant %q which does not exist in any configured gallery", entry.Name, v.Model)
+		}
+		if target.HasVariants() {
+			return nil, fmt.Errorf("model %q references variant %q which declares variants of its own; resolution is a single pass, so those would be silently ignored", entry.Name, v.Model)
+		}
+
+		// Tags come from the REFERENCED entry, not from the declaring one: the
+		// serving feature being ranked is a property of the build that would be
+		// installed, and a parent's tags describe the model family.
+		option := VariantOption{Variant: v, Backend: target.Backend, Tags: target.Tags}
+		if env.ProbeMemory != nil {
+			option.ProbedMemory = env.ProbeMemory(target)
+		}
+		options = append(options, option)
+	}
+
+	// The base is probed like any other candidate. It is never filtered out, but
+	// it IS ranked against the variants, and an unsized base would lose every
+	// contest to a variant whose size nothing could measure.
+	base := VariantOption{
+		Variant: Variant{Model: entry.Name},
+		Backend: entry.Backend,
+		IsBase:  true,
+		Tags:    entry.Tags,
+	}
+	if env.ProbeMemory != nil {
+		base.ProbedMemory = env.ProbeMemory(entry)
+	}
+	return append(options, base), nil
+}
+
+// probeContextLength is the context size the footprint estimate is taken at.
+// Selection compares whole models against whole hosts, so this only has to be a
+// consistent, realistic default rather than the context a user will eventually
+// configure.
+const probeContextLength = 8192
+
+// probeTimeout bounds a single entry's probe. An entry with several variants
+// probes each of them, so an unreachable host has to give up quickly: an
+// unknown size only costs a variant its ranking, whereas a stalled probe costs
+// the user the whole install.
+const probeTimeout = 5 * time.Second
+
+// probeEntryMemory measures what a gallery entry will occupy, without
+// downloading it.
+//
+// pkg/vram does the work and caches its results across calls: it range-fetches
+// the GGUF header for a real estimate, falls back to an HTTP HEAD for the
+// content length, and finally to any declared size:. A HuggingFace repo listing
+// is deliberately NOT used as a further fallback, unlike the gallery UI's
+// estimator: a quantization entry's urls routinely point at the base model's
+// repo, and summing every weight file there would overstate one variant badly
+// enough to wrongly filter it out.
+//
+// Zero means the size could not be determined. Callers must read that as
+// unknown rather than as zero, so an unreachable network downgrades a variant's
+// ranking instead of failing the install.
+func probeEntryMemory(ctx context.Context, entry *GalleryModel) uint64 {
+	input := vram.ModelEstimateInput{Size: entry.Size}
+	for _, f := range entry.AdditionalFiles {
+		if vram.IsWeightFile(f.URI) {
+			input.Files = append(input.Files, vram.FileInput{URI: f.URI})
+		}
+	}
+	if len(input.Files) == 0 && input.Size == "" {
+		return 0
+	}
+
+	ctx, cancel := context.WithTimeout(ctx, probeTimeout)
+	defer cancel()
+
+	estimate, err := vram.EstimateModelMultiContext(ctx, input, []uint32{probeContextLength})
+	if err != nil {
+		xlog.Debug("Could not probe a model variant's size; treating it as unknown", "model", entry.Name, "error", err)
+		return 0
+	}
+	return estimate.VRAMForContext(probeContextLength)
+}
+
+// ResolveVariant picks the gallery entry to install for a host, from the
+// variants an entry declares plus the entry itself, and returns it renamed to
+// the entry's name and carrying the entry's presentation metadata.
+//
+// Why the metadata split: the payload (url, config_file, files, overrides)
+// must come from the chosen variant because that is what actually gets
+// downloaded, while the presentation (name, description, icon, tags) must come
+// from the entry the user asked for, so the installed model presents as that
+// model rather than as one of its variants.
+//
+// The base entry always resolves, whatever the host has. It is a complete entry
+// that every older LocalAI release installs unconditionally, so refusing it here
+// would make the gallery behave worse the newer the client is. Being exempt from
+// the filters does not make it exempt from ranking: it is measured and compared
+// like every declared variant, and wins by default only once they have all been
+// ruled out.
+func ResolveVariant(models []*GalleryModel, entry *GalleryModel, env ResolveEnv, pin string) (*GalleryModel, Variant, error) {
+	options, err := variantOptions(models, entry, env)
+	if err != nil {
+		return nil, Variant{}, err
+	}
+
+	selection, err := SelectVariant(options, env, pin)
+	if err != nil {
+		return nil, Variant{}, fmt.Errorf("resolving variant for model %q: %w", entry.Name, err)
+	}
+	selected := selection.Option
+
+	if selection.FellBackToBase && len(entry.Variants) > 0 {
+		xlog.Warn("No declared variant of this model fits this system; installing the entry's own build",
+			"model", entry.Name, "available_memory", env.AvailableMemory, "reasons", strings.Join(selection.Reasons, "; "))
+	}
+
+	// A pin is an operator override and deliberately bypasses the hardware
+	// checks, but a silent bypass makes a later out-of-memory failure
+	// impossible to trace back to the pin, so it is recorded loudly here.
+	if pin != "" {
+		if need, known := selected.EffectiveMemory(); known && env.AvailableMemory < need {
+			xlog.Warn("Pinned model variant declares more memory than this system reports; installing anyway because the pin overrides hardware resolution",
+				"model", entry.Name, "variant", selected.Variant.Model, "required_memory", need, "available_memory", env.AvailableMemory)
+		}
+	}
+
+	// Selecting the base means installing the entry's own payload, which is the
+	// entry itself; there is no second entry to look up.
+	source := entry
+	if !selected.IsBase {
+		// variantOptions already proved this lookup succeeds.
+		source = FindGalleryElement(models, selected.Variant.Model)
+	}
+
+	resolved := *source
+	resolved.Name = entry.Name
+	resolved.Description = entry.Description
+	resolved.Icon = entry.Icon
+	resolved.License = entry.License
+	// The resolved entry is a concrete install target, so it must not carry the
+	// variant list any more; leaving it would let a second pass resolve the
+	// already-resolved entry all over again.
+	resolved.Variants = nil
+
+	// The struct copy above is shallow, so every reference-typed field still
+	// aliases the gallery's own entries. The install path mutates Overrides in
+	// place (mergo merges the caller's request overrides into it) and appends to
+	// the URL and tag slices, which would write the caller's request into the
+	// gallery catalog itself and leak between installs the moment this path
+	// reads from a cached, long-lived gallery listing. Detach them here.
+	//
+	// Overrides and ConfigFile are copied all the way down rather than cloned at
+	// the top level only: gallery overrides are nested in practice (a
+	// parameters.model map is near-universal), and mergo recurses into nested
+	// maps and overwrites them in place, so a top-level clone would still hand
+	// the caller the gallery's own inner maps. The slices below hold value types,
+	// so cloning them once fully detaches them.
+	resolved.Overrides = deepCopyStringMap(source.Overrides)
+	resolved.ConfigFile = deepCopyStringMap(source.ConfigFile)
+	resolved.AdditionalFiles = slices.Clone(source.AdditionalFiles)
+	resolved.URLs = slices.Clone(entry.URLs)
+	resolved.Tags = slices.Clone(entry.Tags)
+
+	return &resolved, selected.Variant, nil
+}
+
+// HostResolveEnv describes this machine to variant selection.
+//
+// It exists so the install path and every read-only surface that reports what
+// selection WOULD choose derive the host from one place. Two copies of this
+// wiring would eventually disagree, and a picker that shows a different answer
+// than the installer produces is worse than no picker at all.
+func HostResolveEnv(ctx context.Context, systemState *system.SystemState) ResolveEnv {
+	return ResolveEnv{
+		AvailableMemory: availableModelMemory(systemState),
+		// The whole hardware gate. IsBackendCompatible already derives
+		// Darwin-only, NVIDIA-only, ROCm-only and SYCL-only from the backend
+		// name, so a gallery author never has to describe hardware.
+		//
+		// The uri argument is deliberately empty: it exists for backend OCI
+		// images, and passing a model's gallery url here would let an unrelated
+		// substring in a download link decide hardware compatibility.
+		BackendCompatible: func(backend string) bool {
+			return systemState.IsBackendCompatible(backend, "")
+		},
+		// The ranking half of the hardware story. IsBackendCompatible only rules
+		// out what cannot run; on a Mac both an MLX and a llama.cpp build can,
+		// and this is what makes the native runtime win.
+		//
+		// EnginePreferenceTokens, NOT BackendPreferenceTokens: a variant is
+		// matched on its gallery `backend:` engine name, and the latter reports
+		// build tags that no engine name contains.
+		EnginePreference: systemState.EnginePreferenceTokens(),
+		// The third vocabulary, and the only one that is not host derived: no
+		// hardware prefers a plain build over an equivalent faster one, so this
+		// comes from a package function rather than from systemState.
+		ServingFeaturePreference: system.ServingFeaturePreferenceTokens(),
+		ProbeMemory: func(target *GalleryModel) uint64 {
+			return probeEntryMemory(ctx, target)
+		},
+	}
+}
+
+// noGPUDetected is what SystemState.DetectedCapability() reports when no
+// usable GPU was found. The constant is unexported in pkg/system.
+const noGPUDetected = "default"
+
+// availableModelMemory reports how much memory a model may occupy on this host.
+//
+// With a discrete GPU the model lives in VRAM. Otherwise it lives in system
+// RAM, read through xsysinfo because that path is cgroup-aware: under
+// Kubernetes the container's limit, not the node's physical RAM, is what the
+// model actually gets.
+//
+// A detected GPU whose VRAM reads as zero falls back to RAM rather than to
+// zero. That is the normal shape of a unified-memory host, not an error:
+// arm64 macs report the metal capability unconditionally, yet have no discrete
+// VRAM pool for TotalAvailableVRAM to find, so the model's real budget is
+// system RAM. Returning the zero here would strand every Mac on the base
+// build no matter how much memory it has.
+//
+// An unreadable RAM figure yields 0, which drops every variant with a known
+// requirement and installs the base. That is the safe direction: an unknown
+// host should not be talked into a larger download.
+func availableModelMemory(systemState *system.SystemState) uint64 {
+	if systemState.DetectedCapability() != noGPUDetected && systemState.VRAM > 0 {
+		return systemState.VRAM
+	}
+	ram, err := xsysinfo.GetSystemRAMInfo()
+	if err != nil || ram == nil {
+		xlog.Warn("Could not read system RAM; treating this host as having no memory budget for variant selection", "error", err)
+		return 0
+	}
+	return ram.Total
+}
+
+// deepCopyStringMap copies a decoded YAML map so no part of the result, at any
+// depth, is reachable from the original.
+func deepCopyStringMap(m map[string]any) map[string]any {
+	if m == nil {
+		return nil
+	}
+	out := make(map[string]any, len(m))
+	for k, v := range m {
+		out[k] = deepCopyYAMLValue(v)
+	}
+	return out
+}
+
+// deepCopyYAMLValue recurses through the only container shapes a YAML decoder
+// produces. Scalars are returned as-is because they cannot be mutated through
+// the copy. map[any]any is handled as well because gopkg.in/yaml.v2 decodes
+// non-string keys into it, and gallery documents are not guaranteed to have
+// passed through the v3 decoder.
+func deepCopyYAMLValue(v any) any {
+	switch t := v.(type) {
+	case map[string]any:
+		return deepCopyStringMap(t)
+	case map[any]any:
+		out := make(map[any]any, len(t))
+		for k, val := range t {
+			out[k] = deepCopyYAMLValue(val)
+		}
+		return out
+	case []any:
+		out := make([]any, len(t))
+		for i, val := range t {
+			out[i] = deepCopyYAMLValue(val)
+		}
+		return out
+	default:
+		return v
+	}
+}
+
 // Installs a model from the gallery
 func InstallModelFromGallery(
 	ctx context.Context,
@@ -81,7 +377,9 @@ func InstallModelFromGallery(
 	modelLoader *model.ModelLoader,
 	name string, req GalleryModel, downloadStatus func(string, string, string, float64), enforceScan, automaticallyInstallBackend, requireBackendIntegrity bool, options ...InstallOption) error {
 
-	applyModel := func(model *GalleryModel) error {
+	installOpts := applyInstallOptions(options...)
+
+	applyModel := func(model *GalleryModel, record *ModelConfig) error {
 		name = strings.ReplaceAll(name, string(os.PathSeparator), "__")
 
 		var config ModelConfig
@@ -104,13 +402,46 @@ func InstallModelFromGallery(
 				ConfigFile:  string(reYamlConfig),
 				Description: model.Description,
 				License:     model.License,
-				URLs:        model.URLs,
 				Name:        model.Name,
 				Files:       make([]File, 0), // Real values get added below, must be blank
+				// URLs are deliberately not seeded here: they are appended once
+				// below for both branches, and seeding them too would write every
+				// URL twice into the persisted gallery file.
 				// Prompt Template Skipped for now - I expect in this mode that they will be delivered as files.
 			}
 		} else {
-			return fmt.Errorf("invalid gallery model %+v", model)
+			// An entry that names no base config describes itself entirely
+			// through overrides: and files:, so the base is simply empty.
+			//
+			// This is what the several hundred entries pointing at
+			// gallery/virtual.yaml were already getting. That stub carries only
+			// a name, a description and a license, all three of which are
+			// overwritten from the gallery entry a few lines up, and overrides
+			// reach InstallModel as a separate argument rather than being merged
+			// into the fetched config. So the fetch bought nothing beyond a
+			// round trip to GitHub on every install, and an author who omits the
+			// key is asking for exactly the same thing.
+			if !model.installsSomething(req) {
+				return fmt.Errorf("gallery model %q installs nothing: it declares no url, no config_file, no overrides and no files", model.Name)
+			}
+			config = ModelConfig{
+				Description: model.Description,
+				License:     model.License,
+				Name:        model.Name,
+				Files:       make([]File, 0), // Real values get added below, must be blank
+				// URLs are appended once below for every branch, as in the
+				// config_file case above.
+			}
+		}
+
+		if record != nil {
+			config.EntryName = record.EntryName
+			config.ResolvedVariant = record.ResolvedVariant
+			config.PinnedVariant = record.PinnedVariant
+			// The variant's own name would otherwise be persisted here, which
+			// contradicts the whole point of variants: the model is known by
+			// the entry's stable name regardless of which variant backs it.
+			config.Name = record.EntryName
 		}
 
 		installName := model.Name
@@ -157,7 +488,77 @@ func InstallModelFromGallery(
 		return fmt.Errorf("no model found with name %q", name)
 	}
 
-	return applyModel(model)
+	// An entry without variants is installed directly. An entry with them is
+	// still installable as-is; selection below only decides whether one of its
+	// declared alternatives suits this host better.
+	if !model.HasVariants() {
+		// A caller who named a variant asked for something this entry cannot
+		// give. Installing the entry anyway would look like success while
+		// quietly ignoring the choice, so it is refused by name instead.
+		if installOpts.variant != "" {
+			return fmt.Errorf("%w: %q was requested but model %q declares no variants", ErrPinNotFound, installOpts.variant, model.Name)
+		}
+		return applyModel(model, nil)
+	}
+
+	pin := installOpts.variant
+	// A previously recorded pin survives reinstalls and upgrades, so a user
+	// who deliberately chose a variant is not silently re-resolved onto a
+	// different one by a hardware or gallery change.
+	//
+	// The record is keyed by the name the model was installed under, not by the
+	// gallery entry name: applyModel writes it to ._gallery_<installName>.yaml,
+	// where installName is req.Name whenever the caller supplied one. Reading it
+	// back under the entry's own name would miss the record for every custom-named
+	// install and silently re-resolve a deliberately pinned model onto a
+	// different variant, possibly swapping its backend.
+	//
+	// recalledPin tracks where the pin came from, because the two sources must
+	// fail differently when the name no longer resolves. See below.
+	recalledPin := ""
+	if pin == "" {
+		installName := model.Name
+		if req.Name != "" {
+			installName = req.Name
+		}
+		if previous, err := GetLocalModelConfiguration(systemState.Model.ModelsPath, installName); err == nil && previous != nil {
+			recalledPin = previous.PinnedVariant
+			pin = recalledPin
+		}
+	}
+
+	env := HostResolveEnv(ctx, systemState)
+
+	resolved, variant, err := ResolveVariant(models, model, env, pin)
+	// A pin the caller supplied on this request must stay fatal: they named
+	// something this entry cannot give, and installing anything else would
+	// report success for a request that was not honored.
+	//
+	// A pin recalled from disk is different. The gallery can rename or withdraw
+	// a variant long after it was pinned, and the user is not asking for it
+	// again on this call. Failing here would turn one gallery edit into a
+	// permanently unrepairable model whose only remedy is deleting a dotfile
+	// they have never heard of, so the stale pin is dropped and selection runs
+	// as if it had never been recorded.
+	if err != nil && recalledPin != "" && errors.Is(err, ErrPinNotFound) {
+		xlog.Warn("The recorded variant pin for this model no longer exists in the gallery; re-selecting automatically",
+			"model", model.Name, "dropped_pin", recalledPin)
+		pin = ""
+		resolved, variant, err = ResolveVariant(models, model, env, "")
+	}
+	if err != nil {
+		return err
+	}
+
+	xlog.Info("Resolved model to variant",
+		"model", model.Name, "variant", variant.Model,
+		"available_memory", env.AvailableMemory, "pinned", pin != "")
+
+	return applyModel(resolved, &ModelConfig{
+		EntryName:       model.Name,
+		ResolvedVariant: variant.Model,
+		PinnedVariant:   pin,
+	})
 }
 
 func InstallModel(ctx context.Context, systemState *system.SystemState, nameOverride string, galleryConfig *ModelConfig, configOverrides map[string]any, downloadStatus func(string, string, string, float64), enforceScan bool, options ...InstallOption) (*lconfig.ModelConfig, error) {

core/gallery/models_types.go

@@ -15,6 +15,37 @@ type GalleryModel struct {
 	ConfigFile map[string]any `json:"config_file,omitempty" yaml:"config_file,omitempty"`
 	// Overrides are used to override the configuration of the model located at URL
 	Overrides map[string]any `json:"overrides,omitempty" yaml:"overrides,omitempty"`
+	// Variants is an optional, UNORDERED list of alternative builds of the same
+	// model (other backends such as MLX or vLLM, other quantizations) that the
+	// installer may pick instead of this entry's own payload. Authoring is
+	// deliberately dumb: name the models, and the selector works out which one
+	// this host should get.
+	//
+	// The entry itself is always the last resort, so an entry carrying variants
+	// stays a complete, installable entry and older LocalAI releases, which drop
+	// this key, install it exactly as before.
+	Variants []Variant `json:"variants,omitempty" yaml:"variants,omitempty"`
+}
+
+// installsSomething reports whether this entry, combined with the caller's
+// request, would put anything on disk.
+//
+// It exists to keep an authoring mistake loud. Once an entry with no url and no
+// config_file is accepted as an empty base, the only thing separating a
+// deliberate overrides-only entry from a half-written stanza is whether it
+// carries a payload at all. Without this check the stub would install cleanly
+// and leave a model directory holding a config naming no weights, which fails
+// far from the entry that caused it.
+//
+// The request is counted because its overrides and files are merged into the
+// install exactly as the entry's own are, so a caller supplying them really has
+// asked for something installable.
+//
+// An entry with a url or a config_file never reaches this: it has a base config
+// to install, however thin.
+func (m *GalleryModel) installsSomething(req GalleryModel) bool {
+	return len(m.Overrides) > 0 || len(m.AdditionalFiles) > 0 ||
+		len(req.Overrides) > 0 || len(req.AdditionalFiles) > 0
 }
 
 func (m *GalleryModel) GetInstalled() bool {
@@ -45,6 +76,13 @@ func (m GalleryModel) ID() string {
 	return fmt.Sprintf("%s@%s", m.Gallery.Name, m.Name)
 }
 
+// HasVariants reports whether this entry declares alternative builds of itself.
+// It says nothing about the entry being installable: an entry with variants is
+// a complete entry that can always be installed as-is.
+func (m GalleryModel) HasVariants() bool {
+	return len(m.Variants) > 0
+}
+
 func (m *GalleryModel) GetTags() []string {
 	return m.Tags
 }

core/gallery/resolve_variant.go

@@ -0,0 +1,435 @@
+package gallery
+
+import (
+	"errors"
+	"fmt"
+	"slices"
+	"sort"
+	"strings"
+)
+
+var (
+	// ErrNoVariantMatch is returned when nothing at all is selectable, which
+	// only happens for a caller that supplies no base option.
+	ErrNoVariantMatch = errors.New("no model variant matches this system")
+	// ErrPinNotFound is returned when a pinned variant is absent from the list.
+	ErrPinNotFound = errors.New("pinned model variant not found")
+)
+
+// VariantOption is a variant paired with the facts a host is matched against.
+//
+// The install layer builds these by looking the referenced entries up in the
+// live gallery. The selector never touches the catalog, which is what keeps it
+// pure and testable across hardware shapes without touching the machine.
+type VariantOption struct {
+	Variant Variant
+	// Backend is the engine the referenced entry resolves to (e.g. "llama-cpp",
+	// "mlx", "vllm"). Hardware support is derived from this name alone, which is
+	// precisely why a gallery author never has to describe hardware.
+	Backend string
+	// IsBase marks the declaring entry's own payload. It is exempt from every
+	// filter, because the entry must stay installable on every host and for
+	// every client, but it is otherwise an ordinary candidate: it is ranked
+	// against the declared variants rather than consulted only once they have
+	// all been rejected.
+	IsBase bool
+	// ProbedMemory is the footprint measured live from the referenced entry's
+	// weights, in bytes. It is the only source of a variant's size. An author
+	// who needs to correct a bad estimate sets `size:` on the referenced entry,
+	// which the estimator behind the probe already prefers over its own
+	// guesswork, so the correction lands everywhere the size is used.
+	//
+	// Zero means the probe could not determine a size. That is an unknown, not
+	// a zero requirement: a probe that cannot reach the network must never be
+	// able to break an install.
+	ProbedMemory uint64
+	// Tags are the referenced entry's declared tags, and they are the SOLE
+	// serving feature signal. A tag is something an author wrote down on
+	// purpose, whereas an entry name only happens to contain a marker, so
+	// nothing else is consulted.
+	//
+	// Empty therefore means "declares no serving feature", and an entry that
+	// enables one without saying so ranks as a plain build. That is the cost of
+	// making the signal a declaration, and it is the right cost: an install
+	// silently downgraded to the plain build is recoverable, whereas a build
+	// promoted on a marker nobody meant is not visible at all.
+	// .agents/adding-gallery-models.md states the tagging rule.
+	Tags []string
+}
+
+// EffectiveMemory returns this option's memory requirement in bytes and whether
+// one is known at all.
+func (o VariantOption) EffectiveMemory() (uint64, bool) {
+	if o.ProbedMemory > 0 {
+		return o.ProbedMemory, true
+	}
+	return 0, false
+}
+
+// ResolveEnv describes the host a variant is selected for.
+type ResolveEnv struct {
+	// AvailableMemory is what a model may occupy on this host: VRAM when a
+	// usable GPU was detected, system RAM otherwise. A model's footprint is
+	// roughly the same in either, so one number and one comparison cover both.
+	AvailableMemory uint64
+	// BackendCompatible reports whether a backend can run here. The install
+	// layer wires SystemState.IsBackendCompatible, which already derives
+	// Darwin-only, NVIDIA-only, ROCm-only and SYCL-only from the backend name.
+	//
+	// A nil func treats every backend as runnable, the right default for a
+	// caller with no view of the hardware.
+	BackendCompatible func(backend string) bool
+	// EnginePreference lists the ENGINE NAMES this host prefers, best first, as
+	// SystemState.EnginePreferenceTokens reports them (e.g. NVIDIA gives
+	// ["vllm", "sglang", "llama-cpp"], metal gives ["mlx", "llama-cpp"]). A
+	// token is matched as a substring of a variant's backend name.
+	//
+	// The vocabulary is load bearing. VariantOption.Backend is a gallery entry's
+	// `backend:` value, which is an engine name and never carries a build tag,
+	// so build tags like "cuda" or "rocm" match NOTHING here. Do not wire
+	// SystemState.BackendPreferenceTokens into this field: that reports build
+	// tags for installed-build alias resolution, and the mismatch does not
+	// error, it scores every candidate equally and silently reduces selection to
+	// size alone. pkg/system/capabilities.go documents both tables.
+	//
+	// BackendCompatible answers "can this run here at all" and is a filter.
+	// This answers "which of the things that CAN run here should win" and is
+	// only a ranking. The two are deliberately separate: a Mac can run both an
+	// MLX build and a llama.cpp build, so nothing is filtered, yet the native
+	// accelerated runtime should still be installed even when the GGUF build is
+	// the larger download.
+	//
+	// An empty list ranks every backend equally, which reduces selection to
+	// ordering by size alone. That is the right default for a caller with no
+	// view of the hardware, and it is what every host looked like before
+	// preference existed.
+	EnginePreference []string
+	// ServingFeaturePreference lists the SERVING FEATURES to prefer, best first,
+	// as system.ServingFeaturePreferenceTokens reports them (currently
+	// ["dflash", "mtp"]). A token matches when it equals one of the variant's
+	// declared TAGS, compared whole and case-insensitively. Nothing else is
+	// matched: not the entry name, not its backend options.
+	//
+	// A third vocabulary, matched against a third thing. It is neither a build
+	// tag nor an engine name: these name a way of serving the same weights
+	// faster. pkg/system/capabilities.go documents all three tables together
+	// and justifies why this one reads a declaration rather than free text.
+	//
+	// An empty list ranks every build equally on this axis, which is what every
+	// host looked like before serving features were ranked at all.
+	ServingFeaturePreference []string
+	// ProbeMemory measures how much memory a referenced gallery entry needs,
+	// without downloading it. A zero result means "could not tell", never
+	// "needs nothing".
+	//
+	// It is a func field rather than a live network handle so specs can pin an
+	// exact size, or an exact failure, without reaching the internet. A nil func
+	// leaves every variant unknown, which selection already handles.
+	//
+	// SelectVariant never calls this: the install layer resolves every size into
+	// VariantOption.ProbedMemory first, so the selector stays pure.
+	ProbeMemory func(entry *GalleryModel) uint64
+}
+
+func (e ResolveEnv) backendRuns(backend string) bool {
+	if e.BackendCompatible == nil {
+		return true
+	}
+	return e.BackendCompatible(backend)
+}
+
+// preferenceRank scores a backend against this host's preferred engine order,
+// lower being better.
+//
+// It walks EnginePreference generically and names no engine and no capability:
+// everything a new runtime needs is expressed by the table in pkg/system, so
+// adding one never reaches this function.
+//
+// A backend matching no token scores just below the least preferred known one.
+// It is never dropped, and a field where nothing matches scores uniformly, so
+// an unrecognised backend, an unrecognised capability and an absent preference
+// list all degrade to the same predictable place: ordering by size alone.
+func (e ResolveEnv) preferenceRank(backend string) int {
+	name := strings.ToLower(backend)
+	return preferenceIndex(e.EnginePreference, func(token string) bool {
+		return strings.Contains(name, token)
+	})
+}
+
+// servingFeatureRank scores a variant against the preferred serving feature
+// order, lower being better.
+//
+// It names no feature, exactly as preferenceRank names no engine: the ordered
+// list comes from pkg/system, so teaching LocalAI about a new serving feature
+// is a one-line edit to that table and never reaches this file.
+//
+// A declared TAG is the ONLY signal, compared whole and case-insensitively. An
+// entry name is deliberately not consulted: a naming convention is not a
+// contract, and an author is free to spell a build however they like, so
+// reading a marker out of a name infers a capability nobody declared. Tags are
+// a LocalAI-level vocabulary the gallery curators control and they mean the
+// same thing on every backend. `overrides.options` was considered as the
+// signal and rejected for the opposite reason: spec_type:draft-mtp is what
+// actually enables the feature, but that spelling is llama.cpp's config
+// vocabulary, and a cross-backend ranking decision must not depend on one
+// backend's option syntax. Curators check the options at tagging time; the
+// ranker reads only the tag.
+//
+// A build declaring no feature is a plain build and scores just below the
+// least preferred known one, which is the whole point: whenever a faster way
+// to serve the same weights survived the filters, it outranks the plain build.
+func (e ResolveEnv) servingFeatureRank(o VariantOption) int {
+	tags := lowercased(o.Tags)
+	return preferenceIndex(e.ServingFeaturePreference, func(token string) bool {
+		return slices.Contains(tags, token)
+	})
+}
+
+// servingFeaturesOf lists the serving features a variant declares, best first.
+//
+// It is the descriptive half of servingFeatureRank: that function collapses the
+// same match into one score for ordering, this one names what matched so a
+// picker can tell the user WHY one build is preferred over another. Both read
+// the single vocabulary on ResolveEnv, so a feature can never be shown as an
+// advantage the ranker did not reward, nor rewarded without being shown.
+//
+// Preference order rather than the author's tag order, so the list reads best
+// thing first.
+//
+// nil rather than an empty slice for a plain build, so the view field it feeds
+// is omitted from the wire entirely.
+func (e ResolveEnv) servingFeaturesOf(o VariantOption) []string {
+	if len(e.ServingFeaturePreference) == 0 || len(o.Tags) == 0 {
+		return nil
+	}
+	tags := lowercased(o.Tags)
+	var features []string
+	for _, token := range e.ServingFeaturePreference {
+		if slices.Contains(tags, strings.ToLower(strings.TrimSpace(token))) {
+			features = append(features, token)
+		}
+	}
+	return features
+}
+
+// lowercased folds a tag list for comparison, so a gallery author writing
+// "MTP" declares the same feature as one writing "mtp".
+func lowercased(tags []string) []string {
+	if len(tags) == 0 {
+		return nil
+	}
+	out := make([]string, 0, len(tags))
+	for _, t := range tags {
+		out = append(out, strings.ToLower(strings.TrimSpace(t)))
+	}
+	return out
+}
+
+// preferenceIndex returns the position of the first token `matches` accepts.
+//
+// A subject matching no token scores just below the least preferred known one.
+// It is never dropped, and a list where nothing matches scores uniformly, so an
+// unrecognised subject and an absent preference list degrade to the same
+// predictable place: this axis stops discriminating and the next key decides.
+func preferenceIndex(tokens []string, matches func(token string) bool) int {
+	if len(tokens) == 0 {
+		return 0
+	}
+	for i, token := range tokens {
+		if token != "" && matches(token) {
+			return i
+		}
+	}
+	return len(tokens)
+}
+
+// VariantSelection is the outcome of a selection pass.
+type VariantSelection struct {
+	Option VariantOption
+	// FellBackToBase reports that no declared variant survived the filters and
+	// the entry's own payload was all that remained. Callers log this, because a
+	// host quietly taking the base when upgrades were on offer is worth being
+	// able to see.
+	//
+	// It is deliberately narrower than "the base was selected": the base also
+	// wins on merit whenever it outranks every surviving variant, and that is an
+	// ordinary, uninteresting outcome rather than something to warn about.
+	FellBackToBase bool
+	// Reasons explains, one line per rejected variant, why it was dropped.
+	Reasons []string
+}
+
+// SelectVariant picks the option to install for a host.
+//
+// The algorithm, in order:
+//
+//  1. An explicit pin wins outright, fit or no fit. It is a deliberate operator
+//     override, so refusing it would defeat the point; the caller warns.
+//  2. Variants whose backend cannot run here are dropped. This is the whole of
+//     the hardware gate, derived from the backend name.
+//  3. Variants whose known memory requirement exceeds what the host has are
+//     dropped. A variant with an UNKNOWN requirement survives, because nothing
+//     proves it does not fit and refusing on a size the probe could not read
+//     would let a network hiccup silently downgrade what gets installed.
+//  4. The base survives both filters unconditionally, and then competes. It is
+//     a candidate like any other, not a last resort: an entry whose own build is
+//     the largest thing that fits must win against a smaller variant, and a base
+//     of known size must win against a variant whose size nothing could measure.
+//  5. The survivors are ranked and the best one wins: fit tier first (rankOf
+//     below), then this host's engine preference, then the serving feature
+//     preference, then size. Both preferences beat size deliberately, so a Mac
+//     takes an MLX build over a larger GGUF one, and a host that can hold a
+//     speculative-decoding build of the same weights takes it over the plain
+//     one.
+//  6. With no survivor at all, which can only happen when the caller supplied no
+//     base, there is nothing to install and this reports ErrNoVariantMatch.
+func SelectVariant(options []VariantOption, env ResolveEnv, pin string) (VariantSelection, error) {
+	if pin != "" {
+		for _, o := range options {
+			if strings.EqualFold(o.Variant.Model, pin) {
+				return VariantSelection{Option: o}, nil
+			}
+		}
+		return VariantSelection{}, fmt.Errorf("%w: %q is not among the %d variants of this model", ErrPinNotFound, pin, len(options))
+	}
+
+	type ranked struct {
+		option     VariantOption
+		memory     uint64
+		rank       int
+		preference int
+		feature    int
+	}
+
+	survivors := make([]ranked, 0, len(options))
+	reasons := make([]string, 0, len(options))
+	survivingVariants := 0
+
+	for i := range options {
+		o := options[i]
+		memory, known := o.EffectiveMemory()
+
+		// The base skips both gates. There is nothing below it, so refusing it
+		// would make an entry every older LocalAI installs fine uninstallable on
+		// newer ones.
+		if !o.IsBase {
+			if !env.backendRuns(o.Backend) {
+				reasons = append(reasons, fmt.Sprintf("%s needs backend %q, which cannot run on this system", o.Variant.Model, o.Backend))
+				continue
+			}
+			if known && memory > env.AvailableMemory {
+				reasons = append(reasons, fmt.Sprintf("%s needs %s of memory", o.Variant.Model, humanBytes(memory)))
+				continue
+			}
+			survivingVariants++
+		}
+
+		survivors = append(survivors, ranked{
+			option:     o,
+			memory:     memory,
+			rank:       rankOf(o, env),
+			preference: env.preferenceRank(o.Backend),
+			feature:    env.servingFeatureRank(o),
+		})
+	}
+
+	if len(survivors) == 0 {
+		return VariantSelection{}, fmt.Errorf(
+			"%w: %s of memory available; variants: %s",
+			ErrNoVariantMatch, humanBytes(env.AvailableMemory), strings.Join(reasons, "; "),
+		)
+	}
+
+	// Four keys, in this order, and the order is the whole design:
+	//
+	//  1. rank, the fit tier. Fit is a fact about whether the host can hold the
+	//     build at all, so nothing outranks it.
+	//  2. preference, the host's runtime order. Among builds the host can
+	//     equally hold, the one on the more native runtime wins even when it is
+	//     the smaller download. A Mac given an MLX build and a larger llama.cpp
+	//     build should run MLX; ranking by size first would install the GGUF and
+	//     leave the accelerated build unused. Do NOT move this below size.
+	//  3. feature, the serving feature order. Among builds on an equally
+	//     preferred runtime, one that speculates or predicts several tokens per
+	//     step beats the plain build of the same weights, because it answers
+	//     faster for the same output.
+	//
+	//     Engine outranks this deliberately. A serving feature makes the right
+	//     engine faster; it does not make a wrong engine right. Wearing the
+	//     wrong engine costs the whole GPU, whereas a plain build on the native
+	//     engine is merely slower than it could be, so a DFlash build on the
+	//     portable engine must not beat a plain build on the engine this host
+	//     actually serves well.
+	//  4. memory, largest first, because among builds on equally preferred
+	//     runtimes with the same serving feature a bigger build is a higher
+	//     quality quantization of the same model. Neither preference key may
+	//     flatten this: two plain builds on one backend are still ordered by
+	//     size.
+	//
+	// Stable so that options equal on all four keep their authored order, which
+	// is the only thing order still decides.
+	sort.SliceStable(survivors, func(i, j int) bool {
+		if survivors[i].rank != survivors[j].rank {
+			return survivors[i].rank < survivors[j].rank
+		}
+		if survivors[i].preference != survivors[j].preference {
+			return survivors[i].preference < survivors[j].preference
+		}
+		if survivors[i].feature != survivors[j].feature {
+			return survivors[i].feature < survivors[j].feature
+		}
+		return survivors[i].memory > survivors[j].memory
+	})
+
+	winner := survivors[0].option
+	return VariantSelection{
+		Option: winner,
+		// Only a base that won by default is worth reporting. A base that
+		// outranked live competition is an ordinary selection.
+		FellBackToBase: winner.IsBase && survivingVariants == 0,
+		Reasons:        reasons,
+	}, nil
+}
+
+// Fit tiers, best first. Within a tier the host's preferred runtime wins, then
+// the preferred serving feature, then the larger footprint; see the sort in
+// SelectVariant.
+const (
+	// rankProvenFit is a measured size that the host is measured to satisfy.
+	rankProvenFit = iota
+	// rankBase is the entry's own build when it is not a proven fit: either it
+	// needs more memory than the host reports, or its size could not be
+	// measured either. It still outranks any unsized variant, because it is the
+	// payload the entry is guaranteed to be able to install and a variant of
+	// unmeasurable size is a guess. Taking the guess on a host that cannot be
+	// shown to accommodate it is how an unreachable network silently changes
+	// what gets installed.
+	rankBase
+	// rankUnknownFit is a variant whose size nothing could measure. Nothing
+	// proves it does not fit, so it is not dropped, but nothing proves it does
+	// either, so it ranks last.
+	rankUnknownFit
+)
+
+func rankOf(o VariantOption, env ResolveEnv) int {
+	if memory, known := o.EffectiveMemory(); known && memory <= env.AvailableMemory {
+		return rankProvenFit
+	}
+	if o.IsBase {
+		return rankBase
+	}
+	return rankUnknownFit
+}
+
+func humanBytes(b uint64) string {
+	const unit = 1024
+	if b < unit {
+		return fmt.Sprintf("%dB", b)
+	}
+	div, exp := uint64(unit), 0
+	for n := b / unit; n >= unit; n /= unit {
+		div *= unit
+		exp++
+	}
+	return fmt.Sprintf("%.1f%ciB", float64(b)/float64(div), "KMGTPE"[exp])
+}

core/gallery/variant.go

@@ -0,0 +1,18 @@
+package gallery
+
+// Variant is one option in a gallery entry's variant list. It references an
+// existing gallery entry by name, and that is all an author has to write.
+//
+// Authored order carries no meaning. Selection filters out what this host
+// cannot run and then ranks what is left, so hardware knowledge lives in the
+// selector rather than being pushed onto whoever edits the gallery.
+//
+// There is deliberately no authored memory figure here. When the live probe
+// misreads a variant's footprint, the fix is the referenced entry's own `size:`
+// field: the estimator already prefers a declared size over its own guesswork,
+// so correcting it there fixes the figure for every consumer rather than only
+// for this one selection pass.
+type Variant struct {
+	// Model is the name of a gallery entry that declares no variants of its own.
+	Model string `json:"model" yaml:"model"`
+}

core/gallery/variant_quantization.go

@@ -0,0 +1,147 @@
+package gallery
+
+import (
+	"path"
+	"regexp"
+	"strings"
+)
+
+// quantizationToken matches one filename segment that names a weight format.
+//
+// The alternatives, in order: the GGUF k-quant/legacy family with any number of
+// trailing qualifiers (`q8_0`, `q4_k_m`, `iq2_xxs`, `pq2_0`, `q2_0_g128`), the
+// float formats (`f16`, `fp16`, `bf16`, `f32`) including the vendor 4-bit ones
+// (`nvfp4`, `mxfp4`), and the bit-count spellings the MLX and AWQ ecosystems
+// use (`4bit`, `int8`).
+//
+// It is anchored at both ends because it is applied to an already-split
+// segment. Matching mid-string would let a repo or model name containing a
+// quant-shaped substring be reported as the quantization.
+var quantizationToken = regexp.MustCompile(`^(?:p?i?q[0-9]+(?:_[0-9a-z]+)*|f(?:p)?(?:8|16|32)|bf16|(?:nv|mx)fp[0-9]+|int[0-9]+|[0-9]+bit)$`)
+
+// weightExtensions are stripped before a filename is split into segments, so a
+// model whose quantization is the last thing in its name (`...-Q8_0.gguf`) is
+// not read as the segment `gguf`.
+var weightExtensions = []string{".gguf", ".safetensors", ".bin", ".pt", ".pth", ".onnx"}
+
+// quantizationFromFilename reports the weight format named in a model
+// filename, uppercased, or "" when the name does not declare one.
+//
+// Segments are scanned from the END because that is where the qualifier sits by
+// near-universal convention (`Ternary-Bonsai-27B-Q2_g64.gguf`), and because a
+// repo path may itself carry a quant-shaped segment that describes the
+// repository rather than this file.
+//
+// Only `-` and `.` split segments; `_` deliberately does not, because it is the
+// separator INSIDE a quant token (`Q4_K_M`) and splitting on it would report
+// `Q4` for a build that is not Q4.
+//
+// Two passes, in decreasing order of confidence. A whole segment naming a
+// format is the unambiguous case and always wins. Only when no segment does is
+// the `_`-delimited tail of a segment considered, which catches the authoring
+// style that runs the format into the model name (`gemma-4-E2B_q4_0-it.gguf`,
+// a real and populous gallery family). Running the loose pass second rather
+// than inline keeps a precise match from ever losing to a fuzzy one further
+// right in the name.
+//
+// The result is uppercased so one gallery does not show `Q2_g64` next to
+// `Q2_G64` for what is the same format spelled two ways by two authors.
+func quantizationFromFilename(filename string) string {
+	if filename == "" {
+		return ""
+	}
+
+	base := path.Base(strings.TrimSpace(filename))
+	lower := strings.ToLower(base)
+	for _, ext := range weightExtensions {
+		if strings.HasSuffix(lower, ext) {
+			base = base[:len(base)-len(ext)]
+			break
+		}
+	}
+
+	segments := strings.FieldsFunc(base, func(r rune) bool { return r == '-' || r == '.' })
+
+	for i := len(segments) - 1; i >= 0; i-- {
+		if quantizationToken.MatchString(strings.ToLower(segments[i])) {
+			return strings.ToUpper(segments[i])
+		}
+	}
+
+	for i := len(segments) - 1; i >= 0; i-- {
+		if tail := quantizationTail(segments[i]); tail != "" {
+			return tail
+		}
+	}
+	return ""
+}
+
+// quantizationTail reports a weight format run into the tail of a segment,
+// uppercased, or "" when there is none.
+//
+// It walks the `_`-delimited tails from the LONGEST first, so `e2b_q4_0` yields
+// `Q4_0` rather than the `0` a shortest-first walk would reach. The full
+// segment is not retried here; the caller has already ruled it out.
+func quantizationTail(segment string) string {
+	for i, r := range segment {
+		if r != '_' {
+			continue
+		}
+		tail := segment[i+1:]
+		if quantizationToken.MatchString(strings.ToLower(tail)) {
+			return strings.ToUpper(tail)
+		}
+	}
+	return ""
+}
+
+// quantizationOfEntry reports the weight format a gallery entry installs.
+//
+// `overrides.parameters.model` is preferred over the file list because it names
+// the file the backend is actually pointed at. An entry routinely ships more
+// than one weight file (a vision tower alongside the language model, for
+// instance), and those companions carry their own, different quantization: the
+// Bonsai entries pair a Q2_0 model with a Q8_0 mmproj, so reading the file list
+// first would report Q8_0 for a Q2_0 build.
+//
+// Returning "" is a normal outcome, not a failure. A backend served from a
+// directory of safetensors, or an entry whose name simply does not encode a
+// format, has no quantization to report and callers must render its absence
+// rather than a guess.
+func quantizationOfEntry(entry *GalleryModel) string {
+	if entry == nil {
+		return ""
+	}
+
+	if q := quantizationFromFilename(overrideModelParameter(entry)); q != "" {
+		return q
+	}
+
+	// The file list is the fallback for entries that install a config_file
+	// naming no model parameter of their own. First file wins: the language
+	// model is listed first by convention, and there is nothing better to go on
+	// once the authoritative pointer is absent.
+	for _, f := range entry.AdditionalFiles {
+		if q := quantizationFromFilename(f.Filename); q != "" {
+			return q
+		}
+	}
+	return ""
+}
+
+// overrideModelParameter digs `overrides.parameters.model` out of an entry.
+//
+// Overrides are free-form YAML decoded into map[string]any, so every level has
+// to be type-asserted; a gallery author who writes a list or a scalar where a
+// map belongs gets "" here rather than a panic in the listing handler.
+func overrideModelParameter(entry *GalleryModel) string {
+	parameters, ok := entry.Overrides["parameters"].(map[string]any)
+	if !ok {
+		return ""
+	}
+	model, ok := parameters["model"].(string)
+	if !ok {
+		return ""
+	}
+	return model
+}

core/gallery/variant_quantization_internal_test.go

@@ -0,0 +1,126 @@
+package gallery
+
+import (
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+var _ = Describe("quantizationFromFilename", func() {
+	DescribeTable("reads the weight format out of a model filename",
+		func(filename, expected string) {
+			Expect(quantizationFromFilename(filename)).To(Equal(expected))
+		},
+		Entry("a plain legacy quant", "Ternary-Bonsai-27B-Q2_0.gguf", "Q2_0"),
+		Entry("a packed quant", "Ternary-Bonsai-27B-PQ2_0.gguf", "PQ2_0"),
+		// The pair the whole feature exists for: two builds of one model whose
+		// names differ only here, and whose sizes are close enough that a user
+		// cannot tell them apart from the size column.
+		Entry("a group-size qualified quant", "Ternary-Bonsai-27B-Q2_g64.gguf", "Q2_G64"),
+		Entry("a multi-qualifier quant", "Ternary-Bonsai-27B-Q2_0_g128.gguf", "Q2_0_G128"),
+		Entry("a k-quant", "Qwen3-8B-Q4_K_M.gguf", "Q4_K_M"),
+		Entry("an i-quant", "Qwen3-8B-IQ2_XXS.gguf", "IQ2_XXS"),
+		Entry("a float format", "Qwen3-8B-f16.gguf", "F16"),
+		Entry("bfloat16", "Qwen3-8B-bf16.safetensors", "BF16"),
+		Entry("an MLX bit count", "Qwen3-8B-4bit.safetensors", "4BIT"),
+		// A real gallery shape: the format sits mid-name with serving-feature
+		// and repo-suffix segments after it, so the scan cannot simply take
+		// the last segment.
+		Entry("a vendor 4-bit float behind later segments", "Qwen3.6-27B-NVFP4-MTP-GGUF.gguf", "NVFP4"),
+		Entry("the other vendor 4-bit float", "Qwen3-8B-MXFP4.gguf", "MXFP4"),
+		// The gemma QAT authoring style: the format is run into the model name
+		// with an underscore rather than set off by a dash.
+		Entry("a format run into the model name", "gemma-4-E2B_q4_0-it.gguf", "Q4_0"),
+		Entry("a format run in at the end", "model_q8_0.gguf", "Q8_0"),
+		Entry("an integer width", "Qwen3-8B-int8.safetensors", "INT8"),
+		Entry("a dot-separated qualifier", "qwen3-8b.Q8_0.gguf", "Q8_0"),
+		// A repo path is routinely prefixed to the filename in the gallery, and
+		// its own segments must not be mistaken for this file's format.
+		Entry("a path prefix", "bonsai/models/Q4-repo/Model-Q8_0.gguf", "Q8_0"),
+	)
+
+	DescribeTable("reports nothing when the name declares no format",
+		func(filename string) {
+			Expect(quantizationFromFilename(filename)).To(BeEmpty())
+		},
+		Entry("an empty name", ""),
+		// The degrade case the UI has to render: a backend served from a
+		// directory of weights names no format anywhere.
+		Entry("a bare model name", "Qwen3-8B.safetensors"),
+		Entry("a directory", "models/qwen3-8b/"),
+		// A parameter count is quant-shaped to a careless matcher: it is a
+		// digit run next to a letter, and reporting "27B" as a quantization
+		// would be worse than reporting nothing.
+		Entry("a parameter count", "Ternary-Bonsai-27B.gguf"),
+		Entry("an extension alone", "model.gguf"),
+	)
+
+	It("prefers a whole segment over a tail further right in the name", func() {
+		// The two passes exist for this: a precise, dash-delimited format must
+		// never lose to a looser mid-segment match that happens to sit later.
+		Expect(quantizationFromFilename("Model-Q4_K_M-repo_8bit.gguf")).To(Equal("Q4_K_M"))
+	})
+
+	It("takes the longest tail, not the shortest", func() {
+		// A shortest-first walk over `e2b_q4_0` reaches `0` before `q4_0`, and
+		// `0` is not a weight format.
+		Expect(quantizationFromFilename("gemma_q4_0.gguf")).To(Equal("Q4_0"))
+	})
+
+	It("does not split on the separator inside a quant token", func() {
+		// Splitting on `_` as well as `-` would truncate every k-quant to its
+		// family and report a Q4_K_M build as plain "Q4", which names a
+		// different format that the entry does not ship.
+		Expect(quantizationFromFilename("Model-Q4_K_S.gguf")).To(Equal("Q4_K_S"))
+	})
+})
+
+var _ = Describe("quantizationOfEntry", func() {
+	It("prefers the served model parameter over the file list", func() {
+		// The Bonsai shape: a Q2_0 language model shipped alongside a Q8_0
+		// vision tower. Reading the file list first reports the mmproj's
+		// format, which describes a companion the user is not choosing.
+		entry := &GalleryModel{
+			Overrides: map[string]any{
+				"parameters": map[string]any{"model": "bonsai/models/Bonsai-27B-Q2_0.gguf"},
+			},
+		}
+		entry.AdditionalFiles = []File{
+			{Filename: "bonsai/mmproj/Bonsai-27B-mmproj-Q8_0.gguf"},
+			{Filename: "bonsai/models/Bonsai-27B-Q2_0.gguf"},
+		}
+
+		Expect(quantizationOfEntry(entry)).To(Equal("Q2_0"))
+	})
+
+	It("falls back to the file list when no model parameter is set", func() {
+		entry := &GalleryModel{}
+		entry.AdditionalFiles = []File{{Filename: "models/Qwen3-8B-Q6_K.gguf"}}
+
+		Expect(quantizationOfEntry(entry)).To(Equal("Q6_K"))
+	})
+
+	It("reports nothing for a nil entry", func() {
+		Expect(quantizationOfEntry(nil)).To(BeEmpty())
+	})
+
+	It("reports nothing when the entry ships no recognisable format", func() {
+		entry := &GalleryModel{}
+		entry.AdditionalFiles = []File{{Filename: "models/Qwen3-8B/model.safetensors"}}
+
+		Expect(quantizationOfEntry(entry)).To(BeEmpty())
+	})
+
+	DescribeTable("survives overrides that are not shaped like a parameter map",
+		func(overrides map[string]any) {
+			entry := &GalleryModel{Overrides: overrides}
+			// A gallery author's typo must degrade to "unknown format", never
+			// panic inside the listing handler.
+			Expect(quantizationOfEntry(entry)).To(BeEmpty())
+		},
+		Entry("no overrides at all", nil),
+		Entry("parameters is a scalar", map[string]any{"parameters": "Q8_0"}),
+		Entry("parameters is a list", map[string]any{"parameters": []any{"model"}}),
+		Entry("model is not a string", map[string]any{"parameters": map[string]any{"model": 42}}),
+		Entry("model is absent", map[string]any{"parameters": map[string]any{"context_size": 8192}}),
+	)
+})

core/gallery/variants_install_test.go

@@ -0,0 +1,757 @@
+package gallery_test
+
+import (
+	"context"
+	"fmt"
+	"os"
+	"path/filepath"
+	"runtime"
+
+	"dario.cat/mergo"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+	"gopkg.in/yaml.v3"
+
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/gallery"
+	"github.com/mudler/LocalAI/pkg/system"
+)
+
+var _ = Describe("GalleryModel variant declarations", func() {
+	It("declares no variants when the key is absent", func() {
+		m := gallery.GalleryModel{}
+		m.Name = "plain"
+		Expect(m.HasVariants()).To(BeFalse())
+	})
+
+	It("declares variants when the key is present", func() {
+		m := gallery.GalleryModel{Variants: []gallery.Variant{{Model: "x"}}}
+		Expect(m.HasVariants()).To(BeTrue())
+	})
+
+	It("parses a variant list", func() {
+		var m gallery.GalleryModel
+		err := yaml.Unmarshal([]byte(`
+name: qwen3.6-27b
+url: "github:example/repo/qwen3.6-27b.yaml@master"
+variants:
+  - model: qwen3.6-27b-mlx-8bit
+  - model: qwen3.6-27b-gguf-q8
+`), &m)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(m.Name).To(Equal("qwen3.6-27b"))
+		Expect(m.URL).To(Equal("github:example/repo/qwen3.6-27b.yaml@master"))
+		Expect(m.HasVariants()).To(BeTrue())
+		Expect(m.Variants).To(HaveLen(2))
+		// A variant is nothing but a name, which is the whole of the authoring
+		// surface.
+		Expect(m.Variants[0].Model).To(Equal("qwen3.6-27b-mlx-8bit"))
+		Expect(m.Variants[1].Model).To(Equal("qwen3.6-27b-gguf-q8"))
+	})
+})
+
+var _ = Describe("ResolveVariant", func() {
+	gib := func(n uint64) uint64 { return n * 1024 * 1024 * 1024 }
+
+	// runsEverything keeps these specs about resolution rather than about the
+	// hardware of whatever machine runs them.
+	runsEverything := func(string) bool { return true }
+
+	newModel := func(name, url, description, icon string) *gallery.GalleryModel {
+		m := &gallery.GalleryModel{}
+		m.Name = name
+		m.URL = url
+		m.Description = description
+		m.Icon = icon
+		return m
+	}
+
+	var models []*gallery.GalleryModel
+	var base *gallery.GalleryModel
+
+	BeforeEach(func() {
+		upgrade := newModel("qwen3-8b-vllm-awq", "file://vllm.yaml", "AWQ variant", "vllm.png")
+		upgrade.Backend = "vllm"
+		// The base is an ordinary, complete entry that happens to list an
+		// alternative build of itself, which is the whole point of the design.
+		base = newModel("qwen3-8b-gguf-q4", "file://gguf.yaml", "Qwen3 8B Q4", "qwen.png")
+		base.Backend = "llama-cpp"
+		base.Tags = []string{"llm"}
+		base.Variants = []gallery.Variant{{Model: "qwen3-8b-vllm-awq"}}
+		models = []*gallery.GalleryModel{upgrade, base}
+	})
+
+	// The AWQ build measures 20GiB, so the specs below vary only the host and
+	// read off which payload resolution lands on. The probe is injected rather
+	// than performed, so nothing here reaches the network.
+	env := func(memory uint64) gallery.ResolveEnv {
+		return gallery.ResolveEnv{
+			AvailableMemory:   memory,
+			BackendCompatible: runsEverything,
+			ProbeMemory: func(target *gallery.GalleryModel) uint64 {
+				if target.Name == "qwen3-8b-vllm-awq" {
+					return 20 * 1024 * 1024 * 1024
+				}
+				return 0
+			},
+		}
+	}
+
+	It("installs a fitting variant's payload under the entry's name", func() {
+		resolved, variant, err := gallery.ResolveVariant(models, base, env(gib(24)), "")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(variant.Model).To(Equal("qwen3-8b-vllm-awq"))
+		Expect(resolved.Name).To(Equal("qwen3-8b-gguf-q4"))
+		Expect(resolved.URL).To(Equal("file://vllm.yaml"))
+	})
+
+	It("carries the referenced entry's backend into the compatibility check", func() {
+		// The variant itself says nothing about hardware, so the backend can
+		// only come from the entry it names. Rejecting exactly that backend must
+		// therefore be enough to rule the variant out.
+		resolved, variant, err := gallery.ResolveVariant(models, base, gallery.ResolveEnv{
+			AvailableMemory:   gib(64),
+			BackendCompatible: func(backend string) bool { return backend != "vllm" },
+		}, "")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(variant.Model).To(Equal("qwen3-8b-gguf-q4"))
+		Expect(resolved.URL).To(Equal("file://gguf.yaml"))
+	})
+
+	It("carries the referenced entry's tags into the serving feature ranking", func() {
+		// The tags that rank a variant are the REFERENCED entry's, not the
+		// declaring entry's: the feature belongs to the build that would be
+		// installed. Only the GGUF build here is tagged with a feature, and it
+		// is deliberately the smaller of the two, so it can only win if the
+		// lookup read a tag off the entry rather than off the variant name or
+		// off the parent.
+		speculative := gallery.GalleryModel{}
+		speculative.Name = "qwen3-8b-gguf-turbo"
+		speculative.URL = "file://turbo.yaml"
+		speculative.Backend = "llama-cpp"
+		speculative.Tags = []string{"llm", "mtp"}
+
+		parent := gallery.GalleryModel{}
+		parent.Name = "qwen3-8b-gguf-q4"
+		parent.URL = "file://gguf.yaml"
+		parent.Backend = "llama-cpp"
+		parent.Variants = []gallery.Variant{{Model: "qwen3-8b-vllm-awq"}, {Model: "qwen3-8b-gguf-turbo"}}
+
+		catalog := []*gallery.GalleryModel{models[0], &speculative, &parent}
+
+		resolved, variant, err := gallery.ResolveVariant(catalog, &parent, gallery.ResolveEnv{
+			AvailableMemory:   gib(64),
+			BackendCompatible: runsEverything,
+			// Only llama-cpp is ranked, so the vLLM build cannot win on engine
+			// and the contest comes down to the feature axis.
+			EnginePreference:         []string{"llama-cpp"},
+			ServingFeaturePreference: []string{"dflash", "mtp"},
+			ProbeMemory: func(target *gallery.GalleryModel) uint64 {
+				if target.Name == "qwen3-8b-gguf-turbo" {
+					return gib(10)
+				}
+				return gib(20)
+			},
+		}, "")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(variant.Model).To(Equal("qwen3-8b-gguf-turbo"))
+		Expect(resolved.URL).To(Equal("file://turbo.yaml"))
+	})
+
+	It("falls back to the entry's own payload when no variant fits", func() {
+		resolved, variant, err := gallery.ResolveVariant(models, base, env(gib(8)), "")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(variant.Model).To(Equal("qwen3-8b-gguf-q4"))
+		Expect(resolved.URL).To(Equal("file://gguf.yaml"))
+	})
+
+	It("installs the entry on a host with no memory budget at all", func() {
+		// The base carries no floor of its own precisely so this can never
+		// refuse; an unreadable host reports 0 and must still get the entry.
+		resolved, variant, err := gallery.ResolveVariant(models, base, env(0), "")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(variant.Model).To(Equal("qwen3-8b-gguf-q4"))
+		Expect(resolved.URL).To(Equal("file://gguf.yaml"))
+	})
+
+	It("strips the variant list from the resolved entry", func() {
+		// A resolved entry is a concrete install target. Leaving the list on it
+		// would let a second selection pass fire on an already-resolved entry.
+		resolved, _, err := gallery.ResolveVariant(models, base, env(gib(8)), "")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(resolved.HasVariants()).To(BeFalse())
+	})
+
+	Describe("the live size probe", func() {
+		// The probe is injected rather than performed, so these specs pin an
+		// exact size, or an exact failure, without reaching the network.
+		probing := func(memory uint64, sizes map[string]uint64) gallery.ResolveEnv {
+			e := env(memory)
+			e.ProbeMemory = func(target *gallery.GalleryModel) uint64 { return sizes[target.Name] }
+			return e
+		}
+
+		It("selects a variant the probe shows to fit", func() {
+			resolved, variant, err := gallery.ResolveVariant(models, base,
+				probing(gib(24), map[string]uint64{"qwen3-8b-vllm-awq": gib(20)}), "")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(variant.Model).To(Equal("qwen3-8b-vllm-awq"))
+			Expect(resolved.URL).To(Equal("file://vllm.yaml"))
+		})
+
+		It("rejects a variant the probe shows to be too large", func() {
+			// Same variant, same host, only the probed size differs, so nothing
+			// but the probe can account for the different answer.
+			resolved, variant, err := gallery.ResolveVariant(models, base,
+				probing(gib(24), map[string]uint64{"qwen3-8b-vllm-awq": gib(80)}), "")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(variant.Model).To(Equal("qwen3-8b-gguf-q4"))
+			Expect(resolved.URL).To(Equal("file://gguf.yaml"))
+		})
+
+		It("completes the install on the entry's own payload when no probe answers", func() {
+			// A failed probe reports 0. That is an unknown, so nothing is filtered
+			// out and above all the resolution does not fail: a network hiccup
+			// must never be able to break an install. It must not be able to
+			// change what gets installed either, so an unmeasurable variant does
+			// not displace the payload the entry itself ships.
+			resolved, variant, err := gallery.ResolveVariant(models, base,
+				probing(gib(24), map[string]uint64{}), "")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(variant.Model).To(Equal("qwen3-8b-gguf-q4"))
+			Expect(resolved.URL).To(Equal("file://gguf.yaml"))
+		})
+
+		It("still installs the base when every probe fails and nothing else survives", func() {
+			// The one variant this host could otherwise take is ruled out by its
+			// backend and no probe answers, so selection has to terminate on the
+			// base rather than on an error.
+			e := probing(gib(24), map[string]uint64{})
+			e.BackendCompatible = func(backend string) bool { return backend != "vllm" }
+
+			resolved, variant, err := gallery.ResolveVariant(models, base, e, "")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(variant.Model).To(Equal("qwen3-8b-gguf-q4"))
+			Expect(resolved.URL).To(Equal("file://gguf.yaml"))
+		})
+	})
+
+	It("presents the entry's metadata, not the variant's", func() {
+		resolved, _, err := gallery.ResolveVariant(models, base, env(gib(24)), "")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(resolved.Description).To(Equal("Qwen3 8B Q4"))
+		Expect(resolved.Icon).To(Equal("qwen.png"))
+		Expect(resolved.Tags).To(ConsistOf("llm"))
+	})
+
+	It("honors a pin naming the entry itself", func() {
+		// The entry is the last resort of its own variant list, so its own name
+		// has to be a usable pin: it is how an operator declines an upgrade
+		// their hardware would otherwise take.
+		resolved, variant, err := gallery.ResolveVariant(models, base, env(gib(24)), "qwen3-8b-gguf-q4")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(variant.Model).To(Equal("qwen3-8b-gguf-q4"))
+		Expect(resolved.Name).To(Equal("qwen3-8b-gguf-q4"))
+		Expect(resolved.URL).To(Equal("file://gguf.yaml"))
+	})
+
+	It("honors a pin the hardware does not satisfy", func() {
+		resolved, variant, err := gallery.ResolveVariant(models, base, env(gib(2)), "qwen3-8b-vllm-awq")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(variant.Model).To(Equal("qwen3-8b-vllm-awq"))
+		Expect(resolved.URL).To(Equal("file://vllm.yaml"))
+	})
+
+	It("detaches the resolved entry from the gallery's own maps and slices", func() {
+		models[0].Overrides = map[string]any{"f16": true}
+		models[0].AdditionalFiles = []gallery.File{{Filename: "a.bin"}}
+
+		resolved, _, err := gallery.ResolveVariant(models, base, env(gib(24)), "")
+		Expect(err).ToNot(HaveOccurred())
+
+		// The install path merges the caller's request overrides into this map in
+		// place, so aliasing the gallery's map would write one caller's request
+		// into the shared catalog and leak it into every later install.
+		resolved.Overrides["f16"] = false
+		resolved.Overrides["threads"] = 4
+		Expect(models[0].Overrides).To(HaveKeyWithValue("f16", true))
+		Expect(models[0].Overrides).ToNot(HaveKey("threads"))
+
+		resolved.AdditionalFiles[0].Filename = "mutated.bin"
+		Expect(models[0].AdditionalFiles[0].Filename).To(Equal("a.bin"))
+
+		resolved.Tags = append(resolved.Tags, "extra")
+		Expect(base.Tags).To(ConsistOf("llm"))
+	})
+
+	It("detaches the resolved entry even when it resolves to the entry itself", func() {
+		// Resolving to the base returns a copy of the very entry the gallery
+		// holds, which is the case most likely to alias it.
+		base.Overrides = map[string]any{"parameters": map[string]any{"model": "q4.gguf"}}
+
+		resolved, _, err := gallery.ResolveVariant(models, base, env(gib(8)), "")
+		Expect(err).ToNot(HaveOccurred())
+
+		resolved.Overrides["parameters"].(map[string]any)["model"] = "callers-choice.gguf"
+		Expect(base.Overrides["parameters"]).To(HaveKeyWithValue("model", "q4.gguf"))
+	})
+
+	It("detaches nested override maps from the gallery's own entry", func() {
+		models[0].Overrides = map[string]any{
+			"parameters": map[string]any{"model": "real-variant.gguf"},
+			"stopwords":  []any{"</s>"},
+		}
+
+		resolved, _, err := gallery.ResolveVariant(models, base, env(gib(24)), "")
+		Expect(err).ToNot(HaveOccurred())
+
+		// Cloning only the top level would leave this inner map shared with the
+		// gallery entry, so writing through the resolved copy would rewrite the
+		// catalog's own payload.
+		resolved.Overrides["parameters"].(map[string]any)["model"] = "callers-choice.gguf"
+		resolved.Overrides["stopwords"].([]any)[0] = "<|im_end|>"
+
+		Expect(models[0].Overrides["parameters"]).To(HaveKeyWithValue("model", "real-variant.gguf"))
+		Expect(models[0].Overrides["stopwords"]).To(Equal([]any{"</s>"}))
+	})
+
+	It("does not write the caller's overrides back into the gallery entry", func() {
+		models[0].Overrides = map[string]any{"parameters": map[string]any{"model": "real-variant.gguf"}}
+
+		resolved, _, err := gallery.ResolveVariant(models, base, env(gib(24)), "")
+		Expect(err).ToNot(HaveOccurred())
+
+		// This is exactly what the install path does with the caller's request
+		// overrides, and mergo recurses into nested maps and overwrites them in
+		// place. Asserting against the in-memory catalog is the only way to
+		// observe the leak: re-reading the gallery from disk re-unmarshals fresh
+		// maps and would pass whether or not the resolved entry was detached.
+		requestOverrides := map[string]any{"parameters": map[string]any{"model": "callers-choice.gguf"}}
+		Expect(mergo.Merge(&resolved.Overrides, requestOverrides, mergo.WithOverride)).To(Succeed())
+
+		Expect(resolved.Overrides["parameters"]).To(HaveKeyWithValue("model", "callers-choice.gguf"))
+		Expect(models[0].Overrides["parameters"]).To(HaveKeyWithValue("model", "real-variant.gguf"))
+	})
+
+	It("errors when a variant references a missing entry", func() {
+		base.Variants = []gallery.Variant{{Model: "does-not-exist"}}
+		_, _, err := gallery.ResolveVariant(models, base, env(gib(8)), "")
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("does-not-exist"))
+	})
+
+	It("refuses a variant that declares variants of its own", func() {
+		nested := newModel("nested", "file://nested.yaml", "", "")
+		nested.Variants = []gallery.Variant{{Model: "qwen3-8b-vllm-awq"}}
+		models = append(models, nested)
+		base.Variants = []gallery.Variant{{Model: "nested"}}
+		_, _, err := gallery.ResolveVariant(models, base, env(gib(8)), "")
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("nested"))
+	})
+
+	It("surfaces a bad pin", func() {
+		_, _, err := gallery.ResolveVariant(models, base, env(gib(24)), "nope")
+		Expect(err).To(MatchError(gallery.ErrPinNotFound))
+	})
+})
+
+var _ = Describe("InstallModelFromGallery with variant entries", func() {
+	var tempdir string
+	var galleries []config.Gallery
+	var systemState *system.SystemState
+	// galleryRevision keeps every gallery this suite writes distinct from every
+	// other, so the process-wide gallery cache cannot serve one spec's catalog
+	// to another.
+	galleryRevision := 0
+
+	// Every entry is described with an inline config_file rather than a URL so
+	// the whole install runs off the local filesystem with no network access.
+	//
+	// Each call writes a fresh path and a fresh gallery name because the gallery
+	// listing is cached on the name and URL pair. A spec that edits its gallery
+	// mid-flight would otherwise keep reading the version it started with, and
+	// would silently prove nothing.
+	newGallery := func(entries ...gallery.GalleryModel) {
+		out, err := yaml.Marshal(entries)
+		Expect(err).ToNot(HaveOccurred())
+		name := fmt.Sprintf("test-%d", galleryRevision)
+		galleryRevision++
+		galleryPath := filepath.Join(tempdir, name+".yaml")
+		Expect(os.WriteFile(galleryPath, out, 0600)).To(Succeed())
+
+		galleries = []config.Gallery{{Name: name, URL: "file://" + galleryPath}}
+	}
+
+	entry := func(name, backend string) gallery.GalleryModel {
+		m := gallery.GalleryModel{ConfigFile: map[string]any{"backend": backend}}
+		m.Name = name
+		m.Description = "entry " + name
+		return m
+	}
+
+	// sizedEntry declares its footprint through the entry's own size:, which is
+	// the only way an author influences a variant's measured size. The estimator
+	// prefers a declared size over its own guesswork and parses it locally, so
+	// these specs pin an exact figure without touching the network.
+	sizedEntry := func(name, backend, size string) gallery.GalleryModel {
+		m := entry(name, backend)
+		m.Size = size
+		return m
+	}
+
+	// urlEntry describes an entry through a url rather than an inline
+	// config_file. The distinction matters for the recorded name: the url branch
+	// reads a name out of the fetched config, and that name is the referenced
+	// entry's own, so only the entry-name overlay can keep the record under the
+	// name the user asked for. The inline config_file branch seeds the name from
+	// the already-renamed resolved entry and so cannot observe the overlay.
+	urlEntry := func(name, backend, size string) gallery.GalleryModel {
+		payload := gallery.ModelConfig{
+			Name:        name,
+			Description: "entry " + name,
+			ConfigFile:  "backend: " + backend + "\n",
+		}
+		out, err := yaml.Marshal(payload)
+		Expect(err).ToNot(HaveOccurred())
+		payloadPath := filepath.Join(tempdir, "payload-"+name+".yaml")
+		Expect(os.WriteFile(payloadPath, out, 0600)).To(Succeed())
+
+		m := gallery.GalleryModel{}
+		m.Name = name
+		m.Description = "entry " + name
+		m.URL = "file://" + payloadPath
+		m.Size = size
+		return m
+	}
+
+	// withVariants attaches alternative builds to an otherwise ordinary entry.
+	// Where a spec needs a definite size it declares it on the referenced entry
+	// with sizedEntry, in absolute terms rather than relative to the host, so
+	// these specs assert on selection rather than on whatever memory the machine
+	// running them happens to have.
+	withVariants := func(m gallery.GalleryModel, variants ...gallery.Variant) gallery.GalleryModel {
+		m.Variants = variants
+		return m
+	}
+
+	install := func(name string, req gallery.GalleryModel, options ...gallery.InstallOption) error {
+		return gallery.InstallModelFromGallery(
+			context.TODO(), galleries, []config.Gallery{}, systemState, nil,
+			name, req, func(string, string, string, float64) {}, false, false, false, options...)
+	}
+
+	installedBackend := func(name string) string {
+		dat, err := os.ReadFile(filepath.Join(tempdir, name+".yaml"))
+		Expect(err).ToNot(HaveOccurred())
+		content := map[string]any{}
+		Expect(yaml.Unmarshal(dat, &content)).To(Succeed())
+		return content["backend"].(string)
+	}
+
+	BeforeEach(func() {
+		var err error
+		tempdir, err = os.MkdirTemp("", "variant-install")
+		Expect(err).ToNot(HaveOccurred())
+		DeferCleanup(func() { Expect(os.RemoveAll(tempdir)).To(Succeed()) })
+
+		systemState, err = system.GetSystemState(system.WithModelPath(tempdir))
+		Expect(err).ToNot(HaveOccurred())
+	})
+
+	It("installs the entry's own payload when no variant fits the host", func() {
+		newGallery(
+			withVariants(entry("qwen3-8b-q4", "base-backend"),
+				gallery.Variant{Model: "qwen3-8b-q8"}),
+			sizedEntry("qwen3-8b-q8", "upgrade-backend", "10000GiB"),
+		)
+
+		// No machine clears 10000GiB, so this asserts the base is the last
+		// resort and that missing every variant is not an error.
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{})).To(Succeed())
+		Expect(installedBackend("qwen3-8b-q4")).To(Equal("base-backend"))
+	})
+
+	It("installs a fitting variant's payload under the entry's own name", func() {
+		newGallery(
+			withVariants(entry("qwen3-8b-q4", "base-backend"),
+				gallery.Variant{Model: "qwen3-8b-q8"}),
+			sizedEntry("qwen3-8b-q8", "upgrade-backend", "16MiB"),
+		)
+
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{})).To(Succeed())
+		Expect(installedBackend("qwen3-8b-q4")).To(Equal("upgrade-backend"))
+		_, err := os.Stat(filepath.Join(tempdir, "qwen3-8b-q8.yaml"))
+		Expect(os.IsNotExist(err)).To(BeTrue(), "the variant must not be installed under its own name")
+	})
+
+	It("completes the install when a variant's size cannot be determined", func() {
+		// This runs the REAL probe against an entry that declares no size and no
+		// weight files, which is exactly the shape a probe cannot answer for.
+		// It must yield an unknown, not an error: the variant survives the
+		// filter and the install completes, on the variant or on the base, but
+		// never on a failure. Nothing here touches the network.
+		newGallery(
+			withVariants(entry("qwen3-8b-q4", "base-backend"),
+				gallery.Variant{Model: "qwen3-8b-q8"}),
+			entry("qwen3-8b-q8", "upgrade-backend"),
+		)
+
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{})).To(Succeed())
+		Expect(installedBackend("qwen3-8b-q4")).To(BeElementOf("base-backend", "upgrade-backend"))
+	})
+
+	It("installs the largest fitting variant, not the first authored", func() {
+		// Authored smallest-first with all three within reach, so first-match
+		// would take the small one.
+		newGallery(
+			withVariants(entry("qwen3-8b-q4", "base-backend"),
+				gallery.Variant{Model: "qwen3-8b-small"},
+				gallery.Variant{Model: "qwen3-8b-large"},
+			),
+			sizedEntry("qwen3-8b-small", "small-backend", "16MiB"),
+			sizedEntry("qwen3-8b-large", "large-backend", "256MiB"),
+		)
+
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{})).To(Succeed())
+		Expect(installedBackend("qwen3-8b-q4")).To(Equal("large-backend"))
+	})
+
+	It("never installs a variant whose backend cannot run on this host", func() {
+		if runtime.GOOS == "darwin" {
+			Skip("mlx is a compatible backend on darwin, so it proves nothing here")
+		}
+		// The mlx entry is the only variant and fits trivially, so the real
+		// SystemState.IsBackendCompatible rejecting mlx on a non-darwin host is
+		// the only thing that can send this to the base.
+		newGallery(
+			withVariants(entry("qwen3-8b-q4", "base-backend"),
+				gallery.Variant{Model: "qwen3-8b-mlx"}),
+			entry("qwen3-8b-mlx", "mlx"),
+		)
+
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{})).To(Succeed())
+		Expect(installedBackend("qwen3-8b-q4")).To(Equal("base-backend"))
+	})
+
+	It("round-trips the resolution record to disk under the entry's name", func() {
+		// The variant is described by url so its payload carries its own name.
+		// Without the entry-name overlay the record persists as "qwen3-8b-q8",
+		// and the stable name the entry exists to provide is lost the moment
+		// anything reads the record back.
+		// The declared sizes make the upgrade the largest build that fits, so
+		// resolution lands on the variant and the record it writes is the one
+		// under test here.
+		newGallery(
+			withVariants(urlEntry("qwen3-8b-q4", "base-backend", "16MiB"),
+				gallery.Variant{Model: "qwen3-8b-q8"}),
+			urlEntry("qwen3-8b-q8", "upgrade-backend", "256MiB"),
+		)
+
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{})).To(Succeed())
+
+		record, err := gallery.GetLocalModelConfiguration(tempdir, "qwen3-8b-q4")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(record.EntryName).To(Equal("qwen3-8b-q4"))
+		Expect(record.ResolvedVariant).To(Equal("qwen3-8b-q8"))
+		Expect(record.PinnedVariant).To(BeEmpty())
+		Expect(record.Name).To(Equal("qwen3-8b-q4"))
+		Expect(record.Description).To(Equal("entry qwen3-8b-q4"))
+		Expect(installedBackend("qwen3-8b-q4")).To(Equal("upgrade-backend"))
+	})
+
+	It("refuses a variant name the entry does not declare, naming what was asked for", func() {
+		newGallery(
+			withVariants(entry("qwen3-8b-q4", "base-backend"),
+				gallery.Variant{Model: "qwen3-8b-q8"}),
+			entry("qwen3-8b-q8", "upgrade-backend"),
+		)
+
+		err := install("qwen3-8b-q4", gallery.GalleryModel{}, gallery.WithVariant("qwen3-8b-q6"))
+
+		// Auto-selecting instead would report success while silently
+		// installing something other than what the caller asked for, and a
+		// typo'd variant name would be indistinguishable from a working one.
+		Expect(err).To(MatchError(gallery.ErrPinNotFound))
+		Expect(err.Error()).To(ContainSubstring("qwen3-8b-q6"))
+		Expect(filepath.Join(tempdir, "qwen3-8b-q4.yaml")).ToNot(BeAnExistingFile())
+	})
+
+	It("refuses a variant name when the entry declares no variants at all", func() {
+		// The entry is installable and selection never runs for it, so without
+		// an explicit refusal the requested variant would be dropped on the
+		// floor and the install would look like it honored the choice.
+		newGallery(entry("qwen3-8b-q4", "base-backend"))
+
+		err := install("qwen3-8b-q4", gallery.GalleryModel{}, gallery.WithVariant("qwen3-8b-q8"))
+
+		Expect(err).To(MatchError(gallery.ErrPinNotFound))
+		Expect(err.Error()).To(ContainSubstring("qwen3-8b-q8"))
+		Expect(filepath.Join(tempdir, "qwen3-8b-q4.yaml")).ToNot(BeAnExistingFile())
+	})
+
+	It("records a pin and honors it on a plain reinstall", func() {
+		// The upgrade declares a size and the base does not, so auto-selection
+		// genuinely prefers the upgrade here. Without that the second install
+		// below would land on the base whether or not the pin was recalled, and
+		// the spec would prove nothing.
+		newGallery(
+			withVariants(entry("qwen3-8b-q4", "base-backend"),
+				gallery.Variant{Model: "qwen3-8b-q8"}),
+			sizedEntry("qwen3-8b-q8", "upgrade-backend", "16MiB"),
+		)
+
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{}, gallery.WithVariant("qwen3-8b-q4"))).To(Succeed())
+
+		record, err := gallery.GetLocalModelConfiguration(tempdir, "qwen3-8b-q4")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(record.PinnedVariant).To(Equal("qwen3-8b-q4"))
+		Expect(record.ResolvedVariant).To(Equal("qwen3-8b-q4"))
+		Expect(installedBackend("qwen3-8b-q4")).To(Equal("base-backend"))
+
+		// No WithVariant this time: hardware selection would take the upgrade,
+		// so only the recalled pin can keep this on the base payload.
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{})).To(Succeed())
+		Expect(installedBackend("qwen3-8b-q4")).To(Equal("base-backend"))
+
+		record, err = gallery.GetLocalModelConfiguration(tempdir, "qwen3-8b-q4")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(record.PinnedVariant).To(Equal("qwen3-8b-q4"))
+	})
+
+	It("honors a pin recorded under a custom install name", func() {
+		// Sized as above so auto-selection would take the upgrade, which is what
+		// makes the recall observable in the second install.
+		newGallery(
+			withVariants(entry("qwen3-8b-q4", "base-backend"),
+				gallery.Variant{Model: "qwen3-8b-q8"}),
+			sizedEntry("qwen3-8b-q8", "upgrade-backend", "16MiB"),
+		)
+
+		req := gallery.GalleryModel{}
+		req.Name = "prod-llm"
+
+		Expect(install("qwen3-8b-q4", req, gallery.WithVariant("qwen3-8b-q4"))).To(Succeed())
+
+		// The pin is written under the installed name, so the recall must read it
+		// back under that name too and not under the gallery entry's name.
+		record, err := gallery.GetLocalModelConfiguration(tempdir, "prod-llm")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(record.PinnedVariant).To(Equal("qwen3-8b-q4"))
+		Expect(record.EntryName).To(Equal("qwen3-8b-q4"))
+		Expect(installedBackend("prod-llm")).To(Equal("base-backend"))
+
+		Expect(install("qwen3-8b-q4", req)).To(Succeed())
+		Expect(installedBackend("prod-llm")).To(Equal("base-backend"))
+	})
+
+	It("re-selects automatically when a recorded pin no longer exists in the gallery", func() {
+		// A pin is recorded, then the gallery is edited to rename the build it
+		// named. Without dropping the stale pin every later reinstall and every
+		// upgrade of this model fails forever, and the only remedy is deleting a
+		// dotfile the user has never heard of.
+		newGallery(
+			withVariants(entry("qwen3-8b-q4", "base-backend"),
+				gallery.Variant{Model: "qwen3-8b-q8"}),
+			sizedEntry("qwen3-8b-q8", "upgrade-backend", "16MiB"),
+		)
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{}, gallery.WithVariant("qwen3-8b-q8"))).To(Succeed())
+
+		record, err := gallery.GetLocalModelConfiguration(tempdir, "qwen3-8b-q4")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(record.PinnedVariant).To(Equal("qwen3-8b-q8"))
+
+		// The gallery edit: the same build under a new name.
+		newGallery(
+			withVariants(entry("qwen3-8b-q4", "base-backend"),
+				gallery.Variant{Model: "qwen3-8b-q8-v2"}),
+			sizedEntry("qwen3-8b-q8-v2", "renamed-backend", "16MiB"),
+		)
+
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{})).To(Succeed())
+		// Auto-selection ran: the renamed build is the largest that fits.
+		Expect(installedBackend("qwen3-8b-q4")).To(Equal("renamed-backend"))
+
+		// The stale pin is cleared rather than carried forward, so the next
+		// install does not have to rediscover that it is unusable.
+		record, err = gallery.GetLocalModelConfiguration(tempdir, "qwen3-8b-q4")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(record.PinnedVariant).To(BeEmpty())
+		Expect(record.ResolvedVariant).To(Equal("qwen3-8b-q8-v2"))
+	})
+
+	It("still refuses a variant the caller names on this request, recorded pin or not", func() {
+		// The caller-supplied pin and the recalled one must fail differently.
+		// This one is a request that cannot be honored, so reporting success
+		// for it would make a typo indistinguishable from a working choice.
+		newGallery(
+			withVariants(entry("qwen3-8b-q4", "base-backend"),
+				gallery.Variant{Model: "qwen3-8b-q8"}),
+			sizedEntry("qwen3-8b-q8", "upgrade-backend", "16MiB"),
+		)
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{}, gallery.WithVariant("qwen3-8b-q8"))).To(Succeed())
+
+		err := install("qwen3-8b-q4", gallery.GalleryModel{}, gallery.WithVariant("qwen3-8b-q6"))
+		Expect(err).To(MatchError(gallery.ErrPinNotFound))
+		Expect(err.Error()).To(ContainSubstring("qwen3-8b-q6"))
+	})
+
+	It("writes each declared url only once into the persisted gallery file", func() {
+		base := withVariants(entry("qwen3-8b-q4", "base-backend"),
+			gallery.Variant{Model: "qwen3-8b-q8"})
+		base.URLs = []string{"https://example.invalid/qwen3"}
+		newGallery(base, entry("qwen3-8b-q8", "upgrade-backend"))
+
+		Expect(install("qwen3-8b-q4", gallery.GalleryModel{})).To(Succeed())
+
+		record, err := gallery.GetLocalModelConfiguration(tempdir, "qwen3-8b-q4")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(record.URLs).To(ConsistOf("https://example.invalid/qwen3"))
+	})
+})
+
+var _ = Describe("legacy client compatibility", func() {
+	It("keeps every entry that declares variants installable by clients that ignore them", func() {
+		data, err := os.ReadFile(filepath.Join("..", "..", "gallery", "index.yaml"))
+		Expect(err).ToNot(HaveOccurred())
+
+		// Parse exactly as an older LocalAI release would: non-strictly, with
+		// no knowledge of the variants key. Such a client installs whatever
+		// payload the entry carries directly, so the entry must carry one.
+		var legacy []struct {
+			Name       string         `yaml:"name"`
+			URL        string         `yaml:"url"`
+			ConfigFile map[string]any `yaml:"config_file"`
+			Files      []gallery.File `yaml:"files"`
+			Overrides  map[string]any `yaml:"overrides"`
+		}
+		Expect(yaml.Unmarshal(data, &legacy)).To(Succeed())
+
+		var current []gallery.GalleryModel
+		Expect(yaml.Unmarshal(data, &current)).To(Succeed())
+
+		legacyByName := map[string]int{}
+		for i, e := range legacy {
+			legacyByName[e.Name] = i
+		}
+
+		withVariants := 0
+		for _, e := range current {
+			if !e.HasVariants() {
+				continue
+			}
+			withVariants++
+
+			i, ok := legacyByName[e.Name]
+			Expect(ok).To(BeTrue(), "entry %q vanished under a legacy parse", e.Name)
+			old := legacy[i]
+			// An entry whose payload lived only in its variants would install to
+			// nothing on every released LocalAI, which is precisely what making
+			// the entry itself the last resort exists to prevent.
+			Expect(old.URL != "" || len(old.ConfigFile) > 0).To(BeTrue(),
+				"entry %q carries no payload of its own, so older clients would install nothing", e.Name)
+		}
+		Expect(withVariants).To(BeNumerically(">", 0),
+			"expected at least one entry declaring variants in the gallery index")
+	})
+})

core/gallery/variants_lint_test.go

@@ -0,0 +1,400 @@
+package gallery_test
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+	"sync"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+	"gopkg.in/yaml.v3"
+
+	"github.com/mudler/LocalAI/core/gallery"
+)
+
+// variantViolation is one invariant breach found by a lint helper. Helpers
+// return every breach they find instead of stopping at the first, so a single
+// run names them all rather than forcing a fix-one-rerun-repeat cycle.
+type variantViolation struct {
+	Entry   string
+	Variant string
+	Detail  string
+}
+
+func (v variantViolation) String() string {
+	if v.Variant == "" {
+		return fmt.Sprintf("%s: %s", v.Entry, v.Detail)
+	}
+	return fmt.Sprintf("%s -> variant %q: %s", v.Entry, v.Variant, v.Detail)
+}
+
+func formatViolations(violations []variantViolation) string {
+	lines := make([]string, 0, len(violations))
+	for _, v := range violations {
+		lines = append(lines, "  "+v.String())
+	}
+	return "\n" + strings.Join(lines, "\n")
+}
+
+func indexEntriesByName(entries []gallery.GalleryModel) map[string]gallery.GalleryModel {
+	byName := make(map[string]gallery.GalleryModel, len(entries))
+	for _, e := range entries {
+		byName[e.Name] = e
+	}
+	return byName
+}
+
+// checkVariantReferences verifies every variant names an existing entry that
+// declares no variants of its own. Selection is a single pass, so a nested
+// reference would silently ignore the inner list.
+//
+// This is the one structural rule left. The rules about authored order, the
+// hardware vocabulary and floor relationships stopped describing real hazards
+// once selection became a filter plus a ranking: an author writes names, and
+// there is no longer anything about hardware they can get wrong.
+func checkVariantReferences(entries []gallery.GalleryModel) []variantViolation {
+	byName := indexEntriesByName(entries)
+	var violations []variantViolation
+	for _, e := range entries {
+		if !e.HasVariants() {
+			continue
+		}
+		for _, v := range e.Variants {
+			target, ok := byName[v.Model]
+			if !ok {
+				violations = append(violations, variantViolation{Entry: e.Name, Variant: v.Model, Detail: "references unknown model"})
+				continue
+			}
+			if target.HasVariants() {
+				violations = append(violations, variantViolation{Entry: e.Name, Variant: v.Model, Detail: "references an entry that declares variants of its own; nesting is not allowed"})
+			}
+		}
+	}
+	return violations
+}
+
+// checkEntriesInstallSomething verifies every entry would actually put
+// something on disk.
+//
+// It replaces an earlier rule that demanded a url: or a config_file: from every
+// variant target. That demand no longer holds: applyModel now treats an entry
+// declaring neither as an empty base config, which is precisely what the many
+// entries pointing at gallery/virtual.yaml were already getting, minus the
+// fetch. Requiring one of the two fields would now reject perfectly good
+// authoring.
+//
+// What survives is the weaker invariant the relaxation left exposed. An entry
+// with no base config, no overrides: and no files: names nothing to download
+// and nothing to configure, so installing it yields an empty model directory.
+// That is an authoring mistake, and it is worth catching in the catalog rather
+// than on someone's machine.
+//
+// The rule covers every entry, not only variant targets, because the hazard has
+// nothing to do with variants: it is a half-written stanza, and a parent entry
+// can be one just as easily as a target.
+// entryInstallsSomething restates applyModel's acceptance rule in the terms an
+// author reads: a base config, or a payload to lay over an empty one.
+func entryInstallsSomething(e gallery.GalleryModel) bool {
+	return len(e.URL) > 0 || len(e.ConfigFile) > 0 || len(e.Overrides) > 0 || len(e.AdditionalFiles) > 0
+}
+
+func checkEntriesInstallSomething(entries []gallery.GalleryModel) []variantViolation {
+	var violations []variantViolation
+	for _, e := range entries {
+		if entryInstallsSomething(e) {
+			continue
+		}
+		violations = append(violations, variantViolation{
+			Entry: e.Name,
+			Detail: fmt.Sprintf("entry %q installs nothing: it declares no url:, no config_file:, no overrides: and no files:, "+
+				"so installing it would leave an empty model directory. Give it the payload it is missing. "+
+				"Note that urls: (plural) is the informational link list and is not a payload", e.Name),
+		})
+	}
+	return violations
+}
+
+// loadGalleryIndex parses gallery/index.yaml once for the whole suite. The
+// index carries well over a thousand entries, so re-parsing it per spec is
+// pure overhead.
+var loadGalleryIndex = sync.OnceValues(func() ([]gallery.GalleryModel, error) {
+	data, err := os.ReadFile(filepath.Join("..", "..", "gallery", "index.yaml"))
+	if err != nil {
+		return nil, err
+	}
+	var entries []gallery.GalleryModel
+	if err := yaml.Unmarshal(data, &entries); err != nil {
+		return nil, err
+	}
+	return entries, nil
+})
+
+func plainEntry(name, url string) gallery.GalleryModel {
+	e := gallery.GalleryModel{}
+	e.Name = name
+	e.URL = url
+	return e
+}
+
+func entryWithVariants(name, url string, variants ...gallery.Variant) gallery.GalleryModel {
+	e := gallery.GalleryModel{Variants: variants}
+	e.Name = name
+	e.URL = url
+	return e
+}
+
+// variantFixture builds an entry with variants plus the entries it references.
+// Synthetic fixtures keep the invariant logic covered however few entries in
+// gallery/index.yaml declare variants; without them every index-driven spec
+// below is a no-op that passes while checking nothing.
+func variantFixture(base gallery.GalleryModel, referenced ...gallery.GalleryModel) []gallery.GalleryModel {
+	return append([]gallery.GalleryModel{base}, referenced...)
+}
+
+var _ = Describe("gallery variant lint helpers", func() {
+	// An entry declares variants solely by carrying the key. GalleryBackend.IsMeta()
+	// has deliberately different semantics (it requires an EMPTY uri), so a
+	// well-meaning alignment of the two would make every helper skip every
+	// entry and pass silently. Assert the distinction directly.
+	It("treats an entry with variants as such even though it has a url", func() {
+		Expect(entryWithVariants("base", "u://base", gallery.Variant{Model: "big"}).HasVariants()).To(BeTrue())
+		Expect(plainEntry("big", "u://big").HasVariants()).To(BeFalse())
+	})
+
+	It("passes every invariant on a valid entry", func() {
+		// Authoring is nothing but a list of names, so a bare list of them must
+		// be entirely valid.
+		entries := variantFixture(
+			entryWithVariants("base", "u://base",
+				gallery.Variant{Model: "big"},
+				gallery.Variant{Model: "mid"},
+				gallery.Variant{Model: "metal-big"},
+				gallery.Variant{Model: "small"},
+			),
+			plainEntry("big", "u://big"),
+			plainEntry("mid", "u://mid"),
+			plainEntry("metal-big", "u://metal-big"),
+			plainEntry("small", "u://small"),
+		)
+
+		Expect(checkVariantReferences(entries)).To(BeEmpty())
+		Expect(checkEntriesInstallSomething(entries)).To(BeEmpty())
+	})
+
+	Describe("checkEntriesInstallSomething", func() {
+		// The shape the rule exists for: a stanza that got as far as a name and
+		// stopped.
+		empty := func(name string) gallery.GalleryModel {
+			e := gallery.GalleryModel{}
+			e.Name = name
+			// urls: is the informational HuggingFace link list. It reads like a
+			// payload and is not one, so a stub carrying only this must still be
+			// flagged.
+			e.URLs = []string{"https://huggingface.co/example/" + name}
+			return e
+		}
+
+		It("flags an entry with no url, no config_file, no overrides and no files", func() {
+			entries := variantFixture(
+				entryWithVariants("base", "u://base", gallery.Variant{Model: "dead"}),
+				empty("dead"),
+			)
+
+			violations := checkEntriesInstallSomething(entries)
+			Expect(violations).To(HaveLen(1))
+			Expect(violations[0].Entry).To(Equal("dead"))
+			Expect(violations[0].Detail).To(ContainSubstring("installs nothing"))
+		})
+
+		It("accepts an entry carrying only overrides, which applyModel installs on an empty base", func() {
+			target := gallery.GalleryModel{Overrides: map[string]any{"backend": "ds4"}}
+			target.Name = "overrides-only"
+			entries := variantFixture(
+				entryWithVariants("base", "u://base", gallery.Variant{Model: "overrides-only"}),
+				target,
+			)
+
+			Expect(checkEntriesInstallSomething(entries)).To(BeEmpty())
+		})
+
+		It("accepts an entry carrying only files", func() {
+			target := gallery.GalleryModel{}
+			target.Name = "files-only"
+			target.AdditionalFiles = []gallery.File{{Filename: "weights.gguf", URI: "u://weights"}}
+			entries := variantFixture(
+				entryWithVariants("base", "u://base", gallery.Variant{Model: "files-only"}),
+				target,
+			)
+
+			Expect(checkEntriesInstallSomething(entries)).To(BeEmpty())
+		})
+
+		It("accepts an entry described by an inline config_file rather than a url", func() {
+			target := gallery.GalleryModel{ConfigFile: map[string]any{"backend": "llama-cpp"}}
+			target.Name = "inline"
+			entries := variantFixture(
+				entryWithVariants("base", "u://base", gallery.Variant{Model: "inline"}),
+				target,
+			)
+
+			Expect(checkEntriesInstallSomething(entries)).To(BeEmpty())
+		})
+
+		It("reports every breach in one pass rather than stopping at the first", func() {
+			entries := variantFixture(
+				entryWithVariants("base", "u://base",
+					gallery.Variant{Model: "dead-a"},
+					gallery.Variant{Model: "dead-b"},
+				),
+				empty("dead-a"),
+				empty("dead-b"),
+			)
+
+			Expect(checkEntriesInstallSomething(entries)).To(HaveLen(2))
+		})
+	})
+
+	Describe("checkVariantReferences", func() {
+		It("flags a variant naming an entry that does not exist", func() {
+			entries := variantFixture(
+				entryWithVariants("base", "u://base", gallery.Variant{Model: "ghost"}),
+				plainEntry("a", "u://a"),
+			)
+
+			violations := checkVariantReferences(entries)
+			Expect(violations).To(HaveLen(1))
+			Expect(violations[0].Variant).To(Equal("ghost"))
+			Expect(violations[0].Detail).To(ContainSubstring("unknown model"))
+		})
+
+		It("flags a variant naming an entry that declares variants itself", func() {
+			entries := variantFixture(
+				entryWithVariants("base", "u://base", gallery.Variant{Model: "nested"}),
+				entryWithVariants("nested", "u://nested", gallery.Variant{Model: "a"}),
+				plainEntry("a", "u://a"),
+			)
+
+			violations := checkVariantReferences(entries)
+			Expect(violations).To(HaveLen(1))
+			Expect(violations[0].Variant).To(Equal("nested"))
+			Expect(violations[0].Detail).To(ContainSubstring("nesting is not allowed"))
+		})
+
+		It("reports every breach in one pass rather than stopping at the first", func() {
+			entries := variantFixture(
+				entryWithVariants("base", "u://base",
+					gallery.Variant{Model: "ghost"},
+					gallery.Variant{Model: "phantom"},
+				),
+			)
+
+			Expect(checkVariantReferences(entries)).To(HaveLen(2))
+		})
+	})
+
+})
+
+var _ = Describe("gallery/index.yaml variant invariants", Ordered, func() {
+	var entries []gallery.GalleryModel
+
+	BeforeAll(func() {
+		var err error
+		entries, err = loadGalleryIndex()
+		Expect(err).ToNot(HaveOccurred())
+		// A truncated or emptied index unmarshals cleanly and would make every
+		// spec below vacuously pass.
+		Expect(entries).ToNot(BeEmpty())
+	})
+
+	It("references only existing entries that declare no variants themselves", func() {
+		v := checkVariantReferences(entries)
+		Expect(v).To(BeEmpty(), formatViolations(v))
+	})
+
+	// Gallery-wide, not variants-only. The rule this replaced was scoped to
+	// variant targets because the index carried nine unrelated entries it would
+	// have failed on. Those nine declared overrides and files but no url, and
+	// they install cleanly on an empty base now, so there is nothing left to
+	// exempt and the gate covers the whole catalog in one step.
+	It("contains no entry that would install nothing", func() {
+		v := checkEntriesInstallSomething(entries)
+		Expect(v).To(BeEmpty(), formatViolations(v))
+	})
+})
+
+// The lint rules above check the catalog as text. This drives the real
+// resolution path for the entry a user actually clicked and failed to install,
+// so the fix is proven at the layer that broke and not only at the layer that
+// should have caught it.
+//
+// It is a container of its own rather than another spec beside the lint rules
+// because an Ordered container stops at its first failure: sharing one would
+// let a lint breach skip these silently, which is precisely the kind of
+// vacuously-green spec this file exists to avoid.
+var _ = Describe("gallery/index.yaml deepseek-v4-flash resolution", Ordered, func() {
+	var entries []gallery.GalleryModel
+	var models []*gallery.GalleryModel
+	var entry *gallery.GalleryModel
+
+	BeforeAll(func() {
+		var err error
+		entries, err = loadGalleryIndex()
+		Expect(err).ToNot(HaveOccurred())
+		Expect(entries).ToNot(BeEmpty())
+	})
+
+	BeforeEach(func() {
+		models = make([]*gallery.GalleryModel, 0, len(entries))
+		for i := range entries {
+			models = append(models, &entries[i])
+		}
+		entry = gallery.FindGalleryElement(models, "deepseek-v4-flash")
+		Expect(entry).ToNot(BeNil())
+		Expect(entry.HasVariants()).To(BeTrue())
+	})
+
+	// The unpinned pass covers the entry the user clicks. It does NOT prove the
+	// variants are installable: with no probe wired every size is unknown and
+	// the ranking ties, so the base wins and this only ever exercises the
+	// parent's own url. The pin spec below is what covers the four targets.
+	//
+	// Note that a base selection reports Variant.Model as the ENTRY's name
+	// rather than as empty, so asserting a non-empty Model here would look like
+	// a check that a declared variant won while passing on the base every time.
+	It("yields an installable entry for the entry itself", func() {
+		env := gallery.ResolveEnv{
+			AvailableMemory:   512 << 30,
+			BackendCompatible: func(string) bool { return true },
+		}
+
+		resolved, selected, err := gallery.ResolveVariant(models, entry, env, "")
+		Expect(err).ToNot(HaveOccurred())
+		Expect(entryInstallsSomething(*resolved)).To(BeTrue(),
+			"resolved entry %q (variant %q) carries no payload, so InstallModelFromGallery would refuse it",
+			resolved.Name, selected.Model)
+	})
+
+	// Pinning reaches each target directly, which is what actually proves all
+	// four are installable: every one of them is resolved, not just whichever
+	// the ranking happens to prefer.
+	//
+	// None of the four declares a url: any more. They describe themselves
+	// entirely through overrides: and files:, which applyModel lays over an
+	// empty base, so this also pins that dropping the urls left them
+	// installable.
+	It("yields an installable entry for every declared variant pin", func() {
+		env := gallery.ResolveEnv{
+			AvailableMemory:   512 << 30,
+			BackendCompatible: func(string) bool { return true },
+		}
+
+		for _, v := range entry.Variants {
+			resolved, _, err := gallery.ResolveVariant(models, entry, env, v.Model)
+			Expect(err).ToNot(HaveOccurred(), "pinning %q", v.Model)
+			Expect(resolved.URL).To(BeEmpty(), "variant %q should reach the empty-base path, not a fetch", v.Model)
+			Expect(entryInstallsSomething(*resolved)).To(BeTrue(), "variant %q carries no payload", v.Model)
+		}
+	})
+})

core/http/endpoints/localai/gallery.go

@@ -25,6 +25,11 @@ type ModelGalleryEndpointService struct {
 
 type GalleryModel struct {
 	ID string `json:"id"`
+	// Variant installs one specific build of an entry that declares variants,
+	// named as it appears in the entry's `variants` list (see the `variants`
+	// and `auto_variant` fields of the gallery listing). Leave it empty to let
+	// LocalAI auto-select the largest build this host can actually run.
+	Variant string `json:"variant,omitempty"`
 	gallery.GalleryModel
 }
 
@@ -86,6 +91,7 @@ func (mgs *ModelGalleryEndpointService) ApplyModelGalleryEndpoint() echo.Handler
 			Req:                input.GalleryModel,
 			ID:                 uuid.String(),
 			GalleryElementName: input.ID,
+			Variant:            input.Variant,
 			Galleries:          mgs.galleries,
 			BackendGalleries:   mgs.backendGalleries,
 		}

core/http/react-ui/e2e/backends-management.spec.js

@@ -26,3 +26,54 @@ test.describe('Backends management page', () => {
     await expect(page.getByPlaceholder('oci://quay.io/example/backend:latest')).toBeVisible()
   })
 })
+
+// Backend gallery descriptions are Markdown too: 40 of the entries in
+// backend/index.yaml carry headings, inline code, lists or links, and they used
+// to be dumped raw into the truncated table cell.
+const MARKDOWN_DESCRIPTION =
+  '# InsightFace\n\nUse `insightface` for face analysis. See [the docs](https://example.com/docs) for **details**.'
+const STRIPPED_DESCRIPTION =
+  'InsightFace Use insightface for face analysis. See the docs for details.'
+
+test.describe('Backends management page - Markdown descriptions', () => {
+  test.beforeEach(async ({ page }) => {
+    await page.route('**/api/backends*', (route) => {
+      route.fulfill({
+        contentType: 'application/json',
+        body: JSON.stringify({
+          backends: [
+            { name: 'markdown-backend', description: MARKDOWN_DESCRIPTION, installed: false },
+            { name: 'plain-backend', description: '', installed: false },
+          ],
+        }),
+      })
+    })
+    await page.goto('/app/backends')
+    await expect(page.locator('th', { hasText: 'Description' })).toBeVisible({ timeout: 10_000 })
+  })
+
+  test('table cell shows the description as clean text, not raw Markdown', async ({ page }) => {
+    const cell = page.locator('tr', { hasText: 'markdown-backend' }).locator('span[title]', { hasText: 'InsightFace' })
+
+    await expect(cell).toHaveText(STRIPPED_DESCRIPTION)
+    // The syntax itself must be gone, not merely rendered somewhere.
+    await expect(cell).not.toContainText('#')
+    await expect(cell).not.toContainText('`')
+    await expect(cell).not.toContainText('**')
+    await expect(cell).not.toContainText('https://example.com/docs')
+    // A block element here would blow up the row height.
+    await expect(cell.locator('h1')).toHaveCount(0)
+  })
+
+  test('title tooltip carries the stripped text, not raw Markdown', async ({ page }) => {
+    const cell = page.locator('tr', { hasText: 'markdown-backend' }).locator('span[title]', { hasText: 'InsightFace' })
+
+    await expect(cell).toHaveAttribute('title', STRIPPED_DESCRIPTION)
+  })
+
+  test('a backend with no description still shows the placeholder', async ({ page }) => {
+    const row = page.locator('tr', { hasText: 'plain-backend' })
+
+    await expect(row.locator('span[title=""]')).toHaveText('-')
+  })
+})

core/http/react-ui/e2e/models-recommended-panel.spec.js

@@ -0,0 +1,148 @@
+import { test, expect } from "./coverage-fixtures.js";
+
+// The "Recommended for your hardware" strip defaults its own prominence off the
+// installed-model count and remembers both the collapse choice and a dismissal,
+// so every assertion here is about state that must survive a reload.
+
+const DISMISS_KEY = "localai-models-recommended-dismissed";
+const COLLAPSE_KEY = "localai-models-recommended-collapsed";
+
+const REC_MODELS = [
+  { name: "tiny-chat", description: "Tiny", backend: "llama-cpp", installed: false, tags: ["chat"] },
+  { name: "small-chat", description: "Small", backend: "llama-cpp", installed: false, tags: ["chat"] },
+];
+
+function listResponse(installedModels) {
+  return {
+    models: REC_MODELS,
+    allBackends: ["llama-cpp"],
+    allTags: ["chat"],
+    availableModels: REC_MODELS.length,
+    installedModels,
+    totalPages: 1,
+    currentPage: 1,
+  };
+}
+
+const ESTIMATES = {
+  "tiny-chat": { sizeBytes: 512 * 1024 * 1024, sizeDisplay: "512.0 MB", estimates: { 4096: { vramBytes: 700 * 1024 * 1024, vramDisplay: "700.0 MB" } } },
+  "small-chat": { sizeBytes: 1024 * 1024 * 1024, sizeDisplay: "1.00 GB", estimates: { 4096: { vramBytes: 1400 * 1024 * 1024, vramDisplay: "1.40 GB" } } },
+};
+
+// installedModels drives the panel's default state, so each test picks its own.
+async function mockGallery(page, installedModels) {
+  // Registered first so the more specific routes below take precedence:
+  // Playwright matches the most recently added handler.
+  await page.route("**/api/models*", (route) =>
+    route.fulfill({ contentType: "application/json", body: JSON.stringify(listResponse(installedModels)) }),
+  );
+  await page.route("**/api/models/estimate/*", (route) => {
+    const name = decodeURIComponent(new URL(route.request().url()).pathname.split("/").pop());
+    return route.fulfill({ contentType: "application/json", body: JSON.stringify(ESTIMATES[name] || {}) });
+  });
+  // CPU-only host, which is the branch that shows the "no GPU detected" note.
+  await page.route("**/api/resources", (route) =>
+    route.fulfill({ contentType: "application/json", body: JSON.stringify({ type: "cpu", available: false, gpus: [] }) }),
+  );
+}
+
+const panel = (page) => page.getByTestId("recommended-models");
+const toggle = (page) => page.getByTestId("recommended-models-toggle");
+const grid = (page) => page.locator("#rec-models-content");
+
+async function gotoModels(page) {
+  await page.goto("/app/models");
+  await expect(panel(page)).toBeVisible({ timeout: 20_000 });
+}
+
+test.describe("Models gallery - recommended panel prominence", () => {
+  test("first visit with nothing installed shows the panel expanded", async ({ page }) => {
+    await mockGallery(page, 0);
+    await gotoModels(page);
+
+    await expect(toggle(page)).toHaveAttribute("aria-expanded", "true");
+    await expect(grid(page)).toBeVisible();
+    await expect(grid(page).getByText("tiny-chat")).toBeVisible();
+  });
+
+  test("a user with models installed gets it collapsed by default", async ({ page }) => {
+    await mockGallery(page, 12);
+    await gotoModels(page);
+
+    await expect(toggle(page)).toHaveAttribute("aria-expanded", "false");
+    await expect(grid(page)).toBeHidden();
+    // Collapsed is a summary, not a removal: the heading stays on the page.
+    await expect(panel(page).getByText("Recommended for your hardware")).toBeVisible();
+    await expect(panel(page).getByText("2 models suggested")).toBeVisible();
+  });
+
+  test("the collapsed summary expands again on activation", async ({ page }) => {
+    await mockGallery(page, 12);
+    await gotoModels(page);
+
+    await expect(grid(page)).toBeHidden();
+    await toggle(page).click();
+
+    await expect(toggle(page)).toHaveAttribute("aria-expanded", "true");
+    await expect(grid(page)).toBeVisible();
+    await expect(page.evaluate((k) => localStorage.getItem(k), COLLAPSE_KEY)).resolves.toBe("0");
+  });
+
+  test("the collapse choice persists across a reload", async ({ page }) => {
+    await mockGallery(page, 0);
+    await gotoModels(page);
+    await expect(grid(page)).toBeVisible();
+
+    await toggle(page).click();
+    await expect(grid(page)).toBeHidden();
+
+    await page.reload();
+    await expect(panel(page)).toBeVisible({ timeout: 20_000 });
+    await expect(toggle(page)).toHaveAttribute("aria-expanded", "false");
+    await expect(grid(page)).toBeHidden();
+  });
+
+  test("dismissing it persists across a reload", async ({ page }) => {
+    await mockGallery(page, 0);
+    await gotoModels(page);
+
+    await panel(page).getByRole("button", { name: "Dismiss recommendations" }).click();
+    await expect(panel(page)).toHaveCount(0);
+    await expect(page.evaluate((k) => localStorage.getItem(k), DISMISS_KEY)).resolves.toBe("1");
+
+    await page.reload();
+    // The table is the marker that the page finished rendering without the panel.
+    await expect(page.locator("table tbody tr").first()).toBeVisible({ timeout: 20_000 });
+    await expect(panel(page)).toHaveCount(0);
+  });
+
+  test("the toggle is keyboard operable and exposes its state", async ({ page }) => {
+    await mockGallery(page, 12);
+    await gotoModels(page);
+
+    await toggle(page).focus();
+    await expect(toggle(page)).toBeFocused();
+    await page.keyboard.press("Enter");
+    await expect(toggle(page)).toHaveAttribute("aria-expanded", "true");
+    // aria-controls must resolve to the region it actually shows and hides.
+    await expect(toggle(page)).toHaveAttribute("aria-controls", "rec-models-content");
+    await expect(grid(page)).toBeVisible();
+  });
+
+  test("recommendations render and their install buttons still work", async ({ page }) => {
+    await mockGallery(page, 0);
+    let installed = null;
+    await page.route("**/api/models/install/*", (route) => {
+      installed = decodeURIComponent(new URL(route.request().url()).pathname.split("/").pop());
+      return route.fulfill({ contentType: "application/json", body: JSON.stringify({ uuid: "job-1" }) });
+    });
+    await gotoModels(page);
+
+    const card = grid(page).locator(".rec-models-item", { hasText: "tiny-chat" });
+    await expect(card).toBeVisible();
+    await expect(card.getByText("512.0 MB")).toBeVisible();
+    await card.getByRole("button", { name: "Install" }).click();
+
+    await expect.poll(() => installed).toBe("tiny-chat");
+  });
+});

core/http/react-ui/public/locales/de/models.json

@@ -1,6 +1,17 @@
 {
   "title": "Modelle installieren",
   "subtitle": "Durchsuchen und installieren Sie KI-Modelle aus der Galerie",
+  "recommended": {
+    "title": "Empfohlen für Ihre Hardware",
+    "cpuNote": "Keine GPU erkannt - kleine Modelle, die auf der CPU reaktionsschnell bleiben.",
+    "gpuNote": "Passend zum verfügbaren VRAM, mit Spielraum für den Kontext.",
+    "install": "Installieren",
+    "installing": "Installation läuft",
+    "installStarted": "{{model}} wird installiert…",
+    "installFailed": "Installation fehlgeschlagen: {{message}}",
+    "dismiss": "Empfehlungen ausblenden",
+    "summary": "{{n}} Modelle vorgeschlagen"
+  },
   "stats": {
     "available": "Verfügbar",
     "installed": "Installiert"
@@ -24,8 +35,12 @@
     "embedding": "Embedding",
     "rerank": "Rerank",
     "fitsGpu": "Passt in die GPU",
+    "collapseVariants": "Eine Zeile pro Modell",
     "allBackends": "Alle Backends",
-    "searchBackends": "Backends suchen..."
+    "searchBackends": "Backends suchen...",
+    "contextSize": "Kontext:",
+    "useCaseLabel": "Nach Anwendungsfall filtern",
+    "unavailableForBackend": "Für das gewählte Backend nicht verfügbar"
   },
   "search": {
     "placeholder": "Modelle suchen...",
@@ -71,6 +86,7 @@
   "empty": {
     "title": "Keine Modelle gefunden",
     "withFilters": "Keine Modelle entsprechen den aktuellen Such- oder Filterkriterien.",
+    "collapsedVariantsHint": "Alternative Builds, die ein anderer Eintrag bereits anbietet, sind ausgeblendet. Schalte \"Eine Zeile pro Modell\" aus, um sie einzubeziehen.",
     "noFilters": "Die Modellgalerie ist leer."
   },
   "deleteDialog": {
@@ -83,5 +99,27 @@
     "loadFailed": "Laden der Modelle fehlgeschlagen: {{message}}",
     "installFailed": "Installation fehlgeschlagen: {{message}}",
     "deleteFailed": "Löschen fehlgeschlagen: {{message}}"
+  },
+  "variants": {
+    "title": "Variants",
+    "chooseVariant": "Choose a variant",
+    "auto": "Auto",
+    "autoSelected": "Auto-selected",
+    "base": "Base build",
+    "fits": "Fits",
+    "doesNotFit": "Does not fit",
+    "unknownSize": "Unknown size",
+    "unknownBackend": "Unknown backend",
+    "loading": "Loading variants...",
+    "quantizationTitle": "Gewichtsformat",
+    "unknownQuantization": "Format unbekannt",
+    "showDetails": "Alle Details zu {{variant}} anzeigen",
+    "hideDetails": "Alle Details zu {{variant}} ausblenden",
+    "detailsLoading": "Details werden geladen...",
+    "detailsUnavailable": "Details zu {{variant}} konnten nicht geladen werden.",
+    "features": {
+      "dflash": "Schneller: DFlash",
+      "mtp": "Schneller: MTP"
+    }
   }
 }

core/http/react-ui/public/locales/en/models.json

@@ -10,7 +10,8 @@
     "installing": "Installing",
     "installStarted": "Installing {{model}}…",
     "installFailed": "Install failed: {{message}}",
-    "dismiss": "Dismiss recommendations"
+    "dismiss": "Dismiss recommendations",
+    "summary": "{{n}} models suggested"
   },
   "stats": {
     "available": "Available",
@@ -43,8 +44,12 @@
     "vad": "VAD",
     "ner": "NER",
     "fitsGpu": "Fits in GPU",
+    "collapseVariants": "One row per model",
     "allBackends": "All Backends",
-    "searchBackends": "Search backends..."
+    "searchBackends": "Search backends...",
+    "contextSize": "Context:",
+    "useCaseLabel": "Filter by use case",
+    "unavailableForBackend": "Not available for the selected backend"
   },
   "search": {
     "placeholder": "Search models...",
@@ -90,6 +95,7 @@
   "empty": {
     "title": "No models found",
     "withFilters": "No models match your current search or filters.",
+    "collapsedVariantsHint": "Alternative builds that another entry already offers are hidden. Turn off \"One row per model\" to include them.",
     "noFilters": "The model gallery is empty."
   },
   "deleteDialog": {
@@ -108,5 +114,27 @@
     "selectModel": "Select model...",
     "searchPlaceholder": "Search models...",
     "noModels": "No models available"
+  },
+  "variants": {
+    "title": "Variants",
+    "chooseVariant": "Choose a variant",
+    "auto": "Auto",
+    "autoSelected": "Auto-selected",
+    "base": "Base build",
+    "doesNotFit": "Does not fit",
+    "unknownSize": "Unknown size",
+    "unknownBackend": "Unknown backend",
+    "loading": "Loading variants...",
+    "installVariant": "Install {{variant}}",
+    "quantizationTitle": "Weight format",
+    "unknownQuantization": "Unknown format",
+    "showDetails": "Show full details for {{variant}}",
+    "hideDetails": "Hide full details for {{variant}}",
+    "detailsLoading": "Loading details...",
+    "detailsUnavailable": "Details for {{variant}} could not be loaded.",
+    "features": {
+      "dflash": "Faster: DFlash",
+      "mtp": "Faster: MTP"
+    }
   }
 }

core/http/react-ui/public/locales/es/models.json

@@ -1,6 +1,17 @@
 {
   "title": "Instalar modelos",
   "subtitle": "Explora e instala modelos de IA desde la galería",
+  "recommended": {
+    "title": "Recomendado para tu hardware",
+    "cpuNote": "No se detectó GPU - modelos pequeños que responden bien en CPU.",
+    "gpuNote": "Ajustados a tu VRAM disponible, con margen para el contexto.",
+    "install": "Instalar",
+    "installing": "Instalando",
+    "installStarted": "Instalando {{model}}…",
+    "installFailed": "Error al instalar: {{message}}",
+    "dismiss": "Descartar recomendaciones",
+    "summary": "{{n}} modelos sugeridos"
+  },
   "stats": {
     "available": "Disponibles",
     "installed": "Instalados"
@@ -24,8 +35,12 @@
     "embedding": "Embedding",
     "rerank": "Rerank",
     "fitsGpu": "Cabe en la GPU",
+    "collapseVariants": "Una fila por modelo",
     "allBackends": "Todos los backends",
-    "searchBackends": "Buscar backends..."
+    "searchBackends": "Buscar backends...",
+    "contextSize": "Contexto:",
+    "useCaseLabel": "Filtrar por caso de uso",
+    "unavailableForBackend": "No disponible para el backend seleccionado"
   },
   "search": {
     "placeholder": "Buscar modelos...",
@@ -71,6 +86,7 @@
   "empty": {
     "title": "No se encontraron modelos",
     "withFilters": "Ningún modelo coincide con la búsqueda o filtros actuales.",
+    "collapsedVariantsHint": "Las compilaciones alternativas que otra entrada ya ofrece están ocultas. Desactiva \"Una fila por modelo\" para incluirlas.",
     "noFilters": "La galería de modelos está vacía."
   },
   "deleteDialog": {
@@ -83,5 +99,27 @@
     "loadFailed": "Error al cargar modelos: {{message}}",
     "installFailed": "Error al instalar: {{message}}",
     "deleteFailed": "Error al eliminar: {{message}}"
+  },
+  "variants": {
+    "title": "Variants",
+    "chooseVariant": "Choose a variant",
+    "auto": "Auto",
+    "autoSelected": "Auto-selected",
+    "base": "Base build",
+    "fits": "Fits",
+    "doesNotFit": "Does not fit",
+    "unknownSize": "Unknown size",
+    "unknownBackend": "Unknown backend",
+    "loading": "Loading variants...",
+    "quantizationTitle": "Formato de pesos",
+    "unknownQuantization": "Formato desconocido",
+    "showDetails": "Mostrar todos los detalles de {{variant}}",
+    "hideDetails": "Ocultar todos los detalles de {{variant}}",
+    "detailsLoading": "Cargando detalles...",
+    "detailsUnavailable": "No se pudieron cargar los detalles de {{variant}}.",
+    "features": {
+      "dflash": "Más rápido: DFlash",
+      "mtp": "Más rápido: MTP"
+    }
   }
 }

core/http/react-ui/public/locales/id/models.json

@@ -2,6 +2,17 @@
   "title": "Instal Model",
   "subtitle": "Telusuri dan instal model AI dari galeri",
   "models": "Model",
+  "recommended": {
+    "title": "Direkomendasikan untuk perangkat keras Anda",
+    "cpuNote": "GPU tidak terdeteksi - model kecil yang tetap responsif di CPU.",
+    "gpuNote": "Disesuaikan dengan VRAM yang tersedia, menyisakan ruang untuk konteks.",
+    "install": "Instal",
+    "installing": "Menginstal",
+    "installStarted": "Menginstal {{model}}…",
+    "installFailed": "Instalasi gagal: {{message}}",
+    "dismiss": "Tutup rekomendasi",
+    "summary": "{{n}} model disarankan"
+  },
   "stats": {
     "available": "Tersedia",
     "installed": "Terinstal"
@@ -31,8 +42,12 @@
     "detection": "Deteksi",
     "vad": "VAD",
     "fitsGpu": "Muat di GPU",
+    "collapseVariants": "Satu baris per model",
     "allBackends": "Semua Backend",
-    "searchBackends": "Cari backends..."
+    "searchBackends": "Cari backends...",
+    "contextSize": "Konteks:",
+    "useCaseLabel": "Filter berdasarkan kasus penggunaan",
+    "unavailableForBackend": "Tidak tersedia untuk backend yang dipilih"
   },
   "search": {
     "placeholder": "Cari model...",
@@ -78,6 +93,7 @@
   "empty": {
     "title": "Model tidak ditemukan",
     "withFilters": "Tidak ada model yang cocok dengan pencarian atau filter Anda.",
+    "collapsedVariantsHint": "Build alternatif yang sudah ditawarkan entri lain disembunyikan. Matikan \"Satu baris per model\" untuk menyertakannya.",
     "noFilters": "Galeri model kosong."
   },
   "deleteDialog": {
@@ -96,5 +112,27 @@
     "selectModel": "Pilih model...",
     "searchPlaceholder": "Cari model...",
     "noModels": "Model tidak tersedia"
+  },
+  "variants": {
+    "title": "Variants",
+    "chooseVariant": "Choose a variant",
+    "auto": "Auto",
+    "autoSelected": "Auto-selected",
+    "base": "Base build",
+    "fits": "Fits",
+    "doesNotFit": "Does not fit",
+    "unknownSize": "Unknown size",
+    "unknownBackend": "Unknown backend",
+    "loading": "Loading variants...",
+    "quantizationTitle": "Format bobot",
+    "unknownQuantization": "Format tidak diketahui",
+    "showDetails": "Tampilkan detail lengkap {{variant}}",
+    "hideDetails": "Sembunyikan detail lengkap {{variant}}",
+    "detailsLoading": "Memuat detail...",
+    "detailsUnavailable": "Detail untuk {{variant}} tidak dapat dimuat.",
+    "features": {
+      "dflash": "Lebih cepat: DFlash",
+      "mtp": "Lebih cepat: MTP"
+    }
   }
 }

core/http/react-ui/public/locales/it/models.json

@@ -1,6 +1,17 @@
 {
   "title": "Installa modelli",
   "subtitle": "Sfoglia e installa modelli AI dalla galleria",
+  "recommended": {
+    "title": "Consigliati per il tuo hardware",
+    "cpuNote": "Nessuna GPU rilevata - modelli piccoli che restano reattivi su CPU.",
+    "gpuNote": "Dimensionati per la VRAM disponibile, con spazio per il contesto.",
+    "install": "Installa",
+    "installing": "Installazione",
+    "installStarted": "Installazione di {{model}}…",
+    "installFailed": "Installazione non riuscita: {{message}}",
+    "dismiss": "Nascondi i consigli",
+    "summary": "{{n}} modelli suggeriti"
+  },
   "stats": {
     "available": "Disponibili",
     "installed": "Installati"
@@ -24,8 +35,12 @@
     "embedding": "Embedding",
     "rerank": "Rerank",
     "fitsGpu": "Entra nella GPU",
+    "collapseVariants": "Una riga per modello",
     "allBackends": "Tutti i backend",
-    "searchBackends": "Cerca backend..."
+    "searchBackends": "Cerca backend...",
+    "contextSize": "Contesto:",
+    "useCaseLabel": "Filtra per caso d'uso",
+    "unavailableForBackend": "Non disponibile per il backend selezionato"
   },
   "search": {
     "placeholder": "Cerca modelli...",
@@ -71,6 +86,7 @@
   "empty": {
     "title": "Nessun modello trovato",
     "withFilters": "Nessun modello corrisponde ai filtri attuali.",
+    "collapsedVariantsHint": "Le build alternative gia offerte da un'altra voce sono nascoste. Disattiva \"Una riga per modello\" per includerle.",
     "noFilters": "La galleria modelli è vuota."
   },
   "deleteDialog": {
@@ -83,5 +99,27 @@
     "loadFailed": "Caricamento modelli fallito: {{message}}",
     "installFailed": "Installazione fallita: {{message}}",
     "deleteFailed": "Eliminazione fallita: {{message}}"
+  },
+  "variants": {
+    "title": "Variants",
+    "chooseVariant": "Choose a variant",
+    "auto": "Auto",
+    "autoSelected": "Auto-selected",
+    "base": "Base build",
+    "fits": "Fits",
+    "doesNotFit": "Does not fit",
+    "unknownSize": "Unknown size",
+    "unknownBackend": "Unknown backend",
+    "loading": "Loading variants...",
+    "quantizationTitle": "Formato dei pesi",
+    "unknownQuantization": "Formato sconosciuto",
+    "showDetails": "Mostra tutti i dettagli di {{variant}}",
+    "hideDetails": "Nascondi tutti i dettagli di {{variant}}",
+    "detailsLoading": "Caricamento dettagli...",
+    "detailsUnavailable": "Impossibile caricare i dettagli di {{variant}}.",
+    "features": {
+      "dflash": "Più veloce: DFlash",
+      "mtp": "Più veloce: MTP"
+    }
   }
 }

core/http/react-ui/public/locales/ko/models.json

@@ -1,6 +1,17 @@
 {
   "title": "모델 설치",
   "subtitle": "갤러리에서 AI 모델을 둘러보고 설치합니다",
+  "recommended": {
+    "title": "하드웨어에 맞는 추천",
+    "cpuNote": "GPU가 감지되지 않았습니다 - CPU에서도 빠르게 동작하는 소형 모델입니다.",
+    "gpuNote": "사용 가능한 VRAM에 맞추고 컨텍스트 여유를 남겼습니다.",
+    "install": "설치",
+    "installing": "설치 중",
+    "installStarted": "{{model}} 설치 중…",
+    "installFailed": "설치 실패: {{message}}",
+    "dismiss": "추천 닫기",
+    "summary": "추천 모델 {{n}}개"
+  },
   "stats": {
     "available": "사용 가능",
     "installed": "설치됨"
@@ -30,8 +41,12 @@
     "detection": "감지",
     "vad": "VAD",
     "fitsGpu": "GPU에 적합",
+    "collapseVariants": "모델당 한 행",
     "allBackends": "모든 백엔드",
-    "searchBackends": "백엔드 검색..."
+    "searchBackends": "백엔드 검색...",
+    "contextSize": "컨텍스트:",
+    "useCaseLabel": "사용 사례로 필터링",
+    "unavailableForBackend": "선택한 백엔드에서 사용할 수 없음"
   },
   "search": {
     "placeholder": "모델 검색...",
@@ -77,6 +92,7 @@
   "empty": {
     "title": "모델을 찾을 수 없습니다",
     "withFilters": "현재 검색 또는 필터와 일치하는 모델이 없습니다.",
+    "collapsedVariantsHint": "다른 항목이 이미 제공하는 대체 빌드는 숨겨져 있습니다. 포함하려면 \"모델당 한 행\"을 끄세요.",
     "noFilters": "모델 갤러리가 비어 있습니다."
   },
   "deleteDialog": {

core/http/react-ui/public/locales/zh-CN/models.json

@@ -1,6 +1,17 @@
 {
   "title": "安装模型",
   "subtitle": "从模型库浏览和安装 AI 模型",
+  "recommended": {
+    "title": "适合你硬件的推荐",
+    "cpuNote": "未检测到 GPU - 这些小模型在 CPU 上依然流畅。",
+    "gpuNote": "已按可用显存选择，并为上下文留出空间。",
+    "install": "安装",
+    "installing": "安装中",
+    "installStarted": "正在安装 {{model}}…",
+    "installFailed": "安装失败：{{message}}",
+    "dismiss": "关闭推荐",
+    "summary": "已推荐 {{n}} 个模型"
+  },
   "stats": {
     "available": "可用",
     "installed": "已安装"
@@ -24,8 +35,12 @@
     "embedding": "嵌入",
     "rerank": "重排",
     "fitsGpu": "适合 GPU",
+    "collapseVariants": "每个模型一行",
     "allBackends": "所有后端",
-    "searchBackends": "搜索后端..."
+    "searchBackends": "搜索后端...",
+    "contextSize": "上下文：",
+    "useCaseLabel": "按用例筛选",
+    "unavailableForBackend": "所选后端不支持"
   },
   "search": {
     "placeholder": "搜索模型...",
@@ -71,6 +86,7 @@
   "empty": {
     "title": "未找到模型",
     "withFilters": "没有模型与当前搜索或筛选条件匹配。",
+    "collapsedVariantsHint": "其他条目已经提供的替代构建已被隐藏。关闭“每个模型一行”即可将其包含在内。",
     "noFilters": "模型库为空。"
   },
   "deleteDialog": {
@@ -83,5 +99,27 @@
     "loadFailed": "加载模型失败：{{message}}",
     "installFailed": "安装失败：{{message}}",
     "deleteFailed": "删除失败：{{message}}"
+  },
+  "variants": {
+    "title": "Variants",
+    "chooseVariant": "Choose a variant",
+    "auto": "Auto",
+    "autoSelected": "Auto-selected",
+    "base": "Base build",
+    "fits": "Fits",
+    "doesNotFit": "Does not fit",
+    "unknownSize": "Unknown size",
+    "unknownBackend": "Unknown backend",
+    "loading": "Loading variants...",
+    "quantizationTitle": "权重格式",
+    "unknownQuantization": "未知格式",
+    "showDetails": "显示 {{variant}} 的完整详情",
+    "hideDetails": "隐藏 {{variant}} 的完整详情",
+    "detailsLoading": "正在加载详情...",
+    "detailsUnavailable": "无法加载 {{variant}} 的详情。",
+    "features": {
+      "dflash": "更快：DFlash",
+      "mtp": "更快：MTP"
+    }
   }
 }

core/http/react-ui/src/App.css

@@ -2191,6 +2191,68 @@ select.input {
   user-select: none;
   white-space: nowrap;
 }
+/* Models gallery filter block. Three bands (query / taxonomy / refinements)
+   stacked by .filter-bar-group. The refinements live in their own band so the
+   chip row's wrap height can never displace them. */
+.models-filters__query {
+  flex-wrap: nowrap;
+  gap: var(--spacing-sm);
+}
+.models-filters__backend {
+  flex: 0 0 auto;
+  min-width: 0;
+}
+/* .filter-bar carries its own bottom margin for standalone use; inside the
+   group the parent's gap owns the rhythm. */
+.models-filters > .filter-bar {
+  margin-bottom: 0;
+}
+.models-filters__refine {
+  display: flex;
+  align-items: center;
+  flex-wrap: wrap;
+  gap: var(--spacing-sm) var(--spacing-lg);
+  padding-top: var(--spacing-sm);
+  border-top: 1px solid var(--color-border-subtle);
+}
+/* Both refinements read as one secondary group: same size, same colour and the
+   same icon-then-label rhythm as .filter-bar-group__toggle. */
+.models-filters__context {
+  display: flex;
+  align-items: center;
+  gap: var(--spacing-sm);
+  font-size: var(--text-xs);
+  color: var(--color-text-secondary);
+}
+.models-filters__context > label {
+  display: inline-flex;
+  align-items: center;
+  gap: var(--spacing-xs);
+  white-space: nowrap;
+  cursor: pointer;
+}
+.models-filters__context input[type='range'] {
+  width: 140px;
+  accent-color: var(--color-primary);
+  cursor: pointer;
+}
+.models-filters__context-value {
+  font-weight: var(--font-weight-medium);
+  min-width: 3em;
+  font-variant-numeric: tabular-nums;
+}
+
+@media (max-width: 768px) {
+  /* Search and backend select each take a full row rather than squeezing the
+     input below a usable width. */
+  .models-filters__query {
+    flex-wrap: wrap;
+  }
+  .models-filters__backend {
+    flex: 1 1 100%;
+  }
+}
+
 .filter-btn__count {
   display: inline-flex;
   align-items: center;
@@ -2720,6 +2782,40 @@ button.collapsible-header:focus-visible {
   border-color: var(--color-primary-border);
 }
 
+/* The global :focus-visible ring is wrapped in :where(), so it carries the
+   specificity of a bare :focus-visible - the same (0,1,0) as .filter-btn, which
+   is declared later and therefore won with its resting shadow. Chips were left
+   with no visible keyboard focus indicator at all; this restates the ring where
+   it outranks both the resting and the hover shadow. */
+.filter-btn:focus-visible {
+  box-shadow: 0 0 0 3px var(--color-focus-ring);
+  border-color: var(--color-primary);
+}
+
+/* A chip is disabled when the selected backend cannot serve that use case. It
+   keeps its resting colours so the row still reads as a set, dimmed enough to
+   register as out of play. */
+.filter-btn:disabled {
+  opacity: 0.4;
+  cursor: not-allowed;
+  box-shadow: none;
+}
+.filter-btn:disabled:hover {
+  transform: none;
+  box-shadow: none;
+  color: var(--color-text-secondary);
+  border-color: var(--color-border-default);
+}
+
+@media (prefers-reduced-motion: reduce) {
+  .filter-btn {
+    transition: none;
+  }
+  .filter-btn:hover {
+    transform: none;
+  }
+}
+
 /* Login page */
 .login-page {
   min-height: 100vh;
@@ -2890,6 +2986,16 @@ button.collapsible-header:focus-visible {
   margin: 0;
 }
 
+/* Secondary to .empty-state-text: a hint about a default that may be narrowing
+   the result, so it must not compete with the reason stated above it. */
+.empty-state-hint {
+  color: var(--color-text-muted);
+  font-size: var(--text-sm);
+  line-height: var(--leading-normal);
+  max-width: 52ch;
+  margin: 0;
+}
+
 /* Opt-in editorial sub-parts — pages can adopt these class names over time */
 .empty-state__eyebrow {
   display: inline-flex;
@@ -6449,24 +6555,47 @@ button.collapsible-header:focus-visible {
   margin-bottom: var(--spacing-md);
   padding: var(--spacing-md) var(--spacing-lg);
 }
+/* Collapsed keeps the strip and its heading, only dropping the cards, so the
+   panel is recoverable in place rather than gone. Tighter block padding is
+   what actually returns vertical space to the gallery. */
+.rec-models--collapsed {
+  padding-top: var(--spacing-sm);
+  padding-bottom: var(--spacing-sm);
+}
 .rec-models-head {
-  display: flex;
-  align-items: flex-start;
-  justify-content: space-between;
-  gap: var(--spacing-md);
-}
-.rec-models-title {
   display: flex;
   align-items: center;
-  gap: var(--spacing-sm);
+  gap: var(--spacing-sm) var(--spacing-md);
   flex-wrap: wrap;
 }
-.rec-models-title i {
+.rec-models-toggle {
+  display: inline-flex;
+  align-items: center;
+  gap: var(--spacing-sm);
+  background: none;
+  border: none;
+  padding: 0;
+  margin: 0;
+  font: inherit;
+  color: var(--color-text-primary);
+  cursor: pointer;
+  text-align: left;
+}
+.rec-models-toggle i {
   color: var(--color-primary);
 }
+.rec-models-chevron {
+  font-size: 0.75rem;
+  transition: transform 150ms ease;
+}
+.rec-models--collapsed .rec-models-chevron {
+  transform: rotate(-90deg);
+}
 .rec-models-note {
   font-size: 0.8125rem;
   color: var(--color-text-secondary);
+  flex: 1 1 12rem;
+  min-width: 0;
 }
 .rec-models-dismiss {
   background: none;
@@ -6481,9 +6610,36 @@ button.collapsible-header:focus-visible {
 }
 .rec-models-grid {
   display: grid;
-  grid-template-columns: repeat(auto-fill, minmax(220px, 1fr));
+  /* min() keeps the 220px track from forcing horizontal overflow on narrow
+     phones, where one column is all that fits anyway. */
+  grid-template-columns: repeat(auto-fill, minmax(min(220px, 100%), 1fr));
   gap: var(--spacing-sm);
   margin-top: var(--spacing-md);
+  /* Reveal animates opacity and transform only; the collapsed state is the
+     [hidden] attribute, so no height is being animated. */
+  animation: rec-models-reveal 180ms ease-out;
+}
+/* The class sets display:grid, which would otherwise beat the UA [hidden] rule. */
+.rec-models-grid[hidden] {
+  display: none;
+}
+@keyframes rec-models-reveal {
+  from {
+    opacity: 0;
+    transform: translateY(-4px);
+  }
+  to {
+    opacity: 1;
+    transform: none;
+  }
+}
+@media (prefers-reduced-motion: reduce) {
+  .rec-models-grid {
+    animation: none;
+  }
+  .rec-models-chevron {
+    transition: none;
+  }
 }
 .rec-models-item {
   display: flex;
@@ -9164,3 +9320,328 @@ button.collapsible-header:focus-visible {
 .model-chip__state { opacity: 0.85; font-style: normal; }
 .node-filter { margin-bottom: var(--spacing-lg); }
 .node-detail__metrics { display: flex; gap: var(--spacing-xl); margin: var(--spacing-md) 0 var(--spacing-lg); flex-wrap: wrap; }
+
+/* Rendered Markdown ---------------------------------------------------------
+   Gallery descriptions, backend descriptions and voice notes are all
+   author-supplied Markdown pasted from model cards. Left unscoped they inherit
+   browser defaults, so a description that opens with `#` renders a 2em heading
+   inside a 13px surface while a `##` further down is indistinguishable from
+   body text. This maps the whole element set onto the UI type scale so any
+   description reads as part of the app rather than as a document dropped into
+   it. Apply it to every element fed by renderMarkdown(). */
+.markdown-body {
+  color: var(--color-text-secondary);
+  font-size: var(--text-sm);
+  line-height: var(--leading-normal);
+}
+.markdown-body > :first-child { margin-top: 0; }
+.markdown-body > :last-child { margin-bottom: 0; }
+
+.markdown-body h1,
+.markdown-body h2,
+.markdown-body h3,
+.markdown-body h4,
+.markdown-body h5,
+.markdown-body h6 {
+  color: var(--color-text-primary);
+  font-weight: var(--font-weight-semibold);
+  line-height: var(--leading-tight);
+  margin: var(--spacing-md) 0 var(--spacing-xs);
+}
+.markdown-body h1 { font-size: var(--text-lg); }
+.markdown-body h2 { font-size: var(--text-base); }
+.markdown-body h3 { font-size: var(--text-sm); }
+/* Below h3 the scale has no room left to separate levels by size, so the
+   remaining depth is carried by case and weight instead. */
+.markdown-body h4,
+.markdown-body h5,
+.markdown-body h6 {
+  font-size: var(--text-xs);
+  color: var(--color-text-secondary);
+  text-transform: uppercase;
+  letter-spacing: 0.06em;
+}
+
+.markdown-body p { margin: 0 0 var(--spacing-sm); }
+.markdown-body ul,
+.markdown-body ol {
+  margin: 0 0 var(--spacing-sm);
+  padding-left: var(--spacing-lg);
+}
+.markdown-body li { margin-bottom: var(--spacing-xs); }
+.markdown-body li:last-child { margin-bottom: 0; }
+.markdown-body li > ul,
+.markdown-body li > ol { margin-top: var(--spacing-xs); }
+
+.markdown-body a {
+  color: var(--color-primary);
+  text-decoration: underline;
+  text-underline-offset: 2px;
+}
+.markdown-body a:hover { color: var(--color-primary-hover); }
+
+.markdown-body code {
+  font-family: var(--font-mono);
+  font-size: 0.9em;
+  background: var(--color-bg-tertiary);
+  border-radius: var(--radius-sm);
+  padding: 1px 4px;
+}
+.markdown-body pre {
+  margin: 0 0 var(--spacing-sm);
+  padding: var(--spacing-sm);
+  background: var(--color-surface-sunken);
+  border: 1px solid var(--color-border-subtle);
+  border-radius: var(--radius-md);
+  overflow-x: auto;
+}
+.markdown-body pre code {
+  background: none;
+  padding: 0;
+  font-size: var(--text-xs);
+}
+
+.markdown-body blockquote {
+  margin: 0 0 var(--spacing-sm);
+  padding-left: var(--spacing-sm);
+  border-left: 2px solid var(--color-border-strong);
+  color: var(--color-text-muted);
+}
+.markdown-body img {
+  max-width: 100%;
+  height: auto;
+  border-radius: var(--radius-sm);
+}
+.markdown-body hr {
+  border: 0;
+  border-top: 1px solid var(--color-border-divider);
+  margin: var(--spacing-md) 0;
+}
+/* Tables are the one element that can exceed the measure, so they scroll in
+   their own box rather than widening the surface around them. */
+.markdown-body table {
+  display: block;
+  width: max-content;
+  max-width: 100%;
+  overflow-x: auto;
+  border-collapse: collapse;
+  margin: 0 0 var(--spacing-sm);
+  font-size: var(--text-xs);
+}
+.markdown-body th,
+.markdown-body td {
+  border: 1px solid var(--color-border-subtle);
+  padding: var(--spacing-xs) var(--spacing-sm);
+  text-align: left;
+}
+.markdown-body th {
+  background: var(--color-bg-tertiary);
+  font-weight: var(--font-weight-medium);
+}
+
+/* Prose block in a detail pane. The label/value table beside it is right for
+   scalars, but multi-paragraph prose in a value cell runs the full pane width
+   and swamps the rows it sits above, so the description gets its own
+   full-width block with a reading measure. */
+.detail-prose {
+  margin-bottom: var(--spacing-md);
+}
+.detail-prose__label {
+  font-size: var(--text-sm);
+  font-weight: var(--font-weight-medium);
+  color: var(--color-text-secondary);
+  margin-bottom: var(--spacing-xs);
+}
+.detail-prose__body {
+  max-width: 68ch;
+}
+
+/* Variant list --------------------------------------------------------------
+   One row per alternative build of the same model. Names vary in length, so
+   the rows share the parent's tracks and line up as columns; the name leads
+   because it is what the reader is choosing between. Only the informative
+   status is badged: "fits" is true of nearly every row and says nothing, while
+   a build that does not fit, and the one that plain Install would pick, do. */
+.variant-list {
+  display: grid;
+  width: 100%;
+  /* info, name, backend, quantization, size, status, action, filler.
+     Every informative track is content-sized and the filler takes the slack,
+     so the rows stay packed together rather than stranding the install
+     affordance an inch of empty space away from the name it acts on. The
+     grid still spans the pane, which is what lets a revealed detail panel
+     have the full width its file table needs. */
+  grid-template-columns: max-content minmax(0, auto) max-content max-content max-content max-content max-content 1fr;
+  column-gap: var(--spacing-md);
+  row-gap: 2px;
+}
+/* One variant: the info control, the install row, and the details the control
+   reveals. It spans the list's tracks and passes them down, so the install row
+   still shares its columns with every other row despite the extra nesting. */
+.variant-entry {
+  display: grid;
+  grid-column: 1 / -1;
+  grid-template-columns: subgrid;
+  align-items: center;
+}
+.variant-row {
+  display: grid;
+  /* Stops short of the filler track so the hover and focus box hugs the row's
+     own content rather than trailing across the empty right-hand side. */
+  grid-column: 2 / -2;
+  grid-template-columns: subgrid;
+  align-items: center;
+  width: 100%;
+  margin: 0;
+  padding: var(--spacing-xs) var(--spacing-sm);
+  font: inherit;
+  text-align: left;
+  color: inherit;
+  background: none;
+  border: 1px solid transparent;
+  border-radius: var(--radius-sm);
+  cursor: pointer;
+  transition: background-color var(--duration-fast) var(--ease-default),
+              border-color var(--duration-fast) var(--ease-default);
+}
+/* Subgrid keeps the columns shared across rows; without it each row falls back
+   to sizing its own tracks, which is ragged but still legible. */
+@supports not (grid-template-columns: subgrid) {
+  .variant-entry { grid-template-columns: max-content minmax(22ch, auto) max-content max-content max-content max-content max-content 1fr; }
+  .variant-row { grid-template-columns: minmax(22ch, auto) max-content max-content max-content max-content max-content; }
+}
+.variant-row:hover:not(:disabled) {
+  background: var(--color-bg-hover);
+  border-color: var(--color-border-default);
+}
+.variant-row:focus-visible {
+  outline: 2px solid var(--color-focus-ring);
+  outline-offset: 1px;
+}
+.variant-row:disabled {
+  cursor: not-allowed;
+  opacity: 0.55;
+}
+.variant-row__name {
+  font-family: var(--font-mono);
+  font-size: var(--text-sm);
+  font-weight: var(--font-weight-medium);
+  color: var(--color-text-primary);
+  overflow: hidden;
+  text-overflow: ellipsis;
+}
+.variant-row__backend,
+.variant-row__size {
+  font-size: var(--text-xs);
+  color: var(--color-text-muted);
+}
+/* Monospaced like the name above it, because a weight format is a literal
+   token and Q4_K_M vs Q4_K_S has to be told apart at a glance in a column. */
+.variant-row__quant {
+  font-family: var(--font-mono);
+  font-size: var(--text-xs);
+  color: var(--color-text-secondary);
+  white-space: nowrap;
+}
+/* An entry naming no weight format is a real state, not a gap: it recedes to
+   the muted colour and drops the mono face, so it reads as prose saying
+   "unknown" rather than as a token. */
+.variant-row__quant--unknown {
+  font-family: inherit;
+  font-style: italic;
+  color: var(--color-text-disabled);
+}
+.variant-row__size {
+  text-align: right;
+  font-variant-numeric: tabular-nums;
+}
+.variant-row__status {
+  display: inline-flex;
+  align-items: center;
+  gap: var(--spacing-xs);
+  flex-wrap: wrap;
+}
+.variant-row__action {
+  color: var(--color-text-disabled);
+  font-size: var(--text-xs);
+  transition: color var(--duration-fast) var(--ease-default);
+}
+.variant-row:hover:not(:disabled) .variant-row__action,
+.variant-row:focus-visible .variant-row__action {
+  color: var(--color-primary);
+}
+/* A build that does not fit stays installable, an explicit choice being an
+   override the server honours, but it should not read as a peer of the ones
+   that do. */
+.variant-row--unfit .variant-row__name {
+  color: var(--color-text-muted);
+}
+
+/* Quiet until asked for: the row's job is to be compared and installed, so the
+   way into "everything about this one" recedes until the eye or the keyboard
+   lands on it. */
+.variant-row__info {
+  grid-column: 1;
+  display: inline-flex;
+  align-items: center;
+  justify-content: center;
+  width: 1.75rem;
+  height: 1.75rem;
+  margin: 0;
+  padding: 0;
+  font-size: var(--text-xs);
+  color: var(--color-text-disabled);
+  background: none;
+  border: 1px solid transparent;
+  border-radius: var(--radius-sm);
+  cursor: pointer;
+  transition: color var(--duration-fast) var(--ease-default),
+              background-color var(--duration-fast) var(--ease-default);
+}
+.variant-row__info:hover,
+.variant-entry:hover .variant-row__info {
+  color: var(--color-primary);
+}
+.variant-row__info:hover {
+  background: var(--color-bg-hover);
+}
+.variant-row__info:focus-visible {
+  outline: 2px solid var(--color-focus-ring);
+  outline-offset: 1px;
+  color: var(--color-primary);
+}
+.variant-row__info[aria-expanded="true"] {
+  color: var(--color-primary);
+  background: var(--color-bg-hover);
+  border-color: var(--color-border-default);
+}
+
+/* The third level of disclosure on this page, so it leans on an inset and a
+   rule rather than another card: a nested panel with its own border inside an
+   expanded row inside an expanded table reads as debris. */
+.variant-detail {
+  grid-column: 1 / -1;
+  min-width: 0;
+  margin: var(--spacing-xs) 0 var(--spacing-sm) var(--spacing-lg);
+  border-left: 2px solid var(--color-border-default);
+  background: var(--color-surface-sunken);
+  border-radius: 0 var(--radius-md) var(--radius-md) 0;
+  overflow: hidden;
+}
+.variant-detail__state {
+  display: flex;
+  align-items: center;
+  gap: var(--spacing-sm);
+  padding: var(--spacing-sm) var(--spacing-md);
+  font-size: var(--text-sm);
+  color: var(--color-text-muted);
+}
+.variant-detail__state--error {
+  color: var(--color-error);
+}
+
+@media (prefers-reduced-motion: reduce) {
+  .variant-row,
+  .variant-row__info,
+  .variant-row__action { transition: none; }
+}

core/http/react-ui/src/components/RecommendedModels.jsx

@@ -3,28 +3,68 @@ import { useTranslation } from 'react-i18next'
 import { modelsApi } from '../utils/api'
 import { useRecommendedModels, isNvfp4Name } from '../hooks/useRecommendedModels'
 
-const DISMISS_KEY = 'localai_rec_models_dismissed'
+// Page-scoped storage keys, matching the Models page convention
+// (localai-models-*). The underscore key is the pre-rename one: reading it
+// keeps an existing dismissal honoured instead of resurrecting the panel for
+// everyone who already closed it.
+const DISMISS_KEY = 'localai-models-recommended-dismissed'
+const LEGACY_DISMISS_KEY = 'localai_rec_models_dismissed'
+const COLLAPSE_KEY = 'localai-models-recommended-collapsed'
+const CONTENT_ID = 'rec-models-content'
+
+function readDismissed() {
+  try {
+    return localStorage.getItem(DISMISS_KEY) === '1' || localStorage.getItem(LEGACY_DISMISS_KEY) === '1'
+  } catch {
+    return false
+  }
+}
+
+// null means "the user has never chosen", which is what lets the installed
+// count pick the default instead of overriding an explicit preference.
+function readCollapsePref() {
+  try {
+    const raw = localStorage.getItem(COLLAPSE_KEY)
+    if (raw === '1') return true
+    if (raw === '0') return false
+  } catch { /* ignore */ }
+  return null
+}
 
 // "Recommended for your hardware" strip at the top of the Models gallery. Shares
 // the hardware-fit ranking with the empty-state starter widget via
-// useRecommendedModels, but styled for the gallery page and dismissible (the
-// gallery is a repeat-visit surface, so it shouldn't nag).
-export default function RecommendedModels({ addToast }) {
+// useRecommendedModels, but styled for the gallery page.
+//
+// Prominence tracks need: someone with nothing installed is exactly who this is
+// for and gets it expanded, while a stocked instance gets a one-line summary
+// that still expands on demand. Both the collapse choice and the dismissal
+// persist, so the gallery stops re-litigating the decision on every visit.
+export default function RecommendedModels({ addToast, installedCount = null }) {
   const { t } = useTranslation('models')
   const { recommended, tier, loading } = useRecommendedModels({ count: 4 })
   const [installing, setInstalling] = useState(() => new Set())
-  const [dismissed, setDismissed] = useState(() => {
-    try { return localStorage.getItem(DISMISS_KEY) === '1' } catch { return false }
-  })
+  const [dismissed, setDismissed] = useState(readDismissed)
+  const [collapsePref, setCollapsePref] = useState(readCollapsePref)
 
   if (loading || dismissed) return null
   if (!recommended || recommended.length === 0) return null
+  // Wait for the installed count before committing to a default: rendering
+  // expanded and collapsing a frame later would shove the gallery around.
+  if (installedCount === null || installedCount === undefined) return null
+
+  const collapsed = collapsePref === null ? installedCount > 0 : collapsePref
 
   const dismiss = () => {
     try { localStorage.setItem(DISMISS_KEY, '1') } catch { /* ignore */ }
     setDismissed(true)
   }
 
+  const toggle = () => {
+    const next = !collapsed
+    try { localStorage.setItem(COLLAPSE_KEY, next ? '1' : '0') } catch { /* ignore */ }
+    setCollapsePref(next)
+  }
+
   const install = async (name) => {
     setInstalling(prev => new Set(prev).add(name))
     try {
@@ -43,18 +83,33 @@ export default function RecommendedModels({ addToast }) {
   const isGpu = tier.id !== 'cpu'
 
   return (
-    <div className="rec-models card">
+    <div className={`rec-models card${collapsed ? ' rec-models--collapsed' : ''}`} data-testid="recommended-models">
       <div className="rec-models-head">
-        <div className="rec-models-title">
+        {/* The accessible name is the visible title alone; the note sits
+            outside the control so the name stays short and matches the label a
+            voice-control user would speak. State comes from aria-expanded. */}
+        <button
+          type="button"
+          className="rec-models-toggle"
+          onClick={toggle}
+          aria-expanded={!collapsed}
+          aria-controls={CONTENT_ID}
+          data-testid="recommended-models-toggle"
+        >
+          <i className="fas fa-chevron-down rec-models-chevron" aria-hidden="true" />
           <i className={`fas ${isGpu ? 'fa-microchip' : 'fa-memory'}`} aria-hidden="true" />
           <strong>{t('recommended.title')}</strong>
-          <span className="rec-models-note">{isGpu ? t('recommended.gpuNote') : t('recommended.cpuNote')}</span>
-        </div>
+        </button>
+        <span className="rec-models-note">
+          {collapsed
+            ? t('recommended.summary', { n: recommended.length })
+            : (isGpu ? t('recommended.gpuNote') : t('recommended.cpuNote'))}
+        </span>
         <button type="button" className="rec-models-dismiss" onClick={dismiss} aria-label={t('recommended.dismiss')} title={t('recommended.dismiss')}>
           <i className="fas fa-times" aria-hidden="true" />
         </button>
       </div>
-      <div className="rec-models-grid">
+      <div className="rec-models-grid" id={CONTENT_ID} hidden={collapsed}>
         {recommended.map(m => {
           const busy = installing.has(m.name)
           return (

core/http/react-ui/src/pages/Backends.jsx

@@ -8,7 +8,7 @@ import { useOperations } from '../hooks/useOperations'
 import { useDistributedMode } from '../hooks/useDistributedMode'
 import LoadingSpinner from '../components/LoadingSpinner'
 import PageHeader from '../components/PageHeader'
-import { renderMarkdown } from '../utils/markdown'
+import { renderMarkdown, stripMarkdown } from '../utils/markdown'
 import { safeHref } from '../utils/url'
 import ConfirmDialog from '../components/ConfirmDialog'
 import Toggle from '../components/Toggle'
@@ -550,13 +550,21 @@ export default function Backends() {
 
                     {/* Description */}
                     <td>
-                      <span style={{
-                        fontSize: '0.8125rem', color: 'var(--color-text-secondary)',
-                        display: 'inline-block', maxWidth: 300, overflow: 'hidden',
-                        textOverflow: 'ellipsis', whiteSpace: 'nowrap',
-                      }} title={b.description}>
-                        {b.description || '-'}
-                      </span>
+                      {(() => {
+                        // Gallery descriptions are Markdown. This cell is a single
+                        // truncated line, so it gets the text without the syntax;
+                        // the full Markdown is rendered in the detail panel instead.
+                        const desc = stripMarkdown(b.description)
+                        return (
+                          <span style={{
+                            fontSize: '0.8125rem', color: 'var(--color-text-secondary)',
+                            display: 'inline-block', maxWidth: 300, overflow: 'hidden',
+                            textOverflow: 'ellipsis', whiteSpace: 'nowrap',
+                          }} title={desc}>
+                            {desc || '-'}
+                          </span>
+                        )
+                      })()}
                     </td>
 
                     {/* Repository */}
@@ -840,7 +848,7 @@ function BackendDetail({ backend }) {
           <BackendDetailRow label="Description">
             {backend.description && (
               <div
-                style={{ color: 'var(--color-text-secondary)', lineHeight: 1.6 }}
+                className="markdown-body"
                 dangerouslySetInnerHTML={{ __html: renderMarkdown(backend.description) }}
               />
             )}

core/http/react-ui/src/pages/Manage.jsx

@@ -17,7 +17,7 @@ import { useModels } from '../hooks/useModels'
 import { useGalleryEnrichment } from '../hooks/useGalleryEnrichment'
 import { useOperations } from '../hooks/useOperations'
 import { backendControlApi, modelsApi, backendsApi, systemApi, nodesApi } from '../utils/api'
-import { renderMarkdown } from '../utils/markdown'
+import { renderMarkdown, stripMarkdown } from '../utils/markdown'
 import { safeHref } from '../utils/url'
 import {
   CAP_CHAT, CAP_COMPLETION, CAP_IMAGE, CAP_VIDEO, CAP_TTS,
@@ -121,6 +121,15 @@ function formatBackendVersion(metadata) {
   return { label: '—', full: '' }
 }
 
+// Gallery descriptions are Markdown. The row preview is a single truncated
+// line, so it shows the text without the syntax; the full Markdown is rendered
+// in the expanded detail panel instead.
+function ResourceRowDesc({ description }) {
+  const text = stripMarkdown(description)
+  if (!text) return null
+  return <span className="resource-row__desc" title={text}>{text}</span>
+}
+
 export default function Manage() {
   const { addToast } = useOutletContext()
   const navigate = useNavigate()
@@ -640,9 +649,7 @@ export default function Manage() {
                         <div style={{ display: 'flex', flexDirection: 'column', minWidth: 0 }}>
                           <span style={{ fontWeight: 600, fontSize: 'var(--text-sm)' }}>{model.id}</span>
                           {enriched?.description && (
-                            <span className="resource-row__desc" title={enriched.description}>
-                              {enriched.description}
-                            </span>
+                            <ResourceRowDesc description={enriched.description} />
                           )}
                         </div>
                       </td>
@@ -957,9 +964,7 @@ export default function Manage() {
                             )}
                           </div>
                           {(enriched?.description) && (
-                            <span className="resource-row__desc" title={enriched.description}>
-                              {enriched.description}
-                            </span>
+                            <ResourceRowDesc description={enriched.description} />
                           )}
                         </div>
                       </td>
@@ -1074,7 +1079,7 @@ function ModelDetail({ model, enriched, matchedCaps, distributedMode, onNavigate
         <dd>
           {description ? (
             <div
-              className="resource-row__detail-md"
+              className="resource-row__detail-md markdown-body"
               dangerouslySetInnerHTML={{ __html: renderMarkdown(description) }}
             />
           ) : (
@@ -1184,7 +1189,7 @@ function BackendDetail({ backend, enriched, upgradeInfo, nodes, distributedMode
         <dd>
           {description ? (
             <div
-              className="resource-row__detail-md"
+              className="resource-row__detail-md markdown-body"
               dangerouslySetInnerHTML={{ __html: renderMarkdown(description) }}
             />
           ) : (

core/http/react-ui/src/pages/Models.jsx

@@ -1,4 +1,4 @@
-import { useState, useCallback, useEffect } from 'react'
+import { useState, useCallback, useEffect, useRef } from 'react'
 import { useNavigate, useOutletContext, useLocation } from 'react-router-dom'
 import { useTranslation } from 'react-i18next'
 import { fromState } from '../utils/editorNav'
@@ -14,13 +14,42 @@ import GalleryLoader from '../components/GalleryLoader'
 import Toggle from '../components/Toggle'
 import ResponsiveTable from '../components/ResponsiveTable'
 import RecommendedModels from '../components/RecommendedModels'
+import Popover from '../components/Popover'
+import { formatBytes } from '../utils/format'
+import { renderMarkdown, stripMarkdown } from '../utils/markdown'
 import React from 'react'
 
 
 const CONTEXT_SIZES = [8192, 16384, 32768, 65536, 131072, 262144]
 const CONTEXT_LABELS = ['8K', '16K', '32K', '64K', '128K', '256K']
 const FITS_FILTER_STORAGE_KEY = 'localai-models-fits-filter'
+const COLLAPSE_VARIANTS_STORAGE_KEY = 'localai-models-collapse-variants-filter'
+// The deduplicated gallery is what a user asking "what can I install" wants, so
+// that is the default. The control exists for the other job: browsing every
+// build the gallery holds, which the collapsed view makes impossible however
+// many pages you turn.
+const COLLAPSE_VARIANTS_DEFAULT = true
 
+// How many listing rows to ask for when resolving one variant's gallery entry
+// by exact name. The term is the full name, so the entry is always in the
+// match set; the page size only has to be wide enough that the fuzzy matches
+// sharing that name's prefix cannot push it past the first page.
+const VARIANT_DETAIL_SEARCH_ITEMS = 100
+
+// Only 'on'/'off' counts as a choice. An earlier build wrote '1'/'0' from an
+// effect that ran on mount, so those values record that the page was opened
+// rather than that anyone picked a view, and honouring them would pin a
+// visitor to a default they never chose.
+const readCollapseVariantsPreference = () => {
+  try {
+    const stored = localStorage.getItem(COLLAPSE_VARIANTS_STORAGE_KEY)
+    if (stored === 'on') return true
+    if (stored === 'off') return false
+    return COLLAPSE_VARIANTS_DEFAULT
+  } catch {
+    return COLLAPSE_VARIANTS_DEFAULT
+  }
+}
 
 const FILTERS = [
   { key: '', labelKey: 'filters.all', icon: 'fa-layer-group' },
@@ -62,12 +91,29 @@ export default function Models() {
   const [expandedRow, setExpandedRow] = useState(null)
   const [expandedFiles, setExpandedFiles] = useState(false)
   const [stats, setStats] = useState({ total: 0, installed: 0, repositories: 0 })
+  // Distinguishes "nothing installed" from "not asked yet". The recommendations
+  // panel defaults off the installed count, so it must not read the initial 0.
+  const [statsLoaded, setStatsLoaded] = useState(false)
   const [backendFilter, setBackendFilter] = useState('')
   const [allBackends, setAllBackends] = useState([])
   const [backendUsecases, setBackendUsecases] = useState({})
   const [estimates, setEstimates] = useState({})
   const [contextSize, setContextSize] = useState(CONTEXT_SIZES[0])
   const [confirmDialog, setConfirmDialog] = useState(null)
+  // Index of the row whose variant split-menu is open, or null. A single
+  // Popover is re-anchored per row rather than one instance per row.
+  const [variantMenuFor, setVariantMenuFor] = useState(null)
+  const variantMenuAnchorRef = useRef(null)
+  // Variant descriptions, keyed by model name. The listing only tells us
+  // whether an entry declares any; describing them costs the server a network
+  // probe per variant, so we ask for one entry at a time and keep the answer
+  // for the rest of the page session.
+  const [variantData, setVariantData] = useState({})
+  // Gallery entries behind individual variants, keyed by variant name. The
+  // variant description carries only what ranking needs, and a variant the
+  // collapse hides has no listing row of its own, so this is the only place
+  // its description, licence, tags, links and files become reachable.
+  const [variantDetails, setVariantDetails] = useState({})
   const [fitsFilter, setFitsFilter] = useState(() => {
     try {
       return localStorage.getItem(FITS_FILTER_STORAGE_KEY) === '1'
@@ -75,7 +121,11 @@ export default function Models() {
       return false
     }
   })
-
+  // Collapses the listing to one row per model by hiding the individual builds
+  // another entry already offers as variants. Server-side, unlike fitsFilter,
+  // because the listing paginates and a client-side narrowing would leave the
+  // page count describing the unfiltered set.
+  const [collapseVariants, setCollapseVariants] = useState(readCollapseVariantsPreference)
   // Total GPU memory for "fits" check
   const totalGpuMemory = resources?.aggregate?.total_memory || 0
 
@@ -86,10 +136,20 @@ export default function Models() {
       const filtersVal = params.filters !== undefined ? params.filters : filters
       const sortVal = params.sort !== undefined ? params.sort : sort
       const backendVal = params.backendFilter !== undefined ? params.backendFilter : backendFilter
+      const collapseVal = params.collapseVariants !== undefined ? params.collapseVariants : collapseVariants
       const queryParams = {
         page: params.page || page,
         items: 9,
       }
+      // Omitted entirely when off rather than sent as false, so opting out asks
+      // for exactly the listing every other API client gets.
+      //
+      // Sent alongside the term rather than instead of it. The handler matches
+      // the term against every build the gallery holds either way; the collapse
+      // only decides how a match is reported, and grouped, a match on a build
+      // another entry offers comes back as that entry. So a search never dead
+      // ends, and what "collapsed" means stays decided in one place.
+      if (collapseVal) queryParams.collapse_variants = 'true'
       if (filtersVal.length > 0) queryParams.tag = filtersVal.join(',')
       if (searchVal) queryParams.term = searchVal
       if (backendVal) queryParams.backend = backendVal
@@ -104,17 +164,18 @@ export default function Models() {
         total: data?.availableModels || 0,
         installed: data?.installedModels || 0,
       })
+      setStatsLoaded(true)
       setAllBackends(data?.allBackends || [])
     } catch (err) {
       addToast(t('errors.loadFailed', { message: err.message }), 'error')
     } finally {
       setLoading(false)
     }
-  }, [page, search, filters, sort, order, backendFilter, addToast, t])
+  }, [page, search, filters, sort, order, backendFilter, collapseVariants, addToast, t])
 
   useEffect(() => {
     fetchModels()
-  }, [page, filters, sort, order, backendFilter])
+  }, [page, filters, sort, order, backendFilter, collapseVariants])
 
   // Fetch backend→usecase mapping once on mount
   useEffect(() => {
@@ -187,10 +248,53 @@ export default function Models() {
     }
   }
 
-  const handleInstall = async (modelId) => {
+  // Fetches an entry's variant description once. Called from the two points
+  // where a user actually asks to see variants: opening the split-button menu
+  // and expanding the detail row. An entry that declares none never gets here,
+  // so it issues no request at all.
+  const loadVariants = useCallback((id) => {
+    if (!id) return
+    setVariantData(prev => {
+      if (prev[id]) return prev
+      modelsApi.variants(id)
+        .then(data => setVariantData(p => ({ ...p, [id]: { loading: false, ...data } })))
+        .catch(() => setVariantData(p => ({ ...p, [id]: { loading: false, variants: [] } })))
+      return { ...prev, [id]: { loading: true, variants: [] } }
+    })
+  }, [])
+
+  // Resolves one variant's full gallery entry, once, and only when the user
+  // asks to see it.
+  //
+  // The listing already returns every field the detail view renders, so this
+  // keeps the fields off both the listing and DescribeVariants: an expand costs
+  // nothing, and a variant nobody opens costs nothing.
+  //
+  // The query deliberately omits collapse_variants, which is what makes it
+  // reach the build itself. Grouped, the same term would answer with the entry
+  // that offers this build, and the panel is being asked about the build.
+  //
+  // A name the listing does not return is a real outcome, not a bug to hide:
+  // the gallery can be reloaded between describing the variants and asking
+  // about one of them. It is recorded as an error so the panel can say so.
+  const loadVariantDetail = useCallback((variantName) => {
+    if (!variantName) return
+    setVariantDetails(prev => {
+      if (prev[variantName]) return prev
+      modelsApi.list({ term: variantName, items: VARIANT_DETAIL_SEARCH_ITEMS })
+        .then(data => {
+          const entry = (data?.models || []).find(m => (m.name || m.id) === variantName)
+          setVariantDetails(p => ({ ...p, [variantName]: entry ? { entry } : { error: true } }))
+        })
+        .catch(() => setVariantDetails(p => ({ ...p, [variantName]: { error: true } })))
+      return { ...prev, [variantName]: { loading: true } }
+    })
+  }, [])
+
+  const handleInstall = async (modelId, variant) => {
     try {
       setInstalling(prev => new Map(prev).set(modelId, Date.now()))
-      await modelsApi.install(modelId)
+      await modelsApi.install(modelId, variant)
     } catch (err) {
       addToast(t('errors.installFailed', { message: err.message }), 'error')
     }
@@ -264,6 +368,14 @@ export default function Models() {
     }
   }, [fitsFilter])
 
+  useEffect(() => {
+    try {
+      localStorage.setItem(COLLAPSE_VARIANTS_STORAGE_KEY, collapseVariants ? 'on' : 'off')
+    } catch {
+      // Ignore storage errors (e.g., private browsing restrictions).
+    }
+  }, [collapseVariants])
+
   const visibleModels = models.filter((model) => {
     if (!fitsFilter) return true
     const name = model.name || model.id
@@ -302,76 +414,114 @@ export default function Models() {
         }
       />
 
-      <RecommendedModels addToast={addToast} />
+      <RecommendedModels addToast={addToast} installedCount={statsLoaded ? stats.installed : null} />
 
-      {/* Search */}
-      <div className="search-bar" style={{ marginBottom: 'var(--spacing-md)' }}>
-        <i className="fas fa-search search-icon" />
-        <input
-          className="input"
-          type="text"
-          placeholder={t('search.placeholder')}
-          value={search}
-          onChange={(e) => handleSearch(e.target.value)}
-        />
-      </div>
+      {/* Filters, in three deliberate bands.
+          1. Query scope: free-text search plus the backend select. The backend
+             select leads the taxonomy row rather than trailing it because
+             picking a backend disables the use-cases that backend cannot serve
+             (see isFilterAvailable), so it reads as the gate on what follows.
+          2. Taxonomy: the use-case chips, which wrap freely.
+          3. Refinements: one row per model, fits-in-GPU and context size.
+             All three narrow a listing the user is already reading rather than
+             naming what to look at, which is what separates them from the
+             query scope above. Fits-in-GPU and context size are additionally
+             one control group - the context size is the length the VRAM
+             estimate is computed at, and that estimate is exactly what the
+             fits filter tests against.
+          Each band owns its container, so how many chips happen to wrap at a
+          given width can no longer decide where the other controls land. */}
+      <div className="filter-bar-group models-filters">
+        <div className="filter-bar-group__row models-filters__query">
+          <div className="search-bar filter-bar-group__search">
+            <i className="fas fa-search search-icon" aria-hidden="true" />
+            <input
+              className="input"
+              type="text"
+              placeholder={t('search.placeholder')}
+              aria-label={t('search.placeholder')}
+              value={search}
+              onChange={(e) => handleSearch(e.target.value)}
+            />
+          </div>
+          {allBackends.length > 0 && (
+            <div className="models-filters__backend">
+              <SearchableSelect
+                value={backendFilter}
+                onChange={(v) => { setBackendFilter(v); setPage(1) }}
+                options={allBackends}
+                placeholder={t('filters.allBackends')}
+                allOption={t('filters.allBackends')}
+                searchPlaceholder={t('filters.searchBackends')}
+              />
+            </div>
+          )}
+        </div>
 
-      {/* Filter buttons */}
-      <div className="filter-bar">
-        {FILTERS.map(f => {
-          const isAll = f.key === ''
-          const active = isAll ? filters.length === 0 : filters.includes(f.key)
-          const available = isFilterAvailable(f.key)
-          return (
-            <button
-              key={f.key}
-              className={`filter-btn ${active ? 'active' : ''}`}
-              disabled={!available}
-              style={!available ? { opacity: 0.4, cursor: 'not-allowed' } : undefined}
-              onClick={() => toggleFilter(f.key)}
-            >
-              <i className={`fas ${f.icon}`} style={{ marginRight: 4 }} />
-              {t(f.labelKey)}
-            </button>
-          )
-        })}
-        {totalGpuMemory > 0 && (
-          <label className="filter-bar-group__toggle" style={{ marginLeft: 'auto' }}>
-            <Toggle checked={fitsFilter} onChange={setFitsFilter} />
-            <i className="fas fa-microchip" />
-            <span>{t('filters.fitsGpu')}</span>
+        <div className="filter-bar" role="group" aria-label={t('filters.useCaseLabel')}>
+          {FILTERS.map(f => {
+            const isAll = f.key === ''
+            const active = isAll ? filters.length === 0 : filters.includes(f.key)
+            const available = isFilterAvailable(f.key)
+            return (
+              <button
+                key={f.key}
+                type="button"
+                className={`filter-btn ${active ? 'active' : ''}`}
+                disabled={!available}
+                aria-pressed={active}
+                title={!available ? t('filters.unavailableForBackend') : undefined}
+                onClick={() => toggleFilter(f.key)}
+              >
+                <i className={`fas ${f.icon}`} aria-hidden="true" style={{ marginRight: 4 }} />
+                {t(f.labelKey)}
+              </button>
+            )
+          })}
+        </div>
+
+        <div className="models-filters__refine" data-testid="models-filters-refine">
+          {/* Leads the band because it decides how many rows the other two
+              refine over, and because unlike fits-in-GPU it is always present:
+              a host with no GPU still browses builds. Turning it off is the
+              only way to page through every build the gallery holds; searching
+              reaches a specific one but cannot enumerate them. */}
+          <label className="filter-bar-group__toggle" data-testid="models-collapse-variants">
+            <Toggle
+              checked={collapseVariants}
+              onChange={(v) => { setCollapseVariants(v); setPage(1) }}
+            />
+            <i className="fas fa-layer-group" aria-hidden="true" />
+            <span>{t('filters.collapseVariants')}</span>
           </label>
-        )}
-        {allBackends.length > 0 && (
-          <SearchableSelect
-            value={backendFilter}
-            onChange={(v) => { setBackendFilter(v); setPage(1) }}
-            options={allBackends}
-            placeholder={t('filters.allBackends')}
-            allOption={t('filters.allBackends')}
-            searchPlaceholder={t('filters.searchBackends')}
-            style={totalGpuMemory > 0 ? undefined : { marginLeft: 'auto' }}
-          />
-        )}
-      </div>
-
-      {/* Context size slider for VRAM estimates */}
-      <div style={{ display: 'flex', alignItems: 'center', gap: 'var(--spacing-sm)', marginBottom: 'var(--spacing-md)', fontSize: '0.8125rem' }}>
-        <label style={{ color: 'var(--color-text-muted)', whiteSpace: 'nowrap' }}>
-          <i className="fas fa-memory" style={{ marginRight: 4 }} />
-          Context:
-        </label>
-        <input
-          type="range"
-          min={0}
-          max={CONTEXT_SIZES.length - 1}
-          value={CONTEXT_SIZES.indexOf(contextSize)}
-          onChange={(e) => setContextSize(CONTEXT_SIZES[e.target.value])}
-          style={{ width: 140, accentColor: 'var(--color-primary)' }}
-        />
-        <span style={{ fontWeight: 600, minWidth: '3em' }}>
-          {CONTEXT_LABELS[CONTEXT_SIZES.indexOf(contextSize)]}
-        </span>
+          {totalGpuMemory > 0 && (
+            <label className="filter-bar-group__toggle">
+              <Toggle checked={fitsFilter} onChange={setFitsFilter} />
+              <i className="fas fa-microchip" aria-hidden="true" />
+              <span>{t('filters.fitsGpu')}</span>
+            </label>
+          )}
+          <div className="models-filters__context">
+            <label htmlFor="models-context-size">
+              <i className="fas fa-memory" aria-hidden="true" />
+              {t('filters.contextSize')}
+            </label>
+            <input
+              id="models-context-size"
+              type="range"
+              min={0}
+              max={CONTEXT_SIZES.length - 1}
+              value={CONTEXT_SIZES.indexOf(contextSize)}
+              // The slider steps over an index, so the raw value ("2") is
+              // meaningless to a screen reader; announce the size instead.
+              aria-valuetext={CONTEXT_LABELS[CONTEXT_SIZES.indexOf(contextSize)]}
+              onChange={(e) => setContextSize(CONTEXT_SIZES[e.target.value])}
+            />
+            <span className="models-filters__context-value">
+              {CONTEXT_LABELS[CONTEXT_SIZES.indexOf(contextSize)]}
+            </span>
+          </div>
+        </div>
       </div>
 
       {/* Table */}
@@ -382,12 +532,23 @@ export default function Models() {
           <div className="empty-state-icon"><i className="fas fa-search" /></div>
           <h2 className="empty-state-title">{t('empty.title')}</h2>
           <p className="empty-state-text">
-            {search || filters.length > 0 || backendFilter || fitsFilter ? t('empty.withFilters') : t('empty.noFilters')}
+            {search || filters.length > 0 || backendFilter || fitsFilter || !collapseVariants ? t('empty.withFilters') : t('empty.noFilters')}
           </p>
-          {(search || filters.length > 0 || backendFilter || fitsFilter) && (
+          {/* Only the fits filter can leave the collapse to blame. The term,
+              the chips and the backend are applied server-side over every build
+              the gallery holds, and a match there is always reported as some
+              row, so those three can no longer come back empty on account of
+              the collapse. Fits runs here in the browser, after the server
+              substituted a matching build for the entry that offers it, and
+              judges that entry's own size: the build that fits can still be
+              filtered out along with a parent that does not. */}
+          {collapseVariants && fitsFilter && (
+            <p className="empty-state-hint">{t('empty.collapsedVariantsHint')}</p>
+          )}
+          {(search || filters.length > 0 || backendFilter || fitsFilter || !collapseVariants) && (
             <button
               className="btn btn-secondary btn-sm"
-              onClick={() => { handleSearch(''); setFilters([]); setBackendFilter(''); setFitsFilter(false); setPage(1) }}
+              onClick={() => { handleSearch(''); setFilters([]); setBackendFilter(''); setFitsFilter(false); setCollapseVariants(COLLAPSE_VARIANTS_DEFAULT); setPage(1) }}
             >
               <i className="fas fa-times" /> {t('search.clearFilters')}
             </button>
@@ -423,11 +584,16 @@ export default function Models() {
                   const progress = getOperationProgress(name)
                   const fit = fitsGpu(vramBytes)
                   const isExpanded = expandedRow === idx
+                  const hasVariants = !!model.has_variants
 
                   return (
                     <React.Fragment key={name}>
                     <tr
-                      onClick={() => { setExpandedRow(isExpanded ? null : idx); setExpandedFiles(false) }}
+                      onClick={() => {
+                        if (!isExpanded && hasVariants) loadVariants(name)
+                        setExpandedRow(isExpanded ? null : idx)
+                        setExpandedFiles(false)
+                      }}
                       style={{ cursor: 'pointer' }}
                     >
                       {/* Chevron */}
@@ -466,12 +632,19 @@ export default function Models() {
 
                       {/* Description */}
                       <td>
-                        <div style={{
-                          fontSize: '0.8125rem', color: 'var(--color-text-secondary)',
-                          maxWidth: '200px', overflow: 'hidden', textOverflow: 'ellipsis', whiteSpace: 'nowrap',
-                        }} title={model.description}>
-                          {model.description || '—'}
-                        </div>
+                        {(() => {
+                          // Gallery descriptions are Markdown. This cell is a single
+                          // truncated line, so it gets the text without the syntax.
+                          const desc = stripMarkdown(model.description)
+                          return (
+                            <div style={{
+                              fontSize: '0.8125rem', color: 'var(--color-text-secondary)',
+                              maxWidth: '200px', overflow: 'hidden', textOverflow: 'ellipsis', whiteSpace: 'nowrap',
+                            }} title={desc}>
+                              {desc || '—'}
+                            </div>
+                          )
+                        })()}
                       </td>
 
                       {/* Backend */}
@@ -553,6 +726,37 @@ export default function Models() {
                                 <i className="fas fa-trash" />
                               </button>
                             </>
+                          ) : hasVariants ? (
+                            // Split button: the primary keeps installing the
+                            // auto-selected build, so the default path is
+                            // unchanged. The chevron is the deliberate
+                            // override.
+                            <div style={{ display: 'inline-flex' }}>
+                              <button
+                                className="btn btn-primary btn-sm"
+                                onClick={() => handleInstall(name)}
+                                disabled={installing}
+                                title={t('actions.install')}
+                                style={{ borderTopRightRadius: 0, borderBottomRightRadius: 0 }}
+                              >
+                                <i className="fas fa-download" />
+                              </button>
+                              <button
+                                ref={variantMenuFor === idx ? variantMenuAnchorRef : undefined}
+                                className="btn btn-primary btn-sm"
+                                onClick={() => {
+                                  if (variantMenuFor !== idx) loadVariants(name)
+                                  setVariantMenuFor(variantMenuFor === idx ? null : idx)
+                                }}
+                                aria-haspopup="menu"
+                                aria-expanded={variantMenuFor === idx}
+                                aria-label={t('variants.chooseVariant')}
+                                disabled={installing}
+                                style={{ padding: '0 8px', borderLeft: '1px solid rgba(0,0,0,0.15)', borderTopLeftRadius: 0, borderBottomLeftRadius: 0 }}
+                              >
+                                <i className={`fas fa-chevron-${variantMenuFor === idx ? 'up' : 'down'}`} style={{ fontSize: '0.6875rem' }} />
+                              </button>
+                            </div>
                           ) : (
                             <button
                               className="btn btn-primary btn-sm"
@@ -570,7 +774,7 @@ export default function Models() {
                     {isExpanded && (
                       <tr>
                         <td colSpan="8" style={{ padding: 0 }}>
-                          <ModelDetail model={model} fit={fit} sizeDisplay={sizeDisplay} vramDisplay={vramDisplay} expandedFiles={expandedFiles} setExpandedFiles={setExpandedFiles} t={t} />
+                          <ModelDetail model={model} fit={fit} sizeDisplay={sizeDisplay} vramDisplay={vramDisplay} expandedFiles={expandedFiles} setExpandedFiles={setExpandedFiles} variantData={hasVariants ? variantData[name] : null} variantDetails={variantDetails} onLoadVariantDetail={loadVariantDetail} installing={installing} onInstall={handleInstall} t={t} />
                         </td>
                       </tr>
                     )}
@@ -605,10 +809,117 @@ export default function Models() {
         onConfirm={confirmDialog?.onConfirm}
         onCancel={() => setConfirmDialog(null)}
       />
+
+      <VariantMenu
+        anchor={variantMenuAnchorRef}
+        model={variantMenuFor !== null ? visibleModels[variantMenuFor] : null}
+        variantData={variantData}
+        onClose={() => setVariantMenuFor(null)}
+        onChoose={handleInstall}
+        t={t}
+      />
     </div>
   )
 }
 
+// VariantMenu is the split-button dropdown. It is one instance re-anchored to
+// whichever row is active, so Escape, outside-click and focus return come from
+// Popover rather than being reimplemented per row.
+function VariantMenu({ anchor, model, variantData, onClose, onChoose, t }) {
+  const name = model ? (model.name || model.id) : null
+  const data = name ? variantData[name] : null
+  return (
+    <Popover
+      anchor={anchor}
+      open={!!model}
+      onClose={onClose}
+      ariaLabel={t('variants.chooseVariant')}
+    >
+      <div className="action-menu">
+        {data?.loading && (
+          // The description is a round trip, so the menu says so rather than
+          // opening empty and looking broken.
+          <div className="action-menu__item" style={{ color: 'var(--color-text-muted)', cursor: 'default' }}>
+            <i className="fas fa-spinner fa-spin action-menu__icon" />
+            <span>{t('variants.loading')}</span>
+          </div>
+        )}
+        {(data?.variants || []).map(v => {
+          const isAuto = v.model === data?.auto_selected
+          return (
+            <button
+              key={v.model}
+              type="button"
+              className="action-menu__item"
+              onClick={() => {
+                onClose()
+                if (name) onChoose(name, v.model)
+              }}
+              // A variant that does not fit stays selectable: an explicit
+              // choice is an override the server honours with a warning.
+              style={{ opacity: v.fits ? 1 : 0.6 }}
+            >
+              <i className={`fas ${isAuto ? 'fa-circle-check' : 'fa-download'} action-menu__icon`} />
+              <span style={{ display: 'flex', flexDirection: 'column', alignItems: 'flex-start', gap: 2 }}>
+                <span>
+                  {v.model}
+                  {isAuto && <span className="badge badge-success" style={{ fontSize: '0.625rem', marginLeft: 6 }}>{t('variants.auto')}</span>}
+                  {v.is_base && <span className="badge badge-info" style={{ fontSize: '0.625rem', marginLeft: 6 }}>{t('variants.base')}</span>}
+                  {!v.fits && <span className="badge badge-warning" style={{ fontSize: '0.625rem', marginLeft: 6 }}>{t('variants.doesNotFit')}</span>}
+                  {/* The bare token, not the spelled-out name: a dropdown item
+                      has room for a marker, and the sentence explaining what
+                      the marker means belongs in the detail row. */}
+                  {(v.features || []).map(f => (
+                    <span
+                      key={f}
+                      className="badge badge-info"
+                      style={{ fontSize: '0.625rem', marginLeft: 6 }}
+                      title={variantFeatureLabel(f, t)}
+                    >
+                      {f.toUpperCase()}
+                    </span>
+                  ))}
+                </span>
+                {/* Quantization joins backend and size on the one meta line
+                    because it is what most often separates two rows here:
+                    builds of the same model routinely share a backend and land
+                    within a gigabyte of each other, so without it the line
+                    describes nothing the user is actually choosing between.
+                    Joined by filter so a build that names no weight format
+                    drops the segment and its separator rather than rendering a
+                    dangling dot. */}
+                <span style={{ fontSize: '0.6875rem', color: 'var(--color-text-muted)' }}>
+                  {[v.backend || t('variants.unknownBackend'), v.quantization, variantSizeLabel(v, t)]
+                    .filter(Boolean)
+                    .join(' · ')}
+                </span>
+              </span>
+            </button>
+          )
+        })}
+      </div>
+    </Popover>
+  )
+}
+
+// variantSizeLabel renders a variant footprint. memory_bytes is omitempty on
+// the wire, so an absent key means the probe could not determine a size; it
+// must never render as "0 B", which would read as "needs nothing".
+function variantSizeLabel(variant, t) {
+  return variant?.memory_bytes ? formatBytes(variant.memory_bytes) : t('variants.unknownSize')
+}
+
+// variantFeatureLabel spells out a serving feature.
+//
+// The vocabulary is short and curated server-side, so each token has a real
+// translated name. An unrecognised one still renders as its uppercased token
+// rather than being dropped: the server's list can grow ahead of the locale
+// files, and a missing string is a worse outcome than an untranslated one when
+// the alternative is silently hiding a genuine reason to pick a build.
+function variantFeatureLabel(feature, t) {
+  return t(`variants.features.${feature}`, { defaultValue: feature.toUpperCase() })
+}
+
 function DetailRow({ label, children }) {
   if (!children) return null
   return (
@@ -621,17 +932,58 @@ function DetailRow({ label, children }) {
   )
 }
 
-function ModelDetail({ model, fit, sizeDisplay, vramDisplay, expandedFiles, setExpandedFiles, t }) {
-  const files = model.additionalFiles || model.files || []
+// VariantDetailPanel is the same detail view a top-level row gets, rendered for
+// one variant.
+//
+// It reuses ModelDetail rather than restating what an entry looks like, so a
+// field added to the detail view appears here too. variantData is deliberately
+// withheld: a variant's own entry may declare variants of its own, and
+// recursing would nest a picker inside a picker two levels deep already. The
+// file disclosure gets its own state here because each panel opens and closes
+// independently of the parent's.
+function VariantDetailPanel({ model, t }) {
+  const [expandedFiles, setExpandedFiles] = useState(false)
   return (
-    <div style={{ padding: 'var(--spacing-md) var(--spacing-lg)', background: 'var(--color-bg-primary)', borderTop: '1px solid var(--color-border-subtle)' }}>
+    <ModelDetail
+      model={model}
+      nested
+      expandedFiles={expandedFiles}
+      setExpandedFiles={setExpandedFiles}
+      variantData={null}
+      t={t}
+    />
+  )
+}
+
+function ModelDetail({ model, fit, sizeDisplay, vramDisplay, expandedFiles, setExpandedFiles, variantData, variantDetails, onLoadVariantDetail, installing, onInstall, nested, t }) {
+  const files = model.additionalFiles || model.files || []
+  const name = model.name || model.id
+  // Which variant has its details revealed, or null. One at a time: the list is
+  // a comparison, and two open panels push the rows being compared apart.
+  const [openVariant, setOpenVariant] = useState(null)
+  // Escape returns focus to the control that opened the panel, so dismissing by
+  // keyboard does not drop the user back at the top of the document.
+  const infoRefs = useRef({})
+  return (
+    <div style={{
+      padding: 'var(--spacing-md) var(--spacing-lg)',
+      background: nested ? 'transparent' : 'var(--color-bg-primary)',
+      borderTop: nested ? 'none' : '1px solid var(--color-border-subtle)',
+    }}>
+      {model.description && (
+        // Prose sits outside the label/value table: an eight-line value cell
+        // in a grid of one-line ones breaks the rhythm exactly where the eye
+        // enters, and the full pane width is roughly double a readable measure.
+        <div className="detail-prose">
+          <div className="detail-prose__label">{t('detail.description')}</div>
+          <div
+            className="markdown-body detail-prose__body"
+            dangerouslySetInnerHTML={{ __html: renderMarkdown(model.description) }}
+          />
+        </div>
+      )}
       <table style={{ width: '100%', borderCollapse: 'collapse' }}>
         <tbody>
-          <DetailRow label={t('detail.description')}>
-            {model.description && (
-              <span style={{ color: 'var(--color-text-secondary)', lineHeight: 1.6 }}>{model.description}</span>
-            )}
-          </DetailRow>
           <DetailRow label={t('detail.gallery')}>
             {model.gallery && (
               <span className="badge badge-info" style={{ fontSize: '0.6875rem' }}>
@@ -661,6 +1013,136 @@ function ModelDetail({ model, fit, sizeDisplay, vramDisplay, expandedFiles, setE
               </span>
             ) : null}
           </DetailRow>
+          {variantData?.loading && (
+            <DetailRow label={t('variants.title')}>
+              <span style={{ color: 'var(--color-text-muted)' }}>
+                <i className="fas fa-spinner fa-spin" style={{ marginRight: 6 }} />{t('variants.loading')}
+              </span>
+            </DetailRow>
+          )}
+          {variantData?.variants?.length > 0 && (
+            <DetailRow label={t('variants.title')}>
+              <div className="variant-list">
+                {variantData.variants.map(v => {
+                  const isAuto = v.model === variantData.auto_selected
+                  const detail = variantDetails?.[v.model]
+                  const detailOpen = openVariant === v.model
+                  const panelId = `variant-detail-${v.model}`
+                  return (
+                    <div
+                      key={v.model}
+                      className="variant-entry"
+                      onKeyDown={(e) => {
+                        if (e.key !== 'Escape' || !detailOpen) return
+                        // Stops the row's own expansion, and any dialog above
+                        // it, from also closing on the same keystroke.
+                        e.stopPropagation()
+                        setOpenVariant(null)
+                        infoRefs.current[v.model]?.focus()
+                      }}
+                    >
+                    {/* A separate control, not a region of the install button:
+                        nesting it would be invalid markup and, worse, would
+                        make "tell me more" a click on "install this". It leads
+                        the row because it acts on the name that follows it. */}
+                    <button
+                      type="button"
+                      ref={(el) => { infoRefs.current[v.model] = el }}
+                      className="variant-row__info"
+                      aria-expanded={detailOpen}
+                      aria-controls={detailOpen ? panelId : undefined}
+                      // Named after the build it describes: a column of
+                      // identical "Details" buttons tells a screen reader
+                      // user nothing about which row they are on.
+                      aria-label={detailOpen
+                        ? t('variants.hideDetails', { variant: v.model })
+                        : t('variants.showDetails', { variant: v.model })}
+                      onClick={(e) => {
+                        e.stopPropagation()
+                        if (detailOpen) { setOpenVariant(null); return }
+                        setOpenVariant(v.model)
+                        onLoadVariantDetail?.(v.model)
+                      }}
+                    >
+                      <i className="fas fa-circle-info" aria-hidden="true" />
+                    </button>
+                    {/* Listing the alternatives without offering them made the
+                        detail view read as a menu that could not be ordered
+                        from; installing one is the same call the split-button
+                        chevron already makes. */}
+                    <button
+                      type="button"
+                      className={`variant-row${v.fits ? '' : ' variant-row--unfit'}`}
+                      disabled={installing}
+                      aria-label={t('variants.installVariant', { variant: v.model })}
+                      onClick={(e) => { e.stopPropagation(); onInstall(name, v.model) }}
+                    >
+                      <span className="variant-row__name">{v.model}</span>
+                      <span className="variant-row__backend">{v.backend || t('variants.unknownBackend')}</span>
+                      {/* Its own column rather than appended to the backend
+                          cell, so precision lines up down the list and two
+                          builds can be compared by scanning rather than by
+                          reading. An entry naming no weight format says so:
+                          an empty cell in an aligned column reads as a
+                          rendering fault. */}
+                      <span
+                        className={`variant-row__quant${v.quantization ? '' : ' variant-row__quant--unknown'}`}
+                        title={t('variants.quantizationTitle')}
+                      >
+                        {v.quantization || t('variants.unknownQuantization')}
+                      </span>
+                      <span className="variant-row__size">{variantSizeLabel(v, t)}</span>
+                      <span className="variant-row__status">
+                        {isAuto && (
+                          <span className="badge badge-success">
+                            <i className="fas fa-circle-check" /> {t('variants.autoSelected')}
+                          </span>
+                        )}
+                        {!v.fits && <span className="badge badge-warning">{t('variants.doesNotFit')}</span>}
+                        {v.is_base && !isAuto && <span className="badge badge-info">{t('variants.base')}</span>}
+                        {/* The room the detail row has over the dropdown is
+                            spent here: "DFLASH" names nothing to a user who
+                            has not met it, whereas the spelled-out feature
+                            says why this build is worth choosing. */}
+                        {(v.features || []).map(f => (
+                          <span key={f} className="badge badge-info">
+                            <i className="fas fa-bolt" aria-hidden="true" /> {variantFeatureLabel(f, t)}
+                          </span>
+                        ))}
+                      </span>
+                      <i className="fas fa-download variant-row__action" aria-hidden="true" />
+                    </button>
+                    {detailOpen && (
+                      // An inline disclosure rather than a modal. This is
+                      // already inside an expanded row, and a dialog opened
+                      // from there stacks a dismissal on top of a dismissal
+                      // for what is a few more lines of the same entry. The
+                      // rule and inset carry the third level instead.
+                      <div className="variant-detail" id={panelId}>
+                        {(!detail || detail.loading) && (
+                          <div className="variant-detail__state">
+                            <i className="fas fa-spinner fa-spin" aria-hidden="true" />
+                            <span>{t('variants.detailsLoading')}</span>
+                          </div>
+                        )}
+                        {detail?.error && (
+                          // Stated, not blank: an empty panel reads as a
+                          // rendering fault rather than as a lookup that
+                          // failed.
+                          <div className="variant-detail__state variant-detail__state--error" role="status">
+                            <i className="fas fa-triangle-exclamation" aria-hidden="true" />
+                            <span>{t('variants.detailsUnavailable', { variant: v.model })}</span>
+                          </div>
+                        )}
+                        {detail?.entry && <VariantDetailPanel model={detail.entry} t={t} />}
+                      </div>
+                    )}
+                    </div>
+                  )
+                })}
+              </div>
+            </DetailRow>
+          )}
           <DetailRow label={t('detail.license')}>
             {model.license && <span>{model.license}</span>}
           </DetailRow>

core/http/react-ui/src/pages/VoiceLibrary.jsx

@@ -14,6 +14,7 @@ import { useVoiceCloningGallery, useVoiceProfiles } from '../hooks/useVoiceProfi
 import { CAP_TTS } from '../utils/capabilities'
 import { modelsApi, voiceProfilesApi } from '../utils/api'
 import { copyToClipboard } from '../utils/clipboard'
+import { renderMarkdown } from '../utils/markdown'
 
 const ROW_BARS = [35, 58, 78, 44, 68, 92, 55, 72, 38, 82, 64, 46, 74, 52, 88, 42, 66, 48]
 
@@ -317,7 +318,9 @@ JSON` : ''
                 <div>
                   <span className="voice-detail__eyebrow">{t('voiceLibrary.detail.eyebrow')}</span>
                   <h2>{selected.name}</h2>
-                  {selected.description && <p>{selected.description}</p>}
+                  {selected.description && (
+                    <div className="markdown-body" dangerouslySetInnerHTML={{ __html: renderMarkdown(selected.description) }} />
+                  )}
                 </div>
                 <StatusPill status="healthy" label={t('voiceLibrary.status.ready')} />
               </div>

core/http/react-ui/src/utils/api.js

@@ -85,12 +85,22 @@ export const modelsApi = {
   listV1: () => fetchJSON(API_CONFIG.endpoints.modelsList),
   listCapabilities: () => fetchJSON(API_CONFIG.endpoints.modelsCapabilities),
   listAliases: () => fetchJSON(API_CONFIG.endpoints.modelsAliases),
-  install: (id) => postJSON(API_CONFIG.endpoints.installModel(id), {}),
+  // variant is optional. Omitting it lets the server auto-select the best
+  // build for this host, which is what the listing's auto_variant predicted.
+  install: (id, variant) => postJSON(
+    variant
+      ? `${API_CONFIG.endpoints.installModel(id)}?variant=${encodeURIComponent(variant)}`
+      : API_CONFIG.endpoints.installModel(id),
+    {}
+  ),
   delete: (id) => postJSON(API_CONFIG.endpoints.deleteModel(id), {}),
   estimate: (id, contexts) => fetchJSON(
     buildUrl(API_CONFIG.endpoints.modelEstimate(id),
       contexts?.length ? { contexts: contexts.join(',') } : {})
   ),
+  // Companion to estimate: the listing reports only has_variants, so the
+  // description is fetched per entry, on demand.
+  variants: (id) => fetchJSON(API_CONFIG.endpoints.modelVariants(id)),
   getConfig: (id) => postJSON(API_CONFIG.endpoints.modelConfig(id), {}),
   getConfigJson: (name) => fetchJSON(API_CONFIG.endpoints.modelConfigJson(name)),
   getJob: (uid) => fetchJSON(API_CONFIG.endpoints.modelJob(uid)),

core/http/react-ui/src/utils/config.js

@@ -10,6 +10,7 @@ export const API_CONFIG = {
     installModel: (id) => `/api/models/install/${id}`,
     deleteModel: (id) => `/api/models/delete/${id}`,
     modelEstimate: (id) => `/api/models/estimate/${id}`,
+    modelVariants: (id) => `/api/models/variants/${id}`,
     modelConfig: (id) => `/api/models/config/${id}`,
     modelConfigJson: (name) => `/api/models/config-json/${name}`,
     configMetadata: '/api/models/config-metadata',

core/http/react-ui/src/utils/markdown.js

@@ -20,6 +20,53 @@ export function renderMarkdown(text) {
   return DOMPurify.sanitize(html)
 }
 
+// Recursively pull the human-readable text out of a marked token, dropping the
+// syntax that carries it. Link/image tokens keep their label and lose the URL;
+// code keeps its literal content; raw HTML is dropped rather than leaked as
+// angle-bracket noise into a table cell.
+function tokenToPlainText(token) {
+  if (!token) return ''
+  switch (token.type) {
+    case 'space':
+    case 'hr':
+    case 'br':
+    case 'html':
+      return ' '
+    case 'code':
+    case 'codespan':
+    case 'image':
+      return token.text || ''
+    case 'list':
+      return (token.items || []).map(tokenToPlainText).join(' ')
+    case 'table':
+      return [...(token.header || []), ...(token.rows || []).flat()]
+        .map(tokenToPlainText)
+        .join(' ')
+    default:
+      break
+  }
+  if (Array.isArray(token.tokens) && token.tokens.length > 0) {
+    return token.tokens.map(tokenToPlainText).join('')
+  }
+  return token.text || ''
+}
+
+// Reduce Markdown to a single line of readable plain text. Used by the
+// truncated one-line description cells and their `title` tooltips, where
+// rendering real Markdown would turn a leading `#` into an <h1> and wreck the
+// row rhythm. Output lands in JSX text nodes, so React escapes it.
+export function stripMarkdown(text) {
+  if (!text) return ''
+  let plain
+  try {
+    plain = marked.lexer(text).map(tokenToPlainText).join(' ')
+  } catch {
+    // A malformed gallery description must never break the row it labels.
+    plain = text
+  }
+  return plain.replace(/\s+/g, ' ').trim()
+}
+
 export function highlightAll(element) {
   if (!element) return
   element.querySelectorAll('pre code').forEach((block) => {

core/http/routes/ui_api.go

@@ -398,7 +398,11 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 
 	// Model Gallery APIs (admin only)
 	app.GET("/api/models", func(c echo.Context) error {
-		term := c.QueryParam("term")
+		// Trimmed once, here, so "is the user searching?" has a single answer
+		// for both the search itself and the variant collapse below.
+		// Whitespace is not a lookup: an untrimmed blank term used to narrow
+		// the listing to whatever happened to contain a space.
+		term := strings.TrimSpace(c.QueryParam("term"))
 		tag := c.QueryParam("tag")
 		page := c.QueryParam("page")
 		if page == "" {
@@ -416,6 +420,11 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 				"error": err.Error(),
 			})
 		}
+		// The filters below rebind models, so keep the unfiltered gallery for
+		// the questions that are about the gallery as a whole rather than about
+		// the current result set, such as which entries another entry already
+		// offers as a variant.
+		allModels := models
 
 		// Get all available tags
 		allTags := map[string]struct{}{}
@@ -484,6 +493,81 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 			models = filtered
 		}
 
+		// Collapse the listing to one row per model: report every match at the
+		// entry installable in its own right, so an individual build a parent
+		// already offers as a variant is reported as that parent rather than as
+		// a row of its own. What survives is the deduplicated gallery, rather
+		// than one row per quantization.
+		//
+		// Off by default so the response with the parameter absent is exactly
+		// what it was.
+		//
+		// The parent map is computed over the whole gallery rather than over
+		// what the other filters left, so an entry is grouped because a parent
+		// offers it, never because of what the user searched or picked.
+		//
+		// A hidden build is substituted by the entry that offers it rather than
+		// simply dropped. Dropping was sufficient while nothing could narrow the
+		// listing: every parent was present, so a hidden build always had its
+		// parent on screen anyway. Once a term can remove the parent, dropping
+		// answers "no models found" for a build the gallery does hold, which
+		// reads as "that model does not exist". Substituting keeps the promise
+		// of the grouped view instead: searching sees every build, and a match
+		// is reported at the row the user can act on.
+		//
+		// Deliberately after search, tag and backend, so every filter is judged
+		// against the build that really carries the name, tag or backend, never
+		// against a parent that merely offers it. Substituting first would let a
+		// backend filter match a parent whose own backend is something else. The
+		// price is that the surfaced row shows the parent's own metadata while
+		// the match was on one of its variants, which is what grouping means.
+		//
+		// A parent already in the result keeps its own position and absorbs its
+		// matching variants there, so the browsing listing is ordered exactly as
+		// it was; a parent surfaced only by a variant takes the position of the
+		// first variant that surfaced it. Either way it appears exactly once.
+		//
+		// term is already trimmed, so a stray space in the search box does not
+		// count as a search and cannot silently widen the match set.
+		//
+		// Server-side because the listing paginates at 9 items; filtering the
+		// current page in the client would leave the page count describing the
+		// unfiltered set and hand the user empty pages. Substituting here, above
+		// the totals, is what keeps the count and the page math describing the
+		// set the user is actually handed.
+		if c.QueryParam("collapse_variants") == "true" {
+			parents := gallery.VariantParents(allModels)
+			present := make(map[string]struct{}, len(models))
+			for _, m := range models {
+				present[m.ID()] = struct{}{}
+			}
+			collapsed := make(gallery.GalleryElements[*gallery.GalleryModel], 0, len(models))
+			surfaced := make(map[string]struct{}, len(models))
+			for _, m := range models {
+				row := m
+				if parent, hidden := parents[m.ID()]; hidden {
+					// The parent is in the result on its own merits and will be
+					// emitted at its own position, so dropping the variant here
+					// is what keeps that position.
+					if _, parentMatched := present[parent.ID()]; parentMatched {
+						continue
+					}
+					// One hop only. VariantParents never reports an entry that
+					// declares variants, so a parent is never itself hidden and
+					// a second hop cannot be needed; refusing to take one anyway
+					// is what makes a malformed gallery terminate rather than
+					// loop.
+					row = parent
+				}
+				if _, dup := surfaced[row.ID()]; dup {
+					continue
+				}
+				surfaced[row.ID()] = struct{}{}
+				collapsed = append(collapsed, row)
+			}
+			models = collapsed
+		}
+
 		// Capability filters are derived from the effective gallery model
 		// configuration. In particular, voice cloning is variant-sensitive, so
 		// filtering by the TTS usecase or backend name alone would advertise
@@ -589,6 +673,20 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 				"voice_cloning":   m.VoiceCloningCapability(appConfig.SystemState.Model.ModelsPath),
 			}
 
+			// Only the cheap declaration flag, never the description itself.
+			// Describing a variant probes every referenced entry's weight files
+			// over the network, so doing it here would cost one page load
+			// (entries x variants) serial round trips; a client that wants the
+			// description asks /api/models/variants/:id for one entry at a
+			// time, exactly as it already does for VRAM estimates.
+			//
+			// The flag is omitted rather than sent as false so an entry that
+			// declares nothing stays byte-for-byte what it was before variants
+			// existed, and so a client never asks about it.
+			if m.HasVariants() {
+				obj["has_variants"] = true
+			}
+
 			modelsJSON = append(modelsJSON, obj)
 		}
 
@@ -804,6 +902,53 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 		return c.JSON(200, result)
 	}, adminMiddleware)
 
+	// Returns the selectable builds of a single gallery model and which one
+	// auto-selection would install on this host. Companion to estimate/:id and
+	// for the same reason: describing variants probes each referenced entry's
+	// weight files over the network, so the gallery listing must stay free of
+	// it and the frontend fills pickers in per-entry, on demand.
+	//
+	// An entry that declares no variants is not an error; it answers with an
+	// empty description, so a client that asks anyway gets a well-formed reply
+	// rather than a 404 it has to special-case.
+	app.GET("/api/models/variants/:id", func(c echo.Context) error {
+		modelID, err := url.QueryUnescape(c.Param("id"))
+		if err != nil {
+			return c.JSON(http.StatusBadRequest, map[string]any{"error": "invalid model ID"})
+		}
+
+		models, err := gallery.AvailableGalleryModelsCached(appConfig.Galleries, appConfig.SystemState)
+		if err != nil {
+			return c.JSON(http.StatusInternalServerError, map[string]any{"error": err.Error()})
+		}
+
+		model := gallery.FindGalleryElement(models, modelID)
+		if model == nil {
+			return c.JSON(http.StatusNotFound, map[string]any{"error": "model not found"})
+		}
+
+		// An allocated slice rather than the zero value, so "nothing
+		// selectable" serializes as [] and a client can iterate the reply
+		// without first distinguishing it from null.
+		empty := gallery.EntryVariants{Variants: []gallery.VariantView{}}
+
+		// The full, unpaginated list: a variant references another gallery
+		// entry by name and that entry need not be anywhere near this one.
+		env := gallery.HostResolveEnv(c.Request().Context(), appConfig.SystemState)
+		view, err := gallery.DescribeVariants(models, model, env)
+		if err != nil {
+			// A malformed variant list must not break the picker; the entry
+			// just reports nothing selectable and installs as-is.
+			xlog.Debug("could not describe model variants", "model", modelID, "error", err)
+			return c.JSON(200, empty)
+		}
+		if view == nil {
+			return c.JSON(200, empty)
+		}
+
+		return c.JSON(200, view)
+	}, adminMiddleware)
+
 	app.POST("/api/models/install/:id", func(c echo.Context) error {
 		galleryID := c.Param("id")
 		// URL decode the gallery ID (e.g., "localai%40model" -> "localai@model")
@@ -813,7 +958,11 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 				"error": "invalid model ID",
 			})
 		}
-		xlog.Debug("API job submitted to install", "galleryID", galleryID)
+		// Optional: one of the variants the listing reported for this entry.
+		// Absent means auto-select, which is what the listing's auto_variant
+		// already told the client would happen.
+		variant := c.QueryParam("variant")
+		xlog.Debug("API job submitted to install", "galleryID", galleryID, "variant", variant)
 
 		id, err := uuid.NewUUID()
 		if err != nil {
@@ -829,6 +978,7 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 		op := galleryop.ManagementOp[gallery.GalleryModel, gallery.ModelConfig]{
 			ID:                 uid,
 			GalleryElementName: galleryID,
+			Variant:            variant,
 			Galleries:          appConfig.Galleries,
 			BackendGalleries:   appConfig.BackendGalleries,
 			Context:            ctx,

core/http/routes/ui_api_variants_test.go

@@ -0,0 +1,528 @@
+package routes_test
+
+import (
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"sync/atomic"
+
+	"github.com/labstack/echo/v4"
+	"github.com/mudler/LocalAI/core/application"
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/gallery"
+	"github.com/mudler/LocalAI/core/http/routes"
+	"github.com/mudler/LocalAI/core/services/galleryop"
+	"github.com/mudler/LocalAI/pkg/model"
+	"github.com/mudler/LocalAI/pkg/system"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// These specs pin the cost contract of the gallery listing.
+//
+// Variant description probes every referenced entry's weight files over the
+// network. Doing that inline in the listing makes one page load cost
+// (entries x variants) serial round trips, so the listing must not do it at
+// all: it reports only the cheap `has_variants` flag, and a client that wants
+// the description asks for one entry at a time.
+//
+// The probe counter below is what makes that a behavioral assertion rather
+// than a structural one. It counts real HTTP hits on the weight files, so it
+// goes red if DescribeVariants becomes reachable from the listing path again,
+// through any caller.
+var _ = Describe("Model gallery variants API", func() {
+	var (
+		app          *echo.Echo
+		modelsDir    string
+		weightServer *httptest.Server
+		indexServer  *httptest.Server
+		probes       *atomic.Int64
+		appConfig    *config.ApplicationConfig
+	)
+
+	BeforeEach(func() {
+		// The gallery cache is a package global, so a background refresh the
+		// previous spec triggered can otherwise land here and answer with that
+		// spec's now-closed gallery server.
+		gallery.ResetGalleryModelCache()
+
+		var err error
+		modelsDir, err = os.MkdirTemp("", "ui-api-variants-test-*")
+		Expect(err).NotTo(HaveOccurred())
+
+		probes = &atomic.Int64{}
+		// Stands in for the weight files a variant probe would range-fetch.
+		// Every hit is a probe; a listing that describes variants inline
+		// cannot avoid them.
+		weightServer = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			probes.Add(1)
+			w.Header().Set("Content-Length", "1048576")
+			w.WriteHeader(http.StatusOK)
+		}))
+
+		index := fmt.Sprintf(`
+- name: base-entry
+  description: An entry that declares variants
+  backend: llama-cpp
+  files:
+    - filename: base-Q4_K_M.gguf
+      uri: %s/base.gguf
+      sha256: ""
+  variants:
+    - model: big-entry
+    - model: dir-entry
+- name: big-entry
+  description: The alternative build
+  backend: llama-cpp
+  tags:
+    - llm
+    - dflash
+  files:
+    - filename: big-Q8_0.gguf
+      uri: %s/big.gguf
+      sha256: ""
+# A build served from a directory of weights. Its filename names no weight
+# format, which makes it the fixture for the absent-quantization case a client
+# has to render as unknown rather than as a blank cell. Its backend differs
+# from its parent's on purpose: that is what lets the specs pin down which
+# entry a backend filter is judged against once the collapse substitutes.
+- name: dir-entry
+  description: A build whose name declares no weight format
+  backend: vllm
+  files:
+    - filename: weights.safetensors
+      uri: %s/weights.safetensors
+      sha256: ""
+- name: plain-entry
+  description: An entry that declares nothing
+  backend: whisper
+`, weightServer.URL, weightServer.URL, weightServer.URL)
+
+		indexServer = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			_, _ = w.Write([]byte(index))
+		}))
+
+		systemState, err := system.GetSystemState(system.WithModelPath(modelsDir))
+		Expect(err).NotTo(HaveOccurred())
+
+		appConfig = &config.ApplicationConfig{
+			Galleries:   []config.Gallery{{Name: "test", URL: indexServer.URL + "/index.yaml"}},
+			SystemState: systemState,
+		}
+
+		galleryService := galleryop.NewGalleryService(appConfig, nil)
+		app = echo.New()
+		routes.RegisterUIAPIRoutes(app, config.NewModelConfigLoader(modelsDir), model.NewModelLoader(systemState), appConfig,
+			galleryService, galleryop.NewOpCache(galleryService), &application.Application{},
+			func(next echo.HandlerFunc) echo.HandlerFunc { return next })
+	})
+
+	AfterEach(func() {
+		weightServer.Close()
+		indexServer.Close()
+		Expect(os.RemoveAll(modelsDir)).To(Succeed())
+	})
+
+	get := func(path string) (int, map[string]any) {
+		req := httptest.NewRequest(http.MethodGet, path, nil)
+		rec := httptest.NewRecorder()
+		app.ServeHTTP(rec, req)
+		var body map[string]any
+		Expect(json.Unmarshal(rec.Body.Bytes(), &body)).To(Succeed())
+		return rec.Code, body
+	}
+
+	listing := func() []map[string]any {
+		code, body := get("/api/models?items=9999")
+		Expect(code).To(Equal(http.StatusOK))
+		raw, ok := body["models"].([]any)
+		Expect(ok).To(BeTrue(), "listing must return a models array")
+		out := make([]map[string]any, 0, len(raw))
+		for _, m := range raw {
+			out = append(out, m.(map[string]any))
+		}
+		return out
+	}
+
+	find := func(models []map[string]any, name string) map[string]any {
+		for _, m := range models {
+			if m["name"] == name {
+				return m
+			}
+		}
+		Fail("no entry named " + name + " in the listing")
+		return nil
+	}
+
+	Context("the listing", func() {
+		It("issues no variant probes at all", func() {
+			models := listing()
+			Expect(models).NotTo(BeEmpty())
+
+			// The whole point. An entry declaring variants must cost the
+			// listing exactly what an entry declaring none costs it.
+			Expect(probes.Load()).To(BeZero(),
+				"the gallery listing probed variant weight files; variant description must not be reachable from the listing path")
+		})
+
+		It("omits the described variant payload", func() {
+			entry := find(listing(), "base-entry")
+			Expect(entry).NotTo(HaveKey("variants"))
+			Expect(entry).NotTo(HaveKey("auto_variant"))
+		})
+
+		It("reports has_variants so a client knows whether to ask", func() {
+			models := listing()
+			Expect(find(models, "base-entry")["has_variants"]).To(BeTrue())
+			// An entry declaring nothing must look exactly as it did before
+			// variants existed, so a client never asks about it.
+			Expect(find(models, "plain-entry")).NotTo(HaveKey("has_variants"))
+		})
+	})
+
+	Context("the companion endpoint", func() {
+		It("returns the description the listing used to carry", func() {
+			code, body := get("/api/models/variants/test@base-entry")
+			Expect(code).To(Equal(http.StatusOK))
+
+			Expect(body).To(HaveKey("auto_selected"))
+			variants, ok := body["variants"].([]any)
+			Expect(ok).To(BeTrue())
+			Expect(variants).To(HaveLen(3), "the declared variants plus the base")
+
+			byModel := map[string]map[string]any{}
+			for _, v := range variants {
+				vm := v.(map[string]any)
+				byModel[vm["model"].(string)] = vm
+			}
+			Expect(byModel).To(HaveKey("big-entry"))
+			Expect(byModel).To(HaveKey("base-entry"))
+			Expect(byModel["base-entry"]["is_base"]).To(BeTrue())
+			Expect(byModel["big-entry"]["is_base"]).To(BeFalse())
+			Expect(byModel["big-entry"]).To(HaveKey("backend"))
+			Expect(byModel["big-entry"]).To(HaveKey("fits"))
+		})
+
+		It("omits memory_bytes entirely when a size cannot be determined", func() {
+			// The weight server answers without a usable size, so the probe
+			// comes back unknown. An absent key is the contract: a zero would
+			// read as 'needs nothing'.
+			_, body := get("/api/models/variants/test@base-entry")
+			for _, v := range body["variants"].([]any) {
+				vm := v.(map[string]any)
+				if mb, present := vm["memory_bytes"]; present {
+					Expect(mb).NotTo(BeZero(), "memory_bytes must be omitted rather than serialized as zero")
+				}
+			}
+		})
+
+		// Enrichment: name, backend and size are frequently identical across
+		// two builds of one model, so the picker also carries the facts that
+		// actually differ. Both are derived here rather than in the browser, so
+		// every client reads them out of the same entry the installer uses.
+		It("reports each build's quantization", func() {
+			_, body := get("/api/models/variants/test@base-entry")
+
+			byModel := map[string]map[string]any{}
+			for _, v := range body["variants"].([]any) {
+				vm := v.(map[string]any)
+				byModel[vm["model"].(string)] = vm
+			}
+
+			Expect(byModel["base-entry"]["quantization"]).To(Equal("Q4_K_M"))
+			Expect(byModel["big-entry"]["quantization"]).To(Equal("Q8_0"))
+		})
+
+		It("omits quantization entirely when the build names no weight format", func() {
+			// The degrade contract. An absent key is what lets a client render
+			// "unknown format"; an empty string would render as a blank cell
+			// indistinguishable from a rendering bug.
+			_, body := get("/api/models/variants/test@base-entry")
+
+			for _, v := range body["variants"].([]any) {
+				vm := v.(map[string]any)
+				if vm["model"] == "dir-entry" {
+					Expect(vm).NotTo(HaveKey("quantization"))
+					return
+				}
+			}
+			Fail("dir-entry missing from the described variants")
+		})
+
+		It("reports the serving features a build declares", func() {
+			_, body := get("/api/models/variants/test@base-entry")
+
+			byModel := map[string]map[string]any{}
+			for _, v := range body["variants"].([]any) {
+				vm := v.(map[string]any)
+				byModel[vm["model"].(string)] = vm
+			}
+
+			Expect(byModel["big-entry"]["features"]).To(Equal([]any{"dflash"}))
+			// A build declaring no serving feature carries no key at all,
+			// rather than an empty list a client would have to length-check.
+			Expect(byModel["base-entry"]).NotTo(HaveKey("features"))
+			Expect(byModel["dir-entry"]).NotTo(HaveKey("features"))
+		})
+
+		It("returns an empty description for an entry declaring no variants", func() {
+			code, body := get("/api/models/variants/test@plain-entry")
+			Expect(code).To(Equal(http.StatusOK))
+			Expect(body["variants"]).To(BeEmpty())
+			Expect(body["auto_selected"]).To(BeEmpty())
+		})
+
+		It("404s an unknown entry", func() {
+			code, _ := get("/api/models/variants/test@nope")
+			Expect(code).To(Equal(http.StatusNotFound))
+		})
+	})
+
+	// The collapsed view is the deduplicated gallery: every entry installable
+	// in its own right, with nothing shown twice. Off by default, so a client
+	// that never asks for it sees the listing exactly as it was. The web UI
+	// always asks for it; the parameter stays for every other client.
+	Context("the collapse_variants listing filter", func() {
+		names := func(path string) []string {
+			code, body := get(path)
+			Expect(code).To(Equal(http.StatusOK))
+			raw, ok := body["models"].([]any)
+			Expect(ok).To(BeTrue(), "listing must return a models array")
+			out := make([]string, 0, len(raw))
+			for _, m := range raw {
+				out = append(out, m.(map[string]any)["name"].(string))
+			}
+			return out
+		}
+
+		// Everything the collapse parameter can possibly affect: which entries
+		// come back, how they serialize, and the paging that describes them.
+		//
+		// Deliberately NOT the whole response body. The listing envelope also
+		// carries live host telemetry (ramUsed and friends) which drifts
+		// between two calls milliseconds apart, so a byte-for-byte comparison
+		// asserts on the machine's memory pressure rather than on the handler
+		// and fails at random.
+		listingShape := func(path string) map[string]any {
+			req := httptest.NewRequest(http.MethodGet, path, nil)
+			rec := httptest.NewRecorder()
+			app.ServeHTTP(rec, req)
+			Expect(rec.Code).To(Equal(http.StatusOK))
+			var body map[string]any
+			Expect(json.Unmarshal(rec.Body.Bytes(), &body)).To(Succeed())
+
+			shape := map[string]any{}
+			for _, k := range []string{
+				"models", "availableModels", "installedModels",
+				"totalPages", "currentPage", "nextPage", "prevPage",
+				"allBackends", "allTags",
+			} {
+				shape[k] = body[k]
+			}
+			return shape
+		}
+
+		It("returns every entry when off", func() {
+			Expect(names("/api/models?items=9999")).To(ConsistOf("base-entry", "big-entry", "dir-entry", "plain-entry"))
+		})
+
+		It("hides only the builds another entry already offers", func() {
+			// base-entry is a parent and stays. big-entry and dir-entry are the
+			// builds it references, so they drop out: a user reaches them by
+			// installing base-entry. plain-entry is nobody's variant, so it
+			// stays even though it declares none of its own.
+			Expect(names("/api/models?items=9999&collapse_variants=true")).
+				To(ConsistOf("base-entry", "plain-entry"))
+		})
+
+		It("leaves the response untouched for any value other than true", func() {
+			// Default off has to mean off, so an explicit false and an
+			// unparseable value must both behave as absent rather than as a
+			// truthy presence check.
+			base := listingShape("/api/models?items=9999")
+			Expect(listingShape("/api/models?items=9999&collapse_variants=false")).To(Equal(base))
+			Expect(listingShape("/api/models?items=9999&collapse_variants=1")).To(Equal(base))
+		})
+
+		It("serializes a non-declaring entry exactly as it did before", func() {
+			// The whole promise of the migration phase: a client that never
+			// sends the parameter gets byte-for-byte what it got before.
+			entry := find(listing(), "plain-entry")
+			Expect(entry).NotTo(HaveKey("has_variants"))
+			Expect(entry).NotTo(HaveKey("variants"))
+			Expect(entry).NotTo(HaveKey("auto_variant"))
+		})
+
+		It("composes with the backend filter rather than replacing it", func() {
+			// base-entry and big-entry are llama-cpp; plain-entry is whisper.
+			// If either filter overwrote the other, the llama-cpp case would
+			// keep big-entry and the whisper case would lose plain-entry.
+			Expect(names("/api/models?items=9999&collapse_variants=true&backend=llama-cpp")).
+				To(ConsistOf("base-entry"))
+			Expect(names("/api/models?items=9999&collapse_variants=true&backend=whisper")).
+				To(ConsistOf("plain-entry"))
+			Expect(names("/api/models?items=9999&backend=llama-cpp")).
+				To(ConsistOf("base-entry", "big-entry"))
+		})
+
+		It("still applies the search term when nothing is collapsed away", func() {
+			Expect(names("/api/models?items=9999&collapse_variants=true&term=plain")).
+				To(ConsistOf("plain-entry"))
+		})
+
+		It("does not treat an empty or whitespace-only term as a search", func() {
+			// Otherwise a cleared or fat-fingered search box would silently
+			// widen the match set and the browsing view would change under a
+			// user who only brushed the search box.
+			Expect(names("/api/models?items=9999&collapse_variants=true&term=")).
+				To(ConsistOf("base-entry", "plain-entry"))
+			Expect(names("/api/models?items=9999&collapse_variants=true&term=%20%20")).
+				To(ConsistOf("base-entry", "plain-entry"))
+		})
+
+		It("reports the filtered total so pagination stays honest", func() {
+			// The listing paginates at 9, so a filter that narrowed the rows
+			// without narrowing the count would hand the user empty pages.
+			_, body := get("/api/models?items=9999&collapse_variants=true")
+			Expect(body["availableModels"]).To(BeEquivalentTo(2))
+			Expect(body["totalPages"]).To(BeEquivalentTo(1))
+		})
+
+		It("still issues no variant probes when filtering", func() {
+			names("/api/models?items=9999&collapse_variants=true")
+			Expect(probes.Load()).To(BeZero(),
+				"the filter must select on declared metadata, not by describing variants")
+		})
+	})
+
+	// Searching respects the collapse rather than switching it off. The term is
+	// matched against every entry the gallery holds, hidden builds included, so
+	// nothing becomes unfindable; the result is then reported in the grouped
+	// world the user asked for, which means a match on a build another entry
+	// offers surfaces that entry.
+	//
+	// The fixture: base-entry offers big-entry and dir-entry; plain-entry is
+	// nobody's variant. "big" matches one build, "build" matches both of them,
+	// "entry" matches everything, "plain" matches only the standalone.
+	Context("searching a collapsed listing", func() {
+		names := func(path string) []string {
+			code, body := get(path)
+			Expect(code).To(Equal(http.StatusOK))
+			raw, ok := body["models"].([]any)
+			Expect(ok).To(BeTrue(), "listing must return a models array")
+			out := make([]string, 0, len(raw))
+			for _, m := range raw {
+				out = append(out, m.(map[string]any)["name"].(string))
+			}
+			return out
+		}
+
+		It("surfaces the parent when only a hidden build matches", func() {
+			// The whole point. Answering "no models found" for a build the
+			// gallery does hold reads as "that model does not exist", but
+			// answering with the leaf hands back a row the collapsed view has
+			// no place for. The parent is the row the user can act on, and
+			// installing it is how they reach the build they searched for.
+			Expect(names("/api/models?items=9999&collapse_variants=true&term=big")).
+				To(ConsistOf("base-entry"))
+		})
+
+		It("surfaces the parent exactly once when several of its builds match", func() {
+			// "build" is in both variants' descriptions and in neither their
+			// parent's nor plain-entry's. ConsistOf compares as a multiset, so a
+			// second copy of base-entry fails here.
+			Expect(names("/api/models?items=9999&collapse_variants=true&term=build")).
+				To(ConsistOf("base-entry"))
+		})
+
+		It("surfaces a parent once when it matches in its own right and so do its builds", func() {
+			// "entry" is in every name, so base-entry matches on its own and via
+			// both the builds it offers: three reasons to be listed, one row.
+			Expect(names("/api/models?items=9999&collapse_variants=true&term=entry")).
+				To(ConsistOf("base-entry", "plain-entry"))
+		})
+
+		It("returns a matching standalone entry as itself", func() {
+			// Nobody offers plain-entry, so there is nothing to substitute and
+			// the collapse must leave the match alone.
+			Expect(names("/api/models?items=9999&collapse_variants=true&term=plain")).
+				To(ConsistOf("plain-entry"))
+		})
+
+		It("returns the leaves untouched when the collapse is off", func() {
+			// The other half of the toggle's promise: with grouping off, search
+			// answers with the individual builds exactly as it always did.
+			Expect(names("/api/models?items=9999&term=big")).To(ConsistOf("big-entry"))
+			Expect(names("/api/models?items=9999&term=build")).
+				To(ConsistOf("big-entry", "dir-entry"))
+			Expect(names("/api/models?items=9999&term=entry")).
+				To(ConsistOf("base-entry", "big-entry", "dir-entry", "plain-entry"))
+		})
+
+		It("counts and pages the substituted set, not the matches", func() {
+			// Substitution changes the size of the result, so a count taken
+			// before it would promise rows the response cannot deliver: "build"
+			// matches two builds that collapse into one row. At one item per
+			// page that is the difference between one page and two.
+			_, body := get("/api/models?items=9999&collapse_variants=true&term=build")
+			Expect(body["availableModels"]).To(BeEquivalentTo(1))
+			Expect(body["totalPages"]).To(BeEquivalentTo(1))
+
+			_, paged := get("/api/models?items=1&collapse_variants=true&term=entry")
+			Expect(paged["availableModels"]).To(BeEquivalentTo(2))
+			Expect(paged["totalPages"]).To(BeEquivalentTo(2))
+
+			// Uncollapsed, the same term matches four entries, so the counts
+			// above are the substitution's doing and not the fixture's.
+			_, all := get("/api/models?items=1&term=entry")
+			Expect(all["availableModels"]).To(BeEquivalentTo(4))
+			Expect(all["totalPages"]).To(BeEquivalentTo(4))
+		})
+
+		It("hands out every substituted row across the pages, none twice", func() {
+			// A count that matches the set is not enough: the page slices have
+			// to carry that set. A substitution done per page rather than over
+			// the whole result would repeat or drop rows at a page boundary.
+			first := names("/api/models?items=1&collapse_variants=true&term=entry&page=1")
+			second := names("/api/models?items=1&collapse_variants=true&term=entry&page=2")
+			Expect(first).To(HaveLen(1))
+			Expect(second).To(HaveLen(1))
+			Expect(append(first, second...)).To(ConsistOf("base-entry", "plain-entry"))
+		})
+
+		It("judges the backend filter against the build, then surfaces its parent", func() {
+			// dir-entry is vllm; the parent offering it is llama-cpp. The filter
+			// is applied before the substitution, so it selects the build that
+			// really is vllm rather than a parent that merely offers one. The
+			// row that comes back is therefore its parent, whose own backend
+			// column reads llama-cpp: the honest consequence of grouping, and
+			// the alternative is claiming the gallery has no vllm build.
+			Expect(names("/api/models?items=9999&collapse_variants=true&backend=vllm")).
+				To(ConsistOf("base-entry"))
+			// The other order would have matched nothing at all here, since no
+			// visible row in the collapsed gallery declares vllm.
+			Expect(names("/api/models?items=9999&backend=vllm")).To(ConsistOf("dir-entry"))
+		})
+
+		It("judges a tag the same way, and composes with a term", func() {
+			// dflash is big-entry's tag, not its parent's.
+			Expect(names("/api/models?items=9999&collapse_variants=true&tag=dflash")).
+				To(ConsistOf("base-entry"))
+			// Both filters still narrow: a term that misses the tagged build
+			// leaves nothing to surface.
+			Expect(names("/api/models?items=9999&collapse_variants=true&tag=dflash&term=plain")).
+				To(BeEmpty())
+		})
+
+		It("issues no variant probes while searching", func() {
+			// Substitution reads the declared names it already has in memory.
+			// Resolving a match through DescribeVariants would put a network
+			// round trip per variant on the search path.
+			names("/api/models?items=9999&collapse_variants=true&term=build")
+			Expect(probes.Load()).To(BeZero(),
+				"searching a collapsed listing probed variant weight files")
+		})
+	})
+})

core/schema/gallery-model.schema.json

@@ -56,6 +56,22 @@
         "additionalProperties": false
       }
     },
+    "variants": {
+      "type": "array",
+      "description": "Other builds of the same model (other backends, other quantizations) the installer may pick instead of this entry's own payload. Order carries no meaning: the installer drops what this host cannot run or does not have the memory for, then takes the largest of what is left, installing it under this entry's own name. If nothing is left this entry installs itself. Clients that predate variants support drop this key and install the entry unchanged.",
+      "minItems": 1,
+      "items": {
+        "type": "object",
+        "required": ["model"],
+        "properties": {
+          "model": {
+            "type": "string",
+            "description": "Name of a gallery entry that declares no variants of its own"
+          }
+        },
+        "additionalProperties": false
+      }
+    },
     "prompt_templates": {
       "type": "array",
       "description": "Prompt templates written as .tmpl files",

core/services/galleryop/managers_local.go

@@ -62,10 +62,14 @@ func (m *LocalModelManager) InstallModel(ctx context.Context, op *ManagementOp[g
 		}
 		return nil
 	case op.GalleryElementName != "":
+		opts := []gallery.InstallOption{gallery.WithArtifactMaterializer(m.artifactMaterializer)}
+		if op.Variant != "" {
+			opts = append(opts, gallery.WithVariant(op.Variant))
+		}
 		return gallery.InstallModelFromGallery(ctx, op.Galleries, op.BackendGalleries,
 			m.systemState, m.modelLoader, op.GalleryElementName, op.Req, progressCb,
 			m.enforcePredownloadScans, m.automaticallyInstallBackend, m.requireBackendIntegrity,
-			gallery.WithArtifactMaterializer(m.artifactMaterializer))
+			opts...)
 	default:
 		return installModelFromRemoteConfig(ctx, m.systemState, m.modelLoader, op.Req,
 			progressCb, m.enforcePredownloadScans, m.automaticallyInstallBackend, op.BackendGalleries, m.requireBackendIntegrity,

core/services/galleryop/model_variant_test.go

@@ -0,0 +1,107 @@
+package galleryop_test
+
+import (
+	"context"
+	"os"
+	"path/filepath"
+
+	"github.com/mudler/LocalAI/core/config"
+	"github.com/mudler/LocalAI/core/gallery"
+	"github.com/mudler/LocalAI/core/services/galleryop"
+	"github.com/mudler/LocalAI/pkg/model"
+	"github.com/mudler/LocalAI/pkg/system"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+	"gopkg.in/yaml.v3"
+)
+
+// A variant chosen in the UI or over REST arrives here as ManagementOp.Variant.
+// If it is not turned into a gallery.WithVariant install option the install
+// still succeeds, just on the auto-selected build, so the user's choice
+// disappears without a single error anywhere. These specs pin the threading.
+var _ = Describe("LocalModelManager variant selection", func() {
+	var (
+		modelsDir   string
+		mgr         *galleryop.LocalModelManager
+		appConfig   *config.ApplicationConfig
+		systemState *system.SystemState
+	)
+
+	// Every entry carries an inline config_file, so the whole install runs off
+	// the local filesystem and never reaches the network.
+	entry := func(name, backend string, variants ...gallery.Variant) gallery.GalleryModel {
+		m := gallery.GalleryModel{ConfigFile: map[string]any{"backend": backend}}
+		m.Name = name
+		m.Description = "entry " + name
+		m.Variants = variants
+		return m
+	}
+
+	installedBackend := func(name string) string {
+		GinkgoHelper()
+		dat, err := os.ReadFile(filepath.Join(modelsDir, name+".yaml"))
+		Expect(err).ToNot(HaveOccurred())
+		content := map[string]any{}
+		Expect(yaml.Unmarshal(dat, &content)).To(Succeed())
+		return content["backend"].(string)
+	}
+
+	install := func(variant string) error {
+		op := &galleryop.ManagementOp[gallery.GalleryModel, gallery.ModelConfig]{
+			GalleryElementName: "qwen3-8b-q4",
+			Variant:            variant,
+			Galleries:          appConfig.Galleries,
+		}
+		return mgr.InstallModel(context.Background(), op, func(string, string, string, float64) {})
+	}
+
+	BeforeEach(func() {
+		var err error
+		modelsDir, err = os.MkdirTemp("", "variant-op-*")
+		Expect(err).ToNot(HaveOccurred())
+		DeferCleanup(func() { Expect(os.RemoveAll(modelsDir)).To(Succeed()) })
+
+		// The upgrade declares a size no machine fails to clear and the base
+		// declares none, so the upgrade is the only measured fit and
+		// auto-selection takes it everywhere. Only an honored pin can land the
+		// install on the base. The size is read from the entry itself, so
+		// nothing here reaches the network.
+		upgrade := entry("qwen3-8b-q8", "upgrade-backend")
+		upgrade.Size = "16MiB"
+		entries := []gallery.GalleryModel{
+			entry("qwen3-8b-q4", "base-backend", gallery.Variant{Model: "qwen3-8b-q8"}),
+			upgrade,
+		}
+		out, err := yaml.Marshal(entries)
+		Expect(err).ToNot(HaveOccurred())
+		galleryPath := filepath.Join(modelsDir, "gallery.yaml")
+		Expect(os.WriteFile(galleryPath, out, 0o600)).To(Succeed())
+
+		systemState, err = system.GetSystemState(system.WithModelPath(modelsDir))
+		Expect(err).ToNot(HaveOccurred())
+		appConfig = &config.ApplicationConfig{
+			SystemState: systemState,
+			Galleries:   []config.Gallery{{Name: "test", URL: "file://" + galleryPath}},
+		}
+		mgr = galleryop.NewLocalModelManager(appConfig, model.NewModelLoader(systemState))
+	})
+
+	It("installs the entry's own build when the op pins it", func() {
+		Expect(install("qwen3-8b-q4")).To(Succeed())
+		Expect(installedBackend("qwen3-8b-q4")).To(Equal("base-backend"))
+	})
+
+	It("auto-selects when the op names no variant", func() {
+		// The mirror of the spec above: same gallery, same host, only the
+		// pin differs, so nothing but the pin can explain the different build.
+		Expect(install("")).To(Succeed())
+		Expect(installedBackend("qwen3-8b-q4")).To(Equal("upgrade-backend"))
+	})
+
+	It("fails the install when the op names a variant the entry does not declare", func() {
+		err := install("qwen3-8b-q2")
+		Expect(err).To(MatchError(gallery.ErrPinNotFound))
+		Expect(err.Error()).To(ContainSubstring("qwen3-8b-q2"))
+	})
+})

core/services/galleryop/models.go

@@ -6,6 +6,7 @@ import (
 	"errors"
 	"fmt"
 	"os"
+	"slices"
 
 	"github.com/mudler/LocalAI/core/config"
 	"github.com/mudler/LocalAI/core/gallery"
@@ -174,6 +175,9 @@ func installModelFromRemoteConfig(ctx context.Context, systemState *system.Syste
 type galleryModel struct {
 	gallery.GalleryModel `yaml:",inline"` // https://github.com/go-yaml/yaml/issues/63
 	ID                   string           `json:"id"`
+	// Variant pins the install to one of the entry's declared variants. Empty
+	// means auto-select.
+	Variant string `json:"variant,omitempty" yaml:"variant,omitempty"`
 }
 
 func processRequests(systemState *system.SystemState, modelLoader *model.ModelLoader, enforceScan, automaticallyInstallBackend bool, galleries []config.Gallery, backendGalleries []config.Gallery, requests []galleryModel, requireBackendIntegrity bool, options ...gallery.InstallOption) error {
@@ -185,8 +189,15 @@ func processRequests(systemState *system.SystemState, modelLoader *model.ModelLo
 			err = installModelFromRemoteConfig(ctx, systemState, modelLoader, r.GalleryModel, utils.DisplayDownloadFunction, enforceScan, automaticallyInstallBackend, backendGalleries, requireBackendIntegrity, options...)
 
 		} else {
+			// Cloned rather than appended to in place: `options` is shared by
+			// every request in the batch, so appending would leak one request's
+			// pin onto the next request that reuses the same backing array.
+			requestOptions := options
+			if r.Variant != "" {
+				requestOptions = append(slices.Clone(options), gallery.WithVariant(r.Variant))
+			}
 			err = gallery.InstallModelFromGallery(
-				ctx, galleries, backendGalleries, systemState, modelLoader, r.ID, r.GalleryModel, utils.DisplayDownloadFunction, enforceScan, automaticallyInstallBackend, requireBackendIntegrity, options...)
+				ctx, galleries, backendGalleries, systemState, modelLoader, r.ID, r.GalleryModel, utils.DisplayDownloadFunction, enforceScan, automaticallyInstallBackend, requireBackendIntegrity, requestOptions...)
 		}
 	}
 	return err

core/services/galleryop/operation.go

@@ -43,6 +43,15 @@ type ManagementOp[T any, E any] struct {
 	// build on one node without touching the rest of the cluster.
 	TargetNodeID string
 
+	// Variant pins a model install to one of the gallery entry's declared
+	// variants, by that variant's model name. Empty means auto-select: LocalAI
+	// picks the largest variant this host's backend support and memory can
+	// actually run, and falls back to the entry's own build.
+	//
+	// A name that is not among the entry's variants fails the install rather
+	// than quietly auto-selecting, so a typo cannot masquerade as a choice.
+	Variant string
+
 	// Upgrade is true if this is an upgrade operation (not a fresh install)
 	Upgrade bool
 

core/startup/model_preload.go

@@ -5,6 +5,7 @@ import (
 	"encoding/json"
 	"errors"
 	"fmt"
+	"slices"
 	"time"
 
 	"github.com/google/uuid"
@@ -22,9 +23,17 @@ import (
 // It will download the model if it is not already present in the model path
 // It will also try to resolve if the model is an embedded model YAML configuration
 func InstallModels(ctx context.Context, galleryService *galleryop.GalleryService, galleries, backendGalleries []config.Gallery, systemState *system.SystemState, modelLoader *model.ModelLoader, enforceScan, autoloadBackendGalleries, requireBackendIntegrity bool, downloadStatus func(string, string, string, float64), models ...string) error {
+	return InstallModelsWithOptions(ctx, galleryService, galleries, backendGalleries, systemState, modelLoader, enforceScan, autoloadBackendGalleries, requireBackendIntegrity, downloadStatus, nil, models...)
+}
+
+// InstallModelsWithOptions is InstallModels with extra install options, which
+// is how the CLI passes a variant pin. It is a separate entry point rather than
+// a variadic tail on InstallModels because that signature already ends in a
+// variadic model list.
+func InstallModelsWithOptions(ctx context.Context, galleryService *galleryop.GalleryService, galleries, backendGalleries []config.Gallery, systemState *system.SystemState, modelLoader *model.ModelLoader, enforceScan, autoloadBackendGalleries, requireBackendIntegrity bool, downloadStatus func(string, string, string, float64), extraOptions []gallery.InstallOption, models ...string) error {
 	// create an error that groups all errors
 	var err error
-	var installOptions []gallery.InstallOption
+	installOptions := slices.Clone(extraOptions)
 	if galleryService != nil {
 		installOptions = append(installOptions, gallery.WithArtifactMaterializer(galleryService.ModelArtifactMaterializer()))
 	}

docs/content/features/model-gallery.md

@@ -151,6 +151,150 @@ where:
 - `bert-embeddings` is the model name in the gallery
   (read its [config here](https://github.com/mudler/LocalAI/tree/master/gallery/blob/main/bert-embeddings.yaml)).
 
+### Model variants
+
+Some gallery entries offer several builds of the same model: different
+quantizations, or the same weights served by a different engine. Such an entry
+carries a `variants` list, and installing it normally lets LocalAI choose:
+
+- variants whose backend cannot run on this machine are dropped;
+- variants that do not fit the available memory are dropped. That budget is
+  VRAM on a discrete-GPU host, and system RAM otherwise — including on
+  unified-memory machines such as Apple Silicon, where the GPU shares system
+  RAM and reports no separate VRAM pool;
+- the entry's own build is never dropped. It competes with whatever survived
+  rather than waiting for everything else to fail, so an entry that is itself
+  the largest build that fits keeps its own payload;
+- among the remaining builds the engine this machine prefers wins first: a vLLM
+  build on an NVIDIA or AMD host, an MLX build on Apple Silicon, llama.cpp
+  otherwise. The native accelerated runtime is worth more than a bigger
+  download, so preference is settled before size;
+- the largest build on the preferred engine then wins, because a bigger
+  footprint means a higher quality build of the same model. A machine with no
+  preferred engine picks purely by size;
+- a build whose size could not be measured ranks below the entry's own build,
+  so an unreadable size never quietly displaces the payload the entry ships;
+- if nothing else survives, the entry's own build is installed. The entry is
+  always installable, on any machine.
+
+Because the entry's own build competes like every other candidate, the order of
+the list means nothing and a `variants` list may offer smaller builds, larger
+ones, or both.
+
+Sizes are measured from the model's weights rather than downloaded, and cached.
+
+The gallery listing only flags which entries offer variants, with a
+`has_variants` field. It deliberately does not describe them: measuring a
+variant is a network round trip per referenced build, so describing every
+entry inline would make one listing request cost as many round trips as the
+whole page has variants.
+
+```bash
+curl http://localhost:8080/api/models | jq '.models[] | select(.has_variants) | .name'
+```
+
+### Collapsing the listing to one row per model
+
+By default the listing returns every entry, including the individual builds a
+parent entry offers as variants, so one model can occupy several rows. Pass
+`collapse_variants=true` for the deduplicated view: every entry that is
+installable in its own right, with nothing shown twice.
+
+```bash
+curl 'http://localhost:8080/api/models?collapse_variants=true'
+```
+
+An entry is hidden only when another entry already offers it as a variant, so
+it stays reachable by installing that entry. Entries that declare variants are
+always kept, and so is any entry nobody references. The filter is applied
+before pagination, so page counts stay correct.
+
+**Searching respects the collapse.** `term` is matched against every entry the
+gallery holds, builds another entry offers included, so nothing becomes
+unfindable. The collapse then decides how a match is reported: a hit on a build
+another entry offers comes back as that entry, since that is the row installable
+in its own right. Looking a model up by name must not answer "not found" for an
+entry the gallery holds, and must not answer with a row the requested view has
+no place for either. A term that is empty or only whitespace does not count as a
+search:
+
+```bash
+# Collapsed: the parent stands in for the build it offers.
+curl 'http://localhost:8080/api/models?collapse_variants=true'
+# Collapsed, searching a build the parent offers: the parent comes back.
+curl 'http://localhost:8080/api/models?collapse_variants=true&term=nanbeige4.1-3b-q8'
+# Uncollapsed, same term: the build itself comes back.
+curl 'http://localhost:8080/api/models?term=nanbeige4.1-3b-q8'
+```
+
+A parent appears once however many of its builds match, and once when it matches
+in its own right as well. The substitution runs before the count and the page
+math, so both describe the rows actually handed out.
+
+`term`, `tag` and `backend` are all applied before the substitution, so each is
+judged against the build that really carries the name, tag or backend rather
+than against a parent that merely offers it. The consequence is worth knowing:
+filtering by a backend only one variant declares returns that variant's parent,
+whose own `backend` field may say something else. The alternative would be to
+claim the gallery holds no such build.
+
+The web UI requests the collapsed view by default and has a toggle for the other
+one. The parameter stays on the API, off by default, for clients that want
+either view.
+
+Ask for the description one entry at a time, as the web UI does when you open
+a model's variant menu:
+
+```bash
+curl http://localhost:8080/api/models/variants/localai@nanbeige4.1-3b-q4
+```
+
+```json
+{
+  "auto_selected": "nanbeige4.1-3b-q8",
+  "variants": [
+    { "model": "nanbeige4.1-3b-q8", "backend": "llama-cpp", "memory_bytes": 4187593113, "fits": true, "is_base": false },
+    { "model": "nanbeige4.1-3b-q4", "backend": "llama-cpp", "fits": true, "is_base": true }
+  ]
+}
+```
+
+`auto_selected` is what installing without a choice would pick right now. `fits`
+is whether auto-selection would consider that variant on this machine, and
+`is_base` marks the entry's own build. `memory_bytes` is omitted entirely, as on
+the second entry above, when the size could not be measured; read a missing
+`memory_bytes` as unknown rather than as a free build.
+
+An entry that declares no variants carries no `has_variants` field and answers
+this endpoint with an empty list, so a client never has to ask about it.
+
+To install a specific one, pass its name as `variant`:
+
+```bash
+curl $LOCALAI/models/apply -H "Content-Type: application/json" -d '{
+     "id": "localai@nanbeige4.1-3b-q4",
+     "variant": "nanbeige4.1-3b-q8"
+   }'
+```
+
+An explicit choice is honored even when the machine looks too small for it, so
+you can deliberately install a build LocalAI would not have picked. A `variant`
+the entry does not declare fails the install and names what was requested; it
+never quietly falls back to auto-selection. The choice is recorded, so a later
+reinstall or upgrade of the same model stays on the variant you picked.
+
+The same option exists on the CLI:
+
+```bash
+local-ai models install nanbeige4.1-3b-q4 --variant nanbeige4.1-3b-q8
+```
+
+The `install_model` MCP tool takes the same `variant` argument, so an assistant
+managing installs conversationally can pick a build too.
+
+Entries without a `variants` list are unaffected by any of this and install
+exactly as they always have.
+
 ### Artifact-backed models
 
 Gallery models with an `artifacts` declaration are fully materialized during

gallery/index.yaml

@@ -172,6 +172,7 @@
   tags:
     - llm
     - gguf
+    - mtp
   overrides:
     backend: llama-cpp
     function:
@@ -233,6 +234,9 @@
       uri: huggingface://prism-ml/Bonsai-8B-gguf/Bonsai-8B-Q1_0.gguf
 - &ternary-bonsai-8b
   name: "ternary-bonsai-8b"
+  variants:
+    - model: ternary-bonsai-8b-q2-g64
+    - model: ternary-bonsai-8b-pq2
   url: "github:mudler/LocalAI/gallery/qwen3.yaml@master"
   license: apache-2.0
   icon: https://huggingface.co/prism-ml/Ternary-Bonsai-8B-gguf/resolve/main/assets/bonsai-logo.svg
@@ -269,6 +273,7 @@
       uri: huggingface://prism-ml/Ternary-Bonsai-8B-gguf/Ternary-Bonsai-8B-Q2_0.gguf
 - !!merge <<: *ternary-bonsai-8b
   name: "ternary-bonsai-8b-q2-g64"
+  variants: []
   description: |
     Ternary Bonsai 8B (PrismML), GGUF Q2_0 with group-64 packing (each FP16 scale shared
     across 64 weights instead of 128). Slightly larger (~2.31 GB) but matches llama.cpp's
@@ -286,6 +291,7 @@
       uri: huggingface://prism-ml/Ternary-Bonsai-8B-gguf/Ternary-Bonsai-8B-Q2_0_g64.gguf
 - !!merge <<: *ternary-bonsai-8b
   name: "ternary-bonsai-8b-pq2"
+  variants: []
   description: |
     Ternary Bonsai 8B (PrismML), GGUF PQ2_0 (packed Q2_0) ternary variant (~2.18 GB).
     Same {-1, 0, +1} weight alphabet as Q2_0. Runs on LocalAI's `bonsai` backend.
@@ -345,6 +351,9 @@
       uri: huggingface://prism-ml/Bonsai-27B-gguf/Bonsai-27B-mmproj-Q8_0.gguf
 - &ternary-bonsai-27b
   name: "ternary-bonsai-27b"
+  variants:
+    - model: ternary-bonsai-27b-pq2
+    - model: ternary-bonsai-27b-q2-g64
   url: "github:mudler/LocalAI/gallery/qwen3.yaml@master"
   license: apache-2.0
   icon: https://huggingface.co/prism-ml/Ternary-Bonsai-27B-gguf/resolve/main/assets/bonsai-logo.svg
@@ -387,6 +396,7 @@
       uri: huggingface://prism-ml/Ternary-Bonsai-27B-gguf/Ternary-Bonsai-27B-mmproj-Q8_0.gguf
 - !!merge <<: *ternary-bonsai-27b
   name: "ternary-bonsai-27b-pq2"
+  variants: []
   description: |
     Ternary Bonsai 27B (PrismML), GGUF PQ2_0 (packed Q2_0) ternary variant (~7.17 GB) with
     the 4-bit vision tower (mmproj) included. Runs on LocalAI's `bonsai` backend.
@@ -408,6 +418,7 @@
       uri: huggingface://prism-ml/Ternary-Bonsai-27B-gguf/Ternary-Bonsai-27B-mmproj-Q8_0.gguf
 - !!merge <<: *ternary-bonsai-27b
   name: "ternary-bonsai-27b-q2-g64"
+  variants: []
   description: |
     Ternary Bonsai 27B (PrismML), GGUF Q2_0 with group-64 packing (~7.59 GB), matching
     llama.cpp's native 64-value Q2_0 block layout, with the 4-bit vision tower (mmproj)
@@ -485,6 +496,11 @@
       uri: https://huggingface.co/GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF/resolve/main/MiniCPM5-1B-Claude-Opus-Fable5-Thinking-Q4_K_M.gguf
       sha256: b1c3bf2995e96cb792a0031e4e1497a500e9244c68ba17c24a7e6edf1fc59019
 - name: "deepseek-v4-flash"
+  variants:
+    - model: deepseek-v4-flash-q2
+    - model: deepseek-v4-flash-q2-q4
+    - model: deepseek-v4-flash-q4-ssd
+    - model: deepseek-v4-flash-q2-mtp
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
     - https://huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF
@@ -521,6 +537,7 @@
     - gguf
     - vision
     - multimodal
+    - mtp
   icon: https://cdn-uploads.huggingface.co/production/uploads/66309bd090589b7c65950665/ztbyGV_zGhzcLuTCSVyq3.png
   overrides:
     backend: llama-cpp
@@ -556,6 +573,7 @@
   tags:
     - llm
     - gguf
+    - mtp
   overrides:
     backend: llama-cpp
     function:
@@ -607,6 +625,8 @@
       sha256: e7a8eafdd8013443b6bcc4b6fb47b2d2025f772d359650b9ceb7d75971e22cad
       uri: https://huggingface.co/unsloth/Qwen-AgentWorld-35B-A3B-GGUF/resolve/main/Qwen-AgentWorld-35B-A3B-UD-Q4_K_M.gguf
 - name: "ornith-1.0-9b"
+  variants:
+    - model: ornith-1.0-9b-mtp
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
     - https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B-GGUF
@@ -714,6 +734,7 @@
     - llm
     - gguf
     - reasoning
+    - mtp
   icon: https://storage.ko-fi.com/cdn/kofi6.png
   overrides:
     backend: llama-cpp
@@ -785,6 +806,7 @@
     - vision
     - multimodal
     - reasoning
+    - mtp
   icon: https://cdn-uploads.huggingface.co/production/uploads/66309bd090589b7c65950665/sGQKmrMc6L6guMoaB5_Y2.png
   overrides:
     backend: llama-cpp
@@ -1215,6 +1237,7 @@
     - vision
     - multimodal
     - reasoning
+    - mtp
   overrides:
     backend: llama-cpp
     function:
@@ -1249,6 +1272,7 @@
   tags:
     - llm
     - gguf
+    - mtp
   icon: https://raw.githubusercontent.com/zai-org/GLM-5/refs/heads/main/resources/bench_52.png
   overrides:
     backend: llama-cpp
@@ -1570,6 +1594,7 @@
     - vision
     - multimodal
     - reasoning
+    - mtp
   icon: https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3.6/Figures/qwen3.6_27b_score.png
   overrides:
     backend: llama-cpp
@@ -1799,6 +1824,8 @@
       sha256: 88956c71d20444d3ebf890e4495afed3257c6be877d4e82f0c26ce58e79b340f
       uri: https://huggingface.co/ReadyArt/Dark-Scarlett-v0.3-26B-A4B-GGUF/resolve/main/Dark-Scarlett-v0.3-26B-A4B-Q4_K_M.gguf
 - name: "qwopus3.6-27b-coder-mtp"
+  variants:
+    - model: qwopus3.6-27b-coder-mtp-nvfp4
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
     - https://huggingface.co/Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF
@@ -1810,6 +1837,7 @@
     - vision
     - multimodal
     - reasoning
+    - mtp
   overrides:
     backend: llama-cpp
     function:
@@ -1836,6 +1864,8 @@
       sha256: 32f7ea0600c07272547da401d460f8abbd980f3a57b69d6df87be0e2505e0b9c
       uri: https://huggingface.co/Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF/resolve/main/mmproj-F32.gguf
 - name: "gemma-4-26b-a4b-it-qat"
+  variants:
+    - model: gemma-4-26b-a4b-it-qat-q4_0
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
     - https://huggingface.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF
@@ -1890,6 +1920,8 @@
       sha256: ef269e294502d6ee3722cbf129681b2586c2e6ceb79d0507963c92146e058cd4
       uri: https://huggingface.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF/resolve/main/mmproj-F32.gguf
 - name: "gemma-4-12b-it-qat-q4_0"
+  variants:
+    - model: gemma-4-12b-it-qat-mtp
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
     - https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-gguf
@@ -1943,6 +1975,8 @@
       uri: https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-gguf/resolve/main/mmproj-gemma-4-12b-it-qat-q4_0.gguf
       sha256: cb018338a7538a9814d994bfe54644c71eb7ed54e31eae2f721e45fd3c260da7
 - name: "gemma-4-e2b-it-qat-q4_0"
+  variants:
+    - model: gemma-4-e2b-it-qat-mtp
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
     - https://huggingface.co/google/gemma-4-E2B-it-qat-q4_0-gguf
@@ -1985,6 +2019,8 @@
       uri: https://huggingface.co/google/gemma-4-E2B-it-qat-q4_0-gguf/resolve/main/gemma-4-E2B-it-mmproj.gguf
       sha256: 021059cce659fe7f9170d5599761d7bbaf644b798dab9503aca30dc43e6beb14
 - name: "gemma-4-e4b-it-qat-q4_0"
+  variants:
+    - model: gemma-4-e4b-it-qat-mtp
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
     - https://huggingface.co/google/gemma-4-E4B-it-qat-q4_0-gguf
@@ -2070,6 +2106,8 @@
       uri: https://huggingface.co/google/gemma-4-26B-A4B-it-qat-q4_0-gguf/resolve/main/gemma-4-26B-it-mmproj.gguf
       sha256: a359953a076b877db30c31dbbb4c6d93b4a6e017ee5db5784247e4d4c0dd4f3b
 - name: "gemma-4-31b-it-qat-q4_0"
+  variants:
+    - model: gemma-4-31b-it-qat-mtp
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
     - https://huggingface.co/google/gemma-4-31B-it-qat-q4_0-gguf
@@ -2450,6 +2488,8 @@
       sha256: 01b76572f80b7d2ebee80a27cb9c3699c26b04cae1c402eee7664fc17a4b5ce6
       uri: https://huggingface.co/LocalAI-io/privacy-filter-multilingual-GGUF/resolve/main/privacy-filter-multilingual-f16.gguf
 - name: "privacy-filter-nemotron"
+  variants:
+    - model: privacy-filter-nemotron-q8
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   icon: https://cdn-avatars.huggingface.co/v1/production/uploads/5fd5e18a90b6dc4633f6d292/QPiv8pt4JNxr0FdGnpFef.png
   urls:
@@ -2626,6 +2666,7 @@
     - llm
     - gguf
     - reasoning
+    - mtp
   icon: https://cdn-uploads.huggingface.co/production/uploads/66309bd090589b7c65950665/9EnS13MSxNU3snpAgEiLq.jpeg
   overrides:
     backend: llama-cpp
@@ -2653,6 +2694,8 @@
       sha256: f48daca405a1c768a9514e392c3955dcc4a9d66a5cf64cf45e064092b5f20ee4
       uri: https://huggingface.co/Jackrong/Qwopus3.5-9B-Coder-MTP-GGUF/resolve/main/Qwopus3.5-9B-Coder-MTP-mmproj.gguf
 - name: "qwopus3.6-27b-v2-mtp"
+  variants:
+    - model: qwopus3.6-27b-v2-mtp-nvfp4
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
     - https://huggingface.co/Jackrong/Qwopus3.6-27B-v2-MTP-GGUF
@@ -2662,6 +2705,7 @@
     - llm
     - gguf
     - reasoning
+    - mtp
   overrides:
     backend: llama-cpp
     function:
@@ -3242,6 +3286,9 @@
       sha256: 13bd039f95c9ea46ef1d75905faa7be6ca4e47a5af9d4cf62e298a738a5b195f
       uri: https://huggingface.co/KyleHessling1/Qwopus-GLM-18B-Merged-GGUF/resolve/main/Qwopus-GLM-18B-Healed-Q4_K_M.gguf
 - name: qwen3.6-27b
+  variants:
+    - model: qwen3.6-27b-dflash
+    - model: qwen3.6-27b-nvfp4-mtp
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
     - https://huggingface.co/unsloth/Qwen3.6-27B-GGUF
@@ -3532,6 +3579,9 @@
       sha256: b5aa0676be588bf6ef3bbdb89905d7d239b2a809637f0766a6ce23aed6c6b5b4
       uri: https://huggingface.co/mudler/Qwen3.6-35B-A3B-APEX-GGUF/resolve/main/Qwen3.6-35B-A3B-APEX-Quality.gguf
 - name: qwen3.6-35b-a3b
+  variants:
+    - model: qwen3.6-35b-a3b-dflash
+    - model: qwen3.6-35b-a3b-nvfp4-mtp
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
     - https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF
@@ -3670,6 +3720,8 @@
       sha256: 2aa99ffb47033ead4a3f1584fec5283905302c1c16fed59c99e0eec131c6dc53
       uri: huggingface://ggml-org/gemma-4-26B-A4B-it-GGUF/mmproj-gemma-4-26B-A4B-it-bf16.gguf
 - name: gemma-4-e2b-it
+  variants:
+    - model: "gemma-4-e2b-it:sglang-mtp"
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
     - https://huggingface.co/google/gemma-4-E2B-it
@@ -3713,6 +3765,8 @@
       sha256: e42083b71a9e31e0f722171d551f6d92b101544001c4dde040306a8f2160fe8c
       uri: huggingface://ggml-org/gemma-4-E2B-it-GGUF/mmproj-gemma-4-E2B-it-bf16.gguf
 - name: gemma-4-e4b-it
+  variants:
+    - model: "gemma-4-e4b-it:sglang-mtp"
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
     - https://huggingface.co/google/gemma-4-E4B-it
@@ -4130,6 +4184,8 @@
       sha256: 8221b6a48c714db6829a92760c31034d7ecd436f830c61624ccc92b461b4a1c4
       uri: https://huggingface.co/mradermacher/Q3.5-BlueStar-27B-GGUF/resolve/main/Q3.5-BlueStar-27B.mmproj-f16.gguf
 - name: qwen3.5-9b
+  variants:
+    - model: qwen3.5-9b-dflash
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
     - https://huggingface.co/unsloth/Qwen3.5-9B-GGUF
@@ -4408,6 +4464,13 @@
     - code
     - math
   last_checked: "2026-04-30"
+  # This entry installs the Q4_K_M build as-is, on any client and any host.
+  # variants lists other builds of the same model the installer may pick
+  # instead when this host can run them and has the memory for them, sizing
+  # each one live from its weights. Clients that predate variants support drop
+  # the key and install this entry unchanged.
+  variants:
+    - model: nanbeige4.1-3b-q8
   overrides:
     parameters:
       model: nanbeige4.1-3b-q4_k_m.gguf
@@ -4548,6 +4611,8 @@
       sha256: 5c058d9f7b737167195fa54eae4a2ae17658ac2c0a8073f7f116ba006b2ab32c
       uri: https://huggingface.co/mudler/ced-gguf/resolve/main/ced-base-f16.gguf
 - name: ced-base-q8
+  variants:
+    - model: ced-base-f16
   url: github:mudler/LocalAI/gallery/ced.yaml@master
   urls:
     - https://huggingface.co/mudler/ced-gguf
@@ -4594,6 +4659,8 @@
       sha256: af8b81c67bae50bfca4ea83dbba77b3bae4fa6180d36c17d6877f7700aeeb77b
       uri: https://huggingface.co/mudler/ced-gguf/resolve/main/ced-tiny-f16.gguf
 - name: ced-tiny-q8
+  variants:
+    - model: ced-tiny-f16
   url: github:mudler/LocalAI/gallery/ced.yaml@master
   urls:
     - https://huggingface.co/mudler/ced-gguf
@@ -4640,6 +4707,8 @@
       sha256: 3c6a8936c77312f07a9ecb7b4bbbcb1f93ad137920ca6656bae9306571fb0c03
       uri: https://huggingface.co/mudler/ced-gguf/resolve/main/ced-mini-f16.gguf
 - name: ced-mini-q8
+  variants:
+    - model: ced-mini-f16
   url: github:mudler/LocalAI/gallery/ced.yaml@master
   urls:
     - https://huggingface.co/mudler/ced-gguf
@@ -4686,6 +4755,8 @@
       sha256: c391ed8697a1b08d7c1a463e4940a5c3a2f670e0544ab0d8ee23b544583602a8
       uri: https://huggingface.co/mudler/ced-gguf/resolve/main/ced-small-f16.gguf
 - name: ced-small-q8
+  variants:
+    - model: ced-small-f16
   url: github:mudler/LocalAI/gallery/ced.yaml@master
   urls:
     - https://huggingface.co/mudler/ced-gguf
@@ -6200,6 +6271,8 @@
       uri: huggingface://Serveurperso/Qwen3-TTS-GGUF/qwen-tokenizer-12hz-Q4_K_M.gguf
 - !!merge <<: *qwenttscpp_gallery
   name: qwen3-tts-cpp-1.7b-base
+  variants:
+    - model: qwen3-tts-cpp-1.7b-base-q4
   description: |
     Qwen3-TTS 1.7B Base (C++ / GGML, qwentts.cpp), Q8_0 (~2.0 GB talker).
     Higher-quality streaming + voice cloning, 24kHz mono, 11 languages.
@@ -6238,6 +6311,8 @@
       uri: huggingface://Serveurperso/Qwen3-TTS-GGUF/qwen-tokenizer-12hz-Q4_K_M.gguf
 - !!merge <<: *qwenttscpp_gallery
   name: qwen3-tts-cpp-customvoice
+  variants:
+    - model: qwen3-tts-cpp-customvoice-q4
   description: |
     Qwen3-TTS 0.6B CustomVoice (C++ / GGML, qwentts.cpp), Q8_0. Named speakers
     selected via the `voice` field: serena, vivian, uncle_fu, ryan, aiden,
@@ -6295,6 +6370,8 @@
       uri: huggingface://Serveurperso/Qwen3-TTS-GGUF/qwen-tokenizer-12hz-Q4_K_M.gguf
 - !!merge <<: *qwenttscpp_gallery
   name: qwen3-tts-cpp-1.7b-customvoice
+  variants:
+    - model: qwen3-tts-cpp-1.7b-customvoice-q4
   description: |
     Qwen3-TTS 1.7B CustomVoice (C++ / GGML, qwentts.cpp), Q8_0. Named speakers via
     the `voice` field (serena, vivian, ryan, aiden, eric, dylan, ...). Streaming,
@@ -6350,6 +6427,8 @@
       uri: huggingface://Serveurperso/Qwen3-TTS-GGUF/qwen-tokenizer-12hz-Q4_K_M.gguf
 - !!merge <<: *qwenttscpp_gallery
   name: qwen3-tts-cpp-1.7b-voicedesign
+  variants:
+    - model: qwen3-tts-cpp-1.7b-voicedesign-q4
   description: |
     Qwen3-TTS 1.7B VoiceDesign (C++ / GGML, qwentts.cpp), Q8_0. Synthesises a
     speaker from a free-text attribute instruction - REQUIRES the OpenAI
@@ -6407,6 +6486,8 @@
       uri: huggingface://Serveurperso/Qwen3-TTS-GGUF/qwen-tokenizer-12hz-Q4_K_M.gguf
 - &mossttscpp_gallery
   name: moss-tts-cpp-v1_5-q8_0
+  variants:
+    - model: moss-tts-cpp-v1_5-f16
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
     - https://huggingface.co/mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF
@@ -6448,6 +6529,7 @@
       uri: huggingface://mudler/MOSS-TTS-Local-Transformer-v1.5-GGUF/moss-tokenizer-v1_5.gguf
 - !!merge <<: *mossttscpp_gallery
   name: moss-tts-cpp-v1_5-f16
+  variants: []
   description: |
     MOSS-TTS-Local v1.5 (C++/ggml, moss-tts.cpp), F16 (~9.4 GB), code-exact versus the
     reference. 48kHz stereo text-to-speech with reference-audio voice cloning.
@@ -8163,6 +8245,8 @@
       sha256: cd5a851d3928697fa1bd76d459d2cc409b6cf40c9d9682b2f5c8e7c6a9f9630f
       uri: huggingface://unsloth/Qwen3-VL-2B-Instruct-GGUF/mmproj-F16.gguf
 - name: huihui-qwen3-vl-30b-a3b-instruct-abliterated
+  variants:
+    - model: huihui-qwen3-vl-30b-a3b-instruct-abliterated-mxfp4_moe
   url: github:mudler/LocalAI/gallery/qwen3.yaml@master
   urls:
     - https://huggingface.co/noctrex/Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-GGUF
@@ -8682,6 +8766,8 @@
       sha256: c5c38bfa5d8d5100e91a2e0050a0b2f3e082cd4bfd423cb527abc3b6f1ae180c
       uri: huggingface://bartowski/Aurore-Reveil_Koto-Small-7B-IT-GGUF/Aurore-Reveil_Koto-Small-7B-IT-Q4_K_M.gguf
 - name: opengvlab_internvl3_5-30b-a3b
+  variants:
+    - model: opengvlab_internvl3_5-30b-a3b-q8_0
   url: github:mudler/LocalAI/gallery/qwen3.yaml@master
   urls:
     - https://huggingface.co/OpenGVLab/InternVL3_5-30B-A3B
@@ -8779,6 +8865,8 @@
       sha256: c9625c981969d267052464e2d345f8ff5bc7e841871f5284a2bd972461c7356d
       uri: huggingface://bartowski/OpenGVLab_InternVL3_5-14B-GGUF/mmproj-OpenGVLab_InternVL3_5-14B-f16.gguf
 - name: opengvlab_internvl3_5-14b
+  variants:
+    - model: opengvlab_internvl3_5-14b-q8_0
   url: github:mudler/LocalAI/gallery/qwen3.yaml@master
   urls:
     - https://huggingface.co/OpenGVLab/InternVL3_5-14B
@@ -8811,6 +8899,8 @@
       sha256: c9625c981969d267052464e2d345f8ff5bc7e841871f5284a2bd972461c7356d
       uri: huggingface://bartowski/OpenGVLab_InternVL3_5-14B-GGUF/mmproj-OpenGVLab_InternVL3_5-14B-f16.gguf
 - name: opengvlab_internvl3_5-8b
+  variants:
+    - model: opengvlab_internvl3_5-8b-q8_0
   url: github:mudler/LocalAI/gallery/qwen3.yaml@master
   urls:
     - https://huggingface.co/OpenGVLab/InternVL3_5-8B
@@ -8873,6 +8963,8 @@
       sha256: 212cc090f81ea2981b870186d4b424fae69489a5313a14e52ffdb2e877852389
       uri: huggingface://bartowski/OpenGVLab_InternVL3_5-8B-GGUF/mmproj-OpenGVLab_InternVL3_5-8B-f16.gguf
 - name: opengvlab_internvl3_5-4b
+  variants:
+    - model: opengvlab_internvl3_5-4b-q8_0
   url: github:mudler/LocalAI/gallery/qwen3.yaml@master
   urls:
     - https://huggingface.co/OpenGVLab/InternVL3_5-4B
@@ -10100,6 +10192,8 @@
       sha256: ebab7f90c7833fbccd46d3a555410e78d969db5438e169b6524be444862b3676
       uri: https://github.com/yakhyo/face-anti-spoofing/releases/download/weights/MiniFASNetV1SE.onnx
 - name: insightface-opencv
+  variants:
+    - model: insightface-opencv-int8
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
     - https://github.com/opencv/opencv_zoo
@@ -10793,6 +10887,10 @@
       sha256: 0909d8a1aba584b482d501baae032611d1559878be1b7f6606ba516687c5380d
 - &depth-anything-3-base
   name: depth-anything-3-base
+  variants:
+    - model: depth-anything-3-base-q8_0
+    - model: depth-anything-3-base-f16
+    - model: depth-anything-3-base-f32
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
     - https://github.com/mudler/depth-anything.cpp
@@ -10824,6 +10922,7 @@
       sha256: "43cd45d00f9024f4319f4beabd73155db5132e4b575bc52eff4131262c9d78f1"
 - !!merge <<: *depth-anything-3-base
   name: depth-anything-3-base-q8_0
+  variants: []
   description: |
     Depth Anything 3 (base), q8_0 — near-lossless 8-bit quant (~149 MB). Same
     depth + camera pose output as the q4_k default at higher fidelity.
@@ -10837,6 +10936,7 @@
       sha256: "71b1c953113657f9a4fbef43ab6a16fe7a6f87b36da113a184f13c4a564968a0"
 - !!merge <<: *depth-anything-3-base
   name: depth-anything-3-base-f16
+  variants: []
   description: |
     Depth Anything 3 (base), f16 — half precision (~233 MB), no measurable
     accuracy loss vs f32. Depth + camera pose.
@@ -10850,6 +10950,7 @@
       sha256: "2975419c99702ca646d5b7120c53e35c9fee158f0a803577241d16957f75624b"
 - !!merge <<: *depth-anything-3-base
   name: depth-anything-3-base-f32
+  variants: []
   description: |
     Depth Anything 3 (base), f32 — maximum fidelity (~412 MB). Reference-parity
     depth + camera pose.
@@ -10863,6 +10964,7 @@
       sha256: "1b13b166e8a8b4f2c862f42d36edb2f9aab995a18cc527a52b9f160b99c6b8da"
 - !!merge <<: *depth-anything-3-base
   name: depth-anything-3-giant
+  variants: []
   description: |
     Depth Anything 3 (giant / vitg), f32 — the large backbone (~4.9 GB) for
     maximum quality depth + camera pose. GPU recommended.
@@ -10883,6 +10985,7 @@
       sha256: "392edf64626be6a985487beb39c8d54cdc14f7feb2b53323742c96b71e7e7181"
 - !!merge <<: *depth-anything-3-base
   name: depth-anything-3-small
+  variants: []
   description: |
     Depth Anything 3 (small / vits), f32 — the smallest backbone (~131 MB) for
     fast CPU depth + camera pose. Same output as base at lower latency.
@@ -10896,6 +10999,7 @@
       sha256: "eab5597e01dedde1a20c038590ae8c887b85ec35b882581138c08308e92c41e5"
 - !!merge <<: *depth-anything-3-base
   name: depth-anything-3-large
+  variants: []
   description: |
     Depth Anything 3 (large / vitl), f32 (~1.6 GB) — higher quality depth +
     camera pose than base. GPU recommended for interactive use.
@@ -10916,6 +11020,7 @@
       sha256: "a79eb3e19e8ec49f4daac484fb5fb67e15baac61518d229cf819e40c87080906"
 - !!merge <<: *depth-anything-3-base
   name: depth-anything-3-mono-large
+  variants: []
   description: |
     Depth Anything 3 (monocular large / vitl), f32 (~1.3 GB) — single-image
     monocular depth + a sky mask (no camera pose). DPT single-head variant; use
@@ -10930,6 +11035,7 @@
       sha256: "291b1a554af907c3f79986ee225da8933be5f7a31d73c81d06784cda284535de"
 - !!merge <<: *depth-anything-3-base
   name: depth-anything-3-metric-large
+  variants: []
   description: |
     Depth Anything 3 (metric large / vitl), f32 (~1.3 GB) — single-image
     metric-scale depth (meters) + a sky mask. DPT single-head metric variant; use
@@ -10945,6 +11051,7 @@
       sha256: "d10b7450c2238244b2d72e2749537a1876255180149cd630a18bc1619c9286be"
 - !!merge <<: *depth-anything-3-base
   name: depth-anything-3-nested
+  variants: []
   description: |
     Depth Anything 3 (nested giant+large), f32 — the recommended metric model. A
     two-branch pipeline: the anyview GIANT (vitg) branch and a metric ViT-L branch
@@ -10976,6 +11083,10 @@
       sha256: "b54ed50cbc0b0c14fae1f8edd0fea8bd1cac0850485fd6e7eb2422c7a19e570e"
 - &depth-anything-2-base
   name: depth-anything-2-base
+  variants:
+    - model: depth-anything-2-base-q8_0
+    - model: depth-anything-2-base-f16
+    - model: depth-anything-2-base-f32
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
     - https://github.com/mudler/depth-anything.cpp
@@ -11006,6 +11117,7 @@
       sha256: "49e77ec7e593080111242fa76017cae3e26498d550841cf8a70dfcd36bb175f2"
 - !!merge <<: *depth-anything-2-base
   name: depth-anything-2-base-q8_0
+  variants: []
   description: |
     Depth Anything V2 (base / ViT-B), q8_0 — near-lossless 8-bit quant. Same
     relative monocular depth output as the q4_k default at higher fidelity. Use
@@ -11020,6 +11132,7 @@
       sha256: "11920ec7a8dfc2fa7fe8ed44811a46fafe415c641d13144bb733d1437832291b"
 - !!merge <<: *depth-anything-2-base
   name: depth-anything-2-base-f16
+  variants: []
   description: |
     Depth Anything V2 (base / ViT-B), f16 — half precision, no measurable
     accuracy loss vs f32. Relative monocular depth only (no pose). Use
@@ -11034,6 +11147,7 @@
       sha256: "e91c011fbbf90a44639fea55b8d61ca9e80dfb5541220946c8b6e6261fe67ab1"
 - !!merge <<: *depth-anything-2-base
   name: depth-anything-2-base-f32
+  variants: []
   description: |
     Depth Anything V2 (base / ViT-B), f32 — maximum reference fidelity. Relative
     monocular depth only (no pose). Use GenerateImage (src -> depth PNG) or the
@@ -11048,6 +11162,7 @@
       sha256: "2d3d2e4d8fae9646c17577b84c870c7d77a34ded8cf8d5da0a60b2bee1530ccc"
 - !!merge <<: *depth-anything-2-base
   name: depth-anything-2-small
+  variants: []
   description: |
     Depth Anything V2 (small / ViT-S), f32 — the smallest, fastest backbone for
     relative monocular depth on CPU. Depth only (no pose). Use GenerateImage
@@ -11062,6 +11177,7 @@
       sha256: "1f6622aa70cbd0eba34d34e2f635a156ed8c3f8e158fb149eb355366e2deb899"
 - !!merge <<: *depth-anything-2-base
   name: depth-anything-2-large
+  variants: []
   description: |
     Depth Anything V2 (large / ViT-L), f32 — higher-quality relative monocular
     depth than base. Depth only (no pose). Use GenerateImage (src -> depth PNG)
@@ -11076,6 +11192,7 @@
       sha256: "e187658b01b6df62e1553b03df9973606a7287551d4771b09802fc10b26f19f3"
 - !!merge <<: *depth-anything-2-base
   name: depth-anything-2-metric-hypersim-small
+  variants: []
   description: |
     Depth Anything V2 Metric (Hypersim, indoor / ViT-S), q4_k — metric monocular
     depth in METRES (indoor, max_depth 20). Depth only (no pose). Use
@@ -11097,6 +11214,7 @@
       sha256: "decd99f5c756b564aae5ca1a1612f896e7f76889060e1d25ba610549bbc39b52"
 - !!merge <<: *depth-anything-2-base
   name: depth-anything-2-metric-hypersim-base
+  variants: []
   description: |
     Depth Anything V2 Metric (Hypersim, indoor / ViT-B), q4_k — metric monocular
     depth in METRES (indoor, max_depth 20). Depth only (no pose). Use
@@ -11118,6 +11236,7 @@
       sha256: "c7c6a8628ac154f2ad43db09b8146518e863375be2cb03b8b46caec49a4fcf3a"
 - !!merge <<: *depth-anything-2-base
   name: depth-anything-2-metric-hypersim-large
+  variants: []
   description: |
     Depth Anything V2 Metric (Hypersim, indoor / ViT-L), q4_k — highest-quality
     metric monocular depth in METRES (indoor, max_depth 20). Depth only (no
@@ -11139,6 +11258,7 @@
       sha256: "3664506bea55e64926fff2cb112ea5a9ad923d13647b9c69617184a89dd1e473"
 - !!merge <<: *depth-anything-2-base
   name: depth-anything-2-metric-vkitti-small
+  variants: []
   description: |
     Depth Anything V2 Metric (Virtual KITTI, outdoor / ViT-S), q4_k — metric
     monocular depth in METRES (outdoor, max_depth 80). Depth only (no pose). Use
@@ -11160,6 +11280,7 @@
       sha256: "8dcaa5d0f8475c3dc5de59e28faacd6d46e5ef73c73ecc58e365d7751bc2279f"
 - !!merge <<: *depth-anything-2-base
   name: depth-anything-2-metric-vkitti-base
+  variants: []
   description: |
     Depth Anything V2 Metric (Virtual KITTI, outdoor / ViT-B), q4_k — metric
     monocular depth in METRES (outdoor, max_depth 80). Depth only (no pose). Use
@@ -11181,6 +11302,7 @@
       sha256: "1de5a7aae674df6afb8fa5e06d67843dccfbab92cd64b7c816c1218229446d6d"
 - !!merge <<: *depth-anything-2-base
   name: depth-anything-2-metric-vkitti-large
+  variants: []
   description: |
     Depth Anything V2 Metric (Virtual KITTI, outdoor / ViT-L), q4_k —
     highest-quality metric monocular depth in METRES (outdoor, max_depth 80).
@@ -12133,6 +12255,8 @@
       sha256: 376902d50612ecfc5bd8b268f376c04d10ad7e480f99a1483b833f04344a549e
       uri: huggingface://MaziyarPanahi/Qwen3-8B-GGUF/Qwen3-8B.Q4_K_M.gguf
 - name: qwen3-4b
+  variants:
+    - model: qwen3-4b-dflash
   url: github:mudler/LocalAI/gallery/qwen3.yaml@master
   urls:
     - https://huggingface.co/Qwen/Qwen3-4B
@@ -29179,6 +29303,8 @@
 - &arch-router-1_5b
   url: github:mudler/LocalAI/gallery/chatml.yaml@master
   name: arch-router-1.5b-q4
+  variants:
+    - model: arch-router-1.5b-q8
   icon: https://cdn-avatars.huggingface.co/v1/production/uploads/66b681906c8d3b36786b764c/uyP7mxDVv0HbV9Hv_KfHk.jpeg
   license: other
   urls:
@@ -29224,6 +29350,7 @@
       uri: huggingface://mradermacher/Arch-Router-1.5B-GGUF/Arch-Router-1.5B.Q4_K_M.gguf
 - !!merge <<: *arch-router-1_5b
   name: arch-router-1.5b-q8
+  variants: []
   overrides:
     known_usecases:
       - score
@@ -30804,6 +30931,9 @@
       sha256: f4df16c641a05c4a6ca717068ba3ee312875000f6fac0efbd152915553b5fc3e
       uri: huggingface://second-state/stable-diffusion-3.5-large-GGUF/t5xxl-Q5_0.gguf
 - name: flux.1-dev
+  variants:
+    - model: flux.1-dev-ggml
+    - model: flux.1-dev-ggml-q8_0
   url: github:mudler/LocalAI/gallery/flux.yaml@master
   urls:
     - https://huggingface.co/black-forest-labs/FLUX.1-dev
@@ -31053,6 +31183,8 @@
       sha256: 6e480b09fae049a72d2a8c5fbccb8d3e92febeb233bbe9dfe7256958a9167635
       uri: https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp16.safetensors
 - name: flux.1-krea-dev-ggml
+  variants:
+    - model: flux.1-krea-dev-ggml-q8_0
   url: github:mudler/LocalAI/gallery/flux-ggml.yaml@master
   urls:
     - https://huggingface.co/black-forest-labs/FLUX.1-Krea-dev
@@ -31446,6 +31578,8 @@
       sha256: 422f1ae452ade6f30a004d7e5c6a43195e4433bc370bf23fac9cc591f01a8898
       uri: huggingface://ggerganov/whisper.cpp/ggml-base-q5_1.bin
 - name: whisper-base
+  variants:
+    - model: whisper-base-q5_1
   url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
   urls:
     - https://github.com/ggerganov/whisper.cpp
@@ -31500,6 +31634,8 @@
       sha256: 4baf70dd0d7c4247ba2b81fafd9c01005ac77c2f9ef064e00dcf195d0e2fdd2f
       uri: huggingface://ggerganov/whisper.cpp/ggml-base.en-q5_1.bin
 - name: whisper-base-en
+  variants:
+    - model: whisper-base-en-q5_1
   url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
   urls:
     - https://github.com/ggerganov/whisper.cpp
@@ -31554,6 +31690,8 @@
       sha256: d75795ecff3f83b5faa89d1900604ad8c780abd5739fae406de19f23ecd98ad1
       uri: huggingface://ggerganov/whisper.cpp/ggml-large-v3-q5_0.bin
 - name: whisper-medium
+  variants:
+    - model: whisper-medium-q5_0
   url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
   urls:
     - https://github.com/ggerganov/whisper.cpp
@@ -31634,6 +31772,8 @@
       sha256: ae85e4a935d7a567bd102fe55afc16bb595bdb618e11b2fc7591bc08120411bb
       uri: huggingface://ggerganov/whisper.cpp/ggml-small-q5_1.bin
 - name: whisper-small
+  variants:
+    - model: whisper-small-q5_1
   url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
   urls:
     - https://github.com/ggerganov/whisper.cpp
@@ -31688,6 +31828,8 @@
       sha256: bfdff4894dcb76bbf647d56263ea2a96645423f1669176f4844a1bf8e478ad30
       uri: huggingface://ggerganov/whisper.cpp/ggml-small.en-q5_1.bin
 - name: whisper-small-en
+  variants:
+    - model: whisper-small-en-q5_1
   url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
   urls:
     - https://github.com/ggerganov/whisper.cpp
@@ -31742,6 +31884,8 @@
       sha256: ae85e4a935d7a567bd102fe55afc16bb595bdb618e11b2fc7591bc08120411bb
       uri: huggingface://ggerganov/whisper.cpp/ggml-small-q5_1.bin
 - name: whisper-tiny
+  variants:
+    - model: whisper-tiny-q5_1
   url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
   urls:
     - https://github.com/ggerganov/whisper.cpp
@@ -31821,6 +31965,9 @@
       sha256: c77c5766f1cef09b6b7d47f21b546cbddd4157886b3b5d6d4f709e91e66c7c2b
       uri: huggingface://ggerganov/whisper.cpp/ggml-tiny.en-q5_1.bin
 - name: whisper-tiny-en
+  variants:
+    - model: whisper-tiny-en-q5_1
+    - model: whisper-tiny-en-q8_0
   url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
   urls:
     - https://github.com/ggerganov/whisper.cpp
@@ -31872,6 +32019,8 @@
       sha256: 5bc2b3860aa151a4c6e7bb095e1fcce7cf12c7b020ca08dcec0c6d018bb7dd94
       uri: huggingface://ggerganov/whisper.cpp/ggml-tiny.en-q8_0.bin
 - name: whisper-large
+  variants:
+    - model: whisper-large-q5_0
   url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
   urls:
     - https://github.com/ggerganov/whisper.cpp
@@ -31925,6 +32074,9 @@
       sha256: d75795ecff3f83b5faa89d1900604ad8c780abd5739fae406de19f23ecd98ad1
       uri: huggingface://ggerganov/whisper.cpp/ggml-large-v3-q5_0.bin
 - name: whisper-large-turbo
+  variants:
+    - model: whisper-large-turbo-q5_0
+    - model: whisper-large-turbo-q8_0
   url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
   urls:
     - https://github.com/ggerganov/whisper.cpp
@@ -35164,6 +35316,8 @@
       sha256: af8cb9e4ca0bf19eb54d08c612fdf325059264abbbd2c619527e5d2dda8de655
       uri: https://huggingface.co/mradermacher/nomic-embed-text-v1.5-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf
 - name: silero-vad
+  variants:
+    - model: silero-vad-ggml
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
     - https://github.com/snakers4/silero-vad
@@ -36458,6 +36612,8 @@
       sha256: 0a599099e93ad521045e17d82365a73c1738fff0603d6cb2c9557e96fbc907cb
       uri: huggingface://mradermacher/YanoljaNEXT-Rosetta-27B-2511-i1-GGUF/YanoljaNEXT-Rosetta-27B-2511.i1-Q4_K_M.gguf
 - name: orca-agent-v0.1
+  variants:
+    - model: orca-agent-v0.1-i1
   url: github:mudler/LocalAI/gallery/qwen3.yaml@master
   urls:
     - https://huggingface.co/mradermacher/Orca-Agent-v0.1-GGUF
@@ -36683,6 +36839,9 @@
       uri: huggingface://unsloth/LTX-2.3-GGUF/text_encoders/ltx-2.3-22b-dev_embeddings_connectors.safetensors
 - !!merge <<: *ltx-2-3-dev-ggml
   name: ltx-2.3-22b-dev-ggml-q4_k_m
+  variants:
+    - model: ltx-2.3-22b-dev-ggml
+    - model: ltx-2.3-22b-dev-ggml-q8_0
   description: |
     LTX-2.3 22B dev - non-dynamic Q4_K_M quantization (~14.3 GB). Same
     pipeline as ltx-2.3-22b-dev-ggml but with the plain Q4_K_M weights
@@ -36810,6 +36969,9 @@
       uri: huggingface://unsloth/LTX-2.3-GGUF/text_encoders/ltx-2.3-22b-distilled_embeddings_connectors.safetensors
 - !!merge <<: *ltx-2-3-distilled-ggml
   name: ltx-2.3-22b-distilled-ggml-q4_k_m
+  variants:
+    - model: ltx-2.3-22b-distilled-ggml
+    - model: ltx-2.3-22b-distilled-ggml-q8_0
   description: |
     LTX-2.3 22B distilled - non-dynamic Q4_K_M quantization (~14.3 GB).
     Same pipeline as ltx-2.3-22b-distilled-ggml but with the plain Q4_K_M
@@ -36958,6 +37120,7 @@
     - gguf
     - llm
     - chat
+    - mtp
   overrides:
     backend: ds4
     options:

pkg/downloader/partial_resume_test.go

@@ -0,0 +1,173 @@
+package downloader_test
+
+import (
+	"crypto/rand"
+	"crypto/sha256"
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"path/filepath"
+	"strconv"
+	"strings"
+	"sync"
+
+	. "github.com/mudler/LocalAI/pkg/downloader"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+// A download interrupted mid-transfer leaves a `<file>.partial` behind. The
+// next attempt has to cope with it, and for a non-http(s) scheme (every
+// gallery entry uses huggingface://) it used to return an error wrapping a nil
+// error, which made the model permanently uninstallable until someone deleted
+// the partial by hand.
+var _ = Describe("DownloadFile with a leftover .partial", func() {
+	var (
+		payload     []byte
+		payloadSHA  string
+		destDir     string
+		destPath    string
+		partialPath string
+		noProgress  = func(string, string, string, float64) {}
+	)
+
+	// rangeServer serves payload at any path, optionally honouring Range
+	// requests, and records the Range headers it received so a spec can tell
+	// a resume apart from a restart.
+	rangeServer := func(supportsRange bool, seenRanges *[]string, mu *sync.Mutex) *httptest.Server {
+		return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			if r.Method == http.MethodHead {
+				if supportsRange {
+					w.Header().Set("Accept-Ranges", "bytes")
+				}
+				w.WriteHeader(http.StatusOK)
+				return
+			}
+			rangeHeader := r.Header.Get("Range")
+			if seenRanges != nil {
+				mu.Lock()
+				*seenRanges = append(*seenRanges, rangeHeader)
+				mu.Unlock()
+			}
+			start := 0
+			if supportsRange && strings.HasPrefix(rangeHeader, "bytes=") {
+				spec := strings.TrimPrefix(rangeHeader, "bytes=")
+				n, err := strconv.Atoi(strings.TrimSuffix(spec, "-"))
+				Expect(err).ToNot(HaveOccurred())
+				start = n
+			}
+			body := payload[start:]
+			w.Header().Set("Content-Length", strconv.Itoa(len(body)))
+			if start > 0 {
+				w.Header().Set("Content-Range", fmt.Sprintf("bytes %d-%d/%d", start, len(payload)-1, len(payload)))
+				w.WriteHeader(http.StatusPartialContent)
+			} else {
+				w.WriteHeader(http.StatusOK)
+			}
+			_, _ = w.Write(body)
+		}))
+	}
+
+	writePartial := func(content []byte) {
+		Expect(os.WriteFile(partialPath, content, 0600)).To(Succeed())
+	}
+
+	expectDownloadedPayload := func() {
+		got, err := os.ReadFile(destPath)
+		Expect(err).ToNot(HaveOccurred())
+		Expect(got).To(Equal(payload))
+		Expect(partialPath).ToNot(BeAnExistingFile())
+	}
+
+	BeforeEach(func() {
+		payload = make([]byte, 16384)
+		_, err := rand.Read(payload)
+		Expect(err).ToNot(HaveOccurred())
+		sum := sha256.Sum256(payload)
+		payloadSHA = fmt.Sprintf("%x", sum[:])
+
+		destDir, err = os.MkdirTemp("", "partial-resume")
+		Expect(err).ToNot(HaveOccurred())
+		destPath = filepath.Join(destDir, "model.gguf")
+		partialPath = destPath + PartialFileSuffix
+	})
+
+	AfterEach(func() {
+		_ = os.RemoveAll(destDir)
+	})
+
+	// The reported bug: `huggingface://` is not an http(s) URI, so the resume
+	// branch was skipped and the fall-through reported a stat error that had
+	// not happened.
+	It("restarts and produces a correct file for a non-HTTP URI", func() {
+		server := rangeServer(true, nil, nil)
+		defer server.Close()
+
+		originalEndpoint := HF_ENDPOINT
+		HF_ENDPOINT = server.URL
+		defer func() { HF_ENDPOINT = originalEndpoint }()
+
+		writePartial([]byte("stale bytes from an interrupted download"))
+
+		uri := URI("huggingface://owner/repo/model.gguf")
+		Expect(uri.DownloadFile(destPath, payloadSHA, 1, 1, noProgress)).To(Succeed())
+		expectDownloadedPayload()
+	})
+
+	It("resumes an HTTP download when the server supports ranges", func() {
+		var mu sync.Mutex
+		var seenRanges []string
+		server := rangeServer(true, &seenRanges, &mu)
+		defer server.Close()
+
+		writePartial(payload[:8192])
+
+		Expect(URI(server.URL).DownloadFile(destPath, payloadSHA, 1, 1, noProgress)).To(Succeed())
+		expectDownloadedPayload()
+
+		mu.Lock()
+		defer mu.Unlock()
+		Expect(seenRanges).To(ContainElement("bytes=8192-"),
+			"expected a Range request resuming from the partial, got %v", seenRanges)
+	})
+
+	It("restarts an HTTP download when the server does not support ranges", func() {
+		var mu sync.Mutex
+		var seenRanges []string
+		server := rangeServer(false, &seenRanges, &mu)
+		defer server.Close()
+
+		writePartial([]byte("stale bytes"))
+
+		Expect(URI(server.URL).DownloadFile(destPath, payloadSHA, 1, 1, noProgress)).To(Succeed())
+		expectDownloadedPayload()
+
+		mu.Lock()
+		defer mu.Unlock()
+		Expect(seenRanges).To(ConsistOf(""), "expected a full restart, got ranges %v", seenRanges)
+	})
+
+	It("downloads normally when no partial is present", func() {
+		server := rangeServer(true, nil, nil)
+		defer server.Close()
+
+		Expect(URI(server.URL).DownloadFile(destPath, payloadSHA, 1, 1, noProgress)).To(Succeed())
+		expectDownloadedPayload()
+	})
+
+	It("reports an informative error when the partial cannot be stat'd", func() {
+		server := rangeServer(true, nil, nil)
+		defer server.Close()
+
+		// A symlink pointing at itself makes os.Stat fail with ELOOP: a real
+		// stat failure that is not "does not exist".
+		Expect(os.Symlink(partialPath, partialPath)).To(Succeed())
+
+		err := URI(server.URL).DownloadFile(destPath, payloadSHA, 1, 1, noProgress)
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring(partialPath))
+		Expect(err.Error()).ToNot(ContainSubstring("<nil>"))
+		Expect(destPath).ToNot(BeAnExistingFile())
+	})
+})

pkg/downloader/uri.go

@@ -614,23 +614,34 @@ func (uri URI) DownloadFileWithContext(ctx context.Context, filePath, sha string
 	// save partial download to dedicated file
 	tmpFilePath := filePath + ".partial"
 	var startPos int64
-	tmpFileInfo, err := os.Stat(tmpFilePath)
-	if err == nil && uri.LooksLikeHTTPURL() {
-		support, err := uri.checkServerSupportsRangeHeader(ctx, dopts.bearerToken)
-		if err != nil {
-			return fmt.Errorf("failed to check if uri server supports range header: %v", err)
+	tmpFileInfo, statErr := os.Stat(tmpFilePath)
+	switch {
+	case statErr == nil:
+		// A leftover partial is only usable when we can ask the server to
+		// continue from where it stopped. Resume is probed only for raw
+		// http(s) URIs; every other transport (local files, and schemes we do
+		// not probe) has to restart, because the writer opens the partial with
+		// O_APPEND and would otherwise concatenate a fresh full body onto the
+		// stale bytes. Discarding here is what makes a retry after an
+		// interrupted download recover on its own instead of failing forever.
+		resumable := false
+		if uri.LooksLikeHTTPURL() {
+			support, err := uri.checkServerSupportsRangeHeader(ctx, dopts.bearerToken)
+			if err != nil {
+				return fmt.Errorf("failed to check if uri server supports range header: %v", err)
+			}
+			resumable = support
 		}
-		if support {
+		if resumable {
 			startPos = tmpFileInfo.Size()
 			req.Header.Set("Range", fmt.Sprintf("bytes=%d-", startPos))
-		} else {
-			err := removePartialFile(tmpFilePath)
-			if err != nil {
-				return err
-			}
+		} else if err := removePartialFile(tmpFilePath); err != nil {
+			return err
 		}
-	} else if !errors.Is(err, os.ErrNotExist) {
-		return fmt.Errorf("failed to check file %q existence: %v", filePath, err)
+	case errors.Is(statErr, os.ErrNotExist):
+		// Nothing to resume or discard: this is a fresh download.
+	default:
+		return fmt.Errorf("failed to check partial download file %q: %w", tmpFilePath, statErr)
 	}
 
 	var source io.ReadCloser

pkg/mcp/localaitools/dto.go

@@ -67,6 +67,7 @@ type InstallModelRequest struct {
 	GalleryName string         `json:"gallery_name,omitempty" jsonschema:"The gallery the model lives in (from gallery_search). Optional when ModelName is unique across galleries."`
 	ModelName   string         `json:"model_name"             jsonschema:"The canonical model name as returned by gallery_search."`
 	Overrides   map[string]any `json:"overrides,omitempty"    jsonschema:"Optional config overrides to merge into the installed model's YAML."`
+	Variant     string         `json:"variant,omitempty"      jsonschema:"Optional. Installs one specific build of an entry that offers several (different quantizations or engines), named exactly as it appears in that entry's variant list. Leave empty unless the user explicitly asked for a particular build: by default LocalAI drops builds this machine cannot run or cannot fit, prefers the engine this hardware is best served by, and uses size only to decide between equally preferred builds."`
 }
 
 // InstallBackendRequest is the input for install_backend.

pkg/mcp/localaitools/httpapi/client.go

@@ -252,6 +252,9 @@ func (c *Client) InstallModel(ctx context.Context, req localaitools.InstallModel
 	if len(req.Overrides) > 0 {
 		body["overrides"] = req.Overrides
 	}
+	if req.Variant != "" {
+		body["variant"] = req.Variant
+	}
 	var resp struct {
 		ID        string `json:"uuid"`
 		StatusURL string `json:"status"`

pkg/mcp/localaitools/httpapi/client_test.go

@@ -129,6 +129,39 @@ var _ = Describe("httpapi.Client against the LocalAI admin REST surface", func()
 			Expect(err).ToNot(HaveOccurred())
 			Expect(id).To(Equal("job-123"))
 		})
+
+		It("forwards a chosen variant to the apply endpoint", func() {
+			// The assistant's whole ability to honor "install the Q8 one"
+			// rests on this field surviving the hop to REST.
+			var body map[string]any
+			srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+				Expect(json.NewDecoder(r.Body).Decode(&body)).To(Succeed())
+				_ = json.NewEncoder(w).Encode(map[string]any{"uuid": "job-123"})
+			}))
+			DeferCleanup(srv.Close)
+
+			_, err := New(srv.URL, "").InstallModel(context.Background(), localaitools.InstallModelRequest{
+				ModelName: "qwen2.5-7b-instruct",
+				Variant:   "qwen2.5-7b-instruct-q8",
+			})
+			Expect(err).ToNot(HaveOccurred())
+			Expect(body).To(HaveKeyWithValue("variant", "qwen2.5-7b-instruct-q8"))
+		})
+
+		It("omits the variant when none was chosen, so the server auto-selects", func() {
+			var body map[string]any
+			srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+				Expect(json.NewDecoder(r.Body).Decode(&body)).To(Succeed())
+				_ = json.NewEncoder(w).Encode(map[string]any{"uuid": "job-123"})
+			}))
+			DeferCleanup(srv.Close)
+
+			_, err := New(srv.URL, "").InstallModel(context.Background(), localaitools.InstallModelRequest{
+				ModelName: "qwen2.5-7b-instruct",
+			})
+			Expect(err).ToNot(HaveOccurred())
+			Expect(body).ToNot(HaveKey("variant"))
+		})
 	})
 
 	Describe("GetJobStatus", func() {

pkg/mcp/localaitools/inproc/client.go

@@ -203,6 +203,7 @@ func (c *Client) InstallModel(ctx context.Context, req localaitools.InstallModel
 		Req: gallery.GalleryModel{
 			Metadata: gallery.Metadata{Name: req.ModelName},
 		},
+		Variant:          req.Variant,
 		Galleries:        galleries,
 		BackendGalleries: c.AppConfig.BackendGalleries,
 	}

pkg/mcp/localaitools/prompts/skills/install_chat_model.md

@@ -6,7 +6,7 @@ Use this when the user wants to install a chat-capable model — including the c
 2. Show the top results as a numbered list with name, gallery, short description, and license. If none match, say so and ask whether to broaden the search.
 3. Wait for the user to pick.
 4. Summarise the chosen install ("I'll install **`<gallery>/<name>`** — confirm?") and wait for confirmation.
-5. On confirmation, call `install_model` with `gallery_name` and `model_name` from the chosen hit.
+5. On confirmation, call `install_model` with `gallery_name` and `model_name` from the chosen hit. Leave `variant` empty: LocalAI then auto-selects the largest build this machine's engines and free memory can actually run. Only set `variant` when the user names a specific build (e.g. "the Q8 one"), and pass the name exactly as the entry lists it: an unknown name fails the install rather than falling back to auto-selection.
 6. Poll `get_job_status` with the returned job id. Report meaningful progress changes (every ~10–20%, plus completion).
 7. When the job reports `processed: true` and no error, call `reload_models`, then `list_installed_models` with `capability: "chat"` to confirm the model is now visible.
 8. Tell the user the model is ready and how to use it (its name as the `model` field in chat completions).

pkg/mcp/localaitools/tools_models.go

@@ -83,7 +83,7 @@ func registerModelTools(s *mcp.Server, client LocalAIClient, opts Options) {
 
 	mcp.AddTool(s, &mcp.Tool{
 		Name:        ToolInstallModel,
-		Description: "Install a model from a gallery. Requires explicit user confirmation per safety rule 1. Returns a job id; poll with get_job_status.",
+		Description: "Install a model from a gallery. Some entries offer several variants (quantizations or engines) of the same model; leaving `variant` empty auto-selects the largest one this machine can actually run, which is almost always what the user wants. Set `variant` only when the user names a specific build. Requires explicit user confirmation per safety rule 1. Returns a job id; poll with get_job_status.",
 	}, func(ctx context.Context, _ *mcp.CallToolRequest, args InstallModelRequest) (*mcp.CallToolResult, any, error) {
 		// Empty-string check at the tool layer: the SDK schema validator
 		// only enforces presence, not non-empty, and we want a consistent

pkg/system/capabilities.go

@@ -6,6 +6,7 @@ import (
 	"os"
 	"path/filepath"
 	"runtime"
+	"slices"
 	"strings"
 
 	"github.com/mudler/xlog"
@@ -18,8 +19,8 @@ const (
 	Intel  = "intel"
 
 	// Private constants - only used within this package
-	defaultCapability  = "default"
-	disableCapability  = "disable"
+	defaultCapability = "default"
+	disableCapability = "disable"
 	nvidiaL4T         = "nvidia-l4t"
 	darwinX86         = "darwin-x86"
 	metal             = "metal"
@@ -43,8 +44,214 @@ const (
 	backendTokenROCM   = "rocm"
 	backendTokenHIP    = "hip"
 	backendTokenSYCL   = "sycl"
+	backendTokenCPU    = "cpu"
+
+	// Engine names (private). Unlike the tokens above these are whole backend
+	// identities as a gallery entry's `backend:` field spells them, not build
+	// tags. See the two preference tables below for why the distinction matters.
+	engineVLLM     = "vllm"
+	engineSGLang   = "sglang"
+	engineLlamaCpp = "llama-cpp"
+	engineMLX      = "mlx"
+
+	// Serving feature names (private). A third vocabulary again: not a build
+	// tag and not an engine, but the inference-time strategy a published build
+	// enables. They are matched against a gallery entry's declared TAGS and
+	// nothing else, so they are spelled the way gallery authors spell a tag.
+	servingFeatureDFlash = "dflash"
+	servingFeatureMTP    = "mtp"
 )
 
+// There are THREE preference tables below and they speak DIFFERENT VOCABULARIES,
+// matched against three different things. Merging any two looks tempting and
+// silently breaks a consumer, because a token that means something in one
+// vocabulary means nothing in the others. Read this before editing any of them.
+//
+//   - backendBuildTagPreferenceRules holds BUILD TAGS ("cuda", "rocm", "metal").
+//     They are matched as substrings of INSTALLED BACKEND BUILD DIRECTORY NAMES
+//     such as "llama-cpp-cuda-12" or "cuda12-vllm". Consumer: alias resolution
+//     in ListSystemBackends (core/gallery/backends.go), which picks which
+//     installed build of one alias to run.
+//
+//   - engineNamePreferenceRules holds ENGINE NAMES ("vllm", "llama-cpp", "mlx").
+//     They are matched as substrings of a gallery entry's `backend:` value,
+//     which never carries a build tag: no entry in gallery/index.yaml contains
+//     "cuda", "rocm", "sycl" or "vulkan" anywhere in its backend name. Consumer:
+//     gallery variant auto-selection (core/gallery/resolve_variant.go), which
+//     picks which build of one model's weights to install.
+//
+//   - servingFeaturePreferenceTokens holds SERVING FEATURES ("dflash", "mtp").
+//     They are matched against a gallery entry's declared TAGS, whole and
+//     case-insensitively, rather than against a backend, an engine or the entry
+//     name. Same consumer as the engine table, applied one rank below it. See
+//     the token list for why a tag is the only signal.
+//
+// Feeding build tags to the variant ranker matches nothing, which does not
+// error: every candidate simply scores equal and size alone decides, so the
+// preference silently stops existing. That is exactly the bug this split fixes.
+// A serving feature fed to either of the other two tables fails the same way.
+
+// backendPreferenceRule maps a detected capability to preferred tokens, best
+// first. Both tables share this shape; only their vocabulary differs.
+type backendPreferenceRule struct {
+	capabilityPrefix string
+	tokens           []string
+}
+
+// backendBuildTagPreferenceRules is the BUILD TAG table. See the block comment
+// above for the vocabulary contract.
+//
+// Matching is by capability PREFIX, because a detected capability is refined at
+// runtime ("nvidia" becomes "nvidia-cuda-12" when the toolkit is present) and a
+// preference is about the vendor rather than the point release. Rules are tried
+// in order, so a more specific prefix must precede any rule it shares a prefix
+// with.
+//
+// Tokens are matched as substrings of a build directory name, so "cuda" covers
+// "cuda12-llama-cpp". A build matching no token is not an error and is never
+// discarded; it simply sorts below every recognised one.
+var backendBuildTagPreferenceRules = []backendPreferenceRule{
+	{Nvidia, []string{backendTokenCUDA, vulkan, backendTokenCPU}},
+	{AMD, []string{backendTokenROCM, backendTokenHIP, vulkan, backendTokenCPU}},
+	{Intel, []string{backendTokenSYCL, Intel, backendTokenCPU}},
+	{metal, []string{backendTokenMetal, backendTokenCPU}},
+	{darwinX86, []string{darwinX86, backendTokenCPU}},
+	{vulkan, []string{vulkan, backendTokenCPU}},
+}
+
+// defaultBackendBuildTagTokens is what a host with no matching rule prefers.
+// A capability nobody has taught this table about degrades to plain CPU rather
+// than to an error or to an empty list.
+var defaultBackendBuildTagTokens = []string{backendTokenCPU}
+
+// engineNamePreferenceRules is the ENGINE NAME table. See the block comment
+// above for the vocabulary contract.
+//
+// Capability matching is by prefix, exactly as the build tag table does it.
+//
+// Tokens are matched as SUBSTRINGS of an engine name, which is load bearing:
+// "vllm" also covers "vllm-omni", "mlx" also covers "mlx-vlm" and "mlx-audio",
+// and "llama-cpp" also covers "ik-llama-cpp". Each of those is a build of the
+// engine named by the token, so ranking them together is correct. Order the
+// tokens so no token is a substring of an engine that should rank differently.
+//
+// A capability is deliberately ABSENT rather than guessed at when no engine
+// ordering can be justified for it. An absent rule degrades to ordering by size
+// alone, which is the behaviour that predates preference and is always safe.
+//
+// Safe, however, only where every candidate engine is equally at home on the
+// host. Where one is not, the rule has to be written here, because the hardware
+// filter will not write it: IsBackendCompatible derives support from the engine
+// NAME, and "vllm" and "sglang" contain none of the darwin, cuda, rocm or sycl
+// tokens it keys on, so a GPU serving engine is never filtered out on a host
+// with no GPU. Left unranked there, it wins on size alone whenever its build is
+// the larger one, and a CPU-only box installs vLLM over llama.cpp.
+var engineNamePreferenceRules = []backendPreferenceRule{
+	// vLLM first on every host with a dedicated serving engine build. It is the
+	// throughput engine, and a model published with a vLLM build is published
+	// that way precisely because that build is the one worth running.
+	// SGLang sits directly behind it: same class of GPU serving engine, ships
+	// cuda/rocm/intel builds alike, but it is behind vLLM because vLLM covers
+	// far more of the gallery. llama-cpp is the portable fallback.
+	{Nvidia, []string{engineVLLM, engineSGLang, engineLlamaCpp}},
+	{AMD, []string{engineVLLM, engineSGLang, engineLlamaCpp}},
+	{Intel, []string{engineVLLM, engineSGLang, engineLlamaCpp}},
+	// MLX is the native accelerated runtime on Apple silicon, whereas a
+	// metal-enabled GGUF build is the portable engine merely compiled with GPU
+	// offload. No vLLM or SGLang build targets metal, so neither is listed.
+	{metal, []string{engineMLX, engineLlamaCpp}},
+	// A Vulkan host has exactly one LLM engine with a Vulkan build, so a
+	// llama-cpp variant is the only one that will use the GPU at all.
+	{vulkan, []string{engineLlamaCpp}},
+	// No usable accelerator: either nothing was detected, or a GPU was found
+	// with too little VRAM to serve from, both of which report "default".
+	// llama.cpp is the engine built to run well on a CPU; vLLM and SGLang are
+	// GPU serving engines whose CPU paths exist but are not what an unattended
+	// auto-selection should hand a CPU-only box.
+	//
+	// The GPU engines are enumerated behind llama.cpp rather than left
+	// unmatched. Listing llama.cpp alone would already make it win, since an
+	// unmatched engine ranks below every listed one, but it would leave vLLM
+	// and SGLang tied with each other and with any engine nobody has ranked
+	// yet, so download size would decide among them. Naming them fixes that
+	// order and says the omission of a GPU engine from the top spot is a
+	// decision rather than an oversight. Ranking never filters, so a vLLM
+	// variant offered alone is still installed here.
+	{defaultCapability, []string{engineLlamaCpp, engineVLLM, engineSGLang}},
+	// An Intel Mac has no accelerated runtime at all: MLX needs Apple silicon,
+	// and no vLLM or SGLang build targets darwin. Same ordering, same reasons.
+	// MLX is deliberately left unlisted so it ranks last, since
+	// IsBackendCompatible admits any darwin-tokened engine on this capability
+	// and will not drop it.
+	{darwinX86, []string{engineLlamaCpp, engineVLLM, engineSGLang}},
+}
+
+// defaultEnginePreferenceTokens is empty on purpose. It is now only reached by
+// a capability nobody has taught this table about, which an operator can force
+// via LOCALAI_FORCE_META_BACKEND_CAPABILITY; the detected "default" has its own
+// rule above. Guessing an order for hardware we know nothing about would be
+// worse than ordering by size alone, which is what the ranker reads an empty
+// list as and is the behaviour that predates preference.
+var defaultEnginePreferenceTokens = []string{}
+
+// servingFeaturePreferenceTokens is the SERVING FEATURE table, best first. See
+// the block comment above for the vocabulary contract.
+//
+// A serving feature is a way of running the same weights faster rather than a
+// different set of weights: a DFlash build pairs the base GGUF with a drafter
+// so the target accepts several speculated tokens per step, and an MTP build
+// carries extra prediction heads to the same end. Both produce the same
+// distribution as the plain build, so whenever one fits there is no reason to
+// install the plain build instead.
+//
+// DFlash leads because it is the newer pairing and accepts more tokens per step
+// in practice; MTP follows; a build naming neither is plain and ranks last.
+// A build's extra weights make it strictly larger than the plain build, which
+// is why this ranks BELOW fit: the size filter drops the pairing on a host too
+// small for it before this list is ever consulted.
+//
+// Unlike the two tables above this one is NOT keyed by capability. A serving
+// feature is a property of the published build, and no hardware prefers the
+// plain build over an equivalent faster one, so there is no host-shaped
+// ordering to express. Should one ever appear, this becomes a rule table like
+// its neighbours without any consumer changing.
+//
+// A token is matched against an entry's declared TAGS, compared whole and
+// case-insensitively, and against NOTHING ELSE. A tag is the declaration: an
+// author writing "mtp" in a tag list means the feature, so it can be compared
+// exactly without the ambiguity free text carries.
+//
+// Two rejected alternatives, because both look like obvious improvements:
+//
+//   - The ENTRY NAME. It was the original signal and is now gone. A naming
+//     convention is not a contract, and a name is author-supplied free text
+//     where a short marker turns up inside unrelated words ("smtp-assistant"),
+//     or names weights that carry MTP heads without the entry enabling
+//     anything ("qwen3.6-27b-nvfp4-mtp", whose only option is use_jinja).
+//     Reading a name infers a capability nobody declared.
+//
+//   - overrides.options, where "spec_type:draft-mtp" is what actually turns
+//     the feature on. That spelling is llama.cpp's config vocabulary. ds4
+//     spells the same feature "mtp_path" and sglang spells it
+//     "speculative_algorithm" in a referenced config file, so keying a
+//     cross-backend ranking decision on one backend's option syntax would
+//     rank the other backends' builds as plain.
+//
+// Options are the CURATION-TIME check instead: a gallery entry earns the tag
+// when it configures the feature in whatever vocabulary its backend uses. The
+// rule is written down in .agents/adding-gallery-models.md, and the
+// backend-specific syntax never reaches the selection logic.
+var servingFeaturePreferenceTokens = []string{servingFeatureDFlash, servingFeatureMTP}
+
+// ServingFeaturePreferenceTokens returns the serving features to prefer, best
+// first, for ranking gallery model variants against each other.
+//
+// It is a package function rather than a SystemState method precisely because
+// the answer does not depend on the host; see the table for why.
+func ServingFeaturePreferenceTokens() []string {
+	return slices.Clone(servingFeaturePreferenceTokens)
+}
+
 var (
 	cuda13DirExists bool
 	cuda12DirExists bool
@@ -219,28 +426,52 @@ func (s *SystemState) getSystemCapabilities() string {
 // backend implementation order for the current system capability. Callers can use
 // these tokens to select the most appropriate concrete backend among multiple
 // candidates sharing the same alias (e.g., "llama-cpp").
+//
+// These are BUILD TAGS matched against installed build directory names. For
+// engine names as a gallery entry spells them, use EnginePreferenceTokens.
 func (s *SystemState) BackendPreferenceTokens() []string {
-	capStr := strings.ToLower(s.getSystemCapabilities())
-	switch {
-	case strings.HasPrefix(capStr, Nvidia):
-		return []string{backendTokenCUDA, vulkan, "cpu"}
-	case strings.HasPrefix(capStr, AMD):
-		return []string{backendTokenROCM, backendTokenHIP, vulkan, "cpu"}
-	case strings.HasPrefix(capStr, Intel):
-		return []string{backendTokenSYCL, Intel, "cpu"}
-	case strings.HasPrefix(capStr, metal):
-		return []string{backendTokenMetal, "cpu"}
-	case strings.HasPrefix(capStr, darwinX86):
-		return []string{"darwin-x86", "cpu"}
-	case strings.HasPrefix(capStr, vulkan):
-		return []string{vulkan, "cpu"}
-	default:
-		return []string{"cpu"}
-	}
+	return s.preferenceTokens(backendBuildTagPreferenceRules, defaultBackendBuildTagTokens)
 }
 
-// DetectedCapability returns the detected system capability string.
+// EnginePreferenceTokens returns the engine names this host prefers, best first,
+// for ranking gallery model variants against each other.
+//
+// These are ENGINE NAMES matched against a gallery entry's `backend:` value
+// ("vllm", "llama-cpp", "mlx"), never build tags. Feeding this function's output
+// to installed-build alias resolution, or BackendPreferenceTokens' output to
+// variant ranking, matches nothing and silently disables the preference.
+//
+// An empty result is normal and means "no engine ordering applies here", which
+// the ranker reads as ordering by size alone.
+func (s *SystemState) EnginePreferenceTokens() []string {
+	return s.preferenceTokens(engineNamePreferenceRules, defaultEnginePreferenceTokens)
+}
+
+// preferenceTokens resolves the current capability against one preference table.
+// The rules live in the tables above; this only looks them up, so teaching
+// LocalAI about a new runtime never means editing logic.
+func (s *SystemState) preferenceTokens(rules []backendPreferenceRule, fallback []string) []string {
+	capStr := strings.ToLower(s.getSystemCapabilities())
+	for _, rule := range rules {
+		if strings.HasPrefix(capStr, rule.capabilityPrefix) {
+			// Copied so a caller cannot mutate the shared table out from under
+			// every other host lookup.
+			return slices.Clone(rule.tokens)
+		}
+	}
+	return slices.Clone(fallback)
+}
+
+// DetectedCapability returns the raw detected capability string (e.g. "metal",
+// "nvidia-cuda-12", "default") with no map-membership fallback applied.
 // This can be used by the UI to display what capability was detected.
+//
+// Why this exists alongside Capability: Capability resolves against a caller
+// supplied map and falls back to "default" then "cpu" when the detected value
+// is absent from that map, so its answer describes what that caller can serve
+// rather than what the hardware is. A caller reasoning about the hardware
+// itself, or reporting it to a human, cannot tell a genuinely detected
+// "default" apart from a substituted one and needs the undecorated value.
 func (s *SystemState) DetectedCapability() string {
 	return s.getSystemCapabilities()
 }

pkg/system/capabilities_test.go

@@ -128,6 +128,159 @@ var _ = Describe("getSystemCapabilities", func() {
 	)
 })
 
+var _ = Describe("BackendPreferenceTokens", func() {
+	var origEnv string
+
+	BeforeEach(func() {
+		origEnv = os.Getenv(capabilityEnv)
+	})
+
+	AfterEach(func() {
+		if origEnv != "" {
+			Expect(os.Setenv(capabilityEnv, origEnv)).To(Succeed())
+		} else {
+			Expect(os.Unsetenv(capabilityEnv)).To(Succeed())
+		}
+	})
+
+	tokensFor := func(capability string) []string {
+		GinkgoHelper()
+		Expect(os.Setenv(capabilityEnv, capability)).To(Succeed())
+		return (&SystemState{}).BackendPreferenceTokens()
+	}
+
+	// This table is a REGRESSION LOCK, not a description. These are build tags
+	// matched against installed backend build directory names, and the only
+	// consumer is alias resolution in ListSystemBackends. Every value below is
+	// the output this function had before engine preference existed.
+	//
+	// Engine names ("vllm", "mlx", "llama-cpp") belong to EnginePreferenceTokens
+	// and MUST NOT appear here. Merging the two vocabularies is exactly the
+	// mistake this table exists to catch: it does not error, it silently makes
+	// one of the two consumers rank everything equally.
+	DescribeTable("returns the original build tags for every capability",
+		func(capability string, want []string) {
+			Expect(tokensFor(capability)).To(Equal(want))
+		},
+		Entry("nvidia", "nvidia-cuda-12", []string{"cuda", "vulkan", "cpu"}),
+		Entry("amd", AMD, []string{"rocm", "hip", "vulkan", "cpu"}),
+		Entry("intel", Intel, []string{"sycl", "intel", "cpu"}),
+		Entry("metal", metal, []string{"metal", "cpu"}),
+		Entry("darwin-x86", darwinX86, []string{"darwin-x86", "cpu"}),
+		Entry("vulkan", vulkan, []string{"vulkan", "cpu"}),
+		Entry("unknown capability", "some-future-accelerator", []string{"cpu"}),
+	)
+
+	It("carries no engine name in any capability's tokens", func() {
+		// The generic form of the lock above: whatever anyone adds to the build
+		// tag table later, an engine name in it is a merged vocabulary.
+		engineNames := []string{"vllm", "sglang", "llama-cpp", "mlx"}
+		for _, capability := range []string{"nvidia", AMD, Intel, metal, darwinX86, vulkan, "default"} {
+			Expect(tokensFor(capability)).ToNot(ContainElements(engineNames),
+				"capability %q leaked an engine name into the build tag table", capability)
+		}
+	})
+
+	It("matches a capability by prefix so a refined one keeps its vendor order", func() {
+		Expect(tokensFor("nvidia-l4t-cuda-13")).To(Equal(tokensFor("nvidia")))
+	})
+
+	It("hands out a copy so one caller cannot corrupt another's lookup", func() {
+		first := tokensFor("nvidia")
+		first[0] = "clobbered"
+		Expect(tokensFor("nvidia")[0]).To(Equal("cuda"))
+	})
+})
+
+var _ = Describe("EnginePreferenceTokens", func() {
+	var origEnv string
+
+	BeforeEach(func() {
+		origEnv = os.Getenv(capabilityEnv)
+	})
+
+	AfterEach(func() {
+		if origEnv != "" {
+			Expect(os.Setenv(capabilityEnv, origEnv)).To(Succeed())
+		} else {
+			Expect(os.Unsetenv(capabilityEnv)).To(Succeed())
+		}
+	})
+
+	tokensFor := func(capability string) []string {
+		GinkgoHelper()
+		Expect(os.Setenv(capabilityEnv, capability)).To(Succeed())
+		return (&SystemState{}).EnginePreferenceTokens()
+	}
+
+	It("puts vLLM ahead of llama.cpp on nvidia", func() {
+		// The rule the whole engine table exists to express. A build tag list
+		// here would match no engine name at all and this would read empty.
+		Expect(tokensFor("nvidia-cuda-12")).To(Equal([]string{"vllm", "sglang", "llama-cpp"}))
+	})
+
+	It("puts MLX ahead of llama.cpp on apple silicon", func() {
+		Expect(tokensFor(metal)).To(Equal([]string{"mlx", "llama-cpp"}))
+	})
+
+	It("gives the GPU serving engines to every vendor that has builds of them", func() {
+		Expect(tokensFor(AMD)).To(Equal([]string{"vllm", "sglang", "llama-cpp"}))
+		Expect(tokensFor(Intel)).To(Equal([]string{"vllm", "sglang", "llama-cpp"}))
+	})
+
+	It("prefers llama.cpp on vulkan, the only engine with a vulkan build", func() {
+		Expect(tokensFor(vulkan)).To(Equal([]string{"llama-cpp"}))
+	})
+
+	It("names only engines, never a build tag", func() {
+		// The mirror of the lock on BackendPreferenceTokens. A build tag here
+		// matches no gallery `backend:` value and silently disables ranking.
+		buildTags := []string{"cuda", "rocm", "hip", "sycl", "vulkan", "metal", "cpu", "darwin-x86"}
+		for _, capability := range []string{"nvidia", AMD, Intel, metal, vulkan, defaultCapability, darwinX86} {
+			Expect(tokensFor(capability)).ToNot(ContainElements(buildTags),
+				"capability %q leaked a build tag into the engine table", capability)
+		}
+	})
+
+	It("prefers llama.cpp on a host with no usable accelerator", func() {
+		// Nothing filters a GPU serving engine out here: IsBackendCompatible
+		// keys on the engine name, and "vllm" carries no hardware token. Without
+		// this rule a larger vLLM build would win a CPU-only host on size.
+		Expect(tokensFor(defaultCapability)).To(Equal([]string{"llama-cpp", "vllm", "sglang"}))
+	})
+
+	It("prefers llama.cpp on an intel mac, where nothing accelerates", func() {
+		Expect(tokensFor(darwinX86)).To(Equal([]string{"llama-cpp", "vllm", "sglang"}))
+	})
+
+	It("ranks the GPU serving engines below llama.cpp rather than leaving them tied", func() {
+		// Enumerating them is what stops download size deciding between vLLM and
+		// SGLang once llama.cpp is out of the running.
+		for _, capability := range []string{defaultCapability, darwinX86} {
+			tokens := tokensFor(capability)
+			Expect(tokens).To(HaveLen(3), "capability %q", capability)
+			Expect(tokens[0]).To(Equal("llama-cpp"), "capability %q", capability)
+		}
+	})
+
+	It("leaves a capability nobody has taught the table about empty rather than guessing", func() {
+		// Empty is read by the variant ranker as "order by size alone", which is
+		// the behaviour that predates preference and is always safe. Only a
+		// forced, unrecognised capability lands here now.
+		Expect(tokensFor("some-future-accelerator")).To(BeEmpty())
+	})
+
+	It("matches a capability by prefix so a refined one keeps its engine order", func() {
+		Expect(tokensFor("nvidia-l4t-cuda-13")).To(Equal(tokensFor("nvidia")))
+	})
+
+	It("hands out a copy so one caller cannot corrupt another's lookup", func() {
+		first := tokensFor("nvidia")
+		first[0] = "clobbered"
+		Expect(tokensFor("nvidia")[0]).To(Equal("vllm"))
+	})
+})
+
 var _ = Describe("CapabilityFilterDisabled", func() {
 	var origEnv string
 
@@ -137,9 +290,9 @@ var _ = Describe("CapabilityFilterDisabled", func() {
 
 	AfterEach(func() {
 		if origEnv != "" {
-			os.Setenv(capabilityEnv, origEnv)
+			Expect(os.Setenv(capabilityEnv, origEnv)).To(Succeed())
 		} else {
-			os.Unsetenv(capabilityEnv)
+			Expect(os.Unsetenv(capabilityEnv)).To(Succeed())
 		}
 	})
 
@@ -164,3 +317,71 @@ var _ = Describe("CapabilityFilterDisabled", func() {
 		Expect(s.IsBackendCompatible("cpu-whisperx", "quay.io/cpu")).To(BeTrue())
 	})
 })
+
+var _ = Describe("DetectedCapability", func() {
+	var previous string
+
+	BeforeEach(func() {
+		previous = os.Getenv("LOCALAI_FORCE_META_BACKEND_CAPABILITY")
+	})
+
+	AfterEach(func() {
+		Expect(os.Setenv("LOCALAI_FORCE_META_BACKEND_CAPABILITY", previous)).To(Succeed())
+	})
+
+	It("returns the forced capability verbatim without map fallback", func() {
+		Expect(os.Setenv("LOCALAI_FORCE_META_BACKEND_CAPABILITY", "nvidia-cuda-12")).To(Succeed())
+		state := &SystemState{}
+		Expect(state.DetectedCapability()).To(Equal("nvidia-cuda-12"))
+	})
+
+	It("does not fall back to default when the capability is unusual", func() {
+		Expect(os.Setenv("LOCALAI_FORCE_META_BACKEND_CAPABILITY", "metal")).To(Succeed())
+		state := &SystemState{}
+		Expect(state.DetectedCapability()).To(Equal("metal"))
+		Expect(state.DetectedCapability()).NotTo(Equal("default"))
+	})
+})
+
+var _ = Describe("ServingFeaturePreferenceTokens", func() {
+	It("puts dflash ahead of mtp", func() {
+		// The rule the serving feature table exists to express. Both are ways
+		// of serving the same weights faster; only this order separates them.
+		Expect(ServingFeaturePreferenceTokens()).To(Equal([]string{"dflash", "mtp"}))
+	})
+
+	It("carries neither a build tag nor an engine name", func() {
+		// The third vocabulary, and the same merge hazard as the other two. A
+		// build tag or an engine name here would be matched against a gallery
+		// entry's declared tags, where it means nothing.
+		Expect(ServingFeaturePreferenceTokens()).ToNot(ContainElements(
+			"cuda", "rocm", "hip", "sycl", "metal", "vulkan", "cpu", "darwin-x86",
+			"vllm", "sglang", "llama-cpp", "mlx",
+		))
+	})
+
+	It("does not depend on the host capability", func() {
+		// Deliberately not a SystemState method: no hardware prefers the plain
+		// build over an equivalent faster one, so there is no host-shaped
+		// ordering to express here.
+		previous := os.Getenv(capabilityEnv)
+		defer func() {
+			if previous != "" {
+				Expect(os.Setenv(capabilityEnv, previous)).To(Succeed())
+			} else {
+				Expect(os.Unsetenv(capabilityEnv)).To(Succeed())
+			}
+		}()
+
+		Expect(os.Setenv(capabilityEnv, "nvidia-cuda-12")).To(Succeed())
+		onNvidia := ServingFeaturePreferenceTokens()
+		Expect(os.Setenv(capabilityEnv, metal)).To(Succeed())
+		Expect(ServingFeaturePreferenceTokens()).To(Equal(onNvidia))
+	})
+
+	It("hands out a copy so one caller cannot corrupt another's lookup", func() {
+		first := ServingFeaturePreferenceTokens()
+		first[0] = "clobbered"
+		Expect(ServingFeaturePreferenceTokens()[0]).To(Equal("dflash"))
+	})
+})

pkg/vram/estimate.go

@@ -169,8 +169,9 @@ func EstimateMultiContext(ctx context.Context, files []FileInput, contextSizes [
 }
 
 // ParseSizeString parses a human-readable size string (e.g. "500MB", "14.5 GB", "2tb")
-// into bytes. Supports B, KB, MB, GB, TB, PB (case-insensitive, space optional).
-// Uses SI units (1 KB = 1000 B).
+// into bytes. Supports B, KB, MB, GB, TB, PB (case-insensitive, space optional)
+// as SI units (1 KB = 1000 B), and the IEC suffixes KiB, MiB, GiB, TiB, PiB as
+// binary units (1 KiB = 1024 B).
 func ParseSizeString(s string) (uint64, error) {
 	s = strings.TrimSpace(s)
 	if s == "" {
@@ -212,6 +213,18 @@ func ParseSizeString(s string) (uint64, error) {
 		multiplier = 1000 * 1000 * 1000 * 1000
 	case "P", "PB":
 		multiplier = 1000 * 1000 * 1000 * 1000 * 1000
+	// IEC binary suffixes. Model and VRAM sizes are conventionally quoted in
+	// these, so treating "GiB" as "GB" would understate a floor by 7%.
+	case "KIB":
+		multiplier = 1024
+	case "MIB":
+		multiplier = 1024 * 1024
+	case "GIB":
+		multiplier = 1024 * 1024 * 1024
+	case "TIB":
+		multiplier = 1024 * 1024 * 1024 * 1024
+	case "PIB":
+		multiplier = 1024 * 1024 * 1024 * 1024 * 1024
 	default:
 		return 0, fmt.Errorf("unknown size suffix: %q", suffix)
 	}

pkg/vram/hf_estimate_test.go

@@ -25,6 +25,11 @@ var _ = Describe("ParseSizeString", func() {
 		Entry("short suffix 100M", "100M", uint64(100_000_000)),
 		Entry("short suffix 2G", "2G", uint64(2_000_000_000)),
 		Entry("short suffix 1K", "1K", uint64(1_000)),
+		Entry("binary 1KiB", "1KiB", uint64(1024)),
+		Entry("binary 512MiB", "512MiB", uint64(512*1024*1024)),
+		Entry("binary 6GiB", "6GiB", uint64(6*1024*1024*1024)),
+		Entry("binary 2TiB", "2TiB", uint64(2*1024*1024*1024*1024)),
+		Entry("binary 1.5 gib lowercase with space", "1.5 gib", uint64(1.5*1024*1024*1024)),
 	)
 
 	DescribeTable("invalid sizes",