mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-02 20:37:03 -04:00
21e793ad2a8e378b529d80fcc0de67ad8564be35
14 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
LocalAI [bot]
|
d892e4af80 |
feat: add ds4 backend (DeepSeek V4 Flash) with tool calls, thinking, KV cache (#9758)
* test(e2e-backends): allow BACKEND_BINARY for native-built backends
Adds an escape hatch for hardware-gated backends (e.g. ds4) where the
model is too large for Docker build context. When BACKEND_BINARY points
at a run.sh produced by 'make -C backend/cpp/<name> package', the suite
skips docker image extraction and drives the binary directly.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* test(e2e-backends): validate BACKEND_BINARY basename + log actual source
Two follow-ups from the
|
||
|
Richard Palethorpe
|
bb033b16a9 |
feat: add LocalVQE backend and audio transformations UI (#9640)
feat(audio-transform): add LocalVQE backend, bidi gRPC RPC, Studio UI
Introduce a generic "audio transform" capability for any audio-in / audio-out
operation (echo cancellation, noise suppression, dereverberation, voice
conversion, etc.) and ship LocalVQE as the first backend implementation.
Backend protocol:
- Two new gRPC RPCs in backend.proto: unary AudioTransform for batch and
bidirectional AudioTransformStream for low-latency frame-by-frame use.
This is the first bidi stream in the proto; per-frame unary at LocalVQE's
16 ms hop would be RTT-bound. Wire it through pkg/grpc/{client,server,
embed,interface,base} with paired-channel ergonomics.
LocalVQE backend (backend/go/localvqe/):
- Go-Purego wrapper around upstream liblocalvqe.so. CMake builds the upstream
shared lib + its libggml-cpu-*.so runtime variants directly — no MODULE
wrapper needed because LocalVQE handles CPU feature selection internally
via GGML_BACKEND_DL.
- Sets GGML_NTHREADS from opts.Threads (or runtime.NumCPU()-1) — without it
LocalVQE runs single-threaded at ~1× realtime instead of the documented
~9.6×.
- Reference-length policy: zero-pad short refs, truncate long ones (the
trailing portion can't have leaked into a mic that wasn't recording).
- Ginkgo test suite (9 always-on specs + 2 model-gated).
HTTP layer:
- POST /audio/transformations (alias /audio/transform): multipart batch
endpoint, accepts audio + optional reference + params[*]=v form fields.
Persists inputs alongside the output in GeneratedContentDir/audio so the
React UI history can replay past (audio, reference, output) triples.
- GET /audio/transformations/stream: WebSocket bidi, 16 ms PCM frames
(interleaved stereo mic+ref in, mono out). JSON session.update envelope
for config; constants hoisted in core/schema/audio_transform.go.
- ffmpeg-based input normalisation to 16 kHz mono s16 WAV via the existing
utils.AudioToWav (with passthrough fast-path), so the user can upload any
format / rate without seeing the model's strict 16 kHz constraint.
- BackendTraceAudioTransform integration so /api/backend-traces and the
Traces UI light up with audio_snippet base64 and timing.
- Routes registered under routes/localai.go (LocalAI extension; OpenAI has
no /audio/transformations endpoint), traced via TraceMiddleware.
Auth + capability + importer:
- FLAG_AUDIO_TRANSFORM (model_config.go), FeatureAudioTransform (default-on,
in APIFeatures), three RouteFeatureRegistry rows.
- localvqe added to knownPrefOnlyBackends with modality "audio-transform".
- Gallery entry localvqe-v1-1.3m (sha256-pinned, hosted on
huggingface.co/LocalAI-io/LocalVQE).
React UI:
- New /app/transform page surfaced via a dedicated "Enhance" sidebar
section (sibling of Tools / Biometrics) — the page is enhancement, not
generation, so it lives outside Studio. Two AudioInput components
(Upload + Record tabs, drag-drop, mic capture).
- Echo-test button: records mic while playing the loaded reference through
the speakers — the mic naturally picks up speaker bleed, giving a real
(mic, ref) pair for AEC testing without leaving the UI.
- Reusable WaveformPlayer (canvas peaks + click-to-seek + audio controls)
and useAudioPeaks hook (shared module-scoped AudioContext to avoid
hitting browser context limits with three players on one page); migrated
TTS, Sound, Traces audio blocks to use it.
- Past runs saved in localStorage via useMediaHistory('audio-transform') —
the history entry stores all three URLs so clicking re-renders the full
triple, not just the output.
Build + e2e:
- 11 matrix entries removed from .github/workflows/backend.yml (CUDA, ROCm,
SYCL, Metal, L4T): upstream supports only CPU + Vulkan, so we ship those
two and let GPU-class hardware route through Vulkan in the gallery
capabilities map.
- tests-localvqe-grpc-transform job in test-extra.yml (gated on
detect-changes.outputs.localvqe).
- New audio_transform capability + 4 specs in tests/e2e-backends.
- Playwright spec suite in core/http/react-ui/e2e/audio-transform.spec.js
(8 specs covering tabs, file upload, multipart shape, history, errors).
Docs:
- New docs/content/features/audio-transform.md covering the (audio,
reference) mental model, batch + WebSocket wire formats, LocalVQE param
keys, and a YAML config example. Cross-links from text-to-audio and
audio-to-text feature pages.
Assisted-by: Claude:claude-opus-4-7 [Bash Read Edit Write Agent TaskCreate]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
|
||
|
Ettore Di Giacinto
|
2da1a4d230 |
feat(distributed): per-node backend installation from the gallery
In distributed mode the Backends gallery used to fan every install out to every worker — fine for auto-resolving (meta) backends like llama-cpp where each node picks its own variant, but wrong for hardware-specific builds like cpu-llama-cpp that would silently land on every GPU node. Adds a node-targeted install path through the existing POST /api/nodes/:id/backends/install plumbing, with two entry points: - Backends gallery row gets a split-button in distributed mode. Auto- resolving keeps "Install on all nodes" as the primary; chevron menu opens the picker. Hardware-specific routes the primary directly to the picker — no fan-out path on the row. - Nodes-page drawer gets a "+ Add backend" button that navigates to /app/backends?target=<node-id>; the gallery scopes itself to that node (banner, single per-row install button, Reinstall/Remove for already- installed). One gallery, two scopes — no second UI to maintain. The picker (new NodeInstallPicker) shows a 3-state suitability column (Compatible / Override / Installed), an auto-expanding variant override disclosure that fires when selected nodes have no working GPU, parallel per-node installs with inline status and Retry-failed-nodes, and a mismatch confirm that names the consequence on the button itself. A 409 fan-out guard on /api/backends/apply protects CLI/Terraform/script users from the same footgun: hardware-specific installs in distributed mode now return code "concrete_backend_requires_target" with a human- readable error and a meta_alternative pointer. The gallery list payload now surfaces capabilities, metaBackendFor and per-row nodes (NodeBackendRef) so the picker and the new Nodes column have everything they need without re-walking the gallery client-side. GODEBUG=netdns=go is set on the compose services because the cgo DNS resolver follows the container's nsswitch.conf to host systemd-resolved (127.0.0.53), unreachable from inside the container; the pure-Go resolver reads /etc/resolv.conf directly and uses Docker's embedded DNS. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-4-7[1m] [Edit] [Bash] [Read] [Write] |
||
|
Ettore Di Giacinto
|
f0c92610a1 |
feat(importer): expand importer flow to almost all backends (#9466)
* docs(agents): require importer integration when adding backends
Document the importer registry workflow so contributors know that adding
a new backend also requires updating the /import-model dropdown source:
either a new importer in core/gallery/importers/, extending an existing
one for drop-in replacements, or the pref-only slice for backends with
no reliable auto-detect signal. Always covered by a table-driven test.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for Batch 0 primitives
Introduce failing tests that drive Batch 0 of the importer expansion:
- pkg/huggingface-api: assert GetModelDetails populates PipelineTag and
LibraryName from /api/models/{repo}, and that a failing metadata
endpoint still returns file details (best-effort fetch).
- core/gallery/importers/helpers_test.go: new table-driven coverage for
HasFile, HasExtension, HasONNX, HasONNXConfigPair, HasGGMLFile.
- core/gallery/importers/importers_test.go: assert ErrAmbiguousImport
sentinel exists and round-trips through errors.Is.
- core/gallery/importers/local_test.go: extend with detection cases for
ggml-*.bin (whisper), silero_vad.onnx (silero-vad), and the piper
.onnx + .onnx.json pair.
- core/http/endpoints/localai/import_model_test.go: assert
ImportModelURIEndpoint returns HTTP 400 with a structured
{error, detail, hint} body when ErrAmbiguousImport surfaces.
All tests fail in the expected places (missing fields, missing
helpers, missing sentinel, endpoint still wraps as 500).
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): Batch 0 foundation — helpers, sentinel, local detection
Implements the Batch 0 primitives that subsequent importer batches build on:
- pkg/huggingface-api: ModelDetails gains PipelineTag and LibraryName.
GetModelDetails now layers a best-effort GET /api/models/{repo} fetch
on top of ListFiles — a metadata outage leaves the fields empty but
still returns full file details. Uses a dedicated response struct
because the single-model endpoint uses snake_case keys while the list
endpoint historically returned camelCase.
- core/gallery/importers/helpers.go: generic HasFile, HasExtension,
HasONNX, HasONNXConfigPair, HasGGMLFile helpers working on
[]hfapi.ModelFile so per-backend importers can detect artefact
patterns without duplicating string wrangling.
- core/gallery/importers/importers.go: adds the ErrAmbiguousImport
sentinel. DiscoverModelConfig now returns it (wrapped with
fmt.Errorf("%w: ...")) when no importer matched AND the HF
pipeline_tag falls in a whitelist of narrow modalities (ASR, TTS,
sentence-similarity, text-classification, object-detection). The
whitelist is intentionally narrow — unknown tags keep the previous
"no importer matched" behaviour to avoid blocking rare repos.
- core/gallery/importers/local.go: three new local-path detections,
inserted before the existing merged-transformers branch:
* ggml-*.bin → whisper
* silero*.onnx → silero-vad
* *.onnx + *.onnx.json pair → piper
- core/http/endpoints/localai/import_model.go: ImportModelURIEndpoint
surfaces ErrAmbiguousImport as HTTP 400 with
{error, detail, hint} JSON, preserving existing behaviour for
unrelated errors.
Green tests:
go test ./core/gallery/importers/... ./pkg/huggingface-api/... \
./core/http/endpoints/localai/...
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(importers): red tests for KnownBackend endpoint and importer metadata
Add failing tests that drive Batch UI-Dropdown:
- importers_test.go: assert importers expose Name/Modality/AutoDetects
and that LlamaCPPImporter advertises drop-in replacements via a new
AdditionalBackendsProvider interface. A Registry() accessor is also
expected.
- backend_test.go (new): assert GET /backends/known returns
[]schema.KnownBackend, covers every importer, exposes drop-in
llama-cpp replacements, includes curated pref-only backends, has no
duplicates, and is sorted by Modality+Name.
These tests fail at compile time against master; they are intentionally
red so the follow-up green commit is reviewable.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery): add /backends/known endpoint for importer-aware backend list
Extend the Importer interface with Name/Modality/AutoDetects so the
import system can self-describe its registry, and introduce the
AdditionalBackendsProvider interface so importers can advertise drop-in
replacements (llama-cpp advertises ik-llama-cpp and turboquant).
Expose the new GET /backends/known endpoint that merges:
- the importer registry (auto-detect supported),
- drop-in replacements hosted by importers (preference-only),
- a curated knownPrefOnlyBackends slice for backends with no dedicated
importer (sglang, tinygrad, trl, mlx-vlm, whisperx, kokoros, Qwen TTS
variants, sam3-cpp) — kept at the top of backend.go so contributors
adding a new pref-only backend have one obvious place to edit,
- backends installed on disk but unknown to the importer (marked
AutoDetect=false, empty Modality).
The endpoint deliberately does NOT filter by gallery membership or host
capability (unlike /backends/available): LocalAI may auto-install a
backend that is not yet present, so the import form dropdown must show
everything the importer knows about.
Response is deduplicated (importer wins over pref-only) and sorted by
Modality+Name for deterministic output.
Registered in core/http/routes/localai.go next to /backends/available
under the same admin middleware.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(ui): source import form backend dropdown from /backends/known
Replace the hard-coded BACKENDS constant in ImportModel.jsx with a
live fetch of /backends/known on mount. Users now see every backend
the importer layer knows about (including preference-only entries)
grouped by modality, not a stale subset.
Changes:
- config.js: add backendsKnown endpoint constant next to
backendsAvailable.
- api.js: add backendsApi.listKnown() wrapper.
- ImportModel.jsx: remove BACKENDS constant, fetch the list via
useEffect, and derive grouped options via buildBackendOptions.
Preference-only entries render with a " (preference-only)" suffix.
Loading state disables the dropdown with a "Loading backends…"
placeholder; on fetch failure the form falls back to auto-detect
only and surfaces a non-blocking toast.
- SearchableSelect.jsx: accept items flagged isHeader=true and render
them as non-selectable section dividers. Keyboard navigation skips
headers and search queries hide them so filtered output stays
relevant.
Vitest is not set up in this project (devDependencies ship Playwright
only). Per the brief's guard-rail, no frontend test framework is
introduced; coverage is provided by the Go handler tests that assert
the /backends/known contract consumed by the React form.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for whisper importer
Asserts detection on ggerganov/whisper.cpp (via ggml-*.bin filename),
the preferences.backend=whisper override path for arbitrary URIs,
and the Importer interface metadata (name/modality/autodetect).
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add whisper importer
Recognises whisper.cpp GGML models by the "ggml-*.bin" filename
convention (direct URL or HF repo member) and by the explicit
preferences.backend="whisper" override. Emits backend: whisper with
the transcript use-case. Registered before llama-cpp so the narrow
filename signal wins before any generic GGUF match is attempted.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for moonshine importer
Asserts detection on UsefulSensors/moonshine-tiny via owner + ONNX
files, the preferences.backend=moonshine override for arbitrary URIs,
and the Importer interface metadata (name/modality/autodetect).
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add moonshine importer
Matches UsefulSensors-owned HF repos whose artefacts or metadata
identify them as ASR: on-disk .onnx files (the canonical Moonshine
packaging) OR pipeline_tag=automatic-speech-recognition (covers
transformers/safetensors-only sibling repos). preferences.backend=
moonshine overrides detection. Test uses the live moonshine-tiny
repo because the canonical UsefulSensors/moonshine repo currently
hits a recursive-subfolder bug in pkg/huggingface-api ListFiles.
Registered after WhisperImporter but before LlamaCPPImporter and
TransformersImporter so the narrower owner+ASR signal wins before
the generic tokenizer.json check routes the repo to transformers.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for nemo importer
Asserts detection on nvidia/parakeet-tdt-0.6b-v3 via owner + .nemo
file, the preferences.backend=nemo override for arbitrary URIs, and
the Importer interface metadata (name/modality/autodetect).
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add nemo importer
Matches nvidia-owned HF repos that ship a .nemo checkpoint archive,
the canonical NeMo ASR packaging. preferences.backend=nemo forces
detection. Registered between moonshine and llama-cpp so the narrow
owner + extension signal wins before any downstream generic matcher.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for faster-whisper importer
Asserts detection on Systran/faster-whisper-large-v3 (owner +
model.bin + config.json + ASR pipeline), the preferences.backend=
faster-whisper override for arbitrary URIs, and the Importer
interface metadata.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add faster-whisper importer
Recognises CTranslate2-packaged whisper checkpoints distributed for
the faster-whisper runtime: model.bin + config.json + ASR
pipeline_tag, narrowed to Systran-owned repos or repo names
containing "faster-whisper" to avoid falsely claiming vanilla
OpenAI whisper HF repos. preferences.backend=faster-whisper
overrides detection. Registered before llama-cpp and transformers
so the narrow signal wins before tokenizer.json routes the repo to
the generic transformers importer.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for qwen-asr importer
Asserts detection on Qwen/Qwen3-ASR-1.7B via owner + ASR substring
in the repo name, the preferences.backend=qwen-asr override for
arbitrary URIs, and the Importer interface metadata.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add qwen-asr importer
Matches Qwen-owned HF repos whose name contains "ASR"
(case-insensitive), routing them to the qwen-asr backend rather
than the generic transformers/vllm path. The substring check scans
the repo portion only so the owner field cannot leak a false match.
preferences.backend=qwen-asr forces detection. Registered before
llama-cpp and transformers so the narrow owner+name signal wins.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): ASR ambiguity surfaces ErrAmbiguousImport
Locks in the behaviour added in Batch 0: an HF repo whose pipeline_tag
marks it as automatic-speech-recognition but whose artefacts match no
ASR importer (and no generic importer) must fail with
ErrAmbiguousImport so callers know to pass preferences.backend rather
than silently guess. pyannote/voice-activity-detection is the fixture
— its file list is only config.yaml + README, leaving every importer's
artefact check negative.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for piper importer
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add piper importer
Detects piper TTS voices by the canonical <voice>.onnx + <voice>.onnx.json
pair packaging (via HasONNXConfigPair). Narrow enough to skip generic
ONNX repos used by other backends (Moonshine ASR, sentence-transformers).
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for bark importer
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add bark importer
Detects Suno's Bark TTS checkpoints by HF owner "suno" + repo name
prefix "bark". Adds HFOwnerRepoFromURI() helper so importers can fall
back to URI parsing when pkg/huggingface-api's recursive tree listing
errors on repos with nested subdirectories (suno/bark ships a
speaker_embeddings/v2 subtree that trips a pre-existing path-doubling
bug in the listFilesInPath recursion).
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for fish-speech importer
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add fish-speech importer
Detects Fish Audio TTS releases by HF owner "fishaudio" with a URI-based
fallback for repos whose tree recursion trips the pre-existing hfapi
path-doubling bug.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for outetts importer
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add outetts importer
Detects OuteAI's OuteTTS releases by HF owner "OuteAI" or a case-
insensitive "OuteTTS" substring in the repo name, with a URI-based
fallback for recursion-bugged repos.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for voxcpm importer
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add voxcpm importer
Detects OpenBMB's VoxCPM TTS family by repo-name substring (community
mirrors re-host the weights under many owners — mlx-community,
bluryar, callgg, etc).
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for kokoro importer
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add kokoro importer
Detects hexgrad's Kokoro TTS by the "Kokoro" repo-name substring paired
with a PyTorch .pth/.pt checkpoint — the pairing excludes ONNX-only
mirrors (handled by the pref-only `kokoros` Rust runtime) and GGUF
mirrors (handled by llama-cpp).
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for kitten-tts importer
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add kitten-tts importer
Detects KittenML's kitten-tts releases by owner or "kitten-tts" repo-name
substring, with URI-parsing fallback.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for neutts importer
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add neutts importer
Detects Neuphonic's NeuTTS releases by owner "neuphonic" or "neutts"
repo-name substring, with URI-parsing fallback.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for chatterbox importer
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add chatterbox importer
Detects Resemble AI's Chatterbox TTS by owner "ResembleAI" or
"chatterbox" repo-name substring, with URI-parsing fallback.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for vibevoice importer
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add vibevoice importer
Detects Microsoft's VibeVoice TTS by "vibevoice" repo-name substring
(case-insensitive) so community mirrors still route here.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for coqui importer
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add coqui importer
Detects Coqui AI's TTS releases (XTTS-v2, YourTTS, …) by the
authoritative `coqui` HF owner, with URI-parsing fallback.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): TTS ambiguity surfaces ErrAmbiguousImport
Adds a Ginkgo spec that imports nari-labs/Dia-1.6B — a real HF repo
carrying pipeline_tag="text-to-speech" whose artefacts (*.pth, one
safetensors shard, preprocessor_config.json, config.json) match none of
the Batch-2 TTS importers nor the generic text/image importers — and
asserts DiscoverModelConfig wraps ErrAmbiguousImport via errors.Is.
Also pivots the endpoint-level ambiguity fixture from hexgrad/Kokoro-82M
to nari-labs/Dia-1.6B. Batch 2 added a dedicated kokoro importer that
now claims the original fixture; Dia remains genuinely unclaimed and
so exercises the same ambiguity code path at the HTTP layer.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for stablediffusion-ggml importer
Covers HF repo detection (city96/FLUX.1-dev-gguf), raw .gguf URL matching on
filename arch tokens, preference override, and Importer interface metadata.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add stablediffusion-ggml importer
Detects GGUF-packed Stable Diffusion and FLUX checkpoints (leejet owner,
city96 FLUX mirrors, second-state SD dumps, raw .gguf URLs with arch
tokens) and routes them to the stablediffusion-ggml backend. Registered
BEFORE LlamaCPPImporter so .gguf image checkpoints are not stolen by
llama-cpp's generic .gguf match. Reuses HFOwnerRepoFromURI for the
hfapi-recursion-bug fallback. preferences.backend overrides detection.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for ace-step importer
Covers HF repo-name detection (ACE-Step/ACE-Step-v1-3.5B), preference
override, and Importer interface metadata.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add ace-step importer
Routes ACE-Step music generation checkpoints (ACE-Step/ACE-Step-v1-3.5B,
ACE-Step/Ace-Step1.5, community mirrors) to the ace-step backend.
Matching is case-insensitive on the "ace-step" repo-name substring and
owner, with an HFOwnerRepoFromURI fallback for the hfapi recursion bug.
KnownUsecaseStrings mirrors the gallery's ace-step-turbo entry
(sound_generation, tts). preferences.backend overrides.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): surface ErrAmbiguousImport on text-to-image misses
Adds text-to-image to ambiguousModalities whitelist and covers the
h94/IP-Adapter-FaceID case — pipeline_tag=text-to-image but ships only
.bin/.safetensors so diffusers, stablediffusion-ggml, llama-cpp,
transformers, vllm, mlx, and ace-step all miss. DiscoverModelConfig now
surfaces ErrAmbiguousImport for that shape instead of the opaque
"no importer matched" error.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for vllm-omni importer
Introduces the test surface for the forthcoming VLLMOmniImporter:
detection via preferences.backend, Qwen owner + Omni repo token,
URI-only fallback, negative cases (plain Qwen, random OmniX repo), and
Import() emitting backend: vllm-omni with chat + multimodal usecases.
Includes a registration-order assertion via DiscoverModelConfig to pin
the requirement that vllm-omni wins over vllm for Qwen Omni repos
(tokenizer files are usually present too).
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add vllm-omni importer
Adds VLLMOmniImporter for Qwen Omni-style multimodal checkpoints
(Qwen3-Omni, Qwen2.5-Omni, …). Detection is narrow: HF owner "Qwen"
combined with "omni" in the repo name, or a repo name matching the
-Omni-/Omni- naming pattern. preferences.backend="vllm-omni" always
wins; HFOwnerRepoFromURI provides a URI-only fallback for the hfapi
recursion-bug edge case.
Emitted YAML sets backend: vllm-omni and known_usecases: [chat,
multimodal], matching the gallery/index.yaml vllm-omni entries. The
importer is registered ahead of VLLMImporter so Qwen Omni repos —
which also carry tokenizer files — route to vllm-omni rather than the
plain vllm backend.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for llama-cpp drop-in preferences
Pins the expected drop-in replacement behaviour: preferences.backend
of ik-llama-cpp or turboquant must swap the emitted YAML backend
field while keeping the llama-cpp file layout identical. Also covers
the unknown-backend case (must stay llama-cpp) and re-asserts
AdditionalBackends() returns the two curated entries with non-empty
descriptions.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): llama-cpp honours ik-llama-cpp and turboquant drop-in preferences
preferences.backend set to ik-llama-cpp or turboquant now swaps the
emitted YAML backend field while leaving the file layout, model path,
mmproj handling and everything else in the llama-cpp Import pipeline
untouched. Unknown values are ignored and fall back to backend:
llama-cpp so arbitrary input can't leak into the config.
Aligns the AdditionalBackends() descriptions with the user-facing
naming conventions surfaced via /backends/known. No changes to the
pref-only curated list in endpoints/localai/backend.go: the two
drop-in names have always lived on the importer side via
AdditionalBackends.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for silero-vad importer
Add the SileroVADImporter test fixtures covering metadata, preference
overrides, snakers4 + onnx detection, silero_vad.onnx canonical filename,
URI fallback, and live HF discovery. Implementation follows in the next
commit.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add silero-vad importer
Recognise the Silero VAD ONNX packaging: the canonical silero_vad.onnx
filename or any ONNX file under the snakers4 owner. Emits a
backend: silero-vad config with the vad known_usecase, and attaches the
canonical file entry when present so the weights download on import.
Registered before the generic importers so the unique-filename signal
takes precedence over any downstream tokenizer-based matcher.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for rerankers importer
Cover the RerankersImporter contract: interface metadata, preference
override, cross-encoder owner detection, case-insensitive 'reranker'
substring match (BAAI/bge-reranker, Alibaba-NLP/gte-reranker), URI
fallback, and the full-discovery ordering check that a BAAI reranker
repo must route to the rerankers importer rather than transformers.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add rerankers importer
Recognise reranker repositories — cross-encoder owner or any repo whose
name contains 'reranker' (case-insensitive). Emits backend: rerankers
with reranking: true and the rerank known_usecase.
Registered ahead of sentencetransformers and transformers so reranker
repos that happen to ship tokenizer.json or modules.json still route
here.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for sentencetransformers importer
Cover the SentenceTransformersImporter contract: interface metadata,
preference override, modules.json marker file, sentence_bert_config.json
marker file, sentence-transformers owner, URI fallback, and the
full-discovery ordering check that ensures a sentence-transformers HF
URI routes here rather than transformers.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add sentencetransformers importer
Recognise sentence-transformers embedding repos by modules.json,
sentence_bert_config.json, or the sentence-transformers owner. Emits
backend: sentencetransformers with embeddings: true and the embeddings
known_usecase.
Registered ahead of transformers so ST repos that carry tokenizer.json
still route here.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): add failing tests for rfdetr importer
Cover the RFDetrImporter contract: interface metadata, preference
override, case-insensitive rf-detr and rfdetr substring matches, URI
fallback, and negative cases. Implementation follows in the next
commit.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(gallery/importers): add rfdetr importer
Recognise RF-DETR object-detection repositories by a case-insensitive
'rf-detr' / 'rfdetr' substring in the repo name. Emits backend: rfdetr
with the detection known_usecase.
Registered ahead of transformers so RF-DETR repos with tokenizer
artefacts still route here.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(gallery/importers): surface ErrAmbiguousImport on sentence-similarity misses
Add an ambiguity fixture covering the embeddings/rerankers modality.
Qdrant/bm25 carries pipeline_tag=sentence-similarity but ships only
config.json + stopword .txt files — none of the Batch 5 importers
(silero-vad, rerankers, sentencetransformers, rfdetr) or the generic
vllm/transformers/llama-cpp/mlx/diffusers importers match. Because the
modality is in the ambiguous whitelist, DiscoverModelConfig must
surface ErrAmbiguousImport.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(localai/backend): red tests for KnownBackend.Installed flag
Extend the /backends/known suite with three failing cases that pin down
the forthcoming Installed field: JSON field presence on every entry,
flipping to true when an importer-registered backend is also present on
disk (and staying false for non-installed pref-only entries), and
surfacing system-only backends with empty modality and AutoDetect=false.
A small writeFakeSystemBackend helper plants a run.sh under the backends
dir so gallery.ListSystemBackends recognises the fixture.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(schema,localai/backend): add Installed flag to KnownBackend
Add an Installed bool to schema.KnownBackend and populate it from the
/backends/known handler so the React import form can warn users that
picking a not-yet-installed backend will trigger an automatic download
on submit.
Computation: after merging the importer registry, additional backends
provider entries and the curated pref-only slice, the handler walks
gallery.ListSystemBackends(systemState) and either flips the existing
map entry's Installed flag to true (preserving modality / autodetect /
description metadata) or inserts a bare {Installed:true} entry for
system-only backends the importer layer doesn't know about.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(localai/import_model): structured ambiguous-import response
Add red tests covering the extended ambiguity shape the React import
form needs:
- ImportModelURIEndpoint must return an HTTP 400 body that exposes the
detected `modality` (normalised to the importer modality key, e.g.
"tts" for pipeline_tag=text-to-speech) and a list of `candidates`
(backend names filtered by modality, excluding text-LLM backends).
- The importers package must surface a typed AmbiguousImportError so
HTTP consumers can read Modality + Candidates without parsing the
error string. errors.Is against the existing sentinel keeps working.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(localai/import_model): structured ambiguity response with modality + candidates
DiscoverModelConfig now returns a typed AmbiguousImportError that
carries the importer modality key, candidate backend names, the
original URI, and the raw HF pipeline_tag. Its Is() preserves
errors.Is(err, ErrAmbiguousImport) for legacy callers.
The importer modality is pre-mapped from the HF pipeline_tag
(automatic-speech-recognition → asr, text-to-speech → tts, etc) via
PipelineTagToModality — surfaced as an exported helper so downstream
consumers can avoid duplicating the table. CandidatesForModality
filters the default importer registry plus AdditionalBackendsProvider
drop-ins by modality, sorts deterministically, and is the single
source of truth used by ImportModelURIEndpoint.
ImportModelURIEndpoint now returns HTTP 400 with
{ error, detail, modality, candidates, hint }
when ambiguity fires, letting the React form render a modality-scoped
picker inline instead of a generic toast.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(ui/import): manual pick badge + tooltip
Red Playwright coverage for the preference-only → manual pick rename:
- The Backend dropdown renders a "manual pick" badge on every option
whose KnownBackend.auto_detect is false.
- The badge carries a title attribute with hover-tooltip copy that
explains auto-detect won't route to this backend.
- Auto-detectable backends must NOT carry the badge.
- The legacy " (preference-only)" suffix is gone from every label.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* ui(import): replace preference-only suffix with manual pick badge
SearchableSelect option rows now support an optional badge field — a
muted pill rendered to the right of the label with an optional title
attribute for native hover tooltips. Plain text so screen readers read
it alongside the option name.
buildBackendOptions in ImportModel stops appending " (preference-only)"
to the label and instead sets badge="manual pick" plus a descriptive
tooltip on every option whose auto_detect is false. The Backend help
text explains what "manual pick" means so users aren't left wondering
about the badge.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(ui/import): inline ambiguity picker
Red Playwright coverage for Batch A2 — when the server returns a 400
ambiguity body, the form must render an inline alert instead of a
toast, expose one clickable chip per candidate backend, and support
both auto-resubmit on pick and silent dismiss.
- Mocks /api/models/import-uri with the structured ambiguity body
(error, detail, modality, candidates, hint).
- On first click of Import, the alert is visible, carries
modality-specific copy, and shows a chip per candidate.
- Clicking a chip clears the alert, sets the Backend dropdown, and
triggers a second POST to /api/models/import-uri.
- Dismissing the alert leaves the Backend dropdown on Auto-detect —
no implicit backend assignment.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(ui/import): inline ambiguity alert with candidate chips
Adds AmbiguityAlert — a soft, info-coloured card rendered above the URI
input when the server returns a structured 400 with { modality,
candidates }. Message is modality-aware (tts/asr/embeddings/image/
reranker/detection get purpose-written copy, everything else falls back
to a generic template). Each candidate is a clickable chip that shows a
download icon when /backends/known marks the backend as not yet
installed, so users aren't surprised by an implicit install.
ImportModel wires the alert to handleSimpleImport's error path:
- api.handleResponse now attaches { status, body } to the thrown Error
so pages can pattern-match on structured responses instead of string
error messages.
- handleSimpleImport detects `status === 400 && body.error === 'ambiguous
import'` and flips into the inline-picker mode instead of toasting.
- Clicking a chip sets prefs.backend and auto-resubmits (passing the
picked backend as an override so setPrefs's asynchrony doesn't leak
a stale value).
- Dismissing clears the alert; changing the URI or the backend also
clears it so a stale alert never sticks around.
Test fixtures mock GET /backends/known + POST /models/import-uri so the
Playwright specs don't depend on real network reachability.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(ui/import): auto-install warning
Red Playwright coverage for Batch A3 — when the user picks a backend
whose KnownBackend.installed is false, the form must render a muted
inline note under the Backend dropdown warning that submitting will
download the backend first. Picking an installed backend or leaving
Auto-detect selected must keep the note hidden.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(ui/import): auto-install warning under backend dropdown
When the user picks a backend whose KnownBackend.installed is false,
render a muted inline note under the Backend dropdown's help text
warning that submitting will download the backend first. The note
lives inside the same form-group so it lines up with the existing
hint text; it's hidden when Auto-detect is selected (the selected
backend is unknowable at that point) or when the chosen backend is
already on disk.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* ui(import): drop redundant section header, adjust icons, rename HF shortcut
- Remove the "Import from URI" card-level <h2> — the page title already
says "Import New Model" one row up, so the secondary header was
duplicating information.
- Swap the fa-star on "Common Preferences" for fa-sliders (stars imply
favourites/ratings; this is just a preferences block) and move the
Custom Preferences fa-sliders-h to fa-plus-circle so the two blocks
read as distinct rather than as two sliders.
- Rename the HF shortcut from "Search GGUF on HF" → "Browse models on
HF" and drop the `search=gguf` filter on the linked URL. The import
form now supports ~40 backends; hard-coding GGUF in the copy no
longer matches the form's actual reach.
- Pure polish — no behaviour change, covered by the existing Batch A
Playwright suite.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(ui/import): batch B — simple/power switch, options, tabs, dialog
Adds a failing Playwright suite covering the full Batch B surface ahead
of implementation:
- B1: SimplePowerSwitch segmented control renders, toggles, persists to
localStorage across reloads.
- B2: Simple-mode Options disclosure is collapsed by default; expanding
exposes only Backend, Model Name, Description (no quantizations,
mmproj, model type, or custom prefs).
- B3: Power mode has Preferences and YAML tabs with a persistent
selection across reloads; URI/name/description typed in Simple carry
over to Power; YAML tab swaps the primary action to Create.
- B4: Switching Power -> Simple with a custom preference set triggers
the 3-button confirmation dialog (Keep / Discard / Cancel) with the
documented semantics.
Tests fail against master — implementation lands in the following
commits.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(ui/import): add SimplePowerSwitch segmented control
Replaces the previous "Advanced Mode / Simple Mode" toggle button in the
page header with a two-segment control that flips between Simple and
Power. The control reuses the existing .segmented CSS shared with the
Sound page for visual consistency.
Mode state is persisted to localStorage under `import-form-mode` so
reloads land on the same view (default: simple). The boolean alias
`isAdvancedMode` is retained internally to minimise diff — subsequent
commits reshape the Simple and Power surfaces independently.
Closes B1 from the Batch B Playwright suite.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(ui/import): simple mode collapsible options, power tabs, switch dialog
Completes the Batch B surface in a single structural pass so Simple and
Power mode can evolve independently:
Simple mode
- URI input + Ambiguity alert + Import button, plus a collapsible
"Options" disclosure that exposes ONLY Backend, Model Name,
Description. Quantizations / MMProj / Model Type / Diffusers fields
/ Custom Preferences are no longer rendered in Simple mode.
Power mode
- In-page segmented "Preferences · YAML" tab strip. Active tab
persists to localStorage under `import-form-power-tab`.
- Preferences tab = the full existing preferences + custom prefs
panel (no progressive disclosure yet — that's Batch D).
- YAML tab = the existing CodeEditor. Primary button reads "Create"
here, "Import Model" everywhere else.
Switch dialog
- Power -> Simple with non-default prefs (advanced pref keys set,
any custom-pref key non-empty, or YAML edited away from the
template) opens a 3-button dialog: Keep & switch / Discard &
switch / Cancel.
- Keep preserves all state. Discard resets prefs + customPrefs + YAML
to defaults. Cancel leaves the user in Power mode.
Page subtitle reflects the current surface (Simple, Power/Preferences,
Power/YAML). Estimate banner renders everywhere except Power/YAML.
Closes B2/B3/B4 from the Batch B Playwright suite.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(ui/import): expand Options disclosure in Batch A tests
Batch B hid the Backend dropdown behind a collapsible Options disclosure
in Simple mode. The Batch A tests that exercise the dropdown directly
(manual-pick badge, ambiguity chip sets the selected backend, auto-
install warning) now click the disclosure toggle before asserting on
dropdown contents. Test intent is unchanged.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* ui(import): strip decorative icons from field labels
The preference panel had 12 Font Awesome icons decorating field labels
(Backend, Model Name, Description, Quantizations, MMProj Quantizations,
Model Type, Pipeline Type, Scheduler Type, Enable Parameters, Embeddings,
CUDA, plus fa-link on Model URI). Every label screamed equally, flattening
the visual hierarchy.
Remove them. Keep icons where they carry meaning: page-level section
headers, URI format guide entries, primary buttons, the Simple-mode
Options disclosure, the ambiguity alert's fa-lightbulb, the auto-install
note's fa-download, and the Estimated-requirements banner's
fa-memory / fa-microchip / fa-download.
No new behaviour, no layout / spacing changes beyond removing the
orphaned icon margin. Playwright suite green.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(ui/import): progressive disclosure of preference fields
Cover the Batch D visibility matrix for Power > Preferences: Quantizations,
MMProj Quantizations, and Model Type each render only for the backends that
can consume them, stay visible when the backend is unset, and preserve any
value the user already typed when toggled off and back on. Also pin the
shrunk Description textarea at rows=2.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(ui/import): progressive disclosure + shorter description textarea
Gate Quantizations, MMProj Quantizations, and Model Type in the Power >
Preferences tab so each field only renders for the backends that can
actually consume it. Backend unset keeps everything visible. Hidden
fields' state is preserved (the JSX wrapper is guarded, not the
underlying prefs state) so users flipping backends back and forth don't
lose input.
Also shrink the Description textarea from rows=3 to rows=2 — it's
shared between Simple Options and Power Preferences so the change
applies to both.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(ui/import): enter-to-submit in Simple mode
Red test for Batch F3 — pressing Enter in the URI input must POST
/models/import-uri, and Enter in the Description textarea must insert
a newline without submitting the form.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(ui/import): enter-to-submit in Simple mode
Wrap the Simple-mode URI input + ambiguity alert + Options disclosure
in a <form> whose onSubmit calls handleSimpleImport. Pressing Enter in
the URI input (or any Simple-mode text input) now submits the import
without having to move the mouse to the header button. The Description
textarea keeps its native behaviour — Enter inserts a newline.
A hidden submit button is included because the visible Import button
lives outside the form in the page header; some browsers only fire
implicit Enter-submit when the form contains a submit-capable element.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* ui(import,SearchableSelect,components): aria-hidden on decorative icons
Every Font Awesome icon in the import form is decorative — its meaning
is already conveyed by adjacent visible text. Adding aria-hidden="true"
prevents screen readers from announcing the unicode glyph point as
content. Covers ImportModel.jsx (all remaining <i> glyphs) and
SearchableSelect.jsx (the trigger chevron).
AmbiguityAlert and SimplePowerSwitch already set aria-hidden on their
icons when the components landed in Batches A and B — no change needed
there.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* ui(SearchableSelect): responsive dropdown maxHeight + hover focus guard
F2 — replace fixed pixel heights with min(pixel, vh) so the dropdown
and its inner scroll region don't overflow short viewports. Outer
container: 260px -> min(260px, 60vh); inner listbox: 200px ->
min(200px, 50vh). Tall viewports still get the original pixel caps.
F5 — short-circuit onMouseEnter when the hovered row is already the
focused row. Avoids queueing a setFocusIndex call (and a render) for
every mousemove inside the same item — the state would be identical.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* ui(import): aria-label on custom preference rows
The Key / Value inputs and trash button in each Custom Preferences row
previously relied on placeholder text alone. Placeholders are not
accessible names — they vanish on input and screen readers do not
announce them consistently. Add row-indexed aria-labels so assistive
tech can distinguish "Preference key for row 1" from "row 2", and give
the trash button an explicit "Remove this preference" label.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* test(ui/import): modality chip row
Red tests for Batch E — a horizontal modality chip row that filters the
Backend dropdown by modality. Covers visibility in Simple-mode Options
and Power/Preferences (and absence in Power/YAML), filter behaviour,
mismatched-backend clearing with toast, ambiguity-alert auto-selection,
and radiogroup keyboard navigation.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* feat(ui/import): add ModalityChips component + filter integration
Horizontal chip row (Any, Text, Speech, TTS, Image, Embeddings,
Rerankers, Detection, VAD) filters the Backend dropdown options to the
selected modality. Default is Any — no filter, current behaviour.
- New ModalityChips component (radiogroup pattern, roving tabindex,
arrow-key navigation, Home/End).
- buildBackendOptions now accepts an optional modalityFilter so grouped
output is narrowed before rendering.
- Chips render inside Simple-mode Options disclosure and Power >
Preferences tab. Power > YAML stays unaffected.
- Switching the filter drops a mismatched backend selection and
surfaces a toast so the auto-clear is visible.
- Ambiguity alerts auto-activate the matching chip so users see only
relevant backends even if they dismiss the alert.
Tightens the Batch E tests' option-matching to the label <span> so the
"↵" keybind hint on the focused row doesn't break accessible-name
lookups.
Assisted-by: Claude:claude-opus-4-7[1m] [Agent]
* fix(ui/import): rename Power to Advanced + stop URI-formats toggle from submitting form
The "Supported URI Formats" disclosure button inside the Simple-mode form
lacked an explicit type attribute, so it defaulted to type="submit". Every
click triggered the form's onSubmit and surfaced the empty-URI validation
toast ("Please enter a model URI"). Marking it type="button" lets it
behave as a pure toggle.
While here, rename the user-visible "Power" label to "Advanced" in the
mode switch (button text + tooltip) and the Power-mode tab's aria-label,
matching the term users actually expect. The internal mode key stays
'power' so tests, localStorage, and data-testid selectors are untouched.
Assisted-by: Claude:claude-opus-4-7
* fix(system): fall back to cpu when meta backend lacks default capability
Meta backends like vllm and sglang enumerate concrete variants for
nvidia/amd/intel/cpu but omit a default: catch-all entry. On a no-GPU
host the reported capability is "default", so the previous Capability()
returned "default" unconditionally on a miss — IsCompatibleWith then saw
no "default" key and filtered the meta out of AvailableBackends. The
import flow's auto-install step then failed with "no backend found with
name <meta>", contradicting the UI's promise that the backend would be
downloaded on demand.
Try the explicit "default" key first, then fall back to "cpu" before
giving up. vllm now resolves to cpu-vllm on CPU-only Linux without
touching the gallery YAML.
Assisted-by: Claude:claude-opus-4-7
|
||
|
Ettore Di Giacinto
|
8ab0744458 |
feat: backend versioning, upgrade detection and auto-upgrade (#9315)
* feat: add backend versioning data model foundation Add Version, URI, and Digest fields to BackendMetadata for tracking installed backend versions and enabling upgrade detection. Add Version field to GalleryBackend. Add UpgradeAvailable/AvailableVersion fields to SystemBackend. Implement GetImageDigest() for lightweight OCI digest lookups via remote.Head. Record version, URI, and digest at install time in InstallBackend() and propagate version through meta backends. * feat: add backend upgrade detection and execution logic Add CheckBackendUpgrades() to compare installed backend versions/digests against gallery entries, and UpgradeBackend() to perform atomic upgrades with backup-based rollback on failure. Includes Agent A's data model changes (Version/URI/Digest fields, GetImageDigest). * feat: add AutoUpgradeBackends config and runtime settings Add configuration and runtime settings for backend auto-upgrade: - RuntimeSettings field for dynamic config via API/JSON - ApplicationConfig field, option func, and roundtrip conversion - CLI flag with LOCALAI_AUTO_UPGRADE_BACKENDS env var - Config file watcher support for runtime_settings.json - Tests for ToRuntimeSettings, ApplyRuntimeSettings, and roundtrip * feat(ui): add backend version display and upgrade support - Add upgrade check/trigger API endpoints to config and api module - Backends page: version badge, upgrade indicator, upgrade button - Manage page: version in metadata, context-aware upgrade/reinstall button - Settings page: auto-upgrade backends toggle * feat: add upgrade checker service, API endpoints, and CLI command - UpgradeChecker background service: checks every 6h, auto-upgrades when enabled - API endpoints: GET /backends/upgrades, POST /backends/upgrades/check, POST /backends/upgrade/:name - CLI: `localai backends upgrade` command, version display in `backends list` - BackendManager interface: add UpgradeBackend and CheckUpgrades methods - Wire upgrade op through GalleryService backend handler - Distributed mode: fan-out upgrade to worker nodes via NATS * fix: use advisory lock for upgrade checker in distributed mode In distributed mode with multiple frontend instances, use PostgreSQL advisory lock (KeyBackendUpgradeCheck) so only one instance runs periodic upgrade checks and auto-upgrades. Prevents duplicate upgrade operations across replicas. Standalone mode is unchanged (simple ticker loop). * test: add e2e tests for backend upgrade API - Test GET /api/backends/upgrades returns 200 (even with no upgrade checker) - Test POST /api/backends/upgrade/:name accepts request and returns job ID - Test full upgrade flow: trigger upgrade via API, wait for job completion, verify run.sh updated to v2 and metadata.json has version 2.0.0 - Test POST /api/backends/upgrades/check returns 200 - Fix nil check for applicationInstance in upgrade API routes |
||
|
Richard Palethorpe
|
557d0f0f04 |
feat(api): Allow coding agents to interactively discover how to control and configure LocalAI (#9084)
Signed-off-by: Richard Palethorpe <io@richiejp.com> |
||
|
Ettore Di Giacinto
|
59108fbe32 |
feat: add distributed mode (#9124)
* feat: add distributed mode (experimental) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix data races, mutexes, transactions Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactorings Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix events and tool stream in agent chat Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * use ginkgo Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(cron): compute correctly time boundaries avoiding re-triggering Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * enhancements, refactorings Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * do not flood of healthy checks Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * do not list obvious backends as text backends Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * tests fixups Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactoring and consolidation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Drop redundant healthcheck Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * enhancements, refactorings Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
c37785b78c |
chore(refactor): move logging to common package based on slog (#7668)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
1cdcaf0152 |
feat: migrate to echo and enable cancellation of non-streaming requests (#7270)
* WIP: migrate to echo Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
3728552e94 |
feat: import models via URI (#7245)
* feat: initial hook to install elements directly Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * WIP: ui changes Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Move HF api client to pkg Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add simple importer for gguf files Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add opcache Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * wire importers to CLI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add omitempty to config fields Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Fix tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add MLX importer Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Small refactors to star to use HF for discovery Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add tests Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Common preferences Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add support to bare HF repos Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(importer/llama.cpp): add support for mmproj files Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * add mmproj quants to common preferences Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Fix vlm usage in tokenizer mode with llama.cpp Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
089efe05fd |
feat(backends): add system backend, refactor (#6059)
- Add a system backend path - Refactor and consolidate system information in system state - Use system state in all the components to figure out the system paths to used whenever needed - Refactor BackendConfig -> ModelConfig. This was otherway misleading as now we do have a backend configuration which is not the model config. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
b52bfaf1b3 |
fix: do not show invalid backends (#6058)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
9aadfd485f |
chore: update swagger (#5946)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Ettore Di Giacinto
|
2d64269763 |
feat: Add backend gallery (#5607)
* feat: Add backend gallery This PR add support to manage backends as similar to models. There is now available a backend gallery which can be used to install and remove extra backends. The backend gallery can be configured similarly as a model gallery, and API calls allows to install and remove new backends in runtime, and as well during the startup phase of LocalAI. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add backends docs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * wip: Backend Dockerfile for python backends Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat: drop extras images, build python backends separately Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixup on all backends Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test CI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Tweaks Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Drop old backends leftovers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Fixup CI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Move dockerfile upper Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Fix proto Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Feature dropped for consistency - we prefer model galleries Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add missing packages in the build image Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * exllama is ponly available on cublas Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * pin torch on chatterbox Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Fixups to index Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * CI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Debug CI * Install accellerators deps Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Add target arch * Add cuda minor version Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Use self-hosted runners Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci: use quay for test images Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fixups for vllm and chatterbox Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Small fixups on CI Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chatterbox is only available for nvidia Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Simplify CI builds Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Adapt test, use qwen3 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(model gallery): add jina-reranker-v1-tiny-en-gguf Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gguf-parser): recover from potential panics that can happen while reading ggufs with gguf-parser Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Use reranker from llama.cpp in AIO images Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * Limit concurrent jobs Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Ettore Di Giacinto <mudler@users.noreply.github.com> |