When no sampled candidate fits GPU memory, ranking falls back to the
oversized pool and labels its first model Best fit. Keep GPU picks within
the existing 95% budget and hide the section when no candidate qualifies.
Remove static GPU starter picks so Home cannot reintroduce the same error.
Add browser regressions for both sections and document the empty result.
CPU fallback behavior stays unchanged.
Assisted-by: Codex:gpt-6 [Codex]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
JPEG base64 starts with /9j and was classified as an absolute path in distributed mode. Detect valid JPEG base64 by its decoded magic bytes so vision requests reach the backend unchanged while malformed path-like inputs retain existing staging behavior.
Assisted-by: Codex:gpt-5
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* feat(chat): template.system_messages_after_first — merge or forward late system turns
Tokenizer chat templates such as Qwen3.8 / Qwen3.8-Flash-Next raise
'System message must be at the beginning' for system-role messages that
appear after the leading system block, while agent frameworks (cogito tool
selection and adjustment prompts) legitimately append system instructions
mid-conversation. Every such request failed with a 500 (48 errors in one
10-task agent run).
New per-model option template.system_messages_after_first:
merge fold late system turns into the leading system message
user forward them as user-role turns at their original position
Default (unset) keeps the current pass-through behaviour.
Fixes#11876
Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
* docs(model-config): document template.system_messages_after_first
Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
* fix(config/meta): register template.system_messages_after_first in the field registry
TestAllFieldsHaveRegistryEntries requires every model-config field to have
a registry entry. Adds the entry (templates section, select component) and
the option list for the new field so the coverage gate passes.
Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
---------
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
* fix(realtime): support session voice profile switching
Keep the active resolved voice binding on the realtime session so updates can atomically replace model, voice, and profile parameters while releasing leases at the correct lifecycle boundaries.
Assisted-by: Codex:gpt-5
* docs(realtime): explain voice profile switching
Document the session.update payload for selecting a Voice Library URI and clarify precedence when changing the model in the same event.\n\nAssisted-by: Codex:gpt-5
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Capacity guards reject symlink components. On macOS, temporary paths
start with /var, which links to /private/var, so the new staging tests
fail before exercising cleanup or capacity accounting.
Resolve the fixture directories before building guarded paths. Keep
explicit symlinks within the fixtures for containment tests.
Assisted-by: Codex:gpt-6
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Backend processes shared the host temporary directory, so crashes could leave request images and audio behind until the filesystem filled. Give each process a locked LocalAI-owned runtime, remove scratch on exit, and sweep only marked abandoned runtimes at the next start.
Also close known request error-path leaks in the Python media backends, CrispASR, LongCat Video, and stable-diffusion.cpp.
Assisted-by: Codex:gpt-5
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(realtime): resolve saved voice profiles
Realtime pipelines now validate saved voices against the selected TTS model and retain leased audio until session teardown. Each synthesis request receives its own transcript parameter map.
Assisted-by: Codex:GPT-5
* docs(tts): document realtime voice defaults
Show how a realtime pipeline selects a saved Voice Library profile at session start. Clarify which session voice updates remain supported.
Assisted-by: Codex:GPT-5
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Realtime turns could invoke the LLM and TTS even when speech transcription returned only whitespace. This let ambient noise produce unsolicited assistant output and polluted conversation history with an empty user turn.
Require non-blank transcript text before automatic response generation while preserving the completed transcription event.
Assisted-by: Codex:gpt-5 golangci-lint
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
RF-DETR and Locate Anything wrote each decoded request image to the OS
temporary directory. A full temporary filesystem then disabled detection,
even though both native libraries already accept encoded image buffers.
Pass decoded images directly to the native buffer APIs. This removes the
request-time disk dependency and prevents crash-orphaned image files.
Assisted-by: Codex:gpt-5
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* docs: design ephemeral staging retention
High-frequency camera and audio inputs can fill a worker before the current six-hour cleanup window expires.
Define a one-hour retention policy that preserves recently modified request payloads.
Assisted-by: Codex:gpt-6
* docs: make ephemeral staging request-owned
Time-based retention can still fill a worker under bursty or high-rate input. Define request-lifecycle cleanup with capacity reservation and crash recovery.
Assisted-by: Codex:gpt-6
* feat(distributed): release exact staged keys
Request inputs need transport-neutral cleanup after backend calls. Add authenticated exact-key deletion for HTTP and coordinated cache eviction before shared-object deletion for S3/NATS.
Preserve URL metacharacters as filename data, reject unsafe keys, and remove upload sidecars while pruning empty request directories.
Assisted-by: Codex:gpt-6
* fix(distributed): release staged request inputs
Ephemeral inputs remained on workers after inference completed. Release each exact key after synchronous and streaming calls, including partial staging failures.
Use a bounded cleanup context so caller cancellation cannot suppress release. Preserve caller requests and backend results when cleanup fails.
Assisted-by: Codex:gpt-6
* feat(worker): bound ephemeral staging capacity
Concurrent staging can otherwise exceed its byte limit or consume reserved filesystem headroom. Explicit states keep bytes charged through each reservation, write, and commit transition.
Use a synchronized waiter count to prove Commit blocks until bounded writers close, and retain committed baselines across re-reservation.
Assisted-by: Codex:gpt-6
* feat(worker): enforce ephemeral staging bounds
Share capacity accounting across HTTP and S3 request inputs so workers
reject uploads before exhausting their filesystem. Reconcile exact release
and crash recovery with the same guard.
Assisted-by: Codex:gpt-6
* fix(distributed): make staged release race-safe
Pin each release path component before removing request-owned inputs and sidecars. Stop pruning when a directory identity changes.
Assisted-by: Codex:gpt-6
* fix(worker): retain staged input ownership
Keep committed request inputs protected from age recovery until exact release ends their ownership. Startup-scanned files remain reclaimable and can acquire ownership through reservation.
Assisted-by: Codex:gpt-6
* fix(worker): claim cached ephemeral inputs
Keep startup-scanned cache hits owned while inference uses them and reconcile their actual size against capacity.
Assisted-by: Codex:gpt-6
* fix(distributed): enforce staging admission
Propagate multimodal staging failures before inference and claim matching ephemeral HTTP cache entries. Fall back to PUT when an older worker does not support claims.
Assisted-by: Codex:gpt-6
* fix(distributed): close staging accounting gaps
Keep unknown-length reservations charged until bytes reach disk and bound NATS release waits by the lifecycle cleanup deadline.
Assisted-by: Codex:gpt-6
* fix(distributed): restage swept cache hits
Treat files removed between cache probing and ownership claims as misses so HTTP and S3 workers can stage them again.
Assisted-by: Codex:gpt-6
* fix(distributed): release staged inputs by request
Release every input from one inference with one fixed-size worker coordination request. Fence request ingress against cleanup, bound staging capacity and cleanup state, and retain exact-key release for rolling upgrades.
Assisted-by: Codex:gpt-6
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(gallery): default audio-cpp models to backend:best
The audio-cpp engine creates its session on the CPU backend when no
backend option is given, so every gallery model ran CPU-only even on
machines where a CUDA/Vulkan/Metal device was registered. backend:best
selects the best available backend and falls back to CPU.
Assisted-by: Claude:claude-fable-5
Signed-off-by: Plamen K. Kosseff <p.kosseff@gmail.com>
* docs(audio-cpp): explain gallery device selection
Document automatic compute backend selection and the CPU override.
Assisted-by: Codex:gpt-6
Signed-off-by: Plamen K. Kosseff <p.kosseff@gmail.com>
---------
Signed-off-by: Plamen K. Kosseff <p.kosseff@gmail.com>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The device fell back to CPU unless the model config set cuda: true,
while MPS right below was auto-detected — GPU hosts silently rendered
on CPU for any gallery entry missing the flag. Use CUDA whenever torch
reports it available (ROCm builds included), keep cuda: true as an
explicit force, and allow pinning with the device: model option (e.g.
options: ["device:cpu"]). Gallery entries stay untouched.
Assisted-by: Claude:claude-fable-5
Signed-off-by: Plamen K. Kosseff <p.kosseff@gmail.com>
Virtual model names have no primary file to anchor the worker path.
Companion assets still stage successfully, but relative options retain
an incorrect model directory and fail to load.
Derive the worker root from successfully staged option assets when the
primary path is absent. Cover Buffalo packs, files, directories,
overrides, and failed transfers. Document the frontend upgrade.
Assisted-by: Codex:gpt-6 golangci-lint
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
A worker can retain all model bytes with an unfinished-upload marker.
Retries then start at zero and repeatedly fail with HTTP 416.
Verify the existing bytes and finalize same-file retries at full size.
Reuse the normal integrity checks so corrupt content cannot be accepted.
Add regression coverage and document worker recovery.
Assisted-by: Codex:gpt-6 golangci-lint
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Accept original embeddings and timestamps so clients can restore faces
when the in-memory store restarts. Derive stable IDs from exact vectors
to make registration retries preserve identity without duplicate entries.
Assisted-by: Codex:GPT-6 golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(distributed): stage sound detection audio
Sound detection passes frontend temporary paths directly to remote
workers, unlike transcription. Stage the WAV before classification so
CED can read it without a shared temporary directory.
Preserve the original request for retries and propagate staging errors
without calling the backend. Cover staging, request preservation, and
error handling with regression tests.
Assisted-by: Codex:GPT-6 golangci-lint
* test(distributed): verify routed sound staging
Call sound detection through the client returned by SmartRouter.Route.
This checks interface dispatch through both routing wrappers, rather
than constructing FileStagingClient directly.
The test fails without the sound-staging override and passes with it.
Assisted-by: Codex:GPT-6 golangci-lint
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Community-maintained packagings that currently track releases —
Homebrew, ALT Sisyphus and the Gentoo local-ai overlay — with a note
that versions may lag. Placement and scope as discussed in the issue.
Assisted-by: Claude:claude-fable-5
Signed-off-by: Plamen K. Kosseff <p.kosseff@gmail.com>
* ⬆️ Update leejet/stable-diffusion.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(stablediffusion): adapt streaming options
Upstream now selects segmented weight streaming automatically and removes the stream_layers field. Keep the old LocalAI option as a no-op for existing model configurations.
Assisted-by: Codex:gpt-5
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* ⬆️ Update antirez/ds4
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(ds4): link upstream image helpers
The ds4 bump adds vision calls to the engine object. Link the new image preprocessing object into every backend target.
Assisted-by: Codex:gpt-5
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* Update containers.md to fix podman image qualification
Signed-off-by: Alex Mazzariol <alex@alex-maz.info>
* docs(containers): clarify Podman image names
Podman can reject short image names when no registry is configured. Explain why the examples use fully qualified Docker Hub names.
Assisted-by: Codex:gpt-5.6
---------
Signed-off-by: Alex Mazzariol <alex@alex-maz.info>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
