* feat(credentials): parse and match download credential rules
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(credentials): keep secrets out of parse errors and tighten URL matching
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(credentials): resolve secrets lazily and authenticate HTTP per hop
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(credentials): redact secrets in nested and store formatting
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(credentials): add registry keychain and oras credential adapters
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(credentials): match repository rules for Docker Hub in the oras adapter
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(downloads): authenticate HTTP downloads and gallery reads from the credentials store
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(oci): authenticate registry pulls, resumes, blobs and cosign from the credentials store
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* feat(cli): load download credentials from --credentials-file
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(credentials): correct the local-network registry rules
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(credentials): keep secrets out of match and YAML parse errors
A match that fails to parse is no longer quoted in the Parse error, since
it may be a URL with a token in it. Userinfo is detected before the scheme
check, so ftp://user:token@host is refused as userinfo, and a match with a
query string or fragment is refused because it can never apply and a query
string is where signed URLs carry their token. Every YAML decode error is
now redacted, not only type errors: quoted scalars such as a secret under a
mismatched !!int tag are replaced and unquoted map keys are cut off.
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(downloads): make auth errors name the real cause and never retry unresolved secrets
AuthError now appends its cause, so a registry's DENIED or UNAUTHORIZED
detail reaches the operator. HTTP auth errors print only the status text in
place of the cause, because the downloader builds that cause from the
requested URL, which can carry a signed query string. Registry pulls say
that docker config credentials were tried too, and a download that carried
a caller-provided credential (WithBearerToken, or an explicit authorization
on gallery reads) reports that credential as rejected instead of blaming
the store.
The Range probe for a leftover partial file now returns an unresolved
secret as a permanent error, like the download request already did.
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(credentials): keep oras pulls anonymous on a broken docker helper and close bodies
When docker config names a credsStore helper that cannot run, the oras
credential func now logs at debug and returns no credential, so public
pulls keep working as they did before the adapter existed. The transport
closes the request body when a rule's secret cannot be resolved, as the
RoundTripper contract requires. The redirect spec now uses a custom header
rule on the origin, which net/http would not strip on its own, to prove the
transport does not carry credentials to the next hop.
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* test(oci): cover FetchImageBlob authentication against a private registry
FetchImageBlob now has a spec that pulls a layer blob by digest from a
basic-auth registry through the oras credential adapter, and one that shows
the same fetch fails when no rule matches. oras only speaks HTTPS here, so
the registry serves TLS and the spec points http.DefaultTransport, which
retry.DefaultClient falls back to, at the test server's client for its
duration instead of adding a transport seam to production code.
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* docs(credentials): document auth error wording, ollama manifests and registry tokens
The errors section now lists the registry and provided-credential messages
and says the server's reason is appended. ollama:// manifests are fetched
without credentials, so only blob downloads use the file. GHCR, Docker Hub
and Quay need basic auth with the token as password, and match rules must
not carry a query string or fragment. The backend gallery docs and the
container troubleshooting section now point to the private sources page.
Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(credentials): document trusted file path
The credentials path comes from operator configuration. Mark the file
read with a scoped G304 explanation to resolve the gosec false positive.
Assisted-by: Codex:gpt-6 gosec
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
* feat(cli): benchmark configured text models
Compare model aliases through a running LocalAI server. Report full
request latency and server-reported throughput with raw JSON samples.
Keep warmups separate and fail before writing results on request errors.
Assisted-by: Codex:GPT-6
* fix(cli): satisfy benchmark error checks
Explicitly discard errors from buffered report writes, HTTP response cleanup, and test server writes to pass errcheck without changing behavior.
Assisted-by: Codex:gpt-6 golangci-lint
---------
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
The LocalAGI bump in #11985 changed state.NewAgentPool to take a
SkillsProvider and a PoolLimits value. The call site here was not
updated, so master stopped compiling and every Go job went red.
Pass the limits explicitly, mirroring LocalAGI's own defaults, so the
pool prunes conversation dumps and scheduler run history instead of
growing without end.
Assisted-by: Claude Code:claude-opus-5 [Bash] [Edit]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
pinned: true was only honoured by the per-node watchdog. Every distributed
eviction path was pinned-blind: the router's LRU eviction (EvictLRU,
evictLRUAndFreeNode) and the replica reconciler's idle scale-down would
happily unload a pinned model — and since eviction is gated on
in_flight = 0, a pinned model became eviction-eligible the instant each
response completed. Under capacity pressure that surfaces as the backend
being freed immediately after every request (#11101).
Wire the model config loader into the router and reconciler through a new
PinnedModelResolver seam (mirroring ConcurrencyConflictResolver):
- EvictLRU passes the pinned set into FindLRUModel's query so the
next-oldest unpinned model is selected instead of the attempt failing
- evictLRUAndFreeNode filters pinned models inside its locked selection
- scaleDownIdle skips pinned models entirely: trimming to the floor still
means requests beyond the survivor's capacity pay a cold reload
Deliberate teardown (admin unload, model delete, node drain) intentionally
still applies to pinned models, as does dead-row reaping (state correction,
not eviction).
Regression specs verified to fail with the exclusion disabled.
Addresses the cluster-side eviction gap in #11101
Assisted-by: Claude Code:claude-fable-5 [Claude Code]
Signed-off-by: Adira Denis Muhando <dennisadira@gmail.com>
Do not seed streaming reasoning state when the latest prompt thinking marker is already followed by its matching closing marker. This keeps direct Gemma 4 output in content when its template disables thinking with a preclosed channel.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
backend.stop was the one lifecycle subject a worker never answered. The
controller published and returned nil as soon as the local publish
succeeded, so a stop that killed nothing, and a stop that failed
outright, were indistinguishable from one that worked.
The unload endpoint calls model.unload and then StopBackend. Only the
first is acknowledged, so the endpoint answered 200 while the backend
kept running and held its VRAM, and its own "backend stop failed" branch
could never run. The worker logged the failure and nobody saw it.
Give the subject a reply. The worker now enumerates the process keys it
terminated and reports any per-process error, so StopBackend fails when
the stop failed. Resolving to nothing stays a success: stopping a backend
that is not running leaves the caller in the state it asked for, and
eviction paths stop already-gone models routinely. The empty list is what
says nothing matched, and ReportsStoppedProcesses is what makes that
emptiness trustworthy, the same way BackendDeleteReply handles it.
A worker built before this reply still receives the request and still
stops the backend, it only stays silent, so a timeout degrades to the old
assumption rather than failing every stop on a fleet mid-upgrade. Only
silence degrades: a transport error is still reported, because
UnloadRemoteModel skips its registry cleanup for a node it could not
reach and needs to keep hearing about that.
Assisted-by: Claude:claude-opus-5 golangci-lint
Gallery installs merged family defaults at the YAML root and only re-marshaled them on the artifact path. Persist the defaults in the loader-visible parameters map for every install path while preserving authored overrides.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Adds FunASR/SenseVoice as a Python backend for speech-to-text with
support for CPU, CUDA 12/13, ROCm, Intel SYCL, L4T, and Apple MPS.
Co-authored-by: xingyifeng <xingyifeng@users.noreply.github.com>
Agent Status replaced the chat route and unmounted its EventSource. Any response still in flight could then disappear from the conversation.\n\nOpen status in a separate tab so the chat keeps its live connection until the response completes.\n\nAssisted-by: Codex:gpt-5 [eslint]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Saved profiles previously resolved to one audio path and transcript, so
cloning backends could not use several examples of one personality.
Store ordered audio and transcript pairs while preserving the legacy
first-reference fields. Fish Speech and audio.cpp receive all pairs,
including on distributed workers. Other backends retain their
single-reference behavior.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The API mock also matched navigation to /app/tts and returned a WAV download instead of the React page. Let non-POST requests reach the test server.
Assisted-by: Codex:gpt-5 [Playwright]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
TTS instructions add a second textarea to the page. Target the speech input by its placeholder so the history test does not depend on the page having one textarea.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Let studio users guide speech delivery for backends that support request instructions. Blank guidance stays out of requests and media history.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Clients cannot discover the named voices that an installed TTS model accepts without consulting backend-specific documentation. Expose voice metadata through the audio API and let custom model configs declare their own catalog.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The pinned audio.cpp revision supports HIP, but LocalAI neither builds a ROCm image nor accepts its backend option. AMD hosts therefore fall back to the CPU image.
Build and publish the HIP variant, connect it to AMD capability selection, and accept both upstream HIP names.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Expose diffusers audio pipelines through the existing sound-generation RPC. AudioLDM2 can now return PCM WAV output from the model gallery without a separate backend.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add a NATS event contract and exact-residency provider for backend KV cache reports. Keep guessed request observations as the default routing source while maintaining the reported index for future producers.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Add a Darwin-only MLX-Video backend for LTX-2 and converted Wan checkpoints, expose it through the existing video API, and wire packaging, discovery, tests, docs, and an example.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Broadcast forced-disturb events and successful scale-up resets so every frontend shares the same rolling autoscale signal. Deduplicate NATS echoes, expose an origin-only Prometheus counter, and document cluster behavior.
Closes#10083
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Expose original_config_file through the model configuration metadata registry so registry completeness checks and generated configuration surfaces stay in sync.
Assisted-by: Codex:gpt-5 [systematic-debugging]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Route prefix-cache replica selection through composable filters, weighted scorers, and a replaceable picker. Preserve the existing load guard and deterministic selection policy while exposing per-model scorer weights through scheduling configuration.
Assisted-by: Codex:gpt-5 [go-vet]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(auth): bypass API-key auth for CORS preflight (OPTIONS) requests
When API-key auth is enabled, a browser making a cross-origin API call
first sends an OPTIONS CORS preflight, which cannot carry credentials by
HTTP spec. The auth middleware is registered (app.go:324) before the CORS
middleware (app.go:337-347), so the preflight hit auth first and returned
401 before the CORS middleware could answer it, blocking the actual call.
Bypass auth for OPTIONS so the request reaches the CORS middleware, which
answers the preflight with 200 + headers. Real API requests (GET/POST/etc.)
still require auth. Regression test added (red on master, green on branch).
Refs #4576
Signed-off-by: supermario_leo <leo.stack@outlook.com>
* fix(auth): exempt CORS preflights via publicRouteRegistry instead of middleware bypass
Route the global OPTIONS exemption through publicRouteRegistry
(OPTIONS on every path, replacing the OPTIONS-under-/api/auth/ rule
it subsumes) instead of a hardcoded method check inside Middleware,
so "which requests skip auth" has one mechanism. Preflights now flow
through the same authenticate-then-public-rules path as other public
routes, which also lets a credentialed OPTIONS request keep its user
context.
Update the route-coverage allowlist and the near-prefix lookalike
table for the new semantics (OPTIONS is public on every path by
design; near-prefix privacy stays pinned by the non-OPTIONS entries),
and fix the authentication docs' exempt-route enumeration, which
still described OPTIONS as an /api/auth/-only exemption.
Signed-off-by: supermario_leo <leo.stack@outlook.com>
---------
Signed-off-by: supermario_leo <leo.stack@outlook.com>
fix(ui): preserve decoded route parameters
React Router already decodes dynamic path segments before exposing them through useParams. Decoding those values again crashes pages for names containing a literal percent sign and mutates escape-like substrings.
Use route parameters as-is, encode the model editor API path at the outbound boundary, and cover all affected pages with Playwright.
Fixes#11882
Assisted-by: Codex:gpt-5 eslint playwright
Signed-off-by: QiuLG <l237455523@outlook.com>
fix(ds4): resolve repository imports to GGUF
Select a concrete DeepSeek V4 Flash GGUF from Hugging Face metadata while honoring quantization preference order. Reject unresolved repository imports instead of emitting an invalid bare URI.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
When no sampled candidate fits GPU memory, ranking falls back to the
oversized pool and labels its first model Best fit. Keep GPU picks within
the existing 95% budget and hide the section when no candidate qualifies.
Remove static GPU starter picks so Home cannot reintroduce the same error.
Add browser regressions for both sections and document the empty result.
CPU fallback behavior stays unchanged.
Assisted-by: Codex:gpt-6 [Codex]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
JPEG base64 starts with /9j and was classified as an absolute path in distributed mode. Detect valid JPEG base64 by its decoded magic bytes so vision requests reach the backend unchanged while malformed path-like inputs retain existing staging behavior.
Assisted-by: Codex:gpt-5
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* feat(chat): template.system_messages_after_first — merge or forward late system turns
Tokenizer chat templates such as Qwen3.8 / Qwen3.8-Flash-Next raise
'System message must be at the beginning' for system-role messages that
appear after the leading system block, while agent frameworks (cogito tool
selection and adjustment prompts) legitimately append system instructions
mid-conversation. Every such request failed with a 500 (48 errors in one
10-task agent run).
New per-model option template.system_messages_after_first:
merge fold late system turns into the leading system message
user forward them as user-role turns at their original position
Default (unset) keeps the current pass-through behaviour.
Fixes#11876
Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
* docs(model-config): document template.system_messages_after_first
Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
* fix(config/meta): register template.system_messages_after_first in the field registry
TestAllFieldsHaveRegistryEntries requires every model-config field to have
a registry entry. Adds the entry (templates section, select component) and
the option list for the new field so the coverage gate passes.
Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
---------
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
* fix(realtime): support session voice profile switching
Keep the active resolved voice binding on the realtime session so updates can atomically replace model, voice, and profile parameters while releasing leases at the correct lifecycle boundaries.
Assisted-by: Codex:gpt-5
* docs(realtime): explain voice profile switching
Document the session.update payload for selecting a Voice Library URI and clarify precedence when changing the model in the same event.\n\nAssisted-by: Codex:gpt-5
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Capacity guards reject symlink components. On macOS, temporary paths
start with /var, which links to /private/var, so the new staging tests
fail before exercising cleanup or capacity accounting.
Resolve the fixture directories before building guarded paths. Keep
explicit symlinks within the fixtures for containment tests.
Assisted-by: Codex:gpt-6
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
Backend processes shared the host temporary directory, so crashes could leave request images and audio behind until the filesystem filled. Give each process a locked LocalAI-owned runtime, remove scratch on exit, and sweep only marked abandoned runtimes at the next start.
Also close known request error-path leaks in the Python media backends, CrispASR, LongCat Video, and stable-diffusion.cpp.
Assisted-by: Codex:gpt-5
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(realtime): resolve saved voice profiles
Realtime pipelines now validate saved voices against the selected TTS model and retain leased audio until session teardown. Each synthesis request receives its own transcript parameter map.
Assisted-by: Codex:GPT-5
* docs(tts): document realtime voice defaults
Show how a realtime pipeline selects a saved Voice Library profile at session start. Clarify which session voice updates remain supported.
Assisted-by: Codex:GPT-5
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Realtime turns could invoke the LLM and TTS even when speech transcription returned only whitespace. This let ambient noise produce unsolicited assistant output and polluted conversation history with an empty user turn.
Require non-blank transcript text before automatic response generation while preserving the completed transcription event.
Assisted-by: Codex:gpt-5 golangci-lint
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* docs: design ephemeral staging retention
High-frequency camera and audio inputs can fill a worker before the current six-hour cleanup window expires.
Define a one-hour retention policy that preserves recently modified request payloads.
Assisted-by: Codex:gpt-6
* docs: make ephemeral staging request-owned
Time-based retention can still fill a worker under bursty or high-rate input. Define request-lifecycle cleanup with capacity reservation and crash recovery.
Assisted-by: Codex:gpt-6
* feat(distributed): release exact staged keys
Request inputs need transport-neutral cleanup after backend calls. Add authenticated exact-key deletion for HTTP and coordinated cache eviction before shared-object deletion for S3/NATS.
Preserve URL metacharacters as filename data, reject unsafe keys, and remove upload sidecars while pruning empty request directories.
Assisted-by: Codex:gpt-6
* fix(distributed): release staged request inputs
Ephemeral inputs remained on workers after inference completed. Release each exact key after synchronous and streaming calls, including partial staging failures.
Use a bounded cleanup context so caller cancellation cannot suppress release. Preserve caller requests and backend results when cleanup fails.
Assisted-by: Codex:gpt-6
* feat(worker): bound ephemeral staging capacity
Concurrent staging can otherwise exceed its byte limit or consume reserved filesystem headroom. Explicit states keep bytes charged through each reservation, write, and commit transition.
Use a synchronized waiter count to prove Commit blocks until bounded writers close, and retain committed baselines across re-reservation.
Assisted-by: Codex:gpt-6
* feat(worker): enforce ephemeral staging bounds
Share capacity accounting across HTTP and S3 request inputs so workers
reject uploads before exhausting their filesystem. Reconcile exact release
and crash recovery with the same guard.
Assisted-by: Codex:gpt-6
* fix(distributed): make staged release race-safe
Pin each release path component before removing request-owned inputs and sidecars. Stop pruning when a directory identity changes.
Assisted-by: Codex:gpt-6
* fix(worker): retain staged input ownership
Keep committed request inputs protected from age recovery until exact release ends their ownership. Startup-scanned files remain reclaimable and can acquire ownership through reservation.
Assisted-by: Codex:gpt-6
* fix(worker): claim cached ephemeral inputs
Keep startup-scanned cache hits owned while inference uses them and reconcile their actual size against capacity.
Assisted-by: Codex:gpt-6
* fix(distributed): enforce staging admission
Propagate multimodal staging failures before inference and claim matching ephemeral HTTP cache entries. Fall back to PUT when an older worker does not support claims.
Assisted-by: Codex:gpt-6
* fix(distributed): close staging accounting gaps
Keep unknown-length reservations charged until bytes reach disk and bound NATS release waits by the lifecycle cleanup deadline.
Assisted-by: Codex:gpt-6
* fix(distributed): restage swept cache hits
Treat files removed between cache probing and ownership claims as misses so HTTP and S3 workers can stage them again.
Assisted-by: Codex:gpt-6
* fix(distributed): release staged inputs by request
Release every input from one inference with one fixed-size worker coordination request. Fence request ingress against cleanup, bound staging capacity and cleanup state, and retain exact-key release for rolling upgrades.
Assisted-by: Codex:gpt-6
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Virtual model names have no primary file to anchor the worker path.
Companion assets still stage successfully, but relative options retain
an incorrect model directory and fail to load.
Derive the worker root from successfully staged option assets when the
primary path is absent. Cover Buffalo packs, files, directories,
overrides, and failed transfers. Document the frontend upgrade.
Assisted-by: Codex:gpt-6 golangci-lint
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
A worker can retain all model bytes with an unfinished-upload marker.
Retries then start at zero and repeatedly fail with HTTP 416.
Verify the existing bytes and finalize same-file retries at full size.
Reuse the normal integrity checks so corrupt content cannot be accepted.
Add regression coverage and document worker recovery.
Assisted-by: Codex:gpt-6 golangci-lint
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Accept original embeddings and timestamps so clients can restore faces
when the in-memory store restarts. Derive stable IDs from exact vectors
to make registration retries preserve identity without duplicate entries.
Assisted-by: Codex:GPT-6 golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(distributed): stage sound detection audio
Sound detection passes frontend temporary paths directly to remote
workers, unlike transcription. Stage the WAV before classification so
CED can read it without a shared temporary directory.
Preserve the original request for retries and propagate staging errors
without calling the backend. Cover staging, request preservation, and
error handling with regression tests.
Assisted-by: Codex:GPT-6 golangci-lint
* test(distributed): verify routed sound staging
Call sound detection through the client returned by SmartRouter.Route.
This checks interface dispatch through both routing wrappers, rather
than constructing FileStagingClient directly.
The test fails without the sound-staging override and passes with it.
Assisted-by: Codex:GPT-6 golangci-lint
---------
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
A quantization job that runs with no client on its progress stream stays
"queued" forever, in the API and in state.json, while the finished artifact
sits on disk. state.json was written once by StartJob, and the only code that
advanced a job afterwards lived inside the stream callback of StreamProgress,
so job state depended on somebody watching it.
The backend's progress stream cannot simply gain a second reader: each job owns
one queue.Queue and QuantizationProgress pops from it, so two consumers split
the updates rather than both seeing them. The stream has to be opened exactly
once per job.
StartJob now starts watchProgress on the application context (the request
context is done as soon as the handler returns). That goroutine is the single
reader: it applies each update to the job -- in the cross-replica store and in
state.json, terminal statuses still winning over late updates -- and republishes
it in-process. StreamProgress becomes a pure reader over that fan-out and no
longer loads a backend or opens a stream. A client attaching to a job that has
already finished, including one hydrated from disk after a restart, gets a final
event built from the stored job instead of blocking.
Two paths used to end a client's stream by breaking the gRPC connection and now
release it explicitly: StopJob kills the backend, so it publishes the stopped
event itself; and a stream that ends without a terminal update means the backend
is gone, so the job is recorded as failed rather than left running forever.
Signed-off-by: Tai An <antai12232931@outlook.com>
The recommended sampling parameters for a model family were applied at
install and then never took effect. Two things went wrong on the way to
disk.
They were written as top level keys. ModelConfig embeds PredictionOptions
under the "parameters" yaml key, so temperature, top_p, top_k, min_p,
repeat_penalty and presence_penalty are only read from there. At the top
level they parse without error and are then ignored for the life of the
model.
They were also merged in after the YAML had already been marshalled. The
only re-marshal sat behind the artifact binding, which an entry carrying
files: never reaches, so for those entries the defaults were computed and
then dropped before anything was written.
Neither failure was visible in normal use. ApplyInferenceDefaults runs
again at load time and fills the same values from the same table, so the
model ends up tuned correctly while the file on disk pins nothing. It
surfaces when someone edits one of those values expecting it to win, or
when a family is absent from inference_defaults.json and there is nothing
to refill from.
Both install paths are covered: an entry carrying files:, and one that
binds a primary artifact instead.
The empty base spec asserted that the authored parameters block landed
verbatim. It now checks the authored keys individually, because the family
defaults are merged into that same block.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
Web Chat Settings left the System Prompt field empty but still treated a
blank/whitespace value as an explicit system turn. That satisfied
tokenizer chat templates' messages[0].role == system check and suppressed
the model YAML system_prompt on fresh chats.
Omit empty/whitespace system messages in the React and Alpine UIs, strip
them server-side, and inject config.SystemPrompt for tokenizer-template
models when the request has no real system turn.
Fixes#11834
Signed-off-by: lei_lei <96427312+leilei3167@users.noreply.github.com>
