Commit Graph
759 Commits
Author SHA1 Message Date
Leoy aeac5e86dc fix(auth): bypass API-key auth for CORS preflight (OPTIONS) requests (#11113)
* fix(auth): bypass API-key auth for CORS preflight (OPTIONS) requests

When API-key auth is enabled, a browser making a cross-origin API call
first sends an OPTIONS CORS preflight, which cannot carry credentials by
HTTP spec. The auth middleware is registered (app.go:324) before the CORS
middleware (app.go:337-347), so the preflight hit auth first and returned
401 before the CORS middleware could answer it, blocking the actual call.

Bypass auth for OPTIONS so the request reaches the CORS middleware, which
answers the preflight with 200 + headers. Real API requests (GET/POST/etc.)
still require auth. Regression test added (red on master, green on branch).

Refs #4576

Signed-off-by: supermario_leo <leo.stack@outlook.com>

* fix(auth): exempt CORS preflights via publicRouteRegistry instead of middleware bypass

Route the global OPTIONS exemption through publicRouteRegistry
(OPTIONS on every path, replacing the OPTIONS-under-/api/auth/ rule
it subsumes) instead of a hardcoded method check inside Middleware,
so "which requests skip auth" has one mechanism. Preflights now flow
through the same authenticate-then-public-rules path as other public
routes, which also lets a credentialed OPTIONS request keep its user
context.

Update the route-coverage allowlist and the near-prefix lookalike
table for the new semantics (OPTIONS is public on every path by
design; near-prefix privacy stays pinned by the non-OPTIONS entries),
and fix the authentication docs' exempt-route enumeration, which
still described OPTIONS as an /api/auth/-only exemption.

Signed-off-by: supermario_leo <leo.stack@outlook.com>

---------

Signed-off-by: supermario_leo <leo.stack@outlook.com>
2026-09-11 17:57:09 +02:00
QiuLG e588d463dc fix(ui): preserve percent signs in route parameters (#11883)
fix(ui): preserve decoded route parameters

React Router already decodes dynamic path segments before exposing them through useParams. Decoding those values again crashes pages for names containing a literal percent sign and mutates escape-like substrings.

Use route parameters as-is, encode the model editor API path at the outbound boundary, and cover all affected pages with Playwright.

Fixes #11882

Assisted-by: Codex:gpt-5 eslint playwright

Signed-off-by: QiuLG <l237455523@outlook.com>
2026-09-11 16:54:56 +02:00
Abdullah Mansour | عبد الله منصور acc7284526 fix(openresponses): support Codex WebSocket warm-up (#11608)
* fix(openresponses): support Codex WebSocket warmup

Assisted-by: ChatGPT:GPT-5.6-Sol golangci-lint
Signed-off-by: Abdullah Mansour <abdullahmansour.marketing@gmail.com>

* docs(openresponses): document WebSocket responses

Assisted-by: Codex:GPT-5.6-Sol gh Docker golangci-lint
Signed-off-by: Abdullah Mansour <abdullahmansour.marketing@gmail.com>

* fix(openresponses): harden WebSocket response lifecycle

Ensure response ownership, continuation storage, error sequencing, and connection-local resource limits remain correct across HTTP and WebSocket transports.

Assisted-by: Codex:GPT-5.6-Sol [gh] [Docker] [golangci-lint]
Signed-off-by: Abdullah Mansour <abdullahmansour.marketing@gmail.com>

---------

Signed-off-by: Abdullah Mansour <abdullahmansour.marketing@gmail.com>
2026-09-11 16:54:42 +02:00
dependabot[bot] 428898373a chore(deps): bump the npm_and_yarn group across 1 directory with 2 updates (#11957)
Bumps the npm_and_yarn group with 2 updates in the /core/http/react-ui directory: [hono](https://github.com/honojs/hono) and [js-yaml](https://github.com/nodeca/js-yaml).


Updates `hono` from 4.12.34 to 4.13.5
- [Release notes](https://github.com/honojs/hono/releases)
- [Commits](https://github.com/honojs/hono/compare/v4.12.34...v4.13.5)

Updates `js-yaml` from 3.14.2 to 3.15.2
- [Changelog](https://github.com/nodeca/js-yaml/blob/3.15.2/CHANGELOG.md)
- [Commits](https://github.com/nodeca/js-yaml/compare/3.14.2...3.15.2)

---
updated-dependencies:
- dependency-name: hono
  dependency-version: 4.13.5
  dependency-type: direct:production
  dependency-group: npm_and_yarn
- dependency-name: js-yaml
  dependency-version: 3.15.2
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-10 08:58:38 +02:00
localai-org-maint-botandlocalai-org-maint-bot de563f17b5 fix(ui): omit GPU recommendations that do not fit (#11945)
When no sampled candidate fits GPU memory, ranking falls back to the
oversized pool and labels its first model Best fit. Keep GPU picks within
the existing 95% budget and hide the section when no candidate qualifies.
Remove static GPU starter picks so Home cannot reintroduce the same error.

Add browser regressions for both sections and document the empty result.
CPU fallback behavior stays unchanged.

Assisted-by: Codex:gpt-6 [Codex]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
2026-09-10 00:53:47 +02:00
Stefan Walcz 109244a76a [chat] feat: template.system_messages_after_first — merge or forward late system turns (#11906)
* feat(chat): template.system_messages_after_first — merge or forward late system turns

Tokenizer chat templates such as Qwen3.8 / Qwen3.8-Flash-Next raise
'System message must be at the beginning' for system-role messages that
appear after the leading system block, while agent frameworks (cogito tool
selection and adjustment prompts) legitimately append system instructions
mid-conversation. Every such request failed with a 500 (48 errors in one
10-task agent run).

New per-model option template.system_messages_after_first:
  merge  fold late system turns into the leading system message
  user   forward them as user-role turns at their original position
Default (unset) keeps the current pass-through behaviour.

Fixes #11876

Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>

* docs(model-config): document template.system_messages_after_first

Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>

* fix(config/meta): register template.system_messages_after_first in the field registry

TestAllFieldsHaveRegistryEntries requires every model-config field to have
a registry entry. Adds the entry (templates section, select component) and
the option list for the new field so the coverage gate passes.

Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>

---------

Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
2026-09-09 22:21:32 +02:00
localai-org-maint-botandEttore Di Giacinto f12bcfac9a fix(realtime): support voice profile switching (#11948)
* fix(realtime): support session voice profile switching

Keep the active resolved voice binding on the realtime session so updates can atomically replace model, voice, and profile parameters while releasing leases at the correct lifecycle boundaries.

Assisted-by: Codex:gpt-5

* docs(realtime): explain voice profile switching

Document the session.update payload for selecting a Voice Library URI and clarify precedence when changing the model in the same event.\n\nAssisted-by: Codex:gpt-5

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-09-09 10:38:23 +02:00
localai-org-maint-botandEttore Di Giacinto 8c718441f6 fix(realtime): resolve pipeline voice profiles (#11942)
* fix(realtime): resolve saved voice profiles

Realtime pipelines now validate saved voices against the selected TTS model and retain leased audio until session teardown. Each synthesis request receives its own transcript parameter map.

Assisted-by: Codex:GPT-5

* docs(tts): document realtime voice defaults

Show how a realtime pipeline selects a saved Voice Library profile at session start. Clarify which session voice updates remain supported.

Assisted-by: Codex:GPT-5

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-09-08 23:42:31 +02:00
localai-org-maint-botandEttore Di Giacinto 780e458f11 fix(realtime): skip responses for empty transcripts (#11940)
Realtime turns could invoke the LLM and TTS even when speech transcription returned only whitespace. This let ambient noise produce unsolicited assistant output and polluted conversation history with an empty user turn.

Require non-blank transcript text before automatic response generation while preserving the completed transcription event.

Assisted-by: Codex:gpt-5 golangci-lint

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-09-08 23:12:57 +02:00
localai-org-maint-botandEttore Di Giacinto 88d19567b8 feat(faces): replay saved face enrollments (#11908)
Accept original embeddings and timestamps so clients can restore faces
when the in-memory store restarts. Derive stable IDs from exact vectors
to make registration retries preserve identity without duplicate entries.

Assisted-by: Codex:GPT-6 golangci-lint

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-09-07 17:40:27 +02:00
Ettore Di Giacinto 18d20239df fix(ui): keep trace expansion stable during refresh
Squashed merge of #11278.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-09-05 22:09:16 +00:00
pengmin 718357219b fix(ui): send collection intervals as numbers
Squashed merge of #11819.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-09-05 22:09:16 +00:00
Anupam Mediratta 1479aef8cd fix(deps): upgrade path-to-regexp to 8.4.0 (CVE-2026-4926)
Squashed merge of #11633.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-09-05 22:09:16 +00:00
dependabot[bot] ee2920d619 chore(deps): bump the npm_and_yarn group across 1 directory with 3 updates (#11869)
Bumps the npm_and_yarn group with 3 updates in the /core/http/react-ui directory: [@humanfs/node](https://github.com/humanwhocodes/humanfs/tree/HEAD/packages/node), [fast-uri](https://github.com/fastify/fast-uri) and [qs](https://github.com/ljharb/qs).


Updates `@humanfs/node` from 0.16.7 to 0.16.8
- [Release notes](https://github.com/humanwhocodes/humanfs/releases)
- [Changelog](https://github.com/humanwhocodes/humanfs/blob/main/packages/node/CHANGELOG.md)
- [Commits](https://github.com/humanwhocodes/humanfs/commits/node-v0.16.8/packages/node)

Updates `fast-uri` from 3.1.5 to 3.1.7
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.5...v3.1.7)

Updates `qs` from 6.15.3 to 6.16.0
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.15.3...v6.16.0)

---
updated-dependencies:
- dependency-name: "@humanfs/node"
  dependency-version: 0.16.8
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: fast-uri
  dependency-version: 3.1.7
  dependency-type: indirect
  dependency-group: npm_and_yarn
- dependency-name: qs
  dependency-version: 6.16.0
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 11:25:54 +02:00
dependabot[bot] 99a2e855a8 chore(deps): bump the npm_and_yarn group across 1 directory with 2 updates (#11835)
Bumps the npm_and_yarn group with 2 updates in the /core/http/react-ui directory: [react-router](https://github.com/remix-run/react-router/tree/HEAD/packages/react-router) and [browserslist](https://github.com/browserslist/browserslist).


Updates `react-router` from 7.18.1 to 7.18.2
- [Release notes](https://github.com/remix-run/react-router/releases)
- [Changelog](https://github.com/remix-run/react-router/blob/react-router@7.18.2/packages/react-router/CHANGELOG.md)
- [Commits](https://github.com/remix-run/react-router/commits/react-router@7.18.2/packages/react-router)

Updates `browserslist` from 4.28.2 to 4.28.8
- [Release notes](https://github.com/browserslist/browserslist/releases)
- [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md)
- [Commits](https://github.com/browserslist/browserslist/compare/4.28.2...4.28.8)

---
updated-dependencies:
- dependency-name: react-router
  dependency-version: 7.18.2
  dependency-type: direct:production
  dependency-group: npm_and_yarn
- dependency-name: browserslist
  dependency-version: 4.28.8
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-03 23:38:23 +02:00
lei_lei 7c921aa020 fix(ui): omit empty system prompt so model default applies (#11838)
Web Chat Settings left the System Prompt field empty but still treated a
blank/whitespace value as an explicit system turn. That satisfied
tokenizer chat templates' messages[0].role == system check and suppressed
the model YAML system_prompt on fresh chats.

Omit empty/whitespace system messages in the React and Alpine UIs, strip
them server-side, and inject config.SystemPrompt for tokenizer-template
models when the request has no real system turn.

Fixes #11834

Signed-off-by: lei_lei <96427312+leilei3167@users.noreply.github.com>
2026-09-02 11:58:23 +00:00
localai-org-maint-botandlocalai-org-maint-bot 893a45141c fix(realtime): accept GA WebRTC signaling (#11778)
OpenAI GA clients send multipart or raw SDP requests. They expect a bare
SDP answer. LocalAI only accepted its legacy JSON envelope, so signaling
failed before media setup.

Keep the JSON contract for existing clients. Accept both GA request
shapes and choose the matching response format.

Assisted-by: Codex:gpt-5

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
2026-08-29 21:28:37 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 80e3240f2d feat(distributed): key scheduling rules by a model alias (#11771)
Node placement and replica rules could only name a model, so an operator
who pinned "llama3" to the GPU tier had to rewrite the rule whenever a
different model took over that job. An alias already gives a stable name
for whichever model serves it, and a rule on that name makes it a
deployment slot: repoint the alias and the placement follows.

A rule keeps the name the operator chose. Reads resolve that name through
the config loader to the model the rule governs, so the reconciler counts,
schedules and trims replicas of the target, and the router finds an
alias-keyed rule from the target it is already routing. An alias that
resolves to nothing governs nothing loadable, so the reconciler skips it
and the write paths refuse it.

A replica is shared by every name that resolves to it, so only one rule
can decide where it runs. The REST and MCP write paths reject a rule whose
target another rule already governs. A pair that arrives some other way,
such as a seed file or an alias repointed onto a model that already has a
rule, resolves in favour of the rule named after the model itself and then
the oldest, and the rest are listed as shadowed.

The eviction guard is the exception: it matches rules to replicas in raw
SQL inside a locking transaction and cannot resolve an alias. It reads a
stored target that the reconciler refreshes each tick, and falls back to
the rule's own name when that target is empty.


Assisted-by: Claude:claude-opus-5 golangci-lint eslint

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-08-29 09:16:50 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 29899cd1e0 fix(ui): size model fit against the cluster and move node labels into the selector (#11765)
* fix(ui): move node labels into the scheduling selector field

The scheduling page kept a node-label browser open above the rules
whether or not anyone was writing one, while the field that actually
needs labels, the rule's node selector, was two bare text inputs with no
hint of what the cluster reports.

The browser is gone. The selector's key input now completes against the
label keys the cluster uses, and the value input offers only the values
that key takes. The roster already loads for the page, so the
suggestions cost no request, and a roster that fails to load costs the
admin the hints and nothing else.

Suggestions stay suggestions: a key no node reports yet still commits as
typed, which is how an admin writes a rule before labelling the nodes
for it.

Assisted-by: Claude:claude-opus-5 golangci-lint eslint playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(distributed): size model fit against the cluster, not the frontend

The models page asked the frontend how much memory a model may occupy.
In distributed mode the frontend is usually a GPU-less pod while every
model runs on a worker, so a fleet of GPU nodes was told it could only
run the smallest CPU build. The variant picker's fits flag and its
auto-selection came from the same place, as did the hardware
recommendations.

The registry now reports the largest single healthy backend node. The
largest node, not the fleet total: a model loads into one node, so four
16GB workers are not a home for a 40GB model. An operator-set VRAM
budget caps a node's contribution, because the scheduler refuses a load
above that ceiling anyway, and a GPU node beats a CPU node holding more
system RAM.

GET /api/resources and GET /api/models carry this as an additional
cluster object. Their aggregate and ram fields keep reporting the
frontend's own hardware, which is what the resource monitor shows.
Variant selection judges backends against the union of the capabilities
present in the cluster, the way backend discovery already did.

Every path degrades to the local host: no cluster object in single-node
mode, and none when the registry cannot be read, so a hiccup narrows the
answer back to single-node behaviour rather than marking the whole
catalog too large.

The verdicts now name the node they belong to, since a model fits
somewhere or nowhere.

Assisted-by: Claude:claude-opus-5 golangci-lint eslint playwright
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-08-28 22:57:35 +02:00
Plamen K. Kosseff e58dabf75f feat(ui): add 'Focus mode' option in chat settings to persistently toggle the sidebar auto-collapse (#11750)
Assisted-by: Claude:claude-fable-5

Signed-off-by: Plamen K. Kosseff <p.kosseff@gmail.com>
2026-08-27 22:32:32 +02:00
lei_lei f28e8b24e6 fix(ollama): accept :latest tag on model lookup (#11732)
/api/tags appends :latest to untagged names, but chat and the other
model endpoints looked the tagged name up as-is and 404'd.

Signed-off-by: lei_lei <96427312+leilei3167@users.noreply.github.com>
2026-08-26 08:57:36 +02:00
Ettore Di Giacinto 1dc3aeef87 fix(distributed): resolve config revisions through one entry point
A model's revision is published by administration and checked against on
every inference request. Those were computed by separate code: the
request path resolves through the loader, while each publisher hashed
whatever ModelConfig it happened to hold. By then SetDefaults had folded
in the GGUF guess and app-level options, so the published value was one
no request would ever carry and the model became unroutable until the
row was deleted by hand.

Fixing the publishers one at a time did not hold. Three rounds each
found another: the startup resync, then a saved edit and a toggle, then
a rename and the peer-change path.

ModelConfigLoader.RevisionFor is now the only way to obtain a revision,
and the raw hash is unexported, so a caller outside this package cannot
hash a config it holds. A publisher and a request agree by construction
rather than by two implementations happening to match.

The request path no longer falls back to hashing its merged config
either: an unstamped config is routed without a revision rather than
with a wrong one.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
2026-08-24 19:11:16 +00:00
Ettore Di Giacinto df1a40f9c0 fix(distributed): hash the config as persisted, not as defaulted
The revision was computed after SetDefaults, which folds in things that
are not persisted configuration: the GGUF guess, the hardware defaults,
and app-level options such as threads.

The GGUF guess is the damaging one. It parses the model file to fill in
values like context size, and when that parse fails it falls back to a
different default. Whether a multi-gigabyte file on network storage
parses at a given moment is not a property of the configuration, so one
unchanged YAML produced two different revisions depending on when it was
read. The controller rejected every request carrying the other one, and
the model stayed unroutable until the stored value happened to match
again. This is why it never reproduced against a model directory with no
weights in it: the guess is skipped there and both values agree.

The app-level defaults are the same class of bug with a slower fuse:
changing threads in the settings UI changed every model's revision and
made every model unroutable.

The revision is now stamped when the file is parsed, before any defaults
are applied, so it is a function of the file alone.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
2026-08-24 11:45:32 +00:00
localai-org-maint-botandlocalai-org-maint-bot a8bc64cd09 fix(ci): bound Discord release summaries (#11695)
* fix(ci): bound Discord release summaries

The release model can return more than Discord's 2,000-character
message limit. Discord then rejects the entire release notification.

Ask the model for a smaller response and truncate extracted content to
1,800 characters before the notification step. The smaller bound leaves
room below Discord's hard limit when model output varies.

Assisted-by: Codex:gpt-5

* fix(tests): implement node liveness stub

NodeCommandSender now requires PingNode. The endpoint test stub must
implement it before the package can compile.

Assisted-by: Codex:gpt-5 [Codex]

---------

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
2026-08-24 09:32:26 +02:00
Ettore Di Giacinto f3fabe8c5c fix(distributed): order derived usecases deterministically
syncKnownUsecasesFromString rebuilds KnownUsecaseStrings by ranging
GetAllModelConfigUsecases, which is a map. Go randomizes that order per
call, and the field is part of the serialized config, so one unchanged
YAML hashed to a different config revision on every load.

A model that derives a single usecase hid the problem. One that derives
several, such as a chat model with an mmproj, alternated between as many
revisions as there are orderings. The router treats a revision it did
not establish as a config change, so requests failed with "stale model
config revision" until the stored value happened to match again.

Sorting the list makes the revision a function of the file alone.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
2026-08-23 17:20:20 +00:00
Ettore Di Giacinto 04735cd1f6 fix(distributed): stamp config revision at load time
The request middleware merges the caller's prediction parameters into
its copy of the model config. core/backend.ModelOptions then hashed
that copy, so the revision identified the request body rather than the
persisted configuration.

EstablishModelConfigRevision stores the first revision it sees and
requires an exact match afterwards. The first request after a restart
therefore pinned the model to its own temperature, top_p and stop
values, and every later request that sent different ones failed with
"stale model config revision". No config edit was involved.

The loader now stamps the revision when it materializes a config,
before any request override reaches it, and ModelOptions reads that
stamp. Model administration keeps hashing the same persisted config, so
both paths agree on one revision per configuration.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5 [golangci-lint]
2026-08-23 14:35:44 +00:00
localai-org-maint-botandlocalai-org-maint-bot 8f56e4e042 fix(vram): persist remote probe metadata (#11487)
* fix(vram): persist remote probe metadata

The startup warmer repeated remote size and GGUF metadata probes after every restart because both caches lived only in memory. Store successful HTTP probes for 24 hours so frequent restarts reuse the prior results.

Bound the cache, reject invalid records, and purge it when gallery data changes. Local model files continue to bypass persistence.

Assisted-by: Codex:gpt-5

* fix(vram): check temporary file cleanup

The lint gate rejects the unchecked cleanup call in the persistent cache writer.

Assisted-by: Codex:gpt-5.6 [golangci-lint]

* fix(vram): make persistent cache optional

Remote metadata probes can transfer enough data that operators need
control over disk reuse and startup warming. Gallery autoload now gates
both behaviors, and the runtime setting applies changes immediately.

Assisted-by: Codex:gpt-5

* fix(ui): expose gallery startup pre-warm

The existing gallery autoload setting also gates the startup metadata warmer. Name both effects in Settings so operators can find the requested boot control.

Assisted-by: Codex:gpt-5

---------

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
2026-08-23 08:55:13 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 82c191afad fix(distributed): keep model replicas config-consistent (#11664)
* docs: design configurable copy buffering

Document the context-aware copy buffer option and its validation plan.

Assisted-by: Codex:gpt-5

* docs: design durable distributed staging operations

Assisted-by: Codex:gpt-5

* docs: design distributed model config revisions

Assisted-by: Codex:GPT-5 [apply_patch] [exec_command]

* feat(config): add stable model revisions

Hash typed model configuration and effective protobuf options deterministically for distributed revision comparisons.

Assisted-by: Codex:GPT-5 [apply_patch] [exec_command]

* feat(worker): acknowledge exact model stops

Assisted-by: Codex:GPT-5 [apply_patch] [exec_command]

* feat(nodes): track model config revisions

Assisted-by: Codex:GPT-5 [apply_patch]

* fix(distributed): retry quarantined model cleanup

Stop quarantined replicas by exact process identity, retain failed cleanup as durable capped retries, and compare-and-delete only the claimed registry row. Process one sufficiently leased row at a time so multiple frontends cannot duplicate slow cleanup work.

Assisted-by: Codex:gpt-5

* fix(distributed): bind loads to config revisions

Assisted-by: Codex: GPT-5 [OpenAI Codex]

* fix(modeladmin): apply config revisions consistently

Route model edits, patches, state changes, deletion, and peer refreshes through the same revision lifecycle. Quarantine stale replicas before exact cleanup and report durable pending cleanup without failing successful config writes.

Assisted-by: Codex: GPT-5 [OpenAI Codex]

* feat(distributed): expose model config revision state

Document replica revision observability and durable cleanup behavior. Keep pending cleanup explicit in model mutation responses and verify endpoint contracts expose revision state without serialized load options.

Assisted-by: Codex:GPT-5 [OpenAI Codex]

* test(distributed): cover model revision convergence

Exercise cross-frontend quarantine, stale replay rejection, exact cleanup retry, worker re-registration, and current-generation replica convergence against the distributed PostgreSQL harness.

Assisted-by: Codex:gpt-5

* fix(distributed): pass config revision CI checks

Keep configured gallery sources out of authoritative runtime snapshots only after validating their real schema, and harden rollback snapshots against symlink races and non-regular files.

Assisted-by: Codex: GPT-5 [OpenAI Codex]

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-08-22 22:44:03 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 9d92139de4 feat(ui): edit scheduling rules in place (#11667)
* docs(ui): design scheduling rule editing

Document the approved in-place rule editing flow and scalable node-label reference for the scheduling view.

Assisted-by: Codex:gpt-5

* feat(ui): improve scheduling rule management

Add scalable node-label discovery and editable scheduling rules with responsive, accessible controls.

Assisted-by: Codex:gpt-5

* chore(ui): ratchet inline style baseline

Record the static inline style removed by the scheduling view enhancement.

Assisted-by: Codex:gpt-5

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-08-22 00:51:48 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto a0252ad6a1 fix(distributed): keep staging operations stable (#11663)
* docs: design configurable copy buffering

Document the context-aware copy buffer option and its validation plan.

Assisted-by: Codex:gpt-5

* docs: design durable distributed staging operations

Assisted-by: Codex:gpt-5

* fix(distributed): merge durable staging operations

Use active model load jobs as the durable operations baseline and overlay replica-local staging progress without duplication. Preserve tracker-only operations when the registry cannot be read.

Assisted-by: Codex:gpt-5

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-08-21 20:08:47 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 5429f569e0 fix(progress): stop status updates throttling downloads (#11661)
* feat(progress): aggregate and coalesce gallery downloads

Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(ui): show rolling transfer speed

Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(ui): preserve legacy import byte labels

Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-08-21 18:54:28 +02:00
Anupam Mediratta ff6043b811 fix: upgrade react-router to 7.18.2, 8.3.0 (GHSA-qwww-vcr4-c8h2) (#11644)
* fix: GHSA-qwww-vcr4-c8h2 security vulnerability

Automated dependency upgrade by OrbisAI Security

Signed-off-by: anupamme <mediratta@gmail.com>

* fix: upgrade react-router-dom to 7.18.2 to fully remediate GHSA-qwww-vcr4-c8h2

The prior fix pinned react-router@7.18.2 directly but left react-router-dom
at ^7.18.1, which bun resolved to 7.18.1. That package bundles its own
react-router@7.18.1 sub-dep, leaving the vulnerable version in bun.lock
via the react-router-dom/react-router scoped resolution. Pinning
react-router-dom to 7.18.2 and regenerating the lockfile removes all
7.18.1 resolutions.

Assisted-by: Claude Code:claude-sonnet-4-6
Signed-off-by: Anupam Mediratta <mediratta@gmail.com>

---------

Signed-off-by: anupamme <mediratta@gmail.com>
Signed-off-by: Anupam Mediratta <mediratta@gmail.com>
2026-08-21 08:41:23 +02:00
Anupam Mediratta 9ba4bbf9bb fix: upgrade ip-address to 10.3.1 (CVE-2026-69192) (#11632)
fix: CVE-2026-69192 security vulnerability

Automated dependency upgrade by OrbisAI Security

Signed-off-by: anupamme <mediratta@gmail.com>
2026-08-20 20:04:11 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 0ab632b6bd fix(auth): protect HTTP routes by default (#11602)
* fix(auth): default to protected HTTP routes

Use a method-aware registry for the small anonymous bootstrap surface.
Unknown routes now require credentials instead of inheriting fail-open
path classification.

Keep node self-service routes behind their registration-token middleware.
Global auth no longer rejects valid worker credentials first.

Assisted-by: Codex:gpt-5

* docs(auth): document public HTTP surface

Assisted-by: Codex:gpt-5

* test(auth): align route coverage with default denial

Assisted-by: Codex:gpt-5

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-08-19 20:49:19 +02:00
Ettore Di Giacinto 2383726d6d Revert "chore(tests): Avoid network, sleep and more during tests" (#11601)
Revert "chore(tests): Avoid network, sleep and more during tests (#11050)"

This reverts commit cb3bf7af3f.
2026-08-19 16:39:39 +02:00
Richard Palethorpeandlocalai-org-maint-bot cb3bf7af3f chore(tests): Avoid network, sleep and more during tests (#11050)
* test: make coverage failures observable

Keep per-root logs, reject concurrent coverage runs, and avoid relying on /bin/sleep in the worker timeout test.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test: parallelize coverage without remote fixtures

Assisted-by: Codex:gpt-5 [apply_patch] [exec_command]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test: add offline resource infrastructure

Introduce versioned resource manifests, a checksum-verified CAS preparer, offline test wrappers, and a guarded network transport. Replace live Hugging Face, GitHub, and OCI cases with deterministic fixtures and inject fixture metadata into importer discovery.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test: enforce offline resource replay

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test: harden offline resource refresh

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test: expose slow coverage waits

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test: eliminate avoidable wall-clock waits

Inject a clock into Hugging Face retry handling, reuse a process-scoped PostgreSQL container with per-spec schemas in the nodes suite, and poll local import jobs promptly.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test: remove repeated fixture startup waits

Share PostgreSQL fixtures across parallel endpoint and agent suite workers, and make the worker Free deadline injectable so the wedged-backend test does not spend five seconds on wall-clock time.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test: fix offline resource CI portability

Normalize Docker archive metadata before content addressing, derive archive checksums during explicit refreshes, make network lint portable to macOS, and prepare distributed images before running their offline suite.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* ci: cache Go modules before offline tests

Warm the complete module graph before the Linux and macOS test jobs enter offline replay mode, so tool dependencies such as Ginkgo are not fetched through the guarded proxy.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test: drop the static network lint in favour of real isolation

The offline test suite already prevents tests from reaching the network
twice over: run-test-linux-offline.sh puts the test process in a cgroup
and REJECTs egress outside the private ranges, and HardenedTransport
installs testnetwork.LocalGuard to refuse dials that resolve to a public
address. Both fail the test with a precise error at the moment of the
dial.

test-network-lint.sh added neither. Its diff stage defaulted to a HEAD
base, so on a clean checkout it compared the tree against itself and
inspected nothing; the branch's own commits were never examined. It only
produced output when an earlier job step dirtied the tree, and then it
matched a bare https?:// against whatever changed. make react-ui runs
npm install rather than npm ci, so CI rewrote
core/http/react-ui/package-lock.json and the lint reported an npm
registry URL as forbidden test network access:

  +      "resolved": "https://registry.npmjs.org/hono/-/hono-4.12.25.tgz",

Its fingerprint stage was self-defeating in a quieter way: hashing the
whole tree's network-mechanism inventory meant every rebase onto a master
that touched any _test.go needed a manual baseline bump, so the check
mostly caught its own staleness.

Remove the script, its make target and the two prerequisite edges, along
with the test-network: fixture markers that existed only to suppress it.
The isolation itself is untouched.

Assisted-by: Claude:claude-opus-5 [go vet]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* ci: keep hidden files in the offline test bundle artifact

Cherry-picked from 15a37b0ac on the remote branch. The offline bundle lives
under .cache/, which actions/upload-artifact skips by default, so the Linux
job packed an artifact missing the very file the next step restores.

The other half of 15a37b0ac moved test-network-lint out of the `test` and
`test-coverage` prerequisite lists into a recipe line, so parallel make could
not fingerprint the tree while generated fixtures were still changing. That
is dropped: the preceding commit removes the lint entirely, and the race it
worked around is one more reason a whole-tree fingerprint was the wrong
mechanism.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* refactor: share bounded exponential backoff

Use overflow-safe saturating arithmetic for retry delays across model import polling, downloads, registration, node operations, and model loading. Keep model import status checks responsive initially while capping their interval at 500ms.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* ci: mirror Jetson Python wheels

Keep the CUDA aarch64 wheel subset in GHCR and serve it as a local PEP 503 index during L4T backend builds, preserving last-known-good packages through upstream outages.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* docs(agents): index the Jetson wheels mirror

Mention the GHCR-hosted L4T wheel mirror in the CI caching guide summary so maintainers can find its outage and cache documentation.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* ci: add defensive build network proxy

Record build destinations and byte counts, retry observable idempotent HTTP downloads, and isolate explorer database tests that race under coverage.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(kokoros): implement updated backend trait

Return unimplemented for image upscaling, matching the backend's other unsupported modalities after the protobuf API update.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(ci): clear recovered proxy errors

Do not mark a request failed when a later safe retry succeeds.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* ci: require HTTPS build interception

Inject a short-lived proxy CA into BuildKit and Dockerfile RUN steps, reject plain HTTP and opaque tunnels, and retain method/status/byte telemetry for verified HTTPS traffic.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(ci): preserve system trust in unproxied builds

Mount the generated interception CA at a dedicated secret path and add it to the trust bundle only in proxy-aware dependency stages. This prevents optional secret mounts from masking the system CA bundle in ordinary backend test builds.

Install the requested Go toolchain before starting the proxy and satisfy cleanup error checks found by CI lint.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(ci): persist build proxy trust

Install the generated proxy CA through the system-managed local certificate directory so ca-certificates upgrades retain it. Avoid turning canceled matrix jobs into proxy cleanup failures.

Assisted-by: Codex:gpt-5

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(ci): trust proxy in nested build scripts

Install the build proxy CA before nested source fetches, route the DS4 package setup through the HTTPS mirror helper, and avoid repeated OCI setup in gallery behavior tests.

Assisted-by: Codex:gpt-5

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(ci): use HTTPS apt sources for Bonsai

Rewrite ARM64 package sources before installing GCC and check gallery fixture cleanup errors so the optimized tests satisfy errcheck.

Assisted-by: Codex:gpt-5

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(privacy-filter): trust build proxy CA

Install the mounted build proxy certificate before privacy-filter's make target fetches its HTTPS sources, for both source and prebuilt builder paths.\n\nAssisted-by: Codex:gpt-5

Signed-off-by: Richard Palethorpe <io@richiejp.com>

* test: fail on hidden offline egress

Count cgroup-scoped firewall rejects and fail the offline test harness with bounded aggregate diagnostics. Inject the gen-audio GGUF probe so fixture-backed importer tests do not attempt real network access.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(ci): preserve system CA trust

Build a combined runner certificate bundle instead of replacing public roots with the generated proxy CA. Centralize additive container installation in the shared proxy CA helper.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
2026-08-19 10:59:31 +02:00
localai-org-maint-botandlocalai-org-maint-bot 4cad809003 fix(ci): test stale chunks in split bundle (#11595)
The V8 coverage build inlines every dynamic import, so the stale chunk tests cannot intercept a page chunk. Run those tests against the normal code-split bundle and exclude them from the inlined coverage pass.

Assisted-by: Codex:gpt-5

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
2026-08-19 08:45:59 +02:00
Ettore Di Giacinto b806b1fec3 fix(ui): reload once when a page chunk 404s
A deploy replaces the whole content-hashed asset set at once. A tab
holding an older index.html, or one whose request lands on a replica
that the rollout has not swapped yet, asks for a page chunk the server
no longer has. The dynamic import rejects and React Router's default
error boundary replaces the app with "Unexpected Application Error!"
until someone reloads by hand.

The router now reloads the page itself when a chunk fails to load.
index.html is served no-cache, so the reload lands on a self-consistent
asset set. A timestamp in sessionStorage bounds this to one reload per
10 seconds, so a chunk that is genuinely gone reaches the error boundary
instead of looping forever.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude Code:claude-opus-5[1m]
2026-08-18 19:51:15 +00:00
localai-org-maint-botandlocalai-org-maint-bot 0761bd02c7 feat(chat): add end-to-end context compression (#11556)
* feat(config): add context compression policy

Define the opt-in model configuration contract before the chat middleware consumes it. Document each policy field so later request handling does not invent a second schema.\n\nRefs #9534\n\nAssisted-by: Codex:gpt-5

* fix(config): register compression fields

The model editor metadata gate rejects new config fields without descriptions and suitable controls. Register the compression policy so operators can edit its six fields safely.

Assisted-by: Codex:gpt-5 [monitoring-prs]

* feat(chat): compress long contexts

Long conversations currently fail once they reach the model context window. The opt-in policy now summarizes complete older turns before primary inference and preserves the newest tool chains.

Both OpenAI and MCP chat routes share the same transformation. Usage metadata and metrics expose each compression event.

Refs #9534

Assisted-by: Codex:gpt-5

---------

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
2026-08-18 11:31:03 +00:00
Nicholas Ciechanowski cf93c04878 fix: tts text wrap (#11576)
Signed-off-by: Nicholas Ciechanowski <nicholas@ciech.anow.ski>
2026-08-18 12:57:55 +02:00
Richard Palethorpe d10374f849 feat(router): make KNN a first-class classifier with a persisted, curated corpus (#10652)
* feat(router): make KNN a first-class classifier with a persisted, curated corpus

Add `classifier: knn` — similarity-weighted voting over labelled
example prompts. Unlike score/colbert it needs no classifier model:
label knowledge lives in a corpus seeded and curated through the
admin API, so routing decisions are deterministic, auditable, and
grounded in graded experience rather than a model's opinion.

Epistemic gate: corpus entries below knn.similarity_threshold cannot
vote; when none clears it the classifier activates no labels and the
router uses the fallback — a prompt unlike all labelled experience is
treated as undecidable, not guessed. Decisions record
nearest_similarity (also on fallback rows) so admins can see how far
the nearest labelled experience was; the Routing tab explains
out-of-corpus fallbacks and shows per-label corpus counts.

Persistence: one JSONL file per router under
<data path>/router-corpus (text, labels, vector, embedder
fingerprint). The file is the source of truth; the local-store index
is rebuilt from it at classifier build time and stays a pure
in-memory index. Entries recorded under a different embedding model
re-embed on load. Also corrects the docs' false claim that
local-store collections persist — the embedding cache never survived
restarts (and still doesn't); the corpus does.

Corpus input is API-only by design (entries may contain example user
content): POST /api/router/{name}/corpus seeds (labels validated
against declared policies, embedded server-side, indexed
immediately), GET .../corpus/stats inspects — label counts only,
entry texts are never returned by any surface — DELETE .../corpus
wipes. Admin-gated like the sibling router endpoints, and exposed as
MCP tools (seed_router_corpus / get_router_corpus_stats /
clear_router_corpus) in both the httpapi and inproc clients with
coverage-test route mappings.

Plumbing: VectorStore gains SearchK (top-K was hardcoded to 1);
local-store gets InsertBatch/Delete as optional fast paths;
RouterConfig gains a knn block (embedding_model, k,
similarity_threshold, vote_threshold, store_name) with meta-registry
fields; the classifier dropdown now offers knn and the
previously-missing colbert; embedding_cache is ignored (with a
warning) for knn — it IS an embedding-KNN lookup; the stale
/api/instructions intelligent-routing entry is rewritten (it
described a classifier that no longer exists); swagger regenerated.

Tests: KNN vote/gate specs with hand-computed vote shares, corpus
manager suite (restart reload without re-embedding, fingerprint
re-embed, dedupe, hostile store names), middleware specs (corpus
routing, gate fallback, config validation, cache-wrap refusal),
corpus endpoint specs pinning the texts-never-returned contract, MCP
catalog + route-mapping gates, and a Playwright spec for corpus
stats and the out-of-corpus decision detail.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(router): name consulted corpus neighbours in knn decisions

Every knn decision (decision log rows and the /api/router/decide
response) now carries neighbors: the K retrieved corpus entries by
descending similarity - including ones below the epistemic gate, which
is what makes fallback decisions diagnosable - each as {id, similarity,
labels}. The id is the entry's content hash (first 8 bytes of the
SHA-256 of its text, hex): stable across reseeds and re-embeds, and
text-free, so an external platform that seeded the corpus can recompute
text->id on its own copy and bucket decisions by corpus region (per-
region reliability accounting) without corpus text ever leaving the
server. A corrupt index payload surfaces as an id-less neighbour at a
real similarity instead of disappearing.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* refactor(router): deduplicate knn plumbing and cut corpus hot-path waste

Post-review cleanup of the knn-first-class-router branch; no behaviour
changes on the API surface.

Reuse/altitude:
- RouterKNNConfig.ResolvedStoreName is now the single source of the
  router-corpus-<name> default (was hand-derived in four files).
- corpus.ResolveKNNRouter + corpus.Seed carry the shared model
  resolution and seed validation; the REST endpoints and the assistant
  MCP client are thin transport adapters over them, with sentinel
  errors mapped to HTTP statuses at the echo boundary.
- middleware.NewClassifierDeps assembles the classifier dependency set
  once for all five entry points (OpenAI, Anthropic, realtime, decide,
  corpus) instead of five hand-copied literals.
- router.AllClassifiers feeds both the status endpoint and the
  unknown-classifier error, ending the classifier-list drift.
- Per-classifier requirements moved out of validateRouterPolicies into
  their buildClassifier arms; the knn arm owns its embedding_cache
  opt-out instead of a name-check in the shared wrap tail.
- adminOnly replaces four inline copies of the admin gate in the
  middleware routes.
- localVectorStore.Search delegates to SearchK (identical traces).

Efficiency:
- Manager.Add embeds outside the manager mutex and appends to the
  JSONL file (O(new) instead of O(corpus) rewrite); a torn tail from a
  crash mid-append is tolerated on read and repaired on next write.
- Stats memoises per store keyed on the file's stat fingerprint and no
  longer takes the manager mutex, so the 5s status poll stops parsing
  vector-laden JSONL and stops blocking behind seeds.
- KNN Classify decodes each neighbour payload once (was twice) and
  builds refs and votes in a single pass with one fallback return.
- Corpus file writes fsync before rename/close.
- The corpus manager is built eagerly in newApplication (sync.Once
  dropped); test helper dead branch removed.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(router): bind knn corpus vectors to an embedder fingerprint and fail closed on mismatch

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* chore(mcp): align corpus tool prompts and the mutating-tool safety list

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(proto,backend): report embedding shape from the llama-cpp backend

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(embeddings): Go-side pooling — mean/last/decayed_mean with half-life

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* feat(embeddings): accept chat messages[] and per-request pooling on /v1/embeddings

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* chore(middleware): name the failing fields when post-merge validation 400s

An intermittent post-merge validation failure surfaced as an opaque 400
during integration (pooling scheme mismatch that no client had sent).
Log the model, the request's pooling override, and the merged config's
pooling fields at the failure point so the next occurrence identifies
whether the request or the stored config carried the bad value.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(embeddings): scheme override must not inherit the config's half-life

A model config defaulting to decayed_mean pooling carries
pooling_half_life_tokens; a request overriding the scheme to mean/last
without its own half-life inherited that value, and post-merge
validation rejected the pair the server itself had assembled. Zero the
inherited half-life when the overridden scheme is not decayed_mean; a
request that explicitly pairs a half-life with a non-decayed scheme
still 400s.

Assisted-by: Claude:claude-fable-5 [Claude Code]
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix embedding pooling validation and router bounds

Declare backend embedding layouts and reject incompatible pooling modes. Reset local-store dimensions after a full clear, validate KNN thresholds, and add real backend and store integration coverage.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* ci: run local-store integration tests

Build and install the local-store backend in the Linux test job, then run the existing store integration suite so new specs are discovered automatically.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
2026-08-18 09:37:43 +02:00
localai-org-maint-botandlocalai-org-maint-bot a7bce6a128 fix(audio): reject incompatible transform streams (#11565)
The transform WebSocket accepted any model and opened its frame-based RPC. Any-to-any models use a different stream contract, so liquid-audio failed with an unimplemented RPC after the handshake.

Reject incompatible model use cases before loading the backend. Direct realtime-audio callers to the OpenAI Realtime API.

Assisted-by: Codex:gpt-5

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
2026-08-18 09:03:16 +02:00
Richard Palethorpe 799cc9f211 feat: bound global admission and expose running backend traces (#11560)
feat: bound backend admission and expose running traces

Add process-wide backend execution admission without blocking UI or administrative HTTP work. Represent backend operations while they are in flight, surface running traces with immediate log links, and tie streaming admission leases to the gRPC receive lifecycle.

Assisted-by: OpenAI Codex: GPT-5

Signed-off-by: Richard Palethorpe <io@richiejp.com>
2026-08-18 08:56:59 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 0aaff91ebd feat(ui): unify model and backend lifecycle (#11548)
* feat(ui): add installed model lifecycle

Models now owns catalog exploration and installed runtime controls under one canonical route. URL-owned state keeps lifecycle context recoverable through links and browser history.

Assisted-by: Codex:gpt-5 Playwright

* feat(ui): add installed backend lifecycle

Backends split discovery from backend-binary management. The canonical
page now keeps both lifecycle views under one URL-backed shell while it
preserves target-node placement.

Assisted-by: Codex:gpt-5 Playwright

* fix(ui): repair lifecycle state updates

Installed models lost distributed refreshes and kept a deleted selection. Backend searches also stopped tracking URL changes, while batch upgrades stopped after their first error.

Preserve background refreshes and finish each requested batch action. Drive catalog results from URL-backed state without losing full metadata.

Assisted-by: Codex:gpt-5 [Playwright]

* feat(ui): make resource pages canonical

Replace Host navigation with canonical Models and Backends lifecycle routes, preserve legacy management URLs, and surface shared host capacity on the Operate overview.

Assisted-by: Codex:gpt-5 [Playwright]

* feat(ui): complete canonical resource lifecycle

Finish the responsive list-to-detail behavior, remove the retired Host implementation, and keep Explore focused on discovery while Installed owns destructive actions. Update regression coverage, localization, documentation, and development binding for the canonical resource pages.

Assisted-by: Codex:gpt-5 [Playwright]

* docs(ui): record the UI design context

Record the approved users, brand character, and design principles so
future interface work uses the same product direction. Index the context
from the repository's agent instructions.

Assisted-by: Codex:gpt-5

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-08-16 11:57:31 +02:00
88edd7fc7f fix(distributed): run cold model loads as durable jobs instead of holding the advisory lock (#11514)
* fix(advisorylock): set statement_timeout alongside lock_timeout

WithLockCtx already overrides a deployment-wide lock_timeout on its
dedicated connection so a blocking pg_advisory_lock() waits its turn
instead of failing with 55P03. statement_timeout aborts that exact same
statement independently, with SQLSTATE 57014, and was not overridden.

Production roles commonly carry statement_timeout=60s. Any guarded
section longer than that (a cold model load stages for tens of minutes)
therefore killed every concurrent waiter:

  advisorylock: acquiring lock 9003261067483446873: ERROR: canceling
  statement due to statement timeout (SQLSTATE 57014)

Derive it from the same context budget as lock_timeout, with a matching
RESET so the pooled connection is returned clean.

Assisted-by: Claude Opus 5 [claude-code]

* feat(distributed): add ModelLoadJob, the durable cold-load record

A cold load in distributed mode is a long-running background job, but it
was modelled as a synchronous side effect of an inference request: the
whole of it (backend install, multi-GB staging, checkpoint load) ran
inside the per-model advisory lock. Loading a 35.7 GB GGUF held that lock
for ~20 minutes, so every concurrent request for the same model blocked
on pg_advisory_lock and died at the role's 60s statement_timeout.

Introduce the row that lets the lock shrink to a decision. Exactly one
ModelLoadJob may be active per tracking key; that uniqueness — not the
lifetime of a lock — is what de-duplicates concurrent loaders across
replicas. ClaimLoadJob does its read-then-write under the advisory lock
and nothing else: no network, file or gRPC I/O inside the guarded
section, so a claim costs milliseconds no matter how long the resulting
load takes.

LastProgress is a heartbeat rather than a byte counter. A checkpoint load
legitimately moves zero bytes for many minutes, so a reaper keyed on byte
movement would reclaim a healthy job mid-load; byte progress stays the
concern of load_deadline.go. A job whose heartbeat stops for longer than
the orphan window is reclaimable, so a replica killed mid-load cannot
wedge a model permanently.

Failed jobs keep their row for a short grace so an immediately-following
request reports the real cause instead of silently starting a fresh load
of a model that just failed.

No caller yet — the router moves onto this in the next commit.

Assisted-by: Claude Opus 5 [claude-code]

* refactor(distributed): run cold loads as jobs, outside the advisory lock

Route wrapped the entire cold load — node selection, backend install,
multi-GB staging and the remote LoadModel — in the per-model advisory
lock. The lock's job is to de-duplicate concurrent loaders, a decision
that takes milliseconds; holding it for the tens of minutes the resulting
work takes is what turned a dedup mechanism into a cluster-wide outage
for that model.

Split it into a claim and a run. The claim is the only thing left inside
the lock. The run is a background job owned by the claiming replica and
bounded by the same progress-extended deadline as before; every other
request for that model — local or on another replica — attaches as a
waiter and is served the moment the model is ready, with no duplicate
load and no lock contention.

Waiters share one broadcast rather than an ordered queue: they all want
the identical outcome, so ordering them would add fairness machinery that
changes no result. The local channel wakes same-replica waiters instantly
and a 2s DB poll is the authority, because a waiter on another replica
has no channel to close. On wake a waiter re-runs the warm path rather
than trusting the signal — the model may have been evicted in between.

A waiter whose client disconnects returns immediately and the job keeps
running; it belongs to the job record, not to the request. A failure is
recorded on the row so every waiter reports the real cause, and the row
survives briefly so the next request does not read "no job" as "not
loading" and start a duplicate load of a model that just failed.

The runner heartbeats the row on a fixed interval whether or not bytes
are moving, which is what keeps a legitimately silent checkpoint load
from being reclaimed as an orphan. Phase (installing/staging/loading) and
placement ride to the heartbeat on the context, the same seam
load_deadline.go already uses, so single-host paths are untouched.

Non-distributed mode (no DB) keeps the inline load exactly as it was.

Assisted-by: Claude Opus 5 [claude-code]

* feat(distributed): bound the wait for a loading model and answer with progress

A request whose model is cold-loading now attaches to the running job and
is served the moment the model is ready. That wait has to be bounded: a
held HTTP request cannot survive real infrastructure, and an ingress or LB
idle timeout kills a twenty-minute request regardless of what LocalAI
does.

New LOCALAI_MODEL_LOAD_WAIT (default 60s) bounds the CALLER, never the
load — the job keeps running either way. On expiry the request gets 503
with Retry-After and a structured body naming the model, the node, the
phase, byte progress and an ETA. The `error` envelope keeps OpenAI
clients working; `loading` is additive so they ignore it.

The ETA comes from the job's own observed rate and is omitted rather than
guessed until enough bytes have moved for that rate to mean anything: a
confidently wrong ETA on a twenty-minute wait is worse than none.
Retry-After is that ETA when known, clamped to [5s, 300s], and the wait
budget otherwise.

LOCALAI_MODEL_LOAD_WAIT=0 waits unbounded, for deployments with no proxy
in front. Zero in the config struct still means "unset, use the default",
so the CLI records the operator's zero as ModelLoadWaitUnbounded rather
than losing the distinction.

The distributed branch of ModelLoader.loadModel wrapped the router's
error with %s, which flattened it to a string. Use %w: the typed error is
what the HTTP layer keys the 503 off.

Assisted-by: Claude Opus 5 [claude-code]

* feat(api): add GET /api/models/{id}/load-status

A client that receives 503 while a model stages onto a worker needs
somewhere to poll. This returns the same `loading` object the 503 carries
— phase, node, byte progress and ETA — or 404 when no load is running.

Read-only and observability-shaped, so it is deliberately neither
admin-gated nor feature-gated: it explains a 503 the caller just
received, and hiding that behind a per-modality feature would make the
explanation for a failed image request depend on chat permissions. It
also gets no MCP tool, since there is nothing here an admin would manage
conversationally.

Registered on the surfaces from .agents/api-endpoints-and-auth.md: the
swagger block (existing `models` tag, so /api/instructions needs no new
area), the endpoint discovery maps in RegisterLocalAIRoutes, regenerated
swagger, and the distributed-mode docs page. No FLAG_* usecase is
involved, so capabilities.js is unchanged.

Assisted-by: Claude Opus 5 [claude-code]

* feat(ui): show cold-load progress in Chat and retry when the model is ready

A chat request for a model that is still staging onto a worker now gets a
503 carrying live progress instead of an error. Render it: the composer
shows the phase (installing / staging / loading), the node, the percent
and the ETA, then polls load-status and re-sends the request the moment
the model is ready.

Reuses the staging progress idiom the page already had rather than
inventing a second one — the two sources are folded into one
loadProgress, with the load job winning because it is authoritative
across frontend replicas and knows the phase, where the staging operation
only knows about a byte transfer this replica happens to be performing.

Waiting is bounded (three send attempts, ~30 min of polling each), so a
load that never finishes still surfaces as an error rather than as a
spinner nobody questions. An aborted generation stops the polling too.

Assisted-by: Claude Opus 5 [claude-code]

* fix(distributed): check warm-path cleanup errors

The router moved legacy cleanup calls onto newly linted lines. Report
cleanup failures while preserving the fallback to a cold load.

Assisted-by: Codex:gpt-5 [golangci-lint]

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
2026-08-15 13:20:11 +02:00
localai-org-maint-botandlocalai-org-maint-bot 95653f221e fix(ui): keep agent import action visible (#11488)
* fix(ui): keep agent import action visible

The header hid its full import label after the agent list became non-empty. Hide only the nested file input so users can import more agents.

Assisted-by: Codex:gpt-5

* test(ui): match the agent import label

The Agents page renders the action as Import.

The test searched for Import Agent, so it failed before checking visibility.

Mock the observables request to remove backend timing from the fixture.

Assisted-by: Codex:gpt-5

---------

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
2026-08-14 15:06:51 +02:00
Richard Palethorpe 5c63969760 fix: Show MCP connection errors in the UI (#11495)
* fix(mcp): surface configured server failures

Keep model-configured MCP servers visible when discovery or connection setup fails, propagate status through distributed discovery, and let the Chat UI show actionable errors while retrying unavailable servers.

Add model-editor metadata for remote and stdio configuration and document the expected format, deployment networking boundary, and alternate MCP scopes.

Assisted-by: Codex:gpt-5 Ordino golangci-lint
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* build(compose): match CUDA development image

Configure the API image with the cublas, CUDA 13, auth-tagged build settings used by the local development Makefile invocation, including the 24-way Docker build.

Assisted-by: Codex:gpt-5 Ordino
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* revert: keep host build settings out of compose

The CUDA development deployment is managed from ~/docker/localai, not the repository example Compose file. Restore the generic example and keep machine-specific build settings in the host deployment.

Assisted-by: Codex:gpt-5 Ordino
Signed-off-by: Richard Palethorpe <io@richiejp.com>

* fix(docker): exclude local agent artifacts

Keep Claude worktrees and locally installed verification tools out of the Docker build context. These host-only directories added roughly 1.9 GB to every root image build.

Assisted-by: Codex:gpt-5 Ordino
Signed-off-by: Richard Palethorpe <io@richiejp.com>

---------

Signed-off-by: Richard Palethorpe <io@richiejp.com>
2026-08-13 22:25:58 +02:00
Dedy F. Setyawan 9fd7ea7e93 i18n(id): translate admin, media, and nav UI strings to Indonesian (#11493)
Signed-off-by: Dedy F. Setyawan <dedyfajars@gmail.com>
2026-08-13 16:27:45 +02:00
Dimitris Karakasilis 3636fcbd38 fix(realtime): keep the ICE interface allow-list working with a fixed UDP port (#11466)
LOCALAI_WEBRTC_ICE_INTERFACES was silently ignored whenever
LOCALAI_WEBRTC_UDP_PORT was set. Every interface was gathered regardless
of the allow-list, so a browser was handed the docker0/veth addresses the
setting exists to suppress, and the connection established on a good pair
and then dropped when consent checks failed on the unreachable ones.

Two things combine to cause it. A mux built over a wildcard socket makes
pion derive its host candidates by enumerating interfaces itself, with a
nil filter and loopback included. Independently, the muxed gathering path
in pion/ice never consults SetInterfaceFilter, so setting it has no effect
there either.

Bind one socket per admitted interface address via NewMultiUDPMuxFromPort,
which takes the filter, instead of one wildcard socket. All the sockets
share the same port, so the firewall requirement is still a single rule.
Networks are pinned to UDP4 to match the socket family this replaces.

An allow-list that matches no address on the host now reports the
misconfiguration rather than binding nothing and leaving signaling to
succeed while no candidate is ever advertised.

Two tests: one asserts an unmatched allow-list is an error, and one gathers
against a real peer connection and asserts no address outside the allowed
interface appears (skipped on single-interface hosts).

Assisted-by: Claude:claude-opus-5 go vet gofmt

Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>
2026-08-11 18:37:49 +02:00
mudler's LocalAI [bot]andEttore Di Giacinto 45cb3983ee fix(ui): unmerge the class strings that left buttons in browser chrome (#11462)
Eight header controls across seven pages had two or three elements' classes
collapsed into one string. The wrapper or the icon ended up wearing the
button classes, and the buttons themselves were left with no class at all,
so they rendered in the browser's own chrome. Reported on Agent Jobs; the
grep found the rest.

`fas` does not draw anything by itself: it sets
`font-family: "Font Awesome 6 Free"` and weight 900 on whatever carries it,
and the `fa-*` class supplies the glyph via ::before. So

    <button className="btn btn-primary fas fa-plus">

renders its own label "New Task" in the icon font, and

    <div className="hstack btn btn-primary btn-sm fas fa-edit btn-secondary fa-arrow-left">
      <button>Edit</button>
      <button>Back</button>
    </div>

styles the flex wrapper as a button that is both primary and secondary,
points two glyphs at one ::before, and leaves both real buttons bare.

Fixed, all of them keeping the correct `<i>` child they already had:

  - AgentJobs, AgentTaskDetails (x2), AgentCreate - icon classes off the
    button.
  - AgentTaskDetails, AgentJobDetails - wrapper back to plain `hstack`, and
    the two buttons inside each get the variants the wrapper had been
    holding. Back is secondary and leads, Edit/Cancel is the emphatic one
    and trails, matching every other detail header.
  - VoiceLibrary, VoiceProfileCreate - the title `<i>` had swallowed the
    action link's classes, so "Create voice" and "Back to library" were
    unstyled anchors. Back was also drawing a "+" because it had inherited
    fa-plus while its own fa-arrow-left sat up in the title.
  - P2P - a stray fa-circle-info on the title icon.

The ninth instance was ImportModel, where this class of bug was first
found. #11461 rewrote that file and landed first, so nothing is left to fix
there.

Guarded by e2e/class-hygiene.spec.js, which reads the source rather than
walking routes: several of these pages need agent or voice data before they
render a header, so a route walk would skip exactly the pages that had the
bug. It fails on an icon-font class outside an `<i>`/`<span>`, on two glyphs
or two button variants on one element, and on a layout wrapper that is also
a button. Font Awesome modifiers (fa-spin, fa-fw, sizes) are excluded, so
the `fa-spinner fa-spin` idiom stays legal.

e2e: 428 passed.


Assisted-by: Claude Code:claude-opus-5[1m] [Read] [Edit] [Bash] [Playwright]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-08-11 12:07:42 +02:00