mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 09:57:57 -04:00
feat/vllm-cpp-engine-args
7295 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
walcz-de
|
6d1bbb74c4 |
fix(backend/python): don't await sync servicer behaviors in AsyncModelIdentityInterceptor (#10980)
* fix(backend/python): don't await sync servicer behaviors in AsyncModelIdentityInterceptor The model-identity interceptor (added for #10952) is installed on every Python backend's gRPC server. Its grpc.aio variant invokes the wrapped servicer behavior itself and awaits the result unconditionally: result = await original(request, context) # LoadModel return await original_unary(request, context) # guarded RPCs async for response in original_stream(request, context): # streaming But a backend's servicer methods may be plain sync functions. The transformers backend, for one, defines `def LoadModel` and `def Embedding` (not `async def`). grpc.aio's own dispatch adapts both shapes, but this interceptor calls the behavior directly and bypasses that. For a sync method `original(...)` returns a message object, not a coroutine, so the `await` raises: TypeError: object Result can't be used in 'await' expression The model loads, then the LoadModel RPC dies on return; the guarded sync Embedding fails the same way. It happens on every platform, not just one backend build. CI never caught it because AsyncModelIdentityInterceptor had no behavioral test -- only an "is it installed" assertion. Fix: await only when the behavior actually returned an awaitable (inspect.isawaitable), mirroring grpc.aio's own sync/async adaptation. The streaming guard iterates a sync generator with `for` and an async one with `async for`. Adds async-path coverage to model_identity_test.py exercising both sync and async LoadModel / guarded-unary / streaming behaviors. The sync cases fail on the current code with the TypeError above and pass with this fix. Signed-off-by: stefanwalcz <stefan.walcz@walcz.de> * fix(backend/python): dispatch sync servicer behaviors off the event loop Addresses review feedback: awaiting only awaitable results removed the TypeError, but still ran a sync LoadModel/Embedding -- and stepped a sync stream via next() -- on the asyncio event-loop thread, so a slow load/inference/stream could freeze all aio RPC handling. Route sync behavior through run_in_executor (a worker thread) while awaiting native async behavior directly. A callable wrapper that returns an awaitable is run in the thread and its awaitable awaited back on the loop. Sync streaming pulls each item via the executor with a done sentinel, so StopIteration cannot escape through a Future. Adds regression tests that record the handler thread id and assert it differs from the event-loop thread, for LoadModel, a guarded unary RPC and a sync stream. Signed-off-by: stefanwalcz <stefan.walcz@walcz.de> --------- Signed-off-by: stefanwalcz <stefan.walcz@walcz.de> |
||
|
dependabot[bot]
|
f92410b20b |
chore(deps): bump fast-uri from 3.1.2 to 3.1.4 in /core/http/react-ui in the npm_and_yarn group across 1 directory (#11043)
chore(deps): bump fast-uri Bumps the npm_and_yarn group with 1 update in the /core/http/react-ui directory: [fast-uri](https://github.com/fastify/fast-uri). Updates `fast-uri` from 3.1.2 to 3.1.4 - [Release notes](https://github.com/fastify/fast-uri/releases) - [Commits](https://github.com/fastify/fast-uri/compare/v3.1.2...v3.1.4) --- updated-dependencies: - dependency-name: fast-uri dependency-version: 3.1.4 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
54f531f452 |
fix(mcp): bound MCP session connect so an unreachable server can't hang the widget (#10880) (#10884)
Establishing an MCP session held the session-cache mutex across client.Connect with no per-connect timeout. An unreachable remote server (bounded only by the 360s httpClient timeout) or a stdio server whose initialize handshake never completes therefore blocked the caller and, because the mutex was held, every other MCP request for that model too. In the UI this shows up as the MCP "Servers" widget spinning forever. It is most visible for cloud-proxy models: their chat path bails out before the MCP tool block, so it never warms the session cache in the background. The widget's /v1/mcp/servers/<model> call is then the first and only code that connects synchronously, in the request foreground. The session, once established, stays bound to the shared context (it is cancelled later via the cached cancel func on eviction/shutdown), so we can't pass a WithTimeout context to Connect: firing the timeout would tear a healthy session down, and cancelling the shared context would also kill sibling servers that already connected. Instead connectMCP runs Connect on the shared context in a goroutine and stops waiting after the discovery timeout, returning an error for that one server without disturbing the others. A stalled goroutine is reaped when the model's sessions are cancelled. Applied to both SessionsFromMCPConfig and NamedSessionsFromMCPConfig. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
01fca9c9b2 |
fix(distributed): scale the remote model-load deadline with checkpoint size (#11030)
The gRPC deadline for the remote LoadModel call was a fixed 5m. It starts
only after the backend install and file staging have completed, so it
covers the worker's checkpoint read and pipeline init alone - work whose
duration is proportional to the bytes on disk. A fixed value is therefore
a model-size cliff, not a timeout.
Measured in production: a 70 GB video checkpoint (longcat-video-avatar-1.5)
on an NVIDIA Jetson Thor worker failed reproducibly with
"rpc error: code = DeadlineExceeded" after 953.5s of wall clock. Backend
install plus staging consumed ~11m, then LoadModel got its 5m and expired.
The load never had a chance, and the operator saw only a generic
DeadlineExceeded with no hint that a config value was the cause.
Raising the constant does not fix this. It moves the cliff to the next
larger model - the cluster has to support 600 GB checkpoints - and it makes
a genuinely wedged SMALL model hang for the whole inflated duration before
anyone notices, which is a real regression in failure latency.
So derive the budget from the checkpoint size instead:
budget = 5m + 20s/GiB, capped at 6h
2 GiB -> 5m40s, 70 GiB -> 28m20s, 600 GiB -> 3h25m. The per-GiB rate is
deliberately pessimistic (~54 MB/s of weight read) because the errors are
not symmetric: too long costs only failure latency on a load that was going
to fail anyway, too short is a guaranteed false failure on a healthy load.
The size is measured from the frontend's local model files, over the same
path set stageModelFiles uploads. When those files are not present locally -
a backend handed a bare HuggingFace repo id fetches its own weights on the
worker - there is nothing to measure and the budget stays at today's 5m.
An explicit LOCALAI_NATS_MODEL_LOAD_TIMEOUT still wins outright, in both
directions: a shorter override is honoured, so an operator who wants fast
failure is not silently extended by the heuristic.
The cold-load hold needed widening to match. It extends on staging progress,
but LoadModel reports none, so once the last byte lands the hold expires a
stall window later and would cancel a load still well inside its own budget.
scheduleAndLoad now extends the hold by the load budget plus the staging
margin as it enters the load phase; ModelLoadCeilingFor stays the hold's
starting budget rather than its maximum.
Finally, a deadline that does expire now names the budget, the checkpoint
size it was derived from, and the knob that overrides it, instead of
surfacing a bare "context deadline exceeded".
Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
|
||
|
Tai An
|
d7020708f2 |
fix(completions): reject empty PromptStrings in streaming to avoid index-out-of-range panic (#11028)
* fix(completions): reject empty PromptStrings in streaming to avoid index-out-of-range panic The streaming branch of CompletionEndpoint only guarded len(config.PromptStrings) > 1 before unconditionally reading config.PromptStrings[0]. A completion request whose prompt field is an empty array, an array of non-strings, or omitted leaves PromptStrings with length 0, so PromptStrings[0] panics with index out of range and crashes the handler goroutine. Guard for exactly one prompt string instead, returning a clean error for the 0-length case as well as the pre-existing multi-prompt case. Signed-off-by: Tai An <antai12232931@outlook.com> * fix(completions): return 400 for malformed streaming prompt Reject streaming completion requests whose prompt does not resolve to exactly one string (omitted prompt, empty array, or a multi-element array) with an HTTP 400 before writing any SSE headers, instead of returning a plain error that Echo surfaces as a 500. Extract the guard into validateStreamingPromptStrings and cover the three reported payloads with a regression test. Fixes #11021 Signed-off-by: Tai An <antai12232931@outlook.com> --------- Signed-off-by: Tai An <antai12232931@outlook.com> |
||
|
Tai An
|
bf19758e05 |
fix(ollama): cap num_ctx so it cannot wrap negative when cast to int32 (#11032)
* fix(ollama): cap num_ctx so it cannot wrap negative when cast to int32 applyOllamaOptions copied a client-supplied options.num_ctx straight into cfg.ContextSize with only a > 0 check. That value is later cast to int32 before it reaches the backend (core/backend/options.go), so a num_ctx above math.MaxInt32 silently wrapped into a negative context size that was then sent to the LoadModel gRPC call. Both /api/chat and /api/generate share applyOllamaOptions, so both endpoints were affected. Cap num_ctx at math.MaxInt32 so the later cast stays positive, and add internal regression coverage for the overflow, in-range, and unset cases. num_ctx remains an intentional user override, so this does not re-impose the hardware-aware auto context clamp; that policy choice is left to maintainers. Fixes #11022 Signed-off-by: Tai An <antai12232931@outlook.com> * fix(ollama): clamp num_ctx to model context ceiling, not just int32 Per review on #11032: capping only at math.MaxInt32 still let an unauthenticated request replace the hardware/model-derived context limit with ~2.1B tokens, so a real backend could attempt a catastrophic KV-cache allocation. Treat any existing positive cfg.ContextSize as the server ceiling and clamp num_ctx down to it (smaller values still honored), while retaining the int32-safe bound when no smaller ceiling exists. Shared by /api/chat and /api/generate via applyOllamaOptions. Add regression coverage proving num_ctx=2,000,000,000 cannot replace an existing 4096/8192 ceiling. Signed-off-by: Tai An <antai12232931@outlook.com> --------- Signed-off-by: Tai An <antai12232931@outlook.com> |
||
|
mudler's LocalAI [bot]
|
48b7d6d8fd |
docs: ⬆️ update docs version mudler/LocalAI (#11033)
⬆️ Update docs version mudler/LocalAI Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
3154bec357 |
chore(model-gallery): ⬆️ update checksum (#11036)
⬆️ Checksum updates in gallery/index.yaml Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-bot
|
47c0e06198 |
fix(ci): authenticate the nightly dependency-bump API calls (#11042)
The "Bump Backend dependencies" workflow has failed every night for the last two weeks. #11012 fixed one cause (repos renamed under localai-org); what is left is rate limiting. bump_deps.sh fans out to ~25 parallel matrix jobs that each query api.github.com anonymously. Anonymous calls are capped at 60/hour per source IP and GitHub-hosted runners egress through shared NAT addresses, so a random handful of jobs draw HTTP 403 and die at curl exit 22 with an empty response. Last night that hit ggml-org/whisper.cpp and mudler/depth-anything.cpp -- both public and resolvable, nothing wrong with either pin. Route every bump script through a shared gh_curl helper that sends GITHUB_TOKEN when present (1000/hour instead of 60) and retries transient failures, including the 403s that plain --retry ignores. The helper suppresses xtrace around the call so the Authorization header cannot land in a public job log. bump_docs.sh had a sharper version of the same bug: it piped an unchecked response into `jq -r .tag_name`, so a throttled request resolved to the string "null" and would have been published as the docs version. It now refuses to write anything it cannot resolve to a tag. Verified locally by running all four scripts end to end against their real upstreams: correct SHAs/tags written, exit 0; a nonexistent repo now fails with a named diagnostic instead of a bare exit 22 and leaves the pinned file untouched; the token is absent from the xtrace output; and the scripts still work unauthenticated. Assisted-by: Claude:opus-4.8 [Claude Code] Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
5c96e097ba |
feat(gallery): fix stale DFlash drafters and add the APEX families as variant ladders (#11027)
* fix(gallery): repoint qwen3-4b/qwen3.5-9b dflash drafters at post-rename GGUFs The drafters both entries referenced were converted from the pre-merge DFlash PR branch and carry dflash.target_layer_ids. llama.cpp reads dflash.target_layers and refuses the load. The stored values are offset by +1 relative to the HF-side field, so the files cannot be repaired by renaming the key and must be replaced. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): add apexentries HuggingFace client Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(apexentries): build the HF client via pkg/httpclient The apexentries HuggingFace client was constructed as a raw &http.Client{Timeout: 60s}. The repo convention (documented in .golangci.yml, which cannot express this as a forbidigo pattern) is that all outbound HTTP goes through pkg/httpclient, which refuses redirects by default and sets a TLS 1.2 floor. The std client follows redirects and forwards custom credential headers to the redirect target on a cross-host hop (GHSA-3mj3-57v2-4636). Only a User-Agent is sent today, but this calls an external API and an HF_TOKEN header added later would leak. Switch to httpclient.NewWithTimeout, preserving the 60 second timeout. No behaviour change for the current header set. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): discover APEX tiers by filename suffix Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): resolve unsloth counterparts and sharded quants Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): render APEX child entries with the dflash/mtp tag rule Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(apexentries): set backend, known_usecases and cross-repo drafters RenderChild left three gaps against the hand-written gallery entries. The generated entries reference gallery/virtual.yaml, which supplies no backend, so every generated entry named no engine at all. All comparable hand-written entries set backend: llama-cpp in overrides; do the same. Set known_usecases to [chat] alongside it: LocalAI falls back to the backend defaults when it is absent, so this is convention rather than breakage, but generated entries should not read differently from their neighbours. The drafter was also assumed to live in the repo publishing the weights. Speculative pairings routinely cross repos, and a drafter URI built from the weights repo 404s at install time. Add ChildInput.DraftRepo, used for both the drafter URI and its local path, falling back to Repo when empty so pairings that do ship the drafter alongside the weights are unchanged. The dflash/mtp tagging rule is untouched: the tag still follows SpecType and nothing else. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): dedupe generated entries against the existing gallery Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * apexentries: canonicalize HF URIs and dedup the generated batch Merge exists to stop a second gallery entry being added for weights the gallery already ships, but two gaps let duplicates through on a bulk run. The URI key was compared as an exact string while render.go only ever emits https://huggingface.co/{repo}/resolve/main/{file} and the gallery records 1038 of its URIs in huggingface://{repo}/{file} shorthand. A generated unsloth rung whose weights are already shipped in shorthand was therefore not recognised. canonicalURI reduces both spellings to one key and is applied on both sides, taking care that the repo is exactly the first two path segments so sharded quants in a subdirectory still match. A URI in neither form is returned untouched so other hosts dedup on their literal string. Merge also never accounted for entries it had just accepted, so two generated entries sharing a name or a primary URI both landed in add. Several APEX repos share one base model and resolve to the same unsloth counterpart, so the identical rungs are generated twice under the same name. Batch state is tracked locally rather than written back into the caller's ExistingIndex, which a caller may reasonably reuse. Name is still checked before URI: a name collision must block the add regardless of the weights. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): verify variant and tagging invariants in the gallery index Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ci): scope the apex-entries verifier to what it can actually judge The verifier reported 60 problems against the real gallery, 57 of which were llama.cpp assumptions meeting entries from other backends. A gate that is wrong 57 times out of 60 cannot gate anything. - The weight-count check catches a quant label collision in llama-cpp quant discovery, so it now runs only for overrides.backend: llama-cpp. Entries with no declared backend are skipped because their weights are declared in the referenced url: template, which the verifier never reads. - The dflash/mtp tag check now implements the per-backend table in .agents/adding-gallery-models.md instead of assuming llama.cpp's spec_type: vocabulary. ds4 declares mtp_path:/mtp_draft:; sglang declares speculative_algorithm: in a file this verifier cannot follow, so sglang entries are not judged in either direction. The check stays bidirectional within the backends it does judge. - sha256 is now required on .gguf files only, since every non-GGUF asset in the index belongs to a hand-curated entry outside this generator's scope. Against the current gallery this leaves exactly the three genuine problems: two entries setting spec_type:draft-mtp without the mtp tag, and one entry whose overrides.mmproj names a file it does not download. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(apexentries): anchor quant matching and invert the sha256 rule UnaccountedQuants matched files to wanted quants with strings.Contains, which reproduces the substring collision it was written to warn about: Q8_0 is a substring of UD-Q8_0, so a repo publishing only UD-Q8_0 was reported as publishing an unbuilt Q8_0. Subdirectory-sharded UD quants are the normal unsloth layout for large repos, so this fired on realistic input. Match on the quant label as an anchored token instead, the way DiscoverUnslothQuants does, so the diagnostic and the discovery it audits cannot disagree about what a file is. Root-level shards, the layout the diagnostic mainly exists to catch, stay detected. The sha256 requirement was scoped to .gguf, which exempted seven real model weights: wan_2.1_vae.safetensors and clip_vision_h.safetensors across the wan-2.1-*-ggml entries, both load-bearing weights named by gallery/wan-ggml.yaml. Invert the rule so a checksum is required on everything except metadata extensions, which keeps a future weight format covered by default rather than silently exempt. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): wire the apexentries command Adds the generation path to the apexentries command: list the mudler APEX repos, discover each one's quality ladder and its unsloth counterpart's quant rungs from the filenames actually published, render a child entry per build plus a family parent carrying the variants list, dedup against the gallery, and write the additions to -out or append them with -apply. Discovery shortfalls are reported at discovery time rather than left to the verifier. A quant or a tier that discovery drops leaves no trace in a finished gallery file, and because an empty imatrix ladder falls back to the plain one, a repo whose imatrix filenames all fail to match downgrades the whole family silently instead of erroring. Merge's single reused map is split into two reported categories. A URI match means the gallery already ships exactly these weights and referencing the existing entry is correct; a name collision means an unrelated entry owns the name and referencing it would substitute a different build. Multimodal children now declare known_usecases [chat, vision]. An explicit known_usecases suppresses the backend-default fallback, so a chat-only entry carrying an mmproj never matches the vision or multimodal gallery filters. .github/ci is invisible to go list ./..., so a workflow names both generator packages explicitly and their specs finally run on pull requests. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): gather APEX builds under the base model entry The hub for a family is the BASE model entry, never a generated *-apex parent. Somebody looking for qwen3.6-35b-a3b has to find every build of those weights under that one name, so a competing qwen3.6-35b-a3b-apex hub would split the family and leave half of it invisible. When the gallery already ships the base entry, a variants block is spliced into it textually, leaving its description, icon, tags, overrides and files untouched. Only a family whose base model the gallery does not ship gets a new hub, still named for the base model and carrying one of the discovered builds as its own payload so it declares a backend the verifier can judge. The line editing is factored into .github/ci/galleryedit, shared with the variantproposals job, so the two cannot drift apart on where a variants block belongs. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(apexentries): treat an unreadable optional counterpart repo as absent HuggingFace answers 401 Unauthorized, not 404, for a repository that does not exist when the request carries no credentials. FetchRepoFiles treated only 404 as absence, so probing for the OPTIONAL unsloth counterpart hard failed for every family that legitimately has none: 27 of the 45 APEX families are community merges that will never have an unsloth build, and a full run failed all of them. Split the fetch so the two call sites can apply different policies to the same response. The APEX repo itself stays strict: a 401 or 403 on a repo the run requires is a real failure and still errors. Only the optional probe tolerates it, because without a token 401 cannot be told apart from absence. That collapse is lossy in one direction, since a private or gated repo also answers 401, so the skipped candidates are named in the run summary alongside the other silent-shortfall counters instead of being dropped in silence. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(apexentries): report full-precision sources as a known exclusion The 45 APEX repos publish their unquantized F16 sources next to the imatrix ladder, flat or sharded. Discovery correctly emits nothing for them, but they were landing in the unclassified total, leaving a permanent baseline of 24 benign lines on every run. That baseline is what the unclassified check exists to prevent: a standing count of known-benign files is exactly what hides the one file that ever genuinely matters. Count full-precision sources separately and give them their own summary line, so unclassified returns to 0 and stays loud when something really is an unknown shape. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(apexentries): namespace local paths by owner and enable MTP builds localPath namespaced downloads by the repo basename alone, so two repos publishing the same filename under different owners collapsed to one local path. LiquidAI/LFM2.5-8B-A1B-GGUF and unsloth/LFM2.5-8B-A1B-GGUF collided that way, and both were offered from the same hub, so installing the second either overwrote the first model's weights or was skipped as already present while recording a sha256 that did not match the bytes on disk. The owner is now its own path segment: owner/repo is globally unique on HuggingFace and neither half can contain a separator, so uniqueness holds by construction. Verify gains a check for the whole class, that no local filename may map to two different upstream URIs. It surfaces seven pre-existing collisions in the gallery, which are left alone here. Entries built from the *-APEX-MTP-GGUF repos now configure MTP rather than shipping the heads inert, matching the pattern the hand-written MTP entries already use: spec_type:draft-mtp with spec_n_max and spec_p_min, tagged mtp, and no draft_model because the heads live in the weights. RenderChild no longer requires a separate drafter file before it will configure a spec type, while the cross-repo drafter path is unchanged. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add the APEX GGUF families as variant ladders Adds the imatrix quality ladder from each mudler/*-APEX-GGUF repo, a fixed subset of unsloth quant rungs where a counterpart repo exists, and the MTP builds, then attaches them to the base model entry so one entry offers every build of the same weights and LocalAI picks the one that fits the hardware. Ten existing base model entries gain a variants list; twenty-seven families that the gallery had no base entry for get one. Builds are discovered from the filenames each repo actually publishes rather than derived from its name, since six repos ship a stem that differs from their repo name. Every file carries a sha256 taken from the HuggingFace API. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
a4a181d2f7 |
fix(distributed): count staging verification as progress, not as a stall (#11026)
Testing the progress-based cold-load deadline on the live cluster surfaced a false positive. The stall window observed UPLOAD bytes only, but the staging path has a phase that does real work while moving zero upload bytes: the resumable-upload verify phase. When a shard is already present on the worker from an earlier attempt, the frontend HEADs it, hashes the local copy to confirm it matches, and skips the transfer. Staging a 70 GB model with 56 GB already staged: 17:27:34 INFO Upload skipped (file already exists with matching hash) ... 17:28:20 INFO Upload skipped (file already exists with matching hash) ... 17:29:07 INFO Upload skipped (file already exists with matching hash) ... ... six-plus consecutive minutes, no bytes uploaded at all ~45s per skipped ~4 GB shard. That is correct and desirable - it is what makes resume work - but it was indistinguishable from a stall. At 45s per shard it sits inside the 5m window, so the run in flight was fine; the problem is the 600 GB scale this machinery exists to enable, where one shard can plausibly hash for longer than the window. The guard would then fire during verification of a transfer that is working perfectly. Verified mechanism: probeExisting() HEADs the worker and then calls downloader.CalculateSHA(). The staging progress callback is only consulted inside doUpload(), which the skip path never reaches, so observeLoadProgress was called zero times for the whole verify phase. Verification exposed a second, worse bug in the same path: CalculateSHA consults no context at all. An expired cold load kept hashing to completion, compared the hashes, and returned success - reporting a file as staged on a dead load. The failure only surfaced on the NEXT file, whose HEAD died immediately. That is exactly the shape of the red test here, which fails on shard 3. Fix: hash in 1 MiB chunks via hashFileWithActivity(), ticking the cold-load deadline per chunk and checking ctx per chunk. A successful HEAD also counts, since a 200 with a content hash proves the worker is serving right now. Counting hash progress does not make a dead transfer look alive: hashing is bounded, terminating work proportional to file size, in probeExisting it runs only after a HEAD proved the worker was up, and the 24h absolute cap still bounds the whole hold. The alternative of simply widening the window was rejected - it would reintroduce the size cliff this work removes. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
2b61e4bc1d |
fix(upgrade-check): don't filter upgrade candidates by controller capability (#11024)
CheckUpgradesAgainst resolved gallery entries through AvailableBackends, which drops every entry the *local* host cannot run. In distributed mode the host running the check is a CPU-only controller while the GPU backends live on worker nodes, so FindGalleryElement returned nil for every cuda/rocm/l4t entry and those backends were silently skipped. Measured on a live cluster: GET /backends reported 48 installed backends, POST /backends/upgrades/check evaluated 5 — all of them plain or cpu-prefixed. The 43 skipped were all hardware-specific builds. As a result cuda13-nvidia-l4t-arm64-longcat-video-development stayed at sha256:0b8dc851 while the registry tag held sha256:38dae6ff, and a cuDNN packaging fix sat unnoticed on a GPU worker for two days. Every name looked up here is already installed somewhere in the cluster, so hardware compatibility was decided at install time; re-deciding it against the controller is wrong. Switch both CheckUpgradesAgainst and UpgradeBackend to AvailableBackendsUnfiltered. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
3584e0776d |
chore: ⬆️ Update antirez/ds4 to efdadd41e20134af4f3381e1ed90e96fe4faef6f (#11010)
* ⬆️ Update antirez/ds4 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix(ds4): link new tensor parallel objects The updated ds4 revision split tensor-parallel transport and layer placement into separate translation units. Build and link those objects on CPU, CUDA, and Metal builds. Assisted-by: Codex:gpt-5 --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> |
||
|
dependabot[bot]
|
4ce67ccb84 |
chore(deps): bump body-parser from 2.2.2 to 2.3.0 in /core/http/react-ui in the npm_and_yarn group across 1 directory (#11016)
chore(deps): bump body-parser Bumps the npm_and_yarn group with 1 update in the /core/http/react-ui directory: [body-parser](https://github.com/expressjs/body-parser). Updates `body-parser` from 2.2.2 to 2.3.0 - [Release notes](https://github.com/expressjs/body-parser/releases) - [Changelog](https://github.com/expressjs/body-parser/blob/master/HISTORY.md) - [Commits](https://github.com/expressjs/body-parser/compare/v2.2.2...v2.3.0) --- updated-dependencies: - dependency-name: body-parser dependency-version: 2.3.0 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
b700a78ae4 |
fix(distributed): make the cold-load hold scale with progress, not wall-clock (#11019)
A 70 GB video checkpoint (longcat-video-avatar-1.5) could not be loaded on a
distributed cluster. The request failed with HTTP 500 after 1499.98s - exactly
the 25m00s cold-load ceiling - while staging was demonstrably healthy: 26 of 57
files and 39 GB transferred at a sustained ~26 MB/s, zero errors, no stalls. It
was not wedged, it was killed by a timer.
ModelLoadCeilingFor covers node selection, backend install, file staging and the
remote LoadModel. Install and load carry their own budgets; staging was covered
only by a FIXED 5-minute margin. But staging time is bytes over bandwidth, not a
constant: 70 GB at 26 MB/s needs ~45m against a 25m ceiling, so the failure is
deterministic for any sufficiently large model rather than a flake. Simply
raising the constant moves the cliff to the next model size - the deployment
target here is checkpoints of 600 GB and beyond.
The ceiling's real purpose is that "a wedged worker can never pin the lock
indefinitely". Progress, not elapsed time, is what distinguishes a wedged worker
from a large one. The hold is now a deadline that extends whenever the transfer
reports bytes and expires a 5-minute stall window after they stop:
- A large model transferring fine continues, for hours if needed.
- A worker that died mid-transfer still fails within the stall window.
Progress is observed at byte level on the transfer itself, via the existing
staging progress callback. Per-file completion would be too coarse - a single
600 GB shard would be indistinguishable from a stall for hours. The observation
point is back-pressured by the socket, so it reflects the network rather than
local disk reads. Observation is coarsened to one timer touch per stall/20 so
the per-read callback stays cheap.
The base budget (unchanged, and still derived from the install and load
timeouts) continues to cover the steps that report no progress, so
LOCALAI_NATS_MODEL_LOAD_TIMEOUT keeps working exactly as before. An absolute
cap of 24h bounds the hold even while progress keeps arriving, so a peer
trickling bytes forever cannot pin the advisory lock; 600 GB at the measured
26 MB/s is ~6.5h, so the cap sits far above any legitimate transfer.
Also fixes the incoherent layering the same error exposed: the resumable upload
carried a 1h retry budget nested inside the 25m ceiling, so the inner budget was
unreachable and the message still blamed it ("failed after 1 attempts within
1h0m0s budget") while the 25m parent was the actual killer. The upload now
adopts the caller's deadline when there is one, and applies its fixed budget
only when nothing above bounded it - which also stops a fixed 1h from
reintroducing the size cliff under the now-extendable parent.
This is the successor to #10968, where a hardcoded 5-minute LoadModel gRPC
timeout was replaced by this derived ceiling. Fixing the inner timeout exposed
the outer ceiling as the new binding constraint.
Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
|
||
|
Richard Palethorpe
|
0d2124894e |
docs(realtime): fix Opus backend installation (#11018)
The Realtime guide incorrectly sent the Opus backend through the model gallery endpoint. Point users to the backend gallery API and document the UI and CLI alternatives. Assisted-by: Codex:gpt-5 Signed-off-by: Richard Palethorpe <io@richiejp.com> |
||
|
mudler's LocalAI [bot]
|
54d5c18bfb |
fix(model): only announce a load at INFO when a load actually happens (#11017)
backendLoader logged "BackendLoader starting" at INFO as its very first statement, unconditionally. That reads as "a model is being loaded", but backendLoader is not only a load path: in distributed mode Load() deliberately bypasses the local cache and calls backendLoader on every inference request so SmartRouter can re-pick a replica per request. The model is already resident, no process is spawned, and nothing is loaded, yet the banner fires at request rate. On a live cluster this produced ~5 "BackendLoader starting" lines per second for a single embedding model, sustained, starting 22 seconds after the load had already completed. The model was state=loaded with in_flight=0 and exactly one backend process on the worker. It looked exactly like a retry storm and cost real debugging time during an unrelated production investigation. The adjacent "effective runtime tuning" banner, documented as "logged once per load", had the same problem for the same reason. Emit both banners at INFO only when the model is not already resident, and keep the per-call trace at DEBUG for anyone following the routing path. isResident is a plain store lookup with no health probe and no eviction, so it is safe on the per-request hot path (unlike checkIsLoaded, which probes and can evict). Same class of defect as #10985: a log line that sends the reader after the wrong thing. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
49fcdd921f |
chore: ⬆️ Update CrispStrobe/CrispASR to 644a8b1b31ca42e26f641df38e323ed9d698a1ff (#11007)
⬆️ Update CrispStrobe/CrispASR Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
e887a1ccf3 |
chore: ⬆️ Update ServeurpersoCom/qwentts.cpp to ba4c7f7838ecb24a75b0ac94e14fdbebb6bb138c (#11006)
⬆️ Update ServeurpersoCom/qwentts.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
5fbd79a4bb |
chore: ⬆️ Update PrismML-Eng/llama.cpp to 7529fdaaf99ffdc5ca71ace9c7409a56b27ad92f (#11009)
⬆️ Update PrismML-Eng/llama.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
dependabot[bot]
|
2a8eb5a04b |
chore(deps): bump the npm_and_yarn group across 1 directory with 5 updates (#11011)
Bumps the npm_and_yarn group with 5 updates in the /core/http/react-ui directory: | Package | From | To | | --- | --- | --- | | [dompurify](https://github.com/cure53/DOMPurify) | `3.4.0` | `3.4.11` | | [hono](https://github.com/honojs/hono) | `4.12.18` | `4.12.31` | | [qs](https://github.com/ljharb/qs) | `6.15.0` | `6.15.3` | | [react-router](https://github.com/remix-run/react-router/tree/HEAD/packages/react-router) | `7.13.1` | `7.18.1` | | [undici](https://github.com/nodejs/undici) | `7.25.0` | `7.28.0` | Updates `dompurify` from 3.4.0 to 3.4.11 - [Release notes](https://github.com/cure53/DOMPurify/releases) - [Commits](https://github.com/cure53/DOMPurify/compare/3.4.0...3.4.11) Updates `hono` from 4.12.18 to 4.12.31 - [Release notes](https://github.com/honojs/hono/releases) - [Commits](https://github.com/honojs/hono/compare/v4.12.18...v4.12.31) Updates `qs` from 6.15.0 to 6.15.3 - [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md) - [Commits](https://github.com/ljharb/qs/compare/v6.15.0...v6.15.3) Updates `react-router` from 7.13.1 to 7.18.1 - [Release notes](https://github.com/remix-run/react-router/releases) - [Changelog](https://github.com/remix-run/react-router/blob/react-router@7.18.1/packages/react-router/CHANGELOG.md) - [Commits](https://github.com/remix-run/react-router/commits/react-router@7.18.1/packages/react-router) Updates `undici` from 7.25.0 to 7.28.0 - [Release notes](https://github.com/nodejs/undici/releases) - [Commits](https://github.com/nodejs/undici/compare/v7.25.0...v7.28.0) --- updated-dependencies: - dependency-name: dompurify dependency-version: 3.4.11 dependency-type: direct:production dependency-group: npm_and_yarn - dependency-name: hono dependency-version: 4.12.31 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: qs dependency-version: 6.15.3 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: react-router dependency-version: 7.18.1 dependency-type: indirect dependency-group: npm_and_yarn - dependency-name: undici dependency-version: 7.28.0 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
9c8f510021 |
chore(model gallery): 🤖 add 1 new models via gallery agent (#11013)
chore(model gallery): 🤖 add new models via gallery agent Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
localai-org-maint-bot
|
1e0baec2a7 |
fix(ci): repair nightly backend dep bumps for renamed localai-org repos (#11012)
The "Bump Backend dependencies" workflow has failed every night for over ten days. Four upstreams — ced.cpp, moss-transcribe.cpp, voice-detect.cpp and rf-detr.cpp — moved from the mudler org to localai-org, so the GitHub API answers 301 for the old slugs. ced.cpp additionally renamed its default branch to main. bump_deps.sh fetched without -L or -f and never checked the response, so the redirect's JSON body was passed straight to sed, which died with "unterminated `s' command". The loud failure was luck: an error body without slashes would have been substituted into the Makefile as the new pin, silently corrupting the version and shipping it in a bump PR. Point the matrix at the new slugs and branch, and harden the script so a bad response can never reach sed: follow redirects, fail on HTTP errors, and require a bare 40-hex SHA before rewriting anything. Also refresh the now-stale repository URLs in the backend Makefiles, test scripts, backend/index.yaml and the docs. Verified all 25 matrix entries resolve to a commit SHA and that the four previously-failing jobs run end to end against the real API. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
7bda73fd66 |
chore: ⬆️ Update ServeurpersoCom/omnivoice.cpp to f39cc4a3af988091f662313b336dddf8c83a3fb5 (#11002)
⬆️ Update ServeurpersoCom/omnivoice.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
a2c87947a9 |
chore(model-gallery): ⬆️ update checksum (#11005)
⬆️ Checksum updates in gallery/index.yaml Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
7c984f5c81 |
chore: ⬆️ Update vllm-metal (darwin) to v0.3.0.dev20260720105820 (#11003)
⬆️ Update vllm-project/vllm-metal (darwin) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
f8755997cc |
feat(swagger): update swagger (#11001)
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
d0401f9bb4 |
chore: bump go-processmanager to pick up the concurrent-Run fix (#11004)
Picks up mudler/go-processmanager#7, which closes the check-then-act window in Run's "already started" guard. Concurrent Run calls on one handle could each observe a nil p.proc, each start a process and each launch a monitor goroutine; every monitor does `defer close(p.done)` against a channel created once in New, so the second monitor to see its process exit panicked on close of a closed channel. The same window raced on p.proc itself. LocalAI does not hit this today: the only New/Run pair (pkg/model/process.go:178-191) runs a freshly created handle, and core/services/worker/supervisor.go only ever calls Stop on handles it holds. The bump is defensive, and keeps the dependency from drifting further behind a fix in the process lifecycle we rely on. No exported signature changes upstream; ErrProcessAlreadyRun is additive and keeps the historical "command already started" prefix, so any caller matching on that text is unaffected. Nothing in this repo matches it. Verified: go build ./core/... ./pkg/... clean; go vet clean; go test -race ./pkg/model/ ./core/services/worker/ both ok. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Bash] [Edit] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
0cdd781c2d |
ci(gallery): propose variant groupings for review instead of letting them decay (#10992)
ci(gallery): propose variant groupings for review on a schedule A gallery entry may declare `variants:`, references to other entries that are alternative builds of the same weights, and auto-selection then installs the best build for the host. Those families exist only because humans curated them in two manual sweeps. The gallery agent dedupes on the HuggingFace repo URL and picks one quantization per model, so it never adds a second build of a repo it already has, and consequently never creates a family and never joins one. A model published across two repos lands as two unrelated standalone entries. The grouping decays as the gallery grows and nothing notices. Add a scheduled job, in the same shape as the checksum checker: compute offline, edit the index textually, open a pull request against ci-forks. It proposes and never decides. Grouping is a judgement call that has gone wrong in both directions, so the value is catching drift and surfacing candidates with their evidence. Three grouping signals, taken from the manual sweeps: same name once quantization markers are stripped, the `:` config-suffix convention, and the same primary weight filename once quantization markers are stripped. The third requires the same upstream repository. Excluding auxiliary files is not enough on its own: bert-embeddings, an ultravox audio model and a roleplay finetune all declare a primary file called llama-3.2-1b-instruct-q4_k_m.gguf, and grouping on that is the same error that linked four wan-2.1 entries and Z-Image-Turbo to qwen3-4b. Add gallery/variant-exclusions.yaml, a checked-in rejection ledger. A job that re-proposes declined candidates every night becomes noise and gets ignored. Declining a proposal is one flow-mapping line a reviewer adds inside the proposal pull request itself. Seeded with the six -abliterated pairs whose base is also in the gallery, the mistral-small multimodal pair, the whisper-1 alias, the kokoros language set, and the recurring finetune tokens. qat and apex are deliberately not on it: they are quantization techniques. Proposals refuse to nest, to let two parents claim one target, to target an entry that installs nothing, and to touch a merge anchor, since a variants key added to an anchor is inherited by every merging child. The anchor refusal names every entry that would inherit, which is the worklist a human needs. Run against the pre-sweep gallery, the job rediscovers 12 of the 19 groupings the second manual sweep made, with no false positives. The rest it reports as refusals or ledger declines rather than missing silently. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
f01038f479 |
fix(modelartifacts): stage each writer's artifact in its own partial tree (#10995)
Every writer used to stage into the same `.artifacts/.partial/<cacheKey>`. That was safe only because the artifact lock held: two writers that both believed they had it opened the same blob with O_APPEND and interleaved their bytes into one file, while the resume probe read the other writer's in-flight size. SHA verification caught the damage only after both had burned the entire download. #10986 restored the lock's precondition on CIFS but left the dependency in place. Suffix the staging tree with a writer identity drawn once per process run, so concurrent writers cannot corrupt each other whatever the lock does. The lock stops being a correctness dependency and becomes a pure efficiency optimisation: a lock failure now costs a duplicated download, not a corrupted one. Commit stays an atomic rename. The loser of a commit race reconciles onto the winner's tree instead of surfacing a bare ENOTEMPTY for work that actually succeeded, since the artifact is content-addressed and both trees hold the same verified bytes. Writer-unique staging means a crashed writer's tree is no longer overwritten by its successor, so two things are added to keep it from becoming a disk leak and a resume regression: - A sweep reclaims trees whose contents have been untouched for 24h, matching the window the startup reaper already uses for stray *.partial files. It reads the newest mtime anywhere inside the tree, because writing a blob never touches an ancestor, and refuses any name this package did not write. A live download writes continuously, and the downloader's stall watchdog aborts a silent one long before it could look abandoned. - Adoption lets a restarted process claim a dead predecessor's tree for the same artifact and resume from its bytes, which a tens-of-gigabytes repo depends on. The claim is an atomic rename, so racing adopters cannot both win. It runs only under the artifact lock - which is released exactly when the owning process dies - and only on a tree idle for 5 minutes as a second line of defence for when the lock does not exclude. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
0eb8a1188d |
fix(worker): give the worker a real health endpoint and a mode-aware HEALTHCHECK (#10999)
fix(worker): give the worker a real health endpoint (#10987) The image bakes in a single HEALTHCHECK that curls http://localhost:8080/readyz, but the same image also runs `local-ai worker`, which serves HTTP on the gRPC base port minus one and never binds 8080. Every worker container was therefore permanently `unhealthy` (43 consecutive failures observed on a production node), which is worse than having no healthcheck: a genuinely broken worker and a perfectly good one both report `unhealthy`, so the signal carries no information and orchestration that keys on it misbehaves. The worker already served /readyz on that port via the file-transfer server, but as a constant 200 — it only proved the listener was bound, which is precisely the failure mode at issue. Readiness now tracks the live NATS connection: all of a worker's actual work (backend lifecycle events, inference dispatch, file staging) arrives over NATS, so a worker whose link is dead is up and useless. Registration is already implied, since the server only starts after registration succeeds. This reports something the controller cannot already see. The node registry's status/last_heartbeat is fed by an HTTP heartbeat to the frontend, a different network path from NATS — a worker can keep heartbeating while its NATS connection is dead and still look healthy in the registry. /healthz stays a constant 200: liveness must not follow readiness, or a NATS blip becomes a cluster-wide restart storm. The HEALTHCHECK is now a script that derives its endpoint from the mode the container is actually running plus the env vars that configure the bind address, so a frontend moved off 8080 with LOCALAI_ADDRESS (broken the same way) and a worker on a non-default base port are both probed correctly. Modes with no HTTP surface (agent-worker, one-shot commands) report healthy rather than false-unhealthy. HEALTHCHECK_ENDPOINT remains as an explicit override, so the workaround shipped in docker-compose.distributed.yaml keeps working; both overrides in that file are now unnecessary and have been removed. Also fixes the latent --start-period gap. Since #10949 a frontend's startup preload materializes HuggingFace artifacts before the HTTP server binds (31 GB observed on a live cluster), so a healthy replica can legitimately fail probes for a long time. --start-period is Docker's knob for exactly this: failures inside it leave the container `starting` instead of burning retries, and it ends early on the first success, so a generous 60m costs a fast-starting container nothing. --timeout drops from 10m to 10s — it is a per-probe deadline, and a localhost curl that has not answered in 10s is itself the fault being detected. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
d7e04dcc32 |
fix(openresponses): make responses visible and cancellable across replicas (#11000)
In distributed mode the Open Responses store is process-local: a sync.OnceValue over a map behind an RWMutex. With several frontend replicas behind a round-robin load balancer, every request that lands on a replica other than the creator misses. Measured on a live 2-replica cluster (#10993): the same response id returns 200 on the creating replica and 404 on its peer, and a cancel on the peer returns 404 without ever invoking CancelFunc, so generation runs to completion on the other replica while the caller is told the response does not exist. previous_response_id chaining fails through the same lookup. Split the state by what can actually cross a process boundary: - Replicated: response metadata (request, response resource, owner, expiry, stream/background flags) via syncstate.SyncedMap, the same component finetune, quantization and agent tasks already use. A local miss in Get/FindItem now falls back to it and returns a read-only remote view, so polling and chaining resolve on any replica. - Delegated: cancellation. context.CancelFunc is a function pointer and exists only in the creating process, so a cancel that lands elsewhere is broadcast on responses.<id>.cancel and applied by whichever replica holds the function. The broadcast is fire-and-forget rather than request/reply: if the owner crashed or was scaled down nobody answers, and the handler must not block on a reply that will never come. The replicated status moves to cancelled either way, which is truthful, since a dead owner's generation died with its process. - Refused: streaming resume. The resume buffer is a byte log plus a live notification channel and cannot be replicated without shipping every token over the bus. A resume that reaches the wrong replica now returns HTTP 409 naming the owning replica via the new ErrResponseNotLocal, instead of an empty event list that looks like a finished stream. It is deliberately distinct from ErrOffsetLost, which means the owner's buffer evicted the requested events. Standalone deployments never call EnableDistributed and keep exactly the previous process-local behaviour. Fixes #10993 Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
a4bab71f27 |
gallery: remove duplicated entries and lint against them recurring (#10996)
gallery: remove duplicated entries and lint against them coming back gallery/index.yaml declared eight names twice: deepseek-r1-distill-llama-8b, llama3.2-3b-enigma, qwen3-asr-0.6b, qwen3-asr-1.7b, qwopus-glm-18b-merged, voice-en-us-kathleen-low, whisper-large-q5_0 and whisper-small-q5_1. FindGalleryElement resolves a reference by returning the first match, so in every pair the second copy was unreachable: it could not be installed, could not be selected as a variant target, and could not be corrected, because any edit to it went to a copy nobody reads. A reference to such a name is also ambiguous to anything reasoning over the catalog, which is why the variant proposal job refuses to propose against them. Each pair was compared both as parsed entries and as raw text, and all eight were byte-identical apart from position. None of the sixteen blocks defines a YAML anchor or pulls one in with a merge key, so nothing was reachable only through a deleted block, and no entry named a removed copy as a variant target. Removing the second copy of each therefore changes no behaviour: the parsed set loses exactly eight entries and every surviving entry is field-for-field unchanged. The removal is textual, by line range, so the diff is pure deletions rather than a reflow of forty thousand lines. checkNoDuplicateEntryNames is the rule that keeps them out, added beside the existing gallery invariants and reporting in the same style. checkSingleVariantClaim closes the adjacent gap in the same place. VariantParents resolves a build claimed by two parents by taking the first in gallery order and calls that deterministic "for a gallery the linter would reject", but nothing rejected it: the invariant was held by curation alone. Now it is a rule, and the comment describes something real. No target is doubly claimed today, so the rule is green on arrival. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
2f33ad6669 |
fix(modelartifacts): treat CIFS EACCES as lock contention, not failure (#10986)
flock(2) on CIFS/SMB returns EACCES when another client holds the lock: the kernel maps STATUS_LOCK_NOT_GRANTED and STATUS_FILE_LOCK_CONFLICT to -EACCES and never produces EWOULDBLOCK on that path. gofrs/flock only recognises EWOULDBLOCK as contention, so TryLockContext returned a bare "permission denied" and Ensure aborted. Both replicas then fell back to legacy loading, which makes the worker download the whole repo in-band inside LoadModel and blow the remote-load deadline. Replace TryLockContext with an explicit wait loop over a new Locker interface, classifying EWOULDBLOCK/EAGAIN/EACCES/EBUSY as contention. EACCES is ambiguous at the syscall boundary but not here: the lock file is already open O_CREATE|O_RDWR, so a real permission problem would have failed the open with an *fs.PathError, and flock(2) documents no EACCES on Linux at all. The wait is bounded (DefaultLockWait, overridable via WithLockWait), so even a misclassification degrades to a delay. On timeout the committed result is re-checked before reporting the new ErrLockContended, so a peer that finished the work still wins. Locker also exists so the contention path is testable without a network filesystem: nothing in CI can make flock(2) return EACCES on demand. Raise the fallback to error for a managedArtifactBackends backend, via a shared config.LogArtifactFallback used by both call sites. For those backends the legacy path is not graceful degradation, and the operator otherwise sees only a timeout with no causal link. The fallback stays non-fatal. Drop the os.Chmod(layout.Lock, 0o600) after acquisition: flock.New already creates the file 0600, and the chmod was gratuitous risk on a nounix mount that ignores modes. Fixes #10981 Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
1cd7d63c7b |
fix(distributed): reject wrong-model requests on the remaining modalities (#10990)
#10970 gave the four PredictOptions RPCs a model-identity check so a backend reached through a stale distributed route rejects the request instead of answering from whatever model it holds (#10952). Every other modality shares that exposure: the route is cached by host:port, a worker can recycle a stopped backend's port for another model's backend, and a liveness-only probe cannot tell a stale row from a valid one. Extends the same mechanism to the 21 remaining request messages that reach a backend through the router, using the pattern #10970 established rather than a parallel one: - proto: ModelIdentity on each modality request message. - controller: populated from ModelConfig.Model at the call site that also builds ModelOptions, so load-time and request-time values are equal by construction. - backends: one generic guard in pkg/grpc/server.go (27 Go backends), the method set in backend/python/common (36 Python backends), llama-cpp (AudioTranscription/Stream, Rerank, Score) and privacy-filter (TokenClassify). - reconcile already drops the stale row on IsModelMismatch; no change. TTSRequest and SoundGenerationRequest get a SEPARATE ModelIdentity field rather than reusing their existing `model`: FileStagingClient rewrites `model` to a worker-local path, so comparing it would reject valid requests in exactly the configuration this guards. AudioEncode/AudioDecode are deliberately left unguarded: the opus codec backend is loaded from a literal rather than a ModelConfig, so no value carries the equality guarantee the comparison depends on. The four bidirectional stream RPCs are out of scope; they bypass reconcile. Empty means skip on both sides, so an old controller, an old backend, and the bare request structs in tests/e2e-backends all keep working. Assisted-by: Claude Code:claude-opus-4-8 [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
a784cf669f |
fix(ci): rebuild Go backends on linked pkg/ changes and on matrix entry edits (#10988)
PR #10975 taught the backend matrix filter about shared build inputs, but left two paths that still rebuild nothing. Go backends link code from the main tree. `go list -deps ./backend/go/...` resolves to exactly six pkg subtrees (audio, grpc incl. base/grpcerrors/proto, httpclient, sound, store, utils), identical for GOOS/GOARCH in {linux,darwin} x {amd64,arm64}. Editing any of them changes the shipped binary, but they sit outside every backend directory so the prefix match never saw them. Enumerating those six rather than taking all of pkg/ is the point: all of pkg/ changes in ~8.6% of commits, these six in 2.0% — the same order as the already accepted scripts/build/ rule (1.9%). Blast radius 199/417 Linux, 26/56 Darwin; the ~21 core-server-only pkg subtrees still rebuild nothing, and neither do _test.go files. .github/backend-matrix.yml was excluded wholesale because matching its path would rebuild all 417 entries on every new-backend PR. That hid a real hole: editing an existing entry's base-image, build-type or cuda version changes the image it produces while touching no file the filter can see. Since the change is within a structured file, compare it against the base revision and rebuild only the entries whose fields actually differ — 1 entry for a base-image edit, 0 for a comment or whitespace edit, and all 417 only when the previous revision cannot be resolved. This also closes a third hole: a new matrix entry for an existing backend (a new CUDA variant, say) touches nothing under that backend's directory and previously rebuilt nothing. changed-backends.js fetches the base revision via the contents API, and only when the changed-file list actually names the matrix file, so the common path costs no extra request. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
65bdbc4ee3 |
fix(http): make /readyz reflect startup readiness, plus gitignore and coverage-ratchet fixes (#10989)
* fix(http): make /readyz reflect startup readiness instead of always 200 /readyz was registered as a static handler returning 200 unconditionally, so it carried no information: it was green whenever it could be reached at all. Readiness could not distinguish "serving" from "still starting", and any future change that started the HTTP listener earlier would silently turn the probe into a lie. Track startup completion on the Application (atomic flag, flipped at the very end of New() on the success path only) and have the readiness handler consult it per request, returning 503 with a small JSON body while startup is in progress. A nil readiness source fails open so embedders keep the historical behaviour. /healthz is deliberately left readiness-independent. Liveness and readiness answer different questions, and failing liveness during a long preload makes an orchestrator restart the pod mid-download so the preload never finishes. This matters because since #10949 the startup preload materializes HuggingFace artifacts for managed backends: tens of GB for a large model (31 GB observed on a live cluster). Both probes stay in quietPaths and stay exempt from auth. Note the listener is still started only after New() returns, so today the not-ready state is not observable over HTTP. Moving the listener earlier is a separate, deliberate decision and is not made here. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * chore(gitignore): anchor the mock-backend pattern so its source dir is traversable The bare `mock-backend` pattern matched the *directory* tests/e2e/mock-backend/, not just the binary built into it. Git will not descend into an ignored directory even for tracked files, so `git add tests/e2e/mock-backend/main.go` required -f. This was hit while working on #10970. Anchor it to the artifact's full path. The built binary stays ignored (it is also covered by tests/e2e/mock-backend/.gitignore) while the source directory becomes traversable again. Verified with `git check-ignore -v`: a new source file under tests/e2e/mock-backend/ is no longer ignored, and the binary produced by `make build-mock-backend` still is. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * chore(coverage): raise the coverage ratchet from 48.5% to 54.2% The committed baseline had drifted well below reality: it still read 48.5% while a full instrumented run measures 54.2%. A stale-low baseline makes the gate meaningless — coverage could regress by more than 5 percentage points and still pass. Raising a ratchet is a deliberate act, not something to fold into an unrelated fix, so it gets its own commit. The headroom was earned by tests landed in #10946, #10947, #10948, #10949, #10956, #10967, #10968, #10970 and #10975. Measured with `make test-coverage` on this branch (the same instrumented run `make test-coverage-baseline` uses: ginkgo over ./pkg and ./core plus the in-process tests/e2e suite, --covermode=atomic, --coverpkg over core/... and pkg/..., generated protobuf excluded). The run completed with exit 0 and zero spec failures; the total was then written with the exact command the test-coverage-baseline target uses: go tool cover -func=coverage/coverage.out \ | awk '/^total:/{gsub(/%/,"",$NF); print $NF}' > coverage-baseline.txt Verified afterwards with scripts/coverage-check.sh, which reports OK. Note the measured figure includes the readiness specs added earlier on this branch, so it is a demonstrated floor rather than an estimate. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
0d9d07d3a5 |
fix(downloader): distinguish read from write failures and retry transient ones (#10985)
Two independent defects in the download path, both surfaced by the same incident (#10982). A failed `io.Copy` was always reported as "failed to write file", because `io.Copy` folds read and write errors into a single return value. A peer-cancelled HTTP/2 stream therefore presented as a filesystem failure and sent an investigation after mount permissions while the disk was healthy. The source is now wrapped in a recorder so the error names the side that actually broke, and a write failure names the `.partial` it was writing rather than the final blob path. The plan runner returned on the first task error with no retry, so one transient stream cancel discarded every file already downloaded in a multi-file materialization. The `.partial` resume machinery already existed but was unreachable because nothing made a second attempt. Transient failures (dropped transport, mid-stream read failure, stall, 5xx, 429) are now retried with bounded exponential backoff and resume from the partial; permanent ones (4xx, checksum mismatch, local write failure, caller cancellation) fail immediately. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
f381844403 |
gallery: group QAT, APEX and cross-backend builds under their base entry (#10983)
feat(gallery): group 21 more model families under variants Second variant-grouping sweep. QAT and APEX builds are now treated as quantization techniques rather than distinct weights, per maintainer ruling, so they group with their base entry instead of standing alone. Adds 15 new families: quantization and serving-config pairs for llama-3.2-1b/3b-instruct, dolphin-2.9-llama3-8b, phi-2-chat, ideogram-4, meta-llama-3.1-8b-instruct, omnivoice-cpp and qwen3-tts-cpp; the gemma-3 4b/12b/27b QAT families; and three cross-backend pairs (silero-vad plus its sherpa-onnx build, and the vibevoice TTS and ASR builds shared between the vibevoice-cpp and crispasr backends). The cross-backend pairs are the first entries that meaningfully exercise engine-preference ranking during auto-selection. Restructures four gemma-4 families (31b-it, 26b-a4b-it, e2b-it, e4b-it). Those bare entries were skipped by the first sweep, which left a QAT build as parent by default. The bare entry is what installs when nothing else fits, so it reclaims the parent slot and the former parent becomes a plain target. Every pre-existing relationship is preserved; nothing is dropped and nothing nests. gemma-4-12b-it has no bare entry, so it is left as is. qwen3-tts-cpp is a YAML anchor with nine merging children, so the five children that did not already override variants get an explicit empty list to stop them inheriting the parent's. Abliterated builds stay excluded: abliteration edits the weights to remove refusal behaviour, which makes them a different model rather than another build of the same one. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
83a0f16a21 |
feat(gallery): let one gallery entry offer several builds of the same model (#10943)
* feat(system): expose raw detected capability for model meta resolution Model meta gallery entries express hardware fallback through candidate ordering rather than a capability map, so they need the undecorated detected capability string without Capability's default/cpu fallback chain. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * refactor(system): drop duplicate capability accessor, cover DetectedCapability ReportedCapability was added with a body identical to the existing DetectedCapability. Keep one accessor and move the specs onto it, since DetectedCapability had no direct coverage of its no-fallback behavior. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(vram): parse IEC binary size suffixes (KiB..PiB) ParseSizeString accepted only SI suffixes, so a "20GiB" floor was rejected outright. Model and VRAM sizes are conventionally quoted in IEC units, and silently reading GiB as GB would understate a floor by about 7%. Purely additive: these inputs previously returned an unknown-suffix error. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add Candidate type for meta model entries Candidate is one option in a meta entry's ordered variant list. It names a concrete gallery entry and declares when that entry suits the host. EffectiveMinVRAM resolves the VRAM floor, letting an authored min_vram win over a nightly-inferred one. An unparseable floor errors instead of being treated as absent: swallowing a typo would turn a constrained candidate into an unconstrained one and select a too-large variant rather than fail loudly. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add hardware-aware model variant resolver Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): allow gallery model entries to declare variant candidates A gallery entry with a non-empty candidates list is a meta entry: it names an ordered list of concrete entries and resolves to the first one the host can satisfy, instead of describing model files directly. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): resolve meta model entries to hardware-appropriate variants at install Meta gallery entries carry an ordered candidate list; at install time the first candidate the host satisfies is resolved and its payload installed under the meta's name, so the model keeps a stable name regardless of which variant backs it. The resolution is recorded in the installed gallery config so a reinstall honors a prior pin and operators can see the backing variant. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): key meta pin recall on the installed name and detach resolved entries Six review findings on the meta-entry install path. Pin recall was keyed on the gallery entry name while applyModel writes the record under the install name (req.Name when supplied), so a meta installed under a custom name with a pin lost that pin on reinstall and was silently re-resolved onto a different variant, possibly swapping its backend. Compute the install name with applyModel's own precedence before the recall. ResolveMetaModel returned a shallow struct copy, so the resolved entry's Overrides aliased the gallery entry's map and the install path's in-place mergo merge wrote the caller's request into the shared catalog. Detach Overrides, ConfigFile, AdditionalFiles, URLs and Tags. Not exploitable today only because this path re-unmarshals the gallery per call, which is a property nobody should have to rely on. Also: overlay the meta's name onto the persisted config for meta installs so the gallery file no longer records the variant's name; move the pinned-VRAM warning below the variant validation so a pin naming a nonexistent entry does not warn about VRAM before failing for an unrelated reason; and stop seeding config.URLs in the config_file branch, which duplicated every declared URL. Add seven network-free specs driving InstallModelFromGallery with a meta entry: variant payload wins over the meta's legacy url fallback, the resolution record round-trips to disk, a pin is recorded and honored on reinstall including under a custom install name, and the resolved entry does not alias the gallery's maps. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(gallery): deep-copy meta overrides and make two specs functional ResolveMetaModel detached the resolved entry's Overrides and ConfigFile with maps.Clone, which only copies the top level. Gallery overrides are nested in practice (parameters.model is near-universal) and the install path merges the caller's request with mergo.WithOverride, which recurses into nested maps and overwrites them in place, so the gallery entry's own inner maps were still reachable and still got rewritten by the last caller to install. Copy both maps all the way down instead, recursing through the container shapes a YAML decoder produces. ConfigFile is not mutated on the install path today, but it carries the same kind of nested payload and leaving it shallowly cloned would invite the bug back. Also fix two specs that passed whether or not their target fix was present: - "does not write the caller's overrides back into the gallery entry" re-read the catalog from disk, which re-unmarshals fresh structs and so cannot observe in-memory aliasing. It now asserts against the in-memory gallery entry and drives the real mergo merge. - "round-trips the resolution record to disk under the meta's name" asserted a name that is already correct in the config_file branch. It now drives the url branch via a file:// fixture, where the meta-name overlay actually applies. Both were verified red by reverting their fix. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(gallery): lint meta model entry invariants in index.yaml Adds Ginkgo specs that parse the shipped gallery/index.yaml and enforce the invariants that keep meta entries safe: a legacy url fallback equal to the final candidate's url, references only to existing non-meta entries, a min_vram floor on every candidate but the last-resort one, a capability drawn only from the vocabulary the system can report, and descending VRAM floors within a capability group. The capability check is the only compensating control for a typo there. Candidate matching is a case-sensitive exact comparison against SystemState.DetectedCapability(), so an unknown value never matches and falls through silently instead of erroring. The vocabulary therefore mirrors the raw return set of getSystemCapabilities(), which notably excludes "cpu": that is a fallback key inside Capability(capMap) on the meta backend path, never a reported capability. A CPU-only host reports "default". These pass vacuously until the pilot meta entry lands; the guard is intentionally in place before the thing it guards. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * test(gallery): close coverage gaps in the meta entry lint The ordering invariant grouped candidates by capability and asserted floors descend within a group. A candidate with an EMPTY capability matches every host, so it does not belong in its own group: it dominates every later candidate whose floor is at or above its own, across capability groups. Track a running minimum floor over the unconditional candidates instead, which subsumes the old same-group check for the empty capability. Every spec skipped non-meta entries, so with zero meta entries in the index all five bodies were no-ops. Aligning GalleryModel.IsMeta() with GalleryBackend.IsMeta(), whose semantics are deliberately opposite, would have made all of them pass while checking nothing. Extract each invariant into a helper over a slice of entries returning the violations it finds, and cover those helpers with synthetic fixtures so the logic stays tested at zero meta entries. The index-driven specs are now a thin application of already proven logic. Also assert the index parses non-empty, report every violation in one run rather than aborting on the first, and parse the index once for the suite. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(gallery): add nightly denormalization of meta model candidates Fills the read-only backend, quantization and inferred_min_vram fields on meta gallery candidates and opens a PR, modeled on the existing checksum_checker job. Computing these needs network access, so it happens nightly rather than at install time. An authored min_vram is never modified: a human who measured a real load knows more than a pre-download estimate does. The index is rewritten via yaml.Node rather than a document round-trip. A full round-trip reflows all ~26k lines of gallery/index.yaml, which would bury the computed values and make the nightly PR unreviewable. The rewrite touches only the three derived keys, so authored styling survives and a run that computes nothing leaves the file untouched. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ci): keep the gallery denormalize diff reviewable and self-healing The nightly denormalization job edits YAML nodes instead of round-tripping structs so its PR stays small enough for a human to review, but the write path undid that: yaml.Marshal re-encoded the node tree at yaml.v3's default 4-space indent and dropped the leading document marker, reflowing roughly 6000 lines around the handful of real changes. Encode through yaml.NewEncoder at the index's authored 2-space indent and restore the header. A write that changes three fields now changes three lines. Stale inferred_min_vram values were also never cleared. Both skip paths (an authored min_vram is present, or the candidate is the last resort) returned before touching the field, so a candidate that gained a floor or became the last resort after a reorder kept an inferred value that EffectiveMinVRAM reported as a real constraint, failing the meta lint with no way for the job to self-heal. Clear the field before both skips. The workflow discarded a whole night's work on any single failure: the program exits 1 when a candidate cannot be estimated, which aborted the job before the PR step, so one unreachable candidate blocked every other refresh indefinitely. Capture the status, open the PR with what was computed, mark the PR body as partial, and fail the run afterwards so the problem still surfaces. Also preserve the index's existing file mode instead of forcing 0644, and drop the redundant //go:build ignore tag, since Go already skips dot directories and the sibling modelslist.go carries no tag. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add nanbeige4.1-3b meta entry with hardware-resolved variants Adds the first real meta entry to the gallery index. It resolves to the Q8_0 build on hosts with at least 6GiB of VRAM and to the Q4_K_M build everywhere else, installing either payload under the stable name nanbeige4.1-3b. The entry carries a url equal to its final candidate's url. LocalAI releases that predate candidates support parse the index non-strictly and drop the key silently, so without that url they would list the entry and install nothing. A regression spec parses the index the way those releases do and asserts every meta entry stays installable for them. Also teaches core/schema/gallery-model.schema.json about candidates. The schema sets additionalProperties: false at the top level, so an author following CONTRIBUTING.md and adding the yaml-language-server comment would otherwise get a validation error on this entry. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): make candidate entries complete, installable entries Reworks hardware-resolved gallery variants after a design pivot. There is no longer a separate "meta" entry kind. A gallery entry is a normal, complete entry that may additionally carry candidates:, a list of hardware-gated upgrades over itself, and the entry is itself the last-resort candidate. The previous design relied on a bare url: as the fallback for LocalAI releases that predate candidates support. That fallback is empty in practice: none of the 80 gallery/*.yaml files carry a top-level files:, and 1216 of 1281 index entries carry their payload in the index entry itself, so a url alone yields a config template with nothing to download. Since every released LocalAI reads gallery/index.yaml live from master, merging a payload-less entry would have shown every existing user a model that installs to a broken state. Making the entry its own base candidate removes the problem at the root: old clients drop the candidates key and install the entry exactly as they do today. Resolution order is now explicit pin, then capability plus VRAM over the declared upgrades, then the entry itself. The entry ALWAYS installs: when its own min_vram or capability is unmet the installer warns and installs it anyway, because there is nothing below it and refusing would make the gallery behave worse the newer the client is. A pin naming the entry's own name is valid and is how an operator declines an upgrade. IsMeta() becomes HasCandidates(), ResolveMetaModel becomes ResolveVariant, and the persisted meta_name record key becomes entry_name. GalleryBackend.IsMeta() is a separate concept and is untouched. The lint drops the three rules the pivot makes wrong (url equality with the final candidate, no inline payload, unconstrained final candidate) and gains one: the entry's own floor must sit strictly below every candidate's, since a base that outranks a candidate makes that candidate unreachable. The pilot entry is now the existing nanbeige4.1-3b-q4, which gains a 2GiB floor of its own and a single 6GiB upgrade to nanbeige4.1-3b-q8, replacing the separate nanbeige4.1-3b entry added in |
||
|
mudler's LocalAI [bot]
|
6e52d0c2ef |
fix(ci): rebuild backends when shared build inputs change (#10975)
The backend matrix path filter only matched files under a backend's own directory, so a change to shared build infrastructure rebuilt nothing at all: an empty matrix, every job green, and the change reaching no image. PR #10946 fixed scripts/build/package-gpu-libs.sh shipping a partial 4-of-8 cuDNN library set, which mixed versions with the venv's pip cuDNN and produced CUDNN_STATUS_SUBLIBRARY_VERSION_MISMATCH at inference time. It merged 1h48m after the weekly full-matrix cron had already run, so no backend image ever received the fix and nothing signalled that it had been un-shipped. Add a SHARED_BUILD_INPUTS table mapping each shared path to the narrowest set of matrix entries it can honestly invalidate, plus a generic rule for backend/Dockerfile.<x> (which each entry already names). A full matrix is 417 Linux + 56 Darwin builds, so package-gpu-libs.sh now rebuilds the 176 Python entries rather than everything. Unclassified files under scripts/build/ fall back to a full rebuild deliberately: over-building is recoverable, silently shipping nothing is not. Extract the filtering logic to scripts/lib/backend-filter.mjs so it can be unit-tested without bun, js-yaml or a GitHub API round-trip, and run those tests from the existing lint workflow via `make test-ci-scripts`. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
465d488c90 |
fix(distributed): reject wrong-model requests at the backend (#10970)
fix(distributed): reject wrong-model requests at the backend (#10952) In distributed mode the controller caches a NodeModel row naming a backend's host:port. A worker can recycle a stopped backend's gRPC port for a different model's backend, and probeHealth verifies liveness rather than identity, so the probe succeeds against whatever now occupies the port and the request is dispatched to the wrong backend. The caller gets a silent wrong-model answer. Nothing in the request could catch this: PredictOptions had no model field, so model identity crossed the wire only in ModelOptions.Model at LoadModel time, and the cached-hit path issues no LoadModel. Every backend's "model not loaded" guard checks a nil handle, which a process holding a different model passes, so the stale row was never dropped either. Add PredictOptions.ModelIdentity and enforce it at the point of use: - The controller populates it in gRPCPredictOpts from ModelConfig.Model, the same expression ModelOptions feeds to model.WithModel and therefore the same value the backend received as ModelOptions.Model. Both are read from one config value in one function, so they are equal by construction and the comparison cannot false-reject. - Backends compare it against what they loaded and return NOT_FOUND with a fixed sentinel. Enforced in pkg/grpc/server.go (27 Go backends), an interceptor in backend/python/common (all 36 Python backends, no per-backend change), and the llama-cpp / ik-llama-cpp / ds4 C++ servers. That is every backend with real exposure: kokoros answers all four RPCs with unimplemented and privacy-filter implements none of them. - The router's reconcile drops the stale replica row on a mismatch, so the next request reloads somewhere correct. Empty means "skip the check" on both sides: a controller that predates the field sends nothing, a backend loaded by such a controller has nothing to compare, and the C++ server synthesizes PredictOptions internally for ASR. That keeps upgrades working in both directions. Scoped to the four PredictOptions RPCs. TTSRequest.model and SoundGenerationRequest.model are deliberately NOT validated: FileStagingClient already rewrites them to worker-local absolute paths, so in distributed mode they already differ from the load-time value and comparing them would reject valid requests. IsModelMismatch requires both the NOT_FOUND code and the sentinel, unlike the neighbouring helpers which accept either. insightface's Embedding returns NOT_FOUND "no face detected" on a PredictOptions RPC, and a code-only check would drop a healthy replica row on every faceless image. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
1618c2e445 |
chore(model gallery): 🤖 add 1 new models via gallery agent (#10971)
chore(model gallery): 🤖 add new models via gallery agent Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |
||
|
dependabot[bot]
|
9043cbc786 |
chore(deps): bump torch CPU wheels to 2.12.1 (#10969)
* chore(deps): bump the pip group across 6 directories with 1 update Bumps the pip group with 1 update in the /backend/python/ace-step directory: torch. Bumps the pip group with 1 update in the /backend/python/llama-cpp-quantization directory: torch. Bumps the pip group with 1 update in the /backend/python/longcat-video directory: torch. Bumps the pip group with 1 update in the /backend/python/sglang directory: torch. Bumps the pip group with 1 update in the /backend/python/trl directory: torch. Bumps the pip group with 1 update in the /backend/python/vllm-omni directory: torch. Updates `torch` from 2.10.0+rocm7.0 to 2.12.1+cpu Updates `torch` from 2.10.0 to 2.12.1+cpu Updates `torch` from 2.12.1 to 2.12.1+cu130 Updates `torch` from 2.9.0 to 2.12.1+cpu Updates `torch` from 2.10.0 to 2.12.1+cpu Updates `torch` from 2.7.0 to 2.12.1+cu130 --- updated-dependencies: - dependency-name: torch dependency-version: 2.12.1+cpu dependency-type: direct:production dependency-group: pip - dependency-name: torch dependency-version: 2.12.1+cpu dependency-type: direct:production dependency-group: pip - dependency-name: torch dependency-version: 2.12.1+cu130 dependency-type: direct:production dependency-group: pip - dependency-name: torch dependency-version: 2.12.1+cpu dependency-type: direct:production dependency-group: pip - dependency-name: torch dependency-version: 2.12.1+cpu dependency-type: direct:production dependency-group: pip - dependency-name: torch dependency-version: 2.12.1+cu130 dependency-type: direct:production dependency-group: pip ... Signed-off-by: dependabot[bot] <support@github.com> * fix(deps): preserve platform-specific torch requirements Keep the 2.12.1 CPU bump only where uv resolves it cleanly, and restore ROCm, CUDA, MPS, and unrelated transformers constraints that Dependabot rewrote to incompatible wheel variants. Assisted-by: Codex:gpt-5 [uv] --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> |
||
|
localai-org-maint-bot
|
0406741a8c |
fix(vibevoice): install diffusers from PyPI instead of git main (#10972)
Every vibevoice requirements file pulled diffusers straight from git+https://github.com/huggingface/diffusers. That branch now reports itself as 0.40.0.dev0 and requires huggingface-hub>=1.23.0,<2.0, while transformers>=4.51.3,<5.0.0 (which upstream VibeVoice mandates) still caps huggingface-hub at <1.0. Because a git URL offers the resolver exactly one candidate version, uv has nothing to backtrack to and the install fails outright: Because only diffusers==0.40.0.dev0 is available and diffusers==0.40.0.dev0 depends on huggingface-hub>=1.23.0,<2.0 [...] we can conclude that your requirements are unsatisfiable. This broke the vibevoice build on every variant - cpu (amd64/arm64), cuda 12/13, l4t 12/13, intel and rocm, plus the darwin metal job - and has been failing the weekly full-matrix rebuild for three weeks. It is not caught by master pushes because backend builds are path-filtered there, so it only surfaces on the Sunday cron and on release tags. Use the PyPI package instead. That is what upstream VibeVoice declares in its own pyproject.toml, and what every other LocalAI backend already does - vibevoice was the only one tracking the git branch. With a real release series available the resolver settles on diffusers 0.39.0 with huggingface-hub 0.36.2 and transformers 4.57.6, and it can keep backtracking on its own if upstream shifts again. Verified with uv pip compile against cpu, cublas12, cublas13, hipblas, intel, mps and l4t13: all resolve to that same coherent set. l4t12 only resolves on aarch64, since its Jetson index ships no x86_64 torch wheel. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Bash] [uv] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
Nandana Dileep
|
b5e4413eab |
feat: add MiniMax-M3 model support (#10837)
Adds inference parameter defaults for the minimax-m3 model family and includes a vendored patch of upstream llama.cpp PR #24523 to recognize the minimax-m3 architecture. Once the upstream PR merges, the patch can be removed and LLAMA_VERSION bumped normally. Changes: - backend/cpp/llama-cpp/patches/0001-add-minimax-m3-support.patch: vendored patch from ggml-org/llama.cpp#24523 (Preliminary MiniMax-M3 support). Applied by prepare.sh during the build; keeps the pinned LLAMA_VERSION pointing at the latest upstream tag. - core/config/inference_defaults.json: add minimax-m3 family entry (temperature=1.0, top_p=0.95, top_k=40, min_p=0.01, repeat_penalty=1.0, matching the existing minimax defaults) and register it in the patterns list before the shorter minimax-m2.7 entry for correct longest-match-first ordering. Upstream: depends on ggml-org/llama.cpp#24523 Closes: https://github.com/mudler/LocalAI/issues/10820 Signed-off-by: Nandana Dileep <110280757+nandanadileep@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]
|
e55cc3e2a7 |
fix(worker): bound the gRPC port allocator and stop leaking dead backends' ports (#10968)
The worker's gRPC port allocator grew monotonically with no upper bound: nextPort started at the base port and incremented whenever freePorts was empty, and nothing checked 65535. Past that it handed out integers that cannot be bound, surfacing as an opaque "backend won't start". #10961 estimated this needed ~15,000 concurrent-peak allocations, i.e. effectively unreachable. It is not, because of a second defect: the "process died unexpectedly" branch in startBackend deleted the process map entry without releasing its port at all. That port was leaked, never quarantined and never reused. A crash-looping backend leaks one port per restart, so a backend dying every 30s walks 50051 to 65535 in about five days. The leak, not concurrent peak, is the realistic route to exhaustion. Fixing the leak alone would have been wrong. Releasing that port makes it re-bindable, and the death path is the one teardown path with no request/reply to carry StoppedProcessKeys back to the controller (#10952's eager row removal), so a stale NodeModel row could then resolve to a live listener belonging to a different backend. probeHealth verifies liveness, not identity, so the request is silently misrouted. The 15s port quarantine does not cover this: the only reaper is the per-model health check at ~45s, and it can be disabled outright. The residual was masked only because the port was never rebound. So both are fixed together: - The allocator takes an explicit [basePort, LOCALAI_GRPC_MAX_PORT] range and returns ErrNoFreePort naming the range, the live backend count, the quarantined count, and the knob to raise. Exhaustion is now diagnosable instead of surfacing as an unbindable port. - Released ports carry per-key affinity: a port is offered back to the process key that last held it before any other key. Process keys (modelID#replica) and NodeModel rows (nodeID, modelName, replicaIndex) are isomorphic, so a port that can only be re-bound by its previous owner can only ever be named by that owner's row, which that key's re-registration overwrites. Misrouting to a different model becomes impossible by construction rather than by racing the quarantine timer. Affinity is a preference, not a reservation: under range pressure an owned port is stolen with a warning, because a guaranteed outage is worse than a rare misroute window on a port long out of quarantine. Claiming a port evicts its previous owner's entry, keeping ownership injective over ports so the affinity map can never exceed the range width regardless of how many distinct model keys the worker sees. Ownership also expires. It is only load-bearing while a controller row could still name the port, which the per-model reaper bounds at roughly 45s, so it lapses after five minutes and the port becomes ordinary free space again. Holding it indefinitely would have made every distinct model the worker ever served consume a port permanently: every release path is keyed, so nothing would ever be unowned, the allocator would climb to the end of its range on distinct-key count rather than concurrency, stealing would become routine, and the steal warning would tell operators to widen a range that was not the constraint. With expiry, reaching the steal branch means the worker is genuinely out of concurrent capacity, so that advice is correct when it appears. Closes #10961 Closes #10952 Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
9d82c37f98 |
fix(distributed): backend discovery hid worker-installed backends behind the controller's filesystem (#10967)
fix(distributed): backend discovery hid worker-installed backends Backend discovery endpoints filter on installed-state, which on a distributed controller derives from the controller's own filesystem. A backend lives on the worker node that runs it, so every backend an admin installed on a GPU worker read as "not installed" and vanished from the listing. #10947 fixed the sibling capability filter on the same endpoints, so a fine-tuning-capable GPU worker now made the backend listable while the installed-state filter still dropped it: the dropdown stayed empty. The controller cannot derive this locally, but it already aggregates the per-node view that GET /backends renders, so discovery reuses the active BackendManager rather than growing a second path. Three surfaces shared the root cause and route through the same helper now: - GET /backends/available (Installed is now cluster-wide) - GET /api/fine-tuning/backends - GET /api/quantization/backends The response stays a boolean rather than an installed-on-N-of-M count: per-node install state is already served by GET /backends nodes[], and per-node control by POST /api/nodes/:id/backends/install, so a summary is all these dropdowns need. A nil provider (single-node) leaves the local filesystem as the only source and reproduces today's listing exactly, and a registry error degrades to that same listing instead of blanking the catalog. Assisted-by: Claude:claude-opus-4-8 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]
|
f735cb24c0 |
fix(worker): reap deleted backends and stop models that live on a worker (#10956)
* fix(worker): reap deleted backends and stop models that live on a worker
Three related backend-lifecycle defects, all reachable from the same
production incident on a Jetson/Thor worker: a deleted backend's gRPC
process survived ~40 minutes with its directory removed from disk, a later
model load was routed to that orphan and failed with a certifi path pointing
into the deleted directory, and the admin could not stop the model because
the frontend reported it as not loaded.
1. backend.delete orphaned the process it claimed to delete
------------------------------------------------------------
s.processes is keyed by `modelID#replicaIndex` (buildProcessKey), so the
backend name never appeared in a key and was recorded nowhere on the
process. backend.delete resolved its target via isRunning/stopBackend, whose
prefix path only matches a bare *modelID* - a delete keyed on a backend name
resolved to zero keys, the stop silently no-op'd, and the files were removed
out from under a live process.
The install fast path then handed that orphan back out: it returns any live
process for the (model, replica) slot without checking which backend started
it, so a reinstalled variant inherited the deleted backend's port.
- Record backendName on backendProcess, threaded installBackend ->
startBackend.
- Add resolveProcessKeysForBackend, matching the recorded name and resolving
alias <-> concrete via ListSystemBackends *before* DeleteBackendFromSystem
erases the metadata that carries the alias. Alias resolution failure
degrades to name-only matching so a delete never fails on it.
- backend.stop goes through resolveStopTargets, which accepts a backend
name, a model name, or an exact modelID#replica key. Its payload field is
named "backend" but is published with all three meanings: the admin UI
sends a backend name, UnloadRemoteModel sends a model name, and the
router's abandoned-load reap (#10948) sends an exact replica key.
Narrowing it to backend names alone would strand the latter two.
backend.delete stays strict - its identifier is unambiguously a backend.
- Gate the install fast path on processMatchesBackend so a slot held by a
different backend is restarted rather than reused. Processes with no
recorded name (pre-upgrade) are accepted, so rollout does not restart
every running backend.
- stopBackendExact reports a real stop failure - the process still being
alive afterwards, which is precisely what finishBackendStop already
detects to keep the entry and its port reserved - and backend.delete no
longer replies success when it knew about a process and could not kill it.
"No process was running" stays a success but is logged, so the orphan case
is visible rather than silent.
2. /backend/shutdown reported a running model as missing
---------------------------------------------------------
ModelLoader.deleteProcess short-circuits on a miss in this replica's
in-memory store. In distributed mode the authoritative record of "is this
model loaded" is the shared node registry: a frontend replica that never
served the model itself (load balancer picked a peer, or the replica
restarted) has no local entry. The remote unload path that pkg/model
documents ("when ShutdownModel is called for a model with no local process,
UnloadRemoteModel is called") sat behind that short-circuit, unreachable in
exactly the case it exists for. #10865 reworked this function but kept the
short-circuit at the top, so the gap survived that refactor.
- deleteProcess consults the remote unloader on a local-store miss, via a
shared unloadRemote helper so this branch and the existing
no-local-process branch both prefer #10865's RemoteModelContextUnloader,
preserving force propagation across the distributed boundary.
- UnloadRemoteModelContext reports ErrRemoteModelNotLoaded when no node has
the model; it previously returned nil, making a no-op stop
indistinguishable from a real one. The converse case (nodes have it, none
could be stopped) already errors since #10865 joined the per-node
failures, so that half of the original fix was dropped as redundant.
- Only when the model is absent locally AND cluster-wide does the endpoint
report not-found, now 404 naming both scopes rather than a bare 500.
- modelNotFoundErr becomes the exported ErrModelNotFound so the HTTP layer
can map it without string matching; watchdog's identity comparison becomes
errors.Is.
3. Coverage for the bounded Free() that #10865 shipped untested
----------------------------------------------------------------
The original branch also bounded the pre-stop Free(), but #10865 landed that
fix first (workerBackendFreeTimeout, applied in both stopBackendExact and
handleModelUnload). That production change is therefore DROPPED here as
superseded - master's version is strictly better, since it also releases the
supervisor mutex across the call and keeps the port reserved until
termination completes.
What #10865 did not ship is a test, and the bound is load-bearing: the
router-side reap in #10948 sends backend.stop for an abandoned load, and
against a wedged backend an unbounded Free would swallow that stop before it
reached the process. Nothing failed if the bound regressed.
The spec stands up a real gRPC backend server whose Free handler never
returns - what a Python backend looks like when its single worker thread
(PYTHON_GRPC_MAX_WORKERS=1 on 37 backends) is occupied by a stuck LoadModel.
A stub socket is not sufficient and was tried first: without a completed
HTTP/2 handshake, gRPC's own ~20s connect timeout ends the call, so that
version passed against the very bug it targets. With the connection READY,
only the caller's deadline can end it, so the spec hangs to its 60s limit if
the timeout is removed and passes with it.
Its fixture process is deliberately never started. go-processmanager v0.1.1
writes Process.pid from readPID() without synchronization, so a live process
races its own monitor goroutine under -race - reproducible with a bare
Run()+Stop() and unrelated to this spec. Since
scripts/model-lifecycle-conformance.sh runs this package with -race and is
fail-closed, starting one would turn that gate red on an upstream defect. An
unstarted process still proves the point: the stop is reached and the slot
released, which is exactly what an unbounded Free prevents.
Verified: make lint (new-from-merge-base origin/master) reports 0 issues;
scripts/model-lifecycle-conformance.sh passes all three stages including the
FizzBee liveness check (1458 states, IsLive: true).
Assisted-by: Claude:claude-opus-4-8 golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(distributed): keep remote unload idempotent, ask presence separately
|
||
|
mudler's LocalAI [bot]
|
5c607c09d5 |
chore: ⬆️ Update ServeurpersoCom/omnivoice.cpp to 339e3d7fc7161f8ae61d22c291ff40f68b690266 (#10962)
⬆️ Update ServeurpersoCom/omnivoice.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> |