mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-14 15:18:03 -04:00
master
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ettore Di Giacinto
|
dad4d5956a |
ci: move lint back to hosted runners, arc image has no make/gcc
Follow-up to
|
||
|
Ettore Di Giacinto
|
42541dd4f6 |
ci: route site deploy and lint to the self-hosted runner
The GitHub-hosted runner pool is shared per ACCOUNT, not per repo, so a burst in one repo starves every other. On 2026-07-31 it reached zero scheduled jobs for 35 consecutive minutes with 39 jobs queued, while arc-runner-set completed 12 jobs without interruption across the same window. Actions was healthy globally (other public repos were scheduling normally), so this is an account-level throttle we cannot fix from inside the workflows, only route around. Site publishing and lint are small, run on nearly every commit, and gain nothing from waiting behind a saturated hosted queue, so both move to arc-runner-set, the label already proven in generate_intel_image.yaml. lint.yml is routed for PUSH ONLY, and this is the important part: that workflow also triggers on pull_request, and a fork PR executes untrusted contributor code. Running that on a persistent self-hosted runner would be a real compromise vector, so anything that is not a push to mudler/LocalAI stays on the ephemeral hosted pool. gh-pages.yml needs no such clause: it triggers only on push-to-master and workflow_dispatch, so it never runs pull-request code. Both carry a repository guard so forks, which have no such runner label, fall back to hosted instead of queueing forever. Neither workflow uses sudo or apt, and both fetch their own toolchains via setup-go / actions-hugo. A self-hosted image can still be leaner than the hosted one, so each lint job opens with a preflight that names the missing tool (curl/unzip/make for protoc and lint; gcc/ldd/python3 for the packaging-script tests) rather than failing opaquely mid-build. Reverting is one runs-on expression per job. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [ClaudeCode] |
||
|
|
47097041ff |
fix(vllm): apply Options[] engine flags before engine init (#11147)
fix(vllm): apply Options[] engine flags before engine init (#11130) CLI-style flags in a model's `options:` array (`--quantization:gptq_marlin`, `--enable-prefix-caching`, `--kv-cache-dtype:fp8_e5m2`) were discarded: the backend only ever read `tool_parser`/`reasoning_parser` out of Options[], and did so *after* `AsyncLLMEngine.from_engine_args()`, where nothing it set could still reach the engine. Map `--` prefixed options onto the AsyncEngineArgs dataclass before the engine is constructed. Names are normalized the way vLLM's CLI spells them (`--enable-prefix-caching` -> `enable_prefix_caching`), values are coerced to the target field's type (bare flag -> True for booleans), and unknown or uncoercible flags warn and are skipped instead of failing the load, since Options[] is a bag shared with backend-level settings. Field types come from the annotation's base so `Literal["auto", "float16"]` (vLLM's dtype) is not mistaken for a float. Precedence is typed proto fields -> `options:` -> `engine_args:`. The production engine_args defaults seeded in hooks_vllm.go therefore skip any key the user already set as an option, otherwise the later engine_args pass would silently override it. Parser lookups now accept both spellings, so `--reasoning-parser:qwen3` selects LocalAI's parser as well. The helper's tests are stdlib-only and run in the lint workflow's dependency-light job via `make test-python-helpers`. Assisted-by: Claude:claude-opus-5 golangci-lint Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: localai-org-maint-bot <bot-opensource@localaisrl.com> |
||
|
mudler's LocalAI [bot]andEttore Di Giacinto
|
d225e15f0f |
fix(ci): skip the image and Go PR workflows on content they cannot see (#11218)
backend_pr.yml and test-extra.yml already filter themselves, so a gallery-only or docs-only PR costs them about one job each. The image and Go workflows had no filter of any kind, so a one-line gallery/index.yaml edit queued 20 jobs: 7 container image builds, 3 GoReleaser/darwin launcher builds, 3 unit test jobs, 2 golangci-lint, 1 e2e, 1 yamllint, plus the 3 that correctly stop after their detect step. A docs-only PR queued the same. This matters more than the job count suggests. Measured over the week to 2026-07-30, 97% of CI wall-clock is queueing and 3% is execution: a median 5-hour queue against a 4-20 minute median job. Cutting job count is the only lever that shortens feedback time. The volume is there to cut, too: 13 gallery-only PRs merged that week with 10 open at once, and 78 of the 137 PRs opened were bot-generated. Add paths-ignore for gallery/**, docs/**, examples/** and **/*.md to the pull_request trigger of image-pr.yml, build-test.yaml and tests-e2e.yml, and add gallery/** to lint.yml, which already excluded the rest. That drops 13 of the 20 jobs. None of the four can observe such a diff: gallery metadata is parsed at runtime and never copied into an image, docs and markdown never enter one at all, GoReleaser and the launcher take no such input, the e2e suite drives backends over gRPC directly, and golangci-lint runs new-from-merge-base so a diff with no touched Go lines is a no-op. The build-test exclusion also frees macOS capacity, which is the scarcest runner class. The two checks that do validate the gallery are deliberately left alone. test.yml still runs core/gallery/variants_lint_test.go, which reads the real gallery/index.yaml and asserts the index invariants, and yaml-check.yml still lints the syntax. paths-ignore skips a run only when every changed file matches, so a PR touching the gallery and Go code still runs everything. master carries no branch protection and no rulesets, so a skipped workflow reports no status and nothing waits on it; .agents/ci-caching.md records that constraint for whenever required status checks are introduced. image.yml on master push is left unfiltered on purpose: skipping it would stop the master and latest tags being republished for a gallery commit, which is a publishing decision rather than a cost one. Assisted-by: Claude:opus-5 [claude-code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
dependabot[bot]
|
ac9352ef54 |
chore(deps): bump actions/setup-node from 4 to 7 (#11080)
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4 to 7. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](https://github.com/actions/setup-node/compare/v4...v7) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
mudler's LocalAI [bot]andEttore Di Giacinto
|
6e52d0c2ef |
fix(ci): rebuild backends when shared build inputs change (#10975)
The backend matrix path filter only matched files under a backend's own directory, so a change to shared build infrastructure rebuilt nothing at all: an empty matrix, every job green, and the change reaching no image. PR #10946 fixed scripts/build/package-gpu-libs.sh shipping a partial 4-of-8 cuDNN library set, which mixed versions with the venv's pip cuDNN and produced CUDNN_STATUS_SUBLIBRARY_VERSION_MISMATCH at inference time. It merged 1h48m after the weekly full-matrix cron had already run, so no backend image ever received the fix and nothing signalled that it had been un-shipped. Add a SHARED_BUILD_INPUTS table mapping each shared path to the narrowest set of matrix entries it can honestly invalidate, plus a generic rule for backend/Dockerfile.<x> (which each entry already names). A full matrix is 417 Linux + 56 Darwin builds, so package-gpu-libs.sh now rebuilds the 176 Python entries rather than everything. Unclassified files under scripts/build/ fall back to a full rebuild deliberately: over-building is recoverable, silently shipping nothing is not. Extract the filtering logic to scripts/lib/backend-filter.mjs so it can be unit-tested without bun, js-yaml or a GitHub API round-trip, and run those tests from the existing lint workflow via `make test-ci-scripts`. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
mudler's LocalAI [bot]andEttore Di Giacinto
|
963c637130 |
fix(gpu-libs): bundle cuDNN only where it is used, and complete it when it is (#10946)
cuDNN 9 is a dispatcher (libcudnn.so.9) plus seven sublibraries the dispatcher
dlopen()s by bare soname. Only the dispatcher is ever a DT_NEEDED, so ldd finds
it and never the seven. The allowlist force-copied three of them
(libcudnn.so*, libcudnn_ops.so*, libcudnn_cnn.so*) into every CUDA backend,
which is wrong in both directions at once: too few libraries for a backend that
uses cuDNN, and too many for one that does not.
On an L4T fleet, ten of the eleven backends carrying cuDNN were in a broken end
state; the one that was correct was correct by accident, being BUILD_TYPE=cpu
so package_cuda_libs never ran for it.
longcat-video bundled 4 of 8 at 9.24.0 over a complete pip set at 9.20.0.48
in its venv. libbackend.sh puts lib/ on LD_LIBRARY_PATH, searched before
DT_RUNPATH, so the bundle won and the rest still came from the venv:
CUDNN_STATUS_SUBLIBRARY_VERSION_MISMATCH.
Nine others bundled 3 of 8 and had no venv cuDNN. None bundled
libcudnn_graph, which libcudnn_cnn has a hard DT_NEEDED on, so it resolved
out of the runtime image and the process ran bundled 9.22.0 against system
9.23.2.
Five of those nine - llama-cpp, whisper, rfdetr-cpp, sam3-cpp,
stablediffusion-ggml - do not reference cuDNN at all. ggml goes through cuBLAS.
They were carrying ~57 MB of cuDNN with no consumer, and completing the family
for them would have taken that to ~576 MB for nothing.
Sizes overall: backends with no cuDNN consumer shed ~57 MB each (seven
instances on the fleet measured, plus longcat's ~60 MB), while the ones that
genuinely use cuDNN grow from ~57 MB to ~576 MB, because the five missing
sublibraries are ~517 MB, dominated by libcudnn_engines_precompiled. Net on
that fleet is an increase of roughly 570 MB. That growth is the bug being paid
off, not a regression: those backends only work today by silently borrowing the
missing five from the runtime image. Whether the engines set can be trimmed is
an open question, not addressed here.
So bundle per backend, by what that backend actually needs:
- venv has a complete pip cuDNN -> bundle nothing; $ORIGIN resolves the pip
set, which is the one its torch was built against (longcat-video)
- venv has no pip cuDNN -> bundle the complete family. Stays
conservative rather than detecting consumers: for a Python backend they sit
inside the venv (torch, ctranslate2, onnxruntime) where the sweep does not
look (vllm)
- no venv, nothing references cuDNN -> bundle nothing (llama-cpp, whisper,
rfdetr-cpp, sam3-cpp, stablediffusion-ggml)
- no venv, something references it -> bundle the complete family
(face-detect, voice-detect)
The no-venv case needs no new machinery. Go backends stage their own shared
object into package/lib, which IS the target dir, so sweep_transitive_deps
already pulls the dispatcher when it is a genuine dependency - that is exactly
how libcudnn_graph reached longcat. cuDNN simply comes off the force-copy list,
and complete_cudnn_family fills in the seven dlopen'd sublibraries around
whatever the sweep found. Detection is a string scan rather than ldd, so a
consumer that only dlopen()s cuDNN is seen too; over-matching costs an unused
library, under-matching costs a backend that cannot load.
Keeping bundled and pip versions in agreement instead is not viable: nothing
here pins nvidia-cudnn (zero occurrences), torch is unpinned for l4t13 except
longcat-video, and the fleet already runs five concurrent cuDNN versions -
9.19.0.56, 9.20.0.48, 9.22.0, 9.23.2, 9.24.0.
verify_cudnn_bundle asserts the end state: exactly one complete cuDNN visible to
whoever needs one - never both, never partial, and never zero for a backend that
references it. Zero is correct and common otherwise. It deliberately does not
accept the build image's system cuDNN as completing a partial bundle, which is
the shape that had been shipping silently; the build image is not the runtime
image. A version check alone would have missed longcat too, whose four bundled
libs were all 9.24.0 and mutually consistent.
Match per family for the other components for the same dlopen reason: TensorRT
(libnvinfer_plugin, libnvinfer_builder_resource), cuBLAS, cuFFT, cuSPARSE,
cuSOLVER, nvRTC. Exclusions bind inside copy_lib so they cover the sweep.
The packaging scripts' shell tests ran nowhere in CI. Add make
test-build-scripts and a lint workflow job so they gate every PR.
Fixes #10905
Assisted-by: Claude:claude-opus-4-8 golangci-lint shellcheck
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
|
||
|
dependabot[bot]
|
10184b5e28 |
chore(deps): bump actions/checkout from 6 to 7 (#10451)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v6...v7) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Ettore Di Giacinto
|
5a12392570 |
ci(concurrency): make cancel-in-progress event-aware, group by sha on push
Yesterday two PRs (#9724 llama.cpp bump, #9731 llama-cpp-darwin consolidation) merged 11 seconds apart. Both shared the same backend.yml concurrency group (ci-backends-refs/heads/master-...) due to "${{ github.head_ref || github.ref }}" — empty head_ref on push events falls through to the static refs/heads/master. With cancel-in-progress: true that meant the second merge cancelled the first's in-flight backend builds. The first PR's CI never finished; the second PR only touched CI files so its run was a no-op. Two changes per workflow: - group: replace "${{ github.head_ref || github.ref }}" with "${{ github.event.pull_request.number || github.sha }}". On PRs this groups by PR number (same as before, just keyed on number not branch name); on push events it groups per-commit, so two master pushes never share a group. - cancel-in-progress: gate on github.event_name == 'pull_request' so rapid pushes to a PR still cancel old runs (newer push wins) but master pushes never cancel each other. Trade-off vs alternatives: - Merge queue would also solve this and additionally test the merged commit before it lands. Heavier process change; out of scope here. - Allowing per-commit master concurrency means two simultaneous master runs may overlap and race on tag pushes, but each commit's manifest digest is unique and the registry is last-writer-wins on tags — newer commit's tag overwrites older. Applied to 11 workflows that share the same concurrency pattern: backend.yml, backend_pr.yml, image.yml, image-pr.yml, lint.yml, test.yml, test-extra.yml, tests-e2e.yml, tests-aio.yml, tests-ui-e2e.yml, generate_intel_image.yaml. Assisted-by: Claude:claude-opus-4-7 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> |
||
|
dependabot[bot]
|
1caab1de10 |
chore(deps): bump actions/checkout from 4 to 6 (#9663)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 6. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v4...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Richard Palethorpe
|
4443250756 |
chore: add golangci-lint with new-from-merge-base baseline (#9603)
* chore: add golangci-lint with new-from-merge-base baseline
Configure golangci-lint v2 with the standard linter set (errcheck, govet,
ineffassign, unused) plus forbidigo, which enforces the Ginkgo/Gomega-only
test convention from .agents/coding-style.md by rejecting stdlib testing
calls (t.Errorf, t.Fatalf, t.Run, ...). staticcheck is disabled — the
codebase has many pre-existing QF-style suggestions not worth gating on.
issues.new-from-merge-base = master makes the lint job a gate for new
issues only; the ~1300 pre-existing baseline stays visible via
'make lint-all' for incremental cleanup. CI runs 'make lint'.
Backends needing C/C++ headers we don't install in the lint runner are
excluded via a deny list in the Makefile (backend/go/{piper,silero-vad,
llm}, cmd/launcher). Discovery still flows through 'go list ./...', so
new packages are scanned automatically.
To make backend/go/{sam3-cpp,stablediffusion-ggml,whisper} typecheckable,
move their .cpp/.h sources into cpp/ subdirs (matching qwen3-tts-cpp /
acestep-cpp). Without this 'go list' rejects the package because Go does
not allow .cpp alongside .go without cgo.
Fix two real bugs found by lint in tests/integration/ (run only via
'make test-stores', not default CI): a stale zerolog reference left over
from the slog migration (
|