Add a video_lora_dir model option that passes lora_dir as a model-load
extra to the vllm.cpp engine. The engine uses it to resolve
<lora:name:strength> prompt tags at request time, loading the safetensors
LoRA factors and applying per-request deltas without touching base weights
(row ROAD-V1-LORA-RUNTIME).
This is distinct from the existing load-time lora_path/lora_strength fusion
(ROAD-V1-DIT-LORA), which bakes deltas into the DiT weights at load. Both
mechanisms coexist: load-time adapters are always active, while prompt-tag
adapters are selected per request.
The prompt passes through verbatim — the engine strips the tags internally.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:regolo/glm5.2 [maki]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(vulkan): preserve host ICD discovery for packaged backends
Add bundled Mesa manifests through VK_ADD_DRIVER_FILES instead of replacing the system driver list. Merge inherited and model-specific additive paths while preserving explicit operator overrides, with regression coverage.
Assisted-by: Codex:gpt-5 golangci-lint
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* feat(3d): add Kimodo CPU and Vulkan animation backend
Introduce a distinct animation capability and model-described 3D operations, with a typed /3d/animate API, RPC transport, distributed media staging, permissions, and tracing.
Add a persistent kimodo.cpp adapter, skeleton GLB export, CPU/Vulkan packages, model and backend galleries, importer support, CI builds, and documentation. Adapt Studio inputs to each model and provide real-time skeleton playback, seeking, and history.
Cover backend validation, packaging, API behavior, importer inventories, distributed staging, and Studio workflows. Validate real-model CPU/Vulkan generation and deploy the integration to the local QA instance.
Assisted-by: Codex:gpt-5 golangci-lint
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Richard Palethorpe <io@richiejp.com>
* feat(kimodocpp): adopt monolithic encoders and resident inference
Update upstream for resident weights, packed execution paths, and cached motion graphs. Default to all 32 text layers while retaining configurable streaming and legacy bundle support.
Use monolithic Q8_0 encoders by default and offer all six published quantizations through the gallery and importer. Refresh pinned hashes, tests, and documentation; remove the obsolete thread patch and ensure cached source checkouts follow the upstream pin.
Validated CPU and Vulkan generation, lower-bit streaming, gallery/importer suites, packaging, lint, and cold/warm Studio generation on localai-dev.
Assisted-by: Codex:gpt-5 golangci-lint
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Richard Palethorpe <io@richiejp.com>
---------
Signed-off-by: Richard Palethorpe <io@richiejp.com>
feat(vllm-cpp): wire lora_adapters/lora_scales into DiT load-time fusion
The vllm-cpp video backend now converts the standard LocalAI
lora_adapters/lora_scales config fields into indexed lora_path/
lora_strength extras pairs that vllm.cpp's ResolveDitLoraSpecs
consumes (row ROAD-V1-DIT-LORA).
The singular lora_adapter/lora_scale fields are folded in as the
first adapter. Relative paths resolve against the model directory.
Strength defaults to 1.0 when no scale is given. Adapters are fused
into DiT weights at load, so they are always loaded — no per-request
activation in this path.
Six unit tests cover the buildLoraExtras helper.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:regolo/glm5.2 [TOOL]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* ⬆️ Update leejet/stable-diffusion.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(stablediffusion): use effective video frame rate
Capture the frame rate selected by stable-diffusion.cpp and use it when muxing the generated frames. This keeps video timing aligned with upstream model-specific adjustments.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* ⬆️ Update mudler/vllm.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(vllm-cpp): mirror ABI v26 model params
Mirror the new KV-cache dtype and sliding-window fields so the Go POD matches the bumped vllm.cpp header on LP64.
Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* fix(whisper): honour positional listen address argument
The whisper backend parsed its gRPC listen address exclusively through
Go's flag package, while run.sh forwards launcher arguments verbatim.
A bare positional address was silently dropped by flag.Parse(), so the
server always bound the default localhost:50051 instead of the port its
caller allocated — LocalAI then failed to reach it with a misleading
'error reading from server: EOF'.
Fall back to the first positional argument when no explicit -addr value
was given, keeping the default for no-argument launches.
Fixes#11623
Assisted-by: ox-alpha:ox-alpha [go test]
Signed-off-by: Som Samantray <som.samantray@gmail.com>
* fix(whisper): track explicit -addr via flag.Visit and adopt Ginkgo test style
Review follow-up:
- Detect an explicitly set -addr with flag.FlagSet.Visit instead of
comparing against the default sentinel, so '-addr localhost:50051'
plus a positional argument keeps the flag value.
- Treat an explicitly empty -addr as unset rather than binding the
empty address (OS-chosen port on all interfaces).
- Rewrite addr_test.go as Ginkgo v2 specs per .agents/coding-style.md;
stdlib t.Run/t.Errorf are forbidden by .golangci.yml forbidigo.
Assisted-by: ox-alpha:ox-alpha [go test]
Signed-off-by: Som Samantray <som.samantray@gmail.com>
---------
Signed-off-by: Som Samantray <som.samantray@gmail.com>
Backend processes shared the host temporary directory, so crashes could leave request images and audio behind until the filesystem filled. Give each process a locked LocalAI-owned runtime, remove scratch on exit, and sweep only marked abandoned runtimes at the next start.
Also close known request error-path leaks in the Python media backends, CrispASR, LongCat Video, and stable-diffusion.cpp.
Assisted-by: Codex:gpt-5
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
RF-DETR and Locate Anything wrote each decoded request image to the OS
temporary directory. A full temporary filesystem then disabled detection,
even though both native libraries already accept encoded image buffers.
Pass decoded images directly to the native buffer APIs. This removes the
request-time disk dependency and prevents crash-orphaned image files.
Assisted-by: Codex:gpt-5
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
* ⬆️ Update leejet/stable-diffusion.cpp
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(stablediffusion): adapt streaming options
Upstream now selects segmented weight streaming automatically and removes the stream_layers field. Keep the old LocalAI option as a no-op for existing model configurations.
Assisted-by: Codex:gpt-5
---------
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
