LocalAI [bot] 0d57957ebb feat(worker): add LOCALAI_PREFETCH_MODELS for boot-time gallery prefetch (#10108) ...

In LocalAI distributed mode the master streams a model GGUF to a
worker on first inference. On bandwidth-constrained cluster networks
(libp2p circuit-v2 relays under NAT, double-NAT residential, slow
overlays) that transfer can be slow or unreliable — meanwhile each
worker's outbound internet is usually fine.

LOCALAI_PREFETCH_MODELS lets the operator name gallery model IDs to
download at worker boot, BEFORE the worker subscribes to backend.install
events. Reuses gallery.InstallModelFromGallery so the on-disk /models
layout matches what the master would have pushed, and the master can
still push files on demand if the gallery is unreachable at boot
(prefetch is non-fatal on every error path).

The installer is wrapped in a function-value indirection so tests can
swap a fake without touching the real gallery; production never
reassigns the binding.

Assisted-by: Claude:claude-opus-4-7

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

2026-05-31 12:22:45 +02:00

..

application

feat: prefix-cache-aware routing for distributed mode (#10071)

2026-05-30 23:24:22 +02:00

backend

feat: prefix-cache-aware routing for distributed mode (#10071)

2026-05-30 23:24:22 +02:00

cli

feat(ds4): layer-split distributed inference (#10098)

2026-05-31 00:09:55 +02:00

clients

security(http): refuse redirects on outbound clients via hardened pkg/httpclient (#10087)

2026-05-30 12:04:10 +02:00

config

fix(config): register parakeet-cpp as a transcript backend (#9718) (#10106)

2026-05-31 11:15:15 +02:00

dependencies_manager

feat(ui): move to React for frontend (#8772)

2026-03-05 21:47:12 +01:00

explorer

feat(middleware): Model routing, PII filtering, Cloud model proxies (#9802)

2026-05-25 09:28:27 +02:00

gallery

feat(parakeet-cpp): add NVIDIA NeMo Parakeet ASR backend (parakeet.cpp) (#10084)

2026-05-30 14:46:10 +02:00

http

feat(crispasr): add CrispASR backend — multi-architecture ASR + TTS (#10099)

2026-05-31 12:11:03 +02:00

p2p

feat: add distributed mode (#9124)

2026-03-30 00:47:27 +02:00

schema

fix(openresponses): populate Content and accept bare {role,content} items (#10039) (#10040)

2026-05-28 07:21:48 +00:00

services

feat(worker): add LOCALAI_PREFETCH_MODELS for boot-time gallery prefetch (#10108)

2026-05-31 12:22:45 +02:00

startup

feat(gallery): verify backend OCI images with keyless cosign (#9823)

2026-05-18 08:02:20 +02:00

templates

fix(openresponses): populate Content and accept bare {role,content} items (#10039) (#10040)

2026-05-28 07:21:48 +00:00

trace

feat(middleware): Model routing, PII filtering, Cloud model proxies (#9802)

2026-05-25 09:28:27 +02:00