mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-08-02 11:30:44 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat: materialize Hugging Face model artifacts (#10825)

Browse Source 
* feat(config): add model artifact source contract

Assisted-by: Codex:GPT-5 [Codex]

* feat(downloader): add authenticated raw-byte progress

Assisted-by: Codex:GPT-5 [Codex]

* feat(huggingface): resolve immutable snapshot manifests

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): add artifact storage primitives

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): materialize pinned Hugging Face snapshots

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): bind managed snapshots at runtime

Assisted-by: Codex:GPT-5 [Codex]

* feat(gallery): materialize model artifacts during install

Assisted-by: Codex:GPT-5 [Codex]

* feat(gallery): declare managed Hugging Face artifacts

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): preload managed model artifacts

Assisted-by: Codex:GPT-5 [Codex]

* fix(gallery): retain shared artifact caches on delete

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): report artifact acquisition progress

Assisted-by: Codex:GPT-5 [Codex]

* refactor(backends): load managed models from ModelFile

Assisted-by: Codex:GPT-5 [Codex]

* refactor(backends): load staged speech model snapshots

Assisted-by: Codex:GPT-5 [Codex]

* refactor(backends): use staged snapshots in engine backends

Assisted-by: Codex:GPT-5 [Codex]

* test(distributed): cover staged artifact snapshots

Assisted-by: Codex:GPT-5 [Codex]

* docs: explain managed model artifacts

Assisted-by: Codex:GPT-5 [Codex]

* docs: add product design context

Assisted-by: Codex:GPT-5 [Codex]

* feat(ui): show model artifact download progress

Assisted-by: Codex:GPT-5 [Codex]

* Eagerly materialize Hugging Face artifacts

Materialize HF-backed model references as managed GGUF artifacts during load, with lazy download retained only as fallback.

Assisted-by: Codex:GPT-5 [shell]

* Refactor HF
  downloads through a shared executor

Assisted-by: Codex:GPT-5 [shell]

* drop

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
LocalAI [bot]
2026-07-15 01:09:33 +02:00
committed by GitHub
parent d82c38ee77
commit bcc41219f7
83 changed files with 4315 additions and 339 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
backend/python/vllm-omni/backend.py
View File

@@ -33,6 +33,7 @@ import grpc
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'common'))
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'common'))
from grpc_auth import get_auth_interceptors
from model_utils import resolve_model_reference
from vllm_utils import parse_options, messages_to_dicts, setup_parsers
@@ -166,12 +167,16 @@ class BackendServicer(backend_pb2_grpc.BackendServicer):
# Parse options from request.Options using shared helper
self.options = parse_options(request.Options)
opts = self.options
model_ref, _local_only = resolve_model_reference(request)
detection_ref = request.Model or model_ref
print(f"Options: {self.options}", file=sys.stderr)
# Detect model type
self.model_name = request.Model
self.model_type = request.Type if request.Type else self._detect_model_type(request.Model)
self.model_name = model_ref
self.model_type = (
request.Type if request.Type else self._detect_model_type(detection_ref)
)
print(f"Detected model type: {self.model_type}", file=sys.stderr)
# Build DiffusionParallelConfig if diffusion model (image or video)
@@ -205,9 +210,7 @@ class BackendServicer(backend_pb2_grpc.BackendServicer):
}
# Base Omni initialization parameters
omni_kwargs = {
"model": request.Model,
}
omni_kwargs = {"model": model_ref}
# Add diffusion-specific parameters (image/video models)
if self.model_type in ["image", "video"]:
@@ -251,7 +254,7 @@ class BackendServicer(backend_pb2_grpc.BackendServicer):
try:
from vllm.transformers_utils.tokenizer import get_tokenizer
self.tokenizer = get_tokenizer(
request.Model,
model_ref,
trust_remote_code=opts.get("trust_remote_code", False),
)
except Exception as e: