mirror of
https://github.com/mudler/LocalAI.git
synced 2026-08-04 04:12:22 -04:00
feat: materialize Hugging Face model artifacts (#10825)
* feat(config): add model artifact source contract Assisted-by: Codex:GPT-5 [Codex] * feat(downloader): add authenticated raw-byte progress Assisted-by: Codex:GPT-5 [Codex] * feat(huggingface): resolve immutable snapshot manifests Assisted-by: Codex:GPT-5 [Codex] * feat(models): add artifact storage primitives Assisted-by: Codex:GPT-5 [Codex] * feat(models): materialize pinned Hugging Face snapshots Assisted-by: Codex:GPT-5 [Codex] * feat(models): bind managed snapshots at runtime Assisted-by: Codex:GPT-5 [Codex] * feat(gallery): materialize model artifacts during install Assisted-by: Codex:GPT-5 [Codex] * feat(gallery): declare managed Hugging Face artifacts Assisted-by: Codex:GPT-5 [Codex] * feat(models): preload managed model artifacts Assisted-by: Codex:GPT-5 [Codex] * fix(gallery): retain shared artifact caches on delete Assisted-by: Codex:GPT-5 [Codex] * feat(models): report artifact acquisition progress Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): load managed models from ModelFile Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): load staged speech model snapshots Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): use staged snapshots in engine backends Assisted-by: Codex:GPT-5 [Codex] * test(distributed): cover staged artifact snapshots Assisted-by: Codex:GPT-5 [Codex] * docs: explain managed model artifacts Assisted-by: Codex:GPT-5 [Codex] * docs: add product design context Assisted-by: Codex:GPT-5 [Codex] * feat(ui): show model artifact download progress Assisted-by: Codex:GPT-5 [Codex] * Eagerly materialize Hugging Face artifacts Materialize HF-backed model references as managed GGUF artifacts during load, with lazy download retained only as fallback. Assisted-by: Codex:GPT-5 [shell] * Refactor HF downloads through a shared executor Assisted-by: Codex:GPT-5 [shell] * drop Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
@@ -89,6 +89,62 @@ download_files:
|
||||
sha256: abc123...
|
||||
```
|
||||
|
||||
## Model artifacts
|
||||
|
||||
The `artifacts` section makes installation of a Hugging Face model eager and
|
||||
repeatable. LocalAI resolves the requested revision to an immutable commit,
|
||||
downloads the selected repository files, and commits the complete snapshot
|
||||
before the model installation succeeds.
|
||||
|
||||
```yaml
|
||||
artifacts:
|
||||
- name: model
|
||||
target: model
|
||||
source:
|
||||
type: huggingface
|
||||
repo: Qwen/Qwen3-ASR-1.7B
|
||||
revision: main
|
||||
token_env: HF_TOKEN
|
||||
resolved:
|
||||
endpoint: https://huggingface.co
|
||||
revision: 0123456789abcdef0123456789abcdef01234567
|
||||
cache_key: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
|
||||
|
||||
parameters:
|
||||
model: Qwen/Qwen3-ASR-1.7B
|
||||
```
|
||||
|
||||
Declare `source` when authoring a configuration. LocalAI owns the `resolved`
|
||||
block and writes it after installation; do not choose its values manually.
|
||||
For a public repository, omit `token_env`. For a private or gated repository,
|
||||
set it to `HF_TOKEN` and provide that environment variable to the LocalAI
|
||||
controller.
|
||||
|
||||
| Field | Meaning |
|
||||
|-------|---------|
|
||||
| `name` | Logical artifact name; `model` for the initial primary artifact |
|
||||
| `target` | Binding target; only `model` is supported initially |
|
||||
| `source.type` | `huggingface` |
|
||||
| `source.repo` | `owner/repository` or `hf://owner/repository` |
|
||||
| `source.revision` | Branch, tag, or commit; defaults to `main` and resolves to a commit |
|
||||
| `source.token_env` | Empty or `HF_TOKEN`; the secret value is never persisted |
|
||||
| `source.allow_patterns` | Optional slash-separated glob allow-list |
|
||||
| `source.ignore_patterns` | Optional slash-separated glob deny-list |
|
||||
| `resolved` | Installer-owned immutable endpoint, revision, and cache key |
|
||||
|
||||
Managed installation finishes only after every selected file is committed
|
||||
locally. `parameters.model` remains the logical repository ID. Once
|
||||
`resolved.cache_key` is present, LocalAI derives
|
||||
`.artifacts/huggingface/<cache-key>/snapshot` as the runtime `ModelFile`.
|
||||
Configurations without `artifacts` keep the existing lazy repository-ID
|
||||
behavior.
|
||||
|
||||
The initially migrated backend families are `transformers` and its aliases,
|
||||
`diffusers`, `qwen-asr`, `fish-speech`, `nemo`, `voxcpm`, `qwen-tts`,
|
||||
`liquid-audio`, `vllm`, `vllm-omni`, and `sglang`. Automatic imports add
|
||||
artifact declarations only for this set. Compatible external backends may opt
|
||||
in by declaring the artifact explicitly.
|
||||
|
||||
## Parameters Section
|
||||
|
||||
The `parameters` section contains all OpenAI-compatible request parameters and model-specific options.
|
||||
|
||||
Reference in New Issue
Block a user