Files
LocalAI/pkg/modelartifacts/manifest.go
LocalAI [bot] bcc41219f7 feat: materialize Hugging Face model artifacts (#10825)
* feat(config): add model artifact source contract

Assisted-by: Codex:GPT-5 [Codex]

* feat(downloader): add authenticated raw-byte progress

Assisted-by: Codex:GPT-5 [Codex]

* feat(huggingface): resolve immutable snapshot manifests

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): add artifact storage primitives

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): materialize pinned Hugging Face snapshots

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): bind managed snapshots at runtime

Assisted-by: Codex:GPT-5 [Codex]

* feat(gallery): materialize model artifacts during install

Assisted-by: Codex:GPT-5 [Codex]

* feat(gallery): declare managed Hugging Face artifacts

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): preload managed model artifacts

Assisted-by: Codex:GPT-5 [Codex]

* fix(gallery): retain shared artifact caches on delete

Assisted-by: Codex:GPT-5 [Codex]

* feat(models): report artifact acquisition progress

Assisted-by: Codex:GPT-5 [Codex]

* refactor(backends): load managed models from ModelFile

Assisted-by: Codex:GPT-5 [Codex]

* refactor(backends): load staged speech model snapshots

Assisted-by: Codex:GPT-5 [Codex]

* refactor(backends): use staged snapshots in engine backends

Assisted-by: Codex:GPT-5 [Codex]

* test(distributed): cover staged artifact snapshots

Assisted-by: Codex:GPT-5 [Codex]

* docs: explain managed model artifacts

Assisted-by: Codex:GPT-5 [Codex]

* docs: add product design context

Assisted-by: Codex:GPT-5 [Codex]

* feat(ui): show model artifact download progress

Assisted-by: Codex:GPT-5 [Codex]

* Eagerly materialize Hugging Face artifacts

Materialize HF-backed model references as managed GGUF artifacts during load, with lazy download retained only as fallback.

Assisted-by: Codex:GPT-5 [shell]

* Refactor HF
  downloads through a shared executor

Assisted-by: Codex:GPT-5 [shell]

* drop

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-07-15 01:09:33 +02:00

48 lines
1.2 KiB
Go

package modelartifacts
import (
"encoding/json"
"fmt"
"os"
)
const ManifestVersion = 1
type Manifest struct {
Version int `json:"version"`
Artifact Spec `json:"artifact"`
Files []ManifestFile `json:"files"`
}
type ManifestFile struct {
Path string `json:"path"`
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
BlobOID string `json:"blob_oid,omitempty"`
LFSOID string `json:"lfs_oid,omitempty"`
XetHash string `json:"xet_hash,omitempty"`
}
func ReadManifest(fileName string) (Manifest, error) {
data, err := os.ReadFile(fileName)
if err != nil {
return Manifest{}, err
}
var manifest Manifest
if err := json.Unmarshal(data, &manifest); err != nil {
return Manifest{}, err
}
if manifest.Version != ManifestVersion || manifest.Artifact.Resolved == nil {
return Manifest{}, fmt.Errorf("unsupported or incomplete artifact manifest")
}
for _, file := range manifest.Files {
if err := ValidateRelativeHubPath(file.Path); err != nil {
return Manifest{}, err
}
if file.Size < 0 || len(file.SHA256) != 64 {
return Manifest{}, fmt.Errorf("invalid manifest entry %q", file.Path)
}
}
return manifest, nil
}