7 Commits

Author	SHA1	Message	Date
mudler's LocalAI [bot]	ec49548c8e	fix(modelartifacts): resume interrupted materialization per-file, not from scratch (#11071) ... materializeLocked built a download task for every file in the resolved snapshot unconditionally. A completed file is promoted from .downloads/<hash> into snapshot/<path> and its blob deleted, so on any re-entry (a controller pod roll, a resubmit, a crash) the new pass built a task whose .downloads blob no longer existed, re-downloaded the whole file from Hugging Face, and its AfterDownload even removed the already-complete snapshot copy first. The only resume that worked was the downloader's per-file .partial resume for a file caught mid-transfer; completed files were never skipped. Production consequence: installing longcat-video-avatar-1.5 (~35 GB after allow_patterns) on a cluster whose controller rolls hourly (Flux image automation) never converged across ~14 hours. Each roll restarted from the first shard; the completed bytes on disk were repeatedly deleted and re-fetched, and the artifact never promoted. curl of the same files from inside the pod ran fine, proving the loss was the materializer re-fetching, not the network. Before building a task, check whether the file is already materialized and verified in this staging tree's snapshot/ and, if so, keep it and count it complete instead of downloading. "Materialized" means a regular file of the expected size that passes the same verifyDownloadedFile check the download path uses, so the kept manifest entry is byte-for-byte identical to a fresh one and integrity is re-checked. The manifest requires a SHA-256 for every file and non-LFS files carry none to borrow, so a hash is unavoidable for the manifest anyway; a full re-hash of local disk is still orders of magnitude cheaper than re-downloading, and the downloader re-verifies any file it does fetch. Manifest entries are now written at their snapshot index rather than appended in completion order, so a mix of skipped and downloaded files keeps the resolved order that committedResult and staging read. The unconditional root.Remove(destination) now runs only on the fresh-download path; a kept file survives. Skips are logged at INFO with count and bytes so an operator can see resume working. This is the resume-side counterpart to the sibling defects on this path: read/write error conflation and transient retry (#10985), hash-verify progress accounting and silent success on an expired deadline (#11026), and the response-header hang (#11053). The download machinery resumed a single in-flight file; the materializer above it still threw away every completed file on restart. It also makes orphan-partial adoption worth its cost: an adopted tree's completed files were re-downloaded anyway until now. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-23 10:48:01 +02:00
mudler's LocalAI [bot]	f01038f479	fix(modelartifacts): stage each writer's artifact in its own partial tree (#10995) ... Every writer used to stage into the same `.artifacts/.partial/<cacheKey>`. That was safe only because the artifact lock held: two writers that both believed they had it opened the same blob with O_APPEND and interleaved their bytes into one file, while the resume probe read the other writer's in-flight size. SHA verification caught the damage only after both had burned the entire download. #10986 restored the lock's precondition on CIFS but left the dependency in place. Suffix the staging tree with a writer identity drawn once per process run, so concurrent writers cannot corrupt each other whatever the lock does. The lock stops being a correctness dependency and becomes a pure efficiency optimisation: a lock failure now costs a duplicated download, not a corrupted one. Commit stays an atomic rename. The loser of a commit race reconciles onto the winner's tree instead of surfacing a bare ENOTEMPTY for work that actually succeeded, since the artifact is content-addressed and both trees hold the same verified bytes. Writer-unique staging means a crashed writer's tree is no longer overwritten by its successor, so two things are added to keep it from becoming a disk leak and a resume regression: - A sweep reclaims trees whose contents have been untouched for 24h, matching the window the startup reaper already uses for stray *.partial files. It reads the newest mtime anywhere inside the tree, because writing a blob never touches an ancestor, and refuses any name this package did not write. A live download writes continuously, and the downloader's stall watchdog aborts a silent one long before it could look abandoned. - Adoption lets a restarted process claim a dead predecessor's tree for the same artifact and resume from its bytes, which a tens-of-gigabytes repo depends on. The claim is an atomic rename, so racing adopters cannot both win. It runs only under the artifact lock - which is released exactly when the owning process dies - and only on a tree idle for 5 minutes as a second line of defence for when the lock does not exclude. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-20 23:07:43 +02:00
mudler's LocalAI [bot]	2f33ad6669	fix(modelartifacts): treat CIFS EACCES as lock contention, not failure (#10986) ... flock(2) on CIFS/SMB returns EACCES when another client holds the lock: the kernel maps STATUS_LOCK_NOT_GRANTED and STATUS_FILE_LOCK_CONFLICT to -EACCES and never produces EWOULDBLOCK on that path. gofrs/flock only recognises EWOULDBLOCK as contention, so TryLockContext returned a bare "permission denied" and Ensure aborted. Both replicas then fell back to legacy loading, which makes the worker download the whole repo in-band inside LoadModel and blow the remote-load deadline. Replace TryLockContext with an explicit wait loop over a new Locker interface, classifying EWOULDBLOCK/EAGAIN/EACCES/EBUSY as contention. EACCES is ambiguous at the syscall boundary but not here: the lock file is already open O_CREATE|O_RDWR, so a real permission problem would have failed the open with an *fs.PathError, and flock(2) documents no EACCES on Linux at all. The wait is bounded (DefaultLockWait, overridable via WithLockWait), so even a misclassification degrades to a delay. On timeout the committed result is re-checked before reporting the new ErrLockContended, so a peer that finished the work still wins. Locker also exists so the contention path is testable without a network filesystem: nothing in CI can make flock(2) return EACCES on demand. Raise the fallback to error for a managedArtifactBackends backend, via a shared config.LogArtifactFallback used by both call sites. For those backends the legacy path is not graceful degradation, and the operator otherwise sees only a timeout with no causal link. The fallback stays non-fatal. Drop the os.Chmod(layout.Lock, 0o600) after acquisition: flock.New already creates the file 0600, and the chmod was gratuitous risk on a nounix mount that ignores modes. Fixes #10981 Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-20 22:12:39 +02:00
mudler's LocalAI [bot]	626ae4d51e	fix(model-artifacts): materialize longcat-video on the controller, and support companion repos (#10949) ... * fix(model-artifacts): materialize longcat-video checkpoints on the controller longcat-video loads a checkpoint directory: its backend.py takes request.ModelFile when os.path.isdir(request.ModelFile) and otherwise falls back to snapshot_download. That places it in the same class as transformers/vllm/diffusers/sglang, but the allow-list added in #10910 did not enumerate it, so PrimaryArtifactSpec returned no managed artifact for a bare HuggingFace repo id. The consequence in distributed mode: nothing was acquired on the controller, ModelFileName fell through to the raw repo id, and staging skipped the resulting phantom /models/<owner>/<repo> path. The worker received a blank ModelFile, fell back to request.Model, and downloaded ~83GB from HuggingFace inside the remote LoadModel deadline - so the load could only ever fail with DeadlineExceeded while an abandoned backend process kept downloading. Note this materializes the full repository. The backend restricts its own snapshot_download with allow_patterns, and the avatar repo ships both base_model/ and base_model_int8/ where only one is ever loaded; inferred specs have no way to carry patterns today. Tracked separately. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): warn when staging skips a non-existent model path stageModelFiles logs "Staging model files for remote node" up front, then silently drops any path field that does not exist on the controller. The skip itself is legitimate and must stay: a backend outside managedArtifactBackends that takes a bare HuggingFace repo id gets an optimistically constructed path (ModelFileName falls through to the raw model reference) that was never materialized, and sources its own weights on the worker. Erroring would break those configs. But at debug level the operator is left with a reassuring staging line and no trace of the skip, so a genuine controller-side acquisition gap is indistinguishable from a healthy pass-through - it surfaces much later as a remote LoadModel timeout, on a worker that is quietly downloading tens of gigabytes. Raise the skip to warn and name the field, path, node and tracking key. Behavior is unchanged. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(model-artifacts): allow a config to declare companion artifacts A composed pipeline needs more than one HuggingFace snapshot. LongCat-Video-Avatar-1.5 loads its own transformer but takes the tokenizer, text encoder and VAE from the separate LongCat-Video base repo, so a single-artifact config cannot express it and the backend is left to fetch the second repo itself at load time. Widen the artifact model to target: model plus any number of named target: companion entries. Normalize accepts the new target and constrains a companion name to [a-z0-9][a-z0-9_-]{0,63} because that name is the option key the backend later receives; a companion may not claim primary_file, which only means anything for a load target. ModelConfig.Validate requires exactly one primary and requires it first, since Artifacts[0] is what ModelFileName, size estimation and staging all resolve from. Both acquisition paths now loop instead of touching index 0 alone: preloadOne for an already-installed config, bindPrimaryArtifact for a gallery install. Failure policy differs by provenance. An inferred primary keeps its warn-and-fall-back, because the legacy download path still exists for it. Companions are explicit by construction, so they are all-or-nothing: a config naming one is asserting the backend needs it, and failing at the acquisition boundary is far more legible than a missing-weights error surfacing later inside the backend. The cache key is deliberately unchanged. It hashes source identity only, never name or target, so every already-installed managed model still hits its existing snapshot instead of silently re-downloading. Two specs pin that: one proving a companion and a primary with identical sources agree on the key, and one pinning the digest of a known primary outright. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(model-artifacts): hand resolved companion snapshots to the backend A materialized companion is useless until the backend can find it, and its location is a content-addressed cache key that does not exist until the artifact resolves. A static gallery override cannot carry that, and persisting it into the config YAML would rot the moment a re-resolve produced a new key. Synthesize it instead at load time: each resolved companion becomes "<artifact name>:<snapshot path>" in ModelOptions.Options, reusing the key:value convention backends already parse for options like attention_backend. The value stays relative to the models directory so a remote worker can resolve it under its own ModelPath once staging has rewritten the model root. An option the author set explicitly always wins, so pinning a companion to a local checkout still beats the managed snapshot. longcat-video resolves base_model through ModelPath, the same convention qwen-tts, voxcpm, outetts and ace-step already use for companion assets. Its sibling-directory heuristic is deleted: it looked for a LongCat-Video directory next to the model, which cannot exist under the content addressed .artifacts/huggingface/<key>/snapshot layout, so it was dead code the moment the model became managed. The gallery entry declares both repositories and restricts each with allow_patterns. The avatar repo ships base_model/ and base_model_int8/ and only ever loads one, so fetching the whole repo would roughly double the download. The patterns match the entry's own options (use_distill true, use_int8 default false); enabling use_int8 here also requires adding base_model_int8/**, which is called out in the entry. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(distributed): stage managed artifact trees from the models root Staging anchored the worker's models directory on the primary snapshot whenever a model was managed, so a companion snapshot could not reach the worker at all. frontendModelsDir was derived by stripping the Model relative path off the end of ModelFile. For a managed artifact nothing matches: ModelFile is .artifacts/huggingface/<key>/snapshot while Model stays a bare HuggingFace repo id, so the strip was a no-op and the "models directory" came out as the snapshot itself. Two consequences, both silent. Staging keys lost the .artifacts/huggingface/<key>/snapshot prefix, so two snapshots of one model were indistinguishable on the worker. And a companion, which lives in a sibling snapshot directory outside the primary, fell outside that directory entirely: StagingKeyMapper.Key collapsed its files to bare basenames and resolveOptionPath could not resolve the relative option at all, so it was skipped without a word. Derive the models root from the artifact tree instead when the path runs through it, and compute the worker's ModelPath from the file's path relative to that root rather than from the Model field. The legacy layout is unaffected: where Model really is the relative path, the new derivation reduces to the old one, which a regression spec pins. This deliberately changes an invariant that router_dirstage_test.go pinned: for a managed primary, ModelFile and ModelPath were both the snapshot directory, and staging keys were relative to it. Now ModelFile is the snapshot, ModelPath is the models root above it, and keys keep the full relative path. That spec is updated rather than accommodated, with the reasoning recorded inline, because the old invariant is exactly what made a sibling companion unreachable. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-19 12:01:36 +02:00
localai-org-maint-bot	279f5b8a93	fix(model-artifacts): load single-file HF snapshots from the file, not the directory (#10909) ... fix(model-artifacts): load single-file HF snapshots from the file, not the dir The managed Hugging Face artifact materializer (#10825) always pointed backends at the snapshot *directory* (.artifacts/huggingface/<key>/snapshot). For a single-file model reference such as huggingface://nomic-ai/nomic-embed-text-v1.5-GGUF/nomic-embed-text-v1.5.f16.gguf, the GGUF lives *inside* that directory, so llama.cpp was handed a directory and failed with "gguf_init_from_reader: failed to read magic". This has kept the tests-aio job red on master since the feature merged (the embeddings e2e tests could not load text-embedding-ada-002). Record the single file of a one-file snapshot as Resolved.PrimaryFile and have ModelFileName() resolve to snapshot/<PrimaryFile> when it is set. Multi-file snapshots (e.g. transformers repos consumed as a directory) keep pointing at the snapshot directory. PrimaryFile is derived from the resolved contents and is deliberately excluded from the artifact cache key. estimateModelSizeBytes now derives the snapshot directory from the cache key instead of ModelFileName(), so its manifest lookup is unaffected by the file-vs-directory resolution. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-17 22:42:50 +00:00
futurehua	d3ea65a112	refactor: replace Split in loops with more efficient SplitSeq (#10879) ... Signed-off-by: futurehua <futurehua@outlook.com>	2026-07-17 22:07:16 +02:00
LocalAI [bot]	bcc41219f7	feat: materialize Hugging Face model artifacts (#10825) ... * feat(config): add model artifact source contract Assisted-by: Codex:GPT-5 [Codex] * feat(downloader): add authenticated raw-byte progress Assisted-by: Codex:GPT-5 [Codex] * feat(huggingface): resolve immutable snapshot manifests Assisted-by: Codex:GPT-5 [Codex] * feat(models): add artifact storage primitives Assisted-by: Codex:GPT-5 [Codex] * feat(models): materialize pinned Hugging Face snapshots Assisted-by: Codex:GPT-5 [Codex] * feat(models): bind managed snapshots at runtime Assisted-by: Codex:GPT-5 [Codex] * feat(gallery): materialize model artifacts during install Assisted-by: Codex:GPT-5 [Codex] * feat(gallery): declare managed Hugging Face artifacts Assisted-by: Codex:GPT-5 [Codex] * feat(models): preload managed model artifacts Assisted-by: Codex:GPT-5 [Codex] * fix(gallery): retain shared artifact caches on delete Assisted-by: Codex:GPT-5 [Codex] * feat(models): report artifact acquisition progress Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): load managed models from ModelFile Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): load staged speech model snapshots Assisted-by: Codex:GPT-5 [Codex] * refactor(backends): use staged snapshots in engine backends Assisted-by: Codex:GPT-5 [Codex] * test(distributed): cover staged artifact snapshots Assisted-by: Codex:GPT-5 [Codex] * docs: explain managed model artifacts Assisted-by: Codex:GPT-5 [Codex] * docs: add product design context Assisted-by: Codex:GPT-5 [Codex] * feat(ui): show model artifact download progress Assisted-by: Codex:GPT-5 [Codex] * Eagerly materialize Hugging Face artifacts Materialize HF-backed model references as managed GGUF artifacts during load, with lazy download retained only as fallback. Assisted-by: Codex:GPT-5 [shell] * Refactor HF downloads through a shared executor Assisted-by: Codex:GPT-5 [shell] * drop Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>	2026-07-15 01:09:33 +02:00