mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 18:09:05 -04:00
1df5a3ef7f
Follow-up to the companion-persistence fix. On a live distributed cluster the managed base_model companion option still failed to reach the remote worker (nvidia-thor) even with the companion resolved-and-persisted in the config: the backend logged "Downloading required files for meituan-longcat/LongCat-Video" and failed "base_model must point to a LongCat-Video checkpoint". Symlinking the companion into place did not help (the option was simply absent from the worker's LoadModel), while an explicit absolute base_model in options: worked as a control. Trace of where the remote ModelOptions is built and whether the companion is present there: - The *pb.ModelOptions the worker's LoadModel consumes is built on the CONTROLLER by grpcModelOpts (core/backend/options.go) -> withCompanionArtifactOptions, set as gRPCOptions, and sent by direct gRPC via FileStagingClient.LoadModel. It is NOT rebuilt on the worker. The reconciler's replica scale-up instead replays a Postgres-stored proto blob, which already carries whatever grpcModelOpts produced. - withCompanionArtifactOptions is the ONLY builder of ModelOptions.Options in the tree, and it emits base_model iff the config's companion artifact has Resolved != nil. Staging preserves the option and derives the worker ModelPath as the nested per-model staged root, so a resolved companion resolves under it without a download (verified end to end; the path-nesting angle is a red herring here). So the option is absent only when the config the loader is serving from carries the companion WITHOUT a resolved snapshot (its resolved state not reaching the serving config, e.g. a peer-replica reload from local disk or a config loaded before resolution). In that state the old code emitted NOTHING for the companion, and longcat-video fell back to its OWN hardcoded default (BASE_MODEL_ID), which is exactly the observed download-and-fail. Fix: an unresolved-but-declared companion no longer vanishes. It now falls back to its DECLARED source repository id, so the backend fetches the artifact the config actually asked for instead of a hardcoded default; the resolved snapshot path (the staged, no-download fast path) is still preferred whenever the companion is resolved, so the single-node and healthy distributed paths are unchanged. The fallback logs a warning naming the artifact and repo, and the router now logs the exact option strings crossing to the worker at debug, so a recurrence is diagnosable in one load instead of by inference. Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>