mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-07-30 09:57:57 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(cli,mcp): describe variant auto-selection as preference before size

Browse Source 
The CLI flag help and the install_model tool schema both still said
auto-selection takes the largest build that runs. Ranking now puts engine
preference ahead of size, so on NVIDIA a vLLM build wins over a larger
llama.cpp one. An assistant reading the old schema would tell users the
wrong thing.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto
2026-07-19 22:07:04 +00:00
parent 24087e986c
commit c1d2947616
2 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
core/cli/models.go
View File

@@ -39,7 +39,7 @@ type ModelsInstall struct {
RequireBackendIntegrity bool `env:"LOCALAI_REQUIRE_BACKEND_INTEGRITY,REQUIRE_BACKEND_INTEGRITY" help:"If true, reject backend installs without a configured signature verification policy (OCI URIs) or SHA256 (tarball/HTTP URIs)." group:"hardening" default:"false"`
AutoloadBackendGalleries bool `env:"LOCALAI_AUTOLOAD_BACKEND_GALLERIES" help:"If true, automatically loads backend galleries" group:"backends" default:"true"`
ModelArgs []string `arg:"" optional:"" name:"models" help:"Model configuration URLs to load"`
Variant string `name:"variant" help:"Install a specific variant of a gallery entry that declares them, by the variant's model name. Leave unset to let LocalAI auto-select the largest build this machine can run." group:"models"`
Variant string `name:"variant" help:"Install a specific variant of a gallery entry that declares them, by the variant's model name. Leave unset to let LocalAI auto-select: builds this hardware cannot run or cannot fit are dropped, the engine this hardware prefers wins, and size decides among equals." group:"models"`
ModelsCMDFlags `embed:""`
}

2
pkg/mcp/localaitools/dto.go
View File

@@ -67,7 +67,7 @@ type InstallModelRequest struct {
GalleryName string `json:"gallery_name,omitempty" jsonschema:"The gallery the model lives in (from gallery_search). Optional when ModelName is unique across galleries."`
ModelName string `json:"model_name" jsonschema:"The canonical model name as returned by gallery_search."`
Overrides map[string]any `json:"overrides,omitempty" jsonschema:"Optional config overrides to merge into the installed model's YAML."`
Variant string `json:"variant,omitempty" jsonschema:"Optional. Installs one specific build of an entry that offers several (different quantizations or engines), named exactly as it appears in that entry's variant list. Leave empty unless the user explicitly asked for a particular build: by default LocalAI auto-selects the largest variant this machine's engine support and free memory can actually run."`
Variant string `json:"variant,omitempty" jsonschema:"Optional. Installs one specific build of an entry that offers several (different quantizations or engines), named exactly as it appears in that entry's variant list. Leave empty unless the user explicitly asked for a particular build: by default LocalAI drops builds this machine cannot run or cannot fit, prefers the engine this hardware is best served by, and uses size only to decide between equally preferred builds."`
}
// InstallBackendRequest is the input for install_backend.