mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-06-21 23:29:04 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(backend): call vram.EstimateModelMultiContext (master build broken: undefined vram.EstimateModel) (#10426)

Browse Source 
fix(backend): call vram.EstimateModelMultiContext for model size estimate

core/backend/options.go called vram.EstimateModel, which does not exist in
the vram package (it exposes EstimateModelMultiContext). This broke the build
on master (undefined: vram.EstimateModel). Use EstimateModelMultiContext with
a nil context-size slice (defaults to a single 8192 estimate); the returned
MultiContextEstimate.SizeBytes is exactly what the caller consumes, so size
estimation behavior is unchanged.

Assisted-by: Claude:claude-opus-4-8

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
LocalAI [bot]
2026-06-21 17:51:46 +02:00
committed by GitHub
parent aed181e6c1
commit 7b462a0d51
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
core/backend/options.go
View File

@@ -90,7 +90,7 @@ func estimateModelSizeBytes(c config.ModelConfig, modelsPath string) int64 {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
result, err := vram.EstimateModel(ctx, input)
result, err := vram.EstimateModelMultiContext(ctx, input, nil)
if err != nil || result.SizeBytes == 0 {
return 0
}