Files
LocalAI/pkg/vrambudget/budget_test.go
LocalAI [bot] 8cec22c3b7 feat(vram): per-node VRAM allocation budget (LOCALAI_VRAM_BUDGET) (#10833)
* feat(vram): add vrambudget primitive for per-node VRAM caps

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): apply default VRAM budget in xsysinfo aggregate getters

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): wire LOCALAI_VRAM_BUDGET flag to xsysinfo default budget

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): persist VRAM budget via runtime settings with live apply

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(vram): reset process-global VRAM budget after runtime-settings spec

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): add VRAM budget field to Settings page

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): store and enforce per-node VRAM budget in the node registry

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): apply per-node VRAM budget in router hardware defaults

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): report worker VRAM budget in node registration

The distributed worker now reports its operator-set VRAM budget string
(LOCALAI_VRAM_BUDGET) to the server on registration. The worker keeps
reporting RAW total/available VRAM and never sets the xsysinfo
process-global budget (that stays standalone-only); the server resolves
and enforces the budget uniformly (Task 6).

Also closes a Task 6 gap: on re-registration, a struct Updates zero-skips
an empty budget, so a worker that dropped LOCALAI_VRAM_BUDGET left the
stale cap in place. For non-admin-override nodes the budget columns are
now force-written (map Updates) even when empty, so removing the env var
clears the cap; admin overrides are preserved unchanged.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* style(vram): drop em dash from worker-clear comment

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): add node VRAM budget admin endpoints

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): add node VRAM budget control to the node UI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): expose set_node_vram_budget MCP admin tool

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs(vram): document LOCALAI_VRAM_BUDGET and node VRAM budget UI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(vram): avoid double-applying VRAM budget in GetResourceAggregateInfo

The GPU-branch aggregate returned by GetResourceInfo is sourced from
GetGPUAggregateInfo, which already caps total/free/used against the
process-wide VRAM budget. GetResourceAggregateInfo then applied the
budget a second time. For an absolute budget this is idempotent, but for
a percentage budget b.Apply resolves the ceiling as a fraction of its
input total, so a second pass yields P*(P*T) instead of P*T and distorts
UsagePercent (read by the memory reclaimer in pkg/model/watchdog.go).

Remove the redundant second application so the budget is applied exactly
once, against the raw physical totals, upstream in GetGPUAggregateInfo.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(vram): implement SetNodeVRAMBudget on mcp assistant test stub

The LocalAIClient interface gained SetNodeVRAMBudget; the stubClient in
core/http/endpoints/mcp used by the assistant tests is a separate
implementer and needs the method too (broke golangci-lint typecheck and
both test jobs).

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-07-15 09:58:45 +02:00

103 lines
2.9 KiB
Go

package vrambudget_test
import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"github.com/mudler/LocalAI/pkg/vrambudget"
)
const gb = uint64(1000 * 1000 * 1000)
var _ = Describe("Budget", func() {
Describe("Parse", func() {
It("treats empty/zero as unset (no error)", func() {
for _, s := range []string{"", " ", "0", "0%", "0b"} {
b, err := vrambudget.Parse(s)
Expect(err).NotTo(HaveOccurred())
Expect(b.IsSet()).To(BeFalse(), "input %q", s)
}
})
It("parses percentage forms", func() {
for _, s := range []string{"80%", "0.8"} {
b, err := vrambudget.Parse(s)
Expect(err).NotTo(HaveOccurred())
Expect(b.IsSet()).To(BeTrue())
total, free := b.Apply(10*gb, 10*gb)
Expect(total).To(Equal(8 * gb))
Expect(free).To(Equal(8 * gb))
}
})
It("treats 100% / 1.0 as a valid no-op ceiling", func() {
b, err := vrambudget.Parse("100%")
Expect(err).NotTo(HaveOccurred())
total, free := b.Apply(10*gb, 6*gb)
Expect(total).To(Equal(10 * gb))
Expect(free).To(Equal(6 * gb))
})
It("parses absolute forms with decimal and binary suffixes and raw bytes", func() {
cases := map[string]uint64{
"12GB": 12 * gb,
"12000MB": 12 * gb,
"12000000000": 12 * gb,
"12GiB": 12 * 1024 * 1024 * 1024,
}
for s, want := range cases {
b, err := vrambudget.Parse(s)
Expect(err).NotTo(HaveOccurred(), "input %q", s)
Expect(b.Ceiling(64 * gb)).To(Equal(want), "input %q", s)
}
})
It("rejects malformed and out-of-range input", func() {
for _, s := range []string{"120%", "1.5", "-1", "abc", "12ZB", "%", "12 GB x"} {
_, err := vrambudget.Parse(s)
Expect(err).To(HaveOccurred(), "input %q", s)
}
})
})
Describe("Apply", func() {
It("is a passthrough when unset", func() {
b, _ := vrambudget.Parse("")
total, free := b.Apply(16*gb, 9*gb)
Expect(total).To(Equal(16 * gb))
Expect(free).To(Equal(9 * gb))
})
It("caps total and free to the budget ceiling (hard ceiling)", func() {
b, _ := vrambudget.Parse("12GB")
total, free := b.Apply(16*gb, 15*gb)
Expect(total).To(Equal(12 * gb))
Expect(free).To(Equal(12 * gb))
})
It("never raises free above detected when free is already below budget", func() {
b, _ := vrambudget.Parse("12GB")
_, free := b.Apply(16*gb, 5*gb)
Expect(free).To(Equal(5 * gb))
})
It("clamps an absolute budget above physical to detected", func() {
b, _ := vrambudget.Parse("64GB")
total, free := b.Apply(16*gb, 10*gb)
Expect(total).To(Equal(16 * gb))
Expect(free).To(Equal(10 * gb))
})
})
Describe("String", func() {
It("round-trips percentage and absolute canonical forms", func() {
p, _ := vrambudget.Parse("80%")
Expect(p.String()).To(Equal("80%"))
a, _ := vrambudget.Parse("12000MB")
Expect(a.String()).To(Equal("12GB"))
z, _ := vrambudget.Parse("")
Expect(z.String()).To(Equal(""))
})
})
})