Files
LocalAI/pkg/mcp/localaitools/coverage_test.go
LocalAI [bot] 8cec22c3b7 feat(vram): per-node VRAM allocation budget (LOCALAI_VRAM_BUDGET) (#10833)
* feat(vram): add vrambudget primitive for per-node VRAM caps

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): apply default VRAM budget in xsysinfo aggregate getters

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): wire LOCALAI_VRAM_BUDGET flag to xsysinfo default budget

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): persist VRAM budget via runtime settings with live apply

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* test(vram): reset process-global VRAM budget after runtime-settings spec

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): add VRAM budget field to Settings page

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): store and enforce per-node VRAM budget in the node registry

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): apply per-node VRAM budget in router hardware defaults

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): report worker VRAM budget in node registration

The distributed worker now reports its operator-set VRAM budget string
(LOCALAI_VRAM_BUDGET) to the server on registration. The worker keeps
reporting RAW total/available VRAM and never sets the xsysinfo
process-global budget (that stays standalone-only); the server resolves
and enforces the budget uniformly (Task 6).

Also closes a Task 6 gap: on re-registration, a struct Updates zero-skips
an empty budget, so a worker that dropped LOCALAI_VRAM_BUDGET left the
stale cap in place. For non-admin-override nodes the budget columns are
now force-written (map Updates) even when empty, so removing the env var
clears the cap; admin overrides are preserved unchanged.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* style(vram): drop em dash from worker-clear comment

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): add node VRAM budget admin endpoints

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): add node VRAM budget control to the node UI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* feat(vram): expose set_node_vram_budget MCP admin tool

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* docs(vram): document LOCALAI_VRAM_BUDGET and node VRAM budget UI

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(vram): avoid double-applying VRAM budget in GetResourceAggregateInfo

The GPU-branch aggregate returned by GetResourceInfo is sourced from
GetGPUAggregateInfo, which already caps total/free/used against the
process-wide VRAM budget. GetResourceAggregateInfo then applied the
budget a second time. For an absolute budget this is idempotent, but for
a percentage budget b.Apply resolves the ceiling as a fraction of its
input total, so a second pass yields P*(P*T) instead of P*T and distorts
UsagePercent (read by the memory reclaimer in pkg/model/watchdog.go).

Remove the redundant second application so the budget is applied exactly
once, against the raw physical totals, upstream in GetGPUAggregateInfo.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(vram): implement SetNodeVRAMBudget on mcp assistant test stub

The LocalAIClient interface gained SetNodeVRAMBudget; the stubClient in
core/http/endpoints/mcp used by the assistant tests is a separate
implementer and needs the method too (broke golangci-lint typecheck and
both test jobs).

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-07-15 09:58:45 +02:00

103 lines
4.6 KiB
Go

package localaitools
import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// toolToHTTPRoute is the canonical mapping between MCP tools and the
// LocalAI admin REST endpoints they wrap. The httpapi.Client MUST hit the
// listed route for the tool; the inproc.Client may bypass HTTP and call
// services directly, but the on-the-wire shape is documented here so the
// two sides stay aligned.
//
// Updating the map is REQUIRED when:
// - You add a Tool* constant (tools.go).
// - You change which REST endpoint the httpapi.Client calls.
//
// The TestToolHTTPRouteMappingComplete spec below FAILS until every Tool*
// is in the map. That is the drift detector — see
// .agents/localai-assistant-mcp.md for the contributor contract.
//
// "(none)" is a deliberate sentinel for tools whose data is not exposed
// over a single REST endpoint (e.g. system_info aggregates data the
// inproc client picks up directly from services). The httpapi.Client may
// approximate via the welcome JSON; the test still requires an entry so
// the contributor explicitly acknowledges the asymmetry.
var toolToHTTPRoute = map[string]string{
// Read-only tools.
ToolGallerySearch: "GET /models/available",
ToolListInstalledModels: "GET / (welcome JSON, ModelsConfig field)",
ToolListGalleries: "GET /models/galleries",
ToolGetJobStatus: "GET /models/jobs/:uuid",
ToolGetModelConfig: "(none) — no JSON-only REST yet; httpapi.Client returns a documented stub",
ToolListBackends: "GET /backends",
ToolListKnownBackends: "GET /backends/known",
ToolSystemInfo: "GET / (welcome JSON)",
ToolListNodes: "GET /api/nodes",
ToolVRAMEstimate: "POST /api/models/vram-estimate",
ToolGetBranding: "GET /api/branding",
ToolGetUsageStats: "GET /api/usage (or /api/usage/all when all=true)",
ToolGetPIIEvents: "GET /api/pii/events",
ToolGetMiddlewareStatus: "GET /api/middleware/status",
ToolGetRouterDecisions: "GET /api/router/decisions",
ToolListAliases: "GET /api/aliases",
ToolListVoiceProfiles: "GET /api/voice-profiles",
// Mutating tools.
ToolInstallModel: "POST /models/apply",
ToolImportModelURI: "POST /models/import-uri",
ToolDeleteModel: "POST /models/delete/:name",
ToolEditModelConfig: "PATCH /api/models/config-json/:name",
ToolReloadModels: "POST /models/reload",
ToolLoadModel: "POST /backend/load",
ToolInstallBackend: "POST /backends/apply",
ToolUpgradeBackend: "POST /backends/upgrade/:name",
ToolToggleModelState: "PUT /models/toggle-state/:name/:action",
ToolToggleModelPinned: "PUT /models/toggle-pinned/:name/:action",
ToolSetBranding: "POST /api/settings (instance_name, instance_tagline)",
ToolSetAlias: "PATCH /api/models/config-json/:name (swap) or POST /models/import (create)",
ToolCreateVoiceProfile: "POST /api/voice-profiles",
ToolDeleteVoiceProfile: "DELETE /api/voice-profiles/:id",
ToolSetNodeVRAMBudget: "PUT /api/nodes/:id/vram-budget",
}
// allKnownTools is the union of expectedFullCatalog (defined in
// server_test.go). Keeping a single source of truth — the slice from
// server_test — and asserting the route map covers every entry catches
// the case "you added a Tool* but forgot to register it as MCP" indirectly
// (it'd be missing from expectedFullCatalog, which has its own assertion
// in TestServerRegistersExpectedToolCatalog).
var _ = Describe("Tool ↔ HTTP route coverage map", func() {
It("has an entry for every Tool* in the published catalog", func() {
for _, name := range expectedFullCatalog {
_, ok := toolToHTTPRoute[name]
Expect(ok).To(BeTrue(),
"Tool %q is in expectedFullCatalog but not in toolToHTTPRoute. "+
"When adding an MCP tool, update toolToHTTPRoute in coverage_test.go "+
"with the REST endpoint the httpapi.Client calls (or '(none)' with a reason).",
name)
}
})
It("does not document tools that no longer exist in the catalog", func() {
catalog := map[string]struct{}{}
for _, name := range expectedFullCatalog {
catalog[name] = struct{}{}
}
for name := range toolToHTTPRoute {
_, ok := catalog[name]
Expect(ok).To(BeTrue(),
"toolToHTTPRoute documents %q but the tool is not registered. "+
"Remove the stale entry.",
name)
}
})
// Deliberate non-test: we don't enumerate admin REST routes here. That
// would require booting Application or parsing core/http/routes/localai.go,
// both of which are brittle. The contract for "new admin REST endpoint
// → MCP tool" is enforced by the PR checklist in
// .agents/api-endpoints-and-auth.md, not by this test.
})