feat(system): report per-model DRM VRAM (#12026)

* feat(system): report per-model DRM VRAM

Expose optional resident device memory for local backend process trees.
Deduplicate DRM clients and omit unsupported or incomplete readings.
Document accounting limits and preserve a measured zero in JSON.

Closes #11970.

Assisted-by: Codex:gpt-6

* fix(system): document trusted procfs reads

Scope G304 annotations to paths built from the fixed procfs root,
integer process IDs, and kernel directory entries. These reads accept
no user-controlled path components.

Assisted-by: Codex:GPT-6 gosec

---------

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
This commit is contained in:
localai-org-maint-botandlocalai-org-maint-bot authored and GitHub committed 2026-09-27 21:18:34 +02:00
1 parent 5794495a37
commit f154bd990a
11 files changed
+399

No files matched your search

+24
View File
@@ -28,6 +28,29 @@ Returns available backends and currently loaded models.
| `loaded_models[].process.memory_percent` | `number` | `rss_bytes` as a percentage of host RAM |
| `loaded_models[].process.cpu_percent` | `number` | Share of the whole host's CPU used since the previous call, 0-100. Omitted on the first call that sees the process, because there is no earlier reading to compare against |
| `loaded_models[].process.started_at` | `string` | When the process started (RFC 3339) |
| `loaded_models[].size_vram` | `integer` | Optional DRM-accounted resident device memory, in bytes |
### Per-model VRAM
On Linux, `size_vram` reports resident device memory for the local backend
process and its child processes. LocalAI reads `drm-resident-local*` and
`drm-resident-vram*` from `/proc` and counts each DRM client once per GPU.
Host-memory regions are excluded. The reading includes buffers attributed
to the backend, without separating weights, KV cache, and other allocations.
See the [kernel DRM accounting specification](https://docs.kernel.org/gpu/drm-usage-stats.html)
for these counters.
The field is omitted when accounting is unavailable or incomplete. This
includes external and distributed backends, macOS, proprietary NVIDIA
drivers, primary DRM nodes (`/dev/dri/card*`), missing resident counters,
and unreadable process information.
A present value of `0` means the supported counters report zero bytes.
Treat an absent field as unknown.
This is a snapshot of driver accounting, not a memory reservation. Shared
buffers can appear in different clients' counters, and allocations can change
during collection. Do not treat the sum across models as exclusive physical
GPU usage. These readings do not replace capacity checks when scheduling work.
### Usage
@@ -49,6 +72,7 @@ curl http://localhost:8080/system
{
"id": "my-llama-model",
"backend": "llama-cpp",
"size_vram": 5368709120,
"process": {
"pid": 48213,
"rss_bytes": 5368709120,