mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-28 09:05:05 -04:00
feat(system): report per-model DRM VRAM (#12026)
* feat(system): report per-model DRM VRAM Expose optional resident device memory for local backend process trees. Deduplicate DRM clients and omit unsupported or incomplete readings. Document accounting limits and preserve a measured zero in JSON. Closes #11970. Assisted-by: Codex:gpt-6 * fix(system): document trusted procfs reads Scope G304 annotations to paths built from the fixed procfs root, integer process IDs, and kernel directory entries. These reads accept no user-controlled path components. Assisted-by: Codex:GPT-6 gosec --------- Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
This commit is contained in:
1 parent
5794495a37
commit
f154bd990a
11 files changed
+399
No files matched your search
@@ -28,6 +28,29 @@ Returns available backends and currently loaded models.
|
||||
| `loaded_models[].process.memory_percent` | `number` | `rss_bytes` as a percentage of host RAM |
|
||||
| `loaded_models[].process.cpu_percent` | `number` | Share of the whole host's CPU used since the previous call, 0-100. Omitted on the first call that sees the process, because there is no earlier reading to compare against |
|
||||
| `loaded_models[].process.started_at` | `string` | When the process started (RFC 3339) |
|
||||
| `loaded_models[].size_vram` | `integer` | Optional DRM-accounted resident device memory, in bytes |
|
||||
|
||||
### Per-model VRAM
|
||||
|
||||
On Linux, `size_vram` reports resident device memory for the local backend
|
||||
process and its child processes. LocalAI reads `drm-resident-local*` and
|
||||
`drm-resident-vram*` from `/proc` and counts each DRM client once per GPU.
|
||||
Host-memory regions are excluded. The reading includes buffers attributed
|
||||
to the backend, without separating weights, KV cache, and other allocations.
|
||||
See the [kernel DRM accounting specification](https://docs.kernel.org/gpu/drm-usage-stats.html)
|
||||
for these counters.
|
||||
|
||||
The field is omitted when accounting is unavailable or incomplete. This
|
||||
includes external and distributed backends, macOS, proprietary NVIDIA
|
||||
drivers, primary DRM nodes (`/dev/dri/card*`), missing resident counters,
|
||||
and unreadable process information.
|
||||
A present value of `0` means the supported counters report zero bytes.
|
||||
Treat an absent field as unknown.
|
||||
|
||||
This is a snapshot of driver accounting, not a memory reservation. Shared
|
||||
buffers can appear in different clients' counters, and allocations can change
|
||||
during collection. Do not treat the sum across models as exclusive physical
|
||||
GPU usage. These readings do not replace capacity checks when scheduling work.
|
||||
|
||||
### Usage
|
||||
|
||||
@@ -49,6 +72,7 @@ curl http://localhost:8080/system
|
||||
{
|
||||
"id": "my-llama-model",
|
||||
"backend": "llama-cpp",
|
||||
"size_vram": 5368709120,
|
||||
"process": {
|
||||
"pid": 48213,
|
||||
"rss_bytes": 5368709120,
|
||||
|
||||
Reference in new issue
Block a user