fix(compose): request NVIDIA compute capability

The legacy NVIDIA device reservation requests utility without compute.
Docker derives driver capabilities from that list, leaving CUDA libraries
unavailable even when monitoring works.

Include compute in the legacy example and clarify the matching docs.

Assisted-by: Codex:GPT-6
This commit is contained in:
localai-org-maint-bot committed 2026-09-12 10:02:44 +00:00
1 parent 8321234275
commit 10d1e3e5cb
3 files changed
+12 -5

No files matched your search

+2 -1
View File
@@ -59,6 +59,7 @@ services:
# capabilities: [gpu, utility]
#
# For legacy NVIDIA driver (for older NVIDIA Container Toolkit):
# Request compute for CUDA libraries (libcuda.so.1) and utility for NVML.
# environment:
# NVIDIA_DRIVER_CAPABILITIES: "compute,utility"
# init: true
@@ -68,7 +69,7 @@ services:
# devices:
# - driver: nvidia
# count: 1
# capabilities: [gpu, utility]
# capabilities: [gpu, compute, utility]
## Uncomment for PostgreSQL-backed knowledge base (see Agents docs)
# postgres:
+6 -2
View File
@@ -417,8 +417,12 @@ usage is reported back to the frontend:
NVML library (and therefore `nvidia-smi`) is not available inside the
container. CUDA compute still works, but the worker cannot query free VRAM
and the Nodes page will show the node as fully used. Set
`NVIDIA_DRIVER_CAPABILITIES=compute,utility` (or, with the NVIDIA CDI
runtime, list `capabilities: [gpu, utility]` on the device reservation).
`NVIDIA_DRIVER_CAPABILITIES=compute,utility` when using the NVIDIA runtime.
For Docker Compose with `driver: nvidia`, use
`capabilities: [gpu, compute, utility]` on the device reservation.
Docker derives driver capabilities from this reservation, so include `compute`
for CUDA libraries such as `libcuda.so.1`. The `utility` capability alone
enables monitoring but does not provide CUDA libraries.
- **Run the container with `init: true` (or `docker run --init`).** The
worker process becomes PID 1 in the container and cannot reap zombies on
+4 -2
View File
@@ -88,8 +88,10 @@ page in the frontend shows the node as fully used, check two things:
NVML work inside the container. With `--gpus all` alone (or
`--runtime nvidia` without extra flags) only `compute` is wired in on
some driver versions. Add `-e NVIDIA_DRIVER_CAPABILITIES=compute,utility`
to your `docker run`, or `capabilities: [gpu, utility]` in compose /
Kubernetes device reservations.
to your `docker run`. For Docker Compose with `driver: nvidia`, use
`capabilities: [gpu, compute, utility]` on the device reservation.
Include `compute` for CUDA libraries such as `libcuda.so.1`; `utility`
alone only provides monitoring libraries and tools.
2. Pass `--init` to `docker run` (or `init: true` in compose) so the
container has a proper PID 1 reaper - otherwise short-lived child
processes like `nvidia-smi` can intermittently fail with