mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-12 22:33:54 -04:00
fix(compose): request NVIDIA compute capability
The legacy NVIDIA device reservation requests utility without compute. Docker derives driver capabilities from that list, leaving CUDA libraries unavailable even when monitoring works. Include compute in the legacy example and clarify the matching docs. Assisted-by: Codex:GPT-6
This commit is contained in:
1 parent
8321234275
commit
10d1e3e5cb
3 files changed
+12
-5
No files matched your search
+2
-1
@@ -59,6 +59,7 @@ services:
|
||||
# capabilities: [gpu, utility]
|
||||
#
|
||||
# For legacy NVIDIA driver (for older NVIDIA Container Toolkit):
|
||||
# Request compute for CUDA libraries (libcuda.so.1) and utility for NVML.
|
||||
# environment:
|
||||
# NVIDIA_DRIVER_CAPABILITIES: "compute,utility"
|
||||
# init: true
|
||||
@@ -68,7 +69,7 @@ services:
|
||||
# devices:
|
||||
# - driver: nvidia
|
||||
# count: 1
|
||||
# capabilities: [gpu, utility]
|
||||
# capabilities: [gpu, compute, utility]
|
||||
|
||||
## Uncomment for PostgreSQL-backed knowledge base (see Agents docs)
|
||||
# postgres:
|
||||
|
||||
@@ -417,8 +417,12 @@ usage is reported back to the frontend:
|
||||
NVML library (and therefore `nvidia-smi`) is not available inside the
|
||||
container. CUDA compute still works, but the worker cannot query free VRAM
|
||||
and the Nodes page will show the node as fully used. Set
|
||||
`NVIDIA_DRIVER_CAPABILITIES=compute,utility` (or, with the NVIDIA CDI
|
||||
runtime, list `capabilities: [gpu, utility]` on the device reservation).
|
||||
`NVIDIA_DRIVER_CAPABILITIES=compute,utility` when using the NVIDIA runtime.
|
||||
For Docker Compose with `driver: nvidia`, use
|
||||
`capabilities: [gpu, compute, utility]` on the device reservation.
|
||||
Docker derives driver capabilities from this reservation, so include `compute`
|
||||
for CUDA libraries such as `libcuda.so.1`. The `utility` capability alone
|
||||
enables monitoring but does not provide CUDA libraries.
|
||||
|
||||
- **Run the container with `init: true` (or `docker run --init`).** The
|
||||
worker process becomes PID 1 in the container and cannot reap zombies on
|
||||
|
||||
@@ -88,8 +88,10 @@ page in the frontend shows the node as fully used, check two things:
|
||||
NVML work inside the container. With `--gpus all` alone (or
|
||||
`--runtime nvidia` without extra flags) only `compute` is wired in on
|
||||
some driver versions. Add `-e NVIDIA_DRIVER_CAPABILITIES=compute,utility`
|
||||
to your `docker run`, or `capabilities: [gpu, utility]` in compose /
|
||||
Kubernetes device reservations.
|
||||
to your `docker run`. For Docker Compose with `driver: nvidia`, use
|
||||
`capabilities: [gpu, compute, utility]` on the device reservation.
|
||||
Include `compute` for CUDA libraries such as `libcuda.so.1`; `utility`
|
||||
alone only provides monitoring libraries and tools.
|
||||
2. Pass `--init` to `docker run` (or `init: true` in compose) so the
|
||||
container has a proper PID 1 reaper - otherwise short-lived child
|
||||
processes like `nvidia-smi` can intermittently fail with
|
||||
|
||||
Reference in new issue
Block a user