getAMDGPUMemory only asked rocm-smi. ROCm 7+ images no longer ship it
(only amd-smi, whose Python module is missing in the image), so on an
AMD Strix Halo (gfx1151, ROCm 10.1 image) GET /api/resources returned
"gpus": null and aggregate.gpu_count 0, and the Models page said
"No GPU found. Fit is measured against 122.7 GB of memory" while
inference ran on the GPU.
The amdgpu driver exposes the same figures in sysfs, readable inside the
container:
/sys/class/drm/card0/device/vendor 0x1002
/sys/class/drm/card0/device/mem_info_vram_total 536870912
/sys/class/drm/card0/device/mem_info_vram_used 240300032
/sys/class/drm/card0/device/mem_info_gtt_total 128849018880
/sys/class/drm/card0/device/mem_info_gtt_used 33055911936
When rocm-smi is absent or reports nothing, walk the DRM tree with the
existing scanSysfsGPUs (cardN only, connectors skipped, vendor 0x1002)
and report every card that has mem_info_vram_total. The GPU report is
built by one helper shared with the rocm-smi parser, so the
shared-memory APU rule from #12058 (GTT >= 8x VRAM folds GTT into the
totals) applies identically. GetResourceInfo, GetGPUAggregateInfo and
the fit check all read GetGPUMemoryUsage, so they now see the GPU;
DetectGPUVendor already consulted sysfs.
Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
(cherry picked from commit bf222cadba)
* fix(gpu): detect GPUs via sysfs when no pci.ids database is present
ghw.GPU() calls pci.New() before it reads /sys/class/drm and fails
outright when it cannot find a pci.ids database file. jaypipes/pcidb
embeds no database and has network fetch disabled by default, so on an
image that ships no pci.ids, GPU enumeration returns an error and every
detection path downstream goes dark.
The Dockerfile installs pciutils only in the vulkan and cublas branches,
so the Intel image had no pci.ids. A correctly passed-through Arc A310
was reported as "No GPU detected" with zero VRAM even though clinfo and
sycl-ls both enumerated it inside the same container. NVIDIA and AMD
images were shielded by their nvidia-smi / rocm-smi binary fallbacks;
Intel has no equivalent, leaving it fully exposed.
Read PCI vendor IDs directly from /sys/class/drm/card*/device/vendor,
which needs no database, and consult that from DetectGPUVendor. The
same scan replaces the ghw-only guard in getIntelGPUMemory, which is
what had been blocking the working clinfo path and keeping VRAM at
zero. Install hwdata in the base image stage as well, so ghw stops
failing for every image variant rather than only Intel.
Also apply the documented NVIDIA > AMD > Intel priority to the ghw
path, which previously returned whichever card DRM enumerated first
and so reported "intel" on a machine with an Intel iGPU at card0 and
an NVIDIA dGPU at card1.
HasGPU() carried the same blindness plus one of its own: it matched
the requested vendor against ghw's card description with a
case-sensitive Contains, so "nvidia" never matched the pci.ids
spelling "NVIDIA Corporation". It only worked because that same
description embeds the lowercase kernel driver name ("nvidia",
"amdgpu"), and it returned false outright whenever ghw errored. Route
it through the shared vendor lookup so it matches case-insensitively
and falls back to sysfs. It feeds the GPU option and NGPULayers
defaults in core/config/gguf.go.
Fixes#10941
Assisted-by: Claude:claude-opus-4-8 golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* refactor(gpu): key vendor detection off the numeric PCI ID in both paths
The ghw and sysfs legs were identifying vendors by different means: ghw
by substring-matching the pci.ids vendor name, sysfs by the numeric PCI
vendor ID. ghw already exposes that same numeric ID via
DeviceInfo.Vendor.ID, read from the kernel's modalias rather than from
the database, so the name matching was both a duplicate mechanism and
the weaker of the two.
It is weaker because a card absent from an outdated pci.ids gets
Name: "unknown" while its ID is still correct. Detection then failed
even though ghw had enumerated the card successfully. Verified in a
container with a vendor-less pci.ids and an Arc's modalias: before,
DetectGPUVendor returned ""; after, "intel".
Both legs now resolve through the same pciVendorIDs table and share the
hex parsing, with the vendor name kept only as a fallback for devices
exposing no parseable ID.
ghwHasVendor is deliberately not a priority pick, unlike vendorFromGHW:
HasGPU("intel") must stay true on a hybrid-graphics host whose discrete
NVIDIA card outranks the integrated Intel one.
Assisted-by: Claude:claude-opus-4-8 golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
* fix(gpu): silence the gosec G304 on the sysfs attribute read
gosec flags os.ReadFile with a non-literal path. The path here is the
DRM root (a package constant in production, a temp dir under test)
joined with a ReadDir entry name and a fixed attribute filename, so no
external input reaches it.
gosec's suggested autofix, os.Root, cannot be used: /sys/class/drm/cardN
is a symlink into the PCI device tree, and os.Root refuses to traverse
it ("path escapes from parent"), which would disable the whole scan.
Assisted-by: Claude:claude-opus-4-8 gosec golangci-lint
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
---------
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Stefan Walcz