mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-07-30 09:57:57 -04:00
Code Issues Packages Projects Releases Wiki Activity

docs(gpu): add ROCm 7.x and RDNA 3.5 / Strix Halo (gfx1151) to GPU acceleration guide (#9229)

Browse Source 
* docs(gpu): add gfx1151 / ROCm 7.x and fix ROCm section

- Fix typo: "deditated" → "dedicated", "ROCm6" → "ROCm"
- Add ROCm 7.x to requirements (alongside ROCm 6.x)
- Add Ubuntu 24.04 to tested OS list
- Add AMD Strix Halo / gfx1151 section with kernel params,
  required env vars (HSA_OVERRIDE_GFX_VERSION, ROCBLAS_USE_HIPBLASLT),
  and Docker Compose example
- Add gfx1151 to the list of compiled GPU targets
- Add ROCm version column to verified devices table
- Add gfx1151 / Radeon 8060S (ROCm 7.11.0) as verified device

* fix(docs/gpu): correct gfx1151 section — env vars, image tag, safety warning

- Add all 4 required env vars (HSA_OVERRIDE_GFX_VERSION, ROCBLAS_USE_HIPBLASLT,
  HSA_XNACK=1, HSA_ENABLE_SDMA=0) with descriptions in a table
- Fix Docker Compose example to use the ROCm 7.x image tag (-gpu-hipblas-rocm7),
  not the ROCm 6.x image
- Add explicit warning: GGML_CUDA_ENABLE_UNIFIED_MEMORY must NOT be set
  (even =0 activates hipMallocManaged due to getenv != nullptr check)
- Add --force-recreate note (docker restart does not update container env)
- Add tested hardware note (Geekom A9 Mega / Ryzen AI MAX+ 395)

* docs(gpu): single ROCm image — drop -rocm7 tag suffix

Per maintainer feedback on PR #9229: there is only one ROCm/hipblas
main image, and it ships with ROCm 7.x by default — no separate
-rocm7 tag.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
walcz-de
2026-07-29 20:09:52 +02:00
committed by GitHub
parent ecdb32193d
commit 2f33d6dee0
1 changed files with 56 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
docs/content/features/GPU-acceleration.md
View File

@@ -166,12 +166,58 @@ Due to the nature of ROCm it is best to run all implementations in containers as
- Make sure to do not use GPU assigned for compute for desktop rendering.
- Ensure at least 100GB of free space on disk hosting container runtime and storing images prior to installation.
### AMD Strix Halo / gfx1151 (RDNA 3.5)
AMD Ryzen AI MAX+ (Strix Halo) APUs with an integrated Radeon 8060S (gfx1151 / RDNA 3.5) are
supported with ROCm 7.11.0+. These systems provide up to 96 GB of unified VRAM accessible by the GPU.
Tested on: Geekom A9 Mega (AMD Ryzen AI MAX+ 395, ROCm 7.11.0, Ubuntu 24.04, kernel 6.14).
**Required kernel boot parameters** (add to `GRUB_CMDLINE_LINUX` in `/etc/default/grub`, then run `update-grub`):
```
iommu=pt amdgpu.gttsize=126976 ttm.pages_limit=32505856
```
**Required environment variables** for gfx1151 (set automatically in the ROCm/hipblas image):
| Variable | Value | Purpose |
|----------|-------|---------|
| `HSA_OVERRIDE_GFX_VERSION` | `11.5.1` | Tells the HSA runtime to use gfx1151 code objects |
| `ROCBLAS_USE_HIPBLASLT` | `1` | Prefer hipBLASLt over rocBLAS for GEMM (required for gfx1151) |
| `HSA_XNACK` | `1` | Enable XNACK (memory-fault retry) for APU unified memory |
| `HSA_ENABLE_SDMA` | `0` | Disable SDMA engine — causes hangs on APU/iGPU configs |
> **Warning:** Do **not** set `GGML_CUDA_ENABLE_UNIFIED_MEMORY`. The C-level check is
> `getenv(...) != nullptr`, so even `=0` activates `hipMallocManaged` (allocates from
> system RAM instead of the 96 GB VRAM pool).
**Running LocalAI on gfx1151** (use the standard ROCm/hipblas image — there is only one ROCm image, and it ships with ROCm 7.x by default):
```yaml
image: quay.io/go-skynet/local-ai:master-gpu-hipblas
environment:
- HSA_OVERRIDE_GFX_VERSION=11.5.1
- ROCBLAS_USE_HIPBLASLT=1
- HSA_XNACK=1
- HSA_ENABLE_SDMA=0
devices:
- /dev/dri
- /dev/kfd
group_add:
- video
```
> **Note:** When updating the image, always recreate the container (`docker compose up --force-recreate`)
> rather than just restarting it. `docker compose restart` preserves the old container environment
> and will not pick up updated env vars from the image.
For llama.cpp models, enable flash attention (`--flash-attention`) and disable mmap (`--no-mmap`) for best performance on APU systems.
### Limitations
Ongoing verification testing of ROCm compatibility with integrated backends.
Please note the following list of verified backends and devices.
LocalAI hipblas images are built against the following targets: gfx908, gfx90a, gfx942, gfx950, gfx1030, gfx1100, gfx1101, gfx1102, gfx1200, gfx1201
LocalAI hipblas images are built against the following targets: gfx908, gfx90a, gfx942, gfx950, gfx1030, gfx1100, gfx1101, gfx1102, gfx1151, gfx1200, gfx1201
**Note:** Starting with ROCm 6.4, AMD removed rocBLAS kernel support for older architectures (gfx803, gfx900, gfx906). Since llama.cpp and other backends depend on rocBLAS for matrix operations, these GPUs (e.g. Radeon VII) are no longer supported in pre-built images.
@@ -181,14 +227,15 @@ If your device is not one of the above targets, you must specify the correspondi
The devices in the following list have been tested with `hipblas` images.
| Backend | Verified | Devices |
| ---- | ---- | ---- |
| llama.cpp | yes | MI100 (gfx908), MI210/250 (gfx90a) |
| diffusers | yes | MI100 (gfx908), MI210/250 (gfx90a) |
| whisper | no | none |
| coqui | no | none |
| transformers | no | none |
| vllm | no | none |
| Backend | Verified | Devices | ROCm Version |
| ---- | ---- | ---- | ---- |
| llama.cpp | yes | MI100 (gfx908), MI210/250 (gfx90a) | 7.x |
| llama.cpp | yes | Radeon 8060S / gfx1151 (Strix Halo) | 7.11.0 |
| diffusers | yes | MI100 (gfx908), MI210/250 (gfx90a) | 7.x |
| whisper | no | none | - |
| coqui | no | none | - |
| transformers | no | none | - |
| vllm | no | none | - |
**You can help by expanding this list.**