mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-03 12:57:02 -04:00
ef15b4bfda
* fix(vllm): install ROCm vLLM from the AMD wheel index on Python 3.12 The rocm-vllm backend crashed at load with "No module named 'vllm'". requirements-hipblas-after.txt requested a bare `vllm`, which resolves to the CUDA-only PyPI wheel; that wheel is unusable on an AMD GPU. vLLM's prebuilt ROCm wheels live on a dedicated index (https://wheels.vllm.ai/rocm/) and are published only for CPython 3.12, so on the backend's default 3.10 the installer silently falls back to the CUDA wheel. Add a hipblas branch to backend/python/vllm/install.sh that pins Python to 3.12 and installs vllm from the ROCm wheel index, hiding the bare-`vllm` after-file so installRequirements installs only the base ROCm torch/transformers first and does not pull the CUDA wheel. Fixes #10642 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] * chore(vllm): drop the dead hipblas-after requirement and its hide dance requirements-hipblas-after.txt (a bare `vllm`) is never installed for hipblas: installRequirements only adds requirements-${BUILD_PROFILE}-after.txt when BUILD_TYPE != BUILD_PROFILE, and for hipblas they are equal. So the file was dead and the install.sh hide/restore of it was a no-op. Remove both. The hipblas branch already installs vllm explicitly from the ROCm wheel index, so deleting the bare-`vllm` file also removes a latent CUDA-wheel trap should the installRequirements gap ever be closed. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude:claude-opus-4-8 [Claude Code] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Creating a separate environment for the vllm project
make vllm