mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-08 07:18:31 -04:00
fix(vllm): pin L4T arm64 backend to vllm==0.24.0 for GB10 stability (#10725)
The nvidia-l4t-cuda-13-arm64 vLLM backend left `vllm` unpinned, so the
prebuilt image drifted onto whatever aarch64 wheel was latest at build
time (0.23.x). On GB10 / DGX Spark (Grace Blackwell, unified memory),
0.23 crashes deterministically during cold model loads with an empty
"Engine core initialization failed" set and pins GPU memory until a host
reboot.
vLLM 0.24.0 carries vllm-project/vllm#45179 ("release cached device
memory under pressure on UMA GPUs during weight loading"), which the
reporter verified fixes the crash on GB10. Pin the L4T requirements to
0.24.0 to match the already-pinned cublas13 build
(requirements-cublas13-after.txt) and keep the image deterministic.
Editing this file also re-triggers the single-arch L4T image build via
the path filter, republishing the gallery image with 0.24.0 (the
single-arch matrix builds again after #10703).
Closes #10722
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
@@ -1,4 +1,12 @@
|
||||
# vLLM 0.20+ ships an aarch64 manylinux wheel on PyPI whose Requires-Dist pins
|
||||
# torch==2.11.0 / torchvision==0.26.0 / torchaudio==2.11.0, locking an ABI-
|
||||
# consistent set with the cu130 torch wheel installed above.
|
||||
vllm
|
||||
#
|
||||
# Pinned to match the cublas13 build (requirements-cublas13-after.txt) and to
|
||||
# stay deterministic on GB10 / DGX Spark: 0.24.0 carries vllm-project/vllm#45179
|
||||
# ("release cached device memory under pressure on UMA GPUs during weight
|
||||
# loading"), without which cold model loads on the Grace Blackwell unified-
|
||||
# memory architecture crash deterministically with an empty "Engine core init
|
||||
# failed" set (mudler/LocalAI#10722). Leaving this unpinned let the L4T image
|
||||
# drift onto whatever wheel was latest at build time.
|
||||
vllm==0.24.0
|
||||
|
||||
Reference in New Issue
Block a user