From 8565febe45e547784310a68e1fbac53649cc8473 Mon Sep 17 00:00:00 2001 From: "LocalAI [bot]" <139863280+localai-bot@users.noreply.github.com> Date: Tue, 7 Jul 2026 22:18:23 +0200 Subject: [PATCH] fix(vllm): pin L4T arm64 backend to vllm==0.24.0 for GB10 stability (#10725) The nvidia-l4t-cuda-13-arm64 vLLM backend left `vllm` unpinned, so the prebuilt image drifted onto whatever aarch64 wheel was latest at build time (0.23.x). On GB10 / DGX Spark (Grace Blackwell, unified memory), 0.23 crashes deterministically during cold model loads with an empty "Engine core initialization failed" set and pins GPU memory until a host reboot. vLLM 0.24.0 carries vllm-project/vllm#45179 ("release cached device memory under pressure on UMA GPUs during weight loading"), which the reporter verified fixes the crash on GB10. Pin the L4T requirements to 0.24.0 to match the already-pinned cublas13 build (requirements-cublas13-after.txt) and keep the image deterministic. Editing this file also re-triggers the single-arch L4T image build via the path filter, republishing the gallery image with 0.24.0 (the single-arch matrix builds again after #10703). Closes #10722 Assisted-by: Claude:claude-opus-4-8 [Claude Code] Signed-off-by: Ettore Di Giacinto Co-authored-by: Ettore Di Giacinto --- backend/python/vllm/requirements-l4t13-after.txt | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/backend/python/vllm/requirements-l4t13-after.txt b/backend/python/vllm/requirements-l4t13-after.txt index c959c6ae0..31456efab 100644 --- a/backend/python/vllm/requirements-l4t13-after.txt +++ b/backend/python/vllm/requirements-l4t13-after.txt @@ -1,4 +1,12 @@ # vLLM 0.20+ ships an aarch64 manylinux wheel on PyPI whose Requires-Dist pins # torch==2.11.0 / torchvision==0.26.0 / torchaudio==2.11.0, locking an ABI- # consistent set with the cu130 torch wheel installed above. -vllm +# +# Pinned to match the cublas13 build (requirements-cublas13-after.txt) and to +# stay deterministic on GB10 / DGX Spark: 0.24.0 carries vllm-project/vllm#45179 +# ("release cached device memory under pressure on UMA GPUs during weight +# loading"), without which cold model loads on the Grace Blackwell unified- +# memory architecture crash deterministically with an empty "Engine core init +# failed" set (mudler/LocalAI#10722). Leaving this unpinned let the L4T image +# drift onto whatever wheel was latest at build time. +vllm==0.24.0