From 13466917f0ab3eaa84db04c0e0bd673912f02339 Mon Sep 17 00:00:00 2001 From: mudler-agent Date: Fri, 16 Oct 2026 22:32:47 +0200 Subject: [PATCH] fix(vllm-cpp): pin MLX 0.32.0, the version the engine builds against (#12734) * chore(vllm-cpp): bump MLX pin to 0.32.4 The macOS runner's pip index no longer offers mlx==0.29.4 (it lists only 0.32.4), so the darwin vllm-cpp build fails at the MLX venv step. Move the pin to 0.32.4 and update the README default to match. Not verified on macOS: the Makefile was only dry-run on Linux. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * fix(vllm-cpp): pin MLX 0.32.0, the version the engine builds against The macOS build fails against mlx 0.32.4: metal_mlx_provider.mm:147:5: error: no matching member function for call to 'set_data' The vllm.cpp Metal provider calls the five-argument array::set_data(Buffer, size, strides, flags, Deleter). The 0.32.4 headers only declare set_data(allocator::Data) and the Data overload with size, strides, flags and offset; 0.32.0 still has the Buffer one. The engine's own CI installs 'mlx==0.32.0' (ci.yml line 1798, release.yml line 360), so pin that and update the README default. Not verified on macOS: only the Makefile dry-run and the wheel headers were checked on Linux. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto * fix(vllm-cpp): install the MLX 0.32.0 wheel by URL and hash The macOS runner's pip index offers only mlx 0.32.4, so 'pip install mlx==0.32.0' fails with "Could not find a version that satisfies the requirement". 0.32.4 cannot be used: its headers lack the five-argument array::set_data the engine calls. Install the 0.32.0 wheel directly by URL with a sha256 check. The wheel is mlx-metal (py3-none-macosx_14_0_arm64) from PyPI, sha256 5b64b20ac24b0c401f489de01e8209edc4d372125201f19314e6f39e385322aa. It holds mlx/include, mlx/lib/libmlx.dylib and mlx/lib/mlx.metallib, which is all the build and package.sh use, and it is not tied to a CPython version. The URL and hash are Makefile variables; the stamp stays keyed on MLX_VERSION. Not verified on macOS: only the Makefile dry-run, the wheel contents and its sha256 were checked on Linux. Assisted-by: Claude Code:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto --------- Signed-off-by: Ettore Di Giacinto Co-authored-by: Ettore Di Giacinto --- backend/go/vllm-cpp/Makefile | 14 +++++++++++--- backend/go/vllm-cpp/README.md | 9 ++++++--- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/backend/go/vllm-cpp/Makefile b/backend/go/vllm-cpp/Makefile index 2e0b20b7b..9076455ce 100644 --- a/backend/go/vllm-cpp/Makefile +++ b/backend/go/vllm-cpp/Makefile @@ -30,7 +30,15 @@ VLLM_CPP_VERSION?=bf7b654ae2d0cf49028291192a78bd92f2843106 # decode makes ~112 calls per TOKEN. Ungated it does both; gated it does only the # good half. So if this pin is ever moved BACKWARDS, this default must go with it. VLLM_CPP_MLX?=on -MLX_VERSION?=0.29.4 +MLX_VERSION?=0.32.0 +# The macOS runners' pip index offers only the newest MLX (0.32.4), which lacks +# the array::set_data overload the engine calls, so the pinned wheel is fetched +# by URL and verified by sha256. This is the mlx-metal distribution: it holds +# mlx/include, mlx/lib/libmlx.dylib and mlx/lib/mlx.metallib, the only parts the +# build uses, and unlike the `mlx` wheel it is not tied to one CPython version. +# Bumping MLX_VERSION means bumping both values (PyPI JSON: /pypi/mlx-metal//json). +MLX_WHEEL_URL?=https://files.pythonhosted.org/packages/d6/ef/d74ae99cfe9ddb59fd08abd14f47754c3d199291a93756903fa595b31b8a/mlx_metal-0.32.0-py3-none-macosx_14_0_arm64.whl +MLX_WHEEL_SHA256?=5b64b20ac24b0c401f489de01e8209edc4d372125201f19314e6f39e385322aa MLX_VENV?=$(abspath ./mlx-venv) # Resolved lazily (recursive `=`, not `:=`): the glob only matches once the venv # target has run, and the interpreter version in the path varies per runner. @@ -153,14 +161,14 @@ $(MLX_STAMP): @if [ ! -x "$(MLX_VENV)/bin/pip" ]; then \ python3 -m venv "$(MLX_VENV)" || { echo "vllm-cpp: python3 with venv is required to build the MLX provider; pass VLLM_CPP_MLX=off to build Metal without it" >&2; exit 1; }; \ fi - "$(MLX_VENV)"/bin/pip install --quiet --disable-pip-version-check "mlx==$(MLX_VERSION)" + "$(MLX_VENV)"/bin/pip install --quiet --disable-pip-version-check "mlx-metal @ $(MLX_WHEEL_URL)#sha256=$(MLX_WHEEL_SHA256)" @# Resolved in the SHELL, not by $(MLX_ROOT): make expands a whole recipe @# before running its first line, so the glob would still be unmatched here. @# Every later use (the cmake args, package.sh) expands after this target has @# completed, where $(MLX_ROOT) does resolve. @root=$$(echo "$(MLX_VENV)"/lib/python*/site-packages/mlx); \ test -f "$$root/lib/libmlx.dylib" -a -f "$$root/include/mlx/array.h" || \ - { echo "vllm-cpp: mlx==$(MLX_VERSION) did not provide lib/libmlx.dylib + include/mlx/array.h under $$root" >&2; exit 1; } + { echo "vllm-cpp: mlx-metal $(MLX_VERSION) did not provide lib/libmlx.dylib + include/mlx/array.h under $$root" >&2; exit 1; } touch $@ else MLX_STAMP= diff --git a/backend/go/vllm-cpp/README.md b/backend/go/vllm-cpp/README.md index 11427f926..9c5d231d3 100644 --- a/backend/go/vllm-cpp/README.md +++ b/backend/go/vllm-cpp/README.md @@ -181,9 +181,12 @@ Build knobs: - `VLLM_CPP_MLX=off` builds Metal without the provider: ~124 MB smaller, and 96.4% of MLX-LM instead of 99.1%. -- `MLX_VERSION` pins the wheel (default `0.29.4`). MLX is consumed as the - prebuilt pip wheel because building it from source needs `xcrun metal`, i.e. a - full Xcode the macOS runners do not have. +- `MLX_VERSION` names the wheel (default `0.32.0`). MLX is consumed as the + prebuilt `mlx-metal` wheel because building it from source needs `xcrun metal`, + i.e. a full Xcode the macOS runners do not have. The default wheel is installed + by URL and sha256 (`MLX_WHEEL_URL`, `MLX_WHEEL_SHA256`), because the runners' + pip index offers only 0.32.4, whose headers lack the `array::set_data` overload + the engine calls. Change all three together when you bump MLX. Packaging vendors `libmlx.dylib`, `mlx.metallib` and MLX's MIT license into `package/lib/`, and rewrites `libvllm.dylib`'s rpath to `@loader_path/lib`