From 927206ed53a099ef8a804c0c32ed3f2556ba8e4d Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Tue, 28 Jul 2026 08:41:18 +0000 Subject: [PATCH] docs(vllm-cpp): correct the MLX-gated figure to 97.6%, from 99.1% The previous commit quoted 99.1% of MLX-LM for the prefill-gated MLX build. That figure divided by a two-run MLX-LM baseline, 27.135 and 27.744 generation tok/s averaged to 27.44. Re-measured interleaved with ours over four ABBA blocks, MLX-LM's decode is 27.848 with a 0.34% spread across six runs, so the 27.135 was an outlier and averaging it in overstated us by roughly 1.5 points. Corrected: the gated configuration is 24.37 tok/s, or 97.6% of MLX-LM, and the MLX-off build is 23.9 tok/s or 95.9%. Prefill TTFT is unchanged at 524.5 ms against MLX-LM's 532.6, so we remain about 1.5% faster there. Nothing else changes. MLX still wins prefill and loses decode, the shape gate is still the right disposition, and the pin and the flag are still coupled. The gate is worth about 1.7 points over the MLX-off build rather than 2.7. Assisted-by: Claude Code:claude-opus-5 [ClaudeCode] Signed-off-by: Ettore Di Giacinto --- backend/go/vllm-cpp/README.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/backend/go/vllm-cpp/README.md b/backend/go/vllm-cpp/README.md index 3755340d4..65636e1a8 100644 --- a/backend/go/vllm-cpp/README.md +++ b/backend/go/vllm-cpp/README.md @@ -55,9 +55,14 @@ Apple M4, Qwen3-1.7B-bf16 warm at p=512 g=128: | configuration | prefill TTFT | warm throughput | |---|--:|--:| -| MLX **gated to prefill** (pin >= 89c46aeb) | **524.5 ms** | **24.40 tok/s — 99.1% of MLX-LM** | +| MLX **gated to prefill** (pin >= 89c46aeb) | **524.5 ms** | **24.37 tok/s, 97.6% of MLX-LM** | | MLX ungated (older pins) | 537 ms | 12.7 tok/s | -| MLX off | 602 ms | 23.9 tok/s | +| MLX off | 602 ms | 23.9 tok/s, 95.9% | + +Ratios are against an MLX-LM baseline measured INTERLEAVED with ours over four +ABBA blocks (its spread 0.34%, ours 0.12%). An earlier revision of this file +claimed 99.1%; that used a two-run MLX-LM baseline containing an outlier and +overstated us by about 1.5 points. **`VLLM_CPP_VERSION` and this flag are coupled.** Moving the pin back before `89c46aeb` while leaving `VLLM_CPP_MLX=on` would take the middle row — roughly