PR #17004 is merged and already present in our pinned llama.cpp f3e1828.
Measured on DGX Spark (GB10, sm_121, Qwen3-32B-Q4_K_M):
- llama-batched-bench does no sampling (random tokens), so it cannot test
the fix; its ~540 t/s plateau is not sampling-bound.
- Real-sampling A/B via llama-batched (CPU vs -bs GPU sampler): +25% at
np=32, +3% at np=64, GGML_ASSERT(obj_new) graph-alloc crash at np>=128.
- nsys at np=64: GPU-busy time and kernel mix unchanged (392 vs 404 t/s);
sampling kernels negligible. GPU utilization did not rise.
Clean negative: the fix does not break the plateau toward the ~2700 ceiling
or past vLLM 667, and is unusable at the multi-user parallelism in question.
Adoption: code arrives via LLAMA_VERSION bump (prepare.sh vendors the
modified upstream server-context.cpp), but grpc-server must set
params.sampling.backend_sampling to enable it; grammar/tool-call/logprobs
requests fall back to CPU. Defer adoption until #18547/#18550 stabilise it.
Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
