Ettore Di Giacinto fc589b3fad analysis: vLLM GB10 advantage is the SCHEDULER, not the kernel (pivot) ...

Code-grounded vLLM v0.23.0 analysis + DGX measurement: vLLM single-stream W4A16
prefill ~800 t/s (~52 TFLOPS) is TIED with llama.cpp MMQ (718/47), using the exact
XOR-swizzle + 4-stage cp.async Marlin we proved collapses GB10 occupancy. vLLM has
no FP4 cubins on sm_121 (forced W4A16 fallback), so llama.cpp MXFP4 (1153) already
beats vLLM single-stream. vLLM's ~24k headline is the aggregate decode multiplier
(~56x) from paged KV + chunked prefill + continuous batching - a scheduler win.
llama.cpp lacks paged KV + chunked prefill. Kernel work (W4A16 178 t/s, FP4-MMA)
banked as not-the-lever; effort pivots to the scheduler. Detail in
VLLM_DECOMPOSITION.md; W4A16 plan marked STOPPED.

Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2026-06-21 07:09:42 +00:00

..

ds4

chore: ⬆️ Update antirez/ds4 to 80ebbc396aee40eedc1d829222f3362d10fa4c6c (#10378)

2026-06-18 00:32:13 +02:00

grpc

fix: speedup git submodule update with --single-branch (#2847)

2024-07-13 22:32:25 +02:00

ik-llama-cpp

chore: ⬆️ Update ikawrakow/ik_llama.cpp to b3dfb7858cfcb9166e92f366e5af87f19ebc94be (#10395)

2026-06-19 00:03:37 +02:00

llama-cpp

analysis: vLLM GB10 advantage is the SCHEDULER, not the kernel (pivot)

2026-06-21 07:09:42 +00:00

privacy-filter

feat(pii): NER tier engine — privacy-filter.cpp backend + NER-centric PII filter (#10360)

2026-06-18 11:45:22 +01:00

turboquant

chore(turboquant): bump to 7d9715f1 + fix compilation against rebased fork (#10205)

2026-06-07 10:42:06 +02:00