Ettore Di Giacinto 3b59571579 docs(paged): both-engine MoE decode decomposition - the 15% is NOT the Marlin GEMM ...

Ground-truth side-by-side per-kernel ms/step of the MoE decode gap on DGX GB10.
llama (752 t/s, step 169.8ms) vs vLLM graphs-on (901-equiv, step 142.0ms): 27.8ms gap.

Headline: the grouped MoE-expert GEMM is a llama WIN - native FP4-MMA W4A4 47.3ms
vs vLLM Marlin W4A16 50.0ms at the tiny-M decode shape. A Marlin-style W4A16 MoE
GEMM would be slower; it is not the lever (extends the w4a16-marlin DENSE verdict).

The 15% lives elsewhere: bf16 projections + convert glue (+6.5ms), recurrence
state-gather plumbing (+6.6ms, led by k_get_rows 5.2ms), graph coverage + stream
overlap (~+7ms), W4A4 act-quant tax (+3.3ms), router/glue (+5.4ms).

Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2026-06-26 20:11:40 +00:00

..

ds4

chore: ⬆️ Update antirez/ds4 to 80ebbc396aee40eedc1d829222f3362d10fa4c6c (#10378)

2026-06-18 00:32:13 +02:00

grpc

fix: speedup git submodule update with --single-branch (#2847)

2024-07-13 22:32:25 +02:00

ik-llama-cpp

fix(backends): quote $CURDIR in run.sh (fixes backends in paths with spaces) (#10519)

2026-06-26 01:02:48 +02:00

llama-cpp

docs(paged): both-engine MoE decode decomposition - the 15% is NOT the Marlin GEMM

2026-06-26 20:11:40 +00:00

llama-cpp-localai-paged

feat(backend): llama-cpp-localai-paged variant + NVFP4 Qwen3.6 gallery

2026-06-26 12:58:56 +00:00

privacy-filter

feat(backends): darwin/Metal build for the privacy-filter backend (#10513)

2026-06-26 01:18:41 +02:00

turboquant

fix(backends): quote $CURDIR in run.sh (fixes backends in paths with spaces) (#10519)

2026-06-26 01:02:48 +02:00