Ettore Di Giacinto 718b31d063 kernel(P1): W4A16 dispatch seam (gated, byte-identical fallback to MMQ) ...

marlin-w4a16.{cuh,cu} + a gated hook in ggml_cuda_mul_mat (dense path), behind
GGML_CUDA_W4A16 + sm_120/121 + Q4_0/Q4_K + f32. Returns false -> MMQ, so the
default build is byte-identical. Verified on GB10: clean build, test-backend-ops
MUL_MAT 1103/1103, llama-bench pp512 unchanged (717.77 default / 718.26 flagged),
and GGML_CUDA_W4A16=1 reaches the seam ([w4a16] P1 warning) before falling back.
Source + apply steps under kernel/w4a16/ (DGX checkout is volatile). The frame the
P2 correctness kernel + P3 Marlin pipeline fill.

Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2026-06-20 21:46:38 +00:00

..

HOOK.md

kernel(P1): W4A16 dispatch seam (gated, byte-identical fallback to MMQ)

2026-06-20 21:46:38 +00:00

marlin-w4a16.cu

kernel(P1): W4A16 dispatch seam (gated, byte-identical fallback to MMQ)

2026-06-20 21:46:38 +00:00

marlin-w4a16.cuh

kernel(P1): W4A16 dispatch seam (gated, byte-identical fallback to MMQ)

2026-06-20 21:46:38 +00:00