mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-27 09:57:14 -04:00
62c407ed55
Anchors the rigorous same-session A/B validation of patch 0028 (residual conv-state
tap k_get_rows fusion) on this worktree branch with sign-off attribution. The
regenerated 0028 patch + bench-updated LEVER1_GATHER_RESULTS.md first landed via a
concurrent origin/master merge (c1f1d1e8e) that swept the staged files; this records
the provenance and the bench summary in the checkpoint.
Gate (bit-exact, greedy --temp 0 --seed 1 -n 48): dense q36-27b-nvfp4
5951a5b4d624ce891e22ab5fca9bc439, MoE q36-35b-a3b-nvfp4 07db32c2bcb78d17a43ed18bc22705cd
(both == baseline; base == lever1). decode_agg npl128: dense 369.95 -> 377.83 t/s
(+2.13%, 96.6% of vLLM), MoE 763.47 -> 777.95 t/s (+1.90%, 86.3% of vLLM). nsys MoE
decode: k_get_rows_float 17334 -> 15414 inst (-1920), 358.37 -> 133.52 ms, step -3.13 ms.
Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>