mirror of
https://github.com/mudler/LocalAI.git
synced 2026-08-02 03:20:12 -04:00
1edbbae891
The previous commit quoted 99.1% of MLX-LM for the prefill-gated MLX build. That figure divided by a two-run MLX-LM baseline, 27.135 and 27.744 generation tok/s averaged to 27.44. Re-measured interleaved with ours over four ABBA blocks, MLX-LM's decode is 27.848 with a 0.34% spread across six runs, so the 27.135 was an outlier and averaging it in overstated us by roughly 1.5 points. Corrected: the gated configuration is 24.37 tok/s, or 97.6% of MLX-LM, and the MLX-off build is 23.9 tok/s or 95.9%. Prefill TTFT is unchanged at 524.5 ms against MLX-LM's 532.6, so we remain about 1.5% faster there. Nothing else changes. MLX still wins prefill and loses decode, the shape gate is still the right disposition, and the pin and the flag are still coupled. The gate is worth about 1.7 points over the MLX-off build rather than 2.7. Assisted-by: Claude Code:claude-opus-5 [ClaudeCode] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>