mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-12 22:33:54 -04:00
mudler's LocalAI [bot]andEttore Di Giacinto
c61b6f2286
* docs(blog): new DeepSeek and Laguna numbers, visuals, humanizer pass vllm.cpp master moved 26 commits past what the post was written against, and two results changed enough to matter. Both came from the same lever: staging weights device-resident at load instead of reading them from the GGUF mmap over unified memory, which the GB10 reads about 20% slower per GEMV than device memory. - DeepSeek-V4-Flash against DwarfStar: 0.997x parity becomes 1.144x ahead, 18.69 vs 16.33 tok/s decode, same generated tokens. - Laguna-XS-2.1 against vLLM: 87% becomes 1.03x, 44.46 vs 43.10 tok/s. New row in the scoreboard. Adds three visuals. A chart of throughput against every reference engine, which is worth having now that the spread is 0.976 to 1.144 rather than a flat line at parity. The Activity page with four installs running, and the model detail pane with all four pocket-35b variants. Both screenshots were recaptured on 2026-08-04 because #11288, #11305, #11307 and #11222 had all changed those pages since the earlier set. llama.cpp is deliberately absent from the chart: its 1.18x is a prefill ratio, and putting it on the same axis as throughput ratios would be comparing two different measurements. Also carries the media the release notes embed, since a GitHub release body needs URLs that survive publishing and drag-and-drop has no CLI. Supersedes #11364. Humanizer pass on the prose. The post had collected five exactness idioms in one section (token-for-token, byte-exact twice, byte-identical, token-identical). One is precision, five is a tic, so the 27B row keeps its "token-for-token identical" where identical output is the actual claim and the rest say what they mean. That also fixed a hyphen in predicate position ("is token-identical"). Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] * docs(blog): redraw the benchmark chart as a branded card The Flint bar chart was generic: default palette, no brand, and drawn from zero, which made five ratios between 0.976 and 1.144 look like five bars of roughly equal length. Redrawn in the style of recorder-for-agents' render-card.sh cards, the same shape as the vllm.cpp README GIF. Palette taken from the two logos rather than invented (LocalAI navy #0E2632 and teal #469AAF, vllm.cpp teal #3AB4CA), SVG generated by a small JS loop so the geometry is exact at any scale, headless Chrome to PNG at 2x. The substantive change is that bars now run from the 1.00 parity line instead of from zero. Deviation is what the data is about, so DeepSeek's +14.4% and MLX-LM's -2.4% are both legible, and the one row that is behind is the one row in amber. Each bar carries its ratio and the raw measurement under it. Keeps the .html source next to the .png so the chart is editable later: change a number, re-run render-card.sh. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash] --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>