LocalAI

mirror of https://github.com/mudler/LocalAI.git synced 2026-06-23 16:19:07 -04:00

Files

Ettore Di Giacinto ce60737fc5 kernel(doc): dense scope resolved - two FP4 kernels (dense first, then grouped)

Benchmark confirms dense prefill 7.6-32x behind too, so the kernel track needs a
non-grouped FP4 dense GEMM (simpler, land first) + the MoE grouped GEMM. Both
share the e2m1 block-scaled collective; dense is grouped-with-one-group.

Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2026-06-20 03:56:33 +00:00

paged

kernel(doc): dense scope resolved - two FP4 kernels (dense first, then grouped)

2026-06-20 03:56:33 +00:00

patches

bench(dense): Qwen3-32B dense parity - dense has the kernel gap too (PP 7.6-32x)

2026-06-20 03:55:58 +00:00

CMakeLists.txt

fix(turboquant): resolve common.h by detecting llama-common vs common target (#9413 )

2026-04-18 20:30:28 +02:00

grpc-server.cpp

feat: generic chat_template_kwargs (model config + per-request metadata) (#10359 )