Ettore Di Giacinto baa106f457 docs(dllm): record Q4_K_M validation and quantization guidance ...

Q4_K_M validated on GB10: quality holds (cosine 0.9862, coherent
generation, 19/48 stopper exit) but a forward step is ~5x slower than
BF16 (27.5s vs 5.6s: native BF16 tensor cores vs K-quant MoE dequant).
Guidance: prefer BF16 when it fits; Q4_K_M is the memory-bound option.

Assisted-by: Claude Code (Fable 5)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2026-07-29 19:04:53 +00:00

..

_index.en.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

3d-generation.md

feat: Add 3d generation UI/API and trellis2cpp backend (#10979)

2026-07-29 16:15:04 +02:00

agent-actions.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

agents.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

api-discovery.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

audio-classification.md

fix(ci): repair nightly backend dep bumps for renamed localai-org repos (#11012)

2026-07-21 09:40:10 +02:00

audio-diarization.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

audio-to-text.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

audio-transform.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

authentication.md

feat(ui): replace the stacked operations bar with a one-line strip and an Activity page (#11163)

2026-07-28 19:33:48 +02:00

backends.md

feat(stores): add Valkey Search vector store backend (#11196)

2026-07-29 20:12:29 +02:00

constrained_grammars.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

distributed_inferencing.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

distributed-mode.md

fix(worker): report RAM alongside GPU memory (#11167)

2026-07-28 23:55:35 +02:00

embeddings.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

face-recognition.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

fine-tuning.md

fix(trl): disable inline GRPO reward code by default (RCE, #11015) (#11068)

2026-07-23 23:10:34 +02:00

gpt-vision.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

GPU-acceleration.md

docs(gpu): add ROCm 7.x and RDNA 3.5 / Strix Halo (gfx1151) to GPU acceleration guide (#9229)

2026-07-29 20:09:52 +02:00

image-generation.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

interleaved-thinking.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

localai-assistant.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

mcp.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

mlx-distributed.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

model-aliases.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

model-gallery.md

feat(ui): replace the stacked operations bar with a one-line strip and an Activity page (#11163)

2026-07-28 19:33:48 +02:00

object-detection.md

fix(ci): repair nightly backend dep bumps for renamed localai-org repos (#11012)

2026-07-21 09:40:10 +02:00

openai-functions.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

openai-realtime.md

feat(classifier/VAD): support voice control on low power devices (#10804)

2026-07-29 12:50:22 +02:00

quantization.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

reranker.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

runtime-settings.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

stores.md

feat(stores): add Valkey Search vector store backend (#11196)

2026-07-29 20:12:29 +02:00

text-generation.md

docs(dllm): record Q4_K_M validation and quantization guidance

2026-07-29 19:04:53 +00:00

text-to-audio.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

video-generation.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

voice-activity-detection.md

docs: onboarding overhaul, dedup, and error docs (#7711) (#10895)

2026-07-17 22:08:20 +02:00

voice-recognition.md

fix(ci): repair nightly backend dep bumps for renamed localai-org repos (#11012)

2026-07-21 09:40:10 +02:00