Ettore Di Giacinto 595e448714 docs(llama.cpp): note tensor split now works with quantized KV cache (#10135) ...

The split_mode: tensor description claimed tensor parallelism requires
KV-cache quantization to be disabled. ggml-org/llama.cpp#23792 lifts that
restriction by extending the meta backend to preserve shape information
through KV-cache flatten/reshape, so cache_type_k/cache_type_v
quantization can be combined with -sm tensor on builds that include it.

Documentation only: no backend code, grpc-server.cpp comment, or
llama.cpp pin changes.


Assisted-by: Claude Code:claude-opus-4-8

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2026-06-02 15:52:23 +02:00

..

advanced

docs: fix documentation typos (#10125)

2026-06-01 14:31:08 +02:00

features

docs(llama.cpp): note tensor split now works with quantized KV cache (#10135)

2026-06-02 15:52:23 +02:00

getting-started

feat(rocm): bump to 7.x (#9323)

2026-04-12 08:51:30 +02:00

installation

feat(ui): Interactive model config editor with autocomplete (#9149)

2026-04-07 14:42:23 +02:00

reference

fix(distributed): correct VRAM/RAM reporting on NVIDIA unified-memory hosts (#9545)

2026-04-24 22:02:23 +02:00

_index.md

chore(docs): center video

2025-12-08 16:59:11 +01:00

faq.md

feat: docs revamp (#7313)

2025-11-19 22:21:20 +01:00

integrations.md

fix(anthropic): do not emit empty tokens and fix SSE tool calls (#9258)

2026-04-07 00:38:21 +02:00

overview.md

docs: credit the LocalAI maintainers team

2026-05-02 23:37:04 +00:00

whats-new.md

docs: fix documentation typos (#10125)

2026-06-01 14:31:08 +02:00