Ettore Di Giacinto cd6079b2f3 feat(backend): add buun-llama-cpp fork (DFlash + TCQ KV-cache) ...

spiritbuun/buun-llama-cpp is a fork of TheTom/llama-cpp-turboquant that adds
two independent features on top: DFlash block-diffusion speculative decoding
(via a dedicated DFlashDraftModel GGUF arch) and two extra TCQ KV-cache
variants (turbo2_tcq, turbo3_tcq) on top of TurboQuant's turbo2/turbo3/turbo4.

Follows the turboquant thin-wrapper pattern — reuses backend/cpp/llama-cpp
grpc-server sources verbatim, patches only the build copy to extend the KV
allow-list and wire up buun-exclusive tree_budget / draft_topk options.
DraftModel is already wired end-to-end (proto field 39 → params.speculative),
so DFlash activation only needs the existing options passthrough
(spec_type:dflash) plus the drafter path in draft_model.

CacheTypeOptions now surfaces the five turbo* values so the React UI dropdown
shows them — benefits turboquant too (previously users had to type them in
YAML manually).

Assisted-by: Claude:Opus-4.7 [Read] [Edit] [Bash] [WebFetch]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2026-04-24 12:52:53 +00:00

..

apply-patches.sh

feat(backend): add buun-llama-cpp fork (DFlash + TCQ KV-cache)

2026-04-24 12:52:53 +00:00

Makefile

feat(backend): add buun-llama-cpp fork (DFlash + TCQ KV-cache)

2026-04-24 12:52:53 +00:00

package.sh

feat(backend): add buun-llama-cpp fork (DFlash + TCQ KV-cache)

2026-04-24 12:52:53 +00:00

patch-grpc-server.sh

feat(backend): add buun-llama-cpp fork (DFlash + TCQ KV-cache)

2026-04-24 12:52:53 +00:00

run.sh

feat(backend): add buun-llama-cpp fork (DFlash + TCQ KV-cache)

2026-04-24 12:52:53 +00:00