feat(llama-cpp): add lazy_mode model option (#12743)

* feat(llama-cpp): add lazy_mode model option

Expose upstream --lazy-mode (on, auto, off) as a model option, for
example `lazy_mode:on`. It controls on-demand reading of large tensors
such as per-layer embeddings; `on` requires mmap.

The option is also accepted as `lazy-mode` or `lzm`. Invalid values log
a warning and keep the upstream default (auto). The pinned llama.cpp
already provides the lazy_mode field in common_params.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(llama-cpp): bump llama.cpp to c0fd78626405

Update the default llama.cpp pin to the current upstream master. The
changes since the old pin that the backend touches are additive: a new
common_decision_type value, new mtmd audio-stream and download helper
declarations. The two server patches still apply in order, and the
common_params fields and server headers the backend uses are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
authored and GitHub committed 2026-10-17 22:38:23 +02:00
1 parent e715af1def
commit e7cc993b0f
4 files changed
+46 -1

No files matched your search

+1 -1
View File
@@ -1,5 +1,5 @@
LLAMA_VERSION?=b76b13379d3b7d67ab0bc424516bb6fd2522bd8e
LLAMA_VERSION?=c0fd78626405c6d9b934efef0ec7291fcc20e3d6
LLAMA_REPO?=https://github.com/ggerganov/llama.cpp
CMAKE_ARGS?=