mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 09:08:54 -04:00
feat(llama-cpp): add lazy_mode model option (#12743)
* feat(llama-cpp): add lazy_mode model option Expose upstream --lazy-mode (on, auto, off) as a model option, for example `lazy_mode:on`. It controls on-demand reading of large tensors such as per-layer embeddings; `on` requires mmap. The option is also accepted as `lazy-mode` or `lzm`. Invalid values log a warning and keep the upstream default (auto). The pinned llama.cpp already provides the lazy_mode field in common_params. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * chore(llama-cpp): bump llama.cpp to c0fd78626405 Update the default llama.cpp pin to the current upstream master. The changes since the old pin that the backend touches are additive: a new common_decision_type value, new mtmd audio-stream and download helper declarations. The two server patches still apply in order, and the common_params fields and server headers the backend uses are unchanged. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
4 files changed
+46
-1
No files matched your search
@@ -1,5 +1,5 @@
|
||||
|
||||
LLAMA_VERSION?=b76b13379d3b7d67ab0bc424516bb6fd2522bd8e
|
||||
LLAMA_VERSION?=c0fd78626405c6d9b934efef0ec7291fcc20e3d6
|
||||
LLAMA_REPO?=https://github.com/ggerganov/llama.cpp
|
||||
|
||||
CMAKE_ARGS?=
|
||||
|
||||
Reference in new issue
Block a user