3 people e7cc993b0f feat(llama-cpp): add lazy_mode model option (#12743) ...

* feat(llama-cpp): add lazy_mode model option

Expose upstream --lazy-mode (on, auto, off) as a model option, for
example `lazy_mode:on`. It controls on-demand reading of large tensors
such as per-layer embeddings; `on` requires mmap.

The option is also accepted as `lazy-mode` or `lzm`. Invalid values log
a warning and keep the upstream default (auto). The pinned llama.cpp
already provides the lazy_mode field in common_params.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(llama-cpp): bump llama.cpp to c0fd78626405

Update the default llama.cpp pin to the current upstream master. The
changes since the old pin that the backend touches are additive: a new
common_decision_type value, new mtmd audio-stream and download helper
declarations. The two server patches still apply in order, and the
common_params fields and server headers the backend uses are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

2026-10-17 22:38:23 +02:00

..

patches

chore: ⬆️ Update ggml-org/llama.cpp to 51ce9c11a6f2dfa895696c0048c4333e8953b728 (#12461)

2026-10-08 16:40:43 +02:00

tests

feat(router): route with native decision models (#12449)

2026-10-04 09:34:21 +02:00

CMakeLists.txt

feat(llama-cpp): add slot prompt-cache save and restore API (#12730)

2026-10-17 18:23:02 +02:00

decision_compat_test.cpp

feat(router): route with native decision models (#12449)

2026-10-04 09:34:21 +02:00

decision_compat.h

feat(router): route with native decision models (#12449)

2026-10-04 09:34:21 +02:00

decision_images.h

feat(router): route with native decision models (#12449)

2026-10-04 09:34:21 +02:00

disable-score-task.sh

feat(classifier/VAD): support voice control on low power devices (#10804)

2026-07-29 12:50:22 +02:00

disable-tts-task.sh

feat(llama-cpp): serve Qwen3-TTS through the llama.cpp backend (#11392)

2026-08-10 10:18:47 +02:00

grpc-server.cpp

feat(llama-cpp): add lazy_mode model option (#12743)

2026-10-17 22:38:23 +02:00

Makefile

feat(llama-cpp): add lazy_mode model option (#12743)

2026-10-17 22:38:23 +02:00

message_content_test.cpp

fix(llama-cpp): stop reinterpreting plain-string message content as JSON (#10524) (#10538)

2026-06-27 01:42:05 +02:00

message_content.h

chore: ⬆️ Update ggml-org/llama.cpp to f280b26983ad0fdb705a0d9ebf0503e76f2899b0 (#11646)

2026-08-25 12:57:12 +02:00

model_load_error_test.cpp

fix(llama-cpp): explain tensor count mismatch

2026-09-11 21:55:14 +00:00

model_load_error.h

fix(llama-cpp): explain tensor count mismatch

2026-09-11 21:55:14 +00:00

moe_cache_option_test.cpp

feat(llama-cpp): bump llama.cpp and add moe_cache_mib option (#12729)

2026-10-16 00:55:55 +02:00

moe_cache_option.h

feat(llama-cpp): bump llama.cpp and add moe_cache_mib option (#12729)

2026-10-16 00:55:55 +02:00

package.sh

feat(router): route with native decision models (#12449)

2026-10-04 09:34:21 +02:00

parallel_params_test.cpp

fix(llama-cpp): let parallel:1 in the model options win over LLAMACPP_PARALLEL (#12426)

2026-10-02 09:17:44 +02:00

parallel_params.h

fix(llama-cpp): let parallel:1 in the model options win over LLAMACPP_PARALLEL (#12426)

2026-10-02 09:17:44 +02:00

parent_watch_test.cpp

fix(process): give backend workers a parent-death safety net (#10639)

2026-07-02 19:16:48 +02:00

parent_watch.h

fix(process): give backend workers a parent-death safety net (#10639)

2026-07-02 19:16:48 +02:00

passthrough_options_test.cpp

fix(llama-cpp): preserve GPU layers during option passthrough (#11193)

2026-07-30 15:54:05 +02:00

passthrough_options.h

fix(llama-cpp): preserve GPU layers during option passthrough (#11193)

2026-07-30 15:54:05 +02:00

prepare.sh

feat(llama-cpp): add lazy_mode model option (#12743)

2026-10-17 22:38:23 +02:00

run.sh

fix(rocm): point *_TENSILE_LIBPATH at the architecture folder (#12411)

2026-10-05 01:38:09 +02:00

slot_cache_test.cpp

feat(llama-cpp): add slot prompt-cache save and restore API (#12730)

2026-10-17 18:23:02 +02:00

slot_cache.h

feat(llama-cpp): add slot prompt-cache save and restore API (#12730)

2026-10-17 18:23:02 +02:00

stream_peer_test.cpp

fix(llama-cpp): stop a generation whose stream is gone

2026-09-11 23:13:32 +00:00

stream_peer.h

fix(llama-cpp): stop a generation whose stream is gone

2026-09-11 23:13:32 +00:00

thread_params_test.cpp

chore: ⬆️ Update ggml-org/llama.cpp to 0021a77de0a8966059dc94548fb3b96654e0bb12 (#11508)

2026-08-18 09:45:56 +02:00

thread_params.h

chore: ⬆️ Update ggml-org/llama.cpp to 0021a77de0a8966059dc94548fb3b96654e0bb12 (#11508)

2026-08-18 09:45:56 +02:00

tts_request_options_test.cpp

feat(llama-cpp): serve Qwen3-TTS through the llama.cpp backend (#11392)

2026-08-10 10:18:47 +02:00

tts_request_options.h

feat(llama-cpp): serve Qwen3-TTS through the llama.cpp backend (#11392)

2026-08-10 10:18:47 +02:00