LocalAI

mirror/LocalAI

0 Fork 0

mirror of https://github.com/mudler/LocalAI.git synced 2026-10-28 17:18:49 -04:00

Files

T

History

e7cc993b0f feat(llama-cpp): add lazy_mode model option (#12743 )

* feat(llama-cpp): add lazy_mode model option

Expose upstream --lazy-mode (on, auto, off) as a model option, for
example `lazy_mode:on`. It controls on-demand reading of large tensors
such as per-layer embeddings; `on` requires mmap.

The option is also accepted as `lazy-mode` or `lzm`. Invalid values log
a warning and keep the upstream default (auto). The pinned llama.cpp
already provides the lazy_mode field in common_params.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* chore(llama-cpp): bump llama.cpp to c0fd78626405

Update the default llama.cpp pin to the current upstream master. The
changes since the old pin that the backend touches are additive: a new
common_decision_type value, new mtmd audio-stream and download helper
declarations. The two server patches still apply in order, and the
common_params fields and server headers the backend uses are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

2026-10-17 22:38:23 +02:00

_index.en.md

docs: fix dead anchors and a dead section link (#11885 )

2026-09-05 23:47:13 +02:00

advanced-usage.md

docs: point the config example link at the configurations directory

2026-09-27 19:06:33 +00:00

embedded-routing-log.md

feat(router): route with native decision models (#12449 )

2026-10-04 09:34:21 +02:00

model-configuration.md

feat(llama-cpp): add lazy_mode model option (#12743 )