mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-07-31 10:28:43 -04:00
Code Issues Packages Projects Releases Wiki Activity

chore(llama-cpp): bump llama.cpp and adapt to the load-mode refactor (#11140)

Browse Source 
Bump LLAMA_VERSION to 0d47ea7427463093e69128bf2c2f9cd06b3ee5b3 (73 commits
touching common/, src/ and tools/server/). Two upstream changes break the
backend:

* ggml-org/llama.cpp#20834 folded common_params::use_mmap / use_mlock /
  use_direct_io into a single `load_mode` enum. LocalAI still exposes the three
  as independent settings (`mmap`, `mmlock`, and the `direct_io` option), so
  params_parse folds them once all three have been read, keeping the precedence
  the separate booleans had: direct I/O bypasses the page cache, mlock implies
  mmap, everything off is a plain buffered read. turboquant and bonsai compile
  this same grpc-server.cpp against forks that predate the refactor, so
  prepare.sh probes the checkout for LLAMA_LOAD_MODE_MMAP and generates
  llama_compat.h with LOCALAI_LEGACY_LOAD_MODE set accordingly. Probing beats a
  per-fork build flag here because the fork flavor targets disagree on whether
  they forward CMAKE_ARGS or EXTRA_CMAKE_ARGS, and it heals itself once a fork
  rebases past the refactor.

* The MiniMax M3 patch no longer applies. Upstream merged the model half of
  llama.cpp#24523 (LLM_ARCH_MINIMAX_M3, src/models/minimax-m3.cpp, the gguf-py
  constants and conversion/minimax.py) but not the chat half, so the patch is
  re-cut to carry only the common/chat.cpp template detection and PEG parser,
  rebased onto the new pin and onto the thinking_end_tag -> thinking_end_tags
  rename. Dropping it wholesale (as #11008 did, reverted in #11136) would have
  silently regressed MiniMax M3 tool calling and thinking.

Verified with a CPU docker build of the backend plus LoadModel and Predict
against a real GGUF over gRPC in all four load modes.


Assisted-by: Claude:claude-opus-5 [Claude Code]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
mudler's LocalAI [bot]
2026-07-27 09:02:29 +02:00
committed by GitHub
parent d9f3007876
commit 0a8a7fbbb4
5 changed files with 278 additions and 819 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
backend/cpp/llama-cpp/prepare.sh
View File

@@ -31,6 +31,26 @@ cp -r parent_watch_test.cpp llama.cpp/tools/grpc-server/
cp -rfv llama.cpp/vendor/nlohmann/json.hpp llama.cpp/tools/grpc-server/
cp -rfv llama.cpp/vendor/cpp-httplib/httplib.h llama.cpp/tools/grpc-server/
## Fork-skew probe. Upstream folded common_params::use_mmap / use_mlock /
## use_direct_io into a single `load_mode` enum (ggml-org/llama.cpp#20834).
## turboquant and bonsai compile this very same grpc-server.cpp against forks
## that branched before that change, so the field set is decided from the
## checkout in front of us rather than from a per-fork build flag: the flavor
## targets disagree on whether they forward CMAKE_ARGS or EXTRA_CMAKE_ARGS, and
## probing heals itself the moment a fork rebases past the refactor.
if grep -q "LLAMA_LOAD_MODE_MMAP" llama.cpp/include/llama.h; then
echo "==> llama.cpp carries the load-mode enum, using common_params::load_mode"
LEGACY_LOAD_MODE=0
else
echo "==> llama.cpp predates the load-mode enum, using the legacy mmap/mlock/direct-io booleans"
LEGACY_LOAD_MODE=1
fi
cat > llama.cpp/tools/grpc-server/llama_compat.h <<EOF
// Generated by backend/cpp/llama-cpp/prepare.sh. Do not edit.
#pragma once
#define LOCALAI_LEGACY_LOAD_MODE ${LEGACY_LOAD_MODE}
EOF
set +e
if grep -q "grpc-server" llama.cpp/tools/CMakeLists.txt; then
echo "grpc-server already added"