mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-07-30 09:57:57 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(turboquant,bonsai): do not apply vendored llama.cpp patches to fork trees (#10866)

Browse Source 
The turboquant and bonsai backends copy backend/cpp/llama-cpp/ wholesale
into their build directories and reuse its Makefile/prepare.sh against
their own llama.cpp forks. When PR #10837 added
backend/cpp/llama-cpp/patches/0001-add-minimax-m3-support.patch, the
copied patches/ directory was mis-applied to the fork checkouts: the
fork trees diverge from upstream, hunks rejected, and because the
patch-apply loop in prepare.sh ran before set -e took effect the build
kept going and died much later with a confusing compile error
("'LLM_ARCH_MINIMAX_M3' was not declared in this scope"). This broke
tests-turboquant-grpc on that PR.

Two hardening changes:

- turboquant/bonsai Makefiles: delete the copied patches/ directory
  right after the cp -rf of backend/cpp/llama-cpp/. Patches vendored
  for upstream llama.cpp must never be applied to the forks; each fork
  carries its own patch series under backend/cpp/<backend>/patches/,
  applied by its apply-patches.sh.

- llama-cpp prepare.sh: run the patch-apply loop under set -e so a
  rejecting patch fails fast and loudly at apply time instead of
  surfacing as a downstream compile error. A missing or empty patches/
  directory remains a no-op success, so all existing callers (the
  llama-cpp Makefile targets and the turboquant/bonsai copies) are
  unaffected when no patches ship.

Exposed by PR #10837.


Assisted-by: Claude:claude-opus-4-8 [Claude Code]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
LocalAI [bot]
2026-07-17 00:28:10 +02:00
committed by GitHub
parent c1a891662c
commit 1f53dff436
3 changed files with 22 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
backend/cpp/bonsai/Makefile
View File

@@ -36,6 +36,10 @@ PATCHES_DIR := $(CURRENT_MAKEFILE_DIR)/patches
define bonsai-build
rm -rf $(CURRENT_MAKEFILE_DIR)/../bonsai-$(1)-build
cp -rf $(LLAMA_CPP_DIR) $(CURRENT_MAKEFILE_DIR)/../bonsai-$(1)-build
# Drop patches vendored for upstream llama.cpp: the fork tree diverges, so
# they reject there. Fork-specific patches live in backend/cpp/bonsai/patches/
# and are applied by apply-patches.sh below.
rm -rf $(CURRENT_MAKEFILE_DIR)/../bonsai-$(1)-build/patches
$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../bonsai-$(1)-build purge
$(info $(GREEN)I bonsai build info:$(1)$(RESET))
LLAMA_REPO=$(LLAMA_REPO) LLAMA_VERSION=$(BONSAI_VERSION) \
@@ -68,6 +72,10 @@ bonsai-fallback:
bonsai-cpu-all:
rm -rf $(CURRENT_MAKEFILE_DIR)/../bonsai-cpu-all-build
cp -rf $(LLAMA_CPP_DIR) $(CURRENT_MAKEFILE_DIR)/../bonsai-cpu-all-build
# Drop patches vendored for upstream llama.cpp: the fork tree diverges, so
# they reject there. Fork-specific patches live in backend/cpp/bonsai/patches/
# and are applied by apply-patches.sh below.
rm -rf $(CURRENT_MAKEFILE_DIR)/../bonsai-cpu-all-build/patches
$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../bonsai-cpu-all-build purge
$(info $(GREEN)I bonsai build info:cpu-all-variants$(RESET))
LLAMA_REPO=$(LLAMA_REPO) LLAMA_VERSION=$(BONSAI_VERSION) \

10
backend/cpp/llama-cpp/prepare.sh
View File

@@ -1,17 +1,19 @@
#!/bin/bash
set -e
## Patches
## Apply patches from the `patches` directory
## Apply patches from the `patches` directory. Runs under set -e so a
## rejected patch aborts the build here, loudly, instead of surfacing later
## as a confusing compile error. A missing or empty patches dir is a no-op.
if [ -d "patches" ]; then
for patch in $(ls patches); do
echo "Applying patch $patch"
patch -d llama.cpp/ -p1 < patches/$patch
done
done
fi
set -e
for file in $(ls llama.cpp/tools/server/); do
cp -rfv llama.cpp/tools/server/$file llama.cpp/tools/grpc-server/
done

8
backend/cpp/turboquant/Makefile
View File

@@ -37,6 +37,10 @@ PATCHES_DIR := $(CURRENT_MAKEFILE_DIR)/patches
define turboquant-build
rm -rf $(CURRENT_MAKEFILE_DIR)/../turboquant-$(1)-build
cp -rf $(LLAMA_CPP_DIR) $(CURRENT_MAKEFILE_DIR)/../turboquant-$(1)-build
# Drop patches vendored for upstream llama.cpp: the fork tree diverges, so
# they reject there. Fork-specific patches live in backend/cpp/turboquant/patches/
# and are applied by apply-patches.sh below.
rm -rf $(CURRENT_MAKEFILE_DIR)/../turboquant-$(1)-build/patches
$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../turboquant-$(1)-build purge
# Augment the copied grpc-server.cpp's KV-cache allow-list with the
# fork's turbo2/turbo3/turbo4 types. We patch the *copy*, never the
@@ -74,6 +78,10 @@ turboquant-fallback:
turboquant-cpu-all:
rm -rf $(CURRENT_MAKEFILE_DIR)/../turboquant-cpu-all-build
cp -rf $(LLAMA_CPP_DIR) $(CURRENT_MAKEFILE_DIR)/../turboquant-cpu-all-build
# Drop patches vendored for upstream llama.cpp: the fork tree diverges, so
# they reject there. Fork-specific patches live in backend/cpp/turboquant/patches/
# and are applied by apply-patches.sh below.
rm -rf $(CURRENT_MAKEFILE_DIR)/../turboquant-cpu-all-build/patches
$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../turboquant-cpu-all-build purge
bash $(CURRENT_MAKEFILE_DIR)/patch-grpc-server.sh $(CURRENT_MAKEFILE_DIR)/../turboquant-cpu-all-build/grpc-server.cpp
$(info $(GREEN)I turboquant build info:cpu-all-variants$(RESET))