mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-27 18:06:58 -04:00
refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series
Move ALL paged-attention content out of the stock backend/cpp/llama-cpp backend and into backend/cpp/llama-cpp-localai-paged, so the stock backend is pure upstream llama.cpp and the paged backend owns and applies its own vendored patch series. - Delete the dead early-exploration scaffold backend/cpp/llama-cpp/paged/ (kernel/w4a16 Marlin scaffold, standalone paged_kv_manager, bench/loadgen, its own 0001-0002 patches, dense-era design docs, tests). Zero references repo-wide. - Move backend/cpp/llama-cpp/patches/ (the 28-patch paged series + paged/README + 3 operational docs, plus the kernel/ scaffold patch and the top-level paged README/BENCHMARKS) to backend/cpp/llama-cpp-localai-paged/patches/. The stock backend keeps no patches/ dir; it had no non-paged base patches. - Purify the stock backend: remove the LLAMA_PAGED make variable, the patches/paged apply loop, and the LLAMA_PAGED passthrough to prepare.sh; remove the paged-series handling from prepare.sh. The stock llama.cpp target now only clones the pin and applies its own (currently empty) base patches/ series. The runtime paged option hooks in the shared grpc-server.cpp are untouched (inert without the patches). - The paged backend's Makefile now applies its OWN patches/paged/0*.patch onto each freshly cloned tree via strict git apply (apply-paged-patches), after the copied stock infra clones the pin and applies base patches. - Repoint every reference to the old patches/paged path: the upstream canary workflow + apply script, bump_deps.yaml, gallery/index.yaml, the docs, backend/index.yaml, backend-matrix.yml, the top-level Makefile comments, and the moved PIN_SYNC / README docs. Drop the now-removed LLAMA_PAGED=on build-toggle from comments. Verified: the full 28-patch series applies strict-clean (git apply, exit 0) to a clean ggml-org/llama.cpp checkout at the pinned c299a92c, and the repointed canary apply script resolves and applies the series end to end. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
@@ -6,14 +6,6 @@
|
||||
# bump and is advanced only by the manual PIN_SYNC process.
|
||||
LLAMA_VERSION?=9d5d882d8cd0f0a9283d87ed5e6fe3ee0d925fb1
|
||||
LLAMA_REPO?=https://github.com/ggerganov/llama.cpp
|
||||
# LLAMA_PAGED controls whether the vendored paged-attention patch series
|
||||
# (patches/paged/) is applied on top of the pinned llama.cpp. Default on; set
|
||||
# LLAMA_PAGED=off to build a clean-against-upstream backend (e.g. to unblock a
|
||||
# dep-bump if an upstream change breaks a paged hook - the paged carry is then
|
||||
# fixed independently). Runtime behaviour stays gated by the LLAMA_KV_PAGED env
|
||||
# regardless, so an LLAMA_PAGED=on build is byte-identical to stock until that
|
||||
# env is set.
|
||||
LLAMA_PAGED?=on
|
||||
|
||||
CMAKE_ARGS?=
|
||||
BUILD_TYPE?=
|
||||
@@ -187,23 +179,14 @@ llama.cpp:
|
||||
[ -e "$$p" ] || continue; \
|
||||
echo "applying llama.cpp patch: $$p"; \
|
||||
git apply --verbose "$$p" || { echo "patch failed: $$p"; exit 1; }; \
|
||||
done && \
|
||||
if [ "$(LLAMA_PAGED)" = "off" ]; then \
|
||||
echo "LLAMA_PAGED=off: skipping paged-attention patch series"; \
|
||||
else \
|
||||
for p in $(CURRENT_MAKEFILE_DIR)patches/paged/0*.patch; do \
|
||||
[ -e "$$p" ] || continue; \
|
||||
echo "applying llama.cpp PAGED patch: $$p"; \
|
||||
git apply --verbose "$$p" || { echo "paged patch failed: $$p"; exit 1; }; \
|
||||
done; \
|
||||
fi
|
||||
done
|
||||
|
||||
llama.cpp/tools/grpc-server: llama.cpp
|
||||
mkdir -p llama.cpp/tools/grpc-server
|
||||
LLAMA_PAGED=$(LLAMA_PAGED) bash prepare.sh
|
||||
bash prepare.sh
|
||||
|
||||
rebuild:
|
||||
LLAMA_PAGED=$(LLAMA_PAGED) bash prepare.sh
|
||||
bash prepare.sh
|
||||
rm -rf grpc-server
|
||||
$(MAKE) grpc-server
|
||||
|
||||
|
||||
Reference in New Issue
Block a user