mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-27 18:06:58 -04:00
refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series
Move ALL paged-attention content out of the stock backend/cpp/llama-cpp backend and into backend/cpp/llama-cpp-localai-paged, so the stock backend is pure upstream llama.cpp and the paged backend owns and applies its own vendored patch series. - Delete the dead early-exploration scaffold backend/cpp/llama-cpp/paged/ (kernel/w4a16 Marlin scaffold, standalone paged_kv_manager, bench/loadgen, its own 0001-0002 patches, dense-era design docs, tests). Zero references repo-wide. - Move backend/cpp/llama-cpp/patches/ (the 28-patch paged series + paged/README + 3 operational docs, plus the kernel/ scaffold patch and the top-level paged README/BENCHMARKS) to backend/cpp/llama-cpp-localai-paged/patches/. The stock backend keeps no patches/ dir; it had no non-paged base patches. - Purify the stock backend: remove the LLAMA_PAGED make variable, the patches/paged apply loop, and the LLAMA_PAGED passthrough to prepare.sh; remove the paged-series handling from prepare.sh. The stock llama.cpp target now only clones the pin and applies its own (currently empty) base patches/ series. The runtime paged option hooks in the shared grpc-server.cpp are untouched (inert without the patches). - The paged backend's Makefile now applies its OWN patches/paged/0*.patch onto each freshly cloned tree via strict git apply (apply-paged-patches), after the copied stock infra clones the pin and applies base patches. - Repoint every reference to the old patches/paged path: the upstream canary workflow + apply script, bump_deps.yaml, gallery/index.yaml, the docs, backend/index.yaml, backend-matrix.yml, the top-level Makefile comments, and the moved PIN_SYNC / README docs. Drop the now-removed LLAMA_PAGED=on build-toggle from comments. Verified: the full 28-patch series applies strict-clean (git apply, exit 0) to a clean ggml-org/llama.cpp checkout at the pinned c299a92c, and the repointed canary apply script resolves and applies the series end to end. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
2
.github/workflows/bump_deps.yaml
vendored
2
.github/workflows/bump_deps.yaml
vendored
@@ -11,7 +11,7 @@ jobs:
|
||||
matrix:
|
||||
# NOTE: there is intentionally NO entry for the llama-cpp-localai-paged
|
||||
# backend. It carries a vendored paged-attention patch series
|
||||
# (backend/cpp/llama-cpp/patches/paged/) hand-verified bit-exact against
|
||||
# (backend/cpp/llama-cpp-localai-paged/patches/paged/) hand-verified bit-exact against
|
||||
# ONE specific llama.cpp tip; a naive nightly bump would move the tip out
|
||||
# from under the patches and break `git apply` at build time. Its pin is
|
||||
# therefore decoupled (its own LLAMA_VERSION in
|
||||
|
||||
33
.github/workflows/llama-cpp-paged-canary.yml
vendored
33
.github/workflows/llama-cpp-paged-canary.yml
vendored
@@ -1,7 +1,7 @@
|
||||
name: 'llama.cpp paged patches: upstream canary'
|
||||
|
||||
# EARLY-WARNING CANARY for the vendored paged-attention patch series
|
||||
# (backend/cpp/llama-cpp/patches/paged/0001-0030).
|
||||
# (backend/cpp/llama-cpp-localai-paged/patches/paged/0001-0030).
|
||||
#
|
||||
# WHY THIS EXISTS
|
||||
# The paged backend (backend/cpp/llama-cpp-localai-paged) pins its OWN verified
|
||||
@@ -17,7 +17,7 @@ name: 'llama.cpp paged patches: upstream canary'
|
||||
# RED HERE means: time to run a PIN_SYNC (rebase the patches onto the new tip,
|
||||
# pass the bit-exact gate on the GPU, re-export the .patch files, THEN advance
|
||||
# the pin in backend/cpp/llama-cpp-localai-paged/Makefile). See
|
||||
# backend/cpp/llama-cpp/patches/paged/PIN_SYNC_c299a92c.md.
|
||||
# backend/cpp/llama-cpp-localai-paged/patches/paged/PIN_SYNC_c299a92c.md.
|
||||
#
|
||||
# SIGNAL-ONLY: this workflow moves no pinned version, ships nothing, and is fully
|
||||
# decoupled from bump_deps - so the main dep-bump PR stays green regardless. A
|
||||
@@ -91,7 +91,7 @@ jobs:
|
||||
run: |
|
||||
bash .github/scripts/paged-canary-apply.sh \
|
||||
/tmp/llama.cpp \
|
||||
"$PWD/backend/cpp/llama-cpp/patches"
|
||||
"$PWD/backend/cpp/llama-cpp-localai-paged/patches"
|
||||
echo "- apply: full paged series applies to the upstream tip :white_check_mark:" >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
compile:
|
||||
@@ -141,12 +141,16 @@ jobs:
|
||||
cp -a /opt/grpc/. /usr/local/
|
||||
|
||||
# Pre-populate the llama.cpp checkout at the latest tip with the
|
||||
# paged series applied via the tolerant canary apply (so the benign
|
||||
# 0019 dev-doc hunk does not abort the build). Because
|
||||
# backend/cpp/llama-cpp/llama.cpp now exists, the Makefile
|
||||
# llama.cpp target (strict clone + git apply) is skipped and
|
||||
# prepare.sh sees the paged sentinel and skips re-applying - so we
|
||||
# drive the REAL grpc-server build path on top of our apply.
|
||||
# paged series applied via the tolerant canary apply. Because
|
||||
# backend/cpp/llama-cpp/llama.cpp now exists, the stock Makefile's
|
||||
# llama.cpp target (clone + base-patch apply) is skipped and the
|
||||
# now patch-free prepare.sh only copies the grpc-server sources -
|
||||
# so we drive the REAL grpc-server build path on top of our paged
|
||||
# apply. The stock llama-cpp backend no longer carries the paged
|
||||
# series (it lives in backend/cpp/llama-cpp-localai-paged/patches/
|
||||
# paged); we build it here in the stock dir only because that is
|
||||
# where the shared build infra (Makefile / grpc-server.cpp /
|
||||
# CMakeLists.txt / prepare.sh) lives.
|
||||
cd backend/cpp/llama-cpp/
|
||||
mkdir -p llama.cpp
|
||||
cd llama.cpp
|
||||
@@ -157,15 +161,16 @@ jobs:
|
||||
cd /LocalAI
|
||||
bash .github/scripts/paged-canary-apply.sh \
|
||||
backend/cpp/llama-cpp/llama.cpp \
|
||||
"$PWD/backend/cpp/llama-cpp/patches"
|
||||
"$PWD/backend/cpp/llama-cpp-localai-paged/patches"
|
||||
|
||||
# Cheapest real CUDA build that proves the patches compile: one
|
||||
# CUDA arch, cublas, paged on. CMAKE_ARGS is passed via the
|
||||
# environment (not as a make arg) so the Makefile += flags are
|
||||
# still appended, exactly like .docker/llama-cpp-localai-paged-compile.sh.
|
||||
# CUDA arch, cublas. CMAKE_ARGS is passed via the environment (not
|
||||
# as a make arg) so the Makefile += flags are still appended,
|
||||
# exactly like .docker/llama-cpp-localai-paged-compile.sh. The paged
|
||||
# series is already applied to the checkout above, so the stock
|
||||
# build just compiles the patched tree.
|
||||
cd backend/cpp/llama-cpp/
|
||||
BUILD_TYPE=cublas \
|
||||
LLAMA_PAGED=on \
|
||||
CMAKE_ARGS="-DCMAKE_CUDA_ARCHITECTURES=80" \
|
||||
make grpc-server
|
||||
test -x grpc-server
|
||||
|
||||
Reference in New Issue
Block a user