mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-27 18:06:58 -04:00
78fac9a28f
Move ALL paged-attention content out of the stock backend/cpp/llama-cpp backend and into backend/cpp/llama-cpp-localai-paged, so the stock backend is pure upstream llama.cpp and the paged backend owns and applies its own vendored patch series. - Delete the dead early-exploration scaffold backend/cpp/llama-cpp/paged/ (kernel/w4a16 Marlin scaffold, standalone paged_kv_manager, bench/loadgen, its own 0001-0002 patches, dense-era design docs, tests). Zero references repo-wide. - Move backend/cpp/llama-cpp/patches/ (the 28-patch paged series + paged/README + 3 operational docs, plus the kernel/ scaffold patch and the top-level paged README/BENCHMARKS) to backend/cpp/llama-cpp-localai-paged/patches/. The stock backend keeps no patches/ dir; it had no non-paged base patches. - Purify the stock backend: remove the LLAMA_PAGED make variable, the patches/paged apply loop, and the LLAMA_PAGED passthrough to prepare.sh; remove the paged-series handling from prepare.sh. The stock llama.cpp target now only clones the pin and applies its own (currently empty) base patches/ series. The runtime paged option hooks in the shared grpc-server.cpp are untouched (inert without the patches). - The paged backend's Makefile now applies its OWN patches/paged/0*.patch onto each freshly cloned tree via strict git apply (apply-paged-patches), after the copied stock infra clones the pin and applies base patches. - Repoint every reference to the old patches/paged path: the upstream canary workflow + apply script, bump_deps.yaml, gallery/index.yaml, the docs, backend/index.yaml, backend-matrix.yml, the top-level Makefile comments, and the moved PIN_SYNC / README docs. Drop the now-removed LLAMA_PAGED=on build-toggle from comments. Verified: the full 28-patch series applies strict-clean (git apply, exit 0) to a clean ggml-org/llama.cpp checkout at the pinned c299a92c, and the repointed canary apply script resolves and applies the series end to end. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
78 lines
3.5 KiB
Bash
Executable File
78 lines
3.5 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# paged-canary-apply.sh - apply the vendored paged-attention patch series
|
|
# (backend/cpp/llama-cpp-localai-paged/patches/paged/0001-0030) to a llama.cpp checkout, the
|
|
# same way the build does, but tolerating the ONE known-benign pre-existing
|
|
# quirk in the series. Used by the early-warning canary
|
|
# (.github/workflows/llama-cpp-paged-canary.yml) so it only goes red on a REAL
|
|
# upstream break, never on that quirk.
|
|
#
|
|
# Usage: paged-canary-apply.sh <llama.cpp-checkout-dir> <patches-dir>
|
|
# <patches-dir> is normally backend/cpp/llama-cpp-localai-paged/patches (it holds the
|
|
# top-level base series 0*.patch, currently empty, and the paged/ subseries).
|
|
#
|
|
# Exit 0 = the whole series applied -> patches still fit upstream.
|
|
# Exit !=0 = a patch failed to apply = the red signal: an upstream change moved
|
|
# the tree out from under the patches, so it is time to run a PIN_SYNC.
|
|
#
|
|
# Apply method MIRRORS backend/cpp/llama-cpp/Makefile's `llama.cpp` target:
|
|
# plain `git apply --verbose`, which natively tolerates @@ line-number offsets
|
|
# but NOT context-line changes. Matching the build's method is the point - the
|
|
# canary's apply result is exactly what the real build's apply would do.
|
|
#
|
|
# The ONLY tolerance, and it is path-scoped (not a blanket `|| true`): patch
|
|
# 0019 carries a stray *modify* hunk against the dev-only doc
|
|
# SSM_DECODE_FIX_RESULTS.md, a file that exists only on the DGX dev tree and is
|
|
# absent from any clean upstream checkout. `git apply` is atomic, so that single
|
|
# missing-file hunk rejects the whole patch - and because 0021/0022/0026/0028
|
|
# build on 0019's code, the rejection cascades to them too. This is a
|
|
# PRE-EXISTING shipped-series defect, present identically on every pin, NOT an
|
|
# upstream break (see backend/cpp/llama-cpp-localai-paged/patches/paged/PIN_SYNC_c299a92c.md
|
|
# and README.md). We exclude ONLY that dev-doc path and still
|
|
# apply 0019's real code hunks atomically, so a genuine code-hunk break in 0019
|
|
# still fails the canary. prepare.sh tolerates the same hunk via
|
|
# `patch ... || true`; this mirrors that tolerance precisely.
|
|
|
|
set -euo pipefail
|
|
|
|
CHECKOUT="${1:?usage: paged-canary-apply.sh <llama.cpp-checkout> <patches-dir>}"
|
|
PATCHES="${2:?usage: paged-canary-apply.sh <llama.cpp-checkout> <patches-dir>}"
|
|
|
|
# The lone tolerated dev-doc, and the only patch allowed to carry it.
|
|
DEVDOC_GLOB='*SSM_DECODE_FIX_RESULTS.md'
|
|
DEVDOC_PATCH='0019-qwen35-ssm-decode-fused-gather.patch'
|
|
|
|
# Resolve to absolute paths so the apply works after we cd into the checkout.
|
|
PATCHES="$(cd "$PATCHES" && pwd)"
|
|
cd "$CHECKOUT"
|
|
|
|
shopt -s nullglob
|
|
|
|
apply_one() {
|
|
local p="$1"; shift
|
|
echo "paged-canary: applying $(basename "$p")"
|
|
if ! git apply --verbose "$@" "$p"; then
|
|
echo "::error::paged patch no longer applies to the upstream llama.cpp tip: $(basename "$p")"
|
|
echo "::error::upstream drifted past the vendored paged series - run a PIN_SYNC (backend/cpp/llama-cpp-localai-paged/patches/paged/PIN_SYNC_c299a92c.md), do NOT bump the pin blindly"
|
|
exit 1
|
|
fi
|
|
}
|
|
|
|
# Base series first (parity with the build: patches/0*.patch before
|
|
# patches/paged/0*.patch). Currently empty; nullglob makes this a no-op.
|
|
for p in "$PATCHES"/0*.patch; do
|
|
apply_one "$p"
|
|
done
|
|
|
|
# Paged series, in order.
|
|
for p in "$PATCHES"/paged/0*.patch; do
|
|
if [ "$(basename "$p")" = "$DEVDOC_PATCH" ]; then
|
|
# Apply 0019's real code hunks; exclude ONLY the benign dev-doc hunk.
|
|
apply_one "$p" --exclude="$DEVDOC_GLOB"
|
|
else
|
|
apply_one "$p"
|
|
fi
|
|
done
|
|
|
|
echo "paged-canary: the full paged patch series applied cleanly to the upstream tip"
|