refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

Move ALL paged-attention content out of the stock backend/cpp/llama-cpp
backend and into backend/cpp/llama-cpp-localai-paged, so the stock backend is
pure upstream llama.cpp and the paged backend owns and applies its own vendored
patch series.

- Delete the dead early-exploration scaffold backend/cpp/llama-cpp/paged/
  (kernel/w4a16 Marlin scaffold, standalone paged_kv_manager, bench/loadgen,
  its own 0001-0002 patches, dense-era design docs, tests). Zero references
  repo-wide.
- Move backend/cpp/llama-cpp/patches/ (the 28-patch paged series + paged/README
  + 3 operational docs, plus the kernel/ scaffold patch and the top-level paged
  README/BENCHMARKS) to backend/cpp/llama-cpp-localai-paged/patches/. The stock
  backend keeps no patches/ dir; it had no non-paged base patches.
- Purify the stock backend: remove the LLAMA_PAGED make variable, the
  patches/paged apply loop, and the LLAMA_PAGED passthrough to prepare.sh;
  remove the paged-series handling from prepare.sh. The stock llama.cpp target
  now only clones the pin and applies its own (currently empty) base patches/
  series. The runtime paged option hooks in the shared grpc-server.cpp are
  untouched (inert without the patches).
- The paged backend's Makefile now applies its OWN patches/paged/0*.patch onto
  each freshly cloned tree via strict git apply (apply-paged-patches), after the
  copied stock infra clones the pin and applies base patches.
- Repoint every reference to the old patches/paged path: the upstream canary
  workflow + apply script, bump_deps.yaml, gallery/index.yaml, the docs,
  backend/index.yaml, backend-matrix.yml, the top-level Makefile comments, and
  the moved PIN_SYNC / README docs. Drop the now-removed LLAMA_PAGED=on
  build-toggle from comments.

Verified: the full 28-patch series applies strict-clean (git apply, exit 0) to
a clean ggml-org/llama.cpp checkout at the pinned c299a92c, and the repointed
canary apply script resolves and applies the series end to end.

Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

backend/cpp/llama-cpp-localai-paged/Makefile

@@ -1,33 +1,36 @@
 
 # llama-cpp-localai-paged is LocalAI's paged-attention llama.cpp variant. It
 # builds upstream llama.cpp with the LocalAI paged-attention patch series
-# (backend/cpp/llama-cpp/patches/paged/) applied on top (LLAMA_PAGED=on). It
-# reuses backend/cpp/llama-cpp's grpc-server.cpp / CMakeLists.txt / prepare.sh
-# sources verbatim via a thin wrapper.
+# (patches/paged/, vendored in THIS backend) applied on top. It reuses
+# backend/cpp/llama-cpp's grpc-server.cpp / CMakeLists.txt / prepare.sh / Makefile
+# sources verbatim via a thin wrapper - the stock llama-cpp backend is pure
+# upstream and carries NONE of the paged patches; this backend OWNS them.
 #
 # Pin handling (mirrors the turboquant wrapper, the precedent this is modelled
 # on): the paged patch series is hand-verified bit-exact against ONE specific
 # llama.cpp tip and re-exported by the manual PIN_SYNC process
-# (backend/cpp/llama-cpp/patches/paged/PIN_SYNC_*.md). A naive pin bump would
-# move the tip out from under the patches and break `git apply` at build time,
-# so this backend OWNS its pin (LLAMA_VERSION below) instead of inheriting the
-# auto-bumped stock pin from backend/cpp/llama-cpp/Makefile. The override is
-# forced into every copied build via `LLAMA_VERSION=$(LLAMA_VERSION)`. There is
-# deliberately NO bump_deps.yaml entry for it: it is advanced ONLY by PIN_SYNC,
-# never nightly. (turboquant CAN auto-bump because its fork branch carries the
-# patches; the paged series is vendored as .patch files here, so it cannot.)
+# (patches/paged/PIN_SYNC_*.md). A naive pin bump would move the tip out from
+# under the patches and break `git apply` at build time, so this backend OWNS
+# its pin (LLAMA_VERSION below) instead of inheriting the auto-bumped stock pin
+# from backend/cpp/llama-cpp/Makefile. The override is forced into every copied
+# build via `LLAMA_VERSION=$(LLAMA_VERSION)`. There is deliberately NO
+# bump_deps.yaml entry for it: it is advanced ONLY by PIN_SYNC, never nightly.
+# (turboquant CAN auto-bump because its fork branch carries the patches; the
+# paged series is vendored as .patch files here, so it cannot.)
 #
-#   - NO patch-grpc-server.sh and NO apply-patches.sh: the shared
-#     grpc-server.cpp already carries the (runtime-gated) paged option hooks,
-#     and the paged patch series is applied by the copied llama-cpp Makefile's
-#     own `llama.cpp` target whenever LLAMA_PAGED=on (which we force below).
+#   - NO patch-grpc-server.sh and NO apply-patches.sh: the shared grpc-server.cpp
+#     already carries the (runtime-gated) paged option hooks, and the paged patch
+#     series (patches/paged/) is applied by THIS Makefile's own apply step onto
+#     the freshly cloned tree, using the same strict `git apply` method the stock
+#     build uses for base patches. The stock llama-cpp Makefile applies only its
+#     own (currently empty) base patches/ series, never the paged one.
 
 # Manually pin-synced llama.cpp tip the paged patch series is verified against.
 # Decoupled from the auto-bumped stock pin in backend/cpp/llama-cpp/Makefile so
 # the nightly llama.cpp bump cannot silently break the vendored paged patches.
 # Advance ONLY via the PIN_SYNC process (rebase patches + bit-exact gate +
 # re-export), then update this value. See:
-#   backend/cpp/llama-cpp/patches/paged/PIN_SYNC_*.md
+#   backend/cpp/llama-cpp-localai-paged/patches/paged/PIN_SYNC_*.md
 #
 # This pin = the manual, verified sync. The signal telling you WHEN to do the
 # next sync is the early-warning canary
@@ -47,28 +50,49 @@ ARCH?=$(shell uname -m)
 
 CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
 LLAMA_CPP_DIR := $(CURRENT_MAKEFILE_DIR)/../llama-cpp
+# OUR vendored paged-attention patch series. Owned by this backend; the stock
+# llama-cpp backend no longer carries it. Applied onto each freshly cloned
+# llama.cpp tree by apply-paged-patches below (strict git apply).
+PAGED_PATCHES_DIR := $(CURRENT_MAKEFILE_DIR)/patches/paged
 
 GREEN := \033[0;32m
 RESET := \033[0m
 
+# Apply OUR vendored paged-attention patch series (patches/paged/0*.patch) onto a
+# freshly cloned llama.cpp tree ($(1)) using the SAME strict git-apply method the
+# stock build uses for its base patches (backend/cpp/llama-cpp/Makefile `llama.cpp`
+# target). Strict: any patch that no longer applies aborts the build (exit 1) -
+# that is the signal to run a PIN_SYNC, never to bump the pin blindly. The series
+# is owned by THIS backend, not by the now-pure stock llama-cpp backend.
+define apply-paged-patches
+	cd $(1) && \
+	for p in $(PAGED_PATCHES_DIR)/0*.patch; do \
+		[ -e "$$p" ] || continue; \
+		echo "applying llama.cpp PAGED patch: $$p"; \
+		git apply --verbose "$$p" || { echo "paged patch failed: $$p"; exit 1; }; \
+	done
+endef
+
 # Each flavor target:
 #   1. copies backend/cpp/llama-cpp/ (grpc-server.cpp + prepare.sh +
 #      CMakeLists.txt + Makefile) into a sibling
 #      llama-cpp-localai-paged-<flavor>-build directory;
-#   2. clones the SAME upstream llama.cpp pin into that copy and applies the
-#      base AND paged patch series via the copy's own `llama.cpp` target with
-#      LLAMA_PAGED=on;
-#   3. runs the copy's `grpc-server` target (LLAMA_PAGED=on) and copies the
-#      produced binary up as llama-cpp-localai-paged-<flavor>.
-# We patch only the *copy*, never the original under backend/cpp/llama-cpp/, so
-# the stock llama-cpp build stays untouched.
+#   2. clones OUR pinned upstream llama.cpp into that copy via the copy's own
+#      `llama.cpp` target (which applies the stock base patches/ series, normally
+#      empty), then applies THIS backend's paged patch series (patches/paged/)
+#      onto the cloned tree with strict `git apply` (apply-paged-patches);
+#   3. runs the copy's `grpc-server` target and copies the produced binary up as
+#      llama-cpp-localai-paged-<flavor>.
+# We clone+patch only the *copy*, never the original under backend/cpp/llama-cpp/,
+# so the stock llama-cpp build stays untouched and patch-free.
 define paged-build
 	rm -rf $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build
 	cp -rf $(LLAMA_CPP_DIR) $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build
 	$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build purge
 	$(info $(GREEN)I llama-cpp-localai-paged build info:$(1)$(RESET))
-	LLAMA_VERSION=$(LLAMA_VERSION) LLAMA_PAGED=on $(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build llama.cpp
-	CMAKE_ARGS="$(CMAKE_ARGS) $(2)" TARGET="$(3)" LLAMA_VERSION=$(LLAMA_VERSION) LLAMA_PAGED=on \
+	LLAMA_VERSION=$(LLAMA_VERSION) $(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build llama.cpp
+	$(call apply-paged-patches,$(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build/llama.cpp)
+	CMAKE_ARGS="$(CMAKE_ARGS) $(2)" TARGET="$(3)" LLAMA_VERSION=$(LLAMA_VERSION) \
 	$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build grpc-server
 	cp -rfv $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build/grpc-server llama-cpp-localai-paged-$(1)
 endef
@@ -97,8 +121,9 @@ llama-cpp-localai-paged-cpu-all:
 	cp -rf $(LLAMA_CPP_DIR) $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build
 	$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build purge
 	$(info $(GREEN)I llama-cpp-localai-paged build info:cpu-all-variants$(RESET))
-	LLAMA_VERSION=$(LLAMA_VERSION) LLAMA_PAGED=on $(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build llama.cpp
-	SHARED_LIBS=ON EXTRA_CMAKE_ARGS="-DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON" TARGET="--target grpc-server --target ggml" LLAMA_VERSION=$(LLAMA_VERSION) LLAMA_PAGED=on \
+	LLAMA_VERSION=$(LLAMA_VERSION) $(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build llama.cpp
+	$(call apply-paged-patches,$(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build/llama.cpp)
+	SHARED_LIBS=ON EXTRA_CMAKE_ARGS="-DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON" TARGET="--target grpc-server --target ggml" LLAMA_VERSION=$(LLAMA_VERSION) \
 	$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build grpc-server
 	cp -rfv $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build/grpc-server llama-cpp-localai-paged-cpu-all
 	rm -rf ggml-shared-libs && mkdir -p ggml-shared-libs

backend/cpp/llama-cpp-localai-paged/patches/README.md

@@ -16,7 +16,7 @@ patch needs fixing, and the failure points at exactly which step the upstream ch
 
 | # | Patch | What | Verifies |
 |---|-------|------|----------|
-| 0001 | `0001-vendor-paged-kv-manager.patch` | Add `src/paged-kv-manager.{h,cpp}` (vLLM-parity block manager, CPU foundation) + CMake; no behavior change | builds; unit-tested separately under `../paged/` |
+| 0001 | `0001-vendor-paged-kv-manager.patch` | Add `src/paged-kv-manager.{h,cpp}` (vLLM-parity block manager, CPU foundation) + CMake; no behavior change | builds; unit-tested separately |
 | 0002 | `0002-paged-kv-storage.patch` | Shared block-pool KV tensor + `set_rows`-by-slot writes, behind `LLAMA_KV_PAGED` | builds; write/gather round-trip |
 | 0003 | `0003-paged-gather-read.patch` | `build_attn_paged` gather-read in `llama-graph.cpp` | **Gate 0**: token-identical greedy gen, single + multi-seq |
 | 0004 | `0004-paged-ondemand-alloc.patch` | On-demand block allocation via PagedKVManager | max concurrent seqs before OOM |
@@ -35,21 +35,25 @@ git checkout <LLAMA_VERSION from ../Makefile>
 git checkout -b paged
 
 # 2. apply the current series (each becomes a commit), or develop the next patch
-git am /path/to/backend/cpp/llama-cpp/patches/00*.patch     # or `git apply` + commit per patch
+git am /path/to/backend/cpp/llama-cpp-localai-paged/patches/paged/00*.patch     # or `git apply` + commit per patch
 
 # 3. iterate a phase as ONE commit, then export the whole series 1:1
-git format-patch <LLAMA_VERSION>..paged -o /path/to/backend/cpp/llama-cpp/patches/ --zero-commit -N
+git format-patch <LLAMA_VERSION>..paged -o /path/to/backend/cpp/llama-cpp-localai-paged/patches/paged/ --zero-commit -N
 
 # 4. on a pin bump: rebase `paged` onto the new pin; only conflicting patches need edits; re-export.
 ```
 
 ## Build integration
 
-`../Makefile`'s `llama.cpp:` target runs, after `git checkout -b build $(LLAMA_VERSION)`:
+The series is owned by this backend (`backend/cpp/llama-cpp-localai-paged`), not by the stock
+`llama-cpp` backend, which is pure upstream. `../Makefile` (the paged wrapper) clones the pinned
+`llama.cpp` via the copied stock build infra, then applies this series onto the cloned tree with the
+same strict `git apply` the stock build uses for base patches:
 ```
-for p in $(CURRENT_MAKEFILE_DIR)/patches/0*.patch; do git apply --verbose "$p"; done
+for p in $(PAGED_PATCHES_DIR)/0*.patch; do git apply --verbose "$p" || exit 1; done
 ```
-All variants (avx/avx2/avx512/cuda/…) copy the patched `llama.cpp/` tree, so the series ships everywhere.
+All variants (avx/avx2/avx512/cuda/…) clone + apply into their own build copy, so the series ships
+everywhere without ever touching the stock `llama-cpp` source tree.
 
 ## Status
 
@@ -78,5 +82,5 @@ by itself reach vLLM throughput parity, because the measured prefill bottleneck
 (Lever 3: `mul_mat_q<MXFP4>` ~22 TFLOP/s, ~27× behind vLLM) — a *per-token compute* gap that paging does not
 touch. Paged attention closes the **concurrency/memory** gap (more sequences, prefix reuse); the prefill/throughput
 gap additionally needs the tcgen05/CUTLASS grouped-GEMM (deferred, upstream-grade, no shortcut — see
-`../paged/UPSTREAM_GGML_ISSUE.md` and `DGX_BLACKWELL_PLAN.md`). So full vLLM parity = this series **AND** the
+`paged/README.md`). So full vLLM parity = this series **AND** the
 kernel; neither alone suffices.

backend/cpp/llama-cpp-localai-paged/patches/paged/PIN_SYNC_c299a92c.md

@@ -21,7 +21,8 @@ Unlike the `9d5d882d` sync (which needed 4 patch re-exports), this bump required
 **zero patch changes**. The already-shipped source-only series (the result of the
 `7e1832b8` strip that removed all stray dev-doc hunks) applies to a fresh clean
 `ggml-org/llama.cpp` checkout at `c299a92c` with the build's own **strict
-`git apply`** (the `llama.cpp` target in `backend/cpp/llama-cpp/Makefile`:
+`git apply`** (the `apply-paged-patches` step in
+`backend/cpp/llama-cpp-localai-paged/Makefile`:
 `git apply --verbose "$p" || exit 1`) and reaches **exit 0** - every one of the
 28 patches reported "Applied patch ... cleanly", the sentinel
 `src/paged-kv-manager.cpp` was created, and there are **zero** stray
@@ -94,7 +95,7 @@ here to keep the pin-bump diff minimal.
 
 ## Source of truth
 
-The shipped `.patch` files under `backend/cpp/llama-cpp/patches/paged/` are the
+The shipped `.patch` files under `backend/cpp/llama-cpp-localai-paged/patches/paged/` are the
 source of truth and are unchanged by this bump. The DGX dev tree
 (`~/llama-paged-dev`, branch `paged`) was advanced to `c299a92c` for consistency;
 the pre-bump state is retained at `paged-prebump-9d5d882d-backup`.