refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

Move ALL paged-attention content out of the stock backend/cpp/llama-cpp
backend and into backend/cpp/llama-cpp-localai-paged, so the stock backend is
pure upstream llama.cpp and the paged backend owns and applies its own vendored
patch series.

- Delete the dead early-exploration scaffold backend/cpp/llama-cpp/paged/
  (kernel/w4a16 Marlin scaffold, standalone paged_kv_manager, bench/loadgen,
  its own 0001-0002 patches, dense-era design docs, tests). Zero references
  repo-wide.
- Move backend/cpp/llama-cpp/patches/ (the 28-patch paged series + paged/README
  + 3 operational docs, plus the kernel/ scaffold patch and the top-level paged
  README/BENCHMARKS) to backend/cpp/llama-cpp-localai-paged/patches/. The stock
  backend keeps no patches/ dir; it had no non-paged base patches.
- Purify the stock backend: remove the LLAMA_PAGED make variable, the
  patches/paged apply loop, and the LLAMA_PAGED passthrough to prepare.sh;
  remove the paged-series handling from prepare.sh. The stock llama.cpp target
  now only clones the pin and applies its own (currently empty) base patches/
  series. The runtime paged option hooks in the shared grpc-server.cpp are
  untouched (inert without the patches).
- The paged backend's Makefile now applies its OWN patches/paged/0*.patch onto
  each freshly cloned tree via strict git apply (apply-paged-patches), after the
  copied stock infra clones the pin and applies base patches.
- Repoint every reference to the old patches/paged path: the upstream canary
  workflow + apply script, bump_deps.yaml, gallery/index.yaml, the docs,
  backend/index.yaml, backend-matrix.yml, the top-level Makefile comments, and
  the moved PIN_SYNC / README docs. Drop the now-removed LLAMA_PAGED=on
  build-toggle from comments.

Verified: the full 28-patch series applies strict-clean (git apply, exit 0) to
a clean ggml-org/llama.cpp checkout at the pinned c299a92c, and the repointed
canary apply script resolves and applies the series end to end.

Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto committed 2026-06-27 11:01:22 +00:00
1 parent fb2dc33d52
commit 78fac9a28f
87 files changed
+109 -3997

No files matched your search

+1 -1
View File
@@ -5073,7 +5073,7 @@ includeDarwin:
lang: "go"
# llama-cpp-localai-paged on Darwin: same bespoke CPU_ALL_VARIANTS + Metal build
# as stock llama-cpp (driven by make backends/llama-cpp-localai-paged-darwin),
# reusing backend/cpp/llama-cpp sources with LLAMA_PAGED=on. lang=go selects the
# reusing backend/cpp/llama-cpp sources, with the paged patch series applied by the wrapper. lang=go selects the
# runner/toolchain only; the source path is C++. Metal delivers paged-KV (the
# NVFP4 FP4-MMA fast path is CUDA/Blackwell-only) and the GDN/conv fused ops have
# no Metal kernel, so a gated-DeltaNet (qwen35) model falls back to the CPU
+4 -4
View File
@@ -1,14 +1,14 @@
#!/usr/bin/env bash
#
# paged-canary-apply.sh - apply the vendored paged-attention patch series
# (backend/cpp/llama-cpp/patches/paged/0001-0030) to a llama.cpp checkout, the
# (backend/cpp/llama-cpp-localai-paged/patches/paged/0001-0030) to a llama.cpp checkout, the
# same way the build does, but tolerating the ONE known-benign pre-existing
# quirk in the series. Used by the early-warning canary
# (.github/workflows/llama-cpp-paged-canary.yml) so it only goes red on a REAL
# upstream break, never on that quirk.
#
# Usage: paged-canary-apply.sh <llama.cpp-checkout-dir> <patches-dir>
# <patches-dir> is normally backend/cpp/llama-cpp/patches (it holds the
# <patches-dir> is normally backend/cpp/llama-cpp-localai-paged/patches (it holds the
# top-level base series 0*.patch, currently empty, and the paged/ subseries).
#
# Exit 0 = the whole series applied -> patches still fit upstream.
@@ -27,7 +27,7 @@
# missing-file hunk rejects the whole patch - and because 0021/0022/0026/0028
# build on 0019's code, the rejection cascades to them too. This is a
# PRE-EXISTING shipped-series defect, present identically on every pin, NOT an
# upstream break (see backend/cpp/llama-cpp/patches/paged/PIN_SYNC_c299a92c.md
# upstream break (see backend/cpp/llama-cpp-localai-paged/patches/paged/PIN_SYNC_c299a92c.md
# and README.md). We exclude ONLY that dev-doc path and still
# apply 0019's real code hunks atomically, so a genuine code-hunk break in 0019
# still fails the canary. prepare.sh tolerates the same hunk via
@@ -53,7 +53,7 @@ apply_one() {
echo "paged-canary: applying $(basename "$p")"
if ! git apply --verbose "$@" "$p"; then
echo "::error::paged patch no longer applies to the upstream llama.cpp tip: $(basename "$p")"
echo "::error::upstream drifted past the vendored paged series - run a PIN_SYNC (backend/cpp/llama-cpp/patches/paged/PIN_SYNC_c299a92c.md), do NOT bump the pin blindly"
echo "::error::upstream drifted past the vendored paged series - run a PIN_SYNC (backend/cpp/llama-cpp-localai-paged/patches/paged/PIN_SYNC_c299a92c.md), do NOT bump the pin blindly"
exit 1
fi
}
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
matrix:
# NOTE: there is intentionally NO entry for the llama-cpp-localai-paged
# backend. It carries a vendored paged-attention patch series
# (backend/cpp/llama-cpp/patches/paged/) hand-verified bit-exact against
# (backend/cpp/llama-cpp-localai-paged/patches/paged/) hand-verified bit-exact against
# ONE specific llama.cpp tip; a naive nightly bump would move the tip out
# from under the patches and break `git apply` at build time. Its pin is
# therefore decoupled (its own LLAMA_VERSION in
+19 -14
View File
@@ -1,7 +1,7 @@
name: 'llama.cpp paged patches: upstream canary'
# EARLY-WARNING CANARY for the vendored paged-attention patch series
# (backend/cpp/llama-cpp/patches/paged/0001-0030).
# (backend/cpp/llama-cpp-localai-paged/patches/paged/0001-0030).
#
# WHY THIS EXISTS
# The paged backend (backend/cpp/llama-cpp-localai-paged) pins its OWN verified
@@ -17,7 +17,7 @@ name: 'llama.cpp paged patches: upstream canary'
# RED HERE means: time to run a PIN_SYNC (rebase the patches onto the new tip,
# pass the bit-exact gate on the GPU, re-export the .patch files, THEN advance
# the pin in backend/cpp/llama-cpp-localai-paged/Makefile). See
# backend/cpp/llama-cpp/patches/paged/PIN_SYNC_c299a92c.md.
# backend/cpp/llama-cpp-localai-paged/patches/paged/PIN_SYNC_c299a92c.md.
#
# SIGNAL-ONLY: this workflow moves no pinned version, ships nothing, and is fully
# decoupled from bump_deps - so the main dep-bump PR stays green regardless. A
@@ -91,7 +91,7 @@ jobs:
run: |
bash .github/scripts/paged-canary-apply.sh \
/tmp/llama.cpp \
"$PWD/backend/cpp/llama-cpp/patches"
"$PWD/backend/cpp/llama-cpp-localai-paged/patches"
echo "- apply: full paged series applies to the upstream tip :white_check_mark:" >> "$GITHUB_STEP_SUMMARY"
compile:
@@ -141,12 +141,16 @@ jobs:
cp -a /opt/grpc/. /usr/local/
# Pre-populate the llama.cpp checkout at the latest tip with the
# paged series applied via the tolerant canary apply (so the benign
# 0019 dev-doc hunk does not abort the build). Because
# backend/cpp/llama-cpp/llama.cpp now exists, the Makefile
# llama.cpp target (strict clone + git apply) is skipped and
# prepare.sh sees the paged sentinel and skips re-applying - so we
# drive the REAL grpc-server build path on top of our apply.
# paged series applied via the tolerant canary apply. Because
# backend/cpp/llama-cpp/llama.cpp now exists, the stock Makefile's
# llama.cpp target (clone + base-patch apply) is skipped and the
# now patch-free prepare.sh only copies the grpc-server sources -
# so we drive the REAL grpc-server build path on top of our paged
# apply. The stock llama-cpp backend no longer carries the paged
# series (it lives in backend/cpp/llama-cpp-localai-paged/patches/
# paged); we build it here in the stock dir only because that is
# where the shared build infra (Makefile / grpc-server.cpp /
# CMakeLists.txt / prepare.sh) lives.
cd backend/cpp/llama-cpp/
mkdir -p llama.cpp
cd llama.cpp
@@ -157,15 +161,16 @@ jobs:
cd /LocalAI
bash .github/scripts/paged-canary-apply.sh \
backend/cpp/llama-cpp/llama.cpp \
"$PWD/backend/cpp/llama-cpp/patches"
"$PWD/backend/cpp/llama-cpp-localai-paged/patches"
# Cheapest real CUDA build that proves the patches compile: one
# CUDA arch, cublas, paged on. CMAKE_ARGS is passed via the
# environment (not as a make arg) so the Makefile += flags are
# still appended, exactly like .docker/llama-cpp-localai-paged-compile.sh.
# CUDA arch, cublas. CMAKE_ARGS is passed via the environment (not
# as a make arg) so the Makefile += flags are still appended,
# exactly like .docker/llama-cpp-localai-paged-compile.sh. The paged
# series is already applied to the checkout above, so the stock
# build just compiles the patched tree.
cd backend/cpp/llama-cpp/
BUILD_TYPE=cublas \
LLAMA_PAGED=on \
CMAKE_ARGS="-DCMAKE_CUDA_ARCHITECTURES=80" \
make grpc-server
test -x grpc-server
+3 -3
View File
@@ -1142,8 +1142,8 @@ backends/llama-cpp-darwin: build
./local-ai backends install "ocifile://$(abspath ./backend-images/llama-cpp.tar)"
# llama-cpp-localai-paged on Darwin: same bespoke CPU_ALL_VARIANTS + Metal build as
# stock llama-cpp (otool dylib bundling), driven through the paged wrapper Makefile
# with LLAMA_PAGED=on. Mirrors backends/llama-cpp-darwin.
# stock llama-cpp (otool dylib bundling), driven through the paged wrapper Makefile,
# which applies its own vendored paged patch series. Mirrors backends/llama-cpp-darwin.
backends/llama-cpp-localai-paged-darwin: build
bash ./scripts/build/llama-cpp-localai-paged-darwin.sh
./local-ai backends install "ocifile://$(abspath ./backend-images/llama-cpp-localai-paged.tar)"
@@ -1198,7 +1198,7 @@ BACKEND_IK_LLAMA_CPP = ik-llama-cpp|ik-llama-cpp|.|false|false
# Reuses backend/cpp/llama-cpp grpc-server sources via a thin wrapper Makefile.
BACKEND_TURBOQUANT = turboquant|turboquant|.|false|false
# llama-cpp-localai-paged = stock llama.cpp grpc-server + the LocalAI paged-attention
# patch series (LLAMA_PAGED=on). Reuses backend/cpp/llama-cpp sources via a thin
# patch series (vendored in this wrapper backend). Reuses backend/cpp/llama-cpp sources via a thin
# wrapper Makefile (same upstream pin as stock llama-cpp; no fork, no patch-grpc-server).
BACKEND_LLAMA_CPP_LOCALAI_PAGED = llama-cpp-localai-paged|llama-cpp-localai-paged|.|false|false
# ds4 is antirez/ds4, a DeepSeek V4 Flash-specific inference engine.
+52 -27
View File
@@ -1,33 +1,36 @@
# llama-cpp-localai-paged is LocalAI's paged-attention llama.cpp variant. It
# builds upstream llama.cpp with the LocalAI paged-attention patch series
# (backend/cpp/llama-cpp/patches/paged/) applied on top (LLAMA_PAGED=on). It
# reuses backend/cpp/llama-cpp's grpc-server.cpp / CMakeLists.txt / prepare.sh
# sources verbatim via a thin wrapper.
# (patches/paged/, vendored in THIS backend) applied on top. It reuses
# backend/cpp/llama-cpp's grpc-server.cpp / CMakeLists.txt / prepare.sh / Makefile
# sources verbatim via a thin wrapper - the stock llama-cpp backend is pure
# upstream and carries NONE of the paged patches; this backend OWNS them.
#
# Pin handling (mirrors the turboquant wrapper, the precedent this is modelled
# on): the paged patch series is hand-verified bit-exact against ONE specific
# llama.cpp tip and re-exported by the manual PIN_SYNC process
# (backend/cpp/llama-cpp/patches/paged/PIN_SYNC_*.md). A naive pin bump would
# move the tip out from under the patches and break `git apply` at build time,
# so this backend OWNS its pin (LLAMA_VERSION below) instead of inheriting the
# auto-bumped stock pin from backend/cpp/llama-cpp/Makefile. The override is
# forced into every copied build via `LLAMA_VERSION=$(LLAMA_VERSION)`. There is
# deliberately NO bump_deps.yaml entry for it: it is advanced ONLY by PIN_SYNC,
# never nightly. (turboquant CAN auto-bump because its fork branch carries the
# patches; the paged series is vendored as .patch files here, so it cannot.)
# (patches/paged/PIN_SYNC_*.md). A naive pin bump would move the tip out from
# under the patches and break `git apply` at build time, so this backend OWNS
# its pin (LLAMA_VERSION below) instead of inheriting the auto-bumped stock pin
# from backend/cpp/llama-cpp/Makefile. The override is forced into every copied
# build via `LLAMA_VERSION=$(LLAMA_VERSION)`. There is deliberately NO
# bump_deps.yaml entry for it: it is advanced ONLY by PIN_SYNC, never nightly.
# (turboquant CAN auto-bump because its fork branch carries the patches; the
# paged series is vendored as .patch files here, so it cannot.)
#
# - NO patch-grpc-server.sh and NO apply-patches.sh: the shared
# grpc-server.cpp already carries the (runtime-gated) paged option hooks,
# and the paged patch series is applied by the copied llama-cpp Makefile's
# own `llama.cpp` target whenever LLAMA_PAGED=on (which we force below).
# - NO patch-grpc-server.sh and NO apply-patches.sh: the shared grpc-server.cpp
# already carries the (runtime-gated) paged option hooks, and the paged patch
# series (patches/paged/) is applied by THIS Makefile's own apply step onto
# the freshly cloned tree, using the same strict `git apply` method the stock
# build uses for base patches. The stock llama-cpp Makefile applies only its
# own (currently empty) base patches/ series, never the paged one.
# Manually pin-synced llama.cpp tip the paged patch series is verified against.
# Decoupled from the auto-bumped stock pin in backend/cpp/llama-cpp/Makefile so
# the nightly llama.cpp bump cannot silently break the vendored paged patches.
# Advance ONLY via the PIN_SYNC process (rebase patches + bit-exact gate +
# re-export), then update this value. See:
# backend/cpp/llama-cpp/patches/paged/PIN_SYNC_*.md
# backend/cpp/llama-cpp-localai-paged/patches/paged/PIN_SYNC_*.md
#
# This pin = the manual, verified sync. The signal telling you WHEN to do the
# next sync is the early-warning canary
@@ -47,28 +50,49 @@ ARCH?=$(shell uname -m)
CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
LLAMA_CPP_DIR := $(CURRENT_MAKEFILE_DIR)/../llama-cpp
# OUR vendored paged-attention patch series. Owned by this backend; the stock
# llama-cpp backend no longer carries it. Applied onto each freshly cloned
# llama.cpp tree by apply-paged-patches below (strict git apply).
PAGED_PATCHES_DIR := $(CURRENT_MAKEFILE_DIR)/patches/paged
GREEN := \033[0;32m
RESET := \033[0m
# Apply OUR vendored paged-attention patch series (patches/paged/0*.patch) onto a
# freshly cloned llama.cpp tree ($(1)) using the SAME strict git-apply method the
# stock build uses for its base patches (backend/cpp/llama-cpp/Makefile `llama.cpp`
# target). Strict: any patch that no longer applies aborts the build (exit 1) -
# that is the signal to run a PIN_SYNC, never to bump the pin blindly. The series
# is owned by THIS backend, not by the now-pure stock llama-cpp backend.
define apply-paged-patches
cd $(1) && \
for p in $(PAGED_PATCHES_DIR)/0*.patch; do \
[ -e "$$p" ] || continue; \
echo "applying llama.cpp PAGED patch: $$p"; \
git apply --verbose "$$p" || { echo "paged patch failed: $$p"; exit 1; }; \
done
endef
# Each flavor target:
# 1. copies backend/cpp/llama-cpp/ (grpc-server.cpp + prepare.sh +
# CMakeLists.txt + Makefile) into a sibling
# llama-cpp-localai-paged-<flavor>-build directory;
# 2. clones the SAME upstream llama.cpp pin into that copy and applies the
# base AND paged patch series via the copy's own `llama.cpp` target with
# LLAMA_PAGED=on;
# 3. runs the copy's `grpc-server` target (LLAMA_PAGED=on) and copies the
# produced binary up as llama-cpp-localai-paged-<flavor>.
# We patch only the *copy*, never the original under backend/cpp/llama-cpp/, so
# the stock llama-cpp build stays untouched.
# 2. clones OUR pinned upstream llama.cpp into that copy via the copy's own
# `llama.cpp` target (which applies the stock base patches/ series, normally
# empty), then applies THIS backend's paged patch series (patches/paged/)
# onto the cloned tree with strict `git apply` (apply-paged-patches);
# 3. runs the copy's `grpc-server` target and copies the produced binary up as
# llama-cpp-localai-paged-<flavor>.
# We clone+patch only the *copy*, never the original under backend/cpp/llama-cpp/,
# so the stock llama-cpp build stays untouched and patch-free.
define paged-build
rm -rf $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build
cp -rf $(LLAMA_CPP_DIR) $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build
$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build purge
$(info $(GREEN)I llama-cpp-localai-paged build info:$(1)$(RESET))
LLAMA_VERSION=$(LLAMA_VERSION) LLAMA_PAGED=on $(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build llama.cpp
CMAKE_ARGS="$(CMAKE_ARGS) $(2)" TARGET="$(3)" LLAMA_VERSION=$(LLAMA_VERSION) LLAMA_PAGED=on \
LLAMA_VERSION=$(LLAMA_VERSION) $(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build llama.cpp
$(call apply-paged-patches,$(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build/llama.cpp)
CMAKE_ARGS="$(CMAKE_ARGS) $(2)" TARGET="$(3)" LLAMA_VERSION=$(LLAMA_VERSION) \
$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build grpc-server
cp -rfv $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-$(1)-build/grpc-server llama-cpp-localai-paged-$(1)
endef
@@ -97,8 +121,9 @@ llama-cpp-localai-paged-cpu-all:
cp -rf $(LLAMA_CPP_DIR) $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build
$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build purge
$(info $(GREEN)I llama-cpp-localai-paged build info:cpu-all-variants$(RESET))
LLAMA_VERSION=$(LLAMA_VERSION) LLAMA_PAGED=on $(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build llama.cpp
SHARED_LIBS=ON EXTRA_CMAKE_ARGS="-DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON" TARGET="--target grpc-server --target ggml" LLAMA_VERSION=$(LLAMA_VERSION) LLAMA_PAGED=on \
LLAMA_VERSION=$(LLAMA_VERSION) $(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build llama.cpp
$(call apply-paged-patches,$(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build/llama.cpp)
SHARED_LIBS=ON EXTRA_CMAKE_ARGS="-DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON" TARGET="--target grpc-server --target ggml" LLAMA_VERSION=$(LLAMA_VERSION) \
$(MAKE) -C $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build grpc-server
cp -rfv $(CURRENT_MAKEFILE_DIR)/../llama-cpp-localai-paged-cpu-all-build/grpc-server llama-cpp-localai-paged-cpu-all
rm -rf ggml-shared-libs && mkdir -p ggml-shared-libs
@@ -16,7 +16,7 @@ patch needs fixing, and the failure points at exactly which step the upstream ch
| # | Patch | What | Verifies |
|---|-------|------|----------|
| 0001 | `0001-vendor-paged-kv-manager.patch` | Add `src/paged-kv-manager.{h,cpp}` (vLLM-parity block manager, CPU foundation) + CMake; no behavior change | builds; unit-tested separately under `../paged/` |
| 0001 | `0001-vendor-paged-kv-manager.patch` | Add `src/paged-kv-manager.{h,cpp}` (vLLM-parity block manager, CPU foundation) + CMake; no behavior change | builds; unit-tested separately |
| 0002 | `0002-paged-kv-storage.patch` | Shared block-pool KV tensor + `set_rows`-by-slot writes, behind `LLAMA_KV_PAGED` | builds; write/gather round-trip |
| 0003 | `0003-paged-gather-read.patch` | `build_attn_paged` gather-read in `llama-graph.cpp` | **Gate 0**: token-identical greedy gen, single + multi-seq |
| 0004 | `0004-paged-ondemand-alloc.patch` | On-demand block allocation via PagedKVManager | max concurrent seqs before OOM |
@@ -35,21 +35,25 @@ git checkout <LLAMA_VERSION from ../Makefile>
git checkout -b paged
# 2. apply the current series (each becomes a commit), or develop the next patch
git am /path/to/backend/cpp/llama-cpp/patches/00*.patch # or `git apply` + commit per patch
git am /path/to/backend/cpp/llama-cpp-localai-paged/patches/paged/00*.patch # or `git apply` + commit per patch
# 3. iterate a phase as ONE commit, then export the whole series 1:1
git format-patch <LLAMA_VERSION>..paged -o /path/to/backend/cpp/llama-cpp/patches/ --zero-commit -N
git format-patch <LLAMA_VERSION>..paged -o /path/to/backend/cpp/llama-cpp-localai-paged/patches/paged/ --zero-commit -N
# 4. on a pin bump: rebase `paged` onto the new pin; only conflicting patches need edits; re-export.
```
## Build integration
`../Makefile`'s `llama.cpp:` target runs, after `git checkout -b build $(LLAMA_VERSION)`:
The series is owned by this backend (`backend/cpp/llama-cpp-localai-paged`), not by the stock
`llama-cpp` backend, which is pure upstream. `../Makefile` (the paged wrapper) clones the pinned
`llama.cpp` via the copied stock build infra, then applies this series onto the cloned tree with the
same strict `git apply` the stock build uses for base patches:
```
for p in $(CURRENT_MAKEFILE_DIR)/patches/0*.patch; do git apply --verbose "$p"; done
for p in $(PAGED_PATCHES_DIR)/0*.patch; do git apply --verbose "$p" || exit 1; done
```
All variants (avx/avx2/avx512/cuda/…) copy the patched `llama.cpp/` tree, so the series ships everywhere.
All variants (avx/avx2/avx512/cuda/…) clone + apply into their own build copy, so the series ships
everywhere without ever touching the stock `llama-cpp` source tree.
## Status
@@ -78,5 +82,5 @@ by itself reach vLLM throughput parity, because the measured prefill bottleneck
(Lever 3: `mul_mat_q<MXFP4>` ~22 TFLOP/s, ~27× behind vLLM) — a *per-token compute* gap that paging does not
touch. Paged attention closes the **concurrency/memory** gap (more sequences, prefix reuse); the prefill/throughput
gap additionally needs the tcgen05/CUTLASS grouped-GEMM (deferred, upstream-grade, no shortcut — see
`../paged/UPSTREAM_GGML_ISSUE.md` and `DGX_BLACKWELL_PLAN.md`). So full vLLM parity = this series **AND** the
`paged/README.md`). So full vLLM parity = this series **AND** the
kernel; neither alone suffices.
@@ -21,7 +21,8 @@ Unlike the `9d5d882d` sync (which needed 4 patch re-exports), this bump required
**zero patch changes**. The already-shipped source-only series (the result of the
`7e1832b8` strip that removed all stray dev-doc hunks) applies to a fresh clean
`ggml-org/llama.cpp` checkout at `c299a92c` with the build's own **strict
`git apply`** (the `llama.cpp` target in `backend/cpp/llama-cpp/Makefile`:
`git apply`** (the `apply-paged-patches` step in
`backend/cpp/llama-cpp-localai-paged/Makefile`:
`git apply --verbose "$p" || exit 1`) and reaches **exit 0** - every one of the
28 patches reported "Applied patch ... cleanly", the sentinel
`src/paged-kv-manager.cpp` was created, and there are **zero** stray
@@ -94,7 +95,7 @@ here to keep the pin-bump diff minimal.
## Source of truth
The shipped `.patch` files under `backend/cpp/llama-cpp/patches/paged/` are the
The shipped `.patch` files under `backend/cpp/llama-cpp-localai-paged/patches/paged/` are the
source of truth and are unchanged by this bump. The DGX dev tree
(`~/llama-paged-dev`, branch `paged`) was advanced to `c299a92c` for consistency;
the pre-bump state is retained at `paged-prebump-9d5d882d-backup`.
File renamed without changes.
Internal Server Error - Gitea: Git with a cup of tea
500 Internal Server Error

Gitea Version: 1.28.0+dev-477-g8b6ad49a5f