mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-27 09:57:14 -04:00
78fac9a28f
Move ALL paged-attention content out of the stock backend/cpp/llama-cpp backend and into backend/cpp/llama-cpp-localai-paged, so the stock backend is pure upstream llama.cpp and the paged backend owns and applies its own vendored patch series. - Delete the dead early-exploration scaffold backend/cpp/llama-cpp/paged/ (kernel/w4a16 Marlin scaffold, standalone paged_kv_manager, bench/loadgen, its own 0001-0002 patches, dense-era design docs, tests). Zero references repo-wide. - Move backend/cpp/llama-cpp/patches/ (the 28-patch paged series + paged/README + 3 operational docs, plus the kernel/ scaffold patch and the top-level paged README/BENCHMARKS) to backend/cpp/llama-cpp-localai-paged/patches/. The stock backend keeps no patches/ dir; it had no non-paged base patches. - Purify the stock backend: remove the LLAMA_PAGED make variable, the patches/paged apply loop, and the LLAMA_PAGED passthrough to prepare.sh; remove the paged-series handling from prepare.sh. The stock llama.cpp target now only clones the pin and applies its own (currently empty) base patches/ series. The runtime paged option hooks in the shared grpc-server.cpp are untouched (inert without the patches). - The paged backend's Makefile now applies its OWN patches/paged/0*.patch onto each freshly cloned tree via strict git apply (apply-paged-patches), after the copied stock infra clones the pin and applies base patches. - Repoint every reference to the old patches/paged path: the upstream canary workflow + apply script, bump_deps.yaml, gallery/index.yaml, the docs, backend/index.yaml, backend-matrix.yml, the top-level Makefile comments, and the moved PIN_SYNC / README docs. Drop the now-removed LLAMA_PAGED=on build-toggle from comments. Verified: the full 28-patch series applies strict-clean (git apply, exit 0) to a clean ggml-org/llama.cpp checkout at the pinned c299a92c, and the repointed canary apply script resolves and applies the series end to end. Assisted-by: Claude:opus-4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
42 lines
1.5 KiB
Bash
42 lines
1.5 KiB
Bash
#!/bin/bash
|
|
|
|
## Patches
|
|
|
|
## Apply the base `patches/` series (top-level *.patch only; *.md/dirs skipped).
|
|
## The stock llama-cpp backend is patch-free by default, so this normally does
|
|
## nothing. The Makefile `llama.cpp` target already `git apply`s any base patch
|
|
## at checkout, so each apply here is `-N` (skip already-applied): re-applying a
|
|
## git-format patch with `patch` would fuzzily duplicate hunks. This block only
|
|
## does real work if prepare.sh is run against an unpatched checkout.
|
|
if [ -d "patches" ]; then
|
|
for patch in patches/*.patch; do
|
|
[ -e "$patch" ] || continue
|
|
echo "Applying patch $patch"
|
|
patch -d llama.cpp/ -p1 -N -r - < "$patch" || true
|
|
done
|
|
fi
|
|
|
|
set -e
|
|
|
|
for file in $(ls llama.cpp/tools/server/); do
|
|
cp -rfv llama.cpp/tools/server/$file llama.cpp/tools/grpc-server/
|
|
done
|
|
|
|
cp -r CMakeLists.txt llama.cpp/tools/grpc-server/
|
|
cp -r grpc-server.cpp llama.cpp/tools/grpc-server/
|
|
# Shared message-reconstruction helpers (included by grpc-server.cpp) and their
|
|
# unit test (compiled only when -DLLAMA_GRPC_BUILD_TESTS=ON).
|
|
cp -r message_content.h llama.cpp/tools/grpc-server/
|
|
cp -r message_content_test.cpp llama.cpp/tools/grpc-server/
|
|
cp -rfv llama.cpp/vendor/nlohmann/json.hpp llama.cpp/tools/grpc-server/
|
|
cp -rfv llama.cpp/vendor/cpp-httplib/httplib.h llama.cpp/tools/grpc-server/
|
|
|
|
set +e
|
|
if grep -q "grpc-server" llama.cpp/tools/CMakeLists.txt; then
|
|
echo "grpc-server already added"
|
|
else
|
|
echo "add_subdirectory(grpc-server)" >> llama.cpp/tools/CMakeLists.txt
|
|
fi
|
|
set -e
|
|
|