Ettore Di Giacinto 500d653bfa feat(paged): regenerate patch series 0053-0055 (P1 bf16-stream) ...

Additive regen mirroring fork mudler/llama.cpp:localai-paged HEAD 653bb2f3d
(base 1edddc8fe + 3 P1 commits). Patches 0001-0052 are untouched.

- 0053 residual-segment executor + norm-bf16.{cu,cuh} + LLAMA_BF16_CUBLAS_F32_OUT
- 0054 bf16 residual-add + rope op-variants
- 0055 BF16_STREAM_SEGMENT test-backend-ops sentinel

Kill-gate: a fresh detached worktree at pin 0ed235ea2c17a19fc8238668653946721ed136fd
applied all 46 on-disk patches in numeric order (strict git apply) and staged
tree 6cf1523047e0e38679baff20844bdc9e6829eb22, byte-for-byte == fork HEAD tree.
All default-off (LLAMA_BF16_STREAM); default md5 canonical both models
(MoE 8cb0ce23777bf55f92f63d0292c756b0, dense 5951a5b4d624ce891e22ab5fca9bc439).

Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2026-07-02 14:34:14 +00:00

..

0001-vendor-paged-kv-manager.patch

paged-kv-manager.h: add missing <cstddef> for size_t

2026-06-28 04:09:16 +00:00

0002-paged-kv-block-placement-env-LLAMA_KV_PAGED.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0003-paged-gather-read-env-LLAMA_KV_PAGED.patch

paged headers: self-include <cstddef>/<cstdint> for size_t/uintN_t (fix amd64/non-arm64 build; compile-only)

2026-06-28 06:18:56 +00:00

0004-paged-on-demand-block-allocation-env-LLAMA_KV_PAGED.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0006-paged-cross-request-prefix-caching-env-LLAMA_KV_PAGED.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0007-paged-engine-prefix-recompute-skip-env-LLAMA_KV_PAGED.patch

paged headers: self-include <cstddef>/<cstdint> for size_t/uintN_t (fix amd64/non-arm64 build; compile-only)

2026-06-28 06:18:56 +00:00

0008-paged-server-cross-request-prefix-share-env-LLAMA_KV_PAGED.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0009-paged-in-kernel-decode-read-env-LLAMA_KV_PAGED-patch.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0010-paged-tile-in-kernel-read-and-dispatch-guard-env-LLAMA_KV_PAGED.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0011-paged-decode-route-GQA-grouped-tile-kernel-by-defaul.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0012-paged-mask-pad-invariant-assert.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0013-paged-decoupled-prefill-token-budget.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0014-paged-expert-aware-moe-token-tile-cap.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0015-paged-expert-density-aware-moe-token-tile-auto-select.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0016-paged-dynamic-prefill-budget-continuous-batch.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0017-fp4-gemm-decode-tile-tune.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0018-qwen35-ssm-decode-inplace-state.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0019-qwen35-ssm-decode-fused-gather.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0020-qwen35-gdn-oproj-mmq-reshape.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0021-qwen35-conv-state-inplace-fusion.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0022-qwen35-gdn-recurrence-occupancy-retune.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0023-qwen35moe-nvfp4-quant-dedup.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0024-paged-pool-burst-reclaim.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0025-qwen35moe-nvfp4-moe-decode-regraph.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0028-qwen35-recurrent-state-gather-fusion.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0029-qwen35-blocktable-within-step-cache.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0030-fused-op-backend-gate.patch

refactor(paged): stock llama-cpp is patch-free; paged backend owns its patch series

2026-06-27 11:01:22 +00:00

0031-paged-chunked-gdn-prefill-scan-kernel.patch

fix(paged): make patch 0031 apply on the 0001-0030 base; default S3 on under paged KV

2026-06-28 19:37:05 +00:00

0033-fp4-prefill-large-m-bf16-cublas-scaffold.patch

feat(paged): FP4 prefill large-M dequant->bf16 cuBLAS scaffold (patch 0033, default-off)

2026-06-28 17:42:15 +00:00

0034-feat-paged-native-NVFP4-W4A4-FP4-MMA-large-M-prefill.patch

feat(paged): tail-fusion (0042) + full-step decode CUDA graph default-on (0043); FP4-MMA W4A4 (0034) + Marlin W4A16 (0035) MoE-GEMM scaffolds default-off

2026-06-29 06:15:10 +00:00

0035-feat-paged-marlin-w4a16-grouped-moe-prefill-gemm.patch

feat(paged): tail-fusion (0042) + full-step decode CUDA graph default-on (0043); FP4-MMA W4A4 (0034) + Marlin W4A16 (0035) MoE-GEMM scaffolds default-off

2026-06-29 06:15:10 +00:00

0040-feat-paged-S1-paged-decode-graph-reuse-across-servin.patch

feat(paged): close the continuous-serving decode gap (S1+S3, patches 0040/0041)

2026-06-28 18:04:28 +00:00

0041-feat-paged-S3-decode-shape-stable-scheduling-patch-0.patch

fix(paged): revert S3 decode-stable scheduler to default-OFF (A/B regression)

2026-06-29 05:00:11 +00:00

0042-feat-paged-fused-residual-add-RMS-norm-weight-multip.patch

feat(paged): tail-fusion (0042) + full-step decode CUDA graph default-on (0043); FP4-MMA W4A4 (0034) + Marlin W4A16 (0035) MoE-GEMM scaffolds default-off

2026-06-29 06:15:10 +00:00

0043-feat-paged-default-on-full-step-moe-decode-cuda-graph.patch

feat(paged): tail-fusion (0042) + full-step decode CUDA graph default-on (0043); FP4-MMA W4A4 (0034) + Marlin W4A16 (0035) MoE-GEMM scaffolds default-off

2026-06-29 06:15:10 +00:00

0044-feat-paged-fused-gated-RMSNorm-SiLU-gate-mul.patch

patches(paged): track 0044 GatedRMSNorm patch, sync LocalAI series to fork 51168c5

2026-06-30 15:10:13 +00:00

0046-paged-gate-GDN-prefill-geometry-by-scan-length.patch

fix(paged): repair the patch series, sync to the fork branch (drop dev-tree 0044/0045, add f32-only M5 as 0047)

2026-06-30 07:54:46 +00:00

0047-paged-GDN-M5-tensor-core-chunked-scan-f32.patch

fix(paged): repair the patch series, sync to the fork branch (drop dev-tree 0044/0045, add f32-only M5 as 0047)

2026-06-30 07:54:46 +00:00

0048-test-paged-cover-MoE-swiglu-down-chain.patch

patches(paged): trim series to Phase135 routed-FFN line, sync to fork 1edddc8fe

2026-07-02 10:19:10 +00:00

0049-test-paged-cover-MoE-weighted-combine-chain.patch

patches(paged): trim series to Phase135 routed-FFN line, sync to fork 1edddc8fe

2026-07-02 10:19:10 +00:00

0050-test-paged-cover-ragged-MoE-dispatch.patch

patches(paged): trim series to Phase135 routed-FFN line, sync to fork 1edddc8fe

2026-07-02 10:19:10 +00:00

0051-fix-speculative-disable-backend-sampling-for-MTP-dra.patch

patches(paged): trim series to Phase135 routed-FFN line, sync to fork 1edddc8fe

2026-07-02 10:19:10 +00:00

0052-feat-paged-whole-pattern-MoE-matcher-routed-FFN-fuse.patch

patches(paged): trim series to Phase135 routed-FFN line, sync to fork 1edddc8fe

2026-07-02 10:19:10 +00:00

0053-feat-paged-P1-bf16-stream-residual-segment-executor-.patch

feat(paged): regenerate patch series 0053-0055 (P1 bf16-stream)

2026-07-02 14:34:14 +00:00

0054-feat-paged-P1-bf16-stream-bf16-residual-add-rope-op-.patch

feat(paged): regenerate patch series 0053-0055 (P1 bf16-stream)

2026-07-02 14:34:14 +00:00

0055-test-paged-P1-bf16-stream-BF16_STREAM_SEGMENT-sentin.patch

feat(paged): regenerate patch series 0053-0055 (P1 bf16-stream)

2026-07-02 14:34:14 +00:00