Files
LocalAI/backend/cpp
Ettore Di Giacinto 2b57997df0 docs(paged): cudagraph-coverage - GDN serial chain IS graph-covered at B=128
Determine whether the ggml CUDA graph covers the gated-DeltaNet serial chain
at batch=128. It does: nothing in the GDN region forces graph-disable
(check_compability lists only split-buffers and large-batch MUL_MAT_ID), and
the recurrent head is constant for a steady 128-seq batch so the inplace_ids
state_dst offset + rs_head op_param + SSM input shapes are stable across steps.
The fused op does no host-sync / capture-time cudaMalloc. The only re-warm is
the per-256-token full-attention block-table cadence (not a GDN op). The
~40% util is bandwidth-roofline (SSM state traffic 66% of step bytes), not
launch-gap idle - so no GDN graph-safe lever; the only non-covered idle is the
~0.4% between-step host cgraph rebuild.

Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-06-25 14:45:51 +00:00
..