LocalAI

0 Fork 0

mirror of https://github.com/mudler/LocalAI.git synced 2026-09-25 07:34:58 -04:00

Files

T

mudler-agentandEttore Di Giacinto 81eaca8768 fix(turboquant): extend D512 flash-attn patch to all turbo V types (#12234 )

The previous patch only removed DECL_FATTN_VEC_CASE_D512 for turbo2_0
and turbo3_0 V cache types. turbo4_0 also overflows shared memory
(0x10100 bytes > 0xc000 max), causing ptxas errors on CUDA 12/13.

Additionally, the previous patch was incomplete: it only removed the
template instantiations but not the dispatch calls in fattn.cu or the
extern declarations in fattn-vec.cuh. This caused linker errors
(undefined reference to ggml_cuda_flash_attn_ext_vec_case_d512).

This patch removes all three layers for all turbo V types:
- Template instance .cu files (DECL_FATTN_VEC_CASE_D512)
- Dispatch calls in fattn.cu (FATTN_VEC_CASE_D512)
- Extern declarations in fattn-vec.cuh (extern DECL_FATTN_VEC_CASE_D512)

Signed-off-by: mudler <mudler@localai.io>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>

2026-09-24 08:01:28 +02:00

patches

fix(turboquant): extend D512 flash-attn patch to all turbo V types (#12234 )

2026-09-24 08:01:28 +02:00

apply-patches.sh

feat(backend): add turboquant llama.cpp-fork backend (#9355 )

2026-04-15 01:25:04 +02:00

Makefile

chore: ⬆️ Update TheTom/llama-cpp-turboquant to 4deec5587b2963af00bdf80884f3337e02eb7d64 (#12154 )

2026-09-23 21:12:38 +00:00

package.sh

refactor(backends): extract package-system-libs.sh from 31 package.sh (#11095 )