mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-25 07:34:58 -04:00
2ccda5ba92
* ⬆️ Update TheTom/llama-cpp-turboquant Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix(turboquant): patch D512 flash-attn shared memory overflow turboquant 4deec55 added DECL_FATTN_VEC_CASE_D512 for TURBO2_0 and TURBO3_0 V cache types. The D=512 kernel template with these types allocates 65 KB of shared memory, exceeding the 48 KB GPU limit: ptxas error: Entry function uses too much shared data (0x10100 bytes, 0xc000 max) Carry the fix as a patch under backend/cpp/turboquant/patches/ until TheTom/llama-cpp-turboquant#386 is merged upstream. TURBO4_0 (4-bit) does not overflow and is left unchanged. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>