mirror of
https://github.com/mudler/LocalAI.git
synced 2026-06-01 12:42:55 -04:00
798b5b2d84
Bump TURBOQUANT_VERSION from 627ebbc6 to 4d24ad87, which pulls in
upstream commit fa4e8be0a0ce ("fix(cuda): add F16-K + TURBO-V dispatch
cases in fattn.cu"). That commit adds three new template instance files
under ggml-cuda/template-instances/:
- fattn-vec-instance-f16-turbo2_0.cu
- fattn-vec-instance-f16-turbo3_0.cu
- fattn-vec-instance-f16-turbo4_0.cu
and wires matching FATTN_VEC_CASES_ALL_D(GGML_TYPE_F16, GGML_TYPE_TURBO{2,3,4}_0)
dispatch cases into fattn.cu. The dispatch cases are compiled into the
HIP build (fattn.cu is shared with ggml-hip via hipify), but the fork
forgot to mirror the new source files into ggml/src/ggml-hip/CMakeLists.txt.
CMake's ROCm branch carries a hand-curated template-instance list (used
when GGML_CUDA_FA_ALL_QUANTS is OFF, which is the default), so the HIP
build ends up with the extern template declarations but no matching
instantiations — the -gpu-rocm-hipblas-turboquant job failed at link
time (~90min into the 3h+ build).
Add patches/0001-ggml-hip-add-f16-turbo-vec-instances.patch, which the
existing apply-patches.sh machinery applies to the cloned fork sources
after fetch. The patch appends the three new f16-turbo instance files
to ggml-hip's source list in the same interleaved order used by
ggml-cuda's CMakeLists.txt. Drop this patch once the fork syncs the
ROCm list (the build will fail fast if the anchor context goes stale,
which is the signal to retire it).
CUDA builds were unaffected (ggml-cuda's CMakeLists.txt was updated
upstream) — the failure was isolated to HIP.
Assisted-by: Claude:claude-opus-4-7 [Claude Code]