mirror/LocalAI
1
0 Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-09-28 00:55:06 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(turboquant): retain CPU variants in GPU builds (#11276)

Browse Source 
Select the CPU_ALL_VARIANTS target for x86 GPU images so partial offload uses runtime-selected host kernels. Keep GPU arm64 builds on the portable fallback until their toolchains consistently provide gcc-14.

Assisted-by: Codex:gpt-5

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
This commit is contained in:
localai-org-maint-botandlocalai-org-maint-bot authored and GitHub committed 2026-08-01 16:06:36 +02:00
1 parent ad2be8a856
commit 9fe1165f61
4 files changed
+50 -10
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

backend/cpp/turboquant/run.sh
+4 -2
View File
@@ -12,9 +12,11 @@ grep -e "flags" /proc/cpuinfo | head -1
BINARY=turboquant-fallback
# x86/arm64 ship a single turboquant-cpu-all built with ggml CPU_ALL_VARIANTS: ggml's
# CPU images and x86 GPU images ship a single turboquant-cpu-all built with ggml
# CPU_ALL_VARIANTS: ggml's
# backend registry dlopens the best libggml-cpu-*.so for this host, so no shell-side
# probing. ROCm ships only turboquant-fallback, so fall back to it when cpu-all is absent.
# probing. GPU arm64 images still ship turboquant-fallback until their builder toolchains
# support ggml's complete arm variant matrix.
if [ -e "$CURDIR"/turboquant-cpu-all ]; then
BINARY=turboquant-cpu-all
fi