Compare commits

...

4 Commits

Author SHA1 Message Date
localai-org-maint-bot
127a5780f2 fix(qwen-tts): limit CUDA compiler threads
The CUDA 13 FlashAttention build still exhausts hosted-runner memory with a single ninja worker because nvcc can compile multiple threads internally. Limit nvcc to one thread for that profile and guard the setting in the backend test script.

Assisted-by: Codex:gpt-5
2026-08-01 20:03:17 +00:00
localai-org-maint-bot
7916c48f04 test(qwen-tts): guard CUDA 13 build concurrency
Keep the CUDA 13 FlashAttention packaging check tied to its serialized source-build guard so the runner OOM fix cannot be removed unnoticed.\n\nAssisted-by: Codex:gpt-5 [Codex]
2026-08-01 03:03:31 +00:00
localai-org-maint-bot
abb6e2b9d0 fix(qwen-tts): serialize CUDA 13 flash-attn build
Limit the CUDA 13 FlashAttention source build to one ninja worker by default so it stays within CI runner memory. Preserve explicit MAX_JOBS overrides and leave other build profiles unchanged.

Assisted-by: Codex:gpt-5 [web]
2026-08-01 03:03:31 +00:00
localai-org-maint-bot
41d4251cd5 fix(qwen-tts): install flash-attn on CUDA 13
Add the missing cublas13 post-install requirements hook so the CUDA 13 backend uses FlashAttention instead of falling back to SDPA. Keep CUDA packaging parity covered by the backend test script.

Assisted-by: Codex:gpt-5
2026-08-01 03:03:31 +00:00
3 changed files with 17 additions and 0 deletions

View File

@@ -10,4 +10,11 @@ else
source $backend_dir/../common/libbackend.sh
fi
# CUDA 13 has no prebuilt FlashAttention wheel, so the fallback source build
# exceeds the CI runner's memory when ninja compiles multiple units at once.
if [ "x${BUILD_PROFILE}" = "xcublas13" ]; then
export MAX_JOBS="${MAX_JOBS:-1}"
export NVCC_THREADS="${NVCC_THREADS:-1}"
fi
installRequirements

View File

@@ -0,0 +1 @@
flash-attn

View File

@@ -2,6 +2,15 @@
set -e
backend_dir=$(dirname $0)
for cuda_version in 12 13; do
grep -qx "flash-attn" "$backend_dir/requirements-cublas${cuda_version}-after.txt"
done
grep -q 'BUILD_PROFILE.*cublas13' "$backend_dir/install.sh"
grep -q 'MAX_JOBS.*1' "$backend_dir/install.sh"
grep -q 'NVCC_THREADS.*1' "$backend_dir/install.sh"
if [ -d $backend_dir/common ]; then
source $backend_dir/common/libbackend.sh
else