fix(fish-speech): use CUDA toolkit ptxas

Prefer an explicitly configured Triton assembler, otherwise use the executable ptxas from CUDA_HOME so torch.compile can target GPU architectures newer than Triton bundled tooling.

Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

backend/python/fish-speech/run.sh

@@ -10,4 +10,9 @@ fi
 # stale when the backend is relocated under /backends at install time.
 export PYTHONPATH="${EDIR}/fish-speech-src${PYTHONPATH:+:${PYTHONPATH}}"
 
+cuda_home=${CUDA_HOME:-/usr/local/cuda}
+if [ -z "${TRITON_PTXAS_PATH:-}" ] && [ -x "$cuda_home/bin/ptxas" ]; then
+    export TRITON_PTXAS_PATH="$cuda_home/bin/ptxas"
+fi
+
 startBackend "$@"

docs/content/features/text-to-audio.md

@@ -227,6 +227,24 @@ You can use the env variable COQUI_LANGUAGE to set the language used by the coqu
 
 You can also use config files to configure tts models (see section below on how to use config files).
 
+### Fish Speech
+
+Fish Speech models accept the `compile` backend option. Enabling it can improve
+inference performance on CUDA hardware, but the first request after loading the
+model includes the `torch.compile` warmup cost:
+
+```yaml
+backend: fish-speech
+options:
+  - compile:true
+```
+
+When compilation is enabled, the backend uses the CUDA toolkit's executable
+`ptxas` from `$CUDA_HOME/bin` (defaulting to `/usr/local/cuda/bin`) instead of
+the copy bundled with Triton. This allows newer GPU architectures supported by
+the installed CUDA toolkit to compile kernels. Set `TRITON_PTXAS_PATH` on the
+backend explicitly to select a different assembler.
+
 ### Piper
 
 To install the `piper` audio models manually:

scripts/build/fish-speech-ptxas_test.sh

@@ -0,0 +1,39 @@
+#!/bin/bash
+set -euo pipefail
+
+WORK=$(mktemp -d)
+trap 'rm -rf "$WORK"' EXIT
+
+REPO_ROOT=$(dirname "$(dirname "$(dirname "$(realpath "$0")")")")
+BACKEND_DIR="$WORK/fish-speech"
+mkdir -p "$BACKEND_DIR/common" "$WORK/cuda/bin"
+cp "$REPO_ROOT/backend/python/fish-speech/run.sh" "$BACKEND_DIR/run.sh"
+
+cat > "$BACKEND_DIR/common/libbackend.sh" <<'LIBBACKEND'
+startBackend() {
+    printf '%s\n' "${TRITON_PTXAS_PATH:-}"
+}
+LIBBACKEND
+
+fail() {
+    echo "FAIL: $*"
+    exit 1
+}
+
+touch "$WORK/cuda/bin/ptxas"
+chmod +x "$WORK/cuda/bin/ptxas"
+
+got=$(CUDA_HOME="$WORK/cuda" bash "$BACKEND_DIR/run.sh")
+[ "$got" = "$WORK/cuda/bin/ptxas" ] || \
+    fail "expected toolkit ptxas, got '$got'"
+
+got=$(CUDA_HOME="$WORK/cuda" TRITON_PTXAS_PATH=/custom/ptxas \
+    bash "$BACKEND_DIR/run.sh")
+[ "$got" = "/custom/ptxas" ] || \
+    fail "explicit TRITON_PTXAS_PATH was overwritten with '$got'"
+
+chmod -x "$WORK/cuda/bin/ptxas"
+got=$(CUDA_HOME="$WORK/cuda" bash "$BACKEND_DIR/run.sh")
+[ -z "$got" ] || fail "non-executable ptxas was selected as '$got'"
+
+echo "PASS: fish-speech selects a usable toolkit ptxas"