backend(audio-cpp): pin the CUDA architectures, drop the vulkan variant

Upstream sets CUDA_ARCHITECTURES to `native` on the engine_runtime target
whenever CMAKE_CUDA_ARCHITECTURES is unset at root scope, and docs/build/
linux.md says so outright. ggml's own default does not rescue it: it
list(APPEND)s in the ggml subdirectory scope, which never reaches the root
scope where the engine_runtime property is decided. No CI runner has a GPU for
`native` to enumerate, so both cublas entries would have gone red on the very
commit that first turns a CUDA build on.

Pin the list in backend/cpp/audio-cpp/Makefile, selected by CUDA_MAJOR_VERSION,
which Dockerfile.audio-cpp now forwards from the CI build-arg it was previously
discarding. The values are copied from ggml's own version guards rather than
invented, so engine_runtime and ggml compile for the same set: CUDA 12 keeps the
Maxwell/Pascal/Volta virtual archs and stops at 120a-real, CUDA 13 drops them
and adds 121a-real. The `a` suffix is used rather than `f` because the latter
needs CMake 3.31.8 and Ubuntu Noble ships 3.28.3. Verified by driving CMake
3.28.3's own CUDA architecture validator over both lists, with 120f-virtual as
the rejected control.

Drop the vulkan matrix entry, its two gallery entries, the vulkan capability
key on both metas and the Vulkan tag. Every other vulkan backend gets its Mesa
ICD drivers from .docker/install-base-deps.sh, which package-gpu-libs.sh then
bundles; Dockerfile.audio-cpp calls neither and installs only libvulkan-dev and
glslc, so the image would ship a Vulkan loader that finds no GPU. No CI job runs
a vulkan image against real hardware, so that would have passed green and failed
in users' hands. BUILD_TYPE=vulkan stays supported for local builds.

Also note on the cublas entries that cuda-major-version now selects the
architecture list and that cuda-minor-version and the base-image tag encode the
same toolkit, and correct the stale entry counts on matrixEntryKey.

Assisted-by: Claude:claude-opus-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

.github/backend-matrix.yml

@@ -3167,6 +3167,14 @@ include:
   # No ROCm entry: upstream has no HIP configuration. No CUDA arm64 or L4T
   # entry: upstream documents and validates CUDA on x86 only. Darwin/Metal
   # lands with scripts/build/audio-cpp-darwin.sh, which does not exist yet.
+  #
+  # No vulkan entry either, though Dockerfile.audio-cpp and the backend Makefile
+  # both handle BUILD_TYPE=vulkan for local builds. Every other vulkan backend
+  # gets its Mesa ICD drivers from .docker/install-base-deps.sh, which installs
+  # mesa-vulkan-drivers so package-gpu-libs.sh can bundle them; this Dockerfile
+  # calls neither, so the image would ship a Vulkan loader that finds no GPU. No
+  # CI job runs a vulkan image against real hardware, so it would pass green and
+  # fail in users' hands. The entry comes back once the ICD question is settled.
   - build-type: ''
     cuda-major-version: ""
     cuda-minor-version: ""
@@ -3195,6 +3203,11 @@ include:
     dockerfile: "./backend/Dockerfile.audio-cpp"
     context: "./"
     ubuntu-version: '2404'
+  # cuda-major-version is forwarded into the build (Dockerfile.audio-cpp -> the
+  # backend Makefile) and picks the CMAKE_CUDA_ARCHITECTURES list, which upstream
+  # otherwise sets to `native` and no CI runner can enumerate. cuda-minor-version
+  # and the base-image tag encode the same toolkit and must move together;
+  # nothing checks that for you.
   - build-type: 'cublas'
     cuda-major-version: "12"
     cuda-minor-version: "8"
@@ -3221,22 +3234,6 @@ include:
     dockerfile: "./backend/Dockerfile.audio-cpp"
     context: "./"
     ubuntu-version: '2404'
-  # Vulkan: ubuntu:24.04 plus apt's libvulkan-dev + glslc, installed by
-  # Dockerfile.audio-cpp when BUILD_TYPE=vulkan. arm64 vulkan is a one-line add
-  # once amd64 is proven in CI.
-  - build-type: 'vulkan'
-    cuda-major-version: ""
-    cuda-minor-version: ""
-    platforms: 'linux/amd64'
-    tag-latest: 'auto'
-    tag-suffix: '-gpu-vulkan-audio-cpp'
-    runs-on: 'ubuntu-latest'
-    base-image: "ubuntu:24.04"
-    skip-drivers: 'false'
-    backend: "audio-cpp"
-    dockerfile: "./backend/Dockerfile.audio-cpp"
-    context: "./"
-    ubuntu-version: '2404'
   - build-type: ''
     cuda-major-version: ""
     cuda-minor-version: ""

backend/Dockerfile.audio-cpp

@@ -51,8 +51,14 @@ ARG TARGETARCH
 ARG TARGETVARIANT
 ARG APT_MIRROR
 ARG APT_PORTS_MIRROR
+# Selects the CUDA architecture list in backend/cpp/audio-cpp/Makefile. It has
+# to be forwarded: upstream compiles engine_runtime for `native` when
+# CMAKE_CUDA_ARCHITECTURES is unset, and no CI runner has a GPU to enumerate.
+# The value is the same cuda-major-version the matrix entry declares.
+ARG CUDA_MAJOR_VERSION
 
 ENV BUILD_TYPE=${BUILD_TYPE} \
+    CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} \
     APT_MIRROR=${APT_MIRROR} \
     APT_PORTS_MIRROR=${APT_PORTS_MIRROR} \
     DEBIAN_FRONTEND=noninteractive \
@@ -83,7 +89,8 @@ RUN --mount=type=bind,source=.docker/apt-mirror.sh,target=/usr/local/sbin/apt-mi
 COPY . /LocalAI
 
 RUN --mount=type=cache,target=/root/.ccache,id=audio-cpp-ccache-${TARGETARCH}-${BUILD_TYPE},sharing=locked \
-    make -C /LocalAI/backend/cpp/audio-cpp BUILD_TYPE=${BUILD_TYPE} NATIVE=false grpc-server package
+    make -C /LocalAI/backend/cpp/audio-cpp BUILD_TYPE=${BUILD_TYPE} \
+        CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} NATIVE=false grpc-server package
 
 # The package directory is the whole image: run.sh, grpc-server, the dlopened
 # ggml CPU variants, the bundled loader and its library closure, and the

backend/cpp/audio-cpp/Makefile

@@ -26,8 +26,39 @@ UNAME_S := $(shell uname -s)
 # safetensors model tree still resolves its family spec.
 CMAKE_ARGS ?= -DCMAKE_BUILD_TYPE=Release -DAUDIOCPP_DEPLOYMENT_BUILD=ON
 
+# CMAKE_CUDA_ARCHITECTURES must be set explicitly for a cublas build, and this
+# is not a tuning knob: upstream's CMakeLists sets CUDA_ARCHITECTURES to
+# `native` on the engine_runtime target whenever the root-scope variable is
+# unset (audio.cpp/CMakeLists.txt, the `if (CMAKE_CUDA_ARCHITECTURES)` branch
+# next to the istft/torch_random .cu sources), and docs/build/linux.md says so
+# outright: "Leave CMAKE_CUDA_ARCHITECTURES unset to build for the GPUs present
+# at build time (native)". No CI runner has a GPU, so `native` has nothing to
+# enumerate. ggml's own default (external/ggml/src/ggml-cuda/CMakeLists.txt)
+# does not rescue this: it list(APPEND)s in the ggml subdirectory scope, which
+# never reaches the root scope where the engine_runtime property is decided.
+#
+# The values below are ggml's list for the matching toolkit, copied rather than
+# invented, so the two targets compile for exactly the same set:
+#   - CUDA 13 drops the Maxwell/Pascal/Volta virtual archs (50/61/70).
+#   - 121a-real needs CUDA >= 12.9, so the CUDA 12 list (built against 12.8)
+#     stops at 120a-real.
+#   - `a`-suffixed archs are used rather than ggml's rejected 120f-virtual: the
+#     `f` suffix needs CMake >= 3.31.8, and Ubuntu Noble ships 3.28.3. The
+#     3.28 validator (Modules/Internal/CMakeCUDAArchitecturesValidate.cmake)
+#     accepts `[0-9]+a?(-real|-virtual)?`.
+#
+# Setting it here also pins ggml's copy, since its default is guarded by
+# `if (NOT DEFINED CMAKE_CUDA_ARCHITECTURES)`. CUDA_MAJOR_VERSION is the CI
+# build-arg, forwarded by Dockerfile.audio-cpp.
+CUDA_MAJOR_VERSION ?=
+ifeq ($(CUDA_MAJOR_VERSION),13)
+    CUDA_ARCHITECTURES ?= 75-virtual;80-virtual;86-real;89-real;120a-real;121a-real
+else
+    CUDA_ARCHITECTURES ?= 50-virtual;61-virtual;70-virtual;75-virtual;80-virtual;86-real;89-real;120a-real
+endif
+
 ifeq ($(BUILD_TYPE),cublas)
-    CMAKE_ARGS += -DENGINE_ENABLE_CUDA=ON
+    CMAKE_ARGS += -DENGINE_ENABLE_CUDA=ON "-DCMAKE_CUDA_ARCHITECTURES=$(CUDA_ARCHITECTURES)"
 else ifeq ($(BUILD_TYPE),vulkan)
     CMAKE_ARGS += -DENGINE_ENABLE_VULKAN=ON
 else ifeq ($(UNAME_S),Darwin)

backend/index.yaml

@@ -148,16 +148,16 @@
     - audio-transcription
     - CPU
     - CUDA
-    - Vulkan
-  # No metal/metal-darwin-arm64 key: there is no Darwin matrix entry for
-  # audio-cpp yet, and pointing a capability at a tag CI never builds hands the
-  # user a pull failure instead of the CPU fallback below.
+  # No metal/metal-darwin-arm64 key (no Darwin matrix entry yet) and no vulkan
+  # key (the vulkan image would carry a Vulkan loader with no Mesa ICD; see the
+  # audio-cpp block in .github/backend-matrix.yml). Pointing a capability at a
+  # tag CI never builds hands the user a pull failure; leaving it out lets
+  # SystemState.Capability() fall through to `default` and the CPU image.
   capabilities:
     default: "cpu-audio-cpp"
     nvidia: "cuda12-audio-cpp"
     nvidia-cuda-12: "cuda12-audio-cpp"
     nvidia-cuda-13: "cuda13-audio-cpp"
-    vulkan: "vulkan-audio-cpp"
 - &whispercpp
   name: "whisper"
   alias: "whisper"
@@ -2037,7 +2037,6 @@
     nvidia: "cuda12-audio-cpp-development"
     nvidia-cuda-12: "cuda12-audio-cpp-development"
     nvidia-cuda-13: "cuda13-audio-cpp-development"
-    vulkan: "vulkan-audio-cpp-development"
 - !!merge <<: *stablediffusionggml
   name: "stablediffusion-ggml-development"
   capabilities:
@@ -6896,13 +6895,3 @@
   uri: "quay.io/go-skynet/local-ai-backends:master-gpu-nvidia-cuda-13-audio-cpp"
   mirrors:
     - localai/localai-backends:master-gpu-nvidia-cuda-13-audio-cpp
-- !!merge <<: *audiocpp
-  name: "vulkan-audio-cpp"
-  uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-vulkan-audio-cpp"
-  mirrors:
-    - localai/localai-backends:latest-gpu-vulkan-audio-cpp
-- !!merge <<: *audiocpp
-  name: "vulkan-audio-cpp-development"
-  uri: "quay.io/go-skynet/local-ai-backends:master-gpu-vulkan-audio-cpp"
-  mirrors:
-    - localai/localai-backends:master-gpu-vulkan-audio-cpp

scripts/lib/backend-filter.mjs

@@ -473,7 +473,7 @@ export const BACKEND_MATRIX_FILE = ".github/backend-matrix.yml";
 
 // Identity of a matrix entry across revisions. tag-suffix names the image;
 // per-arch legs of the same image are distinguished by platform-tag. Verified
-// unique across all 417 Linux and 56 Darwin entries.
+// unique across all 432 Linux and 57 Darwin entries.
 export function matrixEntryKey(item) {
   return JSON.stringify([item["tag-suffix"] || "", item["platform-tag"] || ""]);
 }