mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 09:57:57 -04:00
5af51273c2
Upstream sets CUDA_ARCHITECTURES to `native` on the engine_runtime target whenever CMAKE_CUDA_ARCHITECTURES is unset at root scope, and docs/build/ linux.md says so outright. ggml's own default does not rescue it: it list(APPEND)s in the ggml subdirectory scope, which never reaches the root scope where the engine_runtime property is decided. No CI runner has a GPU for `native` to enumerate, so both cublas entries would have gone red on the very commit that first turns a CUDA build on. Pin the list in backend/cpp/audio-cpp/Makefile, selected by CUDA_MAJOR_VERSION, which Dockerfile.audio-cpp now forwards from the CI build-arg it was previously discarding. The values are copied from ggml's own version guards rather than invented, so engine_runtime and ggml compile for the same set: CUDA 12 keeps the Maxwell/Pascal/Volta virtual archs and stops at 120a-real, CUDA 13 drops them and adds 121a-real. The `a` suffix is used rather than `f` because the latter needs CMake 3.31.8 and Ubuntu Noble ships 3.28.3. Verified by driving CMake 3.28.3's own CUDA architecture validator over both lists, with 120f-virtual as the rejected control. Drop the vulkan matrix entry, its two gallery entries, the vulkan capability key on both metas and the Vulkan tag. Every other vulkan backend gets its Mesa ICD drivers from .docker/install-base-deps.sh, which package-gpu-libs.sh then bundles; Dockerfile.audio-cpp calls neither and installs only libvulkan-dev and glslc, so the image would ship a Vulkan loader that finds no GPU. No CI job runs a vulkan image against real hardware, so that would have passed green and failed in users' hands. BUILD_TYPE=vulkan stays supported for local builds. Also note on the cublas entries that cuda-major-version now selects the architecture list and that cuda-minor-version and the base-image tag encode the same toolkit, and correct the stale entry counts on matrixEntryKey. Assisted-by: Claude:claude-opus-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
100 lines
4.9 KiB
Docker
100 lines
4.9 KiB
Docker
ARG BASE_IMAGE=ubuntu:24.04
|
|
ARG APT_MIRROR=""
|
|
ARG APT_PORTS_MIRROR=""
|
|
|
|
# audio-cpp: 0xShug0/audio.cpp, a ggml audio inference framework covering TTS,
|
|
# ASR, VAD, diarization, source separation and music generation, wrapped as a
|
|
# LocalAI gRPC backend.
|
|
#
|
|
# BASE_IMAGE is ubuntu:24.04 for cpu and vulkan builds, or
|
|
# nvidia/cuda:<ver>-devel-ubuntu24.04 for cublas builds; both ship apt and
|
|
# Ubuntu Noble packages, and the CUDA base additionally provides
|
|
# /usr/local/cuda. BUILD_TYPE selects the engine backend in the Makefile:
|
|
# "" = portable CPU with all ggml CPU variants, "cublas" ->
|
|
# -DENGINE_ENABLE_CUDA=ON, "vulkan" -> -DENGINE_ENABLE_VULKAN=ON. Darwin
|
|
# (Metal) builds bypass this Dockerfile entirely.
|
|
#
|
|
# Upstream needs GCC 13 or newer, which ubuntu:24.04 and the CUDA 12/13
|
|
# devel-ubuntu24.04 images all provide.
|
|
#
|
|
# THIS BACKEND CANNOT USE .docker/install-base-deps.sh OR THE PREBUILT
|
|
# quay.io/go-skynet/ci-cache:base-grpc-* IMAGES, AND THAT IS NOT A STYLE CHOICE.
|
|
#
|
|
# Both supply gRPC v1.65 built from source at /opt/grpc, which downstream
|
|
# Dockerfiles copy to /usr/local. That gRPC vendors protobuf v26, and protobuf
|
|
# has depended on abseil since v22: google/protobuf/message_lite.h includes
|
|
# absl/strings/cord.h. audio.cpp links sentencepiece, and our CMakeLists sets
|
|
# SPM_PROTOBUF_PROVIDER=package so sentencepiece uses the same protobuf the
|
|
# generated backend.pb.cc was built against (the alternative broke every
|
|
# nested-message parse; the full account is in backend/cpp/audio-cpp/CMakeLists.txt).
|
|
# That makes sentencepiece's init.h include the external message_lite.h while it
|
|
# still includes its own vendored mini-abseil from third_party/absl. The vendored
|
|
# copy declares `namespace absl { namespace internal { ... } }` and real abseil
|
|
# declares `namespace absl { inline namespace lts_20240116 { namespace internal
|
|
# { ... } } }`, so every `absl::internal::` reference becomes ambiguous and the
|
|
# compile dies in absl/base/casts.h. Verified, not theorised: building this image
|
|
# against the base-grpc-amd64 prebuilt fails at
|
|
# sentencepiece-static/error.cc.o with "reference to 'internal' is ambiguous".
|
|
#
|
|
# Ubuntu Noble's apt protobuf is 3.21.12, which predates the abseil dependency,
|
|
# so message_lite.h pulls in no abseil and the vendored copy is the only one in
|
|
# scope. That is also the exact protobuf/gRPC pair every unit and end-to-end run
|
|
# of this backend has been verified against. Keep it: a from-source gRPC here
|
|
# does not buy a faster build, it buys a broken one.
|
|
#
|
|
# The install-base-deps path is additionally unsafe because it drops protoc 27.1
|
|
# into /usr/local/bin, which shadows apt's protoc on PATH and would generate
|
|
# protobuf-27 sources to be compiled against 3.21 headers.
|
|
FROM ${BASE_IMAGE} AS builder
|
|
ARG BUILD_TYPE
|
|
ARG TARGETARCH
|
|
ARG TARGETVARIANT
|
|
ARG APT_MIRROR
|
|
ARG APT_PORTS_MIRROR
|
|
# Selects the CUDA architecture list in backend/cpp/audio-cpp/Makefile. It has
|
|
# to be forwarded: upstream compiles engine_runtime for `native` when
|
|
# CMAKE_CUDA_ARCHITECTURES is unset, and no CI runner has a GPU to enumerate.
|
|
# The value is the same cuda-major-version the matrix entry declares.
|
|
ARG CUDA_MAJOR_VERSION
|
|
|
|
ENV BUILD_TYPE=${BUILD_TYPE} \
|
|
CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} \
|
|
APT_MIRROR=${APT_MIRROR} \
|
|
APT_PORTS_MIRROR=${APT_PORTS_MIRROR} \
|
|
DEBIAN_FRONTEND=noninteractive \
|
|
PATH=/usr/local/cuda/bin:${PATH}
|
|
|
|
WORKDIR /build
|
|
|
|
# gRPC/protobuf from apt, deliberately; see the block above. libgrpc++-dev ships
|
|
# a CMake config so find_package(gRPC CONFIG) resolves, and libprotobuf-dev
|
|
# lands in the layout CMake's FindProtobuf module expects, which matters because
|
|
# sentencepiece runs a bare find_package(Protobuf REQUIRED) with no CONFIG
|
|
# fallback of its own.
|
|
#
|
|
# BUILD_TYPE=vulkan additionally needs the loader headers and glslc; both are in
|
|
# Noble. The CUDA toolkit for BUILD_TYPE=cublas comes from BASE_IMAGE.
|
|
RUN --mount=type=bind,source=.docker/apt-mirror.sh,target=/usr/local/sbin/apt-mirror \
|
|
sh /usr/local/sbin/apt-mirror && \
|
|
apt-get update && \
|
|
apt-get install -y --no-install-recommends \
|
|
git cmake build-essential pkg-config ca-certificates \
|
|
libgrpc++-dev libprotobuf-dev protobuf-compiler protobuf-compiler-grpc && \
|
|
if [ "${BUILD_TYPE}" = "vulkan" ]; then \
|
|
apt-get install -y --no-install-recommends libvulkan-dev glslc; \
|
|
fi && \
|
|
apt-get clean && \
|
|
rm -rf /var/lib/apt/lists/*
|
|
|
|
COPY . /LocalAI
|
|
|
|
RUN --mount=type=cache,target=/root/.ccache,id=audio-cpp-ccache-${TARGETARCH}-${BUILD_TYPE},sharing=locked \
|
|
make -C /LocalAI/backend/cpp/audio-cpp BUILD_TYPE=${BUILD_TYPE} \
|
|
CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} NATIVE=false grpc-server package
|
|
|
|
# The package directory is the whole image: run.sh, grpc-server, the dlopened
|
|
# ggml CPU variants, the bundled loader and its library closure, and the
|
|
# bundled silero_vad / marblenet_vad assets. Nothing else exists at run time.
|
|
FROM scratch
|
|
COPY --from=builder /LocalAI/backend/cpp/audio-cpp/package/. ./
|