mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 01:48:06 -04:00
ab9c6c4a2e
The arm64 CPU image failed to build:
cc1: error: invalid feature modifier 'sme' in
'-march=armv9.2-a+dotprod+fp16+sve+i8mm+sve2+sme'
ggml's CPU_ALL_VARIANTS table includes armv9.2 variants compiled with +sme, and
Ubuntu Noble's default gcc-13 rejects that feature modifier. Every entry in the
table has to compile even though a host only ever dlopens the one its own CPU
supports, so a single unbuildable variant fails the whole image. gcc-14 accepts
it, which is exactly the fix llama-cpp already carries in
.docker/llama-cpp-compile.sh; this is the same problem reached by a different
Dockerfile.
Applied to every arm64 BUILD_TYPE rather than to the CPU one alone, and that
differs from llama-cpp on purpose. llama-cpp needs it only for its pure-CPU
image because its GPU builds run llama-cpp-fallback, which builds no variant
table. This backend's Makefile turns ENGINE_ENABLE_CPU_ALL_VARIANTS on for
every non-Darwin build, GPU included, so an arm64 GPU image would hit the
identical error. The matrix has no arm64 GPU entry today, which is precisely
why gating on an empty BUILD_TYPE would leave the trap armed for whoever adds
the first one.
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]
121 lines
6.1 KiB
Docker
121 lines
6.1 KiB
Docker
ARG BASE_IMAGE=ubuntu:24.04
|
|
ARG APT_MIRROR=""
|
|
ARG APT_PORTS_MIRROR=""
|
|
|
|
# audio-cpp: 0xShug0/audio.cpp, a ggml audio inference framework covering TTS,
|
|
# ASR, VAD, diarization, source separation and music generation, wrapped as a
|
|
# LocalAI gRPC backend.
|
|
#
|
|
# BASE_IMAGE is ubuntu:24.04 for cpu and vulkan builds, or
|
|
# nvidia/cuda:<ver>-devel-ubuntu24.04 for cublas builds; both ship apt and
|
|
# Ubuntu Noble packages, and the CUDA base additionally provides
|
|
# /usr/local/cuda. BUILD_TYPE selects the engine backend in the Makefile:
|
|
# "" = portable CPU with all ggml CPU variants, "cublas" ->
|
|
# -DENGINE_ENABLE_CUDA=ON, "vulkan" -> -DENGINE_ENABLE_VULKAN=ON. Darwin
|
|
# (Metal) builds bypass this Dockerfile entirely.
|
|
#
|
|
# Upstream needs GCC 13 or newer, which ubuntu:24.04 and the CUDA 12/13
|
|
# devel-ubuntu24.04 images all provide.
|
|
#
|
|
# THIS BACKEND CANNOT USE .docker/install-base-deps.sh OR THE PREBUILT
|
|
# quay.io/go-skynet/ci-cache:base-grpc-* IMAGES, AND THAT IS NOT A STYLE CHOICE.
|
|
#
|
|
# Both supply gRPC v1.65 built from source at /opt/grpc, which downstream
|
|
# Dockerfiles copy to /usr/local. That gRPC vendors protobuf v26, and protobuf
|
|
# has depended on abseil since v22: google/protobuf/message_lite.h includes
|
|
# absl/strings/cord.h. audio.cpp links sentencepiece, and our CMakeLists sets
|
|
# SPM_PROTOBUF_PROVIDER=package so sentencepiece uses the same protobuf the
|
|
# generated backend.pb.cc was built against (the alternative broke every
|
|
# nested-message parse; the full account is in backend/cpp/audio-cpp/CMakeLists.txt).
|
|
# That makes sentencepiece's init.h include the external message_lite.h while it
|
|
# still includes its own vendored mini-abseil from third_party/absl. The vendored
|
|
# copy declares `namespace absl { namespace internal { ... } }` and real abseil
|
|
# declares `namespace absl { inline namespace lts_20240116 { namespace internal
|
|
# { ... } } }`, so every `absl::internal::` reference becomes ambiguous and the
|
|
# compile dies in absl/base/casts.h. Verified, not theorised: building this image
|
|
# against the base-grpc-amd64 prebuilt fails at
|
|
# sentencepiece-static/error.cc.o with "reference to 'internal' is ambiguous".
|
|
#
|
|
# Ubuntu Noble's apt protobuf is 3.21.12, which predates the abseil dependency,
|
|
# so message_lite.h pulls in no abseil and the vendored copy is the only one in
|
|
# scope. That is also the exact protobuf/gRPC pair every unit and end-to-end run
|
|
# of this backend has been verified against. Keep it: a from-source gRPC here
|
|
# does not buy a faster build, it buys a broken one.
|
|
#
|
|
# The install-base-deps path is additionally unsafe because it drops protoc 27.1
|
|
# into /usr/local/bin, which shadows apt's protoc on PATH and would generate
|
|
# protobuf-27 sources to be compiled against 3.21 headers.
|
|
FROM ${BASE_IMAGE} AS builder
|
|
ARG BUILD_TYPE
|
|
ARG TARGETARCH
|
|
ARG TARGETVARIANT
|
|
ARG APT_MIRROR
|
|
ARG APT_PORTS_MIRROR
|
|
# Selects the CUDA architecture list in backend/cpp/audio-cpp/Makefile. It has
|
|
# to be forwarded: upstream compiles engine_runtime for `native` when
|
|
# CMAKE_CUDA_ARCHITECTURES is unset, and no CI runner has a GPU to enumerate.
|
|
# The value is the same cuda-major-version the matrix entry declares.
|
|
ARG CUDA_MAJOR_VERSION
|
|
|
|
ENV BUILD_TYPE=${BUILD_TYPE} \
|
|
CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} \
|
|
APT_MIRROR=${APT_MIRROR} \
|
|
APT_PORTS_MIRROR=${APT_PORTS_MIRROR} \
|
|
DEBIAN_FRONTEND=noninteractive \
|
|
PATH=/usr/local/cuda/bin:${PATH}
|
|
|
|
WORKDIR /build
|
|
|
|
# gRPC/protobuf from apt, deliberately; see the block above. libgrpc++-dev ships
|
|
# a CMake config so find_package(gRPC CONFIG) resolves, and libprotobuf-dev
|
|
# lands in the layout CMake's FindProtobuf module expects, which matters because
|
|
# sentencepiece runs a bare find_package(Protobuf REQUIRED) with no CONFIG
|
|
# fallback of its own.
|
|
#
|
|
# BUILD_TYPE=vulkan additionally needs the loader headers and glslc; both are in
|
|
# Noble. The CUDA toolkit for BUILD_TYPE=cublas comes from BASE_IMAGE.
|
|
RUN --mount=type=bind,source=.docker/apt-mirror.sh,target=/usr/local/sbin/apt-mirror \
|
|
sh /usr/local/sbin/apt-mirror && \
|
|
apt-get update && \
|
|
apt-get install -y --no-install-recommends \
|
|
git cmake build-essential pkg-config ca-certificates \
|
|
libgrpc++-dev libprotobuf-dev protobuf-compiler protobuf-compiler-grpc && \
|
|
if [ "${BUILD_TYPE}" = "vulkan" ]; then \
|
|
apt-get install -y --no-install-recommends libvulkan-dev glslc; \
|
|
fi && \
|
|
if [ "${TARGETARCH}" = "arm64" ]; then \
|
|
apt-get install -y --no-install-recommends gcc-14 g++-14; \
|
|
fi && \
|
|
apt-get clean && \
|
|
rm -rf /var/lib/apt/lists/*
|
|
|
|
COPY . /LocalAI
|
|
|
|
# gcc-14 on arm64, for the same reason llama-cpp does it in
|
|
# .docker/llama-cpp-compile.sh: ggml's CPU_ALL_VARIANTS table includes armv9.2
|
|
# variants built with -march=...+sme, and Noble's default gcc-13 rejects that
|
|
# feature modifier outright ("invalid feature modifier 'sme'"). Every variant in
|
|
# the table has to COMPILE even though a host only ever dlopens the one its own
|
|
# CPU supports, so one unbuildable variant fails the whole image.
|
|
#
|
|
# ON EVERY arm64 BUILD_TYPE, which is where this differs from llama-cpp's script
|
|
# and why that difference is spelled out rather than assumed. llama-cpp only
|
|
# needs gcc-14 for its pure-CPU image because its GPU builds run
|
|
# llama-cpp-fallback, which has no variant table at all. This backend's Makefile
|
|
# sets ENGINE_ENABLE_CPU_ALL_VARIANTS for every non-Darwin build, GPU included,
|
|
# so an arm64 GPU image would hit the identical compile error. Gating this on an
|
|
# empty BUILD_TYPE would leave that trap armed for the first arm64 GPU entry
|
|
# added to the matrix, which today has none.
|
|
RUN --mount=type=cache,target=/root/.ccache,id=audio-cpp-ccache-${TARGETARCH}-${BUILD_TYPE},sharing=locked \
|
|
if [ "${TARGETARCH}" = "arm64" ]; then \
|
|
export CC=gcc-14 CXX=g++-14; \
|
|
fi && \
|
|
make -C /LocalAI/backend/cpp/audio-cpp BUILD_TYPE=${BUILD_TYPE} \
|
|
CUDA_MAJOR_VERSION=${CUDA_MAJOR_VERSION} NATIVE=false grpc-server package
|
|
|
|
# The package directory is the whole image: run.sh, grpc-server, the dlopened
|
|
# ggml CPU variants, the bundled loader and its library closure, and the
|
|
# bundled silero_vad / marblenet_vad assets. Nothing else exists at run time.
|
|
FROM scratch
|
|
COPY --from=builder /LocalAI/backend/cpp/audio-cpp/package/. ./
|