mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-07-30 01:48:06 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(sherpa-onnx): install cuDNN in the CUDA builder so the package can bundle it (#11145)

Browse Source 
sherpa-onnx links onnxruntime's CUDA execution provider, and
libonnxruntime_providers_cuda.so carries cuDNN as a hard DT_NEEDED. The
onnxruntime GPU tarball ships no cuDNN of its own, and Dockerfile.golang
only installs libcudnn9 on the arm64 + CUDA 13 branch, so the amd64 CUDA
builders have none at all.

Since #10946 added the packaging guard, that combination is fatal rather
than silent: package-gpu-libs.sh reports 'cuDNN: venv=absent system=absent
-> bundle=detect', correctly detects the reference, finds nothing to copy
and refuses to emit the package. Both -gpu-nvidia-cuda-12-sherpa-onnx and
-gpu-nvidia-cuda-13-sherpa-onnx have failed to build since 2026-07-19, so
neither image has been published. Before the guard existed they shipped
without cuDNN and failed at load time instead.

Install the runtime package for this backend only. The auto-detection
bundles solely what a package references, so no other backend would grow,
but every Go CUDA builder would pay ~1.1 GB of layer and registry cache
for a library ggml never calls.


Assisted-by: Claude Code:claude-opus-5 [Read] [Edit] [Bash]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
mudler's LocalAI [bot]
2026-07-27 22:39:37 +02:00
committed by GitHub
parent 05ff401de8
commit 2ecf893c6c
1 changed files with 27 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
backend/Dockerfile.golang
View File

@@ -221,6 +221,33 @@ RUN if [ "${BACKEND}" = "crispasr" ]; then \
apt-get clean && rm -rf /var/lib/apt/lists/*; \
fi
# sherpa-onnx links onnxruntime's CUDA execution provider, and
# libonnxruntime_providers_cuda.so has cuDNN as a hard DT_NEEDED. The
# onnxruntime GPU tarball does not ship cuDNN itself, so without this the
# builder has none (the arm64 + CUDA 13 branch above is the only other place
# that installs it) and package-gpu-libs.sh correctly refuses to produce a
# package that references cuDNN with no cuDNN available to it.
#
# Installed per-backend rather than for every cublas build: the auto-detection
# in package-gpu-libs.sh bundles only what a package actually references, so
# the ggml backends would not grow either way, but they would all pay ~1.1 GB
# of builder layer and registry cache for a library they never call.
#
# Runtime package only, no -dev: sherpa-onnx consumes onnxruntime's prebuilt
# CUDA provider and never compiles against cuDNN headers. libcudnn9-cuda-N
# carries the dispatcher plus all seven dlopen()ed sublibraries, which is what
# complete_cudnn_family needs to assemble a whole bundle.
RUN <<EOT bash
if [ "${BACKEND}" = "sherpa-onnx" ] && [ "${BUILD_TYPE}" = "cublas" ] && [ "${SKIP_DRIVERS}" = "false" ]; then
apt-get update && \
apt-get install -y --no-install-recommends \
libcudnn9-cuda-${CUDA_MAJOR_VERSION} && \
ldconfig && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
fi
EOT
COPY . /LocalAI
RUN git config --global --add safe.directory /LocalAI