mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 09:57:57 -04:00
c5ad013def
Bundles the dependency closure for the from-scratch image, the dlopened ggml CPU-variant shared objects that ldd cannot see, and upstream's bundled silero_vad and marblenet_vad assets so VAD works with no download. The bundled loader sits in the package ROOT rather than at lib/ld.so. run.sh execs it, which makes /proc/self/exe name the loader, and this backend has two consumers of that path: ggml discovers the libggml-cpu-*.so by listing dirname(/proc/self/exe), and resolve_model_path expands bundled:<name> under the same directory. Rooting the loader makes the binary, the ggml objects and assets/ share the one directory all three resolution mechanisms agree on. llama-cpp's lib/ld.so layout would need assets/ moved into lib/ as well. The image builds against apt gRPC and protobuf, like Dockerfile.ds4 and unlike Dockerfile.privacy-filter. The from-source gRPC that install-base-deps.sh and the base-grpc-* images supply vendors protobuf 26, which pulls abseil into message_lite.h; with SPM_PROTOBUF_PROVIDER=package that collides with sentencepiece's vendored mini-abseil and every absl::internal reference becomes ambiguous. Noble's protobuf 3.21.12 predates the abseil dependency and is the pair every earlier verification of this backend ran against. Assisted-by: Claude:claude-opus-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
49 lines
2.3 KiB
Bash
Executable File
49 lines
2.3 KiB
Bash
Executable File
#!/bin/bash
|
|
# Entry point for the audio-cpp backend image and for BACKEND_BINARY mode.
|
|
#
|
|
# The image's final stage is FROM scratch, so the package root is / and there is
|
|
# no system loader, no system libc and no fallback library path. Everything the
|
|
# process opens has to be inside the package, and it has to be findable by the
|
|
# two mechanisms that actually do the finding: the dynamic linker, and
|
|
# audio.cpp's own directory scans.
|
|
set -e
|
|
|
|
CURDIR=$(dirname "$(realpath "$0")")
|
|
|
|
if [ "$(uname -s)" = "Darwin" ]; then
|
|
export DYLD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:$DYLD_LIBRARY_PATH"
|
|
exec "$CURDIR/grpc-server" "$@"
|
|
fi
|
|
|
|
# $CURDIR is on the path as well as $CURDIR/lib: the ggml shared objects the
|
|
# CPU-all-variants build produces sit in the package root, next to the binary,
|
|
# not in lib/. See the comment below for why they cannot live in lib/.
|
|
export LD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:$LD_LIBRARY_PATH"
|
|
|
|
# THE BUNDLED LOADER IS AT THE PACKAGE ROOT, NOT AT lib/ld.so. DO NOT MOVE IT.
|
|
#
|
|
# Exec'ing the loader is what pins the bundled glibc to the matching ld.so, and
|
|
# every other C++ backend here does it. The cost is that /proc/self/exe then
|
|
# names the LOADER rather than grpc-server, and this backend has two consumers
|
|
# of /proc/self/exe that both have to land on the package root:
|
|
#
|
|
# - ggml's backend registry DISCOVERS the per-microarch libggml-cpu-*.so by
|
|
# scanning dirname(/proc/self/exe) and the current directory. Those files
|
|
# are dlopened, never linked, so no library path and no RUNPATH reaches
|
|
# them: they have to be in a directory ggml scans.
|
|
# - resolve_model_path() turns "bundled:<name>" into
|
|
# dirname(/proc/self/exe)/assets/<name>, which is how the bundled
|
|
# silero_vad and marblenet_vad models resolve with nothing downloaded.
|
|
#
|
|
# backend/cpp/llama-cpp/package.sh answers the first of these by keeping
|
|
# lib/ld.so and moving the ggml objects INTO lib/. That does not generalise
|
|
# here, because it would also drag assets/ into lib/ to keep the second
|
|
# consumer working. Putting the loader in the package root instead makes
|
|
# dirname(/proc/self/exe) the package root, so the binary, the ggml objects and
|
|
# assets/ all sit in the one directory that all three mechanisms agree on.
|
|
if [ -f "$CURDIR/ld.so" ]; then
|
|
exec "$CURDIR/ld.so" "$CURDIR/grpc-server" "$@"
|
|
fi
|
|
|
|
exec "$CURDIR/grpc-server" "$@"
|