mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-07-31 02:18:50 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(cachyllama): use portable ARM build

Browse Source 
Avoid the CPU_ALL_VARIANTS SME matrix on ARM, where the Linux and Darwin backend toolchains fail to compile CachyLLaMA. Keep the existing variant build for x86 and use the fully linked fallback build for ARM.

Assisted-by: Codex:gpt-5 [systematic-debugging]
This commit is contained in:
localai-org-maint-bot
2026-07-30 09:06:50 +00:00
parent 955fed341c
commit 0a063c57af
2 changed files with 12 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
.docker/cachyllama-compile.sh
View File

@@ -20,13 +20,15 @@ fi
cd /LocalAI/backend/cpp/cachyllama
if [ -z "${BUILD_TYPE:-}" ]; then
# Pure CPU image: one ggml CPU_ALL_VARIANTS build replaces the per-microarch binaries.
# arm64: the armv9.2 SME variants need gcc-14 (gcc-13 rejects +sme).
# Keep arm64 on the portable, fully linked build. CachyLLaMA's ARM
# CPU_ALL_VARIANTS build includes SME variants that do not build reliably
# across the Linux and Darwin toolchains used by backend CI.
if [ "${TARGETARCH}" = "arm64" ]; then
apt-get update -qq && apt-get install -y -qq gcc-14 g++-14
export CC=gcc-14 CXX=g++-14
make cachyllama-fallback
else
# One ggml CPU_ALL_VARIANTS build replaces the per-microarch x86 binaries.
make cachyllama-cpu-all
fi
make cachyllama-cpu-all
else
# GPU build (cublas/hipblas/sycl/vulkan/...): single fallback CPU build, the accelerator
# does the compute. Keeps the GPU compile from also building the CPU variant matrix and

25
scripts/build/cachyllama-darwin.sh
View File

@@ -6,11 +6,10 @@ IMAGE_NAME="${IMAGE_NAME:-localai/cachyllama-darwin}"
pushd backend/cpp/cachyllama
# Single build via ggml CPU_ALL_VARIANTS: one binary plus the per-microarch Apple/arm
# dylibs (apple_m1/m2_m3/m4, armv8.x) that ggml selects at runtime. GGML_METAL stays ON
# and --target ggml also builds ggml-metal (via add_dependencies), so the Metal GPU
# backend is still produced as a loadable libggml-metal.dylib.
make cachyllama-cpu-all && \
# CachyLLaMA's ARM CPU_ALL_VARIANTS build includes SME variants that do not build
# reliably across the Darwin toolchains used by backend CI. The fallback build
# remains Metal-enabled on Darwin and is fully linked.
make cachyllama-fallback && \
make cachyllama-grpc && \
make cachyllama-rpc-server
@@ -20,24 +19,10 @@ mkdir -p build/darwin
mkdir -p backend-images
mkdir -p build/darwin/lib
cp -rf backend/cpp/cachyllama/cachyllama-cpu-all build/darwin/
cp -rf backend/cpp/cachyllama/cachyllama-fallback build/darwin/
cp -rf backend/cpp/cachyllama/cachyllama-grpc build/darwin/
cp -rf backend/cpp/cachyllama/cachyllama-rpc-server build/darwin/
# Distribute the shared ggml/llama libraries from the CPU_ALL_VARIANTS build. Unlike the
# old fully-static fallback build, these have @rpath install names, so the otool loop below
# (which only copies deps that exist on disk) will not pick them up. The split is by suffix:
# - ggml emits its loadable backends (per-microarch CPU variants, metal, blas) with a .so
# suffix EVEN ON DARWIN. These go in the package ROOT next to the binary, because darwin
# run.sh execs the binary directly (no bundled ld.so) so ggml's executable-directory
# scan looks there.
# - the core libraries (libggml-base/libggml/libllama/libllama-common/libmtmd) use the
# platform .dylib suffix and are NEEDED deps; they go in lib/, resolved at load time via
# the DYLD_LIBRARY_PATH=lib that run.sh exports. -a preserves the version symlinks.
SHLIBS=backend/cpp/cachyllama/ggml-shared-libs
cp -a $SHLIBS/*.so build/darwin/
cp -a $SHLIBS/*.dylib build/darwin/lib/
# Set default additional libs only for Darwin on M chips (arm64)
if [[ "$(uname -s)" == "Darwin" && "$(uname -m)" == "arm64" ]]; then
ADDITIONAL_LIBS=${ADDITIONAL_LIBS:-$(ls /opt/homebrew/Cellar/protobuf/**/lib/libutf8_validity*.dylib 2>/dev/null)}