mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-06-25 09:09:07 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(llama-cpp,turboquant): arm64 gcc-14 for SME variants + darwin cpu-all packaging

Browse Source 
- arm64: ggml CPU_ALL_VARIANTS builds armv9.2 SME variants whose -march=...+sme
  is rejected by the Ubuntu 24.04 default gcc-13. Build the arm64 variants with
  gcc-14 (installed in the compile step). The host only selects a variant it
  actually supports at runtime, but every variant must still compile.
- darwin: scripts/build/llama-cpp-darwin.sh builds llama-cpp-cpu-all instead of
  the fallback binary, keeps Metal (GGML_METAL stays ON; --target ggml also builds
  ggml-metal). The per-microarch libggml-cpu-*.dylib are placed in the package
  root next to the binary (darwin has no bundled ld.so, so ggml's executable-dir
  scan looks there), while the other shared dylibs go in lib/ for DYLD_LIBRARY_PATH.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-4-8 [Claude Code]
This commit is contained in:
Ettore Di Giacinto
2026-06-24 21:50:29 +00:00
parent 379fa3e525
commit 3b47122e54
3 changed files with 31 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
scripts/build/llama-cpp-darwin.sh
View File

@@ -6,10 +6,11 @@ IMAGE_NAME="${IMAGE_NAME:-localai/llama-cpp-darwin}"
pushd backend/cpp/llama-cpp
# make llama-cpp-avx && \
# make llama-cpp-avx2 && \
# make llama-cpp-avx512 && \
make llama-cpp-fallback && \
# Single build via ggml CPU_ALL_VARIANTS: one binary plus the per-microarch Apple/arm
# dylibs (apple_m1/m2_m3/m4, armv8.x) that ggml selects at runtime. GGML_METAL stays ON
# and --target ggml also builds ggml-metal (via add_dependencies), so the Metal GPU
# backend is still produced as a loadable libggml-metal.dylib.
make llama-cpp-cpu-all && \
make llama-cpp-grpc && \
make llama-cpp-rpc-server
@@ -19,13 +20,22 @@ mkdir -p build/darwin
mkdir -p backend-images
mkdir -p build/darwin/lib
# cp -rf backend/cpp/llama-cpp/llama-cpp-avx build/darwin/
# cp -rf backend/cpp/llama-cpp/llama-cpp-avx2 build/darwin/
# cp -rf backend/cpp/llama-cpp/llama-cpp-avx512 build/darwin/
cp -rf backend/cpp/llama-cpp/llama-cpp-fallback build/darwin/
cp -rf backend/cpp/llama-cpp/llama-cpp-cpu-all build/darwin/
cp -rf backend/cpp/llama-cpp/llama-cpp-grpc build/darwin/
cp -rf backend/cpp/llama-cpp/llama-cpp-rpc-server build/darwin/
# Distribute the shared ggml/llama dylibs from the CPU_ALL_VARIANTS build. Unlike the old
# fully-static fallback build, these are real dylibs with @rpath install names, so the
# otool loop below (which only copies deps that exist on disk) will not pick them up.
# - the per-microarch libggml-cpu-*.dylib go in the package ROOT, next to the binary,
# because on darwin run.sh execs the binary directly (no bundled ld.so) and ggml
# discovers CPU backends by scanning the executable's own directory.
# - everything else (libggml-base/libggml/libllama/libmtmd/libggml-metal/...) goes in
# lib/, resolved at load time via the DYLD_LIBRARY_PATH=lib that run.sh exports.
SHLIBS=backend/cpp/llama-cpp/ggml-shared-libs
cp -rfv $SHLIBS/libggml-cpu-*.dylib build/darwin/
find $SHLIBS -name '*.dylib' ! -name 'libggml-cpu-*.dylib' -exec cp -rfv {} build/darwin/lib/ \;
# Set default additional libs only for Darwin on M chips (arm64)
if [[ "$(uname -s)" == "Darwin" && "$(uname -m)" == "arm64" ]]; then
ADDITIONAL_LIBS=${ADDITIONAL_LIBS:-$(ls /opt/homebrew/Cellar/protobuf/**/lib/libutf8_validity*.dylib 2>/dev/null)}