From 00288b21cc009d29e61d4e247bb158b1de6394a5 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 25 Jun 2026 21:04:25 +0000 Subject: [PATCH] docs(backends): make OS coverage explicit + require darwin for new backends The backend matrix is the source of truth for which OS a backend ships on, but that was never written down, so backends were landing Linux-only by default even when the engine builds fine on macOS. - .github/backend-matrix.yml: header block documenting the two matrices (include = Linux, includeDarwin = macOS/Apple Silicon) and the policy that new backends target every OS they can build for. - .agents/adding-backends.md: a 'Cover every OS' subsection in step 2 (full darwin wiring: includeDarwin entry, index.yaml metal: + metal- entries, run.sh DYLD branch + inferBackendPathDarwin case for C++ backends, the hw_grpc_proto protobuf/grpc link gotcha, and the path-filter touch) plus a verification-checklist item. - AGENTS.md (CLAUDE.md): Quick Reference pointer so it surfaces every session. Signed-off-by: Ettore Di Giacinto Assisted-by: Claude:opus-4.8 [Claude Code] --- .agents/adding-backends.md | 19 +++++++++++++++++++ .github/backend-matrix.yml | 22 ++++++++++++++++++++++ AGENTS.md | 1 + 3 files changed, 42 insertions(+) diff --git a/.agents/adding-backends.md b/.agents/adding-backends.md index ab965f789..fb98c55f2 100644 --- a/.agents/adding-backends.md +++ b/.agents/adding-backends.md @@ -102,6 +102,24 @@ Multi-arch backends are NOT a single matrix entry with `platforms: 'linux/amd64, Entries whose `dockerfile` is `./backend/Dockerfile.{llama-cpp,ik-llama-cpp,turboquant}` must also set a `builder-base-image` field pointing at a prebuilt base from `quay.io/go-skynet/ci-cache:base-grpc-*` (CI builds these via `.github/workflows/base-images.yml`). The mapping is by `(build-type, platforms)` — see existing entries for the pattern. CI uses these prebuilt bases to skip the gRPC compile (~25–35 min cold). Local `make backends/` ignores `builder-base-image` and uses the from-source path inside the Dockerfile, so you don't need quay access for local builds. +### Cover every OS the project supports (Linux **and** Darwin) + +`.github/backend-matrix.yml` has two matrices, and they are the source of truth for which OS a backend ships on: + +- `include:` — the **Linux** matrix (x86_64 + arm64; CPU and CUDA / ROCm / SYCL / Vulkan). +- `includeDarwin:` — the **macOS / Apple Silicon** matrix (arm64; Metal where the engine supports it, otherwise a native arm64 CPU build). + +**A new backend must target every OS it can build for — do not ship Linux-only by default.** A backend that appears only under `include:` is silently unavailable on macOS even when its code would run there. Most C/C++/GGML engines build on Darwin out of the box (ggml defaults `GGML_METAL=ON` on Apple, so a plain build is Metal-enabled), and many Python backends do too (CPU / MPS wheels). If a backend genuinely cannot support an OS (e.g. CUDA-only, no CPU variant), state that in the PR description instead of omitting it silently. + +Wiring a backend into `includeDarwin:` is more than the matrix entry: + +1. **`includeDarwin:` entry** — `tag-suffix: "-metal-darwin-arm64-"`, `build-type: "metal"`, `lang: "go"` for go+ggml backends; omit `build-type` for the bespoke C++ ones (llama-cpp / ds4 / privacy-filter). Match an existing entry of the same shape. +2. **`backend/index.yaml`** — add `metal:` to the backend's `capabilities` map (main and `-development`) and concrete `metal-` / `metal--development` image entries pointing at the `-metal-darwin-arm64-` images. +3. **C/C++ backends only** — add an `inferBackendPathDarwin` case in `scripts/changed-backends.js` returning `backend/cpp//` (the generic fallthrough assumes `backend//`, which is wrong for a C++ source tree driven with `lang: go`), and give `run.sh` a Darwin branch that exports `DYLD_LIBRARY_PATH` instead of `LD_LIBRARY_PATH`. If the build is bespoke (single `grpc-server` + dylib bundling), model it on `scripts/build/ds4-darwin.sh` and add a `backends/-darwin` make target plus a gated step in `.github/workflows/backend_build_darwin.yml`. +4. **C++ proto gotcha** — if the backend compiles the generated gRPC/protobuf in a separate CMake target (e.g. `hw_grpc_proto`), that target must link `protobuf::libprotobuf` + `gRPC::grpc++` so the Homebrew include dirs propagate; otherwise macOS fails with `google/protobuf/runtime_version.h not found` (Linux hides this because apt headers sit in `/usr/include`). + +The CI path filter only builds a backend on a PR when a file under its directory changes, so a darwin-only YAML edit builds nothing — touch a file under `backend///` (a one-line comment is enough) in the same PR. + ## 3. Add Backend Metadata to `backend/index.yaml` **Step 3a: Add Meta Definition** @@ -225,6 +243,7 @@ After adding a new backend, verify: - [ ] Backend directory structure is complete with all necessary files - [ ] Build configurations added to `.github/backend-matrix.yml` for all desired platforms (per-arch entries with `platform-tag` for multi-arch; `builder-base-image` for llama-cpp / ik-llama-cpp / turboquant) +- [ ] **OS coverage considered**: added to `includeDarwin:` (macOS/Apple Silicon) if the backend can build there — with the `backend/index.yaml` `metal:` capability + `metal-` image entries, a `run.sh` Darwin/DYLD branch and `inferBackendPathDarwin` case for C++ backends — or the PR explains why an OS is unsupported. Do not ship Linux-only by default. - [ ] Meta definition added to `backend/index.yaml` in the `## metas` section - [ ] Image entries added to `backend/index.yaml` for all build variants (latest + development) - [ ] Tag suffixes match between workflow file and index.yaml diff --git a/.github/backend-matrix.yml b/.github/backend-matrix.yml index 4d2b85977..b66f1bbf3 100644 --- a/.github/backend-matrix.yml +++ b/.github/backend-matrix.yml @@ -2,6 +2,28 @@ # Matrix data for backend container image builds. # Consumed by scripts/changed-backends.js for both backend.yml and backend_pr.yml. # This file is NOT a workflow — it has no top-level 'on:' or 'jobs:'. +# +# OS / platform coverage — READ THIS WHEN ADDING A BACKEND +# -------------------------------------------------------- +# This file is the source of truth for which OS each backend is built and +# published for. A backend ships ONLY for the matrices it appears in: +# - Linux -> the `include:` matrix below (x86_64 + arm64; CPU and +# CUDA / ROCm / SYCL / Vulkan variants). +# - macOS -> the `includeDarwin:` matrix (Apple Silicon / arm64; Metal where +# the engine supports it, otherwise a native arm64 CPU build). +# +# New backends must target EVERY OS they can build for, not just Linux. A backend +# listed only under `include:` is silently unavailable on macOS even when its code +# would run there. Most C/C++/GGML engines build on Darwin (ggml defaults +# GGML_METAL=ON on Apple, so a plain build is Metal-enabled), and many Python +# backends do too (CPU / MPS). If a backend genuinely cannot support an OS, say so +# in its PR description rather than silently omitting it. +# +# Adding a backend to `includeDarwin:` is more than one line — see the darwin +# checklist in .agents/adding-backends.md (includeDarwin entry, the index.yaml +# `metal:` capability + `metal-` image entries, a `run.sh` Darwin/DYLD +# branch for C/C++ backends, and the inferBackendPathDarwin case in +# scripts/changed-backends.js so the path filter actually builds it). # Linux matrix (consumed by backend-jobs). include: diff --git a/AGENTS.md b/AGENTS.md index 9f397e613..1095ef531 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -43,4 +43,5 @@ LocalAI follows the Linux kernel project's [guidelines for AI coding assistants] - **New API endpoints**: LocalAI advertises its capability surface in several independent places — swagger `@Tags`, `/api/instructions` registry, auth `RouteFeatureRegistry`, React UI `capabilities.js`, docs. Read [.agents/api-endpoints-and-auth.md](.agents/api-endpoints-and-auth.md) and follow its checklist — missing any surface means clients, admins, and the UI won't know the endpoint exists. - **Admin endpoints → MCP tool**: every admin endpoint that an admin would manage conversationally (install/list/edit/toggle/upgrade) MUST also be exposed as an MCP tool in `pkg/mcp/localaitools/`. The LocalAI Assistant chat modality and the standalone `local-ai mcp-server` consume that package; drift between REST and MCP is a real risk. Read [.agents/localai-assistant-mcp.md](.agents/localai-assistant-mcp.md) — the `TestToolHTTPRouteMappingComplete` test fails until you wire the new tool and update the route map. - **Build**: Inspect `Makefile` and `.github/workflows/` — ask the user before running long builds +- **Backend OS coverage**: a new backend must target every OS it can build for, not just Linux. `.github/backend-matrix.yml` has two matrices — `include:` (Linux) and `includeDarwin:` (macOS / Apple Silicon). Most C/C++/GGML and many Python backends build on Darwin too — wire the `includeDarwin` entry + `backend/index.yaml` `metal:` entries, or say in the PR why an OS is unsupported. See the darwin checklist in [.agents/adding-backends.md](.agents/adding-backends.md). - **UI**: The active UI is the React app in `core/http/react-ui/`. The older Alpine.js/HTML UI in `core/http/static/` is pending deprecation — all new UI work goes in the React UI