chore(deps): bump torch

Bumps the pip group with 1 update in the /backend/python/vllm directory: torch.


Updates `torch` from 2.9.1+cpu to 2.12.0+cpu

---
updated-dependencies:
- dependency-name: torch
  dependency-version: 2.12.0+cpu
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

.agents/adding-backends.md

@@ -102,24 +102,6 @@ Multi-arch backends are NOT a single matrix entry with `platforms: 'linux/amd64,
 
 Entries whose `dockerfile` is `./backend/Dockerfile.{llama-cpp,ik-llama-cpp,turboquant}` must also set a `builder-base-image` field pointing at a prebuilt base from `quay.io/go-skynet/ci-cache:base-grpc-*` (CI builds these via `.github/workflows/base-images.yml`). The mapping is by `(build-type, platforms)` — see existing entries for the pattern. CI uses these prebuilt bases to skip the gRPC compile (~25–35 min cold). Local `make backends/<name>` ignores `builder-base-image` and uses the from-source path inside the Dockerfile, so you don't need quay access for local builds.
 
-### Cover every OS the project supports (Linux **and** Darwin)
-
-`.github/backend-matrix.yml` has two matrices, and they are the source of truth for which OS a backend ships on:
-
-- `include:` — the **Linux** matrix (x86_64 + arm64; CPU and CUDA / ROCm / SYCL / Vulkan).
-- `includeDarwin:` — the **macOS / Apple Silicon** matrix (arm64; Metal where the engine supports it, otherwise a native arm64 CPU build).
-
-**A new backend must target every OS it can build for — do not ship Linux-only by default.** A backend that appears only under `include:` is silently unavailable on macOS even when its code would run there. Most C/C++/GGML engines build on Darwin out of the box (ggml defaults `GGML_METAL=ON` on Apple, so a plain build is Metal-enabled), and many Python backends do too (CPU / MPS wheels). If a backend genuinely cannot support an OS (e.g. CUDA-only, no CPU variant), state that in the PR description instead of omitting it silently.
-
-Wiring a backend into `includeDarwin:` is more than the matrix entry:
-
-1. **`includeDarwin:` entry** — `tag-suffix: "-metal-darwin-arm64-<backend>"`, `build-type: "metal"`, `lang: "go"` for go+ggml backends; omit `build-type` for the bespoke C++ ones (llama-cpp / ds4 / privacy-filter). Match an existing entry of the same shape.
-2. **`backend/index.yaml`** — add `metal:` to the backend's `capabilities` map (main and `-development`) and concrete `metal-<backend>` / `metal-<backend>-development` image entries pointing at the `-metal-darwin-arm64-<backend>` images.
-3. **C/C++ backends only** — add an `inferBackendPathDarwin` case in `scripts/changed-backends.js` returning `backend/cpp/<backend>/` (the generic fallthrough assumes `backend/<lang>/`, which is wrong for a C++ source tree driven with `lang: go`), and give `run.sh` a Darwin branch that exports `DYLD_LIBRARY_PATH` instead of `LD_LIBRARY_PATH`. If the build is bespoke (single `grpc-server` + dylib bundling), model it on `scripts/build/ds4-darwin.sh` and add a `backends/<backend>-darwin` make target plus a gated step in `.github/workflows/backend_build_darwin.yml`.
-4. **C++ proto gotcha** — if the backend compiles the generated gRPC/protobuf in a separate CMake target (e.g. `hw_grpc_proto`), that target must link `protobuf::libprotobuf` + `gRPC::grpc++` so the Homebrew include dirs propagate; otherwise macOS fails with `google/protobuf/runtime_version.h not found` (Linux hides this because apt headers sit in `/usr/include`).
-
-The CI path filter only builds a backend on a PR when a file under its directory changes, so a darwin-only YAML edit builds nothing — touch a file under `backend/<lang>/<backend>/` (a one-line comment is enough) in the same PR.
-
 ## 3. Add Backend Metadata to `backend/index.yaml`
 
 **Step 3a: Add Meta Definition**
@@ -243,7 +225,6 @@ After adding a new backend, verify:
 
 - [ ] Backend directory structure is complete with all necessary files
 - [ ] Build configurations added to `.github/backend-matrix.yml` for all desired platforms (per-arch entries with `platform-tag` for multi-arch; `builder-base-image` for llama-cpp / ik-llama-cpp / turboquant)
-- [ ] **OS coverage considered**: added to `includeDarwin:` (macOS/Apple Silicon) if the backend can build there — with the `backend/index.yaml` `metal:` capability + `metal-<backend>` image entries, a `run.sh` Darwin/DYLD branch and `inferBackendPathDarwin` case for C++ backends — or the PR explains why an OS is unsupported. Do not ship Linux-only by default.
 - [ ] Meta definition added to `backend/index.yaml` in the `## metas` section
 - [ ] Image entries added to `backend/index.yaml` for all build variants (latest + development)
 - [ ] Tag suffixes match between workflow file and index.yaml

.github/backend-matrix.yml

@@ -2,28 +2,6 @@
 # Matrix data for backend container image builds.
 # Consumed by scripts/changed-backends.js for both backend.yml and backend_pr.yml.
 # This file is NOT a workflow — it has no top-level 'on:' or 'jobs:'.
-#
-# OS / platform coverage — READ THIS WHEN ADDING A BACKEND
-# --------------------------------------------------------
-# This file is the source of truth for which OS each backend is built and
-# published for. A backend ships ONLY for the matrices it appears in:
-#   - Linux  -> the `include:` matrix below (x86_64 + arm64; CPU and
-#               CUDA / ROCm / SYCL / Vulkan variants).
-#   - macOS  -> the `includeDarwin:` matrix (Apple Silicon / arm64; Metal where
-#               the engine supports it, otherwise a native arm64 CPU build).
-#
-# New backends must target EVERY OS they can build for, not just Linux. A backend
-# listed only under `include:` is silently unavailable on macOS even when its code
-# would run there. Most C/C++/GGML engines build on Darwin (ggml defaults
-# GGML_METAL=ON on Apple, so a plain build is Metal-enabled), and many Python
-# backends do too (CPU / MPS). If a backend genuinely cannot support an OS, say so
-# in its PR description rather than silently omitting it.
-#
-# Adding a backend to `includeDarwin:` is more than one line — see the darwin
-# checklist in .agents/adding-backends.md (includeDarwin entry, the index.yaml
-# `metal:` capability + `metal-<backend>` image entries, a `run.sh` Darwin/DYLD
-# branch for C/C++ backends, and the inferBackendPathDarwin case in
-# scripts/changed-backends.js so the path filter actually builds it).
 
 # Linux matrix (consumed by backend-jobs).
 include:
@@ -4944,37 +4922,6 @@ includeDarwin:
     tag-suffix: "-metal-darwin-arm64-vibevoice-cpp"
     build-type: "metal"
     lang: "go"
-  # Vision/utility C++/ggml backends (go+cgo). Their Makefiles already carry a
-  # Darwin/Metal path (GGML_METAL=ON when build-type=metal); this just builds and
-  # publishes the metal image so Apple Silicon can install them.
-  - backend: "depth-anything-cpp"
-    tag-suffix: "-metal-darwin-arm64-depth-anything-cpp"
-    build-type: "metal"
-    lang: "go"
-  - backend: "locate-anything-cpp"
-    tag-suffix: "-metal-darwin-arm64-locate-anything-cpp"
-    build-type: "metal"
-    lang: "go"
-  - backend: "rfdetr-cpp"
-    tag-suffix: "-metal-darwin-arm64-rfdetr-cpp"
-    build-type: "metal"
-    lang: "go"
-  - backend: "sam3-cpp"
-    tag-suffix: "-metal-darwin-arm64-sam3-cpp"
-    build-type: "metal"
-    lang: "go"
-  # privacy-filter (PII/NER) is a C++/ggml backend built by a bespoke darwin
-  # script (make backends/privacy-filter-darwin); ggml defaults Metal ON on Apple
-  # so the build is Metal-enabled. lang=go drives runner/toolchain selection only.
-  - backend: "privacy-filter"
-    tag-suffix: "-metal-darwin-arm64-privacy-filter"
-    lang: "go"
-  # LocalVQE has no Metal path; on Apple Silicon it builds CPU-only (GGML_METAL
-  # OFF) but is still a native arm64 image. Uses the darwin/metal build profile.
-  - backend: "localvqe"
-    tag-suffix: "-metal-darwin-arm64-localvqe"
-    build-type: "metal"
-    lang: "go"
   - backend: "voxtral"
     tag-suffix: "-metal-darwin-arm64-voxtral"
     build-type: "metal"

.github/workflows/backend_build_darwin.yml

@@ -99,7 +99,6 @@ jobs:
             /opt/homebrew/Cellar/xxhash
             /opt/homebrew/Cellar/zstd
             /opt/homebrew/Cellar/nlohmann-json
-            /opt/homebrew/Cellar/opus
           key: brew-${{ runner.os }}-${{ runner.arch }}-v1-${{ hashFiles('.github/workflows/backend_build_darwin.yml') }}
 
       - name: Dependencies
@@ -114,12 +113,7 @@ jobs:
           # nlohmann-json is header-only and required by the ds4 backend
           # (dsml_renderer.cpp includes <nlohmann/json.hpp>); on Linux it comes
           # from the apt-installed nlohmann-json3-dev in the build image.
-          # opus + pkg-config are required by the opus go backend: its
-          # Makefile/package.sh call `pkg-config --cflags/--libs opus` to build
-          # libopusshim.dylib and to locate libopus.dylib for bundling. brew's
-          # pkg-config defaults its search path to the Homebrew prefix so the
-          # opus.pc is found.
-          brew install protobuf grpc make protoc-gen-go protoc-gen-go-grpc libomp llvm ccache blake3 fmt hiredis xxhash zstd nlohmann-json opus pkg-config
+          brew install protobuf grpc make protoc-gen-go protoc-gen-go-grpc libomp llvm ccache blake3 fmt hiredis xxhash zstd nlohmann-json
           # Force-reinstall ccache so brew re-validates its full runtime-dep
           # closure on every run. This is the durable fix: when the upstream
           # ccache formula gains a new transitive dep (as it has multiple times
@@ -138,7 +132,7 @@ jobs:
           # and decides "already installed" without re-linking, so on a cache-
           # hit run the formulas aren't on PATH. Force-link them; --overwrite
           # tolerates pre-existing symlinks from earlier installs.
-          brew link --overwrite protobuf grpc make protoc-gen-go protoc-gen-go-grpc libomp llvm ccache blake3 fmt hiredis xxhash zstd nlohmann-json opus pkg-config 2>/dev/null || true
+          brew link --overwrite protobuf grpc make protoc-gen-go protoc-gen-go-grpc libomp llvm ccache blake3 fmt hiredis xxhash zstd nlohmann-json 2>/dev/null || true
 
       - name: Save Homebrew cache
         if: github.event_name != 'pull_request' && steps.brew-cache.outputs.cache-hit != 'true'
@@ -159,7 +153,6 @@ jobs:
             /opt/homebrew/Cellar/xxhash
             /opt/homebrew/Cellar/zstd
             /opt/homebrew/Cellar/nlohmann-json
-            /opt/homebrew/Cellar/opus
           key: brew-${{ runner.os }}-${{ runner.arch }}-v1-${{ hashFiles('.github/workflows/backend_build_darwin.yml') }}
 
       # ---- ccache for llama.cpp CMake builds ----
@@ -235,17 +228,8 @@ jobs:
         run: |
           make backends/ds4-darwin
 
-      # privacy-filter is a C++/ggml backend like ds4 - a single grpc-server with
-      # otool dylib bundling - so it gets its own bespoke darwin script rather than
-      # the generic build-darwin-go-backend path.
-      - name: Build privacy-filter backend (Darwin Metal)
-        if: inputs.backend == 'privacy-filter'
-        run: |
-          make protogen-go
-          make backends/privacy-filter-darwin
-
       - name: Build ${{ inputs.backend }}-darwin
-        if: inputs.backend != 'llama-cpp' && inputs.backend != 'ds4' && inputs.backend != 'privacy-filter'
+        if: inputs.backend != 'llama-cpp' && inputs.backend != 'ds4'
         run: |
           make protogen-go
           BACKEND=${{ inputs.backend }} BUILD_TYPE=${{ inputs.build-type }} USE_PIP=${{ inputs.use-pip }} make build-darwin-${{ inputs.lang }}-backend

.github/workflows/release.yaml

@@ -24,11 +24,6 @@ jobs:
           args: release --clean
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          MACOS_SIGN_P12: ${{ secrets.MACOS_CERTIFICATE }}
-          MACOS_SIGN_PASSWORD: ${{ secrets.MACOS_CERTIFICATE_PWD }}
-          MACOS_NOTARY_KEY: ${{ secrets.MACOS_NOTARY_KEY }}
-          MACOS_NOTARY_KEY_ID: ${{ secrets.MACOS_NOTARY_KEY_ID }}
-          MACOS_NOTARY_ISSUER_ID: ${{ secrets.MACOS_NOTARY_ISSUER_ID }}
   launcher-build-darwin:
     runs-on: macos-latest
     steps:
@@ -40,19 +35,9 @@ jobs:
         uses: actions/setup-go@v5
         with:
           go-version: 1.23
-      - name: Import signing certificate
-        env:
-          MACOS_CERTIFICATE: ${{ secrets.MACOS_CERTIFICATE }}
-          MACOS_CERTIFICATE_PWD: ${{ secrets.MACOS_CERTIFICATE_PWD }}
-          MACOS_CI_KEYCHAIN_PWD: ${{ secrets.MACOS_CI_KEYCHAIN_PWD }}
-        run: bash contrib/macos/sign-and-notarize.sh import-cert
-      - name: Build, sign and notarize the DMG
-        env:
-          MACOS_SIGN_IDENTITY: ${{ secrets.MACOS_SIGN_IDENTITY }}
-          MACOS_NOTARY_KEY: ${{ secrets.MACOS_NOTARY_KEY }}
-          MACOS_NOTARY_KEY_ID: ${{ secrets.MACOS_NOTARY_KEY_ID }}
-          MACOS_NOTARY_ISSUER_ID: ${{ secrets.MACOS_NOTARY_ISSUER_ID }}
-        run: make release-launcher-darwin
+      - name: Build launcher for macOS ARM64
+        run: |
+          make build-launcher-darwin
       - name: Upload DMG to Release
         uses: softprops/action-gh-release@v3
         with:

.gitignore

@@ -94,6 +94,3 @@ core/http/react-ui/test-results/
 
 # SDD / brainstorm scratch (agent-driven development)
 .superpowers/
-
-# Local Apple signing material (never commit)
-.certs/

.goreleaser.yaml

@@ -9,8 +9,7 @@ source:
   enabled: true
   name_template: '{{ .ProjectName }}-{{ .Tag }}-source'
 builds:
-  - id: local-ai
-    main: ./cmd/local-ai
+  - main: ./cmd/local-ai
     env:
       - CGO_ENABLED=0
     ldflags:
@@ -36,19 +35,3 @@ snapshot:
   version_template: "{{ .Tag }}-next"
 changelog:
   use: github-native
-# Sign + notarize the macOS server binary via the quill backend (runs on Linux,
-# no macOS runner needed). Disabled automatically when MACOS_SIGN_P12 is unset
-# (forks / PRs), so those builds stay unsigned and green.
-notarize:
-  macos:
-    - enabled: '{{ isEnvSet "MACOS_SIGN_P12" }}'
-      ids:
-        - local-ai
-      sign:
-        certificate: "{{.Env.MACOS_SIGN_P12}}"
-        password: "{{.Env.MACOS_SIGN_PASSWORD}}"
-      notarize:
-        issuer_id: "{{.Env.MACOS_NOTARY_ISSUER_ID}}"
-        key_id: "{{.Env.MACOS_NOTARY_KEY_ID}}"
-        key: "{{.Env.MACOS_NOTARY_KEY}}"
-        wait: true

AGENTS.md

@@ -43,5 +43,4 @@ LocalAI follows the Linux kernel project's [guidelines for AI coding assistants]
 - **New API endpoints**: LocalAI advertises its capability surface in several independent places — swagger `@Tags`, `/api/instructions` registry, auth `RouteFeatureRegistry`, React UI `capabilities.js`, docs. Read [.agents/api-endpoints-and-auth.md](.agents/api-endpoints-and-auth.md) and follow its checklist — missing any surface means clients, admins, and the UI won't know the endpoint exists.
 - **Admin endpoints → MCP tool**: every admin endpoint that an admin would manage conversationally (install/list/edit/toggle/upgrade) MUST also be exposed as an MCP tool in `pkg/mcp/localaitools/`. The LocalAI Assistant chat modality and the standalone `local-ai mcp-server` consume that package; drift between REST and MCP is a real risk. Read [.agents/localai-assistant-mcp.md](.agents/localai-assistant-mcp.md) — the `TestToolHTTPRouteMappingComplete` test fails until you wire the new tool and update the route map.
 - **Build**: Inspect `Makefile` and `.github/workflows/` — ask the user before running long builds
-- **Backend OS coverage**: a new backend must target every OS it can build for, not just Linux. `.github/backend-matrix.yml` has two matrices — `include:` (Linux) and `includeDarwin:` (macOS / Apple Silicon). Most C/C++/GGML and many Python backends build on Darwin too — wire the `includeDarwin` entry + `backend/index.yaml` `metal:` entries, or say in the PR why an OS is unsupported. See the darwin checklist in [.agents/adding-backends.md](.agents/adding-backends.md).
 - **UI**: The active UI is the React app in `core/http/react-ui/`. The older Alpine.js/HTML UI in `core/http/static/` is pending deprecation — all new UI work goes in the React UI

Makefile

@@ -1,5 +1,5 @@
 # Disable parallel execution for backend builds
-.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/outetts backends/piper backends/stablediffusion-ggml backends/whisper backends/crispasr backends/parakeet-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter backends/privacy-filter-darwin
+.NOTPARALLEL: backends/diffusers backends/llama-cpp backends/turboquant backends/outetts backends/piper backends/stablediffusion-ggml backends/whisper backends/crispasr backends/parakeet-cpp backends/faster-whisper backends/silero-vad backends/local-store backends/huggingface backends/rfdetr backends/rfdetr-cpp backends/insightface backends/speaker-recognition backends/kitten-tts backends/kokoro backends/chatterbox backends/llama-cpp-darwin backends/neutts build-darwin-python-backend build-darwin-go-backend backends/mlx backends/diffuser-darwin backends/mlx-vlm backends/mlx-audio backends/mlx-distributed backends/stablediffusion-ggml-darwin backends/vllm backends/vllm-omni backends/sglang backends/moonshine backends/pocket-tts backends/qwen-tts backends/faster-qwen3-tts backends/qwen-asr backends/nemo backends/voxcpm backends/whisperx backends/ace-step backends/acestep-cpp backends/fish-speech backends/voxtral backends/opus backends/trl backends/llama-cpp-quantization backends/kokoros backends/sam3-cpp backends/qwen3-tts-cpp backends/omnivoice-cpp backends/vibevoice-cpp backends/localvqe backends/tinygrad backends/sherpa-onnx backends/ds4 backends/ds4-darwin backends/liquid-audio backends/supertonic backends/depth-anything-cpp backends/privacy-filter
 
 GOCMD=go
 GOTEST=$(GOCMD) test
@@ -1129,10 +1129,6 @@ backends/ds4-darwin: build
 	bash ./scripts/build/ds4-darwin.sh
 	./local-ai backends install "ocifile://$(abspath ./backend-images/ds4.tar)"
 
-backends/privacy-filter-darwin: build
-	bash ./scripts/build/privacy-filter-darwin.sh
-	./local-ai backends install "ocifile://$(abspath ./backend-images/privacy-filter.tar)"
-
 build-darwin-python-backend: build
 	bash ./scripts/build/python-darwin.sh
 
@@ -1453,32 +1449,13 @@ docs: docs/static/gallery.html
 ########################################################
 
 ## fyne cross-platform build
-# Build LocalAI.app from the launcher via fyne (metadata read from cmd/launcher/FyneApp.toml).
-# Signing happens via contrib/macos/sign-and-notarize.sh, which is a no-op when the signing
-# secrets are unset, so unsigned local/fork builds keep working.
-build-launcher-darwin:
-	rm -rf dist/LocalAI.app cmd/launcher/LocalAI.app
-	mkdir -p dist
-	cd cmd/launcher && go run fyne.io/tools/cmd/fyne@latest package -os darwin -icon ../../core/http/static/logo.png --executable $(LAUNCHER_BINARY_NAME)
-	mv cmd/launcher/LocalAI.app dist/LocalAI.app
-	bash contrib/macos/sign-and-notarize.sh sign dist/LocalAI.app
-
-# Wrap the (signed) app into a drag-to-Applications DMG via hdiutil, then sign the DMG.
-dmg-launcher-darwin: build-launcher-darwin
-	rm -rf dist/dmg dist/LocalAI.dmg
-	mkdir -p dist/dmg
-	cp -R dist/LocalAI.app dist/dmg/LocalAI.app
-	ln -s /Applications dist/dmg/Applications
-	hdiutil create -volname "LocalAI" -srcfolder dist/dmg -ov -format UDZO dist/LocalAI.dmg
-	bash contrib/macos/sign-and-notarize.sh sign dist/LocalAI.dmg
-
-# Submit the DMG to Apple notarization and staple the ticket (no-op without notary secrets).
-notarize-launcher-darwin: dmg-launcher-darwin
-	bash contrib/macos/sign-and-notarize.sh notarize dist/LocalAI.dmg
-
-# Single entrypoint for CI: build -> sign app -> dmg -> sign dmg -> notarize -> staple.
-release-launcher-darwin: notarize-launcher-darwin
-	@echo "dist/LocalAI.dmg is ready"
+build-launcher-darwin: build-launcher
+	go run github.com/tiagomelo/macos-dmg-creator/cmd/createdmg@latest \
+	--appName "LocalAI" \
+	--appBinaryPath "$(LAUNCHER_BINARY_NAME)" \
+	--bundleIdentifier "com.localai.launcher" \
+	--iconPath "core/http/static/logo.png" \
+	--outputDir "dist/"
 
 build-launcher-linux:
-	cd cmd/launcher && go run fyne.io/tools/cmd/fyne@latest package -os linux -icon ../../core/http/static/logo.png --executable $(LAUNCHER_BINARY_NAME)-linux && mv LocalAI.tar.xz ../../$(LAUNCHER_BINARY_NAME)-linux.tar.xz
+	cd cmd/launcher && go run fyne.io/tools/cmd/fyne@latest package -os linux -icon ../../core/http/static/logo.png --executable $(LAUNCHER_BINARY_NAME)-linux && mv launcher.tar.xz ../../$(LAUNCHER_BINARY_NAME)-linux.tar.xz

backend/cpp/ik-llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-IK_LLAMA_VERSION?=b84902d2ad27c34f989f23947200c4b91b1568fd
+IK_LLAMA_VERSION?=d5507e33ae7ee2b7b41475f08044d3bde3b839ee
 LLAMA_REPO?=https://github.com/ikawrakow/ik_llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/ik-llama-cpp/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -13,28 +13,28 @@ grep -e "flags" /proc/cpuinfo | head -1
 # ik_llama.cpp requires AVX2 — default to avx2 binary
 BINARY=ik-llama-cpp-avx2
 
-if [ -e "$CURDIR"/ik-llama-cpp-fallback ] && ! grep -q -e "\savx2\s" /proc/cpuinfo ; then
+if [ -e $CURDIR/ik-llama-cpp-fallback ] && ! grep -q -e "\savx2\s" /proc/cpuinfo ; then
 	echo "CPU:    AVX2   NOT found, using fallback"
 	BINARY=ik-llama-cpp-fallback
 fi
 
 # Extend ld library path with the dir where this script is located/lib
 if [ "$(uname)" == "Darwin" ]; then
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
-	#export DYLD_FALLBACK_LIBRARY_PATH="$CURDIR"/lib:$DYLD_FALLBACK_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
+	#export DYLD_FALLBACK_LIBRARY_PATH=$CURDIR/lib:$DYLD_FALLBACK_LIBRARY_PATH
 else
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using binary: $BINARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/$BINARY "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/$BINARY "$@"
 fi
 
 echo "Using binary: $BINARY"
-exec "$CURDIR"/$BINARY "$@"
+exec $CURDIR/$BINARY "$@"
 
 # We should never reach this point, however just in case we do, run fallback
-exec "$CURDIR"/ik-llama-cpp-fallback "$@"
+exec $CURDIR/ik-llama-cpp-fallback "$@"

backend/cpp/llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-LLAMA_VERSION?=9d5d882d8cd0f0a9283d87ed5e6fe3ee0d925fb1
+LLAMA_VERSION?=8be759e6f70d629638a7eb70db3824cbdcea370b
 LLAMA_REPO?=https://github.com/ggerganov/llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/llama-cpp/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -16,37 +16,37 @@ BINARY=llama-cpp-fallback
 # CPU_ALL_VARIANTS: ggml's backend registry dlopens the best libggml-cpu-*.so for this
 # host, so no shell-side AVX probing. GPU images (cublas/sycl/vulkan/hipblas) ship only
 # llama-cpp-fallback (the accelerator does the compute), so fall back to it when absent.
-if [ -e "$CURDIR"/llama-cpp-cpu-all ]; then
+if [ -e $CURDIR/llama-cpp-cpu-all ]; then
 	BINARY=llama-cpp-cpu-all
 fi
 
 if [ -n "$LLAMACPP_GRPC_SERVERS" ]; then
-	if [ -e "$CURDIR"/llama-cpp-grpc ]; then
+	if [ -e $CURDIR/llama-cpp-grpc ]; then
 		BINARY=llama-cpp-grpc
 	fi
 fi
  
 # Extend ld library path with the dir where this script is located/lib
 if [ "$(uname)" == "Darwin" ]; then
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
-	#export DYLD_FALLBACK_LIBRARY_PATH="$CURDIR"/lib:$DYLD_FALLBACK_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
+	#export DYLD_FALLBACK_LIBRARY_PATH=$CURDIR/lib:$DYLD_FALLBACK_LIBRARY_PATH
 else
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 	# Tell rocBLAS where to find TensileLibrary data (GPU kernel tuning files)
 	if [ -d "$CURDIR/lib/rocblas/library" ]; then
-		export ROCBLAS_TENSILE_LIBPATH="$CURDIR"/lib/rocblas/library
+		export ROCBLAS_TENSILE_LIBPATH=$CURDIR/lib/rocblas/library
 	fi
 fi
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using binary: $BINARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/$BINARY "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/$BINARY "$@"
 fi
 
 echo "Using binary: $BINARY"
-exec "$CURDIR"/$BINARY "$@"
+exec $CURDIR/$BINARY "$@"
 
 # We should never reach this point, however just in case we do, run fallback
-exec "$CURDIR"/llama-cpp-fallback "$@"
+exec $CURDIR/llama-cpp-fallback "$@"

backend/cpp/privacy-filter/CMakeLists.txt

@@ -51,14 +51,6 @@ add_library(hw_grpc_proto STATIC
     ${HW_GRPC_SRCS} ${HW_GRPC_HDRS}
     ${HW_PROTO_SRCS} ${HW_PROTO_HDRS})
 target_include_directories(hw_grpc_proto PUBLIC ${CMAKE_CURRENT_BINARY_DIR})
-# The generated proto/grpc sources include protobuf and grpc++ headers, so this
-# library must see their include dirs. Linking the imported targets propagates
-# them. On Linux the apt headers live in /usr/include (default search path) so
-# this was a no-op; on macOS the Homebrew headers are under /opt/homebrew and
-# would otherwise be missed (runtime_version.h not found).
-target_link_libraries(hw_grpc_proto PUBLIC
-    protobuf::libprotobuf
-    gRPC::grpc++)
 
 # Build only the pf static lib (+ ggml) from the engine tree — no CLI/bench/tests.
 # PF_VULKAN is honored when passed on the cmake command line (it lands in the

backend/cpp/privacy-filter/run.sh

@@ -2,13 +2,7 @@
 # Entry point for the privacy-filter backend image / BACKEND_BINARY mode.
 set -e
 CURDIR=$(dirname "$(realpath "$0")")
-# macOS has no bundled ld.so; the darwin package ships only dylibs under lib/,
-# resolved via DYLD_LIBRARY_PATH (the ld.so branch below is skipped there).
-if [ "$(uname)" = "Darwin" ]; then
-    export DYLD_LIBRARY_PATH="$CURDIR/lib:$DYLD_LIBRARY_PATH"
-else
-    export LD_LIBRARY_PATH="$CURDIR/lib:$LD_LIBRARY_PATH"
-fi
+export LD_LIBRARY_PATH="$CURDIR/lib:$LD_LIBRARY_PATH"
 if [ -f "$CURDIR/lib/ld.so" ]; then
     exec "$CURDIR/lib/ld.so" "$CURDIR/grpc-server" "$@"
 fi

backend/cpp/turboquant/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -15,36 +15,36 @@ BINARY=turboquant-fallback
 # x86/arm64 ship a single turboquant-cpu-all built with ggml CPU_ALL_VARIANTS: ggml's
 # backend registry dlopens the best libggml-cpu-*.so for this host, so no shell-side
 # probing. ROCm ships only turboquant-fallback, so fall back to it when cpu-all is absent.
-if [ -e "$CURDIR"/turboquant-cpu-all ]; then
+if [ -e $CURDIR/turboquant-cpu-all ]; then
 	BINARY=turboquant-cpu-all
 fi
 
 if [ -n "$LLAMACPP_GRPC_SERVERS" ]; then
-	if [ -e "$CURDIR"/turboquant-grpc ]; then
+	if [ -e $CURDIR/turboquant-grpc ]; then
 		BINARY=turboquant-grpc
 	fi
 fi
 
 # Extend ld library path with the dir where this script is located/lib
 if [ "$(uname)" == "Darwin" ]; then
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 	# Tell rocBLAS where to find TensileLibrary data (GPU kernel tuning files)
 	if [ -d "$CURDIR/lib/rocblas/library" ]; then
-		export ROCBLAS_TENSILE_LIBPATH="$CURDIR"/lib/rocblas/library
+		export ROCBLAS_TENSILE_LIBPATH=$CURDIR/lib/rocblas/library
 	fi
 fi
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using binary: $BINARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/$BINARY "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/$BINARY "$@"
 fi
 
 echo "Using binary: $BINARY"
-exec "$CURDIR"/$BINARY "$@"
+exec $CURDIR/$BINARY "$@"
 
 # We should never reach this point, however just in case we do, run fallback
-exec "$CURDIR"/turboquant-fallback "$@"
+exec $CURDIR/turboquant-fallback "$@"

backend/go/acestep-cpp/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -21,20 +21,20 @@ if [ "$(uname)" = "Darwin" ]; then
 	if [ ! -e "$LIBRARY" ]; then
 		LIBRARY="$CURDIR/libgoacestepcpp-fallback.so"
 	fi
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/libgoacestepcpp-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/libgoacestepcpp-avx.so ]; then
+		if [ -e $CURDIR/libgoacestepcpp-avx.so ]; then
 			LIBRARY="$CURDIR/libgoacestepcpp-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/libgoacestepcpp-avx2.so ]; then
+		if [ -e $CURDIR/libgoacestepcpp-avx2.so ]; then
 			LIBRARY="$CURDIR/libgoacestepcpp-avx2.so"
 		fi
 	fi
@@ -42,22 +42,22 @@ else
 	# Check avx 512
 	if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX512F found OK"
-		if [ -e "$CURDIR"/libgoacestepcpp-avx512.so ]; then
+		if [ -e $CURDIR/libgoacestepcpp-avx512.so ]; then
 			LIBRARY="$CURDIR/libgoacestepcpp-avx512.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export ACESTEP_LIBRARY=$LIBRARY
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/acestep-cpp "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/acestep-cpp "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/acestep-cpp "$@"
+exec $CURDIR/acestep-cpp "$@"

backend/go/ced/run.sh

@@ -4,10 +4,10 @@ set -e
 CURDIR=$(dirname "$(realpath "$0")")
 
 if [ "$(uname)" = "Darwin" ]; then
-	export DYLD_LIBRARY_PATH="$CURDIR/lib:"$CURDIR":${DYLD_LIBRARY_PATH:-}"
+	export DYLD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:${DYLD_LIBRARY_PATH:-}"
 	export CED_LIBRARY="$CURDIR/lib/libced.dylib"
 else
-	export LD_LIBRARY_PATH="$CURDIR/lib:"$CURDIR":${LD_LIBRARY_PATH:-}"
+	export LD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:${LD_LIBRARY_PATH:-}"
 fi
 
 # If a self-contained ld.so was packaged, route through it so the packaged

backend/go/cloud-proxy/run.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 set -ex
 
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
-exec "$CURDIR"/cloud-proxy "$@"
+exec $CURDIR/cloud-proxy "$@"

backend/go/crispasr/Makefile

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
 
 # CrispASR version (release tag)
 CRISPASR_REPO?=https://github.com/CrispStrobe/CrispASR
-CRISPASR_VERSION?=8f1218141b792b8868861c1af17ba1e361b05dc0
+CRISPASR_VERSION?=96b2a6ee31d30389fed8a7ef1a54239b75231ddc
 SO_TARGET?=libgocrispasr.so
 
 CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

backend/go/crispasr/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -15,20 +15,20 @@ fi
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS: single dylib variant (Metal or Accelerate)
 	LIBRARY="$CURDIR/libgocrispasr-fallback.dylib"
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/libgocrispasr-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/libgocrispasr-avx.so ]; then
+		if [ -e $CURDIR/libgocrispasr-avx.so ]; then
 			LIBRARY="$CURDIR/libgocrispasr-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/libgocrispasr-avx2.so ]; then
+		if [ -e $CURDIR/libgocrispasr-avx2.so ]; then
 			LIBRARY="$CURDIR/libgocrispasr-avx2.so"
 		fi
 	fi
@@ -36,12 +36,12 @@ else
 	# Check avx 512
 	if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX512F found OK"
-		if [ -e "$CURDIR"/libgocrispasr-avx512.so ]; then
+		if [ -e $CURDIR/libgocrispasr-avx512.so ]; then
 			LIBRARY="$CURDIR/libgocrispasr-avx512.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export CRISPASR_LIBRARY=$LIBRARY
@@ -49,14 +49,14 @@ export CRISPASR_LIBRARY=$LIBRARY
 # Point piper's espeak-ng phonemizer at the bundled voice data. The variable
 # names the directory CONTAINING espeak-ng-data (package.sh drops it next to
 # this script). Harmless when espeak-ng wasn't bundled.
-export CRISPASR_ESPEAK_DATA_PATH="$CURDIR"
+export CRISPASR_ESPEAK_DATA_PATH=$CURDIR
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/crispasr "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/crispasr "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/crispasr "$@"
+exec $CURDIR/crispasr "$@"

backend/go/depth-anything-cpp/Makefile

@@ -40,8 +40,6 @@ else ifeq ($(BUILD_TYPE),hipblas)
 else ifeq ($(BUILD_TYPE),vulkan)
 	CMAKE_ARGS+=-DGGML_VULKAN=ON -DDA_GGML_VULKAN=ON
 else ifeq ($(OS),Darwin)
-	# macOS/Metal: built + published as an OCI image by CI (includeDarwin in
-	# .github/backend-matrix.yml) so Apple Silicon users can install this backend.
 	ifneq ($(BUILD_TYPE),metal)
 		CMAKE_ARGS+=-DGGML_METAL=OFF
 	else

backend/go/depth-anything-cpp/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -15,20 +15,20 @@ fi
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS: single dylib variant (Metal or Accelerate)
 	LIBRARY="$CURDIR/libdepthanythingcpp-fallback.dylib"
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/libdepthanythingcpp-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/libdepthanythingcpp-avx.so ]; then
+		if [ -e $CURDIR/libdepthanythingcpp-avx.so ]; then
 			LIBRARY="$CURDIR/libdepthanythingcpp-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/libdepthanythingcpp-avx2.so ]; then
+		if [ -e $CURDIR/libdepthanythingcpp-avx2.so ]; then
 			LIBRARY="$CURDIR/libdepthanythingcpp-avx2.so"
 		fi
 	fi
@@ -36,22 +36,22 @@ else
 	# Check avx 512
 	if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX512F found OK"
-		if [ -e "$CURDIR"/libdepthanythingcpp-avx512.so ]; then
+		if [ -e $CURDIR/libdepthanythingcpp-avx512.so ]; then
 			LIBRARY="$CURDIR/libdepthanythingcpp-avx512.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export DEPTHANYTHING_LIBRARY=$LIBRARY
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/depth-anything-cpp "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/depth-anything-cpp "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/depth-anything-cpp "$@"
+exec $CURDIR/depth-anything-cpp "$@"

backend/go/local-store/run.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 set -ex
 
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
-exec "$CURDIR"/local-store "$@"
+exec $CURDIR/local-store "$@"

backend/go/localvqe/Makefile

@@ -32,8 +32,6 @@ endif
 ifeq ($(BUILD_TYPE),vulkan)
 	CMAKE_ARGS+=-DGGML_VULKAN=ON -DLOCALVQE_VULKAN=ON
 else ifeq ($(OS),Darwin)
-	# Apple Silicon: CPU-only (no Metal upstream); built + published as an arm64
-	# image by CI (includeDarwin in .github/backend-matrix.yml) for macOS install.
 	CMAKE_ARGS+=-DGGML_METAL=OFF
 endif
 

backend/go/localvqe/run.sh

@@ -1,34 +1,34 @@
 #!/bin/bash
 set -ex
 
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 # LocalVQE's runtime CPU-variant loader (ggml_backend_load_all) searches
 # get_executable_path() and current_path() — the second one is what saves us
 # when /proc/self/exe resolves to lib/ld.so under the bundled-loader path.
-# So we cd into "$CURDIR" (where all the libggml-cpu-*.so files live) before
+# So we cd into $CURDIR (where all the libggml-cpu-*.so files live) before
 # exec'ing the binary.
 cd "$CURDIR"
 
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS: LocalVQE is built as a SHARED library, so dyld needs the .dylib +
 	# DYLD_LIBRARY_PATH. Prefer .dylib and fall back to .so just in case.
-	export DYLD_LIBRARY_PATH="$CURDIR":"$CURDIR"/lib:$DYLD_LIBRARY_PATH
-	LOCALVQE_LIBRARY="$CURDIR"/liblocalvqe.dylib
+	export DYLD_LIBRARY_PATH=$CURDIR:$CURDIR/lib:$DYLD_LIBRARY_PATH
+	LOCALVQE_LIBRARY=$CURDIR/liblocalvqe.dylib
 	if [ ! -e "$LOCALVQE_LIBRARY" ]; then
-		LOCALVQE_LIBRARY="$CURDIR"/liblocalvqe.so
+		LOCALVQE_LIBRARY=$CURDIR/liblocalvqe.so
 	fi
 	export LOCALVQE_LIBRARY
 else
-	export LD_LIBRARY_PATH="$CURDIR":"$CURDIR"/lib:$LD_LIBRARY_PATH
-	export LOCALVQE_LIBRARY="$CURDIR"/liblocalvqe.so
+	export LD_LIBRARY_PATH=$CURDIR:$CURDIR/lib:$LD_LIBRARY_PATH
+	export LOCALVQE_LIBRARY=$CURDIR/liblocalvqe.so
 fi
 
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LOCALVQE_LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/localvqe "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/localvqe "$@"
 fi
 
 echo "Using library: $LOCALVQE_LIBRARY"
-exec "$CURDIR"/localvqe "$@"
+exec $CURDIR/localvqe "$@"

backend/go/locate-anything-cpp/Makefile

@@ -33,8 +33,6 @@ else ifeq ($(BUILD_TYPE),hipblas)
 else ifeq ($(BUILD_TYPE),vulkan)
 	CMAKE_ARGS+=-DGGML_VULKAN=ON -DLA_GGML_VULKAN=ON
 else ifeq ($(OS),Darwin)
-	# macOS/Metal: built + published as an OCI image by CI (includeDarwin in
-	# .github/backend-matrix.yml) so Apple Silicon users can install this backend.
 	ifneq ($(BUILD_TYPE),metal)
 		CMAKE_ARGS+=-DGGML_METAL=OFF
 	else

backend/go/locate-anything-cpp/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -15,20 +15,20 @@ fi
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS: single dylib variant (Metal or Accelerate)
 	LIBRARY="$CURDIR/liblocateanythingcpp-fallback.dylib"
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/liblocateanythingcpp-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/liblocateanythingcpp-avx.so ]; then
+		if [ -e $CURDIR/liblocateanythingcpp-avx.so ]; then
 			LIBRARY="$CURDIR/liblocateanythingcpp-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/liblocateanythingcpp-avx2.so ]; then
+		if [ -e $CURDIR/liblocateanythingcpp-avx2.so ]; then
 			LIBRARY="$CURDIR/liblocateanythingcpp-avx2.so"
 		fi
 	fi
@@ -36,22 +36,22 @@ else
 	# Check avx 512
 	if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX512F found OK"
-		if [ -e "$CURDIR"/liblocateanythingcpp-avx512.so ]; then
+		if [ -e $CURDIR/liblocateanythingcpp-avx512.so ]; then
 			LIBRARY="$CURDIR/liblocateanythingcpp-avx512.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export LOCATEANYTHING_LIBRARY=$LIBRARY
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/locate-anything-cpp "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/locate-anything-cpp "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/locate-anything-cpp "$@"
+exec $CURDIR/locate-anything-cpp "$@"

backend/go/omnivoice-cpp/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -15,20 +15,20 @@ fi
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS: single dylib variant (Metal or Accelerate)
 	LIBRARY="$CURDIR/libgomnivoicecpp-fallback.dylib"
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/libgomnivoicecpp-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/libgomnivoicecpp-avx.so ]; then
+		if [ -e $CURDIR/libgomnivoicecpp-avx.so ]; then
 			LIBRARY="$CURDIR/libgomnivoicecpp-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/libgomnivoicecpp-avx2.so ]; then
+		if [ -e $CURDIR/libgomnivoicecpp-avx2.so ]; then
 			LIBRARY="$CURDIR/libgomnivoicecpp-avx2.so"
 		fi
 	fi
@@ -36,22 +36,22 @@ else
 	# Check avx 512
 	if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX512F found OK"
-		if [ -e "$CURDIR"/libgomnivoicecpp-avx512.so ]; then
+		if [ -e $CURDIR/libgomnivoicecpp-avx512.so ]; then
 			LIBRARY="$CURDIR/libgomnivoicecpp-avx512.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export OMNIVOICE_LIBRARY=$LIBRARY
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/omnivoice-cpp "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/omnivoice-cpp "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/omnivoice-cpp "$@"
+exec $CURDIR/omnivoice-cpp "$@"

backend/go/opus/Makefile

@@ -1,30 +1,13 @@
 GOCMD?=go
 GO_TAGS?=
 
-# The opus shim is a small C wrapper around libopus' variadic
-# opus_encoder_ctl (see csrc/opus_shim.c). It is built as a shared library
-# and dlopen'd at runtime by the Go backend (codec.go). The extension is
-# OS-specific: Linux uses .so, macOS uses .dylib. OS is exported by the root
-# Makefile (`export OS := $(shell uname -s)`).
-SHIM_EXT=so
-
 OPUS_CFLAGS := $(shell pkg-config --cflags opus)
 OPUS_LIBS := $(shell pkg-config --libs opus)
-SHIM_LDFLAGS := $(OPUS_LIBS)
 
-ifeq ($(OS),Darwin)
-	SHIM_EXT=dylib
-	# Resolve libopus symbols lazily from the already globally-loaded
-	# libopus (codec.go dlopens it RTLD_GLOBAL before the shim) rather than
-	# recording an absolute Homebrew path in the dylib. This keeps the
-	# packaged shim relocatable on machines that have no Homebrew.
-	SHIM_LDFLAGS := -undefined dynamic_lookup
-endif
+libopusshim.so: csrc/opus_shim.c
+	$(CC) -shared -fPIC -o $@ $< $(OPUS_CFLAGS) $(OPUS_LIBS)
 
-libopusshim.$(SHIM_EXT): csrc/opus_shim.c
-	$(CC) -shared -fPIC -o $@ $< $(OPUS_CFLAGS) $(SHIM_LDFLAGS)
-
-opus: libopusshim.$(SHIM_EXT)
+opus: libopusshim.so
 	$(GOCMD) build -tags "$(GO_TAGS)" -o opus ./
 
 package: opus
@@ -33,7 +16,4 @@ package: opus
 build: package
 
 clean:
-	rm -f opus libopusshim.$(SHIM_EXT)
-	rm -rf package
-
-.PHONY: build package clean
+	rm -f opus libopusshim.so

backend/go/opus/package.sh

@@ -8,23 +8,13 @@ mkdir -p $CURDIR/package/lib
 cp -avf $CURDIR/opus $CURDIR/package/
 cp -avf $CURDIR/run.sh $CURDIR/package/
 
-# The shim extension is OS-specific (.so on Linux, .dylib on macOS).
-SHIM_EXT=so
-if [ "$(uname)" = "Darwin" ]; then
-    SHIM_EXT=dylib
-fi
-
 # Copy the opus shim library
-cp -avf $CURDIR/libopusshim.$SHIM_EXT $CURDIR/package/lib/
+cp -avf $CURDIR/libopusshim.so $CURDIR/package/lib/
 
-# Copy system libopus so the backend is self-contained: the runtime base
-# image has neither libopus-dev (Linux) nor Homebrew (macOS), so codec.go's
-# dlopen would otherwise fail. Both name patterns are attempted; only the
-# host's matching one exists.
+# Copy system libopus
 if command -v pkg-config >/dev/null 2>&1 && pkg-config --exists opus; then
     LIBOPUS_DIR=$(pkg-config --variable=libdir opus)
-    cp -avf $LIBOPUS_DIR/libopus.so* $CURDIR/package/lib/ 2>/dev/null || true
-    cp -avf $LIBOPUS_DIR/libopus*.dylib $CURDIR/package/lib/ 2>/dev/null || true
+    cp -avfL $LIBOPUS_DIR/libopus.so* $CURDIR/package/lib/ 2>/dev/null || true
 fi
 
 # Detect architecture and copy appropriate libraries
@@ -48,8 +38,6 @@ elif [ -f "/lib/ld-linux-aarch64.so.1" ]; then
     cp -arfLv /lib/aarch64-linux-gnu/libdl.so.2 $CURDIR/package/lib/libdl.so.2
     cp -arfLv /lib/aarch64-linux-gnu/librt.so.1 $CURDIR/package/lib/librt.so.1
     cp -arfLv /lib/aarch64-linux-gnu/libpthread.so.0 $CURDIR/package/lib/libpthread.so.0
-elif [ "$(uname -s)" = "Darwin" ]; then
-    echo "Detected Darwin — system libraries linked dynamically, no bundled loader needed"
 else
     echo "Warning: Could not detect architecture for system library bundling"
 fi

backend/go/opus/run.sh

@@ -1,20 +1,15 @@
 #!/bin/bash
 set -ex
 
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
-if [ "$(uname)" = "Darwin" ]; then
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
-	export OPUS_SHIM_LIBRARY="$CURDIR"/lib/libopusshim.dylib
-else
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
-	export OPUS_SHIM_LIBRARY="$CURDIR"/lib/libopusshim.so
-fi
+export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
+export OPUS_SHIM_LIBRARY=$CURDIR/lib/libopusshim.so
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/opus "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/opus "$@"
 fi
 
-exec "$CURDIR"/opus "$@"
+exec $CURDIR/opus "$@"

backend/go/parakeet-cpp/Makefile

@@ -1,6 +1,6 @@
 # parakeet-cpp backend Makefile.
 #
-# Upstream pin lives below as PARAKEET_VERSION?=f469a57270a1cc4554acb15febf60e56619673b9
+# Upstream pin lives below as PARAKEET_VERSION?=89f5e2977b4d8bccd45e7bcc6f2ef7c4ed49e89a
 # (.github/bump_deps.sh) can find and update it - matches the
 # whisper.cpp / ds4 / vibevoice-cpp convention.
 #
@@ -15,7 +15,7 @@
 # That's what the L0 smoke test uses. The default target below does the
 # proper clone-at-pin + cmake build so CI doesn't need a side-checkout.
 
-PARAKEET_VERSION?=f469a57270a1cc4554acb15febf60e56619673b9
+PARAKEET_VERSION?=89f5e2977b4d8bccd45e7bcc6f2ef7c4ed49e89a
 PARAKEET_REPO?=https://github.com/mudler/parakeet.cpp
 
 GOCMD?=go

backend/go/parakeet-cpp/run.sh

@@ -4,10 +4,10 @@ set -e
 CURDIR=$(dirname "$(realpath "$0")")
 
 if [ "$(uname)" = "Darwin" ]; then
-	export DYLD_LIBRARY_PATH="$CURDIR/lib:"$CURDIR":${DYLD_LIBRARY_PATH:-}"
+	export DYLD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:${DYLD_LIBRARY_PATH:-}"
 	export PARAKEET_LIBRARY="$CURDIR/lib/libparakeet.dylib"
 else
-	export LD_LIBRARY_PATH="$CURDIR/lib:"$CURDIR":${LD_LIBRARY_PATH:-}"
+	export LD_LIBRARY_PATH="$CURDIR/lib:$CURDIR:${LD_LIBRARY_PATH:-}"
 	export PARAKEET_LIBRARY="$CURDIR/lib/libparakeet.so"
 fi
 

backend/go/piper/package.sh

@@ -16,15 +16,7 @@ cp -rfv $CURDIR/run.sh $CURDIR/package/
 cp -rfLv $CURDIR/sources/go-piper/piper-phonemize/pi/lib/* $CURDIR/package/lib/
 
 # Detect architecture and copy appropriate libraries
-if [ "$(uname)" = "Darwin" ]; then
-    # macOS has no glibc loader to bundle. The piper binary links its bundled
-    # libs (libucd, libespeak-ng, libpiper_phonemize, libonnxruntime) via
-    # @rpath but ships with no LC_RPATH, so dyld aborts at launch with
-    # "Library not loaded: @rpath/libucd.dylib ... no LC_RPATH's found".
-    # Add an @loader_path/lib rpath so @rpath resolves to package/lib/.
-    echo "Detected macOS; adding @loader_path/lib rpath so bundled libs resolve via @rpath..."
-    install_name_tool -add_rpath @loader_path/lib "$CURDIR/package/piper"
-elif [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
+if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
     # x86_64 architecture
     echo "Detected x86_64 architecture, copying x86_64 libraries..."
     cp -arfLv /lib64/ld-linux-x86-64.so.2 $CURDIR/package/lib/ld.so

backend/go/piper/run.sh

@@ -1,20 +1,15 @@
 #!/bin/bash
 set -ex
 
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
-export ESPEAK_NG_DATA="$CURDIR"/espeak-ng-data
-
-if [ "$(uname)" = "Darwin" ]; then
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
-else
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
-fi
+export ESPEAK_NG_DATA=$CURDIR/espeak-ng-data
+export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/piper "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/piper "$@"
 fi
 
-exec "$CURDIR"/piper "$@"
+exec $CURDIR/piper "$@"

backend/go/qwen3-tts-cpp/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -15,20 +15,20 @@ fi
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS: single dylib variant (Metal or Accelerate)
 	LIBRARY="$CURDIR/libgoqwen3ttscpp-fallback.dylib"
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/libgoqwen3ttscpp-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/libgoqwen3ttscpp-avx.so ]; then
+		if [ -e $CURDIR/libgoqwen3ttscpp-avx.so ]; then
 			LIBRARY="$CURDIR/libgoqwen3ttscpp-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/libgoqwen3ttscpp-avx2.so ]; then
+		if [ -e $CURDIR/libgoqwen3ttscpp-avx2.so ]; then
 			LIBRARY="$CURDIR/libgoqwen3ttscpp-avx2.so"
 		fi
 	fi
@@ -36,22 +36,22 @@ else
 	# Check avx 512
 	if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX512F found OK"
-		if [ -e "$CURDIR"/libgoqwen3ttscpp-avx512.so ]; then
+		if [ -e $CURDIR/libgoqwen3ttscpp-avx512.so ]; then
 			LIBRARY="$CURDIR/libgoqwen3ttscpp-avx512.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export QWEN3TTS_LIBRARY=$LIBRARY
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/qwen3-tts-cpp "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/qwen3-tts-cpp "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/qwen3-tts-cpp "$@"
+exec $CURDIR/qwen3-tts-cpp "$@"

backend/go/rfdetr-cpp/Makefile

@@ -34,8 +34,6 @@ else ifeq ($(BUILD_TYPE),hipblas)
 else ifeq ($(BUILD_TYPE),vulkan)
 	CMAKE_ARGS+=-DGGML_VULKAN=ON -DRFDETR_GGML_VULKAN=ON
 else ifeq ($(OS),Darwin)
-	# macOS/Metal: built + published as an OCI image by CI (includeDarwin in
-	# .github/backend-matrix.yml) so Apple Silicon users can install this backend.
 	ifneq ($(BUILD_TYPE),metal)
 		CMAKE_ARGS+=-DGGML_METAL=OFF
 	else

backend/go/rfdetr-cpp/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -15,20 +15,20 @@ fi
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS: single dylib variant (Metal or Accelerate)
 	LIBRARY="$CURDIR/librfdetrcpp-fallback.dylib"
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/librfdetrcpp-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/librfdetrcpp-avx.so ]; then
+		if [ -e $CURDIR/librfdetrcpp-avx.so ]; then
 			LIBRARY="$CURDIR/librfdetrcpp-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/librfdetrcpp-avx2.so ]; then
+		if [ -e $CURDIR/librfdetrcpp-avx2.so ]; then
 			LIBRARY="$CURDIR/librfdetrcpp-avx2.so"
 		fi
 	fi
@@ -36,22 +36,22 @@ else
 	# Check avx 512
 	if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX512F found OK"
-		if [ -e "$CURDIR"/librfdetrcpp-avx512.so ]; then
+		if [ -e $CURDIR/librfdetrcpp-avx512.so ]; then
 			LIBRARY="$CURDIR/librfdetrcpp-avx512.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export RFDETR_LIBRARY=$LIBRARY
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/rfdetr-cpp "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/rfdetr-cpp "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/rfdetr-cpp "$@"
+exec $CURDIR/rfdetr-cpp "$@"

backend/go/sam3-cpp/Makefile

@@ -31,8 +31,6 @@ else ifeq ($(BUILD_TYPE),hipblas)
 else ifeq ($(BUILD_TYPE),vulkan)
 	CMAKE_ARGS+=-DGGML_VULKAN=ON
 else ifeq ($(OS),Darwin)
-	# macOS/Metal: built + published as an OCI image by CI (includeDarwin in
-	# .github/backend-matrix.yml) so Apple Silicon users can install this backend.
 	ifneq ($(BUILD_TYPE),metal)
 		CMAKE_ARGS+=-DGGML_METAL=OFF
 	else

backend/go/sam3-cpp/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -15,20 +15,20 @@ fi
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS: single dylib variant (Metal or Accelerate)
 	LIBRARY="$CURDIR/libgosam3-fallback.dylib"
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/libgosam3-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/libgosam3-avx.so ]; then
+		if [ -e $CURDIR/libgosam3-avx.so ]; then
 			LIBRARY="$CURDIR/libgosam3-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/libgosam3-avx2.so ]; then
+		if [ -e $CURDIR/libgosam3-avx2.so ]; then
 			LIBRARY="$CURDIR/libgosam3-avx2.so"
 		fi
 	fi
@@ -36,22 +36,22 @@ else
 	# Check avx 512
 	if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX512F found OK"
-		if [ -e "$CURDIR"/libgosam3-avx512.so ]; then
+		if [ -e $CURDIR/libgosam3-avx512.so ]; then
 			LIBRARY="$CURDIR/libgosam3-avx512.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export SAM3_LIBRARY=$LIBRARY
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/sam3-cpp "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/sam3-cpp "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/sam3-cpp "$@"
+exec $CURDIR/sam3-cpp "$@"

backend/go/sherpa-onnx/run.sh

@@ -1,19 +1,19 @@
 #!/bin/bash
 set -ex
 
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 if [ "$(uname)" = "Darwin" ]; then
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
-	export SHERPA_SHIM_LIBRARY="$CURDIR"/lib/libsherpa-shim.dylib
-	export SHERPA_ONNX_LIBRARY="$CURDIR"/lib/libsherpa-onnx-c-api.dylib
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
+	export SHERPA_SHIM_LIBRARY=$CURDIR/lib/libsherpa-shim.dylib
+	export SHERPA_ONNX_LIBRARY=$CURDIR/lib/libsherpa-onnx-c-api.dylib
 else
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/sherpa-onnx "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/sherpa-onnx "$@"
 fi
 
-exec "$CURDIR"/sherpa-onnx "$@"
+exec $CURDIR/sherpa-onnx "$@"

backend/go/silero-vad/package.sh

@@ -15,14 +15,7 @@ cp -avf $CURDIR/run.sh $CURDIR/package/
 cp -rfLv $CURDIR/backend-assets/lib/* $CURDIR/package/lib/
 
 # Detect architecture and copy appropriate libraries
-if [ "$(uname)" = "Darwin" ]; then
-    # macOS has no glibc loader to bundle. silero-vad links its bundled
-    # libonnxruntime via @rpath but ships with no LC_RPATH, so dyld can't find
-    # it at runtime. Add an @loader_path/lib rpath so @rpath resolves to
-    # package/lib/ (matching the piper darwin fix, #10525).
-    echo "Detected macOS; adding @loader_path/lib rpath so bundled libs resolve via @rpath..."
-    install_name_tool -add_rpath @loader_path/lib "$CURDIR/package/silero-vad"
-elif [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
+if [ -f "/lib64/ld-linux-x86-64.so.2" ]; then
     # x86_64 architecture
     echo "Detected x86_64 architecture, copying x86_64 libraries..."
     cp -arfLv /lib64/ld-linux-x86-64.so.2 $CURDIR/package/lib/ld.so

backend/go/silero-vad/run.sh

@@ -1,18 +1,14 @@
 #!/bin/bash
 set -ex
 
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
-if [ "$(uname)" = "Darwin" ]; then
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
-else
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
-fi
+export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/silero-vad "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/silero-vad "$@"
 fi
 
-exec "$CURDIR"/silero-vad "$@"
+exec $CURDIR/silero-vad "$@"

backend/go/stablediffusion-ggml/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -20,20 +20,20 @@ if [ "$(uname)" = "Darwin" ]; then
 	if [ ! -e "$LIBRARY" ]; then
 		LIBRARY="$CURDIR/libgosd-fallback.so"
 	fi
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/libgosd-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/libgosd-avx.so ]; then
+		if [ -e $CURDIR/libgosd-avx.so ]; then
 			LIBRARY="$CURDIR/libgosd-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/libgosd-avx2.so ]; then
+		if [ -e $CURDIR/libgosd-avx2.so ]; then
 			LIBRARY="$CURDIR/libgosd-avx2.so"
 		fi
 	fi
@@ -41,22 +41,22 @@ else
 	# Check avx 512
 	if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX512F found OK"
-		if [ -e "$CURDIR"/libgosd-avx512.so ]; then
+		if [ -e $CURDIR/libgosd-avx512.so ]; then
 			LIBRARY="$CURDIR/libgosd-avx512.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export SD_LIBRARY=$LIBRARY
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/stablediffusion-ggml "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/stablediffusion-ggml "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/stablediffusion-ggml "$@"
+exec $CURDIR/stablediffusion-ggml "$@"

backend/go/supertonic/run.sh

@@ -1,21 +1,21 @@
 #!/bin/bash
 set -ex
 
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS uses dyld: there is no ld.so loader, and the search path env
 	# var is DYLD_LIBRARY_PATH. ONNX Runtime ships as a .dylib here.
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
-	export ONNXRUNTIME_LIB_PATH="$CURDIR"/lib/libonnxruntime.dylib
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
+	export ONNXRUNTIME_LIB_PATH=$CURDIR/lib/libonnxruntime.dylib
 else
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
-	export ONNXRUNTIME_LIB_PATH="$CURDIR"/lib/libonnxruntime.so
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
+	export ONNXRUNTIME_LIB_PATH=$CURDIR/lib/libonnxruntime.so
 
-	if [ -f "$CURDIR"/lib/ld.so ]; then
+	if [ -f $CURDIR/lib/ld.so ]; then
 		echo "Using lib/ld.so"
-		exec "$CURDIR"/lib/ld.so "$CURDIR"/supertonic "$@"
+		exec $CURDIR/lib/ld.so $CURDIR/supertonic "$@"
 	fi
 fi
 
-exec "$CURDIR"/supertonic "$@"
+exec $CURDIR/supertonic "$@"

backend/go/vibevoice-cpp/run.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 set -ex
 
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -14,41 +14,41 @@ fi
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS: single dylib variant (Metal or Accelerate)
 	LIBRARY="$CURDIR/libgovibevoicecpp-fallback.dylib"
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/libgovibevoicecpp-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/libgovibevoicecpp-avx.so ]; then
+		if [ -e $CURDIR/libgovibevoicecpp-avx.so ]; then
 			LIBRARY="$CURDIR/libgovibevoicecpp-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/libgovibevoicecpp-avx2.so ]; then
+		if [ -e $CURDIR/libgovibevoicecpp-avx2.so ]; then
 			LIBRARY="$CURDIR/libgovibevoicecpp-avx2.so"
 		fi
 	fi
 
 	if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX512F found OK"
-		if [ -e "$CURDIR"/libgovibevoicecpp-avx512.so ]; then
+		if [ -e $CURDIR/libgovibevoicecpp-avx512.so ]; then
 			LIBRARY="$CURDIR/libgovibevoicecpp-avx512.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export VIBEVOICECPP_LIBRARY=$LIBRARY
 
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/vibevoice-cpp "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/vibevoice-cpp "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/vibevoice-cpp "$@"
+exec $CURDIR/vibevoice-cpp "$@"

backend/go/voxtral/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -15,35 +15,35 @@ fi
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS: single dylib variant (Metal or Accelerate)
 	LIBRARY="$CURDIR/libgovoxtral-fallback.dylib"
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/libgovoxtral-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/libgovoxtral-avx.so ]; then
+		if [ -e $CURDIR/libgovoxtral-avx.so ]; then
 			LIBRARY="$CURDIR/libgovoxtral-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/libgovoxtral-avx2.so ]; then
+		if [ -e $CURDIR/libgovoxtral-avx2.so ]; then
 			LIBRARY="$CURDIR/libgovoxtral-avx2.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export VOXTRAL_LIBRARY=$LIBRARY
 
 # If there is a lib/ld.so, use it (Linux only)
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/voxtral "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/voxtral "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/voxtral "$@"
+exec $CURDIR/voxtral "$@"

backend/go/whisper/run.sh

@@ -2,7 +2,7 @@
 set -ex
 
 # Get the absolute current dir where the script is located
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
 cd /
 
@@ -15,20 +15,20 @@ fi
 if [ "$(uname)" = "Darwin" ]; then
 	# macOS: single dylib variant (Metal or Accelerate)
 	LIBRARY="$CURDIR/libgowhisper-fallback.dylib"
-	export DYLD_LIBRARY_PATH="$CURDIR"/lib:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH=$CURDIR/lib:$DYLD_LIBRARY_PATH
 else
 	LIBRARY="$CURDIR/libgowhisper-fallback.so"
 
 	if grep -q -e "\savx\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX    found OK"
-		if [ -e "$CURDIR"/libgowhisper-avx.so ]; then
+		if [ -e $CURDIR/libgowhisper-avx.so ]; then
 			LIBRARY="$CURDIR/libgowhisper-avx.so"
 		fi
 	fi
 
 	if grep -q -e "\savx2\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX2   found OK"
-		if [ -e "$CURDIR"/libgowhisper-avx2.so ]; then
+		if [ -e $CURDIR/libgowhisper-avx2.so ]; then
 			LIBRARY="$CURDIR/libgowhisper-avx2.so"
 		fi
 	fi
@@ -36,22 +36,22 @@ else
 	# Check avx 512
 	if grep -q -e "\savx512f\s" /proc/cpuinfo ; then
 		echo "CPU:    AVX512F found OK"
-		if [ -e "$CURDIR"/libgowhisper-avx512.so ]; then
+		if [ -e $CURDIR/libgowhisper-avx512.so ]; then
 			LIBRARY="$CURDIR/libgowhisper-avx512.so"
 		fi
 	fi
 
-	export LD_LIBRARY_PATH="$CURDIR"/lib:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH=$CURDIR/lib:$LD_LIBRARY_PATH
 fi
 
 export WHISPER_LIBRARY=$LIBRARY
 
 # If there is a lib/ld.so, use it
-if [ -f "$CURDIR"/lib/ld.so ]; then
+if [ -f $CURDIR/lib/ld.so ]; then
 	echo "Using lib/ld.so"
 	echo "Using library: $LIBRARY"
-	exec "$CURDIR"/lib/ld.so "$CURDIR"/whisper "$@"
+	exec $CURDIR/lib/ld.so $CURDIR/whisper "$@"
 fi
 
 echo "Using library: $LIBRARY"
-exec "$CURDIR"/whisper "$@"
+exec $CURDIR/whisper "$@"

backend/index.yaml

@@ -340,7 +340,6 @@
     nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-sam3-cpp"
     intel: "intel-sycl-f32-sam3-cpp"
     vulkan: "vulkan-sam3-cpp"
-    metal: "metal-sam3-cpp"
 - &rfdetrcpp
   name: "rfdetr-cpp"
   alias: "rfdetr-cpp"
@@ -369,7 +368,6 @@
     nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-rfdetr-cpp"
     intel: "intel-sycl-f32-rfdetr-cpp"
     vulkan: "vulkan-rfdetr-cpp"
-    metal: "metal-rfdetr-cpp"
 - &locateanything
   name: "locate-anything"
   alias: "locate-anything"
@@ -399,7 +397,6 @@
     nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-locate-anything-cpp"
     intel: "intel-sycl-f32-locate-anything-cpp"
     vulkan: "vulkan-locate-anything-cpp"
-    metal: "metal-locate-anything-cpp"
 - !!merge <<: *locateanything
   name: "locate-anything-development"
   capabilities:
@@ -412,7 +409,6 @@
     nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-locate-anything-cpp-development"
     intel: "intel-sycl-f32-locate-anything-cpp-development"
     vulkan: "vulkan-locate-anything-cpp-development"
-    metal: "metal-locate-anything-cpp-development"
 - !!merge <<: *locateanything
   name: "cpu-locate-anything-cpp"
   uri: "quay.io/go-skynet/local-ai-backends:latest-cpu-locate-anything-cpp"
@@ -423,16 +419,6 @@
   uri: "quay.io/go-skynet/local-ai-backends:master-cpu-locate-anything-cpp"
   mirrors:
     - localai/localai-backends:master-cpu-locate-anything-cpp
-- !!merge <<: *locateanything
-  name: "metal-locate-anything-cpp"
-  uri: "quay.io/go-skynet/local-ai-backends:latest-metal-darwin-arm64-locate-anything-cpp"
-  mirrors:
-    - localai/localai-backends:latest-metal-darwin-arm64-locate-anything-cpp
-- !!merge <<: *locateanything
-  name: "metal-locate-anything-cpp-development"
-  uri: "quay.io/go-skynet/local-ai-backends:master-metal-darwin-arm64-locate-anything-cpp"
-  mirrors:
-    - localai/localai-backends:master-metal-darwin-arm64-locate-anything-cpp
 - !!merge <<: *locateanything
   name: "cuda12-locate-anything-cpp"
   uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-locate-anything-cpp"
@@ -531,7 +517,6 @@
     nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-depth-anything-cpp"
     intel: "intel-sycl-f32-depth-anything-cpp"
     vulkan: "vulkan-depth-anything-cpp"
-    metal: "metal-depth-anything-cpp"
 - !!merge <<: *depthanything
   name: "depth-anything-development"
   capabilities:
@@ -544,7 +529,6 @@
     nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-depth-anything-cpp-development"
     intel: "intel-sycl-f32-depth-anything-cpp-development"
     vulkan: "vulkan-depth-anything-cpp-development"
-    metal: "metal-depth-anything-cpp-development"
 - !!merge <<: *depthanything
   name: "cpu-depth-anything-cpp"
   uri: "quay.io/go-skynet/local-ai-backends:latest-cpu-depth-anything-cpp"
@@ -555,16 +539,6 @@
   uri: "quay.io/go-skynet/local-ai-backends:master-cpu-depth-anything-cpp"
   mirrors:
     - localai/localai-backends:master-cpu-depth-anything-cpp
-- !!merge <<: *depthanything
-  name: "metal-depth-anything-cpp"
-  uri: "quay.io/go-skynet/local-ai-backends:latest-metal-darwin-arm64-depth-anything-cpp"
-  mirrors:
-    - localai/localai-backends:latest-metal-darwin-arm64-depth-anything-cpp
-- !!merge <<: *depthanything
-  name: "metal-depth-anything-cpp-development"
-  uri: "quay.io/go-skynet/local-ai-backends:master-metal-darwin-arm64-depth-anything-cpp"
-  mirrors:
-    - localai/localai-backends:master-metal-darwin-arm64-depth-anything-cpp
 - !!merge <<: *depthanything
   name: "cuda12-depth-anything-cpp"
   uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-depth-anything-cpp"
@@ -1057,8 +1031,6 @@
     nvidia-l4t: "vulkan-localvqe"
     nvidia-l4t-cuda-12: "vulkan-localvqe"
     nvidia-l4t-cuda-13: "vulkan-localvqe"
-    # Apple Silicon: CPU build (LocalVQE has no Metal path); still arm64-native.
-    metal: "metal-localvqe"
 - &privacyfilter
   name: "privacy-filter"
   alias: "privacy-filter"
@@ -1095,7 +1067,6 @@
     amd: "vulkan-privacy-filter"
     intel: "vulkan-privacy-filter"
     vulkan: "vulkan-privacy-filter"
-    metal: "metal-privacy-filter"
 - &faster-whisper
   icon: https://avatars.githubusercontent.com/u/1520500?s=200&v=4
   description: |
@@ -2938,16 +2909,6 @@
   uri: "quay.io/go-skynet/local-ai-backends:master-gpu-vulkan-privacy-filter"
   mirrors:
     - localai/localai-backends:master-gpu-vulkan-privacy-filter
-- !!merge <<: *privacyfilter
-  name: "metal-privacy-filter"
-  uri: "quay.io/go-skynet/local-ai-backends:latest-metal-darwin-arm64-privacy-filter"
-  mirrors:
-    - localai/localai-backends:latest-metal-darwin-arm64-privacy-filter
-- !!merge <<: *privacyfilter
-  name: "metal-privacy-filter-development"
-  uri: "quay.io/go-skynet/local-ai-backends:master-metal-darwin-arm64-privacy-filter"
-  mirrors:
-    - localai/localai-backends:master-metal-darwin-arm64-privacy-filter
 - !!merge <<: *privacyfilter
   name: "cuda13-privacy-filter"
   uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-13-privacy-filter"
@@ -3259,7 +3220,6 @@
     nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-sam3-cpp-development"
     intel: "intel-sycl-f32-sam3-cpp-development"
     vulkan: "vulkan-sam3-cpp-development"
-    metal: "metal-sam3-cpp-development"
 - !!merge <<: *sam3cpp
   name: "cpu-sam3-cpp"
   uri: "quay.io/go-skynet/local-ai-backends:latest-cpu-sam3-cpp"
@@ -3270,16 +3230,6 @@
   uri: "quay.io/go-skynet/local-ai-backends:master-cpu-sam3-cpp"
   mirrors:
     - localai/localai-backends:master-cpu-sam3-cpp
-- !!merge <<: *sam3cpp
-  name: "metal-sam3-cpp"
-  uri: "quay.io/go-skynet/local-ai-backends:latest-metal-darwin-arm64-sam3-cpp"
-  mirrors:
-    - localai/localai-backends:latest-metal-darwin-arm64-sam3-cpp
-- !!merge <<: *sam3cpp
-  name: "metal-sam3-cpp-development"
-  uri: "quay.io/go-skynet/local-ai-backends:master-metal-darwin-arm64-sam3-cpp"
-  mirrors:
-    - localai/localai-backends:master-metal-darwin-arm64-sam3-cpp
 - !!merge <<: *sam3cpp
   name: "cuda12-sam3-cpp"
   uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-sam3-cpp"
@@ -3353,7 +3303,6 @@
     nvidia-l4t-cuda-13: "cuda13-nvidia-l4t-arm64-rfdetr-cpp-development"
     intel: "intel-sycl-f32-rfdetr-cpp-development"
     vulkan: "vulkan-rfdetr-cpp-development"
-    metal: "metal-rfdetr-cpp-development"
 - !!merge <<: *rfdetrcpp
   name: "cpu-rfdetr-cpp"
   uri: "quay.io/go-skynet/local-ai-backends:latest-cpu-rfdetr-cpp"
@@ -3364,16 +3313,6 @@
   uri: "quay.io/go-skynet/local-ai-backends:master-cpu-rfdetr-cpp"
   mirrors:
     - localai/localai-backends:master-cpu-rfdetr-cpp
-- !!merge <<: *rfdetrcpp
-  name: "metal-rfdetr-cpp"
-  uri: "quay.io/go-skynet/local-ai-backends:latest-metal-darwin-arm64-rfdetr-cpp"
-  mirrors:
-    - localai/localai-backends:latest-metal-darwin-arm64-rfdetr-cpp
-- !!merge <<: *rfdetrcpp
-  name: "metal-rfdetr-cpp-development"
-  uri: "quay.io/go-skynet/local-ai-backends:master-metal-darwin-arm64-rfdetr-cpp"
-  mirrors:
-    - localai/localai-backends:master-metal-darwin-arm64-rfdetr-cpp
 - !!merge <<: *rfdetrcpp
   name: "cuda12-rfdetr-cpp"
   uri: "quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-rfdetr-cpp"
@@ -4162,16 +4101,6 @@
   uri: "quay.io/go-skynet/local-ai-backends:master-gpu-vulkan-localvqe"
   mirrors:
     - localai/localai-backends:master-gpu-vulkan-localvqe
-- !!merge <<: *localvqecpp
-  name: "metal-localvqe"
-  uri: "quay.io/go-skynet/local-ai-backends:latest-metal-darwin-arm64-localvqe"
-  mirrors:
-    - localai/localai-backends:latest-metal-darwin-arm64-localvqe
-- !!merge <<: *localvqecpp
-  name: "metal-localvqe-development"
-  uri: "quay.io/go-skynet/local-ai-backends:master-metal-darwin-arm64-localvqe"
-  mirrors:
-    - localai/localai-backends:master-metal-darwin-arm64-localvqe
 ## kokoro
 - !!merge <<: *kokoro
   name: "kokoro-development"

backend/python/vllm/requirements-cpu.txt

@@ -1,6 +1,6 @@
 --extra-index-url https://download.pytorch.org/whl/cpu
 accelerate
-torch==2.9.1+cpu
+torch==2.12.0+cpu
 torchvision
 torchaudio
 transformers

backend/rust/kokoros/run.sh

@@ -1,23 +1,23 @@
 #!/bin/bash
 set -ex
 
-CURDIR=$(dirname "$(realpath "$0")")
+CURDIR=$(dirname "$(realpath $0)")
 
-export LD_LIBRARY_PATH="$CURDIR"/lib:${LD_LIBRARY_PATH:-}
+export LD_LIBRARY_PATH=$CURDIR/lib:${LD_LIBRARY_PATH:-}
 
 # SSL certificates for model auto-download
 if [ -d "$CURDIR/etc/ssl/certs" ]; then
-    export SSL_CERT_DIR="$CURDIR"/etc/ssl/certs
+    export SSL_CERT_DIR=$CURDIR/etc/ssl/certs
 fi
 
 # espeak-ng data directory
 if [ -d "$CURDIR/espeak-ng-data" ]; then
-    export ESPEAK_NG_DATA="$CURDIR"/espeak-ng-data
+    export ESPEAK_NG_DATA=$CURDIR/espeak-ng-data
 fi
 
 # Use bundled ld.so if present (portability)
-if [ -f "$CURDIR"/lib/ld.so ]; then
-    exec "$CURDIR"/lib/ld.so "$CURDIR"/kokoros-grpc "$@"
+if [ -f $CURDIR/lib/ld.so ]; then
+    exec $CURDIR/lib/ld.so $CURDIR/kokoros-grpc "$@"
 fi
 
-exec "$CURDIR"/kokoros-grpc "$@"
+exec $CURDIR/kokoros-grpc "$@"

cmd/launcher/FyneApp.toml

@@ -1,8 +0,0 @@
-Website = "https://localai.io"
-
-[Details]
-Icon = "../../core/http/static/logo.png"
-Name = "LocalAI"
-ID = "com.localai.launcher"
-Version = "0.0.0"
-Build = 1

contrib/macos/Launcher.entitlements

@@ -1,14 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-<dict>
-    <key>com.apple.security.network.client</key>
-    <true/>
-    <key>com.apple.security.network.server</key>
-    <true/>
-    <key>com.apple.security.cs.allow-jit</key>
-    <true/>
-    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
-    <true/>
-</dict>
-</plist>

contrib/macos/sign-and-notarize.sh

@@ -1,84 +0,0 @@
-#!/usr/bin/env bash
-# Code-sign and notarize macOS artifacts for LocalAI.
-# Every sub-command is a no-op (exit 0) when its required secret is unset,
-# so unsigned builds (forks, local dev, PRs) keep working.
-set -euo pipefail
-
-ENTITLEMENTS="contrib/macos/Launcher.entitlements"
-KEYCHAIN="localai-ci.keychain-db"
-
-cmd_import_cert() {
-  if [ -z "${MACOS_CERTIFICATE:-}" ]; then
-    echo "[sign] MACOS_CERTIFICATE unset: skipping cert import (unsigned build)"
-    return 0
-  fi
-  local certfile keychain_pwd default_keychain
-  certfile="$(mktemp).p12"
-  keychain_pwd="${MACOS_CI_KEYCHAIN_PWD:?MACOS_CI_KEYCHAIN_PWD required when signing}"
-  echo "$MACOS_CERTIFICATE" | base64 --decode > "$certfile"
-  security create-keychain -p "$keychain_pwd" "$KEYCHAIN"
-  security set-keychain-settings -lut 21600 "$KEYCHAIN"
-  security unlock-keychain -p "$keychain_pwd" "$KEYCHAIN"
-  security import "$certfile" -k "$KEYCHAIN" -P "${MACOS_CERTIFICATE_PWD:?}" \
-    -T /usr/bin/codesign -T /usr/bin/security
-  security set-key-partition-list -S apple-tool:,apple:,codesign: \
-    -s -k "$keychain_pwd" "$KEYCHAIN" >/dev/null
-  default_keychain="$(security default-keychain | tr -d ' "')"
-  security list-keychains -d user -s "$KEYCHAIN" "$default_keychain"
-  rm -f "$certfile"
-  echo "[sign] certificate imported into $KEYCHAIN"
-}
-
-cmd_sign() {
-  local target="$1"
-  if [ -z "${MACOS_SIGN_IDENTITY:-}" ]; then
-    echo "[sign] MACOS_SIGN_IDENTITY unset: skipping codesign of $target"
-    return 0
-  fi
-  case "$target" in
-    *.app)
-      # Hardened runtime + entitlements are required for notarizing the app bundle.
-      codesign --deep --force --options runtime --timestamp \
-        --entitlements "$ENTITLEMENTS" \
-        --sign "$MACOS_SIGN_IDENTITY" "$target"
-      ;;
-    *)
-      # A disk image carries no entitlements/runtime; just sign the container.
-      codesign --force --timestamp --sign "$MACOS_SIGN_IDENTITY" "$target"
-      ;;
-  esac
-  codesign --verify --strict --verbose=2 "$target"
-  echo "[sign] signed $target"
-}
-
-cmd_notarize() {
-  local dmg="$1"
-  if [ -z "${MACOS_NOTARY_KEY:-}" ]; then
-    echo "[notarize] MACOS_NOTARY_KEY unset: skipping notarization of $dmg"
-    return 0
-  fi
-  local keyfile
-  keyfile="$(mktemp).p8"
-  echo "$MACOS_NOTARY_KEY" | base64 --decode > "$keyfile"
-  xcrun notarytool submit "$dmg" \
-    --key "$keyfile" \
-    --key-id "${MACOS_NOTARY_KEY_ID:?}" \
-    --issuer "${MACOS_NOTARY_ISSUER_ID:?}" \
-    --wait
-  rm -f "$keyfile"
-  xcrun stapler staple "$dmg"
-  xcrun stapler validate "$dmg"
-  echo "[notarize] notarized and stapled $dmg"
-}
-
-main() {
-  local sub="${1:-}"; shift || true
-  case "$sub" in
-    import-cert) cmd_import_cert ;;
-    sign)        cmd_sign "$@" ;;
-    notarize)    cmd_notarize "$@" ;;
-    *) echo "usage: $0 {import-cert|sign <path>|notarize <dmg>}" >&2; exit 2 ;;
-  esac
-}
-
-main "$@"

core/application/agent_jobs.go

@@ -37,8 +37,6 @@ func (a *Application) RestartAgentJobService() error {
 		if d.JobStore != nil {
 			agentJobService.SetDistributedJobStore(d.JobStore)
 		}
-		// Keep agent tasks consistent across replicas (same client the dispatcher uses).
-		agentJobService.SetTaskSyncNATS(d.Nats)
 	}
 
 	// Start the service

core/application/application.go

@@ -604,10 +604,6 @@ func (a *Application) StartAgentPool() {
 			usm.SetJobDBStore(s)
 		}
 	}
-	// Keep per-user agent tasks consistent across replicas (nil in standalone).
-	if d := a.Distributed(); d != nil {
-		usm.SetJobSyncNATS(d.Nats)
-	}
 	aps.SetUserServicesManager(usm)
 
 	a.agentPoolService.Store(aps)

core/application/startup.go

@@ -16,7 +16,6 @@ import (
 	"github.com/mudler/LocalAI/core/services/galleryop"
 	"github.com/mudler/LocalAI/core/services/jobs"
 	"github.com/mudler/LocalAI/core/services/messaging"
-	"github.com/mudler/LocalAI/core/services/modeladmin"
 	"github.com/mudler/LocalAI/core/services/monitoring"
 	"github.com/mudler/LocalAI/core/services/nodes"
 	"github.com/mudler/LocalAI/core/services/routing/admission"
@@ -280,9 +279,6 @@ func New(opts ...config.AppOption) (*Application, error) {
 		if application.agentJobService != nil {
 			application.agentJobService.SetDistributedBackends(distSvc.Dispatcher)
 			application.agentJobService.SetDistributedJobStore(distSvc.JobStore)
-			// Keep agent tasks consistent across replicas (jobs already sync via the
-			// dispatcher + DB read-through). Same NATS client the dispatcher uses.
-			application.agentJobService.SetTaskSyncNATS(distSvc.Nats)
 		}
 		// Wire skill store into AgentPoolService (wired at pool start time via closure)
 		// The actual wiring happens in StartAgentPool since the pool doesn't exist yet.
@@ -334,14 +330,9 @@ func New(opts ...config.AppOption) (*Application, error) {
 			gs := application.galleryService
 			sys := options.SystemState
 			cfgLoaderOpts := options.ToConfigLoaderOptions()
-			gs.OnModelsChanged = func(evt messaging.CacheInvalidateEvent) {
-				// ApplyRemoteChange honors the op: a "delete" prunes the element
-				// (a reload-from-path is additive and cannot drop it), anything
-				// else reloads from disk; a named element's running instance is
-				// shut down so the new config takes effect. The originating
-				// replica reloads inline and never depends on this path.
-				if err := modeladmin.ApplyRemoteChange(application.ModelConfigLoader(), application.modelLoader, sys.Model.ModelsPath, evt, cfgLoaderOpts...); err != nil {
-					xlog.Warn("Failed to apply peer model config change", "error", err)
+			gs.OnModelsChanged = func(_ messaging.CacheInvalidateEvent) {
+				if err := application.ModelConfigLoader().LoadModelConfigsFromPath(sys.Model.ModelsPath, cfgLoaderOpts...); err != nil {
+					xlog.Warn("Failed to reload model configs after peer invalidation", "error", err)
 				}
 			}
 			if err := application.galleryService.SubscribeBroadcasts(); err != nil {

core/cli/run.go

@@ -203,7 +203,6 @@ func (r *RunCMD) Run(ctx *cliContext.Context) error {
 		system.WithBackendImagesReleaseTag(r.BackendImagesReleaseTag),
 		system.WithBackendImagesBranchTag(r.BackendImagesBranchTag),
 		system.WithBackendDevSuffix(r.BackendDevSuffix),
-		system.WithPreferDevelopmentBackends(r.PreferDevelopmentBackends),
 	)
 	if err != nil {
 		return err

core/gallery/backends.go

@@ -59,22 +59,6 @@ func getFallbackTagValues(systemState *system.SystemState) (latestTag, masterTag
 	return latestTag, masterTag, devSuffix
 }
 
-// developmentURI returns the development image URI for a released backend URI by
-// swapping the released tag for the branch tag (e.g.
-// latest-metal-darwin-arm64-llama-cpp -> master-metal-darwin-arm64-llama-cpp).
-// The branch image tracks development. ok is false when uri has no released tag
-// to swap or already uses the branch tag.
-func developmentURI(uri, latestTag, masterTag string) (string, bool) {
-	if strings.Contains(uri, masterTag+"-") {
-		return "", false
-	}
-	branchURI := strings.Replace(uri, latestTag+"-", masterTag+"-", 1)
-	if branchURI == uri {
-		return "", false
-	}
-	return branchURI, true
-}
-
 // backendCandidate represents an installed concrete backend option for a given alias
 type backendCandidate struct {
 	name    string
@@ -311,28 +295,15 @@ func InstallBackend(ctx context.Context, systemState *system.SystemState, modelL
 		return fmt.Errorf("backend %q: %w", config.Name, optsErr)
 	}
 
-	// PreferDevelopmentBackends installs the development image as the primary URI,
-	// keeping the released image reachable as the first fallback — instead of only
-	// reaching development when the released image is missing.
-	primaryURI := string(config.URI)
-	mirrors := config.Mirrors
-	if systemState.PreferDevelopmentBackends {
-		if devURI, ok := developmentURI(string(config.URI), latestTag, masterTag); ok {
-			xlog.Info("PreferDevelopmentBackends: installing development image first", "development", devURI, "released", config.URI)
-			primaryURI = devURI
-			mirrors = append([]string{string(config.URI)}, config.Mirrors...)
-		}
-	}
-
-	uri := downloader.URI(primaryURI)
+	uri := downloader.URI(config.URI)
 	// Check if it is a directory
 	if uri.LooksLikeDir() {
 		// It is a directory, we just copy it over in the backend folder
-		if err := cp.Copy(string(uri), backendPath); err != nil {
+		if err := cp.Copy(config.URI, backendPath); err != nil {
 			return fmt.Errorf("failed copying: %w", err)
 		}
 	} else {
-		xlog.Debug("Downloading backend", "uri", primaryURI, "backendPath", backendPath)
+		xlog.Debug("Downloading backend", "uri", config.URI, "backendPath", backendPath)
 		if err := uri.DownloadFileWithContext(ctx, backendPath, config.SHA256, 1, 1, downloadStatus, downloadOpts...); err != nil {
 			xlog.Debug("Backend download failed, trying fallback", "backendPath", backendPath, "error", err)
 
@@ -345,9 +316,8 @@ func InstallBackend(ctx context.Context, systemState *system.SystemState, modelL
 			}
 
 			success := false
-			// Try to download from mirrors (when development is preferred, the
-			// released image is prepended here as the first fallback).
-			for _, mirror := range mirrors {
+			// Try to download from mirrors
+			for _, mirror := range config.Mirrors {
 				// Check for cancellation before trying next mirror
 				select {
 				case <-ctx.Done():

core/gallery/backends_devuri_test.go

@@ -1,26 +0,0 @@
-package gallery
-
-import (
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-)
-
-var _ = Describe("developmentURI", func() {
-	const latest, master = "latest", "master"
-
-	It("rewrites a released image to its branch (development) image", func() {
-		got, ok := developmentURI("quay.io/go-skynet/local-ai-backends:latest-metal-darwin-arm64-llama-cpp", latest, master)
-		Expect(ok).To(BeTrue())
-		Expect(got).To(Equal("quay.io/go-skynet/local-ai-backends:master-metal-darwin-arm64-llama-cpp"))
-	})
-
-	It("leaves an image already on the branch tag untouched", func() {
-		_, ok := developmentURI("quay.io/go-skynet/local-ai-backends:master-metal-darwin-arm64-llama-cpp", latest, master)
-		Expect(ok).To(BeFalse())
-	})
-
-	It("returns ok=false when there is no released tag to swap", func() {
-		_, ok := developmentURI("oci://localhost/custom-backend:edge", latest, master)
-		Expect(ok).To(BeFalse())
-	})
-})

core/http/app.go

@@ -23,10 +23,8 @@ import (
 
 	"github.com/mudler/LocalAI/core/application"
 	"github.com/mudler/LocalAI/core/schema"
-	"github.com/mudler/LocalAI/core/services/distributed"
 	"github.com/mudler/LocalAI/core/services/finetune"
 	"github.com/mudler/LocalAI/core/services/galleryop"
-	"github.com/mudler/LocalAI/core/services/messaging"
 	"github.com/mudler/LocalAI/core/services/nodes"
 	"github.com/mudler/LocalAI/core/services/quantization"
 
@@ -402,45 +400,25 @@ func API(application *application.Application) (*echo.Echo, error) {
 	routes.RegisterAgentPoolRoutes(e, application, agentsMw, skillsMw, collectionsMw)
 	// Fine-tuning routes
 	fineTuningMw := auth.RequireFeature(application.AuthDB(), auth.FeatureFineTuning)
-	// In distributed mode pass the shared NATS client + PostgreSQL store so
-	// fine-tune jobs stay consistent across replicas (the SyncedMap broadcasts
-	// mutations and hydrates from the DB); standalone passes nil for both.
-	var ftNats messaging.MessagingClient
-	var ftStore *distributed.FineTuneStore
-	if d := application.Distributed(); d != nil {
-		ftNats = d.Nats
-		if d.DistStores != nil && d.DistStores.FineTune != nil {
-			ftStore = d.DistStores.FineTune
-		}
-	}
 	ftService := finetune.NewFineTuneService(
 		application.ApplicationConfig(),
 		application.ModelLoader(),
 		application.ModelConfigLoader(),
-		ftNats,
-		ftStore,
 	)
+	if d := application.Distributed(); d != nil {
+		ftService.SetNATSClient(d.Nats)
+		if d.DistStores != nil && d.DistStores.FineTune != nil {
+			ftService.SetFineTuneStore(d.DistStores.FineTune)
+		}
+	}
 	routes.RegisterFineTuningRoutes(e, ftService, application.ApplicationConfig(), fineTuningMw)
 
 	// Quantization routes
 	quantizationMw := auth.RequireFeature(application.AuthDB(), auth.FeatureQuantization)
-	// In distributed mode pass the shared NATS client + PostgreSQL store so
-	// quantization jobs stay consistent across replicas (the SyncedMap broadcasts
-	// mutations and hydrates from the DB); standalone passes nil for both.
-	var quantNats messaging.MessagingClient
-	var quantStore *distributed.QuantStore
-	if d := application.Distributed(); d != nil {
-		quantNats = d.Nats
-		if d.DistStores != nil && d.DistStores.Quant != nil {
-			quantStore = d.DistStores.Quant
-		}
-	}
 	qService := quantization.NewQuantizationService(
 		application.ApplicationConfig(),
 		application.ModelLoader(),
 		application.ModelConfigLoader(),
-		quantNats,
-		quantStore,
 	)
 	routes.RegisterQuantizationRoutes(e, qService, application.ApplicationConfig(), quantizationMw)
 

core/http/auth/db_sqlite.go

@@ -3,51 +3,10 @@
 package auth
 
 import (
-	"net/url"
-	"strings"
-
 	"gorm.io/driver/sqlite"
 	"gorm.io/gorm"
 )
 
 func openSQLiteDialector(path string) (gorm.Dialector, error) {
-	return sqlite.Open(buildSQLiteDSN(path)), nil
-}
-
-// buildSQLiteDSN augments a SQLite file path with connection pragmas that make
-// the auth DB resilient on slow or contended storage.
-//
-//   - _busy_timeout=5000 makes SQLite retry for up to 5s on SQLITE_BUSY instead
-//     of failing immediately. Network-backed storage (SMB/CIFS/NFS, e.g. Azure
-//     Files) is prone to transient lock contention during migration (see #10506).
-//   - _txlock=immediate takes the write lock at BEGIN, avoiding deadlocks when a
-//     read transaction later upgrades to a write during AutoMigrate.
-//
-// We deliberately do NOT set WAL journal mode: WAL relies on a shared-memory
-// mmap that does not work over SMB/NFS, which is exactly the failing case here.
-//
-// Caller-supplied values for either pragma are preserved.
-func buildSQLiteDSN(path string) string {
-	base := path
-	rawQuery := ""
-	if i := strings.IndexByte(path, '?'); i >= 0 {
-		base = path[:i]
-		rawQuery = path[i+1:]
-	}
-
-	values, err := url.ParseQuery(rawQuery)
-	if err != nil {
-		// An unparseable query string means a hand-crafted DSN we should not
-		// risk corrupting; leave it untouched.
-		return path
-	}
-
-	if values.Get("_busy_timeout") == "" {
-		values.Set("_busy_timeout", "5000")
-	}
-	if values.Get("_txlock") == "" {
-		values.Set("_txlock", "immediate")
-	}
-
-	return base + "?" + values.Encode()
+	return sqlite.Open(path), nil
 }

core/http/auth/db_sqlite_test.go

@@ -1,57 +0,0 @@
-//go:build auth
-
-package auth
-
-import (
-	"net/url"
-	"strings"
-
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-)
-
-// parseDSN splits a "base?query" DSN into its base and decoded query values so
-// assertions don't depend on url.Values.Encode()'s key ordering.
-func parseDSN(dsn string) (string, url.Values) {
-	base := dsn
-	rawQuery := ""
-	if i := strings.IndexByte(dsn, '?'); i >= 0 {
-		base = dsn[:i]
-		rawQuery = dsn[i+1:]
-	}
-	values, err := url.ParseQuery(rawQuery)
-	Expect(err).ToNot(HaveOccurred())
-	return base, values
-}
-
-var _ = Describe("buildSQLiteDSN", func() {
-	It("adds busy_timeout and txlock to a plain file path", func() {
-		base, values := parseDSN(buildSQLiteDSN("/data/database.db"))
-		Expect(base).To(Equal("/data/database.db"))
-		Expect(values.Get("_busy_timeout")).To(Equal("5000"))
-		Expect(values.Get("_txlock")).To(Equal("immediate"))
-	})
-
-	It("adds pragmas to an in-memory database", func() {
-		base, values := parseDSN(buildSQLiteDSN(":memory:"))
-		Expect(base).To(Equal(":memory:"))
-		Expect(values.Get("_busy_timeout")).To(Equal("5000"))
-		Expect(values.Get("_txlock")).To(Equal("immediate"))
-	})
-
-	It("preserves an existing query string", func() {
-		base, values := parseDSN(buildSQLiteDSN("/data/database.db?cache=shared"))
-		Expect(base).To(Equal("/data/database.db"))
-		Expect(values.Get("cache")).To(Equal("shared"))
-		Expect(values.Get("_busy_timeout")).To(Equal("5000"))
-		Expect(values.Get("_txlock")).To(Equal("immediate"))
-	})
-
-	It("does not override a caller-supplied busy_timeout or txlock", func() {
-		_, values := parseDSN(buildSQLiteDSN("/data/database.db?_busy_timeout=1000&_txlock=deferred"))
-		Expect(values["_busy_timeout"]).To(HaveLen(1), "_busy_timeout should not be duplicated")
-		Expect(values.Get("_busy_timeout")).To(Equal("1000"))
-		Expect(values["_txlock"]).To(HaveLen(1), "_txlock should not be duplicated")
-		Expect(values.Get("_txlock")).To(Equal("deferred"))
-	})
-})

core/http/endpoints/localai/config_meta.go

@@ -155,7 +155,7 @@ func AutocompleteEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoader, a
 // @Param name path string true "Model name"
 // @Success 200 {object} map[string]any "success message"
 // @Router /api/models/config-json/{name} [patch]
-func PatchConfigEndpoint(cl *config.ModelConfigLoader, _ *model.ModelLoader, gs *galleryop.GalleryService, appConfig *config.ApplicationConfig) echo.HandlerFunc {
+func PatchConfigEndpoint(cl *config.ModelConfigLoader, _ *model.ModelLoader, appConfig *config.ApplicationConfig) echo.HandlerFunc {
 	svc := modeladmin.NewConfigService(cl, appConfig)
 	return func(c echo.Context) error {
 		modelName := c.Param("name")
@@ -173,14 +173,6 @@ func PatchConfigEndpoint(cl *config.ModelConfigLoader, _ *model.ModelLoader, gs
 		if _, err := svc.PatchConfig(c.Request().Context(), modelName, patchMap); err != nil {
 			return c.JSON(httpStatusForModelAdminError(err), map[string]any{"error": err.Error()})
 		}
-
-		// Patch rewrites the config on disk and reloads only the local loader;
-		// tell peers to refresh so the change is consistent across replicas.
-		// No-op in standalone mode.
-		if gs != nil {
-			gs.BroadcastModelsChanged(modelName, "install")
-		}
-
 		return c.JSON(http.StatusOK, map[string]any{
 			"success": true,
 			"message": fmt.Sprintf("Model '%s' updated successfully", modelName),

core/http/endpoints/localai/config_meta_test.go

@@ -45,7 +45,7 @@ var _ = Describe("Config Metadata Endpoints", func() {
 		app = echo.New()
 		app.GET("/api/models/config-metadata", ConfigMetadataEndpoint())
 		app.GET("/api/models/config-metadata/autocomplete/:provider", AutocompleteEndpoint(configLoader, modelLoader, appConfig))
-		app.PATCH("/api/models/config-json/:name", PatchConfigEndpoint(configLoader, modelLoader, nil, appConfig))
+		app.PATCH("/api/models/config-json/:name", PatchConfigEndpoint(configLoader, modelLoader, appConfig))
 	})
 
 	AfterEach(func() {

core/http/endpoints/localai/edit_model.go

@@ -10,7 +10,6 @@ import (
 	"github.com/labstack/echo/v4"
 	"github.com/mudler/LocalAI/core/config"
 	httpUtils "github.com/mudler/LocalAI/core/http/middleware"
-	"github.com/mudler/LocalAI/core/services/galleryop"
 	"github.com/mudler/LocalAI/core/services/modeladmin"
 	"github.com/mudler/LocalAI/internal"
 	"github.com/mudler/LocalAI/pkg/model"
@@ -56,7 +55,7 @@ func GetEditModelPage(cl *config.ModelConfigLoader, appConfig *config.Applicatio
 }
 
 // EditModelEndpoint handles updating existing model configurations
-func EditModelEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoader, gs *galleryop.GalleryService, appConfig *config.ApplicationConfig) echo.HandlerFunc {
+func EditModelEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoader, appConfig *config.ApplicationConfig) echo.HandlerFunc {
 	svc := modeladmin.NewConfigService(cl, appConfig)
 	return func(c echo.Context) error {
 		modelName := c.Param("name")
@@ -71,17 +70,6 @@ func EditModelEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoader, gs *
 		if err != nil {
 			return c.JSON(httpStatusForModelAdminError(err), ModelResponse{Success: false, Error: err.Error()})
 		}
-
-		// Tell peer replicas to refresh their in-memory config: this endpoint
-		// only reloaded the local loader. A rename is a delete of the old name
-		// plus an install of the new one. No-op in standalone mode.
-		if gs != nil {
-			if result.Renamed {
-				gs.BroadcastModelsChanged(result.OldName, "delete")
-			}
-			gs.BroadcastModelsChanged(result.NewName, "install")
-		}
-
 		msg := fmt.Sprintf("Model '%s' updated successfully. Model has been reloaded with new configuration.", result.NewName)
 		if result.Renamed {
 			msg = fmt.Sprintf("Model '%s' renamed to '%s' and updated successfully.", result.OldName, result.NewName)

core/http/endpoints/localai/edit_model_test.go

@@ -56,7 +56,7 @@ var _ = Describe("Edit Model test", func() {
 			app := echo.New()
 			// Set up a simple renderer for the test
 			app.Renderer = &testRenderer{}
-			app.POST("/import-model", ImportModelEndpoint(modelConfigLoader, nil, applicationConfig))
+			app.POST("/import-model", ImportModelEndpoint(modelConfigLoader, applicationConfig))
 			app.GET("/edit-model/:name", GetEditModelPage(modelConfigLoader, applicationConfig))
 
 			requestBody := bytes.NewBufferString(`{"name": "foo", "backend": "foo", "model": "foo"}`)
@@ -106,7 +106,7 @@ var _ = Describe("Edit Model test", func() {
 			Expect(exists).To(BeTrue())
 
 			app := echo.New()
-			app.POST("/models/edit/:name", EditModelEndpoint(modelConfigLoader, modelLoader, nil, applicationConfig))
+			app.POST("/models/edit/:name", EditModelEndpoint(modelConfigLoader, modelLoader, applicationConfig))
 
 			newYAML := "name: newname\nbackend: llama\nmodel: foo\n"
 			req := httptest.NewRequest("POST", "/models/edit/oldname", bytes.NewBufferString(newYAML))
@@ -163,7 +163,7 @@ var _ = Describe("Edit Model test", func() {
 			Expect(modelConfigLoader.LoadModelConfigsFromPath(tempDir)).To(Succeed())
 
 			app := echo.New()
-			app.POST("/models/edit/:name", EditModelEndpoint(modelConfigLoader, modelLoader, nil, applicationConfig))
+			app.POST("/models/edit/:name", EditModelEndpoint(modelConfigLoader, modelLoader, applicationConfig))
 
 			req := httptest.NewRequest(
 				"POST",
@@ -204,7 +204,7 @@ var _ = Describe("Edit Model test", func() {
 			Expect(modelConfigLoader.LoadModelConfigsFromPath(tempDir)).To(Succeed())
 
 			app := echo.New()
-			app.POST("/models/edit/:name", EditModelEndpoint(modelConfigLoader, modelLoader, nil, applicationConfig))
+			app.POST("/models/edit/:name", EditModelEndpoint(modelConfigLoader, modelLoader, applicationConfig))
 
 			req := httptest.NewRequest(
 				"POST",

core/http/endpoints/localai/import_model.go

@@ -125,7 +125,7 @@ func ImportModelURIEndpoint(cl *config.ModelConfigLoader, appConfig *config.Appl
 }
 
 // ImportModelEndpoint handles creating new model configurations
-func ImportModelEndpoint(cl *config.ModelConfigLoader, gs *galleryop.GalleryService, appConfig *config.ApplicationConfig) echo.HandlerFunc {
+func ImportModelEndpoint(cl *config.ModelConfigLoader, appConfig *config.ApplicationConfig) echo.HandlerFunc {
 	return func(c echo.Context) error {
 		// Get the raw body
 		body, err := io.ReadAll(c.Request().Body)
@@ -245,13 +245,6 @@ func ImportModelEndpoint(cl *config.ModelConfigLoader, gs *galleryop.GalleryServ
 			}
 			return c.JSON(http.StatusInternalServerError, response)
 		}
-		// Tell peer replicas to load the newly-created config from the shared
-		// models dir: this endpoint only reloaded the local loader. No-op in
-		// standalone mode.
-		if gs != nil {
-			gs.BroadcastModelsChanged(modelConfig.Name, "install")
-		}
-
 		// Return success response
 		response := ModelResponse{
 			Success:  true,

core/http/endpoints/localai/nodes.go

@@ -60,10 +60,7 @@ func GetNodeEndpoint(registry *nodes.NodeRegistry) echo.HandlerFunc {
 	return func(c echo.Context) error {
 		ctx := c.Request().Context()
 		id := c.Param("id")
-		// GetWithExtras (not Get) so the response carries the node's labels,
-		// loaded-model count, and in-flight total — the bare BackendNode keeps
-		// labels in a separate table, leaving the detail view's label list empty.
-		node, err := registry.GetWithExtras(ctx, id)
+		node, err := registry.Get(ctx, id)
 		if err != nil {
 			return c.JSON(http.StatusNotFound, nodeError(http.StatusNotFound, "node not found"))
 		}

core/http/endpoints/localai/toggle_model.go

@@ -7,7 +7,6 @@ import (
 
 	"github.com/labstack/echo/v4"
 	"github.com/mudler/LocalAI/core/config"
-	"github.com/mudler/LocalAI/core/services/galleryop"
 	"github.com/mudler/LocalAI/core/services/modeladmin"
 	"github.com/mudler/LocalAI/pkg/model"
 )
@@ -25,7 +24,7 @@ import (
 // @Failure      404  {object}  ModelResponse
 // @Failure      500  {object}  ModelResponse
 // @Router       /api/models/{name}/{action} [put]
-func ToggleStateModelEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoader, gs *galleryop.GalleryService, appConfig *config.ApplicationConfig) echo.HandlerFunc {
+func ToggleStateModelEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoader, appConfig *config.ApplicationConfig) echo.HandlerFunc {
 	svc := modeladmin.NewConfigService(cl, appConfig)
 	return func(c echo.Context) error {
 		modelName := c.Param("name")
@@ -37,14 +36,6 @@ func ToggleStateModelEndpoint(cl *config.ModelConfigLoader, ml *model.ModelLoade
 		if err != nil {
 			return c.JSON(httpStatusForModelAdminError(err), ModelResponse{Success: false, Error: err.Error()})
 		}
-
-		// Enabling/disabling rewrites the config on disk and reloads only the
-		// local loader; tell peers to refresh so the model's availability is
-		// consistent across replicas. No-op in standalone mode.
-		if gs != nil {
-			gs.BroadcastModelsChanged(modelName, "install")
-		}
-
 		msg := fmt.Sprintf("Model '%s' has been %sd successfully.", modelName, action)
 		if action == modeladmin.ActionDisable {
 			msg += " The model will not be loaded on demand until re-enabled."

core/http/routes/localai.go

@@ -72,19 +72,19 @@ func RegisterLocalAIRoutes(router *echo.Echo,
 		router.POST("/backends/upgrades/check", backendGalleryEndpointService.CheckUpgradesEndpoint(), adminMiddleware)
 		router.POST("/backends/upgrade/:name", backendGalleryEndpointService.UpgradeBackendEndpoint(), adminMiddleware)
 		// Custom model import endpoint
-		router.POST("/models/import", localai.ImportModelEndpoint(cl, galleryService, appConfig), adminMiddleware)
+		router.POST("/models/import", localai.ImportModelEndpoint(cl, appConfig), adminMiddleware)
 
 		// URI model import endpoint
 		router.POST("/models/import-uri", localai.ImportModelURIEndpoint(cl, appConfig, galleryService, opcache), adminMiddleware)
 
 		// Custom model edit endpoint
-		router.POST("/models/edit/:name", localai.EditModelEndpoint(cl, ml, galleryService, appConfig), adminMiddleware)
+		router.POST("/models/edit/:name", localai.EditModelEndpoint(cl, ml, appConfig), adminMiddleware)
 
 		// List model aliases endpoint
 		router.GET("/api/aliases", localai.ListAliasesEndpoint(cl), adminMiddleware)
 
 		// Toggle model enable/disable endpoint
-		router.PUT("/models/toggle-state/:name/:action", localai.ToggleStateModelEndpoint(cl, ml, galleryService, appConfig), adminMiddleware)
+		router.PUT("/models/toggle-state/:name/:action", localai.ToggleStateModelEndpoint(cl, ml, appConfig), adminMiddleware)
 
 		// Toggle model pinned status endpoint
 		router.PUT("/models/toggle-pinned/:name/:action", localai.TogglePinnedModelEndpoint(cl, appConfig, func() {

core/http/routes/ui_api.go

@@ -922,7 +922,7 @@ func RegisterUIAPIRoutes(app *echo.Echo, cl *config.ModelConfigLoader, ml *model
 	app.GET("/api/models/config-metadata/autocomplete/:provider", localai.AutocompleteEndpoint(cl, ml, appConfig), adminMiddleware)
 
 	// PATCH config endpoint - partial update using nested JSON merge
-	app.PATCH("/api/models/config-json/:name", localai.PatchConfigEndpoint(cl, ml, galleryService, appConfig), adminMiddleware)
+	app.PATCH("/api/models/config-json/:name", localai.PatchConfigEndpoint(cl, ml, appConfig), adminMiddleware)
 
 	// VRAM estimation endpoint
 	app.POST("/api/models/vram-estimate", localai.VRAMEstimateEndpoint(cl, appConfig), adminMiddleware)

core/services/advisorylock/advisorylock.go

@@ -4,59 +4,14 @@ import (
 	"context"
 	"fmt"
 	"hash/fnv"
-	"strings"
-	"sync"
 
 	"gorm.io/gorm"
 )
 
-// localLocks holds one buffered channel (capacity 1) per lock key, used as an
-// in-process mutex for non-PostgreSQL dialects (SQLite). A SQLite auth DB is
-// effectively single-process, so serializing guarded sections within this
-// process is sufficient - we cannot and need not coordinate across processes
-// the way a PostgreSQL advisory lock does.
-var (
-	localLocksMu sync.Mutex
-	localLocks   = map[int64]chan struct{}{}
-)
-
-// localLockChan returns the per-key buffered channel, creating it on first use.
-func localLockChan(key int64) chan struct{} {
-	localLocksMu.Lock()
-	defer localLocksMu.Unlock()
-	ch, ok := localLocks[key]
-	if !ok {
-		ch = make(chan struct{}, 1)
-		localLocks[key] = ch
-	}
-	return ch
-}
-
-// isPostgres reports whether the gorm dialect is PostgreSQL. Anything else
-// (SQLite and any non-postgres dialect) uses the in-process fallback, because
-// the pg_* advisory lock functions only exist on PostgreSQL.
-func isPostgres(db *gorm.DB) bool {
-	return strings.Contains(db.Dialector.Name(), "postgres")
-}
-
-// TryWithLockCtx attempts to acquire a lock and run fn without blocking.
-// Returns (true, nil) if the lock was acquired and fn executed, (false, nil) if
-// the lock was already held, or (false, error) on failure.
-//
-// On PostgreSQL it uses pg_try_advisory_lock (cross-process). On other dialects
-// (SQLite) it uses a non-blocking in-process lock keyed by key.
+// TryWithLockCtx attempts to acquire a PostgreSQL advisory lock using the provided context.
+// Returns (true, nil) if the lock was acquired and fn executed, (false, nil) if the lock
+// was already held, or (false, error) on failure.
 func TryWithLockCtx(ctx context.Context, db *gorm.DB, key int64, fn func() error) (bool, error) {
-	if !isPostgres(db) {
-		ch := localLockChan(key)
-		select {
-		case ch <- struct{}{}:
-			defer func() { <-ch }()
-			return true, fn()
-		default:
-			return false, nil
-		}
-	}
-
 	sqlDB, err := db.DB()
 	if err != nil {
 		return false, fmt.Errorf("get sql.DB: %w", err)
@@ -95,31 +50,9 @@ func KeyFromString(s string) int64 {
 	return int64(h.Sum64()>>1) | 0x100000000
 }
 
-// WithLockCtx acquires a lock for key, runs fn, then releases it, respecting
-// context cancellation. If ctx is cancelled while waiting for the lock, the
-// function returns ctx.Err().
-//
-// On PostgreSQL it uses pg_advisory_lock (cross-process). On other dialects
-// (SQLite) it falls back to a blocking in-process lock keyed by key, which is
-// sufficient because a SQLite auth DB is effectively single-process.
+// WithLockCtx is like WithLock but respects context cancellation.
+// If ctx is cancelled while waiting for the lock, the function returns ctx.Err().
 func WithLockCtx(ctx context.Context, db *gorm.DB, key int64, fn func() error) error {
-	if !isPostgres(db) {
-		// Honor an already-cancelled context before attempting acquisition:
-		// select picks a ready case at random, so without this an already-free
-		// lock could be taken despite a cancelled ctx.
-		if err := ctx.Err(); err != nil {
-			return err
-		}
-		ch := localLockChan(key)
-		select {
-		case ch <- struct{}{}:
-			defer func() { <-ch }()
-			return fn()
-		case <-ctx.Done():
-			return ctx.Err()
-		}
-	}
-
 	sqlDB, err := db.DB()
 	if err != nil {
 		return fmt.Errorf("advisorylock: getting sql.DB: %w", err)

core/services/advisorylock/advisorylock_sqlite_test.go

@@ -1,129 +0,0 @@
-package advisorylock
-
-import (
-	"context"
-	"sync"
-	"sync/atomic"
-	"time"
-
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-
-	"gorm.io/driver/sqlite"
-	"gorm.io/gorm"
-)
-
-// These specs run against an in-memory SQLite DB and therefore do NOT require
-// Docker, unlike the PostgreSQL testcontainer specs.
-var _ = Describe("AdvisoryLock (SQLite fallback)", Label("sqlite"), func() {
-	var db *gorm.DB
-
-	BeforeEach(func() {
-		var err error
-		db, err = gorm.Open(sqlite.Open("file::memory:?cache=shared"), &gorm.Config{})
-		Expect(err).ToNot(HaveOccurred())
-		Expect(db.Dialector.Name()).To(ContainSubstring("sqlite"))
-	})
-
-	It("WithLockCtx executes fn and returns no error on SQLite", func() {
-		const lockKey int64 = 12001
-		executed := false
-
-		err := WithLockCtx(context.Background(), db, lockKey, func() error {
-			executed = true
-			return nil
-		})
-		Expect(err).ToNot(HaveOccurred())
-		Expect(executed).To(BeTrue(), "function should have run under the in-process lock")
-	})
-
-	It("WithLockCtx serializes concurrent goroutines on the same key", func() {
-		const lockKey int64 = 12002
-
-		var (
-			mu          sync.Mutex
-			maxRunning  int32
-			running     int32
-			concurrency int32
-		)
-
-		var wg sync.WaitGroup
-
-		for range 2 {
-			wg.Go(func() {
-				defer GinkgoRecover()
-				err := WithLockCtx(context.Background(), db, lockKey, func() error {
-					cur := atomic.AddInt32(&running, 1)
-					mu.Lock()
-					if cur > maxRunning {
-						maxRunning = cur
-					}
-					if cur > 1 {
-						atomic.AddInt32(&concurrency, 1)
-					}
-					mu.Unlock()
-
-					time.Sleep(50 * time.Millisecond)
-
-					atomic.AddInt32(&running, -1)
-					return nil
-				})
-				Expect(err).ToNot(HaveOccurred())
-			})
-		}
-
-		wg.Wait()
-
-		Expect(maxRunning).To(BeNumerically("<=", 1), "expected max 1 goroutine inside lock at a time")
-		Expect(concurrency).To(BeZero(), "detected concurrent execution inside advisory lock")
-	})
-
-	It("WithLockCtx returns an error and does not run fn with an already-cancelled context", func() {
-		const lockKey int64 = 12003
-		ctx, cancel := context.WithCancel(context.Background())
-		cancel()
-
-		err := WithLockCtx(ctx, db, lockKey, func() error {
-			Fail("function should not run with a cancelled context")
-			return nil
-		})
-		Expect(err).To(HaveOccurred())
-	})
-
-	It("TryWithLockCtx returns (true, nil) when free and (false, nil) when held", func() {
-		const lockKey int64 = 12004
-
-		acquired, err := TryWithLockCtx(context.Background(), db, lockKey, func() error {
-			return nil
-		})
-		Expect(err).ToNot(HaveOccurred())
-		Expect(acquired).To(BeTrue(), "expected TryWithLockCtx to acquire the free lock")
-
-		// Hold the lock in one goroutine while a concurrent TryWithLockCtx
-		// attempts to acquire the same key.
-		held := make(chan struct{})
-		release := make(chan struct{})
-		var wg sync.WaitGroup
-		wg.Go(func() {
-			defer GinkgoRecover()
-			ok, err := TryWithLockCtx(context.Background(), db, lockKey, func() error {
-				close(held)
-				<-release
-				return nil
-			})
-			Expect(err).ToNot(HaveOccurred())
-			Expect(ok).To(BeTrue())
-		})
-
-		<-held
-		ok, err := TryWithLockCtx(context.Background(), db, lockKey, func() error {
-			Fail("function should not run while lock is held")
-			return nil
-		})
-		Expect(err).ToNot(HaveOccurred())
-		Expect(ok).To(BeFalse(), "expected TryWithLockCtx to fail to acquire a held lock")
-
-		close(release)
-		wg.Wait()
-	})
-})

core/services/agentpool/agent_jobs.go

@@ -30,8 +30,6 @@ import (
 	mcpTools "github.com/mudler/LocalAI/core/http/endpoints/mcp"
 	"github.com/mudler/LocalAI/core/schema"
 	"github.com/mudler/LocalAI/core/services/jobs"
-	"github.com/mudler/LocalAI/core/services/messaging"
-	"github.com/mudler/LocalAI/core/services/syncstate"
 	"github.com/mudler/LocalAI/core/templates"
 	"github.com/mudler/LocalAI/pkg/httpclient"
 	"github.com/mudler/LocalAI/pkg/model"
@@ -45,18 +43,8 @@ type AgentJobService struct {
 	configLoader *config.ModelConfigLoader
 	evaluator    *templates.Evaluator
 
-	// tasks is the cross-replica task store: an in-memory map kept consistent
-	// across replicas via NATS, with read-through to the configured persister
-	// (file in standalone, PostgreSQL in distributed). Unlike jobs - which already
-	// converge via the dispatcher + DB read-through - tasks previously read
-	// in-memory only, so ListTasks went stale on non-originating replicas.
-	tasks *syncstate.SyncedMap[string, schema.Task]
-	// taskNats is the distributed NATS client backing the tasks SyncedMap. It is
-	// not available at construction time, so it is injected via SetTaskSyncNATS
-	// during distributed wiring; nil keeps tasks in-memory-only (standalone).
-	taskNats messaging.MessagingClient
-
 	// Storage (in-memory primary, persister for secondary persistence)
+	tasks     *xsync.SyncedMap[string, schema.Task]
 	jobs      *xsync.SyncedMap[string, schema.Job]
 	persister JobPersister
 	userID    string // Scoping: empty for global (main service), set for per-user instances
@@ -108,31 +96,6 @@ func (s *AgentJobService) SetDistributedJobStore(store *jobs.JobStore) {
 	s.persister = &dbJobPersister{store: store}
 }
 
-// SetTaskSyncNATS wires the distributed NATS client used to keep agent *tasks*
-// consistent across replicas (jobs already converge via the dispatcher + DB
-// read-through, so they are left untouched). The client is not available when the
-// service is constructed, so it is injected here during distributed wiring and the
-// tasks SyncedMap is rebuilt to pick it up. It is always called before Start /
-// hydrate, while the map is still empty, so rebuilding loses no state. Passing nil
-// (standalone) keeps the map in-memory-only with no broadcast.
-func (s *AgentJobService) SetTaskSyncNATS(nats messaging.MessagingClient) {
-	s.taskNats = nats
-	s.buildTasksMap()
-}
-
-// buildTasksMap (re)constructs the cross-replica tasks SyncedMap from the current
-// taskNats. The Store adapter reads s.persister/s.userID live, so a persister swap
-// (SetDistributedJobStore) needs no rebuild; only the NATS client, fixed at
-// New-time, forces one - hence SetTaskSyncNATS calls this.
-func (s *AgentJobService) buildTasksMap() {
-	s.tasks = syncstate.New(syncstate.Config[string, schema.Task]{
-		Name:  "agent.tasks",
-		Key:   func(t schema.Task) string { return t.ID },
-		Nats:  s.taskNats,
-		Store: &taskStoreAdapter{svc: s},
-	})
-}
-
 // Dispatcher returns the distributed dispatcher (nil if not in distributed mode).
 func (s *AgentJobService) Dispatcher() DistributedDispatcher {
 	return s.dispatcher
@@ -143,6 +106,13 @@ func (s *AgentJobService) DBStore() *jobs.JobStore {
 	return s.rawDBStore
 }
 
+// saveTasks persists tasks via the configured persister (file or DB).
+func (s *AgentJobService) saveTasks(task schema.Task) {
+	if err := s.persister.SaveTask(s.userID, task); err != nil {
+		xlog.Warn("Failed to persist task", "error", err, "task_id", task.ID)
+	}
+}
+
 // saveJobs persists jobs via the configured persister (file or DB).
 func (s *AgentJobService) saveJobs(job schema.Job) {
 	if err := s.persister.SaveJob(s.userID, job); err != nil {
@@ -159,8 +129,18 @@ func (s *AgentJobService) LoadFromDB() {
 
 // loadFromPersister loads tasks and jobs from the configured persister into memory.
 func (s *AgentJobService) loadFromPersister() {
-	if err := s.hydrateTasks(s.appConfig.Context); err != nil {
+	if tasks, err := s.persister.LoadTasks(s.userID); err != nil {
 		xlog.Warn("Failed to load tasks from persister", "error", err)
+	} else {
+		for _, task := range tasks {
+			s.tasks.Set(task.ID, task)
+			if task.Enabled && task.Cron != "" {
+				if err := s.ScheduleCronTask(task); err != nil {
+					xlog.Warn("Failed to schedule cron task on load", "error", err, "task_id", task.ID)
+				}
+			}
+		}
+		xlog.Info("Loaded tasks from persister", "count", len(tasks))
 	}
 
 	if loadedJobs, err := s.persister.LoadJobs(s.userID); err != nil {
@@ -173,27 +153,6 @@ func (s *AgentJobService) loadFromPersister() {
 	}
 }
 
-// hydrateTasks loads tasks into the cross-replica SyncedMap and (re)schedules
-// cron entries for enabled tasks. Hydration goes through the SyncedMap's Store
-// read-through (Start), not Set, so it neither re-persists nor re-broadcasts the
-// loaded tasks. Each service instance hydrates exactly once: the main service via
-// Start -> loadFromPersister, per-user services via LoadFromDB or LoadTasksFromFile.
-func (s *AgentJobService) hydrateTasks(ctx context.Context) error {
-	if err := s.tasks.Start(ctx); err != nil {
-		return err
-	}
-	tasks := s.tasks.List()
-	for _, task := range tasks {
-		if task.Enabled && task.Cron != "" {
-			if err := s.ScheduleCronTask(task); err != nil {
-				xlog.Warn("Failed to schedule cron task on load", "error", err, "task_id", task.ID)
-			}
-		}
-	}
-	xlog.Info("Loaded tasks from persister", "count", len(tasks))
-	return nil
-}
-
 // JobExecution represents a job to be executed
 type JobExecution struct {
 	Job    schema.Job
@@ -241,19 +200,21 @@ func NewAgentJobServiceWithPaths(
 ) *AgentJobService {
 	retentionDays := cmp.Or(appConfig.AgentJobRetentionDays, 30)
 
+	tasks := xsync.NewSyncedMap[string, schema.Task]()
 	jobsMap := xsync.NewSyncedMap[string, schema.Job]()
 
-	s := &AgentJobService{
+	return &AgentJobService{
 		appConfig:    appConfig,
 		modelLoader:  modelLoader,
 		configLoader: configLoader,
 		evaluator:    evaluator,
+		tasks:        tasks,
 		jobs:         jobsMap,
 		persister: &fileJobPersister{
+			tasks:     tasks,
 			jobs:      jobsMap,
 			tasksFile: tasksFile,
 			jobsFile:  jobsFile,
-			taskSet:   make(map[string]schema.Task),
 		},
 		jobQueue:      make(chan JobExecution, 100), // Buffer for 100 jobs
 		cancellations: xsync.NewSyncedMap[string, context.CancelFunc](),
@@ -261,17 +222,25 @@ func NewAgentJobServiceWithPaths(
 		cronEntries:   xsync.NewSyncedMap[string, cron.EntryID](),
 		retentionDays: retentionDays,
 	}
-	// Build the cross-replica tasks map standalone (nil NATS); SetTaskSyncNATS
-	// rebuilds it with the distributed client once that is available, before Start.
-	s.buildTasksMap()
-	return s
 }
 
 // LoadTasksFromFile loads tasks from the persister into the in-memory map
 // and schedules cron entries. Named "FromFile" for backward compat; in DB
 // mode it loads from the database.
 func (s *AgentJobService) LoadTasksFromFile() error {
-	return s.hydrateTasks(s.appConfig.Context)
+	tasks, err := s.persister.LoadTasks(s.userID)
+	if err != nil {
+		return err
+	}
+	for _, task := range tasks {
+		s.tasks.Set(task.ID, task)
+		if task.Enabled && task.Cron != "" {
+			if err := s.ScheduleCronTask(task); err != nil {
+				xlog.Warn("Failed to schedule cron task on load", "error", err, "task_id", task.ID)
+			}
+		}
+	}
+	return nil
 }
 
 // SaveTasksToFile flushes the current tasks map via the persister. File
@@ -324,12 +293,8 @@ func (s *AgentJobService) CreateTask(task schema.Task) (string, error) {
 		task.Enabled = true // Default to enabled
 	}
 
-	// Store task: Set updates the in-memory map, write-throughs to the persister
-	// (file or DB), and broadcasts the create to peer replicas. Background ctx
-	// because CreateTask carries no request ctx (mirrors the finetune service).
-	if err := s.tasks.Set(context.Background(), task); err != nil {
-		return "", fmt.Errorf("failed to persist task: %w", err)
-	}
+	// Store task
+	s.tasks.Set(id, task)
 
 	// Schedule cron if enabled and has cron expression
 	if task.Enabled && task.Cron != "" {
@@ -338,15 +303,16 @@ func (s *AgentJobService) CreateTask(task schema.Task) (string, error) {
 		}
 	}
 
+	s.saveTasks(task)
 	return id, nil
 }
 
 // UpdateTask updates an existing task
 func (s *AgentJobService) UpdateTask(id string, task schema.Task) error {
-	existing, ok := s.tasks.Get(id)
-	if !ok {
+	if !s.tasks.Exists(id) {
 		return fmt.Errorf("%w: %s", ErrTaskNotFound, id)
 	}
+	existing := s.tasks.Get(id)
 
 	// Preserve ID and CreatedAt
 	task.ID = id
@@ -358,10 +324,8 @@ func (s *AgentJobService) UpdateTask(id string, task schema.Task) error {
 		s.UnscheduleCronTask(id)
 	}
 
-	// Store updated task: write-through + broadcast (see CreateTask).
-	if err := s.tasks.Set(context.Background(), task); err != nil {
-		return fmt.Errorf("failed to persist task: %w", err)
-	}
+	// Store updated task
+	s.tasks.Set(id, task)
 
 	// Schedule new cron if enabled and has cron expression
 	if task.Enabled && task.Cron != "" {
@@ -370,22 +334,24 @@ func (s *AgentJobService) UpdateTask(id string, task schema.Task) error {
 		}
 	}
 
+	s.saveTasks(task)
 	return nil
 }
 
 // DeleteTask deletes a task
 func (s *AgentJobService) DeleteTask(id string) error {
-	if _, ok := s.tasks.Get(id); !ok {
+	if !s.tasks.Exists(id) {
 		return fmt.Errorf("%w: %s", ErrTaskNotFound, id)
 	}
 
 	// Unschedule cron
 	s.UnscheduleCronTask(id)
 
-	// Delete removes from the in-memory map, deletes from the persister, and
-	// broadcasts the removal to peer replicas.
-	if err := s.tasks.Delete(context.Background(), id); err != nil {
-		xlog.Warn("Failed to delete task from store", "error", err, "task_id", id)
+	// Remove from memory
+	s.tasks.Delete(id)
+
+	if err := s.persister.DeleteTask(id); err != nil {
+		xlog.Warn("Failed to delete task from persister", "error", err, "task_id", id)
 	}
 
 	return nil
@@ -393,8 +359,8 @@ func (s *AgentJobService) DeleteTask(id string) error {
 
 // GetTask retrieves a task by ID
 func (s *AgentJobService) GetTask(id string) (*schema.Task, error) {
-	task, ok := s.tasks.Get(id)
-	if !ok {
+	task := s.tasks.Get(id)
+	if task.ID == "" {
 		return nil, fmt.Errorf("%w: %s", ErrTaskNotFound, id)
 	}
 	return &task, nil
@@ -402,7 +368,7 @@ func (s *AgentJobService) GetTask(id string) (*schema.Task, error) {
 
 // ListTasks returns all tasks, sorted by creation date (newest first)
 func (s *AgentJobService) ListTasks() []schema.Task {
-	tasks := s.tasks.List()
+	tasks := s.tasks.Values()
 	// Sort by CreatedAt descending (newest first), then by Name for stability
 	slices.SortFunc(tasks, func(a, b schema.Task) int {
 		if a.CreatedAt.Equal(b.CreatedAt) {
@@ -431,8 +397,8 @@ func (s *AgentJobService) buildPrompt(templateStr string, params map[string]stri
 // ExecuteJob creates and queues a job for execution
 // multimedia can be nil for backward compatibility
 func (s *AgentJobService) ExecuteJob(taskID string, params map[string]string, triggeredBy string, multimedia *schema.MultimediaAttachment) (string, error) {
-	task, ok := s.tasks.Get(taskID)
-	if !ok {
+	task := s.tasks.Get(taskID)
+	if task.ID == "" {
 		return "", fmt.Errorf("%w: %s", ErrTaskNotFound, taskID)
 	}
 
@@ -1485,12 +1451,6 @@ func (s *AgentJobService) Stop() error {
 	if s.cronScheduler != nil {
 		s.cronScheduler.Stop()
 	}
-	// Release the tasks SyncedMap subscription / background workers.
-	if s.tasks != nil {
-		if err := s.tasks.Close(); err != nil {
-			xlog.Warn("Error closing tasks sync map", "error", err)
-		}
-	}
 	xlog.Info("AgentJobService stopped")
 	return nil
 }

core/services/agentpool/job_persister_file.go

@@ -14,38 +14,24 @@ import (
 )
 
 // fileJobPersister persists tasks and jobs to JSON files.
-//
-// Jobs serialize the service's in-memory jobs syncmap on each save (bulk write).
-// Tasks are kept in this persister's own taskSet map instead: the tasks SyncedMap
-// calls SaveTask/DeleteTask while holding its internal lock (write-through), so
-// reading back the SyncedMap here would re-enter that lock and deadlock. The
-// self-contained taskSet, seeded by LoadTasks, lets a per-task write rewrite the
-// whole bulk file without touching the SyncedMap.
-//
-// Runtime reads (GetJob/ListJobs) return nil (the in-memory state is the
-// authoritative source); LoadTasks/LoadJobs bootstrap state at startup.
+// It holds references to the service's syncmaps and serializes the entire
+// map contents on each save (bulk write). Reads at runtime return nil
+// (the in-memory map is the authoritative source); LoadTasks/LoadJobs
+// are used only at startup to bootstrap the syncmaps.
 type fileJobPersister struct {
+	tasks     *xsync.SyncedMap[string, schema.Task]
 	jobs      *xsync.SyncedMap[string, schema.Job]
 	tasksFile string
 	jobsFile  string
 	mu        sync.Mutex
-	// taskSet is the persister's own view of all tasks, seeded by LoadTasks and
-	// updated by SaveTask/DeleteTask. The bulk JSON file is rewritten from it.
-	taskSet map[string]schema.Task
 }
 
-func (p *fileJobPersister) SaveTask(_ string, task schema.Task) error {
-	p.mu.Lock()
-	defer p.mu.Unlock()
-	p.taskSet[task.ID] = task
-	return p.writeTasksLocked()
+func (p *fileJobPersister) SaveTask(_ string, _ schema.Task) error {
+	return p.saveTasksToFile()
 }
 
-func (p *fileJobPersister) DeleteTask(taskID string) error {
-	p.mu.Lock()
-	defer p.mu.Unlock()
-	delete(p.taskSet, taskID)
-	return p.writeTasksLocked()
+func (p *fileJobPersister) DeleteTask(_ string) error {
+	return p.saveTasksToFile()
 }
 
 func (p *fileJobPersister) SaveJob(_ string, _ schema.Job) error {
@@ -57,9 +43,7 @@ func (p *fileJobPersister) DeleteJob(_ string) error {
 }
 
 func (p *fileJobPersister) FlushTasks() error {
-	p.mu.Lock()
-	defer p.mu.Unlock()
-	return p.writeTasksLocked()
+	return p.saveTasksToFile()
 }
 
 func (p *fileJobPersister) FlushJobs() error {
@@ -99,12 +83,6 @@ func (p *fileJobPersister) LoadTasks(_ string) ([]schema.Task, error) {
 		return nil, fmt.Errorf("failed to parse tasks file: %w", err)
 	}
 
-	// Seed the in-memory set so subsequent per-task SaveTask/DeleteTask merge into
-	// (rather than overwrite) the persisted tasks when the bulk file is rewritten.
-	for _, t := range tf.Tasks {
-		p.taskSet[t.ID] = t
-	}
-
 	xlog.Info("Loaded tasks from file", "count", len(tf.Tasks))
 	return tf.Tasks, nil
 }
@@ -140,19 +118,18 @@ func (p *fileJobPersister) CleanupOldJobs(_ time.Duration) (int64, error) {
 	return 0, nil // cleanup handled via in-memory filtering
 }
 
-// writeTasksLocked serializes the persister's task set to the JSON file. Callers
-// must hold p.mu.
-func (p *fileJobPersister) writeTasksLocked() error {
+// saveTasksToFile serializes the entire tasks map to the JSON file.
+func (p *fileJobPersister) saveTasksToFile() error {
 	if p.tasksFile == "" {
 		return nil
 	}
 
-	tasks := make([]schema.Task, 0, len(p.taskSet))
-	for _, t := range p.taskSet {
-		tasks = append(tasks, t)
-	}
+	p.mu.Lock()
+	defer p.mu.Unlock()
 
-	tf := schema.TasksFile{Tasks: tasks}
+	tf := schema.TasksFile{
+		Tasks: p.tasks.Values(),
+	}
 
 	data, err := json.MarshalIndent(tf, "", "  ")
 	if err != nil {

core/services/agentpool/job_persister_test.go

@@ -20,26 +20,28 @@ var _ = Describe("JobPersister", func() {
 	Context("fileJobPersister", func() {
 		var (
 			p       *fileJobPersister
+			tasks   *xsync.SyncedMap[string, schema.Task]
 			jobsMap *xsync.SyncedMap[string, schema.Job]
 			tmpDir  string
 		)
 
 		BeforeEach(func() {
 			tmpDir = GinkgoT().TempDir()
+			tasks = xsync.NewSyncedMap[string, schema.Task]()
 			jobsMap = xsync.NewSyncedMap[string, schema.Job]()
 			p = &fileJobPersister{
+				tasks:     tasks,
 				jobs:      jobsMap,
 				tasksFile: filepath.Join(tmpDir, "tasks.json"),
 				jobsFile:  filepath.Join(tmpDir, "jobs.json"),
-				// taskSet is the persister's own task view (decoupled from the tasks
-				// SyncedMap to avoid re-entering its lock during write-through).
-				taskSet: make(map[string]schema.Task),
 			}
 		})
 
 		It("SaveTask writes all tasks to file", func() {
-			Expect(p.SaveTask("", schema.Task{ID: "t1", Name: "Task One", Model: "m", Prompt: "p"})).To(Succeed())
-			Expect(p.SaveTask("", schema.Task{ID: "t2", Name: "Task Two", Model: "m", Prompt: "p"})).To(Succeed())
+			tasks.Set("t1", schema.Task{ID: "t1", Name: "Task One", Model: "m", Prompt: "p"})
+			tasks.Set("t2", schema.Task{ID: "t2", Name: "Task Two", Model: "m", Prompt: "p"})
+
+			Expect(p.SaveTask("", schema.Task{})).To(Succeed())
 
 			// Verify file contents
 			data, err := os.ReadFile(p.tasksFile)
@@ -50,9 +52,11 @@ var _ = Describe("JobPersister", func() {
 		})
 
 		It("DeleteTask writes updated tasks to file", func() {
-			Expect(p.SaveTask("", schema.Task{ID: "t1", Name: "Keep"})).To(Succeed())
-			Expect(p.SaveTask("", schema.Task{ID: "t2", Name: "Delete"})).To(Succeed())
+			tasks.Set("t1", schema.Task{ID: "t1", Name: "Keep"})
+			tasks.Set("t2", schema.Task{ID: "t2", Name: "Delete"})
 
+			// Simulate deletion from memory (caller does this before calling persister)
+			tasks.Delete("t2")
 			Expect(p.DeleteTask("t2")).To(Succeed())
 
 			data, err := os.ReadFile(p.tasksFile)

core/services/agentpool/task_sync_test.go

@@ -1,152 +0,0 @@
-package agentpool
-
-// White-box tests (package agentpool) so a spec can build two AgentJobService
-// instances sharing one in-memory bus and assert that agent *tasks* converge
-// across replicas - the bug this migration fixes (ListTasks used to read
-// in-memory only, so a task created on replica A was invisible on replica B).
-// Jobs are deliberately untouched here: they already converge via the dispatcher
-// + DB read-through.
-
-import (
-	"context"
-	"time"
-
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-
-	"github.com/mudler/LocalAI/core/config"
-	"github.com/mudler/LocalAI/core/schema"
-	"github.com/mudler/LocalAI/core/services/messaging"
-	"github.com/mudler/LocalAI/core/services/syncstate"
-	"github.com/mudler/LocalAI/core/services/testutil"
-	"github.com/mudler/LocalAI/pkg/system"
-)
-
-// newTaskSyncService builds an AgentJobService wired to the given bus and a
-// throwaway data dir (so the file persister has somewhere to write). Model/config
-// loaders are nil because the task sync paths under test never touch them.
-func newTaskSyncService(bus messaging.MessagingClient) *AgentJobService {
-	tmpDir := GinkgoT().TempDir()
-	sysState := &system.SystemState{}
-	sysState.Model.ModelsPath = tmpDir
-	appConfig := config.NewApplicationConfig(
-		config.WithDynamicConfigDir(tmpDir),
-		config.WithContext(context.Background()),
-	)
-	appConfig.SystemState = sysState
-
-	svc := NewAgentJobServiceWithPaths(appConfig, nil, nil, nil,
-		// Distinct per-replica files so the file persister write-through never
-		// crosses replicas: convergence here must be proven via the bus alone.
-		tmpDir+"/tasks.json", tmpDir+"/jobs.json")
-	svc.SetTaskSyncNATS(bus)
-	return svc
-}
-
-var _ = Describe("AgentJobService task cross-replica sync", func() {
-	Describe("two replicas sharing one bus", func() {
-		var (
-			bus  *testutil.FakeBus
-			a, b *AgentJobService
-		)
-
-		BeforeEach(func() {
-			// One shared bus, two replicas: exactly the distributed topology where a
-			// round-robin request may land on a replica that did not originate the
-			// change.
-			bus = testutil.NewFakeBus()
-			a = newTaskSyncService(bus)
-			b = newTaskSyncService(bus)
-			// Start hydrates (empty here) and subscribes both replicas to deltas.
-			Expect(a.Start(context.Background())).To(Succeed())
-			Expect(b.Start(context.Background())).To(Succeed())
-		})
-
-		AfterEach(func() {
-			Expect(a.Stop()).To(Succeed())
-			Expect(b.Stop()).To(Succeed())
-		})
-
-		It("makes a task created on A visible via B's GetTask and ListTasks", func() {
-			id, err := a.CreateTask(schema.Task{Name: "Shared", Model: "m", Prompt: "p"})
-			Expect(err).NotTo(HaveOccurred())
-
-			got, err := b.GetTask(id)
-			Expect(err).NotTo(HaveOccurred(), "B must see a task A just created")
-			Expect(got.Name).To(Equal("Shared"))
-
-			listed := b.ListTasks()
-			Expect(listed).To(HaveLen(1))
-			Expect(listed[0].ID).To(Equal(id))
-		})
-
-		It("propagates a task update from A to B", func() {
-			id, err := a.CreateTask(schema.Task{Name: "Before", Model: "m", Prompt: "p"})
-			Expect(err).NotTo(HaveOccurred())
-
-			Expect(a.UpdateTask(id, schema.Task{Name: "After", Model: "m", Prompt: "p"})).To(Succeed())
-
-			got, err := b.GetTask(id)
-			Expect(err).NotTo(HaveOccurred())
-			Expect(got.Name).To(Equal("After"), "an update on A must be visible on B")
-		})
-
-		It("removes a task from B when it is deleted on A", func() {
-			id, err := a.CreateTask(schema.Task{Name: "Doomed", Model: "m", Prompt: "p"})
-			Expect(err).NotTo(HaveOccurred())
-			_, err = b.GetTask(id)
-			Expect(err).NotTo(HaveOccurred(), "precondition: B must have the task before the delete")
-
-			Expect(a.DeleteTask(id)).To(Succeed())
-
-			_, err = b.GetTask(id)
-			Expect(err).To(HaveOccurred(), "a delete on A must remove the task from B")
-			Expect(b.ListTasks()).To(BeEmpty())
-		})
-
-		It("does not re-broadcast a delta it received (echo-loop guard)", func() {
-			subject := messaging.SubjectSyncStateDelta("agent.tasks")
-
-			_, err := a.CreateTask(schema.Task{Name: "Once", Model: "m", Prompt: "p"})
-			Expect(err).NotTo(HaveOccurred())
-
-			// Exactly one publish: A's create. B applies it without re-publishing,
-			// otherwise this would be 2+ and a real bus would storm.
-			Expect(bus.PublishCount(subject)).To(Equal(1))
-		})
-	})
-
-	Describe("ListTasks ordering and scoping", func() {
-		var svc *AgentJobService
-
-		BeforeEach(func() {
-			svc = newTaskSyncService(testutil.NewFakeBus())
-			Expect(svc.Start(context.Background())).To(Succeed())
-		})
-		AfterEach(func() { Expect(svc.Stop()).To(Succeed()) })
-
-		It("sorts newest-first, breaking ties by name", func() {
-			// CreateTask stamps CreatedAt with time.Now(); space them out so ordering
-			// is deterministic rather than relying on the sub-millisecond gap.
-			oldID, err := svc.CreateTask(schema.Task{Name: "Old", Model: "m", Prompt: "p"})
-			Expect(err).NotTo(HaveOccurred())
-			time.Sleep(5 * time.Millisecond)
-			newID, err := svc.CreateTask(schema.Task{Name: "New", Model: "m", Prompt: "p"})
-			Expect(err).NotTo(HaveOccurred())
-
-			listed := svc.ListTasks()
-			Expect(listed).To(HaveLen(2))
-			Expect(listed[0].ID).To(Equal(newID), "newest first")
-			Expect(listed[1].ID).To(Equal(oldID))
-		})
-	})
-
-	Describe("compile-time adapter contract", func() {
-		It("satisfies syncstate.Store for tasks", func() {
-			// Mirrors the var assertion in task_syncstore.go; keeps the type
-			// referenced from a spec so drift surfaces here too.
-			var _ syncstate.Store[string, schema.Task] = (*taskStoreAdapter)(nil)
-			Expect(&taskStoreAdapter{}).ToNot(BeNil())
-		})
-	})
-})

core/services/agentpool/task_syncstore.go

@@ -1,47 +0,0 @@
-package agentpool
-
-import (
-	"context"
-
-	"github.com/mudler/LocalAI/core/schema"
-	"github.com/mudler/LocalAI/core/services/syncstate"
-)
-
-// taskStoreAdapter bridges the existing JobPersister (file- or DB-backed) to the
-// generic syncstate.Store the tasks SyncedMap consumes. Only tasks are migrated:
-// jobs already converge across replicas via the dispatcher (NATS) plus the DB
-// read-through in ListJobs/GetJob, whereas ListTasks read in-memory only and so
-// went stale on replicas that did not originate the change.
-//
-// The adapter reads svc.persister and svc.userID live (rather than capturing
-// them) because both are configured by setters - SetDistributedJobStore swaps the
-// file persister for the DB one, SetUserID scopes per-user queries - AFTER the
-// service, and thus this adapter, is constructed. Reading them at call time means
-// the SyncedMap never has to be rebuilt when the persister is swapped.
-//
-// The SyncedMap value type is schema.Task: the exact shape ListTasks returns, so
-// reads need no conversion and REST responses are provably unchanged.
-type taskStoreAdapter struct {
-	svc *AgentJobService
-}
-
-// compile-time assertion that the adapter satisfies the component's Store.
-var _ syncstate.Store[string, schema.Task] = (*taskStoreAdapter)(nil)
-
-// List hydrates the map from durable storage on Start/reconnect: the file's task
-// list (standalone) or every task row (DB / distributed).
-func (a *taskStoreAdapter) List(_ context.Context) ([]schema.Task, error) {
-	return a.svc.persister.LoadTasks(a.svc.userID)
-}
-
-// Upsert write-through persists a single task created/updated locally; the
-// SyncedMap then broadcasts the delta to peers.
-func (a *taskStoreAdapter) Upsert(_ context.Context, task schema.Task) error {
-	return a.svc.persister.SaveTask(a.svc.userID, task)
-}
-
-// Delete write-through removes a task locally; the SyncedMap then broadcasts the
-// removal to peers.
-func (a *taskStoreAdapter) Delete(_ context.Context, id string) error {
-	return a.svc.persister.DeleteTask(id)
-}

core/services/agentpool/user_services.go

@@ -7,7 +7,6 @@ import (
 	"github.com/mudler/LocalAGI/webui/collections"
 	"github.com/mudler/LocalAI/core/config"
 	"github.com/mudler/LocalAI/core/services/jobs"
-	"github.com/mudler/LocalAI/core/services/messaging"
 	"github.com/mudler/LocalAI/core/templates"
 	"github.com/mudler/LocalAI/pkg/model"
 	"github.com/mudler/xlog"
@@ -29,9 +28,6 @@ type UserServicesManager struct {
 	// Shared distributed backends (set once, inherited by per-user job services)
 	jobDispatcher DistributedDispatcher
 	jobDBStore    *jobs.JobStore
-	// jobNats keeps per-user agent tasks consistent across replicas (nil in
-	// standalone). Inherited by each per-user AgentJobService.
-	jobNats messaging.MessagingClient
 }
 
 // NewUserServicesManager creates a new UserServicesManager.
@@ -166,10 +162,6 @@ func (m *UserServicesManager) GetJobs(userID string) (*AgentJobService, error) {
 	if m.jobDispatcher != nil {
 		svc.SetDistributedBackends(m.jobDispatcher)
 	}
-	// Inherit the NATS client so per-user tasks broadcast across replicas. Must be
-	// set before the hydrate below (LoadFromDB / LoadTasksFromFile) so the tasks
-	// SyncedMap is rebuilt with the client while it is still empty.
-	svc.SetTaskSyncNATS(m.jobNats)
 	if m.jobDBStore != nil {
 		svc.SetDistributedJobStore(m.jobDBStore)
 		// Load tasks/jobs from DB immediately (per-user services skip Start())
@@ -197,12 +189,6 @@ func (m *UserServicesManager) SetJobDBStore(s *jobs.JobStore) {
 	m.jobDBStore = s
 }
 
-// SetJobSyncNATS sets the NATS client used to keep per-user agent tasks consistent
-// across replicas.
-func (m *UserServicesManager) SetJobSyncNATS(nats messaging.MessagingClient) {
-	m.jobNats = nats
-}
-
 // ListAllUserIDs returns all user IDs that have scoped data directories.
 func (m *UserServicesManager) ListAllUserIDs() ([]string, error) {
 	return m.storage.ListUserDirs()

core/services/distributed/finetune.go

@@ -8,7 +8,6 @@ import (
 	"github.com/google/uuid"
 	"github.com/mudler/LocalAI/core/services/advisorylock"
 	"gorm.io/gorm"
-	"gorm.io/gorm/clause"
 )
 
 // FineTuneJobRecord tracks fine-tune jobs in PostgreSQL.
@@ -81,34 +80,6 @@ func (s *FineTuneStore) List(userID string) ([]FineTuneJobRecord, error) {
 	return jobs, q.Find(&jobs).Error
 }
 
-// ListAll returns every fine-tune job across all users. The SyncedMap that backs
-// FineTuneService is a single global map (the REST API filters by user at read
-// time), so hydrate needs the full set rather than the per-user List above.
-func (s *FineTuneStore) ListAll() ([]FineTuneJobRecord, error) {
-	var jobs []FineTuneJobRecord
-	return jobs, s.db.Order("created_at DESC").Find(&jobs).Error
-}
-
-// Upsert idempotently inserts or fully replaces a job row by primary key. The
-// SyncedMap write-through path issues a single Set per mutation regardless of
-// whether the job already exists, so it needs one create-or-update primitive
-// (Create alone fails on a duplicate key, UpdateStatus alone misses new rows and
-// only touches a few columns).
-func (s *FineTuneStore) Upsert(job *FineTuneJobRecord) error {
-	if job.ID == "" {
-		job.ID = uuid.New().String()
-	}
-	now := time.Now()
-	if job.CreatedAt.IsZero() {
-		job.CreatedAt = now
-	}
-	job.UpdatedAt = now
-	return s.db.Clauses(clause.OnConflict{
-		Columns:   []clause.Column{{Name: "id"}},
-		UpdateAll: true,
-	}).Create(job).Error
-}
-
 // UpdateStatus updates the status and message of a fine-tune job.
 func (s *FineTuneStore) UpdateStatus(id, status, message string) error {
 	return s.db.Model(&FineTuneJobRecord{}).Where("id = ?", id).Updates(map[string]any{

core/services/distributed/finetune_suite_test.go

@@ -1,13 +0,0 @@
-package distributed_test
-
-import (
-	"testing"
-
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-)
-
-func TestDistributed(t *testing.T) {
-	RegisterFailHandler(Fail)
-	RunSpecs(t, "Distributed Suite")
-}

core/services/distributed/finetune_test.go

@@ -1,61 +0,0 @@
-package distributed_test
-
-import (
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-
-	"github.com/mudler/LocalAI/core/services/distributed"
-	"github.com/mudler/LocalAI/core/services/testutil"
-)
-
-var _ = Describe("FineTuneStore", func() {
-	var store *distributed.FineTuneStore
-
-	BeforeEach(func() {
-		db := testutil.SetupTestDB()
-		var err error
-		store, err = distributed.NewFineTuneStore(db)
-		Expect(err).ToNot(HaveOccurred())
-	})
-
-	Describe("ListAll", func() {
-		It("returns jobs across all users (unlike per-user List)", func() {
-			Expect(store.Create(&distributed.FineTuneJobRecord{ID: "j1", UserID: "u1", Status: "queued"})).To(Succeed())
-			Expect(store.Create(&distributed.FineTuneJobRecord{ID: "j2", UserID: "u2", Status: "queued"})).To(Succeed())
-
-			all, err := store.ListAll()
-			Expect(err).ToNot(HaveOccurred())
-			Expect(all).To(HaveLen(2))
-
-			perUser, err := store.List("u1")
-			Expect(err).ToNot(HaveOccurred())
-			Expect(perUser).To(HaveLen(1), "List stays per-user")
-		})
-	})
-
-	Describe("Upsert", func() {
-		It("inserts a new row", func() {
-			Expect(store.Upsert(&distributed.FineTuneJobRecord{ID: "up-1", UserID: "u1", Status: "queued"})).To(Succeed())
-
-			got, err := store.Get("up-1")
-			Expect(err).ToNot(HaveOccurred())
-			Expect(got.Status).To(Equal("queued"))
-		})
-
-		It("idempotently updates an existing row on a repeated key", func() {
-			Expect(store.Upsert(&distributed.FineTuneJobRecord{ID: "up-2", UserID: "u1", Status: "queued"})).To(Succeed())
-			// Second Upsert with the same primary key must update, not error on a
-			// duplicate-key violation (this is the SyncedMap write-through contract).
-			Expect(store.Upsert(&distributed.FineTuneJobRecord{ID: "up-2", UserID: "u1", Status: "completed", Message: "done"})).To(Succeed())
-
-			got, err := store.Get("up-2")
-			Expect(err).ToNot(HaveOccurred())
-			Expect(got.Status).To(Equal("completed"))
-			Expect(got.Message).To(Equal("done"))
-
-			all, err := store.ListAll()
-			Expect(err).ToNot(HaveOccurred())
-			Expect(all).To(HaveLen(1), "upsert must not create a duplicate")
-		})
-	})
-})

core/services/distributed/init.go

@@ -11,7 +11,6 @@ import (
 type Stores struct {
 	Gallery  *GalleryStore
 	FineTune *FineTuneStore
-	Quant    *QuantStore
 	Skills   *SkillStore
 }
 
@@ -27,21 +26,15 @@ func InitStores(db *gorm.DB) (*Stores, error) {
 		return nil, fmt.Errorf("fine-tune store: %w", err)
 	}
 
-	quant, err := NewQuantStore(db)
-	if err != nil {
-		return nil, fmt.Errorf("quantization store: %w", err)
-	}
-
 	skills, err := NewSkillStore(db)
 	if err != nil {
 		return nil, fmt.Errorf("skills store: %w", err)
 	}
 
-	xlog.Info("Distributed stores initialized (Gallery, FineTune, Quant, Skills)")
+	xlog.Info("Distributed stores initialized (Gallery, FineTune, Skills)")
 	return &Stores{
 		Gallery:  gallery,
 		FineTune: ft,
-		Quant:    quant,
 		Skills:   skills,
 	}, nil
 }

core/services/distributed/quant.go

@@ -1,105 +0,0 @@
-package distributed
-
-import (
-	"context"
-	"fmt"
-	"time"
-
-	"github.com/google/uuid"
-	"github.com/mudler/LocalAI/core/services/advisorylock"
-	"gorm.io/gorm"
-	"gorm.io/gorm/clause"
-)
-
-// QuantJobRecord tracks quantization jobs in PostgreSQL. The columns mirror the
-// API shape (schema.QuantizationJob); the structured Config and ExtraOptions are
-// serialized into JSON text columns so a record fully reconstructs the job.
-type QuantJobRecord struct {
-	ID               string    `gorm:"primaryKey;size:36" json:"id"`
-	UserID           string    `gorm:"index;size:36" json:"user_id,omitempty"`
-	Model            string    `gorm:"size:255" json:"model"`
-	Backend          string    `gorm:"size:64" json:"backend"`
-	ModelID          string    `gorm:"size:255" json:"model_id,omitempty"`
-	QuantizationType string    `gorm:"size:32" json:"quantization_type"`
-	Status           string    `gorm:"index;size:32;default:queued" json:"status"` // queued, downloading, converting, quantizing, completed, failed, stopped
-	Message          string    `gorm:"type:text" json:"message,omitempty"`
-	OutputDir        string    `gorm:"size:512" json:"output_dir,omitempty"`
-	OutputFile       string    `gorm:"size:512" json:"output_file,omitempty"`
-	ConfigJSON       string    `gorm:"column:config;type:text" json:"-"`
-	ExtraOptsJSON    string    `gorm:"column:extra_options;type:text" json:"-"`
-	ImportStatus     string    `gorm:"size:32" json:"import_status,omitempty"`
-	ImportMessage    string    `gorm:"type:text" json:"import_message,omitempty"`
-	ImportModelName  string    `gorm:"size:255" json:"import_model_name,omitempty"`
-	CreatedAt        time.Time `json:"created_at"`
-	UpdatedAt        time.Time `json:"updated_at"`
-}
-
-func (QuantJobRecord) TableName() string { return "quantization_jobs" }
-
-// QuantStore manages quantization job state in PostgreSQL.
-type QuantStore struct {
-	db *gorm.DB
-}
-
-// NewQuantStore creates a new QuantStore and auto-migrates.
-// Uses a PostgreSQL advisory lock to prevent concurrent migration races
-// when multiple instances (frontend + workers) start at the same time.
-func NewQuantStore(db *gorm.DB) (*QuantStore, error) {
-	if err := advisorylock.WithLockCtx(context.Background(), db, advisorylock.KeySchemaMigrate, func() error {
-		return db.AutoMigrate(&QuantJobRecord{})
-	}); err != nil {
-		return nil, fmt.Errorf("migrating quantization_jobs: %w", err)
-	}
-	return &QuantStore{db: db}, nil
-}
-
-// Create stores a new quantization job.
-func (s *QuantStore) Create(job *QuantJobRecord) error {
-	if job.ID == "" {
-		job.ID = uuid.New().String()
-	}
-	job.CreatedAt = time.Now()
-	job.UpdatedAt = job.CreatedAt
-	return s.db.Create(job).Error
-}
-
-// Get retrieves a quantization job by ID.
-func (s *QuantStore) Get(id string) (*QuantJobRecord, error) {
-	var job QuantJobRecord
-	if err := s.db.First(&job, "id = ?", id).Error; err != nil {
-		return nil, err
-	}
-	return &job, nil
-}
-
-// ListAll returns every quantization job across all users. The SyncedMap that
-// backs QuantizationService is a single global map (the REST API filters by user
-// at read time), so hydrate needs the full set.
-func (s *QuantStore) ListAll() ([]QuantJobRecord, error) {
-	var jobs []QuantJobRecord
-	return jobs, s.db.Order("created_at DESC").Find(&jobs).Error
-}
-
-// Upsert idempotently inserts or fully replaces a job row by primary key. The
-// SyncedMap write-through path issues a single Set per mutation regardless of
-// whether the job already exists, so it needs one create-or-update primitive
-// (Create alone fails on a duplicate key).
-func (s *QuantStore) Upsert(job *QuantJobRecord) error {
-	if job.ID == "" {
-		job.ID = uuid.New().String()
-	}
-	now := time.Now()
-	if job.CreatedAt.IsZero() {
-		job.CreatedAt = now
-	}
-	job.UpdatedAt = now
-	return s.db.Clauses(clause.OnConflict{
-		Columns:   []clause.Column{{Name: "id"}},
-		UpdateAll: true,
-	}).Create(job).Error
-}
-
-// Delete removes a quantization job.
-func (s *QuantStore) Delete(id string) error {
-	return s.db.Where("id = ?", id).Delete(&QuantJobRecord{}).Error
-}

core/services/distributed/quant_test.go

@@ -1,57 +0,0 @@
-package distributed_test
-
-import (
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-
-	"github.com/mudler/LocalAI/core/services/distributed"
-	"github.com/mudler/LocalAI/core/services/testutil"
-)
-
-var _ = Describe("QuantStore", func() {
-	var store *distributed.QuantStore
-
-	BeforeEach(func() {
-		db := testutil.SetupTestDB()
-		var err error
-		store, err = distributed.NewQuantStore(db)
-		Expect(err).ToNot(HaveOccurred())
-	})
-
-	Describe("ListAll", func() {
-		It("returns jobs across all users", func() {
-			Expect(store.Create(&distributed.QuantJobRecord{ID: "j1", UserID: "u1", Status: "queued"})).To(Succeed())
-			Expect(store.Create(&distributed.QuantJobRecord{ID: "j2", UserID: "u2", Status: "queued"})).To(Succeed())
-
-			all, err := store.ListAll()
-			Expect(err).ToNot(HaveOccurred())
-			Expect(all).To(HaveLen(2))
-		})
-	})
-
-	Describe("Upsert", func() {
-		It("inserts a new row", func() {
-			Expect(store.Upsert(&distributed.QuantJobRecord{ID: "up-1", UserID: "u1", Status: "queued"})).To(Succeed())
-
-			got, err := store.Get("up-1")
-			Expect(err).ToNot(HaveOccurred())
-			Expect(got.Status).To(Equal("queued"))
-		})
-
-		It("idempotently updates an existing row on a repeated key", func() {
-			Expect(store.Upsert(&distributed.QuantJobRecord{ID: "up-2", UserID: "u1", Status: "queued"})).To(Succeed())
-			// Second Upsert with the same primary key must update, not error on a
-			// duplicate-key violation (this is the SyncedMap write-through contract).
-			Expect(store.Upsert(&distributed.QuantJobRecord{ID: "up-2", UserID: "u1", Status: "completed", Message: "done"})).To(Succeed())
-
-			got, err := store.Get("up-2")
-			Expect(err).ToNot(HaveOccurred())
-			Expect(got.Status).To(Equal("completed"))
-			Expect(got.Message).To(Equal("done"))
-
-			all, err := store.ListAll()
-			Expect(err).ToNot(HaveOccurred())
-			Expect(all).To(HaveLen(1), "upsert must not create a duplicate")
-		})
-	})
-})

core/services/finetune/finetune_suite_test.go

@@ -1,13 +0,0 @@
-package finetune
-
-import (
-	"testing"
-
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-)
-
-func TestFinetune(t *testing.T) {
-	RegisterFailHandler(Fail)
-	RunSpecs(t, "Finetune Suite")
-}

core/services/finetune/service.go

@@ -19,7 +19,6 @@ import (
 	"github.com/mudler/LocalAI/core/schema"
 	"github.com/mudler/LocalAI/core/services/distributed"
 	"github.com/mudler/LocalAI/core/services/messaging"
-	"github.com/mudler/LocalAI/core/services/syncstate"
 	pb "github.com/mudler/LocalAI/pkg/grpc/proto"
 	"github.com/mudler/LocalAI/pkg/model"
 	"github.com/mudler/LocalAI/pkg/utils"
@@ -33,63 +32,44 @@ type FineTuneService struct {
 	modelLoader  *model.ModelLoader
 	configLoader *config.ModelConfigLoader
 
-	// mu serializes the read-modify-write of job values. The SyncedMap guards its
-	// own map structure, but a job is a pointer mutated in place (e.g. the export
-	// goroutine), so the service still needs a lock to keep those field updates
-	// and the subsequent Set atomic with respect to readers.
-	mu sync.Mutex
+	mu   sync.Mutex
+	jobs map[string]*schema.FineTuneJob
 
-	// jobs is the cross-replica job store: an in-memory map kept consistent across
-	// replicas via NATS, optionally read-through to PostgreSQL in distributed mode.
-	jobs *syncstate.SyncedMap[string, *schema.FineTuneJob]
+	// Distributed mode (nil when not in distributed mode)
+	natsClient    messaging.Publisher
+	fineTuneStore *distributed.FineTuneStore
 }
 
-// NewFineTuneService creates a new FineTuneService. In distributed mode pass the
-// shared NATS client and PostgreSQL store so jobs stay consistent across
-// replicas; pass nil for both in standalone mode, where the disk Loader hydrates
-// the map and there is nothing to broadcast.
+// SetNATSClient sets the NATS client for distributed progress publishing.
+func (s *FineTuneService) SetNATSClient(nc messaging.Publisher) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	s.natsClient = nc
+}
+
+// SetFineTuneStore sets the PostgreSQL fine-tune store for distributed persistence.
+func (s *FineTuneService) SetFineTuneStore(store *distributed.FineTuneStore) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	s.fineTuneStore = store
+}
+
+// NewFineTuneService creates a new FineTuneService.
 func NewFineTuneService(
 	appConfig *config.ApplicationConfig,
 	modelLoader *model.ModelLoader,
 	configLoader *config.ModelConfigLoader,
-	nats messaging.MessagingClient,
-	store *distributed.FineTuneStore,
 ) *FineTuneService {
 	s := &FineTuneService{
 		appConfig:    appConfig,
 		modelLoader:  modelLoader,
 		configLoader: configLoader,
+		jobs:         make(map[string]*schema.FineTuneJob),
 	}
-
-	// Only attach a Store interface when a concrete store exists, otherwise the
-	// SyncedMap would see a non-nil interface wrapping a nil pointer and try to
-	// hydrate/write through a nil DB.
-	var syncStore syncstate.Store[string, *schema.FineTuneJob]
-	if store != nil {
-		syncStore = &fineTuneStoreAdapter{store: store}
-	}
-
-	s.jobs = syncstate.New(syncstate.Config[string, *schema.FineTuneJob]{
-		Name:   "finetune.jobs",
-		Key:    func(j *schema.FineTuneJob) string { return j.ID },
-		Nats:   nats,
-		Store:  syncStore,
-		Loader: s.loadJobsFromDisk, // ignored when Store is set (distributed mode)
-	})
-
-	// Hydrate + subscribe. A hydrate failure must not take the server down: log
-	// and continue degraded (standalone), mirroring the OpCache wiring.
-	if err := s.jobs.Start(appConfig.Context); err != nil {
-		xlog.Warn("FineTune SyncedMap start failed; running degraded", "error", err)
-	}
+	s.loadAllJobs()
 	return s
 }
 
-// Close releases the SyncedMap subscription and background workers.
-func (s *FineTuneService) Close() error {
-	return s.jobs.Close()
-}
-
 // fineTuneBaseDir returns the base directory for fine-tune job data.
 func (s *FineTuneService) fineTuneBaseDir() string {
 	return filepath.Join(s.appConfig.DataPath, "fine-tune")
@@ -120,18 +100,15 @@ func (s *FineTuneService) saveJobState(job *schema.FineTuneJob) {
 	}
 }
 
-// loadJobsFromDisk scans the fine-tune directory for persisted jobs and returns
-// them. It is the SyncedMap Loader used in standalone mode (no DB); the returned
-// slice hydrates the map on Start.
-func (s *FineTuneService) loadJobsFromDisk(_ context.Context) ([]*schema.FineTuneJob, error) {
+// loadAllJobs scans the fine-tune directory for persisted jobs and loads them.
+func (s *FineTuneService) loadAllJobs() {
 	baseDir := s.fineTuneBaseDir()
 	entries, err := os.ReadDir(baseDir)
 	if err != nil {
-		// Directory doesn't exist yet — that's fine, start empty.
-		return nil, nil
+		// Directory doesn't exist yet — that's fine
+		return
 	}
 
-	var jobs []*schema.FineTuneJob
 	for _, entry := range entries {
 		if !entry.IsDir() {
 			continue
@@ -160,13 +137,12 @@ func (s *FineTuneService) loadJobsFromDisk(_ context.Context) ([]*schema.FineTun
 			job.ExportMessage = "Server restarted while export was running"
 		}
 
-		jobs = append(jobs, &job)
+		s.jobs[job.ID] = &job
 	}
 
-	if len(jobs) > 0 {
-		xlog.Info("Loaded persisted fine-tune jobs", "count", len(jobs))
+	if len(s.jobs) > 0 {
+		xlog.Info("Loaded persisted fine-tune jobs", "count", len(s.jobs))
 	}
-	return jobs, nil
 }
 
 // StartJob starts a new fine-tuning job.
@@ -260,14 +236,28 @@ func (s *FineTuneService) StartJob(ctx context.Context, userID string, req schem
 		CreatedAt:      time.Now().UTC().Format(time.RFC3339),
 		Config:         &req,
 	}
-	// Set write-through persists to PostgreSQL (distributed) and broadcasts to
-	// peer replicas; the disk state.json is written separately for restart
-	// recovery / standalone hydrate.
-	if err := s.jobs.Set(ctx, job); err != nil {
-		return nil, fmt.Errorf("failed to persist job: %w", err)
-	}
+	s.jobs[jobID] = job
 	s.saveJobState(job)
 
+	// Persist to PostgreSQL in distributed mode
+	if s.fineTuneStore != nil {
+		configJSON, _ := json.Marshal(req)
+		extraJSON, _ := json.Marshal(req.ExtraOptions)
+		s.fineTuneStore.Create(&distributed.FineTuneJobRecord{
+			ID:             jobID,
+			UserID:         userID,
+			Model:          req.Model,
+			Backend:        backendName,
+			ModelID:        modelID,
+			TrainingType:   req.TrainingType,
+			TrainingMethod: req.TrainingMethod,
+			Status:         "queued",
+			OutputDir:      outputDir,
+			ConfigJSON:     string(configJSON),
+			ExtraOptsJSON:  string(extraJSON),
+		})
+	}
+
 	return &schema.FineTuneJobResponse{
 		ID:      jobID,
 		Status:  "queued",
@@ -280,7 +270,7 @@ func (s *FineTuneService) GetJob(userID, jobID string) (*schema.FineTuneJob, err
 	s.mu.Lock()
 	defer s.mu.Unlock()
 
-	job, ok := s.jobs.Get(jobID)
+	job, ok := s.jobs[jobID]
 	if !ok {
 		return nil, fmt.Errorf("job not found: %s", jobID)
 	}
@@ -296,7 +286,7 @@ func (s *FineTuneService) ListJobs(userID string) []*schema.FineTuneJob {
 	defer s.mu.Unlock()
 
 	var result []*schema.FineTuneJob
-	for _, job := range s.jobs.List() {
+	for _, job := range s.jobs {
 		if userID == "" || job.UserID == userID {
 			result = append(result, job)
 		}
@@ -312,7 +302,7 @@ func (s *FineTuneService) ListJobs(userID string) []*schema.FineTuneJob {
 // StopJob stops a running fine-tuning job.
 func (s *FineTuneService) StopJob(ctx context.Context, userID, jobID string, saveCheckpoint bool) error {
 	s.mu.Lock()
-	job, ok := s.jobs.Get(jobID)
+	job, ok := s.jobs[jobID]
 	if !ok {
 		s.mu.Unlock()
 		return fmt.Errorf("job not found: %s", jobID)
@@ -333,10 +323,10 @@ func (s *FineTuneService) StopJob(ctx context.Context, userID, jobID string, sav
 	s.mu.Lock()
 	job.Status = "stopped"
 	job.Message = "Training stopped by user"
-	if err := s.jobs.Set(ctx, job); err != nil {
-		xlog.Warn("Failed to persist stopped job", "job_id", jobID, "error", err)
-	}
 	s.saveJobState(job)
+	if s.fineTuneStore != nil {
+		s.fineTuneStore.UpdateStatus(jobID, "stopped", "Training stopped by user")
+	}
 	s.mu.Unlock()
 
 	return nil
@@ -345,7 +335,7 @@ func (s *FineTuneService) StopJob(ctx context.Context, userID, jobID string, sav
 // DeleteJob removes a fine-tuning job and its associated data from disk.
 func (s *FineTuneService) DeleteJob(userID, jobID string) error {
 	s.mu.Lock()
-	job, ok := s.jobs.Get(jobID)
+	job, ok := s.jobs[jobID]
 	if !ok {
 		s.mu.Unlock()
 		return fmt.Errorf("job not found: %s", jobID)
@@ -370,10 +360,9 @@ func (s *FineTuneService) DeleteJob(userID, jobID string) error {
 	}
 
 	exportModelName := job.ExportModelName
-	// Delete write-through removes the DB row (distributed) and broadcasts the
-	// removal to peer replicas. DeleteJob has no ctx, so use Background.
-	if err := s.jobs.Delete(context.Background(), jobID); err != nil {
-		xlog.Warn("Failed to delete job from store", "job_id", jobID, "error", err)
+	delete(s.jobs, jobID)
+	if s.fineTuneStore != nil {
+		s.fineTuneStore.Delete(jobID)
 	}
 	s.mu.Unlock()
 
@@ -409,7 +398,7 @@ func (s *FineTuneService) DeleteJob(userID, jobID string) error {
 // StreamProgress opens a gRPC progress stream and calls the callback for each update.
 func (s *FineTuneService) StreamProgress(ctx context.Context, userID, jobID string, callback func(event *schema.FineTuneProgressEvent)) error {
 	s.mu.Lock()
-	job, ok := s.jobs.Get(jobID)
+	job, ok := s.jobs[jobID]
 	if !ok {
 		s.mu.Unlock()
 		return fmt.Errorf("job not found: %s", jobID)
@@ -438,7 +427,7 @@ func (s *FineTuneService) StreamProgress(ctx context.Context, userID, jobID stri
 	}, func(update *pb.FineTuneProgressUpdate) {
 		// Update job status and persist
 		s.mu.Lock()
-		if j, ok := s.jobs.Get(jobID); ok {
+		if j, ok := s.jobs[jobID]; ok {
 			// Don't let progress updates overwrite terminal states
 			isTerminal := j.Status == "stopped" || j.Status == "completed" || j.Status == "failed"
 			if !isTerminal {
@@ -447,10 +436,10 @@ func (s *FineTuneService) StreamProgress(ctx context.Context, userID, jobID stri
 			if update.Message != "" {
 				j.Message = update.Message
 			}
-			if err := s.jobs.Set(ctx, j); err != nil {
-				xlog.Warn("Failed to persist progress update", "job_id", jobID, "error", err)
-			}
 			s.saveJobState(j)
+			if s.fineTuneStore != nil {
+				s.fineTuneStore.UpdateStatus(jobID, j.Status, j.Message)
+			}
 		}
 		s.mu.Unlock()
 
@@ -485,7 +474,7 @@ func (s *FineTuneService) StreamProgress(ctx context.Context, userID, jobID stri
 // ListCheckpoints lists checkpoints for a job.
 func (s *FineTuneService) ListCheckpoints(ctx context.Context, userID, jobID string) ([]*pb.CheckpointInfo, error) {
 	s.mu.Lock()
-	job, ok := s.jobs.Get(jobID)
+	job, ok := s.jobs[jobID]
 	if !ok {
 		s.mu.Unlock()
 		return nil, fmt.Errorf("job not found: %s", jobID)
@@ -531,7 +520,7 @@ func sanitizeModelName(s string) string {
 // ExportModel starts an async model export from a checkpoint and returns the intended model name immediately.
 func (s *FineTuneService) ExportModel(ctx context.Context, userID, jobID string, req schema.ExportRequest) (string, error) {
 	s.mu.Lock()
-	job, ok := s.jobs.Get(jobID)
+	job, ok := s.jobs[jobID]
 	if !ok {
 		s.mu.Unlock()
 		return "", fmt.Errorf("job not found: %s", jobID)
@@ -583,9 +572,6 @@ func (s *FineTuneService) ExportModel(ctx context.Context, userID, jobID string,
 	job.ExportStatus = "exporting"
 	job.ExportMessage = ""
 	job.ExportModelName = ""
-	if err := s.jobs.Set(ctx, job); err != nil {
-		xlog.Warn("Failed to persist export start", "job_id", jobID, "error", err)
-	}
 	s.saveJobState(job)
 	s.mu.Unlock()
 
@@ -676,30 +662,24 @@ func (s *FineTuneService) ExportModel(ctx context.Context, userID, jobID string,
 
 		xlog.Info("Model exported and registered", "job_id", jobID, "model_name", modelName, "format", req.ExportFormat)
 
-		// Runs after the HTTP request returns, so use Background rather than the
-		// (now likely cancelled) request ctx for the write-through.
 		s.mu.Lock()
 		job.ExportStatus = "completed"
 		job.ExportModelName = modelName
 		job.ExportMessage = ""
-		if err := s.jobs.Set(context.Background(), job); err != nil {
-			xlog.Warn("Failed to persist export completion", "job_id", jobID, "error", err)
-		}
 		s.saveJobState(job)
+		if s.fineTuneStore != nil {
+			s.fineTuneStore.UpdateExportStatus(jobID, "completed", "", modelName)
+		}
 		s.mu.Unlock()
 	}()
 
 	return modelName, nil
 }
 
-// setExportMessage updates the export message and persists the job state. Called
-// from the background export goroutine, so it uses Background for write-through.
+// setExportMessage updates the export message and persists the job state.
 func (s *FineTuneService) setExportMessage(job *schema.FineTuneJob, msg string) {
 	s.mu.Lock()
 	job.ExportMessage = msg
-	if err := s.jobs.Set(context.Background(), job); err != nil {
-		xlog.Warn("Failed to persist export message", "job_id", job.ID, "error", err)
-	}
 	s.saveJobState(job)
 	s.mu.Unlock()
 }
@@ -707,7 +687,7 @@ func (s *FineTuneService) setExportMessage(job *schema.FineTuneJob, msg string)
 // GetExportedModelPath returns the path to the exported model directory and its name.
 func (s *FineTuneService) GetExportedModelPath(userID, jobID string) (string, string, error) {
 	s.mu.Lock()
-	job, ok := s.jobs.Get(jobID)
+	job, ok := s.jobs[jobID]
 	if !ok {
 		s.mu.Unlock()
 		return "", "", fmt.Errorf("job not found: %s", jobID)
@@ -743,10 +723,10 @@ func (s *FineTuneService) setExportFailed(job *schema.FineTuneJob, message strin
 	s.mu.Lock()
 	job.ExportStatus = "failed"
 	job.ExportMessage = message
-	if err := s.jobs.Set(context.Background(), job); err != nil {
-		xlog.Warn("Failed to persist export failure", "job_id", job.ID, "error", err)
-	}
 	s.saveJobState(job)
+	if s.fineTuneStore != nil {
+		s.fineTuneStore.UpdateExportStatus(job.ID, "failed", message, "")
+	}
 	s.mu.Unlock()
 }
 

core/services/finetune/service_test.go

@@ -1,185 +0,0 @@
-package finetune
-
-// White-box tests (package finetune) so a spec can drive the service's internal
-// SyncedMap the same way StartJob does (via jobs.Set) without standing up a
-// training backend, then assert the cross-replica reads (GetJob/ListJobs) and
-// the adapter conversions that keep REST responses byte-for-byte unchanged.
-
-import (
-	"context"
-
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-
-	"github.com/mudler/LocalAI/core/config"
-	"github.com/mudler/LocalAI/core/schema"
-	"github.com/mudler/LocalAI/core/services/distributed"
-	"github.com/mudler/LocalAI/core/services/testutil"
-)
-
-// newTestService builds a standalone FineTuneService wired to the given bus. The
-// model/config loaders are nil because the read/sync paths under test never touch
-// them; the data dir is a throwaway temp dir so the disk Loader finds nothing.
-func newTestService(bus *testutil.FakeBus) *FineTuneService {
-	appConfig := &config.ApplicationConfig{
-		Context:  context.Background(),
-		DataPath: GinkgoT().TempDir(),
-	}
-	return NewFineTuneService(appConfig, nil, nil, bus, nil)
-}
-
-var _ = Describe("FineTuneService", func() {
-	ctx := context.Background()
-
-	Describe("cross-replica job visibility", func() {
-		var (
-			bus  *testutil.FakeBus
-			a, b *FineTuneService
-		)
-
-		BeforeEach(func() {
-			// One shared bus, two replicas: exactly the distributed topology where
-			// a round-robin request may land on a replica that did not originate
-			// the change.
-			bus = testutil.NewFakeBus()
-			a = newTestService(bus)
-			b = newTestService(bus)
-		})
-
-		AfterEach(func() {
-			Expect(a.Close()).To(Succeed())
-			Expect(b.Close()).To(Succeed())
-		})
-
-		It("makes a job created on A visible via B's GetJob and ListJobs", func() {
-			job := &schema.FineTuneJob{ID: "job-1", UserID: "user-1", Status: "queued", CreatedAt: "2026-06-27T10:00:00Z"}
-			// StartJob persists via jobs.Set; drive that directly to avoid a backend.
-			Expect(a.jobs.Set(ctx, job)).To(Succeed())
-
-			got, err := b.GetJob("user-1", "job-1")
-			Expect(err).ToNot(HaveOccurred(), "B must see a job A just created")
-			Expect(got.Status).To(Equal("queued"))
-
-			listed := b.ListJobs("user-1")
-			Expect(listed).To(HaveLen(1))
-			Expect(listed[0].ID).To(Equal("job-1"))
-		})
-
-		It("removes a job from B when it is deleted on A", func() {
-			job := &schema.FineTuneJob{ID: "job-2", UserID: "user-1", Status: "completed", CreatedAt: "2026-06-27T10:00:00Z"}
-			Expect(a.jobs.Set(ctx, job)).To(Succeed())
-			_, err := b.GetJob("user-1", "job-2")
-			Expect(err).ToNot(HaveOccurred(), "precondition: B must have the job before the delete")
-
-			Expect(a.jobs.Delete(ctx, "job-2")).To(Succeed())
-
-			_, err = b.GetJob("user-1", "job-2")
-			Expect(err).To(HaveOccurred(), "a delete on A must remove the job from B")
-		})
-
-		It("propagates a status update from A to B", func() {
-			job := &schema.FineTuneJob{ID: "job-3", UserID: "user-1", Status: "training", CreatedAt: "2026-06-27T10:00:00Z"}
-			Expect(a.jobs.Set(ctx, job)).To(Succeed())
-
-			updated := &schema.FineTuneJob{ID: "job-3", UserID: "user-1", Status: "completed", CreatedAt: "2026-06-27T10:00:00Z"}
-			Expect(a.jobs.Set(ctx, updated)).To(Succeed())
-
-			got, err := b.GetJob("user-1", "job-3")
-			Expect(err).ToNot(HaveOccurred())
-			Expect(got.Status).To(Equal("completed"))
-		})
-	})
-
-	Describe("ListJobs", func() {
-		var svc *FineTuneService
-
-		BeforeEach(func() {
-			svc = newTestService(testutil.NewFakeBus())
-		})
-		AfterEach(func() { Expect(svc.Close()).To(Succeed()) })
-
-		It("filters by user and sorts newest-first", func() {
-			Expect(svc.jobs.Set(ctx, &schema.FineTuneJob{ID: "old", UserID: "u1", CreatedAt: "2026-06-25T10:00:00Z"})).To(Succeed())
-			Expect(svc.jobs.Set(ctx, &schema.FineTuneJob{ID: "new", UserID: "u1", CreatedAt: "2026-06-27T10:00:00Z"})).To(Succeed())
-			Expect(svc.jobs.Set(ctx, &schema.FineTuneJob{ID: "other", UserID: "u2", CreatedAt: "2026-06-26T10:00:00Z"})).To(Succeed())
-
-			jobs := svc.ListJobs("u1")
-			Expect(jobs).To(HaveLen(2), "only u1's jobs")
-			Expect(jobs[0].ID).To(Equal("new"), "newest first")
-			Expect(jobs[1].ID).To(Equal("old"))
-		})
-
-		It("returns every user's jobs when the userID filter is empty", func() {
-			Expect(svc.jobs.Set(ctx, &schema.FineTuneJob{ID: "a", UserID: "u1", CreatedAt: "2026-06-25T10:00:00Z"})).To(Succeed())
-			Expect(svc.jobs.Set(ctx, &schema.FineTuneJob{ID: "b", UserID: "u2", CreatedAt: "2026-06-26T10:00:00Z"})).To(Succeed())
-
-			Expect(svc.ListJobs("")).To(HaveLen(2))
-		})
-
-		It("rejects GetJob for a job owned by another user", func() {
-			Expect(svc.jobs.Set(ctx, &schema.FineTuneJob{ID: "x", UserID: "owner", CreatedAt: "2026-06-25T10:00:00Z"})).To(Succeed())
-
-			_, err := svc.GetJob("intruder", "x")
-			Expect(err).To(HaveOccurred(), "a different user must not read someone else's job")
-		})
-	})
-
-	Describe("store adapter conversion", func() {
-		// The SyncedMap value type is *schema.FineTuneJob (the exact REST shape).
-		// These specs prove the DB adapter round-trips it losslessly, so hydrate
-		// and write-through in distributed mode keep responses unchanged.
-		It("round-trips a job through jobToRecord/recordToJob preserving the API shape", func() {
-			original := &schema.FineTuneJob{
-				ID:              "rt-1",
-				UserID:          "user-1",
-				Model:           "base-model",
-				Backend:         "trl",
-				ModelID:         "trl-finetune-rt-1",
-				TrainingType:    "lora",
-				TrainingMethod:  "sft",
-				Status:          "completed",
-				Message:         "done",
-				OutputDir:       "/data/fine-tune/rt-1",
-				ExtraOptions:    map[string]string{"hf_token": "secret"},
-				CreatedAt:       "2026-06-27T10:00:00Z",
-				ExportStatus:    "completed",
-				ExportMessage:   "",
-				ExportModelName: "base-model-ft-rt-1",
-				Config:          &schema.FineTuneJobRequest{Model: "base-model", Backend: "trl", DatasetSource: "data.jsonl"},
-			}
-
-			rec := jobToRecord(original)
-			Expect(rec.ID).To(Equal("rt-1"))
-			Expect(rec.ConfigJSON).ToNot(BeEmpty(), "structured config must serialize into the JSON column")
-			Expect(rec.ExtraOptsJSON).ToNot(BeEmpty())
-
-			back := recordToJob(rec)
-			Expect(back.ID).To(Equal(original.ID))
-			Expect(back.UserID).To(Equal(original.UserID))
-			Expect(back.Model).To(Equal(original.Model))
-			Expect(back.Backend).To(Equal(original.Backend))
-			Expect(back.ModelID).To(Equal(original.ModelID))
-			Expect(back.TrainingType).To(Equal(original.TrainingType))
-			Expect(back.TrainingMethod).To(Equal(original.TrainingMethod))
-			Expect(back.Status).To(Equal(original.Status))
-			Expect(back.Message).To(Equal(original.Message))
-			Expect(back.OutputDir).To(Equal(original.OutputDir))
-			Expect(back.ExportStatus).To(Equal(original.ExportStatus))
-			Expect(back.ExportModelName).To(Equal(original.ExportModelName))
-			Expect(back.CreatedAt).To(Equal(original.CreatedAt))
-			Expect(back.ExtraOptions).To(Equal(original.ExtraOptions))
-			Expect(back.Config).ToNot(BeNil())
-			Expect(back.Config.DatasetSource).To(Equal("data.jsonl"))
-		})
-	})
-
-	Describe("compile-time adapter contract", func() {
-		It("satisfies syncstate.Store for *distributed.FineTuneStore", func() {
-			// Guards against drift between the adapter and the component interface;
-			// the var assertion in syncstore.go covers it at build time, this keeps
-			// the type referenced from a spec too.
-			var _ *distributed.FineTuneStore
-			Expect(&fineTuneStoreAdapter{}).ToNot(BeNil())
-		})
-	})
-})

core/services/finetune/syncstore.go

@@ -1,114 +0,0 @@
-package finetune
-
-import (
-	"context"
-	"encoding/json"
-	"time"
-
-	"github.com/mudler/LocalAI/core/schema"
-	"github.com/mudler/LocalAI/core/services/distributed"
-	"github.com/mudler/LocalAI/core/services/syncstate"
-)
-
-// fineTuneStoreAdapter bridges the distributed PostgreSQL FineTuneStore to the
-// generic syncstate.Store the SyncedMap consumes. It is only wired in distributed
-// mode; standalone leaves Store nil and hydrates from disk via a Loader instead.
-//
-// The SyncedMap value type is *schema.FineTuneJob (the exact shape the REST API
-// returns) so reads need no conversion and the response JSON is provably
-// unchanged. The adapter is the single place that translates between that API
-// shape and the DB FineTuneJobRecord.
-type fineTuneStoreAdapter struct {
-	store *distributed.FineTuneStore
-}
-
-// compile-time assertion that the adapter satisfies the component's Store.
-var _ syncstate.Store[string, *schema.FineTuneJob] = (*fineTuneStoreAdapter)(nil)
-
-func (a *fineTuneStoreAdapter) List(_ context.Context) ([]*schema.FineTuneJob, error) {
-	records, err := a.store.ListAll()
-	if err != nil {
-		return nil, err
-	}
-	jobs := make([]*schema.FineTuneJob, 0, len(records))
-	for i := range records {
-		jobs = append(jobs, recordToJob(&records[i]))
-	}
-	return jobs, nil
-}
-
-func (a *fineTuneStoreAdapter) Upsert(_ context.Context, job *schema.FineTuneJob) error {
-	return a.store.Upsert(jobToRecord(job))
-}
-
-func (a *fineTuneStoreAdapter) Delete(_ context.Context, id string) error {
-	return a.store.Delete(id)
-}
-
-// recordToJob maps a persisted DB record back to the API shape, reconstructing
-// the structured Config / ExtraOptions from their JSON columns.
-func recordToJob(r *distributed.FineTuneJobRecord) *schema.FineTuneJob {
-	job := &schema.FineTuneJob{
-		ID:              r.ID,
-		UserID:          r.UserID,
-		Model:           r.Model,
-		Backend:         r.Backend,
-		ModelID:         r.ModelID,
-		TrainingType:    r.TrainingType,
-		TrainingMethod:  r.TrainingMethod,
-		Status:          r.Status,
-		Message:         r.Message,
-		OutputDir:       r.OutputDir,
-		ExportStatus:    r.ExportStatus,
-		ExportMessage:   r.ExportMessage,
-		ExportModelName: r.ExportModelName,
-		CreatedAt:       r.CreatedAt.UTC().Format(time.RFC3339),
-	}
-	if r.ExtraOptsJSON != "" {
-		// Best-effort: a malformed column must not drop the whole job from the API.
-		_ = json.Unmarshal([]byte(r.ExtraOptsJSON), &job.ExtraOptions)
-	}
-	if r.ConfigJSON != "" {
-		var cfg schema.FineTuneJobRequest
-		if err := json.Unmarshal([]byte(r.ConfigJSON), &cfg); err == nil {
-			job.Config = &cfg
-		}
-	}
-	return job
-}
-
-// jobToRecord maps the API shape to a DB record for write-through, serializing
-// the structured Config / ExtraOptions into their JSON columns. CreatedAt is
-// parsed back from the RFC3339 string the service stamps; an unparseable value
-// is left zero so FineTuneStore.Upsert stamps "now".
-func jobToRecord(job *schema.FineTuneJob) *distributed.FineTuneJobRecord {
-	rec := &distributed.FineTuneJobRecord{
-		ID:              job.ID,
-		UserID:          job.UserID,
-		Model:           job.Model,
-		Backend:         job.Backend,
-		ModelID:         job.ModelID,
-		TrainingType:    job.TrainingType,
-		TrainingMethod:  job.TrainingMethod,
-		Status:          job.Status,
-		Message:         job.Message,
-		OutputDir:       job.OutputDir,
-		ExportStatus:    job.ExportStatus,
-		ExportMessage:   job.ExportMessage,
-		ExportModelName: job.ExportModelName,
-	}
-	if job.Config != nil {
-		if data, err := json.Marshal(job.Config); err == nil {
-			rec.ConfigJSON = string(data)
-		}
-	}
-	if job.ExtraOptions != nil {
-		if data, err := json.Marshal(job.ExtraOptions); err == nil {
-			rec.ExtraOptsJSON = string(data)
-		}
-	}
-	if t, err := time.Parse(time.RFC3339, job.CreatedAt); err == nil {
-		rec.CreatedAt = t
-	}
-	return rec
-}

core/services/galleryop/distributed_sync_test.go

@@ -404,36 +404,6 @@ var _ = Describe("GalleryService cache invalidation broadcasts", func() {
 			Element: "x", Op: "install",
 		})).To(Succeed())
 	})
-
-	It("BroadcastModelsChanged delivers the element and op to a peer's OnModelsChanged", func() {
-		var (
-			mu   sync.Mutex
-			seen []messaging.CacheInvalidateEvent
-		)
-		svcB.OnModelsChanged = func(evt messaging.CacheInvalidateEvent) {
-			mu.Lock()
-			seen = append(seen, evt)
-			mu.Unlock()
-		}
-		Expect(svcA.SubscribeBroadcasts()).To(Succeed())
-		Expect(svcB.SubscribeBroadcasts()).To(Succeed())
-
-		// An admin edit on replica A must reach replica B over the same subject
-		// the gallery path uses, so B refreshes its in-memory config loader.
-		svcA.BroadcastModelsChanged("my-alias", "install")
-
-		mu.Lock()
-		defer mu.Unlock()
-		Expect(seen).To(ContainElement(messaging.CacheInvalidateEvent{
-			Element: "my-alias", Op: "install",
-		}))
-	})
-
-	It("BroadcastModelsChanged is a no-op when NATS is not wired (standalone)", func() {
-		standalone := galleryop.NewGalleryService(&config.ApplicationConfig{}, nil)
-		// No SetNATSClient: must not panic and must simply do nothing.
-		Expect(func() { standalone.BroadcastModelsChanged("x", "delete") }).ToNot(Panic())
-	})
 })
 
 var _ = Describe("GalleryService PostgreSQL hydration", func() {

core/services/galleryop/service.go

@@ -201,24 +201,6 @@ func (g *GalleryService) publishCacheInvalidate(subject string, evt messaging.Ca
 	}
 }
 
-// BroadcastModelsChanged notifies peer replicas that a model config was
-// created, edited, or removed out-of-band of the gallery install/delete
-// channel (e.g. the admin /models/edit, /models/import and
-// /models/toggle-state endpoints, which write the YAML and reload only the
-// local in-memory loader). Peers receive it via OnModelsChanged and refresh
-// their own ModelConfigLoader so a request load-balanced to any replica sees
-// the same config. No-op in standalone mode (no NATS client).
-//
-// op is "install" for a create/edit (the element must be (re)loaded from
-// disk) or "delete" for a removal (the element must be pruned from memory,
-// which a reload-from-path cannot do because the loader is additive).
-func (g *GalleryService) BroadcastModelsChanged(element, op string) {
-	g.publishCacheInvalidate(messaging.SubjectCacheInvalidateModels, messaging.CacheInvalidateEvent{
-		Element: element,
-		Op:      op,
-	})
-}
-
 // mergeStatus is the broadcast-side merge: it updates the in-memory map from
 // a peer's GalleryProgressEvent without re-publishing to NATS or re-writing
 // to PostgreSQL. UpdateStatus is the local-write entry point and does both;

core/services/jobs/sqlite_e2e_test.go

@@ -1,24 +0,0 @@
-//go:build auth
-
-package jobs_test
-
-import (
-	"github.com/mudler/LocalAI/core/http/auth"
-	"github.com/mudler/LocalAI/core/services/jobs"
-
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-)
-
-// Reproduces the #10506 caller chain: auth.InitDB(sqlite) -> jobs.NewJobStore,
-// which previously failed with "no such function: pg_advisory_lock".
-var _ = Describe("NewJobStore on a SQLite auth DB (#10506)", func() {
-	It("migrates without pg_advisory_lock errors", func() {
-		db, err := auth.InitDB(":memory:")
-		Expect(err).ToNot(HaveOccurred())
-
-		store, err := jobs.NewJobStore(db)
-		Expect(err).ToNot(HaveOccurred())
-		Expect(store).ToNot(BeNil())
-	})
-})

core/services/messaging/client.go

@@ -22,14 +22,6 @@ const subscribeConfirmTimeout = 5 * time.Second
 type Client struct {
 	conn *nats.Conn
 	mu   sync.RWMutex
-
-	// reconnectCbs are invoked after the underlying connection is
-	// re-established. nats.go transparently resubscribes existing
-	// subscriptions on reconnect, but it cannot know that a consumer kept
-	// derived in-memory state (e.g. syncstate.SyncedMap) that may have drifted
-	// while the link was down — these callbacks let such consumers re-hydrate.
-	cbMu         sync.Mutex
-	reconnectCbs []func()
 }
 
 // New creates a new NATS client with auto-reconnect.
@@ -39,10 +31,6 @@ func New(url string, opts ...Option) (*Client, error) {
 		o(&cfg)
 	}
 
-	// Allocate the client up front so the reconnect handler closure can reach
-	// it; conn is populated after nats.Connect succeeds below.
-	c := &Client{}
-
 	natsOpts := []nats.Option{
 		nats.RetryOnFailedConnect(true),
 		nats.MaxReconnects(-1),
@@ -53,7 +41,6 @@ func New(url string, opts ...Option) (*Client, error) {
 		}),
 		nats.ReconnectHandler(func(_ *nats.Conn) {
 			xlog.Info("NATS reconnected")
-			c.runReconnectCallbacks()
 		}),
 		nats.ClosedHandler(func(_ *nats.Conn) {
 			xlog.Info("NATS connection closed")
@@ -116,33 +103,7 @@ func New(url string, opts ...Option) (*Client, error) {
 		return nil, fmt.Errorf("connecting to NATS at %s: %w", sanitize.URL(url), err)
 	}
 
-	c.conn = nc
-	return c, nil
-}
-
-// OnReconnect registers a callback invoked after the NATS connection is
-// re-established. It is consumed via an optional interface type-assertion
-// (interface{ OnReconnect(func()) }) rather than being added to MessagingClient,
-// so the messaging abstraction stays minimal and standalone/test clients are not
-// forced to implement reconnect semantics. A nil callback is ignored.
-func (c *Client) OnReconnect(cb func()) {
-	if cb == nil {
-		return
-	}
-	c.cbMu.Lock()
-	c.reconnectCbs = append(c.reconnectCbs, cb)
-	c.cbMu.Unlock()
-}
-
-// runReconnectCallbacks invokes registered reconnect callbacks. It copies the
-// slice under the lock so a callback that (re)registers cannot deadlock.
-func (c *Client) runReconnectCallbacks() {
-	c.cbMu.Lock()
-	cbs := append([]func(){}, c.reconnectCbs...)
-	c.cbMu.Unlock()
-	for _, cb := range cbs {
-		cb()
-	}
+	return &Client{conn: nc}, nil
 }
 
 // Publish marshals data as JSON and publishes it to the given subject.

core/services/messaging/subjects.go

@@ -380,20 +380,6 @@ func SubjectCacheInvalidateCollection(name string) string {
 	return "cache.invalidate.collections." + sanitizeSubjectToken(name)
 }
 
-// SyncedMap State Sync (Pub/Sub — broadcast to all frontends)
-//
-// The reusable syncstate.SyncedMap component publishes a {op,key,value} delta on
-// this subject whenever a replica mutates a piece of cross-replica in-memory
-// state. Peers subscribe and apply the delta to their own map, so a round-robin
-// API request that lands on a replica which did not originate the change still
-// sees it. Convergence on (re)connect is done by re-hydrating from the durable
-// source, so no request/reply snapshot subject is needed here.
-func SubjectSyncStateDelta(name string) string {
-	return subjectSyncStatePrefix + sanitizeSubjectToken(name) + ".delta"
-}
-
-const subjectSyncStatePrefix = "state."
-
 // Prefix-Cache Routing Sync (Pub/Sub - broadcast to all frontends)
 //
 // Frontends share prefix-cache observations so a request routed to any replica

core/services/modeladmin/remote_sync.go

@@ -1,53 +0,0 @@
-package modeladmin
-
-import (
-	"github.com/mudler/LocalAI/core/config"
-	"github.com/mudler/LocalAI/core/services/messaging"
-	"github.com/mudler/LocalAI/pkg/model"
-
-	"github.com/mudler/xlog"
-)
-
-// opDelete is the CacheInvalidateEvent.Op value the gallery delete path and the
-// admin delete endpoint use; a delete must prune (a reload-from-path cannot).
-const opDelete = "delete"
-
-// ApplyRemoteChange refreshes this replica's in-memory model state from a peer
-// replica's model-config change broadcast (messaging.CacheInvalidateEvent on
-// SubjectCacheInvalidateModels). It is the subscriber-side counterpart to
-// GalleryService.BroadcastModelsChanged.
-//
-// The op matters because LoadModelConfigsFromPath is additive: it loads every
-// YAML on disk into the loader but never removes an entry whose file is gone.
-// So a delete cannot be propagated by a plain reload - the deleted element must
-// be explicitly pruned. Specifically:
-//
-//   - op == "delete" with a named element: prune that element from the loader.
-//   - otherwise: reload all configs from disk (picks up creates and edits).
-//
-// In both cases, when an element is named, any running instance on this replica
-// is shut down (best-effort) so the next request rebuilds it from the new
-// config instead of serving the stale one - mirroring what the originating
-// replica does on a local edit/delete.
-//
-// ml may be nil (no running instances to shut down). modelsPath and opts are
-// forwarded to LoadModelConfigsFromPath.
-func ApplyRemoteChange(cl *config.ModelConfigLoader, ml *model.ModelLoader, modelsPath string, evt messaging.CacheInvalidateEvent, opts ...config.ConfigLoaderOption) error {
-	if evt.Op == opDelete && evt.Element != "" {
-		cl.RemoveModelConfig(evt.Element)
-	} else if err := cl.LoadModelConfigsFromPath(modelsPath, opts...); err != nil {
-		return err
-	}
-
-	// Drop any running instance of the affected model so the next request
-	// rebuilds it from the refreshed config instead of serving the stale one.
-	// Best-effort: the model may not be loaded on this replica, which surfaces
-	// as a benign error here.
-	if ml != nil && evt.Element != "" {
-		if err := ml.ShutdownModel(evt.Element); err != nil {
-			xlog.Debug("ApplyRemoteChange: could not shut down model instance (likely not loaded)",
-				"model", evt.Element, "error", err)
-		}
-	}
-	return nil
-}

core/services/modeladmin/remote_sync_test.go

@@ -1,80 +0,0 @@
-package modeladmin
-
-import (
-	"os"
-	"path/filepath"
-
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-	"gopkg.in/yaml.v3"
-
-	"github.com/mudler/LocalAI/core/config"
-	"github.com/mudler/LocalAI/core/services/messaging"
-)
-
-var _ = Describe("ApplyRemoteChange", func() {
-	var (
-		dir    string
-		loader *config.ModelConfigLoader
-	)
-
-	BeforeEach(func() {
-		dir = GinkgoT().TempDir()
-		loader = config.NewModelConfigLoader(dir)
-	})
-
-	writeYAML := func(name string, body map[string]any) {
-		body["name"] = name
-		data, err := yaml.Marshal(body)
-		Expect(err).ToNot(HaveOccurred())
-		Expect(os.WriteFile(filepath.Join(dir, name+".yaml"), data, 0644)).To(Succeed())
-	}
-
-	It("loads a peer-created config from disk on an install event", func() {
-		// Peer wrote the YAML to the shared models dir; this replica has not
-		// loaded it yet (empty in-memory loader).
-		writeYAML("peer-alias", map[string]any{"alias": "qwen"})
-		_, ok := loader.GetModelConfig("peer-alias")
-		Expect(ok).To(BeFalse(), "precondition: not yet in memory")
-
-		err := ApplyRemoteChange(loader, nil, dir, messaging.CacheInvalidateEvent{
-			Element: "peer-alias", Op: "install",
-		})
-		Expect(err).ToNot(HaveOccurred())
-
-		_, ok = loader.GetModelConfig("peer-alias")
-		Expect(ok).To(BeTrue(), "install event must reload the new config from disk")
-	})
-
-	It("prunes a peer-deleted config that a reload-from-path cannot drop", func() {
-		// Model is present in memory (loaded earlier) but its file is now gone
-		// from the shared dir. LoadModelConfigsFromPath is additive, so only an
-		// explicit prune can remove it - this is the cross-replica delete bug.
-		writeYAML("doomed", map[string]any{"alias": "qwen"})
-		Expect(loader.LoadModelConfigsFromPath(dir)).To(Succeed())
-		_, ok := loader.GetModelConfig("doomed")
-		Expect(ok).To(BeTrue(), "precondition: in memory")
-		Expect(os.Remove(filepath.Join(dir, "doomed.yaml"))).To(Succeed())
-
-		err := ApplyRemoteChange(loader, nil, dir, messaging.CacheInvalidateEvent{
-			Element: "doomed", Op: "delete",
-		})
-		Expect(err).ToNot(HaveOccurred())
-
-		_, ok = loader.GetModelConfig("doomed")
-		Expect(ok).To(BeFalse(), "delete event must prune the element from memory")
-	})
-
-	It("does a full reload when no element is named", func() {
-		writeYAML("m1", map[string]any{"alias": "qwen"})
-		writeYAML("m2", map[string]any{"alias": "qwen"})
-
-		err := ApplyRemoteChange(loader, nil, dir, messaging.CacheInvalidateEvent{})
-		Expect(err).ToNot(HaveOccurred())
-
-		_, ok1 := loader.GetModelConfig("m1")
-		_, ok2 := loader.GetModelConfig("m2")
-		Expect(ok1).To(BeTrue())
-		Expect(ok2).To(BeTrue())
-	})
-})

core/services/nodes/registry.go

@@ -673,49 +673,6 @@ func (r *NodeRegistry) Get(ctx context.Context, nodeID string) (*BackendNode, er
 	return &node, nil
 }
 
-// GetWithExtras returns a single node enriched with the same computed fields as
-// ListWithExtras (labels, loaded-model count, in-flight total). The plain Get
-// returns a bare BackendNode whose Labels live in a separate table, so the node
-// detail view needs this to show a node's existing labels and live counts.
-func (r *NodeRegistry) GetWithExtras(ctx context.Context, nodeID string) (*NodeWithExtras, error) {
-	node, err := r.Get(ctx, nodeID)
-	if err != nil {
-		return nil, err
-	}
-
-	labels := make(map[string]string)
-	nodeLabels, err := r.GetNodeLabels(ctx, nodeID)
-	if err != nil {
-		xlog.Warn("GetWithExtras: failed to get labels", "node", nodeID, "error", err)
-	} else {
-		for _, l := range nodeLabels {
-			labels[l.Key] = l.Value
-		}
-	}
-
-	var modelCount int64
-	if err := r.db.WithContext(ctx).Model(&NodeModel{}).
-		Where("node_id = ? AND state = ?", nodeID, "loaded").
-		Count(&modelCount).Error; err != nil {
-		xlog.Warn("GetWithExtras: failed to get model count", "node", nodeID, "error", err)
-	}
-
-	var inFlight struct{ Total int }
-	if err := r.db.WithContext(ctx).Model(&NodeModel{}).
-		Select("COALESCE(SUM(in_flight), 0) as total").
-		Where("node_id = ? AND state IN ?", nodeID, []string{"loaded", "unloading"}).
-		Scan(&inFlight).Error; err != nil {
-		xlog.Warn("GetWithExtras: failed to get in-flight count", "node", nodeID, "error", err)
-	}
-
-	return &NodeWithExtras{
-		BackendNode:   *node,
-		ModelCount:    int(modelCount),
-		InFlightCount: inFlight.Total,
-		Labels:        labels,
-	}, nil
-}
-
 // GetByName returns a single node by name.
 func (r *NodeRegistry) GetByName(ctx context.Context, name string) (*BackendNode, error) {
 	var node BackendNode

core/services/nodes/registry_test.go

@@ -646,38 +646,6 @@ var _ = Describe("NodeRegistry", func() {
 		})
 	})
 
-	Describe("GetWithExtras", func() {
-		It("returns the node enriched with its labels map", func() {
-			node := makeNode("extras-node", "10.0.0.80:50051", 8_000_000_000)
-			Expect(registry.Register(context.Background(), node, true)).To(Succeed())
-			Expect(registry.SetNodeLabel(context.Background(), node.ID, "env", "prod")).To(Succeed())
-			Expect(registry.SetNodeLabel(context.Background(), node.ID, "region", "us-east")).To(Succeed())
-
-			got, err := registry.GetWithExtras(context.Background(), node.ID)
-			Expect(err).ToNot(HaveOccurred())
-			Expect(got).ToNot(BeNil())
-			Expect(got.ID).To(Equal(node.ID))
-			Expect(got.Name).To(Equal("extras-node"))
-			Expect(got.Labels).To(Equal(map[string]string{"env": "prod", "region": "us-east"}))
-		})
-
-		It("returns an empty (non-nil) labels map when the node has none", func() {
-			node := makeNode("extras-no-labels", "10.0.0.81:50051", 8_000_000_000)
-			Expect(registry.Register(context.Background(), node, true)).To(Succeed())
-
-			got, err := registry.GetWithExtras(context.Background(), node.ID)
-			Expect(err).ToNot(HaveOccurred())
-			Expect(got).ToNot(BeNil())
-			Expect(got.Labels).ToNot(BeNil())
-			Expect(got.Labels).To(BeEmpty())
-		})
-
-		It("returns an error for an unknown node", func() {
-			_, err := registry.GetWithExtras(context.Background(), "does-not-exist")
-			Expect(err).To(HaveOccurred())
-		})
-	})
-
 	Describe("FindNodesBySelector", func() {
 		It("returns nodes matching all labels in selector", func() {
 			n1 := makeNode("sel-match", "10.0.0.80:50051", 8_000_000_000)

core/services/quantization/quantization_suite_test.go

@@ -1,13 +0,0 @@
-package quantization
-
-import (
-	"testing"
-
-	. "github.com/onsi/ginkgo/v2"
-	. "github.com/onsi/gomega"
-)
-
-func TestQuantization(t *testing.T) {
-	RegisterFailHandler(Fail)
-	RunSpecs(t, "Quantization Suite")
-}