fix(ui): expand traces without IDs

Keep ID-based expansion stable across refreshes while falling back to the row index for backend trace summaries that do not carry an ID.

Assisted-by: Codex:gpt-5 [systematic-debugging]

.agents/ci-caching.md

@@ -125,7 +125,7 @@ The per-backend prefix match only sees files under a backend's own directory, so
 | `backend/backend.proto` | nothing if the edit is additive-only, otherwise everything (see below) |
 | `backend/Dockerfile.<x>` | the Linux entries whose `dockerfile:` names it |
 | `backend/python/common/` | Python, Linux + Darwin |
-| `scripts/build/package-gpu-libs.sh` | Python, Linux only |
+| `scripts/build/package-gpu-libs.sh` | every Linux entry (Python, Go and C++ all run it) |
 | `scripts/build/<lang>-darwin.sh` | the Darwin entries that build target routes to |
 | `.github/workflows/backend_build[_darwin].yml` | everything on that OS |
 | anything else under `scripts/build/` (except `*_test.sh`) | everything — conservative default for unclassified packaging inputs |
@@ -359,6 +359,26 @@ GitHub Actions caches are limited to 10 GB per repo. Steady-state worst case: ~8
 
 One residual self-hosted reference remains in `test-extra.yml` (`tests-vibevoice-cpp-grpc-transcription` uses `bigger-runner` for the 30s JFK-decode timeout headroom). That's a separate concern.
 
+### Small always-on jobs routed to `arc-runner-set`
+
+The hosted pool is shared across the whole *account*, not per repo, so a burst in one repo starves the others. On 2026-07-31 it went to **zero scheduled jobs for 35 consecutive minutes** with 39 jobs queued, while `arc-runner-set` completed 12 jobs without interruption over the same window. Actions was healthy globally at the time (other public repos were scheduling normally), so this is an account-level throttle, not an outage.
+
+`gh-pages.yml` (`build` + `deploy`) is therefore routed to `arc-runner-set` when `github.repository == 'mudler/LocalAI'`. It needs no fork-safety clause because it only triggers on push-to-master and `workflow_dispatch`, so it never executes pull-request code. The repository guard keeps forks (which have no such runner label) from queueing forever. It fetches its own toolchains via `setup-go` / `actions-hugo` and uses no `sudo`/`apt`.
+
+#### What the `arc-runner-set` image actually contains
+
+Measured 2026-07-31 on run `30637392862` by a preflight step, not assumed:
+
+| present | **absent** |
+|---|---|
+| `git`, `curl`, `unzip`, `tar`, `ldd`, `python3` | **`make`**, **`gcc`** |
+
+That is why `lint.yml` is **not** on the self-hosted pool. Both of its jobs were routed there and both failed in one second: `golangci-lint` needs `make` (for `make protogen-go`, itself needing `curl`+`unzip` to fetch protoc, and for `make lint`), and `build-scripts` additionally needs a C toolchain because the packaging-script tests compile a throwaway binary and inspect it with `ldd`. Both jobs are back on `ubuntu-latest`.
+
+The preflight steps were deliberately left in place. They cost about a second on the hosted pool and mean that whenever the runner image gains `make` + `gcc`, re-routing is one `runs-on:` line per job and any remaining gap reports itself by name rather than as an opaque mid-build failure.
+
+Note for any future re-route: `lint.yml` also triggers on `pull_request`, and a fork PR runs untrusted contributor code. That must never reach a persistent self-hosted runner, so any re-route has to stay push-only, e.g. `${{ (github.event_name == 'push' && github.repository == 'mudler/LocalAI') && 'arc-runner-set' || 'ubuntu-latest' }}`.
+
 ## Touching the cache pipeline
 
 When changing `image_build.yml`, `backend_build.yml`, any of the `backend/Dockerfile.*` files, `Dockerfile.base-grpc-builder`, `.docker/install-base-deps.sh`, `.docker/<backend>-compile.sh`, or `scripts/changed-backends.js`:

.agents/preparing-a-release.md

@@ -0,0 +1,26 @@
+# Preparing a Release
+
+A release is not finished when the tag is pushed. The GitHub release, the blog post and the demo clips ship together, because the changelog says what moved and the post and the clips are what make anyone care.
+
+## What a release must include
+
+1. **Labels on the merged PRs.** GitHub generates the raw notes from PR labels, so label first, generate second. Wrong labels mean a miscategorised changelog that has to be edited by hand.
+2. **`RELEASE_NOTES_vX.Y.Z.md`** at the repository root, in the house style: what changed, why it matters, PR numbers so people can read the diffs.
+3. **A blog post under `website/content/blog/`.** One post per release, front matter with `title`, `date`, `author`, `category: "Release"`, `tags`, `summary` and `extracss: ["blog.css"]`. Cover the two or three changes that alter what a user does day to day, not the whole changelog, and link the PR numbers. See `website/content/blog/what-landed-in-localai-4-8.md` for the shape.
+4. **Demo clips for the notable features.** Anything visible (a new backend, a UI change, a new endpoint, a measured speedup) gets a short screen recording. Put the file in `website/static/media/`, reference it from the blog post, and reuse it on the marketing pages where it fits.
+
+A release without a post and without clips is incomplete, in the same way a user-facing code change without a docs update is incomplete.
+
+## Clip conventions
+
+- MP4, H.264, no audio track unless the feature is about audio. Keep them short (10 to 30 seconds) and loopable.
+- Record the real thing. A clip from the engine's own benchmark suite or a real session, never a mockup.
+- Where the change is a speedup, record both sides on the same machine on the same input, so the comparison is honest.
+- Name the file after the feature, not the release (`vllm-race.mp4`, not `v4-8-demo.mp4`), so it stays reusable once the release is old.
+- The marketing site plays clips with `muted loop playsinline preload="none"` and a `data-lazy` attribute, which the site's IntersectionObserver uses to play and pause them on scroll. Follow that pattern for anything you add.
+
+## Order of work
+
+Label the PRs, generate and edit the release notes, cut the draft release, record the clips while the branch is still fresh in your head, then write the post against the notes and the clips. Publishing the release and merging the post should happen on the same day.
+
+The `creating-localai-releases` skill drives steps 1 to 3 and captures the React UI screenshots that go into the notes.

.docker/install-base-deps.sh

@@ -113,6 +113,54 @@ if [ "${BUILD_TYPE:-}" = "vulkan" ] && [ "${SKIP_DRIVERS:-false}" = "false" ]; t
     rm -rf /var/lib/apt/lists/*
 fi
 
+# --- 2b. Intel graphics driver (BUILD_TYPE=sycl*) ---
+# The Intel oneAPI base image brings the compilers and the oneAPI libraries, but
+# not the driver that talks to the graphics card. The packaging step copies that
+# driver into the backend, so that the backend works on a machine which has no
+# Intel graphics packages of its own, for the same reason the Vulkan section
+# above installs the Mesa drivers. Install it here so there is something to copy.
+#
+# Only the sycl builds are covered, because those are the ones whose packaging
+# copies the driver. See package_intel_libs in scripts/build/package-gpu-libs.sh.
+#
+# The driver comes from Intel's own package repository, not from the Ubuntu
+# archive. The archive has 23.43 from late 2023, which does not know any card
+# released since, so a machine with a recent Intel GPU would end up carrying a
+# driver that cannot drive it. Intel's repository has 25.18 for the same Ubuntu
+# release.
+#
+# Anything that goes wrong here fails the build, on purpose. An unreachable
+# repository is a passing problem that a retry fixes, whereas carrying a
+# different driver than intended, or none, is a difference nobody would notice
+# until a user reports an idle GPU.
+if case "${BUILD_TYPE:-}" in sycl*) true;; *) false;; esac \
+    && [ "${SKIP_DRIVERS:-false}" = "false" ]; then
+    # Ubuntu release name, which is what the repository is indexed by.
+    ubuntu_codename=$(. /etc/os-release && echo "${VERSION_CODENAME:-}")
+    if [ -z "$ubuntu_codename" ]; then
+        echo "ERROR: cannot tell which Ubuntu release this image is, so cannot pick the Intel driver repository" >&2
+        exit 1
+    fi
+
+    # The key is armored text, which apt reads directly from a .asc file, so
+    # there is no need for gnupg here. "unified" is the component Intel ships
+    # its current driver in.
+    mkdir -p /usr/share/keyrings
+    curl -fsSL https://repositories.intel.com/gpu/intel-graphics.key \
+        -o /usr/share/keyrings/intel-graphics.asc
+    echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.asc] https://repositories.intel.com/gpu/ubuntu ${ubuntu_codename} unified" \
+        > /etc/apt/sources.list.d/intel-graphics.list
+    apt-get update
+    # The first package holds the driver OpenCL talks to, the second the driver
+    # Level Zero talks to. Between them they pull in the compiler and the memory
+    # manager that both need.
+    apt-get install -y --no-install-recommends \
+        intel-opencl-icd \
+        libze-intel-gpu1
+    apt-get clean
+    rm -rf /var/lib/apt/lists/*
+fi
+
 # --- 3. CUDA toolkit (BUILD_TYPE=cublas|l4t) ---
 if { [ "${BUILD_TYPE:-}" = "cublas" ] || [ "${BUILD_TYPE:-}" = "l4t" ]; } && [ "${SKIP_DRIVERS:-false}" = "false" ]; then
     apt-get update

.docker/llama-cpp-build-target.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+arch=${1:?target architecture is required}
+build_type=${2-}
+
+# GPU arm64 base images do not consistently provide the gcc-14 toolchain needed
+# to compile ggml's armv9.2 CPU variants. Keep their portable fallback until the
+# builder images can supply that compiler.
+if [ "$arch" = "arm64" ] && [ -n "$build_type" ]; then
+  echo llama-cpp-fallback
+else
+  echo llama-cpp-cpu-all
+fi

.docker/llama-cpp-compile.sh

@@ -18,10 +18,12 @@ if [[ -n "${CUDA_DOCKER_ARCH:-}" ]]; then
 fi
 
 cd /LocalAI/backend/cpp/llama-cpp
-if [ -z "${BUILD_TYPE:-}" ]; then
-  # Pure CPU image (BUILD_TYPE empty): one build with ggml CPU_ALL_VARIANTS replaces the
-  # per-microarch binaries (x86: avx/avx2/avx512/fallback; arm64: armv8.x/armv9.x). ggml
-  # dlopens the best libggml-cpu-*.so at runtime by probing host CPU features.
+BUILD_TARGET=$(/LocalAI/.docker/llama-cpp-build-target.sh "${TARGETARCH}" "${BUILD_TYPE:-}")
+if [ "$BUILD_TARGET" = "llama-cpp-cpu-all" ]; then
+  # One build with ggml CPU_ALL_VARIANTS replaces the per-microarch binaries (x86:
+  # avx/avx2/avx512/fallback; arm64: armv8.x/armv9.x). BUILD_TYPE remains in the
+  # environment, so GPU builds retain their accelerator backend while ggml dlopens the
+  # best CPU library when work is offloaded to the host.
   #
   # arm64: the CPU_ALL_VARIANTS table includes armv9.2 SME variants whose -march=...+sme is
   # rejected by the Ubuntu 24.04 default gcc-13. gcc-14 accepts it, so build the arm64
@@ -35,14 +37,8 @@ if [ -z "${BUILD_TYPE:-}" ]; then
     apt-get update -qq && apt-get install -y -qq gcc-14 g++-14
     export CC=gcc-14 CXX=g++-14
   fi
-  make llama-cpp-cpu-all
-else
-  # GPU build (cublas/hipblas/sycl/vulkan/...): the accelerator does the compute, so a
-  # single fallback CPU build is enough - no per-microarch CPU variants needed. (This also
-  # keeps the heavy GPU backend compile from also building the whole CPU variant matrix,
-  # and avoids the gcc-14 apt step on GPU base images such as nvidia l4t.)
-  make llama-cpp-fallback
 fi
+make "$BUILD_TARGET"
 make llama-cpp-grpc
 make llama-cpp-rpc-server
 

.docker/turboquant-build-target.sh

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+arch=${1:?target architecture is required}
+build_type=${2-}
+
+# GPU arm64 base images do not consistently provide the gcc-14 toolchain needed
+# to compile ggml's armv9.2 CPU variants. Keep their portable fallback until the
+# builder images can supply that compiler.
+if [ "$arch" = "arm64" ] && [ -n "$build_type" ]; then
+  echo turboquant-fallback
+else
+  echo turboquant-cpu-all
+fi

.docker/turboquant-compile.sh

@@ -19,20 +19,18 @@ fi
 
 cd /LocalAI/backend/cpp/turboquant
 
-if [ -z "${BUILD_TYPE:-}" ]; then
-  # Pure CPU image: one ggml CPU_ALL_VARIANTS build replaces the per-microarch binaries.
+BUILD_TARGET=$(/LocalAI/.docker/turboquant-build-target.sh "${TARGETARCH}" "${BUILD_TYPE:-}")
+if [ "$BUILD_TARGET" = "turboquant-cpu-all" ]; then
+  # BUILD_TYPE remains in the environment, so GPU builds retain their accelerator while
+  # ggml selects the best CPU library when model work is offloaded to the host.
   # arm64: the armv9.2 SME variants need gcc-14 (gcc-13 rejects +sme).
   if [ "${TARGETARCH}" = "arm64" ]; then
+    sh /LocalAI/.docker/apt-mirror.sh || true
     apt-get update -qq && apt-get install -y -qq gcc-14 g++-14
     export CC=gcc-14 CXX=g++-14
   fi
-  make turboquant-cpu-all
-else
-  # GPU build (cublas/hipblas/sycl/vulkan/...): single fallback CPU build, the accelerator
-  # does the compute. Keeps the GPU compile from also building the CPU variant matrix and
-  # avoids the gcc-14 apt step on GPU base images such as nvidia l4t.
-  make turboquant-fallback
 fi
+make "$BUILD_TARGET"
 make turboquant-grpc
 make turboquant-rpc-server
 

.github/ci/gen-redirects.sh

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+#
+# Generate client-side redirects for the documentation URLs that used to live at
+# the site root.
+#
+# Until this site existed, the Hugo docs site WAS localai.io, so pages
+# were published at /features/..., /getting-started/..., /faq/ and so on. The
+# docs now build under /docs/, and GitHub Pages serves static files only: there
+# is no server-side rewrite, no .htaccess, no _redirects. The only way to keep
+# every published, bookmarked and search-indexed URL alive is to leave a real
+# HTML file at the old address that sends the browser to the new one.
+#
+# Anything the main site already publishes wins: it owns /, /engines/,
+# /blog/ and friends, so an existing file is never replaced.
+#
+# Usage: gen-redirects.sh <public-dir> [base-url]
+#   public-dir  merged output directory (main site with docs/ inside it)
+#   base-url    absolute or root-relative prefix the deployment is served from,
+#               trailing slash optional (default "/")
+
+set -euo pipefail
+
+PUBLIC_DIR=${1:?usage: gen-redirects.sh <public-dir> [base-url]}
+BASE_URL=${2:-/}
+
+# Normalise to exactly one trailing slash so concatenation below is predictable.
+BASE_URL="${BASE_URL%/}/"
+
+DOCS_DIR="${PUBLIC_DIR}/docs"
+
+if [ ! -d "$DOCS_DIR" ]; then
+  echo "gen-redirects: no docs output at ${DOCS_DIR}" >&2
+  exit 1
+fi
+
+created=0
+skipped=0
+
+# Every .html file is a reachable old URL, not just directory indexes: the
+# generated model gallery ships as a bare gallery.html and used to sit at the
+# root too.
+while IFS= read -r src; do
+  rel=${src#"$DOCS_DIR"/}
+  dst="${PUBLIC_DIR}/${rel}"
+
+  if [ -e "$dst" ]; then
+    skipped=$((skipped + 1))
+    continue
+  fi
+
+  # Link to the directory, not to its index.html, so the redirect target is the
+  # canonical URL the docs site itself advertises.
+  target="${BASE_URL}docs/${rel%index.html}"
+
+  mkdir -p "$(dirname "$dst")"
+  printf '%s' '<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<title>Moved</title>
+<link rel="canonical" href="'"$target"'">
+<meta name="robots" content="noindex">
+<meta http-equiv="refresh" content="0; url='"$target"'">
+</head>
+<body>
+<p>This page moved to <a href="'"$target"'">'"$target"'</a>.</p>
+</body>
+</html>
+' > "$dst"
+
+  created=$((created + 1))
+done <<EOF
+$(find "$DOCS_DIR" -type f -name '*.html' | sort)
+EOF
+
+echo "gen-redirects: ${created} redirect(s) written, ${skipped} path(s) left to the main site"

.github/ci/refresh-site-counters.sh

@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+# Refreshes the counters shown on the landing page from the GitHub API.
+#
+# The numbers used to be typed into the templates by hand, which meant they
+# only moved when somebody remembered, and a stale star count on the front
+# page is worse than no star count. Everything the API can answer for lives
+# in website/data/stats.yaml and is rewritten wholesale by this script.
+#
+# Anything the API cannot answer for (the Discord member count) is read back
+# out of the existing file and carried through untouched.
+set -euo pipefail
+
+REPO="${REPO:-mudler/LocalAI}"
+OUT="${OUT:-website/data/stats.yaml}"
+
+# The contributors and releases endpoints are paginated and never report a
+# total. Asking for one item per page makes the last page number equal to the
+# item count, which the Link header hands over.
+count_via_link_header() {
+  local path="$1" link last
+  link=$(gh api -i "${path}?per_page=1" 2>/dev/null | tr -d '\r' | grep -i '^link:' || true)
+  if [ -z "$link" ]; then
+    # No Link header means a single page, so count that page directly.
+    gh api "${path}?per_page=100" --jq 'length'
+    return
+  fi
+  last=$(sed -n 's/.*[?&]page=\([0-9]*\)>; rel="last".*/\1/p' <<<"$link")
+  [ -n "$last" ] || { gh api "${path}?per_page=100" --jq 'length'; return; }
+  printf '%s\n' "$last"
+}
+
+read -r stars forks < <(gh api "repos/${REPO}" --jq '"\(.stargazers_count) \(.forks_count)"')
+contributors=$(count_via_link_header "repos/${REPO}/contributors")
+releases=$(count_via_link_header "repos/${REPO}/releases")
+
+# Not derivable from the GitHub API, so keep whatever is already on disk.
+discord=$(sed -n 's/^discord: *\([0-9]*\).*/\1/p' "$OUT" 2>/dev/null | head -1)
+discord="${discord:-0}"
+
+for n in stars forks contributors releases; do
+  v="${!n}"
+  [[ "$v" =~ ^[0-9]+$ ]] && [ "$v" -gt 0 ] || {
+    echo "refusing to write: ${n} came back as '${v}'" >&2
+    exit 1
+  }
+done
+
+cat > "$OUT" <<YAML
+# Counters shown on the landing page.
+#
+# The four GitHub fields are rewritten by .github/ci/refresh-site-counters.sh,
+# which runs weekly from .github/workflows/refresh-site-counters.yml. Editing
+# them by hand works but will be overwritten on the next run.
+stars: ${stars}
+forks: ${forks}
+contributors: ${contributors}
+releases: ${releases}
+
+# The GitHub API cannot answer for this one, so it is maintained by hand and
+# the refresh script carries it through untouched.
+discord: ${discord}
+YAML
+
+echo "stars=${stars} forks=${forks} contributors=${contributors} releases=${releases} discord=${discord}"

.github/workflows/gh-pages.yml

@@ -1,4 +1,4 @@
-name: Deploy docs to GitHub Pages
+name: Deploy site to GitHub Pages
 
 on:
   push:
@@ -6,9 +6,11 @@ on:
       - master
     paths:
       - 'docs/**'
+      - 'website/**'
       - 'gallery/**'
       - 'images/**'
       - '.github/ci/modelslist.go'
+      - '.github/ci/gen-redirects.sh'
       - '.github/workflows/gh-pages.yml'
   workflow_dispatch:
 
@@ -23,7 +25,20 @@ concurrency:
 
 jobs:
   build:
-    runs-on: ubuntu-latest
+    # Self-hosted. This workflow is push-to-master + workflow_dispatch only, so
+    # it never executes pull-request code and a fork cannot reach the runner
+    # with untrusted changes. The repository guard keeps forks (whose own master
+    # pushes would otherwise queue forever against a label they do not have) on
+    # the hosted pool.
+    #
+    # Why: the GitHub-hosted pool is shared account-wide and has repeatedly
+    # starved (2026-07-31: 35 consecutive minutes at zero scheduled jobs, while
+    # arc-runner-set kept completing work throughout). Publishing the site is
+    # small, frequent, and must not sit behind a saturated hosted queue.
+    #
+    # Needs only git, tar and curl on the runner: setup-go and actions-hugo
+    # fetch their own toolchains, and no step uses sudo, apt, make or unzip.
+    runs-on: ${{ github.repository == 'mudler/LocalAI' && 'arc-runner-set' || 'ubuntu-latest' }}
     env:
       HUGO_VERSION: "0.146.3"
     steps:
@@ -49,25 +64,46 @@ jobs:
         id: pages
         uses: actions/configure-pages@v6
 
+      # The gallery page is generated from the model index and shipped as a
+      # static asset of the docs site, so it has to exist before Hugo runs.
       - name: Generate gallery
         run: go run ./.github/ci/modelslist.go ./gallery/index.yaml > docs/static/gallery.html
 
-      - name: Build site
+      # Two Hugo sites, one Pages artifact: the main site owns the root,
+      # the docs site is nested under /docs/.
+      - name: Build the main site
+        working-directory: website
+        run: hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/"
+
+      - name: Build documentation site
         working-directory: docs
         run: |
           mkdir -p layouts/_default
-          hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/"
+          hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/docs/"
+
+      - name: Merge documentation into the main site
+        run: |
+          mkdir -p website/public/docs
+          cp -R docs/public/. website/public/docs/
+
+      # Keeps the pre-split URLs alive; see the script header.
+      - name: Generate legacy URL redirects
+        run: .github/ci/gen-redirects.sh website/public "${{ steps.pages.outputs.base_url }}/"
 
       - name: Upload artifact
         uses: actions/upload-pages-artifact@v5
         with:
-          path: docs/public
+          path: website/public
 
   deploy:
     environment:
       name: github-pages
       url: ${{ steps.deployment.outputs.page_url }}
-    runs-on: ubuntu-latest
+    # Same routing as build: a hosted slot for a ~10s deploy is exactly the kind
+    # of job that should not block on a starved pool. deploy-pages authenticates
+    # with the job's OIDC token (id-token: write above), which self-hosted
+    # runners issue the same way hosted ones do.
+    runs-on: ${{ github.repository == 'mudler/LocalAI' && 'arc-runner-set' || 'ubuntu-latest' }}
     needs: build
     steps:
       - name: Deploy to GitHub Pages

.github/workflows/lint.yml

@@ -21,8 +21,41 @@ concurrency:
 
 jobs:
   golangci-lint:
+    # Self-hosted for PUSH only, and only in the canonical repo.
+    #
+    # This workflow also runs on pull_request, which for a fork PR means
+    # executing untrusted contributor code. That must never land on a
+    # self-hosted runner, so anything that is not a push to mudler/LocalAI stays
+    # on the ephemeral hosted pool. Pushes to master are trusted code that has
+    # already been reviewed and merged.
+    #
+    # Why at all: the hosted pool is shared account-wide and starved for 35
+    # straight minutes on 2026-07-31 while arc-runner-set kept completing jobs.
+    # Lint is small and runs on every commit, so it is a good candidate to move
+    # off the contended pool.
+    # REVERTED to hosted: the arc-runner-set image has git, curl, unzip, tar,
+    # ldd and python3, but NOT make (nor gcc). Measured on run 30637392862,
+    # where the preflight below named both. Re-route here once the runner image
+    # ships a C toolchain and make; the preflight stays so the next attempt
+    # fails by name in one second instead of opaquely mid-build.
     runs-on: ubuntu-latest
     steps:
+      - name: Preflight - required host tools
+        # The hosted images ship these; a self-hosted container image may not.
+        # Check up front so a missing tool reports itself by name instead of
+        # surfacing as an opaque failure inside `make protogen-go` (which needs
+        # curl + unzip for protoc) or `make lint`.
+        run: |
+          missing=""
+          for t in git curl unzip make tar; do
+            command -v "$t" >/dev/null 2>&1 || missing="$missing $t"
+          done
+          echo "runner: ${RUNNER_NAME:-unknown}  os: $(uname -sm)"
+          if [ -n "$missing" ]; then
+            echo "::error::missing required tools on this runner:$missing"
+            exit 1
+          fi
+          echo "all required tools present"
       - uses: actions/checkout@v7
         with:
           # Full history so golangci-lint's new-from-merge-base can reach
@@ -55,8 +88,30 @@ jobs:
     # container build (a missing transitive dep, a partial cuDNN family). Their
     # shell tests need nothing but bash + gcc + ldd, so run them on every PR
     # rather than waiting on a multi-GB cross-arch backend image build.
+    #
+    # Push-only self-hosted routing, same fork-safety reasoning as
+    # golangci-lint above.
+    # REVERTED to hosted: the arc-runner-set image has git, curl, unzip, tar,
+    # ldd and python3, but NOT make (nor gcc). Measured on run 30637392862,
+    # where the preflight below named both. Re-route here once the runner image
+    # ships a C toolchain and make; the preflight stays so the next attempt
+    # fails by name in one second instead of opaquely mid-build.
     runs-on: ubuntu-latest
     steps:
+      - name: Preflight - required host tools
+        # This job additionally needs a C toolchain: the packaging-script tests
+        # compile a throwaway binary and inspect it with ldd.
+        run: |
+          missing=""
+          for t in git make gcc ldd python3; do
+            command -v "$t" >/dev/null 2>&1 || missing="$missing $t"
+          done
+          echo "runner: ${RUNNER_NAME:-unknown}  os: $(uname -sm)"
+          if [ -n "$missing" ]; then
+            echo "::error::missing required tools on this runner:$missing"
+            exit 1
+          fi
+          echo "all required tools present"
       - uses: actions/checkout@v7
       - name: run packaging script tests
         run: make test-build-scripts

.github/workflows/refresh-site-counters.yml

@@ -0,0 +1,44 @@
+name: Refresh site counters
+
+# The landing page shows a star count, a contributor count and a release
+# count. They were typed in by hand, so they drifted the moment somebody
+# forgot. This pulls the real numbers once a week and commits them only when
+# they have actually moved, which in turn triggers the usual Pages deploy.
+
+on:
+  schedule:
+    # Mondays, 06:17 UTC. Off the hour on purpose, since the scheduler queues
+    # everything that asks for :00 and drops what it cannot run.
+    - cron: '17 6 * * 1'
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+concurrency:
+  group: refresh-site-counters
+  cancel-in-progress: false
+
+jobs:
+  refresh:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Read the counts off the GitHub API
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: ./.github/ci/refresh-site-counters.sh
+
+      - name: Commit only if something moved
+        run: |
+          if git diff --quiet -- website/data/stats.yaml; then
+            echo "counters unchanged, nothing to commit"
+            exit 0
+          fi
+          git diff --unified=0 -- website/data/stats.yaml
+          git config user.name  "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+          git add website/data/stats.yaml
+          git commit -m "chore(website): refresh the counters"
+          git push

.gitignore

@@ -62,6 +62,11 @@ prepare
 /ggml-metal.metal
 docs/static/gallery.html
 
+# Hugo build output and lock files (docs/ and website/)
+docs/public/
+website/public/
+.hugo_build.lock
+
 # Protobuf generated files
 *.pb.go
 *pb2.py

ADOPTERS.md

@@ -0,0 +1,48 @@
+# Adopters
+
+Organisations running LocalAI, listed by the people who run it.
+
+If your organisation uses LocalAI and you are happy to say so publicly, open a
+pull request adding a row to the table below. That pull request is how we know
+we have permission to list you, which is why we do not add anybody ourselves.
+
+You do not need to be a large company, and you do not need to disclose anything
+sensitive. A sentence on what you use it for is more useful to other readers
+than a logo.
+
+## How to add yourself
+
+1. Add a row to the table, in alphabetical order.
+2. Use your organisation's usual name and a link to your site.
+3. Say briefly what you use LocalAI for, and whether it is in production.
+4. Open the pull request from an account that makes it plausible you speak for
+   the organisation, or say in the description who you are. We may ask.
+
+To be removed, open a pull request deleting your row, or email
+[info@localai.io](mailto:info@localai.io). We will not ask why.
+
+## Who is using LocalAI
+
+<!-- Keep alphabetical. Columns: Organisation | What for | Status -->
+
+| Organisation | What they use it for | Status |
+|---|---|---|
+| _Your organisation here_ | | |
+
+## What this list is not
+
+This is not a list of everyone who has ever starred the repository, and it is
+not a list of the employers of people who have contributed a patch. Both of
+those are easy to scrape and neither means what a logo wall implies.
+
+The website shows two separate things, both of which are checkable without
+anybody's permission:
+
+- **Engineers from these companies have contributed code.** Evidence is the
+  commit history plus the employer on that person's public GitHub profile. It
+  is a claim about a person, not about their employer.
+- **These projects integrate LocalAI.** Evidence is a reference to LocalAI in
+  that project's own repository or documentation.
+
+Those two lists live in [`website/data/ecosystem.yaml`](website/data/ecosystem.yaml).
+This file is the third, stronger thing: organisations that chose to say so.

AGENTS.md

@@ -32,6 +32,7 @@ LocalAI follows the Linux kernel project's [guidelines for AI coding assistants]
 | [.agents/adding-gallery-models.md](.agents/adding-gallery-models.md) | Adding GGUF models from HuggingFace to the model gallery |
 | [.agents/localai-assistant-mcp.md](.agents/localai-assistant-mcp.md) | LocalAI Assistant chat modality — adding admin tools to the in-process MCP server, editing skill prompts, keeping REST + MCP + skills in sync |
 | [.agents/backend-signing.md](.agents/backend-signing.md) | Backend OCI image signing (keyless cosign + sigstore-go) — producer-side CI setup, consumer-side gallery `verification:` block, strict mode (`LOCALAI_REQUIRE_BACKEND_INTEGRITY`), revocation via `not_before` |
+| [.agents/preparing-a-release.md](.agents/preparing-a-release.md) | Cutting a release: PR labels, `RELEASE_NOTES_vX.Y.Z.md`, the blog post under `website/content/blog/`, and the demo clips under `website/static/media/` |
 
 ## Quick Reference
 
@@ -42,6 +43,7 @@ LocalAI follows the Linux kernel project's [guidelines for AI coding assistants]
 - **Docs (docs-with-code rule)**: When you change user-facing behavior (API endpoints, CLI flags, config keys, or features), update the corresponding page under `docs/content/` in the SAME change, not as a follow-up. A user-facing change without a matching docs update is incomplete. See also the documentation conventions in [.agents/coding-style.md](.agents/coding-style.md).
 - **New API endpoints**: LocalAI advertises its capability surface in several independent places — swagger `@Tags`, `/api/instructions` registry, auth `RouteFeatureRegistry`, React UI `capabilities.js`, docs. Read [.agents/api-endpoints-and-auth.md](.agents/api-endpoints-and-auth.md) and follow its checklist — missing any surface means clients, admins, and the UI won't know the endpoint exists.
 - **Admin endpoints → MCP tool**: every admin endpoint that an admin would manage conversationally (install/list/edit/toggle/upgrade) MUST also be exposed as an MCP tool in `pkg/mcp/localaitools/`. The LocalAI Assistant chat modality and the standalone `local-ai mcp-server` consume that package; drift between REST and MCP is a real risk. Read [.agents/localai-assistant-mcp.md](.agents/localai-assistant-mcp.md) — the `TestToolHTTPRouteMappingComplete` test fails until you wire the new tool and update the route map.
+- **Releases ship with a post and clips**: a release is not done at the tag. It needs labelled PRs, `RELEASE_NOTES_vX.Y.Z.md`, a blog post under `website/content/blog/`, and a short demo clip in `website/static/media/` for each notable feature. See [.agents/preparing-a-release.md](.agents/preparing-a-release.md).
 - **Build**: Inspect `Makefile` and `.github/workflows/` — ask the user before running long builds
 - **Backend OS coverage**: a new backend must target every OS it can build for, not just Linux. `.github/backend-matrix.yml` has two matrices — `include:` (Linux) and `includeDarwin:` (macOS / Apple Silicon). Most C/C++/GGML and many Python backends build on Darwin too — wire the `includeDarwin` entry + `backend/index.yaml` `metal:` entries, or say in the PR why an OS is unsupported. See the darwin checklist in [.agents/adding-backends.md](.agents/adding-backends.md).
 - **Gallery variant ranking**: a gallery entry can declare `variants` (alternative builds of the same weights), and LocalAI ranks the ones a host can run by engine preference first, size second. A new backend that should be preferred on some hardware must be listed in `engineNamePreferenceRules` in `pkg/system/capabilities.go`; the sibling `backendBuildTagPreferenceRules` speaks build tags rather than engine names, and using the wrong table matches nothing without erroring. See [.agents/adding-backends.md](.agents/adding-backends.md).

Makefile

@@ -1548,7 +1548,12 @@ swagger:
 gen-assets:
 	$(GOCMD) run core/dependencies_manager/manager.go webui_static.yaml core/http/static/assets
 
-## Documentation
+## Documentation and website
+# The published site is two Hugo sites: website/ owns the root, docs/ is nested
+# under /docs/. Serve them separately while editing; use `make site` to get the
+# merged tree (including the legacy URL redirects) that GitHub Pages deploys.
+SITE_BASE_URL?=http://localhost:8000
+
 docs/layouts/_default:
 	mkdir -p docs/layouts/_default
 
@@ -1560,12 +1565,30 @@ docs/public: docs/layouts/_default docs/static/gallery.html
 
 docs-clean:
 	rm -rf docs/public
+	rm -rf website/public
 	rm -rf docs/static/gallery.html
 
 .PHONY: docs
 docs: docs/static/gallery.html
 	cd docs && hugo serve
 
+.PHONY: website
+website:
+	cd website && hugo serve
+
+.PHONY: site
+site: docs/static/gallery.html
+	rm -rf website/public docs/public
+	cd website && hugo --minify --baseURL "$(SITE_BASE_URL)/"
+	cd docs && hugo --minify --baseURL "$(SITE_BASE_URL)/docs/"
+	mkdir -p website/public/docs
+	cp -R docs/public/. website/public/docs/
+	./.github/ci/gen-redirects.sh website/public "$(SITE_BASE_URL)/"
+
+.PHONY: site-serve
+site-serve: site
+	cd website/public && python3 -m http.server 8000
+
 ########################################################
 ## Platform-specific builds
 ########################################################

README.md

@@ -238,7 +238,7 @@ Most backends wrap a best-in-class upstream engine. A handful of them are native
 | [magpie-tts.cpp](https://github.com/mudler/magpie-tts.cpp) | C++/GGML port of NVIDIA's Magpie TTS Multilingual 357M: 22.05 kHz mono text-to-speech in 5 voices and 9+ languages, with the NanoCodec neural codec and tokenizer/G2P embedded in a single GGUF |
 | [ced.cpp](https://github.com/localai-org/ced.cpp) | C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition |
 | [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) | Speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend |
-| [voxtral-tts.c](https://github.com/mudler/voxtral-tts.c) | Voxtral Realtime 4B speech-to-text in pure C |
+| [voxtral-tts.c](https://github.com/mudler/voxtral-tts.c) | Mistral Voxtral-4B-TTS text-to-speech in pure C: 20 preset voices across 9 languages, 24 kHz WAV output, no dependencies beyond libc |
 | [vibevoice.cpp](https://github.com/mudler/vibevoice.cpp) | Native port of Microsoft VibeVoice for TTS (voice cloning) and long-form ASR with speaker diarization |
 | [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation |
 | [locate-anything.cpp](https://github.com/mudler/locate-anything.cpp) | Open-vocabulary object detection and visual grounding (LocateAnything-3B) |

backend/cpp/audio-cpp/Makefile

@@ -9,7 +9,7 @@
 # recipe is a make target (not a prepare.sh) so 'make purge && make' is a clean
 # rebuild and so the bump bot can see the pin.
 
-AUDIO_CPP_VERSION?=e800d435d130dc776baf6f3e6129bb62b1495c89
+AUDIO_CPP_VERSION?=f78227c52736a4792a50aa3f82ead7e7385c891b
 AUDIO_CPP_REPO?=https://github.com/0xShug0/audio.cpp
 
 CURRENT_MAKEFILE_DIR := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))

backend/cpp/bonsai/Makefile

@@ -1,7 +1,7 @@
 
 # Pinned to the HEAD of the `prism` branch on https://github.com/PrismML-Eng/llama.cpp.
 # Auto-bumped nightly by .github/workflows/bump_deps.yaml.
-BONSAI_VERSION?=7529fdaaf99ffdc5ca71ace9c7409a56b27ad92f
+BONSAI_VERSION?=4dd165625bb6c020285eec8b342af25cf60233dd
 LLAMA_REPO?=https://github.com/PrismML-Eng/llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/bonsai/run.sh

@@ -40,6 +40,27 @@ else
 	if [ -d "$CURDIR/lib/hipblaslt/library" ]; then
 		export HIPBLASLT_TENSILE_LIBPATH="$CURDIR"/lib/hipblaslt/library
 	fi
+	# Backends built for Intel GPUs carry a copy of the Intel graphics driver,
+	# and libze_loader is only there in those builds. Level Zero looks for a
+	# driver on its own, so point it at the copy that came with this backend: it
+	# was built against the same C library, while the machine's own driver may
+	# not have been, and loading that one can crash on start.
+	#
+	# Anything the user set is left alone, so a machine with a graphics card
+	# newer than the driver carried here can still be told to use its own.
+	# Nothing is said about OpenCL: no OpenCL driver is carried, so anything we
+	# set there would leave OpenCL worse off than the machine's own setup.
+	if [ -e "$CURDIR/lib/libze_loader.so.1" ]; then
+		if [ -e "$CURDIR/lib/libze_intel_gpu.so.1" ] && [ -z "${ZE_ENABLE_ALT_DRIVERS:-}" ]; then
+			export ZE_ENABLE_ALT_DRIVERS="$CURDIR"/lib/libze_intel_gpu.so.1
+		fi
+		# Ask the driver how much graphics memory is free. Without this, the
+		# backend reads zero on an integrated graphics chip, because such a chip
+		# shares the system memory instead of having its own.
+		if [ -z "${ZES_ENABLE_SYSMAN:-}" ]; then
+			export ZES_ENABLE_SYSMAN=1
+		fi
+	fi
 fi
 
 # If there is a lib/ld.so, use it

backend/cpp/ik-llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-IK_LLAMA_VERSION?=6647db9c27760044950fd6f99060456ae3d15df3
+IK_LLAMA_VERSION?=3f53a059024039358e9fef75b5dc0c99dbcb40f9
 LLAMA_REPO?=https://github.com/ikawrakow/ik_llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-LLAMA_VERSION?=1cbfd1988311775425d36c0ce066590f7d3049cf
+LLAMA_VERSION?=876a4321163249c43ca4e986818fab5ab081f282
 LLAMA_REPO?=https://github.com/ggerganov/llama.cpp
 
 CMAKE_ARGS?=

backend/cpp/llama-cpp/patches/0001-add-minimax-m3-chat-parser.patch

@@ -1,225 +0,0 @@
-# MiniMax-M3 chat-template parser, vendored from upstream llama.cpp PR #24523.
-#
-# Upstream has since merged the *model* half of #24523 (LLM_ARCH_MINIMAX_M3,
-# src/models/minimax-m3.cpp, the gguf-py constants and conversion/minimax.py), so
-# only the chat half is carried here: M3's namespace token "]<]minimax[>[" collides
-# with the autoparser's markup delimiters, so common/chat.cpp needs a dedicated
-# template detection + PEG parser that upstream does not have yet.
-#
-# Rebased against LLAMA_VERSION 0d47ea7427463093e69128bf2c2f9cd06b3ee5b3, which also
-# renamed common_chat_params::thinking_end_tag to thinking_end_tags (a vector).
-# LLAMA_VERSION is auto-bumped nightly; if a bump rejects this patch, re-vendor from
-# #24523 — or, once the chat half merges upstream, delete this file.
-# See https://github.com/mudler/LocalAI/issues/10820 and PR #10837.
-diff --git a/common/chat.cpp b/common/chat.cpp
-index 7a6e7238c..2dd015a2e 100644
---- a/common/chat.cpp
-+++ b/common/chat.cpp
-@@ -2121,6 +2121,191 @@ static common_chat_params common_chat_params_init_deepseek_v3_2(const common_cha
-     return data;
- }
- 
-+static common_chat_params common_chat_params_init_minimax_m3(const common_chat_template &    tmpl,
-+                                                             const autoparser::generation_params & inputs) {
-+    common_chat_params data;
-+
-+    data.prompt             = common_chat_template_direct_apply_impl(tmpl, inputs);
-+    data.generation_prompt  = common_chat_template_generation_prompt_impl(tmpl, inputs);
-+    data.format             = COMMON_CHAT_FORMAT_PEG_NATIVE;
-+    data.supports_thinking  = true;
-+    data.thinking_start_tag = "<mm:think>";
-+    data.thinking_end_tags  = {"</mm:think>"};
-+
-+    // M3 prefixes every tool tag with the namespace token "]<]minimax[>[";
-+    // params use the parameter name as the tag (<file_path>...</file_path>).
-+    const std::string NS           = "]<]minimax[>[";
-+    const std::string THINK_START  = "<mm:think>";
-+    const std::string THINK_END    = "</mm:think>";
-+    const std::string FC_START     = NS + "<tool_call>";
-+    const std::string FC_END       = NS + "</tool_call>";
-+    const std::string INVOKE_END   = NS + "</invoke>";
-+
-+    data.preserved_tokens   = {
-+        NS,
-+        "<tool_call>",
-+        "</tool_call>",
-+        THINK_START,
-+        THINK_END,
-+    };
-+
-+    auto has_tools           = inputs.tools.is_array() && !inputs.tools.empty();
-+    auto has_response_format = !inputs.json_schema.is_null() && inputs.json_schema.is_object();
-+    auto extract_reasoning   = inputs.reasoning_format != COMMON_REASONING_FORMAT_NONE;
-+    auto include_grammar     = has_response_format || (has_tools && inputs.tool_choice != COMMON_CHAT_TOOL_CHOICE_NONE);
-+
-+    const std::string GEN_PROMPT = data.generation_prompt;
-+
-+    if (inputs.has_continuation()) {
-+        const auto & msg = inputs.continue_msg;
-+
-+        data.generation_prompt = GEN_PROMPT + THINK_START + msg.reasoning_content;
-+        if (inputs.continue_final_message == COMMON_CHAT_CONTINUATION_CONTENT) {
-+            data.generation_prompt += THINK_END + msg.render_content();
-+        }
-+
-+        data.prompt += data.generation_prompt;
-+    }
-+
-+    auto parser = build_chat_peg_parser([&](common_chat_peg_builder & p) {
-+        auto generation_prompt = p.literal(GEN_PROMPT);
-+        auto end = p.end();
-+
-+        auto reasoning = p.eps();
-+        // M3 can emit a bare </mm:think> (no opener) after tool results; keep the opener optional.
-+        if (extract_reasoning && inputs.enable_thinking) {
-+            reasoning = p.optional(p.optional(p.literal(THINK_START)) + p.reasoning(p.until(THINK_END)) + THINK_END);
-+        } else if (extract_reasoning) {
-+            reasoning = p.optional(p.optional(p.literal(THINK_START)) + p.until(THINK_END) + p.literal(THINK_END));
-+        }
-+
-+        if (has_response_format) {
-+            auto response_format = p.rule("response-format",
-+                p.literal("```json") + p.space() +
-+                p.content(p.schema(p.json(), "response-format-schema", inputs.json_schema)) +
-+                p.space() + p.literal("```"));
-+            return generation_prompt + reasoning + response_format + end;
-+        }
-+
-+        if (!has_tools || inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_NONE) {
-+            return generation_prompt + reasoning + p.content(p.rest()) + end;
-+        }
-+
-+        auto tool_choice = p.choice();
-+        foreach_function(inputs.tools, [&](const json & tool) {
-+            const auto & function = tool.at("function");
-+            std::string  name     = function.at("name");
-+            auto params   = function.contains("parameters") ? function.at("parameters") : json::object();
-+            const auto & props    = params.contains("properties") ? params.at("properties") : json::object();
-+
-+            std::set<std::string> required;
-+            if (params.contains("required")) {
-+                params.at("required").get_to(required);
-+            }
-+
-+            auto schema_info = common_schema_info();
-+            schema_info.resolve_refs(params);
-+
-+            std::vector<common_peg_parser> required_parsers;
-+            std::vector<common_peg_parser> optional_parsers;
-+            for (const auto & [param_name, param_schema] : props.items()) {
-+                bool is_required = required.find(param_name) != required.end();
-+                bool is_string   = schema_info.resolves_to_string(param_schema);
-+
-+                const std::string p_close = NS + "</" + param_name + ">";
-+
-+                auto arg = p.tool_arg(
-+                    p.tool_arg_open(
-+                        p.literal(NS + "<") +
-+                        p.tool_arg_name(p.literal(param_name)) +
-+                        p.literal(">")) +
-+                    (is_string
-+                         ? p.ac(p.tool_arg_string_value(p.until(p_close)) +
-+                                p.tool_arg_close(p.literal(p_close)), p_close)
-+                         : p.tool_arg_json_value(p.schema(p.json(),
-+                                                          "tool-" + name + "-arg-" + param_name + "-schema",
-+                                                          param_schema, false)) +
-+                           p.tool_arg_close(p.literal(p_close))));
-+
-+                auto named_arg = p.rule("tool-" + name + "-arg-" + param_name, arg);
-+                if (is_required) {
-+                    required_parsers.push_back(named_arg);
-+                } else {
-+                    optional_parsers.push_back(named_arg);
-+                }
-+            }
-+
-+            common_peg_parser args_seq = p.eps();
-+            for (size_t i = 0; i < required_parsers.size(); i++) {
-+                if (i > 0) {
-+                    args_seq = args_seq + p.space();
-+                }
-+                args_seq = args_seq + required_parsers[i];
-+            }
-+
-+            if (!optional_parsers.empty()) {
-+                common_peg_parser any_opt = p.choice();
-+                for (const auto & opt : optional_parsers) {
-+                    any_opt |= opt;
-+                }
-+                args_seq = args_seq + p.repeat(p.space() + any_opt, 0, -1);
-+            }
-+
-+            common_peg_parser invoke_body = args_seq;
-+            auto func_parser = p.tool(
-+                p.tool_open(p.literal(NS + "<invoke name=\"") +
-+                            p.tool_name(p.literal(name)) + p.literal("\">")) +
-+                p.space() + invoke_body + p.space() +
-+                p.tool_close(p.literal(INVOKE_END)));
-+
-+            tool_choice |= p.rule("tool-" + name, func_parser);
-+        });
-+
-+        auto require_tools = inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_REQUIRED;
-+
-+        common_peg_parser tool_calls = p.eps();
-+        if (inputs.parallel_tool_calls) {
-+            tool_calls = p.trigger_rule("tool-call",
-+                p.literal(FC_START) + p.space() + tool_choice +
-+                p.zero_or_more(p.space() + tool_choice) + p.space() + p.literal(FC_END));
-+        } else {
-+            tool_calls = p.trigger_rule("tool-call",
-+                p.literal(FC_START) + p.space() + tool_choice + p.space() + p.literal(FC_END));
-+        }
-+
-+        if (!require_tools) {
-+            tool_calls = p.optional(tool_calls);
-+        }
-+
-+        auto content_before_tools = p.content(p.until(FC_START));
-+        return generation_prompt + reasoning + content_before_tools + tool_calls + end;
-+    });
-+
-+    data.parser = parser.save();
-+
-+    if (include_grammar) {
-+        data.grammar_lazy = !(has_response_format || (has_tools && inputs.tool_choice == COMMON_CHAT_TOOL_CHOICE_REQUIRED));
-+        data.grammar      = build_grammar([&](const common_grammar_builder & builder) {
-+            foreach_function(inputs.tools, [&](const json & tool) {
-+                const auto & function = tool.at("function");
-+                auto         schema   = function.contains("parameters") ? function.at("parameters") : json::object();
-+                builder.resolve_refs(schema);
-+            });
-+            if (has_response_format) {
-+                auto schema = inputs.json_schema;
-+                builder.resolve_refs(schema);
-+            }
-+            parser.build_grammar(builder, data.grammar_lazy);
-+        });
-+
-+        data.grammar_triggers = {
-+            { COMMON_GRAMMAR_TRIGGER_TYPE_WORD, FC_START },
-+        };
-+    }
-+
-+    return data;
-+}
-+
- // Cohere2 MoE (a.k.a. "North Code") parser.
- //
- // The assistant turn is fully marker-wrapped:
-@@ -2707,6 +2892,15 @@ std::optional<common_chat_params> common_chat_try_specialized_template(
-         return common_chat_params_init_gigachat_v3(tmpl, params);
-     }
- 
-+    // MiniMax-M3: the namespace token "]<]minimax[>[" collides with the autoparser's
-+    // markup delimiters, so detect the template and use a dedicated parser.
-+    if (src.find("]<]minimax[>[") != std::string::npos &&
-+        src.find("<tool_call>") != std::string::npos &&
-+        src.find("<invoke name=") != std::string::npos) {
-+        LOG_DBG("Using specialized template: MiniMax-M3\n");
-+        return common_chat_params_init_minimax_m3(tmpl, params);
-+    }
-+
-     // DeepSeek V3.2/V4 format detection: template defines dsml_token and uses it for tool calls.
-     // The template source contains the token as a variable assignment, not as a literal in markup.
-     // V3.2 names the tool call block "function_calls", V4 names it "tool_calls".

backend/cpp/llama-cpp/run.sh

@@ -12,10 +12,10 @@ grep -e "flags" /proc/cpuinfo | head -1
 
 BINARY=llama-cpp-fallback
 
-# CPU images (x86, arm64, darwin) ship a single llama-cpp-cpu-all built with ggml
+# CPU images and x86 GPU images ship a single llama-cpp-cpu-all built with ggml
 # CPU_ALL_VARIANTS: ggml's backend registry dlopens the best libggml-cpu-*.so for this
-# host, so no shell-side AVX probing. GPU images (cublas/sycl/vulkan/hipblas) ship only
-# llama-cpp-fallback (the accelerator does the compute), so fall back to it when absent.
+# host, so no shell-side AVX probing. GPU arm64 images still ship llama-cpp-fallback
+# until their builder toolchains support ggml's complete arm variant matrix.
 if [ -e "$CURDIR"/llama-cpp-cpu-all ]; then
 	BINARY=llama-cpp-cpu-all
 fi
@@ -42,6 +42,27 @@ else
 	if [ -d "$CURDIR/lib/hipblaslt/library" ]; then
 		export HIPBLASLT_TENSILE_LIBPATH="$CURDIR"/lib/hipblaslt/library
 	fi
+	# Backends built for Intel GPUs carry a copy of the Intel graphics driver,
+	# and libze_loader is only there in those builds. Level Zero looks for a
+	# driver on its own, so point it at the copy that came with this backend: it
+	# was built against the same C library, while the machine's own driver may
+	# not have been, and loading that one can crash on start.
+	#
+	# Anything the user set is left alone, so a machine with a graphics card
+	# newer than the driver carried here can still be told to use its own.
+	# Nothing is said about OpenCL: no OpenCL driver is carried, so anything we
+	# set there would leave OpenCL worse off than the machine's own setup.
+	if [ -e "$CURDIR/lib/libze_loader.so.1" ]; then
+		if [ -e "$CURDIR/lib/libze_intel_gpu.so.1" ] && [ -z "${ZE_ENABLE_ALT_DRIVERS:-}" ]; then
+			export ZE_ENABLE_ALT_DRIVERS="$CURDIR"/lib/libze_intel_gpu.so.1
+		fi
+		# Ask the driver how much graphics memory is free. Without this,
+		# llama.cpp reads zero on an integrated graphics chip, because such a
+		# chip shares the system memory instead of having its own.
+		if [ -z "${ZES_ENABLE_SYSMAN:-}" ]; then
+			export ZES_ENABLE_SYSMAN=1
+		fi
+	fi
 fi
 
 # If there is a lib/ld.so, use it
@@ -55,4 +76,4 @@ echo "Using binary: $BINARY"
 exec "$CURDIR"/$BINARY "$@"
 
 # We should never reach this point, however just in case we do, run fallback
-exec "$CURDIR"/llama-cpp-fallback "$@"
+exec "$CURDIR"/llama-cpp-fallback "$@"

backend/cpp/turboquant/Makefile

@@ -1,7 +1,7 @@
 
 # Pinned to the HEAD of feature/turboquant-kv-cache on https://github.com/TheTom/llama-cpp-turboquant.
 # Auto-bumped nightly by .github/workflows/bump_deps.yaml.
-TURBOQUANT_VERSION?=c26cbdffcf6fc9b7430cd6b117757e9a3f70b7ea
+TURBOQUANT_VERSION?=8a891f4b566efdbd3cea92fafee3227a0a267683
 LLAMA_REPO?=https://github.com/TheTom/llama-cpp-turboquant
 
 CMAKE_ARGS?=

backend/cpp/turboquant/run.sh

@@ -12,9 +12,11 @@ grep -e "flags" /proc/cpuinfo | head -1
 
 BINARY=turboquant-fallback
 
-# x86/arm64 ship a single turboquant-cpu-all built with ggml CPU_ALL_VARIANTS: ggml's
+# CPU images and x86 GPU images ship a single turboquant-cpu-all built with ggml
+# CPU_ALL_VARIANTS: ggml's
 # backend registry dlopens the best libggml-cpu-*.so for this host, so no shell-side
-# probing. ROCm ships only turboquant-fallback, so fall back to it when cpu-all is absent.
+# probing. GPU arm64 images still ship turboquant-fallback until their builder toolchains
+# support ggml's complete arm variant matrix.
 if [ -e "$CURDIR"/turboquant-cpu-all ]; then
 	BINARY=turboquant-cpu-all
 fi
@@ -40,6 +42,27 @@ else
 	if [ -d "$CURDIR/lib/hipblaslt/library" ]; then
 		export HIPBLASLT_TENSILE_LIBPATH="$CURDIR"/lib/hipblaslt/library
 	fi
+	# Backends built for Intel GPUs carry a copy of the Intel graphics driver,
+	# and libze_loader is only there in those builds. Level Zero looks for a
+	# driver on its own, so point it at the copy that came with this backend: it
+	# was built against the same C library, while the machine's own driver may
+	# not have been, and loading that one can crash on start.
+	#
+	# Anything the user set is left alone, so a machine with a graphics card
+	# newer than the driver carried here can still be told to use its own.
+	# Nothing is said about OpenCL: no OpenCL driver is carried, so anything we
+	# set there would leave OpenCL worse off than the machine's own setup.
+	if [ -e "$CURDIR/lib/libze_loader.so.1" ]; then
+		if [ -e "$CURDIR/lib/libze_intel_gpu.so.1" ] && [ -z "${ZE_ENABLE_ALT_DRIVERS:-}" ]; then
+			export ZE_ENABLE_ALT_DRIVERS="$CURDIR"/lib/libze_intel_gpu.so.1
+		fi
+		# Ask the driver how much graphics memory is free. Without this, the
+		# backend reads zero on an integrated graphics chip, because such a chip
+		# shares the system memory instead of having its own.
+		if [ -z "${ZES_ENABLE_SYSMAN:-}" ]; then
+			export ZES_ENABLE_SYSMAN=1
+		fi
+	fi
 fi
 
 # If there is a lib/ld.so, use it

backend/go/crispasr/Makefile

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
 
 # CrispASR version (release tag)
 CRISPASR_REPO?=https://github.com/CrispStrobe/CrispASR
-CRISPASR_VERSION?=4e863bae52aa76a875e4aca57db54ae6d4145c5c
+CRISPASR_VERSION?=b5211ac635489049ee8ce86a82d69faa18e8d8da
 SO_TARGET?=libgocrispasr.so
 
 CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

backend/go/qwen3-tts-cpp/Makefile

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
 
 # qwentts.cpp version
 QWEN3TTS_REPO?=https://github.com/ServeurpersoCom/qwentts.cpp
-QWEN3TTS_CPP_VERSION?=35ebe5376b82a0a59d008586d55bbe623d449011
+QWEN3TTS_CPP_VERSION?=abab6b3bf317cfa1b788efce1d25f4f9239395ad
 SO_TARGET?=libgoqwen3ttscpp.so
 
 CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

backend/go/rfdetr-cpp/Makefile

@@ -11,7 +11,7 @@ JOBS?=$(shell nproc --ignore=1)
 # build; leaving this on `master` always picks up the latest C-API surface
 # (incl. the per-detection accessor functions used by gorfdetrcpp.go).
 RFDETR_REPO?=https://github.com/localai-org/rf-detr.cpp.git
-RFDETR_VERSION?=65c0ffcc9a9bc9dae38252f63d0417c9845a6cf7
+RFDETR_VERSION?=98d0f381b832ef08a608b65c7dd78db066ed8b9a
 
 ifeq ($(NATIVE),false)
 	CMAKE_ARGS+=-DGGML_NATIVE=OFF

backend/go/stablediffusion-ggml/Makefile

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
 
 # stablediffusion.cpp (ggml)
 STABLEDIFFUSION_GGML_REPO?=https://github.com/leejet/stable-diffusion.cpp
-STABLEDIFFUSION_GGML_VERSION?=e92e86fb11b3028ac9edaf63d93709801d106b12
+STABLEDIFFUSION_GGML_VERSION?=e31a86ce9110b11a98bd5990c329093244c2d1e3
 
 CMAKE_ARGS+=-DGGML_MAX_NAME=128
 

backend/go/whisper/Makefile

@@ -8,7 +8,7 @@ JOBS?=$(shell nproc --ignore=1)
 
 # whisper.cpp version
 WHISPER_REPO?=https://github.com/ggml-org/whisper.cpp
-WHISPER_CPP_VERSION?=a630b35c6fc02c8879f751ec3f39a61327f01dc7
+WHISPER_CPP_VERSION?=2ca53bb45e38748d07b310eeb36245a7157ac882
 SO_TARGET?=libgowhisper.so
 
 CMAKE_ARGS+=-DBUILD_SHARED_LIBS=OFF

core/gallery/importers/mlx.go

@@ -3,6 +3,7 @@ package importers
 import (
 	"encoding/json"
 	"path/filepath"
+	"slices"
 	"strings"
 
 	"github.com/mudler/LocalAI/core/config"
@@ -31,7 +32,7 @@ func (i *MLXImporter) Match(details Details) bool {
 	}
 
 	b, ok := preferencesMap["backend"].(string)
-	if ok && b == "mlx" || b == "mlx-vlm" {
+	if ok && slices.Contains([]string{"mlx", "mlx-vlm", "mlx-audio"}, b) {
 		return true
 	}
 
@@ -71,19 +72,32 @@ func (i *MLXImporter) Import(details Details) (gallery.ModelConfig, error) {
 	// (issue #10269). Send them to the mlx-vlm backend, which applies the
 	// processor-aware chat template.
 	backend := "mlx"
-	if details.HuggingFace != nil && details.HuggingFace.PipelineTag == "image-text-to-text" {
-		backend = "mlx-vlm"
+	usecases := []string{config.UsecaseChat}
+	useTokenizerTemplate := true
+	if details.HuggingFace != nil {
+		switch details.HuggingFace.PipelineTag {
+		case "image-text-to-text":
+			backend = "mlx-vlm"
+		case "text-to-speech":
+			backend = "mlx-audio"
+			usecases = []string{config.UsecaseTTS}
+			useTokenizerTemplate = false
+		}
 	}
 	// An explicit backend preference always wins.
 	b, ok := preferencesMap["backend"].(string)
 	if ok {
 		backend = b
+		if backend == "mlx-audio" {
+			usecases = []string{config.UsecaseTTS}
+			useTokenizerTemplate = false
+		}
 	}
 
 	modelConfig := config.ModelConfig{
 		Name:                name,
 		Description:         description,
-		KnownUsecaseStrings: []string{config.UsecaseChat},
+		KnownUsecaseStrings: usecases,
 		Backend:             backend,
 		PredictionOptions: schema.PredictionOptions{
 			BasicModelRequest: schema.BasicModelRequest{
@@ -91,7 +105,7 @@ func (i *MLXImporter) Import(details Details) (gallery.ModelConfig, error) {
 			},
 		},
 		TemplateConfig: config.TemplateConfig{
-			UseTokenizerTemplate: true,
+			UseTokenizerTemplate: useTokenizerTemplate,
 		},
 	}
 

core/gallery/importers/mlx_test.go

@@ -48,6 +48,16 @@ var _ = Describe("MLXImporter", func() {
 			Expect(result).To(BeTrue())
 		})
 
+		It("should match when backend preference is mlx-audio", func() {
+			preferences := json.RawMessage(`{"backend": "mlx-audio"}`)
+			details := importers.Details{
+				URI:         "https://example.com/model",
+				Preferences: preferences,
+			}
+
+			Expect(importer.Match(details)).To(BeTrue())
+		})
+
 		It("should not match when URI does not contain mlx-community/ and no backend preference", func() {
 			details := importers.Details{
 				URI: "https://huggingface.co/other-org/test-model",
@@ -123,6 +133,21 @@ var _ = Describe("MLXImporter", func() {
 			Expect(modelConfig.ConfigFile).To(ContainSubstring("backend: mlx-vlm"))
 		})
 
+		It("should configure explicit mlx-audio imports for text-to-speech", func() {
+			preferences := json.RawMessage(`{"backend": "mlx-audio"}`)
+			details := importers.Details{
+				URI:         "https://huggingface.co/mlx-community/Kokoro-82M-4bit",
+				Preferences: preferences,
+			}
+
+			modelConfig, err := importer.Import(details)
+
+			Expect(err).ToNot(HaveOccurred())
+			Expect(modelConfig.ConfigFile).To(ContainSubstring("backend: mlx-audio"))
+			Expect(modelConfig.ConfigFile).To(ContainSubstring("- tts"))
+			Expect(modelConfig.ConfigFile).ToNot(ContainSubstring("use_tokenizer_template: true"))
+		})
+
 		It("should auto-route vision-language models to the mlx-vlm backend", func() {
 			// gemma-4 E4B and similar VLMs declare pipeline_tag
 			// "image-text-to-text" on HuggingFace. The text-only mlx-lm
@@ -143,6 +168,23 @@ var _ = Describe("MLXImporter", func() {
 			Expect(modelConfig.ConfigFile).To(ContainSubstring("backend: mlx-vlm"))
 		})
 
+		It("should auto-route text-to-speech models to the mlx-audio backend", func() {
+			details := importers.Details{
+				URI: "https://huggingface.co/mlx-community/Kokoro-82M-4bit",
+				HuggingFace: &hfapi.ModelDetails{
+					ModelID:     "mlx-community/Kokoro-82M-4bit",
+					PipelineTag: "text-to-speech",
+				},
+			}
+
+			modelConfig, err := importer.Import(details)
+
+			Expect(err).ToNot(HaveOccurred())
+			Expect(modelConfig.ConfigFile).To(ContainSubstring("backend: mlx-audio"))
+			Expect(modelConfig.ConfigFile).To(ContainSubstring("- tts"))
+			Expect(modelConfig.ConfigFile).ToNot(ContainSubstring("use_tokenizer_template: true"))
+		})
+
 		It("should keep text-only models on the plain mlx backend", func() {
 			details := importers.Details{
 				URI: "https://huggingface.co/mlx-community/Llama-3.2-1B-Instruct-4bit",

core/http/endpoints/localai/backend.go

@@ -38,6 +38,7 @@ var knownPrefOnlyBackends = []schema.KnownBackend{
 	{Name: "whisperx", Modality: "asr", AutoDetect: false, Description: "WhisperX transcription (preference-only)"},
 	{Name: "crispasr", Modality: "asr", AutoDetect: false, Description: "CrispASR multi-architecture transcription (preference-only)"},
 	// TTS
+	{Name: "mlx-audio", Modality: "tts", AutoDetect: false, Description: "MLX-Audio text-to-speech models (auto-detected; pref-only fallback)"},
 	{Name: "kokoros", Modality: "tts", AutoDetect: false, Description: "Kokoros TTS (preference-only)"},
 	{Name: "qwen-tts", Modality: "tts", AutoDetect: false, Description: "Qwen TTS (preference-only)"},
 	{Name: "qwen3-tts-cpp", Modality: "tts", AutoDetect: false, Description: "Qwen3 TTS C++ (preference-only)"},

core/http/endpoints/localai/backend_test.go

@@ -152,6 +152,7 @@ var _ = Describe("Backend Endpoints", func() {
 			expectPrefOnly("tinygrad", "text")
 			expectPrefOnly("trl", "text")
 			expectPrefOnly("mlx-vlm", "text")
+			expectPrefOnly("mlx-audio", "tts")
 			expectPrefOnly("whisperx", "asr")
 			expectPrefOnly("crispasr", "asr")
 			expectPrefOnly("kokoros", "tts")

core/http/react-ui/e2e/forking-chat.spec.js

@@ -3,6 +3,7 @@ import { test, expect } from './coverage-fixtures.js'
 // Seeds two-message chat into localStorage so we don't need a live model.
 async function seedChat(page, history) {
   await page.addInitScript((h) => {
+    if (localStorage.getItem('localai_chats_data')) return
     const chat = {
       id: 'seed1', name: 'Seeded Chat', model: 'test-model',
       history: h, systemPrompt: '', mcpMode: false, mcpServers: [],
@@ -33,6 +34,56 @@ const TWO_TURNS = [
   { role: 'assistant', content: 'second answer' },
 ]
 
+test('saved message edits persist without sending a completion request', async ({ page }) => {
+  await mockModels(page)
+  let completionRequests = 0
+  await page.route('**/v1/chat/completions', (route) => {
+    completionRequests++
+    route.abort()
+  })
+  await seedChat(page, TWO_TURNS)
+  await page.goto('/app/chat')
+
+  const firstUser = page.locator('.chat-message-user').first()
+  await firstUser.hover()
+  await firstUser.getByTitle('Edit').click()
+  await firstUser.getByRole('textbox').fill('edited first question')
+  await firstUser.getByRole('button', { name: 'Save' }).click()
+
+  const firstAssistant = page.locator('.chat-message-assistant').first()
+  await firstAssistant.hover()
+  await firstAssistant.getByTitle('Edit').click()
+  await firstAssistant.getByRole('textbox').fill('edited first answer')
+  await firstAssistant.getByRole('button', { name: 'Save' }).click()
+
+  await expect(firstUser).toContainText('edited first question')
+  await expect(firstAssistant).toContainText('edited first answer')
+  await expect.poll(() => page.evaluate(() => {
+    const data = JSON.parse(localStorage.getItem('localai_chats_data'))
+    return data.chats[0].history.slice(0, 2).map(message => message.content)
+  })).toEqual(['edited first question', 'edited first answer'])
+
+  await page.reload()
+  await expect(page.locator('.chat-message-user').first()).toContainText('edited first question')
+  await expect(page.locator('.chat-message-assistant').first()).toContainText('edited first answer')
+  expect(completionRequests).toBe(0)
+})
+
+test('cancelling a message edit leaves the original content unchanged', async ({ page }) => {
+  await mockModels(page)
+  await seedChat(page, TWO_TURNS)
+  await page.goto('/app/chat')
+
+  const firstUser = page.locator('.chat-message-user').first()
+  await firstUser.hover()
+  await firstUser.getByTitle('Edit').click()
+  await firstUser.getByRole('textbox').fill('discard this draft')
+  await firstUser.getByRole('button', { name: 'Cancel' }).click()
+
+  await expect(firstUser).toContainText('first question')
+  await expect(firstUser).not.toContainText('discard this draft')
+})
+
 test('duplicate creates an independent copy and switches to it', async ({ page }) => {
   await mockModels(page)
   await seedChat(page, TWO_TURNS)
@@ -112,6 +163,29 @@ const FILE_TURNS = [
   { role: 'assistant', content: 'nope, that is it' },
 ]
 
+test('editing a file prompt preserves its content blocks and attachment metadata', async ({ page }) => {
+  await mockModels(page)
+  await seedChat(page, FILE_TURNS)
+  await page.goto('/app/chat')
+
+  const firstUser = page.locator('.chat-message-user').first()
+  await firstUser.hover()
+  await firstUser.getByTitle('Edit').click()
+  await firstUser.getByRole('textbox').fill('edited file question')
+  await firstUser.getByRole('button', { name: 'Save' }).click()
+
+  await expect.poll(() => page.evaluate(() => {
+    const data = JSON.parse(localStorage.getItem('localai_chats_data'))
+    return data.chats[0].history[0]
+  })).toEqual({
+    ...FILE_TURNS[0],
+    content: [
+      { type: 'text', text: 'edited file question' },
+      FILE_TURNS[0].content[1],
+    ],
+  })
+})
+
 test('regenerating a non-last answer in a fork still sends the uploaded file content', async ({ page }) => {
   await mockModels(page)
   let sentMessages = null

core/http/react-ui/e2e/traces-pagination.spec.js

@@ -79,4 +79,30 @@ test.describe('Traces - bounded list and on-demand detail', () => {
     await expect(page.locator('text=hello from the response body')).toBeVisible()
     await expect(page.locator('text=203.0.113.9').first()).toBeVisible()
   })
+
+  test('keeps the expanded trace open when a refresh prepends a new row', async ({ page }) => {
+    await page.locator('tr', { hasText: '/v1/chat/completions' }).first().click()
+    await expect(page.locator('text=hello from the request body')).toBeVisible()
+
+    await page.route('**/api/traces?*', (route) => {
+      route.fulfill({
+        contentType: 'application/json',
+        headers: { 'X-Total-Count': '843' },
+        body: JSON.stringify([
+          {
+            id: '8',
+            request: { method: 'GET', path: '/v1/models', body: null },
+            response: { status: 200, body: null },
+          },
+          ...LIST_BODY,
+        ]),
+      })
+    })
+
+    await page.getByRole('button', { name: 'Refresh' }).click()
+
+    await expect(page.locator('text=hello from the request body')).toBeVisible()
+    const originalRow = page.locator('tr', { hasText: '/v1/chat/completions' }).first()
+    await expect(originalRow.locator('i.fa-chevron-down')).toBeVisible()
+  })
 })

core/http/react-ui/public/locales/de/chat.json

@@ -71,6 +71,10 @@
   },
   "actions": {
     "copy": "Kopieren",
+    "edit": "Bearbeiten",
+    "editMessage": "Nachricht bearbeiten",
+    "save": "Speichern",
+    "cancel": "Abbrechen",
     "regenerate": "Neu generieren",
     "jumpToLatest": "Jump to latest"
   },

core/http/react-ui/public/locales/en/chat.json

@@ -71,6 +71,10 @@
   },
   "actions": {
     "copy": "Copy",
+    "edit": "Edit",
+    "editMessage": "Edit message",
+    "save": "Save",
+    "cancel": "Cancel",
     "regenerate": "Regenerate",
     "branch": "Branch from here",
     "jumpToLatest": "Jump to latest"

core/http/react-ui/public/locales/es/chat.json

@@ -71,6 +71,10 @@
   },
   "actions": {
     "copy": "Copiar",
+    "edit": "Editar",
+    "editMessage": "Editar mensaje",
+    "save": "Guardar",
+    "cancel": "Cancelar",
     "regenerate": "Regenerar",
     "jumpToLatest": "Jump to latest"
   },

core/http/react-ui/public/locales/id/chat.json

@@ -71,6 +71,10 @@
   },
   "actions": {
     "copy": "Salin",
+    "edit": "Edit",
+    "editMessage": "Edit pesan",
+    "save": "Simpan",
+    "cancel": "Batal",
     "regenerate": "Hasilkan ulang",
     "jumpToLatest": "Lompat ke terbaru"
   },

core/http/react-ui/public/locales/it/chat.json

@@ -71,6 +71,10 @@
   },
   "actions": {
     "copy": "Copia",
+    "edit": "Modifica",
+    "editMessage": "Modifica messaggio",
+    "save": "Salva",
+    "cancel": "Annulla",
     "regenerate": "Rigenera",
     "jumpToLatest": "Torna in fondo"
   },

core/http/react-ui/public/locales/ko/chat.json

@@ -71,6 +71,10 @@
   },
   "actions": {
     "copy": "복사",
+    "edit": "편집",
+    "editMessage": "메시지 편집",
+    "save": "저장",
+    "cancel": "취소",
     "regenerate": "다시 생성",
     "jumpToLatest": "Jump to latest"
   },

core/http/react-ui/public/locales/zh-CN/chat.json

@@ -71,6 +71,10 @@
   },
   "actions": {
     "copy": "复制",
+    "edit": "编辑",
+    "editMessage": "编辑消息",
+    "save": "保存",
+    "cancel": "取消",
     "regenerate": "重新生成",
     "jumpToLatest": "Jump to latest"
   },

core/http/react-ui/src/App.css

@@ -3540,6 +3540,37 @@ button.collapsible-header:focus-visible {
   background: var(--color-primary-light);
 }
 
+.chat-message-edit {
+  display: flex;
+  flex-direction: column;
+  gap: var(--spacing-sm);
+  min-width: min(32rem, 60vw);
+}
+
+.chat-message-edit-input {
+  width: 100%;
+  min-height: 6rem;
+  resize: vertical;
+  border: 1px solid var(--color-primary-border);
+  border-radius: var(--radius-md);
+  background: var(--color-bg-primary);
+  color: var(--color-text-primary);
+  font: inherit;
+  line-height: 1.5;
+  padding: var(--spacing-sm);
+}
+
+.chat-message-edit-input:focus {
+  outline: 2px solid var(--color-primary-light);
+  outline-offset: 1px;
+}
+
+.chat-message-edit-actions {
+  display: flex;
+  justify-content: flex-end;
+  gap: var(--spacing-xs);
+}
+
 .chat-message-system {
   align-self: center;
   max-width: 90%;

core/http/react-ui/src/pages/Chat.jsx

@@ -287,6 +287,24 @@ function UserMessageContent({ content, files }) {
   )
 }
 
+function editableMessageText(message) {
+  if (typeof message.content === 'string') return message.content
+  if (!Array.isArray(message.content)) return null
+  const textBlock = message.content.find(block => block?.type === 'text')
+  return typeof textBlock?.text === 'string' ? textBlock.text : null
+}
+
+function withEditedMessageText(message, text) {
+  if (typeof message.content === 'string') return { ...message, content: text }
+  const textIndex = message.content.findIndex(block => block?.type === 'text')
+  return {
+    ...message,
+    content: message.content.map((block, index) =>
+      index === textIndex ? { ...block, text } : block
+    ),
+  }
+}
+
 export default function Chat() {
   const { model: urlModel } = useParams()
   const { addToast } = useOutletContext()
@@ -329,6 +347,8 @@ export default function Chat() {
   const [clientMCPServers, setClientMCPServers] = useState(() => loadClientMCPServers())
   const [confirmDialog, setConfirmDialog] = useState(null)
   const [completionGlowIdx, setCompletionGlowIdx] = useState(-1)
+  const [editingMessageIndex, setEditingMessageIndex] = useState(null)
+  const [messageEditDraft, setMessageEditDraft] = useState('')
   const prevStreamingRef = useRef(false)
   const {
     connect: mcpConnect, disconnect: mcpDisconnect, disconnectAll: mcpDisconnectAll,
@@ -545,6 +565,33 @@ export default function Chat() {
     updateChatSettings(activeChat.id, { clientMCPServers: next })
   }, [activeChat, updateChatSettings])
 
+  const startMessageEdit = useCallback((index, message) => {
+    const text = editableMessageText(message)
+    if (text === null) return
+    setEditingMessageIndex(index)
+    setMessageEditDraft(text)
+  }, [])
+
+  const cancelMessageEdit = useCallback(() => {
+    setEditingMessageIndex(null)
+    setMessageEditDraft('')
+  }, [])
+
+  const saveMessageEdit = useCallback(() => {
+    if (!activeChat || isStreaming || editingMessageIndex === null || !messageEditDraft.trim()) return
+    const message = activeChat.history[editingMessageIndex]
+    if (!message || editableMessageText(message) === null) return
+    const history = activeChat.history.map((item, index) =>
+      index === editingMessageIndex ? withEditedMessageText(item, messageEditDraft) : item
+    )
+    updateChatSettings(activeChat.id, { history })
+    cancelMessageEdit()
+  }, [activeChat, isStreaming, editingMessageIndex, messageEditDraft, updateChatSettings, cancelMessageEdit])
+
+  useEffect(() => {
+    cancelMessageEdit()
+  }, [activeChat?.id, isStreaming, cancelMessageEdit])
+
   // Load initial message from home page
   const homeDataProcessed = useRef(false)
   useEffect(() => {
@@ -1170,40 +1217,80 @@ export default function Chat() {
                     {msg.role === 'assistant' && activeChat.model && (
                       <span className="chat-message-model">{activeChat.model}</span>
                     )}
-                    <div className="chat-message-content">
-                      {msg.role === 'user' ? (
-                        <UserMessageContent content={msg.content} files={msg.files} />
-                      ) : (
-                        <div dangerouslySetInnerHTML={{
-                          __html: canvasMode
-                            ? renderMarkdownWithArtifacts(typeof msg.content === 'string' ? msg.content : '', i)
-                            : renderMarkdown(typeof msg.content === 'string' ? msg.content : '')
-                        }} />
-                      )}
-                    </div>
+                    {editingMessageIndex === i ? (
+                      <div className="chat-message-edit">
+                        <textarea
+                          autoFocus
+                          className="chat-message-edit-input"
+                          value={messageEditDraft}
+                          onChange={(event) => setMessageEditDraft(event.target.value)}
+                          onKeyDown={(event) => {
+                            if (event.key === 'Escape') cancelMessageEdit()
+                          }}
+                          aria-label={t('actions.editMessage')}
+                        />
+                        <div className="chat-message-edit-actions">
+                          <button
+                            type="button"
+                            className="btn btn-primary btn-sm"
+                            onClick={saveMessageEdit}
+                            disabled={!messageEditDraft.trim()}
+                          >
+                            {t('actions.save')}
+                          </button>
+                          <button
+                            type="button"
+                            className="btn btn-secondary btn-sm"
+                            onClick={cancelMessageEdit}
+                          >
+                            {t('actions.cancel')}
+                          </button>
+                        </div>
+                      </div>
+                    ) : (
+                      <div className="chat-message-content">
+                        {msg.role === 'user' ? (
+                          <UserMessageContent content={msg.content} files={msg.files} />
+                        ) : (
+                          <div dangerouslySetInnerHTML={{
+                            __html: canvasMode
+                              ? renderMarkdownWithArtifacts(typeof msg.content === 'string' ? msg.content : '', i)
+                              : renderMarkdown(typeof msg.content === 'string' ? msg.content : '')
+                          }} />
+                        )}
+                      </div>
+                    )}
                     {msg.role === 'assistant' && typeof msg.content === 'string' && msg.content.includes('Error:') && (
                       <a href="/app/traces?tab=backend" className="chat-error-trace-link">
                         <i className="fas fa-wave-square" /> {t('errors.viewTraces')}
                       </a>
                     )}
-                    <div className="chat-message-actions">
-                      <button onClick={() => copyMessage(msg.content)} title={t('actions.copy')}>
-                        <i className="fas fa-copy" />
-                      </button>
-                      {msg.role === 'assistant' && !isStreaming && (
-                        <button onClick={() => handleRegenerate(i)} title={t('actions.regenerate')}>
-                          <i className="fas fa-rotate" />
+                    {editingMessageIndex !== i && (
+                      <div className="chat-message-actions">
+                        <button onClick={() => copyMessage(msg.content)} title={t('actions.copy')}>
+                          <i className="fas fa-copy" />
                         </button>
-                      )}
-                      {msg.role === 'assistant' && !isStreaming && (
-                        <button
-                          onClick={() => { forkChat(activeChat.id, i + 1); addToast(t('toasts.forked'), 'success', 2000) }}
-                          title={t('actions.branch')}
-                        >
-                          <i className="fas fa-code-branch" />
-                        </button>
-                      )}
-                    </div>
+                        {(msg.role === 'user' || msg.role === 'assistant') &&
+                          editableMessageText(msg) !== null && !isStreaming && (
+                            <button onClick={() => startMessageEdit(i, msg)} title={t('actions.edit')}>
+                              <i className="fas fa-pen" />
+                            </button>
+                          )}
+                        {msg.role === 'assistant' && !isStreaming && (
+                          <button onClick={() => handleRegenerate(i)} title={t('actions.regenerate')}>
+                            <i className="fas fa-rotate" />
+                          </button>
+                        )}
+                        {msg.role === 'assistant' && !isStreaming && (
+                          <button
+                            onClick={() => { forkChat(activeChat.id, i + 1); addToast(t('toasts.forked'), 'success', 2000) }}
+                            title={t('actions.branch')}
+                          >
+                            <i className="fas fa-code-branch" />
+                          </button>
+                        )}
+                      </div>
+                    )}
                   </div>
                 </div>
               )

core/http/react-ui/src/pages/Traces.jsx

@@ -342,7 +342,7 @@ export default function Traces() {
   const [apiCount, setApiCount] = useState(0)
   const [backendCount, setBackendCount] = useState(0)
   const [loading, setLoading] = useState(true)
-  const [expandedRow, setExpandedRow] = useState(null)
+  const [expandedTraceId, setExpandedTraceId] = useState(null)
   // detail holds the full record for the currently expanded row, fetched on
   // demand from /api/traces/:id (the list response omits the bodies).
   const [detail, setDetail] = useState(null)
@@ -360,7 +360,8 @@ export default function Traces() {
     duration: (a, b) => (a.duration || 0) - (b.duration || 0),
   }
   const toggleSort = (key) => {
-    setExpandedRow(null)
+    setExpandedTraceId(null)
+    setDetail(null)
     setSort(s => s.key === key ? { key, dir: s.dir === 'asc' ? 'desc' : 'asc' } : { key, dir: 'asc' })
   }
   const sortableTh = (key, label, props = {}) => (
@@ -433,20 +434,21 @@ export default function Traces() {
 
   useEffect(() => {
     setLoading(true)
-    setExpandedRow(null)
+    setExpandedTraceId(null)
     setDetail(null)
     fetchTraces()
   }, [fetchTraces])
 
   // Expanding a row pulls the full record (bodies, data fields, audio
   // snippets) that the list response deliberately omits.
-  const toggleRow = useCallback(async (index, row) => {
-    if (expandedRow === index) {
-      setExpandedRow(null)
+  const toggleRow = useCallback(async (row, index) => {
+    const traceKey = row?.id ?? index
+    if (expandedTraceId === traceKey) {
+      setExpandedTraceId(null)
       setDetail(null)
       return
     }
-    setExpandedRow(index)
+    setExpandedTraceId(traceKey)
     setDetail(null)
     if (!row?.id) return
     try {
@@ -457,7 +459,7 @@ export default function Traces() {
     } catch {
       // Fall back to the summary view; the row still renders what it has.
     }
-  }, [expandedRow, activeTab])
+  }, [expandedTraceId, activeTab])
 
   // Auto-refresh every 5 seconds
   useEffect(() => {
@@ -470,7 +472,7 @@ export default function Traces() {
       if (activeTab === 'api') await tracesApi.clear()
       else await tracesApi.clearBackend()
       setTraces([])
-      setExpandedRow(null)
+      setExpandedTraceId(null)
       setDetail(null)
       addToast('Traces cleared', 'success')
     } catch (err) {
@@ -500,7 +502,7 @@ export default function Traces() {
   }
 
   // Reset sort + expansion when switching trace tabs (columns differ).
-  useEffect(() => { setSort({ key: null, dir: 'asc' }); setExpandedRow(null); setDetail(null) }, [activeTab])
+  useEffect(() => { setSort({ key: null, dir: 'asc' }); setExpandedTraceId(null); setDetail(null) }, [activeTab])
 
   const sortedTraces = sort.key && TRACE_SORT[sort.key]
     ? [...traces].sort((a, b) => sort.dir === 'asc' ? TRACE_SORT[sort.key](a, b) : TRACE_SORT[sort.key](b, a))
@@ -635,9 +637,9 @@ export default function Traces() {
             </thead>
             <tbody>
               {sortedTraces.map((trace, i) => (
-                <React.Fragment key={i}>
-                  <tr onClick={() => toggleRow(i, trace)} className="clickable">
-                    <td><i className={`fas fa-chevron-${expandedRow === i ? 'down' : 'right'} text-xs`} /></td>
+                <React.Fragment key={trace.id ?? i}>
+                  <tr onClick={() => toggleRow(trace, i)} className="clickable">
+                    <td><i className={`fas fa-chevron-${expandedTraceId === (trace.id ?? i) ? 'down' : 'right'} text-xs`} /></td>
                     <td><span className="badge badge-info">{trace.request?.method || '-'}</span></td>
                     <td className="text-mono text-sm">{trace.request?.path || '-'}</td>
                     <td className="text-sub cell-clip" title={trace.user_name || trace.user_id || ''}>{trace.user_name || trace.user_id || '-'}</td>
@@ -648,7 +650,7 @@ export default function Traces() {
                         : <i className="fas fa-check-circle text-success" />}
                     </td>
                   </tr>
-                  {expandedRow === i && (
+                  {expandedTraceId === (trace.id ?? i) && (
                     <tr>
                       <td colSpan="6" className="p-0">
                         <ApiTraceDetail trace={detail && detail.id === trace.id ? detail : trace} />
@@ -674,9 +676,9 @@ export default function Traces() {
             </thead>
             <tbody>
               {sortedTraces.map((trace, i) => (
-                <React.Fragment key={i}>
-                  <tr onClick={() => toggleRow(i, trace)} className="clickable">
-                    <td><i className={`fas fa-chevron-${expandedRow === i ? 'down' : 'right'} text-xs`} /></td>
+                <React.Fragment key={trace.id ?? i}>
+                  <tr onClick={() => toggleRow(trace, i)} className="clickable">
+                    <td><i className={`fas fa-chevron-${expandedTraceId === (trace.id ?? i) ? 'down' : 'right'} text-xs`} /></td>
                     <td><span style={typeBadgeStyle(trace.type)}>{trace.type || '-'}</span></td>
                     <td className="text-sub nowrap">{formatDateTime(trace.timestamp)}</td>
                     <td className="text-mono text-sm">{trace.model_name || '-'}</td>
@@ -690,7 +692,7 @@ export default function Traces() {
                         : <i className="fas fa-check-circle text-success" />}
                     </td>
                   </tr>
-                  {expandedRow === i && (
+                  {expandedTraceId === (trace.id ?? i) && (
                     <tr>
                       <td colSpan="7" className="p-0">
                         <BackendTraceDetail trace={detail && detail.id === trace.id ? detail : trace} />

docs/README.md

@@ -27,9 +27,17 @@ Building and running the site locally requires a recent `extended` version of [H
 You can find out more about how to install Hugo for your environment in our
 [Getting started](https://www.docsy.dev/docs/getting-started/#prerequisites-and-installation) guide.
 
-Once you've made your working copy of the site repo, from the repo root folder, run:
+From the LocalAI repository root, run:
 
+```bash
+make docs
 ```
+
+The Hugo configuration lives in the `docs` directory. To invoke Hugo
+directly instead, run:
+
+```bash
+cd docs
 hugo server
 ```
 

docs/content/_index.md

@@ -1,86 +1,48 @@
 +++
-title = "LocalAI"
-description = "The open, modular AI runtime. Run text, vision, voice, image, video, agents, and more on hardware you control."
+disableToc = false
+title = "LocalAI documentation"
+description = "Install LocalAI, run models, and operate it in production."
 type = "home"
 +++
 
-<div class="lai-home">
-  <section class="lai-hero">
-    <div class="lai-hero__copy">
-      <p class="lai-signal"><span></span> Open source · MIT licensed</p>
-      <h1>One runtime.<br><strong>Every kind of AI.</strong><br>Your hardware.</h1>
-      <p class="lai-hero__lede">LocalAI runs text, vision, speech, sound, images, video, embeddings, reranking, and autonomous agents behind one modular stack-from a CPU laptop to a distributed GPU cluster.</p>
-      <div class="lai-actions">
-        <a class="lai-button" href="/installation/">Install LocalAI <b>→</b></a>
-        <a class="lai-link" href="https://github.com/mudler/LocalAI">View on GitHub ↗</a>
-      </div>
-      <div class="lai-proof"><span>60+ backends</span><span>CPU to cluster</span><span>OpenAI · Anthropic · Ollama · ElevenLabs APIs</span></div>
-    </div>
-  </section>
+LocalAI is the open source AI runtime: a small core that speaks the OpenAI and
+Anthropic APIs, with each inference backend added only when a model needs it.
+It runs text, vision, speech, sound, images, video, embeddings, reranking, and
+autonomous agents on hardware you control, from a CPU laptop to a distributed
+GPU cluster.
 
-  <section class="lai-breadth">
-    <header><p>The runtime, not just the endpoint.</p><h2>Bring the model. Choose the engine. Keep control.</h2></header>
-    <div class="lai-lanes">
-      <a href="/features/text-generation/"><span>Reason</span><b>Language models · tools · structured output</b><em>Text</em></a>
-      <a href="/features/openai-realtime/"><span>Listen & speak</span><b>Realtime WebRTC · transcription · TTS · diarization</b><em>Voice</em></a>
-      <a href="/features/image-generation/"><span>Create</span><b>Images · video · music · sound</b><em>Media</em></a>
-      <a href="/features/object-detection/"><span>See</span><b>Vision · detection · recognition · depth</b><em>Perception</em></a>
-      <a href="/features/agents/"><span>Act</span><b>Agents · MCP · skills · RAG · interactive tools</b><em>Agentic</em></a>
-    </div>
-  </section>
+New here? Read the [Overview]({{% relref "overview" %}}) for what LocalAI is
+and how the pieces fit together, then follow the
+[Quickstart]({{% relref "getting-started/quickstart" %}}).
 
-  <section class="lai-architecture">
-    <div class="lai-architecture__copy">
-      <p>A small core, not a giant bundle.</p>
-      <h2>Backends arrive when the model needs them.</h2>
-      <p>LocalAI keeps the core lean. Each backend wraps a best-in-class engine-llama.cpp, vLLM, SGLang, MLX, whisper.cpp, diffusion engines, and many more-as an isolated service pulled on demand.</p>
-      <ul><li>Install, update, or remove engines independently.</li><li>Mix CPU, NVIDIA, AMD, Intel, Apple Silicon, Vulkan, and Jetson.</li><li>Build your own backend in any language through an open gRPC contract.</li></ul>
-      <a href="/reference/architecture/">Explore the architecture →</a>
-    </div>
-    <figure><img src="/images/diagrams/composable-core.png" alt="LocalAI's small core connected to independent on-demand model backends" /></figure>
-  </section>
+```bash
+docker run -ti --name local-ai -p 8080:8080 localai/localai:latest
+```
 
-  <section class="lai-engines">
-    <div class="lai-engines__intro">
-      <p>We integrate the best engines. We build new ones, too.</p>
-      <h2>Inference work that moves the open ecosystem forward.</h2>
-      <p>The LocalAI team develops native C, C++, Rust, and GGML engines when the available stack is too heavy, too closed, or simply does not exist yet.</p>
-      <a href="https://github.com/mudler/LocalAI#backends-built-by-us">See the engines we maintain ↗</a>
-    </div>
-    <div class="lai-engine-reel">
-      <div><span>Speech</span><b>parakeet.cpp</b><small>Streaming multilingual ASR</small></div>
-      <div><span>Voice</span><b>vibevoice.cpp</b><small>Long-form TTS and ASR</small></div>
-      <div><span>Identity</span><b>voice-detect.cpp</b><small>Speaker recognition and analysis</small></div>
-      <div><span>Vision</span><b>face-detect.cpp</b><small>Recognition and anti-spoofing</small></div>
-      <div><span>Perception</span><b>locate-anything.cpp</b><small>Open-vocabulary detection</small></div>
-      <div><span>Privacy</span><b>privacy-filter.cpp</b><small>Native PII detection</small></div>
-      <div><span>3D</span><b>free-splatter.cpp</b><small>Pose-free reconstruction</small></div>
-      <div><span>Quantization</span><b>apex-quant</b><small>MoE-aware GGUF recipes</small></div>
-    </div>
-  </section>
+## Sections
 
-  <section class="lai-scale">
-    <header><p>Start on one machine. Keep going.</p><h2>The same runtime from workstation to private AI fabric.</h2></header>
-    <div class="lai-scale__path">
-      <div><span>01</span><b>Laptop</b><p>Run useful models locally, including CPU-only setups.</p></div>
-      <div><span>02</span><b>Team server</b><p>Add authentication, API keys, roles, quotas, and usage visibility.</p></div>
-      <div><span>03</span><b>Distributed cluster</b><p>Route across workers, fit models across devices, and scale with demand.</p></div>
-    </div>
-  </section>
+- **[Getting started]({{% relref "getting-started" %}})** - install LocalAI,
+  run your first model, call the API, and fix the common startup problems.
+- **[Features]({{% relref "features" %}})** - every capability, grouped by
+  modality: text, agents, audio, vision, image and video, retrieval,
+  distributed inference, and model management.
+- **[Advanced]({{% relref "advanced" %}})** - model configuration, VRAM
+  management, reverse proxies and TLS, and the rest of the fine-grained
+  control surface.
+- **[Operations]({{% relref "operations" %}})** - running and governing an
+  instance: middleware, cloud and MITM proxies, backend monitoring.
+- **[Reference]({{% relref "reference" %}})** - architecture, CLI flags, the
+  compatibility table, API and runtime errors, system info, and binaries.
+- **[FAQ]({{% relref "faq" %}})** - short answers to the questions that come up
+  most often.
 
-  <section class="lai-platform">
-    <div><p>More than inference</p><h2>A complete local AI control plane.</h2></div>
-    <div class="lai-platform__list">
-      <article><b>Agents built in</b><p>Create agents with MCP tools, skills, memory, RAG, citations, and streamed execution from the UI or API.</p></article>
-      <article><b>Realtime by design</b><p>Build interruptible voice experiences with WebRTC, streaming STT, LLM output, and TTS.</p></article>
-      <article><b>Privacy you can enforce</b><p>Keep data on your infrastructure and add PII analysis, redaction, policy middleware, and audit visibility.</p></article>
-      <article><b>Models under your control</b><p>Discover capabilities, import models, fine-tune, quantize, route, and monitor them in one place.</p></article>
-    </div>
-  </section>
+## Also useful
 
-  <section class="lai-start">
-    <div><p>One command to begin</p><h2>Run your first local AI stack.</h2></div>
-    <pre><code>docker run -ti --name local-ai -p 8080:8080 localai/localai:latest</code></pre>
-    <div class="lai-start__links"><a href="/installation/">Installation options</a><a href="https://models.localai.io">Browse models</a><a href="/model-compatibility/">Compare backends</a><a href="https://discord.gg/uJAeKSAGDy">Join Discord</a></div>
-  </section>
-</div>
+- **[Integrations]({{% relref "integrations" %}})** - projects and tools built
+  on top of LocalAI.
+- **[News]({{% relref "whats-new" %}})** - where release notes live.
+- **[Model gallery](https://models.localai.io)** - browse the models you can
+  install with one click.
+- **[GitHub](https://github.com/mudler/LocalAI)** and
+  **[Discord](https://discord.gg/uJAeKSAGDy)** - report an issue or ask a
+  question.

docs/content/features/GPU-acceleration.md

@@ -329,7 +329,23 @@ This configuration has been tested on a 'custom' cluster managed by SUSE Rancher
 
 ### Requirements
 
-If building from source, you need to install [Intel oneAPI Base Toolkit](https://software.intel.com/content/www/us/en/develop/tools/oneapi/base-toolkit/download.html) and have the Intel drivers available in the system.
+You need a machine with an Intel GPU and a kernel that drives it, which every current Linux kernel does. You do not need to install any Intel graphics packages: the backends carry their own copy of the Intel graphics driver, so they work on a machine that has none installed, and on a machine whose own driver was built against a newer C library than the backend.
+
+If you build from source instead of using the images, you need the [Intel oneAPI Base Toolkit](https://software.intel.com/content/www/us/en/develop/tools/oneapi/base-toolkit/download.html).
+
+#### Using your own Intel driver instead
+
+The carried driver comes from Intel's own package repository, so it knows the cards released up to the point the image was built. If your GPU is newer than that, or you would rather use the driver your distribution ships, point the backend at it:
+
+```bash
+docker run --rm -ti --device /dev/dri -p 8080:8080 \
+  -e ZE_ENABLE_ALT_DRIVERS=/usr/lib/x86_64-linux-gnu/libze_intel_gpu.so.1 \
+  -v $PWD/models:/models quay.io/go-skynet/local-ai:{{< version >}}-gpu-intel
+```
+
+Set the path to wherever your distribution keeps that file. Whatever you set is used as is, and the carried driver is left alone.
+
+The backends carry only the driver Level Zero uses, which is how llama.cpp reaches an Intel GPU. They do not carry an OpenCL driver, so OpenCL inside a container continues to use whatever the image itself provides.
 
 ### Container images
 
@@ -355,6 +371,8 @@ docker run --rm -ti --device /dev/dri -p 8080:8080 -e DEBUG=true -e MODELS_PATH=
 
 Note also that sycl does have a known issue to hang with `mmap: true`. You have to disable it in the model configuration if explicitly enabled.
 
+On an integrated Intel GPU, the amount of free graphics memory can only be read if the driver is asked to report it. The backends do that for you by setting `ZES_ENABLE_SYSMAN=1`. If you set that variable yourself, your value is kept, and setting it to `0` makes the backend read zero free memory, because an integrated GPU has no memory of its own and shares the system's.
+
 ## Vulkan acceleration
 
 ### Requirements
@@ -456,7 +474,7 @@ sycl-ls
 
 - **NVIDIA**: Ensure `nvidia-container-toolkit` is installed and the Docker runtime is configured. Test with `docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu24.04 nvidia-smi`.
 - **AMD**: Ensure `/dev/dri` and `/dev/kfd` are passed to the container and that `amdgpu-dkms` is installed on the host.
-- **Intel**: Ensure `/dev/dri` is passed to the container and Intel GPU drivers are installed on the host.
+- **Intel**: Ensure `/dev/dri` is passed to the container. No Intel graphics packages are needed on the host, since the backends bring their own driver. If the GPU is a recent model that the carried driver does not know, point the backend at the host's own driver as shown in [Intel acceleration](#intel-acceleration-sycl).
 
 ### Model loads on CPU instead of GPU
 

docs/content/getting-started/customize-model.md

@@ -2,7 +2,7 @@
 disableToc = false
 title = "Customizing the Model"
 weight = 5
-url = "/docs/getting-started/customize-model"
+url = "/getting-started/customize-model"
 icon = "rocket_launch"
 
 +++

docs/content/getting-started/models.md

@@ -80,6 +80,12 @@ The WebUI provides a powerful model import interface that supports both simple a
    - Custom preferences
 5. Click "Import Model" to start the import process
 
+Repositories under `mlx-community` are imported with the native MLX backend.
+LocalAI uses Hugging Face's pipeline metadata to select `mlx-vlm` for
+vision-language models and `mlx-audio` for text-to-speech models; other MLX
+repositories use `mlx`. An explicit backend selection in the import form always
+overrides this automatic routing.
+
 ### Advanced Import Mode
 
 For full control over model configuration:

docs/content/getting-started/quickstart.md

@@ -63,6 +63,10 @@ To get your first chat working:
 1. Open the **Models** page and search for `qwen3-4b`. Click **Install** on the `qwen3-4b` entry and wait for the download to finish. (`qwen3-4b` is a small, CPU-friendly Qwen3 model that also supports tool calling, so you can reuse it later in the [Build your first agent]({{% relref "getting-started/first-agent" %}}) walkthrough.)
 2. Open the **Chat** page, select `qwen3-4b` from the model dropdown, type a message, and send it. You should get a reply within a few seconds.
 
+To correct an earlier prompt or response without running the model again, hover
+over the saved message and select **Edit**. **Save** updates that conversation's
+local history; **Cancel** discards the draft.
+
 ### Downloading models from the CLI
 
 When starting LocalAI (either via Docker or via CLI) you can specify as argument a list of models to install automatically before starting the API, for example:

docs/content/overview.md

@@ -5,7 +5,7 @@ toc = true
 description = "What is LocalAI?"
 tags = ["Beginners"]
 categories = [""]
-url = "/docs/overview"
+url = "/overview"
 author = "Ettore Di Giacinto"
 icon = "info"
 +++

docs/hugo.toml

@@ -1,4 +1,7 @@
-baseURL = 'https://localai.io/'
+# The documentation is a second Hugo site, served under /docs/ of the marketing
+# site (see ../website). CI passes the same path via --baseURL, so keeping it
+# here means a local build produces the same links as production.
+baseURL = 'https://localai.io/docs/'
 languageCode = 'en-GB'
 defaultContentLanguage = 'en'
 

docs/static/css/custom.css

@@ -1,100 +0,0 @@
-@import url("./localai-home.css");
-
-/* The homepage deliberately fits Relearn's documentation content column. */
-#R-body-inner.home article.home > h1#localai {
-  display: none;
-}
-
-#R-body-inner.home > .flex-block-wrapper {
-  max-width: 960px;
-}
-
-/* Relearn scrolls the content pane; keep that behavior without a rail beside the hero. */
-#R-body-inner.home {
-  overflow-x: clip;
-  scrollbar-width: none;
-}
-
-#R-body-inner.home::-webkit-scrollbar {
-  display: none;
-}
-
-.lai-home,
-.lai-home > section,
-.lai-home header,
-.lai-home figure,
-.lai-home [class*="__copy"] {
-  box-sizing: border-box;
-  min-width: 0;
-  max-width: 100%;
-}
-
-.lai-home h1,
-.lai-home h2,
-.lai-home h3,
-.lai-home p {
-  overflow-wrap: anywhere;
-}
-
-.lai-home img {
-  max-width: 100%;
-}
-
-.lai-home * {
-  scrollbar-width: none;
-}
-
-.lai-home *::-webkit-scrollbar {
-  display: none;
-}
-
-.lai-start pre {
-  overflow: visible !important;
-  white-space: pre-wrap;
-  overflow-wrap: anywhere;
-  word-break: break-word;
-}
-
-.lai-hero {
-  grid-template-columns: minmax(0, 1fr) !important;
-  min-height: auto;
-  padding-block: clamp(3rem, 6vw, 5rem);
-}
-
-.lai-hero__copy {
-  max-width: 48rem;
-}
-
-.lai-hero h1 {
-  max-width: 12ch;
-  font-size: clamp(3rem, 5.5vw, 4.65rem);
-  text-align: left;
-}
-
-.lai-hero__lede {
-  max-width: 43rem;
-}
-
-.lai-product-shot {
-  width: 100%;
-  max-width: 54rem;
-  transform: none;
-}
-
-@media (max-width: 47.99rem) {
-  .lai-home,
-  .lai-home > section,
-  .lai-hero,
-  .lai-hero__copy,
-  .lai-product-shot {
-    box-sizing: border-box;
-    width: 100%;
-    min-width: 0;
-    max-width: 100%;
-  }
-
-  .lai-hero h1 {
-    max-width: 10ch;
-    font-size: clamp(2.4rem, 11vw, 3.5rem);
-  }
-}

docs/static/css/localai-home.css

@@ -1,4 +0,0 @@
-.lai-home{--lai-bg:#0d1117;--lai-surface:#131a23;--lai-surface-2:#192330;--lai-line:#29384a;--lai-ink:#edf4fc;--lai-muted:#9aabc0;--lai-blue:#4f8cff;--lai-green:#56d6a4;--lai-amber:#f1b95d;max-width:1180px;margin:0 auto;color:var(--lai-ink)}.lai-home *{box-sizing:border-box}.lai-home h1,.lai-home h2{color:var(--lai-ink);letter-spacing:-.04em;line-height:.96;text-wrap:balance}.lai-home h2{font-size:clamp(2.2rem,5vw,4.4rem)}.lai-home p{color:var(--lai-muted);text-wrap:pretty}.lai-home a{text-decoration:none}.lai-hero{display:grid;min-height:78vh;align-items:center;gap:clamp(2rem,6vw,5.5rem);padding:clamp(4rem,10vw,8rem) 0}.lai-signal{display:flex;align-items:center;gap:.5rem;margin:0 0 1.1rem!important;font-size:.73rem;font-weight:800;letter-spacing:.07em;text-transform:uppercase}.lai-signal span{width:.5rem;height:.5rem;border-radius:50%;background:var(--lai-green);box-shadow:0 0 0 .25rem color-mix(in srgb,var(--lai-green) 15%,transparent)}.lai-hero h1{margin:0 0 1.4rem;font-size:clamp(3.2rem,7vw,5.8rem)}.lai-hero h1 strong{color:var(--lai-blue);font-weight:760}.lai-hero__lede{max-width:42rem;margin:0!important;font-size:clamp(1.03rem,1.5vw,1.25rem);line-height:1.65}.lai-actions{display:flex;flex-wrap:wrap;align-items:center;gap:1rem;margin-top:1.8rem}.lai-button{display:inline-flex;min-height:3.1rem;align-items:center;gap:.75rem;padding:.7rem 1rem;border:1px solid var(--lai-blue);border-radius:7px;background:var(--lai-blue);color:white!important;font-weight:800;transition:transform .18s cubic-bezier(.16,1,.3,1),background .18s ease}.lai-button:hover{background:#70a2ff;transform:translateY(-2px)}.lai-link{color:var(--lai-ink)!important;font-weight:750;text-decoration:underline!important;text-underline-offset:.25rem}.lai-proof{display:flex;flex-wrap:wrap;gap:.55rem 1rem;margin-top:1.6rem}.lai-proof span{display:flex;align-items:center;gap:.35rem;color:var(--lai-muted);font-size:.68rem;font-weight:750}.lai-proof span::before{width:.32rem;height:.32rem;border-radius:50%;background:var(--lai-green);content:""}.lai-product-shot{overflow:hidden;margin:0;border:1px solid var(--lai-line);border-radius:12px;background:var(--lai-surface);transform:rotate(1deg)}.lai-product-shot img{display:block;width:100%;height:auto}.lai-product-shot figcaption{display:flex;align-items:center;justify-content:space-between;gap:.7rem;padding:.75rem .9rem;border-top:1px solid var(--lai-line);color:var(--lai-muted);font-size:.64rem}.lai-product-shot figcaption span{display:flex;align-items:center;gap:.35rem}.lai-product-shot figcaption i{width:.42rem;height:.42rem;border-radius:50%;background:var(--lai-green)}
-.lai-breadth,.lai-scale,.lai-platform,.lai-start{padding:clamp(5rem,10vw,8rem) 0;border-top:1px solid var(--lai-line)}.lai-breadth header,.lai-scale header{display:grid;gap:.7rem;margin-bottom:2.5rem}.lai-breadth header p,.lai-scale header p,.lai-architecture__copy>p:first-child,.lai-engines__intro>p:first-child,.lai-platform>div:first-child>p,.lai-start>div>p{margin:0!important;color:var(--lai-green);font-size:.72rem;font-weight:800;letter-spacing:.06em;text-transform:uppercase}.lai-breadth h2,.lai-scale h2,.lai-platform h2,.lai-start h2{max-width:16ch;margin:0}.lai-lanes{display:grid;border-top:1px solid var(--lai-line)}.lai-lanes a{display:grid;grid-template-columns:minmax(6rem,.45fr) minmax(0,1.55fr) auto;align-items:center;gap:1rem;padding:1.1rem .2rem;border-bottom:1px solid var(--lai-line);color:var(--lai-ink)!important;transition:padding .18s ease,background .18s ease}.lai-lanes a:hover{padding-inline:.8rem;background:var(--lai-surface)}.lai-lanes span{color:var(--lai-blue);font-size:.73rem;font-weight:800;text-transform:uppercase}.lai-lanes b{font-size:clamp(.9rem,1.6vw,1.15rem)}.lai-lanes em{color:var(--lai-muted);font-size:.66rem;font-style:normal}.lai-architecture{display:grid;gap:clamp(2.5rem,7vw,6rem);align-items:center;padding:clamp(5rem,11vw,9rem) 0;border-top:1px solid var(--lai-line)}.lai-architecture__copy h2,.lai-engines__intro h2{max-width:14ch;margin:.7rem 0 1.2rem;font-size:clamp(2.2rem,5vw,4.4rem)}.lai-architecture__copy>p:not(:first-child),.lai-engines__intro>p:not(:first-child){max-width:43rem;line-height:1.65}.lai-architecture ul{display:grid;gap:.55rem;margin:1.4rem 0;padding:0;list-style:none}.lai-architecture li{display:flex;gap:.5rem;color:var(--lai-muted)}.lai-architecture li::before{color:var(--lai-green);content:"✓"}.lai-architecture__copy>a,.lai-engines__intro>a{color:var(--lai-blue)!important;font-weight:800}.lai-architecture figure{margin:0;padding:1rem;border:1px solid var(--lai-line);border-radius:10px;background:var(--lai-surface)}.lai-architecture figure img{display:block;width:100%}
-.lai-engines{display:grid;gap:clamp(2.5rem,7vw,6rem);padding:clamp(5rem,11vw,9rem) 0;border-top:1px solid var(--lai-line);background:linear-gradient(90deg,transparent,var(--lai-surface),transparent)}.lai-engine-reel{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:1px;border:1px solid var(--lai-line);background:var(--lai-line)}.lai-engine-reel>div{display:grid;min-height:7.3rem;align-content:center;gap:.15rem;padding:1rem;background:var(--lai-bg)}.lai-engine-reel span{color:var(--lai-green);font-size:.62rem;font-weight:800;text-transform:uppercase}.lai-engine-reel b{font-family:monospace;font-size:.88rem}.lai-engine-reel small{color:var(--lai-muted);font-size:.65rem}.lai-scale__path{display:grid;gap:1px;border:1px solid var(--lai-line);background:var(--lai-line)}.lai-scale__path>div{min-height:12rem;padding:1.25rem;background:var(--lai-surface)}.lai-scale__path span{display:block;margin-bottom:2.4rem;color:var(--lai-blue);font-family:monospace;font-size:.7rem}.lai-scale__path b{font-size:1.2rem}.lai-scale__path p{margin:.45rem 0 0!important;font-size:.84rem}.lai-platform{display:grid;gap:3rem}.lai-platform__list{border-top:1px solid var(--lai-line)}.lai-platform article{display:grid;grid-template-columns:minmax(9rem,.6fr) minmax(0,1.4fr);gap:1rem;padding:1.15rem 0;border-bottom:1px solid var(--lai-line)}.lai-platform article b{color:var(--lai-ink);font-size:1rem}.lai-platform article p{margin:0!important}.lai-start{display:grid;gap:2rem}.lai-start pre{overflow:auto;margin:0!important;padding:1.2rem!important;border:1px solid var(--lai-line);border-radius:8px;background:#080b0f!important}.lai-start code{color:var(--lai-green)!important}.lai-start__links{display:flex;flex-wrap:wrap;gap:.6rem}.lai-start__links a{padding:.5rem .65rem;border:1px solid var(--lai-line);border-radius:5px;color:var(--lai-ink)!important;font-size:.72rem;font-weight:750}.lai-start__links a:hover{border-color:var(--lai-blue)}
-@media(min-width:48rem){.lai-hero{grid-template-columns:minmax(22rem,.9fr) minmax(24rem,1.1fr)}.lai-breadth header,.lai-scale header{grid-template-columns:1fr 1.3fr;align-items:end}.lai-architecture{grid-template-columns:minmax(22rem,.9fr) minmax(24rem,1.1fr)}.lai-engines{grid-template-columns:minmax(22rem,.9fr) minmax(25rem,1.1fr)}.lai-engine-reel{grid-template-columns:repeat(2,minmax(0,1fr))}.lai-scale__path{grid-template-columns:repeat(3,1fr)}.lai-platform{grid-template-columns:.75fr 1.25fr}.lai-start{grid-template-columns:.75fr 1.25fr}.lai-start__links{grid-column:2}}@media(max-width:47.99rem){.lai-home{padding-inline:.2rem}.lai-lanes a{grid-template-columns:1fr auto}.lai-lanes b{grid-column:1/3}.lai-product-shot{transform:none}.lai-platform article{grid-template-columns:1fr}.lai-start__links{gap:.4rem}}@media(prefers-reduced-motion:reduce){.lai-home *{scroll-behavior:auto!important;transition-duration:.01ms!important}.lai-product-shot{transform:none}}

gallery/index.yaml

@@ -1,7 +1,194 @@
 ---
+- name: "deepseek-v4-flash-0731"
+  url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
+  urls:
+    - https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF
+  description: |
+    # DeepSeek-V4-Flash-0731
+
+    Technical Report👁️
+
+    ## Introduction
+
+    **DeepSeek-V4-Flash-0731** is the official release of **DeepSeek-V4-Flash**, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.
+
+    DeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available.
+
+    Notes:
+
+    1. For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the `max` reasoning effort level with `temperature = 1.0, top_p = 0.95`.
+    2. † DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems.
+
+    ## Chat Template
+
+    ...
+  license: "mit"
+  tags:
+    - llm
+    - gguf
+    - deepseek
+  icon: https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/logo.svg
+  overrides:
+    backend: ds4
+    function:
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    parameters:
+      model: ds4flash.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: ds4flash.gguf
+      sha256: ""
+      uri: https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF
+- name: "parable-granite-4.1-3b-claude-fable-5"
+  url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
+  urls:
+    - https://huggingface.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF
+  description: |
+    # Parable-Granite-4.1-3B-Claude-Fable-5
+
+    Granite 4.1 3B fine-tuned on genuine Claude Fable 5 and GPT-5.5 agent
+    traces (planning, tool use, <think> reasoning from real agent sessions).
+    Agent-flavored small model: terminal workflows, idiomatic code fixes,
+    explanations. v2 recipe: completion-masked SFT, replay mix, seed-averaged
+    weights. Published corpus and eval harness.
+  license: "apache-2.0"
+  tags:
+    - llm
+    - gguf
+    - agent
+    - coding
+    - thinking
+    - tool-use
+  overrides:
+    backend: llama-cpp
+    function:
+      automatic_tool_parsing_fallback: true
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/Parable-Granite-4.1-3B-Claude-Fable-5-Q4_K_M/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Parable-Granite-4.1-3B-Claude-Fable-5-Q4_K_M/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+      uri: https://huggingface.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF/resolve/main/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+      sha256: 67dc7695d92939c713165761f115c9d892fdff74fcbd987c8bb453b9b8ab645d
+- name: "parable-qwen3-4b-claude-fable-5"
+  url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
+  urls:
+    - https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-GGUF
+  description: |
+    # Parable-Qwen3-4B-Claude-Fable-5
+
+    Qwen3 4B fine-tuned on genuine Claude Fable 5 agent traces. Thinking-mode
+    reasoning, agent/terminal task flavor, tool-call formatting.
+  license: "apache-2.0"
+  tags:
+    - llm
+    - gguf
+    - agent
+    - coding
+    - thinking
+  overrides:
+    backend: llama-cpp
+    function:
+      automatic_tool_parsing_fallback: true
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/Parable-Qwen3-4B-Claude-Fable-5-Q4_K_M/Parable-Qwen3-4B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Parable-Qwen3-4B-Claude-Fable-5-Q4_K_M/Parable-Qwen3-4B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+      uri: https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-GGUF/resolve/main/Parable-Qwen3-4B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+      sha256: c94b06a912aa901f3da5689754577ad534415efafc50dcee3f389594a153bf38
+- name: "parable-granite-4.1-8b-claude-fable-5"
+  url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
+  urls:
+    - https://huggingface.co/AnkitAI/Parable-Granite-4.1-8B-Claude-Fable-5-GGUF
+  description: |
+    # Parable-Granite-4.1-8B-Claude-Fable-5
+
+    Granite 4.1 8B fine-tuned on genuine Claude Fable 5 and GPT-5.5 agent
+    traces. Strongest Parable model: multi-step scripts, configs, terminal
+    workflows, <think> reasoning.
+  license: "apache-2.0"
+  tags:
+    - llm
+    - gguf
+    - agent
+    - coding
+    - thinking
+    - tool-use
+  overrides:
+    backend: llama-cpp
+    function:
+      automatic_tool_parsing_fallback: true
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/Parable-Granite-4.1-8B-Claude-Fable-5-Q4_K_M/Parable-Granite-4.1-8B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Parable-Granite-4.1-8B-Claude-Fable-5-Q4_K_M/Parable-Granite-4.1-8B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+      uri: https://huggingface.co/AnkitAI/Parable-Granite-4.1-8B-Claude-Fable-5-GGUF/resolve/main/Parable-Granite-4.1-8B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+      sha256: 61a8133c344a0d0a00188395afe33c803e3b973cb4bbfd5ef1fa7110e80bc1c3
+- name: "parable-qwen3-8b-claude-fable-5"
+  url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
+  urls:
+    - https://huggingface.co/AnkitAI/Parable-Qwen3-8B-Claude-Fable-5-GGUF
+  description: |
+    # Parable-Qwen3-8B-Claude-Fable-5
+
+    Qwen3 8B fine-tuned on genuine Claude Fable 5 agent traces. Thinking-mode
+    reasoning with agent/terminal flavor and tool-call formatting.
+  license: "apache-2.0"
+  tags:
+    - llm
+    - gguf
+    - agent
+    - coding
+    - thinking
+  overrides:
+    backend: llama-cpp
+    function:
+      automatic_tool_parsing_fallback: true
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/Parable-Qwen3-8B-Claude-Fable-5-Q4_K_M/Parable-Qwen3-8B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Parable-Qwen3-8B-Claude-Fable-5-Q4_K_M/Parable-Qwen3-8B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+      uri: https://huggingface.co/AnkitAI/Parable-Qwen3-8B-Claude-Fable-5-GGUF/resolve/main/Parable-Qwen3-8B-Claude-Fable-5-GGUF-Q4_K_M.gguf
+      sha256: 956070afc8023b8665fe450842f7be76b505b53d142460fd9b588222f4e16112
 - &pocket-35b
   name: "pocket-35b"
   variants:
+    - model: pocket-35b-q3
     - model: pocket-35b-q2
     - model: pocket-35b-iq1
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
@@ -38,6 +225,31 @@
     - filename: llama-cpp/models/FINAL-Bench/POCKET-35B-GGUF/POCKET-35B-Q4_K_M.gguf
       sha256: 6f479f637c8fb932df39b9cfabdc454568eee48c0e7c0584e1815a27558e8ffe
       uri: huggingface://FINAL-Bench/POCKET-35B-GGUF/POCKET-35B-Q4_K_M.gguf
+- !!merge <<: *pocket-35b
+  name: "pocket-35b-q3"
+  variants: []
+  description: |
+    POCKET-35B is an Apache-2.0 Qwen3.5-family mixture-of-experts model from
+    FINAL-Bench/VIDRAFT, derived from Darwin-36B-Opus and packaged for stock
+    llama.cpp. This entry uses the balanced Q3_K_M GGUF quantization.
+  overrides:
+    backend: llama-cpp
+    function:
+      automatic_tool_parsing_fallback: true
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/FINAL-Bench/POCKET-35B-GGUF/POCKET-35B-Q3_K_M.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/FINAL-Bench/POCKET-35B-GGUF/POCKET-35B-Q3_K_M.gguf
+      sha256: 9ab4184f0f5af0cc1cf8eee7e553664b692c3f837a8868c4a16137bb3a3c97a2
+      uri: huggingface://FINAL-Bench/POCKET-35B-GGUF/POCKET-35B-Q3_K_M.gguf
 - !!merge <<: *pocket-35b
   name: "pocket-35b-q2"
   variants: []
@@ -729,6 +941,90 @@
     - filename: llama-cpp/mmproj/inkling-UD-Q4_K_XL/mmproj-BF16.gguf
       sha256: 662c925e1df293cfba16ffd6bd53dac31d3c73160ba65dff7270d7a70f351e91
       uri: https://huggingface.co/unsloth/inkling-GGUF/resolve/main/mmproj-BF16.gguf
+- &inkling-small
+  name: "inkling-small"
+  url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
+  urls:
+    - https://huggingface.co/thinkingmachines/Inkling-Small
+    - https://huggingface.co/unsloth/Inkling-Small-GGUF
+  description: |
+    Inkling Small is a 276B-parameter mixture-of-experts multimodal model with 12B active parameters for text, image, and audio understanding, instruction following, coding, and tool use. This entry uses the Q4_K_M GGUF quantization, whose five language-model shards total approximately 162.5 GB.
+  license: "apache-2.0"
+  tags:
+    - llm
+    - gguf
+    - vision
+    - audio
+    - multimodal
+  overrides:
+    backend: llama-cpp
+    function:
+      automatic_tool_parsing_fallback: true
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    mmproj: llama-cpp/mmproj/Inkling-Small-UD-Q4_K_M/mmproj-BF16.gguf
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/Inkling-Small-UD-Q4_K_M/Inkling-Small-UD-Q4_K_M-00001-of-00005.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Inkling-Small-UD-Q4_K_M/Inkling-Small-UD-Q4_K_M-00001-of-00005.gguf
+      sha256: a51ac3f439198f2817219edd582be4b600c273be24e78cbd58ebff982d9f007e
+      uri: https://huggingface.co/unsloth/Inkling-Small-GGUF/resolve/main/UD-Q4_K_M/Inkling-Small-UD-Q4_K_M-00001-of-00005.gguf
+    - filename: llama-cpp/models/Inkling-Small-UD-Q4_K_M/Inkling-Small-UD-Q4_K_M-00002-of-00005.gguf
+      sha256: 3dccdd473cc3a191e6028f6105b01831ed3a8dc30ec4e02679f9e0c2ccb59671
+      uri: https://huggingface.co/unsloth/Inkling-Small-GGUF/resolve/main/UD-Q4_K_M/Inkling-Small-UD-Q4_K_M-00002-of-00005.gguf
+    - filename: llama-cpp/models/Inkling-Small-UD-Q4_K_M/Inkling-Small-UD-Q4_K_M-00003-of-00005.gguf
+      sha256: 1a7edf29bda1d278b4668e1a082d7db634b845a53ea58c918a8cea1f9006c21c
+      uri: https://huggingface.co/unsloth/Inkling-Small-GGUF/resolve/main/UD-Q4_K_M/Inkling-Small-UD-Q4_K_M-00003-of-00005.gguf
+    - filename: llama-cpp/models/Inkling-Small-UD-Q4_K_M/Inkling-Small-UD-Q4_K_M-00004-of-00005.gguf
+      sha256: 376f67568438da96b10566730e8a9e17e3f665ab1b9d82eba48ec33708b172f7
+      uri: https://huggingface.co/unsloth/Inkling-Small-GGUF/resolve/main/UD-Q4_K_M/Inkling-Small-UD-Q4_K_M-00004-of-00005.gguf
+    - filename: llama-cpp/models/Inkling-Small-UD-Q4_K_M/Inkling-Small-UD-Q4_K_M-00005-of-00005.gguf
+      sha256: e34364af0d04d2d295bc374f1a4fa80002e4277db1bcf4b56240dbd443ac21d3
+      uri: https://huggingface.co/unsloth/Inkling-Small-GGUF/resolve/main/UD-Q4_K_M/Inkling-Small-UD-Q4_K_M-00005-of-00005.gguf
+    - filename: llama-cpp/mmproj/Inkling-Small-UD-Q4_K_M/mmproj-BF16.gguf
+      sha256: 05d4475a956030be87b099865d6552a541a476db8cc3e266fcfa7c5a24846248
+      uri: https://huggingface.co/unsloth/Inkling-Small-GGUF/resolve/main/mmproj-BF16.gguf
+  variants:
+    - model: inkling-small-iq2-m
+- !!merge <<: *inkling-small
+  name: "inkling-small-iq2-m"
+  description: |
+    Inkling Small is a 276B-parameter mixture-of-experts multimodal model with 12B active parameters for text, image, and audio understanding, instruction following, coding, and tool use. This entry uses the IQ2_M GGUF quantization, whose three language-model shards total approximately 82.4 GB.
+  overrides:
+    backend: llama-cpp
+    function:
+      automatic_tool_parsing_fallback: true
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    mmproj: llama-cpp/mmproj/Inkling-Small-UD-IQ2_M/mmproj-BF16.gguf
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/Inkling-Small-UD-IQ2_M/Inkling-Small-UD-IQ2_M-00001-of-00003.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Inkling-Small-UD-IQ2_M/Inkling-Small-UD-IQ2_M-00001-of-00003.gguf
+      sha256: 3b6ace30e488ad26e816cdba4e42714f40110a3142a210bd5c2e48f69e27cb31
+      uri: https://huggingface.co/unsloth/Inkling-Small-GGUF/resolve/main/UD-IQ2_M/Inkling-Small-UD-IQ2_M-00001-of-00003.gguf
+    - filename: llama-cpp/models/Inkling-Small-UD-IQ2_M/Inkling-Small-UD-IQ2_M-00002-of-00003.gguf
+      sha256: 5ca94e858ae116eb513a2af1facd35844d42ef4a209e1d85cc4ecc73cd21b894
+      uri: https://huggingface.co/unsloth/Inkling-Small-GGUF/resolve/main/UD-IQ2_M/Inkling-Small-UD-IQ2_M-00002-of-00003.gguf
+    - filename: llama-cpp/models/Inkling-Small-UD-IQ2_M/Inkling-Small-UD-IQ2_M-00003-of-00003.gguf
+      sha256: 8a84e00d4625d52491969f88f24f0999bc77527b7fa1d15cbe44ef88235bb377
+      uri: https://huggingface.co/unsloth/Inkling-Small-GGUF/resolve/main/UD-IQ2_M/Inkling-Small-UD-IQ2_M-00003-of-00003.gguf
+    - filename: llama-cpp/mmproj/Inkling-Small-UD-IQ2_M/mmproj-BF16.gguf
+      sha256: 05d4475a956030be87b099865d6552a541a476db8cc3e266fcfa7c5a24846248
+      uri: https://huggingface.co/unsloth/Inkling-Small-GGUF/resolve/main/mmproj-BF16.gguf
+  variants: []
 - name: "qwythos-9b-v2"
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
@@ -828,6 +1124,74 @@
     - filename: llama-cpp/mmproj/Qwen3.6-27B-Fable-Fus-711-UnHeretic-NM-DAU-NEO-MAX-NEO-MTP-Q4_K_M/mmproj-F32.gguf
       sha256: fdc443e974cad1f61c45af1cfd5580855855ddce0d6c14cc500a5714c486ac1d
       uri: https://huggingface.co/DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF/resolve/main/mmproj-F32.gguf
+- &qwopus3-6-27b-fusion
+  name: "qwopus3.6-27b-fusion"
+  variants:
+    - model: qwopus3.6-27b-fusion-q8
+  url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
+  urls:
+    - https://huggingface.co/Qwen/Qwen3.6-27B
+    - https://huggingface.co/KyleHessling1/Qwopus3.6-27B-Fusion-GGUF
+  description: |
+    Qwopus3.6-27B Fusion is an experimental 27B Qwen3.6 merge that combines the
+    Qwopus reasoning and coding fine-tunes. It targets agentic coding and
+    general reasoning, supports the Qwen3.6 262K context window, and retains
+    the base model's MTP head. This default entry uses the 16.8 GB Q4_K_M GGUF.
+  license: qwen
+  tags:
+    - llm
+    - gguf
+    - reasoning
+    - thinking
+    - mtp
+  overrides:
+    backend: llama-cpp
+    function:
+      automatic_tool_parsing_fallback: true
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    options:
+      - use_jinja:true
+      - spec_type:draft-mtp
+      - spec_n_max:6
+      - spec_p_min:0.75
+    parameters:
+      model: llama-cpp/models/Qwopus3.6-27B-Fusion-Q4_K_M/Qwopus3.6-27B-Fusion-Q4_K_M.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Qwopus3.6-27B-Fusion-Q4_K_M/Qwopus3.6-27B-Fusion-Q4_K_M.gguf
+      sha256: 6d4e6e9ffba84ffe5166969303d4918f3a31682d75862416827412cee8f20d5d
+      uri: huggingface://KyleHessling1/Qwopus3.6-27B-Fusion-GGUF/Qwopus3.6-27B-Fusion-Q4_K_M.gguf
+- !!merge <<: *qwopus3-6-27b-fusion
+  name: "qwopus3.6-27b-fusion-q8"
+  variants: null
+  description: |
+    Qwopus3.6-27B Fusion in the higher-fidelity 29.0 GB Q8_0 GGUF format.
+    It retains the base model's MTP head for speculative decoding.
+  overrides:
+    backend: llama-cpp
+    function:
+      automatic_tool_parsing_fallback: true
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+    options:
+      - use_jinja:true
+      - spec_type:draft-mtp
+      - spec_n_max:6
+      - spec_p_min:0.75
+    parameters:
+      model: llama-cpp/models/Qwopus3.6-27B-Fusion-Q8_0/Qwopus3.6-27B-Fusion-Q8_0.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Qwopus3.6-27B-Fusion-Q8_0/Qwopus3.6-27B-Fusion-Q8_0.gguf
+      sha256: 5594e1776b75beedf4a54b933bba386dc83a0883417e3bcd9ef53fdfd120d5b6
+      uri: huggingface://KyleHessling1/Qwopus3.6-27B-Fusion-GGUF/Qwopus3.6-27B-Fusion-Q8_0.gguf
 - name: "minicpm5-1b-claude-opus-fable5-v2-thinking"
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
@@ -1241,7 +1605,7 @@
   files:
     - filename: ds4flash.gguf
       uri: https://huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF
-      sha256: ea2be54e4e989cb8cc2a88c0791b15eaa50eb97fb46a89843f6415d7e1a73e33
+      sha256: 7d6d1691bc2d02c5a8194afb8bd9b57519343afebf020231c61d9236504b4a5c
 - name: "qwopus3.6-35b-a3b-coder-mtp"
   url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
   urls:
@@ -4988,97 +5352,6 @@
     - filename: mmproj-F16.gguf
       sha256: 6edcca228213c28d3567a35d22f849eea52d8360875093851959adf5d2f270eb
       uri: https://huggingface.co/unsloth/gemma-4-31B-it-GGUF/resolve/main/mmproj-F16.gguf
-- &pocket-35b
-  name: pocket-35b-q4-k-m
-  url: github:mudler/LocalAI/gallery/virtual.yaml@master
-  urls:
-    - https://huggingface.co/FINAL-Bench/Darwin-36B-Opus
-    - https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF
-  description: |
-    POCKET-35B is an Apache-2.0 Qwen3.5-family sparse MoE model derived from
-    Darwin-36B-Opus. Its official GGUF ladder spans Q4_K_M, Q3_K_M, Q2_K, and
-    IQ1_M so LocalAI can select a build for the available memory. All builds run
-    with the stock llama.cpp backend.
-  license: apache-2.0
-  icon: https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF/resolve/main/pocket_hero.svg
-  tags:
-    - llm
-    - gguf
-    - gpu
-    - cpu
-    - qwen
-    - qwen3.5
-    - moe
-  last_checked: "2026-07-29"
-  overrides:
-    backend: llama-cpp
-    known_usecases:
-      - chat
-    options:
-      - use_jinja:true
-    parameters:
-      model: POCKET-35B-Q4_K_M.gguf
-    template:
-      use_tokenizer_template: true
-  files:
-    - filename: POCKET-35B-Q4_K_M.gguf
-      uri: huggingface://FINAL-Bench/POCKET-35B-GGUF/POCKET-35B-Q4_K_M.gguf
-      sha256: 6f479f637c8fb932df39b9cfabdc454568eee48c0e7c0584e1815a27558e8ffe
-  variants:
-    - model: pocket-35b-q3-k-m
-    - model: pocket-35b-q2-k
-    - model: pocket-35b-iq1-m
-- !!merge <<: *pocket-35b
-  name: pocket-35b-q3-k-m
-  overrides:
-    backend: llama-cpp
-    known_usecases:
-      - chat
-    options:
-      - use_jinja:true
-    parameters:
-      model: POCKET-35B-Q3_K_M.gguf
-    template:
-      use_tokenizer_template: true
-  files:
-    - filename: POCKET-35B-Q3_K_M.gguf
-      uri: huggingface://FINAL-Bench/POCKET-35B-GGUF/POCKET-35B-Q3_K_M.gguf
-      sha256: 9ab4184f0f5af0cc1cf8eee7e553664b692c3f837a8868c4a16137bb3a3c97a2
-  variants: []
-- !!merge <<: *pocket-35b
-  name: pocket-35b-q2-k
-  overrides:
-    backend: llama-cpp
-    known_usecases:
-      - chat
-    options:
-      - use_jinja:true
-    parameters:
-      model: POCKET-35B-Q2_K.gguf
-    template:
-      use_tokenizer_template: true
-  files:
-    - filename: POCKET-35B-Q2_K.gguf
-      uri: huggingface://FINAL-Bench/POCKET-35B-GGUF/POCKET-35B-Q2_K.gguf
-      sha256: 2567ed710fb5b5bdf0df99ecfa241ab1a2b35edc618d2830b91d468e3a3ce191
-  variants: []
-- !!merge <<: *pocket-35b
-  name: pocket-35b-iq1-m
-  overrides:
-    backend: llama-cpp
-    known_usecases:
-      - chat
-    options:
-      - use_jinja:true
-    parameters:
-      model: POCKET-35B-IQ1_M.gguf
-    template:
-      use_tokenizer_template: true
-  files:
-    - filename: POCKET-35B-IQ1_M.gguf
-      uri: huggingface://FINAL-Bench/POCKET-35B-GGUF/POCKET-35B-IQ1_M.gguf
-      sha256: c56c77d158786f6dfb084e8cf3467456668e400cc7632c366cc0444b9f8565a1
-  variants: []
 - name: qwen3.5-35b-a3b-apex
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
@@ -5117,6 +5390,82 @@
     - filename: llama-cpp/models/Qwen3.5-35B-A3B-APEX-GGUF/Qwen3.5-35B-A3B-APEX-Quality.gguf
       sha256: 50887b60c77ee5c95bc3657814ae993abcab7b2d71868b9af1e84d6badd09a57
       uri: https://huggingface.co/mudler/Qwen3.5-35B-A3B-APEX-GGUF/resolve/main/Qwen3.5-35B-A3B-APEX-Quality.gguf
+- &fara1-5-9b
+  name: fara1.5-9b
+  url: github:mudler/LocalAI/gallery/virtual.yaml@master
+  variants:
+    - model: fara1.5-9b-q8
+  urls:
+    - https://huggingface.co/microsoft/Fara1.5-9B
+    - https://huggingface.co/bartowski/Fara1.5-9B-GGUF
+  description: |
+    Fara1.5-9B is Microsoft's 9B-parameter multimodal computer-use agent for web browsers, fine-tuned from Qwen3.5-9B. It accepts screenshots and text, emits structured browser actions, supports a 262K-token context, and should be deployed with appropriate sandboxing and user-confirmation controls. This entry uses the recommended Q4_K_M GGUF quantization.
+  license: mit
+  tags:
+    - fara
+    - qwen
+    - qwen3.5
+    - 9b
+    - llm
+    - gguf
+    - quantized
+    - chat
+    - vision
+    - multimodal
+    - agent
+    - computer-use
+    - gpu
+    - cpu
+  last_checked: "2026-08-01"
+  overrides:
+    backend: llama-cpp
+    function:
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+      - vision
+    mmproj: llama-cpp/mmproj/Fara1.5-9B-GGUF/mmproj-Fara1.5-9B-f16.gguf
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/Fara1.5-9B-GGUF/Fara1.5-9B-Q4_K_M.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Fara1.5-9B-GGUF/Fara1.5-9B-Q4_K_M.gguf
+      sha256: a02e7220337b87290bca7ef7225ba4afa7104efa05c0851de6c29ec9c5d04c7d
+      uri: huggingface://bartowski/Fara1.5-9B-GGUF/Fara1.5-9B-Q4_K_M.gguf
+    - filename: llama-cpp/mmproj/Fara1.5-9B-GGUF/mmproj-Fara1.5-9B-f16.gguf
+      sha256: 97b423c81719ffc367124a9739d6feb6f62d62f60869a6d385a701b963ce1906
+      uri: huggingface://bartowski/Fara1.5-9B-GGUF/mmproj-Fara1.5-9B-f16.gguf
+- !!merge <<: *fara1-5-9b
+  name: fara1.5-9b-q8
+  variants: []
+  description: |
+    Fara1.5-9B is Microsoft's 9B-parameter multimodal computer-use agent for web browsers, fine-tuned from Qwen3.5-9B. It accepts screenshots and text, emits structured browser actions, supports a 262K-token context, and should be deployed with appropriate sandboxing and user-confirmation controls. This entry uses the higher-quality Q8_0 GGUF quantization.
+  overrides:
+    backend: llama-cpp
+    function:
+      grammar:
+        disable: true
+    known_usecases:
+      - chat
+      - vision
+    mmproj: llama-cpp/mmproj/Fara1.5-9B-GGUF/mmproj-Fara1.5-9B-f16.gguf
+    options:
+      - use_jinja:true
+    parameters:
+      model: llama-cpp/models/Fara1.5-9B-GGUF/Fara1.5-9B-Q8_0.gguf
+    template:
+      use_tokenizer_template: true
+  files:
+    - filename: llama-cpp/models/Fara1.5-9B-GGUF/Fara1.5-9B-Q8_0.gguf
+      sha256: a2e30cca7aec006266308153ae781347505af16baa514bbd4e0e3f4a79ea3a22
+      uri: huggingface://bartowski/Fara1.5-9B-GGUF/Fara1.5-9B-Q8_0.gguf
+    - filename: llama-cpp/mmproj/Fara1.5-9B-GGUF/mmproj-Fara1.5-9B-f16.gguf
+      sha256: 97b423c81719ffc367124a9739d6feb6f62d62f60869a6d385a701b963ce1906
+      uri: huggingface://bartowski/Fara1.5-9B-GGUF/mmproj-Fara1.5-9B-f16.gguf
 - name: fara1.5-27b
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   variants:
@@ -5865,14 +6214,14 @@
     - instruction-tuned
     - code
     - math
-  last_checked: "2026-07-28"
+  last_checked: "2026-08-01"
   overrides:
     parameters:
-      model: nanbeige4.2-3b-Q8_0.gguf
+      model: Nanbeige4.2-3B-Q8_0.gguf
   files:
-    - filename: nanbeige4.2-3b-Q8_0.gguf
-      sha256: 44707bb25e7ba3f2b0b5f3c2311da95ee3676986dd2014ce2aaeb14113590e33
-      uri: huggingface://owao/Nanbeige4.2-3B-GGUF/nanbeige4.2-3b-Q8_0.gguf
+    - filename: Nanbeige4.2-3B-Q8_0.gguf
+      sha256: 4f8bd17cdf58bea2a94aef03457e0b8f019c26fe4daee7ae49b61bfa935a9126
+      uri: huggingface://owao/Nanbeige4.2-3B-GGUF/Nanbeige4.2-3B-Q8_0.gguf
 - name: nanbeige4.2-3b
   url: github:mudler/LocalAI/gallery/nanbeige4.2.yaml@master
   urls:
@@ -5895,16 +6244,16 @@
     - instruction-tuned
     - code
     - math
-  last_checked: "2026-07-28"
+  last_checked: "2026-08-01"
   variants:
     - model: nanbeige4.2-3b-q8
   overrides:
     parameters:
-      model: nanbeige4.2-3b-Q4_K_M.gguf
+      model: Nanbeige4.2-3B-Q4_K_M.gguf
   files:
-    - filename: nanbeige4.2-3b-Q4_K_M.gguf
-      sha256: 9ffd17d14472ff208409b3f51a6d87a5e5ec1b878b9a6f4dfe15c2a883366104
-      uri: huggingface://owao/Nanbeige4.2-3B-GGUF/nanbeige4.2-3b-Q4_K_M.gguf
+    - filename: Nanbeige4.2-3B-Q4_K_M.gguf
+      sha256: ffe1b9b8ee95ec4b962c379905aa8be6f72ae9c4645c6c70e3b6ff7b197e6ef4
+      uri: huggingface://owao/Nanbeige4.2-3B-GGUF/Nanbeige4.2-3B-Q4_K_M.gguf
 - name: nemo-parakeet-tdt-0.6b
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
@@ -10043,6 +10392,54 @@
     - filename: ai21labs_AI21-Jamba-Reasoning-3B-Q4_K_M.gguf
       sha256: ac7ec0648dea62d1efb5ef6e7268c748ffc71f1c26eebe97eccff0a8d41608e6
       uri: huggingface://bartowski/ai21labs_AI21-Jamba-Reasoning-3B-GGUF/ai21labs_AI21-Jamba-Reasoning-3B-Q4_K_M.gguf
+- &antares-1b
+  name: "antares-1b"
+  variants:
+    - model: antares-1b-q8
+  url: "github:mudler/LocalAI/gallery/granite4.yaml@master"
+  urls:
+    - https://huggingface.co/fdtn-ai/antares-1b
+    - https://huggingface.co/HolkViking/antares-1b-Q4_K_M-GGUF
+    - https://huggingface.co/mitkox/antares-1b-Q8_0-GGUF
+  description: |
+    Antares-1B is an Apache-2.0 Granite 4.0 model specialized for vulnerability
+    localization in real-world codebases. It operates as a terminal agent that
+    navigates repositories, inspects source files, and identifies vulnerable
+    file paths. This entry uses the Q4_K_M GGUF quantization.
+  license: "apache-2.0"
+  tags:
+    - granite
+    - 1b
+    - gguf
+    - quantized
+    - llm
+    - security
+    - vulnerability-detection
+    - agentic
+    - chat
+  last_checked: "2026-07-31"
+  overrides:
+    parameters:
+      model: antares-1b-q4_k_m.gguf
+  files:
+    - filename: antares-1b-q4_k_m.gguf
+      sha256: 562693472633ad1b0fe2d0cda54edfead53aea2ff2c7e0bdbf5f393291f2ba2c
+      uri: huggingface://HolkViking/antares-1b-Q4_K_M-GGUF/antares-1b-q4_k_m.gguf
+- !!merge <<: *antares-1b
+  name: "antares-1b-q8"
+  variants: []
+  description: |
+    Antares-1B is an Apache-2.0 Granite 4.0 model specialized for vulnerability
+    localization in real-world codebases. It operates as a terminal agent that
+    navigates repositories, inspects source files, and identifies vulnerable
+    file paths. This entry uses the higher-quality Q8_0 GGUF quantization.
+  overrides:
+    parameters:
+      model: antares-1b-q8_0.gguf
+  files:
+    - filename: antares-1b-q8_0.gguf
+      sha256: f0b147c42d9056d59eb3f0ab51d4d57a2259bf1b4e37ae986ef96a6c2e0ac6b7
+      uri: huggingface://mitkox/antares-1b-Q8_0-GGUF/antares-1b-q8_0.gguf
 - name: ibm-granite_granite-4.0-h-small
   url: github:mudler/LocalAI/gallery/granite4.yaml@master
   urls:
@@ -12971,8 +13368,8 @@
       model: rfdetr-seg-medium-f16.gguf
   files:
     - filename: rfdetr-seg-medium-f16.gguf
-      sha256: 885d85ed6935495fc50ff464e06b6ea3bd8e8386865852d68a8be0f649d65afe
       uri: huggingface://mudler/rfdetr-cpp-seg-medium/rfdetr-seg-medium-f16.gguf
+      sha256: dd7c8da7cf0a2e64a1002f5ff66d7fede45b00e612457f249bcd9d4a0c122566
 - name: rfdetr-cpp-seg-large
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
@@ -13000,8 +13397,8 @@
       model: rfdetr-seg-large-f16.gguf
   files:
     - filename: rfdetr-seg-large-f16.gguf
-      sha256: 90423066d0791b4ae249f3986cce1f095a1e4090bf46800bf7f9e371ea80d559
       uri: huggingface://mudler/rfdetr-cpp-seg-large/rfdetr-seg-large-f16.gguf
+      sha256: ffc631b8e6115b11bdbb8e876c77aaa3e3e5d8c41c00ce8417ebbf183b1e6404
 - name: rfdetr-cpp-seg-xlarge
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
@@ -13029,8 +13426,8 @@
       model: rfdetr-seg-xlarge-f16.gguf
   files:
     - filename: rfdetr-seg-xlarge-f16.gguf
-      sha256: 0b82de4a6e65a40bc930979a1a4281cb24de35203d30eeefd797c858101a7bec
       uri: huggingface://mudler/rfdetr-cpp-seg-xlarge/rfdetr-seg-xlarge-f16.gguf
+      sha256: 72b6210e255ebb89a8d471c06a6f4b5307205b1a34b9e4292de90d1488da4e26
 - name: rfdetr-cpp-seg-2xlarge
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:
@@ -13059,8 +13456,8 @@
       model: rfdetr-seg-2xlarge-f16.gguf
   files:
     - filename: rfdetr-seg-2xlarge-f16.gguf
-      sha256: 7f957997db23e844194ea8266a95b4adc3deb6d0b71c0924922b20fbdeafa299
       uri: huggingface://mudler/rfdetr-cpp-seg-2xlarge/rfdetr-seg-2xlarge-f16.gguf
+      sha256: 00f3988bdf9a382b06610c200b3938b65a7731d14eafa84b73f6c3b5be4af8d9
 - name: edgetam
   url: github:mudler/LocalAI/gallery/virtual.yaml@master
   urls:

internal/version.go

@@ -8,3 +8,11 @@ var Commit = ""
 func PrintableVersion() string {
 	return fmt.Sprintf("%s (%s)", Version, Commit)
 }
+
+// UserAgent returns the version-aware client identity used for outbound requests.
+func UserAgent() string {
+	if Version == "" {
+		return "LocalAI"
+	}
+	return fmt.Sprintf("LocalAI/%s", Version)
+}

pkg/oci/cosignverify/useragent_internal_test.go

@@ -0,0 +1,56 @@
+package cosignverify
+
+import (
+	"context"
+	"io"
+	"net/http"
+	"strings"
+
+	"github.com/google/go-containerregistry/pkg/name"
+	"github.com/google/go-containerregistry/pkg/v1/remote"
+	"github.com/mudler/LocalAI/internal"
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+)
+
+type roundTripFunc func(*http.Request) (*http.Response, error)
+
+func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
+	return f(req)
+}
+
+var _ = Describe("registry User-Agent", func() {
+	It("identifies LocalAI during signature verification requests", func() {
+		originalVersion := internal.Version
+		internal.Version = "v-test"
+		DeferCleanup(func() {
+			internal.Version = originalVersion
+		})
+
+		var userAgent string
+		transport := roundTripFunc(func(req *http.Request) (*http.Response, error) {
+			userAgent = req.Header.Get("User-Agent")
+			return &http.Response{
+				StatusCode: http.StatusOK,
+				Header: http.Header{
+					"Content-Type":          []string{"application/vnd.oci.image.manifest.v1+json"},
+					"Docker-Content-Digest": []string{"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"},
+				},
+				Body:    io.NopCloser(strings.NewReader("")),
+				Request: req,
+			}, nil
+		})
+
+		verifier, err := NewVerifier(Policy{
+			Issuer:        "https://token.actions.githubusercontent.com",
+			IdentityRegex: `^https://github\.com/example/.*`,
+		}, nil, transport)
+		Expect(err).NotTo(HaveOccurred())
+
+		ref, err := name.ParseReference("registry.example.com/localai/backend:latest")
+		Expect(err).NotTo(HaveOccurred())
+		_, err = remote.Head(ref, verifier.remoteOptions(context.Background())...)
+		Expect(err).NotTo(HaveOccurred())
+		Expect(userAgent).To(ContainSubstring("LocalAI/v-test"))
+	})
+})

pkg/oci/cosignverify/verify.go

@@ -30,6 +30,7 @@ import (
 	"github.com/google/go-containerregistry/pkg/v1/remote"
 	"github.com/google/go-containerregistry/pkg/v1/remote/transport"
 
+	"github.com/mudler/LocalAI/internal"
 	"github.com/sigstore/sigstore-go/pkg/root"
 	"github.com/sigstore/sigstore-go/pkg/tuf"
 	"github.com/sigstore/sigstore-go/pkg/verify"
@@ -297,6 +298,7 @@ func (v *Verifier) remoteOptions(ctx context.Context) []remote.Option {
 	opts := []remote.Option{
 		remote.WithContext(ctx),
 		remote.WithTransport(t),
+		remote.WithUserAgent(internal.UserAgent()),
 	}
 	if v.auth != nil {
 		opts = append(opts, remote.WithAuth(staticAuth{auth: v.auth}))

pkg/oci/useragent.go

@@ -1,10 +1,6 @@
 package oci
 
-import (
-	"fmt"
-
-	"github.com/mudler/LocalAI/internal"
-)
+import "github.com/mudler/LocalAI/internal"
 
 // UserAgent returns the User-Agent string LocalAI sends on outbound registry
 // requests (OCI registries and Ollama). It identifies the client as LocalAI
@@ -12,8 +8,5 @@ import (
 // can attribute client-side usage to LocalAI rather than to the generic
 // User-Agent of the underlying transport library.
 func UserAgent() string {
-	if internal.Version == "" {
-		return "LocalAI"
-	}
-	return fmt.Sprintf("LocalAI/%s", internal.Version)
+	return internal.UserAgent()
 }

pkg/utils/untar.go

@@ -70,6 +70,11 @@ func ExtractArchive(archive, dst string) error {
 		if f.FileInfo.Mode()&os.ModeSymlink != 0 {
 			return fmt.Errorf("archive contains a symlink")
 		}
+		if linkname, ok := archiveMemberLinkname(f); ok {
+			if err := validateArchiveMemberPath(extractRoot, linkname); err != nil {
+				return err
+			}
+		}
 		return nil
 	})
 
@@ -95,6 +100,18 @@ func archiveMemberName(f archiver.File) string {
 	}
 }
 
+// archiveMemberLinkname reports the target of a tar hardlink member, which carries a regular file mode and so is not caught by the symlink check.
+func archiveMemberLinkname(f archiver.File) (string, bool) {
+	switch h := f.Header.(type) {
+	case tar.Header:
+		return h.Linkname, h.Typeflag == tar.TypeLink
+	case *tar.Header:
+		return h.Linkname, h.Typeflag == tar.TypeLink
+	default:
+		return "", false
+	}
+}
+
 func validateArchiveMemberPath(root, name string) error {
 	if name == "" {
 		return fmt.Errorf("archive contains an empty path")

pkg/utils/untar_test.go

@@ -3,6 +3,7 @@ package utils_test
 import (
 	"archive/tar"
 	"archive/zip"
+	"compress/gzip"
 	"os"
 	"path/filepath"
 
@@ -59,6 +60,53 @@ var _ = Describe("utils/archive tests", func() {
 		Expect(err.Error()).To(ContainSubstring("unsafe path"))
 		Expect(filepath.Join(tmpDir, "escaped.txt")).ToNot(BeAnExistingFile())
 	})
+
+	It("rejects tar hardlinks that overwrite a file outside the destination", func() {
+		tmpDir := GinkgoT().TempDir()
+		archivePath := filepath.Join(tmpDir, "model.tar.gz")
+		extractPath := filepath.Join(tmpDir, "models")
+		outsidePath := filepath.Join(tmpDir, "outside.txt")
+
+		Expect(os.WriteFile(outsidePath, []byte("original"), 0o600)).To(Succeed())
+		Expect(writeTarGzArchiveWithHardlinkedFile(archivePath, "payload.bin", "../outside.txt", "overwritten")).To(Succeed())
+
+		err := ExtractArchive(archivePath, extractPath)
+
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("unsafe path"))
+
+		contents, readErr := os.ReadFile(outsidePath)
+		Expect(readErr).ToNot(HaveOccurred())
+		Expect(string(contents)).To(Equal("original"))
+	})
+
+	It("extracts tar hardlinks that stay inside the destination", func() {
+		tmpDir := GinkgoT().TempDir()
+		archivePath := filepath.Join(tmpDir, "model.tar.gz")
+		extractPath := filepath.Join(tmpDir, "models")
+
+		Expect(writeTarGzArchiveWithInternalHardlink(archivePath, "model.bin", "alias.bin", "weights")).To(Succeed())
+
+		Expect(ExtractArchive(archivePath, extractPath)).To(Succeed())
+
+		extracted, err := os.ReadFile(filepath.Join(extractPath, "alias.bin"))
+		Expect(err).ToNot(HaveOccurred())
+		Expect(string(extracted)).To(Equal("weights"))
+	})
+
+	It("rejects tar hardlinks that point outside the destination", func() {
+		tmpDir := GinkgoT().TempDir()
+		archivePath := filepath.Join(tmpDir, "model.tar")
+		extractPath := filepath.Join(tmpDir, "models")
+
+		Expect(writeTarArchiveWithHardlink(archivePath, "payload.bin", "../escaped.txt")).To(Succeed())
+
+		err := ExtractArchive(archivePath, extractPath)
+
+		Expect(err).To(HaveOccurred())
+		Expect(err.Error()).To(ContainSubstring("unsafe path"))
+		Expect(filepath.Join(tmpDir, "escaped.txt")).ToNot(BeAnExistingFile())
+	})
 })
 
 func writeZipArchive(path string, files map[string]string) (err error) {
@@ -126,3 +174,121 @@ func writeTarArchive(path string, files map[string]string) (err error) {
 
 	return nil
 }
+
+func writeTarArchiveWithHardlink(path, name, linkname string) (err error) {
+	out, err := os.Create(path)
+	if err != nil {
+		return err
+	}
+	defer func() {
+		if closeErr := out.Close(); err == nil {
+			err = closeErr
+		}
+	}()
+
+	writer := tar.NewWriter(out)
+	defer func() {
+		if closeErr := writer.Close(); err == nil {
+			err = closeErr
+		}
+	}()
+
+	return writer.WriteHeader(&tar.Header{
+		Name:     name,
+		Linkname: linkname,
+		Typeflag: tar.TypeLink,
+		Mode:     0o600,
+	})
+}
+
+func writeTarGzArchiveWithHardlinkedFile(path, name, linkname, contents string) (err error) {
+	out, err := os.Create(path)
+	if err != nil {
+		return err
+	}
+	defer func() {
+		if closeErr := out.Close(); err == nil {
+			err = closeErr
+		}
+	}()
+
+	compressor := gzip.NewWriter(out)
+	defer func() {
+		if closeErr := compressor.Close(); err == nil {
+			err = closeErr
+		}
+	}()
+
+	writer := tar.NewWriter(compressor)
+	defer func() {
+		if closeErr := writer.Close(); err == nil {
+			err = closeErr
+		}
+	}()
+
+	if err := writer.WriteHeader(&tar.Header{
+		Name:     name,
+		Linkname: linkname,
+		Typeflag: tar.TypeLink,
+		Mode:     0o600,
+	}); err != nil {
+		return err
+	}
+
+	data := []byte(contents)
+	if err := writer.WriteHeader(&tar.Header{
+		Name: name,
+		Mode: 0o600,
+		Size: int64(len(data)),
+	}); err != nil {
+		return err
+	}
+	_, err = writer.Write(data)
+
+	return err
+}
+
+func writeTarGzArchiveWithInternalHardlink(path, targetName, linkName, contents string) (err error) {
+	out, err := os.Create(path)
+	if err != nil {
+		return err
+	}
+	defer func() {
+		if closeErr := out.Close(); err == nil {
+			err = closeErr
+		}
+	}()
+
+	compressor := gzip.NewWriter(out)
+	defer func() {
+		if closeErr := compressor.Close(); err == nil {
+			err = closeErr
+		}
+	}()
+
+	writer := tar.NewWriter(compressor)
+	defer func() {
+		if closeErr := writer.Close(); err == nil {
+			err = closeErr
+		}
+	}()
+
+	data := []byte(contents)
+	if err := writer.WriteHeader(&tar.Header{
+		Name: targetName,
+		Mode: 0o600,
+		Size: int64(len(data)),
+	}); err != nil {
+		return err
+	}
+	if _, err := writer.Write(data); err != nil {
+		return err
+	}
+
+	return writer.WriteHeader(&tar.Header{
+		Name:     linkName,
+		Linkname: targetName,
+		Typeflag: tar.TypeLink,
+		Mode:     0o600,
+	})
+}

scripts/build/backend-run-intel-env_test.sh

@@ -0,0 +1,139 @@
+#!/bin/bash
+# Checks how the run.sh of each C++ backend sets up the Intel graphics driver.
+#
+# A backend built for Intel GPUs carries its own copy of the Intel graphics
+# driver. run.sh has to tell Level Zero, which is how llama.cpp reaches the
+# card, to use that copy. Three things must hold, and all three have broken in
+# the past:
+#
+#   1. If the user already chose a driver, keep the user's choice. Otherwise a
+#      machine with a graphics card too new for the carried driver stops
+#      working, with no way to get back to the driver that did work.
+#   2. Say nothing about OpenCL. No OpenCL driver is carried, so pointing
+#      OpenCL at the backend's own directory would leave it with no driver at
+#      all, where saying nothing leaves it the machine's own.
+#   3. Ask the driver for the amount of free memory. Without this, llama.cpp
+#      reads zero free memory on an integrated graphics chip, because such a
+#      chip has no memory of its own and shares the system's.
+#
+# The test builds a fake backend directory for each run.sh, runs it, and reads
+# back the variables it exported.
+set -euo pipefail
+
+WORK=$(mktemp -d)
+trap 'rm -rf "$WORK"' EXIT
+
+REPO_ROOT=$(dirname "$(dirname "$(dirname "$(realpath "$0")")")")
+
+RUN_SCRIPTS=(
+    "backend/cpp/llama-cpp/run.sh llama-cpp"
+    "backend/cpp/turboquant/run.sh turboquant"
+    "backend/cpp/bonsai/run.sh bonsai"
+)
+
+failures=0
+
+fail() {
+    echo "FAIL: $*"
+    failures=$((failures + 1))
+}
+
+# Builds a fake backend directory: the real run.sh, a stand-in for the backend
+# program that prints the variables we care about, and whichever libraries the
+# caller asked for.
+#
+# Usage: make_backend <dir> <program-prefix> [library ...]
+make_backend() {
+    local dir="$1" prefix="$2"
+    shift 2
+
+    mkdir -p "$dir/lib"
+    cp "$RUN_SH" "$dir/run.sh"
+    chmod +x "$dir/run.sh"
+
+    local lib
+    for lib in "$@"; do
+        : > "$dir/lib/$lib"
+    done
+
+    cat > "$dir/${prefix}-fallback" <<'PROGRAM'
+#!/bin/bash
+echo "level_zero_driver=${ZE_ENABLE_ALT_DRIVERS:-}"
+echo "opencl_driver_list=${OCL_ICD_VENDORS:-}"
+echo "report_free_memory=${ZES_ENABLE_SYSMAN:-}"
+PROGRAM
+    chmod +x "$dir/${prefix}-fallback"
+}
+
+# Runs a fake backend and prints the one variable asked for.
+# Usage: read_variable <dir> <name>
+read_variable() {
+    local dir="$1" name="$2"
+    bash "$dir/run.sh" 2>/dev/null | sed -n "s/^${name}=//p"
+}
+
+for entry in "${RUN_SCRIPTS[@]}"; do
+    read -r script prefix <<< "$entry"
+    RUN_SH="$REPO_ROOT/$script"
+
+    if [ ! -f "$RUN_SH" ]; then
+        fail "$script does not exist"
+        continue
+    fi
+
+    # An Intel build with its own graphics driver: point Level Zero and OpenCL
+    # at the bundled copies and ask for the free memory reading.
+    bundled="$WORK/$prefix-bundled"
+    make_backend "$bundled" "$prefix" \
+        libze_loader.so.1 libze_intel_gpu.so.1 libigdrcl.so
+    mkdir -p "$bundled/etc/OpenCL/vendors"
+    echo "libigdrcl.so" > "$bundled/etc/OpenCL/vendors/intel.icd"
+
+    got=$(read_variable "$bundled" level_zero_driver)
+    if [ "$got" != "$bundled/lib/libze_intel_gpu.so.1" ]; then
+        fail "$script: expected Level Zero to use the bundled driver, got '$got'"
+    fi
+
+    # Even with an OpenCL driver and a driver list sitting in the backend, which
+    # is what an older packaging left behind, OpenCL must be left alone.
+    got=$(read_variable "$bundled" opencl_driver_list)
+    if [ -n "$got" ]; then
+        fail "$script: OpenCL was pointed at the backend's own directory ('$got')"
+    fi
+
+    got=$(read_variable "$bundled" report_free_memory)
+    if [ "$got" != "1" ]; then
+        fail "$script: expected the free memory reading to be turned on, got '$got'"
+    fi
+
+    # The user picked a driver already. Both choices must survive.
+    got=$(ZE_ENABLE_ALT_DRIVERS=/usr/lib/host-driver.so \
+        read_variable "$bundled" level_zero_driver)
+    if [ "$got" != "/usr/lib/host-driver.so" ]; then
+        fail "$script: the user's Level Zero driver was overwritten with '$got'"
+    fi
+
+    got=$(ZES_ENABLE_SYSMAN=0 read_variable "$bundled" report_free_memory)
+    if [ "$got" != "0" ]; then
+        fail "$script: the user's free memory setting was overwritten with '$got'"
+    fi
+
+    # A build for some other kind of graphics card. None of the Intel
+    # variables belong here.
+    other="$WORK/$prefix-other"
+    make_backend "$other" "$prefix" libcublas.so.12
+
+    for name in level_zero_driver opencl_driver_list report_free_memory; do
+        got=$(read_variable "$other" "$name")
+        if [ -n "$got" ]; then
+            fail "$script: $name was set on a build with no Intel libraries ('$got')"
+        fi
+    done
+done
+
+if [ "$failures" -gt 0 ]; then
+    echo "$failures check(s) failed"
+    exit 1
+fi
+
+echo "PASS: every run.sh sets up the Intel graphics driver correctly"

scripts/build/llama-cpp-build-target_test.sh

@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+CURDIR=$(dirname "$(realpath "$0")")
+SELECTOR="$CURDIR/../../.docker/llama-cpp-build-target.sh"
+
+assert_target() {
+  local arch=$1
+  local build_type=$2
+  local expected=$3
+  local actual
+
+  actual=$("$SELECTOR" "$arch" "$build_type")
+  if [ "$actual" != "$expected" ]; then
+    echo "FAIL: $arch/$build_type selected $actual, expected $expected"
+    exit 1
+  fi
+}
+
+assert_target amd64 cublas llama-cpp-cpu-all
+assert_target amd64 vulkan llama-cpp-cpu-all
+assert_target amd64 "" llama-cpp-cpu-all
+assert_target arm64 cublas llama-cpp-fallback
+assert_target arm64 "" llama-cpp-cpu-all
+
+echo "PASS: llama.cpp build target preserves CPU variants where supported"

scripts/build/package-gpu-libs-intel_test.sh

@@ -0,0 +1,171 @@
+#!/bin/bash
+# Checks what package_intel_libs puts in a backend built for Intel GPUs.
+#
+# The packager copies the libraries a backend needs next to the backend itself,
+# so it can run on a machine that has none of them installed. Four things have
+# to happen, and each one has been missing at some point:
+#
+#   1. Copy the libraries the backend program is linked against. Some of them
+#      are only reachable from the program, not from any other copied library,
+#      so looking at the copied libraries alone is not enough.
+#   2. Copy the libraries that are opened by name while the program runs. Those
+#      are invisible to any tool that reads the list of libraries a file is
+#      linked against, so they have to be named one by one.
+#   3. Copy the Intel graphics driver, which is also opened by name at run
+#      time.
+#   4. Copy only the driver Level Zero talks to, and leave the OpenCL one out.
+#      llama.cpp reaches an Intel GPU through Level Zero; the OpenCL driver
+#      brings a second copy of the graphics compiler with it, which is about
+#      139 MB for a path nothing here uses.
+#
+# The test builds a stand-in for an oneAPI installation, a stand-in for a
+# driver installation and two fake backend programs, runs the real packager and
+# checks the result.
+set -euo pipefail
+
+CURDIR=$(dirname "$(realpath "$0")")
+SCRIPT="$CURDIR/package-gpu-libs.sh"
+
+if ! command -v gcc >/dev/null 2>&1 || ! command -v ldd >/dev/null 2>&1; then
+    echo "SKIP: gcc/ldd not available"
+    exit 0
+fi
+
+WORK=$(mktemp -d)
+trap 'rm -rf "$WORK"' EXIT
+
+# Stand-in for /opt/intel/oneapi/*/lib.
+ONEAPI="$WORK/oneapi/lib"
+mkdir -p "$ONEAPI"
+
+# Two libraries the backend programs are linked against, one each. Nothing else
+# refers to them, so they can only be found by looking at the programs.
+echo 'int first_fn(void){return 1;}' > "$WORK/first.c"
+gcc -shared -fPIC -o "$ONEAPI/libfakeoneapifirst.so.2" "$WORK/first.c"
+echo 'int second_fn(void){return 2;}' > "$WORK/second.c"
+gcc -shared -fPIC -o "$ONEAPI/libfakeoneapisecond.so.2" "$WORK/second.c"
+
+# A library that is opened by name while the program runs. Nothing is linked
+# against it, so only the list of names in the packager can find it.
+echo 'int adapter_fn(void){return 3;}' > "$WORK/adapter.c"
+gcc -shared -fPIC -o "$ONEAPI/libur_adapter_level_zero.so.0" "$WORK/adapter.c"
+
+# Two fake backend programs, in the directory the real packaging script uses:
+# package/, one level above package/lib. One is named after llama.cpp, the
+# other is not, because the same packager serves several backends.
+PKG="$WORK/package"
+TARGET="$PKG/lib"
+mkdir -p "$TARGET"
+echo 'int first_fn(void); int main(void){return first_fn();}' > "$WORK/main1.c"
+gcc -o "$PKG/llama-cpp-grpc" "$WORK/main1.c" \
+    -L"$ONEAPI" -l:libfakeoneapifirst.so.2 -Wl,-rpath,"$ONEAPI"
+echo 'int second_fn(void); int main(void){return second_fn();}' > "$WORK/main2.c"
+gcc -o "$PKG/bonsai-grpc" "$WORK/main2.c" \
+    -L"$ONEAPI" -l:libfakeoneapisecond.so.2 -Wl,-rpath,"$ONEAPI"
+
+# The real directory also holds the script that starts the backend. Looking at a
+# shell script for libraries has to be harmless.
+printf '#!/bin/bash\necho started\n' > "$PKG/run.sh"
+chmod +x "$PKG/run.sh"
+
+# Stand-in for the Intel graphics driver installation. These files are opened by
+# name at run time rather than linked, so the packager has to name the ones it
+# wants.
+DRV="$WORK/driver"
+mkdir -p "$DRV/intel-opencl"
+echo 'int ze_drv(void){return 4;}' > "$WORK/zedrv.c"
+gcc -shared -fPIC -o "$DRV/libze_intel_gpu.so.1" "$WORK/zedrv.c"
+echo 'int cl_drv(void){return 5;}' > "$WORK/cldrv.c"
+gcc -shared -fPIC -o "$DRV/intel-opencl/libigdrcl.so" "$WORK/cldrv.c"
+
+# The compiler front end the OpenCL driver needs, and the large library it is
+# linked against. The link is what makes the big one arrive on its own if the
+# front end is ever copied again, so the fake mirrors it.
+echo 'int clang_fn(void){return 6;}' > "$WORK/clang.c"
+gcc -shared -fPIC -o "$DRV/libopencl-clang.so.15" "$WORK/clang.c"
+echo 'int clang_fn(void); int fcl_fn(void){return clang_fn();}' > "$WORK/fcl.c"
+gcc -shared -fPIC -o "$DRV/libigdfcl.so.2" "$WORK/fcl.c" \
+    -L"$DRV" -l:libopencl-clang.so.15 -Wl,-rpath,"$DRV"
+
+# Let the fake oneAPI libraries be found the way the real ones are on the build
+# machine.
+export LD_LIBRARY_PATH="$ONEAPI:${LD_LIBRARY_PATH:-}"
+
+# shellcheck source=/dev/null
+source "$SCRIPT" "$TARGET"
+
+export BUILD_TYPE=sycl_f16
+export INTEL_ONEAPI_LIB_DIRS="$ONEAPI"
+export INTEL_DRIVER_LIB_DIRS="$DRV $DRV/intel-opencl"
+package_intel_libs
+
+fail=false
+
+for lib in libfakeoneapifirst.so.2 libfakeoneapisecond.so.2; do
+    if [ ! -e "$TARGET/$lib" ]; then
+        echo "FAIL: $lib is missing; the backend programs' own libraries were not copied"
+        fail=true
+    fi
+done
+
+if [ ! -e "$TARGET/libur_adapter_level_zero.so.0" ]; then
+    echo "FAIL: the Level Zero adapter, which is opened by name, was not copied"
+    fail=true
+fi
+
+if [ ! -e "$TARGET/libze_intel_gpu.so.1" ]; then
+    echo "FAIL: the Level Zero graphics driver was not copied"
+    fail=true
+fi
+
+# The OpenCL driver and the compiler front end that hangs off it are left out,
+# and so is the driver list that would name them.
+for lib in libigdrcl.so libigdfcl.so.2 libopencl-clang.so.15; do
+    if [ -e "$TARGET/$lib" ]; then
+        echo "FAIL: $lib was copied, but nothing here uses the OpenCL path"
+        fail=true
+    fi
+done
+if [ -e "$TARGET/../etc/OpenCL" ]; then
+    echo "FAIL: an OpenCL driver list was created for a path nothing uses"
+    fail=true
+fi
+
+# The Python backends for Intel GPUs, built as BUILD_TYPE=intel, start without
+# run.sh and so never load a copied driver. Copying one for them would add
+# several hundred megabytes that nothing reads.
+PYTHON_STYLE="$WORK/python-backend/lib"
+mkdir -p "$PYTHON_STYLE"
+(
+    BUILD_TYPE=intel \
+    INTEL_ONEAPI_LIB_DIRS="$ONEAPI" \
+    INTEL_DRIVER_LIB_DIRS="$DRV $DRV/intel-opencl" \
+    bash -c 'source "$0" "$1"; package_intel_libs' "$SCRIPT" "$PYTHON_STYLE"
+) >/dev/null 2>&1
+if [ -e "$PYTHON_STYLE/libze_intel_gpu.so.1" ]; then
+    echo "FAIL: the graphics driver was copied into a backend that cannot load it"
+    fail=true
+fi
+
+# A build for Intel GPUs that ends up with no driver still works, but only on a
+# machine that has its own. That is easy to cause by accident and impossible to
+# see afterwards, so the packager has to say so.
+warning=$(
+    BUILD_TYPE=sycl_f16 \
+    INTEL_ONEAPI_LIB_DIRS="$ONEAPI" \
+    INTEL_DRIVER_LIB_DIRS="$WORK/empty" \
+    bash -c 'source "$0" "$1"; package_intel_libs' \
+        "$SCRIPT" "$WORK/nodriver/lib" 2>&1 >/dev/null || true
+)
+if ! grep -qi "no intel graphics driver" <<< "$warning"; then
+    echo "FAIL: no warning when the graphics driver could not be copied"
+    fail=true
+fi
+
+if [ "$fail" = true ]; then
+    ls -la "$TARGET" || true
+    exit 1
+fi
+
+echo "PASS: the oneAPI libraries, the adapter and the Level Zero graphics driver were all handled"
+exit 0

scripts/build/package-gpu-libs.sh

@@ -675,21 +675,43 @@ package_rocm_libs() {
 package_intel_libs() {
     echo "Packaging Intel oneAPI/SYCL libraries for BUILD_TYPE=${BUILD_TYPE}..."
 
-    local intel_lib_paths=(
-        "/opt/intel/oneapi/compiler/latest/lib"
-        "/opt/intel/oneapi/mkl/latest/lib/intel64"
-        "/opt/intel/oneapi/tbb/latest/lib/intel64/gcc4.8"
-    )
+    # Where to look for the oneAPI libraries. The default is the standard install
+    # layout. The list can be overridden with a space-separated one, which lets
+    # the tests run without a real oneAPI install, the same way ROCM_BASE_DIRS
+    # works. Both the current and the older math library layouts are listed, and
+    # the check below skips whichever of them is absent.
+    local intel_lib_paths
+    if [ -n "${INTEL_ONEAPI_LIB_DIRS:-}" ]; then
+        # shellcheck disable=SC2206  # intentional word-split of the override
+        intel_lib_paths=(${INTEL_ONEAPI_LIB_DIRS})
+    else
+        intel_lib_paths=(
+            "/opt/intel/oneapi/compiler/latest/lib"
+            "/opt/intel/oneapi/mkl/latest/lib"
+            "/opt/intel/oneapi/mkl/latest/lib/intel64"
+            "/opt/intel/oneapi/dnnl/latest/lib"
+            "/opt/intel/oneapi/tbb/latest/lib/intel64/gcc4.8"
+        )
+    fi
 
-    # Core Intel oneAPI runtime libraries
+    # The oneAPI libraries a backend needs at run time. The math library entries
+    # cover both of its number formats and both of its threading layers, because
+    # the llama.cpp build for Intel GPUs uses a different combination than the
+    # rest. The libur_adapter_* entries have to be named here even though nothing
+    # is linked against them: oneAPI opens them by name while the program runs,
+    # so the dependency scan later in this function cannot see them.
     local intel_libs=(
         "libsycl.so*"
         "libOpenCL.so*"
         "libmkl_core.so*"
         "libmkl_intel_lp64.so*"
+        "libmkl_intel_ilp64.so*"
         "libmkl_intel_thread.so*"
+        "libmkl_tbb_thread.so*"
         "libmkl_sequential.so*"
         "libmkl_sycl.so*"
+        "libmkl_sycl_blas.so*"
+        "libdnnl.so*"
         "libiomp5.so*"
         "libsvml.so*"
         "libirng.so*"
@@ -697,6 +719,10 @@ package_intel_libs() {
         "libintlc.so*"
         "libtbb.so*"
         "libtbbmalloc.so*"
+        "libur_loader.so*"
+        "libur_adapter_level_zero.so*"
+        "libur_adapter_level_zero_v2.so*"
+        "libur_adapter_opencl.so*"
         "libpi_level_zero.so*"
         "libpi_opencl.so*"
         "libze_loader.so*"
@@ -710,10 +736,92 @@ package_intel_libs() {
         fi
     done
 
-    # Pull in transitive deps the allowlist misses so the backend is
-    # self-contained (same class of failure as #10537).
+    # Copy the libraries the backend programs themselves are linked against. The
+    # list above is not enough on its own: the programs are linked directly
+    # against several oneAPI libraries that no copied library refers to, so
+    # without this step the backend only ran inside the build image, where oneAPI
+    # happens to be on the library path.
+    #
+    # The programs sit one level above the target directory, in package/, next to
+    # the run.sh that starts them. Every backend that builds for Intel GPUs is
+    # covered by looking at all of them, rather than at one set of names, because
+    # llama.cpp, turboquant and bonsai all come through here.
+    local pkg_dir="$TARGET_LIB_DIR/.."
+    local bin
+    for bin in "$pkg_dir"/*; do
+        if [ -f "$bin" ] && [ -x "$bin" ]; then
+            copy_elf_deps "$bin"
+        fi
+    done
+
+    # Copy the Intel graphics driver itself, the way the Vulkan packaging copies
+    # the Mesa driver. Level Zero opens the driver by name while the program
+    # runs, so no dependency scan can find it and it has to be named here.
+    #
+    # This is what lets the backend run on a machine with no Intel graphics
+    # packages installed, and also on a machine whose own driver was built
+    # against a newer C library than the one this backend carries, where loading
+    # the host's driver crashes. Carrying the driver is safe across kernel
+    # versions because it reaches the graphics hardware through an interface the
+    # kernel keeps stable. The NVIDIA driver has no such interface, which is why
+    # that one is never copied.
+    #
+    # Only the builds that start through run.sh get a driver: run.sh is what
+    # tells Level Zero to use it. The Python backends built for Intel GPUs start
+    # differently and keep using the host's driver, so copying one for them would
+    # add several hundred megabytes that nothing would ever load.
+    #
+    # Only the Level Zero side is copied. llama.cpp reaches an Intel GPU through
+    # Level Zero, which hands the driver ready-compiled programs and so needs
+    # only the compiler's back end. The OpenCL driver can be handed source code
+    # instead, so it also needs the compiler's front end, and that pulls in a
+    # copy of clang: around 139 MB for a path nothing here takes. A user who
+    # wants OpenCL has their machine's own.
+    case "${BUILD_TYPE:-}" in
+        sycl*)
+            local intel_driver_lib_dirs
+            if [ -n "${INTEL_DRIVER_LIB_DIRS:-}" ]; then
+                # shellcheck disable=SC2206  # split the override into words on purpose
+                intel_driver_lib_dirs=(${INTEL_DRIVER_LIB_DIRS})
+            else
+                intel_driver_lib_dirs=(
+                    "/usr/lib/x86_64-linux-gnu"
+                    "/usr/lib"
+                )
+            fi
+            local driver_libs=(
+                "libze_intel_gpu.so*"   # the driver Level Zero talks to
+                "libigc.so*"            # turns compute programs into instructions for the card
+                "libigdgmm.so*"         # manages graphics memory
+            )
+            local drv_dir pat
+            for drv_dir in "${intel_driver_lib_dirs[@]}"; do
+                [ -d "$drv_dir" ] || continue
+                for pat in "${driver_libs[@]}"; do
+                    copy_libs_glob "${drv_dir}/${pat}"
+                done
+            done
+            ;;
+    esac
+
+    # Copy whatever the steps above still missed. Each library copied so far can
+    # need further libraries of its own, and a missing one stops the backend from
+    # starting at all (issue #10537).
     sweep_transitive_deps "$TARGET_LIB_DIR"
 
+    # Say so when a build meant for Intel GPUs ends up without a driver. It still
+    # works on a machine that has its own, so nothing fails here, and the only
+    # other way to notice is a user reporting that their GPU is not used. The
+    # usual cause is a build image that predates the driver being installed in
+    # .docker/install-base-deps.sh.
+    case "${BUILD_TYPE:-}" in
+        sycl*)
+            if [ ! -e "$TARGET_LIB_DIR/libze_intel_gpu.so.1" ]; then
+                echo "WARNING: no Intel graphics driver was found to copy. This backend will only use a GPU on a machine that has its own Intel driver installed." >&2
+            fi
+            ;;
+    esac
+
     echo "Intel oneAPI libraries packaged successfully"
 }
 

scripts/build/turboquant-build-target_test.sh

@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+CURDIR=$(dirname "$(realpath "$0")")
+SELECTOR="$CURDIR/../../.docker/turboquant-build-target.sh"
+
+assert_target() {
+  local arch=$1
+  local build_type=$2
+  local expected=$3
+  local actual
+
+  actual=$("$SELECTOR" "$arch" "$build_type")
+  if [ "$actual" != "$expected" ]; then
+    echo "FAIL: $arch/$build_type selected $actual, expected $expected"
+    exit 1
+  fi
+}
+
+assert_target amd64 cublas turboquant-cpu-all
+assert_target amd64 vulkan turboquant-cpu-all
+assert_target amd64 "" turboquant-cpu-all
+assert_target arm64 cublas turboquant-fallback
+assert_target arm64 "" turboquant-cpu-all
+
+echo "PASS: turboquant build target preserves CPU variants where supported"

scripts/lib/backend-filter.mjs

@@ -389,10 +389,14 @@ export const SHARED_BUILD_INPUTS = [
     darwin: always,
   },
   {
-    // Stages the CUDA/ROCm runtime libraries into every Python image's lib/.
-    // COPY'd and run by Dockerfile.python only. This is the #10946 case.
+    // Decides which GPU libraries end up inside an image. Every Linux image
+    // runs it: Dockerfile.python calls it directly, and the Go and C++ backends
+    // call it from their own package.sh. Naming only the Python images here is
+    // how a packaging fix for the Intel llama.cpp backend could merge and reach
+    // no image, which is the #10946 case all over again. The Darwin builds have
+    // their own packaging scripts and never call this one.
     matches: file => file === "scripts/build/package-gpu-libs.sh",
-    linux: isLinuxPython,
+    linux: always,
     darwin: never,
   },
   {

scripts/lib/backend-filter_test.mjs

@@ -86,20 +86,25 @@ const run = (changedFiles, previousMatrix) =>
 
 const names = entries => entries.map(e => e.backend).sort();
 
-test("a change to only package-gpu-libs.sh rebuilds every Python image", () => {
-  // The PR #10946 regression: this script is COPY'd and run by
-  // Dockerfile.python for every Python backend, but lives under scripts/, so
-  // the per-backend prefix match produced an empty matrix and the cuDNN
-  // packaging fix shipped to nothing.
+test("a change to only package-gpu-libs.sh rebuilds every Linux image", () => {
+  // The PR #10946 regression: this script decides which GPU libraries end up
+  // inside an image, but lives under scripts/, so the per-backend prefix match
+  // produced an empty matrix and the packaging fix shipped to nothing.
+  //
+  // Every Linux image runs it, not only the Python ones: Dockerfile.python
+  // calls it directly, and the Go and C++ backends call it from their own
+  // package.sh (see backend/cpp/llama-cpp/package.sh and backend/go/*/
+  // package.sh). Leaving those out is how a fix aimed at the Intel llama.cpp
+  // backend could merge and reach no image.
   const { filtered, filteredDarwin, changedBackends } = run([
     "scripts/build/package-gpu-libs.sh",
   ]);
 
-  assert.notEqual(filtered.length, 0, "expected a non-empty Linux matrix");
-  assert.deepEqual(names(filtered), ["diffusers", "vllm"]);
+  assert.equal(filtered.length, includes.length);
   assert.ok(changedBackends.has("vllm"));
+  assert.ok(changedBackends.has("llama-cpp"));
 
-  // Darwin Python builds never invoke it (see scripts/build/python-darwin.sh).
+  // The Darwin builds have their own packaging scripts and never call this one.
   assert.deepEqual(filteredDarwin, []);
 });
 

swagger/docs.go

@@ -1798,7 +1798,7 @@ const docTemplate = `{
                     },
                     {
                         "type": "integer",
-                        "description": "desired output sample rate",
+                        "description": "desired output sample rate in Hz; omit for the backend's own rate, otherwise 8000-192000",
                         "name": "sample_rate",
                         "in": "formData"
                     }
@@ -1855,7 +1855,7 @@ const docTemplate = `{
                     },
                     {
                         "type": "integer",
-                        "description": "desired output sample rate",
+                        "description": "desired output sample rate in Hz; omit for the backend's own rate, otherwise 8000-192000",
                         "name": "sample_rate",
                         "in": "formData"
                     }

swagger/swagger.json

@@ -1795,7 +1795,7 @@
                     },
                     {
                         "type": "integer",
-                        "description": "desired output sample rate",
+                        "description": "desired output sample rate in Hz; omit for the backend's own rate, otherwise 8000-192000",
                         "name": "sample_rate",
                         "in": "formData"
                     }
@@ -1852,7 +1852,7 @@
                     },
                     {
                         "type": "integer",
-                        "description": "desired output sample rate",
+                        "description": "desired output sample rate in Hz; omit for the backend's own rate, otherwise 8000-192000",
                         "name": "sample_rate",
                         "in": "formData"
                     }

swagger/swagger.yaml

@@ -3984,7 +3984,8 @@ paths:
         in: formData
         name: response_format
         type: string
-      - description: desired output sample rate
+      - description: desired output sample rate in Hz; omit for the backend's own
+          rate, otherwise 8000-192000
         in: formData
         name: sample_rate
         type: integer
@@ -4028,7 +4029,8 @@ paths:
         in: formData
         name: response_format
         type: string
-      - description: desired output sample rate
+      - description: desired output sample rate in Hz; omit for the backend's own
+          rate, otherwise 8000-192000
         in: formData
         name: sample_rate
         type: integer

website/README.md

@@ -0,0 +1,29 @@
+# localai.io
+
+The Hugo site served at the root of localai.io. The documentation is a separate
+Hugo site in [`../docs`](../docs) and is served under `/docs/`. CI builds both
+and uploads them as a single GitHub Pages artifact
+(see [`.github/workflows/gh-pages.yml`](../.github/workflows/gh-pages.yml)).
+
+## Running locally
+
+From the repository root:
+
+```bash
+make website    # this site only, http://localhost:1313/
+make docs       # documentation only, http://localhost:1313/docs/
+make site       # build both, merged into website/public, exactly as CI does
+make site-serve # the same merged build, served on http://localhost:8000
+```
+
+`make website` and `make docs` run `hugo serve`, so they pick up edits live but
+only cover one site at a time. Use `make site-serve` when you need the cross
+links between the two sites, or the redirects from the pre-split URLs, to work.
+
+`make site` also runs `.github/ci/gen-redirects.sh`, which leaves a meta-refresh
+page at every URL the documentation used to occupy before it moved under
+`/docs/`. GitHub Pages has no server-side redirects, so those files are the only
+thing keeping the old links alive.
+
+Set `SITE_BASE_URL` to change the base URL the merged build is generated for
+(default `http://localhost:8000`).

website/content/_index.md

@@ -0,0 +1,4 @@
+---
+title: "LocalAI"
+description: "LocalAI is the open source AI engine. Run any model, LLMs, vision, voice, image and video, on any hardware. No GPU required."
+---

website/content/blog/_index.md

@@ -0,0 +1,5 @@
+---
+title: "Blog"
+description: "Release write-ups, benchmark reports and engineering notes from the LocalAI team. Every number here comes out of a benchmark suite, a release or a commit, and the source is named so you can check it."
+extracss: ["blog.css"]
+---

website/content/blog/apex-moe-quantization.md

@@ -0,0 +1,112 @@
+---
+title: "APEX: a 35B MoE model at 12.2 GB, and faster than F16"
+date: 2026-04-10
+author: "Ettore Di Giacinto"
+category: "Research"
+tags: ["quantization", "APEX", "mixture-of-experts", "llama.cpp", "benchmarks"]
+summary: "Qwen3.5-35B-A3B goes from 64.6 GB to 12.2 GB and speeds up from 30.4 to 74.4 tokens per second. Perplexity moves from 6.537 to 7.088. Here is the precision assignment that does it, and where it costs you."
+extracss: ["blog.css"]
+---
+
+A 35B mixture-of-experts model at full precision is a 64.6 GB file, which puts it out of reach of every consumer GPU. APEX gets Qwen3.5-35B-A3B down to 12.2 GB, where it fits a 16 GB card with room for context, and it generates at 74.4 tokens per second instead of 30.4. The output is an ordinary GGUF that stock llama.cpp opens with no patches and no custom build.
+
+The compression is not free at that tier, and the numbers below say exactly what it costs. At the 21.3 GB tier it is closer to free than we expected: APEX Quality has a lower perplexity than the F16 model it was quantized from.
+
+## The measurements
+
+All of this is Qwen3.5-35B-A3B on an NVIDIA DGX Spark (GB10, 122 GB unified VRAM, CUDA 13). Perplexity is wikitext-2-raw at context 2048 over the full test set. HellaSwag and the other accuracy benchmarks run through llama.cpp at 400 tasks. `tg128` is token generation throughput.
+
+<div class="tw">
+<table>
+<thead><tr><th>Build</th><th>Size</th><th>Perplexity</th><th>KL mean</th><th>HellaSwag</th><th>MMLU</th><th>tg128 t/s</th></tr></thead>
+<tbody>
+<tr><td>F16</td><td>64.6 GB</td><td>6.537</td><td>-</td><td>82.5%</td><td>41.5%</td><td>30.4</td></tr>
+<tr><td>Q8_0</td><td>34.4 GB</td><td>6.533</td><td>0.0046</td><td>83.0%</td><td>41.2%</td><td>52.5</td></tr>
+<tr><td>Unsloth UD-Q8_K_XL</td><td>45.3 GB</td><td>6.536</td><td>0.0025</td><td>82.5%</td><td>41.3%</td><td>36.4</td></tr>
+<tr><td>Unsloth UD-Q4_K_XL</td><td>20.7 GB</td><td>6.554</td><td>0.0097</td><td>83.0%</td><td>40.6%</td><td>58.1</td></tr>
+<tr><td>bartowski IQ2_M</td><td>11.3 GB</td><td>7.303</td><td>0.1113</td><td>80.3%</td><td>39.6%</td><td>76.2</td></tr>
+<tr><td><b>APEX Quality</b></td><td><b>21.3 GB</b></td><td><b>6.527</b></td><td>0.0114</td><td>83.0%</td><td>41.2%</td><td><b>62.3</b></td></tr>
+<tr><td><b>APEX I-Quality</b></td><td><b>21.3 GB</b></td><td>6.552</td><td>0.0102</td><td><b>83.5%</b></td><td>41.4%</td><td>63.1</td></tr>
+<tr><td><b>APEX Balanced</b></td><td>23.6 GB</td><td>6.533</td><td>0.0088</td><td>83.0%</td><td>41.3%</td><td>60.8</td></tr>
+<tr><td><b>APEX Compact</b></td><td>16.1 GB</td><td>6.783</td><td>0.0469</td><td>82.5%</td><td>40.9%</td><td>69.8</td></tr>
+<tr><td><b>APEX I-Compact</b></td><td>16.1 GB</td><td>6.669</td><td>0.0332</td><td>81.8%</td><td>41.7%</td><td>69.8</td></tr>
+<tr><td><b>APEX Mini</b></td><td><b>12.2 GB</b></td><td>7.088</td><td>0.0870</td><td>81.0%</td><td>41.3%</td><td><b>74.4</b></td></tr>
+</tbody>
+</table>
+</div>
+
+Three things in that table are worth stopping on.
+
+APEX Quality is 21.3 GB, a third of F16, and its perplexity of 6.527 is lower than F16's 6.537 and lower than Q8_0's 6.533. Quantization noise acting as mild regularization on a wikitext evaluation is a known effect and we are not claiming the quantized model is smarter. The honest reading is that at this tier the loss is below the measurement floor.
+
+Against Unsloth's UD-Q8_K_XL, APEX I-Quality is half the size (21.3 GB against 45.3 GB), one point ahead on HellaSwag (83.5% against 82.5%), within 0.016 on perplexity, and 73% faster (63.1 t/s against 36.4). That is the comparison that matters for anyone choosing a published quant today.
+
+At the bottom end, APEX Mini beats bartowski IQ2_M on every metric while being 0.9 GB larger: perplexity 7.088 against 7.303, HellaSwag 81.0% against 80.3%, MMLU 41.3% against 39.6%.
+
+## Why it gets faster, not just smaller
+
+Token generation on a single stream is bound by memory bandwidth, not by arithmetic. Every generated token requires reading the active weights out of memory, so halving the bytes roughly halves the time spent waiting for them. Going from 64.6 GB to 12.2 GB takes throughput from 30.4 to 74.4 tokens per second, a 2.45x gain on the same hardware with the same kernels. Every APEX tier clears 60 t/s.
+
+That is also why a large well-behaved quant such as UD-Q8_K_XL is slower than a smaller one with equal quality. Size is a speed knob as much as a memory knob.
+
+## Per-tensor and per-layer precision
+
+Uniform quantization gives every tensor the same bit width, which spends the same precision on a weight that fires for every token and one that fires for 3% of them. In a mixture-of-experts model those two populations are enormous and they are easy to tell apart.
+
+APEX classifies every tensor into one of three roles and treats them differently.
+
+**Routed expert weights** (the gate, up and down projections inside the experts) are the bulk of the parameters, and only 8 of 256 experts are active per token. That 97% structural sparsity is what makes aggressive quantization safe here. The routing decision itself reads full-precision gate weights, so quantization noise inside an expert that was not selected never reaches the output at all. When an expert is selected, its contribution is one of eight summed paths, which further dilutes per-tensor error.
+
+**Shared expert weights** run for every single token and their weight distribution is heavy-tailed, with a kurtosis of 13.10 against 3.41 for routed experts. Those outliers carry real signal and low-bit formats clip them. Q8_0 is the minimum viable precision here, and dropping it is the fastest way to wreck a build.
+
+**Attention and SSM weights** are dense, contribute few parameters relative to the experts, and matter for generation quality. They sit at Q6_K throughout.
+
+On top of the role split there is a layer-wise gradient. The first and last five transformer layers do input embedding alignment and output logit generation, and they are measurably more sensitive than the middle layers, which perform more redundant intermediate processing. So routed experts get Q6_K at the edges (L0-4 and L35-39), Q5_K near the edges (L5-9 and L30-34), and Q4_K or IQ4_XS in the middle (L10-29). The tiers differ mostly in how far down that middle band is pushed: Compact runs Q4_K edges with Q3_K middle, and Mini goes to IQ2_S in the middle.
+
+None of this needs a patched llama.cpp. The assignments are expressed with the stock `--tensor-type-file` flag for per-layer rules and `--tensor-type` for component-level ones.
+
+## What the experiments ruled out
+
+Twenty-five or so systematic runs produced a few results that saved a lot of time later.
+
+Going from Q6_K to Q8_0 on routed experts costs 7.5 GB and buys zero perplexity improvement. Going below Q5_K on them causes measurable degradation. Q6_K is the ceiling worth paying for.
+
+Layer position matters more than uniform bit width. A two-tier gradient of Q6_K edges and Q5_K middle matches Q8_0 quality; a uniform Q5_K assignment at a similar size does not.
+
+IQ formats underperform K-quants on MoE experts. IQ3_S gives worse perplexity than Q3_K on routed expert tensors at a similar bit rate, because the near-Gaussian expert weight distribution (kurtosis 3.41) suits the K-quant block structure better.
+
+Five C-level modifications to the quantization algorithms themselves, including error feedback, enhanced scale search, super-block refinement and Gaussian-density weighting, all showed zero improvement. Stock llama.cpp quantization is already good. The gains here come entirely from deciding where to spend bits.
+
+## The I-variants and their calibration set
+
+Standard imatrix calibration uses Wikipedia text, which is also what wikitext perplexity measures, so the calibration and the benchmark agree with each other by construction. The I-variants calibrate on a diverse set spanning chat, code, reasoning and tool-calling, with no Wikipedia in it.
+
+That trade shows up clearly. I-Compact drops perplexity from 6.783 to 6.669, cuts KL max from 7.56 to 5.50, and lifts MMLU from 40.9% to 41.7%. At the Quality tier, I-Quality gives up 0.025 perplexity against Quality and takes the highest HellaSwag score of anything tested (83.5%), the best TruthfulQA (38.4%), and a lower KL divergence. If your workload is chat, code or agents rather than encyclopedic prose, take the I variant.
+
+## Where it costs you
+
+The Compact and Mini tiers are real compression, and they are not free.
+
+Compact at 16.1 GB moves perplexity from 6.537 to 6.783, a 3.8% increase, and its KL mean rises tenfold against Q8_0, from 0.0046 to 0.0469. Mini at 12.2 GB goes to 7.088, an 8.4% increase, with a KL mean of 0.0870 and HellaSwag down 1.5 points to 81.0%. Those are the numbers to weigh against the fact that the model now runs at all on a 16 GB card.
+
+The accuracy benchmarks in that table run at 400 tasks, so a single point of difference sits inside the noise. MMLU in particular stays between 39.6% and 41.7% across every build including F16, which says more about the resolution of a 400-task evaluation than about the quantizations. Perplexity and KL divergence are the metrics that separate these builds cleanly, and KL max is the one that shows worst-case outlier behaviour rather than the average.
+
+Running the accuracy evaluations at all required fixing llama.cpp's hybrid memory path for recurrent architectures, since Qwen3.5-35B-A3B uses both attention and SSM blocks and the evaluations crashed before the fix. That went upstream as [llama.cpp #21224](https://github.com/ggml-org/llama.cpp/pull/21224).
+
+## Running one
+
+The GGUFs are published in the [APEX collection on Hugging Face](https://huggingface.co/collections/mudler/apex-quants) and open in any current llama.cpp build. With LocalAI:
+
+```bash
+local-ai run mudler/Qwen3.5-35B-A3B-APEX-GGUF@Qwen3.5-35B-A3B-APEX-Balanced.gguf
+```
+
+To quantize your own MoE model, the recipes are shell scripts over stock llama.cpp:
+
+```bash
+git clone https://github.com/mudler/apex-quant.git
+cd apex-quant
+./scripts/quantize.sh --i-quality model-f16.gguf model-apex-i-quality.gguf
+```
+
+The default configuration assumes 40 transformer layers, as in Qwen3.5-35B-A3B. Set `NUM_LAYERS` for a different depth and the edge and middle boundaries move with it. The full methodology, every experiment and all the plots are in the [technical report](https://github.com/localai-org/apex-quant).

website/content/blog/localai-since-march-2023.md

@@ -0,0 +1,79 @@
+---
+title: "LocalAI, from March 2023 to now"
+date: 2026-07-29
+author: "Ettore Di Giacinto"
+category: "History"
+tags: ["history", "architecture", "releases", "community"]
+summary: "Three years, 133 releases and 224 contributors later. The four changes that mattered most were making the core small, adding agents, making it a cluster, and giving it eyes and ears."
+extracss: ["blog.css"]
+---
+
+I created the LocalAI repository on 18 March 2023. It was an OpenAI-compatible HTTP API in front of llama.cpp, and that was the whole idea: change one URL in code you already wrote, and the answers start coming from your own laptop instead of somebody's datacenter.
+
+Three years on, 224 people have put code into it, across 133 releases. It has picked up 48,042 stars along the way, which still surprises me. It now runs 73 different backends, you can install any of 1,585 models from the gallery without writing a line of Python, and eighteen of the engines doing the actual work are C or C++ ports we sat down and wrote ourselves.
+
+None of those numbers are rounded up. You can read every one of them off the repository or the GitHub API right now, and the ones further down this post come from the release notes.
+
+{{< starchart >}}
+
+The curve is not the point, but it is a useful map. The four marks on it are the four decisions below, and you can see each of them in the slope afterwards.
+
+What follows is how it got here. Not the feature list, which you can read in the releases, but the four decisions that changed the shape of the thing.
+
+## 2023 to 2024: an API in front of llama.cpp
+
+One rule has not moved since the first commit: if a feature only works on a GPU, it does not ship as the main path. Every modality gets a CPU path, and that path is tested. Most people do not have a spare A100 sitting around, and the ones who do still want to develop on the train.
+
+The first two years were spent widening what sat behind the API while keeping that constraint: whisper.cpp for transcription, stable-diffusion for images, embeddings, then reranking, then constrained grammars and the function-calling surface. The API compatibility list grew alongside it, and later picked up the Anthropic and ElevenLabs shapes as well as OpenAI's, so that the same server answers whichever client somebody already has.
+
+The bill for all that breadth came due in the binary. Everything was compiled in, so you downloaded CUDA kernels whether or not you owned a GPU, a Python runtime whether or not you wanted one, and image models when all you asked for was chat. Building from source meant building the lot. It got embarrassing.
+
+## July 2025: the core gets small
+
+Every backend moved out of the main binary in [v3.2.0](https://github.com/mudler/LocalAI/releases/tag/v3.2.0). Backends became separate OCI images, pulled on demand the first time a model asks for one, with a small core that speaks the API and manages processes.
+
+You install one thing and it stays small. Ask for a GGUF model and llama-cpp arrives. Ask for transcription and whisper or parakeet arrives. Nothing else is fetched, and a machine that only ever serves one model never downloads the other sixty-nine backends.
+
+That one change is what made everything after it possible. Adding a backend stopped meaning adding weight to everybody's install, so "should we support this engine" stopped being an argument about download size and went back to being an argument about whether the engine is any good. It is also the reason we can afford to maintain eighteen engines of our own, which comes later.
+
+## March 2026: agents, and a new interface
+
+[LocalAI 4.0.0](https://github.com/mudler/LocalAI/releases/tag/v4.0.0) added native agentic orchestration with the [Agenthub](https://agenthub.localai.io) community hub, so agents with tool use, RAG, skills and streaming run inside the same server rather than as a separate stack you wire up yourself.
+
+The web interface was rewritten in React at the same time, with a Canvas mode, MCP Apps and client-side tools with tool streaming ([#8947](https://github.com/mudler/LocalAI/pull/8947)), and WebRTC realtime audio ([#8790](https://github.com/mudler/LocalAI/pull/8790)). MLX gained a distributed mode ([#8801](https://github.com/mudler/LocalAI/pull/8801)).
+
+The realtime audio path is the piece that changed what people built. Speech in, tool calls in the middle, speech out, over WebRTC, fast enough that it feels like a conversation rather than a walkie-talkie. It had landed as the Realtime API in February 2026 ([#6245](https://github.com/mudler/LocalAI/pull/6245)), and the interface rewrite finally gave it a face.
+
+## April 2026: it becomes a cluster
+
+[LocalAI 4.1.0](https://github.com/mudler/LocalAI/releases/tag/v4.1.0) added distributed cluster mode. You start a worker on another box, it reports its hardware and the backends it can run, and it joins the pool. Requests are placed by a scheduler that knows real free VRAM rather than a static guess, and the pool autoscales.
+
+The same release turned LocalAI into something you can put more than one person on: OIDC and API keys, per-user quotas with predictive analytics, in-UI fine-tuning with TRL that exports straight to GGUF, an on-the-fly quantization backend, and a visual pipeline editor.
+
+[4.3.0](https://github.com/mudler/LocalAI/releases/tag/v4.3.0) in May followed with per-request replica routing, per-API-key and per-user usage attribution ([#9920](https://github.com/mudler/LocalAI/pull/9920)), keyless cosign signing of the backend OCI images ([#9823](https://github.com/mudler/LocalAI/pull/9823)), and llama.cpp prompt caching on by default ([#9925](https://github.com/mudler/LocalAI/pull/9925)), which collapses repeated system prompts from minutes to seconds. In June, prefix-cache-aware routing ([#10071](https://github.com/mudler/LocalAI/pull/10071)) started sending a request to the replica that already holds the matching prefix cache rather than to whichever node was least busy.
+
+Distributed mode is also where most of the painful bugs have lived since. In the 4.8.0 cycle alone we fixed a reaper that cheerfully deleted rows for backends that were alive and busy, frontend stubs that outlived the replicas behind them, and `in_flight` counters that leaked and then pinned VRAM against eviction. Running across machines finds failure modes a single process will never show you, and it finds them in production.
+
+## May and June 2026: it sees and hears
+
+[LocalAI 4.2.0](https://github.com/mudler/LocalAI/releases/tag/v4.2.0) added voice recognition ([#9500](https://github.com/mudler/LocalAI/pull/9500)), face recognition with anti-spoofing liveness ([#9480](https://github.com/mudler/LocalAI/pull/9480)) and speaker diarization, alongside video generation ([#9420](https://github.com/mudler/LocalAI/pull/9420)), a drop-in Ollama API ([#9284](https://github.com/mudler/LocalAI/pull/9284)) and eleven new backends.
+
+A chat model only knows what somebody typed at it. Recognition widens that: who is in the room, who is talking, whether the face at the camera is a live person or a photo somebody is holding up. All of it runs on the same machine as the model, which for biometrics is the only arrangement most people can honestly deploy at all.
+
+In June those two capabilities stopped being Python. [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) and [face-detect.cpp](https://github.com/mudler/face-detect.cpp) replaced the Python `speaker-recognition` and `insightface` backends with from-scratch C++ and ggml engines, self-contained GGUF weights, no onnxruntime at inference, and bit-exact parity with the references they replaced ([#10441](https://github.com/mudler/LocalAI/pull/10441)).
+
+## Mid 2026: eighteen engines of our own
+
+The README now has a table called "Backends built by us". It lists eighteen native C and C++ engines, plus [apex-quant](https://github.com/localai-org/apex-quant), our quantization recipe for mixture-of-experts models.
+
+Each one exists for one of three reasons. Some replace a Python dependency that was simply too heavy to ship, like insightface, speaker-recognition, or vLLM itself. Some port a model nobody had written in C++ yet: Depth Anything 3, LocateAnything, CED audio tagging, TRELLIS.2. And some fill a hole in what a local assistant can do at all, like acoustic echo cancellation, which is the unglamorous thing that stops a voice loop from sitting there transcribing its own speaker.
+
+The recipe is the same every time. A model shows up as PyTorch or ONNX plus a Python package. We port the graph to ggml, convert the weights into one GGUF file, and gate every component against reference tensors dumped from the original. Only after it is correct do we let ourselves look at the clock. LocalAI then loads the shared library through purego over a flat C ABI, so there is no Python process anywhere on the serving path.
+
+The most recent one is [vllm.cpp](https://github.com/mudler/vllm.cpp), a C++20 port of vLLM's V1 serving architecture with paged KV cache, continuous batching and prefix caching, which shipped as the `vllm-cpp` backend in 4.8.0.
+
+## Where it stands
+
+Still MIT, still a community project. 224 people have put code in, and the README is kept translated into eight languages because the people using this are not all in one place. The [contributors graph](https://github.com/mudler/LocalAI/graphs/contributors) is the honest picture of who actually built this, and it is not me.
+
+If you want to add something, backends and gallery entries are the two places a first contribution lands cleanly. There is a step-by-step checklist for a new backend in `.agents/adding-backends.md`, and a gallery entry is just a YAML block. Come say hello in [Discord](https://discord.gg/uJAeKSAGDy) if you get stuck.

website/content/blog/parakeet-cpp-asr-on-cpu.md

@@ -0,0 +1,104 @@
+---
+title: "parakeet.cpp: NeMo transcripts, byte for byte, without the Python"
+date: 2026-06-05
+author: "Ettore Di Giacinto"
+category: "Benchmarks"
+tags: ["parakeet.cpp", "ASR", "ggml", "streaming", "benchmarks"]
+summary: "Same transcript as NVIDIA NeMo, character for character, at a median 1.40x on CPU and about 27x the speed of whisper.cpp. One binary, one GGUF file, no Python at inference."
+extracss: ["blog.css"]
+---
+
+You can drop a single binary and a GGUF file onto a machine with no GPU and get NVIDIA NeMo Parakeet transcription out of it, at a median 1.40x NeMo's own PyTorch CPU speed, with a transcript that matches NeMo character for character. That is [parakeet.cpp](https://github.com/mudler/parakeet.cpp), a C++17 port of the Parakeet speech-recognition family built on ggml.
+
+Accuracy came first and speed came second, in that order, because a faster transcriber that disagrees with the reference is a different model, not a port.
+
+## WER 0 against NeMo
+
+Every published checkpoint is validated at WER 0 against NeMo. Across the LibriSpeech test-clean set the mean f32 agreement WER, meaning the word error rate between our transcript and NeMo's on the same audio, is 0.0155%. On seven of the ten models it is exactly 0.0000%, which is a byte-identical transcript.
+
+That number is what makes the speed comparison meaningful. Both engines did the same work and produced the same output, so the only difference left is how long they took.
+
+## CPU, against NeMo's own runtime
+
+Measured on a 20-core x86 host with 8 threads for both engines, LibriSpeech test-clean, batch size 1 on both sides. NeMo 2.7.3 on PyTorch CPU is the reference. RTFx is audio seconds divided by processing seconds, so higher is faster.
+
+<div class="tw">
+<table>
+<thead><tr><th>Model</th><th>RTFx NeMo</th><th>RTFx f32</th><th>Speedup f32</th><th>Speedup q8_0</th><th>Agreement WER</th><th>RSS NeMo</th><th>RSS ours</th></tr></thead>
+<tbody>
+<tr><td>ctc-0.6b</td><td>30.7</td><td>41.7</td><td>1.36x</td><td>1.61x</td><td>0.0000%</td><td>5447 MB</td><td>2457 MB</td></tr>
+<tr><td>ctc-1.1b</td><td>17.8</td><td>26.1</td><td>1.46x</td><td>1.63x</td><td>0.0222%</td><td>8914 MB</td><td>4201 MB</td></tr>
+<tr><td>rnnt-0.6b</td><td>24.3</td><td>34.1</td><td>1.40x</td><td>1.52x</td><td>0.0000%</td><td>5516 MB</td><td>2487 MB</td></tr>
+<tr><td>rnnt-1.1b</td><td>15.3</td><td>21.4</td><td>1.40x</td><td>1.57x</td><td>0.0000%</td><td>8982 MB</td><td>4231 MB</td></tr>
+<tr><td>tdt-0.6b-v2</td><td>22.4</td><td>32.4</td><td>1.45x</td><td>1.55x</td><td>0.0000%</td><td>5499 MB</td><td>2545 MB</td></tr>
+<tr><td>tdt-1.1b</td><td>14.6</td><td>22.6</td><td>1.54x</td><td>1.74x</td><td>0.0000%</td><td>8956 MB</td><td>4231 MB</td></tr>
+<tr><td>tdt_ctc-1.1b</td><td>13.3</td><td>22.5</td><td><b>1.69x</b></td><td><b>1.89x</b></td><td>0.0985%</td><td>8909 MB</td><td>4236 MB</td></tr>
+<tr><td>tdt_ctc-110m</td><td>72.9</td><td>81.1</td><td>1.11x</td><td>1.26x</td><td>0.0196%</td><td>1650 MB</td><td>563 MB</td></tr>
+<tr><td>rt-eou-120m-v1</td><td>61.8</td><td>70.6</td><td>1.14x</td><td>1.23x</td><td>0.0000%</td><td>1714 MB</td><td>621 MB</td></tr>
+</tbody>
+</table>
+</div>
+
+<figure>
+  <img src="/img/parakeet-speedup.jpg" alt="Bar chart of parakeet.cpp CPU speedup over NeMo, per model and per GGUF dtype" loading="lazy">
+  <figcaption>CPU speedup over NeMo per model and per dtype, from the benchmark suite that runs in CI. Values above 1.0 mean parakeet.cpp finished first on the same audio with the same transcript.</figcaption>
+</figure>
+
+The range across all ten models is 1.11x to 1.69x at f32, with a median of 1.40x. Quantizing to q8_0 shrinks the file to 37% of f32 and pushes the best case to 1.86x, staying near-lossless. f16 is 57% of the size and reaches 1.70x. K-quants below that keep shrinking the model at a small and monotonic accuracy cost, which the per-model tables in the repository lay out.
+
+Peak resident memory is roughly half NeMo's on every model, and lower again once quantized. The 110M model runs in 563 MB against NeMo's 1650 MB, which is the difference between fitting on a small edge box and not.
+
+Against whisper.cpp turbo on the same clip and at the same accuracy (1.6% WER on that audio), parakeet.cpp is about 27x faster on CPU and about 12x faster on GPU. That gap is mostly architectural: Whisper is an encoder-decoder that processes fixed 30 second windows, and Parakeet's FastConformer plus transducer decodes only the frames it has.
+
+## Where the CPU speed came from
+
+The decisive win was on the decode side. A transducer decodes autoregressively, and profiling showed the prediction-network LSTM taking about 97% of RNN-T decode time while producing the same output over and over: on a non-emitting frame the prediction network's input has not changed, so its forward pass is redundant. Caching that forward across non-emitting frames removed most of the decode cost.
+
+The encoder side is a set of smaller wins with no single hero: a persistent ggml backend with `gallocr`, zero-copy weights straight out of the GGUF mapping, one fused graph rather than per-layer graph building, and tinyBLAS through `GGML_LLAMAFILE`.
+
+## On the GPU
+
+On an NVIDIA GB10 (Grace-Blackwell), parakeet.cpp wins on all ten models, with a median of 1.25x and up to 4.3x on the large TDT and hybrid models. The reference here is NeMo-GPU inside the `nvcr.io/nvidia/nemo` container, because NeMo cannot run on that host's torch and CUDA stack directly.
+
+The 4.3x cases have a specific cause. NeMo's TDT greedy decode is not CUDA-graph accelerated and falls back to a per-step Python loop, while ours is a lean C++ loop. Where NeMo's decode is CUDA-graph accelerated, as it is for RNN-T, the gap narrows to about 1.16x at f32 and 1.30x at q8_0. On the pure-encoder CTC models the margin is around 1.2x, because ggml's generic CUDA conv and attention kernels still trail NVIDIA's tuned cuDNN. That is the main piece of GPU headroom left in the project and we say so in the README rather than averaging it away.
+
+Batching several clips through the decoder together reaches about 10x to 12x at batch size 16 on the GB10, and about 3x to 5x on CPU. It applies to transducer models only, since CTC has no autoregressive decode to batch, and the batched path is bit-identical to running the clips one at a time.
+
+On Apple M4 through ggml's Metal backend, the larger models run about 3x to 5x faster than the same models on that machine's CPU.
+
+## Cache-aware streaming, and what end-of-utterance detection buys you
+
+Offline transcription hands you a file and waits. A voice assistant cannot do that, so `parakeet_realtime_eou_120m-v1` runs a cache-aware streaming path instead: you feed it 16 kHz mono PCM as it arrives and it returns newly finalized text as it becomes stable.
+
+Cache-aware means the cost per chunk stays flat. Each chunk's forward pass carries per-layer convolution and attention caches plus the transducer decoder state forward, so nothing before the current chunk is recomputed. Without that, every chunk would re-run the encoder over the whole session so far, and the per-chunk cost would grow with the length of the conversation until the loop fell behind. The implementation covers layer norm with causal convolution, causal subsampling, and chunked-limited attention, and its transcript matches NeMo's own cache-aware streaming byte for byte.
+
+End-of-utterance detection is the part that changes how an assistant feels. The model emits `<EOU>` when the speaker has finished a turn and `<EOB>` for a backchannel, as events alongside the text. A voice loop can start generating a reply the moment `<EOU>` arrives rather than waiting out a fixed silence timer, which is where most of the perceived lag in a spoken assistant comes from. The alternative, a VAD with a 700 ms hangover, either cuts people off mid-sentence or makes the assistant feel slow, and it cannot tell "mm-hm" from the end of a thought. `finalize` flushes the tail at end of stream without fabricating an `<EOU>` that NeMo would not have emitted.
+
+The streaming path measures at RTFx 3.80 on a 7.43 second clip. That sits well below the offline number by design, because streaming runs many small chunked passes rather than one large one, and it is still several times faster than real time on a CPU.
+
+There is also a multilingual streaming model, `nemotron-3.5-asr-streaming-0.6b`, covering 40 or more locales with a per-language prompt. On CPU it runs at 2.40x NeMo at f32 and 2.52x at q8_0, with agreement WER 0.0000% in both cases, offline and streaming.
+
+## Long audio without the memory cliff
+
+The FastConformer encoder uses global relative-position self-attention, which is O(T squared) in time and in memory. A 16.6 minute file subsamples to roughly 12,000 encoder frames, and the score and mask tensors alone reach tens of gigabytes, which is enough to take down a node.
+
+parakeet.cpp ports NeMo's `rel_pos_local_attn`, a banded attention where each query attends only to keys within a window, making attention O(T times W). It turns on automatically past 8192 encoder frames, and `PARAKEET_ATT_CONTEXT` forces a specific window.
+
+<div class="tw">
+<table>
+<thead><tr><th>Attention</th><th>Window</th><th>Wall clock</th><th>RTFx</th><th>Peak RSS</th></tr></thead>
+<tbody>
+<tr><td>full (global)</td><td>-</td><td>148.3 s</td><td>6.7x</td><td>54.0 GB</td></tr>
+<tr><td>banded</td><td>W=32</td><td>39.5 s</td><td>25.2x</td><td>8.9 GB</td></tr>
+<tr><td><b>banded</b></td><td><b>W=128</b></td><td><b>36.9 s</b></td><td><b>27.0x</b></td><td><b>9.4 GB</b></td></tr>
+</tbody>
+</table>
+</div>
+
+At NeMo's full W=128 window that is about 4x faster and about 5.7x less peak memory than the global path. The band is built with a chunk-matmul construction, overlapping key and value chunks feeding one batched GEMM plus a diagonal skew view, so the graph node count does not depend on the window. The wide window costs the same as the narrow one. Short clips stay on the global path and remain byte-identical to before.
+
+## Using it
+
+parakeet.cpp ships prebuilt `parakeet-cli` bundles for Linux x64 (CPU, Vulkan, CUDA), Linux arm64, macOS arm64 with Metal, macOS x64 and Windows x64. In LocalAI it is the `parakeet-cpp` backend, which dlopens `libparakeet.so` through purego and calls the C ABI directly, so there is no Python process in the serving path and transcription comes back on the standard OpenAI-compatible endpoint.
+
+The full benchmark suite, methodology and per-model plots are in [benchmarks/BENCHMARK.md](https://github.com/mudler/parakeet.cpp/blob/main/benchmarks/BENCHMARK.md), and the parity matrix per checkpoint is in `docs/parity.md`.

website/content/blog/what-landed-in-localai-4-8.md

@@ -0,0 +1,125 @@
+---
+title: "What landed in LocalAI 4.8"
+date: 2026-07-28
+author: "Ettore Di Giacinto"
+category: "Release"
+tags: ["release", "vllm.cpp", "gallery", "distributed", "performance"]
+summary: "The web interface got 3.48x lighter, gallery entries now install the build your hardware can actually run, and there is a new inference engine in the box. 214 pull requests in thirteen days."
+extracss: ["blog.css"]
+---
+
+LocalAI 4.8.0 is out. It took thirteen days and 214 merged pull requests, and the changes you will notice first are the boring ones: the web interface loads faster, model installs stop asking you to pick a quantization, and a cluster no longer reports models as loaded when they are gone.
+
+The full notes list everything. This post covers the parts that change what you do day to day, with the pull request numbers so you can read the diffs.
+
+## The web interface got 3.48x lighter
+
+Open the UI over a slow link and you now wait about a third as long. Three separate HTTP problems were fixed together in [#11056](https://github.com/mudler/LocalAI/pull/11056), all measured on a live deployment.
+
+The server was sending no `Content-Encoding` at all, whatever the client asked for. There is now gzip middleware, on by default, with `--disable-http-compression` and `--http-compression-min-length` (default 1024) if you want to change it. Streaming responses are skipped explicitly, because buffering an SSE stream behind a gzip writer defeats incremental flushing and looks to the client like a hung request. Completion, realtime, speech, transcription, agent-job and log-tail paths are all on that skip list, as are already-compressed formats, which gzip made marginally larger.
+
+Vite content-hashes the bundle filenames, so an `/assets/` URL can never change content, yet the assets shipped with no `Cache-Control`, `ETag` or `Last-Modified`. They now carry `public, max-age=31536000, immutable`, and `index.html` is explicitly `no-cache` so a deploy is still picked up.
+
+The third one was `/api/traces` returning a 21 MB unpaginated blob that the UI polled every five seconds. Both trace endpoints now take `limit` (default 50, max 1000, `0` for all), `offset` and `full`, and summarize by default, dropping bodies and headers but keeping the byte counters so the UI can still say what it dropped. Every trace carries a process-lifetime `id`, and `GET /api/traces/{id}` serves the full record when you expand a row.
+
+<div class="tw">
+<table>
+<thead><tr><th>Measurement</th><th>Before</th><th>After</th><th>Change</th></tr></thead>
+<tbody>
+<tr><td>React JS + CSS over the wire</td><td>2,815,513 B</td><td>807,918 B</td><td><b>3.48x smaller</b></td></tr>
+<tr><td>All embedded assets, fonts included</td><td>3,953,917 B</td><td>1,559,787 B</td><td>2.53x smaller</td></tr>
+<tr><td>Repeat navigation asset transfer</td><td>full re-download</td><td>0 bytes</td><td>eliminated</td></tr>
+<tr><td><code>/api/backend-traces</code> poll payload</td><td>21,131,097 B</td><td>7,201 B</td><td><b>~2900x smaller</b></td></tr>
+</tbody>
+</table>
+</div>
+
+## One gallery entry, several builds
+
+Installing a model no longer means reading a list of quantizations and guessing which one your card will hold. A gallery entry can now declare `variants:`, a list of references to other entries that are alternative builds of the same weights:
+
+```yaml
+- name: nanbeige4.1-3b-q4
+  url: github:mudler/LocalAI/gallery/nanbeige4.1.yaml@master
+  overrides: {parameters: {model: nanbeige4.1-3b-q4_k_m.gguf}}
+  files: [...]
+  variants:
+    - model: nanbeige4.1-3b-q8
+```
+
+At install time LocalAI drops the variants this host cannot run, which it derives from the backend name rather than from hardware conditions an author would have to write by hand, so MLX disappears on Linux and CUDA disappears on a Mac. It then drops the ones that will not fit, using VRAM on GPU hosts and cgroup-aware system RAM on CPU hosts so a container sees its own limit rather than the machine's. Of what is left it picks the largest, on the assumption that a bigger footprint is a better build of the same weights. The entry's own build always competes and is never filtered out, so selection ends with something installable.
+
+Sizes come from the existing `pkg/vram` estimator, which reads the remote GGUF header, falls back to an HTTP `HEAD`, then the declared `size:`, then the Hugging Face repo listing. Nothing is downloaded to make the decision, and a failed probe never fails an install.
+
+Every surface can override the choice: `variant` on `POST /models/apply`, `local-ai models install --variant`, the `install_model` MCP tool, and a split-button in the models table. An explicit choice is honored even when it does not fit, with a warning, because that is a deliberate operator decision. Older clients read the same live `gallery/index.yaml`, ignore the key they do not understand, and install exactly as before.
+
+One gap worth knowing about: in distributed mode `InstallModel` resolves against the frontend rather than the worker that will serve the model, so a cluster with a small frontend and large workers selects conservatively. PRs [#10943](https://github.com/mudler/LocalAI/pull/10943), [#10983](https://github.com/mudler/LocalAI/pull/10983), [#10992](https://github.com/mudler/LocalAI/pull/10992), [#11027](https://github.com/mudler/LocalAI/pull/11027) and [#11139](https://github.com/mudler/LocalAI/pull/11139).
+
+## A new engine: vllm.cpp
+
+[vllm.cpp](https://github.com/mudler/vllm.cpp) is a from-scratch C++20 port of vLLM, written and maintained by the LocalAI team under Apache-2.0, and it ships here as the `vllm-cpp` backend ([#11100](https://github.com/mudler/LocalAI/pull/11100)). It mirrors vLLM's V1 architecture, so paged KV cache, continuous batching, prefix caching, scheduler and sampler, on a portable tensor runtime with no Python, no PyTorch and no ggml at inference. It loads Hugging Face safetensors and GGUF, enforces structured output inside the engine (JSON schema, regex, choice, GBNF), and builds for CPU amd64 and arm64, CUDA 12 and 13 including Blackwell, L4T for GB10, Vulkan and Darwin Metal.
+
+Tool calling is at llama.cpp parity by construction, because chat deliberately reuses the same autoparser path: full minja chat templates, `tool_choice: auto` lowered to a lazy structural-tag decode constraint, 30 tool dialects, 7 reasoning parsers, and streamed `ChatDelta` and `ToolCallDelta`.
+
+Configuration is a normal backend install:
+
+```yaml
+name: qwen3-vllm
+backend: vllm-cpp
+context_size: 8192
+parameters:
+  model: Qwen3-4B      # a safetensors directory or a .gguf file
+options:
+- max_num_seqs:16      # also: block_size:<n>, num_blocks:<n>
+```
+
+The CPU path is verified end to end against `Qwen3.5-2B-UD-Q8_K_XL.gguf` with the full Ginkgo suite, covering blocking and streaming byte-parity, greedy determinism, stop words, GBNF-constrained generation, concurrent streams, reasoning split and both `required` and `auto` tool calls. The maturity statement from the release notes is worth repeating in full:
+
+> The GPU images build and ship, but their runtime behavior has not been through the same e2e gate yet. This is a first release of a young engine: no throughput comparison against upstream vLLM is claimed here, and `llama-cpp` remains the default recommendation for general use. Try it, and please report what breaks.
+
+## VRAM budgets, per node
+
+You can now cap how much of a card LocalAI is allowed to use, as a percentage or an absolute amount ([#10833](https://github.com/mudler/LocalAI/pull/10833)):
+
+```
+LOCALAI_VRAM_BUDGET=80%
+LOCALAI_VRAM_BUDGET=12GB
+```
+
+Everywhere LocalAI reads VRAM to make an allocation decision it now uses `min(detected, budget)`. Percentages above 100 are rejected and absolute values above physical are clamped, so the ceiling can only ever lower usable VRAM. Standalone it is a hard per-process cap that hardware defaults, context auto-fit, GGUF warnings and the watchdog all inherit. Distributed it is a placement ceiling: the worker reports raw VRAM plus its budget string, the node registry resolves it to a byte ceiling on registration and heartbeat, and the SQL scheduler needed no query change. Admin overrides through `PUT` and `DELETE /api/nodes/:id/vram-budget` survive worker restarts, and the same thing is available as the `set_node_vram_budget` MCP tool. Unset means all detected VRAM, so existing deployments do not change.
+
+## Three new backends for speech and small quants
+
+`magpie-tts-cpp` wraps [magpie-tts.cpp](https://github.com/mudler/magpie-tts.cpp), a C++17 and ggml port of NVIDIA Magpie TTS Multilingual 357M with the NanoCodec vocoder embedded. Five voices, nine or more languages, 22.05 kHz mono, out of one self-contained GGUF. The upstream engine is parity-gated against NeMo per component, with a teacher-forced replay maximum absolute difference of 3.6e-5.
+
+`moss-tts-cpp` wraps [moss-tts.cpp](https://github.com/mudler/moss-tts.cpp) and serves MOSS-TTS-Local v1.5 at 48 kHz stereo, with optional reference-audio voice cloning. Images cover CPU, CUDA 12 and 13, Intel SYCL, Vulkan, ROCm, L4T and Darwin Metal. Both landed in [#11115](https://github.com/mudler/LocalAI/pull/11115), [#10860](https://github.com/mudler/LocalAI/pull/10860) and [#10877](https://github.com/mudler/LocalAI/pull/10877).
+
+The `bonsai` backend serves the 1-bit (Q1_0) and ternary (Q2_0) Bonsai quantizations of Qwen3 8B and Qwen3.6-27B, from about 1.15 GB. Stock llama.cpp has no kernels for those formats, so the backend builds against the PrismML fork through a wrapper Makefile that swaps only `LLAMA_REPO` and `LLAMA_VERSION`, reusing the same `grpc-server.cpp` with zero skew patches. Eight gallery entries ship with it. If Q1_0 and Q2_0 reach mainline llama.cpp, this backend retires into a routine version bump ([#10834](https://github.com/mudler/LocalAI/pull/10834), [#10866](https://github.com/mudler/LocalAI/pull/10866)).
+
+## Distributed mode stops reaping live backends
+
+A model that showed as loaded on the home page but appeared on no node in the cluster turned out to be four separate bugs, all fixed in this cycle.
+
+The reaper was deleting `node_models` rows for backends that were alive and working. `probeLoadedModels` reaped a row after a single failed one second gRPC health check, and a backend that is merely busy cannot answer one, because a single-threaded Python backend blocks for minutes inside a request. The worker spawned the backend and holds the process handle, so its answer is not blocked by whatever the backend is doing. A new `models.running` request-reply subject asks the worker directly, and the reconciler diffs the worker's process keys against the registry rows before any port probe runs. A worker that does not answer is skipped rather than assumed empty, so a NATS blip cannot delete a node's rows. Where the port probe still runs, it no longer conflates failure modes: `DeadlineExceeded` means busy, `Unavailable` means gone, and only the second counts toward a threshold that now needs three consecutive misses.
+
+Every routed model also left an in-process stub in the frontend's `ModelLoader`, and removal paths deleted only the database row, so the stub outlived the replica and the model was reported as loaded forever. The replica-removed hook became a list, and a new invalidator drops the local stub once no healthy replica remains anywhere in the cluster.
+
+Alongside those, `in_flight` counters no longer leak high and pin a replica's VRAM against eviction, model-load deadlines scale with checkpoint size and with progress rather than wall-clock, staging verification counts as progress rather than as a stall, backend discovery stopped hiding worker-installed and GPU-only backends behind the controller's own filesystem, and the scheduler will not place a model on a node that cannot store it. The full list is in [#11142](https://github.com/mudler/LocalAI/pull/11142) and the eighteen PRs around it.
+
+## A security fix you should read
+
+`POST /api/fine-tuning/jobs` accepted `reward_functions[].code`, an inline Python body that ran against a hand-rolled builtin allowlist. That allowlist was not a security boundary. Standard CPython introspection reaches the real `os` module from inside it, which is arbitrary code execution on the host, and the execution happened during a smoke test at job start on an endpoint that is unauthenticated by default.
+
+Inline reward code is now refused unless the operator sets `LOCALAI_TRL_ALLOW_INLINE_REWARD=true` on the backend. Builtin reward functions are unaffected and need no configuration. The documentation no longer describes the allowlist as a sandbox ([#11068](https://github.com/mudler/LocalAI/pull/11068)). This release also picks up hono 4.12.25 for CVE-2026-54290 ([#11023](https://github.com/mudler/LocalAI/pull/11023)).
+
+## The rest, briefly
+
+Hugging Face model artifacts are now a managed snapshot flow: immutable snapshot resolution, authenticated downloads with real progress, materialization on gallery install and preload, runtime binding to staged artifacts, and per-file resume of an interrupted download rather than starting over. Python backends reuse the Go download path instead of fetching on their own.
+
+The traces panel gained a sortable User column, plus client IP and user agent in the expanded row, taken from echo's `RealIP()` so a trusted proxy is honored.
+
+Documentation got an onboarding overhaul driven by a per-page audit: one model, `qwen3-4b`, now carries through install, the web UI and a curl call; there is a new walkthrough for building your first agent; and a new runtime-errors reference is keyed on the literal error strings users actually see.
+
+Fifteen people contributed to this release, five of them for the first time. The gallery went from 1,221 entries to 1,476.
+
+To upgrade, pull `localai/localai:latest` or re-run the install script. The [full changelog](https://github.com/mudler/LocalAI/compare/v4.7.1...v4.8.0) has the other 180 pull requests.

website/content/blog/why-we-write-our-own-engines.md

@@ -0,0 +1,88 @@
+---
+title: "Why we write our own C and C++ engines"
+date: 2026-07-24
+author: "Ettore Di Giacinto"
+category: "Engineering"
+tags: ["engineering", "ggml", "vllm.cpp", "depth-anything.cpp", "parity"]
+summary: "A 66 MiB binary instead of a 9.1 GiB virtualenv, depth estimation that beats PyTorch on CPU in half the memory, and biometrics that match insightface bit for bit. The method, the measurements, and what it costs us."
+extracss: ["blog.css"]
+---
+
+Most LocalAI backends wrap somebody else's engine, and that is the right default. llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX and the rest are maintained by people who are better at those models than we are, and wrapping them costs a Dockerfile and a gRPC shim.
+
+Eighteen of our backends do not wrap anything. They are C or C++ ports we wrote from scratch, and each one exists because wrapping the upstream engine would have meant shipping something we could not ship: a multi-gigabyte Python install, a non-portable CUDA-only stack, or a model that had no C++ implementation at all. This post is about what those ports buy, measured, and what they cost.
+
+## What you get: one file, and memory you can predict
+
+Deploying a Python inference stack means resolving a dependency tree at install time, on the target machine, against whatever CUDA and glibc it has. Deploying a ggml port means copying a shared library and a GGUF file.
+
+The clearest measurement of that difference is [vllm.cpp](https://github.com/mudler/vllm.cpp), our C++20 port of vLLM's V1 serving architecture. Installing vLLM produces a 9.1 GiB virtualenv. Installing vllm.cpp produces a 66 MiB binary. The engine implements the same things the Python original does, including paged KV cache, continuous batching, prefix caching, the scheduler and the sampler, with no Python, no PyTorch and no ggml at inference.
+
+The obvious question is what that costs in throughput. On an NVIDIA GB10 running Qwen3.6-27B in NVFP4, greedy, closed loop, against vLLM in its production graphed configuration rather than `--enforce-eager`:
+
+<div class="tw">
+<table>
+<thead><tr><th>Concurrency</th><th>1</th><th>2</th><th>4</th><th>8</th><th>16</th><th>32</th></tr></thead>
+<tbody>
+<tr><td><b>vllm.cpp</b> tok/s</td><td><b>86.05</b></td><td><b>159.68</b></td><td><b>292.34</b></td><td><b>508.77</b></td><td><b>801.76</b></td><td><b>1095.01</b></td></tr>
+<tr><td>vLLM tok/s</td><td>82.32</td><td>158.03</td><td>290.31</td><td>505.46</td><td>789.16</td><td>1076.25</td></tr>
+<tr><td>Ratio</td><td>1.045x</td><td>1.011x</td><td>1.007x</td><td>1.007x</td><td>1.016x</td><td>1.017x</td></tr>
+</tbody>
+</table>
+</div>
+
+We are ahead at all six points, and five of those six are ties. Our run-to-run noise band is 0.5%, and concurrency 2 through 32 land between 0.7% and 1.7%, so the honest reading is that only the single-stream case (4.5%) is clearly outside noise. Output is token-for-token identical to vLLM at every point on that curve. Peak host memory is 24.88 GiB against 28.18 GiB.
+
+A tie against a mature CUDA stack is a good result for a 66 MiB binary, and it means the footprint saving is not paid for in throughput. Against llama.cpp on CPU from the same GGUF file, prefill runs 1.18x faster (223.8 against 177.3 tok/s), decode is a tie inside llama.cpp's own spread, and the tokens are byte-identical to its greedy decode. Against MLX-LM on an Apple M4, prefill time to first token is 1.5% ahead and warm total throughput is 97.6% of MLX-LM, a real 2.4% gap that sits entirely in decode.
+
+## Sometimes the port is simply faster
+
+[depth-anything.cpp](https://github.com/mudler/depth-anything.cpp) is a port of ByteDance's Depth Anything 3, which gives you metric depth in metres from one ordinary photo, plus per-pixel confidence, camera intrinsics and extrinsics, and a back-projected point cloud. On CPU it is faster than PyTorch running the same model.
+
+<div class="tw">
+<table>
+<thead><tr><th>Engine</th><th>Quant</th><th>Model MB</th><th>Load ms</th><th>Infer ms</th><th>Peak RAM MB</th><th>vs PyTorch</th></tr></thead>
+<tbody>
+<tr><td>PyTorch</td><td>f32</td><td>516</td><td>749</td><td>416.9</td><td>1328</td><td>1.00x</td></tr>
+<tr><td><b>C++/ggml</b></td><td>q8_0</td><td>142</td><td><b>40</b></td><td><b>319.4</b></td><td><b>363</b></td><td><b>1.31x</b></td></tr>
+</tbody>
+</table>
+</div>
+
+Same model, 1.31x the speed, 27% of the memory, and a load that finishes in 40 ms instead of 749 ms, on a Ryzen 9 9950X3D at 504x336 with 16 threads. The quantized q4_k build is a 99 MB file and stays near-lossless. Output correlates 1.0 with the reference forward pass, component by component, across 37 parity tests.
+
+The reason it is faster has nothing to do with writing better matmul kernels than PyTorch. Two positional embeddings, the DPT head's UV embedding and the backbone's bicubic position embedding, were being recomputed on every forward pass with single-threaded scalar sin, cos and bicubic loops, even though they depend only on the input geometry and are identical every call. Caching them removed about 95 ms of host-side overhead per forward, which is most of the gap. PyTorch builds the same embeddings with vectorized operations and never paid that cost.
+
+That is the general shape of these wins. The heavy GEMMs are close to a wash, because everyone is calling into the same class of BLAS kernel. The difference sits in host-side work that a Python reference implementation never bothered to optimize, and in not loading an interpreter and a framework to do inference. On GPU the picture flips back to parity: with the ggml CUDA backend and flash attention on a GB10, depth-anything.cpp ties PyTorch's tuned cuDNN at 47.3 ms per forward, and wins only the cold start, loading 1.75x to 2.9x faster.
+
+## Parity is the gate, speed is the follow-up
+
+[face-detect.cpp](https://github.com/mudler/face-detect.cpp) and [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) replaced LocalAI's Python `insightface` and `speaker-recognition` backends. Both are the case where we do not claim a CPU speed win, and both shipped anyway.
+
+face-detect.cpp runs the whole insightface buffalo chain, so SCRFD detection, five-landmark similarity-transform alignment to 112x112, and the ArcFace embedding, out of one self-contained GGUF with no Python and no onnxruntime. Detector boxes and landmarks match insightface to within 1 pixel, and the recognition embedding matches to cosine 1.000000, held at any thread count. On CPU it is slower than onnxruntime: SCRFD detect runs at about 0.83x at one thread and 0.69x at eight, ArcFace embed at about 0.61x and 0.84x. onnxruntime's MLAS convolution kernels sit at the FMA-port peak, and a custom AVX2 Winograd path narrowed the gap without closing it. On GPU, routing the same convolutions through cuDNN takes SCRFD from 14.8 ms to 6.4 ms and lands at torch-cuDNN parity.
+
+voice-detect.cpp is the same story with a memory result attached. A WeSpeaker verification peaks at about 62 MB in our binary against about 334 MB for the CPU-only Python, torch and onnxruntime path, roughly 5.4x lower, with an identical verdict and embedding cosine 1.000000. End to end on CPU the two land within 10 to 15% of each other, trading the lead by model and thread count, and on GPU the conv encoders match the reference.
+
+For a biometric pipeline, matching the reference exactly matters more than being faster than it. An embedding that differs in the fourth decimal place changes verification decisions at a threshold, and every enrolled template in a deployment would have to be recomputed. Parity is what makes the replacement a drop-in rather than a migration.
+
+## The method
+
+Every port follows the same sequence, and the order is the important part.
+
+Convert the weights first, into one GGUF with the tokenizer, the vocabulary and any auxiliary model embedded, so that deploying the model is copying a file.
+
+Port the graph second, and gate it component by component against reference tensors dumped from the original implementation. depth-anything.cpp has 37 ctest cases covering preprocessing, backbone, attention, the DPT head, depth, pose, the ray head, the ray to pose solver and the exporters. parakeet.cpp gates on transcript agreement with NeMo at WER 0. face-detect.cpp gates on box and landmark distance in pixels and embedding cosine. A port that is fast and slightly wrong is worthless, and without a per-component gate you find out it is wrong months later.
+
+Optimize third, with a profiler, and only after parity holds. In parakeet.cpp the decisive win was caching a prediction-network LSTM forward pass that was 97% of transducer decode time and mostly redundant. In depth-anything.cpp it was two cached positional embeddings. Neither was a kernel rewrite, and neither would have been findable without a working baseline to profile.
+
+Expose a flat C ABI last. LocalAI dlopens the shared library through purego and calls that ABI directly, so there is no subprocess, no gRPC hop to a Python server, and no interpreter in the serving path.
+
+## What it costs
+
+Maintenance, mostly. Each engine is a repository with its own CI, its own benchmark suite, its own GGUF conversion script and its own parity baselines, and upstream keeps releasing new checkpoints that need converter work.
+
+GPU kernels are the weak spot. ggml's generic CUDA convolution and attention kernels trail NVIDIA's tuned cuDNN on the conv-heavy models, which is why face-detect.cpp needs an explicit cuDNN path to reach parity, and why parakeet.cpp's GPU margin over NeMo is a median 1.25x while its CPU margin is wider.
+
+Porting also does not scale to everything. llama.cpp, vLLM, whisper.cpp, MLX and diffusers stay wrapped, because those projects are large, fast-moving and already excellent at what they do. We write an engine when a model has no C++ implementation, when the Python dependency is heavier than the model, or when the thing we need does not exist yet. Everything else we install from somebody else.
+
+Every engine listed above keeps its own benchmark suite, its parity gates and its methodology in its own repository, including the runs that did not work. The full list of them is the "Backends built by us" table in the [LocalAI README](https://github.com/mudler/LocalAI#backends-built-by-us).

website/content/engines/_index.md

@@ -0,0 +1,5 @@
+---
+title: "Engines"
+description: "Nineteen native C, C++ and Go engines written by the LocalAI team. No Python at inference, checked against the reference implementation in CI, and small enough to ship as one file."
+extracss: ["engines.css"]
+---

website/data/ecosystem.yaml

@@ -0,0 +1,195 @@
+# Who shows up around LocalAI, and the evidence for each claim.
+#
+# Three lists, three different strengths of evidence. Keep them separate: a
+# person fixing a typo is not their employer adopting the project, and saying
+# so invites a takedown request.
+#
+#   contributors  someone with commits in this repository whose public GitHub
+#                 profile names that employer. Evidence is the commit history
+#                 plus a self-declared field, so the claim is about the person.
+#   integrations  the other project's own repository or documentation
+#                 references LocalAI. Checkable by anyone, needs no permission.
+#   press         somebody else published a piece about LocalAI.
+#
+# Companies that want to say they use LocalAI add themselves to ADOPTERS.md at
+# the repository root. That pull request is the permission, which is why the
+# adopters list is kept apart from this file.
+#
+# Refreshed on 2026-07-31. Contribution counts are from the GitHub API.
+
+contributors:
+  note: "Engineers whose public profile names these employers have commits in the repository."
+  # The landing page prints the names only, not the per-company counts: a "1
+  # commit" stamp next to a large employer reads as weakness even though the
+  # claim is exactly as true. The counts stay here because they are the
+  # provenance for the list, and anyone re-checking it needs them.
+  companies:
+    - { name: "Microsoft",       people: 3, commits: 20 }
+    - { name: "Spectro Cloud",   people: 3, commits: 11 }
+    - { name: "VictoriaMetrics", people: 1, commits: 5 }
+    - { name: "Acronis",         people: 1, commits: 5 }
+    - { name: "BearingPoint",    people: 1, commits: 4 }
+    - { name: "Zendesk",         people: 1, commits: 3 }
+    - { name: "Nextcloud",       people: 1, commits: 2 }
+    - { name: "Scaleway",        people: 1, commits: 2 }
+    - { name: "Kubermatic",      people: 1, commits: 2 }
+    - { name: "Avanade",         people: 1, commits: 2 }
+    - { name: "IBM",             people: 1, commits: 1 }
+    - { name: "Ableton",         people: 1, commits: 1 }
+    - { name: "Atlassian",       people: 1, commits: 1 }
+    - { name: "TikTok",          people: 1, commits: 1 }
+    - { name: "CockroachDB",     people: 1, commits: 1 }
+    - { name: "MediaTek",        people: 1, commits: 1 }
+    - { name: "Samsung SDS",     people: 1, commits: 1 }
+    - { name: "Ant Group",       people: 1, commits: 1 }
+    - { name: "Deloitte",        people: 1, commits: 1 }
+    - { name: "solo.io",         people: 1, commits: 1 }
+    - { name: "Qonto",           people: 1, commits: 1 }
+  academia:
+    - MIT
+    - University of Edinburgh
+    - RMIT
+    - Rochester Institute of Technology
+    - University of Miami
+    - USC
+
+integrations:
+  note: "These projects reference LocalAI in their own repository or documentation."
+  # Admission rule: the project's OWN repository or official docs must name
+  # LocalAI as a provider, backend or integration. A blog post saying so is not
+  # enough, and neither is a generic "any OpenAI-compatible server" line that
+  # never names us. Every entry below was opened and read before being added,
+  # and the quote that justifies it is in the pull request that added it.
+  #
+  # Deliberately excluded after checking: LiteLLM, n8n, Home Assistant core,
+  # BionicGPT, Semantic Kernel, Spring AI, Jan, CrewAI, Haystack, Onyx,
+  # SillyTavern, Portkey and Kong all have zero mentions in their own repos.
+  # PrivateGPT and Langflow name us only in passing comparisons or a tooltip.
+  # Tools4AI has a LocalAI processor whose methods return null.
+  projects:
+    - name: AnythingLLM
+      by: Mintplex Labs
+      url: https://github.com/Mintplex-Labs/anything-llm
+      what: "Runs LocalAI as a model provider for its chat and document workspaces."
+    - name: LangChain
+      url: https://python.langchain.com/docs/integrations/providers/localai/
+      what: "Documents LocalAI as a provider in the Python integrations."
+    - name: LlamaIndex
+      url: https://docs.llamaindex.ai/en/stable/examples/llm/localai/
+      what: "Documents LocalAI as an OpenAI-compatible LLM you can point it at."
+    - name: Open WebUI
+      url: https://docs.openwebui.com/getting-started/quick-start/connect-a-provider/starting-with-openai-compatible
+      what: "Connects to LocalAI as an OpenAI-compatible local server."
+    - name: Dify
+      by: LangGenius
+      url: https://marketplace.dify.ai/plugin/langgenius/localai
+      what: "Ships an official LocalAI plugin for inference and embeddings."
+    - name: LibreChat
+      url: https://www.librechat.ai/docs/configuration/librechat_yaml/object_structure/speech
+      what: "Uses LocalAI as a speech provider for text to speech."
+    - name: RAGFlow
+      by: InfiniFlow
+      url: https://ragflow.io/docs/supported_models
+      what: "Lists LocalAI as a model provider, and ships a driver for it including rerank."
+    - name: Flowise
+      by: FlowiseAI
+      url: https://github.com/FlowiseAI/Flowise
+      what: "Offers LocalAI as a node for chat and embeddings in its visual builder."
+    - name: Continue
+      url: https://docs.continue.dev/customize/model-providers/top-level/openai
+      what: "Lists LocalAI among the OpenAI-compatible providers the editor can target."
+    - name: big-AGI
+      url: https://github.com/enricoros/big-AGI/blob/main/docs/config-local-localai.md
+      what: "Adds LocalAI as a model source, with its own setup page and env vars."
+    - name: K8sGPT
+      by: CNCF Sandbox
+      url: https://docs.k8sgpt.ai/reference/providers/backend/
+      what: "Lists LocalAI as an analysis backend, alongside Bedrock and Azure OpenAI."
+    - name: k8sgpt-operator
+      by: k8sgpt-ai
+      url: https://github.com/k8sgpt-ai/k8sgpt-operator
+      what: "Sets LocalAI as the backend on the K8sGPT custom resource, with no OpenAI secret."
+    - name: Nextcloud
+      url: https://apps.nextcloud.com/apps/integration_openai
+      what: "Points its text, image and speech features at a self-hosted LocalAI."
+    - name: Frigate
+      url: https://github.com/blakeblackshear/frigate/blob/dev/docs/docs/configuration/genai/config.md
+      what: "Names LocalAI as a server for its generative AI features."
+      # Present in the dev branch docs source; not yet on the published site.
+    - name: Kairos
+      url: https://github.com/kairos-io/kairos
+      what: "Ships LocalAI as part of its immutable Linux distribution."
+    - name: AIKit
+      by: Sertac Ozercan
+      url: https://github.com/sozercan/aikit
+      what: "Builds fine-tuned models into images that serve through LocalAI."
+    - name: langchain4j
+      url: https://github.com/langchain4j/langchain4j
+      what: "Ships a LocalAI module, so JVM applications can target it directly."
+    - name: promptfoo
+      url: https://www.promptfoo.dev/docs/providers/localai/
+      what: "Provides a localai: prefix so evals run against chat, completion and embeddings."
+    - name: Mods
+      by: Charm
+      url: https://github.com/charmbracelet/mods
+      what: "Reads a LocalAI endpoint from its config, so the CLI pipes shell output to a local model."
+    - name: TypingMind
+      url: https://docs.typingmind.com/manage-and-connect-ai-models/local-ai
+      what: "Documents pointing a custom model endpoint at a LocalAI server."
+    - name: baibot
+      by: etke.cc
+      url: https://github.com/etkecc/baibot/blob/main/docs/providers.md
+      what: "Has a first-class localai provider for text, speech to text and text to speech."
+    - name: LLM Vision
+      by: Home Assistant
+      url: https://github.com/valentinfrlch/ha-llmvision
+      what: "Lists LocalAI as a provider for analysing camera images and video feeds."
+    - name: VoxInput
+      url: https://github.com/richiejp/VoxInput
+      what: "Sends desktop voice input to LocalAI for transcription and a realtime assistant."
+    - name: Obsidian BMO Chatbot
+      url: https://github.com/longy2k/obsidian-bmo-chatbot
+      what: "Lists LocalAI as a self-hosted endpoint for the note-taking chat plugin."
+    - name: ShellOracle
+      url: https://github.com/djcopley/ShellOracle
+      what: "Ships a LocalAI provider for turning natural language into shell commands."
+    - name: QA-Pilot
+      url: https://github.com/reid41/QA-Pilot
+      what: "Runs its repository chat against a LocalAI base URL."
+
+press:
+  note: "Written about LocalAI, by people who do not work on it."
+  # One card per outlet. SUSE published four posts, and listing them as four
+  # entries made a single vendor look like the whole of the coverage; as one
+  # series entry the repetition becomes the point instead of the problem.
+  #
+  # Every entry here has been opened and checked to confirm it is about this
+  # project. Two earlier "coverage" links were not: a modelslab.com piece that
+  # reviewed Frikallo/parakeet.cpp, an unrelated project of the same name, and
+  # a snailtext.app benchmark of Parakeet through ONNX Runtime that never
+  # mentioned LocalAI at all. Verify before adding.
+  articles:
+    - outlet: Pulumi
+      title: "Deploy low-code LLM apps on AWS with Flowise and LocalAI"
+      author: "Engin Diri"
+      date: "2024-02-26"
+      url: https://www.pulumi.com/blog/low-code-llm-apps-with-local-ai-flowise-and-pulumi/
+      what: "A full EKS deployment, with LocalAI doing inference and no GPU in the cluster."
+    - outlet: Semaphore
+      title: "LocalAI: replacing OpenAI API with open-source"
+      author: "Tomas Fernandez"
+      date: "2023-12-19"
+      url: https://semaphore.io/blog/localai
+      what: "Swapping an existing OpenAI integration over to LocalAI, endpoint by endpoint."
+    - outlet: Spectro Cloud
+      title: "K8sGPT + LocalAI: unlock Kubernetes superpowers for free"
+      author: "Tyler Gillson"
+      date: "2023-04-29"
+      url: https://www.spectrocloud.com/blog/k8sgpt-localai-unlock-kubernetes-superpowers-for-free
+      what: "Cluster diagnosis through K8sGPT, answered by a model running on CPU."
+    - outlet: SUSE
+      title: "A four-part series on running AI locally"
+      author: "Christian Huller"
+      date: "2025-11-24"
+      url: https://www.suse.com/c/running-ai-locally/
+      what: "Install on openSUSE and drive it from VS Code, add your own documents with RAG, put a Confluence space behind a local model, then train a LoRA and serve it."

website/data/engines.yaml

@@ -0,0 +1,281 @@
+# The nineteen native engines the LocalAI team wrote, and the one quantization
+# recipe that feeds them. This file is the single source of truth for the
+# /engines/ page: the layout renders whatever is here, in this order, and adds
+# nothing of its own. Numbers in `highlights` come from each engine's own
+# benchmark suite, so if a README moves, move the number here too.
+#
+# Fields per engine:
+#   name       display name, matching how the repo calls itself
+#   tagline    one sentence, benefit first, what the user gets
+#   category   must match a category id below
+#   language   implementation language
+#   repo       canonical GitHub URL
+#   featured   optional, renders the entry wide with its clip
+#   status     optional badge for anything not generally announced
+#   media      optional clip under /media/
+#   poster     optional still under /img/
+#   clips      optional extra clips, only shown on featured entries
+#   highlights optional list of concrete facts
+
+categories:
+  - id: hearing
+    label: Hearing
+    blurb: Turning sound into something a model can act on, words first and then everything else in the room.
+  - id: voice
+    label: Voice
+    blurb: Speech coming back out, in a voice you chose or one you cloned from a few seconds of audio.
+  - id: identity
+    label: Identity
+    blurb: Working out who is in front of the microphone or the camera, and whether they are really there.
+  - id: vision
+    label: Vision
+    blurb: Finding things in an image and naming them, including things nobody trained a class for.
+  - id: space
+    label: Space
+    blurb: Reading distance, camera pose and shape out of ordinary photos, with no rig and no capture setup.
+  - id: text
+    label: Text
+    blurb: Serving language models, and cleaning what goes into them before it leaves the machine.
+  - id: data
+    label: Data
+    blurb: The storage and quantization work that decides what actually fits on your hardware.
+
+engines:
+  # ---------------------------------------------------------------- hearing
+  - name: parakeet.cpp
+    tagline: Transcribe a meeting on a laptop CPU and be finished before whisper.cpp has cleared the first minute.
+    category: hearing
+    language: C++17
+    repo: https://github.com/mudler/parakeet.cpp
+    featured: true
+    media: /media/parakeet-duel.mp4
+    clips:
+      - src: /media/parakeet-long.mp4
+        caption: Long-form audio, ours against NeMo on the same machine
+    highlights:
+      - About 27x faster than whisper.cpp turbo on CPU, and about 12x on GPU
+      - WER 0 against NVIDIA NeMo on every published checkpoint, so the transcript is identical
+      - Cache-aware streaming with end-of-utterance detection, for live audio
+      - Ten checkpoints, from 110M to 1.1B, and 40 or more locales on the streaming multilingual model
+
+  - name: moss-transcribe.cpp
+    tagline: Get the transcript, the speaker labels and the timestamps out of a single pass, then export straight to srt or json.
+    category: hearing
+    language: C++17
+    repo: https://github.com/localai-org/moss-transcribe.cpp
+    highlights:
+      - 1.58x to 1.78x faster than PyTorch on CPU, on about 1.5x less memory
+      - Byte-identical transcript against the reference, cosine 1.0 component by component
+      - 3.4 GB at f32 down to 511 MB at q4_k, still byte-identical through q5_k
+
+  - name: ced.cpp
+    tagline: Let the model hear a smoke alarm, a dog or breaking glass, not only the words somebody typed.
+    category: hearing
+    language: C++17
+    repo: https://github.com/localai-org/ced.cpp
+    media: /media/ced.mp4
+    highlights:
+      - 527 AudioSet sound classes, multi-label, tagged in about 55 ms
+      - 6 MB on disk at ced-tiny q8_0, 111 MB at ced-base
+      - About 1.25x faster than PyTorch at f32 on half the memory
+      - Works over REST and live over the realtime websocket
+
+  - name: LocalVQE
+    tagline: Keep a voice loop usable in a real room, with the echo, the noise and the reverb removed before the model ever hears it.
+    category: hearing
+    language: C++
+    repo: https://github.com/localai-org/LocalVQE
+    highlights:
+      - Echo cancellation, noise suppression and dereverberation in one pass
+      - Streaming and causal, 16 ms latency, 5x realtime on a desktop CPU
+      - From a 17 KB linear filter to a 19 MB joint model, pick what your CPU can afford
+      - About 21x realtime on a single Raspberry Pi 5 core
+
+  # ------------------------------------------------------------------ voice
+  - name: moss-tts.cpp
+    tagline: Clone a voice from a short reference clip and have it read anything back at 48 kHz stereo.
+    category: voice
+    language: C++17
+    repo: https://github.com/mudler/moss-tts.cpp
+    media: /media/moss.mp4
+    highlights:
+      - About 1.9x faster per frame than PyTorch on CPU, both at fp32
+      - Codec decode matches the reference at 114.9 dB SNR
+      - 48 kHz stereo out of MOSS-TTS-Local v1.5, 12 codebooks
+      - Flat C API as well as a CLI, so it embeds anywhere
+
+  - name: magpie-tts.cpp
+    tagline: Ship multilingual speech from one GGUF that already carries the codec, the tokenizer and the pronunciation dictionaries.
+    category: voice
+    language: C++17
+    repo: https://github.com/mudler/magpie-tts.cpp
+    media: /media/magpie.mp4
+    highlights:
+      - 63x faster than the NeMo reference at f32, 73x at q8_0
+      - 5 named voices, 9 languages plus 3 Arabic variants
+      - 541 MB at q4_k, everything bundled in the single file
+      - Deterministic from a seed, and parity holds to 3.6e-5 on the full decode
+
+  - name: vibevoice.cpp
+    tagline: Read a multi-speaker script in cloned voices, and transcribe long recordings back with speaker labels, from the same binary.
+    category: voice
+    language: C++
+    repo: https://github.com/localai-org/vibevoice.cpp
+    highlights:
+      - Voice cloning from roughly 5 seconds of reference audio
+      - Long-form ASR with diarization on the same engine as the synthesis
+      - 11 GB down to 6.8 GB at Q8_0 with no measurable recall loss
+      - Realtime 0.5B, 1.5B and a 7B ASR model
+
+  - name: voxtral-tts.c
+    tagline: Run a 4B speech model with nothing but a C compiler and libm, reading the weights straight off the safetensors file.
+    category: voice
+    language: C
+    repo: https://github.com/mudler/voxtral-tts.c
+    status: Experimental
+    highlights:
+      - Pure C, no dependency beyond the C standard library and math
+      - BF16 weights read from mmap, no conversion step
+      - 20 preset voices across 9 languages, 24 kHz output
+      - Optional BLAS, Apple Accelerate, NEON and CUDA paths
+
+  # --------------------------------------------------------------- identity
+  - name: voice-detect.cpp
+    tagline: Tell who is speaking, and read their age, gender and mood, without an onnxruntime install anywhere near it.
+    category: identity
+    language: C++17
+    repo: https://github.com/localai-org/voice-detect.cpp
+    media: /media/voice.mp4
+    highlights:
+      - Embedding cosine 0.9999 or better against the reference, often exactly 1.0
+      - 5.4x lower peak memory than the Python path, 62 MB against 334 MB
+      - Six model families, ECAPA-TDNN and WeSpeaker through ERes2Net and CAM++
+      - Verification, identification against a registry, plus age, gender and emotion
+
+  - name: face-detect.cpp
+    tagline: Detect, recognise and verify a face, and catch a photo held up to the camera, all from one shared library.
+    category: identity
+    language: C++17
+    repo: https://github.com/mudler/face-detect.cpp
+    featured: true
+    media: /media/face.mp4
+    clips:
+      - src: /media/face-id.mp4
+        caption: The same person found again in a different photo, one against many
+    highlights:
+      - Boxes and landmarks land within 1 pixel of insightface
+      - Recognition embedding cosine 1.000000 against the reference
+      - Detect, align, recognise, demographics and anti-spoofing in one pipeline
+      - The yunet-sface pack is Apache-2.0, so it is usable commercially
+
+  # ----------------------------------------------------------------- vision
+  - name: locate-anything.cpp
+    tagline: Ask for the red mug on the left in plain words and get coordinates back, not a caption.
+    category: vision
+    language: C++17
+    repo: https://github.com/mudler/locate-anything.cpp
+    featured: true
+    media: /media/locate.mp4
+    highlights:
+      - 1.66x to 3.09x faster than the official PyTorch on CPU
+      - Identical detections, IoU 1.000 against the reference
+      - At q8_0 it is about 4.8x faster than PyTorch f32 and still box-identical
+      - 9.2 GB at f16, 4.7 GB at q4_k, and an annotated PNG out of the box
+
+  - name: rf-detr.cpp
+    tagline: Get boxes and instance masks at COCO quality out of any image, with no PyTorch anywhere in the process.
+    category: vision
+    language: C++17
+    repo: https://github.com/localai-org/rf-detr.cpp
+    highlights:
+      - 11 variants, 5 detection and 6 segmentation, from Nano to 2XLarge
+      - About 9% faster than PyTorch on CPU at F16, and 1.86x smaller
+      - Mean mask IoU 0.99 against PyTorch on the small segmentation variants
+      - 44 published GGUFs, F32 through Q4_K
+
+  # ------------------------------------------------------------------ space
+  - name: depth-anything.cpp
+    tagline: Turn one ordinary photo into distance in metres, a camera pose and a point cloud you can open in a 3D viewer.
+    category: space
+    language: C++17
+    repo: https://github.com/localai-org/depth-anything.cpp
+    featured: true
+    media: /media/depth-race.mp4
+    clips:
+      - src: /media/depth.mp4
+        caption: Metric depth on CPU, against PyTorch on the same box
+    highlights:
+      - 1.31x faster than PyTorch on CPU at q8_0, in half the memory
+      - Loads about 6.7x faster, 112 ms against 749 ms
+      - 99 MB at q4_k, and correlation 1.0 with the reference component by component
+      - Exports to glb, COLMAP and PLY, plus confidence and a sky mask
+
+  - name: free-splatter.cpp
+    tagline: Turn a handful of snapshots into a 3D Gaussian scene with no camera poses, no rig and no GPU.
+    category: space
+    language: C++
+    repo: https://github.com/localai-org/free-splatter.cpp
+    highlights:
+      - 0.22 s per forward pass on Vulkan, against 1.37 s for the PyTorch reference on CUDA
+      - 14 s on 12 CPU threads, roughly 4x the reference, with no GPU at all
+      - Pose-free, so ordinary photos are enough
+      - One 3D Gaussian per pixel, ready for any splat viewer
+
+  - name: trellis2.cpp
+    tagline: Drop in one image and get back a watertight textured mesh you can hand straight to a 3D tool.
+    category: space
+    language: C++
+    repo: https://github.com/localai-org/trellis2cpp
+    highlights:
+      - Single image to GLB with PBR materials, all inference in C++
+      - Prebuilt f16 GGUFs, so no safetensors download and no conversion
+      - Flat C ABI plus a Go demo server with a browser mesh viewer
+      - Metal on by default on Apple, CUDA and CPU elsewhere
+
+  # ------------------------------------------------------------------- text
+  - name: vllm.cpp
+    tagline: Serve a language model with vLLM's throughput from a 66 MiB binary instead of a 9 GB virtualenv.
+    category: text
+    language: C++20
+    repo: https://github.com/mudler/vllm.cpp
+    status: In development
+    media: /media/vllm-race.mp4
+    highlights:
+      - 66 MiB to install, against 9.1 GiB for a vLLM environment
+      - Continuous batching, paged KV cache, prefix caching and speculative decoding
+      - 25 or more architectures, gated token for token against a pinned vLLM oracle
+      - CPU, CUDA, Metal and Vulkan from the same source
+
+  - name: privacy-filter.cpp
+    tagline: Catch names, addresses and card numbers on the machine, before any of it reaches a model or a log.
+    category: text
+    language: C++
+    repo: https://github.com/localai-org/privacy-filter.cpp
+    highlights:
+      - 7.7x faster than HF Transformers on an 8k token document, on CPU
+      - Runs flat to 131k tokens on GPU where HF runs out of memory at about 16k
+      - Exact UTF-8 byte offsets for every span it finds
+      - 360 tokens a second on a Raspberry Pi 5, on-device
+
+  # ------------------------------------------------------------------- data
+  - name: local-store
+    tagline: Get vector search inside LocalAI with nothing to deploy, nothing to configure and no second service to run.
+    category: data
+    language: Go
+    repo: https://github.com/mudler/LocalAI
+    highlights:
+      - Ships in-tree and is the default, so embeddings work on a fresh install
+      - Exact cosine similarity, zero configuration
+      - Backs RAG, the face and voice registries and the semantic router cache
+      - Swap in valkey-store per request when you need durability
+
+  - name: apex-quant
+    tagline: Fit a 35B mixture-of-experts model on a card you already own, and watch it run faster than the full-size build.
+    category: data
+    language: Shell
+    repo: https://github.com/localai-org/apex-quant
+    highlights:
+      - 64.6 GB down to 12.2 GB, at 74.4 tokens a second against 30.4
+      - APEX Quality beats F16 perplexity at a third of the size
+      - Ordinary GGUF files, so stock llama.cpp opens them unpatched
+      - 201 builds already sitting in the LocalAI gallery

website/data/milestones.json

@@ -0,0 +1,6 @@
+[
+  {"date": "2025-07-15", "label": "Backends leave the binary", "tag": "v3.2.0"},
+  {"date": "2026-03-15", "label": "Agents, and a new interface", "tag": "v4.0.0"},
+  {"date": "2026-04-15", "label": "It becomes a cluster", "tag": "v4.1.0"},
+  {"date": "2026-05-15", "label": "It sees and hears", "tag": "v4.2.0"}
+]

website/data/stars.json

@@ -0,0 +1 @@
+[["2023-03-31", 100], ["2023-04-25", 900], ["2023-05-03", 1700], ["2023-05-14", 2500], ["2023-05-17", 3300], ["2023-05-22", 4100], ["2023-05-30", 4900], ["2023-06-07", 5700], ["2023-06-16", 6500], ["2023-07-02", 7300], ["2023-07-22", 8100], ["2023-08-08", 8900], ["2023-08-29", 9700], ["2023-09-21", 10500], ["2023-10-15", 11300], ["2023-11-08", 12100], ["2023-11-29", 12900], ["2023-12-22", 13700], ["2024-01-14", 14500], ["2024-02-05", 15300], ["2024-02-28", 16100], ["2024-03-20", 16900], ["2024-04-07", 17700], ["2024-04-22", 18500], ["2024-05-13", 19300], ["2024-06-10", 20100], ["2024-07-14", 20900], ["2024-08-17", 21700], ["2024-09-19", 22500], ["2024-10-26", 23300], ["2024-11-12", 24100], ["2024-11-18", 24900], ["2024-12-04", 25700], ["2025-01-03", 26500], ["2025-01-13", 27300], ["2025-01-21", 28100], ["2025-02-01", 28900], ["2025-02-28", 29700], ["2025-04-04", 30500], ["2025-04-29", 31300], ["2025-06-03", 32100], ["2025-07-10", 32900], ["2025-08-12", 33700], ["2025-09-15", 34500], ["2025-10-24", 35300], ["2025-11-04", 36100], ["2025-11-07", 36900], ["2025-11-14", 37700], ["2025-11-26", 38500], ["2025-12-09", 39300], ["2025-12-24", 40000], ["2026-07-31", 48042]]

website/data/starsmeta.json

@@ -0,0 +1 @@
+{"measuredUntil": "2025-12-24", "measuredStars": 40000, "note": "GitHub caps stargazers pagination at 400 pages"}

website/data/stats.yaml

@@ -0,0 +1,13 @@
+# Counters shown on the landing page.
+#
+# The four GitHub fields are rewritten by .github/ci/refresh-site-counters.sh,
+# which runs weekly from .github/workflows/refresh-site-counters.yml. Editing
+# them by hand works but will be overwritten on the next run.
+stars: 48067
+forks: 4320
+contributors: 225
+releases: 133
+
+# The GitHub API cannot answer for this one, so it is maintained by hand and
+# the refresh script carries it through untouched.
+discord: 3187

website/hugo.toml

@@ -0,0 +1,31 @@
+baseURL = 'https://localai.io/'
+languageCode = 'en-GB'
+defaultContentLanguage = 'en'
+title = 'LocalAI'
+enableEmoji = true
+
+# The main site. Documentation is a second Hugo site under ../docs,
+# built with baseURL <root>/docs/ and merged into this site's public/ by CI.
+# See .github/workflows/gh-pages.yml.
+
+[params]
+  description = 'LocalAI is the open source AI engine. Run any model, LLMs, vision, voice, image and video, on any hardware. No GPU required.'
+  author = 'Ettore Di Giacinto'
+  docsURL = '/docs/'
+  github = 'https://github.com/mudler/LocalAI'
+  discord = 'https://discord.gg/uJAeKSAGDy'
+  x = 'https://twitter.com/LocalAI_API'
+  huggingface = 'https://huggingface.co/mudler'
+  # Counters live in data/stats.yaml, which .github/ci/refresh-site-counters.sh
+  # rewrites weekly. They used to sit here as hand-typed strings and drifted.
+
+[markup.goldmark.renderer]
+  unsafe = true
+
+[outputs]
+  home = ['html']
+  section = ['html', 'rss']
+  page = ['html']
+
+[taxonomies]
+  tag = 'tags'

website/layouts/_default/baseof.html

@@ -0,0 +1,12 @@
+<!doctype html>
+<html lang="en">
+{{ partial "head.html" . }}
+<body>
+  <div class="prog"><i id="pbar"></i></div>
+  <canvas id="field" aria-hidden="true"></canvas>
+  {{ partial "nav.html" . }}
+  {{ block "main" . }}{{ end }}
+  {{ partial "footer.html" . }}
+  <script src="{{ "js/site.js" | relURL }}" defer></script>
+</body>
+</html>

website/layouts/_default/taxonomy.html

@@ -0,0 +1,17 @@
+{{ define "main" }}
+<main id="top">
+  <section class="navy">
+    <div class="shell">
+      <div class="bars" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+      <p class="kicker">Tags</p>
+      <h2 class="mt1">Everything we have written about</h2>
+      <div class="chips mt2">
+        {{ range .Data.Terms.Alphabetical }}
+        <a href="{{ .Page.RelPermalink }}"><span>{{ .Page.Title }} ({{ .Count }})</span></a>
+        {{ end }}
+      </div>
+      <p class="mt3"><a class="btn btn--o" href="{{ "blog/" | relURL }}">All posts &#8594;</a></p>
+    </div>
+  </section>
+</main>
+{{ end }}

website/layouts/_default/term.html

@@ -0,0 +1,23 @@
+{{ define "main" }}
+<main id="top">
+  <section class="navy">
+    <div class="shell">
+      <div class="bars" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+      <p class="kicker">Tagged</p>
+      <h2 class="mt1">{{ .Title }}</h2>
+      <p class="lede mt2">{{ len .Pages }} post{{ if ne (len .Pages) 1 }}s{{ end }} tagged {{ .Title }}.</p>
+      <div class="cards">
+        {{ range .Pages.ByDate.Reverse }}
+        <a class="cd" href="{{ .RelPermalink }}">
+          <p class="cd__k">{{ with .Params.category }}{{ . }}{{ else }}Post{{ end }}</p>
+          <h3>{{ .Title }}</h3>
+          <p>{{ .Params.summary | default .Summary }}</p>
+          <span class="cd__go">Read the post &#8594;</span>
+        </a>
+        {{ end }}
+      </div>
+      <p class="mt3"><a class="btn btn--o" href="{{ "blog/" | relURL }}">All posts &#8594;</a></p>
+    </div>
+  </section>
+</main>
+{{ end }}

website/layouts/blog/list.html

@@ -0,0 +1,45 @@
+{{ define "main" }}
+<main id="top">
+
+<section class="navy bp-top">
+  <div class="shell">
+    <div class="bars" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker">From the team</p>
+    <h1 class="bp-h1 mt1">{{ .Title }}</h1>
+    {{ with .Description }}<p class="lede mt2">{{ . }}</p>{{ end }}
+    {{ with .Content }}<div class="bp-intro">{{ . }}</div>{{ end }}
+  </div>
+</section>
+
+<section class="navy bp-index">
+  <div class="shell">
+    <div class="bp-list">
+      {{ range .Pages.ByDate.Reverse }}
+      <a class="bp-item rv" href="{{ .RelPermalink }}">
+        <span class="bp-item__m">
+          <time datetime="{{ .Date.Format "2006-01-02" }}">{{ .Date.Format "2 Jan 2006" }}</time>
+          {{ with .Params.category }}<em>{{ . }}</em>{{ end }}
+        </span>
+        <span class="bp-item__b">
+          <span class="bp-item__t">{{ .Title }}</span>
+          <span class="bp-item__s">{{ with .Params.summary }}{{ . }}{{ else }}{{ .Summary }}{{ end }}</span>
+        </span>
+        <span class="bp-item__go">Read &#8594;</span>
+      </a>
+      {{ end }}
+    </div>
+
+    <div class="bp-end rv">
+      <p class="kicker">Next</p>
+      <h2 class="bp-end__h">Install it and check the numbers yourself.</h2>
+      <p>Every figure in these posts comes out of a benchmark suite or a release that you can run on your own hardware.</p>
+      <div class="acts">
+        <a class="btn" href="{{ .Site.Params.docsURL }}">Read the documentation &#8594;</a>
+        <a class="btn btn--o" href="{{ .Site.Params.github }}">LocalAI on GitHub &#8599;</a>
+      </div>
+    </div>
+  </div>
+</section>
+
+</main>
+{{ end }}

website/layouts/blog/single.html

@@ -0,0 +1,66 @@
+{{ define "main" }}
+<main id="top">
+
+<article class="navy bp-article">
+  <div class="shell">
+
+    <p class="bp-back"><a href="{{ "/blog/" | relURL }}">&#8592; All posts</a></p>
+
+    <header class="bp-hd">
+      <div class="bars" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+      {{ with .Params.category }}<p class="kicker">{{ . }}</p>{{ end }}
+      <h1 class="bp-h1 mt1">{{ .Title }}</h1>
+      {{ with .Params.summary }}<p class="lede mt2 bp-sum">{{ . }}</p>{{ end }}
+      <p class="bp-meta">
+        {{ with .Params.author }}<span>{{ . }}</span>{{ end }}
+        <span><time datetime="{{ .Date.Format "2006-01-02" }}">{{ .Date.Format "2 January 2006" }}</time></span>
+        <span>{{ .ReadingTime }} min read</span>
+      </p>
+    </header>
+
+    <div class="bp-body">
+      {{ .Content }}
+    </div>
+
+    {{ with .Params.tags }}
+    <div class="bp-tags">
+      {{ range . }}<span>{{ . }}</span>{{ end }}
+    </div>
+    {{ end }}
+
+    <div class="bp-cta">
+      <div>
+        <p class="kicker">Try it</p>
+        <h2 class="bp-cta__h">Run this on the machine you are reading it on.</h2>
+        <p>LocalAI installs as a container, a binary, a macOS DMG or a Helm chart, and pulls a backend the first time a model asks for one.</p>
+      </div>
+      <div class="acts">
+        <a class="btn" href="{{ .Site.Params.docsURL }}">Read the documentation &#8594;</a>
+        <a class="btn btn--o" href="{{ .Site.Params.github }}">LocalAI on GitHub &#8599;</a>
+      </div>
+    </div>
+
+    {{ if or .PrevInSection .NextInSection }}
+    <nav class="bp-pn" aria-label="More posts">
+      {{ with .NextInSection }}
+      <a class="bp-pn__i" href="{{ .RelPermalink }}">
+        <span class="bp-pn__k">&#8592; Newer</span>
+        <span class="bp-pn__t">{{ .Title }}</span>
+      </a>
+      {{ else }}<span></span>{{ end }}
+      {{ with .PrevInSection }}
+      <a class="bp-pn__i bp-pn__i--r" href="{{ .RelPermalink }}">
+        <span class="bp-pn__k">Older &#8594;</span>
+        <span class="bp-pn__t">{{ .Title }}</span>
+      </a>
+      {{ end }}
+    </nav>
+    {{ end }}
+
+    <p class="bp-back bp-back--b"><a href="{{ "/blog/" | relURL }}">&#8592; All posts</a></p>
+
+  </div>
+</article>
+
+</main>
+{{ end }}

website/layouts/engines/list.html

@@ -0,0 +1,130 @@
+{{ define "main" }}
+{{ $data := .Site.Data.engines }}
+{{ $cats := $data.categories }}
+{{ $engines := $data.engines }}
+<main id="top">
+
+<!-- HEAD -->
+<section class="hero navy">
+  <div class="shell">
+    <p class="kicker fd" style="margin-top:0">Engines we build</p>
+    <h1 class="eng-h1"><u><b>Eighteen engines,</b></u><u><b><s>written from scratch.</s></b></u></h1>
+    <div class="bars" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="lede fd mt2">Most backends wrap somebody else's engine. These do not. Each one exists because the thing we needed was a multi-gigabyte Python install, or closed, or nobody had built it yet. What you get instead is a binary and a GGUF file, checked against the reference implementation in CI, running on the machine you already own.</p>
+    <div class="acts fd">
+      <a class="btn" href="/#start">Install LocalAI <span>&#8594;</span></a>
+      <a class="btn btn--o" href="/docs/features/backends/">How backends work</a>
+    </div>
+    <div class="figures fd">
+      <div><b class="tnum" data-count="{{ len $engines }}">0</b><span>Engines</span></div>
+      <div><b class="tnum" data-count="{{ len $cats }}">0</b><span>Modalities</span></div>
+      <div><b class="tnum" data-count="0">0</b><span>Python at inference</span></div>
+      <div><b class="tnum" data-count="0" data-text="MIT">0</b><span>Licence</span></div>
+    </div>
+  </div>
+</section>
+
+<!-- CATALOGUE -->
+<section class="navy" id="catalogue">
+  <div class="shell">
+
+    <div class="engf rv" role="group" aria-label="Filter engines by modality">
+      <button type="button" class="engf__c" data-cat="all" aria-pressed="true">All <b>{{ len $engines }}</b></button>
+      {{ range $cats }}
+      {{ $n := len (where $engines "category" .id) }}
+      {{ if gt $n 0 }}<button type="button" class="engf__c" data-cat="{{ .id }}" aria-pressed="false">{{ .label }} <b>{{ $n }}</b></button>{{ end }}
+      {{ end }}
+    </div>
+
+    {{ range $cats }}
+    {{ $inCat := where $engines "category" .id }}
+    {{ if gt (len $inCat) 0 }}
+    <div class="engg" data-group="{{ .id }}" id="{{ .id }}">
+      <div class="engg__h rv">
+        <p class="kicker">{{ .label }}</p>
+        <p class="engg__b">{{ .blurb }}</p>
+      </div>
+      <div class="engs">
+        {{ range $inCat }}
+        {{ $e := . }}
+        <a class="eng rv{{ if $e.featured }} eng--f{{ end }}" data-cat="{{ $e.category }}" href="{{ $e.repo }}">
+          {{ with $e.media }}
+          <div class="eng__m">
+            <video src="{{ . }}"{{ with $e.poster }} poster="{{ . }}"{{ end }} muted loop playsinline preload="none" data-lazy aria-label="{{ $e.name }}: {{ $e.tagline }}"></video>
+          </div>
+          {{ end }}
+          <div class="eng__b">
+            <p class="eng__k"><span>{{ $e.language }}</span>{{ with $e.status }}<b class="eng__s">{{ . }}</b>{{ end }}</p>
+            <h3 class="eng__n">{{ $e.name }}</h3>
+            <p class="eng__t">{{ $e.tagline }}</p>
+            {{ with $e.highlights }}
+            <ul class="eng__l">{{ range . }}<li>{{ . }}</li>{{ end }}</ul>
+            {{ end }}
+            {{ with $e.clips }}
+            <div class="eng__x">
+              {{ range . }}
+              <figure>
+                <video src="{{ .src }}" muted loop playsinline preload="none" data-lazy aria-label="{{ .caption }}"></video>
+                <figcaption>{{ .caption }}</figcaption>
+              </figure>
+              {{ end }}
+            </div>
+            {{ end }}
+            <span class="eng__go">{{ $e.name }} on GitHub &#8599;</span>
+          </div>
+        </a>
+        {{ end }}
+      </div>
+    </div>
+    {{ end }}
+    {{ end }}
+
+    <p class="engn rv" role="status" hidden>Nothing in that modality yet.</p>
+  </div>
+</section>
+
+<!-- CLOSE -->
+<section class="deepbg" id="why">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">The rule we hold them to</p>
+    <h2 class="rv mt1" style="max-width:20ch">A port only ships once it matches the original.</h2>
+    <p class="lede rv mt2">Every engine here is gated against the framework it replaces, on the same input, on the same machine. That means a transcript that comes out word for word identical, boxes that land on the same pixels, or a waveform inside a stated tolerance. Speed is the part we then go and win, and the numbers on this page come out of each engine's own benchmark suite, not a marketing run.</p>
+    <div class="acts rv">
+      <a class="btn" href="/#start">Install LocalAI &#8594;</a>
+      <a class="btn btn--o" href="https://github.com/mudler/LocalAI">LocalAI on GitHub &#8599;</a>
+    </div>
+  </div>
+</section>
+
+</main>
+
+<script>
+(function(){
+  var chips = [].slice.call(document.querySelectorAll('.engf__c'));
+  var cards = [].slice.call(document.querySelectorAll('.eng'));
+  var groups = [].slice.call(document.querySelectorAll('.engg'));
+  var none = document.querySelector('.engn');
+  if (!chips.length) return;
+  function apply(cat){
+    var shown = 0;
+    cards.forEach(function(c){
+      var on = (cat === 'all' || c.dataset.cat === cat);
+      c.hidden = !on;
+      if (on) shown++;
+    });
+    groups.forEach(function(g){
+      g.hidden = !(cat === 'all' || g.dataset.group === cat);
+    });
+    chips.forEach(function(c){ c.setAttribute('aria-pressed', String(c.dataset.cat === cat)); });
+    if (none) none.hidden = shown > 0;
+    try { history.replaceState(null, '', cat === 'all' ? location.pathname : location.pathname + '#' + cat); } catch (e) {}
+  }
+  chips.forEach(function(c){
+    c.addEventListener('click', function(){ apply(c.dataset.cat); });
+  });
+  var start = (location.hash || '').replace('#','');
+  if (start && chips.some(function(c){ return c.dataset.cat === start; })) apply(start);
+})();
+</script>
+{{ end }}

website/layouts/index.html

@@ -0,0 +1,581 @@
+{{ define "main" }}
+<main id="top">
+
+<!-- HERO -->
+<section class="hero navy">
+  <div class="shell">
+    <div class="hero__grid">
+      <div>
+        <p class="kicker fd" style="margin-top:0">Open source · MIT · v4.8.0</p>
+        <h1><u><b>Make AI run on</b></u><u><b><s>every machine.</s></b></u></h1>
+        <div class="bars" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+        <p class="lede fd mt2">Text, voice, vision, images, video, 3D and agents, from one open runtime. It works on the laptop you already own, and scales to a room full of GPUs when you have one. When the engine we need is too heavy, too closed, or does not exist, we write it.</p>
+        <div class="acts fd">
+          <a class="btn" href="#start">Install LocalAI <span>→</span></a>
+          <a class="btn btn--o" href="/docs/">Read the docs</a>
+          <a class="btn btn--o" href="{{ .Site.Params.github }}">&#9733; Star on GitHub</a>
+        </div>
+        <div class="figures fd">
+          <div><b class="tnum" data-count="{{ .Site.Data.stats.stars }}">0</b><span>GitHub stars</span></div>
+          <div><b class="tnum" data-count="73">0</b><span>Backends</span></div>
+          <div><b class="tnum" data-count="{{ len .Site.Data.engines.engines }}">0</b><span>Engines we wrote</span></div>
+          <div><b class="tnum" data-count="1585">0</b><span>Models, one click</span></div>
+        </div>
+      </div>
+      <div class="fd">
+        <figure class="screen" style="margin:0">
+          <figcaption class="screen__bar"><i></i> localai · chat <b>CPU only, no GPU</b></figcaption>
+          <video src="/media/hero-ui.mp4" poster="/img/hero-poster.jpg" autoplay muted loop playsinline preload="metadata" aria-label="The LocalAI interface running a chat completion on CPU"></video>
+        </figure>
+      </div>
+    </div>
+  </div>
+</section>
+
+<!-- LOCALAI -->
+<section class="navy" id="localai">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">The runtime</p>
+    <h2 class="rv mt1" style="max-width:21ch">LocalAI is the engine everything else plugs into.</h2>
+    <p class="lede rv mt2">One binary with an OpenAI-compatible API in front of it. Point an existing client at it and the calls keep working, except now the model is on your machine. It also speaks the Anthropic, Ollama and ElevenLabs APIs, so most tools need a URL change and nothing else.</p>
+    <p class="lede rv mt2">Underneath, a small core pulls each engine in as a separate backend, only when a model asks for it. That is why one install covers this much ground without becoming a 9 GB download.</p>
+    <div class="apis rv">
+      <span>OpenAI API</span><span>Anthropic API</span><span>Ollama API</span><span>ElevenLabs API</span><span>Realtime over WebRTC</span>
+    </div>
+    <div class="duo mt3">
+      <div>
+        <div class="lanes rv">
+          <a class="lane" href="/docs/features/text-generation/"><span class="lane__k">Reason</span><span class="lane__d">Language models, tool calling, structured output</span><span class="lane__t">llama.cpp · vLLM · MLX</span></a>
+          <a class="lane" href="/docs/features/openai-realtime/"><span class="lane__k">Listen</span><span class="lane__d">Realtime voice, transcription, diarization</span><span class="lane__t">parakeet · whisper</span></a>
+          <a class="lane" href="/docs/features/text-to-audio/"><span class="lane__k">Speak</span><span class="lane__d">Speech synthesis and voice cloning</span><span class="lane__t">moss-tts · piper</span></a>
+          <a class="lane" href="/docs/features/object-detection/"><span class="lane__k">See</span><span class="lane__d">Vision, detection, recognition, depth, 3D</span><span class="lane__t">rf-detr · depth-anything</span></a>
+          <a class="lane" href="/docs/features/image-generation/"><span class="lane__k">Create</span><span class="lane__d">Images, video, music and sound</span><span class="lane__t">diffusers · ace-step</span></a>
+          <a class="lane" href="/docs/features/agents/"><span class="lane__k">Act</span><span class="lane__d">Agents, MCP, skills, RAG, interactive tools</span><span class="lane__t">agents · MCP apps</span></a>
+        </div>
+        <p class="rv mt2"><a class="btn btn--o" href="/docs/">Read the documentation →</a></p>
+      </div>
+      <div class="duo__m rv">
+        <figure class="screen" style="margin:0">
+          <figcaption class="screen__bar"><i></i> localai · model gallery <b>1,585 models</b></figcaption>
+          <video src="/media/gallery.mp4" muted loop playsinline preload="none" data-lazy aria-label="Installing a model from the LocalAI gallery"></video>
+        </figure>
+      </div>
+    </div>
+  </div>
+</section>
+
+<!-- MISSION -->
+<section class="navy" id="mission">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">Why the project exists</p>
+    <h2 class="rv mt1" style="max-width:19ch">Local AI should not need a datacenter.</h2>
+    <div class="mission">
+      <div class="mi rv">
+        <p class="mi__n">01 / HARDWARE</p>
+        <h3>Every feature ships a CPU path first.</h3>
+        <p>Not a degraded mode that technically runs. The real one, tested in CI, on the hardware most people already have. GPUs make it faster, they are not the price of entry.</p>
+        <p class="mi__meta">x86_64 · ARM64 · CUDA · ROCm · SYCL · Metal · Vulkan</p>
+      </div>
+      <div class="mi rv">
+        <p class="mi__n">02 / REALTIME</p>
+        <h3>You can talk to it, and it answers.</h3>
+        <p>Speech in, tool calls in the middle, speech out over WebRTC, fast enough to feel like a conversation. Transcription, diarization and speech synthesis all run without a GPU.</p>
+        <p class="mi__meta">Realtime API · WebRTC · streaming ASR · TTS · VAD</p>
+      </div>
+      <div class="mi rv">
+        <p class="mi__n">03 / DISTRIBUTED</p>
+        <h3>Plug in a second machine and stop there.</h3>
+        <p>Routing, VRAM-aware placement, prefix-cache affinity and failover are the runtime's problem. You add hardware, the cluster works out what to do with it.</p>
+        <p class="mi__meta">Smart routing · autoscaling · P2P · NATS · federation</p>
+      </div>
+    </div>
+  </div>
+</section>
+
+<!-- SENSES -->
+<section class="navy" id="senses">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">Senses</p>
+    <h2 class="rv mt1" style="max-width:20ch">Give the model eyes and ears.</h2>
+    <p class="lede rv mt2">A chat model can only work with what somebody types at it. Our engines change what it has access to: what is happening in the room, who walked into it, where things are in space, and how to answer out loud. All of it runs locally, most of it on a CPU.</p>
+    <div class="senses">
+      <div class="sn rv"><p class="sn__v">Hear <u>words</u></p>
+        <p>Live transcription with speaker labels and timestamps, fast enough to keep up with a meeting while it is still happening.</p>
+        <div class="sn__e"><span>parakeet.cpp</span><span>moss-transcribe.cpp</span></div></div>
+      <div class="sn rv"><p class="sn__v">Hear <u>the room</u></p>
+        <p>527 kinds of sound event: a door, a dog, breaking glass, a smoke alarm. The model notices things nobody thought to type.</p>
+        <div class="sn__e"><span>ced.cpp</span></div></div>
+      <div class="sn rv"><p class="sn__v">Know <u>who</u></p>
+        <p>Recognise a voice, recognise a face, and tell a live person from a photo held up to the camera.</p>
+        <div class="sn__e"><span>voice-detect.cpp</span><span>face-detect.cpp</span></div></div>
+      <div class="sn rv"><p class="sn__v">See <u>things</u></p>
+        <p>Ask for "the red mug on the left" in plain language and get back coordinates, not a caption.</p>
+        <div class="sn__e"><span>locate-anything.cpp</span><span>rf-detr.cpp</span></div></div>
+      <div class="sn rv"><p class="sn__v">See <u>space</u></p>
+        <p>Distance in metres from one ordinary photo, and a full 3D reconstruction from a handful of them. No rig, no camera poses, no GPU.</p>
+        <div class="sn__e"><span>depth-anything.cpp</span><span>free-splatter.cpp</span><span>trellis2.cpp</span></div></div>
+      <div class="sn rv"><p class="sn__v">Speak</p>
+        <p>Long-form speech in a cloned voice, across dozens of languages, up to 48 kHz.</p>
+        <div class="sn__e"><span>moss-tts.cpp</span><span>magpie-tts.cpp</span><span>vibevoice.cpp</span><span>voxtral-tts.c</span></div></div>
+      <div class="sn rv"><p class="sn__v">Hear <u>clearly</u></p>
+        <p>Echo cancellation, noise suppression and dereverberation, so a voice loop survives a real room with a real speaker in it.</p>
+        <div class="sn__e"><span>LocalVQE</span></div></div>
+      <div class="sn rv"><p class="sn__v">Forget <u>on purpose</u></p>
+        <p>Names, addresses and card numbers get caught and redacted on the machine, before anything is sent anywhere.</p>
+        <div class="sn__e"><span>privacy-filter.cpp</span></div></div>
+    </div>
+    <p class="lede rv mt3" style="max-width:74ch;color:var(--ink)">One session can do all of it at once: hear the room, work out who is talking, read what is on the desk, call a tool, and answer out loud. One API, one machine, nothing leaves the building.</p>
+  </div>
+</section>
+
+<!-- MADE WITH -->
+<section class="navy" id="made">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">Made with LocalAI</p>
+    <h2 class="rv mt1" style="max-width:23ch">Nobody was at the keyboard.</h2>
+    <div class="made">
+      <div class="rv">
+        <div class="player" id="player">
+          <span class="player__tag">Sound on</span>
+          <video id="pv" src="/media/presenter.mp4" poster="/img/presenter-poster.jpg" playsinline preload="none" aria-label="A generated presenter explaining voice cloning, produced entirely with LocalAI"></video>
+          <div class="player__ui"><span class="player__btn"><i></i> Play with sound</span></div>
+        </div>
+      </div>
+      <div>
+        <p class="lede rv">Every part of this clip came out of LocalAI, and nib drove the machine that made it. The agent opened the app, ran the demo and captured the screen, while a local model wrote the script, a cloned voice read it, and the video endpoint generated and lip-synced the presenter. No human touched the keyboard, and nothing left the building.</p>
+        <div class="recipe rv">
+          <div><b>Direction</b><span>nib, our agent harness, drove the machine end to end</span></div>
+          <div><b>Script</b><span>Written by a local language model</span></div>
+          <div><b>Voice</b><span>Cloned from a few seconds of reference audio</span></div>
+          <div><b>Presenter</b><span>Generated and lip-synced through the video endpoint</span></div>
+        </div>
+        <p class="rv mt2"><a class="btn btn--o" href="/docs/features/text-to-audio/">How the voice pipeline works &#8594;</a></p>
+      </div>
+    </div>
+  </div>
+</section>
+
+<!-- ENGINES -->
+<section class="navy" id="engines">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">Engines we build</p>
+    <h2 class="rv mt1" style="max-width:22ch">{{ len .Site.Data.engines.engines }} engines, written from scratch.</h2>
+    {{/* Names the link back to the runtime section explicitly. Readers were
+         arriving here and taking these for unrelated side projects, because
+         nothing on the page said they are the backends the core loads. */}}
+    <p class="lede rv mt2">Those backends the core pulls in on demand are mostly ours. Most projects wrap somebody else's engine. We wrote ours, because the thing we needed came as a 9 GB Python install, or was closed, or did not exist. Each one is a binary and a GGUF file, checked against the reference implementation in CI.</p>
+
+    <div class="spot rv">
+      <div>
+        <h3>parakeet.cpp</h3>
+        <p class="spot__h">Twenty-seven times faster than whisper.cpp, on a CPU.</p>
+        <p>NVIDIA NeMo Parakeet, ported to C++ and ggml. Ten checkpoints, all of them verified at WER 0 against NeMo, which means the transcript comes out byte for byte identical while finishing first. Cache-aware streaming with end-of-utterance detection handles live audio, and the multilingual streaming model covers 40 or more locales.</p>
+        <div class="facts">
+          <div><b>27x</b><span>vs whisper.cpp, CPU</span></div>
+          <div><b>1.40x</b><span>vs NeMo, CPU median</span></div>
+          <div><b>WER 0</b><span>Parity with NeMo</span></div>
+          <div><b>37%</b><span>Size at q8_0</span></div>
+        </div>
+        <p class="mt2"><a class="btn btn--o" href="https://github.com/mudler/parakeet.cpp">parakeet.cpp on GitHub ↗</a></p>
+      </div>
+      <div class="spot__m">
+        <figure><video src="/media/parakeet-duel.mp4" muted loop playsinline preload="none" data-lazy aria-label="parakeet.cpp finishing ahead of NeMo on the same audio"></video>
+          <figcaption>Same audio, same words, ours finishes first</figcaption></figure>
+        <figure><img src="/img/parakeet-speedup.jpg" alt="Bar chart of parakeet.cpp CPU speedup over NeMo per dtype" loading="lazy">
+          <figcaption>CPU speedup over NeMo, by dtype</figcaption></figure>
+      </div>
+    </div>
+
+    <div class="spot spot--r rv">
+      <div>
+        <h3>depth-anything.cpp</h3>
+        <p class="spot__h">Beats PyTorch on CPU, in half the memory.</p>
+        <p>Depth Anything 3 as a 99 MB file. It gives you metric depth, per-pixel confidence, camera intrinsics and extrinsics, and a back-projected point cloud you can export to glb or COLMAP. Output correlates 1.0 with the reference implementation, component by component, and there is no Python or CUDA toolkit anywhere at inference.</p>
+        <div class="facts">
+          <div><b>1.31x</b><span>vs PyTorch, CPU</span></div>
+          <div><b>363 MB</b><span>Peak RAM, q8_0</span></div>
+          <div><b>6.7x</b><span>Faster to load</span></div>
+          <div><b>99 MB</b><span>Smallest build</span></div>
+        </div>
+        <p class="mt2"><a class="btn btn--o" href="https://github.com/mudler/depth-anything.cpp">depth-anything.cpp on GitHub ↗</a></p>
+      </div>
+      <div class="spot__m">
+        <figure><video src="/media/depth-race.mp4" muted loop playsinline preload="none" data-lazy aria-label="depth-anything.cpp finishing ahead of PyTorch on CPU"></video>
+          <figcaption>One photo in, distance in metres out, ahead of PyTorch on the same CPU</figcaption></figure>
+      </div>
+    </div>
+
+    <div class="wallhead rv">
+      <h3>And the rest of them, running.</h3>
+      <p>Every clip comes out of the benchmark suite that runs in CI on that engine. Where you see two panes, ours is racing the reference implementation on the same machine, on the same input.</p>
+    </div>
+<div class="wall">
+      <a class="wi rv" href="https://github.com/mudler/parakeet.cpp"><video src="/media/parakeet-long.mp4" muted loop playsinline preload="none" data-lazy aria-label="parakeet.cpp racing NeMo on long audio"></video>
+        <div class="wi__c"><em>parakeet.cpp</em> long-form transcription <b>vs NeMo</b></div></a>
+      <a class="wi rv" href="https://github.com/localai-org/ced.cpp"><video src="/media/ced.mp4" muted loop playsinline preload="none" data-lazy aria-label="ced.cpp tagging sound events live"></video>
+        <div class="wi__c"><em>ced.cpp</em> sound events, live <b>527 classes</b></div></a>
+      <a class="wi rv" href="https://github.com/mudler/face-detect.cpp"><video src="/media/face.mp4" muted loop playsinline preload="none" data-lazy aria-label="face-detect.cpp detecting, landmarking and recognising faces"></video>
+        <div class="wi__c"><em>face-detect.cpp</em> detect, landmark, recognise <b>no Python</b></div></a>
+      <a class="wi rv" href="https://github.com/mudler/face-detect.cpp"><video src="/media/face-id.mp4" muted loop playsinline preload="none" data-lazy aria-label="face-detect.cpp finding the same person in a lineup"></video>
+        <div class="wi__c"><em>face-detect.cpp</em> same person, different photo <b>1 to N</b></div></a>
+      <a class="wi rv" href="https://github.com/localai-org/voice-detect.cpp"><video src="/media/voice.mp4" muted loop playsinline preload="none" data-lazy aria-label="voice-detect.cpp speaker recognition benchmark"></video>
+        <div class="wi__c"><em>voice-detect.cpp</em> who is speaking <b>vs reference</b></div></a>
+      <a class="wi rv" href="https://github.com/mudler/depth-anything.cpp"><video src="/media/depth.mp4" muted loop playsinline preload="none" data-lazy aria-label="depth-anything.cpp racing PyTorch on CPU"></video>
+        <div class="wi__c"><em>depth-anything.cpp</em> metric depth <b>vs PyTorch, CPU</b></div></a>
+      <a class="wi rv" href="https://github.com/mudler/locate-anything.cpp"><video src="/media/locate.mp4" muted loop playsinline preload="none" data-lazy aria-label="locate-anything.cpp open vocabulary detection race"></video>
+        <div class="wi__c"><em>locate-anything.cpp</em> say it, find it <b>open vocabulary</b></div></a>
+      <a class="wi rv" href="https://github.com/mudler/moss-tts.cpp"><video src="/media/moss.mp4" muted loop playsinline preload="none" data-lazy aria-label="moss-tts.cpp synthesis benchmark"></video>
+        <div class="wi__c"><em>moss-tts.cpp</em> 48 kHz voice cloning <b>vs reference</b></div></a>
+      <a class="wi rv" href="https://github.com/mudler/magpie-tts.cpp"><video src="/media/magpie.mp4" muted loop playsinline preload="none" data-lazy aria-label="magpie-tts.cpp synthesis benchmark"></video>
+        <div class="wi__c"><em>magpie-tts.cpp</em> 9 languages, 5 voices <b>vs reference</b></div></a>
+    </div>
+
+    <div class="reel rv" aria-hidden="true">
+      <div class="reel__t">
+        <span>vllm.cpp</span><span>parakeet.cpp</span><span>moss-transcribe.cpp</span><span>moss-tts.cpp</span><span>magpie-tts.cpp</span><span>ced.cpp</span><span>voice-detect.cpp</span><span>voxtral-tts.c</span><span>vibevoice.cpp</span><span>rf-detr.cpp</span><span>locate-anything.cpp</span><span>depth-anything.cpp</span><span>face-detect.cpp</span><span>free-splatter.cpp</span><span>trellis2.cpp</span><span>privacy-filter.cpp</span><span>LocalVQE</span><span>local-store</span><span>apex-quant</span>
+        <span>vllm.cpp</span><span>parakeet.cpp</span><span>moss-transcribe.cpp</span><span>moss-tts.cpp</span><span>magpie-tts.cpp</span><span>ced.cpp</span><span>voice-detect.cpp</span><span>voxtral-tts.c</span><span>vibevoice.cpp</span><span>rf-detr.cpp</span><span>locate-anything.cpp</span><span>depth-anything.cpp</span><span>face-detect.cpp</span><span>free-splatter.cpp</span><span>trellis2.cpp</span><span>privacy-filter.cpp</span><span>LocalVQE</span><span>local-store</span><span>apex-quant</span>
+      </div>
+    </div>
+    <div class="quiet rv">
+      <div>
+        <p class="quiet__tag">In development, not announced yet</p>
+        <h4>vllm.cpp</h4>
+        <p>vLLM ported to C++20, with paged attention, continuous batching and prefix caching, on CPU, CUDA, Metal and Vulkan. It installs as 66 MB instead of a 9.1 GB virtualenv, and it stays ahead of vLLM at every concurrency level we have measured so far. Still being finished, so treat the numbers as provisional.</p>
+      </div>
+      <video src="/media/vllm-race.mp4" muted loop playsinline preload="none" data-lazy aria-label="vllm.cpp ahead of vLLM at every concurrency level"></video>
+    </div>
+
+    <p class="rv mt2"><a class="btn btn--o" href="/engines/">All {{ len .Site.Data.engines.engines }} engines →</a></p>
+  </div>
+</section>
+
+<!-- APEX -->
+<section class="paper" id="apex">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">APEX quantization</p>
+    <h2 class="rv mt1" style="max-width:19ch">The model you could not fit, on the card you already own.</h2>
+    {{/* APEX was being used as a known term on first appearance, in a section
+         that opened straight onto a benchmark table. The first two sentences
+         say what it is and why it follows the engines. */}}
+    <p class="lede rv mt2">The engine decides how fast a model runs. The weights decide whether it runs at all, so we build those too. APEX assigns a different precision to every tensor and every layer: a 35B mixture-of-experts model goes from 64.6 GB, out of reach of any consumer GPU, to 12.2 GB at 74 tokens a second. That is more than twice the speed of the original, and quality barely moves. The file is an ordinary GGUF, so stock llama.cpp opens it with no patches, and 201 of them are already sitting in the LocalAI gallery.</p>
+    <div class="sizes rv">
+      <div class="sz"><span>F16 · 64.6 GB</span><i style="width:100%"></i><u>30.4 t/s</u></div>
+      <div class="sz"><span>Q8_0 · 34.4 GB</span><i style="width:53%"></i><u>52.5 t/s</u></div>
+      <div class="sz"><span>APEX Quality · 21.3 GB</span><i style="width:33%"></i><u>62.3 t/s</u></div>
+      <div class="sz"><span>APEX Mini · 12.2 GB</span><i style="width:19%"></i><u>74.4 t/s</u></div>
+    </div>
+    <div class="tw rv">
+      <table>
+        <thead><tr><th>Build</th><th>Size</th><th>Perplexity</th><th>HellaSwag</th><th>MMLU</th><th>tg128 t/s</th></tr></thead>
+        <tbody>
+          <tr><td>F16</td><td>64.6 GB</td><td>6.537</td><td>82.5%</td><td>41.5%</td><td>30.4</td></tr>
+          <tr><td>Q8_0</td><td>34.4 GB</td><td>6.533</td><td>83.0%</td><td>41.2%</td><td>52.5</td></tr>
+          <tr><td>Unsloth UD-Q8_K_XL</td><td>45.3 GB</td><td>6.536</td><td>82.5%</td><td>41.3%</td><td>36.4</td></tr>
+          <tr data-a><td>APEX Quality</td><td>21.3 GB</td><td>6.527</td><td>83.0%</td><td>41.2%</td><td>62.3</td></tr>
+          <tr data-a><td>APEX I-Quality</td><td>21.3 GB</td><td>6.552</td><td>83.5%</td><td>41.4%</td><td>63.1</td></tr>
+          <tr data-a><td>APEX Compact</td><td>16.1 GB</td><td>6.783</td><td>82.5%</td><td>40.9%</td><td>69.8</td></tr>
+          <tr data-a><td>APEX Mini</td><td>12.2 GB</td><td>7.088</td><td>81.0%</td><td>41.3%</td><td>74.4</td></tr>
+        </tbody>
+      </table>
+    </div>
+    <p class="mono rv" style="margin-top:.9rem;font-size:.66rem;color:var(--p-dim)">Qwen3.5-35B-A3B on an NVIDIA DGX Spark (GB10). Perplexity on wikitext-2-raw at context 2048. Full methodology in the technical report.</p>
+    <div class="acts rv">
+      <a class="btn" href="https://huggingface.co/collections/mudler/apex-quants-gguf">APEX models on Hugging Face ↗</a>
+      <a class="btn btn--o" href="https://github.com/localai-org/apex-quant">Technical report ↗</a>
+    </div>
+  </div>
+</section>
+
+<!-- CLUSTER -->
+<section class="navy" id="cluster">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">Distributed</p>
+    <h2 class="rv mt1" style="max-width:17ch">Two machines behave like one.</h2>
+    <div class="scene">
+      <div>
+        <div class="st rv"><p class="st__k">01</p><h3>Start a worker</h3>
+          <p>One command on any box. It reports what hardware it has and which backends it can run, then joins the pool.</p></div>
+        <div class="st rv"><p class="st__k">02</p><h3>Work goes where it is cheapest</h3>
+          <p>Requests land on the replica that already holds the model and the matching prefix cache, sized against real free VRAM rather than a guess.</p></div>
+        <div class="st rv"><p class="st__k">03</p><h3>Losing a node is boring</h3>
+          <p>In-flight work reschedules, the model loads somewhere else, and the client never finds out.</p></div>
+      </div>
+      <div class="pin rv">
+        <div class="rig">
+          <div class="nodes" id="nodes">
+            <div class="nd"><b>node-01</b>24 GB · CUDA</div>
+            <div class="nd"><b>node-02</b>16 GB · ROCm</div>
+            <div class="nd"><b>node-03</b>CPU · 64 GB</div>
+          </div>
+          <div class="link"></div>
+          <div class="nodes"><div class="nd" style="grid-column:1/4"><b>router</b>prefix affinity · VRAM aware · autoscaling</div></div>
+        </div>
+      </div>
+    </div>
+  </div>
+</section>
+
+<!-- MODELS -->
+<section class="navy" id="models">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">The gallery</p>
+    <h2 class="rv mt1" style="max-width:20ch">1,585 models. No notebook, no conversion script.</h2>
+    <div class="cards">
+      <a class="cd rv" href="/docs/getting-started/models/"><p class="cd__k">Quantizations</p><h3>201 APEX builds</h3>
+        <p>Every tier of every model we quantize, ranked against the hardware you actually have and installed with one click.</p><span class="cd__go">Browse the gallery →</span></a>
+      <a class="cd rv" href="#"><p class="cd__k">Speech</p><h3>italian-asr</h3>
+        <p>Italian speech recognition trained and published by the team, streaming on a CPU through parakeet.cpp.</p><span class="cd__go">Model card →</span></a>
+      <a class="cd rv" href="/docs/features/text-to-audio/"><p class="cd__k">Voices</p><h3>60 Piper voices</h3>
+        <p>Forty-two languages of text to speech, small enough to run on a Raspberry Pi, installed from the web interface.</p><span class="cd__go">Voice catalogue →</span></a>
+    </div>
+  </div>
+</section>
+
+<!-- NIB -->
+<section class="deepbg" id="nib">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">nib</p>
+    <div class="duo">
+      <div>
+        <h2 class="rv mt1" style="max-width:16ch">An agent you can drop on any box you SSH into.</h2>
+        <p class="lede rv mt2">One Go binary, about 20 MB, no runtime and no daemon. Press <span class="mono" style="color:var(--cyan-hi)">Ctrl+Space</span> anywhere and it opens. Point it at any OpenAI-compatible endpoint, including a model running on your own laptop, and it works. Tool calls go through an approval gate you control, and Claude Code plugins load as they are.</p>
+        <div class="chips rv"><span>Go</span><span>zero dependencies</span><span>MCP</span><span>plugins</span><span>skills</span><span>sub-agents</span></div>
+        <p class="rv mt2"><a class="btn btn--o" href="https://github.com/mudler/nib">nib on GitHub ↗</a></p>
+      </div>
+      <div class="duo__m rv">
+        <figure class="screen" style="margin:0">
+          <figcaption class="screen__bar"><i></i> nib <b>~20 MB, one binary</b></figcaption>
+          <video src="/media/nib.mp4" muted loop playsinline preload="none" data-lazy aria-label="The nib terminal agent running a task"></video>
+        </figure>
+      </div>
+    </div>
+  </div>
+</section>
+
+<!-- PROOF -->
+<section class="navy" id="proof">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">Since March 2023</p>
+    <h2 class="rv mt1" style="max-width:22ch">Forty-eight thousand stars, and still shipping every week.</h2>
+    <div class="trend rv">
+      <img src="/img/trendshift.svg" alt="LocalAI on Trendshift">
+      <p>LocalAI has been <b>trending on GitHub</b> repeatedly since it launched, and it is one of the most starred self-hosted AI projects there is. <b>{{ lang.FormatNumberCustom 0 .Site.Data.stats.contributors }} people</b> have contributed code, <b>{{ lang.FormatNumberCustom 0 .Site.Data.stats.discord }}</b> are in the Discord, and the README is kept translated into <b>eight languages</b> because the users are everywhere.</p>
+    </div>
+    <div class="big rv">
+      <div><b class="tnum" data-count="{{ .Site.Data.stats.stars }}">0</b><span>Stars</span></div>
+      <div><b class="tnum" data-count="{{ .Site.Data.stats.forks }}">0</b><span>Forks</span></div>
+      <div><b class="tnum" data-count="{{ .Site.Data.stats.contributors }}">0</b><span>Contributors</span></div>
+      <div><b class="tnum" data-count="{{ .Site.Data.stats.releases }}">0</b><span>Releases</span></div>
+      <div><b class="tnum" data-count="{{ .Site.Data.stats.discord }}">0</b><span>In Discord</span></div>
+      <div><b class="tnum" data-count="0" data-text="3 yrs">0</b><span>Shipping since</span></div>
+    </div>
+    <div class="tl rv">
+      <div class="tl__t">
+        <div class="tl__i"><p class="tl__d">MAR 2023</p><h4>First commit</h4>
+          <p>An OpenAI-compatible API in front of llama.cpp, so a laptop could answer the same calls as the cloud.</p></div>
+        <div class="tl__i"><p class="tl__d">JUL 2025</p><h4>The core gets small</h4>
+          <p>Every backend moves out of the binary. You install only the engines your models need.</p></div>
+        <div class="tl__i"><p class="tl__d">MAR 2026</p><h4>Agents and a new interface</h4>
+          <p>Native agentic orchestration, a full React rewrite with canvas mode, WebRTC realtime audio.</p></div>
+        <div class="tl__i"><p class="tl__d">APR 2026</p><h4>It becomes a cluster</h4>
+          <p>Distributed mode with VRAM-aware routing, autoscaling, multi-user auth and per-user quotas.</p></div>
+        <div class="tl__i"><p class="tl__d">MAY 2026</p><h4>It sees and hears</h4>
+          <p>Voice recognition, face recognition with liveness, diarization, video generation, drop-in Ollama API.</p></div>
+        <div class="tl__i"><p class="tl__d">JUL 2026</p><h4>Nineteen engines of our own</h4>
+          <p>The native C and C++ ports take over the heavy Python backends, one modality at a time.</p></div>
+      </div>
+    </div>
+    <p class="kicker rv mt3">Where it runs</p>
+    <div class="marks rv">
+      <span>NVIDIA CUDA</span><span>AMD ROCm</span><span>Intel SYCL</span><span>Apple Metal</span><span>Vulkan</span>
+      <span>Jetson</span><span>Raspberry Pi</span><span>x86_64</span><span>ARM64</span><span>Kubernetes</span><span>Docker</span>
+    </div>
+    <div class="langs rv">
+      <span>Deutsch</span><span>Español</span><span>français</span><span>日本語</span>
+      <span>한국어</span><span>Português</span><span>Русский</span><span>中文</span>
+    </div>
+    {{/* The strongest sentence in this section is somebody else's, so it opens
+         the section rather than closing it. Everything below is supporting
+         evidence for what these three people already said. */}}
+    <p class="kicker rv mt3">What other people say</p>
+    <div class="headline">
+      <a class="hq rv" href="https://x.com/ggerganov/status/2065447087311917459">
+        <blockquote>&ldquo;Some cool ggml-based work by @mudler_it recently, make sure to check it out.&rdquo;</blockquote>
+        <p class="hq__w">@ggerganov<b>Georgi Gerganov, author of llama.cpp and ggml</b></p>
+        <span class="hq__go">On X, 2026 &#8599;</span></a>
+      <a class="hq rv" href="https://x.com/badlogicgames/status/2061201400059531729">
+        <blockquote>&ldquo;What a wonderful project: parakeet.cpp. A ggml based parakeet inference pipeline that is 2x faster than my ONNX parakeet pipeline on Apple Silicon.&rdquo;</blockquote>
+        <p class="hq__w">@badlogicgames<b>Mario Zechner, author of pi.agent</b></p>
+        <span class="hq__go">On X, 2026 &#8599;</span></a>
+      <a class="hq rv" href="https://www.linkedin.com/posts/adimargolink_spinoza-saw-all-things-straining-to-become-activity-7468135820076634113-u3Hc">
+        <blockquote>&ldquo;Build something good enough that the community chooses to carry it beyond your reach. This week, Ettore Di Giacinto brought NVIDIA Parakeet to the CPU.&rdquo;</blockquote>
+        <p class="hq__w">Adi Margolin<b>On LinkedIn</b></p>
+        <span class="hq__go">Read the post &#8599;</span></a>
+    </div>
+
+    {{/* Names run as a sentence rather than a grid of pills. A pill wall of
+         employers reads as a customer logo wall, which is a claim we are not
+         making; a sentence keeps it about the people, which is the true one. */}}
+    <p class="kicker rv mt3">Who shows up</p>
+    <h3 class="eco__h rv">{{ lang.FormatNumberCustom 0 .Site.Data.stats.contributors }} people have put code in this repository.</h3>
+    {{- $co := slice }}
+    {{- range .Site.Data.ecosystem.contributors.companies }}{{ $co = $co | append (printf "<b>%s</b>" .name) }}{{ end }}
+    {{- $ac := slice }}
+    {{- range .Site.Data.ecosystem.contributors.academia }}{{ $ac = $ac | append (printf "<b>%s</b>" .) }}{{ end }}
+    <p class="eco__names rv">Some of them do it from a desk at {{ delimit $co ", " " and " | safeHTML }}.
+      Others from labs at {{ delimit $ac ", " " and " | safeHTML }}.</p>
+    <p class="eco__note rv">That is where they work, not who sent them.
+      <a href="{{ .Site.Params.github }}/graphs/contributors">Check out the full contributor list, 200+ and growing &#8599;</a></p>
+
+    {{/* The list runs as a marquee because the count is the argument: any one
+         of these is a weak signal, and the whole moving line is the strong one.
+         It pauses on hover so the links stay usable. */}}
+    <h3 class="eco__h rv mt3">And many projects integrate it</h3>
+    <p class="eco__note rv">Click any name to see where they say so.</p>
+    <div class="reel reel--int rv">
+      <div class="reel__t">
+        {{- range .Site.Data.ecosystem.integrations.projects }}
+        <a href="{{ .url }}">{{ .name }}</a>
+        {{- end }}
+        {{- range .Site.Data.ecosystem.integrations.projects }}
+        <a href="{{ .url }}" tabindex="-1" aria-hidden="true">{{ .name }}</a>
+        {{- end }}
+      </div>
+    </div>
+    <p class="eco__aside rv">Using LocalAI at work? Add your organisation to
+      <a href="{{ .Site.Params.github }}/blob/master/ADOPTERS.md">ADOPTERS.md</a>.</p>
+
+    <h3 class="eco__h rv mt3">Written about elsewhere</h3>
+    <div class="eco eco--press rv">
+      {{- range .Site.Data.ecosystem.press.articles }}
+      <a href="{{ .url }}"><b>{{ .outlet }}</b><span>{{ .title }}</span><em>{{ .what }}</em></a>
+      {{- end }}
+    </div>
+
+    <p class="kicker rv mt3">Built on, integrated with, written about</p>
+    <div class="posts">
+      <a class="post rv" href="https://x.com/sozercan/status/1769769695081546236">
+        <p class="post__h">@sozercan <em>builds on</em></p>
+        <p>AIKit now offers an extensible solution for finetuning LLMs! Thanks to @UnslothAI, you can finetune fast and efficiently. Then, deploy seamlessly with AIKit using @LocalAI_API for an end-to-end solution!</p>
+        <span class="post__go">On X, 2024 &#8599;</span></a>
+      <a class="post rv" href="https://x.com/ivanfioravanti/status/2062526685484851440">
+        <p class="post__h">@ivanfioravanti <em>benchmarks</em></p>
+        <p>An M5 Max with 40 GPU cores just beat an M3 Ultra with 80 on parakeet.cpp. Every model. ~1.7x faster on average, up to ~2x. Half the cores.</p>
+        <span class="post__go">On X, 2026 &#8599;</span></a>
+      <a class="post rv" href="https://x.com/PulumiCorp/status/1794038185061663083">
+        <p class="post__h">@PulumiCorp <em>integration</em></p>
+        <p>Explore how to build and deploy a LLM app using @FlowiseAI and @LocalAI_API with AWS EKS, Pulumi, and TypeScript! Run your models locally or on-prem.</p>
+        <span class="post__go">On X, 2024 &#8599;</span></a>
+      <a class="post rv" href="https://x.com/enricoros/status/1912401037794898354">
+        <p class="post__h">@enricoros <em>ecosystem</em></p>
+        <p>Congrats to @LocalAI_API for launching LocalAGI (Agents), and LocalRecall (Memory). The Local stack is well designed and expanding.</p>
+        <span class="post__go">On X, 2025 &#8599;</span></a>
+      <a class="post rv" href="https://x.com/UniverseAdam/status/1779854715519459432">
+        <p class="post__h">@UniverseAdam <em>in print</em></p>
+        <p>My hardcopies of the official @Raspberry_Pi magazine @TheMagPi have arrived! And they have my Automatic Speech Recognition project based around @NordVPN&#x27;s Meshnet and a self-hosted @LocalAI_API language model inside.</p>
+        <span class="post__go">On X, 2024 &#8599;</span></a>
+      <a class="post rv" href="https://x.com/ivanfioravanti/status/2038141571678212580">
+        <p class="post__h">@ivanfioravanti <em>community</em></p>
+        <p>LocalAI is becoming stronger and better release, after release! Keep pushing @mudler_it and @LocalAI_API</p>
+        <span class="post__go">On X, 2026 &#8599;</span></a>
+      <a class="post rv" href="https://x.com/alepiad/status/1654502947697442816">
+        <p class="post__h">@alepiad <em>early days</em></p>
+        <p>I&#x27;m really excited about the prospect of open-source LLMs. In that respect, take a look at @LocalAI_API, a drop-in replacement for OpenAI&#x27;s API but serving GGML models right on your own infrastructure.</p>
+        <span class="post__go">On X, 2023 &#8599;</span></a>
+      <a class="post rv" href="https://x.com/mattapperson/status/1727390465543041423">
+        <p class="post__h">@mattapperson <em>builds on</em></p>
+        <p>Oh, high there @LocalAI_API, nice to see a terminal based UI for ya! (it&#x27;s a WIP, but just wanted something cleaner then CURL calls)</p>
+        <span class="post__go">On X, 2023 &#8599;</span></a>
+    </div>
+  </div>
+</section>
+
+<!-- BLOG -->
+<section class="deepbg" id="blog">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">From the team</p>
+    <h2 class="rv mt1" style="max-width:22ch">We publish the numbers, including the ones that cost us.</h2>
+    <p class="lede rv mt2">APEX takes a 35B model from 64.6 GB to 12.2 GB, and from 30.4 to 74.4 tokens a second. Perplexity goes from 6.537 to 7.088. That trade is in the post too, with the command that produced it.</p>
+    {{/* Pulled from the posts themselves. The cards used to be hand-written,
+         which is how one of them ended up advertising a post that did not
+         exist, and how all three linked to the index instead of the article. */}}
+    <div class="cards">
+      {{- range first 3 (where .Site.RegularPages "Section" "blog") }}
+      <a class="cd rv" href="{{ .RelPermalink }}">
+        <p class="cd__k">{{ .Date.Format "2 January 2006" }}</p><h3>{{ .Title }}</h3>
+        <p>{{ .Params.summary | truncate 155 }}</p><span class="cd__go">Read the post →</span></a>
+      {{- end }}
+    </div>
+  </div>
+</section>
+
+<!-- START -->
+<section class="navy" id="start">
+  <div class="shell">
+    <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
+    <p class="kicker rv">Get started</p>
+    <div class="duo">
+      <div>
+        <h2 class="rv mt1" style="max-width:14ch">Running in about a minute.</h2>
+        <p class="lede rv mt2">A container on any platform, a DMG on macOS, a binary on Linux, or a chart on Kubernetes. Backends download themselves the first time a model asks for one, so the base install stays small.</p>
+        <div class="acts rv">
+          <a class="btn" href="/docs/getting-started/">Installation guide →</a>
+          <a class="btn btn--o" href="https://discord.gg/uJAeKSAGDy">Join the Discord</a>
+        </div>
+      </div>
+      <div class="rv">
+        <div class="tabs" role="tablist" aria-label="Installation method">
+          <button class="tab" type="button" role="tab" aria-selected="true" data-pane="p-script">Script</button>
+          <button class="tab" type="button" role="tab" aria-selected="false" data-pane="p-docker">Docker</button>
+          <button class="tab" type="button" role="tab" aria-selected="false" data-pane="p-k8s">Kubernetes</button>
+        </div>
+        <div class="term">
+          <div class="term__b"><span id="term-label">bash</span> <button class="cpy" type="button" id="cpy" data-copy="curl -sSL https://localai.io/install.sh | sh">copy</button></div>
+          <div class="pane" id="p-script">
+            <pre><span class="p">$</span> <span class="c">curl -sSL https://localai.io/install.sh | sh</span>
+<span class="o">detecting platform</span> <span class="h">linux/amd64</span>
+<span class="o">fetching latest release binary</span>
+<span class="o">installed to</span> <span class="h">/usr/local/bin/local-ai</span>
+
+<span class="p">$</span> <span class="c">local-ai run qwen3.5-35b-a3b-apex</span>
+<span class="o">pulling backend llama-cpp</span>
+<span class="o">API ready on</span> <span class="h">http://localhost:8080</span>
+<span class="o">model ready ·</span> <span class="h">74.4 tok/s</span></pre>
+          </div>
+          <div class="pane" id="p-docker" hidden>
+            <pre><span class="p">$</span> <span class="c">docker run -p 8080:8080 --name local-ai -ti localai/localai:latest</span>
+<span class="o">starting local-ai</span>
+<span class="o">detected: CPU (AVX-512), 32 GB RAM</span>
+<span class="o">API ready on</span> <span class="h">http://localhost:8080</span>
+
+<span class="o"># Podman works the same way</span>
+<span class="p">$</span> <span class="c">podman run -p 8080:8080 --name local-ai -ti localai/localai:latest</span></pre>
+          </div>
+          <div class="pane" id="p-k8s" hidden>
+            <pre><span class="p">$</span> <span class="c">kubectl apply -f https://localai.io/install/kubernetes.yaml</span>
+<span class="o">deployment.apps/local-ai created</span>
+<span class="o">service/local-ai created</span>
+
+<span class="o"># or with Helm</span>
+<span class="p">$</span> <span class="c">helm install local-ai go-skynet/local-ai</span></pre>
+          </div>
+        </div>
+        <div class="alts"><span>macOS: DMG</span><span>Linux: binary</span><span>Podman</span><span>Helm chart</span><span>Build from source</span></div>
+      </div>
+    </div>
+  </div>
+</section>
+
+</main>
+{{ end }}

website/layouts/partials/footer.html

@@ -0,0 +1,14 @@
+<footer>
+  <div class="shell">
+    <div class="foot">
+      <div>
+        <img src="/img/logo-full.png" alt="LocalAI">
+        <p style="margin-top:1rem;font-size:.88rem;max-width:34ch;color:var(--dim)">AI that runs on the hardware you already own. MIT licensed, built in the open.</p>
+      </div>
+      <div><h4>Product</h4><ul><li><a href="/docs/">Documentation</a></li><li><a href="/engines/">Engines</a></li><li><a href="/blog/">Blog</a></li><li><a href="/docs/getting-started/">Install</a></li></ul></div>
+      <div><h4>Community</h4><ul><li><a href="https://github.com/mudler/LocalAI">GitHub</a></li><li><a href="https://discord.gg/uJAeKSAGDy">Discord</a></li><li><a href="https://twitter.com/LocalAI_API">X</a></li><li><a href="https://huggingface.co/mudler">Hugging Face</a></li></ul></div>
+      <div><h4>Support us</h4><ul><li><a href="https://github.com/sponsors/mudler">Sponsor</a></li><li><a href="https://github.com/mudler/LocalAI/graphs/contributors">Contributors</a></li><li><a href="/docs/">Contributing</a></li></ul></div>
+    </div>
+    <div class="foot__n"><span>© 2026 LocalAI</span><span>MIT licence</span><span><a href="{{ .Site.Params.github }}">Source on GitHub</a></span></div>
+  </div>
+</footer>

website/layouts/partials/head.html

@@ -0,0 +1,26 @@
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <title>{{ if .IsHome }}{{ .Site.Title }} · Make AI run on every machine{{ else }}{{ .Title }} · {{ .Site.Title }}{{ end }}</title>
+  {{ $desc := or .Description .Site.Params.description }}
+  <meta name="description" content="{{ $desc }}">
+  <meta name="author" content="{{ .Site.Params.author }}">
+  {{ if not hugo.IsProduction }}<meta name="robots" content="noindex">{{ end }}
+  <link rel="canonical" href="{{ .Permalink }}">
+
+  <meta property="og:type" content="website">
+  <meta property="og:site_name" content="{{ .Site.Title }}">
+  <meta property="og:title" content="{{ if .IsHome }}Make AI run on every machine{{ else }}{{ .Title }}{{ end }}">
+  <meta property="og:description" content="{{ $desc }}">
+  <meta property="og:url" content="{{ .Permalink }}">
+  <meta property="og:image" content="{{ "img/logo-full.png" | absURL }}">
+  <meta name="twitter:card" content="summary_large_image">
+  <meta name="twitter:site" content="@LocalAI_API">
+
+  <link rel="icon" href="{{ "img/logo-mark.png" | relURL }}">
+  <link rel="preload" as="font" type="font/woff2" href="{{ "fonts/sora-700.woff2" | relURL }}" crossorigin>
+  <link rel="preload" as="font" type="font/woff2" href="{{ "fonts/geist-mono.woff2" | relURL }}" crossorigin>
+  <link rel="stylesheet" href="{{ "css/site.css" | relURL }}">
+  {{ range .Params.extracss }}<link rel="stylesheet" href="{{ printf "css/%s" . | relURL }}">
+  {{ end }}
+</head>

website/layouts/partials/nav.html

@@ -0,0 +1,19 @@
+{{- $home := .Site.Home.RelPermalink -}}
+<header class="top">
+  <a class="brand" href="{{ $home }}"><img src="{{ "img/logo-mark.png" | relURL }}" alt=""> <span>LocalAI</span></a>
+  <nav>
+    {{/* Anchors resolve against the home page so the nav still works from
+         /engines/ and /blog/, not only from the landing page. */}}
+    <a href="{{ $home }}#localai">LocalAI</a>
+    <a href="{{ $home }}#senses">Senses</a>
+    <a href="{{ $home }}#made">Made with it</a>
+    <a href="{{ "engines/" | relURL }}">Engines</a>
+    <a href="{{ $home }}#apex">APEX</a>
+    <a href="{{ "blog/" | relURL }}">Blog</a>
+    <a href="{{ .Site.Params.docsURL }}">Docs</a>
+  </nav>
+  <div class="topright">
+    <a class="pill" href="{{ .Site.Params.github }}">&#9733; {{ lang.FormatNumberCustom 0 .Site.Data.stats.stars }}</a>
+    <a class="go-btn" href="{{ $home }}#start">Install</a>
+  </div>
+</header>