docs: preserve 32 untracked blog drafts

These 32 posts have been sitting untracked in docs/content/blog/ in a local
working tree, never committed to any branch. They are 404 on the live site,
so nothing here changes what is published.

Committing them so they exist in git. During this session a rebase running in
that checkout briefly took the directory out of the working tree, which is
exactly the failure mode untracked files have.

They are clean writing. A no-ai-slop detect pass over all 32 returns zero hits
on every pattern, which is unsurprising since they are short release notes,
median 83 words, in the same register as what-landed-in-localai-4-8.md.

Not resolved here, needs a decision before this is merged:

- They use TOML +++ front matter with an explicit url = "/blog/<slug>/", while
  the five live posts in website/content/blog/ use YAML --- and no url
  override. These look like leftovers from before the site split in 94d5affce,
  when the whole site was under docs/ and the blog lived at /blog/.
- docs/hugo.toml mounts content wholesale, so merging this to master would
  feed 32 new pages into the docs site build. I could not verify the resulting
  URLs because the docs theme module is not available in a bare worktree.
- If they are meant to be published they probably belong in
  website/content/blog/ with the front matter converted. If they are not, they
  want draft = true.

I have left the content byte-identical to the working-tree originals rather
than guess at any of that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.agents/api-endpoints-and-auth.md

@@ -304,7 +304,9 @@ React pages that want to filter the ModelSelector by capability import this symb
 
 ### 4. `docs/content/` (user-facing documentation)
 
-A new capability deserves its own page under `docs/content/features/`, plus cross-links from related features and an entry in `docs/content/whats-new.md`. See the pattern used by `face-recognition.md` / `object-detection.md`.
+A new capability deserves its own page under `docs/content/features/`, plus cross-links from related features. See the pattern used by `face-recognition.md` / `object-detection.md`.
+
+Announcing it is the release's job, not this page's: the capability gets covered in the release blog post under `website/content/blog/`. See [preparing-a-release.md](preparing-a-release.md). `docs/content/whats-new.md` is only a pointer at the blog and GitHub Releases, so there is nothing to add there.
 
 ## Path protection rules
 
@@ -334,7 +336,7 @@ When adding a new endpoint:
 - [ ] Swagger block on the handler: `@Summary`, `@Tags`, `@Param`, `@Success`, `@Router`
 - [ ] If new capability area (new swagger tag): entry in `instructionDefs` in `core/http/endpoints/localai/api_instructions.go` + test count bumped in `api_instructions_test.go`
 - [ ] If new `FLAG_*` usecase flag: matching `CAP_*` symbol exported from `core/http/react-ui/src/utils/capabilities.js`
-- [ ] `docs/content/features/<feature>.md` created; cross-links from related feature pages; entry in `docs/content/whats-new.md`
+- [ ] `docs/content/features/<feature>.md` created; cross-links from related feature pages; capability covered in the release blog post (see [preparing-a-release.md](preparing-a-release.md))
 
 **Quality**
 - [ ] Error responses use `schema.ErrorResponse` format (or `echo.NewHTTPError` with a mapped gRPC status — see the `mapBackendError` helper in `core/http/endpoints/localai/images.go`)

README.md

@@ -195,7 +195,7 @@ For more details, see the [Getting Started guide](https://localai.io/basics/gett
 - **August 2025**: MLX, MLX-VLM, Diffusers, llama.cpp now supported on Apple Silicon
 - **July 2025**: All backends migrated outside the main binary — [lightweight, modular architecture](https://github.com/mudler/LocalAI/releases/tag/v3.2.0)
 
-For older news and full release notes, see [GitHub Releases](https://github.com/mudler/LocalAI/releases) and the [News page](https://localai.io/basics/news/).
+For older news and full release notes, see [GitHub Releases](https://github.com/mudler/LocalAI/releases) and the [blog](https://localai.io/blog/).
 
 ## Features
 
@@ -260,7 +260,7 @@ We also maintain [apex-quant](https://github.com/localai-org/apex-quant), a per-
 - [Kubernetes installation](https://localai.io/basics/getting_started/#run-localai-in-kubernetes)
 - [Integrations & community projects](https://localai.io/docs/integrations/)
 - [Installation video walkthrough](https://www.youtube.com/watch?v=cMVNnlqwfw4)
-- [Media & blog posts](https://localai.io/basics/news/#media-blogs-social)
+- [Blog: release write-ups, benchmarks and engineering notes](https://localai.io/blog/)
 - [Examples](https://github.com/mudler/LocalAI-examples) — including the [realtime voice assistant demo](https://github.com/localai-org/localai-realtime-demo) (Go client for the Realtime API with tool calling)
 
 ## Team

backend/cpp/llama-cpp/Makefile

@@ -1,5 +1,5 @@
 
-LLAMA_VERSION?=a7a6d0d269c896218b6c78e0933bd6a17519d3f6
+LLAMA_VERSION?=221f0f6356efe2260023208365705ec5d5a7c8f5
 LLAMA_REPO?=https://github.com/ggerganov/llama.cpp
 
 CMAKE_ARGS?=

core/http/endpoints/mcp/localai_assistant_test.go

@@ -154,22 +154,6 @@ func (stubClient) GetRouterDecisions(_ context.Context, _ localaitools.RouterDec
 	return []localaitools.RouterDecision{}, nil
 }
 
-func (stubClient) ListScheduling(_ context.Context) ([]localaitools.ModelSchedulingConfig, error) {
-	return []localaitools.ModelSchedulingConfig{}, nil
-}
-
-func (stubClient) GetScheduling(_ context.Context, _ string) (*localaitools.ModelSchedulingConfig, error) {
-	return &localaitools.ModelSchedulingConfig{}, nil
-}
-
-func (stubClient) SetScheduling(_ context.Context, _ localaitools.SetSchedulingRequest) (*localaitools.ModelSchedulingConfig, error) {
-	return &localaitools.ModelSchedulingConfig{}, nil
-}
-
-func (stubClient) DeleteScheduling(_ context.Context, _ string) error {
-	return nil
-}
-
 var _ = Describe("LocalAIAssistantHolder", func() {
 	var ctx context.Context
 

docs/content/blog/2024-04-24-reranker-api.md

@@ -0,0 +1,10 @@
++++
+title = "Reranker API"
+date = 2024-04-24
+description = "A new reranker backend implementing the Jina rerankers API."
+url = "/blog/reranker-api/"
++++
+
+A new reranker backend lands, implementing the Jina rerankers API, in [PR #2121](https://github.com/mudler/LocalAI/pull/2121).
+
+See [Reranker]({{% relref "features/reranker" %}}).

docs/content/blog/2024-05-14-distributed-and-p2p-inferencing.md

@@ -0,0 +1,13 @@
++++
+title = "Distributed and decentralized P2P inferencing"
+date = 2024-05-14
+description = "Distributed llama.cpp inferencing, followed by fully decentralized peer-to-peer inference."
+url = "/blog/distributed-and-p2p-inferencing/"
++++
+
+Two changes that set up everything LocalAI later built on top of:
+
+- [Distributed llama.cpp inferencing](https://github.com/mudler/LocalAI/pull/2324), splitting a model across machines.
+- [Totally decentralized, private, distributed peer-to-peer inference](https://github.com/mudler/LocalAI/pull/2343).
+
+See [Distributed inferencing]({{% relref "features/distributed_inferencing" %}}).

docs/content/blog/2024-08-02-p2p-federation-and-swarms.md

@@ -0,0 +1,15 @@
++++
+title = "P2P dashboard, federated mode and AI swarms"
+date = 2024-08-02
+description = "A P2P dashboard, federation, AI swarms, global community pools, FLUX-1 support and the P2P Explorer."
+url = "/blog/p2p-federation-and-swarms/"
++++
+
+The peer-to-peer work matured over July and August:
+
+- [A P2P dashboard, federated mode and AI swarms](https://github.com/mudler/LocalAI/pull/2723).
+- [Global community pools](https://github.com/mudler/LocalAI/issues/3113), for sharing federated instances and workers.
+- FLUX-1 support.
+- The [P2P Explorer](https://explorer.localai.io).
+
+See [Distributed inferencing]({{% relref "features/distributed_inferencing" %}}).

docs/content/blog/2024-10-01-examples-moved-out.md

@@ -0,0 +1,8 @@
++++
+title = "Examples move to LocalAI-examples"
+date = 2024-10-01
+description = "The examples directory leaves the main repository and gets its own home."
+url = "/blog/examples-moved-out/"
++++
+
+The examples have moved out of the main repository into [LocalAI-examples](https://github.com/mudler/LocalAI-examples), where they can be versioned and maintained independently of the runtime.

docs/content/blog/2024-11-20-vad-and-bark-cpp.md

@@ -0,0 +1,9 @@
++++
+title = "Voice Activity Detection and bark.cpp"
+date = 2024-11-20
+description = "Silero-based Voice Activity Detection, plus a bark.cpp backend for audio generation."
+url = "/blog/vad-and-bark-cpp/"
++++
+
+- [Voice Activity Detection](https://github.com/mudler/LocalAI/pull/4204), via a Silero VAD backend. See [Voice activity detection]({{% relref "features/voice-activity-detection" %}}).
+- [A bark.cpp backend](https://github.com/mudler/LocalAI/pull/4287) for audio generation.

docs/content/blog/2024-12-03-stablediffusion-cpp-backend.md

@@ -0,0 +1,10 @@
++++
+title = "stablediffusion.cpp backend (ggml)"
+date = 2024-12-03
+description = "A ggml-based stablediffusion.cpp backend for image generation."
+url = "/blog/stablediffusion-cpp-backend/"
++++
+
+A ggml-based `stablediffusion.cpp` backend lands for image generation, in [PR #4289](https://github.com/mudler/LocalAI/pull/4289).
+
+See [Image generation]({{% relref "features/image-generation" %}}).

docs/content/blog/2025-07-24-modular-backend-architecture.md

@@ -0,0 +1,12 @@
++++
+title = "Backends move outside the main binary"
+date = 2025-07-24
+description = "All backends migrate out of the main binary, leaving a lightweight modular core that pulls engines on demand."
+url = "/blog/modular-backend-architecture/"
++++
+
+All backends have been migrated outside the main binary. The core stays small, and each backend is an isolated service installed on demand.
+
+This is the architecture LocalAI still runs on: install, update or remove engines independently, and mix CPU, NVIDIA, AMD, Intel, Apple Silicon, Vulkan and Jetson in one deployment.
+
+See [Backends]({{% relref "features/backends" %}}) and the [v3.2.0 release notes](https://github.com/mudler/LocalAI/releases/tag/v3.2.0).

docs/content/blog/2025-08-12-apple-silicon-backends.md

@@ -0,0 +1,10 @@
++++
+title = "MLX, MLX-VLM, Diffusers and llama.cpp on Apple Silicon"
+date = 2025-08-12
+description = "Apple Silicon gains first-class backend coverage."
+url = "/blog/apple-silicon-backends/"
++++
+
+MLX, MLX-VLM, Diffusers and llama.cpp are now supported on Apple Silicon, giving Mac users the same backend choice available elsewhere.
+
+Released as part of [v3.4.0](https://github.com/mudler/LocalAI/releases/tag/v3.4.0).

docs/content/blog/2025-09-03-launcher-and-extended-backends.md

@@ -0,0 +1,13 @@
++++
+title = "New launcher, extended backend support, MLX-Audio and WAN 2.2"
+date = 2025-09-03
+description = "A desktop launcher for macOS and Linux, wider backend coverage for Mac and Nvidia L4T, MLX-Audio and WAN 2.2."
+url = "/blog/launcher-and-extended-backends/"
++++
+
+- A new [launcher app](https://github.com/mudler/LocalAI/pull/6127) for macOS and Linux, so LocalAI can be started and managed without the terminal.
+- Extended backend support for Mac and Nvidia L4T.
+- MLX-Audio.
+- WAN 2.2.
+
+Released as part of [v3.5.0](https://github.com/mudler/LocalAI/releases/tag/v3.5.0).

docs/content/blog/2025-10-05-mcp-support.md

@@ -0,0 +1,10 @@
++++
+title = "Model Context Protocol (MCP) support"
+date = 2025-10-05
+description = "Agentic capabilities through MCP, with a new chat/completion endpoint that can call MCP tools."
+url = "/blog/mcp-support/"
++++
+
+LocalAI gains [Model Context Protocol](https://modelcontextprotocol.io) support for agentic capabilities, through [a new chat/completion endpoint](https://github.com/mudler/LocalAI/pull/6381) that can reach MCP tools, plus [a UI toggle to enable it](https://github.com/mudler/LocalAI/pull/6400).
+
+See [MCP]({{% relref "features/mcp" %}}).

docs/content/blog/2025-11-24-import-models-via-url-and-chat-history.md

@@ -0,0 +1,11 @@
++++
+title = "Import models via URL, multiple chats and history"
+date = 2025-11-24
+description = "Point LocalAI at a model URL to import it, and keep several chat threads with their history in the UI."
+url = "/blog/import-models-via-url-and-chat-history/"
++++
+
+Two usability changes:
+
+- [Import models via URL](https://github.com/mudler/LocalAI/pull/7245). Paste a model URL and LocalAI handles the download and configuration.
+- [Multiple chats and history](https://github.com/mudler/LocalAI/pull/7325) in the UI, so conversations persist and can run in parallel.

docs/content/blog/2025-12-16-memory-reclaimer-and-multi-gpu-fitting.md

@@ -0,0 +1,12 @@
++++
+title = "Dynamic memory reclaimer, multi-GPU fitting and Vibevoice"
+date = 2025-12-16
+description = "Reclaim GPU memory from idle models, fit llama.cpp models across multiple GPUs automatically, and generate long-form speech with Vibevoice."
+url = "/blog/memory-reclaimer-and-multi-gpu-fitting/"
++++
+
+Three additions this month:
+
+- [A dynamic memory resource reclaimer](https://github.com/mudler/LocalAI/pull/7583), which frees GPU memory held by idle models.
+- [Automatic multi-GPU model fitting for llama.cpp](https://github.com/mudler/LocalAI/pull/7584), so a model too large for one device is split across several without hand-tuning.
+- [The Vibevoice backend](https://github.com/mudler/LocalAI/pull/7494) for long-form speech.

docs/content/blog/2026-01-18-localai-3-10-0.md

@@ -0,0 +1,17 @@
++++
+title = "LocalAI 3.10.0"
+date = 2026-01-18
+description = "Anthropic API support, the Open Responses API, video and image generation with LTX-2, unified GPU backends, tool streaming, Moonshine and Pocket-TTS."
+url = "/blog/localai-3-10-0/"
++++
+
+LocalAI 3.10.0 is out.
+
+- Anthropic API support.
+- The Open Responses API.
+- Video and image generation with LTX-2.
+- Unified GPU backends.
+- Tool streaming.
+- Moonshine and Pocket-TTS.
+
+[Full release notes](https://github.com/mudler/LocalAI/releases/tag/v3.10.0).

docs/content/blog/2026-02-05-realtime-api-and-ace-step.md

@@ -0,0 +1,11 @@
++++
+title = "Realtime API and ACE-Step 1.5"
+date = 2026-02-05
+description = "Audio-to-audio with tool calling through the Realtime API, plus ACE-Step 1.5 music generation."
+url = "/blog/realtime-api-and-ace-step/"
++++
+
+Two additions this month:
+
+- [The Realtime API for audio-to-audio with tool calling](https://github.com/mudler/LocalAI/pull/6245). See [Realtime API]({{% relref "features/openai-realtime" %}}).
+- [ACE-Step 1.5 support](https://github.com/mudler/LocalAI/pull/8396) for music generation.

docs/content/blog/2026-03-14-localai-4-0-0.md

@@ -0,0 +1,16 @@
++++
+title = "LocalAI 4.0.0: native agentic orchestration"
+date = 2026-03-14
+description = "The Agenthub community hub, a full React UI rewrite with Canvas mode, MCP Apps with tool streaming, WebRTC realtime audio, and MLX-distributed."
+url = "/blog/localai-4-0-0/"
++++
+
+LocalAI 4.0.0 brings agentic orchestration into the core.
+
+- Native agentic orchestration, with the new [Agenthub](https://agenthub.localai.io) community hub.
+- A full React UI rewrite, including Canvas mode.
+- [MCP Apps and client-side MCP](https://github.com/mudler/LocalAI/pull/8947) with tool streaming.
+- [WebRTC realtime audio](https://github.com/mudler/LocalAI/pull/8790).
+- [MLX-distributed](https://github.com/mudler/LocalAI/pull/8801).
+
+[Full release notes](https://github.com/mudler/LocalAI/releases/tag/v4.0.0).

docs/content/blog/2026-04-02-localai-4-1-0.md

@@ -0,0 +1,17 @@
++++
+title = "LocalAI 4.1.0: LocalAI becomes a control tower"
+date = 2026-04-02
+description = "Distributed cluster mode with VRAM-aware routing and autoscaling, a multi-user platform with OIDC, per-user quotas, in-UI fine-tuning, and a visual pipeline editor."
+url = "/blog/localai-4-1-0/"
++++
+
+LocalAI 4.1.0 turns LocalAI into a control tower rather than a single inference server.
+
+- Distributed cluster mode, with VRAM-aware smart routing and autoscaling.
+- A multi-user platform with OIDC and API keys.
+- Per-user quotas with predictive analytics.
+- In-UI fine-tuning with TRL, including automatic export to GGUF.
+- An on-the-fly quantization backend.
+- A visual pipeline editor.
+
+[Full release notes](https://github.com/mudler/LocalAI/releases/tag/v4.1.0).

docs/content/blog/2026-04-22-face-recognition-backend.md

@@ -0,0 +1,20 @@
++++
+title = "Face recognition backend"
+date = 2026-04-22
+description = "insightface-powered 1:1 verification, 1:N identification, face embedding, detection and demographic analysis."
+url = "/blog/face-recognition-backend/"
++++
+
+A new face recognition backend, powered by `insightface`, covering:
+
+- 1:1 verification
+- 1:N identification
+- Face embedding
+- Face detection
+- Demographic analysis
+
+It ships with two model options: the non-commercial `buffalo_l`, and an Apache 2.0 alternative from the OpenCV Zoo.
+
+See [Face recognition]({{% relref "features/face-recognition" %}}). Shipped in [PR #9480](https://github.com/mudler/LocalAI/pull/9480).
+
+The engine was later rewritten from scratch in C++/ggml: see [Native biometric backends]({{% relref "blog/2026-06-28-native-biometric-backends" %}}).

docs/content/blog/2026-05-04-audio-transform.md

@@ -0,0 +1,19 @@
++++
+title = "Audio Transform"
+date = 2026-05-04
+description = "A generic audio-in / audio-out endpoint with an optional reference signal. First implementation: LocalVQE, a joint AEC, noise suppression and dereverberation engine."
+url = "/blog/audio-transform/"
++++
+
+Audio Transform is a generic audio-in / audio-out endpoint, with an optional reference signal for tasks that need one.
+
+The first implementation is [LocalVQE](https://github.com/localai-org/LocalVQE), a C++ backend doing joint acoustic echo cancellation, noise suppression and dereverberation in a DeepVQE-style model.
+
+Both call styles are supported:
+
+- Batch, via `POST /audio/transformations`.
+- Bidirectional streaming, via the `/audio/transformations/stream` WebSocket.
+
+Studio gains a "Transform" tab with synchronized waveform players for the input, reference and output signals.
+
+See [Audio transform]({{% relref "features/audio-transform" %}}). Shipped in [PR #9640](https://github.com/mudler/LocalAI/pull/9640).

docs/content/blog/2026-05-05-speaker-diarization.md

@@ -0,0 +1,17 @@
++++
+title = "Speaker diarization"
+date = 2026-05-05
+description = "A /v1/audio/diarization endpoint returning who spoke when, backed by sherpa-onnx and vibevoice-cpp."
+url = "/blog/speaker-diarization/"
++++
+
+`POST /v1/audio/diarization` is a new endpoint that returns "who spoke when" as a list of segments.
+
+Two backends serve it:
+
+- `sherpa-onnx` for pure diarization, combining pyannote-3.0, speaker embeddings and clustering.
+- `vibevoice-cpp` for diarization bundled with long-form ASR.
+
+Responses are available as `json`, `verbose_json` or `rttm`.
+
+See [Audio diarization]({{% relref "features/audio-diarization" %}}). Shipped in [PR #9654](https://github.com/mudler/LocalAI/pull/9654).

docs/content/blog/2026-05-11-localai-4-2-0.md

@@ -0,0 +1,120 @@
++++
+title = "LocalAI 4.2.0: who spoke when, and whose face is that"
+date = 2026-05-11
+description = "Speaker diarization, voice and face recognition, and an Ollama-compatible API."
+url = "/blog/localai-4-2-0/"
++++
+
+![Diarization: segment, embed, and cluster into speaker-labelled segments](/images/diagrams/diarization-pipeline.png)
+
+You record an hour of standup, run it through Whisper, and get back one long wall of text. Every word is correct. You still have no idea who said any of them, so you end up scrubbing through the audio with the transcript open in another window, guessing at voices.
+
+4.2.0 is mostly about that class of problem. Audio and images carry more than "here are the words" or "here is a picture", and until now LocalAI had nowhere to put the rest of it.
+
+Enough chitchat, let's look at what's in it.
+
+## Who spoke when
+
+There's a new `/v1/audio/diarization` endpoint, shaped like `/v1/audio/transcriptions` so your existing multipart code mostly carries over:
+
+```bash
+curl http://localhost:8080/v1/audio/diarization \
+  -H "Content-Type: multipart/form-data" \
+  -F file="@meeting.wav" \
+  -F model="vibevoice-cpp-asr" \
+  -F num_speakers=3
+```
+
+```json
+{
+  "task": "diarize",
+  "duration": 12.34,
+  "num_speakers": 2,
+  "segments": [
+    {"id": 0, "speaker": "SPEAKER_00", "label": "0", "start": 0.00, "end": 2.34},
+    {"id": 1, "speaker": "SPEAKER_01", "label": "1", "start": 2.34, "end": 4.10}
+  ]
+}
+```
+
+Two backends serve it. [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx) does pure diarization (pyannote-3.0 segmentation, a speaker-embedding extractor, then clustering) and never transcribes, so you don't pay for ASR you didn't ask for. `vibevoice-cpp` emits speaker-labelled segments as a by-product of its long-form ASR pass, so with `include_text=true` you get a transcript per segment for free! `response_format` gives you `json`, `verbose_json`, or `rttm` if you want to feed the output to `dscore`.
+
+One thing to know before you build on it: `SPEAKER_00` is local to a single request. Run the same meeting twice and the numbering can come out differently, and nothing promises that `SPEAKER_00` in Monday's recording is the same human as `SPEAKER_00` in Tuesday's. If you need identity across files, pair it with `/v1/voice/embed` and keep your own embedding store. Which brings me to..
+
+## Voices and faces
+
+`/v1/voice/*` is new: verify (are these two clips the same person?), identify (which of my enrolled speakers is this?), embed (give me the vector, I'll do the rest myself), and analyze (age, gender, emotion).
+
+```bash
+local-ai models install speechbrain-ecapa-tdnn
+
+curl -sX POST http://localhost:8080/v1/voice/verify \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "speechbrain-ecapa-tdnn",
+    "audio1": "https://example.com/alice_1.wav",
+    "audio2": "https://example.com/alice_2.wav"
+  }'
+```
+
+```json
+{"verified": true, "distance": 0.18, "threshold": 0.25}
+```
+
+The default threshold is around 0.25 for ECAPA-TDNN, and it moves per engine, so pass `threshold` explicitly if you swap the model out.
+
+`/v1/face/*` does the same thing for faces, plus detection and demographics, and 4.2.0 adds antispoofing. Holding a printed photo or a phone screen up to the camera is the oldest attack on face auth there is, and the liveness check rejects it.
+
+Some honest limits. Liveness is an arms race and this is not bank-grade. The demographic heads emit confident-looking numbers for age and emotion that you should read as a rough signal and not as a fact about a person. And the default `insightface` buffalo packs are released for non-commercial research use only, so if you're shipping this in a product, pick the OpenCV Zoo entry instead. That's in the docs, but people skip docs, so it's here too.
+
+The samples never leave your machine, which is the part I actually care about. They go from your process to the backend running next to it and nowhere else. Doing biometrics against somebody else's cloud API always felt like the worst possible trade.
+
+## Point your ollama client at LocalAI
+
+```sh
+OLLAMA_HOST=http://localhost:8080 ollama run qwen3
+```
+
+LocalAI answers the Ollama API now, so a tool that only ever learned to talk to Ollama keeps working with no code change on your side. `/api/chat`, `/api/generate`, `/api/embed`, `/api/tags`, `/api/show`, `/api/ps` and `/api/version` all land on the engine you were already running, and your existing `/v1/*` clients are untouched.
+
+There's no `/api/pull` in there. Models come from the LocalAI gallery or from a URL you hand it, so `ollama run` against something you haven't installed yet won't go and fetch it for you.
+
+## Video, and a UI repaint
+
+`stable-diffusion.ggml` generates video now! There are gallery entries for Wan 2.1 FLF2V 14B 720P and Wan i2v 720p, including first-last-frame interpolation.
+
+The React UI got a long cycle of work. The chat is redesigned, the palette moved to Nord, and there's i18n across English, Italiano, Español, Deutsch and 简体中文. You can brand your instance too - name, tagline, logo, favicon - and the login page, sidebar, footer and browser tab all pick it up. Handy if you run LocalAI for a team and would rather it didn't look like somebody's side project.
+
+The model config editor is interactive now, with autocomplete over known fields and live validation, and it renames the file on save so you stop accumulating three copies of the same config.
+
+## Eleven new backends
+
+sglang, ik-llama.cpp, TurboQuant, sam.cpp, Kokoros, qwen3tts.cpp, tinygrad-multimodal (experimental, don't build anything load-bearing on it yet), vibevoice.cpp, LocalVQE, insightface, and voice-rec.
+
+vLLM reached feature parity with llama.cpp in this cycle. The full `AsyncEngineArgs` surface is exposed as a generic YAML map, and tensor-parallel distributed workers let a single model span nodes. There are CUDA 13 builds for vLLM, vLLM-omni and sglang, plus L4T arm64 for Jetson-class boards.
+
+## The unglamorous half
+
+Most of the 279 pull requests here are not features. A sample of what actually went in:
+
+- llama.cpp renamed its `common` target to `llama-common`, which broke the TurboQuant build until the detection was fixed.
+- ik-llama.cpp needed a patch to `clip.cpp` for the new `ggml_quantize_chunk` signature, plus adapting to the `common_grammar` struct in `sampling.h`.
+- `mlx-vlm` is pinned to v0.4.4 to unblock CUDA builds.
+- vLLM dropped the flash-attn wheel to avoid a torch 2.10 ABI mismatch.
+- Whisper transcriptions can be cancelled by the client, through the ggml `abort_callback`, so aborting a request frees the GPU instead of letting it run to completion in the background.
+- faster-whisper emits word-level timestamps.
+- gfx1151 (Strix Halo / Ryzen AI MAX) works, with `AMDGPU_TARGETS` exposed as a build-arg.
+
+On the security side: an unsafe `sprintf()` came out of the C++ grpc-server, env-supplied API keys are stripped from Settings API requests before they get persisted so they can't leak back out through the config, and deleting a user on PostgreSQL cascades across everything they owned instead of leaving orphaned rows behind.
+
+Distributed mode got a hardening pass. Round-robin across replicas of the same model, "Upgrade All" scoped to the nodes that actually have the backend installed, NATS `backend.upgrade` split off from install, and correct VRAM/RAM reporting on NVIDIA unified-memory hosts.
+
+## Thanks!
+
+This one had a lot of hands on it. Thanks to @richiejp for the model config editor, Kokoros and a pile of build fixes, @Anai-Guo, @russell, @leinasi2014, @keithmattix for gfx1151, @orbisai0security and @SAY-5 for the security work, @walcz-de, @thelittlefireman, @sec171, @pjbrzozowski, @mvanhorn, @arteven, @Dennisadira, @eglia, @arbrick, @neurocis and @ER-EPR.
+
+If you're wiring up diarization or the voice endpoints and get stuck, open an issue or reach out, I'm genuinely happy to help you get it working. And if LocalAI is useful to you, consider [donating](https://github.com/sponsors/mudler) or just telling somebody about it. The more the merrier!
+
+[Full release notes](https://github.com/mudler/LocalAI/releases/tag/v4.2.0). See [Speaker diarization]({{% relref "features/audio-diarization" %}}), [Voice recognition]({{% relref "features/voice-recognition" %}}) and [Face recognition]({{% relref "features/face-recognition" %}}).
+
+Cheers!

docs/content/blog/2026-05-24-localai-4-3-0.md

@@ -0,0 +1,15 @@
++++
+title = "LocalAI 4.3.0"
+date = 2026-05-24
+description = "llama.cpp prompt cache on by default, keyless cosign signing of backend images, per-key and per-user usage attribution, and Distributed v3."
+url = "/blog/localai-4-3-0/"
++++
+
+LocalAI 4.3.0 is out.
+
+- [Prompt cache on by default for llama.cpp](https://github.com/mudler/LocalAI/pull/9925). Repeated system prompts collapse from minutes to seconds.
+- [Keyless cosign signing of backend OCI images](https://github.com/mudler/LocalAI/pull/9823).
+- [Per-API-key and per-user usage attribution](https://github.com/mudler/LocalAI/pull/9920).
+- Distributed v3, with [per-request replica routing](https://github.com/mudler/LocalAI/pull/9968).
+
+[Full release notes](https://github.com/mudler/LocalAI/releases/tag/v4.3.0).

docs/content/blog/2026-06-11-realtime-voice-assistant-demo.md

@@ -0,0 +1,12 @@
++++
+title = "Realtime voice assistant demo and pipeline streaming"
+date = 2026-06-11
+description = "A tiny Go client for the Realtime API with a full talk-back loop and tool calling, plus streaming of the realtime pipeline stages."
+url = "/blog/realtime-voice-assistant-demo/"
++++
+
+The new [realtime voice assistant demo](https://github.com/localai-org/localai-realtime-demo) is a small Go client for the Realtime API with a complete talk-back voice loop and tool calling. It is intended as a reference you can read end to end.
+
+On the server side, two supporting changes landed: [streaming of the realtime LLM, TTS and transcription pipeline stages](https://github.com/mudler/LocalAI/pull/10176), and [configurable WebRTC ICE candidates](https://github.com/mudler/LocalAI/pull/10231).
+
+See [Realtime API]({{% relref "features/openai-realtime" %}}).

docs/content/blog/2026-06-12-distributed-mode-hardening.md

@@ -0,0 +1,16 @@
++++
+title = "Distributed mode hardening"
+date = 2026-06-12
+description = "Prefix-cache-aware routing, a production-ready request router, ds4 layer-split inference, NATS JWT auth with TLS/mTLS, and resumable uploads."
+url = "/blog/distributed-mode-hardening/"
++++
+
+Distributed mode picked up a round of production hardening:
+
+- [Prefix-cache-aware routing](https://github.com/mudler/LocalAI/pull/10071), so requests sharing a prompt prefix land on the replica that already holds it.
+- [A production-ready request router with auto-sized embedding and rerank batches](https://github.com/mudler/LocalAI/pull/10104).
+- [ds4 layer-split distributed inference](https://github.com/mudler/LocalAI/pull/10098).
+- [NATS JWT auth plus TLS/mTLS](https://github.com/mudler/LocalAI/pull/10159).
+- [Resumable file uploads](https://github.com/mudler/LocalAI/pull/10109).
+
+See [Distributed inferencing]({{% relref "features/distributed_inferencing" %}}).

docs/content/blog/2026-06-12-new-backends-and-models.md

@@ -0,0 +1,15 @@
++++
+title = "New backends and models: locate-anything.cpp, Ideogram4, Gemma 4"
+date = 2026-06-12
+description = "Open-vocabulary object detection via ggml, Ideogram4 image generation, llama.cpp video input, and the Gemma 4 QAT family with MTP pairs."
+url = "/blog/new-backends-and-models-june-2026/"
++++
+
+A batch of new capability this month:
+
+- [locate-anything.cpp](https://github.com/mudler/LocalAI/pull/10264) for open-vocabulary object detection via ggml.
+- [Ideogram4 image generation](https://github.com/mudler/LocalAI/pull/10201) in `stablediffusion-ggml`.
+- [llama.cpp video input](https://github.com/mudler/LocalAI/pull/10216).
+- [The Gemma 4 QAT family with MTP speculative-decoding pairs](https://github.com/mudler/LocalAI/pull/10215).
+
+Plus two usability additions: an [interactive CLI chat mode](https://github.com/mudler/LocalAI/pull/10226) and [RAG source citations in agent responses](https://github.com/mudler/LocalAI/pull/10228).

docs/content/blog/2026-06-13-speech-push-parakeet-crispasr-piper.md

@@ -0,0 +1,17 @@
++++
+title = "A big speech push: parakeet.cpp, CrispASR and 60 Piper voices"
+date = 2026-06-13
+description = "Segment timestamps, multilingual streaming, dynamic batching and CUDA graphs for parakeet.cpp, plus a new ASR/TTS backend and a large Piper voice drop."
+url = "/blog/speech-push-parakeet-crispasr-piper/"
++++
+
+A concentrated round of speech work landed this month.
+
+[parakeet.cpp](https://github.com/mudler/parakeet.cpp), our ASR engine, gained:
+
+- [NeMo-faithful segment timestamps](https://github.com/mudler/LocalAI/pull/10207)
+- [a multilingual streaming Nemotron-3.5 model](https://github.com/mudler/LocalAI/pull/10199)
+- [dynamic batching for concurrent transcription](https://github.com/mudler/LocalAI/pull/10112)
+- [CUDA graphs](https://github.com/mudler/LocalAI/pull/10273)
+
+Alongside it, the new [CrispASR backend](https://github.com/mudler/LocalAI/pull/10099) adds multi-architecture ASR and TTS, and [60 Piper TTS voices across 42 languages](https://github.com/mudler/LocalAI/pull/10296) land in the gallery, together with [per-request TTS instructions and parameters](https://github.com/mudler/LocalAI/pull/10172).

docs/content/blog/2026-06-18-pii-analyze-redact-api.md

@@ -0,0 +1,15 @@
++++
+title = "PII analyze and redact API"
+date = 2026-06-18
+description = "The PII detection pipeline becomes a standalone service, callable without routing a chat request through the middleware."
+url = "/blog/pii-analyze-redact-api/"
++++
+
+The PII detection pipeline (NER plus restricted-regex pattern tiers) is now reachable directly, without routing a chat request through the middleware:
+
+- `POST /api/pii/analyze` returns the detected entity spans.
+- `POST /api/pii/redact` returns the sanitised text, or `400 pii_blocked`.
+
+Events also gain an `origin` field (`middleware`, `proxy`, `pii_analyze`, `pii_redact`), so `/api/pii/events` can be filtered by which surface produced them.
+
+See [Middleware]({{% relref "operations/middleware" %}}#analyze--redact-api). Shipped in [PR #10360](https://github.com/mudler/LocalAI/pull/10360).

docs/content/blog/2026-06-22-sound-classification.md

@@ -0,0 +1,12 @@
++++
+title = "Sound classification with ced.cpp"
+date = 2026-06-22
+description = "A new /v1/audio/classification endpoint for audio tagging, returning scored AudioSet labels."
+url = "/blog/sound-classification/"
++++
+
+`POST /v1/audio/classification` is a new endpoint for audio tagging and sound-event classification. It returns scored [AudioSet](https://research.google.com/audioset/) labels: baby cry, glass breaking, alarms, and several hundred others.
+
+It is backed by [ced.cpp](https://github.com/localai-org/ced.cpp), a 527-class AudioSet tagger ported to ggml by the LocalAI team.
+
+See [Audio classification]({{% relref "features/audio-classification" %}}). Shipped in [PR #10425](https://github.com/mudler/LocalAI/pull/10425).

docs/content/blog/2026-06-28-native-biometric-backends.md

@@ -0,0 +1,15 @@
++++
+title = "Native biometric backends: voice-detect.cpp and face-detect.cpp"
+date = 2026-06-28
+description = "Two from-scratch C++/ggml engines replace the heavier Python insightface and speaker-recognition backends."
+url = "/blog/native-biometric-backends/"
++++
+
+Two new biometric engines built by the LocalAI team, both from-scratch C++/ggml implementations with no Python and no onnxruntime at inference time:
+
+- [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) for speaker recognition and voice analysis: ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, and wav2vec2 age/gender/emotion.
+- [face-detect.cpp](https://github.com/mudler/face-detect.cpp) for face detection, recognition, demographics and anti-spoofing: SCRFD/ArcFace and YuNet/SFace.
+
+Both ship self-contained GGUF weights, hold bit-exact parity with the reference implementations, and reach cuDNN parity on GPU. They replace the heavier Python `insightface` and `speaker-recognition` backends.
+
+Shipped in [PR #10441](https://github.com/mudler/LocalAI/pull/10441).

docs/content/blog/2026-06-30-concurrent-scoring-and-pii-ner.md

@@ -0,0 +1,15 @@
++++
+title = "Concurrent scoring and PII NER on llama.cpp"
+date = 2026-06-30
+description = "Score and TokenClassify now ride llama.cpp's server task queue instead of locking the context, so they run alongside chat traffic."
+url = "/blog/concurrent-scoring-and-pii-ner/"
++++
+
+The `Score` primitive (used by the router classifier) and `TokenClassify` (used by the PII NER tier) previously locked the llama.cpp context for the duration of the call. They now ride llama.cpp's server task queue instead.
+
+What changes as a result:
+
+- Scoring and token classification run concurrently with chat, completion and embedding traffic, and with each other.
+- The `known_usecases` restriction that forced dedicated scorer and NER model configs on `llama-cpp` is lifted.
+- Repeated scoring calls reuse the prompt KV cache across candidates.
+- Scoring inputs are no longer capped by the physical batch size.

docs/content/blog/2026-07-05-model-capabilities-endpoint.md

@@ -0,0 +1,14 @@
++++
+title = "Model capabilities endpoint"
+date = 2026-07-05
+description = "GET /v1/models/capabilities reports what each model can do and which modalities it accepts, so clients stop guessing from backend names."
+url = "/blog/model-capabilities-endpoint/"
++++
+
+`GET /v1/models/capabilities` is a new endpoint: an additive superset of `/v1/models` that reports each model's `capabilities` alongside its `input_modalities` and `output_modalities` (`text`, `image`, `audio`, `video`).
+
+The practical effect is that a client can decide where to send an attachment by asking the server, instead of pattern-matching on backend names. Modalities are either inferred by LocalAI or declared explicitly in the model config.
+
+Because the endpoint is additive, existing `/v1/models` consumers are unaffected.
+
+See [API discovery]({{% relref "features/api-discovery" %}}#model-capabilities). Shipped in [PR #10687](https://github.com/mudler/LocalAI/pull/10687).

docs/content/blog/2026-07-12-longcat-video-and-avatar-generation.md

@@ -0,0 +1,16 @@
++++
+title = "LongCat video and avatar generation"
+date = 2026-07-12
+description = "A dedicated CUDA backend for LongCat-Video text/image-to-video and LongCat-Video-Avatar-1.5 speech-driven avatars."
+url = "/blog/longcat-video-and-avatar-generation/"
++++
+
+LocalAI gains a dedicated CUDA backend for the LongCat family: `LongCat-Video` for text-to-video and image-to-video, and `LongCat-Video-Avatar-1.5` for speech-driven avatars.
+
+Highlights:
+
+- Multi-segment continuation, so a clip can be extended beyond a single generation window.
+- Portrait and recorded-audio inputs wired into Studio.
+- An SDPA CUDA 13 ARM64 build, which makes the backend usable on DGX Spark.
+
+See [Video generation]({{% relref "features/video-generation" %}}) for configuration and the available model entries. Shipped in [PR #10792](https://github.com/mudler/LocalAI/pull/10792).

docs/content/blog/_index.md

@@ -0,0 +1,17 @@
++++
+title = "Blog"
+weight = 10
+icon = "newspaper"
+alwaysopen = false
+aliases = ["/basics/news/", "/whats-new/"]
++++
+
+Announcements, release write-ups and feature notes from the LocalAI team.
+
+Full changelogs for every version live on [GitHub Releases](https://github.com/mudler/LocalAI/releases). This page is the narrative archive: what shipped, and why it matters.
+
+{{% notice tip %}}
+Prefer a feed reader? Subscribe to [/blog/index.xml](/blog/index.xml).
+{{% /notice %}}
+
+{{< postlist >}}

docs/content/whats-new.md

@@ -6,27 +6,11 @@ url = '/basics/news/'
 icon = "newspaper"
 +++
 
-Release notes have been now moved completely over Github releases.
+LocalAI news is published in two places, both kept current:
 
-You can see the release notes [here](https://github.com/mudler/LocalAI/releases).
+- **[Blog](https://localai.io/blog/)** for release write-ups, benchmark reports and engineering notes.
+- **[GitHub Releases](https://github.com/mudler/LocalAI/releases)** for the full changelog of every version.
 
-## 2026 Highlights
+For how the project got here, read [LocalAI, from March 2023 to now](https://localai.io/blog/localai-since-march-2023/).
 
-- **August 2026**: [Text moderation](/features/moderation/) - new OpenAI-compatible `POST /v1/moderations` endpoint. It uses any local completion model with a constrained JSON grammar and returns the standard safety categories, scores, and per-input flags.
-- **July 2026**: [LongCat video and avatar generation](/features/video-generation/) - dedicated CUDA backend for `LongCat-Video` text/image-to-video and `LongCat-Video-Avatar-1.5` speech-driven avatars. Includes multi-segment continuation, portrait and recorded-audio inputs in Studio, and an SDPA CUDA 13 ARM64 build for DGX Spark.
-- **April 2026**: [Audio Transform](/features/audio-transform/) - generic audio-in / audio-out endpoint with optional reference signal. First implementation: [LocalVQE](https://github.com/localai-org/LocalVQE) C++ backend (joint AEC + noise suppression + dereverberation, DeepVQE-style). Both batch (`POST /audio/transformations`) and bidirectional WebSocket streaming (`/audio/transformations/stream`). Studio "Transform" tab with synchronized waveform players for input / reference / output.
-- **April 2026**: [Face recognition backend](/features/face-recognition/) - `insightface`-powered 1:1 verification, 1:N identification, face embedding, face detection, and demographic analysis. Ships both a non-commercial `buffalo_l` model and an Apache 2.0 OpenCV Zoo alternative.
-- **May 2026**: [Speaker diarization](/features/audio-diarization/) - new `/v1/audio/diarization` endpoint returning "who spoke when" segments. Backed by `sherpa-onnx` (pyannote-3.0 + speaker embeddings + clustering) for pure diarization, and `vibevoice-cpp` for diarization bundled with long-form ASR. Supports `json` / `verbose_json` / `rttm` response formats.
-- **June 2026**: [Sound classification](/features/audio-classification/) - new `/v1/audio/classification` endpoint for audio tagging / sound-event classification, returning scored [AudioSet](https://research.google.com/audioset/) labels (baby cry, glass breaking, alarms, ...). Backed by [ced.cpp](https://github.com/localai-org/ced.cpp), a 527-class AudioSet tagger ported to ggml.
-- **June 2026**: [PII analyze / redact API](/features/middleware/#analyze--redact-api) - the PII detection pipeline (NER + restricted-regex pattern tiers) is now a standalone service: `POST /api/pii/analyze` returns detected entity spans and `POST /api/pii/redact` returns the sanitised text (or `400 pii_blocked`), without routing a chat request through the middleware. Events gain an `origin` (`middleware` / `proxy` / `pii_analyze` / `pii_redact`) so `/api/pii/events` can be filtered by source.
-- **July 2026**: [Model capabilities endpoint](/features/api-discovery/#model-capabilities) - `GET /v1/models/capabilities`, an additive superset of `/v1/models` that reports each model's `capabilities` plus its `input_modalities` / `output_modalities` (`text` / `image` / `audio` / `video`). Lets clients route attachments using inferred or explicitly declared model modalities instead of backend-name checks.
-- **June 2026**: Concurrent scoring and PII NER on llama.cpp - the `Score` (router classifier) and `TokenClassify` (PII NER) primitives now ride llama.cpp's server task queue instead of locking the context, so they run concurrently with chat/completion/embedding traffic and with each other. The `known_usecases` restriction that forced dedicated scorer/NER model configs on llama-cpp is lifted, repeated scoring calls reuse the prompt KV cache across candidates, and scoring inputs are no longer capped by the physical batch size.
-
-## 2024 Highlights
-
-- **April 2024**: [Reranker API](https://github.com/mudler/LocalAI/pull/2121)
-- **May 2024**: [Distributed inferencing](https://github.com/mudler/LocalAI/pull/2324), [Decentralized P2P llama.cpp](https://github.com/mudler/LocalAI/pull/2343) - [Docs](https://localai.io/features/distribute/)
-- **July/August 2024**: [P2P Dashboard, Federated mode and AI Swarms](https://github.com/mudler/LocalAI/pull/2723), [P2P Global community pools](https://github.com/mudler/LocalAI/issues/3113), FLUX-1 support, [P2P Explorer](https://explorer.localai.io)
-- **October 2024**: Examples moved to [LocalAI-examples](https://github.com/mudler/LocalAI-examples)
-- **November 2024**: [Voice Activity Detection (VAD)](https://github.com/mudler/LocalAI/pull/4204), [Bark.cpp backend](https://github.com/mudler/LocalAI/pull/4287)
-- **December 2024**: [stablediffusion.cpp backend (ggml)](https://github.com/mudler/LocalAI/pull/4289)
+This page used to carry a hand-maintained highlights list. It drifted against both sources above, so it now points at them instead.

website/content/blog/_index.md

@@ -1,5 +1,5 @@
 ---
 title: "Blog"
-description: "Release write-ups, benchmark reports and engineering notes from the LocalAI team. Every number here comes out of a benchmark suite, a release or a commit, and the source is named so you can check it."
+description: "Release write-ups, benchmark reports and engineering notes from the LocalAI team. Numbers link to the release, commit or benchmark run they came from."
 extracss: ["blog.css"]
 ---

website/content/blog/apex-moe-quantization.md

@@ -4,13 +4,13 @@ date: 2026-04-10
 author: "Ettore Di Giacinto"
 category: "Research"
 tags: ["quantization", "APEX", "mixture-of-experts", "llama.cpp", "benchmarks"]
-summary: "Qwen3.5-35B-A3B goes from 64.6 GB to 12.2 GB and speeds up from 30.4 to 74.4 tokens per second. Perplexity moves from 6.537 to 7.088. Here is the precision assignment that does it, and where it costs you."
+summary: "Qwen3.5-35B-A3B goes from 64.6 GB to 12.2 GB and speeds up from 30.4 to 74.4 tokens per second. Perplexity moves from 6.537 to 7.088. Here is the precision assignment that does it, and where the quality drops."
 extracss: ["blog.css"]
 ---
 
 A 35B mixture-of-experts model at full precision is a 64.6 GB file, which puts it out of reach of every consumer GPU. APEX gets Qwen3.5-35B-A3B down to 12.2 GB, where it fits a 16 GB card with room for context, and it generates at 74.4 tokens per second instead of 30.4. The output is an ordinary GGUF that stock llama.cpp opens with no patches and no custom build.
 
-The compression is not free at that tier, and the numbers below say exactly what it costs. At the 21.3 GB tier it is closer to free than we expected: APEX Quality has a lower perplexity than the F16 model it was quantized from.
+At that tier the quality does drop, and the numbers below say by how much. At the 21.3 GB tier it barely drops at all: APEX Quality has a lower perplexity than the F16 model it was quantized from.
 
 ## The measurements
 
@@ -37,17 +37,17 @@ All of this is Qwen3.5-35B-A3B on an NVIDIA DGX Spark (GB10, 122 GB unified VRAM
 
 Three things in that table are worth stopping on.
 
-APEX Quality is 21.3 GB, a third of F16, and its perplexity of 6.527 is lower than F16's 6.537 and lower than Q8_0's 6.533. Quantization noise acting as mild regularization on a wikitext evaluation is a known effect and we are not claiming the quantized model is smarter. The honest reading is that at this tier the loss is below the measurement floor.
+APEX Quality is 21.3 GB, a third of F16, and its perplexity of 6.527 is lower than F16's 6.537 and lower than Q8_0's 6.533. Quantization noise acting as mild regularization on a wikitext evaluation is a known effect and we are not claiming the quantized model is smarter. At this tier the loss is below the measurement floor.
 
-Against Unsloth's UD-Q8_K_XL, APEX I-Quality is half the size (21.3 GB against 45.3 GB), one point ahead on HellaSwag (83.5% against 82.5%), within 0.016 on perplexity, and 73% faster (63.1 t/s against 36.4). That is the comparison that matters for anyone choosing a published quant today.
+Against Unsloth's UD-Q8_K_XL, APEX I-Quality is half the size (21.3 GB against 45.3 GB), one point ahead on HellaSwag (83.5% against 82.5%), within 0.016 on perplexity, and 73% faster (63.1 t/s against 36.4).
 
 At the bottom end, APEX Mini beats bartowski IQ2_M on every metric while being 0.9 GB larger: perplexity 7.088 against 7.303, HellaSwag 81.0% against 80.3%, MMLU 41.3% against 39.6%.
 
-## Why it gets faster, not just smaller
+## Why it also gets faster
 
 Token generation on a single stream is bound by memory bandwidth, not by arithmetic. Every generated token requires reading the active weights out of memory, so halving the bytes roughly halves the time spent waiting for them. Going from 64.6 GB to 12.2 GB takes throughput from 30.4 to 74.4 tokens per second, a 2.45x gain on the same hardware with the same kernels. Every APEX tier clears 60 t/s.
 
-That is also why a large well-behaved quant such as UD-Q8_K_XL is slower than a smaller one with equal quality. Size is a speed knob as much as a memory knob.
+That is also why a large well-behaved quant such as UD-Q8_K_XL is slower than a smaller one with equal quality.
 
 ## Per-tensor and per-layer precision
 
@@ -55,9 +55,9 @@ Uniform quantization gives every tensor the same bit width, which spends the sam
 
 APEX classifies every tensor into one of three roles and treats them differently.
 
-**Routed expert weights** (the gate, up and down projections inside the experts) are the bulk of the parameters, and only 8 of 256 experts are active per token. That 97% structural sparsity is what makes aggressive quantization safe here. The routing decision itself reads full-precision gate weights, so quantization noise inside an expert that was not selected never reaches the output at all. When an expert is selected, its contribution is one of eight summed paths, which further dilutes per-tensor error.
+**Routed expert weights** (the gate, up and down projections inside the experts) are the bulk of the parameters, and only 8 of 256 experts are active per token. That 97% structural sparsity is why aggressive quantization is safe here. The routing decision itself reads full-precision gate weights, so quantization noise inside an expert that was not selected never reaches the output at all. When an expert is selected, its contribution is one of eight summed paths, which further dilutes per-tensor error.
 
-**Shared expert weights** run for every single token and their weight distribution is heavy-tailed, with a kurtosis of 13.10 against 3.41 for routed experts. Those outliers carry real signal and low-bit formats clip them. Q8_0 is the minimum viable precision here, and dropping it is the fastest way to wreck a build.
+**Shared expert weights** run for every single token and their weight distribution is heavy-tailed, with a kurtosis of 13.10 against 3.41 for routed experts. Those outliers carry real signal and low-bit formats clip them. Q8_0 is the minimum viable precision here, and dropping it degrades the build quickly.
 
 **Attention and SSM weights** are dense, contribute few parameters relative to the experts, and matter for generation quality. They sit at Q6_K throughout.
 
@@ -69,23 +69,23 @@ None of this needs a patched llama.cpp. The assignments are expressed with the s
 
 Twenty-five or so systematic runs produced a few results that saved a lot of time later.
 
-Going from Q6_K to Q8_0 on routed experts costs 7.5 GB and buys zero perplexity improvement. Going below Q5_K on them causes measurable degradation. Q6_K is the ceiling worth paying for.
+Going from Q6_K to Q8_0 on routed experts costs 7.5 GB and gives zero perplexity improvement. Going below Q5_K on them causes measurable degradation. Q6_K is the ceiling.
 
 Layer position matters more than uniform bit width. A two-tier gradient of Q6_K edges and Q5_K middle matches Q8_0 quality; a uniform Q5_K assignment at a similar size does not.
 
 IQ formats underperform K-quants on MoE experts. IQ3_S gives worse perplexity than Q3_K on routed expert tensors at a similar bit rate, because the near-Gaussian expert weight distribution (kurtosis 3.41) suits the K-quant block structure better.
 
-Five C-level modifications to the quantization algorithms themselves, including error feedback, enhanced scale search, super-block refinement and Gaussian-density weighting, all showed zero improvement. Stock llama.cpp quantization is already good. The gains here come entirely from deciding where to spend bits.
+Five C-level modifications to the quantization algorithms themselves, including error feedback, enhanced scale search, super-block refinement and Gaussian-density weighting, all showed zero improvement. Stock llama.cpp quantization is already good. The gains here come entirely from deciding where to put the bits.
 
 ## The I-variants and their calibration set
 
 Standard imatrix calibration uses Wikipedia text, which is also what wikitext perplexity measures, so the calibration and the benchmark agree with each other by construction. The I-variants calibrate on a diverse set spanning chat, code, reasoning and tool-calling, with no Wikipedia in it.
 
-That trade shows up clearly. I-Compact drops perplexity from 6.783 to 6.669, cuts KL max from 7.56 to 5.50, and lifts MMLU from 40.9% to 41.7%. At the Quality tier, I-Quality gives up 0.025 perplexity against Quality and takes the highest HellaSwag score of anything tested (83.5%), the best TruthfulQA (38.4%), and a lower KL divergence. If your workload is chat, code or agents rather than encyclopedic prose, take the I variant.
+It shows up in the numbers. I-Compact drops perplexity from 6.783 to 6.669, cuts KL max from 7.56 to 5.50, and lifts MMLU from 40.9% to 41.7%. At the Quality tier, I-Quality gives up 0.025 perplexity against Quality and takes the highest HellaSwag score of anything tested (83.5%), the best TruthfulQA (38.4%), and a lower KL divergence. If your workload is chat, code or agents rather than encyclopedic prose, take the I variant.
 
-## Where it costs you
+## Where the quality drops
 
-The Compact and Mini tiers are real compression, and they are not free.
+The Compact and Mini tiers lose real quality.
 
 Compact at 16.1 GB moves perplexity from 6.537 to 6.783, a 3.8% increase, and its KL mean rises tenfold against Q8_0, from 0.0046 to 0.0469. Mini at 12.2 GB goes to 7.088, an 8.4% increase, with a KL mean of 0.0870 and HellaSwag down 1.5 points to 81.0%. Those are the numbers to weigh against the fact that the model now runs at all on a 16 GB card.
 

website/content/blog/localai-since-march-2023.md

@@ -4,7 +4,7 @@ date: 2026-07-29
 author: "Ettore Di Giacinto"
 category: "History"
 tags: ["history", "architecture", "releases", "community"]
-summary: "Three years, 133 releases and 224 contributors later. The four changes that mattered most were making the core small, adding agents, making it a cluster, and giving it eyes and ears."
+summary: "Three years, 133 releases and 224 contributors later. Here are the four decisions that shaped it: making the core small, adding agents, making it a cluster, and giving it eyes and ears."
 extracss: ["blog.css"]
 ---
 
@@ -16,9 +16,9 @@ None of those numbers are rounded up. You can read every one of them off the rep
 
 {{< starchart >}}
 
-The curve is not the point, but it is a useful map. The four marks on it are the four decisions below, and you can see each of them in the slope afterwards.
+The four marks on it are the four decisions below, and you can see each of them in the slope afterwards.
 
-What follows is how it got here. Not the feature list, which you can read in the releases, but the four decisions that changed the shape of the thing.
+What follows is the four decisions that changed the shape of the thing. The full feature list is in the releases.
 
 ## 2023 to 2024: an API in front of llama.cpp
 
@@ -34,7 +34,7 @@ Every backend moved out of the main binary in [v3.2.0](https://github.com/mudler
 
 You install one thing and it stays small. Ask for a GGUF model and llama-cpp arrives. Ask for transcription and whisper or parakeet arrives. Nothing else is fetched, and a machine that only ever serves one model never downloads the other sixty-nine backends.
 
-That one change is what made everything after it possible. Adding a backend stopped meaning adding weight to everybody's install, so "should we support this engine" stopped being an argument about download size and went back to being an argument about whether the engine is any good. It is also the reason we can afford to maintain eighteen engines of our own, which comes later.
+Everything after it depended on that one change. Adding a backend stopped meaning adding weight to everybody's install, so "should we support this engine" stopped being an argument about download size and went back to being an argument about whether the engine is any good. It is also the reason we can afford to maintain eighteen engines of our own, which comes later.
 
 ## March 2026: agents, and a new interface
 
@@ -42,7 +42,7 @@ That one change is what made everything after it possible. Adding a backend stop
 
 The web interface was rewritten in React at the same time, with a Canvas mode, MCP Apps and client-side tools with tool streaming ([#8947](https://github.com/mudler/LocalAI/pull/8947)), and WebRTC realtime audio ([#8790](https://github.com/mudler/LocalAI/pull/8790)). MLX gained a distributed mode ([#8801](https://github.com/mudler/LocalAI/pull/8801)).
 
-The realtime audio path is the piece that changed what people built. Speech in, tool calls in the middle, speech out, over WebRTC, fast enough that it feels like a conversation rather than a walkie-talkie. It had landed as the Realtime API in February 2026 ([#6245](https://github.com/mudler/LocalAI/pull/6245)), and the interface rewrite finally gave it a face.
+The realtime audio path changed what people built with it. Speech in, tool calls in the middle, speech out, over WebRTC, fast enough that it feels like a conversation rather than a walkie-talkie. It had landed as the Realtime API in February 2026 ([#6245](https://github.com/mudler/LocalAI/pull/6245)), and the interface rewrite finally gave it a face.
 
 ## April 2026: it becomes a cluster
 
@@ -74,6 +74,6 @@ The most recent one is [vllm.cpp](https://github.com/mudler/vllm.cpp), a C++20 p
 
 ## Where it stands
 
-Still MIT, still a community project. 224 people have put code in, and the README is kept translated into eight languages because the people using this are not all in one place. The [contributors graph](https://github.com/mudler/LocalAI/graphs/contributors) is the honest picture of who actually built this, and it is not me.
+Still MIT, still a community project. 224 people have put code in, and the README is kept translated into eight languages because the people using this are not all in one place. The [contributors graph](https://github.com/mudler/LocalAI/graphs/contributors) shows who actually built this, and it is not me.
 
 If you want to add something, backends and gallery entries are the two places a first contribution lands cleanly. There is a step-by-step checklist for a new backend in `.agents/adding-backends.md`, and a gallery entry is just a YAML block. Come say hello in [Discord](https://discord.gg/uJAeKSAGDy) if you get stuck.

website/content/blog/parakeet-cpp-asr-on-cpu.md

@@ -1,22 +1,22 @@
 ---
-title: "parakeet.cpp: NeMo transcripts, byte for byte, without the Python"
+title: "parakeet.cpp: the same NeMo transcript, without the Python"
 date: 2026-06-05
 author: "Ettore Di Giacinto"
 category: "Benchmarks"
 tags: ["parakeet.cpp", "ASR", "ggml", "streaming", "benchmarks"]
-summary: "Same transcript as NVIDIA NeMo, character for character, at a median 1.40x on CPU and about 27x the speed of whisper.cpp. One binary, one GGUF file, no Python at inference."
+summary: "The same transcript as NVIDIA NeMo at a median 1.40x on CPU, and about 27x the speed of whisper.cpp, from one binary and one GGUF file."
 extracss: ["blog.css"]
 ---
 
-You can drop a single binary and a GGUF file onto a machine with no GPU and get NVIDIA NeMo Parakeet transcription out of it, at a median 1.40x NeMo's own PyTorch CPU speed, with a transcript that matches NeMo character for character. That is [parakeet.cpp](https://github.com/mudler/parakeet.cpp), a C++17 port of the Parakeet speech-recognition family built on ggml.
+You can drop a single binary and a GGUF file onto a machine with no GPU and get NVIDIA NeMo Parakeet transcription out of it, at a median 1.40x NeMo's own PyTorch CPU speed, with the same transcript NeMo produces. That is [parakeet.cpp](https://github.com/mudler/parakeet.cpp), a C++17 port of the Parakeet speech-recognition family built on ggml.
 
-Accuracy came first and speed came second, in that order, because a faster transcriber that disagrees with the reference is a different model, not a port.
+We checked the accuracy before touching the speed, because a transcriber that disagrees with the reference is not a port of it.
 
 ## WER 0 against NeMo
 
 Every published checkpoint is validated at WER 0 against NeMo. Across the LibriSpeech test-clean set the mean f32 agreement WER, meaning the word error rate between our transcript and NeMo's on the same audio, is 0.0155%. On seven of the ten models it is exactly 0.0000%, which is a byte-identical transcript.
 
-That number is what makes the speed comparison meaningful. Both engines did the same work and produced the same output, so the only difference left is how long they took.
+Both engines did the same work and produced the same output, so the only difference left is how long they took.
 
 ## CPU, against NeMo's own runtime
 
@@ -54,25 +54,25 @@ Against whisper.cpp turbo on the same clip and at the same accuracy (1.6% WER on
 
 The decisive win was on the decode side. A transducer decodes autoregressively, and profiling showed the prediction-network LSTM taking about 97% of RNN-T decode time while producing the same output over and over: on a non-emitting frame the prediction network's input has not changed, so its forward pass is redundant. Caching that forward across non-emitting frames removed most of the decode cost.
 
-The encoder side is a set of smaller wins with no single hero: a persistent ggml backend with `gallocr`, zero-copy weights straight out of the GGUF mapping, one fused graph rather than per-layer graph building, and tinyBLAS through `GGML_LLAMAFILE`.
+The encoder side is a set of smaller wins: a persistent ggml backend with `gallocr`, zero-copy weights straight out of the GGUF mapping, one fused graph rather than per-layer graph building, and tinyBLAS through `GGML_LLAMAFILE`.
 
 ## On the GPU
 
 On an NVIDIA GB10 (Grace-Blackwell), parakeet.cpp wins on all ten models, with a median of 1.25x and up to 4.3x on the large TDT and hybrid models. The reference here is NeMo-GPU inside the `nvcr.io/nvidia/nemo` container, because NeMo cannot run on that host's torch and CUDA stack directly.
 
-The 4.3x cases have a specific cause. NeMo's TDT greedy decode is not CUDA-graph accelerated and falls back to a per-step Python loop, while ours is a lean C++ loop. Where NeMo's decode is CUDA-graph accelerated, as it is for RNN-T, the gap narrows to about 1.16x at f32 and 1.30x at q8_0. On the pure-encoder CTC models the margin is around 1.2x, because ggml's generic CUDA conv and attention kernels still trail NVIDIA's tuned cuDNN. That is the main piece of GPU headroom left in the project and we say so in the README rather than averaging it away.
+The 4.3x cases have a specific cause. NeMo's TDT greedy decode is not CUDA-graph accelerated and falls back to a per-step Python loop, while ours is a lean C++ loop. Where NeMo's decode is CUDA-graph accelerated, as it is for RNN-T, the gap narrows to about 1.16x at f32 and 1.30x at q8_0. On the pure-encoder CTC models the margin is around 1.2x, because ggml's generic CUDA conv and attention kernels still trail NVIDIA's tuned cuDNN. That is the main piece of GPU headroom left in the project, and the README lists it per model.
 
-Batching several clips through the decoder together reaches about 10x to 12x at batch size 16 on the GB10, and about 3x to 5x on CPU. It applies to transducer models only, since CTC has no autoregressive decode to batch, and the batched path is bit-identical to running the clips one at a time.
+Batching several clips through the decoder together reaches about 10x to 12x at batch size 16 on the GB10, and about 3x to 5x on CPU. It applies to transducer models only, since CTC has no autoregressive decode to batch, and the batched path produces the same output as running the clips one at a time.
 
 On Apple M4 through ggml's Metal backend, the larger models run about 3x to 5x faster than the same models on that machine's CPU.
 
-## Cache-aware streaming, and what end-of-utterance detection buys you
+## Cache-aware streaming and end-of-utterance detection
 
 Offline transcription hands you a file and waits. A voice assistant cannot do that, so `parakeet_realtime_eou_120m-v1` runs a cache-aware streaming path instead: you feed it 16 kHz mono PCM as it arrives and it returns newly finalized text as it becomes stable.
 
-Cache-aware means the cost per chunk stays flat. Each chunk's forward pass carries per-layer convolution and attention caches plus the transducer decoder state forward, so nothing before the current chunk is recomputed. Without that, every chunk would re-run the encoder over the whole session so far, and the per-chunk cost would grow with the length of the conversation until the loop fell behind. The implementation covers layer norm with causal convolution, causal subsampling, and chunked-limited attention, and its transcript matches NeMo's own cache-aware streaming byte for byte.
+Cache-aware means the cost per chunk stays flat. Each chunk's forward pass carries per-layer convolution and attention caches plus the transducer decoder state forward, so nothing before the current chunk is recomputed. Without that, every chunk would re-run the encoder over the whole session so far, and the per-chunk cost would grow with the length of the conversation until the loop fell behind. The implementation covers layer norm with causal convolution, causal subsampling, and chunked-limited attention, and its transcript matches NeMo's own cache-aware streaming exactly.
 
-End-of-utterance detection is the part that changes how an assistant feels. The model emits `<EOU>` when the speaker has finished a turn and `<EOB>` for a backchannel, as events alongside the text. A voice loop can start generating a reply the moment `<EOU>` arrives rather than waiting out a fixed silence timer, which is where most of the perceived lag in a spoken assistant comes from. The alternative, a VAD with a 700 ms hangover, either cuts people off mid-sentence or makes the assistant feel slow, and it cannot tell "mm-hm" from the end of a thought. `finalize` flushes the tail at end of stream without fabricating an `<EOU>` that NeMo would not have emitted.
+End-of-utterance detection changes how an assistant feels. The model emits `<EOU>` when the speaker has finished a turn and `<EOB>` for a backchannel, as events alongside the text. A voice loop can start generating a reply the moment `<EOU>` arrives rather than waiting out a fixed silence timer, which is where most of the perceived lag in a spoken assistant comes from. The alternative, a VAD with a 700 ms hangover, either cuts people off mid-sentence or makes the assistant feel slow, and it cannot tell "mm-hm" from the end of a thought. `finalize` flushes the tail at end of stream without fabricating an `<EOU>` that NeMo would not have emitted.
 
 The streaming path measures at RTFx 3.80 on a 7.43 second clip. That sits well below the offline number by design, because streaming runs many small chunked passes rather than one large one, and it is still several times faster than real time on a CPU.
 
@@ -95,7 +95,7 @@ parakeet.cpp ports NeMo's `rel_pos_local_attn`, a banded attention where each qu
 </table>
 </div>
 
-At NeMo's full W=128 window that is about 4x faster and about 5.7x less peak memory than the global path. The band is built with a chunk-matmul construction, overlapping key and value chunks feeding one batched GEMM plus a diagonal skew view, so the graph node count does not depend on the window. The wide window costs the same as the narrow one. Short clips stay on the global path and remain byte-identical to before.
+At NeMo's full W=128 window that is about 4x faster and about 5.7x less peak memory than the global path. The band is built with a chunk-matmul construction, overlapping key and value chunks feeding one batched GEMM plus a diagonal skew view, so the graph node count does not depend on the window. The wide window costs the same as the narrow one. Short clips stay on the global path and produce the same output as before.
 
 ## Using it
 

website/content/blog/why-we-write-our-own-engines.md

@@ -4,21 +4,21 @@ date: 2026-07-24
 author: "Ettore Di Giacinto"
 category: "Engineering"
 tags: ["engineering", "ggml", "vllm.cpp", "depth-anything.cpp", "parity"]
-summary: "A 66 MiB binary instead of a 9.1 GiB virtualenv, depth estimation that beats PyTorch on CPU in half the memory, and biometrics that match insightface bit for bit. The method, the measurements, and what it costs us."
+summary: "Eighteen of our backends are C or C++ ports we wrote from scratch instead of wrapping an upstream engine. Here is why we did it and what we measured."
 extracss: ["blog.css"]
 ---
 
-Most LocalAI backends wrap somebody else's engine, and that is the right default. llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX and the rest are maintained by people who are better at those models than we are, and wrapping them costs a Dockerfile and a gRPC shim.
+Most LocalAI backends wrap somebody else's engine. llama.cpp, vLLM, whisper.cpp, stable-diffusion and MLX are maintained by people who work on those models full time, and wrapping one of them costs us a Dockerfile and a gRPC shim. We do that wherever we can.
 
-Eighteen of our backends do not wrap anything. They are C or C++ ports we wrote from scratch, and each one exists because wrapping the upstream engine would have meant shipping something we could not ship: a multi-gigabyte Python install, a non-portable CUDA-only stack, or a model that had no C++ implementation at all. This post is about what those ports buy, measured, and what they cost.
+Eighteen of our backends do not wrap anything. They are C or C++ ports we wrote from scratch, and each one exists because wrapping the upstream engine would have meant shipping something we could not ship: a multi-gigabyte Python install, a CUDA-only stack that will not run on half the machines our users have, or, in a few cases, a model with no C++ implementation to wrap in the first place. Below are the numbers for four of them, and what keeping them alive takes.
 
-## What you get: one file, and memory you can predict
+## vllm.cpp: 66 MiB instead of 9.1 GiB
 
-Deploying a Python inference stack means resolving a dependency tree at install time, on the target machine, against whatever CUDA and glibc it has. Deploying a ggml port means copying a shared library and a GGUF file.
+Deploying a Python inference stack means resolving a dependency tree at install time, on the target machine, against whatever CUDA and glibc that machine has. Deploying a ggml port means copying a shared library and a GGUF file.
 
-The clearest measurement of that difference is [vllm.cpp](https://github.com/mudler/vllm.cpp), our C++20 port of vLLM's V1 serving architecture. Installing vLLM produces a 9.1 GiB virtualenv. Installing vllm.cpp produces a 66 MiB binary. The engine implements the same things the Python original does, including paged KV cache, continuous batching, prefix caching, the scheduler and the sampler, with no Python, no PyTorch and no ggml at inference.
+[vllm.cpp](https://github.com/mudler/vllm.cpp) is our C++20 port of vLLM's V1 serving architecture. Installing vLLM produces a 9.1 GiB virtualenv. Installing vllm.cpp produces a 66 MiB binary. It implements the same things the Python original does, including paged KV cache, continuous batching, prefix caching, the scheduler and the sampler, with no Python, no PyTorch and no ggml at inference.
 
-The obvious question is what that costs in throughput. On an NVIDIA GB10 running Qwen3.6-27B in NVFP4, greedy, closed loop, against vLLM in its production graphed configuration rather than `--enforce-eager`:
+The question is what that does to throughput. On an NVIDIA GB10 running Qwen3.6-27B in NVFP4, greedy, closed loop, against vLLM in its production graphed configuration rather than `--enforce-eager`:
 
 <div class="tw">
 <table>
@@ -31,13 +31,15 @@ The obvious question is what that costs in throughput. On an NVIDIA GB10 running
 </table>
 </div>
 
-We are ahead at all six points, and five of those six are ties. Our run-to-run noise band is 0.5%, and concurrency 2 through 32 land between 0.7% and 1.7%, so the honest reading is that only the single-stream case (4.5%) is clearly outside noise. Output is token-for-token identical to vLLM at every point on that curve. Peak host memory is 24.88 GiB against 28.18 GiB.
+Those are ties. Our run-to-run noise band is 0.5%, and concurrency 2 through 32 land between 0.7% and 1.7%, so those five points sit inside the noise or close enough to it not to matter. Only the single-stream case, at 4.5%, is clearly outside. Output is token-for-token identical to vLLM at every point on the curve, and peak host memory is 24.88 GiB against 28.18 GiB.
 
-A tie against a mature CUDA stack is a good result for a 66 MiB binary, and it means the footprint saving is not paid for in throughput. Against llama.cpp on CPU from the same GGUF file, prefill runs 1.18x faster (223.8 against 177.3 tok/s), decode is a tie inside llama.cpp's own spread, and the tokens are byte-identical to its greedy decode. Against MLX-LM on an Apple M4, prefill time to first token is 1.5% ahead and warm total throughput is 97.6% of MLX-LM, a real 2.4% gap that sits entirely in decode.
+The install drops from 9.1 GiB to 66 MiB and the throughput stays where it was, which is what we were after.
 
-## Sometimes the port is simply faster
+Against llama.cpp on CPU from the same GGUF file, prefill runs 1.18x faster (223.8 against 177.3 tok/s), decode is a tie inside llama.cpp's own spread, and the tokens match its greedy decode exactly. Against MLX-LM on an Apple M4, prefill time to first token is 1.5% ahead and warm total throughput is 97.6% of MLX-LM, a real 2.4% gap that sits entirely in decode.
 
-[depth-anything.cpp](https://github.com/mudler/depth-anything.cpp) is a port of ByteDance's Depth Anything 3, which gives you metric depth in metres from one ordinary photo, plus per-pixel confidence, camera intrinsics and extrinsics, and a back-projected point cloud. On CPU it is faster than PyTorch running the same model.
+## depth-anything.cpp is faster on CPU
+
+[depth-anything.cpp](https://github.com/mudler/depth-anything.cpp) is a port of ByteDance's Depth Anything 3, which gives you metric depth in metres from one ordinary photo, plus per-pixel confidence, camera intrinsics and extrinsics, and a back-projected point cloud. On CPU it runs faster than PyTorch on the same model.
 
 <div class="tw">
 <table>
@@ -49,40 +51,42 @@ A tie against a mature CUDA stack is a good result for a 66 MiB binary, and it m
 </table>
 </div>
 
-Same model, 1.31x the speed, 27% of the memory, and a load that finishes in 40 ms instead of 749 ms, on a Ryzen 9 9950X3D at 504x336 with 16 threads. The quantized q4_k build is a 99 MB file and stays near-lossless. Output correlates 1.0 with the reference forward pass, component by component, across 37 parity tests.
+That is on a Ryzen 9 9950X3D at 504x336 with 16 threads. The C++ build runs the same model 1.31x faster, uses 363 MB of RAM against 1328 MB, and loads in 40 ms instead of 749 ms. The quantized q4_k build is a 99 MB file and stays near-lossless. Output correlates 1.0 with the reference forward pass across 37 parity tests.
 
-The reason it is faster has nothing to do with writing better matmul kernels than PyTorch. Two positional embeddings, the DPT head's UV embedding and the backbone's bicubic position embedding, were being recomputed on every forward pass with single-threaded scalar sin, cos and bicubic loops, even though they depend only on the input geometry and are identical every call. Caching them removed about 95 ms of host-side overhead per forward, which is most of the gap. PyTorch builds the same embeddings with vectorized operations and never paid that cost.
+We did not write a better matmul kernel than PyTorch. Two positional embeddings, the DPT head's UV embedding and the backbone's bicubic position embedding, were being recomputed on every forward pass with single-threaded scalar sin, cos and bicubic loops, even though they depend only on the input geometry and are identical every call. Caching them removed about 95 ms of host-side overhead per forward, which is most of the gap. PyTorch builds the same embeddings with vectorized operations and never had that overhead to begin with.
 
-That is the general shape of these wins. The heavy GEMMs are close to a wash, because everyone is calling into the same class of BLAS kernel. The difference sits in host-side work that a Python reference implementation never bothered to optimize, and in not loading an interpreter and a framework to do inference. On GPU the picture flips back to parity: with the ggml CUDA backend and flash attention on a GB10, depth-anything.cpp ties PyTorch's tuned cuDNN at 47.3 ms per forward, and wins only the cold start, loading 1.75x to 2.9x faster.
+The heavy GEMMs are close to a wash, because everyone is calling into the same class of BLAS kernel. What is left is host-side work that a Python reference implementation never bothered to optimize, plus not loading an interpreter and a framework to do inference. On GPU it goes back to parity: with the ggml CUDA backend and flash attention on a GB10, depth-anything.cpp ties PyTorch's tuned cuDNN at 47.3 ms per forward, and wins only the cold start, loading 1.75x to 2.9x faster.
 
-## Parity is the gate, speed is the follow-up
+## The two where we are slower
 
-[face-detect.cpp](https://github.com/mudler/face-detect.cpp) and [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) replaced LocalAI's Python `insightface` and `speaker-recognition` backends. Both are the case where we do not claim a CPU speed win, and both shipped anyway.
+[face-detect.cpp](https://github.com/mudler/face-detect.cpp) and [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) replaced LocalAI's Python `insightface` and `speaker-recognition` backends. Neither of them is faster than what it replaced on CPU, and we shipped them anyway.
 
-face-detect.cpp runs the whole insightface buffalo chain, so SCRFD detection, five-landmark similarity-transform alignment to 112x112, and the ArcFace embedding, out of one self-contained GGUF with no Python and no onnxruntime. Detector boxes and landmarks match insightface to within 1 pixel, and the recognition embedding matches to cosine 1.000000, held at any thread count. On CPU it is slower than onnxruntime: SCRFD detect runs at about 0.83x at one thread and 0.69x at eight, ArcFace embed at about 0.61x and 0.84x. onnxruntime's MLAS convolution kernels sit at the FMA-port peak, and a custom AVX2 Winograd path narrowed the gap without closing it. On GPU, routing the same convolutions through cuDNN takes SCRFD from 14.8 ms to 6.4 ms and lands at torch-cuDNN parity.
+face-detect.cpp runs the whole insightface buffalo chain, so SCRFD detection, five-landmark similarity-transform alignment to 112x112, and the ArcFace embedding, out of one self-contained GGUF with no Python and no onnxruntime. Detector boxes and landmarks match insightface to within 1 pixel, and the recognition embedding matches to cosine 1.000000 at any thread count. On CPU it is slower than onnxruntime: SCRFD detect runs at about 0.83x at one thread and 0.69x at eight, ArcFace embed at about 0.61x and 0.84x. onnxruntime's MLAS convolution kernels sit at the FMA-port peak, and a custom AVX2 Winograd path narrowed the gap without closing it. On GPU, routing the same convolutions through cuDNN takes SCRFD from 14.8 ms to 6.4 ms, which lands at torch-cuDNN parity.
 
-voice-detect.cpp is the same story with a memory result attached. A WeSpeaker verification peaks at about 62 MB in our binary against about 334 MB for the CPU-only Python, torch and onnxruntime path, roughly 5.4x lower, with an identical verdict and embedding cosine 1.000000. End to end on CPU the two land within 10 to 15% of each other, trading the lead by model and thread count, and on GPU the conv encoders match the reference.
+voice-detect.cpp has a memory result instead. A WeSpeaker verification peaks at about 62 MB in our binary against about 334 MB for the CPU-only Python, torch and onnxruntime path, roughly 5.4x lower, with an identical verdict and embedding cosine 1.000000. End to end on CPU the two land within 10 to 15% of each other, trading the lead by model and thread count, and on GPU the conv encoders match the reference.
 
-For a biometric pipeline, matching the reference exactly matters more than being faster than it. An embedding that differs in the fourth decimal place changes verification decisions at a threshold, and every enrolled template in a deployment would have to be recomputed. Parity is what makes the replacement a drop-in rather than a migration.
+For a biometric pipeline we would rather have the exact match than the speed. An embedding that differs in the fourth decimal place changes verification decisions at a threshold, and every enrolled template in a deployment would have to be recomputed. Matching insightface exactly is what lets somebody swap the backend out without re-enrolling their users.
 
-## The method
+## How we do it
 
-Every port follows the same sequence, and the order is the important part.
+Every port follows the same four steps.
 
 Convert the weights first, into one GGUF with the tokenizer, the vocabulary and any auxiliary model embedded, so that deploying the model is copying a file.
 
-Port the graph second, and gate it component by component against reference tensors dumped from the original implementation. depth-anything.cpp has 37 ctest cases covering preprocessing, backbone, attention, the DPT head, depth, pose, the ray head, the ray to pose solver and the exporters. parakeet.cpp gates on transcript agreement with NeMo at WER 0. face-detect.cpp gates on box and landmark distance in pixels and embedding cosine. A port that is fast and slightly wrong is worthless, and without a per-component gate you find out it is wrong months later.
+Port the graph second, and check it component by component against reference tensors dumped from the original implementation. depth-anything.cpp has 37 ctest cases covering preprocessing, backbone, attention, the DPT head, depth, pose, the ray head, the ray to pose solver and the exporters. parakeet.cpp checks transcript agreement with NeMo at WER 0. face-detect.cpp checks box and landmark distance in pixels, and embedding cosine. Skip this step and you find out the port is wrong months later, from a user, on a model you had stopped thinking about.
 
-Optimize third, with a profiler, and only after parity holds. In parakeet.cpp the decisive win was caching a prediction-network LSTM forward pass that was 97% of transducer decode time and mostly redundant. In depth-anything.cpp it was two cached positional embeddings. Neither was a kernel rewrite, and neither would have been findable without a working baseline to profile.
+Optimize third, with a profiler, and only once the parity checks pass. In parakeet.cpp the win was caching a prediction-network LSTM forward pass that was 97% of transducer decode time and mostly redundant. In depth-anything.cpp it was the two positional embeddings above. Neither was a kernel rewrite, and neither would have turned up without a working baseline to profile.
 
 Expose a flat C ABI last. LocalAI dlopens the shared library through purego and calls that ABI directly, so there is no subprocess, no gRPC hop to a Python server, and no interpreter in the serving path.
 
-## What it costs
+## What it takes to maintain
 
-Maintenance, mostly. Each engine is a repository with its own CI, its own benchmark suite, its own GGUF conversion script and its own parity baselines, and upstream keeps releasing new checkpoints that need converter work.
+Each engine is its own repository with its own CI, benchmark suite, GGUF conversion script and parity baselines, and upstream keeps releasing checkpoints that need converter work.
 
 GPU kernels are the weak spot. ggml's generic CUDA convolution and attention kernels trail NVIDIA's tuned cuDNN on the conv-heavy models, which is why face-detect.cpp needs an explicit cuDNN path to reach parity, and why parakeet.cpp's GPU margin over NeMo is a median 1.25x while its CPU margin is wider.
 
-Porting also does not scale to everything. llama.cpp, vLLM, whisper.cpp, MLX and diffusers stay wrapped, because those projects are large, fast-moving and already excellent at what they do. We write an engine when a model has no C++ implementation, when the Python dependency is heavier than the model, or when the thing we need does not exist yet. Everything else we install from somebody else.
+It also does not scale to everything. llama.cpp, vLLM, whisper.cpp, MLX and diffusers stay wrapped, because those projects are large, fast-moving and already good at what they do. We write an engine when a model has no C++ implementation, when the Python dependency is heavier than the model itself, or when the thing we need does not exist yet. The rest we install like everybody else.
 
-Every engine listed above keeps its own benchmark suite, its parity gates and its methodology in its own repository, including the runs that did not work. The full list of them is the "Backends built by us" table in the [LocalAI README](https://github.com/mudler/LocalAI#backends-built-by-us).
+One thing that confuses people reading the tree for the first time: LocalAI's own core is Go, and each backend is written in whatever its model's ecosystem needs, which is why there is C++ sitting next to Python in the same repository.
+
+Every engine above keeps its benchmark suite, its parity checks and its methodology in its own repository, including the runs that did not work out. The full list is the "Backends built by us" table in the [LocalAI README](https://github.com/mudler/LocalAI#backends-built-by-us).

website/content/engines/_index.md

@@ -1,5 +1,5 @@
 ---
 title: "Engines"
-description: "Nineteen native C, C++ and Go engines written by the LocalAI team. No Python at inference, checked against the reference implementation in CI, and small enough to ship as one file."
+description: "Eighteen native C, C++ and Go engines written by the LocalAI team. No Python at inference, checked against the reference implementation in CI, and small enough to ship as one file."
 extracss: ["engines.css"]
 ---

website/data/engines.yaml

@@ -1,4 +1,4 @@
-# The nineteen native engines the LocalAI team wrote, and the one quantization
+# The eighteen native engines the LocalAI team wrote, and the one quantization
 # recipe that feeds them. This file is the single source of truth for the
 # /engines/ page: the layout renders whatever is here, in this order, and adds
 # nothing of its own. Numbers in `highlights` come from each engine's own

website/layouts/engines/list.html

@@ -10,7 +10,7 @@
     <p class="kicker fd" style="margin-top:0">Engines we build</p>
     <h1 class="eng-h1"><u><b>Eighteen engines,</b></u><u><b><s>written from scratch.</s></b></u></h1>
     <div class="bars" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
-    <p class="lede fd mt2">Most backends wrap somebody else's engine. These do not. Each one exists because the thing we needed was a multi-gigabyte Python install, or closed, or nobody had built it yet. What you get instead is a binary and a GGUF file, checked against the reference implementation in CI, running on the machine you already own.</p>
+    <p class="lede fd mt2">Most LocalAI backends wrap somebody else's engine. These were written from scratch, each one because the thing we needed was a multi-gigabyte Python install, or closed, or nobody had built it yet. What ships instead is a binary and a GGUF file, checked against the reference implementation in CI, running on the machine you already own.</p>
     <div class="acts fd">
       <a class="btn" href="/#start">Install LocalAI <span>&#8594;</span></a>
       <a class="btn btn--o" href="/docs/features/backends/">How backends work</a>
@@ -88,8 +88,8 @@
   <div class="shell">
     <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
     <p class="kicker rv">The rule we hold them to</p>
-    <h2 class="rv mt1" style="max-width:20ch">A port only ships once it matches the original.</h2>
-    <p class="lede rv mt2">Every engine here is gated against the framework it replaces, on the same input, on the same machine. That means a transcript that comes out word for word identical, boxes that land on the same pixels, or a waveform inside a stated tolerance. Speed is the part we then go and win, and the numbers on this page come out of each engine's own benchmark suite, not a marketing run.</p>
+    <h2 class="rv mt1" style="max-width:20ch">We do not ship a port until it matches the original.</h2>
+    <p class="lede rv mt2">Every engine here is gated against the framework it replaces, on the same input, on the same machine. That means a transcript identical to the reference, boxes that land on the same pixels, or a waveform inside a stated tolerance. Speed work comes after that, and the numbers on this page come out of each engine's own benchmark suite.</p>
     <div class="acts rv">
       <a class="btn" href="/#start">Install LocalAI &#8594;</a>
       <a class="btn btn--o" href="https://github.com/mudler/LocalAI">LocalAI on GitHub &#8599;</a>

website/layouts/index.html

@@ -37,7 +37,7 @@
   <div class="shell">
     <div class="bars rv" aria-hidden="true"><i></i><i></i><i></i><i></i></div>
     <p class="kicker rv">The runtime</p>
-    <h2 class="rv mt1" style="max-width:21ch">LocalAI is the engine everything else plugs into.</h2>
+    <h2 class="rv mt1" style="max-width:21ch">Everything else plugs into LocalAI.</h2>
     <p class="lede rv mt2">One binary with an OpenAI-compatible API in front of it. Point an existing client at it and the calls keep working, except now the model is on your machine. It also speaks the Anthropic, Ollama and ElevenLabs APIs, so most tools need a URL change and nothing else.</p>
     <p class="lede rv mt2">Underneath, a small core pulls each engine in as a separate backend, only when a model asks for it. That is why one install covers this much ground without becoming a 9 GB download.</p>
     <div class="apis rv">
@@ -75,7 +75,7 @@
       <div class="mi rv">
         <p class="mi__n">01 / HARDWARE</p>
         <h3>Every feature ships a CPU path first.</h3>
-        <p>Not a degraded mode that technically runs. The real one, tested in CI, on the hardware most people already have. GPUs make it faster, they are not the price of entry.</p>
+        <p>That path is tested in CI, on the hardware most people already have, and it is not a degraded fallback. A GPU makes it faster but is not required.</p>
         <p class="mi__meta">x86_64 · ARM64 · CUDA · ROCm · SYCL · Metal · Vulkan</p>
       </div>
       <div class="mi rv">
@@ -86,7 +86,7 @@
       </div>
       <div class="mi rv">
         <p class="mi__n">03 / DISTRIBUTED</p>
-        <h3>Plug in a second machine and stop there.</h3>
+        <h3>Add a second machine.</h3>
         <p>Routing, VRAM-aware placement, prefix-cache affinity and failover are the runtime's problem. You add hardware, the cluster works out what to do with it.</p>
         <p class="mi__meta">Smart routing · autoscaling · P2P · NATS · federation</p>
       </div>
@@ -174,7 +174,7 @@
       <div>
         <h3>parakeet.cpp</h3>
         <p class="spot__h">Twenty-seven times faster than whisper.cpp, on a CPU.</p>
-        <p>NVIDIA NeMo Parakeet, ported to C++ and ggml. Ten checkpoints, all of them verified at WER 0 against NeMo, which means the transcript comes out byte for byte identical while finishing first. Cache-aware streaming with end-of-utterance detection handles live audio, and the multilingual streaming model covers 40 or more locales.</p>
+        <p>NVIDIA NeMo Parakeet, ported to C++ and ggml. Ten checkpoints, all of them verified at WER 0 against NeMo, which means the transcript is identical to NeMo's while finishing first. Cache-aware streaming with end-of-utterance detection handles live audio, and the multilingual streaming model covers 40 or more locales.</p>
         <div class="facts">
           <div><b>27x</b><span>vs whisper.cpp, CPU</span></div>
           <div><b>1.40x</b><span>vs NeMo, CPU median</span></div>
@@ -395,7 +395,7 @@
           <p>Distributed mode with VRAM-aware routing, autoscaling, multi-user auth and per-user quotas.</p></div>
         <div class="tl__i"><p class="tl__d">MAY 2026</p><h4>It sees and hears</h4>
           <p>Voice recognition, face recognition with liveness, diarization, video generation, drop-in Ollama API.</p></div>
-        <div class="tl__i"><p class="tl__d">JUL 2026</p><h4>Nineteen engines of our own</h4>
+        <div class="tl__i"><p class="tl__d">JUL 2026</p><h4>Eighteen engines of our own</h4>
           <p>The native C and C++ ports take over the heavy Python backends, one modality at a time.</p></div>
       </div>
     </div>