chore: merge master into distributed transport PR

Bring the distributed branch onto current master before the CI fix.

Assisted-by: Codex:gpt-6

docs/content/advanced/model-configuration.md

@@ -1066,7 +1066,9 @@ known_usecases:
   - embeddings
 ```
 
-Available flags: `chat`, `completion`, `edit`, `embeddings`, `rerank`, `image`, `transcript`, `tts`, `sound_generation`, `tokenize`, `vad`, `video`, `detection`, `llm` (combination of CHAT, COMPLETION, EDIT).
+Available flags: `chat`, `completion`, `edit`, `embeddings`, `rerank`, `image`, `transcript`, `tts`, `sound_generation`, `tokenize`, `vad`, `video`, `detection`, `score`, `token_classify`, `decisions`, `llm` (combination of CHAT, COMPLETION, EDIT).
+
+`decisions` marks a model as a decision model for the [Decisions API]({{% relref "features/decisions" %}}) (`POST /v1/systemone`). It is never guessed, and a model that declares it is not listed as a chat, completion or embeddings model.
 
 `token_classify` marks a model as a token-classification (NER) provider for the PII filter (e.g. an `openai-privacy-filter` GGUF). Declare it explicitly together with `embeddings: true` (the classifier loads via TOKEN_CLS pooling). It runs on the dedicated `privacy-filter` backend (`backend/cpp/privacy-filter`), a standalone GGML engine for the `openai-privacy-filter` family - separate from `llama-cpp`, which no longer carries the token-classification path.
 

docs/content/features/audio-classification.md

@@ -9,6 +9,8 @@ Sound-event classification (audio tagging) answers the question **"what am I hea
 
 LocalAI exposes this through the `/v1/audio/classification` endpoint, modelled after `/v1/audio/transcriptions`. The reference backend is **[ced.cpp](https://github.com/localai-org/ced.cpp)** (CED, a 527-class AudioSet tagger), a small ViT over a log-mel spectrogram ported to ggml with full PyTorch parity. Apache-2.0 weights are redistributable as GGUF.
 
+**[parakeet.cpp](https://github.com/mudler/parakeet.cpp)** can also load a CED model (through `third_party/ced.cpp`) and serve `/v1/audio/classification` from the same backend used for ASR and diarization. It scores the clip in 10 s windows and averages each class's score across the windows before sorting and applying `top_k`/`threshold` - CED's own method for clips longer than one window. Install `parakeet-cpp-ced-tiny` or `parakeet-cpp-ced-base` from the gallery, or point `parameters.model` at a CED GGUF under `backend: parakeet-cpp`. A parakeet-cpp ASR model can also point `sound_model` at a CED GGUF to add live sound events during realtime transcription - see [Realtime API]({{% relref "openai-realtime" %}}).
+
 Because classification is exposed as a regular OpenAI-style endpoint, any HTTP client works - there is no Python dependency on the consumer side.
 
 In distributed mode, LocalAI stages uploaded audio and realtime sound-detection
@@ -59,6 +61,25 @@ curl http://localhost:8080/v1/audio/classification \
   -F top_k=10
 ```
 
+The same request works unchanged against a parakeet-cpp CED model:
+
+```yaml
+name: parakeet-ced-tiny
+backend: parakeet-cpp
+parameters:
+  model: ced-tiny-q8_0.gguf
+known_usecases:
+  - sound_classification
+```
+
+```bash
+curl http://localhost:8080/v1/audio/classification \
+  -H "Content-Type: multipart/form-data" \
+  -F file="@/path/to/clip.wav" \
+  -F model="parakeet-ced-tiny" \
+  -F top_k=10
+```
+
 ## See also
 
 - [Audio to Text]({{% relref "audio-to-text" %}}) - speech transcription

docs/content/features/audio-diarization.md

@@ -9,12 +9,13 @@ url = "/features/audio-diarization/"
 
 Speaker diarization answers the question **"who spoke when?"** - given an audio clip with multiple speakers, it returns time-stamped segments labelled with a stable speaker ID (`SPEAKER_00`, `SPEAKER_01`, …).
 
-LocalAI exposes this through the `/v1/audio/diarization` endpoint, modelled after `/v1/audio/transcriptions`. Four backends are supported today:
+LocalAI exposes this through the `/v1/audio/diarization` endpoint, modelled after `/v1/audio/transcriptions`. Five backends are supported today:
 
 - **[sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx)** - pyannote-3.0 segmentation + a speaker-embedding extractor (3D-Speaker, NeMo, WeSpeaker) + fast clustering. Pure diarization - no transcription cost. Recommended when you only need speaker turns.
 - **[vibevoice.cpp](https://github.com/microsoft/VibeVoice)** - produces speaker-labelled segments as a by-product of its long-form ASR pass, so you can optionally get a transcript per segment for free.
 - **[NeMo-Speech.cpp](https://github.com/NVIDIA/NeMo-Speech.cpp)** - NVIDIA Sortformer, served standalone by the [NeMo-Speech.cpp backend]({{%relref "features/nemo-speech-cpp" %}}). It is end to end, so the speaker capacity is fixed by the checkpoint and the count hints are ignored. The same backend can instead put speaker tags on a transcript, by attaching a Sortformer model to an ASR one.
 - **[audio.cpp](https://github.com/0xShug0/audio.cpp)** - the `sortformer_diar` family, served by the multi-modality [audio.cpp backend]({{%relref "features/audio-cpp" %}}).
+- **[parakeet.cpp](https://github.com/mudler/parakeet.cpp)** - NVIDIA Nemotron-3-Diarization (Sortformer, up to 8 speakers), served standalone or paired with a Parakeet ASR model for per-segment text. See the [Audio to Text]({{% relref "audio-to-text" %}}) page for the parakeet-cpp option reference.
 
 Because diarization is exposed as a regular OpenAI-compatible endpoint, any HTTP client works. There is no Python dependency on pyannote or NeMo on the consumer side.
 
@@ -157,10 +158,38 @@ curl http://localhost:8080/v1/audio/diarization \
   -F response_format=verbose_json
 ```
 
+## Backend setup - parakeet-cpp (Nemotron-3-Diarization)
+
+Nemotron-3-Diarization is Sortformer, served standalone or paired with a Parakeet ASR model. Install `parakeet-cpp-nemotron-3-diarization` from the gallery for diarization only, or `parakeet-cpp-nemotron-3-diarization-asr` for the same model paired with `parakeet-cpp-tdt_ctc-110m` through the `asr_model` option:
+
+```yaml
+name: parakeet-diarize
+backend: parakeet-cpp
+parameters:
+  model: nemotron-3-diarization-q8_0.gguf
+options:
+  - asr_model:tdt_ctc-110m-f16.gguf
+known_usecases:
+  - diarization
+```
+
+Getting text on each segment needs both: an `asr_model` companion loaded on the model, and `include_text=true` on the request. With only one of the two, segments carry no text and no error is raised. Sortformer has a fixed speaker capacity and no clustering stage, so `num_speakers`, `min_speakers`, `max_speakers` and `clustering_threshold` are ignored (logged at debug); `min_duration_on` and `min_duration_off` are honored. Speaker labels are the decimal index the model assigned (`"0"`, `"1"`, …), or `"unknown"` when a segment has no diarized speaker.
+
+```bash
+curl http://localhost:8080/v1/audio/diarization \
+  -H "Content-Type: multipart/form-data" \
+  -F file="@meeting.wav" \
+  -F model="parakeet-diarize" \
+  -F include_text=true \
+  -F response_format=verbose_json
+```
+
+Sortformer clusters on voice-like characteristics, not on "is this a human". A loud non-speech sound with voice-like pitch and rhythm (a rooster crow, in one test clip) can come back as its own speaker segment alongside the real speakers. This is model behavior, not a bug in the LocalAI integration: treat an unexpected extra speaker as a hint the clip may contain a non-speech sound, and use [Sound Classification]({{% relref "audio-classification" %}}) to confirm what it is.
+
 ## Notes
 
 - **Speaker identity across files**: speaker IDs (`SPEAKER_00`, `SPEAKER_01`, …) are local to each request. To track the same person across multiple recordings, combine `/v1/audio/diarization` with `/v1/voice/embed` (speaker embedding) and maintain your own embedding store.
-- **Hints vs. forces**: `num_speakers` overrides clustering when set; `min_speakers` / `max_speakers` are advisory and only honored by backends that expose a range hint. vibevoice.cpp ignores them - its model picks the count itself.
+- **Hints vs. forces**: `num_speakers` overrides clustering when set; `min_speakers` / `max_speakers` are advisory and only honored by backends that expose a range hint. vibevoice.cpp and parakeet-cpp (Sortformer) ignore them - the model picks the count itself.
 - **Sample rate**: input is automatically converted to 16 kHz mono via ffmpeg before the backend sees it; sherpa-onnx pyannote-3.0 requires 16 kHz.
 
 ## See also

docs/content/features/audio-to-text.md

@@ -12,7 +12,7 @@ The transcription endpoint allows to convert audio files to text. The endpoint s
 - **moonshine**: Ultra-fast transcription engine optimized for low-end devices
 - **faster-whisper**: Fast Whisper implementation with CTranslate2
 - **WhisperX**: Whisper transcription with word alignment and optional speaker diarization. Set `HF_TOKEN` and pass `diarize=true` to load WhisperX's gated pyannote diarization pipeline.
-- **[parakeet-cpp](https://github.com/mudler/parakeet.cpp)**: A C++/ggml port of NVIDIA NeMo Parakeet (FastConformer TDT/CTC/RNNT/hybrid). Runs quantized GGUFs on CPU or GPU, emits word-level timestamps, and supports cache-aware streaming (the `realtime_eou` model surfaces end-of-utterance events).
+- **[parakeet-cpp](https://github.com/mudler/parakeet.cpp)**: A C++/ggml port of NVIDIA NeMo Parakeet (FastConformer TDT/CTC/RNNT/hybrid). Runs quantized GGUFs on CPU or GPU, emits word-level timestamps, and supports cache-aware streaming (the `realtime_eou` model surfaces end-of-utterance events). The same backend also loads Nemotron-3-Diarization (`/v1/audio/diarization`) and CED sound models (`/v1/audio/classification`), and can attach either as a companion to a transcription model.
 - **llama-cpp**: Route transcription to any multimodal-audio GGUF model served by the `llama-cpp` backend (e.g. [Qwen3-ASR](https://huggingface.co/ggml-org/Qwen3-ASR-0.6B-GGUF), Voxtral, Qwen2-Audio). Under the hood the request is converted into a chat completion with the audio attached via the model's audio encoder - the same path the upstream llama.cpp server uses. Set `backend: llama-cpp` in the model YAML and point `mmproj` at the matching audio encoder.
 - **voxtral**: Voxtral-family models served by a dedicated backend
 - **[NeMo-Speech.cpp](https://github.com/NVIDIA/NeMo-Speech.cpp)**: NVIDIA's C++/ggml runtime for the Nemotron Speech models. Serves offline, streaming and live transcription, with VAD, punctuation, inverse text normalization and Sortformer speaker tags attached through model options, and covers diarization, speech synthesis and translation from the same backend. See the [NeMo-Speech.cpp backend]({{%relref "features/nemo-speech-cpp" %}}) page for the model options.
@@ -190,6 +190,21 @@ curl http://localhost:8080/v1/audio/transcriptions \
 
 For real-time use, load a cache-aware streaming model (e.g. `realtime_eou_120m-v1-*.gguf`) and pass `-F stream=true`. Deltas are emitted as the audio is decoded, with end-of-utterance events closing each segment.
 
+### Diarization and sound classification
+
+The same backend also serves the `/v1/audio/diarization` and `/v1/audio/classification` endpoints, and can attach a diarization or sound model to a live transcription session. `options:` accepts paths relative to the models directory, or absolute:
+
+| Option | Allowed on | Used for |
+|---|---|---|
+| `asr_model:<path>` | a diarization model | `include_text` on `/v1/audio/diarization` |
+| `diarization_model:<path>` | an ASR model | a `speaker` on transcript segments (and words), and speaker segments during realtime live transcription |
+| `sound_model:<path>` | an ASR model | sound events during realtime live transcription |
+| `diarization_latency:<model\|low\|very_low\|ultra_low>` | a model with a diarization companion | latency mode for the live speaker stream; default `low` |
+
+With a `diarization_model` companion, `/v1/audio/transcriptions` labels each segment with its `speaker` (`"0"`, `"1"`, ... in order of first appearance) and splits segments where the speaker changes; with `timestamp_granularities[]=word` each word carries its speaker too. With `stream=true` the closing `transcript.text.done` event lists the segments with their speakers. Pass `-F diarize=false` to skip diarization for one request. The diarization GGUF can also be imported directly: `local-ai models import https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/nemotron-3-diarization-f16.gguf`.
+
+The loader rejects a companion whose role duplicates the primary's own (for example `asr_model:` on an already-ASR primary, or `sound_model:` on a CED primary), and rejects a companion GGUF that does not match the role its option names (for example `sound_model:` pointing at an ASR GGUF fails to load, naming the kind it expected). See [Speaker Diarization]({{% relref "audio-diarization" %}}) for the `Diarize` RPC and [Sound Classification]({{% relref "audio-classification" %}}) for `SoundDetection`, and [Realtime API]({{% relref "openai-realtime" %}}) for the live speaker/sound events emitted during a realtime session.
+
 ### Segment timestamps
 
 Transcriptions are split into segments the same way NVIDIA NeMo does: a new segment starts after sentence-ending punctuation (`.`, `?`, `!`), and each segment carries `start`/`end` times. This is the default (NeMo's punctuation-only segmentation) and needs no configuration. While streaming, each end-of-utterance closes a segment, now with timestamps.

docs/content/features/decisions.md

@@ -0,0 +1,164 @@
++++
+disableToc = false
+title = "Decisions API"
+weight = 66
+url = "/features/decisions/"
++++
+
+The Decisions API is a fast, typed decision layer. You send a piece of text (the
+*state*) and a set of named questions. A decision model answers each question
+with a value and a confidence, in one pass. The model does not generate text, so
+there is nothing to parse and no free-form output to validate.
+
+LocalAI serves it on the `/v1/systemone` routes. The request and response shapes
+follow the [kev](https://github.com/jaredpalmer/kev) project, and the field names
+and question types are the same ones Ollama serves on its `/v1/systemone`
+endpoint (Ollama 0.35 and later). The wire contract is called SystemOne; the
+capability a model declares is called `decisions`. See
+[Compatibility with Ollama](#compatibility-with-ollama) for what differs.
+
+OpenAI announced its own Decisions API in limited preview on 2026-09-29. It has no
+public request or response schema yet, so LocalAI does not serve a `/v1/decisions`
+route.
+
+## Endpoints
+
+| Endpoint | Method | Description |
+|---|---|---|
+| `/v1/systemone` | POST | Answer all questions in one pass |
+| `/v1/systemone/permute` | POST | Re-run one choice question under `n_perm` option orders |
+| `/v1/systemone/separate` | POST | Answer each question in its own pass |
+
+Which route a model can serve depends on its kind:
+
+| Model kind | `/v1/systemone` | `/permute` and `/separate` |
+|---|---|---|
+| Decision model (`decisions`), such as Laya or GLiNER2.5-Decide | Yes | No, returns `400` |
+| Zero-shot NER model (`token_classify`), such as GLiNER2.5 | Yes, through the NER path | Yes |
+
+## Question types
+
+| Type | Answer | Fields in the answer |
+|---|---|---|
+| `choice` | One option out of a named set | `choice`, `probabilities`, `confidence` |
+| `noul` | Yes, no or unknown for a statement | `noul` (0 to 1), `entities` |
+| `score` | One level on a scale | `score`, `legend`, `probabilities`, `confidence` |
+
+## Example
+
+```bash
+curl http://localhost:8080/v1/systemone -H "Content-Type: application/json" -d '{
+  "model": "laya-vllm-cpp",
+  "state": "My order arrived broken and I want my money back. This is the second time.",
+  "questions": {
+    "team": {
+      "type": "choice",
+      "instructions": "Which team should handle this ticket?",
+      "criteria": {
+        "billing": "Payments, invoices and refunds",
+        "shipping": "Delivery and damaged goods",
+        "product": "Questions about how the product works"
+      }
+    },
+    "refund_requested": {
+      "type": "noul",
+      "instructions": "The customer explicitly asks for a refund"
+    },
+    "urgency": {
+      "type": "score",
+      "instructions": "How urgent is this ticket?",
+      "criteria": ["not urgent", "somewhat urgent", "urgent", "critical"]
+    }
+  }
+}'
+```
+
+Answers from a decision model carry a `confidence` value, and the response
+reports token usage and `latency_ms`. The NER path does not report token usage.
+
+## Choosing a model
+
+A model can serve the Decisions API only if it is a decision model. Declare the usecase
+in the model config:
+
+```yaml
+name: laya
+backend: vllm-cpp
+known_usecases:
+  - decisions
+parameters:
+  model: convaiinnovations/laya
+```
+
+`decisions` is never guessed, and a model that declares it is not listed as a
+chat, completion or embeddings model. A model that declares usecases without
+`decisions` or `token_classify` gets a `400` from these endpoints that names the
+missing usecase. A model that declares `token_classify` and not `decisions` is
+served by the zero-shot NER path. A vllm-cpp config that declares no usecases is
+treated as a decision model, so setups that predate the flag keep working, but a
+config that declares only `chat` (as an older `laya` gallery entry did) now gets
+the `400` and needs `known_usecases: [decisions]`.
+
+Install one from the gallery and filter on the `decisions` tag:
+
+| Gallery entry | Model | Notes |
+|---|---|---|
+| `laya-vllm-cpp` | Laya | ModernBERT-large, non-autoregressive, about 800 MB |
+| `gliner25-decide-vllm-cpp` | GLiNER2.5-Decide | DeBERTa-v3-large with a classification head, about 2 GB |
+| `tev1-4b-vllm-cpp` | Tev1 4B | Autoregressive Qwen3.5-4B fine-tune that answers with an option letter, about 9.3 GB |
+| `tev1-0.8b-vllm-cpp` | Tev1 0.8B | Autoregressive Qwen3.5-0.8B fine-tune that answers with an option letter, about 1.8 GB |
+| `kev-0.8b-vllm-cpp` | kev 0.8B | Qwen3.5-0.8B-Base with a merged LoRA and a PointerHead readout, converted for vllm.cpp only, about 1.53 GB |
+
+The engine, [vllm.cpp]({{% relref "features/vllm-cpp" %}}), also supports the
+CLM and xor decision models. Those checkpoints need a conversion step, so
+they are not gallery entries yet. The kev entry installs a checkpoint that was
+already converted with the vllm.cpp `convert-kev.py` script.
+
+Tev1 is an autoregressive decision model. The engine answers each question by
+scoring the option letters, so its `confidence` is the entropy measure Ollama
+uses. A Tev1 `choice` or `score` question accepts at most 24 options (Ollama
+allows 26), because the model is trained on the letters A to X, and every
+option needs a nonempty description. The published checkpoints name another
+architecture in `config.json`, so the Tev1 gallery entries set
+`engine_args.hf_overrides` to load them as `Tev1Model` (see
+[Overriding config.json keys]({{% relref "features/vllm-cpp" %}}#overriding-configjson-keys-hf_overrides)).
+The same model also answers `/v1/chat/completions` requests.
+
+## Request limits
+
+A request is refused with `400` (or `413` for the body size) when:
+
+- the body is larger than 64 KiB,
+- `state` is missing or blank,
+- there are no questions, or more than 64,
+- a question id is blank,
+- a `choice` question has fewer than 2 options or a blank option key,
+- a `score` question has fewer than 2 levels,
+- a `noul` question has `criteria` with keys other than `"false"` and `"true"`.
+
+A `noul` question may carry `criteria` with a description for each outcome, for
+example `{"false": "No refund is requested", "true": "The customer requests a refund"}`.
+Some models cap the number of options for a `choice` or `score` question. Models
+that answer with a letter accept at most 26, and Tev1 accepts at most 24. The engine refuses more options than
+the model supports and the error names the limit.
+
+## Compatibility with Ollama
+
+The field names, question types and answer fields are the same as Ollama's
+`/v1/systemone`, so a client written for one works against the other for the
+common case. These behaviors differ:
+
+| | Ollama | LocalAI |
+|---|---|---|
+| `confidence` | `1 - H(p) / ln(N)`, an entropy measure | Computed by the model's pipeline. For kev and Laya it is a normalized margin, so the same probabilities give a different value |
+| Errors | `{"error": "message"}` | `{"error": {"message": "...", "type": "invalid_request"}}` |
+| `keep_alive` | Sets how long the model stays loaded | Accepted and ignored. Model lifetime follows the LocalAI idle and watchdog settings |
+| `state` given as an object | Serialized as JSON text | Rendered as labeled lines, the way kev does it |
+| `noul` answer on the NER path | `{type, noul}` | Also carries `entities` |
+| Token `usage` | Full prompt lengths across all questions | Whatever the backend reports; the NER path reports 0 |
+
+## Access control
+
+When authentication is on, the three routes need the `decisions` feature. It is
+on by default for every user, like the other API features, and an administrator
+can turn it off per user.

docs/content/features/openai-realtime.md

@@ -129,6 +129,113 @@ A client `session.update` still overrides `type` and `eagerness` per session.
 - `false` (default): the transcript accumulated from the live stream is used as-is - the model runs once per utterance and the LLM starts immediately at commit.
 - `true`: the committed audio is re-transcribed offline. If the batch decode also ends with the end-of-utterance token the turn proceeds (using the batch transcript); if it does **not**, the commit is cancelled and the session keeps listening - treating the streaming token as a false positive. Both transcripts are compared and logged, which makes this mode a useful diagnostic for how well the streaming and batch decodes align, at the cost of one extra decode per turn.
 
+### Live speaker and sound events (parakeet-cpp)
+
+When the `semantic_vad` transcription model is a parakeet-cpp model loaded with a `diarization_model` and/or `sound_model` companion (see [Audio to Text]({{% relref "audio-to-text" %}})), the realtime session also streams speaker and sound events while a turn is live, alongside the transcript deltas. Nothing needs to change on the client: unrecognized event types are ignored by standard OpenAI Realtime clients.
+
+The transcription model, with its companions:
+
+```yaml
+name: parakeet-realtime-scene
+backend: parakeet-cpp
+parameters:
+  model: realtime_eou_120m-v1-f16.gguf
+options:
+  - diarization_model:nemotron-3-diarization-q8_0.gguf
+  - sound_model:ced-tiny-q8_0.gguf
+```
+
+The realtime pipeline that uses it:
+
+```yaml
+name: gpt-realtime
+pipeline:
+  vad: silero-vad-ggml
+  transcription: parakeet-realtime-scene
+  llm: qwen3-4b
+  tts: tts-1
+  turn_detection:
+    type: semantic_vad
+```
+
+Each closed speaker segment emits a `conversation.item.input_audio_transcription.segment` event under the turn's item id, with an empty `text` (the event exists to carry the speaker boundary, not a transcript - the transcript still comes from the ordinary delta/completed events):
+
+```json
+{
+  "type": "conversation.item.input_audio_transcription.segment",
+  "item_id": "item_abc",
+  "content_index": 0,
+  "speaker": "0",
+  "start": 1.92,
+  "end": 4.10,
+  "text": ""
+}
+```
+
+Each sound event emits a `conversation.item.sound_detection` event with one tag and the detection window's `start`/`end`:
+
+```json
+{
+  "type": "conversation.item.sound_detection",
+  "item_id": "item_abc",
+  "content_index": 0,
+  "detections": [{"label": "Chicken, rooster", "score": 0.91, "index": 99}],
+  "start": 24.0,
+  "end": 30.0
+}
+```
+
+The `start`/`end` on both event types are seconds measured from the start of the current turn's own audio, not the session or the WebSocket connection - the same base the streamed transcript words use.
+
+The companion stream is opened fresh for each speech turn, alongside that turn's ASR live session, and closed when the turn commits: whatever it had not yet emitted is drained and sent at that point. Because the diarization model runs a brand new session every turn, its speaker indices are scoped to the turn too - `"speaker": "0"` in one turn and `"speaker": "0"` in the next are not guaranteed to be the same person, even within the same conversation.
+
+`score` is the peak score seen for that tag while the sound was live, not an average.
+
+**Limitation**: under `semantic_vad`, live transcription (and so this companion stream) only runs during speech turns - it does not see audio between turns. A sound that happens while nobody is speaking is not detected this way. If you need sound events independent of speech turns, use the pipeline's `sound_detection` model instead (see [Sound Classification]({{% relref "audio-classification" %}})), which classifies each VAD-committed utterance on its own. Use one or the other, not both, on the same session - they overlap in purpose and would emit sound detections twice.
+
+### Speaker and sound events with an offline model (Parakeet TDT v3)
+
+The live events above need a cache-aware streaming transcription model. An offline model such as Parakeet TDT 0.6B v3 (25 languages) runs under `server_vad` instead: each VAD-committed turn is transcribed as a whole. The gallery model `parakeet-cpp-realtime-scene-tdt` bundles it with Nemotron-3-Diarization and CED-Tiny, so one parakeet-cpp backend handles transcription, speakers and sounds. Point both `transcription` and `sound_detection` at it and turn on `diarization`:
+
+```yaml
+name: gpt-realtime-scene
+pipeline:
+  vad: silero-vad-ggml
+  transcription: parakeet-cpp-realtime-scene-tdt
+  sound_detection: parakeet-cpp-realtime-scene-tdt
+  diarization: true
+  llm: qwen3-4b
+  tts: tts-1
+```
+
+`pipeline.diarization` asks the transcription model for speaker labels on each committed turn and emits every labelled segment as a `conversation.item.input_audio_transcription.segment` event before the turn's `completed` event. Unlike the live path, these segments carry their `text`:
+
+```json
+{
+  "type": "conversation.item.input_audio_transcription.segment",
+  "item_id": "item_abc",
+  "content_index": 0,
+  "id": "seg_1",
+  "speaker": "1",
+  "start": 6.85,
+  "end": 10.82,
+  "text": "Well, I don't wish to see it any more, observed Phoebe, turning away her eyes."
+}
+```
+
+`sound_detection` classifies the same committed audio and emits one `conversation.item.sound_detection` event per turn (see [Sound Classification]({{% relref "audio-classification" %}})). As on the live path, times are relative to the turn's audio and speaker labels are only consistent within a turn. `pipeline.diarization` is off by default: it needs a transcription model that diarizes (parakeet-cpp with a `diarization_model` companion), and some other backends fail a diarization request they cannot serve.
+
+#### Choosing the sound model
+
+Both scene models ship with CED-Tiny, the cheapest to run all the time. `parakeet-cpp-realtime-scene-base` and `parakeet-cpp-realtime-scene-tdt-base` are the same pipelines with CED-Base (86M, the largest CED), which tags sounds more confidently. Any CED GGUF from [`mudler/ced-gguf`](https://huggingface.co/mudler/ced-gguf) (tiny, mini, small, base) works as `sound_model`. Measured on CPU (Ryzen 9 9950X3D) over a 37 s clip with two speakers and a rooster, as a fraction of real time:
+
+| | CED-Tiny | CED-Base |
+|---|---|---|
+| Live scene stream (diarization `low` + sound), EOU path | 0.103 | 0.125 |
+| Sound detection per committed turn, TDT path | 0.005 | 0.031 |
+
+The EOU model's own ASR stream adds 0.016. Diarization dominates the live cost, so CED-Base keeps the live path about 7x faster than real time.
+
 ### Disabling thinking
 
 For reasoning models, you can force the pipeline LLM's thinking off without editing the LLM model config:

docs/content/features/text-generation.md

@@ -1094,6 +1094,7 @@ engine_args:
 | `tokenizer_config` | Override the `tokenizer_config.json` the chat template is read from | `<model_dir>/tokenizer_config.json` |
 | `speculative_config` | Speculative decoding (see below) | disabled |
 | `kv_transfer_config` | External KV connector / LMCache (see below) | none |
+| `hf_overrides` | JSON object of `config.json` keys merged over the model directory's own, as vLLM's `--hf-overrides` (see the [vllm.cpp backend page]({{% relref "features/vllm-cpp" %}}#overriding-configjson-keys-hf_overrides)) | none |
 
 Raising `max_num_batched_tokens` lets more prefill land in a single step, at the
 cost of decode latency for requests queued behind it. The default deliberately

docs/content/features/vllm-cpp.md

@@ -133,6 +133,39 @@ engine_args:
   tool_parser: qwen3_coder
 ```
 
+## Overriding config.json keys (`hf_overrides`)
+
+`engine_args.hf_overrides` is a JSON object of top-level `config.json` keys that
+the backend merges over the model directory's own `config.json` before the
+engine loads it, like vLLM's `--hf-overrides`. The main use is to opt a published
+checkpoint into an engine adapter that its config does not name. For example,
+the Tev1 repositories declare `Qwen3_5ForConditionalGeneration`, and vllm.cpp
+serves them as decision models only when the architecture is `Tev1Model`:
+
+```yaml
+engine_args:
+  hf_overrides:
+    architectures: ["Tev1Model"]
+```
+
+The downloaded model files do not change. At load the backend creates a private
+temporary directory. It writes the merged `config.json` there and adds a
+symlink for each other entry of the model directory (weights, tokenizer files,
+a `tokenizer/` subdirectory). Then it gives that directory to the engine. When
+the model unloads, the backend removes the directory.
+
+Rules:
+
+- The merge is top-level only. An override key replaces the whole value of that
+  key, including a nested object such as `text_config`.
+- The model must be a directory that contains a `config.json`. A `.gguf` file
+  or a directory without `config.json` fails the load.
+- A value that is not a JSON object (an array, a scalar, or JSON that does not
+  parse) fails the load. The backend does not ignore it, because loading the
+  unchanged config would serve a different architecture than the one you
+  configured.
+- An empty object (`{}`) does nothing.
+
 ## Named entity recognition (GLiNER2.5)
 
 The `vllm-cpp` backend serves [GLiNER2.5](https://huggingface.co/fastino/gliner2.5-multi-v1),
@@ -160,22 +193,25 @@ forward, which is the required contract for pooling models in vllm.cpp. A
 device-resident forward is tracked as a performance optimization, not a
 correctness gap.
 
-### SystemOne structured-extraction API
+### Decisions API
 
-The `vllm-cpp` backend also exposes kev-compatible SystemOne endpoints that
-turn zero-shot NER into structured question answering. These mirror the API
-from the [kev](https://github.com/jaredpalmer/kev) project:
+The `vllm-cpp` backend serves the kev-compatible SystemOne endpoints (the Decisions API): typed
+`choice`, `noul` and `score` questions over a state text, answered by a
+non-generative decision model in one pass. A decision model declares
+`known_usecases: [decisions]`. See [Decisions API]({{% relref "features/decisions" %}})
+for the request shape, the models you can install and the access rules.
 
 | Endpoint | Method | Description |
 |---|---|---|
-| `/v1/systemone` | POST | Answer all questions in one NER pass |
+| `/v1/systemone` | POST | Answer all questions in one pass |
 | `/v1/systemone/permute` | POST | Re-run one choice question under n_perm option orders |
-| `/v1/systemone/separate` | POST | Answer each question in its own NER pass (N passes) |
+| `/v1/systemone/separate` | POST | Answer each question in its own pass (N passes) |
 
-Each question has a `type` of `noul` (binary entity presence), `choice` (pick
-one option), or `score` (pick one level). The `model` field in the request body
-selects the NER model. Labels are derived from the question definition, so no
-`ner_labels` configuration is needed for these endpoints.
+The GLiNER2.5 zero-shot NER model (`token_classify`) also serves
+`/v1/systemone`, through the NER path, and it is the model to use for
+`/v1/systemone/permute` and `/v1/systemone/separate`, which decision models
+refuse with a `400`. It derives its NER labels from the question definitions, so
+no `ner_labels` configuration is needed.
 
 ## Beyond text generation