mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 01:48:06 -04:00
fix(ci): repair nightly backend dep bumps for renamed localai-org repos (#11012)
The "Bump Backend dependencies" workflow has failed every night for over ten days. Four upstreams — ced.cpp, moss-transcribe.cpp, voice-detect.cpp and rf-detr.cpp — moved from the mudler org to localai-org, so the GitHub API answers 301 for the old slugs. ced.cpp additionally renamed its default branch to main. bump_deps.sh fetched without -L or -f and never checked the response, so the redirect's JSON body was passed straight to sed, which died with "unterminated `s' command". The loud failure was luck: an error body without slashes would have been substituted into the Makefile as the new pin, silently corrupting the version and shipping it in a bump PR. Point the matrix at the new slugs and branch, and harden the script so a bad response can never reach sed: follow redirects, fail on HTTP errors, and require a bare 40-hex SHA before rewriting anything. Also refresh the now-stale repository URLs in the backend Makefiles, test scripts, backend/index.yaml and the docs. Verified all 25 matrix entries resolve to a commit SHA and that the four previously-failing jobs run end to end against the real API. Assisted-by: Claude:claude-opus-4-8 [Claude Code] Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
This commit is contained in:
committed by
GitHub
parent
7bda73fd66
commit
1e0baec2a7
14
.github/bump_deps.sh
vendored
14
.github/bump_deps.sh
vendored
@@ -9,7 +9,19 @@ if [ -z "$FILE" ]; then
|
||||
FILE="Makefile"
|
||||
fi
|
||||
|
||||
LAST_COMMIT=$(curl -s -H "Accept: application/vnd.github.VERSION.sha" "https://api.github.com/repos/$REPO/commits/$BRANCH")
|
||||
# -L so a renamed/transferred upstream repo (GitHub answers 301) still
|
||||
# resolves instead of handing us the redirect body, and -f so an HTTP error
|
||||
# aborts the run rather than letting an error page reach sed below.
|
||||
LAST_COMMIT=$(curl -sfL -H "Accept: application/vnd.github.VERSION.sha" "https://api.github.com/repos/$REPO/commits/$BRANCH")
|
||||
|
||||
# Guard the sed input: anything that is not a bare 40-hex SHA (an API error
|
||||
# body, an empty response) would otherwise be spliced into the Makefile pin —
|
||||
# either corrupting it silently or blowing up sed with an unterminated
|
||||
# expression, which is how this job failed for a renamed repo.
|
||||
if ! [[ "$LAST_COMMIT" =~ ^[0-9a-f]{40}$ ]]; then
|
||||
echo "Refusing to bump $VAR: expected a 40-char commit SHA for $REPO@$BRANCH, got: $LAST_COMMIT" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Read $VAR from Makefile (only first match)
|
||||
set +e
|
||||
|
||||
10
.github/workflows/bump_deps.yaml
vendored
10
.github/workflows/bump_deps.yaml
vendored
@@ -50,15 +50,15 @@ jobs:
|
||||
variable: "PARAKEET_VERSION"
|
||||
branch: "master"
|
||||
file: "backend/go/parakeet-cpp/Makefile"
|
||||
- repository: "mudler/moss-transcribe.cpp"
|
||||
- repository: "localai-org/moss-transcribe.cpp"
|
||||
variable: "MOSS_VERSION"
|
||||
branch: "master"
|
||||
file: "backend/go/moss-transcribe-cpp/Makefile"
|
||||
- repository: "mudler/ced.cpp"
|
||||
- repository: "localai-org/ced.cpp"
|
||||
variable: "CED_VERSION"
|
||||
branch: "master"
|
||||
branch: "main"
|
||||
file: "backend/go/ced/Makefile"
|
||||
- repository: "mudler/voice-detect.cpp"
|
||||
- repository: "localai-org/voice-detect.cpp"
|
||||
variable: "VOICEDETECT_VERSION"
|
||||
branch: "master"
|
||||
file: "backend/go/voice-detect/Makefile"
|
||||
@@ -90,7 +90,7 @@ jobs:
|
||||
variable: "SAM3_VERSION"
|
||||
branch: "main"
|
||||
file: "backend/go/sam3-cpp/Makefile"
|
||||
- repository: "mudler/rf-detr.cpp"
|
||||
- repository: "localai-org/rf-detr.cpp"
|
||||
variable: "RFDETR_VERSION"
|
||||
branch: "main"
|
||||
file: "backend/go/rfdetr-cpp/Makefile"
|
||||
|
||||
10
README.md
10
README.md
@@ -177,7 +177,7 @@ For more details, see the [Getting Started guide](https://localai.io/basics/gett
|
||||
|
||||
## Latest News
|
||||
|
||||
- **June 2026**: New native biometric backends from the LocalAI team: [voice-detect.cpp](https://github.com/mudler/voice-detect.cpp) for speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion) and [face-detect.cpp](https://github.com/mudler/face-detect.cpp) for face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace). Both are from-scratch C++/ggml engines with no Python or onnxruntime at inference, self-contained GGUF weights, bit-exact parity with the reference, and GPU cuDNN parity, replacing the heavier Python `insightface` and `speaker-recognition` backends ([PR #10441](https://github.com/mudler/LocalAI/pull/10441)).
|
||||
- **June 2026**: New native biometric backends from the LocalAI team: [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) for speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion) and [face-detect.cpp](https://github.com/mudler/face-detect.cpp) for face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace). Both are from-scratch C++/ggml engines with no Python or onnxruntime at inference, self-contained GGUF weights, bit-exact parity with the reference, and GPU cuDNN parity, replacing the heavier Python `insightface` and `speaker-recognition` backends ([PR #10441](https://github.com/mudler/LocalAI/pull/10441)).
|
||||
- **June 2026**: New [realtime voice assistant demo](https://github.com/localai-org/localai-realtime-demo) (a tiny Go client for the Realtime API with a full talk-back voice loop and tool calling), plus [streaming of the realtime LLM / TTS / transcription pipeline stages](https://github.com/mudler/LocalAI/pull/10176) and [configurable WebRTC ICE candidates](https://github.com/mudler/LocalAI/pull/10231).
|
||||
- **June 2026**: Big speech push: the [parakeet.cpp](https://github.com/mudler/parakeet.cpp) ASR engine gains [NeMo-faithful segment timestamps](https://github.com/mudler/LocalAI/pull/10207), a [multilingual streaming Nemotron-3.5 model](https://github.com/mudler/LocalAI/pull/10199), [dynamic batching for concurrent transcription](https://github.com/mudler/LocalAI/pull/10112) and [CUDA graphs](https://github.com/mudler/LocalAI/pull/10273); the new [CrispASR backend](https://github.com/mudler/LocalAI/pull/10099) adds multi-architecture ASR + TTS, and [60 Piper TTS voices across 42 languages](https://github.com/mudler/LocalAI/pull/10296) land in the gallery (plus [per-request TTS instructions and params](https://github.com/mudler/LocalAI/pull/10172)).
|
||||
- **June 2026**: New backends and models: [locate-anything.cpp](https://github.com/mudler/LocalAI/pull/10264) for open-vocabulary object detection via ggml, [Ideogram4 image generation](https://github.com/mudler/LocalAI/pull/10201) in stablediffusion-ggml, [llama.cpp video input](https://github.com/mudler/LocalAI/pull/10216), and the [Gemma 4 QAT family with MTP speculative-decoding pairs](https://github.com/mudler/LocalAI/pull/10215). Plus an [interactive CLI chat mode](https://github.com/mudler/LocalAI/pull/10226) and [RAG source citations in agent responses](https://github.com/mudler/LocalAI/pull/10228).
|
||||
@@ -232,13 +232,13 @@ Most backends wrap a best-in-class upstream engine. A handful of them are native
|
||||
| Backend | What it does |
|
||||
|---------|-------------|
|
||||
| [parakeet.cpp](https://github.com/mudler/parakeet.cpp) | C++/GGML port of NVIDIA NeMo Parakeet ASR (tdt/ctc/rnnt/hybrid), with cache-aware streaming transcription |
|
||||
| [moss-transcribe.cpp](https://github.com/mudler/moss-transcribe.cpp) | C++/GGML port of OpenMOSS MOSS-Transcribe-Diarize: joint long-form transcription, speaker diarization and timestamping in a single pass |
|
||||
| [moss-transcribe.cpp](https://github.com/localai-org/moss-transcribe.cpp) | C++/GGML port of OpenMOSS MOSS-Transcribe-Diarize: joint long-form transcription, speaker diarization and timestamping in a single pass |
|
||||
| [moss-tts.cpp](https://github.com/mudler/moss-tts.cpp) | C++/GGML port of the OpenMOSS MOSS-TTS family: text-to-speech (MOSS-TTS-Local v1.5, 48 kHz stereo) with reference-audio voice cloning, through the MOSS-Audio-Tokenizer neural codec |
|
||||
| [ced.cpp](https://github.com/mudler/ced.cpp) | C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition |
|
||||
| [voice-detect.cpp](https://github.com/mudler/voice-detect.cpp) | Speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend |
|
||||
| [ced.cpp](https://github.com/localai-org/ced.cpp) | C++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition |
|
||||
| [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) | Speaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend |
|
||||
| [voxtral-tts.c](https://github.com/mudler/voxtral-tts.c) | Voxtral Realtime 4B speech-to-text in pure C |
|
||||
| [vibevoice.cpp](https://github.com/mudler/vibevoice.cpp) | Native port of Microsoft VibeVoice for TTS (voice cloning) and long-form ASR with speaker diarization |
|
||||
| [rf-detr.cpp](https://github.com/mudler/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation |
|
||||
| [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation |
|
||||
| [locate-anything.cpp](https://github.com/mudler/locate-anything.cpp) | Open-vocabulary object detection and visual grounding (LocateAnything-3B) |
|
||||
| [depth-anything.cpp](https://github.com/mudler/depth-anything.cpp) | Depth Anything 3 monocular metric depth + camera pose estimation |
|
||||
| [face-detect.cpp](https://github.com/mudler/face-detect.cpp) | Face detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace), replacing the Python insightface backend |
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
# go build -o ced-grpc .
|
||||
|
||||
CED_VERSION?=c04ac14b7992d00584d9e812c9bb6268598a6ce7
|
||||
CED_REPO?=https://github.com/mudler/ced.cpp
|
||||
CED_REPO?=https://github.com/localai-org/ced.cpp
|
||||
|
||||
GOCMD?=go
|
||||
GO_TAGS?=
|
||||
|
||||
@@ -25,7 +25,7 @@ fi
|
||||
# Depth estimation needs real content; a synthetic image would be degenerate.
|
||||
TEST_IMAGE_DIR="$CURDIR/test-data"
|
||||
TEST_IMAGE_FILE="$TEST_IMAGE_DIR/test.jpg"
|
||||
TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/mudler/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}"
|
||||
TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/localai-org/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}"
|
||||
|
||||
mkdir -p "$TEST_IMAGE_DIR"
|
||||
if [ ! -f "$TEST_IMAGE_FILE" ]; then
|
||||
|
||||
@@ -27,7 +27,7 @@ fi
|
||||
# synthetic image would trivially yield zero detections.
|
||||
TEST_IMAGE_DIR="$CURDIR/test-data"
|
||||
TEST_IMAGE_FILE="$TEST_IMAGE_DIR/test.jpg"
|
||||
TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/mudler/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}"
|
||||
TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/localai-org/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}"
|
||||
|
||||
mkdir -p "$TEST_IMAGE_DIR"
|
||||
if [ ! -f "$TEST_IMAGE_FILE" ]; then
|
||||
|
||||
@@ -16,7 +16,7 @@
|
||||
# clone-at-pin + cmake build so CI doesn't need a side-checkout.
|
||||
|
||||
MOSS_VERSION?=190a569c13b4b247450f2fb3b2a431244e84833e
|
||||
MOSS_REPO?=https://github.com/mudler/moss-transcribe.cpp
|
||||
MOSS_REPO?=https://github.com/localai-org/moss-transcribe.cpp
|
||||
|
||||
GOCMD?=go
|
||||
GO_TAGS?=
|
||||
|
||||
@@ -7,10 +7,10 @@ GO_TAGS?=
|
||||
JOBS?=$(shell nproc --ignore=1)
|
||||
|
||||
# rt-detr.cpp (GitHub redirects the historical mudler/rt-detr.cpp to the new
|
||||
# mudler/rf-detr.cpp slug). Pin to a specific commit if you need a stable
|
||||
# localai-org/rf-detr.cpp slug). Pin to a specific commit if you need a stable
|
||||
# build; leaving this on `master` always picks up the latest C-API surface
|
||||
# (incl. the per-detection accessor functions used by gorfdetrcpp.go).
|
||||
RFDETR_REPO?=https://github.com/mudler/rf-detr.cpp.git
|
||||
RFDETR_REPO?=https://github.com/localai-org/rf-detr.cpp.git
|
||||
RFDETR_VERSION?=65c0ffcc9a9bc9dae38252f63d0417c9845a6cf7
|
||||
|
||||
ifeq ($(NATIVE),false)
|
||||
|
||||
@@ -34,7 +34,7 @@ fi
|
||||
# detections.
|
||||
TEST_IMAGE_DIR="$CURDIR/test-data"
|
||||
TEST_IMAGE_FILE="$TEST_IMAGE_DIR/test.jpg"
|
||||
TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/mudler/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}"
|
||||
TEST_IMAGE_URL="${TEST_IMAGE_URL:-https://raw.githubusercontent.com/localai-org/rf-detr.cpp/main/tests/fixtures/ci/test_image.jpg}"
|
||||
|
||||
mkdir -p "$TEST_IMAGE_DIR"
|
||||
if [ ! -f "$TEST_IMAGE_FILE" ]; then
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
# not need a side-checkout.
|
||||
|
||||
VOICEDETECT_VERSION?=1db1759572c90faef6f3a78c36b5941a096a9f89
|
||||
VOICEDETECT_REPO?=https://github.com/mudler/voice-detect.cpp
|
||||
VOICEDETECT_REPO?=https://github.com/localai-org/voice-detect.cpp
|
||||
|
||||
GOCMD?=go
|
||||
GO_TAGS?=
|
||||
|
||||
@@ -222,7 +222,7 @@
|
||||
It does joint long-form transcription, speaker diarization and timestamping in a single pass, emitting a speaker-labelled,
|
||||
time-aligned transcript, and runs on CPU, NVIDIA CUDA, AMD ROCm/HIP, Intel SYCL, Vulkan, Apple Metal and NVIDIA Jetson (L4T) targets.
|
||||
urls:
|
||||
- https://github.com/mudler/moss-transcribe.cpp
|
||||
- https://github.com/localai-org/moss-transcribe.cpp
|
||||
tags:
|
||||
- audio-transcription
|
||||
- CPU
|
||||
@@ -253,7 +253,7 @@
|
||||
/v1/audio/classification REST endpoint. It runs on CPU, NVIDIA CUDA,
|
||||
AMD ROCm/HIP, Intel SYCL, Vulkan and NVIDIA Jetson (L4T) targets.
|
||||
urls:
|
||||
- https://github.com/mudler/ced.cpp
|
||||
- https://github.com/localai-org/ced.cpp
|
||||
tags:
|
||||
- audio-classification
|
||||
- CPU
|
||||
@@ -287,7 +287,7 @@
|
||||
gRPC rpcs and the /v1/voice/* REST endpoints. It runs on CPU, NVIDIA
|
||||
CUDA, AMD ROCm/HIP, Intel SYCL, Vulkan and NVIDIA Jetson (L4T) targets.
|
||||
urls:
|
||||
- https://github.com/mudler/voice-detect.cpp
|
||||
- https://github.com/localai-org/voice-detect.cpp
|
||||
tags:
|
||||
- voice-recognition
|
||||
- speaker-verification
|
||||
@@ -487,7 +487,7 @@
|
||||
and returns bounding boxes, class labels, confidence scores, and
|
||||
(for segmentation variants) PNG-encoded per-detection masks.
|
||||
urls:
|
||||
- https://github.com/mudler/rf-detr.cpp
|
||||
- https://github.com/localai-org/rf-detr.cpp
|
||||
tags:
|
||||
- object-detection
|
||||
- image-segmentation
|
||||
|
||||
@@ -7,7 +7,7 @@ url = "/features/audio-classification/"
|
||||
|
||||
Sound-event classification (audio tagging) answers the question **"what am I hearing?"** - given an audio clip, it returns a list of scored [AudioSet](https://research.google.com/audioset/) labels (e.g. *Baby cry, infant cry*, *Glass breaking*, *Dog bark*, *Alarm*).
|
||||
|
||||
LocalAI exposes this through the `/v1/audio/classification` endpoint, modelled after `/v1/audio/transcriptions`. The reference backend is **[ced.cpp](https://github.com/mudler/ced.cpp)** (CED, a 527-class AudioSet tagger), a small ViT over a log-mel spectrogram ported to ggml with full PyTorch parity. Apache-2.0 weights are redistributable as GGUF.
|
||||
LocalAI exposes this through the `/v1/audio/classification` endpoint, modelled after `/v1/audio/transcriptions`. The reference backend is **[ced.cpp](https://github.com/localai-org/ced.cpp)** (CED, a 527-class AudioSet tagger), a small ViT over a log-mel spectrogram ported to ggml with full PyTorch parity. Apache-2.0 weights are redistributable as GGUF.
|
||||
|
||||
Because classification is exposed as a regular OpenAI-style endpoint, any HTTP client works - there is no Python dependency on the consumer side.
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ weight = 41
|
||||
url = "/features/object-detection/"
|
||||
+++
|
||||
|
||||
LocalAI supports object detection and image segmentation through various backends. This feature allows you to identify and locate objects within images with high accuracy and real-time performance. Available backends include [RF-DETR](https://github.com/roboflow/rf-detr) (Python) and [rf-detr.cpp](https://github.com/mudler/rf-detr.cpp) (native C++/ggml) for object detection and segmentation, and [sam3.cpp](https://github.com/PABannier/sam3.cpp) for image segmentation (SAM 3/2/EdgeTAM).
|
||||
LocalAI supports object detection and image segmentation through various backends. This feature allows you to identify and locate objects within images with high accuracy and real-time performance. Available backends include [RF-DETR](https://github.com/roboflow/rf-detr) (Python) and [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp) (native C++/ggml) for object detection and segmentation, and [sam3.cpp](https://github.com/PABannier/sam3.cpp) for image segmentation (SAM 3/2/EdgeTAM).
|
||||
|
||||
For detecting **faces** specifically, see the dedicated
|
||||
[Face Recognition](/features/face-recognition/) feature - its
|
||||
@@ -138,7 +138,7 @@ You can browse and install this model through the LocalAI web interface or using
|
||||
### RF-DETR Native Backend (rfdetr-cpp)
|
||||
|
||||
The `rfdetr-cpp` backend is a native C++/ggml implementation of RF-DETR
|
||||
inference based on [rf-detr.cpp](https://github.com/mudler/rf-detr.cpp). It
|
||||
inference based on [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp). It
|
||||
runs as a Go gRPC service that dlopens a per-CPU-variant shared library, so
|
||||
there is no Python runtime on the inference path - startup is fast and the
|
||||
binary is self-contained.
|
||||
|
||||
@@ -16,7 +16,7 @@ The audio analog to [Face Recognition](/features/face-recognition/),
|
||||
served over the same `/v1/voice/*` HTTP API by two backends:
|
||||
|
||||
- **`voice-detect` (recommended, default).** A standalone C++/ggml
|
||||
engine ([voice-detect.cpp](https://github.com/mudler/voice-detect.cpp)):
|
||||
engine ([voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp)):
|
||||
no Python, no onnxruntime, no torch runtime. Each gallery entry is a
|
||||
single self-describing GGUF. This is the recommended option for new
|
||||
deployments.
|
||||
|
||||
@@ -94,12 +94,12 @@ All backends listed here can be installed on demand from the [Backend Gallery]({
|
||||
| Backend | Description | Acceleration |
|
||||
|---------|-------------|-------------|
|
||||
| [RF-DETR](https://github.com/roboflow/rf-detr) | Real-time transformer-based object detection (Python) | CPU, CUDA 12/13, Intel SYCL, Metal, Jetson L4T |
|
||||
| [rf-detr.cpp](https://github.com/mudler/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation in C/C++ using GGML | CPU, CUDA 12/13, Intel SYCL, Vulkan, Jetson L4T |
|
||||
| [rf-detr.cpp](https://github.com/localai-org/rf-detr.cpp) | Native RF-DETR object detection and instance segmentation in C/C++ using GGML | CPU, CUDA 12/13, Intel SYCL, Vulkan, Jetson L4T |
|
||||
| [locate-anything.cpp](https://github.com/mudler/locate-anything.cpp) | Open-vocabulary object detection and visual grounding (LocateAnything-3B) in C/C++ using GGML | CPU, CUDA 12/13, Intel SYCL, Vulkan, Jetson L4T |
|
||||
| [depth-anything.cpp](https://github.com/mudler/depth-anything.cpp) | Depth Anything 3 monocular metric depth + camera pose in C/C++ using GGML | CPU, CUDA 12/13, Intel SYCL, Vulkan, Jetson L4T |
|
||||
| [sam3.cpp](https://github.com/PABannier/sam3.cpp) | Segment Anything (SAM 3/2/EdgeTAM) with text/point/box prompts in C/C++ using GGML | CPU, CUDA 12/13, Intel SYCL, Vulkan, Jetson L4T |
|
||||
| [face-detect.cpp](https://github.com/mudler/face-detect.cpp) | Native face detection, recognition, embedding, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace) in C/C++ using GGML | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T |
|
||||
| [voice-detect.cpp](https://github.com/mudler/voice-detect.cpp) | Native speaker (voice) recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2) in C/C++ using GGML | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T |
|
||||
| [voice-detect.cpp](https://github.com/localai-org/voice-detect.cpp) | Native speaker (voice) recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2) in C/C++ using GGML | CPU, CUDA 12/13, ROCm, Intel SYCL, Vulkan, Metal, Jetson L4T |
|
||||
| [insightface](https://github.com/deepinsight/insightface) | Face verification, embedding, and anti-spoofing liveness (ONNX Runtime) | CPU, CUDA 12 |
|
||||
| [speaker-recognition](https://speechbrain.github.io/) | Speaker (voice) recognition via SpeechBrain ECAPA-TDNN | CPU, CUDA 12, Metal |
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ You can see the release notes [here](https://github.com/mudler/LocalAI/releases)
|
||||
- **April 2026**: [Audio Transform](/features/audio-transform/) - generic audio-in / audio-out endpoint with optional reference signal. First implementation: [LocalVQE](https://github.com/localai-org/LocalVQE) C++ backend (joint AEC + noise suppression + dereverberation, DeepVQE-style). Both batch (`POST /audio/transformations`) and bidirectional WebSocket streaming (`/audio/transformations/stream`). Studio "Transform" tab with synchronized waveform players for input / reference / output.
|
||||
- **April 2026**: [Face recognition backend](/features/face-recognition/) - `insightface`-powered 1:1 verification, 1:N identification, face embedding, face detection, and demographic analysis. Ships both a non-commercial `buffalo_l` model and an Apache 2.0 OpenCV Zoo alternative.
|
||||
- **May 2026**: [Speaker diarization](/features/audio-diarization/) - new `/v1/audio/diarization` endpoint returning "who spoke when" segments. Backed by `sherpa-onnx` (pyannote-3.0 + speaker embeddings + clustering) for pure diarization, and `vibevoice-cpp` for diarization bundled with long-form ASR. Supports `json` / `verbose_json` / `rttm` response formats.
|
||||
- **June 2026**: [Sound classification](/features/audio-classification/) - new `/v1/audio/classification` endpoint for audio tagging / sound-event classification, returning scored [AudioSet](https://research.google.com/audioset/) labels (baby cry, glass breaking, alarms, ...). Backed by [ced.cpp](https://github.com/mudler/ced.cpp), a 527-class AudioSet tagger ported to ggml.
|
||||
- **June 2026**: [Sound classification](/features/audio-classification/) - new `/v1/audio/classification` endpoint for audio tagging / sound-event classification, returning scored [AudioSet](https://research.google.com/audioset/) labels (baby cry, glass breaking, alarms, ...). Backed by [ced.cpp](https://github.com/localai-org/ced.cpp), a 527-class AudioSet tagger ported to ggml.
|
||||
- **June 2026**: [PII analyze / redact API](/features/middleware/#analyze--redact-api) - the PII detection pipeline (NER + restricted-regex pattern tiers) is now a standalone service: `POST /api/pii/analyze` returns detected entity spans and `POST /api/pii/redact` returns the sanitised text (or `400 pii_blocked`), without routing a chat request through the middleware. Events gain an `origin` (`middleware` / `proxy` / `pii_analyze` / `pii_redact`) so `/api/pii/events` can be filtered by source.
|
||||
- **July 2026**: [Model capabilities endpoint](/features/api-discovery/#model-capabilities) - `GET /v1/models/capabilities`, an additive superset of `/v1/models` that reports each model's `capabilities` plus its `input_modalities` / `output_modalities` (`text` / `image` / `audio` / `video`). Lets clients route attachments using inferred or explicitly declared model modalities instead of backend-name checks.
|
||||
- **June 2026**: Concurrent scoring and PII NER on llama.cpp - the `Score` (router classifier) and `TokenClassify` (PII NER) primitives now ride llama.cpp's server task queue instead of locking the context, so they run concurrently with chat/completion/embedding traffic and with each other. The `known_usecases` restriction that forced dedicated scorer/NER model configs on llama-cpp is lifted, repeated scoring calls reuse the prompt KV cache across candidates, and scoring inputs are no longer capped by the physical batch size.
|
||||
|
||||
Reference in New Issue
Block a user