mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-31 10:28:43 -04:00
40d35c0385
* docs: fix CPU image tag (latest, not latest-cpu) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: use canonical localai/localai registry in models guide Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: replace dead llama-stable backend with llama-cpp Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: correct mitm-proxy intercept config and redaction tier Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: fix text-to-audio endpoint and broken notice block Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: fix VAD example, stale FAQ, broken link, CLI list, whats-new dump Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: render advanced/reference section indexes (consolidate _index) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: remove duplicate getting-started build/kubernetes pages Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: fold container image reference into installation/containers Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: remove stale advanced fine-tuning page (superseded by features/fine-tuning) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: fold distribution/longcat/sound pages into their parents Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: make getting-started index accurate and complete Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: carry one concrete model through the getting-started path Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add end-to-end 'build your first agent' walkthrough Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add runtime errors reference; consolidate troubleshooting from FAQ Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add agent actions catalog Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: agent-scoped MCP, skills walkthrough, agentic disambiguation Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add concrete gallery install lines to media feature pages Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: merge installation into getting-started (URLs preserved via aliases) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add Operations section; move operator pages and P2P API reference Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: journey-ordered top nav and grouped feature sections Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: add docs-with-code process gate (PR template + agent instructions) Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * docs: remove em/en dashes from documentation prose Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
62 lines
2.1 KiB
Markdown
62 lines
2.1 KiB
Markdown
+++
|
|
disableToc = false
|
|
title = "Sound Classification"
|
|
weight = 32
|
|
url = "/features/audio-classification/"
|
|
+++
|
|
|
|
Sound-event classification (audio tagging) answers the question **"what am I hearing?"** - given an audio clip, it returns a list of scored [AudioSet](https://research.google.com/audioset/) labels (e.g. *Baby cry, infant cry*, *Glass breaking*, *Dog bark*, *Alarm*).
|
|
|
|
LocalAI exposes this through the `/v1/audio/classification` endpoint, modelled after `/v1/audio/transcriptions`. The reference backend is **[ced.cpp](https://github.com/mudler/ced.cpp)** (CED, a 527-class AudioSet tagger), a small ViT over a log-mel spectrogram ported to ggml with full PyTorch parity. Apache-2.0 weights are redistributable as GGUF.
|
|
|
|
Because classification is exposed as a regular OpenAI-style endpoint, any HTTP client works - there is no Python dependency on the consumer side.
|
|
|
|
## Endpoint
|
|
|
|
```
|
|
POST /v1/audio/classification
|
|
Content-Type: multipart/form-data
|
|
```
|
|
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `file` | file (required) | audio file in any format `ffmpeg` accepts |
|
|
| `model` | string (required) | name of the sound-classification-capable model (e.g. `ced-base-f16`) |
|
|
| `top_k` | int | number of top tags to return (0 = backend default) |
|
|
| `threshold` | float | drop tags scoring below this value |
|
|
|
|
### Response
|
|
|
|
```json
|
|
{
|
|
"model": "ced-base-f16",
|
|
"detections": [
|
|
{"index": 23, "label": "Baby cry, infant cry", "score": 0.87},
|
|
{"index": 22, "label": "Crying, sobbing", "score": 0.41}
|
|
]
|
|
}
|
|
```
|
|
|
|
Detections are returned in score-descending order. Scores are per-class probabilities (multi-label, independent), so they do not sum to 1.
|
|
|
|
## Example
|
|
|
|
First install a classification model from the gallery (the example below uses `ced-base-f16`):
|
|
|
|
```bash
|
|
local-ai run ced-base-f16
|
|
```
|
|
|
|
```bash
|
|
curl http://localhost:8080/v1/audio/classification \
|
|
-H "Content-Type: multipart/form-data" \
|
|
-F file="@/path/to/clip.wav" \
|
|
-F model="ced-base-f16" \
|
|
-F top_k=10
|
|
```
|
|
|
|
## See also
|
|
|
|
- [Audio to Text]({{% relref "audio-to-text" %}}) - speech transcription
|
|
- [Speaker Diarization]({{% relref "audio-diarization" %}}) - who spoke when
|