docs(audio-cpp): document params[text] and the pinned transform tasks

The text channel and the two task pins are both invisible from the endpoint
contract alone: nothing in the AudioTransform form tells a reader that a
speech-to-speech model needs the line it is resynthesising, and nothing says
that asking for singing voice conversion without task:svc silently gets plain
voice conversion instead. Both are the kind of thing a user only discovers
from a refusal or, worse, from output that looks right and is not.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5 [Claude Code]
This commit is contained in:
Ettore Di Giacinto
2026-07-27 14:23:44 +00:00
committed by localai-org-maint-bot
parent ed14796a15
commit f72e409457
2 changed files with 53 additions and 0 deletions

View File

@@ -180,6 +180,36 @@ the body, is listed in the `X-Audio-Stems` response header. See
`params[stem]` against a model that produces a single unnamed output is refused rather than
ignored.
## Text-conditioned transforms
`s2s` is a text-and-prosody route, not an audio-only one: `vevo2` resynthesises a line in
another voice and reads the line itself, refusing the run without it. `AudioTransform` has
no text field, so the text arrives as `params[text]` (or `params[target_text]`, the name
vevo2's own option table uses) and the backend lifts it into the engine's text input.
`params[language]` rides along when the line is not English.
```bash
curl http://localhost:8080/audio/transformations \
-F model=audio-cpp-vevo2-speech-to-speech \
-F audio=@source.wav -F reference=@target-speaker.wav \
-F 'params[text]=The quick brown fox jumps over the lazy dog.' -o converted.wav
```
A model that takes no text ignores the keys, so separation and plain voice conversion are
unaffected.
## Pinned tasks
Two routes are unreachable by auto-routing and need `task:` in the model config, because
nothing in a request distinguishes them from a task the same family also advertises:
- `task:svc` for singing voice conversion. `seed_vc` and `vevo2` advertise both `svc` and
ordinary `vc`, and no request signal means "this input is singing", so `vc` always wins.
- `task:s2s` for speech to speech, for the same reason against the same `vc`.
The gallery entries that ship these routes carry the pin already. Removing it gives plain
voice conversion from the same weights.
## Family notes
- **Supertonic**: use the `orig` GGUF package, whose weights are f32. The f16 package was

View File

@@ -56,6 +56,7 @@ form-data, returns audio bytes.
| `sample_rate` | int | no | Desired output sample rate in Hz. Omit it for the backend's own rate; otherwise it must be between 8000 and 192000, and a value outside that range is refused with a 400 |
| `params[<key>]` | string | no | Repeated; forwarded to backend |
| `params[stem]` | string | no | Multi-output transforms only; picks which named output the body carries (see [stems](#multi-output-transforms-source-separation-stems)) |
| `params[text]` | string | no | Text-conditioned transforms only; the line being resynthesised (see [text-conditioned transforms](#text-conditioned-transforms-speech-to-speech)) |
First install an audio-transform model from the gallery (the examples below use `localvqe-v1.3-4.8m`):
@@ -143,6 +144,28 @@ The stems live in the generated-content directory beside the main output and are
served from `/generated-audio/`. Like every other generated artifact, they are
not swept automatically.
### Text-conditioned transforms (speech to speech)
Most transforms are audio in, audio out. A few are not: a speech-to-speech
model resynthesises a line in another voice and needs to know what the line
says. `/audio/transform` has no text field, so the text travels as
`params[text]`.
```bash
curl -sS -X POST http://localhost:8080/audio/transformations \
-F model=audio-cpp-vevo2-speech-to-speech \
-F audio=@source.wav \
-F reference=@target-speaker.wav \
-F 'params[text]=The quick brown fox jumps over the lazy dog.' \
-o converted.wav
```
For these models the text is required, not a hint: the run is refused without
it. `params[target_text]` is accepted as the same field, and
`params[language]` sets the language when the line is not English. A model that
does not take text ignores all three, so an ordinary separation or
noise-suppression request is unaffected.
## Streaming endpoint
`GET /audio/transformations/stream` - bidirectional WebSocket. The first