From f72e4094577dc8f6a19214ab229af5edd3bcd8d1 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Mon, 27 Jul 2026 14:23:44 +0000 Subject: [PATCH] docs(audio-cpp): document params[text] and the pinned transform tasks The text channel and the two task pins are both invisible from the endpoint contract alone: nothing in the AudioTransform form tells a reader that a speech-to-speech model needs the line it is resynthesising, and nothing says that asking for singing voice conversion without task:svc silently gets plain voice conversion instead. Both are the kind of thing a user only discovers from a refusal or, worse, from output that looks right and is not. Signed-off-by: Ettore Di Giacinto Assisted-by: Claude:claude-opus-5 [Claude Code] --- docs/content/features/audio-cpp.md | 30 ++++++++++++++++++++++++ docs/content/features/audio-transform.md | 23 ++++++++++++++++++ 2 files changed, 53 insertions(+) diff --git a/docs/content/features/audio-cpp.md b/docs/content/features/audio-cpp.md index edeb76aee..ae71cd384 100644 --- a/docs/content/features/audio-cpp.md +++ b/docs/content/features/audio-cpp.md @@ -180,6 +180,36 @@ the body, is listed in the `X-Audio-Stems` response header. See `params[stem]` against a model that produces a single unnamed output is refused rather than ignored. +## Text-conditioned transforms + +`s2s` is a text-and-prosody route, not an audio-only one: `vevo2` resynthesises a line in +another voice and reads the line itself, refusing the run without it. `AudioTransform` has +no text field, so the text arrives as `params[text]` (or `params[target_text]`, the name +vevo2's own option table uses) and the backend lifts it into the engine's text input. +`params[language]` rides along when the line is not English. + +```bash +curl http://localhost:8080/audio/transformations \ + -F model=audio-cpp-vevo2-speech-to-speech \ + -F audio=@source.wav -F reference=@target-speaker.wav \ + -F 'params[text]=The quick brown fox jumps over the lazy dog.' -o converted.wav +``` + +A model that takes no text ignores the keys, so separation and plain voice conversion are +unaffected. + +## Pinned tasks + +Two routes are unreachable by auto-routing and need `task:` in the model config, because +nothing in a request distinguishes them from a task the same family also advertises: + +- `task:svc` for singing voice conversion. `seed_vc` and `vevo2` advertise both `svc` and + ordinary `vc`, and no request signal means "this input is singing", so `vc` always wins. +- `task:s2s` for speech to speech, for the same reason against the same `vc`. + +The gallery entries that ship these routes carry the pin already. Removing it gives plain +voice conversion from the same weights. + ## Family notes - **Supertonic**: use the `orig` GGUF package, whose weights are f32. The f16 package was diff --git a/docs/content/features/audio-transform.md b/docs/content/features/audio-transform.md index 6b32a2fe6..8b37f42cd 100644 --- a/docs/content/features/audio-transform.md +++ b/docs/content/features/audio-transform.md @@ -56,6 +56,7 @@ form-data, returns audio bytes. | `sample_rate` | int | no | Desired output sample rate in Hz. Omit it for the backend's own rate; otherwise it must be between 8000 and 192000, and a value outside that range is refused with a 400 | | `params[]` | string | no | Repeated; forwarded to backend | | `params[stem]` | string | no | Multi-output transforms only; picks which named output the body carries (see [stems](#multi-output-transforms-source-separation-stems)) | +| `params[text]` | string | no | Text-conditioned transforms only; the line being resynthesised (see [text-conditioned transforms](#text-conditioned-transforms-speech-to-speech)) | First install an audio-transform model from the gallery (the examples below use `localvqe-v1.3-4.8m`): @@ -143,6 +144,28 @@ The stems live in the generated-content directory beside the main output and are served from `/generated-audio/`. Like every other generated artifact, they are not swept automatically. +### Text-conditioned transforms (speech to speech) + +Most transforms are audio in, audio out. A few are not: a speech-to-speech +model resynthesises a line in another voice and needs to know what the line +says. `/audio/transform` has no text field, so the text travels as +`params[text]`. + +```bash +curl -sS -X POST http://localhost:8080/audio/transformations \ + -F model=audio-cpp-vevo2-speech-to-speech \ + -F audio=@source.wav \ + -F reference=@target-speaker.wav \ + -F 'params[text]=The quick brown fox jumps over the lazy dog.' \ + -o converted.wav +``` + +For these models the text is required, not a hint: the run is refused without +it. `params[target_text]` is accepted as the same field, and +`params[language]` sets the language when the line is not English. A model that +does not take text ignores all three, so an ordinary separation or +noise-suppression request is unaffected. + ## Streaming endpoint `GET /audio/transformations/stream` - bidirectional WebSocket. The first