feat(tts): support multi-reference personalities

Saved profiles previously resolved to one audio path and transcript, so
cloning backends could not use several examples of one personality.

Store ordered audio and transcript pairs while preserving the legacy
first-reference fields. Fish Speech and audio.cpp receive all pairs,
including on distributed workers. Other backends retain their
single-reference behavior.

Assisted-by: Codex:gpt-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
Ettore Di Giacinto committed 2026-09-11 21:56:08 +00:00
1 parent ddda0436a3
commit db09452d54
19 files changed
+463 -118

No files matched your search

@@ -128,6 +128,14 @@ static void test_tts_shape() {
check(!shape.has_voice_reference,
"shape: a preset name is not a voice reference");
}
{
backend::TTSRequest request;
(*request.mutable_params())["multi_reference_cond"] =
R"([{"audio":"one.wav","text":"one"}])";
const auto shape = build_tts_shape(request);
check(shape.has_voice_reference,
"shape: multi-reference conditioning is a voice reference");
}
{
backend::TTSRequest request;
request.set_voice(dir.string());