* fix(realtime): accept legacy 'modalities' alias for output_modalities
OpenAI's Realtime *beta* used the field name `modalities`; the GA field is
`output_modalities`. LocalAI only binds `output_modalities`, so a client
sending the still-common beta field `modalities: ["text"]` has it silently
dropped by encoding/json and the session falls back to audio: TTS runs and the
client receives large response.output_audio.* frames even though it asked for
text-only.
Accept `modalities` as an alias on both session.update (RealtimeSession) and
response.create (ResponseCreateParams). The GA `output_modalities` wins when
both are present, so GA clients are unaffected. Applied at the two existing
resolution points via a small modalitiesWithAlias helper.
Fixes#11103
Signed-off-by: Anai-Guo <antai12232931@anaiguo.com>
* test(realtime): add JSON-boundary regression for modalities alias
Decode representative session.update and response.create payloads that
carry only the legacy beta `modalities` key and assert the effective
output modality resolves to text (not audio), reproducing the exact
expressions used in updateSession and triggerResponseAtTurn. This guards
against a wrong JSON tag or a missed call site letting encoding/json drop
the alias silently.
Also document output_modalities (and the accepted legacy modalities
alias) for text-only sessions in the realtime feature docs.
Signed-off-by: Tai An <antai12232931@outlook.com>
---------
Signed-off-by: Anai-Guo <antai12232931@anaiguo.com>
Signed-off-by: Tai An <antai12232931@outlook.com>
Co-authored-by: Anai-Guo <antai12232931@anaiguo.com>
53006bb8e1