mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-05 12:34:43 -04:00
localai-org-maint-botandEttore Di Giacinto
2ae6cae70d
* feat(voice): list registered voices and record which encoder made them The voice registry could register, identify and forget but not list, and it did not remember which speaker encoder produced an embedding. Add Metadata.Model and Registry.List, answered from the index the store registry already keeps for Forget. Needed so a backend can be given the registered voices that match its own speaker encoder. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(voice): store the encoder model with a registered voice Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(voice): pick the registered voices that match a speaker model Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(proto): carry known voices and speaker names on diarize and live messages Assisted-by: Claude:claude-haiku-4-5 [Claude Code] * feat(diarization): name speakers from the voice registry When a diarization model has a speaker_model option, the endpoint sends the registered voices made by that encoder to the backend. The backend's name and name_score come back as extra fields next to the normalized SPEAKER_NN speaker, and the speakers summary carries the first name seen for each speaker. RTTM output and results without names are unchanged. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(live): pass registered voices to a live session and surface speaker names Live sessions now send the registered voices that match the model's speaker_model to the backend, and each speaker segment carries the name the backend matched. The realtime segment event gains an optional speaker_name field. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(parakeet-cpp): load a speaker model and build per-request voice registries Adds the speaker bindings (ABI v9 and v10, probed separately), the speaker_model, speaker_threshold and speaker_margin options, and a per-request registry builder over the known voices. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(parakeet-cpp): name the speakers in Diarize from the known voices Diarize builds a per-request speaker registry from the known voices when a speaker model is loaded, calls the named C functions, and puts each slot's registered name and score on the segments. The registry is freed on every path. A library without ABI 10 reports Unimplemented instead of dropping the names. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(parakeet-cpp): name speakers in the live scene stream The live scene stream now begins with a known-voice registry when a speaker model is loaded and the live config carries voices, and each closed speaker segment takes its slot's current name from the feed's names map. A segment that closes before its slot is identified has an empty name. The registry is freed after the stream, on every path. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * feat(gallery): speaker naming entries and docs for parakeet-cpp Add three gallery entries that load the WeSpeaker ResNet34 speaker model next to the diarization or realtime scene models, and document speaker names in the voice recognition, diarization, audio to text and realtime pages. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * fix(parakeet-cpp): skip an unusable registered voice instead of failing the request A registered voice with the wrong embedding size, or one the C side refused, failed the whole diarization request, so one legacy voice broke the model for every user. Skip such voices with a warning that does not carry the voice name, and take the plain path when none is left. Also map an exact 0 speaker threshold or margin to a tiny positive value, since the C side reads 0 as "use the default", and fix a stale comment about which contexts Free() walks. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * fix(diarization): warn once per model about voices from another encoder; document the privacy limit The different-encoder warning fired on every request. Log it once per feature and speaker model, then at debug level. Document that the global voice registry lets any caller of a speaker_model model learn matching names, and that skipped wrong-sized voices are logged. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] * chore(parakeet-cpp): bump parakeet.cpp to 8c8cec0 (C-API v10) and check speaker naming against the real library The pin moves from 623a968 to 8c8cec0, which brings in everything merged in parakeet.cpp since: the voice identification change (C-API v9, #78) and raw-embedding enroll plus diarize-only speaker naming (C-API v10, #79). New real-library specs (gated on PARAKEET_BACKEND_TEST_SPEAKER_MODEL, _DIAR_MODEL, _WAV and, for the live path, _STREAM_MODEL) name the two speakers of two_speakers.wav from a committed pair of WeSpeaker embeddings, with the voices passed in reversed order. They also check that the float32 threshold reaches C through purego. The shared test loader now registers the v9/v10 and scene symbols as main.go does. The rebase onto origin/master had no conflicts. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
1 line
4.9 KiB
JSON
1 line
4.9 KiB
JSON
{"model":"voice-detect-wespeaker-resnet34.gguf","dim":256,"voices":[{"name":"voice_a","clip":"two_speakers.wav 0.6-4.6 s","embedding":[0.011315,0.055604,-0.071566,0.00961,0.040386,-0.016421,-0.033611,-0.019875,0.012314,-0.048251,0.025119,-0.035261,-0.051,-0.065269,0.044619,-0.032789,-0.077872,-0.059979,0.077165,0.034743,-0.034322,0.051458,0.119578,-0.060482,-0.032685,-0.066351,-0.043352,0.014827,-0.008393,0.079046,0.075357,0.177104,-0.002038,0.020706,-0.018464,0.004545,0.026542,0.00754,0.126408,0.011555,-0.083333,0.046795,0.029405,0.039988,-0.051861,0.034671,0.063438,0.010646,0.089453,0.009784,0.161807,-0.060077,0.080398,0.0026,-0.179952,-0.092161,0.008115,-0.043436,0.032408,-0.009307,0.04841,0.029929,-0.066182,0.069376,0.017271,0.076435,-0.013476,-0.091337,0.027061,0.070812,0.039052,-0.025303,0.062032,-0.129511,-0.035932,0.123741,0.059323,0.084191,0.017091,-0.071467,0.00336,0.109306,-0.032808,0.071844,0.013484,0.01657,-0.04768,-0.087006,0.07233,-0.004725,-0.058554,-0.061997,0.088107,0.069736,0.008408,-0.104471,0.049595,0.070141,0.071505,0.03694,0.030285,-0.103312,-0.028439,0.053189,0.132217,0.061057,-0.115166,0.04954,-0.04504,-0.044936,0.041552,0.07976,0.005178,0.027334,0.036235,-0.096866,-0.087907,-0.095822,0.10443,0.041865,-0.034412,0.065312,0.008648,-0.075287,0.125829,0.046572,-0.006843,0.106429,0.024523,0.013693,0.011473,-0.053549,0.078732,0.094865,-0.06333,0.03013,-0.090245,-0.013319,-0.090633,0.04209,-0.012844,-0.055208,-0.096489,-0.093557,0.038501,0.033988,0.032093,-0.015028,-0.067271,-0.100653,-0.00632,0.055982,-0.026291,-0.088451,0.019825,0.18585,-0.013341,-0.016392,0.05616,0.022352,0.006584,-0.097643,-0.02696,-0.019335,0.089462,-0.082992,0.02947,-0.014762,-0.055757,-0.011026,0.046157,-0.01546,0.029421,0.005409,-0.080892,-0.009961,0.029963,0.013844,-0.021823,-0.036825,0.001308,0.044549,0.011387,0.058622,0.047924,-0.034883,-0.038361,0.092906,0.006849,-0.076708,0.080052,0.011746,0.168146,-0.017433,-0.017833,0.010594,0.05945,-0.034766,-0.008519,0.02914,0.062197,-0.032722,-0.029624,0.103967,-0.07483,-0.052394,0.022729,-0.036227,0.060867,0.144592,-0.054725,-0.009984,0.056824,-0.076816,-0.005553,-0.054548,-0.047348,0.011424,0.091265,-0.012273,0.013987,-0.019944,0.054455,0.050702,0.004049,0.013685,0.022356,0.003249,-0.081608,-0.06926,-0.077244,-0.048076,0.059592,0.009222,0.007737,0.014863,-0.070568,-0.070963,0.056541,-0.067229,-0.034265,0.072919,0.032005,-0.06045,-0.148719,0.050373,-0.06677,0.020233,-0.031594,-0.084966,0.033239,-0.020293,-0.049292,0.120206,-0.056133,0.090007]},{"name":"voice_b","clip":"two_speakers.wav 6.9-10.9 s","embedding":[-0.106236,0.026909,0.032522,-0.040702,0.06418,0.058867,-0.043462,0.029255,-0.069755,0.02625,-0.012982,0.011717,0.037221,0.004934,-0.148783,0.073643,0.065623,0.176729,0.000483,0.109778,-0.105427,0.02092,0.038877,-0.020431,0.06949,-0.027571,-0.023125,-0.036426,0.050819,-0.047508,-0.000816,-0.073777,0.060473,0.023495,-0.035937,-0.064082,-0.085108,-0.045895,0.029146,0.007555,0.003923,-0.081975,0.038573,0.072211,-0.063173,0.006889,-0.011189,-0.018597,-0.027534,-0.001236,0.056054,-0.055961,-0.016422,0.064765,-0.114341,-0.028348,-0.016785,-0.001291,0.00695,-0.073718,0.146327,-0.030716,-0.029492,0.066045,-0.049192,0.055391,0.105924,0.033385,0.049666,0.055198,-0.0327,-0.051139,-0.009429,0.113493,0.065922,-0.027484,-0.062549,-0.027209,0.006105,-0.016623,0.012859,-0.042844,0.046082,-0.029788,-0.132396,-0.005784,-0.020116,-0.042434,0.045863,-0.00114,0.022023,-0.117683,0.002397,0.187938,0.067782,-0.053011,0.018449,0.043227,-0.007033,0.034628,-0.036276,0.044694,-0.105351,-0.113562,0.051461,-0.040255,0.088879,0.101067,0.110482,-0.011473,0.030959,0.028627,0.042808,0.106958,-0.020069,-0.048371,0.002144,-0.063363,0.065032,-0.032523,0.025647,0.12314,0.026488,0.054077,0.026475,0.088169,-0.112363,-0.03501,-0.061824,-0.085917,0.027558,0.010396,-0.092906,-0.024477,0.027928,-0.012543,0.069895,0.01406,0.196947,0.006955,-0.012518,-0.048722,0.035787,-0.051044,0.007964,-0.009535,-0.045574,0.013616,-0.023598,-0.027236,0.023735,-0.023832,-0.053049,0.023769,0.080018,-0.039285,0.009429,-0.046602,0.023376,-0.011104,-0.118228,-0.089595,0.047087,-0.041417,-0.030876,-0.015441,-0.004501,-0.038595,0.070107,-0.078626,-0.128752,0.012564,-0.034697,-0.069278,0.058026,0.059071,0.082451,0.041674,0.031116,-0.007566,-0.007439,-0.079061,-0.059093,0.014351,-0.006355,-0.010857,-0.050468,0.059146,-0.047707,-0.08106,0.07763,0.065864,0.061567,-0.126551,-0.119901,-0.002173,0.076158,0.066434,0.053258,-0.065215,0.012435,0.015349,-0.00617,-0.033529,-0.024387,0.001493,0.113286,0.039088,0.051456,0.166047,-0.020073,-0.07846,-0.029372,0.022419,-0.06691,0.097686,-0.060915,-0.032454,-0.095959,-0.026277,0.026242,-0.123124,0.092298,-0.036004,0.031126,0.119571,0.09179,-0.060572,0.028013,0.016614,0.09969,-0.113456,-0.094285,-0.058706,-0.071713,0.131803,-0.066991,0.048721,-0.050628,-0.030655,0.010101,-0.046536,-0.024472,-2.4e-05,-0.04725,0.046721,0.036714,-0.082382,0.010479,0.053549,0.068698,0.006237,0.020804,-0.081368,-0.024734,0.051612]}]} |