mirror of
https://github.com/mudler/LocalAI.git
synced 2026-08-03 03:53:15 -04:00
backend(audio-cpp): correct the status, lifetime and state contracts of LoadedModel
An environment fault during session creation was reported as UNIMPLEMENTED. A missing libggml-cpu-*.so surfaced to the client as 'family silero_vad advertises vad/offline but refused to create the session: Failed to initialize CPU backend', which tells LocalAI the model cannot do this and must never be retried, and sends an operator hunting a capability bug instead of a packaging one. A throw from create_task_session is now a plain runtime_error, so it maps to INTERNAL. Only a null return, where the family genuinely declined, stays a CapabilityError. The model.'s task: option was parsed and then dropped: it lived in a local that died at the end of LoadModel and had no route to RequestShape::pinned_task. LoadedModel now keeps it and exposes pinned_task(). The global model becomes a shared_ptr reached through snapshot(). An audio RPC runs for seconds and cannot hold g_model_mu for its duration, so under a unique_ptr a Free arriving mid-request would destroy the model underneath it. Handlers now take a counted reference and whichever finishes last does the teardown, outside the lock. session_for documents the streaming state contract rather than resetting the session itself. Resetting on a cache hit was tried first and is not possible: silero_vad throws 'session prepare() must be called before Silero VAD reset()', so it would turn an ordinary second fetch into a hard error. start_stream's base implementation is already a reset, so a caller that runs prepare then start_stream per stream gets a clean session; a probe against the bundled silero_vad confirms an identical replay when it does and a carried-over stream when it does not. Also: an unknown backend: name is rejected before the model loads rather than after; MainGPU is parsed instead of passed through std::atoi, which turned 'gpu1' into device 0 silently; and device carries a device_set flag, because 0 is both the default and a real device index, so MainGPU was overriding an explicit device:0 that the neighbouring threads: handling promises will win. Assisted-by: Claude:claude-opus-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
committed by
localai-org-maint-bot
parent
b4153fc6eb
commit
e2cad2bf3d
@@ -20,6 +20,11 @@ struct ModelOptions {
|
||||
// ggml backend: cpu, cuda, vulkan, metal, best.
|
||||
std::string backend = "cpu";
|
||||
int device = 0;
|
||||
// True once a `device:` entry has been seen. 0 is both the default and a
|
||||
// legitimate device index, so the value alone cannot tell an explicit
|
||||
// `device:0` from an unset option, and a caller merging in its own fallback
|
||||
// would silently override the explicit choice.
|
||||
bool device_set = false;
|
||||
// 0 means "let the runtime decide".
|
||||
int threads = 0;
|
||||
std::string model_spec_override;
|
||||
|
||||
Reference in New Issue
Block a user