mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-31 18:38:23 -04:00
e2cad2bf3d
An environment fault during session creation was reported as UNIMPLEMENTED. A missing libggml-cpu-*.so surfaced to the client as 'family silero_vad advertises vad/offline but refused to create the session: Failed to initialize CPU backend', which tells LocalAI the model cannot do this and must never be retried, and sends an operator hunting a capability bug instead of a packaging one. A throw from create_task_session is now a plain runtime_error, so it maps to INTERNAL. Only a null return, where the family genuinely declined, stays a CapabilityError. The model.'s task: option was parsed and then dropped: it lived in a local that died at the end of LoadModel and had no route to RequestShape::pinned_task. LoadedModel now keeps it and exposes pinned_task(). The global model becomes a shared_ptr reached through snapshot(). An audio RPC runs for seconds and cannot hold g_model_mu for its duration, so under a unique_ptr a Free arriving mid-request would destroy the model underneath it. Handlers now take a counted reference and whichever finishes last does the teardown, outside the lock. session_for documents the streaming state contract rather than resetting the session itself. Resetting on a cache hit was tried first and is not possible: silero_vad throws 'session prepare() must be called before Silero VAD reset()', so it would turn an ordinary second fetch into a hard error. start_stream's base implementation is already a reset, so a caller that runs prepare then start_stream per stream gets a clean session; a probe against the bundled silero_vad confirms an identical replay when it does and a carried-over stream when it does not. Also: an unknown backend: name is rejected before the model loads rather than after; MainGPU is parsed instead of passed through std::atoi, which turned 'gpu1' into device 0 silently; and device carries a device_set flag, because 0 is both the default and a real device index, so MainGPU was overriding an explicit device:0 that the neighbouring threads: handling promises will win. Assisted-by: Claude:claude-opus-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
67 lines
2.3 KiB
C++
67 lines
2.3 KiB
C++
#include "family_gate.h"
|
|
|
|
#include <cctype>
|
|
|
|
namespace audiocpp_backend {
|
|
namespace {
|
|
|
|
// PRECONDITION: `suffix` must already be lowercase. Both sides are folded, so
|
|
// this reads as symmetric, but only `value` can carry case in practice and a
|
|
// caller passing ".GGUF" would still work today for that reason alone. Do not
|
|
// rely on it: the fold on the suffix side is the only thing standing between
|
|
// this and a helper that answers false for every input, and it is not covered
|
|
// by any test, because with a lowercase suffix no input can distinguish it.
|
|
bool ends_with_ci(const std::string &value, const std::string &suffix) {
|
|
if (value.size() <= suffix.size()) {
|
|
return false; // a bare ".gguf" is an extension, not a model file
|
|
}
|
|
const size_t offset = value.size() - suffix.size();
|
|
for (size_t i = 0; i < suffix.size(); ++i) {
|
|
const auto lhs = static_cast<unsigned char>(value[offset + i]);
|
|
const auto rhs = static_cast<unsigned char>(suffix[i]);
|
|
if (std::tolower(lhs) != std::tolower(rhs)) {
|
|
return false;
|
|
}
|
|
}
|
|
return true;
|
|
}
|
|
|
|
} // namespace
|
|
|
|
bool path_looks_like_gguf(const std::string &path) {
|
|
return ends_with_ci(path, ".gguf");
|
|
}
|
|
|
|
FamilyDecision decide_family(bool path_is_gguf, const std::string &embedded_family,
|
|
const std::string &configured_family) {
|
|
FamilyDecision decision;
|
|
|
|
if (!configured_family.empty()) {
|
|
decision.ok = true;
|
|
decision.family = configured_family;
|
|
return decision;
|
|
}
|
|
|
|
if (path_is_gguf) {
|
|
if (!embedded_family.empty()) {
|
|
decision.ok = true;
|
|
decision.family = embedded_family;
|
|
return decision;
|
|
}
|
|
decision.error =
|
|
"audio-cpp: this GGUF carries no 'audiocpp.model_spec.family' "
|
|
"metadata key, so it is not an audio.cpp model. Convert it with "
|
|
"audiocpp_gguf, or name the family explicitly with the model option "
|
|
"'family:<name>'";
|
|
return decision;
|
|
}
|
|
|
|
decision.error =
|
|
"audio-cpp: a model path that is not a standalone audio.cpp GGUF needs "
|
|
"an explicit 'family:<name>' model option, because the audio.cpp family "
|
|
"cannot be inferred from a safetensors or package directory";
|
|
return decision;
|
|
}
|
|
|
|
} // namespace audiocpp_backend
|