mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-08 15:28:23 -04:00
eb32cd9073
Realtime sessions previously lazy-loaded each pipeline sub-model (VAD,
transcription, LLM, TTS) on first use, so every cold session paid a
per-request model-load stall and load errors only surfaced mid-stream.
Warm the whole pipeline eagerly and blockingly at session start
(including the voice-gate speaker-recognition model, which an enforced
gate blocks each utterance on; compaction's summary_model stays lazy
since it only runs off the response path):
- Add backend.PreloadModel / PreloadModelByName as the single load path
for every modality (no transcription special-case; backend-omitted
configs are deprecated).
- The realtime session blocks on Model.Warmup and returns a
model_load_error to the client if any stage fails to load;
updateSession warms in the background. Opt out per pipeline with
pipeline.disable_warmup, exposed as a UI toggle via the
config-metadata registry.
Add a LocalAI-native POST /backend/load (and /v1/backend/load) that
pre-loads a model -- expanding realtime pipelines into their sub-models
-- as the inverse of /backend/shutdown. There is one preload engine
(backend.PreloadStages): the realtime Warmup methods, /backend/load and
the --load-to-memory startup flag all use it, so --load-to-memory now
also expands pipeline models and records load-failure traces. Pipeline
sub-model alias resolution is likewise shared
(ModelConfigLoader.LoadResolvedModelConfig). Surface the endpoint
everywhere an admin manages models:
- MCP admin tool load_model (httpapi + inproc clients, safety/catalog
prompts, catalog/dispatch tests).
- "Load into memory" action in the React models UI.
- Swagger regenerated; docs moved to the general backend-monitor page
since it is not realtime-specific.
Fix a Traces UI crash ("json: unsupported value: -Inf"): audio-snippet
RMS/peak now floor at a finite dBFS, and backend-trace data is sanitized
to drop non-finite floats before marshaling. The sanitizer is
copy-on-write -- it runs on every RecordBackendTrace, so containers are
only re-allocated on the paths that actually changed.
Migrate core/http/openresponses_test.go onto the prebuilt mock-backend
the rest of the http suite already uses -- it was the last spec still
pointing at a real HuggingFace model, so it 404'd wherever no vision
backend was built -- and fix its item_reference specs to send the
spec's "id" field instead of "item_id", which the handler never
accepted.
Assisted-by: Claude:claude-opus-4-8 Claude Code
Signed-off-by: Richard Palethorpe <io@richiejp.com>
60 lines
1.9 KiB
Go
60 lines
1.9 KiB
Go
package httpapi
|
|
|
|
import (
|
|
"fmt"
|
|
"net/url"
|
|
)
|
|
|
|
// Route paths for the LocalAI admin REST surface that this client targets.
|
|
// Static paths are constants; dynamic paths are builders that handle
|
|
// url.PathEscape on segment values. Keep these aligned with the server-side
|
|
// registrations in core/http/routes/localai.go — the Tool↔REST drift detector
|
|
// in coverage_test.go documents the mapping.
|
|
const (
|
|
routeWelcome = "/"
|
|
routeModelsApply = "/models/apply"
|
|
routeModelsAvail = "/models/available"
|
|
routeModelsGall = "/models/galleries"
|
|
routeModelsImport = "/models/import-uri"
|
|
routeModelImport = "/models/import"
|
|
routeAliases = "/api/aliases"
|
|
routeModelsReload = "/models/reload"
|
|
routeBackendLoad = "/backend/load"
|
|
routeBackends = "/backends"
|
|
routeBackendsKnown = "/backends/known"
|
|
routeBackendsApply = "/backends/apply"
|
|
routeNodes = "/api/nodes"
|
|
routeVRAMEstimate = "/api/models/vram-estimate"
|
|
routeBranding = "/api/branding"
|
|
routeSettings = "/api/settings"
|
|
routeUsage = "/api/usage"
|
|
routeUsageAll = "/api/usage/all"
|
|
routePIIEvents = "/api/pii/events"
|
|
routeMiddleware = "/api/middleware/status"
|
|
routeRouterDecisions = "/api/router/decisions"
|
|
)
|
|
|
|
func routeJobStatus(jobID string) string {
|
|
return "/models/jobs/" + url.PathEscape(jobID)
|
|
}
|
|
|
|
func routeModelDelete(name string) string {
|
|
return "/models/delete/" + url.PathEscape(name)
|
|
}
|
|
|
|
func routeModelConfigJSON(name string) string {
|
|
return "/api/models/config-json/" + url.PathEscape(name)
|
|
}
|
|
|
|
func routeBackendUpgrade(name string) string {
|
|
return "/backends/upgrade/" + url.PathEscape(name)
|
|
}
|
|
|
|
func routeToggleModelState(name, action string) string {
|
|
return fmt.Sprintf("/models/toggle-state/%s/%s", url.PathEscape(name), url.PathEscape(action))
|
|
}
|
|
|
|
func routeToggleModelPinned(name, action string) string {
|
|
return fmt.Sprintf("/models/toggle-pinned/%s/%s", url.PathEscape(name), url.PathEscape(action))
|
|
}
|