mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-08-01 19:09:42 -04:00
Code Issues Packages Projects Releases Wiki Activity

fix(runtime-settings): apply persisted threads/context_size/f16 at startup (#10853)

Browse Source 
ApplyRuntimeSettings persists the performance settings (threads,
context_size, f16) on the live /api/settings path, but the startup
loader loadRuntimeSettingsFromFile never read them back, so a value
saved via the Middleware UI was silently ignored on the next restart:
the model booted with the CLI/physical-core default and GET /api/settings
echoed that default instead of the saved value (#10845).

Threads needs special handling: unlike context_size/f16, WithThreads
eagerly resolves an unset (0) value to xsysinfo.CPUPhysicalCores() at
option-apply time, so options.Threads is never 0 in the loader and the
usual "== default" heuristic cannot tell an env/CLI value from the
physical-core fallback. Detect LOCALAI_THREADS/THREADS explicitly so the
env still wins over the persisted file value.

Signed-off-by: Anai-Guo <Anai-Guo@users.noreply.github.com>
Co-authored-by: Anai-Guo <Anai-Guo@users.noreply.github.com>
This commit is contained in:
Tai An
2026-07-15 23:52:11 -07:00
committed by GitHub
parent 8c9b3b2e33
commit 688f904a10
1 changed files with 30 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
core/application/startup.go
View File

@@ -675,6 +675,36 @@ func loadRuntimeSettingsFromFile(options *config.ApplicationConfig) {
options.AgentJobRetentionDays = *settings.AgentJobRetentionDays
}
}
// Performance settings (threads / context size / f16). ApplyRuntimeSettings
// already persists these on the live /api/settings path, but the startup
// loader dropped them, so a value saved via the Middleware UI was silently
// ignored on restart (the model booted with the CLI/physical-core default).
//
// Threads is special: unlike ContextSize/F16, WithThreads eagerly resolves
// an unset (0) value to xsysinfo.CPUPhysicalCores() at option-apply time
// (see application_config.go), so options.Threads is never 0 here and the
// usual "== default" heuristic can't distinguish an env/CLI value from the
// physical-core fallback. Detect the env/CLI explicitly so
// LOCALAI_THREADS/THREADS still win over the persisted file value.
if settings.Threads != nil {
if os.Getenv("LOCALAI_THREADS") == "" && os.Getenv("THREADS") == "" {
options.Threads = *settings.Threads
}
}
if settings.ContextSize != nil {
// Only apply if current value is default (0), suggesting it wasn't set from env var
if options.ContextSize == 0 {
options.ContextSize = *settings.ContextSize
}
}
if settings.F16 != nil {
// Only apply if current value is default (false), suggesting it wasn't set from env var
if !options.F16 {
options.F16 = *settings.F16
}
}
if !options.WatchDogIdle && !options.WatchDogBusy {
if settings.WatchdogEnabled != nil && *settings.WatchdogEnabled {
options.WatchDog = true