mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 09:08:54 -04:00
mudler-agentandEttore Di Giacinto
0e0e91f95c
* feat(diagnostics): add private opt-in profiler Keep profiling on a dedicated loopback listener rather than the public API. Validate settings and bind synchronously so startup reports configuration and socket failures. Own sampling and shutdown for the run process. Cover private handlers, blocked cmdline access, sampling cleanup, and serving failures with focused Ginkgo tests. Assisted-by: nib:gpt-6-astra * feat(diagnostics): add sanitized phase timing Carry generated request and reload identities through private contexts. Keep timing disabled without allocations and emit only validated fields at Info level when enabled. Cover identity isolation, nested durations, inheritance, and sanitized logging with focused race-tested Ginkgo specs. Assisted-by: nib:gpt-6-astra * fix(diagnostics): retain request observations Reuse an existing request observation across recorders to avoid splitting correlation. Keep the original context, identity, and sink without new allocations. Preserve nil-recorder behavior and fresh reload identities. Verify the regression fails before the fix and passes with the full Diagnostics race suite afterward. Assisted-by: nib:gpt-6-astra * feat(cli): own opt-in diagnostics lifecycle Expose profiling and phase timing flags without enabling them for debug. Bind the private profiler before application initialization and propagate serving failures through the run lifecycle. Bound signal cleanup and preserve preload-only backend startup. Cover flag parsing, preload isolation, cancellation, and once-only shutdown. Document initialization and shutdown cancellation limitations. Assisted-by: nib:gpt-6-astra * feat(config): measure filter and reload phases Separate request lock waits from filtering and held work. Aggregate reload filesystem and parsing durations without logging inside the config lock. Preserve caller option execution under the lock after metadata collection. Extract only the internal diagnostics option before waiting for the lock. Keep direct callers uninstrumented unless they supply diagnostics options. Verify focused Diagnostics examples with the race detector, including option ordering, gated contention, errors, panic propagation, and parity. Assisted-by: nib:gpt-6-astra * fix(config): separate early reload observation Opaque callbacks cannot select timing before their execution point. Carry the startup recorder explicitly through loader construction. Apply late options once under the lock and honor composed last writes. Document early and late phase boundaries and observation identities. Report malformed gallery syntax as an internal parse failure without changing strict or non-strict loader returns. Assisted-by: nib:gpt-6-astra * fix(config): preserve per-model option callbacks The resolved snapshot suppresses existing callbacks during model defaults, even when diagnostics are disabled. Pass original options through the existing defaults pathway without adding diagnostics-driven replay. Test stateful options with two models in enabled and disabled modes. Preserve three calls and thread defaults of two and three. Assisted-by: nib:gpt-6-astra * feat(model): observe discovery phases Carry request diagnostics through config and loose-file discovery to separate directory enumeration from in-memory filtering and fallback. Keep existing APIs as background-context wrappers and preserve discovery policies, ordering, duplicates, errors, and filesystem operation counts. Add focused parity, privacy, phase, and disabled-allocation tests. The focused diagnostics race gate passes for model and galleryop. Assisted-by: nib:gpt-6-astra * feat(http): observe extraction phases Generate diagnostic identity before request discovery, independently of client correlation headers. End each extraction segment before inference so downstream latency does not inflate discovery measurements. Carry observations through discovery helpers and rebuilt request contexts without changing cancellation or model selection. Cover disabled parity, error outcomes, privacy, concurrent identity isolation and route contexts. Assisted-by: nib:gpt-6-astra * fix(http): retain diagnostics across failover Failover restores its entry request before retrying. Initialize the observation before that capture so direct extraction retains one ID. Keep extraction spans per attempt and end them before downstream work. Exercise two real targets without default middleware, including replay, shared identity, phase boundaries, and timing-off response parity. Assisted-by: nib:gpt-6-astra * feat(model): observe initialization and routing Mark loader entry before cache and admission checks. Measure only the ModelRouter callback, without attributing transport or worker latency. Test inherited identity and sanitized events with fake backends. Exercise the actual Echo router to keep profiles private across debug and timing options. Cover capture shutdown, forced close, and lifecycle failures. Assisted-by: nib:gpt-6-astra * docs(diagnostics): guide bounded investigations Document private profiling, sampling ownership, and phase boundaries so operators can interpret captures without assuming request attribution. Add Ginkgo drift checks for flags, defaults, phases, links, and capture warnings. These checks do not certify the operator prose. The focused documentation race test passes after four expected failures. The consolidated Diagnostics race gate passes across all eight packages. This gate does not validate the full suite, auth, or production UI. Assisted-by: nib:gpt-6-astra * refactor(cli): restore inline run lifecycle Remove diagnostic startup hooks so application startup stays readable in Run. Keep the profiler as an optional sidecar with bounded cleanup and an error watcher, without a general lifecycle runner. Replace hook simulations with real CLI startup checks and component capture tests. Document that application cleanup can exceed the private listener budget. The eight-package Diagnostics race gate passes. A cleanup mutation fails the real CLI listener-release regression as expected. Assisted-by: nib:gpt-6-astra * fix(cli): retain concurrent startup errors Cancellation must not hide a public API bind failure. Join the public serve result, cancellation cause, and HTTP shutdown errors inline. Ignore ErrServerClosed only after intentional shutdown. Exercise real startup with an occupied API port and a failed private listener. Isolate the Linux socket fixture in a subprocess and wait for cancellation acknowledgment without timing sleeps. Assisted-by: nib:gpt-6-astra * refactor(http): make request context primary Move context preparation into the request extractor. Diagnostics remain an optional addition, not the entry point for request processing. Preserve context initialization before failover captures the request. Assisted-by: nib:gpt-6-astra --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
76 lines
3.0 KiB
Go
76 lines
3.0 KiB
Go
// SPDX-License-Identifier: MIT
|
|
package model
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"os"
|
|
"path/filepath"
|
|
"testing"
|
|
|
|
"github.com/mudler/LocalAI/pkg/diagnostics"
|
|
. "github.com/onsi/ginkgo/v2"
|
|
. "github.com/onsi/gomega"
|
|
)
|
|
|
|
var _ = Describe("Diagnostics model directory enumeration", func() {
|
|
It("counts only the existing ReadDir and separates filename work without leaking paths", func() {
|
|
dir := GinkgoT().TempDir()
|
|
for _, name := range []string{"secret-candidate", "z-model", "weights.GGUF", "config.yaml", "CACHEDIR.TAG", "x.bak-save"} {
|
|
Expect(os.WriteFile(filepath.Join(dir, name), nil, 0600)).To(Succeed())
|
|
}
|
|
Expect(os.Mkdir(filepath.Join(dir, "secret-directory"), 0700)).To(Succeed())
|
|
ml := &ModelLoader{ModelPath: dir}
|
|
var events []diagnostics.Event
|
|
ctx := diagnostics.NewRecorder(func(e diagnostics.Event) { events = append(events, e) }).Request(context.Background())
|
|
calls := 0
|
|
readDir := func(path string) ([]os.DirEntry, error) {
|
|
calls++
|
|
Expect(events).To(BeEmpty()) // Emission happens after ReadDir, outside the measured operation.
|
|
return os.ReadDir(path)
|
|
}
|
|
plain, err := ml.listFilesInModelPathContext(context.Background(), readDir)
|
|
Expect(err).NotTo(HaveOccurred())
|
|
got, err := ml.listFilesInModelPathContext(ctx, readDir)
|
|
Expect(err).NotTo(HaveOccurred())
|
|
Expect(got).To(Equal(plain))
|
|
Expect(got).To(Equal([]string{"secret-candidate", "z-model"}))
|
|
Expect(calls).To(Equal(2))
|
|
Expect(events).To(HaveLen(3))
|
|
Expect(events[0].Count).To(Equal(1))
|
|
Expect(events[2].Count).To(Equal(7))
|
|
Expect(events[2].Phase).To(Equal(diagnostics.PhaseLooseFilter))
|
|
for _, e := range events {
|
|
Expect(e.ID).To(Equal(events[0].ID))
|
|
Expect(e.Outcome).To(Equal(diagnostics.OutcomeOK))
|
|
}
|
|
Expect(fmt.Sprint(events)).NotTo(ContainSubstring("secret"))
|
|
Expect(fmt.Sprint(events)).NotTo(ContainSubstring(dir))
|
|
wrapped, err := ml.ListFilesInModelPath()
|
|
Expect(err).NotTo(HaveOccurred())
|
|
Expect(wrapped).To(Equal(got))
|
|
Expect(events).To(HaveLen(3))
|
|
observed, err := ml.ListFilesInModelPathContext(ctx)
|
|
Expect(err).NotTo(HaveOccurred())
|
|
Expect(observed).To(Equal(got))
|
|
})
|
|
It("preserves missing-directory errors and does not run filename filtering", func() {
|
|
ml := &ModelLoader{ModelPath: filepath.Join(GinkgoT().TempDir(), "secret-missing")}
|
|
var events []diagnostics.Event
|
|
ctx := diagnostics.NewRecorder(func(e diagnostics.Event) { events = append(events, e) }).Request(context.Background())
|
|
plain, plainErr := ml.ListFilesInModelPath()
|
|
got, err := ml.ListFilesInModelPathContext(ctx)
|
|
Expect(os.IsNotExist(err)).To(BeTrue())
|
|
Expect(err.Error()).To(Equal(plainErr.Error()))
|
|
Expect(got).To(Equal(plain))
|
|
Expect(events).To(HaveLen(1))
|
|
Expect(events[0].Outcome).To(Equal(diagnostics.OutcomeError))
|
|
Expect(events[0].Count).To(Equal(1))
|
|
})
|
|
It("adds no allocations for disabled timing on empty enumeration", func() {
|
|
ml := &ModelLoader{}
|
|
readDir := func(string) ([]os.DirEntry, error) { return nil, nil }
|
|
Expect(testing.AllocsPerRun(100, func() { _, _ = ml.listFilesInModelPathContext(context.Background(), readDir) })).To(BeZero())
|
|
})
|
|
})
|