mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 09:57:57 -04:00
06b4a29387
The gRPC configuration table only listed the two fields with a one-line description each, without defaults, without explaining what the total load window looks like, and without hinting when a user should adjust them. In practice the default 20 attempts x 2 s = 40 s window is way too tight for large NVFP4 / FP8 models on slow storage or first-run CUDA-graph capture, and the resulting kill (exitCode=120, 'context canceled') looks like a backend crash even though the backend is still making legitimate forward progress. Extend the section with: - Defaults column (20 and 2) added to the table - Prose explaining that these govern the readiness handshake between LocalAI and a freshly spawned backend (Health polling loop) - Total-load-window formula - Concrete failure signature so users can recognize a timeout-kill vs. a real backend crash - Example configuration for a ~10 min cold-load window (grpc.attempts 140, attempts_sleep_time 5), with a note that inference-timeouts and the watchdog are unaffected.