LocalAI

0 Fork 0

mirror of https://github.com/mudler/LocalAI.git synced 2026-09-25 15:44:56 -04:00

Files

T

Ettore Di Giacinto 7f3029e0d3 feat: return 429 when backends are saturated

When backends are at capacity (per-model max_concurrent or the
process-wide --max-concurrent-backend-requests ceiling), the response
was 503. The OpenAI SDK, litellm, and most agent harnesses key on 429
for rate-limit backoff and treat 503 as a hard error.

Both saturation paths now return 429 with the existing Retry-After
header and type: "rate_limit_error" in the JSON body. The per-model
admission middleware keeps admission_rejected as the code field so
existing alerts that match on it still fire.

Non-saturation 503s are unchanged: model cold-loading (with progress
body), model-load failure cooldown, PII detector fail-closed, and
classifier unavailable. These mean "not ready" rather than "busy".

Assisted-by: AGENT:regolo/glm5.2 [TOOL]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2026-09-17 21:04:07 +00:00

_index.en.md

docs: fix dead anchors and a dead section link (#11885 )

2026-09-05 23:47:13 +02:00

ai-coding-assistants.md

docs(dco): let maintainer-operated automation sign off (#11850 )

2026-09-03 23:08:38 +02:00

api-errors.md

feat: return 429 when backends are saturated

2026-09-17 21:04:07 +00:00

architecture.md

docs: onboarding overhaul, dedup, and error docs (#7711 ) (#10895 )