Ettore Di Giacinto 7f3029e0d3 feat: return 429 when backends are saturated ...

When backends are at capacity (per-model max_concurrent or the
process-wide --max-concurrent-backend-requests ceiling), the response
was 503. The OpenAI SDK, litellm, and most agent harnesses key on 429
for rate-limit backoff and treat 503 as a hard error.

Both saturation paths now return 429 with the existing Retry-After
header and type: "rate_limit_error" in the JSON body. The per-model
admission middleware keeps admission_rejected as the code field so
existing alerts that match on it still fire.

Non-saturation 503s are unchanged: model cold-loading (with progress
body), model-load failure cooldown, PII detector fail-closed, and
classifier unavailable. These mean "not ready" rather than "busy".

Assisted-by: AGENT:regolo/glm5.2 [TOOL]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2026-09-17 21:04:07 +00:00

..

admission_test.go

feat: return 429 when backends are saturated

2026-09-17 21:04:07 +00:00

admission.go

feat: return 429 when backends are saturated

2026-09-17 21:04:07 +00:00

baseurl_test.go

fix: correct scheme/host on self-referential URLs behind an HTTPS reverse proxy (#10482) (#10504)

2026-06-25 08:10:59 +02:00

baseurl.go

fix: correct scheme/host on self-referential URLs behind an HTTPS reverse proxy (#10482) (#10504)

2026-06-25 08:10:59 +02:00

compress_test.go

perf(http): gzip responses, cache hashed assets, bound the trace endpoints (#11056)

2026-07-22 22:51:25 +02:00

compress.go

perf(http): gzip responses, cache hashed assets, bound the trace endpoints (#11056)

2026-07-22 22:51:25 +02:00

compression_test.go

feat(chat): add end-to-end context compression (#11556)

2026-08-18 11:31:03 +00:00

compression.go

feat(chat): add end-to-end context compression (#11556)

2026-08-18 11:31:03 +00:00

context_keys.go

feat(middleware): Model routing, PII filtering, Cloud model proxies (#9802)

2026-05-25 09:28:27 +02:00

forwarded_prefix_test.go

chore: Security hardening (#9719)

2026-05-08 16:25:45 +02:00

forwarded_prefix.go

chore: Security hardening (#9719)

2026-05-08 16:25:45 +02:00

middleware_suite_test.go

feat: migrate to echo and enable cancellation of non-streaming requests (#7270)

2025-11-14 22:57:53 +01:00

node_header_concurrency_test.go

feat(distributed): gated X-LocalAI-Node response header (middleware + wrapper) (#9976)

2026-05-25 10:51:48 +02:00

node_header_integration_test.go

feat(distributed): gated X-LocalAI-Node response header (middleware + wrapper) (#9976)

2026-05-25 10:51:48 +02:00

node_header_test.go

feat(distributed): gated X-LocalAI-Node response header (middleware + wrapper) (#9976)

2026-05-25 10:51:48 +02:00

node_header.go

feat(distributed): gated X-LocalAI-Node response header (middleware + wrapper) (#9976)

2026-05-25 10:51:48 +02:00

probe_trim_test.go

fix(router): production-ready request router + auto-size batch for embedding/rerank (#10104)

2026-06-12 16:21:15 +02:00

request_config_revision_test.go

fix(distributed): hash the config as persisted, not as defaulted

2026-08-24 11:45:32 +00:00

request_test.go

fix(ollama): accept :latest tag on model lookup (#11732)

2026-08-26 08:57:36 +02:00

request.go

fix(ollama): accept :latest tag on model lookup (#11732)

2026-08-26 08:57:36 +02:00

route_model_test.go

feat(router): make KNN a first-class classifier with a persisted, curated corpus (#10652)

2026-08-18 09:37:43 +02:00

route_model.go

feat(router): make KNN a first-class classifier with a persisted, curated corpus (#10652)

2026-08-18 09:37:43 +02:00

security_headers_test.go

feat(localvqe/audio): v1.3 release and add spectrograms to audio transform UI (#10113)

2026-05-31 23:56:46 +02:00

security_headers.go

feat(localvqe/audio): v1.3 release and add spectrograms to audio transform UI (#10113)

2026-05-31 23:56:46 +02:00

strippathprefix_test.go

feat: migrate to echo and enable cancellation of non-streaming requests (#7270)

2025-11-14 22:57:53 +01:00

strippathprefix.go

chore: Security hardening (#9719)

2026-05-08 16:25:45 +02:00

trace_body_cap_test.go

fix(traces): cap captured body size to keep admin Traces UI responsive (#9946)

2026-05-22 15:29:24 +02:00

trace_live_test.go

feat(traces): show in-flight API requests (#11368)

2026-08-05 09:37:05 +02:00

trace_pagination_test.go

perf(http): gzip responses, cache hashed assets, bound the trace endpoints (#11056)

2026-07-22 22:51:25 +02:00

trace_persistence_test.go

feat(tracing): persist bounded trace histories (#11203)

2026-07-30 12:20:01 +02:00

trace_redact_test.go

chore: Security hardening (#9719)

2026-05-08 16:25:45 +02:00

trace_summary_test.go

feat(ui): give Operate and Studio a front door, and fix two layout regressions (#11305)

2026-08-03 00:16:08 +02:00

trace_summary.go

feat(ui): give Operate and Studio a front door, and fix two layout regressions (#11305)

2026-08-03 00:16:08 +02:00

trace.go

feat(traces): show in-flight API requests (#11368)

2026-08-05 09:37:05 +02:00

usage_stamp.go

feat(middleware): Model routing, PII filtering, Cloud model proxies (#9802)

2026-05-25 09:28:27 +02:00

usage_test.go

feat(middleware): Model routing, PII filtering, Cloud model proxies (#9802)

2026-05-25 09:28:27 +02:00

usage.go

feat(middleware): Model routing, PII filtering, Cloud model proxies (#9802)

2026-05-25 09:28:27 +02:00