perf(http): gzip responses, cache hashed assets, bound the trace endpoints (#11056)

Three measured HTTP-layer regressions on a live deployment, fixed together
because they all shape the bytes on the wire.

1. No compression. The server sent no Content-Encoding regardless of what
   the client asked for, confirmed with curl straight at 127.0.0.1:8080 so
   it was not an ingress artefact. Adds gzip middleware, on by default and
   configurable via LOCALAI_DISABLE_HTTP_COMPRESSION and
   LOCALAI_HTTP_COMPRESSION_MIN_LENGTH (default 1024 bytes so tiny bodies
   are not wastefully wrapped). Streaming routes are skipped explicitly:
   an SSE Accept header, a WebSocket upgrade, and the completion / SSE /
   log-tail path prefixes, because whether a completion request streams is
   decided by the request body, which the middleware runs too early to see.
   Already-compressed formats (woff2, png, mp4, ...) are skipped too; gzip
   made those marginally larger. Measured over the embedded React build:
   JS+CSS 2815 KB raw to 808 KB gzipped (3.48x).

2. No cache headers on content-hashed assets. Vite hashes the filenames,
   so a given /assets/ URL can never change content, yet they shipped with
   no Cache-Control, ETag or Last-Modified, and the browser re-fetched the
   whole bundle on every navigation with no conditional request available.
   /assets/* now carries public, max-age=31536000, immutable. index.html
   stays no-cache so a deploy is picked up, and the unhashed locale JSONs
   get a short TTL rather than the immutable one.

3. Unbounded trace endpoints. /api/traces returned 21,033,606 bytes in
   4.65s and /api/backend-traces 3,471,682 bytes in 1.50s, and the admin
   UI polls both every few seconds. The ring buffer holds up to 1024
   entries, each embedding full input_text payloads. Both list endpoints
   now take limit / offset / full, default to 50 entries, and strip the
   heavy fields (request and response bodies plus headers for API traces,
   body and data for backend traces) unless full=true. Every trace gets a
   process-lifetime ID and GET /api/traces/{id} and
   /api/backend-traces/{id} serve the full record, which is what the UI
   fetches when a row is expanded. The list body stays a JSON array;
   paging metadata rides in X-Total-Count, X-Trace-Offset and
   X-Trace-Limit. Reproducing the live shape in a test, the polled payload
   goes from 21,131,097 bytes to 7,201 bytes.


Assisted-by: Claude:claude-opus-4-8 [Claude Code]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
mudler's LocalAI [bot]
2026-07-22 22:51:25 +02:00
committed by GitHub
parent 8eb8376596
commit ff299df453
24 changed files with 1439 additions and 49 deletions

View File

@@ -83,6 +83,8 @@ For more information on VRAM management, see [VRAM and Memory Management]({{%rel
| `--cors` | `false` | Enable CORS (Cross-Origin Resource Sharing) | `$LOCALAI_CORS`, `$CORS` |
| `--cors-allow-origins` | | Comma-separated list of allowed CORS origins | `$LOCALAI_CORS_ALLOW_ORIGINS`, `$CORS_ALLOW_ORIGINS` |
| `--csrf` | `false` | Enable Fiber CSRF middleware | `$LOCALAI_CSRF` |
| `--disable-http-compression` | `false` | Disable gzip compression of HTTP responses. Compression is enabled by default; streaming endpoints (streaming chat completions, SSE bridges, WebSocket upgrades) and already-compressed formats are never compressed | `$LOCALAI_DISABLE_HTTP_COMPRESSION` |
| `--http-compression-min-length` | `1024` | Minimum response size in bytes before gzip compression is applied. Smaller responses are sent as-is because the gzip envelope would outweigh the saving | `$LOCALAI_HTTP_COMPRESSION_MIN_LENGTH` |
| `--upload-limit` | `15` | Default upload-limit in MB | `$LOCALAI_UPLOAD_LIMIT`, `$UPLOAD_LIMIT` |
| `--api-keys` | | List of API Keys to enable API authentication. When this is set, all requests must be authenticated with one of these API keys | `$LOCALAI_API_KEY`, `$API_KEY` |
| `--disable-webui` | `false` | Disables the web user interface. When set to true, the server will only expose API endpoints without serving the web interface | `$LOCALAI_DISABLE_WEBUI`, `$DISABLE_WEBUI` |