Ettore Di Giacinto ad6d1dbc8b feat(grpc): request cancellation for Go backends via the Cancellable capability ...

The llama.cpp C++ backend aborts generation when its gRPC context is
cancelled (grpc-server.cpp polls context->IsCancelled() in the result
loops), but Go backends served by pkg/grpc never observed context
cancellation: a disconnected client left the generation running to
completion. Add an optional Cancellable capability; the server registers
context.AfterFunc on the request/stream context (after the Locking block
so queued requests cannot abort the current owner) covering both rich
and legacy paths. dllm implements it: measured cancel latency ~10ms vs
~10s of orphaned generation, and follow-up requests no longer queue
behind cancelled ones (~220ms vs ~9s in the e2e proof).

Assisted-by: Claude Code (Fable 5)
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

2026-06-11 17:50:04 +00:00

..

advanced

feat: forward reasoning_effort to the backend so jinja models honor it (#10184)

2026-06-05 13:45:43 +00:00

features

feat(grpc): request cancellation for Go backends via the Cancellable capability

2026-06-11 17:50:04 +00:00

getting-started

feat(cli): add interactive chat mode (#10226)

2026-06-09 14:58:44 +00:00

installation

feat(ui): Interactive model config editor with autocomplete (#9149)

2026-04-07 14:42:23 +02:00

reference

feat(cli): add interactive chat mode (#10226)

2026-06-09 14:58:44 +00:00

_index.md

docs: position LocalAI as a composable engine, not a bundle (#10136)

2026-06-02 17:34:43 +02:00

faq.md

docs: position LocalAI as a composable engine, not a bundle (#10136)

2026-06-02 17:34:43 +02:00

integrations.md

fix(anthropic): do not emit empty tokens and fix SSE tool calls (#9258)

2026-04-07 00:38:21 +02:00

overview.md

docs: architecture & feature diagrams (blueprint style) (#10137)

2026-06-02 18:43:22 +02:00

whats-new.md

docs: fix documentation typos (#10125)

2026-06-01 14:31:08 +02:00