Ettore Di Giacinto 10f7b1ca60 fix(localai-proxy): cancel the upstream request when the caller goes away ...

The proxy sent every chat and completion request with
context.Background, and the gRPC server gave rich backends no context
at all. When a client disconnected, or failover gave up on the target,
the upstream kept generating to the end, which costs tokens on a paid
or shared upstream. A silent upstream held the backend forever.

Add the optional AIModelRichContext interface. The gRPC server prefers
it and passes the call's context, like the Score and Rerank
extensions. The proxy implements it, so the upstream request ends with
the gRPC call.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5-5 [Claude Code]

2026-09-27 20:16:02 +00:00

..

base

feat: Add kimodo.cpp and 3D animation API/UI (#12095)

2026-09-18 06:12:03 +01:00

grpcerrors

fix(distributed): reject wrong-model requests at the backend (#10970)

2026-07-20 13:05:47 +02:00

metadata

feat(kimodocpp): track usage through generic backend metadata (#12162)

2026-09-20 19:39:06 +01:00

animation_usage_test.go

feat(kimodocpp): track usage through generic backend metadata (#12162)

2026-09-20 19:39:06 +01:00

auth_test.go

feat: add distributed mode (#9124)

2026-03-30 00:47:27 +02:00

backend.go

feat: Add kimodo.cpp and 3D animation API/UI (#12095)

2026-09-18 06:12:03 +01:00

client_busy_test.go

fix(model): make backend shutdown model-scoped (#10865)

2026-07-19 08:43:17 +02:00

client.go

feat: Add kimodo.cpp and 3D animation API/UI (#12095)

2026-09-18 06:12:03 +01:00

embed.go

feat: Add kimodo.cpp and 3D animation API/UI (#12095)

2026-09-18 06:12:03 +01:00

forward_test.go

feat(middleware): Model routing, PII filtering, Cloud model proxies (#9802)

2026-05-25 09:28:27 +02:00

grpc_suite_test.go

feat: add distributed mode (#9124)

2026-03-30 00:47:27 +02:00

integration_toolcalls_test.go

feat(middleware): Model routing, PII filtering, Cloud model proxies (#9802)

2026-05-25 09:28:27 +02:00

interface.go

fix(localai-proxy): cancel the upstream request when the caller goes away

2026-09-27 20:16:02 +00:00

model_identity_modalities_test.go

feat(grpc): serve Rerank from Go backends and skip Unimplemented targets

2026-09-27 07:42:21 +00:00

model_identity_test.go

fix(distributed): reject wrong-model requests at the backend (#10970)

2026-07-20 13:05:47 +02:00

parentwatch_test.go

fix(process): give backend workers a parent-death safety net (#10639)

2026-07-02 19:16:48 +02:00

parentwatch.go

fix(process): give backend workers a parent-death safety net (#10639)

2026-07-02 19:16:48 +02:00

rich_context_test.go

fix(localai-proxy): cancel the upstream request when the caller goes away

2026-09-27 20:16:02 +00:00

rich_test.go

feat(middleware): Model routing, PII filtering, Cloud model proxies (#9802)

2026-05-25 09:28:27 +02:00

server_rerank_test.go

feat(grpc): serve Rerank from Go backends and skip Unimplemented targets

2026-09-27 07:42:21 +00:00

server.go

fix(localai-proxy): cancel the upstream request when the caller goes away

2026-09-27 20:16:02 +00:00

stream_cleanup_test.go

feat: bound global admission and expose running backend traces (#11560)

2026-08-18 08:56:59 +02:00

transcription_live_conn_test.go

feat(realtime): Semantic VAD EOU token (#10444)

2026-06-30 09:01:22 +02:00

transcription_live_test.go

feat(realtime): Semantic VAD EOU token (#10444)

2026-06-30 09:01:22 +02:00