Ettore Di Giacinto 10f7b1ca60 fix(localai-proxy): cancel the upstream request when the caller goes away ...

The proxy sent every chat and completion request with
context.Background, and the gRPC server gave rich backends no context
at all. When a client disconnected, or failover gave up on the target,
the upstream kept generating to the end, which costs tokens on a paid
or shared upstream. A silent upstream held the backend forever.

Add the optional AIModelRichContext interface. The gRPC server prefers
it and passes the call's context, like the Score and Rerank
extensions. The proxy implements it, so the upstream request ends with
the gRPC call.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:claude-opus-5-5 [Claude Code]

2026-09-27 20:16:02 +00:00

..

audio

feat(localvqe/audio): v1.3 release and add spectrograms to audio transform UI (#10113)

2026-05-31 23:56:46 +02:00

clusterrouting

refactor(routing): extract replica picker into pkg/clusterrouting (#10123)

2026-06-01 09:38:55 +02:00

concurrency

feat: add distributed mode (#9124)

2026-03-30 00:47:27 +02:00

credentials

feat(downloads): authenticate registries, galleries and downloads from a credentials file (#12028)

2026-09-13 23:11:40 +02:00

distributedhdr

feat: prefix-cache-aware routing for distributed mode (#10071)

2026-05-30 23:24:22 +02:00

downloader

fix(gallery): verification follow-ups for oci:// galleries (#12243)

2026-09-24 21:20:21 +02:00

functions

fix(grammars): restore backslash escaping in llama31 grammar fixture (#11242)

2026-07-30 23:53:23 +02:00

grpc

fix(localai-proxy): cancel the upstream request when the caller goes away

2026-09-27 20:16:02 +00:00

httpclient

Revert "chore(tests): Avoid network, sleep and more during tests" (#11601)

2026-08-19 16:39:39 +02:00

huggingface-api

Revert "chore(tests): Avoid network, sleep and more during tests" (#11601)

2026-08-19 16:39:39 +02:00

mcp/localaitools

refactor: name the proxy backends with constants

2026-09-27 07:42:21 +00:00

model

fix: resolve gosec alerts in files this change touches

2026-09-27 08:42:12 +00:00

modelartifacts

fix(modelartifacts): reuse committed sibling files for narrowed allow_patterns (#11484)

2026-09-27 21:10:35 +02:00

natsauth

fix(distributed): aggregate prefix cache pressure

2026-09-11 21:48:07 +00:00

oci

fix(gallery): verification follow-ups for oci:// galleries (#12243)

2026-09-24 21:20:21 +02:00

radixtree

feat: prefix-cache-aware routing for distributed mode (#10071)

2026-05-30 23:24:22 +02:00

reasoning

fix(reasoning): ignore preclosed prompt markers

2026-09-11 22:44:39 +00:00

safefile

fix(distributed): bound ephemeral staging (#11924)

2026-09-08 18:13:40 +02:00

sanitize

feat: add distributed mode (#9124)

2026-03-30 00:47:27 +02:00

signals

feat: add distributed mode (#9124)

2026-03-30 00:47:27 +02:00

sound

fix: correct stale symbol names in doc comments (#12139)

2026-09-19 23:39:44 +02:00

store

fix(backends): refuse foreign model loads in opus and local-store (#10769)

2026-07-11 09:17:34 +02:00

system

feat(gallery): galleries published as OCI artifacts (#12167)

2026-09-21 09:23:05 +02:00

tokens

feat(chat): add end-to-end context compression (#11556)

2026-08-18 11:31:03 +00:00

utils

fix(utils): reject tar hardlinks that escape the extraction root (#11266)

2026-08-01 09:25:35 +02:00

vram

fix(vram): persist remote probe metadata (#11487)

2026-08-23 08:55:13 +02:00

vrambudget

feat(vram): per-node VRAM allocation budget (LOCALAI_VRAM_BUDGET) (#10833)

2026-07-15 09:58:45 +02:00

xio

feat(xio): make copy buffer size configurable (#11660)

2026-08-21 16:14:26 +02:00

xsync

chore: fix go.mod module (#2635)

2024-06-23 08:24:36 +00:00

xsysinfo

feat(system): report per-model DRM VRAM (#12026)

2026-09-27 21:18:34 +02:00