mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-29 09:35:02 -04:00
fix(failover): send remote targets their own upstream model
A chain request reached a cloud-proxy target with the client's model, the chain name, whenever the target set no upstream_model: passthrough forwards the body's model and translate falls back to it. The upstream answered 404, which neither retries nor trips, while the liveness probe, which checks the target's own name, kept passing. PrepareTarget now sets the upstream model of a remote target to proxy.upstream_model or the target name, the same name the probe uses. The request pipeline and realtime chain stages both call it. Assisted-by: Claude:claude-opus-5-5
This commit is contained in:
1 parent
d1a59aae41
commit
e183ea11d1
8 files changed
+98
-7
No files matched your search
@@ -50,6 +50,9 @@ Rules:
|
||||
- A chain cannot also set `alias` or `backend`.
|
||||
- Responses name the chain as the model. The `X-LocalAI-Served-Model` header
|
||||
names the target that served the request.
|
||||
- A remote (`cloud-proxy`) target receives its own model name, never the chain
|
||||
name: `proxy.upstream_model`, or the target name when `upstream_model` is
|
||||
empty. The health check looks for the same name.
|
||||
|
||||
## How the target is chosen
|
||||
|
||||
|
||||
@@ -100,6 +100,12 @@ The behaviour matches aliases. Responses echo the chain name. Usage and traces
|
||||
record `requested=<chain>` and `served=<target>` through the existing
|
||||
`ContextKeyRequestedModel` and `ContextKeyServedModel` keys.
|
||||
|
||||
The upstream of a remote target never sees the chain name. A request served
|
||||
through a chain reaches a remote target with that target's upstream model:
|
||||
`proxy.upstream_model`, or the target name when it is empty. This holds in
|
||||
passthrough and translate mode, and it is the same name the liveness probe
|
||||
looks for in `/v1/models` (one helper derives both).
|
||||
|
||||
## Failover manager
|
||||
|
||||
New package: `core/services/failover`. The application creates one `Manager` at
|
||||
|
||||
Reference in new issue
Block a user