mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-05 12:34:43 -04:00
fix(cloud-proxy): surface Anthropic refusals instead of empty replies (#12424)
When an Anthropic model declines to answer, the Messages API returns HTTP 200 with `stop_reason: "refusal"` and empty `content`. The translate mode mapped that to a normal reply with no content, so the OpenAI-compatible response looked like a successful completion (`finish_reason: "stop"`, empty message). Routers, agents and UIs could not tell "the model declined" from "the model had nothing to say", and no fallback was triggered. Return an explicit error for `stop_reason: "refusal"` in both the non-streaming path and the streaming path (`message_delta`). Regular replies, including empty `end_turn` replies, are unchanged. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
This commit is contained in:
1 parent
985df46d24
commit
1056c62f4c
3 files changed
+96
-3
No files matched your search
@@ -198,6 +198,8 @@ image blocks, and per-request usage tokens are dropped through the
|
||||
internal `Predict()` signature. Use passthrough mode when your clients need
|
||||
the upstream's full feature set.
|
||||
|
||||
In translate mode, an Anthropic response with `stop_reason: "refusal"` is returned to the client as an error instead of an empty successful reply, for both non-streaming and streaming requests. A streamed response may already have delivered partial content when the refusal arrives. Responses that end normally (`end_turn`) are unaffected, even when their content is empty.
|
||||
|
||||
#### Anthropic prompt caching
|
||||
|
||||
`proxy.cache_prompt: true` makes the translator add Anthropic
|
||||
|
||||
Reference in new issue
Block a user