Stefan Walcz c3bea567fe fix(llama-cpp): do not stream the error text as content on pre-stream failures (#12425) ...

When the first result of a streamed request is an error (for example a
prompt that exceeds the context), PredictStream wrote the error message
as a Reply and only then returned the error status. LocalAI treated that
Reply as the first token: it sent the assistant role chunk and the error
text as `content` on an HTTP 200 stream. Because a chunk had already been
written, the pre-stream HTTP error path from #12204 never triggered, so
streaming clients still got a 200 with the error as model output, while
the same request without streaming correctly returns a 400.

Return the error only as the gRPC status. The e2e backend suite gets a
`context_overflow` capability (enabled for llama-cpp) that streams an
over-long prompt and asserts an error status with no content.

Assisted-by: Claude:claude-opus-5-5

Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>

2026-10-02 09:23:57 +02:00

..

backend_test.go

fix(llama-cpp): do not stream the error text as content on pre-stream failures (#12425)

2026-10-02 09:23:57 +02:00

suite_test.go

feat(backends): add ik-llama-cpp (#9326)

2026-04-12 13:51:28 +02:00