diff --git a/docs/content/features/text-generation.md b/docs/content/features/text-generation.md index d987fcf9c..f7cf77be9 100644 --- a/docs/content/features/text-generation.md +++ b/docs/content/features/text-generation.md @@ -31,6 +31,8 @@ curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/jso Available additional parameters: `top_p`, `top_k`, `max_tokens` +When a request omits `max_tokens`, the vLLM and SGLang backends do not cap generation at 200 tokens. Generation is bounded only by the context window of the model. + Reasoning models return their thinking in the `reasoning` field. When a model reasons and calls a tool in the same turn, see [Interleaved Thinking with Tool Calls]({{%relref "features/interleaved-thinking" %}}). When `stream: true` is set and the llama.cpp backend fails before the first chunk, for example because the prompt exceeds the context size, the request fails with an HTTP error. The error message is not streamed as assistant content. An error after streaming has started is reported inside the stream.