From 3302c769643d055f6ce05f82b3c242b696f2ee72 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Sun, 25 Oct 2026 08:50:18 +0000 Subject: [PATCH] docs(vllm,sglang): note that omitted max_tokens is not capped The vLLM and SGLang backends no longer stop at 200 tokens when a request has no max_tokens. Generation is now bounded by the context window. Assisted-by: Claude:claude-sonnet-5-5 Signed-off-by: Ettore Di Giacinto --- docs/content/features/text-generation.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/content/features/text-generation.md b/docs/content/features/text-generation.md index d987fcf9c..f7cf77be9 100644 --- a/docs/content/features/text-generation.md +++ b/docs/content/features/text-generation.md @@ -31,6 +31,8 @@ curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/jso Available additional parameters: `top_p`, `top_k`, `max_tokens` +When a request omits `max_tokens`, the vLLM and SGLang backends do not cap generation at 200 tokens. Generation is bounded only by the context window of the model. + Reasoning models return their thinking in the `reasoning` field. When a model reasons and calls a tool in the same turn, see [Interleaved Thinking with Tool Calls]({{%relref "features/interleaved-thinking" %}}). When `stream: true` is set and the llama.cpp backend fails before the first chunk, for example because the prompt exceeds the context size, the request fails with an HTTP error. The error message is not streamed as assistant content. An error after streaming has started is reported inside the stream.