From 05af7b072c64a738178924e5a0dbd54c9f03e6d1 Mon Sep 17 00:00:00 2001 From: pos-ei-don <1822533+pos-ei-don@users.noreply.github.com> Date: Sat, 24 Oct 2026 19:30:35 +0000 Subject: [PATCH] docs(vllm): explain sampling validation Describe stop strings, greedy decoding, and seed handling after sampling parameters are validated during construction. Signed-off-by: pos-ei-don <1822533+pos-ei-don@users.noreply.github.com> --- docs/content/advanced/model-configuration.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/content/advanced/model-configuration.md b/docs/content/advanced/model-configuration.md index bea83b6f5..5fd89323c 100644 --- a/docs/content/advanced/model-configuration.md +++ b/docs/content/advanced/model-configuration.md @@ -833,6 +833,12 @@ The fields below come from upstream llama.cpp's **CLI completion tool** and are These options apply when using the `vllm` backend: +The `vllm` backend validates sampling parameters before generation. Stop strings +work with streaming and non-streaming requests; an empty stop string is invalid. +Presence and frequency penalties must be between -2 and 2. A temperature of `0` +selects greedy decoding, which ignores `top_p` and `top_k`. A seed of `-1` uses +vLLM's default random seed behavior. + | Field | Type | Description | |-------|------|-------------| | `gpu_memory_utilization` | float32 | GPU memory utilization (0.0-1.0, default 0.9) |