mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 17:18:49 -04:00
Stefan Walcz
5600a1cabe
Agent jobs (/api/agent/jobs, local and on distributed agent workers) run cogito against a LocalAI client that sends no max_tokens of its own, so every completion gets the client's fixed default of 16384. max_iterations bounds the number of turns and loop_detection repeated tool calls, but neither stops one completion that keeps generating, and 16384 is the same for a small tool-picking model and a reasoning model that thinks at length. Add agent.max_generation_tokens to the model config. When set, both job paths apply it to the client with SetMaxTokens; requests that carry their own cap still win, so cogito's length retry can raise it. Unset keeps today's behaviour. The specs run each job path against an MCP fixture and a recording LLM endpoint and check the max_tokens the completions carry; they fail when either call site drops the cap. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>