mirror of
https://github.com/mudler/LocalAI.git
synced 2026-08-02 11:30:44 -04:00
bf19758e05
* fix(ollama): cap num_ctx so it cannot wrap negative when cast to int32 applyOllamaOptions copied a client-supplied options.num_ctx straight into cfg.ContextSize with only a > 0 check. That value is later cast to int32 before it reaches the backend (core/backend/options.go), so a num_ctx above math.MaxInt32 silently wrapped into a negative context size that was then sent to the LoadModel gRPC call. Both /api/chat and /api/generate share applyOllamaOptions, so both endpoints were affected. Cap num_ctx at math.MaxInt32 so the later cast stays positive, and add internal regression coverage for the overflow, in-range, and unset cases. num_ctx remains an intentional user override, so this does not re-impose the hardware-aware auto context clamp; that policy choice is left to maintainers. Fixes #11022 Signed-off-by: Tai An <antai12232931@outlook.com> * fix(ollama): clamp num_ctx to model context ceiling, not just int32 Per review on #11032: capping only at math.MaxInt32 still let an unauthenticated request replace the hardware/model-derived context limit with ~2.1B tokens, so a real backend could attempt a catastrophic KV-cache allocation. Treat any existing positive cfg.ContextSize as the server ceiling and clamp num_ctx down to it (smaller values still honored), while retaining the int32-safe bound when no smaller ceiling exists. Shared by /api/chat and /api/generate via applyOllamaOptions. Add regression coverage proving num_ctx=2,000,000,000 cannot replace an existing 4096/8192 ceiling. Signed-off-by: Tai An <antai12232931@outlook.com> --------- Signed-off-by: Tai An <antai12232931@outlook.com>