mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-21 21:54:52 -04:00
The Q4 and Q8 overrides place context_size inside parameters, where PredictionOptions ignores it. Move it beside parameters so both builds use the intended 8,192-token context, matching F16. Assisted-by: Codex:GPT-6