mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-12 22:33:54 -04:00
Dimitris Karakasilis
8aeea4cdde
The recommended sampling parameters for a model family were applied at install and then never took effect. Two things went wrong on the way to disk. They were written as top level keys. ModelConfig embeds PredictionOptions under the "parameters" yaml key, so temperature, top_p, top_k, min_p, repeat_penalty and presence_penalty are only read from there. At the top level they parse without error and are then ignored for the life of the model. They were also merged in after the YAML had already been marshalled. The only re-marshal sat behind the artifact binding, which an entry carrying files: never reaches, so for those entries the defaults were computed and then dropped before anything was written. Neither failure was visible in normal use. ApplyInferenceDefaults runs again at load time and fills the same values from the same table, so the model ends up tuned correctly while the file on disk pins nothing. It surfaces when someone edits one of those values expecting it to win, or when a family is absent from inference_defaults.json and there is nothing to refill from. Both install paths are covered: an entry carrying files:, and one that binds a primary artifact instead. The empty base spec asserted that the authored parameters block landed verbatim. It now checks the authored keys individually, because the family defaults are merged into that same block. Assisted-by: Claude:claude-opus-5 Signed-off-by: Dimitris Karakasilis <dimitris@karakasilis.me>