diff --git a/docs/content/features/openai-functions.md b/docs/content/features/openai-functions.md index 0ed724daa..5e7a384ca 100644 --- a/docs/content/features/openai-functions.md +++ b/docs/content/features/openai-functions.md @@ -36,6 +36,34 @@ Reasoning content (`...` blocks from DeepSeek R1, Qwen3, Gemma 4, No configuration required - the autoparser detects the tool call format for any `ggml`/`gguf` model that was trained with tool support. +#### Migrating older Ministral 3 configurations + +The `mistralai_ministral-3-14b-reasoning-2512-multimodal` gallery entry uses the model's embedded template and llama.cpp's native tool parser. +Older installations inherit the Mistral 0.3 prompt and JSON parser, which can return tool calls such as `pick_tool{...}` as text. + +For an existing installation, replace the model YAML's `template` and `function` sections with: + +```yaml +template: + use_tokenizer_template: true +function: + disable_no_action: true + automatic_tool_parsing_fallback: true + grammar: + disable: true +``` + +Set `use_jinja:true` in the existing `options` list: + +```yaml +options: + - use_jinja:true +``` + +Remove the inherited `stopwords` list so llama.cpp controls the model's end-of-turn markers. +Keep your model path, `mmproj`, sampling settings, and MCP configuration. Reload the model after saving the YAML. +Gallery changes do not rewrite installed model configurations. + ### vLLM / vLLM Omni The parser must be specified explicitly because vLLM itself doesn't auto-detect one. Pass it via the model `options`: @@ -312,4 +340,4 @@ Grammars and function tools can be used as well in conjunction with vision APIs: ## 💡 Examples -A full e2e example with `docker-compose` is available [here](https://github.com/mudler/LocalAI-examples/tree/main/functions). \ No newline at end of file +A full e2e example with `docker-compose` is available [here](https://github.com/mudler/LocalAI-examples/tree/main/functions). diff --git a/gallery/index.yaml b/gallery/index.yaml index 74610a5b1..104827cc3 100644 --- a/gallery/index.yaml +++ b/gallery/index.yaml @@ -41311,7 +41311,7 @@ sha256: 2740ba9e9b30b09be4282a9a9f617ec43dc47b89aed416cb09b5f698f90783b5 uri: huggingface://unsloth/Ministral-3-14B-Instruct-2512-GGUF/mmproj-F32.gguf - name: mistralai_ministral-3-14b-reasoning-2512-multimodal - url: github:mudler/LocalAI/gallery/mistral-0.3.yaml@master + url: github:mudler/LocalAI/gallery/virtual.yaml@master urls: - https://huggingface.co/mistralai/Ministral-3-14B-Reasoning-2512 - https://huggingface.co/unsloth/Ministral-3-14B-Reasoning-2512-GGUF @@ -41359,8 +41359,19 @@ - vision last_checked: "2026-05-04" overrides: + backend: llama-cpp context_size: 32768 + mmap: true mmproj: llama-cpp/mmproj/mmproj-mistralai_Ministral-3-14B-Reasoning-2512-f32.gguf + options: + - use_jinja:true + template: + use_tokenizer_template: true + function: + disable_no_action: true + automatic_tool_parsing_fallback: true + grammar: + disable: true parameters: model: llama-cpp/models/mistralai_Ministral-3-14B-Reasoning-2512-Q4_K_M.gguf temperature: 0.7