fix(gallery): use native Ministral tool parsing (#12022)

The Ministral 3 14B Reasoning entry inherits a Mistral 0.3 prompt and
JSON parser. Its name-first tool calls can therefore reach clients as
plain text.

Use the embedded template and llama.cpp's native tool parser. Document
migration for installed configurations, which gallery updates do not
rewrite.

Assisted-by: Codex:gpt-6

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
This commit is contained in:
localai-org-maint-botandlocalai-org-maint-bot authored and GitHub committed 2026-09-13 19:05:33 +02:00
1 parent d463316caf
commit 9abe2aead5
2 files changed
+41 -2

No files matched your search

+29 -1
View File
@@ -36,6 +36,34 @@ Reasoning content (`<think>...</think>` blocks from DeepSeek R1, Qwen3, Gemma 4,
No configuration required - the autoparser detects the tool call format for any `ggml`/`gguf` model that was trained with tool support.
#### Migrating older Ministral 3 configurations
The `mistralai_ministral-3-14b-reasoning-2512-multimodal` gallery entry uses the model's embedded template and llama.cpp's native tool parser.
Older installations inherit the Mistral 0.3 prompt and JSON parser, which can return tool calls such as `pick_tool{...}` as text.
For an existing installation, replace the model YAML's `template` and `function` sections with:
```yaml
template:
use_tokenizer_template: true
function:
disable_no_action: true
automatic_tool_parsing_fallback: true
grammar:
disable: true
```
Set `use_jinja:true` in the existing `options` list:
```yaml
options:
- use_jinja:true
```
Remove the inherited `stopwords` list so llama.cpp controls the model's end-of-turn markers.
Keep your model path, `mmproj`, sampling settings, and MCP configuration. Reload the model after saving the YAML.
Gallery changes do not rewrite installed model configurations.
### vLLM / vLLM Omni
The parser must be specified explicitly because vLLM itself doesn't auto-detect one. Pass it via the model `options`:
@@ -312,4 +340,4 @@ Grammars and function tools can be used as well in conjunction with vision APIs:
## 💡 Examples
A full e2e example with `docker-compose` is available [here](https://github.com/mudler/LocalAI-examples/tree/main/functions).
A full e2e example with `docker-compose` is available [here](https://github.com/mudler/LocalAI-examples/tree/main/functions).
+12 -1
View File
@@ -41311,7 +41311,7 @@
sha256: 2740ba9e9b30b09be4282a9a9f617ec43dc47b89aed416cb09b5f698f90783b5
uri: huggingface://unsloth/Ministral-3-14B-Instruct-2512-GGUF/mmproj-F32.gguf
- name: mistralai_ministral-3-14b-reasoning-2512-multimodal
url: github:mudler/LocalAI/gallery/mistral-0.3.yaml@master
url: github:mudler/LocalAI/gallery/virtual.yaml@master
urls:
- https://huggingface.co/mistralai/Ministral-3-14B-Reasoning-2512
- https://huggingface.co/unsloth/Ministral-3-14B-Reasoning-2512-GGUF
@@ -41359,8 +41359,19 @@
- vision
last_checked: "2026-05-04"
overrides:
backend: llama-cpp
context_size: 32768
mmap: true
mmproj: llama-cpp/mmproj/mmproj-mistralai_Ministral-3-14B-Reasoning-2512-f32.gguf
options:
- use_jinja:true
template:
use_tokenizer_template: true
function:
disable_no_action: true
automatic_tool_parsing_fallback: true
grammar:
disable: true
parameters:
model: llama-cpp/models/mistralai_Ministral-3-14B-Reasoning-2512-Q4_K_M.gguf
temperature: 0.7