mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-14 15:18:03 -04:00
fix(gallery): use native Ministral tool parsing (#12022)
The Ministral 3 14B Reasoning entry inherits a Mistral 0.3 prompt and JSON parser. Its name-first tool calls can therefore reach clients as plain text. Use the embedded template and llama.cpp's native tool parser. Document migration for installed configurations, which gallery updates do not rewrite. Assisted-by: Codex:gpt-6 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
This commit is contained in:
1 parent
d463316caf
commit
9abe2aead5
2 files changed
+41
-2
No files matched your search
@@ -36,6 +36,34 @@ Reasoning content (`<think>...</think>` blocks from DeepSeek R1, Qwen3, Gemma 4,
|
||||
|
||||
No configuration required - the autoparser detects the tool call format for any `ggml`/`gguf` model that was trained with tool support.
|
||||
|
||||
#### Migrating older Ministral 3 configurations
|
||||
|
||||
The `mistralai_ministral-3-14b-reasoning-2512-multimodal` gallery entry uses the model's embedded template and llama.cpp's native tool parser.
|
||||
Older installations inherit the Mistral 0.3 prompt and JSON parser, which can return tool calls such as `pick_tool{...}` as text.
|
||||
|
||||
For an existing installation, replace the model YAML's `template` and `function` sections with:
|
||||
|
||||
```yaml
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
function:
|
||||
disable_no_action: true
|
||||
automatic_tool_parsing_fallback: true
|
||||
grammar:
|
||||
disable: true
|
||||
```
|
||||
|
||||
Set `use_jinja:true` in the existing `options` list:
|
||||
|
||||
```yaml
|
||||
options:
|
||||
- use_jinja:true
|
||||
```
|
||||
|
||||
Remove the inherited `stopwords` list so llama.cpp controls the model's end-of-turn markers.
|
||||
Keep your model path, `mmproj`, sampling settings, and MCP configuration. Reload the model after saving the YAML.
|
||||
Gallery changes do not rewrite installed model configurations.
|
||||
|
||||
### vLLM / vLLM Omni
|
||||
|
||||
The parser must be specified explicitly because vLLM itself doesn't auto-detect one. Pass it via the model `options`:
|
||||
@@ -312,4 +340,4 @@ Grammars and function tools can be used as well in conjunction with vision APIs:
|
||||
|
||||
## 💡 Examples
|
||||
|
||||
A full e2e example with `docker-compose` is available [here](https://github.com/mudler/LocalAI-examples/tree/main/functions).
|
||||
A full e2e example with `docker-compose` is available [here](https://github.com/mudler/LocalAI-examples/tree/main/functions).
|
||||
+12
-1
@@ -41311,7 +41311,7 @@
|
||||
sha256: 2740ba9e9b30b09be4282a9a9f617ec43dc47b89aed416cb09b5f698f90783b5
|
||||
uri: huggingface://unsloth/Ministral-3-14B-Instruct-2512-GGUF/mmproj-F32.gguf
|
||||
- name: mistralai_ministral-3-14b-reasoning-2512-multimodal
|
||||
url: github:mudler/LocalAI/gallery/mistral-0.3.yaml@master
|
||||
url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
urls:
|
||||
- https://huggingface.co/mistralai/Ministral-3-14B-Reasoning-2512
|
||||
- https://huggingface.co/unsloth/Ministral-3-14B-Reasoning-2512-GGUF
|
||||
@@ -41359,8 +41359,19 @@
|
||||
- vision
|
||||
last_checked: "2026-05-04"
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 32768
|
||||
mmap: true
|
||||
mmproj: llama-cpp/mmproj/mmproj-mistralai_Ministral-3-14B-Reasoning-2512-f32.gguf
|
||||
options:
|
||||
- use_jinja:true
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
function:
|
||||
disable_no_action: true
|
||||
automatic_tool_parsing_fallback: true
|
||||
grammar:
|
||||
disable: true
|
||||
parameters:
|
||||
model: llama-cpp/models/mistralai_Ministral-3-14B-Reasoning-2512-Q4_K_M.gguf
|
||||
temperature: 0.7
|
||||
|
||||
Reference in new issue
Block a user