mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 09:08:54 -04:00
mudler-agentandEttore Di Giacinto
855958960f
The Qwen3 template nests context_size, f16, and mmap under parameters, where the typed model configuration ignores them. Move these options to the root and correct three model-specific context overrides. Test the real gallery data and generated qwen3-vl-4b-thinking YAML. Document manual repair because template updates do not rewrite installed configurations. Fixes #12528 Assisted-by: nib:gpt-6-astra Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
29 lines
987 B
YAML
29 lines
987 B
YAML
config_file: |
|
|
backend: llama-cpp
|
|
known_usecases:
|
|
- chat
|
|
context_size: 8192
|
|
f16: true
|
|
mmap: true
|
|
stopwords:
|
|
- <|im_end|>
|
|
- <dummy32000>
|
|
- </s>
|
|
- <|endoftext|>
|
|
# Delegate templating to llama.cpp's jinja runtime so the C++ autoparser
|
|
# can classify <think>…</think> blocks into reasoning_content natively
|
|
# (issue #9985). Without use_jinja the autoparser falls back to a
|
|
# "pure content" PEG parser that leaks reasoning tags into content.
|
|
options:
|
|
- use_jinja:true
|
|
# With use_tokenizer_template the backend (llama.cpp) owns tool-call
|
|
# grammar generation and parsing too. Disabling LocalAI's own grammar lets
|
|
# llama.cpp's native name-first tool pipeline run; otherwise the generated
|
|
# grammar overrides it and the tool-call JSON leaks into content (#10052).
|
|
function:
|
|
grammar:
|
|
disable: true
|
|
template:
|
|
use_tokenizer_template: true
|
|
name: qwen3
|