mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-20 21:28:16 -04:00
fix(gallery): set MiniCPM5 context at the top level
The Q4 and Q8 overrides place context_size inside parameters, where PredictionOptions ignores it. Move it beside parameters so both builds use the intended 8,192-token context, matching F16. Assisted-by: Codex:GPT-6
This commit is contained in:
1 parent
96ccb8e96c
commit
9ac61f271c
2 files changed
+4
-3
No files matched your search
@@ -104,7 +104,8 @@ local-ai models install minicpm5-2b --variant minicpm5-2b-f16
|
||||
```
|
||||
|
||||
The F16 weights require a 5.04 GB download, plus additional memory for inference.
|
||||
This entry uses the embedded chat template and an 8,192-token context.
|
||||
All three builds use the embedded chat template and an 8,192-token default context.
|
||||
To change the context in a model configuration, set `context_size` at the top level, alongside `parameters`.
|
||||
See the [official GGUF repository](https://huggingface.co/openbmb/MiniCPM5-2B-GGUF).
|
||||
|
||||
## VRAM and download size estimates
|
||||
|
||||
+2
-2
@@ -27748,6 +27748,7 @@
|
||||
- model: minicpm5-2b-f16
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 8192
|
||||
known_usecases:
|
||||
- chat
|
||||
options:
|
||||
@@ -27756,7 +27757,6 @@
|
||||
use_tokenizer_template: true
|
||||
parameters:
|
||||
model: llama-cpp/models/minicpm5-2b/MiniCPM5-2B-Q4_K_M.gguf
|
||||
context_size: 8192
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
files:
|
||||
@@ -27772,6 +27772,7 @@
|
||||
variants: null
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 8192
|
||||
known_usecases:
|
||||
- chat
|
||||
options:
|
||||
@@ -27780,7 +27781,6 @@
|
||||
use_tokenizer_template: true
|
||||
parameters:
|
||||
model: llama-cpp/models/minicpm5-2b/MiniCPM5-2B-Q8_0.gguf
|
||||
context_size: 8192
|
||||
temperature: 1.0
|
||||
top_p: 0.95
|
||||
files:
|
||||
|
||||
Reference in new issue
Block a user