mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-31 18:49:22 -04:00
chore(gallery): add Mellum2.1 Thinking variants
Add JetBrains Q4_K_M and Q8_0 GGUF builds for llama.cpp, with variant selection and installation documentation. Pin both downloads and verify their SHA256 values against LFS metadata and HTTP headers. Assisted-by: Codex:gpt-6
This commit is contained in:
1 parent
6343a2dc6f
commit
a5accc7488
2 files changed
+100
No files matched your search
@@ -902,3 +902,27 @@ is still running is reported as queued until the installer picks it up:
|
||||
A job ID is queryable from the moment `/models/apply` returns it, so a `404`/`500`
|
||||
from this endpoint means the ID is genuinely unknown rather than merely waiting
|
||||
its turn.
|
||||
|
||||
### Mellum2.1 Thinking
|
||||
|
||||
[Mellum2.1 Thinking](https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking)
|
||||
is a JetBrains reasoning model with 12 billion total parameters and 2.5 billion
|
||||
active parameters per token. The gallery provides Q4_K_M and Q8_0 GGUF builds
|
||||
for the llama-cpp backend.
|
||||
|
||||
Install the default entry with automatic variant selection:
|
||||
|
||||
```bash
|
||||
local-ai models install mellum2.1-12b-a2.5b-thinking
|
||||
```
|
||||
|
||||
To select Q8_0 explicitly:
|
||||
|
||||
```bash
|
||||
local-ai models install mellum2.1-12b-a2.5b-thinking --variant mellum2.1-12b-a2.5b-thinking-q8
|
||||
```
|
||||
|
||||
Both entries use the embedded chat template, temperature 0.6, top-p 0.95,
|
||||
and top-k 20. The default context is 32,768 tokens; the model supports up to
|
||||
131,072 tokens. Increasing the context requires more memory. The model emits
|
||||
reasoning inside `<think>...</think>` blocks before its final answer.
|
||||
@@ -8766,6 +8766,82 @@
|
||||
- filename: llama-cpp/models/FINAL-Bench/POCKET-26B-GGUF/POCKET-26B-Q2_K.gguf
|
||||
sha256: 490687be6493769dc8322df396cd31d412a15877eaccea4d2f16ca1f93383334
|
||||
uri: huggingface://FINAL-Bench/POCKET-26B-GGUF/POCKET-26B-Q2_K.gguf
|
||||
- name: mellum2.1-12b-a2.5b-thinking
|
||||
variants:
|
||||
- model: mellum2.1-12b-a2.5b-thinking-q8
|
||||
url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
urls:
|
||||
- https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking
|
||||
- https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF
|
||||
description: |
|
||||
Mellum2.1 Thinking is a JetBrains mixture-of-experts reasoning model with
|
||||
12 billion total parameters and 2.5 billion active parameters per token.
|
||||
It emits reasoning in think blocks and supports a 131,072-token context.
|
||||
This entry uses Q4_K_M GGUF weights with a 32,768-token default context.
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
- moe
|
||||
- reasoning
|
||||
last_checked: "2026-10-08"
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 32768
|
||||
known_usecases:
|
||||
- chat
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: llama-cpp/models/JetBrains/Mellum2.1-12B-A2.5B-Thinking/Mellum2.1-12B-A2.5B-Thinking-Q4_K_M.gguf
|
||||
temperature: 0.6
|
||||
top_p: 0.95
|
||||
top_k: 20
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
files:
|
||||
- filename: llama-cpp/models/JetBrains/Mellum2.1-12B-A2.5B-Thinking/Mellum2.1-12B-A2.5B-Thinking-Q4_K_M.gguf
|
||||
sha256: ecc4d5b8107fc219e23c494b2d107552137637b6c6002ee713e0ba8af7fa2755
|
||||
uri: https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF/resolve/20b439426f1e997ba575eaccaf4856ac0f76f5d5/Mellum2.1-12B-A2.5B-Thinking-Q4_K_M.gguf
|
||||
- name: mellum2.1-12b-a2.5b-thinking-q8
|
||||
url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
urls:
|
||||
- https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking
|
||||
- https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF
|
||||
description: |
|
||||
Mellum2.1 Thinking is a JetBrains mixture-of-experts reasoning model with
|
||||
12 billion total parameters and 2.5 billion active parameters per token.
|
||||
It emits reasoning in think blocks and supports a 131,072-token context.
|
||||
This entry uses Q8_0 GGUF weights with a 32,768-token default context.
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
- moe
|
||||
- reasoning
|
||||
last_checked: "2026-10-08"
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 32768
|
||||
known_usecases:
|
||||
- chat
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: llama-cpp/models/JetBrains/Mellum2.1-12B-A2.5B-Thinking/Mellum2.1-12B-A2.5B-Thinking-Q8_0.gguf
|
||||
temperature: 0.6
|
||||
top_p: 0.95
|
||||
top_k: 20
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
files:
|
||||
- filename: llama-cpp/models/JetBrains/Mellum2.1-12B-A2.5B-Thinking/Mellum2.1-12B-A2.5B-Thinking-Q8_0.gguf
|
||||
sha256: 69e0ac02a16586c2c9e991f1e3b8a1eb403882e7ab9e69ea01eb23dbbbc6841f
|
||||
uri: https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF/resolve/20b439426f1e997ba575eaccaf4856ac0f76f5d5/Mellum2.1-12B-A2.5B-Thinking-Q8_0.gguf
|
||||
- &mellum2-12b-a2-5b-instruct
|
||||
name: mellum2-12b-a2.5b-instruct
|
||||
variants:
|
||||
|
||||
Reference in new issue
Block a user