feat(gallery): add Saluki abliterated variants

Add Kujira's mixed IQ2 Saluki fine-tune for llama.cpp with plain and MTP
decoding. Pin the weights and document the text-only entries.

Assisted-by: Codex:gpt-6
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
localai-org-maint-bot committed 2026-10-23 12:03:56 +00:00
1 parent 0a44303c21
commit 24f23a66aa
3 files changed
+205

No files matched your search

+15
View File
@@ -61,6 +61,21 @@ Select a specific build with `--variant minicpm-v-4_7-1b-q4`,
`--variant minicpm-v-4_7-1b-q5`, `--variant minicpm-v-4_7-1b-q6`,
or `--variant minicpm-v-4_7-1b-q8`.
### Underdog Saluki abliterated builds
`underdog-saluki-27b-1.0-abliterated` provides Kujira's mixed IQ2 adaptation
of Underdog Saluki for text chat, reasoning, and tool use. Its MTP variant
enables multi-token prediction with up to two draft tokens. Both builds use
the same weights, the embedded chat template, and a 32,768-token context.
To select the MTP build explicitly:
```bash
local-ai models install --variant underdog-saluki-27b-1.0-abliterated-mtp underdog-saluki-27b-1.0-abliterated
```
These entries are text-only and do not include a vision projector.
### Repairing an existing model configuration
Gallery template updates do not rewrite YAML files for already installed models.
+87
View File
@@ -3,6 +3,93 @@
# written by scripts/models/cmd/modelcatalog and stays only so releases before
# the cutover keep working. CI fails when it differs from the generated copy:
# go run ./scripts/models/cmd/modelcatalog --library gallery/models/index.yaml --out gallery/index.yaml
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: underdog-saluki-27b-1.0-abliterated
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
uses ordinary decoding; the MTP variant enables speculative decoding.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
variants:
- model: underdog-saluki-27b-1.0-abliterated-mtp
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: underdog-saluki-27b-1.0-abliterated-mtp
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
enables multi-token prediction with up to two draft tokens.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
- mtp
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
- spec_type:draft-mtp
- spec_n_max:2
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: adavar-3b-q4
description: |
+103
View File
@@ -4,6 +4,109 @@
# changes and never edit it by hand. CI fails when it is stale. A build from
# master reads it as the default model source; a tagged build reads the gallery
# until the release cutover.
- kind: model
id: underdog-saluki-27b-1.0-abliterated
name: underdog-saluki-27b-1.0-abliterated
version: 0.0.0-gallery.cdab61f388f4
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
uses ordinary decoding; the MTP variant enables speculative decoding.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
variants:
- model: underdog-saluki-27b-1.0-abliterated-mtp
- kind: model
id: underdog-saluki-27b-1.0-abliterated-mtp
name: underdog-saluki-27b-1.0-abliterated-mtp
version: 0.0.0-gallery.774b2b47f0dc
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
enables multi-token prediction with up to two draft tokens.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
- mtp
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
- spec_type:draft-mtp
- spec_n_max:2
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- kind: model
id: adavar-3b-q4
name: adavar-3b-q4