mirror/LocalAI
1
0 Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-10-28 00:59:28 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(gallery): add Saluki abliterated variants

Browse Source 
Add Kujira's mixed IQ2 Saluki fine-tune for llama.cpp with plain and MTP
decoding. Pin the weights and document the text-only entries.

Assisted-by: Codex:gpt-6
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

(cherry picked from commit 24f23a66aa, mudler/LocalAI#12688)
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
localai-org-maint-bot authored and Ettore Di Giacinto committed 2026-10-25 09:55:39 +00:00
1 parent 1ae638dee2
commit d62ea54aa3
2 files changed
+190
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

gallery/index.yaml
+87
View File
@@ -3,6 +3,93 @@
# written by scripts/models/cmd/modelcatalog and stays only so releases before
# the cutover keep working. CI fails when it differs from the generated copy:
# go run ./scripts/models/cmd/modelcatalog --library gallery/models/index.yaml --out gallery/index.yaml
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: underdog-saluki-27b-1.0-abliterated
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
uses ordinary decoding; the MTP variant enables speculative decoding.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
variants:
- model: underdog-saluki-27b-1.0-abliterated-mtp
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: underdog-saluki-27b-1.0-abliterated-mtp
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
enables multi-token prediction with up to two draft tokens.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
- mtp
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
- spec_type:draft-mtp
- spec_n_max:2
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: huihui-qwen3.8-flash-next-abliterated-v2-q4
description: |
gallery/models/index.yaml
+103
View File
@@ -4,6 +4,109 @@
# changes and never edit it by hand. CI fails when it is stale. A build from
# master reads it as the default model source; a tagged build reads the gallery
# until the release cutover.
- kind: model
id: underdog-saluki-27b-1.0-abliterated
name: underdog-saluki-27b-1.0-abliterated
version: 0.0.0-gallery.cdab61f388f4
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
uses ordinary decoding; the MTP variant enables speculative decoding.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
variants:
- model: underdog-saluki-27b-1.0-abliterated-mtp
- kind: model
id: underdog-saluki-27b-1.0-abliterated-mtp
name: underdog-saluki-27b-1.0-abliterated-mtp
version: 0.0.0-gallery.774b2b47f0dc
description: |
Kujira's abliterated adaptation of Underdog Saluki 27B 1.0 uses mixed
IQ2 weights with an embedded MTP head. This text-only llama.cpp build
enables multi-token prediction with up to two draft tokens.
license: apache-2.0
urls:
- https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
- https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- reasoning
- function-calling
- uncensored
- mtp
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
sha256: b9b231748c94e13ecc0f841942798eb36b27eaea60014fea896218bc86742bbc
uri: https://huggingface.co/Kujira/Underdog-Saluki-27B-1.0-MTP-Abliterated-GGUF/resolve/c1d826d73831efcb819d9d64c625f0fab56ae8fc/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
overrides:
backend: llama-cpp
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
options:
- use_jinja:true
- spec_type:draft-mtp
- spec_n_max:2
parameters:
model: llama-cpp/models/underdog-saluki-27b-1.0-abliterated/Underdog-Saluki-27B-1.0-IQ2-mix-MTP-abliterated.gguf
presence_penalty: 1.1
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- kind: model
id: huihui-qwen3.8-flash-next-abliterated-v2-q4
name: huihui-qwen3.8-flash-next-abliterated-v2-q4