mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 00:59:28 -04:00
feat(gallery): add Defiance 27B variants
Add IQ4_XS and Q6_K_P builds with pinned weights and linked variants.
Document installation and the text-only, ordinary-decoding defaults.
Assisted-by: Codex:gpt-6
(cherry picked from commit 1c5eea9580, mudler/LocalAI#12819)
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
localai-org-maint-bot authored and Ettore Di Giacinto committed
1 parent
e2925e20ef
commit
5d5df48169
2 files changed
+148
No files matched your search
@@ -3,6 +3,72 @@
|
||||
# written by scripts/models/cmd/modelcatalog and stays only so releases before
|
||||
# the cutover keep working. CI fails when it differs from the generated copy:
|
||||
# go run ./scripts/models/cmd/modelcatalog --library gallery/models/index.yaml --out gallery/index.yaml
|
||||
- url: github:mudler/LocalAI/gallery/qwen3.yaml@master
|
||||
name: qwen3.8-27b-defiance-iq4
|
||||
description: |
|
||||
Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a
|
||||
customized Turbo Brilliance chat template for reasoning and roleplay.
|
||||
This IQ4_XS GGUF build uses llama.cpp and the embedded template,
|
||||
with a 32,768-token context. Text input only; MTP is not enabled.
|
||||
license: apache-2.0
|
||||
urls:
|
||||
- https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF
|
||||
- https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF
|
||||
- https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
- reasoning
|
||||
- text-generation
|
||||
files:
|
||||
- filename: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
|
||||
sha256: d48cdb6e627b4e74b862938daf5ac2bfb7b82c5f02627a02a3b4927e7bb35f2f
|
||||
uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
|
||||
overrides:
|
||||
context_size: 32768
|
||||
parameters:
|
||||
min_p: 0.05
|
||||
model: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
|
||||
repeat_penalty: 1
|
||||
temperature: 0.7
|
||||
top_k: 64
|
||||
top_p: 0.95
|
||||
variants:
|
||||
- model: qwen3.8-27b-defiance-q6
|
||||
- url: github:mudler/LocalAI/gallery/qwen3.yaml@master
|
||||
name: qwen3.8-27b-defiance-q6
|
||||
description: |
|
||||
Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a
|
||||
customized Turbo Brilliance chat template for reasoning and roleplay.
|
||||
This Q6_K_P GGUF build uses llama.cpp and the embedded template,
|
||||
with a 32,768-token context. Text input only; MTP is not enabled.
|
||||
license: apache-2.0
|
||||
urls:
|
||||
- https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF
|
||||
- https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF
|
||||
- https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
- reasoning
|
||||
- text-generation
|
||||
files:
|
||||
- filename: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
|
||||
sha256: 0ce1abf7e488a5efbd4408af763f18e21bdc220c93106f658d95607dc97868a3
|
||||
uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
|
||||
overrides:
|
||||
context_size: 32768
|
||||
parameters:
|
||||
min_p: 0.05
|
||||
model: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
|
||||
repeat_penalty: 1
|
||||
temperature: 0.7
|
||||
top_k: 64
|
||||
top_p: 0.95
|
||||
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
name: ornith-1.5-122b-ud-q4-k-xl
|
||||
description: |
|
||||
|
||||
@@ -4,6 +4,88 @@
|
||||
# changes and never edit it by hand. CI fails when it is stale. A build from
|
||||
# master reads it as the default model source; a tagged build reads the gallery
|
||||
# until the release cutover.
|
||||
- kind: model
|
||||
id: qwen3.8-27b-defiance-iq4
|
||||
name: qwen3.8-27b-defiance-iq4
|
||||
version: 0.0.0-gallery.9833dd29edab
|
||||
description: |
|
||||
Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a
|
||||
customized Turbo Brilliance chat template for reasoning and roleplay.
|
||||
This IQ4_XS GGUF build uses llama.cpp and the embedded template,
|
||||
with a 32,768-token context. Text input only; MTP is not enabled.
|
||||
license: apache-2.0
|
||||
urls:
|
||||
- https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF
|
||||
- https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF
|
||||
- https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
- reasoning
|
||||
- text-generation
|
||||
channel: gallery
|
||||
policy:
|
||||
requireSignature: false
|
||||
schemaVersion: 1
|
||||
model:
|
||||
url: github:mudler/LocalAI/gallery/qwen3.yaml@master
|
||||
files:
|
||||
- filename: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
|
||||
sha256: d48cdb6e627b4e74b862938daf5ac2bfb7b82c5f02627a02a3b4927e7bb35f2f
|
||||
uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
|
||||
overrides:
|
||||
context_size: 32768
|
||||
parameters:
|
||||
min_p: 0.05
|
||||
model: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf
|
||||
repeat_penalty: 1
|
||||
temperature: 0.7
|
||||
top_k: 64
|
||||
top_p: 0.95
|
||||
variants:
|
||||
- model: qwen3.8-27b-defiance-q6
|
||||
- kind: model
|
||||
id: qwen3.8-27b-defiance-q6
|
||||
name: qwen3.8-27b-defiance-q6
|
||||
version: 0.0.0-gallery.34421ee644a5
|
||||
description: |
|
||||
Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a
|
||||
customized Turbo Brilliance chat template for reasoning and roleplay.
|
||||
This Q6_K_P GGUF build uses llama.cpp and the embedded template,
|
||||
with a 32,768-token context. Text input only; MTP is not enabled.
|
||||
license: apache-2.0
|
||||
urls:
|
||||
- https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF
|
||||
- https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF
|
||||
- https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
- reasoning
|
||||
- text-generation
|
||||
channel: gallery
|
||||
policy:
|
||||
requireSignature: false
|
||||
schemaVersion: 1
|
||||
model:
|
||||
url: github:mudler/LocalAI/gallery/qwen3.yaml@master
|
||||
files:
|
||||
- filename: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
|
||||
sha256: 0ce1abf7e488a5efbd4408af763f18e21bdc220c93106f658d95607dc97868a3
|
||||
uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
|
||||
overrides:
|
||||
context_size: 32768
|
||||
parameters:
|
||||
min_p: 0.05
|
||||
model: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf
|
||||
repeat_penalty: 1
|
||||
temperature: 0.7
|
||||
top_k: 64
|
||||
top_p: 0.95
|
||||
- kind: model
|
||||
id: ornith-1.5-122b-ud-q4-k-xl
|
||||
name: ornith-1.5-122b-ud-q4-k-xl
|
||||
|
||||
Reference in new issue
Block a user