diff --git a/docs/content/features/model-gallery.md b/docs/content/features/model-gallery.md index 56dde2118..e886cb940 100644 --- a/docs/content/features/model-gallery.md +++ b/docs/content/features/model-gallery.md @@ -606,6 +606,29 @@ where: - `bert-embeddings` is the model name in the gallery (read its [config here](https://github.com/mudler/LocalAI/blob/master/gallery/index.yaml)). +### Defiance 27B + +[Defiance](https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF) +combines HauhauCS's Qwen3.8-27B weights with a customized Turbo Brilliance chat +template. The gallery offers IQ4_XS and Q6_K_P GGUF builds for llama.cpp. +Both use the embedded template and a default context of 32,768 tokens. + +Install with automatic variant selection: + +```bash +local-ai models install qwen3.8-27b-defiance-iq4 +``` + +To select the Q6_K_P build explicitly: + +```bash +local-ai models install qwen3.8-27b-defiance-iq4 --variant qwen3.8-27b-defiance-q6 +``` + +These entries support text input and ordinary decoding. They do not install a +vision projector or enable MTP. They use the template embedded in the GGUF, +which predates the publisher's separate template updates. + ### Model variants Some gallery entries offer several builds of the same model: different diff --git a/gallery/index.yaml b/gallery/index.yaml index 981cc3563..151b37609 100644 --- a/gallery/index.yaml +++ b/gallery/index.yaml @@ -3,6 +3,72 @@ # written by scripts/models/cmd/modelcatalog and stays only so releases before # the cutover keep working. CI fails when it differs from the generated copy: # go run ./scripts/models/cmd/modelcatalog --library gallery/models/index.yaml --out gallery/index.yaml +- url: github:mudler/LocalAI/gallery/qwen3.yaml@master + name: qwen3.8-27b-defiance-iq4 + description: | + Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a + customized Turbo Brilliance chat template for reasoning and roleplay. + This IQ4_XS GGUF build uses llama.cpp and the embedded template, + with a 32,768-token context. Text input only; MTP is not enabled. + license: apache-2.0 + urls: + - https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF + - https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF + - https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF + tags: + - llm + - gguf + - cpu + - gpu + - reasoning + - text-generation + files: + - filename: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf + sha256: d48cdb6e627b4e74b862938daf5ac2bfb7b82c5f02627a02a3b4927e7bb35f2f + uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf + overrides: + context_size: 32768 + parameters: + min_p: 0.05 + model: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf + repeat_penalty: 1 + temperature: 0.7 + top_k: 64 + top_p: 0.95 + variants: + - model: qwen3.8-27b-defiance-q6 +- url: github:mudler/LocalAI/gallery/qwen3.yaml@master + name: qwen3.8-27b-defiance-q6 + description: | + Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a + customized Turbo Brilliance chat template for reasoning and roleplay. + This Q6_K_P GGUF build uses llama.cpp and the embedded template, + with a 32,768-token context. Text input only; MTP is not enabled. + license: apache-2.0 + urls: + - https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF + - https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF + - https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF + tags: + - llm + - gguf + - cpu + - gpu + - reasoning + - text-generation + files: + - filename: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf + sha256: 0ce1abf7e488a5efbd4408af763f18e21bdc220c93106f658d95607dc97868a3 + uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf + overrides: + context_size: 32768 + parameters: + min_p: 0.05 + model: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf + repeat_penalty: 1 + temperature: 0.7 + top_k: 64 + top_p: 0.95 - url: github:mudler/LocalAI/gallery/virtual.yaml@master name: adavar-3b-q4 description: | diff --git a/gallery/models/index.yaml b/gallery/models/index.yaml index da124ab3a..faa5c7019 100644 --- a/gallery/models/index.yaml +++ b/gallery/models/index.yaml @@ -4,6 +4,88 @@ # changes and never edit it by hand. CI fails when it is stale. A build from # master reads it as the default model source; a tagged build reads the gallery # until the release cutover. +- kind: model + id: qwen3.8-27b-defiance-iq4 + name: qwen3.8-27b-defiance-iq4 + version: 0.0.0-gallery.9833dd29edab + description: | + Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a + customized Turbo Brilliance chat template for reasoning and roleplay. + This IQ4_XS GGUF build uses llama.cpp and the embedded template, + with a 32,768-token context. Text input only; MTP is not enabled. + license: apache-2.0 + urls: + - https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF + - https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF + - https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF + tags: + - llm + - gguf + - cpu + - gpu + - reasoning + - text-generation + channel: gallery + policy: + requireSignature: false + schemaVersion: 1 + model: + url: github:mudler/LocalAI/gallery/qwen3.yaml@master + files: + - filename: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf + sha256: d48cdb6e627b4e74b862938daf5ac2bfb7b82c5f02627a02a3b4927e7bb35f2f + uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf + overrides: + context_size: 32768 + parameters: + min_p: 0.05 + model: Qwen3.8-27B-Defiance-Uncensored-IQ4_XS.gguf + repeat_penalty: 1 + temperature: 0.7 + top_k: 64 + top_p: 0.95 + variants: + - model: qwen3.8-27b-defiance-q6 +- kind: model + id: qwen3.8-27b-defiance-q6 + name: qwen3.8-27b-defiance-q6 + version: 0.0.0-gallery.34421ee644a5 + description: | + Defiance combines HauhauCS's uncensored Qwen3.8-27B weights with a + customized Turbo Brilliance chat template for reasoning and roleplay. + This Q6_K_P GGUF build uses llama.cpp and the embedded template, + with a 32,768-token context. Text input only; MTP is not enabled. + license: apache-2.0 + urls: + - https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF + - https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF + - https://huggingface.co/DavidAU/Qwen3.8-27B-Turbo-Brilliance-50X-Reasoning-Instruct-NEO-MAX-MTP-GGUF + tags: + - llm + - gguf + - cpu + - gpu + - reasoning + - text-generation + channel: gallery + policy: + requireSignature: false + schemaVersion: 1 + model: + url: github:mudler/LocalAI/gallery/qwen3.yaml@master + files: + - filename: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf + sha256: 0ce1abf7e488a5efbd4408af763f18e21bdc220c93106f658d95607dc97868a3 + uri: https://huggingface.co/ttindall/Qwen3.8-27B-Defiance-Uncensored-MTP-GGUF/resolve/1e664931451831589ce16389a55e0d5605dd4bc5/Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf + overrides: + context_size: 32768 + parameters: + min_p: 0.05 + model: Qwen3.8-27B-Defiance-Uncensored-Q6_K_P.gguf + repeat_penalty: 1 + temperature: 0.7 + top_k: 64 + top_p: 0.95 - kind: model id: adavar-3b-q4 name: adavar-3b-q4