Compare commits

...
Author SHA1 Message Date
localai-org-maint-bot 97f0375ae8 feat(gallery): add Huihui Qwen3.8 Flash Next
Add the published UD-Q4_K_XL shards and BF16 vision projector.
Document installation and the default context size.

Assisted-by: Codex:gpt-6
2026-09-08 00:06:00 +00:00
2 changed files with 76 additions and 0 deletions

No files matched your search

+18
View File
@@ -536,6 +536,24 @@ curl $LOCALAI/models/apply -H "Content-Type: application/json" -d '{
## Examples
### Huihui Qwen3.8 Flash Next
Install the abliterated Qwen3.8-Flash-Next build for text chat and image input:
```bash
local-ai models install huihui-qwen3.8-flash-next-abliterated-q4
```
The entry downloads the publisher's four UD-Q4_K_XL GGUF shards and BF16
vision projector, approximately 112 GB in total. It uses llama.cpp with the
embedded chat template and a 32,768-token default context. The publisher
currently provides one quantization, so this entry has no alternative variants.
This model is modified to reduce refusals. See the
[model card](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-GGUF)
and [Qwen Community License 1.0](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-GGUF/blob/main/LICENSE)
for model details and usage terms.
### Embeddings: Bert
<details>
+58
View File
@@ -1014,6 +1014,64 @@
- filename: llama-cpp/models/llm-jp-4-33b-thinking/llm-jp-4-33b-thinking-BF16.gguf
uri: huggingface://llm-jp/llm-jp-4-33b-thinking-gguf/llm-jp-4-33b-thinking-BF16.gguf
sha256: 7bb8465702b5c4a5d94e03e62921d5917181a2eda56da31edfdcd47fab2e9964
- name: "huihui-qwen3.8-flash-next-abliterated-q4"
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
urls:
- https://huggingface.co/Qwen/Qwen3.8-Flash-Next
- https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-GGUF
description: |
Huihui's abliterated Qwen3.8-Flash-Next is a vision-language
mixture-of-experts model modified to reduce refusals. This entry uses
the publisher's UD-Q4_K_XL GGUF and BF16 vision projector for text chat
and image input through llama.cpp. The default context is 32,768 tokens.
Model weights use the Qwen Community License 1.0.
license: other
tags:
- llm
- gguf
- cpu
- gpu
- qwen
- moe
- vision
- multimodal
- abliterated
icon: https://qianwen-res.oss-cn-beijing.aliyuncs.com/logo_qwen.jpg
overrides:
backend: llama-cpp
context_size: 32768
known_usecases:
- chat
- vision
mmproj: llama-cpp/mmproj/huihui-qwen3.8-flash-next-abliterated-q4/mmproj-model-bf16.gguf
options:
- use_jinja:true
parameters:
model: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
temperature: 1
top_k: 20
top_p: 0.95
min_p: 0
repeat_penalty: 1
presence_penalty: 0
template:
use_tokenizer_template: true
files:
- filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
uri: huggingface://huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-GGUF/UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
sha256: 290a31ca20b74ed2c21b257d378a5678d821ad9fdd8feb1fc52c719761668903
- filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00002-of-00004.gguf
uri: huggingface://huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-GGUF/UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00002-of-00004.gguf
sha256: 0ebf70d076e0e98f74dc414a90b92c67d7267876ae524bc7799cf463e27b754b
- filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00003-of-00004.gguf
uri: huggingface://huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-GGUF/UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00003-of-00004.gguf
sha256: e9b12f44e13e60d6ef9396449f7b4356b2e2911222b8183a596830c2418db19a
- filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00004-of-00004.gguf
uri: huggingface://huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-GGUF/UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00004-of-00004.gguf
sha256: 983901b3e47686f56f5fcec87ba8c51e7e2c6996277dd77bc3fc804a1d563da3
- filename: llama-cpp/mmproj/huihui-qwen3.8-flash-next-abliterated-q4/mmproj-model-bf16.gguf
uri: huggingface://huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-GGUF/mmproj-model-bf16.gguf
sha256: dbac7334a8a433abd6cd1d1db82398ba123faa2f328696da20848e605e645ca1
- &qwen3-8-flash-next
name: "qwen3.8-flash-next-q4"
variants: