chore(gallery): add Huihui Flash Next v2 variants

Add Q4 and IQ4 GGUF builds with a BF16 vision projector and pinned Hugging Face downloads. Keep the original release available. Assisted-by: Codex:gpt-6 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
2026-10-28 09:08:54 -04:00 · 2026-10-23 12:04:00 +00:00
1 parent 0a44303c21
commit 9c9e18450e
@@ -61,6 +61,18 @@ Select a specific build with `--variant minicpm-v-4_7-1b-q4`,
 `--variant minicpm-v-4_7-1b-q5`, `--variant minicpm-v-4_7-1b-q6`,
 or `--variant minicpm-v-4_7-1b-q8`.

+### Huihui Qwen3.8-Flash-Next v2
+
+Install `huihui-qwen3.8-flash-next-abliterated-v2-q4` for the publisher's
+UD-Q4_K_XL build, or select the linked
+`huihui-qwen3.8-flash-next-abliterated-v2-iq4` variant for UD-IQ4_XS.
+Both entries use llama.cpp, include a BF16 vision projector, and default to
+32,768 context tokens. They support text chat and image input.
+
+These entries provide Huihui's second abliteration release and keep the first
+release available separately. See the [publisher's model card](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF)
+for the changes and the Qwen Community License 1.0 terms.
+
 ### Repairing an existing model configuration

 Gallery template updates do not rewrite YAML files for already installed models.
@@ -3,6 +3,123 @@
 # written by scripts/models/cmd/modelcatalog and stays only so releases before
 # the cutover keep working. CI fails when it differs from the generated copy:
 #   go run ./scripts/models/cmd/modelcatalog --library gallery/models/index.yaml --out gallery/index.yaml
+- url: github:mudler/LocalAI/gallery/virtual.yaml@master
+  name: huihui-qwen3.8-flash-next-abliterated-v2-q4
+  description: |
+    Huihui's second abliterated Qwen3.8-Flash-Next release adjusts the
+    abliteration without modifying the experts. This vision-language
+    mixture-of-experts model uses the publisher's UD-Q4_K_XL GGUF
+    and BF16 projector for text chat and image input through llama.cpp.
+    The default context is 32,768 tokens. Model weights use the
+    Qwen Community License 1.0.
+  license: other
+  urls:
+    - https://huggingface.co/Qwen/Qwen3.8-Flash-Next
+    - https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF
+  icon: https://qianwen-res.oss-cn-beijing.aliyuncs.com/logo_qwen.jpg
+  tags:
+    - llm
+    - gguf
+    - cpu
+    - gpu
+    - qwen
+    - moe
+    - vision
+    - multimodal
+    - abliterated
+  files:
+    - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
+      sha256: 4448186216b3af4cc558bbce2c3213f01608f8f8b2e5267a9767971dd3ec8082
+      uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
+    - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00002-of-00004.gguf
+      sha256: 6f3e21a5ae10c9124c2405061d89f90b575a0704b2927f059621b7936850b276
+      uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00002-of-00004.gguf
+    - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00003-of-00004.gguf
+      sha256: bb24c1d9c42c129ad93d427d3be6ca9a8d5ca6bf07ca9a50ddca03328d08c702
+      uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00003-of-00004.gguf
+    - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00004-of-00004.gguf
+      sha256: 3d6a9aee4ade5740394da9bcb9fa22ea9343471f246af6914f8fcc8327ceab8e
+      uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00004-of-00004.gguf
+    - filename: llama-cpp/mmproj/huihui-qwen3.8-flash-next-abliterated-v2/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+      sha256: b1a82259702816a5330d7bd7607cd9676b11780e79ff7348c21103ff3ce49bd0
+      uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+  overrides:
+    backend: llama-cpp
+    context_size: 32768
+    known_usecases:
+      - chat
+      - vision
+    mmproj: llama-cpp/mmproj/huihui-qwen3.8-flash-next-abliterated-v2/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+    options:
+      - use_jinja:true
+    parameters:
+      min_p: 0
+      model: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
+      presence_penalty: 0
+      repeat_penalty: 1
+      temperature: 1
+      top_k: 20
+      top_p: 0.95
+    template:
+      use_tokenizer_template: true
+  variants:
+    - model: huihui-qwen3.8-flash-next-abliterated-v2-iq4
+- url: github:mudler/LocalAI/gallery/virtual.yaml@master
+  name: huihui-qwen3.8-flash-next-abliterated-v2-iq4
+  description: |
+    Huihui's second abliterated Qwen3.8-Flash-Next release adjusts the
+    abliteration without modifying the experts. This vision-language
+    mixture-of-experts model uses the publisher's UD-IQ4_XS GGUF
+    and BF16 projector for text chat and image input through llama.cpp.
+    The default context is 32,768 tokens. Model weights use the
+    Qwen Community License 1.0.
+  license: other
+  urls:
+    - https://huggingface.co/Qwen/Qwen3.8-Flash-Next
+    - https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF
+  icon: https://qianwen-res.oss-cn-beijing.aliyuncs.com/logo_qwen.jpg
+  tags:
+    - llm
+    - gguf
+    - cpu
+    - gpu
+    - qwen
+    - moe
+    - vision
+    - multimodal
+    - abliterated
+  files:
+    - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-iq4/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf
+      sha256: 5ce89370720f8bf90890f439361282104c1aa1482d4013bb9a50923e758e71a4
+      uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf
+    - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-iq4/Qwen3.8-Flash-Next-UD-IQ4_XS-00002-of-00003.gguf
+      sha256: ede87d3bd3e354d43f0e710b2df6791d3479e396847e53adbe2b48be7ba6d663
+      uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00002-of-00003.gguf
+    - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-iq4/Qwen3.8-Flash-Next-UD-IQ4_XS-00003-of-00003.gguf
+      sha256: f8aa8bd7ba134b3b7fe4fc95b9131fc8f6678ace374a0fa6e632225031665995
+      uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00003-of-00003.gguf
+    - filename: llama-cpp/mmproj/huihui-qwen3.8-flash-next-abliterated-v2/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+      sha256: b1a82259702816a5330d7bd7607cd9676b11780e79ff7348c21103ff3ce49bd0
+      uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+  overrides:
+    backend: llama-cpp
+    context_size: 32768
+    known_usecases:
+      - chat
+      - vision
+    mmproj: llama-cpp/mmproj/huihui-qwen3.8-flash-next-abliterated-v2/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+    options:
+      - use_jinja:true
+    parameters:
+      min_p: 0
+      model: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-iq4/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf
+      presence_penalty: 0
+      repeat_penalty: 1
+      temperature: 1
+      top_k: 20
+      top_p: 0.95
+    template:
+      use_tokenizer_template: true
 - url: github:mudler/LocalAI/gallery/virtual.yaml@master
  name: adavar-3b-q4
  description: |
@@ -4,6 +4,139 @@
 # changes and never edit it by hand. CI fails when it is stale. A build from
 # master reads it as the default model source; a tagged build reads the gallery
 # until the release cutover.
+- kind: model
+  id: huihui-qwen3.8-flash-next-abliterated-v2-q4
+  name: huihui-qwen3.8-flash-next-abliterated-v2-q4
+  version: 0.0.0-gallery.a7ae4b3465ef
+  description: |
+    Huihui's second abliterated Qwen3.8-Flash-Next release adjusts the
+    abliteration without modifying the experts. This vision-language
+    mixture-of-experts model uses the publisher's UD-Q4_K_XL GGUF
+    and BF16 projector for text chat and image input through llama.cpp.
+    The default context is 32,768 tokens. Model weights use the
+    Qwen Community License 1.0.
+  license: other
+  urls:
+    - https://huggingface.co/Qwen/Qwen3.8-Flash-Next
+    - https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF
+  tags:
+    - llm
+    - gguf
+    - cpu
+    - gpu
+    - qwen
+    - moe
+    - vision
+    - multimodal
+    - abliterated
+  icon: https://qianwen-res.oss-cn-beijing.aliyuncs.com/logo_qwen.jpg
+  channel: gallery
+  policy:
+    requireSignature: false
+  schemaVersion: 1
+  model:
+    url: github:mudler/LocalAI/gallery/virtual.yaml@master
+    files:
+      - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
+        sha256: 4448186216b3af4cc558bbce2c3213f01608f8f8b2e5267a9767971dd3ec8082
+        uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
+      - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00002-of-00004.gguf
+        sha256: 6f3e21a5ae10c9124c2405061d89f90b575a0704b2927f059621b7936850b276
+        uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00002-of-00004.gguf
+      - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00003-of-00004.gguf
+        sha256: bb24c1d9c42c129ad93d427d3be6ca9a8d5ca6bf07ca9a50ddca03328d08c702
+        uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00003-of-00004.gguf
+      - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00004-of-00004.gguf
+        sha256: 3d6a9aee4ade5740394da9bcb9fa22ea9343471f246af6914f8fcc8327ceab8e
+        uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00004-of-00004.gguf
+      - filename: llama-cpp/mmproj/huihui-qwen3.8-flash-next-abliterated-v2/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+        sha256: b1a82259702816a5330d7bd7607cd9676b11780e79ff7348c21103ff3ce49bd0
+        uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+    overrides:
+      backend: llama-cpp
+      context_size: 32768
+      known_usecases:
+        - chat
+        - vision
+      mmproj: llama-cpp/mmproj/huihui-qwen3.8-flash-next-abliterated-v2/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+      options:
+        - use_jinja:true
+      parameters:
+        min_p: 0
+        model: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-q4/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf
+        presence_penalty: 0
+        repeat_penalty: 1
+        temperature: 1
+        top_k: 20
+        top_p: 0.95
+      template:
+        use_tokenizer_template: true
+    variants:
+      - model: huihui-qwen3.8-flash-next-abliterated-v2-iq4
+- kind: model
+  id: huihui-qwen3.8-flash-next-abliterated-v2-iq4
+  name: huihui-qwen3.8-flash-next-abliterated-v2-iq4
+  version: 0.0.0-gallery.ee04c2bad7d7
+  description: |
+    Huihui's second abliterated Qwen3.8-Flash-Next release adjusts the
+    abliteration without modifying the experts. This vision-language
+    mixture-of-experts model uses the publisher's UD-IQ4_XS GGUF
+    and BF16 projector for text chat and image input through llama.cpp.
+    The default context is 32,768 tokens. Model weights use the
+    Qwen Community License 1.0.
+  license: other
+  urls:
+    - https://huggingface.co/Qwen/Qwen3.8-Flash-Next
+    - https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF
+  tags:
+    - llm
+    - gguf
+    - cpu
+    - gpu
+    - qwen
+    - moe
+    - vision
+    - multimodal
+    - abliterated
+  icon: https://qianwen-res.oss-cn-beijing.aliyuncs.com/logo_qwen.jpg
+  channel: gallery
+  policy:
+    requireSignature: false
+  schemaVersion: 1
+  model:
+    url: github:mudler/LocalAI/gallery/virtual.yaml@master
+    files:
+      - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-iq4/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf
+        sha256: 5ce89370720f8bf90890f439361282104c1aa1482d4013bb9a50923e758e71a4
+        uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf
+      - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-iq4/Qwen3.8-Flash-Next-UD-IQ4_XS-00002-of-00003.gguf
+        sha256: ede87d3bd3e354d43f0e710b2df6791d3479e396847e53adbe2b48be7ba6d663
+        uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00002-of-00003.gguf
+      - filename: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-iq4/Qwen3.8-Flash-Next-UD-IQ4_XS-00003-of-00003.gguf
+        sha256: f8aa8bd7ba134b3b7fe4fc95b9131fc8f6678ace374a0fa6e632225031665995
+        uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/unsloth-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00003-of-00003.gguf
+      - filename: llama-cpp/mmproj/huihui-qwen3.8-flash-next-abliterated-v2/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+        sha256: b1a82259702816a5330d7bd7607cd9676b11780e79ff7348c21103ff3ce49bd0
+        uri: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF/resolve/d7b7fd6c2bda71446542da4e8e325f430f69ec8c/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+    overrides:
+      backend: llama-cpp
+      context_size: 32768
+      known_usecases:
+        - chat
+        - vision
+      mmproj: llama-cpp/mmproj/huihui-qwen3.8-flash-next-abliterated-v2/mmproj-Qwen3.8-Flash-Next-BF16.gguf
+      options:
+        - use_jinja:true
+      parameters:
+        min_p: 0
+        model: llama-cpp/models/huihui-qwen3.8-flash-next-abliterated-v2-iq4/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf
+        presence_penalty: 0
+        repeat_penalty: 1
+        temperature: 1
+        top_k: 20
+        top_p: 0.95
+      template:
+        use_tokenizer_template: true
 - kind: model
  id: adavar-3b-q4
  name: adavar-3b-q4