mirror/LocalAI
1
0 Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-10-28 00:59:28 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(gallery): add Ternary Bonsai 2 9B variants

Browse Source 
Add three GGUF packings with the Q8_0 vision projector and document
installation. Pin asset revisions and verify hashes against HF metadata
and download headers.

Assisted-by: Codex:gpt-6

(cherry picked from commit 77f1ab29c8, mudler/LocalAI#12771)
Assisted-by: Claude:claude-sonnet-5-5
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
localai-org-maint-bot authored and Ettore Di Giacinto committed 2026-10-25 09:55:39 +00:00
1 parent bf32819177
commit d6e81e4ba0
2 files changed
+312
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

gallery/index.yaml
+144
View File
@@ -3,6 +3,150 @@
# written by scripts/models/cmd/modelcatalog and stays only so releases before
# the cutover keep working. CI fails when it differs from the generated copy:
# go run ./scripts/models/cmd/modelcatalog --library gallery/models/index.yaml --out gallery/index.yaml
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: ternary-bonsai-2-9b
description: |
Ternary Bonsai 2 9B is a vision-language model derived from Qwen3.5-9B
with ternary transformer weights. This PQ2_0 build uses the Bonsai
backend and includes the Q8_0 vision projector. The three standard
packings contain the same weights in different storage layouts.
license: apache-2.0
urls:
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf
- https://huggingface.co/Qwen/Qwen3.5-9B
tags:
- llm
- gguf
- reasoning
- vision
- multimodal
files:
- filename: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-PQ2_0.gguf
sha256: 7a89ab48f5038c704480f60781f9b8d4035351f5f6beb7d11fad80001fde23f9
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-PQ2_0.gguf
- filename: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
sha256: e0cb6f6988cea323ef798cd77d019bbb819af75582f034eaab0e7232314953db
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
overrides:
backend: bonsai
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- vision
mmproj: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
options:
- use_jinja:true
parameters:
min_p: 0.0
model: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-PQ2_0.gguf
presence_penalty: 1.0
repeat_penalty: 1.0
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
variants:
- model: ternary-bonsai-2-9b-ptq1
- model: ternary-bonsai-2-9b-q2
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: ternary-bonsai-2-9b-ptq1
description: |
Ternary Bonsai 2 9B is a vision-language model derived from Qwen3.5-9B
with ternary transformer weights. This PTQ1_0 build uses the Bonsai
backend and includes the Q8_0 vision projector. The three standard
packings contain the same weights in different storage layouts.
license: apache-2.0
urls:
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf
- https://huggingface.co/Qwen/Qwen3.5-9B
tags:
- llm
- gguf
- reasoning
- vision
- multimodal
files:
- filename: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-PTQ1_0.gguf
sha256: 3561df7ac7d26a71ffa991ac2db740b10841c3a350aa2676ccd3ff5d428b4e8b
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-PTQ1_0.gguf
- filename: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
sha256: e0cb6f6988cea323ef798cd77d019bbb819af75582f034eaab0e7232314953db
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
overrides:
backend: bonsai
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- vision
mmproj: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
options:
- use_jinja:true
parameters:
min_p: 0.0
model: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-PTQ1_0.gguf
presence_penalty: 1.0
repeat_penalty: 1.0
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: ternary-bonsai-2-9b-q2
description: |
Ternary Bonsai 2 9B is a vision-language model derived from Qwen3.5-9B
with ternary transformer weights. This Q2_0 build uses the Bonsai
backend and includes the Q8_0 vision projector. The three standard
packings contain the same weights in different storage layouts.
license: apache-2.0
urls:
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf
- https://huggingface.co/Qwen/Qwen3.5-9B
tags:
- llm
- gguf
- reasoning
- vision
- multimodal
files:
- filename: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-Q2_0.gguf
sha256: 1f884aa56aacd2bbbb2e4b7abaead2b1102b74a9c75a23fc33dab916813a384d
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-Q2_0.gguf
- filename: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
sha256: e0cb6f6988cea323ef798cd77d019bbb819af75582f034eaab0e7232314953db
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
overrides:
backend: bonsai
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- vision
mmproj: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
options:
- use_jinja:true
parameters:
min_p: 0.0
model: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-Q2_0.gguf
presence_penalty: 1.0
repeat_penalty: 1.0
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: qwen3.8-27b-imatrix-nvfp4-mtp
description: |
gallery/models/index.yaml
+168
View File
@@ -4,6 +4,174 @@
# changes and never edit it by hand. CI fails when it is stale. A build from
# master reads it as the default model source; a tagged build reads the gallery
# until the release cutover.
- kind: model
id: ternary-bonsai-2-9b
name: ternary-bonsai-2-9b
version: 0.0.0-gallery.275b12980f1b
description: |
Ternary Bonsai 2 9B is a vision-language model derived from Qwen3.5-9B
with ternary transformer weights. This PQ2_0 build uses the Bonsai
backend and includes the Q8_0 vision projector. The three standard
packings contain the same weights in different storage layouts.
license: apache-2.0
urls:
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf
- https://huggingface.co/Qwen/Qwen3.5-9B
tags:
- llm
- gguf
- reasoning
- vision
- multimodal
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-PQ2_0.gguf
sha256: 7a89ab48f5038c704480f60781f9b8d4035351f5f6beb7d11fad80001fde23f9
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-PQ2_0.gguf
- filename: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
sha256: e0cb6f6988cea323ef798cd77d019bbb819af75582f034eaab0e7232314953db
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
overrides:
backend: bonsai
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- vision
mmproj: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
options:
- use_jinja:true
parameters:
min_p: 0.0
model: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-PQ2_0.gguf
presence_penalty: 1.0
repeat_penalty: 1.0
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
variants:
- model: ternary-bonsai-2-9b-ptq1
- model: ternary-bonsai-2-9b-q2
- kind: model
id: ternary-bonsai-2-9b-ptq1
name: ternary-bonsai-2-9b-ptq1
version: 0.0.0-gallery.fd060a8d2e1e
description: |
Ternary Bonsai 2 9B is a vision-language model derived from Qwen3.5-9B
with ternary transformer weights. This PTQ1_0 build uses the Bonsai
backend and includes the Q8_0 vision projector. The three standard
packings contain the same weights in different storage layouts.
license: apache-2.0
urls:
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf
- https://huggingface.co/Qwen/Qwen3.5-9B
tags:
- llm
- gguf
- reasoning
- vision
- multimodal
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-PTQ1_0.gguf
sha256: 3561df7ac7d26a71ffa991ac2db740b10841c3a350aa2676ccd3ff5d428b4e8b
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-PTQ1_0.gguf
- filename: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
sha256: e0cb6f6988cea323ef798cd77d019bbb819af75582f034eaab0e7232314953db
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
overrides:
backend: bonsai
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- vision
mmproj: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
options:
- use_jinja:true
parameters:
min_p: 0.0
model: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-PTQ1_0.gguf
presence_penalty: 1.0
repeat_penalty: 1.0
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- kind: model
id: ternary-bonsai-2-9b-q2
name: ternary-bonsai-2-9b-q2
version: 0.0.0-gallery.ba7c9dff50f8
description: |
Ternary Bonsai 2 9B is a vision-language model derived from Qwen3.5-9B
with ternary transformer weights. This Q2_0 build uses the Bonsai
backend and includes the Q8_0 vision projector. The three standard
packings contain the same weights in different storage layouts.
license: apache-2.0
urls:
- https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf
- https://huggingface.co/Qwen/Qwen3.5-9B
tags:
- llm
- gguf
- reasoning
- vision
- multimodal
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-Q2_0.gguf
sha256: 1f884aa56aacd2bbbb2e4b7abaead2b1102b74a9c75a23fc33dab916813a384d
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-Q2_0.gguf
- filename: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
sha256: e0cb6f6988cea323ef798cd77d019bbb819af75582f034eaab0e7232314953db
uri: https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf/resolve/5b2abb02362efbed46bbe204014b02837778b2c2/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
overrides:
backend: bonsai
context_size: 32768
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- vision
mmproj: bonsai/mmproj/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
options:
- use_jinja:true
parameters:
min_p: 0.0
model: bonsai/models/ternary-bonsai-2-9b/Ternary-Bonsai-2-9B-Q2_0.gguf
presence_penalty: 1.0
repeat_penalty: 1.0
temperature: 1.0
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- kind: model
id: qwen3.8-27b-imatrix-nvfp4-mtp
name: qwen3.8-27b-imatrix-nvfp4-mtp