feat(gallery): add Tess 4 27B variants (#11547)

Add Q4_K_M and Q8_0 multimodal builds. Include an MTP-enabled Q4_K_M build for speculative decoding.

Assisted-by: Codex:gpt-5

Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
This commit is contained in:
localai-org-maint-botandlocalai-org-maint-bot authored and GitHub committed 2026-08-16 21:44:22 +02:00
1 parent d666f1a0f0
commit 804dc10968
1 file changed
+134
+134
View File
@@ -2370,6 +2370,140 @@
- filename: llama-cpp/mmproj/Qwythos-9B-v2-MTP-Q4_K_M/mmproj-Qwythos-9B-v2-BF16.gguf
sha256: 0d1687cb33124c78acab788b342d4a2eaf85b3035e87c3abe4ee9d0b84ddb4f5
uri: https://huggingface.co/empero-ai/Qwythos-9B-v2-GGUF/resolve/main/mmproj-Qwythos-9B-v2-BF16.gguf
- &tess-4-27b
name: "tess-4-27b"
variants:
- model: tess-4-27b-q8
- model: tess-4-27b-mtp
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
urls:
- https://huggingface.co/migtissera/Tess-4-27B
- https://huggingface.co/migtissera/Tess-4-27B-GGUF
description: |
Tess-4-27B is an Apache-2.0 agentic and reasoning model built on
Qwen3.6-27B. It scales its thinking depth to the task and supports tool
use, long-context work, and image input. This default entry uses the
Q4_K_M GGUF quantization and the shared F16 vision projector.
license: "apache-2.0"
tags:
- llm
- gguf
- cpu
- gpu
- qwen
- reasoning
- thinking
- agent
- tools
- long-context
- vision
- multimodal
last_checked: "2026-08-16"
overrides:
backend: llama-cpp
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- vision
mmproj: llama-cpp/mmproj/tess-4-27b/mmproj-Tess-4-27B-F16.gguf
options:
- use_jinja:true
parameters:
model: llama-cpp/models/tess-4-27b/Tess-4-27B-Q4_K_M.gguf
template:
use_tokenizer_template: true
files:
- filename: llama-cpp/models/tess-4-27b/Tess-4-27B-Q4_K_M.gguf
sha256: 99b94644e6b31dff920ec5eb1b8598edc95eed7fcc8c3058a98f42319fb2fef4
uri: huggingface://migtissera/Tess-4-27B-GGUF/Tess-4-27B-Q4_K_M.gguf
- filename: llama-cpp/mmproj/tess-4-27b/mmproj-Tess-4-27B-F16.gguf
sha256: 6be9dd0b2298225381277c2b7df9c4bbcc5b9d95b43b7299123f8392527cde69
uri: huggingface://migtissera/Tess-4-27B-GGUF/mmproj-Tess-4-27B-F16.gguf
- !!merge <<: *tess-4-27b
name: "tess-4-27b-q8"
variants: []
description: |
Tess-4-27B is an Apache-2.0 agentic and reasoning model built on
Qwen3.6-27B. This entry uses the near-lossless Q8_0 GGUF quantization and
the shared F16 vision projector.
overrides:
backend: llama-cpp
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- vision
mmproj: llama-cpp/mmproj/tess-4-27b-q8/mmproj-Tess-4-27B-F16.gguf
options:
- use_jinja:true
parameters:
model: llama-cpp/models/tess-4-27b-q8/Tess-4-27B-Q8_0.gguf
template:
use_tokenizer_template: true
files:
- filename: llama-cpp/models/tess-4-27b-q8/Tess-4-27B-Q8_0.gguf
sha256: 35d0b640edc1a9c00f8089bada11c4a58fc825d9d0fbf4ac44f4054ec3cb3a57
uri: huggingface://migtissera/Tess-4-27B-GGUF/Tess-4-27B-Q8_0.gguf
- filename: llama-cpp/mmproj/tess-4-27b-q8/mmproj-Tess-4-27B-F16.gguf
sha256: 6be9dd0b2298225381277c2b7df9c4bbcc5b9d95b43b7299123f8392527cde69
uri: huggingface://migtissera/Tess-4-27B-GGUF/mmproj-Tess-4-27B-F16.gguf
- !!merge <<: *tess-4-27b
name: "tess-4-27b-mtp"
variants: []
description: |
Tess-4-27B with its Q4_K_M multi-token prediction draft enabled for
speculative decoding. The main model verifies every proposed token, and
the entry also includes the shared F16 vision projector.
tags:
- llm
- gguf
- cpu
- gpu
- qwen
- reasoning
- thinking
- agent
- tools
- long-context
- vision
- multimodal
- mtp
- speculative
overrides:
backend: llama-cpp
draft_model: llama-cpp/models/tess-4-27b-mtp/mtp-Tess-4-27B-Q4_K_M.gguf
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- vision
mmproj: llama-cpp/mmproj/tess-4-27b-mtp/mmproj-Tess-4-27B-F16.gguf
options:
- use_jinja:true
- spec_type:draft-mtp
- spec_n_max:6
- spec_p_min:0.75
parameters:
model: llama-cpp/models/tess-4-27b-mtp/Tess-4-27B-Q4_K_M.gguf
template:
use_tokenizer_template: true
files:
- filename: llama-cpp/models/tess-4-27b-mtp/Tess-4-27B-Q4_K_M.gguf
sha256: 99b94644e6b31dff920ec5eb1b8598edc95eed7fcc8c3058a98f42319fb2fef4
uri: huggingface://migtissera/Tess-4-27B-GGUF/Tess-4-27B-Q4_K_M.gguf
- filename: llama-cpp/models/tess-4-27b-mtp/mtp-Tess-4-27B-Q4_K_M.gguf
sha256: c8dd872f71f584eef1693779c98d9c6c81daaa9bf97a7d9b24ae9cb968a30a77
uri: huggingface://migtissera/Tess-4-27B-GGUF/mtp-Tess-4-27B-Q4_K_M.gguf
- filename: llama-cpp/mmproj/tess-4-27b-mtp/mmproj-Tess-4-27B-F16.gguf
sha256: 6be9dd0b2298225381277c2b7df9c4bbcc5b9d95b43b7299123f8392527cde69
uri: huggingface://migtissera/Tess-4-27B-GGUF/mmproj-Tess-4-27B-F16.gguf
- &qwen3-6-14b-a3b-fablevibes
name: "qwen3.6-14b-a3b-fablevibes"
variants: