mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-28 09:05:05 -04:00
feat(gallery): add Cyber-Tiel-Coder GGUF variants
Add Q4, Q6, and Q8 builds with the BF16 vision projector and native chat template. Document installation and pin verified artifact hashes. Assisted-by: Codex:gpt-6
This commit is contained in:
localai-org-maint-bot committed
1 parent
d463316caf
commit
2de795f3f1
2 files changed
+145
No files matched your search
@@ -23,6 +23,27 @@ GPT and text generation models might have a license which is not permissive for
|
||||
|
||||
|
||||
|
||||
## Cyber-Tiel-Coder 35B A3B
|
||||
|
||||
Install Cyber-Tiel-Coder with automatic selection between UD-Q4_K_XL,
|
||||
UD-Q6_K_XL, and UD-Q8_K_XL GGUF builds:
|
||||
|
||||
```bash
|
||||
local-ai models install cyber-tiel-coder-35b-a3b-q4
|
||||
```
|
||||
|
||||
To select the Q8 build explicitly:
|
||||
|
||||
```bash
|
||||
local-ai models install cyber-tiel-coder-35b-a3b-q4 --variant cyber-tiel-coder-35b-a3b-q8
|
||||
```
|
||||
|
||||
These llama.cpp entries include the BF16 vision projector and embedded chat
|
||||
template. They use a 131,072-token context and the publisher's coding sampling
|
||||
settings. Cyber-Tiel-Coder is based on Huihui's abliterated Ornith-1.5 model;
|
||||
see the [model card](https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF)
|
||||
for its capabilities and limitations.
|
||||
|
||||
## Useful Links and resources
|
||||
|
||||
- [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) - here you can find a list of the most performing models on the Open LLM benchmark. Keep in mind models compatible with LocalAI must be quantized in the `gguf` format.
|
||||
|
||||
@@ -3735,6 +3735,130 @@
|
||||
- filename: llama-cpp/mmproj/thomson-1.0-small/mmproj-bf16.gguf
|
||||
uri: huggingface://bartowski/thomsonreuters_Thomson-1.0-Small-GGUF/mmproj-thomsonreuters_Thomson-1.0-Small-bf16.gguf
|
||||
sha256: 11634fcccd59c23f1b95e34e5cf479dec86290eeb3dda980324aabd8b0b48f41
|
||||
- &cyber-tiel-coder-35b-a3b
|
||||
name: cyber-tiel-coder-35b-a3b-q4
|
||||
url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
urls:
|
||||
- https://huggingface.co/huihui-ai/Huihui-Ornith-1.5-35B-A3B-abliterated
|
||||
- https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF
|
||||
license: mit
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
- moe
|
||||
- coding
|
||||
- reasoning
|
||||
- vision
|
||||
variants:
|
||||
- model: cyber-tiel-coder-35b-a3b-q6
|
||||
- model: cyber-tiel-coder-35b-a3b-q8
|
||||
description: |
|
||||
Cyber-Tiel-Coder is a 35B-parameter MoE coding model with 3B active
|
||||
parameters, based on Huihui's abliterated Ornith-1.5. This UD-Q4_K_XL
|
||||
GGUF build includes the BF16 vision projector and embedded chat template.
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 131072
|
||||
function:
|
||||
automatic_tool_parsing_fallback: true
|
||||
grammar:
|
||||
disable: true
|
||||
known_usecases:
|
||||
- chat
|
||||
- vision
|
||||
mmproj: llama-cpp/mmproj/cyber-tiel-coder-35b-a3b/mmproj-BF16.gguf
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: llama-cpp/models/cyber-tiel-coder-35b-a3b/Cyber-Tiel-Coder-35B-A3B-UD-Q4_K_XL.gguf
|
||||
temperature: 0.6
|
||||
top_p: 0.95
|
||||
top_k: 20
|
||||
min_p: 0
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
files:
|
||||
- filename: llama-cpp/models/cyber-tiel-coder-35b-a3b/Cyber-Tiel-Coder-35B-A3B-UD-Q4_K_XL.gguf
|
||||
uri: https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF/resolve/644bee2e6b9a75aa64c3bb818aaeeebe43d862bb/Cyber-Tiel-Coder-35B-A3B-UD-Q4_K_XL.gguf
|
||||
sha256: 21bff5a96fa051809f4a8260befe977771c9927cae9c32eadfcb0718bae960cd
|
||||
- filename: llama-cpp/mmproj/cyber-tiel-coder-35b-a3b/mmproj-BF16.gguf
|
||||
uri: https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF/resolve/644bee2e6b9a75aa64c3bb818aaeeebe43d862bb/mmproj-BF16.gguf
|
||||
sha256: d9ce31026d1cb1f3f8d5152e2e2a014d9d2b302b6c93a7dc07bb0a0487f52837
|
||||
|
||||
- !!merge <<: *cyber-tiel-coder-35b-a3b
|
||||
name: cyber-tiel-coder-35b-a3b-q6
|
||||
variants: []
|
||||
description: |
|
||||
Cyber-Tiel-Coder is a 35B-parameter MoE coding model with 3B active
|
||||
parameters, based on Huihui's abliterated Ornith-1.5. This UD-Q6_K_XL
|
||||
GGUF build includes the BF16 vision projector and embedded chat template.
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 131072
|
||||
function:
|
||||
automatic_tool_parsing_fallback: true
|
||||
grammar:
|
||||
disable: true
|
||||
known_usecases:
|
||||
- chat
|
||||
- vision
|
||||
mmproj: llama-cpp/mmproj/cyber-tiel-coder-35b-a3b/mmproj-BF16.gguf
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: llama-cpp/models/cyber-tiel-coder-35b-a3b/Cyber-Tiel-Coder-35B-A3B-UD-Q6_K_XL.gguf
|
||||
temperature: 0.6
|
||||
top_p: 0.95
|
||||
top_k: 20
|
||||
min_p: 0
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
files:
|
||||
- filename: llama-cpp/models/cyber-tiel-coder-35b-a3b/Cyber-Tiel-Coder-35B-A3B-UD-Q6_K_XL.gguf
|
||||
uri: https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF/resolve/644bee2e6b9a75aa64c3bb818aaeeebe43d862bb/Cyber-Tiel-Coder-35B-A3B-UD-Q6_K_XL.gguf
|
||||
sha256: 86b112ccaf6f2f1d3c07f79abbea3ed6a9cadfa8b26f089f7c41085ab369ca97
|
||||
- filename: llama-cpp/mmproj/cyber-tiel-coder-35b-a3b/mmproj-BF16.gguf
|
||||
uri: https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF/resolve/644bee2e6b9a75aa64c3bb818aaeeebe43d862bb/mmproj-BF16.gguf
|
||||
sha256: d9ce31026d1cb1f3f8d5152e2e2a014d9d2b302b6c93a7dc07bb0a0487f52837
|
||||
|
||||
- !!merge <<: *cyber-tiel-coder-35b-a3b
|
||||
name: cyber-tiel-coder-35b-a3b-q8
|
||||
variants: []
|
||||
description: |
|
||||
Cyber-Tiel-Coder is a 35B-parameter MoE coding model with 3B active
|
||||
parameters, based on Huihui's abliterated Ornith-1.5. This UD-Q8_K_XL
|
||||
GGUF build includes the BF16 vision projector and embedded chat template.
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 131072
|
||||
function:
|
||||
automatic_tool_parsing_fallback: true
|
||||
grammar:
|
||||
disable: true
|
||||
known_usecases:
|
||||
- chat
|
||||
- vision
|
||||
mmproj: llama-cpp/mmproj/cyber-tiel-coder-35b-a3b/mmproj-BF16.gguf
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: llama-cpp/models/cyber-tiel-coder-35b-a3b/Cyber-Tiel-Coder-35B-A3B-UD-Q8_K_XL.gguf
|
||||
temperature: 0.6
|
||||
top_p: 0.95
|
||||
top_k: 20
|
||||
min_p: 0
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
files:
|
||||
- filename: llama-cpp/models/cyber-tiel-coder-35b-a3b/Cyber-Tiel-Coder-35B-A3B-UD-Q8_K_XL.gguf
|
||||
uri: https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF/resolve/644bee2e6b9a75aa64c3bb818aaeeebe43d862bb/Cyber-Tiel-Coder-35B-A3B-UD-Q8_K_XL.gguf
|
||||
sha256: 7509f8c8e3b2df4c7bc5966fa711478c12c2c424f2832f7933f1e46dd180feb2
|
||||
- filename: llama-cpp/mmproj/cyber-tiel-coder-35b-a3b/mmproj-BF16.gguf
|
||||
uri: https://huggingface.co/peculiar-ragdoll/Cyber-Tiel-Coder-35B-A3B-GGUF/resolve/644bee2e6b9a75aa64c3bb818aaeeebe43d862bb/mmproj-BF16.gguf
|
||||
sha256: d9ce31026d1cb1f3f8d5152e2e2a014d9d2b302b6c93a7dc07bb0a0487f52837
|
||||
|
||||
- &tiel-coder-35b-a3b
|
||||
name: "tiel-coder-35b-a3b-q4"
|
||||
variants:
|
||||
|
||||
Reference in new issue
Block a user