mirror of
https://github.com/mudler/LocalAI.git
synced 2026-09-25 07:34:58 -04:00
feat(gallery): add Ortenzya 31B GGUF variants
Add Q4_K_M and Q8_0 builds with the BF16 vision projector. Pin downloads and document installation and variant selection. Assisted-by: Codex:gpt-6
This commit is contained in:
1 parent
3dfb34376a
commit
504dfb5281
2 files changed
+102
No files matched your search
@@ -73,6 +73,26 @@ When browsing the gallery or importing a model by URI, LocalAI can show **estima
|
||||
- **Hardware fit indicator**: When your system reports GPU or RAM capacity, the gallery shows whether the estimated VRAM fits (green) or may not fit (red) using a 95% headroom rule.
|
||||
- Estimates are best-effort and may be missing if the server does not support HEAD/Range or the request times out.
|
||||
|
||||
## Gemma 4 Ortenzya 31B
|
||||
|
||||
[Ortenzya](https://huggingface.co/llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-GGUF)
|
||||
is a Gemma 4 fine-tune for creative writing, translation, and roleplay.
|
||||
Install it with automatic selection between Q4_K_M and Q8_0:
|
||||
|
||||
```bash
|
||||
local-ai models install gemma-4-31b-ortenzya-q4
|
||||
```
|
||||
|
||||
To select Q8_0 explicitly:
|
||||
|
||||
```bash
|
||||
local-ai models install gemma-4-31b-ortenzya-q4 --variant gemma-4-31b-ortenzya-q8
|
||||
```
|
||||
|
||||
Both builds use llama.cpp, the embedded Jinja chat template, and the BF16
|
||||
vision projector for image prompts. The gallery uses a 32,768-token context
|
||||
to limit memory use, with temperature 1, top_p 0.95, and top_k 64.
|
||||
|
||||
## Gemma 4 12B IT
|
||||
|
||||
Install `gemma-4-12b-it-q4` for chat, tool use, and image prompts with
|
||||
|
||||
@@ -11153,6 +11153,88 @@
|
||||
- filename: llama-cpp/mmproj/Qwopus3.6-27B-Coder-MTP-GGUF/mmproj-F32.gguf
|
||||
sha256: 32f7ea0600c07272547da401d460f8abbd980f3a57b69d6df87be0e2505e0b9c
|
||||
uri: https://huggingface.co/Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF/resolve/main/mmproj-F32.gguf
|
||||
- &gemma-4-31b-ortenzya
|
||||
name: gemma-4-31b-ortenzya-q4
|
||||
variants:
|
||||
- model: gemma-4-31b-ortenzya-q8
|
||||
url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
urls:
|
||||
- https://huggingface.co/llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic
|
||||
- https://huggingface.co/llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-GGUF
|
||||
license: apache-2.0
|
||||
icon: https://ai.google.dev/gemma/images/gemma4_banner.png
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- gemma4
|
||||
- multimodal
|
||||
- reasoning
|
||||
- creative-writing
|
||||
description: |
|
||||
Ortenzya is a Gemma 4 31B instruction-tuned model from llmfan46 for
|
||||
creative writing, translation, and roleplay. This entry uses the Q4_K_M
|
||||
GGUF with the BF16 vision projector and a 32,768-token context.
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 32768
|
||||
function:
|
||||
automatic_tool_parsing_fallback: true
|
||||
grammar:
|
||||
disable: true
|
||||
known_usecases:
|
||||
- chat
|
||||
- vision
|
||||
mmproj: llama-cpp/mmproj/gemma-4-31b-ortenzya/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-mmproj-BF16.gguf
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: llama-cpp/models/gemma-4-31b-ortenzya/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-Q4_K_M.gguf
|
||||
temperature: 1
|
||||
top_p: 0.95
|
||||
top_k: 64
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
files:
|
||||
- filename: llama-cpp/models/gemma-4-31b-ortenzya/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-Q4_K_M.gguf
|
||||
uri: https://huggingface.co/llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-GGUF/resolve/d7c96e816759b50ba7d0afe3a9e36deb7b096f16/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-Q4_K_M.gguf
|
||||
sha256: aefb0b5a93362b8880e2639c15ea807220351c7abfa8e0f6903ca25f31844981
|
||||
- filename: llama-cpp/mmproj/gemma-4-31b-ortenzya/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-mmproj-BF16.gguf
|
||||
uri: https://huggingface.co/llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-GGUF/resolve/d7c96e816759b50ba7d0afe3a9e36deb7b096f16/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-mmproj-BF16.gguf
|
||||
sha256: c040f82999134f7ba8dddf1c9e3451822c8ae9af51b60a01619df43a2a3a7128
|
||||
- !!merge <<: *gemma-4-31b-ortenzya
|
||||
name: gemma-4-31b-ortenzya-q8
|
||||
variants: []
|
||||
description: |
|
||||
Ortenzya is a Gemma 4 31B instruction-tuned model from llmfan46 for
|
||||
creative writing, translation, and roleplay. This entry uses the Q8_0
|
||||
GGUF with the BF16 vision projector and a 32,768-token context.
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 32768
|
||||
function:
|
||||
automatic_tool_parsing_fallback: true
|
||||
grammar:
|
||||
disable: true
|
||||
known_usecases:
|
||||
- chat
|
||||
- vision
|
||||
mmproj: llama-cpp/mmproj/gemma-4-31b-ortenzya/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-mmproj-BF16.gguf
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: llama-cpp/models/gemma-4-31b-ortenzya/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-Q8_0.gguf
|
||||
temperature: 1
|
||||
top_p: 0.95
|
||||
top_k: 64
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
files:
|
||||
- filename: llama-cpp/models/gemma-4-31b-ortenzya/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-Q8_0.gguf
|
||||
uri: https://huggingface.co/llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-GGUF/resolve/d7c96e816759b50ba7d0afe3a9e36deb7b096f16/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-Q8_0.gguf
|
||||
sha256: 688170f6d6e03edc33ac86891eb3c37ec6cb1739f5db8625a84a1e4cdba17586
|
||||
- filename: llama-cpp/mmproj/gemma-4-31b-ortenzya/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-mmproj-BF16.gguf
|
||||
uri: https://huggingface.co/llmfan46/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-GGUF/resolve/d7c96e816759b50ba7d0afe3a9e36deb7b096f16/gemma-4-Ortenzya-The-Creative-Wordsmith-31B-it-uncensored-heretic-mmproj-BF16.gguf
|
||||
sha256: c040f82999134f7ba8dddf1c9e3451822c8ae9af51b60a01619df43a2a3a7128
|
||||
- &gemma-4-31b-scotoma-2
|
||||
name: "gemma-4-31b-scotoma-2-q4"
|
||||
variants:
|
||||
|
||||
Reference in new issue
Block a user