mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 09:08:54 -04:00
chore(gallery): add Prism Coder 9B variants
Offer Q4_K_M and Q8_0 builds of the Qwen3.5 tool-routing fine-tune. Keep the entries text-only because the quantization repository does not publish a vision projector. Pin downloads and document selection. Assisted-by: Codex:gpt-6
This commit is contained in:
1 parent
89a0955b5b
commit
98cad4a19f
2 files changed
+90
No files matched your search
@@ -167,6 +167,22 @@ These entries use `stablediffusion-ggml` and include the text encoder, vision pr
|
||||
The invalid `qwen-image-2.1-uncensored` chat entry was removed because llama.cpp cannot load its diffusion weights.
|
||||
This removal does not delete previously installed models. Remove that configuration before installing an image-generation entry.
|
||||
|
||||
## Prism Coder 9B
|
||||
|
||||
[Prism Coder 9B](https://huggingface.co/dcostenco/prism-coder-9b) is a
|
||||
Qwen3.5-9B fine-tune for tool routing and function calling. Install it with:
|
||||
|
||||
```bash
|
||||
local-ai models install prism-coder-9b
|
||||
```
|
||||
|
||||
The gallery offers Q4_K_M and Q8_0 GGUF variants for the `llama-cpp` backend,
|
||||
with a 32768-token context and the embedded chat template. To select Q8_0
|
||||
explicitly, use `local-ai models install prism-coder-9b --variant prism-coder-9b-q8`.
|
||||
These entries support text only because the quantization repository does not
|
||||
provide a vision projector. They load the model weights; the publisher's
|
||||
application-specific tool remapping rules are not included.
|
||||
|
||||
## VRAM and download size estimates
|
||||
|
||||
When browsing the gallery or importing a model by URI, LocalAI can show **estimated download size** and **estimated VRAM** for models.
|
||||
|
||||
@@ -66772,6 +66772,80 @@
|
||||
- filename: mmproj-Qwen3.8-3.6-27B-blend-BF16.gguf
|
||||
sha256: 4568f7e9f79ef5b942be9acc8be717156482d0659a6fde111df660f57a1875c8
|
||||
uri: https://huggingface.co/JetBrains/Qwen3.8-3.6-27B-blend-GGUF/resolve/2770acba861928558b9d706cc06462e6a6f30b7b/mmproj-Qwen3.8-3.6-27B-blend-BF16.gguf
|
||||
- name: prism-coder-9b
|
||||
url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
urls:
|
||||
- https://huggingface.co/dcostenco/prism-coder-9b
|
||||
- https://huggingface.co/mradermacher/prism-coder-9b-GGUF
|
||||
description: |
|
||||
Prism Coder 9B is a Qwen3.5-9B fine-tune for tool routing and function
|
||||
calling. This Q4_K_M GGUF uses the embedded chat template and a 32768-token
|
||||
context. This entry supports text only; the quantization repository does
|
||||
not provide a vision projector.
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
- function-calling
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 32768
|
||||
known_usecases:
|
||||
- chat
|
||||
function:
|
||||
automatic_tool_parsing_fallback: true
|
||||
grammar:
|
||||
disable: true
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: llama-cpp/models/prism-coder-9b/prism-coder-9b.Q4_K_M.gguf
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
variants:
|
||||
- model: prism-coder-9b-q8
|
||||
files:
|
||||
- filename: llama-cpp/models/prism-coder-9b/prism-coder-9b.Q4_K_M.gguf
|
||||
uri: https://huggingface.co/mradermacher/prism-coder-9b-GGUF/resolve/edf75a38dfa056195f1632a9694ff2e0f79054c6/prism-coder-9b.Q4_K_M.gguf
|
||||
sha256: 8143b554f879953b55514c07df0d729e184c0031c93b938cce9b8aa278305821
|
||||
- name: prism-coder-9b-q8
|
||||
url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
urls:
|
||||
- https://huggingface.co/dcostenco/prism-coder-9b
|
||||
- https://huggingface.co/mradermacher/prism-coder-9b-GGUF
|
||||
description: |
|
||||
Prism Coder 9B is a Qwen3.5-9B fine-tune for tool routing and function
|
||||
calling. This Q8_0 GGUF uses the embedded chat template and a 32768-token
|
||||
context. This entry supports text only; the quantization repository does
|
||||
not provide a vision projector.
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- llm
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
- function-calling
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
context_size: 32768
|
||||
known_usecases:
|
||||
- chat
|
||||
function:
|
||||
automatic_tool_parsing_fallback: true
|
||||
grammar:
|
||||
disable: true
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: llama-cpp/models/prism-coder-9b/prism-coder-9b.Q8_0.gguf
|
||||
template:
|
||||
use_tokenizer_template: true
|
||||
files:
|
||||
- filename: llama-cpp/models/prism-coder-9b/prism-coder-9b.Q8_0.gguf
|
||||
uri: https://huggingface.co/mradermacher/prism-coder-9b-GGUF/resolve/edf75a38dfa056195f1632a9694ff2e0f79054c6/prism-coder-9b.Q8_0.gguf
|
||||
sha256: e9ec3446685a5076a1595b2c85fd180271ee8e3e069fd0db5816c3f68465328c
|
||||
- name: qwen3.8-27b-humanlike-chat
|
||||
url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
urls:
|
||||
|
||||
Reference in new issue
Block a user