mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 09:08:54 -04:00
chore(gallery): add Clef GGUF variants
Add the 27B Clef decision model with Q4_K_M and Q8_0 builds and a matching vision projector. Pin artifacts and document installation. Assisted-by: Codex:gpt-6
This commit is contained in:
1 parent
883e4c56eb
commit
ab88ee821e
2 files changed
+87
-1
No files matched your search
@@ -341,6 +341,26 @@ tokens. LocalAI records supplied usage once; explicit zero counts are distinct
|
||||
from missing usage. Missing counts are not estimated, and invalid negative counts
|
||||
are rejected rather than billed.
|
||||
|
||||
### Clef text and image decisions
|
||||
|
||||
Install Cloudflare's 27B Clef decision model with its Q8_0 vision projector:
|
||||
|
||||
```sh
|
||||
local-ai models install clef --variant clef
|
||||
```
|
||||
|
||||
This selects the Q4_K_M build. Use `--variant clef-q8` for Q8_0,
|
||||
or omit `--variant` to let the gallery choose a build for available memory.
|
||||
Both builds use Apache-2.0 weights from
|
||||
[ggml-org/Clef-GGUF](https://huggingface.co/ggml-org/Clef-GGUF), pinned to
|
||||
revision `63840a1a68cb7084c88610cffc328509356b04cb` with SHA-256 checksums.
|
||||
|
||||
Send requests to `/v1/systemone` or `/v1/decisions` with the installed model's
|
||||
name. Clef returns typed decisions; it does not generate chat completions.
|
||||
Image requests use the bounded base64 image contract described above and require
|
||||
the matching projector included in both builds. The gallery sets an 8192-token
|
||||
context. Use a current stock `llama-cpp` backend with Clef decision support.
|
||||
|
||||
### Julia-1 CPU example
|
||||
|
||||
Install the separate stock llama.cpp entry (existing vllm-cpp entries are unchanged):
|
||||
@@ -470,7 +490,7 @@ fit; context overflow remains an error. Size the context for the actual workload
|
||||
Native image decisions require both a decision format that accepts images and a
|
||||
loaded projector that supports **vision input**. A missing projector, an
|
||||
audio-only projector, or a text-only decision model does not silently fall back
|
||||
to a text decision. Other decision gallery entries remain text-only.
|
||||
to a text decision. Clef image decisions also require the matching vision projector.
|
||||
|
||||
Direct native RPC callers receive the same image count, encoded/decoded byte,
|
||||
dimension and aggregate pixel bounds as public callers. Validation precedes
|
||||
|
||||
@@ -65746,6 +65746,72 @@
|
||||
- filename: Kev-4B-Q4_K_M.gguf
|
||||
uri: https://huggingface.co/ggml-org/Kev-4B-GGUF/resolve/d924f2e2c3872da8b8aaf3eb4453b4126deceb79/Kev-4B-Q4_K_M.gguf
|
||||
sha256: 33ae6b18926502b2209a1bf7d3b61a350d65938441515c686bb87d226a19eff9
|
||||
- name: clef
|
||||
url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
variants:
|
||||
- model: clef-q8
|
||||
urls:
|
||||
- https://huggingface.co/Cloudflare/clef
|
||||
- https://huggingface.co/ggml-org/Clef-GGUF
|
||||
license: apache-2.0
|
||||
description: |
|
||||
Clef is Cloudflare's 27B Qwen3.8-based model for typed decisions
|
||||
from text and images. This Q4_K_M GGUF includes the Q8_0 vision projector
|
||||
and uses native llama.cpp decisions through /v1/systemone and /v1/decisions.
|
||||
It answers choice, score, and predicate questions without generating text.
|
||||
tags:
|
||||
- decisions
|
||||
- systemone
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
known_usecases:
|
||||
- decisions
|
||||
context_size: 8192
|
||||
mmproj: llama-cpp/mmproj/clef/mmproj-Clef-Q8_0.gguf
|
||||
parameters:
|
||||
model: llama-cpp/models/clef/Clef-Q4_K_M.gguf
|
||||
files:
|
||||
- filename: llama-cpp/models/clef/Clef-Q4_K_M.gguf
|
||||
uri: https://huggingface.co/ggml-org/Clef-GGUF/resolve/63840a1a68cb7084c88610cffc328509356b04cb/Clef-Q4_K_M.gguf
|
||||
sha256: bd9b8ae24c5752a2bb5e2733d5168340d5f04457e02b8bf0d1bc9285ec3d4f61
|
||||
- filename: llama-cpp/mmproj/clef/mmproj-Clef-Q8_0.gguf
|
||||
uri: https://huggingface.co/ggml-org/Clef-GGUF/resolve/63840a1a68cb7084c88610cffc328509356b04cb/mmproj-Clef-Q8_0.gguf
|
||||
sha256: 0d901ea999ae122ba0afeb4a3f4c4ac9a90baa07f1d120df9c350082db1190a8
|
||||
- name: clef-q8
|
||||
url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
urls:
|
||||
- https://huggingface.co/Cloudflare/clef
|
||||
- https://huggingface.co/ggml-org/Clef-GGUF
|
||||
license: apache-2.0
|
||||
description: |
|
||||
Clef is Cloudflare's 27B Qwen3.8-based model for typed decisions
|
||||
from text and images. This Q8_0 GGUF includes the Q8_0 vision projector
|
||||
and uses native llama.cpp decisions through /v1/systemone and /v1/decisions.
|
||||
It answers choice, score, and predicate questions without generating text.
|
||||
tags:
|
||||
- decisions
|
||||
- systemone
|
||||
- gguf
|
||||
- cpu
|
||||
- gpu
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
known_usecases:
|
||||
- decisions
|
||||
context_size: 8192
|
||||
mmproj: llama-cpp/mmproj/clef/mmproj-Clef-Q8_0.gguf
|
||||
parameters:
|
||||
model: llama-cpp/models/clef/Clef-Q8_0.gguf
|
||||
files:
|
||||
- filename: llama-cpp/models/clef/Clef-Q8_0.gguf
|
||||
uri: https://huggingface.co/ggml-org/Clef-GGUF/resolve/63840a1a68cb7084c88610cffc328509356b04cb/Clef-Q8_0.gguf
|
||||
sha256: 07c6410af7011e0e56873a3b0b3f4ad9e31fb176f0ca80d6fd1525fe6f036548
|
||||
- filename: llama-cpp/mmproj/clef/mmproj-Clef-Q8_0.gguf
|
||||
uri: https://huggingface.co/ggml-org/Clef-GGUF/resolve/63840a1a68cb7084c88610cffc328509356b04cb/mmproj-Clef-Q8_0.gguf
|
||||
sha256: 0d901ea999ae122ba0afeb4a3f4c4ac9a90baa07f1d120df9c350082db1190a8
|
||||
- name: lev-llama-cpp
|
||||
urls:
|
||||
- https://huggingface.co/interfaze-ai/lev
|
||||
|
||||
Reference in new issue
Block a user