feat(gallery): add official Flint INT4 variant

Offer the publisher Q4_1 export alongside the existing Flint builds.
Pin the weights and document explicit variant installation.

Assisted-by: Codex:gpt-6
This commit is contained in:
localai-org-maint-bot committed 2026-10-23 20:04:42 +00:00
1 parent 3934ee8e34
commit 819e49482e
3 files changed
+106

No files matched your search

+14
View File
@@ -44,6 +44,20 @@ Both views store the view, search, filter, and selection in the URL. Installing
from Explore does not move you away from the catalog; the entry updates in place
when the operation finishes.
### Flint INT4 GGUF
[Recursive's official Flint Q4_1 build](https://huggingface.co/RecursiveSI/flint-int4-gguf)
is available as a variant of `flint` for the llama.cpp backend.
The 26 GB GGUF preserves the trained INT4 codes and includes the chat template.
The publisher recommends at least 48 GB of unified memory on Apple Silicon.
This entry uses a 32,768-token default context.
Install this build explicitly:
```bash
local-ai models install flint --variant flint-int4-q4-1
```
### MiniCPM-V 4.7 1B
[MiniCPM-V 4.7 1B](https://huggingface.co/openbmb/MiniCPM-V-4.7-1B)
+42
View File
@@ -77085,6 +77085,7 @@
variants:
- model: flint-q2
- model: flint-q6
- model: flint-int4-q4-1
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: flint-q2
description: |
@@ -77161,3 +77162,44 @@
top_p: 0.95
template:
use_tokenizer_template: true
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: flint-int4-q4-1
description: |
Flint is a coding-agent mixture-of-experts model from Recursive, with
39.8B total and about 2.5B active parameters per token. This official
Q4_1 GGUF preserves the trained INT4 codes from the MLX export, with
F32 floating tensors. The 26 GB file includes the tokenizer and chat
template. This entry uses a 32,768-token default context.
license: apache-2.0
urls:
- https://huggingface.co/RecursiveSI/flint
- https://huggingface.co/RecursiveSI/flint-int4-gguf
tags:
- llm
- gguf
- cpu
- gpu
- moe
- coding
- agent
- reasoning
- tools
files:
- filename: llama-cpp/models/RecursiveSI/flint-int4-gguf/flint-int4-q4_1.gguf
sha256: 43c07baccdba42eb7528a9630ee15699b1942c297291dc1b39a061f508ee1a4b
uri: https://huggingface.co/RecursiveSI/flint-int4-gguf/resolve/9c47151036bc2c34331142dda04488bceab2bc9f/flint-int4-q4_1.gguf
overrides:
backend: llama-cpp
context_size: 32768
known_usecases:
- chat
options:
- use_jinja:true
parameters:
min_p: 0
model: llama-cpp/models/RecursiveSI/flint-int4-gguf/flint-int4-q4_1.gguf
temperature: 1
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
+50
View File
@@ -94790,6 +94790,7 @@
variants:
- model: flint-q2
- model: flint-q6
- model: flint-int4-q4-1
- kind: model
id: flint-q2
name: flint-q2
@@ -94882,3 +94883,52 @@
top_p: 0.95
template:
use_tokenizer_template: true
- kind: model
id: flint-int4-q4-1
name: flint-int4-q4-1
version: 0.1.0
description: |
Flint is a coding-agent mixture-of-experts model from Recursive, with
39.8B total and about 2.5B active parameters per token. This official
Q4_1 GGUF preserves the trained INT4 codes from the MLX export, with
F32 floating tensors. The 26 GB file includes the tokenizer and chat
template. This entry uses a 32,768-token default context.
license: apache-2.0
urls:
- https://huggingface.co/RecursiveSI/flint
- https://huggingface.co/RecursiveSI/flint-int4-gguf
tags:
- llm
- gguf
- cpu
- gpu
- moe
- coding
- agent
- reasoning
- tools
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/RecursiveSI/flint-int4-gguf/flint-int4-q4_1.gguf
sha256: 43c07baccdba42eb7528a9630ee15699b1942c297291dc1b39a061f508ee1a4b
uri: https://huggingface.co/RecursiveSI/flint-int4-gguf/resolve/9c47151036bc2c34331142dda04488bceab2bc9f/flint-int4-q4_1.gguf
overrides:
backend: llama-cpp
context_size: 32768
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/RecursiveSI/flint-int4-gguf/flint-int4-q4_1.gguf
temperature: 1
top_k: 20
top_p: 0.95
min_p: 0
template:
use_tokenizer_template: true