mirror/LocalAI
1
0 Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-10-28 09:08:54 -04:00
Code Issues Packages Projects Releases Wiki Activity

feat(gallery): add flint and bump llama.cpp to a37cc461 (#12854)

Browse Source 
* chore(llama-cpp): bump to a37cc461 and drop the TTS patch

Move the llama.cpp pin to a37cc461, the merge of the Flint
architecture (ggml-org/llama.cpp#30928).

The TTS task patch is gone: upstream now carries
SERVER_TASK_TYPE_TTS, server_task::tts_inp, server_task_result_tts,
server_context_meta::has_cap_tts and the mtmd gen-audio helpers. The
patch is detected as already applied on the new pin. The gRPC server
uses the same symbols. The bonsai and turboquant forks still lack
them and keep using disable-tts-task.sh, so they are unchanged.

The score task patch is still needed: upstream has no
SERVER_TASK_TYPE_SCORE. It is regenerated against the new pin and
applies with no offset or fuzz. Upstream added
SERVER_TASK_TYPE_TRANSCRIBE_RT to server_task::need_logits(), which
broke one hunk, so SCORE is added next to it by hand.

The C++ backend was not built for this change.

Assisted-by: Claude:claude-sonnet-5-5

* feat(gallery): add flint

Add Recursive Flint, a 39.8B mixture-of-experts coding model with
about 2.5B active parameters, from the Unsloth GGUF repository. It
runs on llama.cpp with the Flint architecture support added upstream
in ggml-org/llama.cpp#30928.

Three quantizations are listed, with the default declaring the others
as variants: UD-Q4_K_XL (25 GB), UD-Q2_K_XL (13.6 GB) for small hosts
and UD-Q6_K_XL (36 GB) for larger ones. Sampling defaults follow the
model card (temperature 1, top_p 0.95, top_k 20). The library file
gallery/models/index.yaml is edited and gallery/index.yaml is
regenerated from it.

Assisted-by: Claude:claude-sonnet-5-5

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
mudler-agentandEttore Di Giacinto authored and GitHub committed 2026-10-23 15:58:25 +02:00
1 parent 08de1cb919
commit 86994fac46
5 files changed
+313 -905
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

gallery/index.yaml
+117
View File
@@ -77044,3 +77044,120 @@
- detector_interval:1
parameters:
model: gem-x/gem-x-contact-f32.gguf
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: flint
description: |
Flint is a mixture-of-experts coding-agent model from Recursive, with
39.8B total and about 2.5B active parameters per token. It takes text in
and gives text out, and supports reasoning and multi-turn tool use. This
entry uses the Unsloth UD-Q4_K_XL GGUF quantization (25 GB).
license: apache-2.0
urls:
- https://huggingface.co/RecursiveSI/flint
- https://huggingface.co/unsloth/flint-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- moe
- coding
- agent
- reasoning
- tools
files:
- filename: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q4_K_XL.gguf
sha256: 63c2ecc461ca545134b670c660e76fa20f47240f8d39b6510bfd21c4dd6e072f
uri: huggingface://unsloth/flint-GGUF/flint-UD-Q4_K_XL.gguf
overrides:
backend: llama-cpp
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q4_K_XL.gguf
temperature: 1
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
variants:
- model: flint-q2
- model: flint-q6
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: flint-q2
description: |
Flint is a mixture-of-experts coding-agent model from Recursive, with
39.8B total and about 2.5B active parameters per token. It takes text in
and gives text out, and supports reasoning and multi-turn tool use. This
entry uses the Unsloth UD-Q2_K_XL GGUF quantization (13.6 GB).
license: apache-2.0
urls:
- https://huggingface.co/RecursiveSI/flint
- https://huggingface.co/unsloth/flint-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- moe
- coding
- agent
- reasoning
- tools
files:
- filename: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q2_K_XL.gguf
sha256: 7f43dd99e7d574c43c3f1b24e0e8269cf1ec92ba6c25f11b3ff126cde4c7a0cb
uri: huggingface://unsloth/flint-GGUF/flint-UD-Q2_K_XL.gguf
overrides:
backend: llama-cpp
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q2_K_XL.gguf
temperature: 1
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- url: github:mudler/LocalAI/gallery/virtual.yaml@master
name: flint-q6
description: |
Flint is a mixture-of-experts coding-agent model from Recursive, with
39.8B total and about 2.5B active parameters per token. It takes text in
and gives text out, and supports reasoning and multi-turn tool use. This
entry uses the Unsloth UD-Q6_K_XL GGUF quantization (36 GB).
license: apache-2.0
urls:
- https://huggingface.co/RecursiveSI/flint
- https://huggingface.co/unsloth/flint-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- moe
- coding
- agent
- reasoning
- tools
files:
- filename: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q6_K_XL.gguf
sha256: 086231093581c67f8bf2964f8b49198889c4f0775b39857ba63b4c37041279c2
uri: huggingface://unsloth/flint-GGUF/flint-UD-Q6_K_XL.gguf
overrides:
backend: llama-cpp
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q6_K_XL.gguf
temperature: 1
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
gallery/models/index.yaml
+141
View File
@@ -94741,3 +94741,144 @@
- detector_interval:1
parameters:
model: gem-x/gem-x-contact-f32.gguf
- kind: model
id: flint
name: flint
version: 0.0.0-gallery.a824f12de5cd
description: |
Flint is a mixture-of-experts coding-agent model from Recursive, with
39.8B total and about 2.5B active parameters per token. It takes text in
and gives text out, and supports reasoning and multi-turn tool use. This
entry uses the Unsloth UD-Q4_K_XL GGUF quantization (25 GB).
license: apache-2.0
urls:
- https://huggingface.co/RecursiveSI/flint
- https://huggingface.co/unsloth/flint-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- moe
- coding
- agent
- reasoning
- tools
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q4_K_XL.gguf
sha256: 63c2ecc461ca545134b670c660e76fa20f47240f8d39b6510bfd21c4dd6e072f
uri: huggingface://unsloth/flint-GGUF/flint-UD-Q4_K_XL.gguf
overrides:
backend: llama-cpp
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q4_K_XL.gguf
temperature: 1
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
variants:
- model: flint-q2
- model: flint-q6
- kind: model
id: flint-q2
name: flint-q2
version: 0.0.0-gallery.0522ec7a82c2
description: |
Flint is a mixture-of-experts coding-agent model from Recursive, with
39.8B total and about 2.5B active parameters per token. It takes text in
and gives text out, and supports reasoning and multi-turn tool use. This
entry uses the Unsloth UD-Q2_K_XL GGUF quantization (13.6 GB).
license: apache-2.0
urls:
- https://huggingface.co/RecursiveSI/flint
- https://huggingface.co/unsloth/flint-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- moe
- coding
- agent
- reasoning
- tools
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q2_K_XL.gguf
sha256: 7f43dd99e7d574c43c3f1b24e0e8269cf1ec92ba6c25f11b3ff126cde4c7a0cb
uri: huggingface://unsloth/flint-GGUF/flint-UD-Q2_K_XL.gguf
overrides:
backend: llama-cpp
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q2_K_XL.gguf
temperature: 1
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true
- kind: model
id: flint-q6
name: flint-q6
version: 0.0.0-gallery.6df3c309be93
description: |
Flint is a mixture-of-experts coding-agent model from Recursive, with
39.8B total and about 2.5B active parameters per token. It takes text in
and gives text out, and supports reasoning and multi-turn tool use. This
entry uses the Unsloth UD-Q6_K_XL GGUF quantization (36 GB).
license: apache-2.0
urls:
- https://huggingface.co/RecursiveSI/flint
- https://huggingface.co/unsloth/flint-GGUF
tags:
- llm
- gguf
- cpu
- gpu
- moe
- coding
- agent
- reasoning
- tools
channel: gallery
policy:
requireSignature: false
schemaVersion: 1
model:
url: github:mudler/LocalAI/gallery/virtual.yaml@master
files:
- filename: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q6_K_XL.gguf
sha256: 086231093581c67f8bf2964f8b49198889c4f0775b39857ba63b4c37041279c2
uri: huggingface://unsloth/flint-GGUF/flint-UD-Q6_K_XL.gguf
overrides:
backend: llama-cpp
known_usecases:
- chat
options:
- use_jinja:true
parameters:
model: llama-cpp/models/unsloth/flint-GGUF/flint-UD-Q6_K_XL.gguf
temperature: 1
top_k: 20
top_p: 0.95
template:
use_tokenizer_template: true