mirror/LocalAI
1
0
Fork 0
mirror of https://github.com/mudler/LocalAI.git synced 2026-07-30 09:57:57 -04:00
Code Issues Packages Projects Releases Wiki Activity

gallery: remove duplicated entries and lint against them recurring (#10996)

Browse Source 
gallery: remove duplicated entries and lint against them coming back

gallery/index.yaml declared eight names twice: deepseek-r1-distill-llama-8b,
llama3.2-3b-enigma, qwen3-asr-0.6b, qwen3-asr-1.7b, qwopus-glm-18b-merged,
voice-en-us-kathleen-low, whisper-large-q5_0 and whisper-small-q5_1.

FindGalleryElement resolves a reference by returning the first match, so in
every pair the second copy was unreachable: it could not be installed, could
not be selected as a variant target, and could not be corrected, because any
edit to it went to a copy nobody reads. A reference to such a name is also
ambiguous to anything reasoning over the catalog, which is why the variant
proposal job refuses to propose against them.

Each pair was compared both as parsed entries and as raw text, and all eight
were byte-identical apart from position. None of the sixteen blocks defines a
YAML anchor or pulls one in with a merge key, so nothing was reachable only
through a deleted block, and no entry named a removed copy as a variant target.
Removing the second copy of each therefore changes no behaviour: the parsed set
loses exactly eight entries and every surviving entry is field-for-field
unchanged.

The removal is textual, by line range, so the diff is pure deletions rather than
a reflow of forty thousand lines.

checkNoDuplicateEntryNames is the rule that keeps them out, added beside the
existing gallery invariants and reporting in the same style.

checkSingleVariantClaim closes the adjacent gap in the same place. VariantParents
resolves a build claimed by two parents by taking the first in gallery order and
calls that deterministic "for a gallery the linter would reject", but nothing
rejected it: the invariant was held by curation alone. Now it is a rule, and the
comment describes something real. No target is doubly claimed today, so the rule
is green on arrival.

Assisted-by: Claude:claude-opus-4-8

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
mudler's LocalAI [bot]
2026-07-20 22:58:48 +02:00
committed by GitHub
parent 2f33ad6669
commit a4bab71f27
2 changed files with 193 additions and 212 deletions
Download Patch File Download Diff File Expand all files Collapse all files

193
core/gallery/variants_lint_test.go
View File

@@ -116,6 +116,77 @@ func checkEntriesInstallSomething(entries []gallery.GalleryModel) []variantViola
return violations
}
// checkNoDuplicateEntryNames verifies no two entries share a name.
//
// FindGalleryElement resolves a reference by scanning and returning the first
// match, so a second entry carrying an already-used name is unreachable: it can
// never be installed, never be selected as a variant target, and never be
// corrected, because every edit to it goes to a copy nobody reads. A reference
// to such a name is also ambiguous to a reader and to any tooling that reasons
// over the catalog, which is the harm even when the two copies happen to agree
// today.
func checkNoDuplicateEntryNames(entries []gallery.GalleryModel) []variantViolation {
seen := make(map[string]struct{}, len(entries))
var violations []variantViolation
for _, e := range entries {
if _, dup := seen[e.Name]; dup {
violations = append(violations, variantViolation{
Entry: e.Name,
Detail: fmt.Sprintf("entry %q is declared more than once: FindGalleryElement returns the first match, "+
"so only the first declaration is reachable and this one is dead weight. "+
"Remove the extra copy, or rename it if the two are meant to be different models", e.Name),
})
continue
}
seen[e.Name] = struct{}{}
}
return violations
}
// checkSingleVariantClaim verifies no entry is offered as a variant by more
// than one parent.
//
// VariantParents hides a claimed build from a collapsed listing and answers
// matches on it with the parent instead. With two parents claiming the same
// build it picks the first in gallery order, which is deterministic but
// arbitrary: the user is shown one of two equally valid rows, and which one
// depends on authoring order rather than on anything about the models. That
// resolution exists so the listing stays coherent for a catalog carrying the
// mistake; this is the rule that keeps the catalog from carrying it.
func checkSingleVariantClaim(entries []gallery.GalleryModel) []variantViolation {
byName := indexEntriesByName(entries)
// Only the first claim per parent counts, so a parent listing the same
// target twice is a redundancy inside one stanza and not two parents
// fighting over a build.
claimedBy := make(map[string]string, len(entries))
var violations []variantViolation
for _, e := range entries {
for _, v := range e.Variants {
// A dangling reference and a nested one are checkVariantReferences'
// business, and neither claims anything VariantParents would hide.
target, ok := byName[v.Model]
if !ok || target.HasVariants() || v.Model == e.Name {
continue
}
first, claimed := claimedBy[v.Model]
if !claimed {
claimedBy[v.Model] = e.Name
continue
}
if first == e.Name {
continue
}
violations = append(violations, variantViolation{
Entry: e.Name,
Variant: v.Model,
Detail: fmt.Sprintf("already offered as a variant by %q; a build may have only one parent, "+
"otherwise which row a collapsed listing shows for it depends on authoring order", first),
})
}
}
return violations
}
// loadGalleryIndex parses gallery/index.yaml once for the whole suite. The
// index carries well over a thousand entries, so re-parsing it per spec is
// pure overhead.
@@ -181,6 +252,118 @@ var _ = Describe("gallery variant lint helpers", func() {
Expect(checkVariantReferences(entries)).To(BeEmpty())
Expect(checkEntriesInstallSomething(entries)).To(BeEmpty())
Expect(checkNoDuplicateEntryNames(entries)).To(BeEmpty())
Expect(checkSingleVariantClaim(entries)).To(BeEmpty())
})
Describe("checkNoDuplicateEntryNames", func() {
It("flags a second entry carrying an already-used name", func() {
entries := []gallery.GalleryModel{
plainEntry("twin", "u://first"),
plainEntry("other", "u://other"),
plainEntry("twin", "u://second"),
}
violations := checkNoDuplicateEntryNames(entries)
Expect(violations).To(HaveLen(1))
Expect(violations[0].Entry).To(Equal("twin"))
Expect(violations[0].Detail).To(ContainSubstring("declared more than once"))
Expect(violations[0].Detail).To(ContainSubstring("only the first declaration is reachable"))
})
// The copies in gallery/index.yaml were identical, so a rule that only
// fired on differing copies would have let every one of them through.
It("flags a duplicate whose fields match the first copy exactly", func() {
entries := []gallery.GalleryModel{
plainEntry("twin", "u://same"),
plainEntry("twin", "u://same"),
}
Expect(checkNoDuplicateEntryNames(entries)).To(HaveLen(1))
})
It("reports every breach in one pass rather than stopping at the first", func() {
entries := []gallery.GalleryModel{
plainEntry("a", "u://a"),
plainEntry("b", "u://b"),
plainEntry("a", "u://a2"),
plainEntry("b", "u://b2"),
}
Expect(checkNoDuplicateEntryNames(entries)).To(HaveLen(2))
})
// A name used three times costs two removals, so all the copies past
// the first have to be named rather than only the second.
It("flags every copy past the first", func() {
entries := []gallery.GalleryModel{
plainEntry("a", "u://1"),
plainEntry("a", "u://2"),
plainEntry("a", "u://3"),
}
Expect(checkNoDuplicateEntryNames(entries)).To(HaveLen(2))
})
})
Describe("checkSingleVariantClaim", func() {
It("flags a build offered as a variant by two parents", func() {
entries := variantFixture(
entryWithVariants("base-a", "u://a", gallery.Variant{Model: "shared"}),
entryWithVariants("base-b", "u://b", gallery.Variant{Model: "shared"}),
plainEntry("shared", "u://shared"),
)
violations := checkSingleVariantClaim(entries)
Expect(violations).To(HaveLen(1))
Expect(violations[0].Entry).To(Equal("base-b"))
Expect(violations[0].Variant).To(Equal("shared"))
Expect(violations[0].Detail).To(ContainSubstring(`already offered as a variant by "base-a"`))
})
It("accepts two parents offering different builds", func() {
entries := variantFixture(
entryWithVariants("base-a", "u://a", gallery.Variant{Model: "one"}),
entryWithVariants("base-b", "u://b", gallery.Variant{Model: "two"}),
plainEntry("one", "u://one"),
plainEntry("two", "u://two"),
)
Expect(checkSingleVariantClaim(entries)).To(BeEmpty())
})
// VariantParents takes the first claim per parent, so a repeat inside
// one stanza hides nothing extra and is not this rule's business.
It("does not treat one parent listing a build twice as two claims", func() {
entries := variantFixture(
entryWithVariants("base", "u://base",
gallery.Variant{Model: "dup"},
gallery.Variant{Model: "dup"},
),
plainEntry("dup", "u://dup"),
)
Expect(checkSingleVariantClaim(entries)).To(BeEmpty())
})
// VariantParents skips these too, so flagging them here would report
// the same authoring mistake twice under two different names.
It("leaves dangling and nested references to checkVariantReferences", func() {
entries := variantFixture(
entryWithVariants("base-a", "u://a",
gallery.Variant{Model: "ghost"},
gallery.Variant{Model: "nested"},
),
entryWithVariants("base-b", "u://b",
gallery.Variant{Model: "ghost"},
gallery.Variant{Model: "nested"},
),
entryWithVariants("nested", "u://nested", gallery.Variant{Model: "leaf"}),
plainEntry("leaf", "u://leaf"),
)
Expect(checkSingleVariantClaim(entries)).To(BeEmpty())
})
})
Describe("checkEntriesInstallSomething", func() {
@@ -322,6 +505,16 @@ var _ = Describe("gallery/index.yaml variant invariants", Ordered, func() {
v := checkEntriesInstallSomething(entries)
Expect(v).To(BeEmpty(), formatViolations(v))
})
It("declares every entry name exactly once", func() {
v := checkNoDuplicateEntryNames(entries)
Expect(v).To(BeEmpty(), formatViolations(v))
})
It("gives every variant build a single parent", func() {
v := checkSingleVariantClaim(entries)
Expect(v).To(BeEmpty(), formatViolations(v))
})
})
// The lint rules above check the catalog as text. This drives the real

212
gallery/index.yaml
View File

@@ -3473,37 +3473,6 @@
- filename: llama-cpp/models/supergemma4-26b-uncensored-gguf-v2/supergemma4-26b-uncensored-fast-v2-Q4_K_M.gguf
sha256: e773b0a209d48524f9d485bca0818247f75d7ddde7cce951367a7e441fb59137
uri: https://huggingface.co/Jiunsong/supergemma4-26b-uncensored-gguf-v2/resolve/main/supergemma4-26b-uncensored-fast-v2-Q4_K_M.gguf
- name: qwopus-glm-18b-merged
url: github:mudler/LocalAI/gallery/virtual.yaml@master
urls:
- https://huggingface.co/KyleHessling1/Qwopus-GLM-18B-Merged-GGUF
description: "# \U0001FA90 Qwen3.5-9B-GLM5.1-Distill-v1\n\n## \U0001F4CC Model Overview\n\n**Model Name:** `Jackrong/Qwen3.5-9B-GLM5.1-Distill-v1`\n**Base Model:** Qwen3.5-9B\n**Training Type:** Supervised Fine-Tuning (SFT, Distillation)\n**Parameter Scale:** 9B\n**Training Framework:** Unsloth\n\nThis model is a distilled variant of **Qwen3.5-9B**, trained on high-quality reasoning data derived from **GLM-5.1**.\n\nThe primary goals are to:\n\n - Improve **structured reasoning ability**\n - Enhance **instruction-following consistency**\n - Activate **latent knowledge via better reasoning structure**\n\n## \U0001F4CA Training Data\n\n### Main Dataset\n\n - `Jackrong/GLM-5.1-Reasoning-1M-Cleaned`\n - Cleaned from the original `Kassadin88/GLM-5.1-1000000x` dataset.\n - Generated from a **GLM-5.1 teacher model**\n - Approximately **700x** the scale of `Qwen3.5-reasoning-700x`\n - Training used a **filtered subset**, not the full source dataset.\n\n### Auxiliary Dataset\n\n - `Jackrong/Qwen3.5-reasoning-700x`\n\n...\n"
license: apache-2.0
icon: https://cdn-uploads.huggingface.co/production/uploads/66309bd090589b7c65950665/BnSg_x99v9bG9T5-8sKa1.png
tags:
- llm
- gguf
- reasoning
last_checked: "2026-04-30"
overrides:
backend: llama-cpp
function:
automatic_tool_parsing_fallback: true
grammar:
disable: true
known_usecases:
- chat
- completion
options:
- use_jinja:true
parameters:
model: llama-cpp/models/Qwopus-GLM-18B-Merged-GGUF/Qwopus-GLM-18B-Healed-Q4_K_M.gguf
template:
use_tokenizer_template: true
files:
- filename: llama-cpp/models/Qwopus-GLM-18B-Merged-GGUF/Qwopus-GLM-18B-Healed-Q4_K_M.gguf
sha256: 13bd039f95c9ea46ef1d75905faa7be6ca4e47a5af9d4cf62e298a738a5b195f
uri: https://huggingface.co/KyleHessling1/Qwopus-GLM-18B-Merged-GGUF/resolve/main/Qwopus-GLM-18B-Healed-Q4_K_M.gguf
- name: qwen3.6-35b-a3b-apex
url: github:mudler/LocalAI/gallery/virtual.yaml@master
urls:
@@ -8455,54 +8424,6 @@
- filename: mmproj-Qwen3-Omni-30B-A3B-Thinking-Q8_0.gguf
sha256: 2bd5459571f8230a0c251d3d0dd36267753f0800ed145449a34f220a31f93898
uri: huggingface://ggml-org/Qwen3-Omni-30B-A3B-Thinking-GGUF/mmproj-Qwen3-Omni-30B-A3B-Thinking-Q8_0.gguf
- name: qwen3-asr-0.6b
url: github:mudler/LocalAI/gallery/virtual.yaml@master
urls:
- https://huggingface.co/Qwen/Qwen3-ASR-0.6B
description: |
Qwen3-ASR is an automatic speech recognition model supporting multiple languages and batch inference.
license: apache-2.0
icon: https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/-s1gyJfvbE1RgO5iBeNOi.png
tags:
- speech-recognition
- asr
last_checked: "2026-04-30"
overrides:
backend: qwen-asr
known_usecases:
- transcript
parameters:
model: Qwen/Qwen3-ASR-0.6B
artifacts:
- name: model
target: model
source:
type: huggingface
repo: Qwen/Qwen3-ASR-0.6B
- name: qwen3-asr-1.7b
url: github:mudler/LocalAI/gallery/virtual.yaml@master
urls:
- https://huggingface.co/Qwen/Qwen3-ASR-1.7B
description: |
Qwen3-ASR is an automatic speech recognition model supporting multiple languages and batch inference.
license: apache-2.0
icon: https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/-s1gyJfvbE1RgO5iBeNOi.png
tags:
- speech-recognition
- asr
last_checked: "2026-04-30"
overrides:
backend: qwen-asr
known_usecases:
- transcript
parameters:
model: Qwen/Qwen3-ASR-1.7B
artifacts:
- name: model
target: model
source:
type: huggingface
repo: Qwen/Qwen3-ASR-1.7B
- name: glm-ocr
url: github:mudler/LocalAI/gallery/virtual.yaml@master
urls:
@@ -20064,33 +19985,6 @@
- filename: calme-3.1-llamaloi-3b.Q5_K_M.gguf
sha256: 06b900c7252423329ca57a02a8b8d18a1294934709861d09af96e74694c9a3f1
uri: huggingface://MaziyarPanahi/calme-3.1-llamaloi-3b-GGUF/calme-3.1-llamaloi-3b.Q5_K_M.gguf
- name: llama3.2-3b-enigma
url: github:mudler/LocalAI/gallery/llama3.2-quantized.yaml@master
urls:
- https://huggingface.co/QuantFactory/Llama3.2-3B-Enigma-GGUF
description: |
Enigma is a code-instruct model built on Llama 3.2 3b. It is a high quality code instruct model with the Llama 3.2 Instruct chat format. The model is finetuned on synthetic code-instruct data generated with Llama 3.1 405b and supplemented with generalist synthetic data. It uses the Llama 3.2 Instruct prompt format.
license: llama3.2
icon: https://cdn-uploads.huggingface.co/production/uploads/64f267a8a4f79a118e0fcc89/it7MY5MyLCLpFQev5dUis.jpeg
tags:
- llama
- llama-3.2
- 3b
- gguf
- quantized
- code
- code-instruct
- chat
- instruct
- llm
last_checked: "2026-05-04"
overrides:
parameters:
model: Llama3.2-3B-Enigma.Q4_K_M.gguf
files:
- filename: Llama3.2-3B-Enigma.Q4_K_M.gguf
sha256: 4304e6ee1e348b228470700ec1e9423f5972333d376295195ce6cd5c70cae5e4
uri: huggingface://QuantFactory/Llama3.2-3B-Enigma-GGUF/Llama3.2-3B-Enigma.Q4_K_M.gguf
- name: llama3.2-3b-shiningvaliant2-i1
url: github:mudler/LocalAI/gallery/llama3.2-quantized.yaml@master
urls:
@@ -25401,38 +25295,6 @@
- filename: DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gguf
sha256: bed9b0f551f5b95bf9da5888a48f0f87c37ad6b72519c4cbd775f54ac0b9fc62
uri: huggingface://bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF/DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gguf
- name: deepseek-r1-distill-llama-8b
url: github:mudler/LocalAI/gallery/llama3.1-instruct.yaml@master
urls:
- https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B
- https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF
description: |
DeepSeek-R1 is our advanced first-generation reasoning model designed to enhance performance in reasoning tasks.
Building on the foundation laid by its predecessor, DeepSeek-R1-Zero, which was trained using large-scale reinforcement learning (RL) without supervised fine-tuning, DeepSeek-R1 addresses the challenges faced by R1-Zero, such as endless repetition, poor readability, and language mixing.
By incorporating cold-start data prior to the RL phase,DeepSeek-R1 significantly improves reasoning capabilities and achieves performance levels comparable to OpenAI-o1 across a variety of domains, including mathematics, coding, and complex reasoning tasks.
license: llama3.1
icon: https://avatars.githubusercontent.com/u/148330874
tags:
- deepseek
- llama
- llama-3
- gguf
- quantized
- 8b
- llm
- reasoning
- code
- math
- distilled
- chat
last_checked: "2026-05-04"
overrides:
parameters:
model: deepseek-r1-distill-llama-8b-Q4_K_M.gguf
files:
- filename: deepseek-r1-distill-llama-8b-Q4_K_M.gguf
sha256: 0addb1339a82385bcd973186cd80d18dcc71885d45eabd899781a118d03827d9
uri: huggingface://unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF/DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf
- name: deepseek-r1-distill-llama-70b
url: github:mudler/LocalAI/gallery/deepseek-r1.yaml@master
urls:
@@ -31893,33 +31755,6 @@
- filename: ggml-small.en.bin
sha256: c6138d6d58ecc8322097e0f987c32f1be8bb0a18532a3f88f734d1bbf9c41e5d
uri: huggingface://ggerganov/whisper.cpp/ggml-small.en.bin
- name: whisper-small-q5_1
url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
urls:
- https://github.com/ggerganov/whisper.cpp
- https://huggingface.co/ggerganov/whisper.cpp
description: |
Port of OpenAI's Whisper model in C/C++
license: mit
icon: https://avatars.githubusercontent.com/u/14957082
tags:
- whisper
- small
- gguf
- quantized
- q5_1
- speech-recognition
- multilingual
- transcription
- whisper.cpp
last_checked: "2026-05-04"
overrides:
parameters:
model: ggml-small-q5_1.bin
files:
- filename: ggml-small-q5_1.bin
sha256: ae85e4a935d7a567bd102fe55afc16bb595bdb618e11b2fc7591bc08120411bb
uri: huggingface://ggerganov/whisper.cpp/ggml-small-q5_1.bin
- name: whisper-tiny
variants:
- model: whisper-tiny-q5_1
@@ -32082,34 +31917,6 @@
- filename: ggml-large-v3.bin
sha256: 64d182b440b98d5203c4f9bd541544d84c605196c4f7b845dfa11fb23594d1e2
uri: huggingface://ggerganov/whisper.cpp/ggml-large-v3.bin
- name: whisper-large-q5_0
url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
urls:
- https://github.com/ggerganov/whisper.cpp
- https://huggingface.co/ggerganov/whisper.cpp
description: |
Port of OpenAI's Whisper model in C/C++
license: mit
icon: https://avatars.githubusercontent.com/u/14957082
tags:
- whisper
- gguf
- quantized
- q5_0
- large
- asr
- speech-recognition
- transcription
- multilingual
- audio
last_checked: "2026-05-04"
overrides:
parameters:
model: ggml-large-v3-q5_0.bin
files:
- filename: ggml-large-v3-q5_0.bin
sha256: d75795ecff3f83b5faa89d1900604ad8c780abd5739fae406de19f23ecd98ad1
uri: huggingface://ggerganov/whisper.cpp/ggml-large-v3-q5_0.bin
- name: whisper-large-turbo
variants:
- model: whisper-large-turbo-q5_0
@@ -32506,25 +32313,6 @@
- filename: voice-en-us-danny-low.tar.gz
sha256: 0c8fbb42526d5fbd3a0bded5f18041c0a893a70a7fb8756f97866624b932264b
uri: https://github.com/rhasspy/piper/releases/download/v0.0.2/voice-en-us-danny-low.tar.gz
- name: voice-en-us-kathleen-low
url: github:mudler/LocalAI/gallery/piper.yaml@master
urls:
- https://github.com/rhasspy/piper
description: |
A fast, local neural text to speech system that sounds great and is optimized for the Raspberry Pi 4. Piper is used in a variety of [projects](https://github.com/rhasspy/piper#people-using-piper).
license: mit
icon: https://github.com/rhasspy/piper/raw/master/etc/logo.png
tags:
- tts
- text-to-speech
- cpu
overrides:
parameters:
model: en-us-kathleen-low.onnx
files:
- filename: voice-en-us-kathleen-low.tar.gz
sha256: 18e32f009f864d8061af8a4be4ae9018b5aa8b49c37f9e108bbfd782c6a38fbf
uri: https://github.com/rhasspy/piper/releases/download/v0.0.2/voice-en-us-kathleen-low.tar.gz
- name: voice-en-us-lessac-low
url: github:mudler/LocalAI/gallery/piper.yaml@master
urls: