mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 01:48:06 -04:00
ci(gallery): propose variant groupings for review instead of letting them decay (#10992)
ci(gallery): propose variant groupings for review on a schedule A gallery entry may declare `variants:`, references to other entries that are alternative builds of the same weights, and auto-selection then installs the best build for the host. Those families exist only because humans curated them in two manual sweeps. The gallery agent dedupes on the HuggingFace repo URL and picks one quantization per model, so it never adds a second build of a repo it already has, and consequently never creates a family and never joins one. A model published across two repos lands as two unrelated standalone entries. The grouping decays as the gallery grows and nothing notices. Add a scheduled job, in the same shape as the checksum checker: compute offline, edit the index textually, open a pull request against ci-forks. It proposes and never decides. Grouping is a judgement call that has gone wrong in both directions, so the value is catching drift and surfacing candidates with their evidence. Three grouping signals, taken from the manual sweeps: same name once quantization markers are stripped, the `:` config-suffix convention, and the same primary weight filename once quantization markers are stripped. The third requires the same upstream repository. Excluding auxiliary files is not enough on its own: bert-embeddings, an ultravox audio model and a roleplay finetune all declare a primary file called llama-3.2-1b-instruct-q4_k_m.gguf, and grouping on that is the same error that linked four wan-2.1 entries and Z-Image-Turbo to qwen3-4b. Add gallery/variant-exclusions.yaml, a checked-in rejection ledger. A job that re-proposes declined candidates every night becomes noise and gets ignored. Declining a proposal is one flow-mapping line a reviewer adds inside the proposal pull request itself. Seeded with the six -abliterated pairs whose base is also in the gallery, the mistral-small multimodal pair, the whisper-1 alias, the kokoros language set, and the recurring finetune tokens. qat and apex are deliberately not on it: they are quantization techniques. Proposals refuse to nest, to let two parents claim one target, to target an entry that installs nothing, and to touch a merge anchor, since a variants key added to an anchor is inherited by every merging child. The anchor refusal names every entry that would inherit, which is the worklist a human needs. Run against the pre-sweep gallery, the job rediscovers 12 of the 19 groupings the second manual sweep made, with no false positives. The rest it reports as refusals or ledger declines rather than missing silently. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
mudler's LocalAI [bot]
committed by
GitHub
GitHub
parent
f01038f479
commit
0cdd781c2d
133
.github/ci/variantproposals/body.go
vendored
Normal file
133
.github/ci/variantproposals/body.go
vendored
Normal file
@@ -0,0 +1,133 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// RenderBody writes the pull request body.
|
||||
//
|
||||
// The body is the product of this job, not the diff. Grouping is a judgement
|
||||
// call that has gone wrong in both directions before, so a reviewer has to be
|
||||
// able to accept or reject each family from the body alone, without opening
|
||||
// HuggingFace to work out whether two entries hold the same weights.
|
||||
func RenderBody(r *Result, ledgerPath string) string {
|
||||
var b strings.Builder
|
||||
|
||||
b.WriteString("## Proposed gallery variant groupings\n\n")
|
||||
b.WriteString("This is a proposal, not a decision. The gallery agent adds one build per model and never joins an existing family, so entries that are alternative builds of the same weights drift apart as the gallery grows. This job re-applies the grouping heuristics from the manual sweeps and asks a human to confirm.\n\n")
|
||||
b.WriteString("Each family below lists the parent, the variants, and the evidence that they are the same weights. **Reject anything whose evidence you do not believe.**\n\n")
|
||||
b.WriteString(fmt.Sprintf("To decline a family permanently, add one line to `%s` in this pull request and close it:\n\n", ledgerPath))
|
||||
b.WriteString("```yaml\npairs:\n - {parent: some-model, variant: some-model-thing, reason: \"different finetune\"}\n```\n\n")
|
||||
|
||||
b.WriteString(fmt.Sprintf("### Proposed families (%d)\n\n", len(r.Families)))
|
||||
if len(r.Families) == 0 {
|
||||
b.WriteString("None.\n\n")
|
||||
}
|
||||
for _, f := range r.Families {
|
||||
b.WriteString(fmt.Sprintf("#### `%s`\n\n", f.Parent))
|
||||
b.WriteString("| variant | signals | evidence |\n|---|---|---|\n")
|
||||
for _, p := range f.Proposals {
|
||||
b.WriteString(fmt.Sprintf("| `%s` | %s | %s |\n", p.Variant, joinSignals(p.Evidence.Signals), describeEvidence(p.Evidence)))
|
||||
}
|
||||
b.WriteString("\n")
|
||||
}
|
||||
|
||||
b.WriteString(fmt.Sprintf("### Declined by the ledger (%d)\n\n", len(r.Suppressed)))
|
||||
if len(r.Suppressed) == 0 {
|
||||
b.WriteString("Nothing the heuristics found was already on the ledger.\n\n")
|
||||
} else {
|
||||
b.WriteString("Candidates the heuristics found and the ledger has already settled. They are listed so the ledger's effect stays visible rather than silently shrinking the job's output.\n\n")
|
||||
for _, s := range r.Suppressed {
|
||||
b.WriteString(fmt.Sprintf("- `%s` + `%s`: %s\n", s.A, s.B, s.Reason))
|
||||
}
|
||||
b.WriteString("\n")
|
||||
}
|
||||
|
||||
if len(r.AliasSkipped) > 0 {
|
||||
b.WriteString(fmt.Sprintf("### Aliases, not variants (%d)\n\n", len(r.AliasSkipped)))
|
||||
b.WriteString("These entries install byte for byte the same payload. An alias exists so clients can send a particular name; folding it under another entry would hide that name.\n\n")
|
||||
for _, s := range r.AliasSkipped {
|
||||
b.WriteString(fmt.Sprintf("- `%s` + `%s`: %s\n", s.A, s.B, s.Reason))
|
||||
}
|
||||
b.WriteString("\n")
|
||||
}
|
||||
|
||||
if len(r.Refusals) > 0 {
|
||||
b.WriteString(fmt.Sprintf("### Found but refused (%d)\n\n", len(r.Refusals)))
|
||||
b.WriteString("Candidates the heuristics found but the authoring rules would not let this job write. They need a human edit or a rule change.\n\n")
|
||||
for _, ref := range r.Refusals {
|
||||
b.WriteString(fmt.Sprintf("- %s: %s\n", codeList(ref.Members), ref.Reason))
|
||||
}
|
||||
b.WriteString("\n")
|
||||
}
|
||||
|
||||
b.WriteString("---\n\nOpened by `.github/ci/variantproposals`. Heuristics and the rejection ledger live there and in the ledger file; a wrong proposal is a bug in one of the two.\n")
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func joinSignals(signals []Signal) string {
|
||||
if len(signals) == 0 {
|
||||
return "inferred through another member of the family"
|
||||
}
|
||||
out := make([]string, 0, len(signals))
|
||||
for _, s := range signals {
|
||||
out = append(out, "`"+string(s)+"`")
|
||||
}
|
||||
return strings.Join(out, ", ")
|
||||
}
|
||||
|
||||
func describeEvidence(e Evidence) string {
|
||||
var parts []string
|
||||
if e.SharedStem != "" {
|
||||
parts = append(parts, fmt.Sprintf("same name once quantization markers are stripped: `%s`", e.SharedStem))
|
||||
}
|
||||
if e.SharedFile != "" {
|
||||
parts = append(parts, fmt.Sprintf("same primary weight filename once quantization markers are stripped: `%s`", e.SharedFile))
|
||||
}
|
||||
if e.SharedRepo != "" {
|
||||
parts = append(parts, fmt.Sprintf("same upstream repo `%s`", e.SharedRepo))
|
||||
}
|
||||
if len(e.QuantTokens) > 0 {
|
||||
parts = append(parts, "differing quantization tokens: `"+strings.Join(e.QuantTokens, "`, `")+"`")
|
||||
}
|
||||
if len(parts) == 0 {
|
||||
return "reached this family through another member"
|
||||
}
|
||||
return strings.Join(parts, "; ")
|
||||
}
|
||||
|
||||
func codeList(names []string) string {
|
||||
out := make([]string, 0, len(names))
|
||||
for _, n := range names {
|
||||
out = append(out, "`"+n+"`")
|
||||
}
|
||||
return strings.Join(out, " + ")
|
||||
}
|
||||
|
||||
// RenderSummary is the terminal-facing digest of a run, so the workflow log
|
||||
// says what happened without anyone opening the pull request.
|
||||
func RenderSummary(r *Result) string {
|
||||
var b strings.Builder
|
||||
fmt.Fprintf(&b, "families proposed: %d\n", len(r.Families))
|
||||
for _, f := range r.Families {
|
||||
names := make([]string, 0, len(f.Proposals))
|
||||
for _, p := range f.Proposals {
|
||||
names = append(names, p.Variant)
|
||||
}
|
||||
fmt.Fprintf(&b, " %s <- %s\n", f.Parent, strings.Join(names, ", "))
|
||||
}
|
||||
fmt.Fprintf(&b, "declined by ledger: %d\n", len(r.Suppressed))
|
||||
for _, s := range r.Suppressed {
|
||||
fmt.Fprintf(&b, " %s\n", s)
|
||||
}
|
||||
fmt.Fprintf(&b, "aliases skipped: %d\n", len(r.AliasSkipped))
|
||||
for _, s := range r.AliasSkipped {
|
||||
fmt.Fprintf(&b, " %s\n", s)
|
||||
}
|
||||
fmt.Fprintf(&b, "refused: %d\n", len(r.Refusals))
|
||||
for _, ref := range r.Refusals {
|
||||
fmt.Fprintf(&b, " %s: %s\n", strings.Join(ref.Members, " + "), ref.Reason)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
120
.github/ci/variantproposals/edit.go
vendored
Normal file
120
.github/ci/variantproposals/edit.go
vendored
Normal file
@@ -0,0 +1,120 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var (
|
||||
inlineName = regexp.MustCompile(`^- (?:&\S+ )?name:`)
|
||||
keyName = regexp.MustCompile(`^ name:`)
|
||||
keyVariants = regexp.MustCompile(`^ variants:\s*(.*)$`)
|
||||
variantItem = regexp.MustCompile(`^ - `)
|
||||
unsafeInName = regexp.MustCompile(`[:#{}\[\],&*?|>'"%@` + "`" + `]|^\s|\s$`)
|
||||
)
|
||||
|
||||
// ApplyFamilies writes the proposed variant lists into the index text.
|
||||
//
|
||||
// The edit is textual on purpose. Re-serialising the index through a YAML
|
||||
// marshaller would reflow 40,000 lines, drop the anchors and merge keys the
|
||||
// gallery relies on, and produce a diff no reviewer could read, which would
|
||||
// make the pull request worthless even when the proposals inside it are right.
|
||||
func ApplyFamilies(ix *Index, families []Family) ([]string, error) {
|
||||
byName, _ := ix.ByName()
|
||||
|
||||
type edit struct {
|
||||
at int
|
||||
remove int
|
||||
insert []string
|
||||
ordinal int
|
||||
}
|
||||
var edits []edit
|
||||
|
||||
for _, f := range families {
|
||||
entry, ok := byName[strings.ToLower(f.Parent)]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("parent %q is not in the index", f.Parent)
|
||||
}
|
||||
items := make([]string, 0, len(f.Proposals))
|
||||
for _, p := range f.Proposals {
|
||||
items = append(items, " - model: "+quoteName(p.Variant))
|
||||
}
|
||||
|
||||
at, remove, err := insertionPoint(ix, entry)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
insert := items
|
||||
if remove > 0 || !hasVariantsKey(ix, entry) {
|
||||
insert = append([]string{" variants:"}, items...)
|
||||
}
|
||||
edits = append(edits, edit{at: at, remove: remove, insert: insert, ordinal: entry.Index})
|
||||
}
|
||||
|
||||
// Applying from the bottom up keeps every line number computed against the
|
||||
// original text valid while earlier edits are still pending.
|
||||
sort.Slice(edits, func(i, j int) bool { return edits[i].at > edits[j].at })
|
||||
|
||||
lines := append([]string(nil), ix.Lines...)
|
||||
for _, e := range edits {
|
||||
tail := append([]string(nil), lines[e.at+e.remove:]...)
|
||||
lines = append(lines[:e.at], append(append([]string(nil), e.insert...), tail...)...)
|
||||
}
|
||||
return lines, nil
|
||||
}
|
||||
|
||||
func hasVariantsKey(ix *Index, e *GalleryEntry) bool {
|
||||
for i := e.StartLine; i < e.EndLine; i++ {
|
||||
if keyVariants.MatchString(ix.Lines[i]) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// insertionPoint reports where new variant items belong, and how many existing
|
||||
// lines the insertion replaces.
|
||||
//
|
||||
// An entry with no variants key gets one right after its name, which is where
|
||||
// the hand-written families put it. An entry with an empty "variants: []" has
|
||||
// that line replaced by a block. An entry with a block gets its items appended.
|
||||
func insertionPoint(ix *Index, e *GalleryEntry) (at int, remove int, err error) {
|
||||
for i := e.StartLine; i < e.EndLine; i++ {
|
||||
m := keyVariants.FindStringSubmatch(ix.Lines[i])
|
||||
if m == nil {
|
||||
continue
|
||||
}
|
||||
if strings.TrimSpace(m[1]) == "[]" {
|
||||
return i, 1, nil
|
||||
}
|
||||
if strings.TrimSpace(m[1]) != "" {
|
||||
return 0, 0, fmt.Errorf("entry %q writes its variants inline (%q); this job only edits block lists", e.Name, strings.TrimSpace(m[1]))
|
||||
}
|
||||
last := i
|
||||
for j := i + 1; j < e.EndLine && variantItem.MatchString(ix.Lines[j]); j++ {
|
||||
last = j
|
||||
}
|
||||
return last + 1, 0, nil
|
||||
}
|
||||
|
||||
if inlineName.MatchString(ix.Lines[e.StartLine]) {
|
||||
return e.StartLine + 1, 0, nil
|
||||
}
|
||||
for i := e.StartLine; i < e.EndLine; i++ {
|
||||
if keyName.MatchString(ix.Lines[i]) {
|
||||
return i + 1, 0, nil
|
||||
}
|
||||
}
|
||||
return 0, 0, fmt.Errorf("entry %q has no name line to anchor the insertion to", e.Name)
|
||||
}
|
||||
|
||||
// quoteName quotes a variant reference when the name would otherwise change
|
||||
// meaning as bare YAML. Config-suffixed names carry a ":" and always need it.
|
||||
func quoteName(name string) string {
|
||||
if unsafeInName.MatchString(name) {
|
||||
return `"` + strings.ReplaceAll(name, `"`, `\"`) + `"`
|
||||
}
|
||||
return name
|
||||
}
|
||||
153
.github/ci/variantproposals/edit_test.go
vendored
Normal file
153
.github/ci/variantproposals/edit_test.go
vendored
Normal file
@@ -0,0 +1,153 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("ApplyFamilies", func() {
|
||||
apply := func(ix *Index, families []Family) []string {
|
||||
lines, err := ApplyFamilies(ix, families)
|
||||
ExpectWithOffset(1, err).ToNot(HaveOccurred())
|
||||
return lines
|
||||
}
|
||||
|
||||
// insertedLines is what a reviewer would see in the diff. A textual editor
|
||||
// that reflowed the file would show thousands here, which is the failure
|
||||
// this whole approach exists to avoid.
|
||||
insertedLines := func(before, after []string) int {
|
||||
remaining := map[string]int{}
|
||||
for _, l := range before {
|
||||
remaining[l]++
|
||||
}
|
||||
n := 0
|
||||
for _, l := range after {
|
||||
if remaining[l] > 0 {
|
||||
remaining[l]--
|
||||
continue
|
||||
}
|
||||
n++
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
It("adds a variants block right after the entry's name and touches nothing else", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("foo-model", "acme/repo", "foo-model-Q4_K_M.gguf", "aa"),
|
||||
entryYAML("foo-model-q8_0", "acme/repo", "foo-model-Q8_0.gguf", "bb"),
|
||||
)
|
||||
out := apply(ix, []Family{{Parent: "foo-model", Proposals: []Proposal{{Variant: "foo-model-q8_0"}}}})
|
||||
|
||||
Expect(out[0]).To(Equal("- name: foo-model"))
|
||||
Expect(out[1]).To(Equal(" variants:"))
|
||||
Expect(out[2]).To(Equal(" - model: foo-model-q8_0"))
|
||||
Expect(len(out)).To(Equal(len(ix.Lines) + 2))
|
||||
Expect(insertedLines(ix.Lines, out)).To(Equal(2))
|
||||
})
|
||||
|
||||
It("appends to a variants block that already exists", func() {
|
||||
ix := indexOf(`- name: partial
|
||||
variants:
|
||||
- model: partial-q8_0
|
||||
url: u
|
||||
overrides:
|
||||
parameters:
|
||||
model: partial-Q4_K_M.gguf
|
||||
`, entryYAML("partial-f16", "acme/repo", "partial-f16.gguf", "cc"))
|
||||
out := apply(ix, []Family{{Parent: "partial", Proposals: []Proposal{{Variant: "partial-f16"}}}})
|
||||
|
||||
Expect(out[1]).To(Equal(" variants:"))
|
||||
Expect(out[2]).To(Equal(" - model: partial-q8_0"))
|
||||
Expect(out[3]).To(Equal(" - model: partial-f16"))
|
||||
Expect(out[4]).To(Equal(" url: u"))
|
||||
})
|
||||
|
||||
It("replaces an explicit empty list rather than leaving two variants keys", func() {
|
||||
ix := indexOf(`- name: emptied
|
||||
variants: []
|
||||
url: u
|
||||
`, entryYAML("emptied-q8_0", "acme/repo", "emptied-Q8_0.gguf", "cc"))
|
||||
out := apply(ix, []Family{{Parent: "emptied", Proposals: []Proposal{{Variant: "emptied-q8_0"}}}})
|
||||
|
||||
Expect(strings.Join(out[:4], "\n")).To(Equal("- name: emptied\n variants:\n - model: emptied-q8_0\n url: u"))
|
||||
Expect(strings.Count(strings.Join(out, "\n"), "variants:")).To(Equal(1))
|
||||
})
|
||||
|
||||
It("quotes a config-suffixed name so the reference stays a string", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("phi-2-chat", "acme/repo", "phi-2-chat-Q4_K_M.gguf", "aa"),
|
||||
entryYAML("phi-2-chat:Q8_0", "acme/repo", "phi-2-chat-Q8_0.gguf", "bb"),
|
||||
)
|
||||
out := apply(ix, []Family{{Parent: "phi-2-chat", Proposals: []Proposal{{Variant: "phi-2-chat:Q8_0"}}}})
|
||||
Expect(out[2]).To(Equal(` - model: "phi-2-chat:Q8_0"`))
|
||||
|
||||
// The result has to still be a gallery, and the reference has to
|
||||
// resolve to the entry it names.
|
||||
reparsed, err := ParseIndex(strings.Join(out, "\n"))
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(reparsed.Entries[0].Variants).To(ConsistOf(VariantRef{Model: "phi-2-chat:Q8_0"}))
|
||||
})
|
||||
|
||||
It("keeps line numbers correct when several entries are edited at once", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("alpha", "acme/repo", "alpha-Q4_K_M.gguf", "aa"),
|
||||
entryYAML("alpha-q8_0", "acme/repo", "alpha-Q8_0.gguf", "bb"),
|
||||
entryYAML("beta", "acme/repo", "beta-Q4_K_M.gguf", "cc"),
|
||||
entryYAML("beta-q8_0", "acme/repo", "beta-Q8_0.gguf", "dd"),
|
||||
)
|
||||
out := apply(ix, []Family{
|
||||
{Parent: "alpha", Proposals: []Proposal{{Variant: "alpha-q8_0"}}},
|
||||
{Parent: "beta", Proposals: []Proposal{{Variant: "beta-q8_0"}}},
|
||||
})
|
||||
|
||||
reparsed, err := ParseIndex(strings.Join(out, "\n"))
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(reparsed.Entries).To(HaveLen(4))
|
||||
Expect(reparsed.Entries[0].Variants).To(ConsistOf(VariantRef{Model: "alpha-q8_0"}))
|
||||
Expect(reparsed.Entries[2].Variants).To(ConsistOf(VariantRef{Model: "beta-q8_0"}))
|
||||
Expect(reparsed.Entries[1].Variants).To(BeEmpty())
|
||||
Expect(reparsed.Entries[3].Variants).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("fails loudly rather than editing an entry it cannot find", func() {
|
||||
ix := indexOf(entryYAML("only", "acme/repo", "only-Q4_K_M.gguf", "aa"))
|
||||
_, err := ApplyFamilies(ix, []Family{{Parent: "missing", Proposals: []Proposal{{Variant: "x"}}}})
|
||||
Expect(err).To(MatchError(ContainSubstring("not in the index")))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("ParseIndex", func() {
|
||||
It("records the anchor an entry defines and the anchor an entry merges", func() {
|
||||
ix := indexOf(`- &anc
|
||||
name: anchored
|
||||
url: u
|
||||
`, `- !!merge <<: *anc
|
||||
name: child
|
||||
`)
|
||||
Expect(ix.Entries[0].AnchorName).To(Equal("anc"))
|
||||
Expect(ix.Entries[1].MergesFrom).To(Equal("anc"))
|
||||
Expect(ix.MergeChildren("anc")).To(HaveLen(1))
|
||||
})
|
||||
|
||||
It("carries merged values into the child, so an inherited variants key is visible", func() {
|
||||
ix := indexOf(`- &anc
|
||||
name: anchored
|
||||
url: u
|
||||
variants:
|
||||
- model: something
|
||||
`, `- !!merge <<: *anc
|
||||
name: child
|
||||
`)
|
||||
Expect(ix.Entries[1].HasVariants()).To(BeTrue())
|
||||
})
|
||||
|
||||
It("refuses a list item that decodes to nothing", func() {
|
||||
// Every line number the editor works from comes from pairing decoded
|
||||
// entries with top level list items. If those two views can disagree,
|
||||
// the editor writes into the wrong entry, so the parse refuses instead.
|
||||
_, err := ParseIndex("- name: one\n url: u\n-\n")
|
||||
Expect(err).To(MatchError(ContainSubstring("empty")))
|
||||
})
|
||||
})
|
||||
286
.github/ci/variantproposals/index.go
vendored
Normal file
286
.github/ci/variantproposals/index.go
vendored
Normal file
@@ -0,0 +1,286 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
// File is the subset of a gallery file entry the proposer reads.
|
||||
type File struct {
|
||||
Filename string `yaml:"filename"`
|
||||
URI string `yaml:"uri"`
|
||||
SHA256 string `yaml:"sha256"`
|
||||
}
|
||||
|
||||
// VariantRef mirrors the gallery's variant reference.
|
||||
type VariantRef struct {
|
||||
Model string `yaml:"model"`
|
||||
}
|
||||
|
||||
// GalleryEntry is one gallery entry, carrying both the semantics the heuristics need
|
||||
// and the text range the editor needs.
|
||||
//
|
||||
// The two views are kept together deliberately. The editor must not round-trip
|
||||
// the index through a YAML marshaller: the gallery is 40,000 lines and a
|
||||
// reflowed diff cannot be reviewed, which defeats the entire point of a job
|
||||
// whose output is a human decision.
|
||||
type GalleryEntry struct {
|
||||
Name string `yaml:"name"`
|
||||
URL string `yaml:"url"`
|
||||
ConfigFile map[string]any `yaml:"config_file"`
|
||||
Overrides map[string]any `yaml:"overrides"`
|
||||
Files []File `yaml:"files"`
|
||||
Variants []VariantRef `yaml:"variants"`
|
||||
|
||||
// Index is the entry's position in gallery order.
|
||||
Index int `yaml:"-"`
|
||||
// StartLine and EndLine bound the entry's lines, zero based and half open.
|
||||
StartLine int `yaml:"-"`
|
||||
EndLine int `yaml:"-"`
|
||||
// AnchorName is set when the entry defines a YAML anchor. Adding a variants
|
||||
// key to such an entry is inherited by everything that merges it, which is
|
||||
// why proposals involving anchors get special treatment.
|
||||
AnchorName string `yaml:"-"`
|
||||
// MergesFrom is the anchor this entry pulls in with "!!merge <<:".
|
||||
MergesFrom string `yaml:"-"`
|
||||
}
|
||||
|
||||
// Index is a parsed gallery index: entries plus the exact lines they came from.
|
||||
type Index struct {
|
||||
Lines []string
|
||||
Entries []*GalleryEntry
|
||||
}
|
||||
|
||||
var (
|
||||
entryStart = regexp.MustCompile(`^-(?: |$)`)
|
||||
anchorStart = regexp.MustCompile(`^- &(\S+)`)
|
||||
mergeStart = regexp.MustCompile(`^- !!merge <<: \*(\S+)`)
|
||||
)
|
||||
|
||||
// LoadIndex reads and parses a gallery index file.
|
||||
func LoadIndex(path string) (*Index, error) {
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ParseIndex(string(data))
|
||||
}
|
||||
|
||||
// ParseIndex builds an Index from the raw text of a gallery index.
|
||||
//
|
||||
// The YAML decode and the textual scan are cross checked against each other: if
|
||||
// they disagree on how many entries there are, every line number the editor
|
||||
// would use is suspect, so the run fails rather than editing the wrong entry.
|
||||
func ParseIndex(text string) (*Index, error) {
|
||||
var entries []*GalleryEntry
|
||||
if err := yaml.Unmarshal([]byte(text), &entries); err != nil {
|
||||
return nil, fmt.Errorf("decoding gallery index: %w", err)
|
||||
}
|
||||
|
||||
lines := strings.Split(text, "\n")
|
||||
var starts []int
|
||||
for i, line := range lines {
|
||||
if entryStart.MatchString(line) {
|
||||
starts = append(starts, i)
|
||||
}
|
||||
}
|
||||
if len(starts) != len(entries) {
|
||||
return nil, fmt.Errorf("gallery index has %d decoded entries but %d top level list items; refusing to edit by line number", len(entries), len(starts))
|
||||
}
|
||||
|
||||
for i, e := range entries {
|
||||
if e == nil {
|
||||
return nil, fmt.Errorf("gallery index list item %d is empty; refusing to edit by line number", i)
|
||||
}
|
||||
e.Index = i
|
||||
e.StartLine = starts[i]
|
||||
if i+1 < len(starts) {
|
||||
e.EndLine = starts[i+1]
|
||||
} else {
|
||||
e.EndLine = len(lines)
|
||||
}
|
||||
if m := anchorStart.FindStringSubmatch(lines[e.StartLine]); m != nil {
|
||||
e.AnchorName = m[1]
|
||||
}
|
||||
if m := mergeStart.FindStringSubmatch(lines[e.StartLine]); m != nil {
|
||||
e.MergesFrom = m[1]
|
||||
}
|
||||
}
|
||||
|
||||
return &Index{Lines: lines, Entries: entries}, nil
|
||||
}
|
||||
|
||||
// MergeChildren lists the entries that pull in the given anchor.
|
||||
func (ix *Index) MergeChildren(anchor string) []*GalleryEntry {
|
||||
var out []*GalleryEntry
|
||||
for _, e := range ix.Entries {
|
||||
if e.MergesFrom == anchor {
|
||||
out = append(out, e)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ByName indexes entries by lowercased name. A name appearing twice keeps the
|
||||
// first occurrence, matching the gallery's own first-match-wins resolution, and
|
||||
// the duplicates are returned so the caller can refuse to touch them: a
|
||||
// proposal naming an ambiguous entry cannot be reviewed.
|
||||
func (ix *Index) ByName() (map[string]*GalleryEntry, map[string]int) {
|
||||
byName := make(map[string]*GalleryEntry, len(ix.Entries))
|
||||
counts := make(map[string]int, len(ix.Entries))
|
||||
for _, e := range ix.Entries {
|
||||
key := strings.ToLower(e.Name)
|
||||
counts[key]++
|
||||
if _, seen := byName[key]; !seen {
|
||||
byName[key] = e
|
||||
}
|
||||
}
|
||||
dupes := map[string]int{}
|
||||
for name, n := range counts {
|
||||
if n > 1 {
|
||||
dupes[name] = n
|
||||
}
|
||||
}
|
||||
return byName, dupes
|
||||
}
|
||||
|
||||
// Installable reports whether installing this entry would put anything on disk.
|
||||
// A variant target that installs nothing is a dead end for the selector, so it
|
||||
// is never proposed as one.
|
||||
func (e *GalleryEntry) Installable() bool {
|
||||
return e.URL != "" || len(e.ConfigFile) > 0 || len(e.Overrides) > 0 || len(e.Files) > 0
|
||||
}
|
||||
|
||||
// HasVariants reports whether the entry already offers builds of its own. Such
|
||||
// an entry cannot be a variant target: nesting is what the gallery's own
|
||||
// resolution refuses.
|
||||
func (e *GalleryEntry) HasVariants() bool {
|
||||
return len(e.Variants) > 0
|
||||
}
|
||||
|
||||
// auxiliaryFile matches the shared side files that several unrelated models
|
||||
// legitimately hand out the same copy of. Grouping on one of these is how an
|
||||
// earlier sweep linked four wan-2.1 entries to each other and Z-Image-Turbo to
|
||||
// qwen3-4b: they shared a text encoder, not weights.
|
||||
var auxiliaryFile = regexp.MustCompile(`(?i)(mmproj|vae|clip|t5|umt5|text_?encoder|tokenizer|\bae\b|^ae\.|scheduler|config)`)
|
||||
|
||||
// IsAuxiliaryFile reports whether a filename is a side file rather than the
|
||||
// model's own weights.
|
||||
func IsAuxiliaryFile(filename string) bool {
|
||||
base := filename
|
||||
if i := strings.LastIndex(base, "/"); i >= 0 {
|
||||
base = base[i+1:]
|
||||
}
|
||||
return auxiliaryFile.MatchString(base)
|
||||
}
|
||||
|
||||
// PrimaryWeightFile returns the filename of the entry's own weights, and
|
||||
// whether one could be identified unambiguously.
|
||||
//
|
||||
// The declared overrides.parameters.model wins because that is the file the
|
||||
// backend is actually pointed at. Falling back to the file list only works when
|
||||
// exactly one non-auxiliary file is present; anything else is ambiguous, and
|
||||
// guessing is precisely the failure mode this heuristic has already had.
|
||||
func (e *GalleryEntry) PrimaryWeightFile() (string, bool) {
|
||||
if params, ok := e.Overrides["parameters"].(map[string]any); ok {
|
||||
if model, ok := params["model"].(string); ok && model != "" && !IsAuxiliaryFile(model) {
|
||||
return model, true
|
||||
}
|
||||
}
|
||||
var candidates []string
|
||||
for _, f := range e.Files {
|
||||
if f.Filename == "" || IsAuxiliaryFile(f.Filename) {
|
||||
continue
|
||||
}
|
||||
candidates = append(candidates, f.Filename)
|
||||
}
|
||||
if len(candidates) == 1 {
|
||||
return candidates[0], true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// SourceRepo returns the upstream repository the entry's files come from, as a
|
||||
// coarse "host + owner + repo" key.
|
||||
func (e *GalleryEntry) SourceRepo() string {
|
||||
for _, f := range e.Files {
|
||||
if f.URI == "" {
|
||||
continue
|
||||
}
|
||||
return repoKey(f.URI)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func repoKey(uri string) string {
|
||||
u := strings.ToLower(uri)
|
||||
u = strings.TrimPrefix(u, "huggingface://")
|
||||
u = strings.TrimPrefix(u, "https://huggingface.co/")
|
||||
u = strings.TrimPrefix(u, "http://huggingface.co/")
|
||||
parts := strings.Split(u, "/")
|
||||
if len(parts) >= 2 {
|
||||
return parts[0] + "/" + parts[1]
|
||||
}
|
||||
return u
|
||||
}
|
||||
|
||||
// SameInstallPayload reports whether two entries install byte for byte the same
|
||||
// thing.
|
||||
//
|
||||
// Entries like this are aliases, not variants. whisper-1 exists so a client
|
||||
// speaking the OpenAI API can send that name and get whisper-base; folding it
|
||||
// under whisper-base as a variant would hide the very name clients send.
|
||||
func SameInstallPayload(a, b *GalleryEntry) bool {
|
||||
if a.URL != b.URL {
|
||||
return false
|
||||
}
|
||||
if !sameYAML(a.Overrides, b.Overrides) || !sameYAML(a.ConfigFile, b.ConfigFile) {
|
||||
return false
|
||||
}
|
||||
return sameChecksums(a.Files, b.Files)
|
||||
}
|
||||
|
||||
func sameChecksums(a, b []File) bool {
|
||||
if len(a) != len(b) || len(a) == 0 {
|
||||
return false
|
||||
}
|
||||
ha := make([]string, 0, len(a))
|
||||
hb := make([]string, 0, len(b))
|
||||
for _, f := range a {
|
||||
if f.SHA256 == "" {
|
||||
return false
|
||||
}
|
||||
ha = append(ha, f.SHA256)
|
||||
}
|
||||
for _, f := range b {
|
||||
if f.SHA256 == "" {
|
||||
return false
|
||||
}
|
||||
hb = append(hb, f.SHA256)
|
||||
}
|
||||
sort.Strings(ha)
|
||||
sort.Strings(hb)
|
||||
for i := range ha {
|
||||
if ha[i] != hb[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func sameYAML(a, b any) bool {
|
||||
ba, err := yaml.Marshal(a)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
bb, err := yaml.Marshal(b)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
return string(ba) == string(bb)
|
||||
}
|
||||
180
.github/ci/variantproposals/ledger.go
vendored
Normal file
180
.github/ci/variantproposals/ledger.go
vendored
Normal file
@@ -0,0 +1,180 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
// Ledger records the grouping decisions a human has already made against the
|
||||
// proposer, so a declined candidate stays declined instead of coming back every
|
||||
// night until reviewers stop reading the job's pull requests.
|
||||
//
|
||||
// It is checked in next to the gallery and is meant to be edited inside the
|
||||
// proposal pull request itself: declining a family is adding one flow-mapping
|
||||
// line under pairs or groups and closing the PR.
|
||||
type Ledger struct {
|
||||
// Tokens are name segments that mark a distinct model rather than another
|
||||
// build of the same one: finetune names, language codes, product suffixes.
|
||||
// A candidate whose two names differ by any of these is never proposed.
|
||||
Tokens []LedgerToken `yaml:"tokens"`
|
||||
// Pairs are individual candidates a human considered and declined. Order
|
||||
// does not matter: the pair is matched both ways round.
|
||||
Pairs []LedgerPair `yaml:"pairs"`
|
||||
// Groups decline every pair drawn from a set at once, for families like a
|
||||
// per-language release where listing each pair would be unreadable.
|
||||
Groups []LedgerGroup `yaml:"groups"`
|
||||
}
|
||||
|
||||
type LedgerToken struct {
|
||||
Token string `yaml:"token"`
|
||||
Reason string `yaml:"reason"`
|
||||
}
|
||||
|
||||
type LedgerPair struct {
|
||||
Parent string `yaml:"parent"`
|
||||
Variant string `yaml:"variant"`
|
||||
Reason string `yaml:"reason"`
|
||||
}
|
||||
|
||||
type LedgerGroup struct {
|
||||
Members []string `yaml:"members"`
|
||||
Reason string `yaml:"reason"`
|
||||
}
|
||||
|
||||
// LoadLedger reads a ledger file. A missing file is not an error: a gallery
|
||||
// that has declined nothing yet is a legitimate state, and failing the job over
|
||||
// it would only teach people to keep an empty file around.
|
||||
func LoadLedger(path string) (*Ledger, error) {
|
||||
data, err := os.ReadFile(path)
|
||||
if os.IsNotExist(err) {
|
||||
return &Ledger{}, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ParseLedger(data)
|
||||
}
|
||||
|
||||
func ParseLedger(data []byte) (*Ledger, error) {
|
||||
l := &Ledger{}
|
||||
if err := yaml.Unmarshal(data, l); err != nil {
|
||||
return nil, fmt.Errorf("parsing ledger: %w", err)
|
||||
}
|
||||
for i, t := range l.Tokens {
|
||||
if strings.TrimSpace(t.Token) == "" {
|
||||
return nil, fmt.Errorf("ledger tokens[%d] has an empty token", i)
|
||||
}
|
||||
}
|
||||
for i, p := range l.Pairs {
|
||||
if strings.TrimSpace(p.Parent) == "" || strings.TrimSpace(p.Variant) == "" {
|
||||
return nil, fmt.Errorf("ledger pairs[%d] needs both parent and variant", i)
|
||||
}
|
||||
}
|
||||
return l, nil
|
||||
}
|
||||
|
||||
// Suppression is a ledger hit: why a candidate was not proposed, in words a
|
||||
// reviewer can check against the ledger file.
|
||||
type Suppression struct {
|
||||
A string
|
||||
B string
|
||||
Reason string
|
||||
}
|
||||
|
||||
func (s Suppression) String() string {
|
||||
return fmt.Sprintf("%s + %s: %s", s.A, s.B, s.Reason)
|
||||
}
|
||||
|
||||
// Suppresses reports whether the ledger has already declined pairing these two
|
||||
// entries, and why.
|
||||
//
|
||||
// The token rule is applied to the segments the two names do not share. Two
|
||||
// builds of the same weights differ only in quantization markers, so any
|
||||
// ledgered token showing up in that difference is by construction a claim that
|
||||
// the entries are different models.
|
||||
func (l *Ledger) Suppresses(a, b string) (Suppression, bool) {
|
||||
la, lb := strings.ToLower(a), strings.ToLower(b)
|
||||
for _, p := range l.Pairs {
|
||||
lp, lv := strings.ToLower(p.Parent), strings.ToLower(p.Variant)
|
||||
if (lp == la && lv == lb) || (lp == lb && lv == la) {
|
||||
return Suppression{A: a, B: b, Reason: p.Reason}, true
|
||||
}
|
||||
}
|
||||
for _, g := range l.Groups {
|
||||
var seenA, seenB bool
|
||||
for _, m := range g.Members {
|
||||
lm := strings.ToLower(m)
|
||||
if lm == la {
|
||||
seenA = true
|
||||
}
|
||||
if lm == lb {
|
||||
seenB = true
|
||||
}
|
||||
}
|
||||
if seenA && seenB {
|
||||
return Suppression{A: a, B: b, Reason: g.Reason}, true
|
||||
}
|
||||
}
|
||||
diff := differingSegments(la, lb)
|
||||
for _, t := range l.Tokens {
|
||||
token := strings.ToLower(strings.TrimSpace(t.Token))
|
||||
if _, ok := diff[token]; ok {
|
||||
reason := t.Reason
|
||||
if reason == "" {
|
||||
reason = fmt.Sprintf("names differ by %q", token)
|
||||
}
|
||||
return Suppression{A: a, B: b, Reason: fmt.Sprintf("%s (token %q)", reason, token)}, true
|
||||
}
|
||||
}
|
||||
return Suppression{}, false
|
||||
}
|
||||
|
||||
// segments splits a name into the atoms the token rules are written against.
|
||||
func segments(name string) []string {
|
||||
fields := strings.FieldsFunc(strings.ToLower(name), func(r rune) bool {
|
||||
return r == '-' || r == '_' || r == '.' || r == ':' || r == '/'
|
||||
})
|
||||
return fields
|
||||
}
|
||||
|
||||
// differingSegments returns the set of segments present in exactly one of the
|
||||
// two names.
|
||||
func differingSegments(a, b string) map[string]struct{} {
|
||||
setA := map[string]int{}
|
||||
for _, s := range segments(a) {
|
||||
setA[s]++
|
||||
}
|
||||
setB := map[string]int{}
|
||||
for _, s := range segments(b) {
|
||||
setB[s]++
|
||||
}
|
||||
diff := map[string]struct{}{}
|
||||
for s := range setA {
|
||||
if setB[s] == 0 {
|
||||
diff[s] = struct{}{}
|
||||
}
|
||||
}
|
||||
for s := range setB {
|
||||
if setA[s] == 0 {
|
||||
diff[s] = struct{}{}
|
||||
}
|
||||
}
|
||||
return diff
|
||||
}
|
||||
|
||||
// SortedSuppressions gives the ledger's effect on one run in a stable order, so
|
||||
// the pull request body reads the same way for the same gallery.
|
||||
func SortedSuppressions(in []Suppression) []Suppression {
|
||||
out := append([]Suppression(nil), in...)
|
||||
sort.Slice(out, func(i, j int) bool {
|
||||
if out[i].A != out[j].A {
|
||||
return out[i].A < out[j].A
|
||||
}
|
||||
return out[i].B < out[j].B
|
||||
})
|
||||
return out
|
||||
}
|
||||
65
.github/ci/variantproposals/main.go
vendored
Normal file
65
.github/ci/variantproposals/main.go
vendored
Normal file
@@ -0,0 +1,65 @@
|
||||
// Command variant-proposals looks for gallery entries that are alternative
|
||||
// builds of the same weights but are not grouped under one another, and writes
|
||||
// a proposal for a human to accept or reject.
|
||||
//
|
||||
// It never decides. Grouping has gone wrong repeatedly in both directions, so
|
||||
// the job's value is catching drift and surfacing candidates with their
|
||||
// evidence, not automating the call. The scheduled workflow feeds its output to
|
||||
// a pull request in the same shape as .github/checksum_checker.sh.
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func main() {
|
||||
index := flag.String("index", "gallery/index.yaml", "path to the gallery index")
|
||||
ledger := flag.String("ledger", "gallery/variant-exclusions.yaml", "path to the rejection ledger")
|
||||
bodyOut := flag.String("body-out", "", "write the pull request body here")
|
||||
apply := flag.Bool("apply", false, "write the proposed groupings back into the index")
|
||||
flag.Parse()
|
||||
|
||||
if err := run(*index, *ledger, *bodyOut, *apply); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "variant-proposals:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func run(indexPath, ledgerPath, bodyOut string, apply bool) error {
|
||||
ix, err := LoadIndex(indexPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
ledger, err := LoadLedger(ledgerPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
result := Propose(ix, ledger)
|
||||
fmt.Print(RenderSummary(result))
|
||||
|
||||
if !result.HasProposals() {
|
||||
// An empty pull request every night is how a proposal job gets muted.
|
||||
fmt.Println("nothing to propose")
|
||||
return nil
|
||||
}
|
||||
|
||||
if bodyOut != "" {
|
||||
if err := os.WriteFile(bodyOut, []byte(RenderBody(result, ledgerPath)), 0o644); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if !apply {
|
||||
return nil
|
||||
}
|
||||
|
||||
lines, err := ApplyFamilies(ix, result.Families)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return os.WriteFile(indexPath, []byte(strings.Join(lines, "\n")), 0o644)
|
||||
}
|
||||
615
.github/ci/variantproposals/propose.go
vendored
Normal file
615
.github/ci/variantproposals/propose.go
vendored
Normal file
@@ -0,0 +1,615 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Signal names the grouping heuristic that linked two entries.
|
||||
type Signal string
|
||||
|
||||
const (
|
||||
// SignalName is "same name once quantization markers are stripped".
|
||||
SignalName Signal = "name-modulo-quant"
|
||||
// SignalConfigSuffix is the ":" convention, foo:q8_0 as a build of foo.
|
||||
SignalConfigSuffix Signal = "config-suffix"
|
||||
// SignalWeightFile is "same primary weight filename once quantization
|
||||
// markers are stripped", auxiliary files excluded.
|
||||
SignalWeightFile Signal = "weight-filename"
|
||||
)
|
||||
|
||||
// Evidence is what a reviewer needs in order to agree or disagree without
|
||||
// opening HuggingFace: what the two entries share, and what differs.
|
||||
type Evidence struct {
|
||||
Signals []Signal
|
||||
SharedStem string
|
||||
SharedFile string
|
||||
SharedRepo string
|
||||
QuantTokens []string
|
||||
}
|
||||
|
||||
// Proposal is one variant target offered to one parent.
|
||||
type Proposal struct {
|
||||
Variant string
|
||||
Evidence Evidence
|
||||
}
|
||||
|
||||
// Family is a complete proposal: one parent gaining one or more variants.
|
||||
type Family struct {
|
||||
Parent string
|
||||
Proposals []Proposal
|
||||
}
|
||||
|
||||
// Refusal is a family the heuristics found but the rules would not let through.
|
||||
// Refusals are reported rather than dropped: a candidate the job keeps refusing
|
||||
// is either a rule worth revisiting or a gallery bug worth fixing.
|
||||
type Refusal struct {
|
||||
Members []string
|
||||
Reason string
|
||||
}
|
||||
|
||||
// Result is one run of the proposer.
|
||||
type Result struct {
|
||||
Families []Family
|
||||
Refusals []Refusal
|
||||
Suppressed []Suppression
|
||||
AliasSkipped []Suppression
|
||||
}
|
||||
|
||||
// HasProposals reports whether the run found anything to open a pull request
|
||||
// about. A job that opens an empty pull request every night is a job people
|
||||
// filter out of their inbox.
|
||||
func (r *Result) HasProposals() bool {
|
||||
return len(r.Families) > 0
|
||||
}
|
||||
|
||||
// sizeToken matches a parameter-count marker: 8b, 1.7b, a3b for an active
|
||||
// expert count, e2b for the Gemma effective sizes, 8x7b for a mixture.
|
||||
//
|
||||
// This is a structural rule rather than a ledger entry because it is about the
|
||||
// shape of the token, not about any one model. Different parameter sizes were
|
||||
// mis-grouped by an earlier sweep and the failure is systematic.
|
||||
var sizeToken = regexp.MustCompile(`^(?:[0-9]+(?:\.[0-9]+)?[bm]|[ae][0-9]+(?:\.[0-9]+)?b|[0-9]+x[0-9]+(?:\.[0-9]+)?b)$`)
|
||||
|
||||
func differsByParameterSize(a, b string) (string, bool) {
|
||||
for seg := range differingSegments(a, b) {
|
||||
if sizeToken.MatchString(seg) {
|
||||
return seg, true
|
||||
}
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// genericFileStem lists weight filenames too generic to be evidence of
|
||||
// anything. Two entries both shipping "model.safetensors" share a convention,
|
||||
// not a set of weights.
|
||||
var genericFileStem = map[string]struct{}{
|
||||
"model": {}, "weights": {}, "pytorch_model": {}, "diffusion_pytorch_model": {},
|
||||
"consolidated": {}, "ggml-model": {}, "model-00001-of-00002": {},
|
||||
}
|
||||
|
||||
// minFileStemLength keeps short, collision-prone filename stems from linking
|
||||
// unrelated entries.
|
||||
const minFileStemLength = 6
|
||||
|
||||
type pair struct {
|
||||
a, b int
|
||||
evidence Evidence
|
||||
}
|
||||
|
||||
// Propose runs the grouping heuristics over a gallery index and returns what it
|
||||
// would offer a human, what it refused, and what the ledger silenced.
|
||||
//
|
||||
// Nothing here touches the network or git, and the index is not modified.
|
||||
func Propose(ix *Index, ledger *Ledger) *Result {
|
||||
if ledger == nil {
|
||||
ledger = &Ledger{}
|
||||
}
|
||||
result := &Result{}
|
||||
|
||||
byName, dupes := ix.ByName()
|
||||
|
||||
// Existing relationships. A target already claimed must not be claimed
|
||||
// again, and two entries already in one family need no proposal.
|
||||
claimedBy := map[string]string{}
|
||||
familyOf := map[string]string{}
|
||||
for _, e := range ix.Entries {
|
||||
if !e.HasVariants() {
|
||||
continue
|
||||
}
|
||||
familyOf[strings.ToLower(e.Name)] = strings.ToLower(e.Name)
|
||||
for _, v := range e.Variants {
|
||||
target := strings.ToLower(v.Model)
|
||||
if _, taken := claimedBy[target]; !taken {
|
||||
claimedBy[target] = strings.ToLower(e.Name)
|
||||
}
|
||||
familyOf[target] = strings.ToLower(e.Name)
|
||||
}
|
||||
}
|
||||
|
||||
candidates := map[[2]int]*Evidence{}
|
||||
|
||||
addPair := func(i, j int, sig Signal, apply func(*Evidence)) {
|
||||
if i == j {
|
||||
return
|
||||
}
|
||||
if i > j {
|
||||
i, j = j, i
|
||||
}
|
||||
key := [2]int{i, j}
|
||||
ev, ok := candidates[key]
|
||||
if !ok {
|
||||
ev = &Evidence{}
|
||||
candidates[key] = ev
|
||||
}
|
||||
for _, s := range ev.Signals {
|
||||
if s == sig {
|
||||
apply(ev)
|
||||
return
|
||||
}
|
||||
}
|
||||
ev.Signals = append(ev.Signals, sig)
|
||||
apply(ev)
|
||||
}
|
||||
|
||||
// Signal 1 and 2: entries sharing a name stem.
|
||||
byStem := map[string][]int{}
|
||||
for _, e := range ix.Entries {
|
||||
if e.Name == "" {
|
||||
continue
|
||||
}
|
||||
byStem[NameStem(e.Name)] = append(byStem[NameStem(e.Name)], e.Index)
|
||||
}
|
||||
for stem, members := range byStem {
|
||||
if len(members) < 2 {
|
||||
continue
|
||||
}
|
||||
for i := 0; i < len(members); i++ {
|
||||
for j := i + 1; j < len(members); j++ {
|
||||
a, b := ix.Entries[members[i]], ix.Entries[members[j]]
|
||||
sig := SignalName
|
||||
if HasConfigSuffix(a.Name) || HasConfigSuffix(b.Name) {
|
||||
sig = SignalConfigSuffix
|
||||
}
|
||||
// The bare parent carries no marker in its name, so the
|
||||
// evidence would read "differs by q8_0" and say nothing about
|
||||
// what the parent is. The weight filenames fill that in.
|
||||
fa, _ := a.PrimaryWeightFile()
|
||||
fb, _ := b.PrimaryWeightFile()
|
||||
addPair(members[i], members[j], sig, func(ev *Evidence) {
|
||||
ev.SharedStem = stem
|
||||
ev.QuantTokens = quantDifference(a.Name, b.Name, fa, fb)
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Signal 3: entries whose own weight file is the same file at a different
|
||||
// quantization. Auxiliary files never take part.
|
||||
byFile := map[string][]int{}
|
||||
for _, e := range ix.Entries {
|
||||
primary, ok := e.PrimaryWeightFile()
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
stem := FileStem(primary)
|
||||
if len(stem) < minFileStemLength {
|
||||
continue
|
||||
}
|
||||
if _, generic := genericFileStem[stem]; generic {
|
||||
continue
|
||||
}
|
||||
byFile[stem] = append(byFile[stem], e.Index)
|
||||
}
|
||||
for stem, members := range byFile {
|
||||
if len(members) < 2 {
|
||||
continue
|
||||
}
|
||||
for i := 0; i < len(members); i++ {
|
||||
for j := i + 1; j < len(members); j++ {
|
||||
a, b := ix.Entries[members[i]], ix.Entries[members[j]]
|
||||
// The filename alone is not evidence. Publishers reuse the
|
||||
// upstream filename for finetunes and for models that merely
|
||||
// embed the base weights: bert-embeddings, an ultravox audio
|
||||
// model and a roleplay finetune all ship a file called
|
||||
// llama-3.2-1b-instruct-q4_k_m.gguf. Requiring the same
|
||||
// upstream repository turns the signal back into what it
|
||||
// claims to be, one repo publishing one file at two
|
||||
// quantizations. Two repos holding the same weights is a fact
|
||||
// no filename proves, so it stays a human call.
|
||||
repo := a.SourceRepo()
|
||||
if repo == "" || repo != b.SourceRepo() {
|
||||
continue
|
||||
}
|
||||
fa, _ := a.PrimaryWeightFile()
|
||||
fb, _ := b.PrimaryWeightFile()
|
||||
addPair(members[i], members[j], SignalWeightFile, func(ev *Evidence) {
|
||||
ev.SharedFile = stem
|
||||
ev.SharedRepo = repo
|
||||
if len(ev.QuantTokens) == 0 {
|
||||
ev.QuantTokens = quantDifference(fa, fb)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Filter candidates. Everything dropped here is dropped for a reason a
|
||||
// reviewer can read back off the ledger or the rules.
|
||||
var kept []pair
|
||||
for key, ev := range candidates {
|
||||
a, b := ix.Entries[key[0]], ix.Entries[key[1]]
|
||||
la, lb := strings.ToLower(a.Name), strings.ToLower(b.Name)
|
||||
if la == lb {
|
||||
continue
|
||||
}
|
||||
if dupes[la] > 0 || dupes[lb] > 0 {
|
||||
result.Refusals = append(result.Refusals, Refusal{
|
||||
Members: []string{a.Name, b.Name},
|
||||
Reason: "one of these names appears more than once in the gallery, so a variant reference to it is ambiguous",
|
||||
})
|
||||
continue
|
||||
}
|
||||
if fa, fb := familyOf[la], familyOf[lb]; fa != "" && fa == fb {
|
||||
continue
|
||||
}
|
||||
if seg, differs := differsByParameterSize(la, lb); differs {
|
||||
result.Suppressed = append(result.Suppressed, Suppression{
|
||||
A: a.Name, B: b.Name, Reason: fmt.Sprintf("different parameter sizes (segment %q)", seg),
|
||||
})
|
||||
continue
|
||||
}
|
||||
if s, ok := ledger.Suppresses(a.Name, b.Name); ok {
|
||||
result.Suppressed = append(result.Suppressed, s)
|
||||
continue
|
||||
}
|
||||
if SameInstallPayload(a, b) {
|
||||
result.AliasSkipped = append(result.AliasSkipped, Suppression{
|
||||
A: a.Name, B: b.Name,
|
||||
Reason: "identical install payload; these are aliases of one build, not alternative builds",
|
||||
})
|
||||
continue
|
||||
}
|
||||
kept = append(kept, pair{a: key[0], b: key[1], evidence: *ev})
|
||||
}
|
||||
|
||||
sort.Slice(kept, func(i, j int) bool {
|
||||
if kept[i].a != kept[j].a {
|
||||
return kept[i].a < kept[j].a
|
||||
}
|
||||
return kept[i].b < kept[j].b
|
||||
})
|
||||
|
||||
// Components. A pair from either signal joins the same family, so a chain
|
||||
// of alternative builds discovered by different signals stays one family
|
||||
// rather than two overlapping ones that would double claim a target.
|
||||
parent := map[int]int{}
|
||||
var find func(int) int
|
||||
find = func(x int) int {
|
||||
if p, ok := parent[x]; ok && p != x {
|
||||
parent[x] = find(p)
|
||||
return parent[x]
|
||||
}
|
||||
if _, ok := parent[x]; !ok {
|
||||
parent[x] = x
|
||||
}
|
||||
return parent[x]
|
||||
}
|
||||
union := func(x, y int) {
|
||||
rx, ry := find(x), find(y)
|
||||
if rx != ry {
|
||||
parent[ry] = rx
|
||||
}
|
||||
}
|
||||
evidenceFor := map[[2]int]Evidence{}
|
||||
for _, p := range kept {
|
||||
union(p.a, p.b)
|
||||
evidenceFor[[2]int{p.a, p.b}] = p.evidence
|
||||
}
|
||||
|
||||
components := map[int][]int{}
|
||||
for _, p := range kept {
|
||||
for _, m := range []int{p.a, p.b} {
|
||||
root := find(m)
|
||||
if !contains(components[root], m) {
|
||||
components[root] = append(components[root], m)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
roots := make([]int, 0, len(components))
|
||||
for r := range components {
|
||||
roots = append(roots, r)
|
||||
}
|
||||
sort.Ints(roots)
|
||||
|
||||
proposedTargets := map[string]string{}
|
||||
for _, root := range roots {
|
||||
members := components[root]
|
||||
sort.Ints(members)
|
||||
family, refusal := buildFamily(ix, members, evidenceFor, claimedBy, proposedTargets, byName)
|
||||
if refusal != nil {
|
||||
result.Refusals = append(result.Refusals, *refusal)
|
||||
continue
|
||||
}
|
||||
if family == nil {
|
||||
continue
|
||||
}
|
||||
for _, p := range family.Proposals {
|
||||
proposedTargets[strings.ToLower(p.Variant)] = family.Parent
|
||||
}
|
||||
result.Families = append(result.Families, *family)
|
||||
}
|
||||
|
||||
sort.Slice(result.Families, func(i, j int) bool { return result.Families[i].Parent < result.Families[j].Parent })
|
||||
result.Suppressed = SortedSuppressions(result.Suppressed)
|
||||
result.AliasSkipped = SortedSuppressions(result.AliasSkipped)
|
||||
result.Refusals = dedupeRefusals(result.Refusals)
|
||||
return result
|
||||
}
|
||||
|
||||
// dedupeRefusals collapses the same refusal reached from both orderings of a
|
||||
// pair, and sorts what is left. A reviewer reading the same complaint twice
|
||||
// learns to skim the section.
|
||||
func dedupeRefusals(in []Refusal) []Refusal {
|
||||
seen := map[string]struct{}{}
|
||||
var out []Refusal
|
||||
for _, r := range in {
|
||||
members := append([]string(nil), r.Members...)
|
||||
sort.Strings(members)
|
||||
key := strings.Join(members, "\x00") + "\x00" + r.Reason
|
||||
if _, dup := seen[key]; dup {
|
||||
continue
|
||||
}
|
||||
seen[key] = struct{}{}
|
||||
out = append(out, r)
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool {
|
||||
if a, b := strings.Join(out[i].Members, ","), strings.Join(out[j].Members, ","); a != b {
|
||||
return a < b
|
||||
}
|
||||
return out[i].Reason < out[j].Reason
|
||||
})
|
||||
return out
|
||||
}
|
||||
|
||||
func contains(xs []int, x int) bool {
|
||||
for _, v := range xs {
|
||||
if v == x {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// buildFamily turns a connected component into a proposal, or refuses it.
|
||||
func buildFamily(ix *Index, members []int, evidenceFor map[[2]int]Evidence, claimedBy map[string]string, proposedTargets map[string]string, byName map[string]*GalleryEntry) (*Family, *Refusal) {
|
||||
names := make([]string, 0, len(members))
|
||||
for _, m := range members {
|
||||
names = append(names, ix.Entries[m].Name)
|
||||
}
|
||||
|
||||
parentIdx, err := selectParent(ix, members)
|
||||
if err != nil {
|
||||
return nil, &Refusal{Members: names, Reason: err.Error()}
|
||||
}
|
||||
parentEntry := ix.Entries[parentIdx]
|
||||
parentName := strings.ToLower(parentEntry.Name)
|
||||
|
||||
// A parent that is itself somebody's variant would create a chain, which
|
||||
// the gallery's own resolution refuses to install.
|
||||
if owner, claimed := claimedBy[parentName]; claimed {
|
||||
return nil, &Refusal{Members: names, Reason: fmt.Sprintf("the natural parent %q is already a variant of %q; proposing it as a parent would nest variants", parentEntry.Name, owner)}
|
||||
}
|
||||
if owner, claimed := proposedTargets[parentName]; claimed {
|
||||
return nil, &Refusal{Members: names, Reason: fmt.Sprintf("the natural parent %q is already proposed as a variant of %q; proposing it as a parent would nest variants", parentEntry.Name, owner)}
|
||||
}
|
||||
|
||||
// Adding a variants key to an anchor is inherited by every entry that
|
||||
// merges it, silently grouping models nobody proposed. Handling that means
|
||||
// editing each merging child too, which is a larger change than this job
|
||||
// should make unsupervised, so it refuses and hands the reviewer the list.
|
||||
if parentEntry.AnchorName != "" {
|
||||
children := ix.MergeChildren(parentEntry.AnchorName)
|
||||
if len(children) > 0 {
|
||||
childNames := make([]string, 0, len(children))
|
||||
for _, c := range children {
|
||||
childNames = append(childNames, c.Name)
|
||||
}
|
||||
return nil, &Refusal{
|
||||
Members: names,
|
||||
Reason: fmt.Sprintf("the parent %q defines YAML anchor &%s, and a variants key added there is inherited by the %d entries that merge it (%s). Grouping this family by hand also means adding an explicit `variants: []` to each of those entries",
|
||||
parentEntry.Name, parentEntry.AnchorName, len(children), strings.Join(childNames, ", ")),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
existing := map[string]struct{}{}
|
||||
for _, v := range parentEntry.Variants {
|
||||
existing[strings.ToLower(v.Model)] = struct{}{}
|
||||
}
|
||||
|
||||
family := &Family{Parent: parentEntry.Name}
|
||||
for _, m := range members {
|
||||
if m == parentIdx {
|
||||
continue
|
||||
}
|
||||
target := ix.Entries[m]
|
||||
lower := strings.ToLower(target.Name)
|
||||
if _, already := existing[lower]; already {
|
||||
continue
|
||||
}
|
||||
if target.HasVariants() {
|
||||
return nil, &Refusal{Members: names, Reason: fmt.Sprintf("%q already offers variants of its own, so it cannot itself be a variant target", target.Name)}
|
||||
}
|
||||
if !target.Installable() {
|
||||
return nil, &Refusal{Members: names, Reason: fmt.Sprintf("%q has no url, config_file, overrides or files, so it is not independently installable", target.Name)}
|
||||
}
|
||||
if owner, claimed := claimedBy[lower]; claimed && owner != parentName {
|
||||
return nil, &Refusal{Members: names, Reason: fmt.Sprintf("%q is already a variant of %q; a target claimed by two parents is not something the gallery resolves predictably", target.Name, owner)}
|
||||
}
|
||||
if owner, claimed := proposedTargets[lower]; claimed && owner != parentEntry.Name {
|
||||
return nil, &Refusal{Members: names, Reason: fmt.Sprintf("%q is already proposed as a variant of %q in this same run", target.Name, owner)}
|
||||
}
|
||||
family.Proposals = append(family.Proposals, Proposal{
|
||||
Variant: target.Name,
|
||||
Evidence: lookupEvidence(evidenceFor, parentIdx, m),
|
||||
})
|
||||
}
|
||||
|
||||
if len(family.Proposals) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
sort.Slice(family.Proposals, func(i, j int) bool { return family.Proposals[i].Variant < family.Proposals[j].Variant })
|
||||
return family, nil
|
||||
}
|
||||
|
||||
func lookupEvidence(evidenceFor map[[2]int]Evidence, a, b int) Evidence {
|
||||
if a > b {
|
||||
a, b = b, a
|
||||
}
|
||||
if ev, ok := evidenceFor[[2]int{a, b}]; ok {
|
||||
return ev
|
||||
}
|
||||
// The two entries reached the same family through a third one. Say so
|
||||
// rather than inventing evidence that was never observed for this pair.
|
||||
return Evidence{Signals: []Signal{SignalName}}
|
||||
}
|
||||
|
||||
// selectParent picks the entry the others should hang off.
|
||||
//
|
||||
// The bare name wins when there is one: it is the name a user types and the one
|
||||
// documentation links to. Otherwise the smallest build wins, judged by the
|
||||
// quantization token in the entry's own weight filename, so the default install
|
||||
// is the one most hosts can actually run.
|
||||
func selectParent(ix *Index, members []int) (int, error) {
|
||||
// The family's own stem: the one the most members reduce to, shortest name
|
||||
// breaking a tie. An entry named exactly that is the bare entry.
|
||||
stemCount := map[string]int{}
|
||||
for _, m := range members {
|
||||
stemCount[NameStem(ix.Entries[m].Name)]++
|
||||
}
|
||||
// Only a stem two or more members reduce to is the family's own stem. A
|
||||
// stem reached by exactly one member is just that member's name, and
|
||||
// treating it as the family stem would crown whichever name happens to be
|
||||
// shortest rather than whichever build is the base one.
|
||||
familyStem := ""
|
||||
for stem, n := range stemCount {
|
||||
if n < 2 {
|
||||
continue
|
||||
}
|
||||
if familyStem == "" || n > stemCount[familyStem] ||
|
||||
(n == stemCount[familyStem] && len(stem) < len(familyStem)) ||
|
||||
(n == stemCount[familyStem] && len(stem) == len(familyStem) && stem < familyStem) {
|
||||
familyStem = stem
|
||||
}
|
||||
}
|
||||
|
||||
var bare []int
|
||||
for _, m := range members {
|
||||
e := ix.Entries[m]
|
||||
if HasConfigSuffix(e.Name) {
|
||||
continue
|
||||
}
|
||||
if strings.ToLower(e.Name) == familyStem {
|
||||
bare = append(bare, m)
|
||||
}
|
||||
}
|
||||
if len(bare) == 1 {
|
||||
return bare[0], nil
|
||||
}
|
||||
if len(bare) > 1 {
|
||||
names := make([]string, 0, len(bare))
|
||||
for _, m := range bare {
|
||||
names = append(names, ix.Entries[m].Name)
|
||||
}
|
||||
return 0, fmt.Errorf("more than one entry is named exactly %q (%s), so which one is the base build is a judgement this job will not make", familyStem, strings.Join(names, ", "))
|
||||
}
|
||||
|
||||
// No shared stem to be named after. An entry whose name every other member
|
||||
// extends is still recognisably the base one, and this is the only handle
|
||||
// left for families whose weights carry no readable quantization token at
|
||||
// all, such as the ONNX builds.
|
||||
if prefix, ok := uniquePrefixMember(ix, members); ok {
|
||||
return prefix, nil
|
||||
}
|
||||
|
||||
best := -1
|
||||
bestWidth := 1 << 20
|
||||
for _, m := range members {
|
||||
e := ix.Entries[m]
|
||||
width := unknownWidth
|
||||
if primary, ok := e.PrimaryWeightFile(); ok {
|
||||
width = BuildWidth(primary)
|
||||
}
|
||||
// Members are visited in gallery order, so a strict comparison leaves
|
||||
// the earliest entry holding a tie and the choice is deterministic.
|
||||
if width < bestWidth {
|
||||
best, bestWidth = m, width
|
||||
}
|
||||
}
|
||||
if best < 0 {
|
||||
return 0, fmt.Errorf("no member could be identified as the smallest build")
|
||||
}
|
||||
if bestWidth == unknownWidth {
|
||||
names := make([]string, 0, len(members))
|
||||
for _, m := range members {
|
||||
names = append(names, ix.Entries[m].Name)
|
||||
}
|
||||
return 0, fmt.Errorf("no member declares a weight file whose quantization can be read (%s), so the smallest build cannot be identified", strings.Join(names, ", "))
|
||||
}
|
||||
return best, nil
|
||||
}
|
||||
|
||||
// uniquePrefixMember reports the single member whose name every other member's
|
||||
// name starts with, if there is exactly one.
|
||||
func uniquePrefixMember(ix *Index, members []int) (int, bool) {
|
||||
found := -1
|
||||
for _, m := range members {
|
||||
name := strings.ToLower(ix.Entries[m].Name)
|
||||
isPrefix := true
|
||||
for _, other := range members {
|
||||
if other == m {
|
||||
continue
|
||||
}
|
||||
if !strings.HasPrefix(strings.ToLower(ix.Entries[other].Name), name) {
|
||||
isPrefix = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if !isPrefix {
|
||||
continue
|
||||
}
|
||||
if found >= 0 {
|
||||
return 0, false
|
||||
}
|
||||
found = m
|
||||
}
|
||||
return found, found >= 0
|
||||
}
|
||||
|
||||
// quantDifference lists the quantization tokens that tell two names apart. It
|
||||
// is the compact form of the evidence: "these differ only by q4_k_m vs q8_0".
|
||||
func quantDifference(names ...string) []string {
|
||||
var out []string
|
||||
seen := map[string]struct{}{}
|
||||
for _, name := range names {
|
||||
// Filenames arrive here too, so the extension goes first and "/" counts
|
||||
// as a separator. "_" deliberately does not: it holds "q4_k_m" together.
|
||||
trimmed := weightExtension.ReplaceAllString(name, "")
|
||||
for _, seg := range strings.FieldsFunc(strings.ToLower(trimmed), func(r rune) bool { return r == '-' || r == '/' }) {
|
||||
if !IsQuantToken(seg) {
|
||||
continue
|
||||
}
|
||||
if _, ok := seen[seg]; ok {
|
||||
continue
|
||||
}
|
||||
seen[seg] = struct{}{}
|
||||
out = append(out, seg)
|
||||
}
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
429
.github/ci/variantproposals/propose_test.go
vendored
Normal file
429
.github/ci/variantproposals/propose_test.go
vendored
Normal file
@@ -0,0 +1,429 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
// entryYAML writes one gallery entry with a single weight file, which is the
|
||||
// shape almost every real entry has. Specs that need something else write the
|
||||
// YAML out by hand.
|
||||
func entryYAML(name, repo, filename, sha string) string {
|
||||
return fmt.Sprintf(`- name: %s
|
||||
url: github:mudler/LocalAI/gallery/virtual.yaml@master
|
||||
overrides:
|
||||
parameters:
|
||||
model: %s
|
||||
files:
|
||||
- filename: %s
|
||||
uri: huggingface://%s/%s
|
||||
sha256: %s
|
||||
`, name, filename, filename, repo, filename, sha)
|
||||
}
|
||||
|
||||
func indexOf(entries ...string) *Index {
|
||||
ix, err := ParseIndex(strings.Join(entries, ""))
|
||||
ExpectWithOffset(1, err).ToNot(HaveOccurred())
|
||||
return ix
|
||||
}
|
||||
|
||||
// familyNames flattens a result into "parent <- variant, variant" strings, the
|
||||
// form the specs assert against.
|
||||
func familyNames(r *Result) []string {
|
||||
out := make([]string, 0, len(r.Families))
|
||||
for _, f := range r.Families {
|
||||
names := make([]string, 0, len(f.Proposals))
|
||||
for _, p := range f.Proposals {
|
||||
names = append(names, p.Variant)
|
||||
}
|
||||
out = append(out, f.Parent+" <- "+strings.Join(names, ", "))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func refusalReasons(r *Result) string {
|
||||
var b strings.Builder
|
||||
for _, ref := range r.Refusals {
|
||||
b.WriteString(strings.Join(ref.Members, " + ") + ": " + ref.Reason + "\n")
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func suppressionReasons(r *Result) string {
|
||||
var b strings.Builder
|
||||
for _, s := range r.Suppressed {
|
||||
b.WriteString(s.String() + "\n")
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
var _ = Describe("Propose", func() {
|
||||
Describe("the grouping signals", func() {
|
||||
It("groups entries whose names differ only by a quantization marker", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("foo-model", "acme/foo-GGUF", "foo-model-Q4_K_M.gguf", "aa"),
|
||||
entryYAML("foo-model-q8_0", "acme/foo-GGUF", "foo-model-Q8_0.gguf", "bb"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(ConsistOf("foo-model <- foo-model-q8_0"))
|
||||
Expect(r.Families[0].Proposals[0].Evidence.Signals).To(ContainElement(SignalName))
|
||||
Expect(r.Families[0].Proposals[0].Evidence.SharedStem).To(Equal("foo-model"))
|
||||
Expect(r.Families[0].Proposals[0].Evidence.QuantTokens).To(ContainElements("q4_k_m", "q8_0"))
|
||||
})
|
||||
|
||||
It("groups entries that use the colon config-suffix convention", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("bar-model", "acme/bar-GGUF", "bar-model-Q4_K_M.gguf", "aa"),
|
||||
entryYAML("bar-model:grammar-functioncall", "acme/bar-GGUF", "bar-model-Q4_K_M-grammar.gguf", "bb"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(ConsistOf("bar-model <- bar-model:grammar-functioncall"))
|
||||
Expect(r.Families[0].Proposals[0].Evidence.Signals).To(ContainElement(SignalConfigSuffix))
|
||||
})
|
||||
|
||||
It("groups entries whose own weight file is the same file at another quantization", func() {
|
||||
// The names share no stem, so only the filename signal can link
|
||||
// these two.
|
||||
ix := indexOf(
|
||||
entryYAML("omni-cpp", "Serveurperso/Omni-GGUF", "omnivoice-base-Q8_0.gguf", "aa"),
|
||||
entryYAML("omni-cpp-hq", "Serveurperso/Omni-GGUF", "omnivoice-base-BF16.gguf", "bb"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(ConsistOf("omni-cpp <- omni-cpp-hq"))
|
||||
ev := r.Families[0].Proposals[0].Evidence
|
||||
Expect(ev.Signals).To(ConsistOf(SignalWeightFile))
|
||||
Expect(ev.SharedFile).To(Equal("omnivoice-base"))
|
||||
Expect(ev.SharedRepo).To(Equal("serveurperso/omni-gguf"))
|
||||
})
|
||||
|
||||
It("does not let a shared auxiliary file link unrelated models", func() {
|
||||
// Both entries ship the same text encoder. That is a packaging
|
||||
// convention, not evidence of shared weights: this is how an
|
||||
// earlier sweep linked four wan-2.1 entries to each other.
|
||||
ix := indexOf(`- name: wan-2.1-t2v
|
||||
url: u
|
||||
files:
|
||||
- filename: wan-2.1-t2v-Q4_K_M.gguf
|
||||
uri: huggingface://acme/wan/wan-2.1-t2v-Q4_K_M.gguf
|
||||
sha256: aa
|
||||
- filename: umt5-xxl-encoder-Q8_0.gguf
|
||||
uri: huggingface://acme/wan/umt5-xxl-encoder-Q8_0.gguf
|
||||
sha256: cc
|
||||
`, `- name: z-image-turbo
|
||||
url: u
|
||||
files:
|
||||
- filename: z-image-turbo-Q4_K_M.gguf
|
||||
uri: huggingface://acme/wan/z-image-turbo-Q4_K_M.gguf
|
||||
sha256: bb
|
||||
- filename: umt5-xxl-encoder-Q8_0.gguf
|
||||
uri: huggingface://acme/wan/umt5-xxl-encoder-Q8_0.gguf
|
||||
sha256: cc
|
||||
`)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("does not treat a shared filename in two different repos as evidence", func() {
|
||||
// A finetune republished under the base model's filename is the
|
||||
// most common way this signal misfires.
|
||||
ix := indexOf(
|
||||
entryYAML("llama-3.2-3b-instruct", "hugging-quants/Llama-3.2-3B-Instruct-GGUF", "llama-3.2-3b-instruct-q4_k_m.gguf", "aa"),
|
||||
entryYAML("llama-3.2-3b-shiro-roleplay", "someone/Shiro-GGUF", "Llama-3.2-3B-Instruct.Q8_0.gguf", "bb"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
})
|
||||
})
|
||||
|
||||
Describe("what must never be proposed", func() {
|
||||
It("does not group different parameter sizes that share a prefix", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("qwen3-tts-cpp-0.6b-base", "Serveurperso/Qwen3-TTS-GGUF", "qwen3-tts-talker-Q4_K_M.gguf", "aa"),
|
||||
entryYAML("qwen3-tts-cpp-1.7b-base", "Serveurperso/Qwen3-TTS-GGUF", "qwen3-tts-talker-Q8_0.gguf", "bb"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
Expect(suppressionReasons(r)).To(ContainSubstring("different parameter sizes"))
|
||||
})
|
||||
|
||||
It("does not group the Gemma effective sizes", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("gemma-4-e2b-it", "google/gemma-GGUF", "gemma-4-it-Q4_K_M.gguf", "aa"),
|
||||
entryYAML("gemma-4-e4b-it", "google/gemma-GGUF", "gemma-4-it-Q8_0.gguf", "bb"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
Expect(suppressionReasons(r)).To(ContainSubstring("different parameter sizes"))
|
||||
})
|
||||
|
||||
It("does not group entries with a byte-identical install payload", func() {
|
||||
// whisper-1 exists so OpenAI-compatible clients can send that name.
|
||||
// Folding it under whisper-base would hide the name they send.
|
||||
payload := ` url: github:mudler/LocalAI/gallery/whisper-base.yaml@master
|
||||
overrides:
|
||||
parameters:
|
||||
model: ggml-whisper-base.bin
|
||||
files:
|
||||
- filename: ggml-whisper-base.bin
|
||||
uri: huggingface://ggerganov/whisper.cpp/ggml-base.bin
|
||||
sha256: aa
|
||||
`
|
||||
ix := indexOf("- name: whisper-base\n"+payload, "- name: whisper-1\n"+payload)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
Expect(r.AliasSkipped).To(HaveLen(1))
|
||||
Expect(r.AliasSkipped[0].Reason).To(ContainSubstring("aliases"))
|
||||
})
|
||||
|
||||
DescribeTable("declines the categories the ledger records",
|
||||
func(nameA, nameB string, ledgerYAML string) {
|
||||
ix := indexOf(
|
||||
entryYAML(nameA, "acme/repo", "shared-weights-Q4_K_M.gguf", "aa"),
|
||||
entryYAML(nameB, "acme/repo", "shared-weights-Q8_0.gguf", "bb"),
|
||||
)
|
||||
ledger, err := ParseLedger([]byte(ledgerYAML))
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
|
||||
// Without the ledger these would be proposed, which is what
|
||||
// makes the ledger load bearing rather than decorative.
|
||||
Expect(familyNames(Propose(ix, nil))).ToNot(BeEmpty())
|
||||
|
||||
r := Propose(ix, ledger)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
Expect(r.Suppressed).To(HaveLen(1))
|
||||
},
|
||||
Entry("a finetune", "base-model", "base-model-abliterated",
|
||||
"tokens:\n - {token: abliterated, reason: finetune}\n"),
|
||||
Entry("a distill", "base-model", "base-model-distilled",
|
||||
"tokens:\n - {token: distilled, reason: distilled}\n"),
|
||||
Entry("English-only versus multilingual ASR", "whisper-small", "whisper-small-en",
|
||||
"pairs:\n - {parent: whisper-small, variant: whisper-small-en, reason: English-only versus multilingual}\n"),
|
||||
Entry("two products sharing a prefix", "vibevoice-cpp", "vibevoice-cpp-asr",
|
||||
"pairs:\n - {parent: vibevoice-cpp, variant: vibevoice-cpp-asr, reason: different products}\n"),
|
||||
Entry("a per-language release", "kokoros-de", "kokoros-ja",
|
||||
"groups:\n - {members: [kokoros, kokoros-de, kokoros-ja], reason: different languages}\n"),
|
||||
)
|
||||
|
||||
It("reports the ledger's reason so its effect stays visible", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("base-model", "acme/repo", "shared-weights-Q4_K_M.gguf", "aa"),
|
||||
entryYAML("base-model-heretic", "acme/repo", "shared-weights-Q8_0.gguf", "bb"),
|
||||
)
|
||||
ledger, err := ParseLedger([]byte("tokens:\n - {token: heretic, reason: \"finetune, not a re-quantization\"}\n"))
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
r := Propose(ix, ledger)
|
||||
Expect(suppressionReasons(r)).To(ContainSubstring("finetune, not a re-quantization"))
|
||||
Expect(suppressionReasons(r)).To(ContainSubstring(`token "heretic"`))
|
||||
})
|
||||
})
|
||||
|
||||
Describe("parent selection", func() {
|
||||
It("picks the bare-named entry when one exists", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("base-model-q8_0", "acme/repo", "base-model-Q8_0.gguf", "aa"),
|
||||
entryYAML("base-model", "acme/repo", "base-model-Q4_K_M.gguf", "bb"),
|
||||
entryYAML("base-model-f16", "acme/repo", "base-model-f16.gguf", "cc"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(ConsistOf("base-model <- base-model-f16, base-model-q8_0"))
|
||||
})
|
||||
|
||||
It("picks the smallest build when no entry is bare-named", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("ced-base-f16", "acme/repo", "ced-base-f16.gguf", "aa"),
|
||||
entryYAML("ced-base-q8", "acme/repo", "ced-base-Q8_0.gguf", "bb"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(ConsistOf("ced-base-q8 <- ced-base-f16"))
|
||||
})
|
||||
|
||||
It("judges the smallest build by the quantization in the model filename, not the name", func() {
|
||||
// The names carry no marker at all; only the filenames say which
|
||||
// build is which.
|
||||
ix := indexOf(
|
||||
entryYAML("thing-hq", "acme/repo", "thing-weights-BF16.gguf", "aa"),
|
||||
entryYAML("thing-lite", "acme/repo", "thing-weights-Q4_K_M.gguf", "bb"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(ConsistOf("thing-lite <- thing-hq"))
|
||||
})
|
||||
})
|
||||
|
||||
Describe("the rules a proposal has to respect", func() {
|
||||
It("refuses to nest: a target that already offers variants of its own", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("nest-model", "acme/repo", "nest-model-Q4_K_M.gguf", "aa"),
|
||||
`- name: nest-model-q8_0
|
||||
url: u
|
||||
variants:
|
||||
- model: nest-model-q8_0-mtp
|
||||
overrides:
|
||||
parameters:
|
||||
model: nest-model-Q8_0.gguf
|
||||
files:
|
||||
- filename: nest-model-Q8_0.gguf
|
||||
uri: huggingface://acme/repo/nest-model-Q8_0.gguf
|
||||
sha256: bb
|
||||
`,
|
||||
entryYAML("nest-model-q8_0-mtp", "other/repo", "nest-model-mtp.gguf", "cc"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
Expect(refusalReasons(r)).To(ContainSubstring("already offers variants of its own"))
|
||||
})
|
||||
|
||||
It("refuses to nest: a parent that is already somebody else's variant", func() {
|
||||
ix := indexOf(
|
||||
`- name: outer
|
||||
url: u
|
||||
variants:
|
||||
- model: middle
|
||||
overrides:
|
||||
parameters:
|
||||
model: outer-Q4_K_M.gguf
|
||||
files:
|
||||
- filename: outer-Q4_K_M.gguf
|
||||
uri: huggingface://acme/repo/outer-Q4_K_M.gguf
|
||||
sha256: aa
|
||||
`,
|
||||
entryYAML("middle", "acme/other", "middle-Q4_K_M.gguf", "bb"),
|
||||
entryYAML("middle-q8_0", "acme/other", "middle-Q8_0.gguf", "cc"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
Expect(refusalReasons(r)).To(ContainSubstring("would nest variants"))
|
||||
})
|
||||
|
||||
It("refuses to let two parents claim one target", func() {
|
||||
ix := indexOf(
|
||||
`- name: claimant
|
||||
url: u
|
||||
variants:
|
||||
- model: contested-q8_0
|
||||
overrides:
|
||||
parameters:
|
||||
model: claimant-Q4_K_M.gguf
|
||||
files:
|
||||
- filename: claimant-Q4_K_M.gguf
|
||||
uri: huggingface://acme/repo/claimant-Q4_K_M.gguf
|
||||
sha256: aa
|
||||
`,
|
||||
entryYAML("contested", "acme/other", "contested-Q4_K_M.gguf", "bb"),
|
||||
entryYAML("contested-q8_0", "acme/other", "contested-Q8_0.gguf", "cc"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
Expect(refusalReasons(r)).To(ContainSubstring("already a variant of"))
|
||||
})
|
||||
|
||||
It("refuses a target that is not independently installable", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("stub-model", "acme/repo", "stub-model-Q4_K_M.gguf", "aa"),
|
||||
"- name: stub-model-q8_0\n description: a stanza nobody finished\n",
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
Expect(refusalReasons(r)).To(ContainSubstring("not independently installable"))
|
||||
})
|
||||
|
||||
It("refuses a family whose parent defines a merge anchor, naming the entries that would inherit", func() {
|
||||
ix := indexOf(
|
||||
`- &anchored
|
||||
name: anchored-model
|
||||
url: u
|
||||
overrides:
|
||||
parameters:
|
||||
model: anchored-Q4_K_M.gguf
|
||||
files:
|
||||
- filename: anchored-Q4_K_M.gguf
|
||||
uri: huggingface://acme/repo/anchored-Q4_K_M.gguf
|
||||
sha256: aa
|
||||
`,
|
||||
`- !!merge <<: *anchored
|
||||
name: anchored-child
|
||||
variants: []
|
||||
overrides:
|
||||
parameters:
|
||||
model: unrelated-child-Q4_K_M.gguf
|
||||
files:
|
||||
- filename: unrelated-child-Q4_K_M.gguf
|
||||
uri: huggingface://other/repo/unrelated-child-Q4_K_M.gguf
|
||||
sha256: cc
|
||||
`,
|
||||
entryYAML("anchored-model-q8_0", "acme/repo", "anchored-Q8_0.gguf", "bb"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
Expect(refusalReasons(r)).To(ContainSubstring("defines YAML anchor &anchored"))
|
||||
Expect(refusalReasons(r)).To(ContainSubstring("anchored-child"))
|
||||
Expect(refusalReasons(r)).To(ContainSubstring("variants: []"))
|
||||
})
|
||||
|
||||
It("refuses an entry whose name is not unique in the gallery", func() {
|
||||
ix := indexOf(
|
||||
entryYAML("twin", "acme/repo", "twin-Q4_K_M.gguf", "aa"),
|
||||
entryYAML("twin", "acme/repo", "twin-Q4_K_M.gguf", "aa"),
|
||||
entryYAML("twin-q8_0", "acme/repo", "twin-Q8_0.gguf", "bb"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(BeEmpty())
|
||||
Expect(refusalReasons(r)).To(ContainSubstring("appears more than once"))
|
||||
})
|
||||
|
||||
It("says nothing about a pair that is already grouped", func() {
|
||||
ix := indexOf(
|
||||
`- name: settled
|
||||
url: u
|
||||
variants:
|
||||
- model: settled-q8_0
|
||||
overrides:
|
||||
parameters:
|
||||
model: settled-Q4_K_M.gguf
|
||||
files:
|
||||
- filename: settled-Q4_K_M.gguf
|
||||
uri: huggingface://acme/repo/settled-Q4_K_M.gguf
|
||||
sha256: aa
|
||||
`,
|
||||
entryYAML("settled-q8_0", "acme/repo", "settled-Q8_0.gguf", "bb"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(r.HasProposals()).To(BeFalse())
|
||||
Expect(r.Refusals).To(BeEmpty())
|
||||
Expect(r.Suppressed).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("adds only the missing members to a family that already exists", func() {
|
||||
ix := indexOf(
|
||||
`- name: partial
|
||||
url: u
|
||||
variants:
|
||||
- model: partial-q8_0
|
||||
overrides:
|
||||
parameters:
|
||||
model: partial-Q4_K_M.gguf
|
||||
files:
|
||||
- filename: partial-Q4_K_M.gguf
|
||||
uri: huggingface://acme/repo/partial-Q4_K_M.gguf
|
||||
sha256: aa
|
||||
`,
|
||||
entryYAML("partial-q8_0", "acme/repo", "partial-Q8_0.gguf", "bb"),
|
||||
entryYAML("partial-f16", "acme/repo", "partial-f16.gguf", "cc"),
|
||||
)
|
||||
r := Propose(ix, nil)
|
||||
Expect(familyNames(r)).To(ConsistOf("partial <- partial-f16"))
|
||||
})
|
||||
})
|
||||
|
||||
It("does not modify the index it was given", func() {
|
||||
text := entryYAML("foo-model", "acme/foo-GGUF", "foo-model-Q4_K_M.gguf", "aa") +
|
||||
entryYAML("foo-model-q8_0", "acme/foo-GGUF", "foo-model-Q8_0.gguf", "bb")
|
||||
ix, err := ParseIndex(text)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
before := strings.Join(ix.Lines, "\n")
|
||||
Propose(ix, nil)
|
||||
Expect(strings.Join(ix.Lines, "\n")).To(Equal(before))
|
||||
})
|
||||
})
|
||||
151
.github/ci/variantproposals/quant.go
vendored
Normal file
151
.github/ci/variantproposals/quant.go
vendored
Normal file
@@ -0,0 +1,151 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Quantization and precision markers that distinguish one build of a set of
|
||||
// weights from another build of the same weights. Stripping them from a name
|
||||
// is what lets the proposer notice that two entries are the same model.
|
||||
//
|
||||
// qat and apex are in this list on a maintainer ruling: they are quantization
|
||||
// techniques applied to published weights, not separate weights. Names that use
|
||||
// "apex" to mean a finetune are handled by the rejection ledger instead, because
|
||||
// no amount of pattern matching can tell the two uses apart.
|
||||
const quantAlternation = `q[2-8](?:_[0-9a-z]+)*|pq[2-8](?:_[0-9a-z]+)*|iq[1-9][0-9a-z]*(?:_[0-9a-z]+)*|i1|` +
|
||||
`f16|f32|bf16|fp16|fp32|fp8|fp4|nvfp4|mxfp4(?:_moe)*|awq|gptq|qat|apex|gguf|ggml|[0-9]+bit|g[0-9]+`
|
||||
|
||||
// quantSegment matches a whole hyphen-delimited segment of an entry name.
|
||||
// Names separate their parts with "-" and keep quantization tokens internally
|
||||
// joined with "_", so a segment is the right unit here: "q4_k_m" arrives whole.
|
||||
var quantSegment = regexp.MustCompile(`^(?:` + quantAlternation + `)$`)
|
||||
|
||||
// quantFileSuffix matches a trailing quantization token in a weight filename.
|
||||
// Filenames mix "-", "_" and "." as separators, so unlike entry names they
|
||||
// cannot be split into segments up front without tearing "Q4_K_M" apart.
|
||||
var quantFileSuffix = regexp.MustCompile(`(?i)[-_.](?:` + quantAlternation + `)$`)
|
||||
|
||||
var weightExtension = regexp.MustCompile(`(?i)\.(gguf|ggml|safetensors|bin|pt|pth|onnx)$`)
|
||||
|
||||
// IsQuantToken reports whether a single name segment is a quantization or
|
||||
// precision marker rather than part of the model's identity.
|
||||
func IsQuantToken(segment string) bool {
|
||||
return quantSegment.MatchString(strings.ToLower(segment))
|
||||
}
|
||||
|
||||
// NameStem reduces an entry name to the identity it shares with its alternative
|
||||
// builds: the config suffix after ":" is dropped, then trailing quantization
|
||||
// segments are stripped.
|
||||
//
|
||||
// It implements the first two grouping signals together because they answer the
|
||||
// same question. "foo:q8_0" and "foo-q8_0" are both alternative builds of "foo",
|
||||
// and the caller that needs to report which convention was used can compare the
|
||||
// name against the stem itself.
|
||||
//
|
||||
// At least one segment always survives, so a name made entirely of quantization
|
||||
// tokens does not collapse to the empty stem and swallow every other such name.
|
||||
func NameStem(name string) string {
|
||||
base := strings.ToLower(strings.TrimSpace(name))
|
||||
if i := strings.Index(base, ":"); i >= 0 {
|
||||
base = base[:i]
|
||||
}
|
||||
segments := strings.Split(base, "-")
|
||||
for len(segments) > 1 && quantSegment.MatchString(segments[len(segments)-1]) {
|
||||
segments = segments[:len(segments)-1]
|
||||
}
|
||||
return strings.Join(segments, "-")
|
||||
}
|
||||
|
||||
// HasConfigSuffix reports whether a name uses the ":" convention for naming a
|
||||
// config variant of another entry.
|
||||
func HasConfigSuffix(name string) bool {
|
||||
return strings.Contains(name, ":")
|
||||
}
|
||||
|
||||
// FileStem reduces a weight filename to the identity shared by its other
|
||||
// quantizations: directories, extension and trailing quantization tokens go.
|
||||
//
|
||||
// This is the third grouping signal. It is the one that has misfired before, so
|
||||
// callers must filter auxiliary files out before handing a filename here: a
|
||||
// shared text encoder is not evidence of shared weights.
|
||||
func FileStem(filename string) string {
|
||||
base := filename
|
||||
if i := strings.LastIndex(base, "/"); i >= 0 {
|
||||
base = base[i+1:]
|
||||
}
|
||||
base = weightExtension.ReplaceAllString(base, "")
|
||||
for {
|
||||
stripped := quantFileSuffix.ReplaceAllString(base, "")
|
||||
if stripped == base {
|
||||
break
|
||||
}
|
||||
base = stripped
|
||||
}
|
||||
return strings.ToLower(base)
|
||||
}
|
||||
|
||||
// bitsPerWeight ranks quantization tokens so the smallest build of a family can
|
||||
// be identified when no bare-named entry exists to be the parent.
|
||||
//
|
||||
// The figures are nominal bits per weight, not measured file sizes. Ranking is
|
||||
// all that is asked of them, and a nominal figure is available from the name
|
||||
// alone without downloading anything.
|
||||
func bitsPerWeight(token string) (int, bool) {
|
||||
t := strings.ToLower(token)
|
||||
switch {
|
||||
case t == "i1":
|
||||
return 1, true
|
||||
case strings.HasPrefix(t, "nvfp4"), strings.HasPrefix(t, "mxfp4"), t == "fp4":
|
||||
return 4, true
|
||||
case t == "fp8":
|
||||
return 8, true
|
||||
case t == "f16", t == "bf16", t == "fp16":
|
||||
return 16, true
|
||||
case t == "f32", t == "fp32":
|
||||
return 32, true
|
||||
case t == "awq", t == "gptq":
|
||||
return 4, true
|
||||
}
|
||||
if m := regexp.MustCompile(`^p?q([1-9])`).FindStringSubmatch(t); m != nil {
|
||||
n, _ := strconv.Atoi(m[1])
|
||||
return n, true
|
||||
}
|
||||
if m := regexp.MustCompile(`^iq([1-9])`).FindStringSubmatch(t); m != nil {
|
||||
n, _ := strconv.Atoi(m[1])
|
||||
return n, true
|
||||
}
|
||||
if m := regexp.MustCompile(`^([0-9]+)bit$`).FindStringSubmatch(t); m != nil {
|
||||
n, _ := strconv.Atoi(m[1])
|
||||
return n, true
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
// unknownWidth sorts after every recognised quantization so an entry whose
|
||||
// build cannot be read from its filename never wins the "smallest build" tie
|
||||
// break by accident.
|
||||
const unknownWidth = 1 << 10
|
||||
|
||||
// BuildWidth reports the nominal bits per weight of the build a filename holds.
|
||||
// An unreadable filename gets unknownWidth.
|
||||
func BuildWidth(filename string) int {
|
||||
base := filename
|
||||
if i := strings.LastIndex(base, "/"); i >= 0 {
|
||||
base = base[i+1:]
|
||||
}
|
||||
base = weightExtension.ReplaceAllString(base, "")
|
||||
best := unknownWidth
|
||||
for {
|
||||
m := quantFileSuffix.FindString(base)
|
||||
if m == "" {
|
||||
break
|
||||
}
|
||||
if bits, ok := bitsPerWeight(m[1:]); ok && bits < best {
|
||||
best = bits
|
||||
}
|
||||
base = base[:len(base)-len(m)]
|
||||
}
|
||||
return best
|
||||
}
|
||||
92
.github/ci/variantproposals/quant_test.go
vendored
Normal file
92
.github/ci/variantproposals/quant_test.go
vendored
Normal file
@@ -0,0 +1,92 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("quantization markers", func() {
|
||||
DescribeTable("NameStem strips the markers that distinguish builds, not models",
|
||||
func(name, expected string) {
|
||||
Expect(NameStem(name)).To(Equal(expected))
|
||||
},
|
||||
Entry("plain q4", "foo-model-q4_k_m", "foo-model"),
|
||||
Entry("q8_0", "foo-model-q8_0", "foo-model"),
|
||||
Entry("q5_1", "foo-model-q5_1", "foo-model"),
|
||||
Entry("q2 with group size", "ternary-bonsai-8b-q2-g64", "ternary-bonsai-8b"),
|
||||
Entry("iq variant", "ideogram-4-iq4nl-ggml", "ideogram-4"),
|
||||
Entry("i1 imatrix", "orca-agent-v0.1-i1", "orca-agent-v0.1"),
|
||||
Entry("f16", "ced-base-f16", "ced-base"),
|
||||
Entry("bf16", "some-model-bf16", "some-model"),
|
||||
Entry("fp8", "some-model-fp8", "some-model"),
|
||||
Entry("nvfp4", "qwen3.6-27b-nvfp4", "qwen3.6-27b"),
|
||||
Entry("mxfp4_moe", "huihui-qwen3-vl-30b-a3b-instruct-abliterated-mxfp4_moe", "huihui-qwen3-vl-30b-a3b-instruct-abliterated"),
|
||||
Entry("pq2", "ternary-bonsai-8b-pq2", "ternary-bonsai-8b"),
|
||||
Entry("awq", "some-model-awq", "some-model"),
|
||||
Entry("gptq", "some-model-gptq", "some-model"),
|
||||
Entry("Nbit", "qwen3-8b-mlx-4bit", "qwen3-8b-mlx"),
|
||||
Entry("gguf", "some-model-gguf", "some-model"),
|
||||
Entry("ggml", "flux.1-dev-ggml", "flux.1-dev"),
|
||||
Entry("qat is a quantization technique", "gemma-3-27b-it-qat", "gemma-3-27b-it"),
|
||||
Entry("apex is a quantization technique", "qwen3.6-35b-a3b-apex", "qwen3.6-35b-a3b"),
|
||||
Entry("stacked markers", "gemma-4-e2b-it-qat-q4_0", "gemma-4-e2b-it"),
|
||||
Entry("the config suffix is dropped", "phi-2-chat:Q8_0", "phi-2-chat"),
|
||||
Entry("a non-quant config suffix is dropped too", "meta-llama-3.1-8b-instruct:grammar-functioncall", "meta-llama-3.1-8b-instruct"),
|
||||
)
|
||||
|
||||
DescribeTable("NameStem leaves alone what identifies a different model",
|
||||
func(name, expected string) {
|
||||
Expect(NameStem(name)).To(Equal(expected))
|
||||
},
|
||||
Entry("parameter size", "qwen3-tts-cpp-0.6b-base", "qwen3-tts-cpp-0.6b-base"),
|
||||
Entry("language suffix", "kokoros-de", "kokoros-de"),
|
||||
Entry("English-only ASR", "whisper-small-en", "whisper-small-en"),
|
||||
Entry("finetune", "qwen3-30b-a3b-abliterated", "qwen3-30b-a3b-abliterated"),
|
||||
Entry("product suffix", "vibevoice-cpp-asr", "vibevoice-cpp-asr"),
|
||||
)
|
||||
|
||||
It("never strips a name down to nothing", func() {
|
||||
Expect(NameStem("q4_k_m")).To(Equal("q4_k_m"))
|
||||
Expect(NameStem("f16-q8_0")).To(Equal("f16"))
|
||||
})
|
||||
|
||||
DescribeTable("FileStem reduces a weight filename to the weights it holds",
|
||||
func(filename, expected string) {
|
||||
Expect(FileStem(filename)).To(Equal(expected))
|
||||
},
|
||||
Entry("directory and extension go", "bonsai/models/Ternary-Bonsai-8B-gguf/Ternary-Bonsai-8B-Q2_0.gguf", "ternary-bonsai-8b"),
|
||||
Entry("underscored quant token stays whole", "Llama-3.2-1B-Instruct-Q4_K_M.gguf", "llama-3.2-1b-instruct"),
|
||||
Entry("dot separated quant token", "Llama-3.2-3B-Instruct.Q4_K_M.gguf", "llama-3.2-3b-instruct"),
|
||||
Entry("group size suffix", "Ternary-Bonsai-8B-Q2_0_g64.gguf", "ternary-bonsai-8b"),
|
||||
Entry("bf16", "omnivoice-cpp-hq/omnivoice-base-BF16.gguf", "omnivoice-base"),
|
||||
Entry("safetensors", "some/dir/Model-Name-fp8.safetensors", "model-name"),
|
||||
)
|
||||
|
||||
DescribeTable("BuildWidth reads the nominal width out of a filename",
|
||||
func(filename string, expected int) {
|
||||
Expect(BuildWidth(filename)).To(Equal(expected))
|
||||
},
|
||||
Entry("q4", "foo-Q4_K_M.gguf", 4),
|
||||
Entry("q8", "foo-Q8_0.gguf", 8),
|
||||
Entry("q2", "foo-Q2_0.gguf", 2),
|
||||
Entry("f16", "foo-f16.gguf", 16),
|
||||
Entry("bf16", "foo-BF16.gguf", 16),
|
||||
Entry("iq3", "foo-iq3_xxs.gguf", 3),
|
||||
Entry("nothing readable sorts last", "foo.gguf", unknownWidth),
|
||||
)
|
||||
|
||||
It("treats an auxiliary file as never being the model's own weights", func() {
|
||||
for _, f := range []string{
|
||||
"mmproj-model-f16.gguf",
|
||||
"dir/vae-BF16.gguf",
|
||||
"clip_l.safetensors",
|
||||
"umt5-xxl-encoder-Q8_0.gguf",
|
||||
"t5xxl_fp16.safetensors",
|
||||
"ae.safetensors",
|
||||
"omnivoice-tokenizer-Q8_0.gguf",
|
||||
} {
|
||||
Expect(IsAuxiliaryFile(f)).To(BeTrue(), "expected %q to be auxiliary", f)
|
||||
}
|
||||
Expect(IsAuxiliaryFile("gemma-3-27b-it-Q4_K_M.gguf")).To(BeFalse())
|
||||
})
|
||||
})
|
||||
13
.github/ci/variantproposals/variantproposals_suite_test.go
vendored
Normal file
13
.github/ci/variantproposals/variantproposals_suite_test.go
vendored
Normal file
@@ -0,0 +1,13 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
func TestVariantProposals(t *testing.T) {
|
||||
RegisterFailHandler(Fail)
|
||||
RunSpecs(t, "gallery variant proposals")
|
||||
}
|
||||
54
.github/workflows/gallery_variant_proposals.yaml
vendored
Normal file
54
.github/workflows/gallery_variant_proposals.yaml
vendored
Normal file
@@ -0,0 +1,54 @@
|
||||
name: Propose gallery variant groupings
|
||||
on:
|
||||
schedule:
|
||||
- cron: 0 4 * * 1
|
||||
workflow_dispatch:
|
||||
jobs:
|
||||
variant_proposals:
|
||||
if: github.repository == 'mudler/LocalAI'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version-file: go.mod
|
||||
cache: false
|
||||
|
||||
# The heuristics are the risky part of this job. A regression in them
|
||||
# produces confident, wrong proposals, which is worse than no job at all.
|
||||
- name: Test the proposer
|
||||
run: go test ./.github/ci/variantproposals/
|
||||
|
||||
- name: Propose groupings 🔧
|
||||
id: propose
|
||||
run: |
|
||||
rm -f /tmp/variant-proposals-body.md
|
||||
go run ./.github/ci/variantproposals \
|
||||
-index gallery/index.yaml \
|
||||
-ledger gallery/variant-exclusions.yaml \
|
||||
-body-out /tmp/variant-proposals-body.md \
|
||||
-apply
|
||||
if [ -s /tmp/variant-proposals-body.md ]; then
|
||||
echo "have_proposals=true" >> "$GITHUB_OUTPUT"
|
||||
{
|
||||
echo 'body<<VARIANT_PROPOSAL_BODY_EOF'
|
||||
cat /tmp/variant-proposals-body.md
|
||||
echo VARIANT_PROPOSAL_BODY_EOF
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "have_proposals=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
# No body file means the proposer found nothing. Opening an empty pull
|
||||
# request every run is how a proposal job gets muted by its reviewers.
|
||||
- name: Create Pull Request
|
||||
if: steps.propose.outputs.have_proposals == 'true'
|
||||
uses: peter-evans/create-pull-request@v8
|
||||
with:
|
||||
token: ${{ secrets.UPDATE_BOT_TOKEN }}
|
||||
push-to-fork: ci-forks/LocalAI
|
||||
commit-message: 'chore(model-gallery): propose variant groupings'
|
||||
title: 'chore(model-gallery): propose variant groupings for review'
|
||||
branch: "propose/variant-groupings"
|
||||
body: ${{ steps.propose.outputs.body }}
|
||||
signoff: true
|
||||
83
gallery/variant-exclusions.yaml
Normal file
83
gallery/variant-exclusions.yaml
Normal file
@@ -0,0 +1,83 @@
|
||||
# Rejection ledger for the gallery variant proposal job.
|
||||
#
|
||||
# `.github/ci/variantproposals` looks for gallery entries that are alternative
|
||||
# builds of the same weights and proposes grouping them under one another with
|
||||
# `variants:`. Grouping is a judgement call that has gone wrong in both
|
||||
# directions, so the job only proposes. This file is where the answers live once
|
||||
# a human has given them.
|
||||
#
|
||||
# The job reads this file and stays silent about everything it names. Without it
|
||||
# a declined candidate comes back every night until reviewers stop reading the
|
||||
# job's pull requests, at which point the job is worse than useless.
|
||||
#
|
||||
# To decline a proposal, add one line to `pairs` inside the proposal pull
|
||||
# request itself and close the pull request:
|
||||
#
|
||||
# - {parent: some-model, variant: some-model-thing, reason: "different finetune"}
|
||||
#
|
||||
# Three kinds of entry:
|
||||
#
|
||||
# tokens a name segment that marks a distinct model rather than another build
|
||||
# of the same one. Two candidates whose names differ by one of these
|
||||
# are never proposed. Use this for a finetune or language suffix that
|
||||
# will keep recurring across models.
|
||||
# pairs one specific candidate, matched in either order.
|
||||
# groups a set where no two members may ever be paired, for releases like a
|
||||
# per-language family where listing every pair would be unreadable.
|
||||
#
|
||||
# Things the job refuses structurally, so they do NOT need to be listed here:
|
||||
# different parameter sizes (8b vs 1.7b, a3b vs a22b, e2b vs e4b), entries with
|
||||
# a byte-identical install payload, and any target that already has a parent.
|
||||
|
||||
tokens:
|
||||
- {token: abliterated, reason: "safety-ablated finetune, not a re-quantization of the base weights"}
|
||||
- {token: uncensored, reason: "finetune, not a re-quantization of the base weights"}
|
||||
- {token: heretic, reason: "finetune, not a re-quantization of the base weights"}
|
||||
- {token: horror, reason: "finetune, not a re-quantization of the base weights"}
|
||||
- {token: glitter, reason: "finetune, not a re-quantization of the base weights"}
|
||||
- {token: starshine, reason: "finetune, not a re-quantization of the base weights"}
|
||||
- {token: deckard, reason: "finetune, not a re-quantization of the base weights"}
|
||||
- {token: esper, reason: "finetune, not a re-quantization of the base weights"}
|
||||
- {token: sft, reason: "supervised finetune, not a re-quantization of the base weights"}
|
||||
- {token: distill, reason: "distilled model, different weights from the teacher"}
|
||||
- {token: distilled, reason: "distilled model, different weights from the teacher"}
|
||||
- {token: roleplay, reason: "finetune, not a re-quantization of the base weights"}
|
||||
# `apex` and `qat` are quantization techniques and are deliberately NOT here:
|
||||
# the job strips them like any other quantization marker. A model that uses
|
||||
# "apex" as a finetune name instead belongs in `pairs` below.
|
||||
|
||||
pairs:
|
||||
# Aliases. Byte-identical payloads are not alternative builds. whisper-1
|
||||
# exists so OpenAI-compatible clients can send that name; grouping it under
|
||||
# whisper-base would hide the name clients actually send.
|
||||
- {parent: whisper-base, variant: whisper-1, reason: "alias: identical payload, whisper-1 is the name OpenAI-compatible clients send"}
|
||||
|
||||
# Different capability surface, not a different build. The multimodal entry
|
||||
# adds a vision projector and a different config; installing one instead of
|
||||
# the other silently changes what the model can do.
|
||||
- {parent: mistralai_mistral-small-3.1-24b-instruct-2503, variant: mistralai_mistral-small-3.1-24b-instruct-2503-multimodal, reason: "different capability surface: the multimodal entry adds a vision projector"}
|
||||
|
||||
# Safety-ablated finetunes that share a base name with the model they were
|
||||
# derived from. The token rule above already covers these; they are spelled
|
||||
# out because each was considered individually during the manual sweeps.
|
||||
- {parent: dolphin-2.9.2-phi-3-medium, variant: dolphin-2.9.2-phi-3-medium-abliterated, reason: "finetune, not a re-quantization"}
|
||||
- {parent: falcon3-1b-instruct, variant: falcon3-1b-instruct-abliterated, reason: "finetune, not a re-quantization"}
|
||||
- {parent: falcon3-3b-instruct, variant: falcon3-3b-instruct-abliterated, reason: "finetune, not a re-quantization"}
|
||||
- {parent: falcon3-10b-instruct, variant: falcon3-10b-instruct-abliterated, reason: "finetune, not a re-quantization"}
|
||||
- {parent: meta-llama-3.1-8b-instruct, variant: meta-llama-3.1-8b-instruct-abliterated, reason: "finetune, not a re-quantization"}
|
||||
- {parent: qwen3-30b-a3b, variant: qwen3-30b-a3b-abliterated, reason: "finetune, not a re-quantization"}
|
||||
|
||||
# English-only and multilingual ASR are different models trained on different
|
||||
# data. They share a repo and a filename shape, so the heuristics will keep
|
||||
# finding them.
|
||||
- {parent: whisper-tiny, variant: whisper-tiny-en, reason: "English-only versus multilingual ASR: different weights"}
|
||||
- {parent: whisper-base, variant: whisper-base-en, reason: "English-only versus multilingual ASR: different weights"}
|
||||
- {parent: whisper-small, variant: whisper-small-en, reason: "English-only versus multilingual ASR: different weights"}
|
||||
|
||||
# Different products that happen to share a prefix.
|
||||
- {parent: vibevoice-cpp, variant: vibevoice-cpp-asr, reason: "different products: a TTS engine and an ASR engine, not two builds of one model"}
|
||||
|
||||
groups:
|
||||
# One release per language. Every pair here is a different model.
|
||||
- members: [kokoros, kokoros-de, kokoros-ja, kokoros-cmn]
|
||||
reason: "different languages: one Kokoro release per language, not alternative builds of one model"
|
||||
Reference in New Issue
Block a user