mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 01:48:06 -04:00
feat(gallery): fix stale DFlash drafters and add the APEX families as variant ladders (#11027)
* fix(gallery): repoint qwen3-4b/qwen3.5-9b dflash drafters at post-rename GGUFs The drafters both entries referenced were converted from the pre-merge DFlash PR branch and carry dflash.target_layer_ids. llama.cpp reads dflash.target_layers and refuses the load. The stored values are offset by +1 relative to the HF-side field, so the files cannot be repaired by renaming the key and must be replaced. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): add apexentries HuggingFace client Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(apexentries): build the HF client via pkg/httpclient The apexentries HuggingFace client was constructed as a raw &http.Client{Timeout: 60s}. The repo convention (documented in .golangci.yml, which cannot express this as a forbidigo pattern) is that all outbound HTTP goes through pkg/httpclient, which refuses redirects by default and sets a TLS 1.2 floor. The std client follows redirects and forwards custom credential headers to the redirect target on a cross-host hop (GHSA-3mj3-57v2-4636). Only a User-Agent is sent today, but this calls an external API and an HF_TOKEN header added later would leak. Switch to httpclient.NewWithTimeout, preserving the 60 second timeout. No behaviour change for the current header set. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): discover APEX tiers by filename suffix Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): resolve unsloth counterparts and sharded quants Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): render APEX child entries with the dflash/mtp tag rule Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(apexentries): set backend, known_usecases and cross-repo drafters RenderChild left three gaps against the hand-written gallery entries. The generated entries reference gallery/virtual.yaml, which supplies no backend, so every generated entry named no engine at all. All comparable hand-written entries set backend: llama-cpp in overrides; do the same. Set known_usecases to [chat] alongside it: LocalAI falls back to the backend defaults when it is absent, so this is convention rather than breakage, but generated entries should not read differently from their neighbours. The drafter was also assumed to live in the repo publishing the weights. Speculative pairings routinely cross repos, and a drafter URI built from the weights repo 404s at install time. Add ChildInput.DraftRepo, used for both the drafter URI and its local path, falling back to Repo when empty so pairings that do ship the drafter alongside the weights are unchanged. The dflash/mtp tagging rule is untouched: the tag still follows SpecType and nothing else. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): dedupe generated entries against the existing gallery Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * apexentries: canonicalize HF URIs and dedup the generated batch Merge exists to stop a second gallery entry being added for weights the gallery already ships, but two gaps let duplicates through on a bulk run. The URI key was compared as an exact string while render.go only ever emits https://huggingface.co/{repo}/resolve/main/{file} and the gallery records 1038 of its URIs in huggingface://{repo}/{file} shorthand. A generated unsloth rung whose weights are already shipped in shorthand was therefore not recognised. canonicalURI reduces both spellings to one key and is applied on both sides, taking care that the repo is exactly the first two path segments so sharded quants in a subdirectory still match. A URI in neither form is returned untouched so other hosts dedup on their literal string. Merge also never accounted for entries it had just accepted, so two generated entries sharing a name or a primary URI both landed in add. Several APEX repos share one base model and resolve to the same unsloth counterpart, so the identical rungs are generated twice under the same name. Batch state is tracked locally rather than written back into the caller's ExistingIndex, which a caller may reasonably reuse. Name is still checked before URI: a name collision must block the add regardless of the weights. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): verify variant and tagging invariants in the gallery index Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(ci): scope the apex-entries verifier to what it can actually judge The verifier reported 60 problems against the real gallery, 57 of which were llama.cpp assumptions meeting entries from other backends. A gate that is wrong 57 times out of 60 cannot gate anything. - The weight-count check catches a quant label collision in llama-cpp quant discovery, so it now runs only for overrides.backend: llama-cpp. Entries with no declared backend are skipped because their weights are declared in the referenced url: template, which the verifier never reads. - The dflash/mtp tag check now implements the per-backend table in .agents/adding-gallery-models.md instead of assuming llama.cpp's spec_type: vocabulary. ds4 declares mtp_path:/mtp_draft:; sglang declares speculative_algorithm: in a file this verifier cannot follow, so sglang entries are not judged in either direction. The check stays bidirectional within the backends it does judge. - sha256 is now required on .gguf files only, since every non-GGUF asset in the index belongs to a hand-curated entry outside this generator's scope. Against the current gallery this leaves exactly the three genuine problems: two entries setting spec_type:draft-mtp without the mtp tag, and one entry whose overrides.mmproj names a file it does not download. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(apexentries): anchor quant matching and invert the sha256 rule UnaccountedQuants matched files to wanted quants with strings.Contains, which reproduces the substring collision it was written to warn about: Q8_0 is a substring of UD-Q8_0, so a repo publishing only UD-Q8_0 was reported as publishing an unbuilt Q8_0. Subdirectory-sharded UD quants are the normal unsloth layout for large repos, so this fired on realistic input. Match on the quant label as an anchored token instead, the way DiscoverUnslothQuants does, so the diagnostic and the discovery it audits cannot disagree about what a file is. Root-level shards, the layout the diagnostic mainly exists to catch, stay detected. The sha256 requirement was scoped to .gguf, which exempted seven real model weights: wan_2.1_vae.safetensors and clip_vision_h.safetensors across the wan-2.1-*-ggml entries, both load-bearing weights named by gallery/wan-ggml.yaml. Invert the rule so a checksum is required on everything except metadata extensions, which keeps a future weight format covered by default rather than silently exempt. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): wire the apexentries command Adds the generation path to the apexentries command: list the mudler APEX repos, discover each one's quality ladder and its unsloth counterpart's quant rungs from the filenames actually published, render a child entry per build plus a family parent carrying the variants list, dedup against the gallery, and write the additions to -out or append them with -apply. Discovery shortfalls are reported at discovery time rather than left to the verifier. A quant or a tier that discovery drops leaves no trace in a finished gallery file, and because an empty imatrix ladder falls back to the plain one, a repo whose imatrix filenames all fail to match downgrades the whole family silently instead of erroring. Merge's single reused map is split into two reported categories. A URI match means the gallery already ships exactly these weights and referencing the existing entry is correct; a name collision means an unrelated entry owns the name and referencing it would substitute a different build. Multimodal children now declare known_usecases [chat, vision]. An explicit known_usecases suppresses the backend-default fallback, so a chat-only entry carrying an mmproj never matches the vision or multimodal gallery filters. .github/ci is invisible to go list ./..., so a workflow names both generator packages explicitly and their specs finally run on pull requests. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(ci): gather APEX builds under the base model entry The hub for a family is the BASE model entry, never a generated *-apex parent. Somebody looking for qwen3.6-35b-a3b has to find every build of those weights under that one name, so a competing qwen3.6-35b-a3b-apex hub would split the family and leave half of it invisible. When the gallery already ships the base entry, a variants block is spliced into it textually, leaving its description, icon, tags, overrides and files untouched. Only a family whose base model the gallery does not ship gets a new hub, still named for the base model and carrying one of the discovered builds as its own payload so it declares a backend the verifier can judge. The line editing is factored into .github/ci/galleryedit, shared with the variantproposals job, so the two cannot drift apart on where a variants block belongs. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(apexentries): treat an unreadable optional counterpart repo as absent HuggingFace answers 401 Unauthorized, not 404, for a repository that does not exist when the request carries no credentials. FetchRepoFiles treated only 404 as absence, so probing for the OPTIONAL unsloth counterpart hard failed for every family that legitimately has none: 27 of the 45 APEX families are community merges that will never have an unsloth build, and a full run failed all of them. Split the fetch so the two call sites can apply different policies to the same response. The APEX repo itself stays strict: a 401 or 403 on a repo the run requires is a real failure and still errors. Only the optional probe tolerates it, because without a token 401 cannot be told apart from absence. That collapse is lossy in one direction, since a private or gated repo also answers 401, so the skipped candidates are named in the run summary alongside the other silent-shortfall counters instead of being dropped in silence. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(apexentries): report full-precision sources as a known exclusion The 45 APEX repos publish their unquantized F16 sources next to the imatrix ladder, flat or sharded. Discovery correctly emits nothing for them, but they were landing in the unclassified total, leaving a permanent baseline of 24 benign lines on every run. That baseline is what the unclassified check exists to prevent: a standing count of known-benign files is exactly what hides the one file that ever genuinely matters. Count full-precision sources separately and give them their own summary line, so unclassified returns to 0 and stays loud when something really is an unknown shape. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * fix(apexentries): namespace local paths by owner and enable MTP builds localPath namespaced downloads by the repo basename alone, so two repos publishing the same filename under different owners collapsed to one local path. LiquidAI/LFM2.5-8B-A1B-GGUF and unsloth/LFM2.5-8B-A1B-GGUF collided that way, and both were offered from the same hub, so installing the second either overwrote the first model's weights or was skipped as already present while recording a sha256 that did not match the bytes on disk. The owner is now its own path segment: owner/repo is globally unique on HuggingFace and neither half can contain a separator, so uniqueness holds by construction. Verify gains a check for the whole class, that no local filename may map to two different upstream URIs. It surfaces seven pre-existing collisions in the gallery, which are left alone here. Entries built from the *-APEX-MTP-GGUF repos now configure MTP rather than shipping the heads inert, matching the pattern the hand-written MTP entries already use: spec_type:draft-mtp with spec_n_max and spec_p_min, tagged mtp, and no draft_model because the heads live in the weights. RenderChild no longer requires a separate drafter file before it will configure a spec type, while the cross-repo drafter path is unchanged. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * feat(gallery): add the APEX GGUF families as variant ladders Adds the imatrix quality ladder from each mudler/*-APEX-GGUF repo, a fixed subset of unsloth quant rungs where a counterpart repo exists, and the MTP builds, then attaches them to the base model entry so one entry offers every build of the same weights and LocalAI picks the one that fits the hardware. Ten existing base model entries gain a variants list; twenty-seven families that the gallery had no base entry for get one. Builds are discovered from the filenames each repo actually publishes rather than derived from its name, since six repos ship a stem that differs from their repo name. Every file carries a sha256 taken from the HuggingFace API. Assisted-by: Claude Opus 4.8 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
This commit is contained in:
mudler's LocalAI [bot]
committed by
GitHub
GitHub
parent
a4a181d2f7
commit
5c96e097ba
194
.github/ci/apexentries/README.md
vendored
Normal file
194
.github/ci/apexentries/README.md
vendored
Normal file
@@ -0,0 +1,194 @@
|
||||
# apexentries
|
||||
|
||||
Generates gallery entries for the `mudler/*-APEX-GGUF` HuggingFace repositories.
|
||||
|
||||
Each APEX repo becomes one **family**: one entry per quality rung the repo
|
||||
publishes and one per quantization rung its unsloth counterpart publishes, all
|
||||
gathered under the **base model's** entry. LocalAI's variant selector then picks
|
||||
the build that fits the hardware in front of it.
|
||||
|
||||
## The hub is the base model entry, never a generated `*-apex` parent
|
||||
|
||||
Somebody looking for `qwen3.6-35b-a3b` must find every build of those weights
|
||||
under that one name: the APEX imatrix rungs, the unsloth quant rungs and any
|
||||
speculative build. A separate `qwen3.6-35b-a3b-apex` hub competing with the base
|
||||
entry would split the family in two and leave whichever half the user did not
|
||||
search for effectively invisible.
|
||||
|
||||
So the generator resolves the hub by stripping the `-APEX`, `-MTP` and `-TQ`
|
||||
markers and looking the result up in the index, trying both the repo-derived and
|
||||
the stem-derived candidate the same way `CounterpartCandidates` does. Then:
|
||||
|
||||
- **The hub exists** (14 of the 45 repos, resolving to 10 distinct entries).
|
||||
Nothing new is emitted for the family root. A `variants:` block is spliced into
|
||||
the entry that is already there, textually, leaving its description, icon,
|
||||
tags, overrides and files untouched. The line editing is shared with the
|
||||
`variantproposals` job via `.github/ci/galleryedit`.
|
||||
- **The hub is absent** (the other 31). A new hub is emitted, named for the base
|
||||
model and never for the APEX repo. It carries one of the discovered builds as
|
||||
its own payload so it is a complete installable entry rather than a bare index,
|
||||
and that payload is what gives it an `overrides.backend`. Without a declared
|
||||
backend the verifier would skip it, so a hub carrying feature tags would escape
|
||||
the tagging check in silence.
|
||||
|
||||
Several APEX repos routinely resolve to one base model, so both paths accumulate
|
||||
by hub name rather than assuming one family per hub.
|
||||
|
||||
Two references are always filtered out of a hub's list: anything the entry
|
||||
already declares, and the hub's own name. The self reference is not merely
|
||||
redundant. An unsloth rung whose weights the gallery already ships under the base
|
||||
name resolves, through the merge, straight back to the hub, and the verifier
|
||||
reads a self reference as a variant that declares variants of its own.
|
||||
|
||||
The four hand-written `*-apex` entries (`qwen3.6-35b-a3b-apex`,
|
||||
`gemma-4-26b-a4b-it-apex`, `qwen3.5-35b-a3b-apex`,
|
||||
`nemotron-3-nano-omni-30b-a3b-reasoning-apex`) are **ordinary builds**, not hubs.
|
||||
They are referenced from their hub's variants list like any other rung, and are
|
||||
never deleted or renamed.
|
||||
|
||||
## Flags
|
||||
|
||||
| Flag | Default | Meaning |
|
||||
|------|---------|---------|
|
||||
| `-index <path>` | `gallery/index.yaml` | Gallery index to dedup against. Read only, unless `-apply` is passed. |
|
||||
| `-only <a,b,c>` | (all) | Comma-separated full repo names (`mudler/Foo-APEX-GGUF`) to restrict generation to. A name that matches nothing is reported as a warning, since it is a typo rather than an empty result. |
|
||||
| `-out <path>` | (none) | Write the entries to add to this file. Nothing is written to the gallery. |
|
||||
| `-apply` | `false` | Splice the variants into `-index` and append the new entries to it. |
|
||||
| `-verify <path>` | (none) | Verify a gallery index and exit. Ignores every other flag. |
|
||||
|
||||
Either `-out` or `-apply` is required, otherwise the run has nothing to do.
|
||||
|
||||
`-apply` splices variant lines into existing entries and **appends** new ones. It
|
||||
never re-serialises the index: it is roughly 40,000 lines, and a YAML round trip
|
||||
would reflow the whole file, drop the anchors and merge keys the gallery relies
|
||||
on, and produce a diff nobody can review. On the three-family sample the splice
|
||||
is 24 added lines across 3 hunks with zero deletions.
|
||||
|
||||
## Discovery is by filename suffix, never by repo name
|
||||
|
||||
Builds come from the files a repo actually publishes. A filename is never
|
||||
constructed from a repo name, because the two disagree:
|
||||
`mudler/gemma-4-26B-A4B-it-APEX-GGUF` ships `gemma-4-26B-A4B-APEX-*.gguf`, and
|
||||
five other repos likewise drop a suffix (`-it`, `-2603`) or a vendor prefix
|
||||
(`NVIDIA-`) that the repo name carries. Composing a URL from the repo name would
|
||||
produce a 404 for every one of them, and the 404 would only surface after the
|
||||
entry shipped.
|
||||
|
||||
The quality ladder is matched on the trailing tier marker, `-(I-)?(Quality|
|
||||
Balanced|Compact|Mini|Nano).gguf`. The `I-` prefix marks the imatrix ladder. The
|
||||
imatrix ladder is emitted when it is non-empty and the plain ladder is used only
|
||||
as a fallback, because two of the 45 repos publish no imatrix tiers at all and
|
||||
must still contribute. Eleven repos carry a fifth `I-Nano` rung, so nothing
|
||||
assumes a fixed number of rungs.
|
||||
|
||||
Every run prints, per repo, the counts that discovery accounted for. If the
|
||||
number of classified files is short of the number of `.gguf` files the repo
|
||||
publishes, the shortfall is printed as `UNCLASSIFIED`. That check is a set
|
||||
difference on counts rather than a second pass over filenames: a second matcher
|
||||
would duplicate the tier regex and the two copies would drift. The failure it
|
||||
catches is quiet. A publishing-script typo that breaks every imatrix filename in
|
||||
a repo does not produce a short ladder; it makes the imatrix ladder empty, and
|
||||
the fallback then downgrades the whole family to the plain ladder with nothing
|
||||
said. A downstream HTTP check cannot catch it either, because it validates the
|
||||
URLs that were emitted, and an undiscovered tier emits none.
|
||||
|
||||
The same reasoning applies to `UNACCOUNTED QUANT`, printed when the unsloth
|
||||
counterpart demonstrably publishes a wanted quant that produced no build. It is
|
||||
reported at discovery time because a dropped quant leaves no trace at all in the
|
||||
finished gallery file.
|
||||
|
||||
## sha256 always comes from the API
|
||||
|
||||
Every file stanza takes its `sha256` from the HuggingFace models API
|
||||
(`lfs.sha256`). A GGUF the API describes without one is a fatal error for that
|
||||
family: the repo is reported by name and the run ends non-zero. It is never
|
||||
substituted from another field, because that is exactly how a Xet hash ends up
|
||||
masquerading as a content hash.
|
||||
|
||||
## The dflash / mtp tagging rule
|
||||
|
||||
An entry is tagged `dflash` or `mtp` **if and only if** it configures the
|
||||
matching `spec_type:draft-<feature>`. Variant ranking reads tags and nothing
|
||||
else, so a tag that does not match the configuration either promotes a build
|
||||
that is no faster or hides one that genuinely is.
|
||||
|
||||
A repo name is not configuration. `mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF` ships
|
||||
weights that carry MTP heads; an entry that does not enable them is not an MTP
|
||||
entry and is not tagged as one.
|
||||
|
||||
A generated hub inherits the tags of the build it carries as its payload, rather
|
||||
than rebuilding them from the base set, so a hub whose payload configures a
|
||||
`spec_type` stays tagged consistently with the overrides copied alongside it.
|
||||
|
||||
## Reuse reporting: two categories, not one
|
||||
|
||||
Generated entries are deduped against the gallery and against the batch itself.
|
||||
The run prints the result under two separate headings, because the two cases are
|
||||
not equivalent:
|
||||
|
||||
- **URI MATCHES** mean the gallery, or an earlier entry in this batch, already
|
||||
ships exactly these weights. Pointing the hub at the existing entry is correct
|
||||
and needs no thought.
|
||||
- **NAME COLLISIONS** mean an entry already owns the name but holds different
|
||||
weights. Referencing it would point the hub at a build other than the one
|
||||
generated. Every one of these must be inspected by hand.
|
||||
|
||||
The run then prints `HUBS SPLICED`, listing every reference that will be added to
|
||||
an entry the gallery already ships along with the line it will be added at, and
|
||||
`HUBS CREATED` for the families that get a new hub. The splices are the part a
|
||||
review has to read closely, because they modify entries somebody else wrote.
|
||||
|
||||
Hubs are deliberately kept out of the merge. A new hub carries the family's top
|
||||
rung as its own payload, so URI dedup would fold the hub into that rung and the
|
||||
family would lose the very entry point this command exists to create.
|
||||
|
||||
## Workflow: sample first, then the full set
|
||||
|
||||
Never run the full generation straight into the gallery. Generate a small,
|
||||
deliberately awkward sample, have it reviewed, then run the rest.
|
||||
|
||||
```bash
|
||||
# 1. Sample three families that between them cover the awkward shapes:
|
||||
# a standard four-rung repo, one with the extra I-Nano rung AND a file stem
|
||||
# that differs from its repo name, and one whose unsloth counterpart shards
|
||||
# its quants across subdirectories.
|
||||
go run ./.github/ci/apexentries \
|
||||
-index gallery/index.yaml \
|
||||
-only mudler/Qwen3.6-35B-A3B-APEX-GGUF,mudler/gemma-4-26B-A4B-it-APEX-GGUF,mudler/Step-3.7-Flash-APEX-GGUF \
|
||||
-out /tmp/sample.yaml
|
||||
|
||||
# 2. Verify the sample against the gallery it would join, splices included. Apply
|
||||
# to a COPY, never to the real index, and check that the diff is only the
|
||||
# intended variant lines. Compare the verifier output to the gallery's own
|
||||
# baseline: what matters is that the sample adds no new problem, not that the
|
||||
# total is zero.
|
||||
cp gallery/index.yaml /tmp/index-copy.yaml
|
||||
go run ./.github/ci/apexentries -index /tmp/index-copy.yaml -only <same list> -apply
|
||||
diff -u gallery/index.yaml /tmp/index-copy.yaml # expect zero deletions
|
||||
|
||||
go run ./.github/ci/apexentries -verify gallery/index.yaml > /tmp/baseline.log 2>&1
|
||||
go run ./.github/ci/apexentries -verify /tmp/index-copy.yaml > /tmp/spliced.log 2>&1
|
||||
diff /tmp/baseline.log /tmp/spliced.log
|
||||
|
||||
# 3. Have a human review /tmp/sample.yaml and every reported name collision.
|
||||
|
||||
# 4. Only then, the full set.
|
||||
go run ./.github/ci/apexentries -index gallery/index.yaml -apply
|
||||
```
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
go test ./.github/ci/apexentries/
|
||||
```
|
||||
|
||||
The shared line editor has its own package:
|
||||
|
||||
```bash
|
||||
go test ./.github/ci/galleryedit/
|
||||
```
|
||||
|
||||
`.github/ci/` is invisible to `go list ./...`, so these specs are not covered by
|
||||
`make lint` or the repository test run. `.github/workflows/ci-tools-tests.yaml`
|
||||
names the package explicitly; keep that workflow in step with any package added
|
||||
under `.github/ci/`.
|
||||
70
.github/ci/apexentries/discover.go
vendored
Normal file
70
.github/ci/apexentries/discover.go
vendored
Normal file
@@ -0,0 +1,70 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// tierRE matches the tier marker APEX repos put at the end of a weight
|
||||
// filename. Discovery is by suffix because the stem is not predictable from
|
||||
// the repo name: six of the 45 repos drop a suffix ("-it", "-2603") or a
|
||||
// vendor prefix ("NVIDIA-") that the repo name carries.
|
||||
var tierRE = regexp.MustCompile(`-(I-)?(Quality|Balanced|Compact|Mini|Nano)\.gguf$`)
|
||||
|
||||
// fullPrecisionRE matches the unquantized source weights an APEX repo publishes
|
||||
// alongside its ladder, flat (-F16.gguf) or sharded across a numbered set
|
||||
// (-F16-00001-of-00010.gguf). bf16 is accepted because some repos publish that
|
||||
// instead, and the match is case-insensitive because the casing varies between
|
||||
// publishing scripts.
|
||||
//
|
||||
// These are deliberately not tiers: they are the weights the ladder is quantized
|
||||
// FROM, and generation is scoped to the ladder itself.
|
||||
var fullPrecisionRE = regexp.MustCompile(`(?i)-b?f16(-\d{5}-of-\d{5})?\.gguf$`)
|
||||
|
||||
// IsFullPrecision reports whether a weight filename is an unquantized source.
|
||||
func IsFullPrecision(name string) bool {
|
||||
return fullPrecisionRE.MatchString(name)
|
||||
}
|
||||
|
||||
// Tier is one discovered build of an APEX repo.
|
||||
type Tier struct {
|
||||
Label string
|
||||
File GGUFFile
|
||||
}
|
||||
|
||||
// DiscoverAPEXTiers splits a repo's weight files into the imatrix ladder and
|
||||
// the plain ladder. mmproj files are never tiers.
|
||||
func DiscoverAPEXTiers(files []GGUFFile) (imatrix, plain []Tier) {
|
||||
for _, f := range files {
|
||||
if strings.HasPrefix(f.Name, "mmproj") {
|
||||
continue
|
||||
}
|
||||
m := tierRE.FindStringSubmatch(f.Name)
|
||||
if m == nil {
|
||||
continue
|
||||
}
|
||||
if m[1] != "" {
|
||||
imatrix = append(imatrix, Tier{Label: "I-" + m[2], File: f})
|
||||
continue
|
||||
}
|
||||
plain = append(plain, Tier{Label: m[2], File: f})
|
||||
}
|
||||
return imatrix, plain
|
||||
}
|
||||
|
||||
// DiscoverMMProj returns the repo's projector file, if it publishes one. The
|
||||
// name varies across repos (mmproj.gguf, mmproj-F16.gguf,
|
||||
// mmproj-step3.7-flash-f16.gguf), so match the prefix rather than a fixed name.
|
||||
func DiscoverMMProj(files []GGUFFile) (GGUFFile, bool) {
|
||||
for _, f := range files {
|
||||
if strings.HasPrefix(f.Name, "mmproj") {
|
||||
return f, true
|
||||
}
|
||||
}
|
||||
return GGUFFile{}, false
|
||||
}
|
||||
|
||||
// FileStem returns a tier's filename with its tier suffix removed.
|
||||
func FileStem(t Tier) string {
|
||||
return tierRE.ReplaceAllString(t.File.Name, "")
|
||||
}
|
||||
68
.github/ci/apexentries/discover_test.go
vendored
Normal file
68
.github/ci/apexentries/discover_test.go
vendored
Normal file
@@ -0,0 +1,68 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("DiscoverAPEXTiers", func() {
|
||||
It("finds tiers regardless of how the stem relates to the repo name", func() {
|
||||
// This repo is mudler/gemma-4-26B-A4B-it-APEX-GGUF but its files drop "-it".
|
||||
files := []GGUFFile{
|
||||
{Name: "gemma-4-26B-A4B-APEX-I-Quality.gguf", SHA256: "a"},
|
||||
{Name: "gemma-4-26B-A4B-APEX-I-Nano.gguf", SHA256: "b"},
|
||||
{Name: "gemma-4-26B-A4B-APEX-Quality.gguf", SHA256: "c"},
|
||||
{Name: "mmproj-F16.gguf", SHA256: "d"},
|
||||
}
|
||||
|
||||
imatrix, plain := DiscoverAPEXTiers(files)
|
||||
|
||||
Expect(labels(imatrix)).To(ConsistOf("I-Quality", "I-Nano"))
|
||||
Expect(labels(plain)).To(ConsistOf("Quality"))
|
||||
})
|
||||
|
||||
It("excludes mmproj from the tier list", func() {
|
||||
files := []GGUFFile{{Name: "mmproj.gguf", SHA256: "d"}}
|
||||
|
||||
imatrix, plain := DiscoverAPEXTiers(files)
|
||||
|
||||
Expect(imatrix).To(BeEmpty())
|
||||
Expect(plain).To(BeEmpty())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("DiscoverMMProj", func() {
|
||||
It("finds an mmproj whatever its suffix", func() {
|
||||
files := []GGUFFile{
|
||||
{Name: "Model-APEX-I-Mini.gguf", SHA256: "a"},
|
||||
{Name: "mmproj-step3.7-flash-f16.gguf", SHA256: "b"},
|
||||
}
|
||||
|
||||
got, ok := DiscoverMMProj(files)
|
||||
|
||||
Expect(ok).To(BeTrue())
|
||||
Expect(got.Name).To(Equal("mmproj-step3.7-flash-f16.gguf"))
|
||||
})
|
||||
|
||||
It("reports absence when the repo ships none", func() {
|
||||
_, ok := DiscoverMMProj([]GGUFFile{{Name: "Model-APEX-Quality.gguf", SHA256: "a"}})
|
||||
|
||||
Expect(ok).To(BeFalse())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("FileStem", func() {
|
||||
It("strips the tier suffix", func() {
|
||||
t := Tier{Label: "I-Quality", File: GGUFFile{Name: "gemma-4-26B-A4B-APEX-I-Quality.gguf"}}
|
||||
|
||||
Expect(FileStem(t)).To(Equal("gemma-4-26B-A4B-APEX"))
|
||||
})
|
||||
})
|
||||
|
||||
func labels(ts []Tier) []string {
|
||||
out := make([]string, 0, len(ts))
|
||||
for _, t := range ts {
|
||||
out = append(out, t.Label)
|
||||
}
|
||||
return out
|
||||
}
|
||||
130
.github/ci/apexentries/hf.go
vendored
Normal file
130
.github/ci/apexentries/hf.go
vendored
Normal file
@@ -0,0 +1,130 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/mudler/LocalAI/pkg/httpclient"
|
||||
)
|
||||
|
||||
// ErrNoSHA256 marks a GGUF the HuggingFace API describes without an
|
||||
// lfs.sha256. Emitting an entry without a hash would ship an unverifiable
|
||||
// download, and guessing one from another field is how a Xet hash ends up
|
||||
// masquerading as a content hash, so this is fatal rather than skippable.
|
||||
var ErrNoSHA256 = errors.New("gguf file has no lfs.sha256")
|
||||
|
||||
// GGUFFile is one .gguf sibling of a HuggingFace repo.
|
||||
type GGUFFile struct {
|
||||
Name string
|
||||
Size int64
|
||||
SHA256 string
|
||||
}
|
||||
|
||||
type apiSibling struct {
|
||||
RFilename string `json:"rfilename"`
|
||||
Size int64 `json:"size"`
|
||||
LFS *struct {
|
||||
SHA256 string `json:"sha256"`
|
||||
} `json:"lfs"`
|
||||
}
|
||||
|
||||
type apiModel struct {
|
||||
Siblings []apiSibling `json:"siblings"`
|
||||
}
|
||||
|
||||
// ParseRepoFiles returns every .gguf sibling described by a models API body.
|
||||
func ParseRepoFiles(body []byte) ([]GGUFFile, error) {
|
||||
var m apiModel
|
||||
if err := json.Unmarshal(body, &m); err != nil {
|
||||
return nil, fmt.Errorf("decoding model response: %w", err)
|
||||
}
|
||||
|
||||
var out []GGUFFile
|
||||
for _, s := range m.Siblings {
|
||||
if !strings.HasSuffix(s.RFilename, ".gguf") {
|
||||
continue
|
||||
}
|
||||
if s.LFS == nil || s.LFS.SHA256 == "" {
|
||||
return nil, fmt.Errorf("%s: %w", s.RFilename, ErrNoSHA256)
|
||||
}
|
||||
out = append(out, GGUFFile{Name: s.RFilename, Size: s.Size, SHA256: s.LFS.SHA256})
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// FetchOptionalRepoFiles asks the models API for a repo the caller can do
|
||||
// without, and reports separately whether the repo was merely unreadable.
|
||||
//
|
||||
// HuggingFace answers 401 Unauthorized, not 404, for a repo that does not exist
|
||||
// when the request carries no credentials. Without a token there is therefore no
|
||||
// way to tell "this repo was never published" from "this repo is private", so an
|
||||
// optional probe has to treat 401 and 403 exactly like 404: whatever the reason,
|
||||
// there is nothing here for us to read, so there is no counterpart.
|
||||
//
|
||||
// The second return value exists because that collapse is lossy in one
|
||||
// direction: 401/403 can also mean a real, gated repo whose quants we would
|
||||
// genuinely want. The caller reports those repos so a silently dropped
|
||||
// counterpart is visible to a human rather than invisible.
|
||||
func FetchOptionalRepoFiles(client *http.Client, repo string) ([]GGUFFile, bool, error) {
|
||||
files, status, err := fetchRepoFiles(client, repo)
|
||||
if err != nil && (status == http.StatusUnauthorized || status == http.StatusForbidden) {
|
||||
return nil, true, nil
|
||||
}
|
||||
return files, false, err
|
||||
}
|
||||
|
||||
// FetchRepoFiles asks the models API for one repo. A 404 yields (nil, nil) so
|
||||
// that probing for an optional counterpart repo is not an error. Every other
|
||||
// non-200, 401 and 403 included, is an error: for a repo the run REQUIRES there
|
||||
// is no benign reading of "we cannot see it".
|
||||
func FetchRepoFiles(client *http.Client, repo string) ([]GGUFFile, error) {
|
||||
files, _, err := fetchRepoFiles(client, repo)
|
||||
return files, err
|
||||
}
|
||||
|
||||
// fetchRepoFiles does the request and returns the HTTP status alongside the
|
||||
// result, so the optional and required callers can apply different policies to
|
||||
// the same response without duplicating the request.
|
||||
func fetchRepoFiles(client *http.Client, repo string) ([]GGUFFile, int, error) {
|
||||
url := fmt.Sprintf("https://huggingface.co/api/models/%s?blobs=true", repo)
|
||||
req, err := http.NewRequest(http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
req.Header.Set("User-Agent", "localai-apexentries/1.0")
|
||||
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode == http.StatusNotFound {
|
||||
return nil, resp.StatusCode, nil
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, resp.StatusCode, fmt.Errorf("%s: unexpected status %d", repo, resp.StatusCode)
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, resp.StatusCode, err
|
||||
}
|
||||
files, err := ParseRepoFiles(body)
|
||||
return files, resp.StatusCode, err
|
||||
}
|
||||
|
||||
// newHTTPClient builds the client used against the HuggingFace API. It goes
|
||||
// through pkg/httpclient rather than a bare &http.Client{} because the std
|
||||
// client follows redirects and forwards custom credential headers to the
|
||||
// redirect target on a cross-host hop (GHSA-3mj3-57v2-4636). This caller sends
|
||||
// only a User-Agent today, but it talks to an external API that could start
|
||||
// redirecting, and an HF_TOKEN header here later would then leak.
|
||||
func newHTTPClient() *http.Client {
|
||||
return httpclient.NewWithTimeout(60 * time.Second)
|
||||
}
|
||||
142
.github/ci/apexentries/hf_test.go
vendored
Normal file
142
.github/ci/apexentries/hf_test.go
vendored
Normal file
@@ -0,0 +1,142 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io"
|
||||
"net/http"
|
||||
"testing"
|
||||
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
func TestApexEntries(t *testing.T) {
|
||||
RegisterFailHandler(Fail)
|
||||
RunSpecs(t, "apexentries")
|
||||
}
|
||||
|
||||
// stubTransport answers every request with one canned status and body, so the
|
||||
// status handling of the fetchers can be exercised without reaching the real
|
||||
// HuggingFace API.
|
||||
type stubTransport struct {
|
||||
status int
|
||||
body string
|
||||
}
|
||||
|
||||
func (t stubTransport) RoundTrip(req *http.Request) (*http.Response, error) {
|
||||
return &http.Response{
|
||||
StatusCode: t.status,
|
||||
Body: io.NopCloser(bytes.NewBufferString(t.body)),
|
||||
Header: make(http.Header),
|
||||
Request: req,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func stubClient(status int, body string) *http.Client {
|
||||
return &http.Client{Transport: stubTransport{status: status, body: body}}
|
||||
}
|
||||
|
||||
const oneGGUFBody = `{"siblings":[{"rfilename":"Model-APEX-I-Quality.gguf","size":10,"lfs":{"sha256":"aa","size":10}}]}`
|
||||
|
||||
var _ = Describe("FetchOptionalRepoFiles", func() {
|
||||
// HuggingFace answers 401 rather than 404 for a repo that does not exist
|
||||
// when the client carries no credentials, so an optional probe cannot tell
|
||||
// "absent" from "unauthorized" and must treat both as "no counterpart".
|
||||
It("treats a 401 as an absent repo and flags it as unavailable", func() {
|
||||
files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusUnauthorized, ""), "unsloth/Nope-GGUF")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(files).To(BeEmpty())
|
||||
Expect(unavailable).To(BeTrue())
|
||||
})
|
||||
|
||||
It("treats a 403 as an absent repo and flags it as unavailable", func() {
|
||||
files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusForbidden, ""), "unsloth/Gated-GGUF")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(files).To(BeEmpty())
|
||||
Expect(unavailable).To(BeTrue())
|
||||
})
|
||||
|
||||
// A clean 404 is an unambiguous absence, so it must NOT be reported as
|
||||
// unavailable: the whole point of the flag is to separate the ambiguous
|
||||
// case a human may need to look at from the settled one.
|
||||
It("treats a 404 as an absent repo without flagging it as unavailable", func() {
|
||||
files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusNotFound, ""), "unsloth/Nope-GGUF")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(files).To(BeEmpty())
|
||||
Expect(unavailable).To(BeFalse())
|
||||
})
|
||||
|
||||
It("parses a 200 body as usual", func() {
|
||||
files, unavailable, err := FetchOptionalRepoFiles(stubClient(http.StatusOK, oneGGUFBody), "unsloth/Real-GGUF")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(unavailable).To(BeFalse())
|
||||
Expect(files).To(HaveLen(1))
|
||||
Expect(files[0].Name).To(Equal("Model-APEX-I-Quality.gguf"))
|
||||
Expect(files[0].SHA256).To(Equal("aa"))
|
||||
})
|
||||
|
||||
// Tolerating 401/403 must not widen into tolerating everything: a 500 is a
|
||||
// broken API, not evidence about whether the repo exists.
|
||||
It("still errors on a 500", func() {
|
||||
_, _, err := FetchOptionalRepoFiles(stubClient(http.StatusInternalServerError, ""), "unsloth/Real-GGUF")
|
||||
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("unexpected status 500"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("FetchRepoFiles", func() {
|
||||
// The APEX repo itself is not optional. A 401 there means the repo the run
|
||||
// was asked to publish cannot be read, which is a real failure and must not
|
||||
// be quietly downgraded to "no files".
|
||||
It("errors on a 401 for a required repo", func() {
|
||||
_, err := FetchRepoFiles(stubClient(http.StatusUnauthorized, ""), "mudler/Model-APEX-GGUF")
|
||||
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("unexpected status 401"))
|
||||
})
|
||||
|
||||
It("errors on a 403 for a required repo", func() {
|
||||
_, err := FetchRepoFiles(stubClient(http.StatusForbidden, ""), "mudler/Model-APEX-GGUF")
|
||||
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("unexpected status 403"))
|
||||
})
|
||||
|
||||
It("still treats a 404 as an absent repo", func() {
|
||||
files, err := FetchRepoFiles(stubClient(http.StatusNotFound, ""), "mudler/Model-APEX-GGUF")
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(files).To(BeEmpty())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("ParseRepoFiles", func() {
|
||||
It("returns gguf siblings with their lfs sha256", func() {
|
||||
body := []byte(`{"siblings":[
|
||||
{"rfilename":"Model-APEX-I-Quality.gguf","size":10,"lfs":{"sha256":"aa","size":10}},
|
||||
{"rfilename":"README.md"},
|
||||
{"rfilename":"mmproj.gguf","size":5,"lfs":{"sha256":"bb","size":5}}
|
||||
]}`)
|
||||
|
||||
files, err := ParseRepoFiles(body)
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(files).To(HaveLen(2))
|
||||
Expect(files[0].Name).To(Equal("Model-APEX-I-Quality.gguf"))
|
||||
Expect(files[0].SHA256).To(Equal("aa"))
|
||||
Expect(files[1].Name).To(Equal("mmproj.gguf"))
|
||||
})
|
||||
|
||||
It("reports a gguf that carries no lfs sha256", func() {
|
||||
body := []byte(`{"siblings":[{"rfilename":"mmproj.gguf","size":5}]}`)
|
||||
|
||||
_, err := ParseRepoFiles(body)
|
||||
|
||||
Expect(err).To(MatchError(ErrNoSHA256))
|
||||
})
|
||||
})
|
||||
143
.github/ci/apexentries/hub.go
vendored
Normal file
143
.github/ci/apexentries/hub.go
vendored
Normal file
@@ -0,0 +1,143 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
|
||||
"github.com/mudler/LocalAI/.github/ci/galleryedit"
|
||||
)
|
||||
|
||||
// IndexText is the gallery index seen as text: the entries it declares plus the
|
||||
// exact lines each one occupies, which is what splicing a variants block into an
|
||||
// entry the gallery already ships requires.
|
||||
//
|
||||
// It is a second, narrower read of the same file LoadExisting parses. The two
|
||||
// answer different questions: LoadExisting answers "do these weights already
|
||||
// exist anywhere", this one answers "where in the file does this entry live".
|
||||
type IndexText struct {
|
||||
Lines []string
|
||||
Entries []*indexEntry
|
||||
|
||||
byName map[string]*indexEntry
|
||||
}
|
||||
|
||||
// indexEntry is one entry of the index: its name, the variants it already
|
||||
// declares, and its coordinates in the file.
|
||||
type indexEntry struct {
|
||||
Name string `yaml:"name"`
|
||||
Variants []VariantRef `yaml:"variants"`
|
||||
|
||||
Pos galleryedit.Entry `yaml:"-"`
|
||||
}
|
||||
|
||||
// LoadIndexText reads the gallery index for editing.
|
||||
func LoadIndexText(path string) (*IndexText, error) {
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return ParseIndexText(string(raw))
|
||||
}
|
||||
|
||||
// ParseIndexText pairs the decoded entries with the top level list items the
|
||||
// text actually contains.
|
||||
//
|
||||
// If the two views disagree on how many entries there are then every line number
|
||||
// a splice would compute is suspect, and the failure mode is writing a variants
|
||||
// block into the wrong model. The parse refuses instead.
|
||||
func ParseIndexText(text string) (*IndexText, error) {
|
||||
var entries []*indexEntry
|
||||
if err := yaml.Unmarshal([]byte(text), &entries); err != nil {
|
||||
return nil, fmt.Errorf("decoding gallery index: %w", err)
|
||||
}
|
||||
|
||||
lines, starts := galleryedit.Scan(text)
|
||||
if len(starts) != len(entries) {
|
||||
return nil, fmt.Errorf("gallery index has %d decoded entries but %d top level list items; refusing to edit by line number",
|
||||
len(entries), len(starts))
|
||||
}
|
||||
|
||||
ix := &IndexText{Lines: lines, Entries: entries, byName: map[string]*indexEntry{}}
|
||||
for i, e := range entries {
|
||||
if e == nil {
|
||||
return nil, fmt.Errorf("gallery index list item %d is empty; refusing to edit by line number", i)
|
||||
}
|
||||
end := len(lines)
|
||||
if i+1 < len(starts) {
|
||||
end = starts[i+1]
|
||||
}
|
||||
e.Pos = galleryedit.Entry{Name: e.Name, StartLine: starts[i], EndLine: end}
|
||||
|
||||
// First occurrence wins, matching the gallery's own resolution.
|
||||
key := strings.ToLower(e.Name)
|
||||
if _, seen := ix.byName[key]; !seen {
|
||||
ix.byName[key] = e
|
||||
}
|
||||
}
|
||||
return ix, nil
|
||||
}
|
||||
|
||||
// Find looks an entry up by name, case insensitively.
|
||||
func (ix *IndexText) Find(name string) *indexEntry {
|
||||
return ix.byName[strings.ToLower(name)]
|
||||
}
|
||||
|
||||
// ResolveHub returns the gallery name of a family's hub and whether the gallery
|
||||
// already ships an entry under it.
|
||||
//
|
||||
// The hub is the BASE model entry, never a generated *-apex parent. Somebody
|
||||
// looking for qwen3.6-35b-a3b has to find every build of those weights under
|
||||
// that one name: the APEX imatrix rungs, the unsloth quant rungs and any
|
||||
// speculative build. A separate qwen3.6-35b-a3b-apex hub competing with the base
|
||||
// entry would split the family in two and leave whichever half the user did not
|
||||
// search for invisible.
|
||||
//
|
||||
// Both candidates are tried for the same reason CounterpartCandidates tries
|
||||
// both. The repo name and the published file stem disagree for several of these
|
||||
// repos, and either one may be what the base entry was named after.
|
||||
func ResolveHub(ix *IndexText, repoBase, stem string) (name string, exists bool) {
|
||||
candidates := CounterpartCandidates(repoBase, stem)
|
||||
for _, c := range candidates {
|
||||
if n := slug(c); ix.Find(n) != nil {
|
||||
return n, true
|
||||
}
|
||||
}
|
||||
// Nothing matched, so the family needs a hub of its own under the repo
|
||||
// derived name, which is the more reliable of the two.
|
||||
return slug(candidates[0]), false
|
||||
}
|
||||
|
||||
// HubLabel is the human-cased base model name, for prose rather than lookup.
|
||||
func HubLabel(repoBase, stem string) string {
|
||||
return CounterpartCandidates(repoBase, stem)[0]
|
||||
}
|
||||
|
||||
// filterVariants drops the references a hub must not carry: itself, and anything
|
||||
// it already lists.
|
||||
//
|
||||
// The self reference is not merely redundant. A hub that names itself makes the
|
||||
// verifier resolve the reference back to the hub, see that the hub declares
|
||||
// variants, and report a variant that declares variants of its own. It arises
|
||||
// for real rather than in theory: an unsloth rung whose weights the gallery
|
||||
// already ships under the base model name resolves, through Merge, straight back
|
||||
// to the hub that is about to reference it.
|
||||
func filterVariants(hub string, already []VariantRef, want []string) []string {
|
||||
seen := map[string]bool{strings.ToLower(hub): true}
|
||||
for _, v := range already {
|
||||
seen[strings.ToLower(v.Model)] = true
|
||||
}
|
||||
|
||||
var out []string
|
||||
for _, w := range want {
|
||||
key := strings.ToLower(w)
|
||||
if seen[key] {
|
||||
continue
|
||||
}
|
||||
seen[key] = true
|
||||
out = append(out, w)
|
||||
}
|
||||
return out
|
||||
}
|
||||
795
.github/ci/apexentries/main.go
vendored
Normal file
795
.github/ci/apexentries/main.go
vendored
Normal file
@@ -0,0 +1,795 @@
|
||||
// Command apexentries generates gallery entries for the mudler APEX GGUF
|
||||
// repositories: one entry per imatrix tier and per unsloth quant rung, all
|
||||
// gathered under the BASE model's entry. Builds off a *-APEX-MTP-GGUF repo turn
|
||||
// speculative decoding on, because those weights retain the model's MTP heads
|
||||
// and are only worth their extra size with the heads in use.
|
||||
//
|
||||
// The base model entry is the hub. Somebody looking for qwen3.6-35b-a3b must
|
||||
// find every build of those weights under that one name, so when the gallery
|
||||
// already ships the base entry this command splices a variants block into it
|
||||
// rather than emitting a competing *-apex parent beside it. Only a family whose
|
||||
// base model the gallery does not ship at all gets a new hub entry, and that one
|
||||
// is still named for the base model.
|
||||
//
|
||||
// Builds are discovered by inspecting the filenames a repo actually publishes.
|
||||
// Repo names do not reliably predict them: mudler/gemma-4-26B-A4B-it-APEX-GGUF
|
||||
// ships gemma-4-26B-A4B-APEX-*.gguf, and six of the 45 repos drop a suffix or a
|
||||
// vendor prefix in the same way.
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
|
||||
"github.com/mudler/LocalAI/.github/ci/galleryedit"
|
||||
)
|
||||
|
||||
const (
|
||||
// entryTemplate carries no backend and no parameters of its own, which is
|
||||
// why RenderChild states everything inline.
|
||||
entryTemplate = "virtual.yaml"
|
||||
unslothOwner = "unsloth"
|
||||
authorListURL = "https://huggingface.co/api/models?author=mudler&limit=300"
|
||||
)
|
||||
|
||||
// rungRank orders the quality ladder from best to smallest. The HuggingFace API
|
||||
// returns siblings alphabetically and DiscoverAPEXTiers preserves that order, so
|
||||
// an unsorted variants list reads I-Balanced, I-Compact, I-Mini, I-Nano,
|
||||
// I-Quality. Selection ignores authored order, so this is purely so the file a
|
||||
// human reviews scans in a meaningful sequence.
|
||||
var rungRank = map[string]int{
|
||||
"I-Quality": 0, "I-Balanced": 1, "I-Compact": 2, "I-Mini": 3, "I-Nano": 4,
|
||||
"Quality": 5, "Balanced": 6, "Compact": 7, "Mini": 8, "Nano": 9,
|
||||
}
|
||||
|
||||
// baseTags are the tags every generated entry carries. dflash and mtp are never
|
||||
// among them: RenderChild adds those if and only if the entry configures the
|
||||
// matching spec_type.
|
||||
var baseTags = []string{"llm", "gguf", "cpu", "gpu"}
|
||||
|
||||
// childBuild pairs a rendered entry with its position on the quality ladder, so
|
||||
// the parent's variants list can be sorted without re-parsing entry names.
|
||||
type childBuild struct {
|
||||
entry GalleryEntry
|
||||
rank int
|
||||
}
|
||||
|
||||
// family is one APEX repo's full generated output.
|
||||
type family struct {
|
||||
repo string
|
||||
repoBase string
|
||||
stem string
|
||||
hasMMProj bool
|
||||
children []childBuild
|
||||
|
||||
// skippedRepos are counterpart candidates HuggingFace would not describe.
|
||||
// Carried on the family rather than printed and forgotten so the run can
|
||||
// summarize them next to everything else a reviewer has to eyeball.
|
||||
skippedRepos []string
|
||||
census fileCensus
|
||||
unaccounted int
|
||||
}
|
||||
|
||||
// fileCensus splits the files discovery emitted nothing for into the ones a
|
||||
// reviewer must chase and the ones that are deliberately out of scope.
|
||||
//
|
||||
// Full-precision sources are the second kind: they are the unquantized weights
|
||||
// the ladder is derived FROM, not a rung of it. Folding them into the
|
||||
// unclassified total would leave a permanent benign baseline, and a permanent
|
||||
// baseline is exactly what hides the one file that ever genuinely matters.
|
||||
type fileCensus struct {
|
||||
unclassified int
|
||||
fullPrecision int
|
||||
}
|
||||
|
||||
// add accumulates one repo's census into a running total.
|
||||
func (c *fileCensus) add(o fileCensus) {
|
||||
c.unclassified += o.unclassified
|
||||
c.fullPrecision += o.fullPrecision
|
||||
}
|
||||
|
||||
// sortedChildren returns the family's builds in ladder order, best first.
|
||||
func (f *family) sortedChildren() []childBuild {
|
||||
sorted := append([]childBuild{}, f.children...)
|
||||
sort.SliceStable(sorted, func(i, j int) bool { return sorted[i].rank < sorted[j].rank })
|
||||
return sorted
|
||||
}
|
||||
|
||||
func main() {
|
||||
verify := flag.String("verify", "", "verify a gallery index and exit")
|
||||
index := flag.String("index", "gallery/index.yaml", "gallery index to dedup against")
|
||||
only := flag.String("only", "", "comma-separated repo names to restrict generation to")
|
||||
out := flag.String("out", "", "write the entries to add to this file")
|
||||
apply := flag.Bool("apply", false, "append the entries to add to -index")
|
||||
flag.Parse()
|
||||
|
||||
if *verify != "" {
|
||||
problems := Verify(*verify)
|
||||
for _, p := range problems {
|
||||
fmt.Fprintln(os.Stderr, p)
|
||||
}
|
||||
if len(problems) > 0 {
|
||||
fmt.Fprintf(os.Stderr, "%d problem(s)\n", len(problems))
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Println("index is sound")
|
||||
return
|
||||
}
|
||||
|
||||
if err := generate(*index, *only, *out, *apply); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func generate(indexPath, only, outPath string, apply bool) error {
|
||||
if outPath == "" && !apply {
|
||||
return fmt.Errorf("nothing to do: pass -out <file> or -apply")
|
||||
}
|
||||
|
||||
client := newHTTPClient()
|
||||
|
||||
repos, err := listAPEXRepos(client)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if only != "" {
|
||||
repos = restrict(repos, only)
|
||||
}
|
||||
if len(repos) == 0 {
|
||||
return fmt.Errorf("no APEX repos selected")
|
||||
}
|
||||
fmt.Printf("repos selected: %d\n", len(repos))
|
||||
|
||||
var families []family
|
||||
var failed []string
|
||||
|
||||
for _, repo := range repos {
|
||||
f, err := buildFamily(client, repo)
|
||||
if err != nil {
|
||||
// A missing sha256 is fatal for the family rather than skippable: an
|
||||
// entry without one ships an unverifiable download. Report which repo
|
||||
// and keep going, so one bad repo does not hide the state of the rest.
|
||||
fmt.Fprintf(os.Stderr, "FAILED %s: %v\n", repo, err)
|
||||
failed = append(failed, repo)
|
||||
continue
|
||||
}
|
||||
families = append(families, *f)
|
||||
}
|
||||
|
||||
existing, err := LoadExisting(indexPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
ixText, err := LoadIndexText(indexPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("existing index: %d names, %d weight URIs, %d lines\n",
|
||||
len(existing.ByName), len(existing.ByURI), len(ixText.Lines))
|
||||
|
||||
// Only the builds go through Merge. A hub is deliberately kept out of it: a
|
||||
// new hub carries the family's top rung as its own payload, so Merge's URI
|
||||
// dedup would fold the hub into that rung and the family would lose the very
|
||||
// entry point this command exists to create. Hub names are checked against
|
||||
// the index directly, by ResolveHub.
|
||||
var generated []GalleryEntry
|
||||
for _, f := range families {
|
||||
for _, c := range f.children {
|
||||
generated = append(generated, c.entry)
|
||||
}
|
||||
}
|
||||
|
||||
add, reused := Merge(existing, generated)
|
||||
reportReuse(existing, generated, reused)
|
||||
|
||||
// Variant references are resolved from `reused`, never used to decide what to
|
||||
// emit: on a within-batch name collision Merge records reused[name] = name
|
||||
// while the first entry of that name is still in `add`, so treating presence
|
||||
// in `reused` as "dropped" would silently emit nothing for it.
|
||||
added := map[string]bool{}
|
||||
for _, e := range add {
|
||||
added[e.Name] = true
|
||||
}
|
||||
|
||||
inserts, newHubs, err := planHubs(families, ixText, reused, added)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
reportHubs(ixText, inserts, newHubs)
|
||||
|
||||
skipped, census, fullPrecisionRepos, unaccounted := reportSkipped(families)
|
||||
|
||||
add = append(add, newHubs...)
|
||||
|
||||
fmt.Printf("\nentries generated: %d\nentries to add: %d\nentries reused: %d\nhubs spliced: %d\nhubs created: %d\nrepos skipped: %d\nexcluded (full precision): %d files across %d repos\nunclassified: %d\nunaccounted: %d\n",
|
||||
len(generated), len(add), len(reused), len(inserts), len(newHubs), len(skipped),
|
||||
census.fullPrecision, fullPrecisionRepos, census.unclassified, unaccounted)
|
||||
|
||||
lines, err := galleryedit.Apply(ixText.Lines, inserts)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if err := writeEntries(add, lines, outPath, apply, indexPath); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if len(failed) > 0 {
|
||||
return fmt.Errorf("%d repo(s) failed: %s", len(failed), strings.Join(failed, ", "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// resolveVariant maps a generated child name onto whatever entry actually stands
|
||||
// for it after the merge. `added` is consulted first because a within-batch name
|
||||
// collision puts a name in BOTH add and reused, and the entry that was emitted
|
||||
// is the one the parent must reference.
|
||||
func resolveVariant(name string, reused map[string]string, added map[string]bool) string {
|
||||
if added[name] {
|
||||
return name
|
||||
}
|
||||
if target, ok := reused[name]; ok {
|
||||
return target
|
||||
}
|
||||
return name
|
||||
}
|
||||
|
||||
// SpecTypeForRepo reports the speculative decoding mechanism a repo's builds can
|
||||
// turn on with no extra download.
|
||||
//
|
||||
// The *-APEX-MTP-GGUF repos republish the base weights with the model's own MTP
|
||||
// heads retained, so those builds are only worth their extra size if the heads
|
||||
// are actually used. Every other APEX repo drops them, and switching MTP on
|
||||
// there would name a mechanism the weights cannot serve.
|
||||
//
|
||||
// The suffix is read off the repo the FILES come from, so nothing downstream has
|
||||
// to infer a capability from an entry name.
|
||||
func SpecTypeForRepo(repo string) string {
|
||||
if strings.HasSuffix(path.Base(repo), "-APEX-MTP-GGUF") {
|
||||
return "draft-mtp"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// buildFamily discovers everything one APEX repo and its unsloth counterpart
|
||||
// publish, and renders it.
|
||||
func buildFamily(client *http.Client, repo string) (*family, error) {
|
||||
files, err := FetchRepoFiles(client, repo)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(files) == 0 {
|
||||
return nil, fmt.Errorf("no gguf files")
|
||||
}
|
||||
|
||||
imatrix, plain := DiscoverAPEXTiers(files)
|
||||
mmproj, hasMMProj := DiscoverMMProj(files)
|
||||
|
||||
census := reportUnclassified(repo, files, imatrix, plain)
|
||||
|
||||
// The imatrix ladder is preferred, but two of the 45 repos publish no
|
||||
// imatrix tiers at all and must still contribute their plain ladder.
|
||||
ladder := imatrix
|
||||
ladderKind := "imatrix"
|
||||
if len(ladder) == 0 {
|
||||
ladder = plain
|
||||
ladderKind = "plain"
|
||||
}
|
||||
if len(ladder) == 0 {
|
||||
return nil, fmt.Errorf("no tiers discovered")
|
||||
}
|
||||
|
||||
sortTiers(ladder)
|
||||
|
||||
var mm *GGUFFile
|
||||
if hasMMProj {
|
||||
mm = &mmproj
|
||||
}
|
||||
|
||||
repoBase := strings.TrimSuffix(path.Base(repo), "-GGUF")
|
||||
f := &family{repo: repo, repoBase: repoBase, hasMMProj: hasMMProj, census: census}
|
||||
|
||||
// Only the APEX ladder can carry MTP heads; the unsloth counterpart quantizes
|
||||
// the plain weights and gets nothing from this.
|
||||
specType := SpecTypeForRepo(repo)
|
||||
|
||||
for _, t := range ladder {
|
||||
f.children = append(f.children, childBuild{
|
||||
rank: rungRank[t.Label],
|
||||
entry: RenderChild(ChildInput{
|
||||
Name: slug(repoBase) + "-" + slug(t.Label),
|
||||
Repo: repo,
|
||||
Template: entryTemplate,
|
||||
SpecType: specType,
|
||||
Weights: []GGUFFile{t.File},
|
||||
MMProj: mm,
|
||||
BaseTags: baseTags,
|
||||
}),
|
||||
})
|
||||
}
|
||||
|
||||
stem := FileStem(ladder[0])
|
||||
f.stem = stem
|
||||
fmt.Printf("%s: %d %s tier(s) [%s], stem %s, mmproj %v\n",
|
||||
repo, len(ladder), ladderKind, tierLabels(ladder), stem, hasMMProj)
|
||||
|
||||
counterpart, cpFiles, skipped, err := resolveCounterpart(client, repoBase, stem)
|
||||
f.skippedRepos = skipped
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if counterpart != "" {
|
||||
builds := DiscoverUnslothQuants(cpFiles)
|
||||
|
||||
// Called here rather than inside Verify: a quant dropped at discovery
|
||||
// leaves no trace at all in the finished gallery file, so the only place
|
||||
// the shortfall is still visible is the moment of discovery.
|
||||
unaccounted := UnaccountedQuants(cpFiles, builds)
|
||||
f.unaccounted = len(unaccounted)
|
||||
for _, p := range unaccounted {
|
||||
fmt.Fprintf(os.Stderr, "UNACCOUNTED QUANT %s: %s\n", counterpart, p)
|
||||
}
|
||||
|
||||
cpMMProj, hasCPMMProj := DiscoverMMProj(cpFiles)
|
||||
var cpMM *GGUFFile
|
||||
if hasCPMMProj {
|
||||
cpMM = &cpMMProj
|
||||
}
|
||||
cpBase := strings.TrimSuffix(path.Base(counterpart), "-GGUF")
|
||||
|
||||
for i, b := range builds {
|
||||
f.children = append(f.children, childBuild{
|
||||
rank: 100 + i,
|
||||
entry: RenderChild(ChildInput{
|
||||
Name: slug(cpBase) + "-" + slug(b.Quant),
|
||||
Repo: counterpart,
|
||||
Template: entryTemplate,
|
||||
Weights: b.Files,
|
||||
MMProj: cpMM,
|
||||
BaseTags: baseTags,
|
||||
}),
|
||||
})
|
||||
}
|
||||
fmt.Printf("%s: counterpart %s, %d quant build(s) %s\n", repo, counterpart, len(builds), quantLabels(builds))
|
||||
} else {
|
||||
fmt.Printf("%s: no unsloth counterpart\n", repo)
|
||||
}
|
||||
|
||||
return f, nil
|
||||
}
|
||||
|
||||
// planHubs decides, per family, whether the family's builds are spliced into a
|
||||
// base model entry the gallery already ships or gathered under a new hub.
|
||||
//
|
||||
// Splicing is strongly preferred and is the measured majority-adjacent case. The
|
||||
// existing entry keeps its description, icon, tags, overrides and files
|
||||
// untouched; only variant lines are added to it.
|
||||
func planHubs(families []family, ix *IndexText, reused map[string]string, added map[string]bool) ([]galleryedit.Insert, []GalleryEntry, error) {
|
||||
// Several APEX repos can resolve to one base model, so both paths accumulate
|
||||
// by hub name rather than assuming one family per hub.
|
||||
wantByHub := map[string][]string{}
|
||||
var spliceOrder []string
|
||||
|
||||
var newHubs []GalleryEntry
|
||||
hubAt := map[string]int{}
|
||||
|
||||
for i := range families {
|
||||
f := &families[i]
|
||||
|
||||
hubName, exists := ResolveHub(ix, f.repoBase, f.stem)
|
||||
want := hubVariants(f, ix, reused, added)
|
||||
|
||||
if exists {
|
||||
if _, seen := wantByHub[hubName]; !seen {
|
||||
spliceOrder = append(spliceOrder, hubName)
|
||||
}
|
||||
wantByHub[hubName] = append(wantByHub[hubName], want...)
|
||||
continue
|
||||
}
|
||||
|
||||
if at, dup := hubAt[hubName]; dup {
|
||||
for _, v := range filterVariants(hubName, newHubs[at].Variants, want) {
|
||||
newHubs[at].Variants = append(newHubs[at].Variants, VariantRef{Model: v})
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
builds := f.sortedChildren()
|
||||
if len(builds) == 0 {
|
||||
return nil, nil, fmt.Errorf("%s: no builds to hang a hub on", f.repo)
|
||||
}
|
||||
hubAt[hubName] = len(newHubs)
|
||||
newHubs = append(newHubs, renderHub(hubName, f, builds[0], filterVariants(hubName, nil, want)))
|
||||
}
|
||||
|
||||
var inserts []galleryedit.Insert
|
||||
for _, name := range spliceOrder {
|
||||
e := ix.Find(name)
|
||||
items := filterVariants(name, e.Variants, wantByHub[name])
|
||||
if len(items) == 0 {
|
||||
continue
|
||||
}
|
||||
inserts = append(inserts, galleryedit.Insert{Entry: e.Pos, Variants: items})
|
||||
}
|
||||
return inserts, newHubs, nil
|
||||
}
|
||||
|
||||
// hubVariants is a family's full build list, in ladder order, named as the hub
|
||||
// must reference them after the merge.
|
||||
func hubVariants(f *family, ix *IndexText, reused map[string]string, added map[string]bool) []string {
|
||||
var out []string
|
||||
|
||||
// A hand-written *-apex entry is an ordinary build of these weights. It is
|
||||
// never deleted, never renamed and never treated as a hub; it is simply
|
||||
// referenced like any other rung.
|
||||
if apex := slug(f.repoBase); ix.Find(apex) != nil {
|
||||
out = append(out, apex)
|
||||
}
|
||||
for _, c := range f.sortedChildren() {
|
||||
out = append(out, resolveVariant(c.entry.Name, reused, added))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// renderHub builds the hub for a family whose base model the gallery does not
|
||||
// ship at all. It is named for the BASE model, never for the APEX repo.
|
||||
//
|
||||
// It carries one of the discovered builds as its own payload so it is a complete
|
||||
// installable entry rather than a bare index pointing at other entries. That
|
||||
// payload is what supplies overrides.backend, which matters beyond installation:
|
||||
// the verifier can only judge the tagging rule for a backend it can read, so a
|
||||
// hub carrying feature tags and no backend would escape the check in silence.
|
||||
//
|
||||
// The payload's own tags are kept rather than rebuilt from baseTags, so a hub
|
||||
// whose payload configures a spec_type stays tagged for it and consistent with
|
||||
// the overrides copied alongside.
|
||||
func renderHub(name string, f *family, payload childBuild, variants []string) GalleryEntry {
|
||||
e := payload.entry
|
||||
e.Name = name
|
||||
e.Description = fmt.Sprintf(
|
||||
"%s. Quality ladder and quantization rungs published by %s and its unsloth counterpart; LocalAI picks the build that fits the hardware.",
|
||||
HubLabel(f.repoBase, f.stem), f.repo)
|
||||
|
||||
e.Tags = append([]string{}, payload.entry.Tags...)
|
||||
if f.hasMMProj && !hasTag(e.Tags, "vision") {
|
||||
e.Tags = append(e.Tags, "vision")
|
||||
}
|
||||
|
||||
e.Variants = nil
|
||||
for _, v := range variants {
|
||||
e.Variants = append(e.Variants, VariantRef{Model: v})
|
||||
}
|
||||
return e
|
||||
}
|
||||
|
||||
func hasTag(tags []string, want string) bool {
|
||||
for _, t := range tags {
|
||||
if t == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// resolveCounterpart probes the unsloth candidates in order and returns the
|
||||
// first that publishes files.
|
||||
//
|
||||
// CounterpartCandidates is handed a BARE repo name: its cleaner does not strip
|
||||
// an owner prefix, so passing "mudler/Foo-APEX-GGUF" would yield "mudler/Foo"
|
||||
// and compose into the nonsense probe "unsloth/mudler/Foo".
|
||||
//
|
||||
// It also returns the candidates HuggingFace refused to describe. Those are
|
||||
// indistinguishable from absent without credentials, so they are skipped, but
|
||||
// they are named rather than dropped: one of them could be a real gated repo
|
||||
// whose quants belong in the gallery.
|
||||
func resolveCounterpart(client *http.Client, repoBase, stem string) (string, []GGUFFile, []string, error) {
|
||||
var unavailable []string
|
||||
for _, cand := range CounterpartCandidates(repoBase, stem) {
|
||||
repo := unslothOwner + "/" + cand + "-GGUF"
|
||||
files, unreadable, err := FetchOptionalRepoFiles(client, repo)
|
||||
if err != nil {
|
||||
return "", nil, unavailable, fmt.Errorf("probing %s: %w", repo, err)
|
||||
}
|
||||
if unreadable {
|
||||
unavailable = append(unavailable, repo)
|
||||
continue
|
||||
}
|
||||
if len(files) > 0 {
|
||||
return repo, files, unavailable, nil
|
||||
}
|
||||
}
|
||||
return "", nil, unavailable, nil
|
||||
}
|
||||
|
||||
// reportUnclassified prints the files discovery turned into nothing.
|
||||
//
|
||||
// It is a set difference on COUNTS, not a re-match of filenames: re-matching
|
||||
// would duplicate the tier regex from discover.go and the two copies would
|
||||
// drift. The likeliest trigger is a typo or case change from a publishing script
|
||||
// rather than a genuine sixth tier, and because generation falls back to the
|
||||
// plain ladder when the imatrix one is empty, a repo whose imatrix files all
|
||||
// fail to match silently downgrades the whole family instead of erroring. The
|
||||
// downstream HTTP check cannot catch that: it validates URLs that were emitted,
|
||||
// and an undiscovered tier emits none.
|
||||
// It returns the census so the run can total it.
|
||||
func reportUnclassified(repo string, files []GGUFFile, imatrix, plain []Tier) fileCensus {
|
||||
mmprojCount, fullPrecision := 0, 0
|
||||
for _, f := range files {
|
||||
// The mmproj test comes first because projectors are themselves often
|
||||
// published at f16 (mmproj-F16.gguf), and counting such a file in both
|
||||
// buckets would understate the unclassified remainder.
|
||||
if strings.HasPrefix(f.Name, "mmproj") {
|
||||
mmprojCount++
|
||||
continue
|
||||
}
|
||||
if IsFullPrecision(f.Name) {
|
||||
fullPrecision++
|
||||
}
|
||||
}
|
||||
|
||||
classified := len(imatrix) + len(plain) + mmprojCount + fullPrecision
|
||||
if classified >= len(files) {
|
||||
return fileCensus{fullPrecision: fullPrecision}
|
||||
}
|
||||
fmt.Fprintf(os.Stderr, "UNCLASSIFIED %s: %d of %d .gguf files classified, %d unaccounted for\n",
|
||||
repo, classified, len(files), len(files)-classified)
|
||||
return fileCensus{unclassified: len(files) - classified, fullPrecision: fullPrecision}
|
||||
}
|
||||
|
||||
// reportReuse splits Merge's single reused map into the two cases it conflates.
|
||||
//
|
||||
// A URI match means the gallery already ships exactly these weights, and
|
||||
// pointing the parent at the existing entry is correct. A NAME match with a
|
||||
// different URI means an unrelated entry happens to own the name, and
|
||||
// referencing it would point the parent at different weights than were
|
||||
// generated, substituting a build without saying so. Only the first is safe to
|
||||
// wave through.
|
||||
func reportReuse(existing *ExistingIndex, generated []GalleryEntry, reused map[string]string) {
|
||||
byName := map[string]GalleryEntry{}
|
||||
for _, e := range generated {
|
||||
if _, seen := byName[e.Name]; !seen {
|
||||
byName[e.Name] = e
|
||||
}
|
||||
}
|
||||
|
||||
var nameCollisions, uriMatches []string
|
||||
for name, target := range reused {
|
||||
gen := byName[name]
|
||||
uri := ""
|
||||
if len(gen.Files) > 0 {
|
||||
uri = gen.Files[0].URI
|
||||
}
|
||||
|
||||
switch {
|
||||
case hasName(existing, name):
|
||||
nameCollisions = append(nameCollisions,
|
||||
fmt.Sprintf(" %s -> gallery entry of the same name (generated uri: %s)", name, orNone(uri)))
|
||||
case target == name:
|
||||
nameCollisions = append(nameCollisions,
|
||||
fmt.Sprintf(" %s -> earlier entry of the same name in this batch (generated uri: %s)", name, orNone(uri)))
|
||||
default:
|
||||
uriMatches = append(uriMatches, fmt.Sprintf(" %s -> %s (same weights: %s)", name, target, orNone(uri)))
|
||||
}
|
||||
}
|
||||
sort.Strings(nameCollisions)
|
||||
sort.Strings(uriMatches)
|
||||
|
||||
fmt.Printf("\nNAME COLLISIONS (%d) - inspect each by hand, the target may hold different weights\n", len(nameCollisions))
|
||||
for _, l := range nameCollisions {
|
||||
fmt.Println(l)
|
||||
}
|
||||
fmt.Printf("\nURI MATCHES (%d) - the gallery or this batch already ships these exact weights\n", len(uriMatches))
|
||||
for _, l := range uriMatches {
|
||||
fmt.Println(l)
|
||||
}
|
||||
}
|
||||
|
||||
// reportHubs prints exactly what will be written where. The splices are the part
|
||||
// a human has to read: they modify entries the gallery already ships, so the
|
||||
// review needs the target, the line, and every added reference spelled out.
|
||||
func reportHubs(ix *IndexText, inserts []galleryedit.Insert, newHubs []GalleryEntry) {
|
||||
fmt.Printf("\nHUBS SPLICED (%d) - variants added to the EXISTING base model entry, nothing else touched\n", len(inserts))
|
||||
for _, in := range inserts {
|
||||
e := ix.Find(in.Entry.Name)
|
||||
fmt.Printf(" %s (line %d, %d variant(s) already declared):\n", in.Entry.Name, in.Entry.StartLine+1, len(e.Variants))
|
||||
for _, v := range in.Variants {
|
||||
fmt.Printf(" + - model: %s\n", galleryedit.QuoteName(v))
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Printf("\nHUBS CREATED (%d) - the gallery ships no base model entry, so one is emitted for it\n", len(newHubs))
|
||||
for _, h := range newHubs {
|
||||
fmt.Printf(" %s:\n", h.Name)
|
||||
for _, v := range h.Variants {
|
||||
fmt.Printf(" - model: %s\n", v.Model)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// reportSkipped names the counterpart repos HuggingFace would not describe, and
|
||||
// totals the other two silent-shortfall counters alongside them.
|
||||
//
|
||||
// A skipped repo is not the same as a clean 404. HuggingFace answers 401 for a
|
||||
// nonexistent repo to an unauthenticated client, so the overwhelmingly likely
|
||||
// reading is "there is no such counterpart", which is the normal case for the
|
||||
// community merges. But a private or gated repo answers 401 too, and that one
|
||||
// WOULD have quants worth shipping. Printing the list is what keeps that
|
||||
// possibility auditable instead of silently discarded.
|
||||
func reportSkipped(families []family) ([]string, fileCensus, int, int) {
|
||||
var skipped []string
|
||||
var census fileCensus
|
||||
fullPrecisionRepos, unaccounted := 0, 0
|
||||
for _, f := range families {
|
||||
skipped = append(skipped, f.skippedRepos...)
|
||||
census.add(f.census)
|
||||
if f.census.fullPrecision > 0 {
|
||||
fullPrecisionRepos++
|
||||
}
|
||||
unaccounted += f.unaccounted
|
||||
}
|
||||
sort.Strings(skipped)
|
||||
|
||||
fmt.Printf("\nREPOS SKIPPED AS UNAVAILABLE (%d) - HuggingFace answered 401/403, which is indistinguishable from absent without a token; check none of these is a real gated repo\n", len(skipped))
|
||||
for _, r := range skipped {
|
||||
fmt.Printf(" %s\n", r)
|
||||
}
|
||||
return skipped, census, fullPrecisionRepos, unaccounted
|
||||
}
|
||||
|
||||
func hasName(ix *ExistingIndex, name string) bool {
|
||||
_, ok := ix.ByName[name]
|
||||
return ok
|
||||
}
|
||||
|
||||
func orNone(s string) string {
|
||||
if s == "" {
|
||||
return "(no files)"
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// writeEntries emits the additions.
|
||||
//
|
||||
// -apply does two things in one pass: it writes back the spliced lines, which
|
||||
// differ from the original only by the variant lines galleryedit inserted, and
|
||||
// then appends the new entries. New entries are APPENDED rather than merged into
|
||||
// the structure, for the same reason the splice is textual: a YAML round trip
|
||||
// over 40,000 lines would reflow the whole file into an unreviewable diff.
|
||||
func writeEntries(add []GalleryEntry, lines []string, outPath string, apply bool, indexPath string) error {
|
||||
if apply {
|
||||
if err := os.WriteFile(indexPath, []byte(strings.Join(lines, "\n")), 0o644); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("spliced %s\n", indexPath)
|
||||
}
|
||||
|
||||
if len(add) == 0 {
|
||||
fmt.Println("nothing to append")
|
||||
return nil
|
||||
}
|
||||
|
||||
blob, err := yaml.Marshal(add)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if outPath != "" {
|
||||
if err := os.WriteFile(outPath, blob, 0o644); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("wrote %d entries to %s\n", len(add), outPath)
|
||||
}
|
||||
|
||||
if apply {
|
||||
f, err := os.OpenFile(indexPath, os.O_APPEND|os.O_WRONLY, 0o644)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
if _, err := f.Write(blob); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("appended %d entries to %s\n", len(add), indexPath)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// listAPEXRepos returns the mudler repos whose name marks them as APEX builds.
|
||||
func listAPEXRepos(client *http.Client) ([]string, error) {
|
||||
req, err := http.NewRequest(http.MethodGet, authorListURL, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("User-Agent", "localai-apexentries/1.0")
|
||||
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("listing models: unexpected status %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var models []struct {
|
||||
ID string `json:"id"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &models); err != nil {
|
||||
return nil, fmt.Errorf("decoding model list: %w", err)
|
||||
}
|
||||
|
||||
var out []string
|
||||
for _, m := range models {
|
||||
if strings.Contains(m.ID, "APEX") {
|
||||
out = append(out, m.ID)
|
||||
}
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func restrict(repos []string, only string) []string {
|
||||
want := map[string]bool{}
|
||||
for _, r := range strings.Split(only, ",") {
|
||||
if r = strings.TrimSpace(r); r != "" {
|
||||
want[r] = true
|
||||
}
|
||||
}
|
||||
|
||||
var out []string
|
||||
for _, r := range repos {
|
||||
if want[r] {
|
||||
out = append(out, r)
|
||||
delete(want, r)
|
||||
}
|
||||
}
|
||||
// A name in -only that matched nothing is a typo, not an empty result.
|
||||
for r := range want {
|
||||
fmt.Fprintf(os.Stderr, "WARNING: -only names %s, which is not an APEX repo of this author\n", r)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func sortTiers(tiers []Tier) {
|
||||
sort.SliceStable(tiers, func(i, j int) bool { return rungRank[tiers[i].Label] < rungRank[tiers[j].Label] })
|
||||
}
|
||||
|
||||
func tierLabels(tiers []Tier) string {
|
||||
var out []string
|
||||
for _, t := range tiers {
|
||||
out = append(out, t.Label)
|
||||
}
|
||||
return strings.Join(out, ",")
|
||||
}
|
||||
|
||||
func quantLabels(builds []QuantBuild) string {
|
||||
var out []string
|
||||
for _, b := range builds {
|
||||
l := b.Quant
|
||||
if b.Sharded {
|
||||
l += fmt.Sprintf("(%d shards)", len(b.Files))
|
||||
}
|
||||
out = append(out, l)
|
||||
}
|
||||
return strings.Join(out, ",")
|
||||
}
|
||||
|
||||
// slug turns a repo, tier or quant label into a gallery entry name component.
|
||||
func slug(s string) string {
|
||||
return strings.ReplaceAll(strings.ToLower(s), "_", "-")
|
||||
}
|
||||
344
.github/ci/apexentries/main_test.go
vendored
Normal file
344
.github/ci/apexentries/main_test.go
vendored
Normal file
@@ -0,0 +1,344 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
|
||||
"github.com/mudler/LocalAI/.github/ci/galleryedit"
|
||||
)
|
||||
|
||||
func mustIndex(text string) *IndexText {
|
||||
ix, err := ParseIndexText(text)
|
||||
ExpectWithOffset(1, err).ToNot(HaveOccurred())
|
||||
return ix
|
||||
}
|
||||
|
||||
// buildOf renders a realistic child so the specs exercise the payload a hub
|
||||
// actually inherits rather than a bare name.
|
||||
func buildOf(name, repo, file string, rank int) childBuild {
|
||||
return childBuild{
|
||||
rank: rank,
|
||||
entry: RenderChild(ChildInput{
|
||||
Name: name,
|
||||
Repo: repo,
|
||||
Template: entryTemplate,
|
||||
Weights: []GGUFFile{{Name: file, SHA256: "aa"}},
|
||||
BaseTags: baseTags,
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
var _ = Describe("ResolveHub", func() {
|
||||
It("picks the base model name over the APEX name, even when both are in the gallery", func() {
|
||||
// The hub is the entry a user searches for. If the *-apex entry were
|
||||
// chosen the family would be gathered under a name nobody looks up, and
|
||||
// the base entry would go on advertising only its own build.
|
||||
ix := mustIndex("- name: qwen3.6-35b-a3b\n url: u\n- name: qwen3.6-35b-a3b-apex\n url: u\n")
|
||||
|
||||
name, exists := ResolveHub(ix, "Qwen3.6-35B-A3B-APEX", "Qwen3.6-35B-A3B-APEX")
|
||||
|
||||
Expect(name).To(Equal("qwen3.6-35b-a3b"))
|
||||
Expect(exists).To(BeTrue())
|
||||
})
|
||||
|
||||
It("falls back to the stem-derived candidate when the repo-derived one is absent", func() {
|
||||
// gemma's repo says "-it" and its published files do not, so only one of
|
||||
// the two candidates can match whatever the base entry was named after.
|
||||
ix := mustIndex("- name: gemma-4-26b-a4b\n url: u\n")
|
||||
|
||||
name, exists := ResolveHub(ix, "gemma-4-26B-A4B-it-APEX", "gemma-4-26B-A4B-APEX")
|
||||
|
||||
Expect(name).To(Equal("gemma-4-26b-a4b"))
|
||||
Expect(exists).To(BeTrue())
|
||||
})
|
||||
|
||||
It("reports the base name as absent rather than settling for the APEX entry", func() {
|
||||
ix := mustIndex("- name: qwen3.5-35b-a3b-apex\n url: u\n")
|
||||
|
||||
name, exists := ResolveHub(ix, "Qwen3.5-35B-A3B-APEX", "Qwen3.5-35B-A3B-APEX")
|
||||
|
||||
Expect(name).To(Equal("qwen3.5-35b-a3b"))
|
||||
Expect(exists).To(BeFalse())
|
||||
})
|
||||
|
||||
It("strips the MTP and TQ markers as well as APEX", func() {
|
||||
ix := mustIndex("- name: qwen3.6-35b-a3b\n url: u\n")
|
||||
|
||||
name, exists := ResolveHub(ix, "Qwen3.6-35B-A3B-APEX-MTP", "Qwen3.6-35B-A3B-APEX-MTP")
|
||||
|
||||
Expect(name).To(Equal("qwen3.6-35b-a3b"))
|
||||
Expect(exists).To(BeTrue())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("planHubs", func() {
|
||||
noReuse := map[string]string{}
|
||||
allAdded := func(names ...string) map[string]bool {
|
||||
out := map[string]bool{}
|
||||
for _, n := range names {
|
||||
out[n] = true
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
It("splices into the existing base entry instead of emitting an *-apex parent", func() {
|
||||
ix := mustIndex("- name: step-3.7-flash\n url: u\n- name: other\n url: u\n")
|
||||
fams := []family{{
|
||||
repo: "mudler/Step-3.7-Flash-APEX-GGUF",
|
||||
repoBase: "Step-3.7-Flash-APEX",
|
||||
stem: "Step-3.7-Flash-APEX",
|
||||
children: []childBuild{buildOf("step-3.7-flash-apex-i-quality", "mudler/Step-3.7-Flash-APEX-GGUF", "a.gguf", 0)},
|
||||
}}
|
||||
|
||||
inserts, newHubs, err := planHubs(fams, ix, noReuse, allAdded("step-3.7-flash-apex-i-quality"))
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(newHubs).To(BeEmpty())
|
||||
Expect(inserts).To(HaveLen(1))
|
||||
Expect(inserts[0].Entry.Name).To(Equal("step-3.7-flash"))
|
||||
Expect(inserts[0].Variants).To(Equal([]string{"step-3.7-flash-apex-i-quality"}))
|
||||
})
|
||||
|
||||
It("merges into an entry that already declares variants, without repeating one", func() {
|
||||
// The gallery's qwen3.6-35b-a3b already lists its APEX build. Re-adding it
|
||||
// would put a duplicate key's worth of noise in the diff and a duplicate
|
||||
// reference in the entry.
|
||||
ix := mustIndex("- name: qwen3.6-35b-a3b\n variants:\n - model: qwen3.6-35b-a3b-apex\n url: u\n" +
|
||||
"- name: qwen3.6-35b-a3b-apex\n url: u\n")
|
||||
fams := []family{{
|
||||
repo: "mudler/Qwen3.6-35B-A3B-APEX-GGUF",
|
||||
repoBase: "Qwen3.6-35B-A3B-APEX",
|
||||
stem: "Qwen3.6-35B-A3B-APEX",
|
||||
children: []childBuild{buildOf("qwen3.6-35b-a3b-apex-i-quality", "mudler/Qwen3.6-35B-A3B-APEX-GGUF", "a.gguf", 0)},
|
||||
}}
|
||||
|
||||
inserts, newHubs, err := planHubs(fams, ix, noReuse, allAdded("qwen3.6-35b-a3b-apex-i-quality"))
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(newHubs).To(BeEmpty())
|
||||
Expect(inserts[0].Variants).To(Equal([]string{"qwen3.6-35b-a3b-apex-i-quality"}))
|
||||
|
||||
out, err := galleryedit.Apply(ix.Lines, inserts)
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(strings.Count(strings.Join(out, "\n"), "variants:")).To(Equal(1))
|
||||
Expect(out).To(HaveLen(len(ix.Lines) + 1))
|
||||
})
|
||||
|
||||
It("never lets the hub reference itself", func() {
|
||||
// An unsloth rung whose weights the gallery already ships under the base
|
||||
// name resolves, through Merge, straight back to the hub. The verifier
|
||||
// reads a self reference as a variant that declares variants of its own.
|
||||
ix := mustIndex("- name: step-3.7-flash\n url: u\n")
|
||||
fams := []family{{
|
||||
repo: "mudler/Step-3.7-Flash-APEX-GGUF",
|
||||
repoBase: "Step-3.7-Flash-APEX",
|
||||
stem: "Step-3.7-Flash-APEX",
|
||||
children: []childBuild{buildOf("step-3.7-flash-ud-q4-k-m", "unsloth/Step-3.7-Flash-GGUF", "a.gguf", 100)},
|
||||
}}
|
||||
|
||||
inserts, _, err := planHubs(fams, ix, map[string]string{"step-3.7-flash-ud-q4-k-m": "step-3.7-flash"}, map[string]bool{})
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(inserts).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("emits a hub named for the base model when the gallery has none", func() {
|
||||
ix := mustIndex("- name: qwen3.5-35b-a3b-apex\n url: u\n")
|
||||
fams := []family{{
|
||||
repo: "mudler/Qwen3.5-35B-A3B-APEX-GGUF",
|
||||
repoBase: "Qwen3.5-35B-A3B-APEX",
|
||||
stem: "Qwen3.5-35B-A3B-APEX",
|
||||
hasMMProj: true,
|
||||
children: []childBuild{
|
||||
buildOf("qwen3.5-35b-a3b-apex-i-quality", "mudler/Qwen3.5-35B-A3B-APEX-GGUF", "a.gguf", 0),
|
||||
buildOf("qwen3.5-35b-a3b-ud-q6-k", "unsloth/Qwen3.5-35B-A3B-GGUF", "b.gguf", 102),
|
||||
},
|
||||
}}
|
||||
|
||||
inserts, newHubs, err := planHubs(fams, ix, noReuse,
|
||||
allAdded("qwen3.5-35b-a3b-apex-i-quality", "qwen3.5-35b-a3b-ud-q6-k"))
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(inserts).To(BeEmpty())
|
||||
Expect(newHubs).To(HaveLen(1))
|
||||
|
||||
hub := newHubs[0]
|
||||
Expect(hub.Name).To(Equal("qwen3.5-35b-a3b"))
|
||||
Expect(hub.Name).ToNot(HaveSuffix("-apex"))
|
||||
|
||||
// A hand-written *-apex entry is an ordinary build, referenced like any
|
||||
// other rung and never deleted or renamed.
|
||||
Expect(hub.Variants).To(Equal([]VariantRef{
|
||||
{Model: "qwen3.5-35b-a3b-apex"},
|
||||
{Model: "qwen3.5-35b-a3b-apex-i-quality"},
|
||||
{Model: "qwen3.5-35b-a3b-ud-q6-k"},
|
||||
}))
|
||||
|
||||
// The verifier skips entries with no declared backend, so a hub without
|
||||
// one would escape the tagging check in silence.
|
||||
Expect(hub.Overrides).To(HaveKeyWithValue("backend", "llama-cpp"))
|
||||
Expect(hub.Files).ToNot(BeEmpty())
|
||||
Expect(hub.Tags).To(ContainElement("vision"))
|
||||
})
|
||||
|
||||
It("gathers two APEX repos that share one base model under a single hub", func() {
|
||||
ix := mustIndex("- name: unrelated\n url: u\n")
|
||||
fams := []family{
|
||||
{
|
||||
repo: "mudler/Solo-APEX-GGUF",
|
||||
repoBase: "Solo-APEX",
|
||||
stem: "Solo-APEX",
|
||||
children: []childBuild{buildOf("solo-apex-i-quality", "mudler/Solo-APEX-GGUF", "a.gguf", 0)},
|
||||
},
|
||||
{
|
||||
repo: "mudler/Solo-APEX-MTP-GGUF",
|
||||
repoBase: "Solo-APEX-MTP",
|
||||
stem: "Solo-APEX-MTP",
|
||||
children: []childBuild{buildOf("solo-apex-mtp-i-quality", "mudler/Solo-APEX-MTP-GGUF", "b.gguf", 0)},
|
||||
},
|
||||
}
|
||||
|
||||
_, newHubs, err := planHubs(fams, ix, noReuse, allAdded("solo-apex-i-quality", "solo-apex-mtp-i-quality"))
|
||||
|
||||
Expect(err).ToNot(HaveOccurred())
|
||||
Expect(newHubs).To(HaveLen(1))
|
||||
Expect(newHubs[0].Name).To(Equal("solo"))
|
||||
Expect(newHubs[0].Variants).To(Equal([]VariantRef{
|
||||
{Model: "solo-apex-i-quality"},
|
||||
{Model: "solo-apex-mtp-i-quality"},
|
||||
}))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("hubVariants", func() {
|
||||
It("orders builds by quality rung rather than discovery order", func() {
|
||||
// DiscoverAPEXTiers preserves input order and the HF API returns siblings
|
||||
// alphabetically, so an unsorted list reads I-Balanced, I-Compact, I-Mini,
|
||||
// I-Nano, I-Quality. Selection ignores authored order; this is for the
|
||||
// human reading the file.
|
||||
f := family{repoBase: "X-APEX", stem: "X-APEX", children: []childBuild{
|
||||
{rank: rungRank["I-Nano"], entry: GalleryEntry{Name: "x-i-nano"}},
|
||||
{rank: 100, entry: GalleryEntry{Name: "x-ud-q4-k-m"}},
|
||||
{rank: rungRank["I-Quality"], entry: GalleryEntry{Name: "x-i-quality"}},
|
||||
{rank: rungRank["I-Compact"], entry: GalleryEntry{Name: "x-i-compact"}},
|
||||
}}
|
||||
|
||||
got := hubVariants(&f, mustIndex("- name: x\n url: u\n"), map[string]string{}, map[string]bool{})
|
||||
|
||||
Expect(got).To(Equal([]string{"x-i-quality", "x-i-compact", "x-i-nano", "x-ud-q4-k-m"}))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("ParseIndexText", func() {
|
||||
It("refuses to edit by line number when the two views of the file disagree", func() {
|
||||
_, err := ParseIndexText("- name: one\n url: u\n-\n")
|
||||
Expect(err).To(MatchError(ContainSubstring("empty")))
|
||||
})
|
||||
|
||||
It("records the line range of each entry", func() {
|
||||
ix := mustIndex("- name: first\n url: u\n- name: second\n url: u\n")
|
||||
|
||||
Expect(ix.Find("FIRST").Pos.StartLine).To(Equal(0))
|
||||
Expect(ix.Find("first").Pos.EndLine).To(Equal(2))
|
||||
Expect(ix.Find("second").Pos.StartLine).To(Equal(2))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("resolveVariant", func() {
|
||||
It("keeps an entry that was emitted even when it is also in reused", func() {
|
||||
// A within-batch name collision records reused[name] = name while the
|
||||
// FIRST entry of that name is still in add. Treating presence in reused as
|
||||
// "dropped" would emit nothing for it.
|
||||
added := map[string]bool{"dup": true}
|
||||
reused := map[string]string{"dup": "dup"}
|
||||
|
||||
Expect(resolveVariant("dup", reused, added)).To(Equal("dup"))
|
||||
})
|
||||
|
||||
It("redirects a reused name at the entry that stands in for it", func() {
|
||||
added := map[string]bool{}
|
||||
reused := map[string]string{"generated": "already-in-gallery"}
|
||||
|
||||
Expect(resolveVariant("generated", reused, added)).To(Equal("already-in-gallery"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("slug", func() {
|
||||
It("lowercases and turns quant underscores into hyphens", func() {
|
||||
Expect(slug("UD-Q4_K_M")).To(Equal("ud-q4-k-m"))
|
||||
Expect(slug("gemma-4-26B-A4B-it-APEX")).To(Equal("gemma-4-26b-a4b-it-apex"))
|
||||
Expect(slug("I-Nano")).To(Equal("i-nano"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("sortTiers", func() {
|
||||
It("puts the imatrix ladder in descending quality order", func() {
|
||||
tiers := []Tier{
|
||||
{Label: "I-Balanced"}, {Label: "I-Compact"}, {Label: "I-Mini"},
|
||||
{Label: "I-Nano"}, {Label: "I-Quality"},
|
||||
}
|
||||
sortTiers(tiers)
|
||||
Expect(tierLabels(tiers)).To(Equal("I-Quality,I-Balanced,I-Compact,I-Mini,I-Nano"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("restrict", func() {
|
||||
It("keeps only the named repos", func() {
|
||||
got := restrict([]string{"mudler/A-APEX-GGUF", "mudler/B-APEX-GGUF"}, "mudler/B-APEX-GGUF")
|
||||
Expect(got).To(Equal([]string{"mudler/B-APEX-GGUF"}))
|
||||
})
|
||||
|
||||
It("returns nothing when the filter matches nothing", func() {
|
||||
Expect(restrict([]string{"mudler/A-APEX-GGUF"}, "mudler/typo")).To(BeEmpty())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("reportUnclassified", func() {
|
||||
// One real imatrix rung is always present so the specs measure how the
|
||||
// remaining files are bucketed, not an empty-repo edge case.
|
||||
tier := Tier{Label: "I-Quality", File: GGUFFile{Name: "Model-APEX-I-Quality.gguf"}}
|
||||
|
||||
censusOf := func(names ...string) fileCensus {
|
||||
files := []GGUFFile{tier.File}
|
||||
for _, n := range names {
|
||||
files = append(files, GGUFFile{Name: n})
|
||||
}
|
||||
return reportUnclassified("mudler/Model-APEX-GGUF", files, []Tier{tier}, nil)
|
||||
}
|
||||
|
||||
It("counts a flat full-precision source as excluded, not unclassified", func() {
|
||||
got := censusOf("Carnice-MoE-35B-A3B-F16.gguf")
|
||||
Expect(got.fullPrecision).To(Equal(1))
|
||||
Expect(got.unclassified).To(Equal(0))
|
||||
})
|
||||
|
||||
It("counts every shard of a sharded full-precision source as excluded", func() {
|
||||
got := censusOf(
|
||||
"MiniMax-M2.7-APEX-F16-00001-of-00003.gguf",
|
||||
"MiniMax-M2.7-APEX-F16-00002-of-00003.gguf",
|
||||
"MiniMax-M2.7-APEX-F16-00003-of-00003.gguf",
|
||||
)
|
||||
Expect(got.fullPrecision).To(Equal(3))
|
||||
Expect(got.unclassified).To(Equal(0))
|
||||
})
|
||||
|
||||
It("treats bf16 the same as f16, in either case", func() {
|
||||
got := censusOf("Model-APEX-BF16.gguf", "Model-APEX-bf16-00001-of-00002.gguf", "Model-APEX-f16.gguf")
|
||||
Expect(got.fullPrecision).To(Equal(3))
|
||||
Expect(got.unclassified).To(Equal(0))
|
||||
})
|
||||
|
||||
It("still reports a genuinely unknown filename as unclassified", func() {
|
||||
got := censusOf("Model-APEX-Turbo.gguf")
|
||||
Expect(got.unclassified).To(Equal(1))
|
||||
Expect(got.fullPrecision).To(Equal(0))
|
||||
})
|
||||
|
||||
It("separates the two kinds when a repo publishes both", func() {
|
||||
got := censusOf("Model-APEX-F16.gguf", "Model-APEX-Turbo.gguf")
|
||||
Expect(got.fullPrecision).To(Equal(1))
|
||||
Expect(got.unclassified).To(Equal(1))
|
||||
})
|
||||
})
|
||||
143
.github/ci/apexentries/merge.go
vendored
Normal file
143
.github/ci/apexentries/merge.go
vendored
Normal file
@@ -0,0 +1,143 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
const (
|
||||
hfShorthandPrefix = "huggingface://"
|
||||
hfResolvePrefix = "https://huggingface.co/"
|
||||
hfResolveInfix = "/resolve/main/"
|
||||
)
|
||||
|
||||
// canonicalURI reduces the two interchangeable spellings of a HuggingFace file
|
||||
// to one key, so a generated resolve/main URI dedups against the shorthand the
|
||||
// gallery uses for the majority of its entries.
|
||||
//
|
||||
// The repo is exactly the first two path segments; everything after is the file
|
||||
// path, which may itself contain slashes because sharded quants live in a
|
||||
// subdirectory. Anything that is not recognisably one of the two forms is
|
||||
// returned unchanged rather than guessed at, so mirrors and other hosts still
|
||||
// dedup on their literal string.
|
||||
func canonicalURI(uri string) string {
|
||||
switch {
|
||||
case strings.HasPrefix(uri, hfShorthandPrefix):
|
||||
rest := strings.TrimPrefix(uri, hfShorthandPrefix)
|
||||
owner, after, ok := strings.Cut(rest, "/")
|
||||
if !ok {
|
||||
return uri
|
||||
}
|
||||
name, file, ok := strings.Cut(after, "/")
|
||||
if !ok || owner == "" || name == "" || file == "" {
|
||||
return uri
|
||||
}
|
||||
return hfShorthandPrefix + owner + "/" + name + "/" + file
|
||||
|
||||
case strings.HasPrefix(uri, hfResolvePrefix):
|
||||
rest := strings.TrimPrefix(uri, hfResolvePrefix)
|
||||
repo, file, ok := strings.Cut(rest, hfResolveInfix)
|
||||
if !ok || file == "" {
|
||||
return uri
|
||||
}
|
||||
// A repo is owner/name and nothing more; a longer prefix means this is
|
||||
// some other huggingface.co URL that must not be rewritten.
|
||||
owner, name, ok := strings.Cut(repo, "/")
|
||||
if !ok || owner == "" || name == "" || strings.Contains(name, "/") {
|
||||
return uri
|
||||
}
|
||||
return hfShorthandPrefix + repo + "/" + file
|
||||
|
||||
default:
|
||||
return uri
|
||||
}
|
||||
}
|
||||
|
||||
// ExistingIndex is the lookup built from the current gallery: entry names, and
|
||||
// which entry claims each weight URI.
|
||||
type ExistingIndex struct {
|
||||
ByName map[string]int
|
||||
ByURI map[string]string
|
||||
}
|
||||
|
||||
// LoadExisting reads the gallery index for dedup purposes only. It is
|
||||
// deliberately not used to rewrite the file: the index is 40,000 lines, and a
|
||||
// YAML round trip would reflow the whole thing into an unreviewable diff.
|
||||
func LoadExisting(path string) (*ExistingIndex, error) {
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var entries []struct {
|
||||
Name string `yaml:"name"`
|
||||
Files []struct {
|
||||
URI string `yaml:"uri"`
|
||||
} `yaml:"files"`
|
||||
}
|
||||
if err := yaml.Unmarshal(raw, &entries); err != nil {
|
||||
return nil, fmt.Errorf("parsing %s: %w", path, err)
|
||||
}
|
||||
|
||||
ix := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
|
||||
for i, e := range entries {
|
||||
ix.ByName[e.Name] = i
|
||||
for _, f := range e.Files {
|
||||
if f.URI != "" {
|
||||
ix.ByURI[canonicalURI(f.URI)] = e.Name
|
||||
}
|
||||
}
|
||||
}
|
||||
return ix, nil
|
||||
}
|
||||
|
||||
// Merge splits generated entries into those to add and those already covered.
|
||||
// reused maps a generated name to the existing entry that stands in for it, so
|
||||
// a parent can reference what is already there instead of duplicating weights.
|
||||
// Several APEX repos share one base model, so the same counterpart rungs are
|
||||
// generated more than once in a batch. The batch has to dedup against itself as
|
||||
// well as against the gallery, tracked locally because the caller may reuse the
|
||||
// ExistingIndex it passed in.
|
||||
func Merge(existing *ExistingIndex, generated []GalleryEntry) (add []GalleryEntry, reused map[string]string) {
|
||||
reused = map[string]string{}
|
||||
batchNames := map[string]string{}
|
||||
batchURIs := map[string]string{}
|
||||
|
||||
// Canonicalized into a local copy rather than in place: an ExistingIndex may
|
||||
// be hand-built or reused by the caller, so Merge must not rewrite it.
|
||||
existingURIs := make(map[string]string, len(existing.ByURI))
|
||||
for uri, owner := range existing.ByURI {
|
||||
existingURIs[canonicalURI(uri)] = owner
|
||||
}
|
||||
|
||||
for _, e := range generated {
|
||||
// Name is checked before URI: a name collision must block the add
|
||||
// whatever the weights say, since duplicate names corrupt the index.
|
||||
if _, clash := existing.ByName[e.Name]; clash {
|
||||
reused[e.Name] = e.Name
|
||||
continue
|
||||
}
|
||||
if claimant, clash := batchNames[e.Name]; clash {
|
||||
reused[e.Name] = claimant
|
||||
continue
|
||||
}
|
||||
if len(e.Files) > 0 {
|
||||
uri := canonicalURI(e.Files[0].URI)
|
||||
if owner, ok := existingURIs[uri]; ok {
|
||||
reused[e.Name] = owner
|
||||
continue
|
||||
}
|
||||
if claimant, ok := batchURIs[uri]; ok {
|
||||
reused[e.Name] = claimant
|
||||
continue
|
||||
}
|
||||
batchURIs[uri] = e.Name
|
||||
}
|
||||
batchNames[e.Name] = e.Name
|
||||
add = append(add, e)
|
||||
}
|
||||
return add, reused
|
||||
}
|
||||
183
.github/ci/apexentries/merge_test.go
vendored
Normal file
183
.github/ci/apexentries/merge_test.go
vendored
Normal file
@@ -0,0 +1,183 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("Merge", func() {
|
||||
It("drops a generated entry whose weight URI already exists and reports the existing name", func() {
|
||||
existing := &ExistingIndex{
|
||||
ByName: map[string]int{"qwen3.6-35b-a3b-apex": 0},
|
||||
ByURI: map[string]string{
|
||||
"https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/X-APEX-I-Quality.gguf": "qwen3.6-35b-a3b-apex",
|
||||
},
|
||||
}
|
||||
gen := []GalleryEntry{{
|
||||
Name: "x-apex-i-quality",
|
||||
Files: []EntryFile{{URI: "https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/X-APEX-I-Quality.gguf"}},
|
||||
}}
|
||||
|
||||
add, reused := Merge(existing, gen)
|
||||
|
||||
Expect(add).To(BeEmpty())
|
||||
Expect(reused).To(HaveKeyWithValue("x-apex-i-quality", "qwen3.6-35b-a3b-apex"))
|
||||
})
|
||||
|
||||
It("keeps a generated entry whose weights are new", func() {
|
||||
existing := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
|
||||
gen := []GalleryEntry{{
|
||||
Name: "x-apex-i-mini",
|
||||
Files: []EntryFile{{URI: "https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/X-APEX-I-Mini.gguf"}},
|
||||
}}
|
||||
|
||||
add, reused := Merge(existing, gen)
|
||||
|
||||
Expect(add).To(HaveLen(1))
|
||||
Expect(reused).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("refuses to add an entry whose name collides with an existing one", func() {
|
||||
existing := &ExistingIndex{
|
||||
ByName: map[string]int{"x-apex-i-mini": 0},
|
||||
ByURI: map[string]string{},
|
||||
}
|
||||
gen := []GalleryEntry{{
|
||||
Name: "x-apex-i-mini",
|
||||
Files: []EntryFile{{URI: "https://huggingface.co/mudler/X-APEX-GGUF/resolve/main/other.gguf"}},
|
||||
}}
|
||||
|
||||
add, reused := Merge(existing, gen)
|
||||
|
||||
Expect(add).To(BeEmpty())
|
||||
Expect(reused).To(HaveKeyWithValue("x-apex-i-mini", "x-apex-i-mini"))
|
||||
})
|
||||
|
||||
// The gallery records most of its URIs in huggingface:// shorthand while
|
||||
// render.go only ever emits the resolve/main form, so without
|
||||
// canonicalization the majority of the file is invisible to the dedup.
|
||||
It("matches a generated https URI against the shorthand form recorded in the gallery", func() {
|
||||
existing := &ExistingIndex{
|
||||
ByName: map[string]int{"foo-gguf-q8-0": 0},
|
||||
ByURI: map[string]string{
|
||||
"huggingface://unsloth/Foo-GGUF/Foo-Q8_0.gguf": "foo-gguf-q8-0",
|
||||
},
|
||||
}
|
||||
gen := []GalleryEntry{{
|
||||
Name: "foo-apex-q8-0",
|
||||
Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Foo-GGUF/resolve/main/Foo-Q8_0.gguf"}},
|
||||
}}
|
||||
|
||||
add, reused := Merge(existing, gen)
|
||||
|
||||
Expect(add).To(BeEmpty())
|
||||
Expect(reused).To(HaveKeyWithValue("foo-apex-q8-0", "foo-gguf-q8-0"))
|
||||
})
|
||||
|
||||
It("matches a generated shorthand URI against the https form recorded in the gallery", func() {
|
||||
existing := &ExistingIndex{
|
||||
ByName: map[string]int{"foo-gguf-q8-0": 0},
|
||||
ByURI: map[string]string{
|
||||
"https://huggingface.co/unsloth/Foo-GGUF/resolve/main/Foo-Q8_0.gguf": "foo-gguf-q8-0",
|
||||
},
|
||||
}
|
||||
gen := []GalleryEntry{{
|
||||
Name: "foo-apex-q8-0",
|
||||
Files: []EntryFile{{URI: "huggingface://unsloth/Foo-GGUF/Foo-Q8_0.gguf"}},
|
||||
}}
|
||||
|
||||
add, reused := Merge(existing, gen)
|
||||
|
||||
Expect(add).To(BeEmpty())
|
||||
Expect(reused).To(HaveKeyWithValue("foo-apex-q8-0", "foo-gguf-q8-0"))
|
||||
})
|
||||
|
||||
// Sharded quants live under a subdirectory, so the file path carries slashes
|
||||
// of its own and only the first two segments are the repo.
|
||||
It("matches across both forms when the file path has a subdirectory", func() {
|
||||
existing := &ExistingIndex{
|
||||
ByName: map[string]int{"model-ud-q4-k-m": 0},
|
||||
ByURI: map[string]string{
|
||||
"huggingface://unsloth/Model-GGUF/UD-Q4_K_M/Model-UD-Q4_K_M-00001-of-00002.gguf": "model-ud-q4-k-m",
|
||||
},
|
||||
}
|
||||
gen := []GalleryEntry{{
|
||||
Name: "model-apex-ud-q4-k-m",
|
||||
Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Model-GGUF/resolve/main/UD-Q4_K_M/Model-UD-Q4_K_M-00001-of-00002.gguf"}},
|
||||
}}
|
||||
|
||||
add, reused := Merge(existing, gen)
|
||||
|
||||
Expect(add).To(BeEmpty())
|
||||
Expect(reused).To(HaveKeyWithValue("model-apex-ud-q4-k-m", "model-ud-q4-k-m"))
|
||||
})
|
||||
|
||||
// Several APEX repos share one base model, so the same unsloth rungs are
|
||||
// generated more than once in a single batch.
|
||||
It("adds only the first of two generated entries sharing a name", func() {
|
||||
existing := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
|
||||
gen := []GalleryEntry{
|
||||
{
|
||||
Name: "shared-rung-q8-0",
|
||||
Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Shared-GGUF/resolve/main/Shared-Q8_0.gguf"}},
|
||||
},
|
||||
{
|
||||
Name: "shared-rung-q8-0",
|
||||
Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Other-GGUF/resolve/main/Other-Q8_0.gguf"}},
|
||||
},
|
||||
}
|
||||
|
||||
add, reused := Merge(existing, gen)
|
||||
|
||||
Expect(add).To(HaveLen(1))
|
||||
Expect(add[0].Files[0].URI).To(Equal("https://huggingface.co/unsloth/Shared-GGUF/resolve/main/Shared-Q8_0.gguf"))
|
||||
Expect(reused).To(HaveKeyWithValue("shared-rung-q8-0", "shared-rung-q8-0"))
|
||||
})
|
||||
|
||||
It("adds only the first of two generated entries sharing a primary URI", func() {
|
||||
existing := &ExistingIndex{ByName: map[string]int{}, ByURI: map[string]string{}}
|
||||
gen := []GalleryEntry{
|
||||
{
|
||||
Name: "shared-rung-from-apex",
|
||||
Files: []EntryFile{{URI: "https://huggingface.co/unsloth/Shared-GGUF/resolve/main/Shared-Q8_0.gguf"}},
|
||||
},
|
||||
{
|
||||
Name: "shared-rung-from-apex-mtp",
|
||||
Files: []EntryFile{{URI: "huggingface://unsloth/Shared-GGUF/Shared-Q8_0.gguf"}},
|
||||
},
|
||||
}
|
||||
|
||||
add, reused := Merge(existing, gen)
|
||||
|
||||
Expect(add).To(HaveLen(1))
|
||||
Expect(add[0].Name).To(Equal("shared-rung-from-apex"))
|
||||
Expect(reused).To(HaveKeyWithValue("shared-rung-from-apex-mtp", "shared-rung-from-apex"))
|
||||
})
|
||||
|
||||
// Anything that is not a HuggingFace URI must survive untouched, so an
|
||||
// unrecognised scheme still dedups against the very same string.
|
||||
It("leaves a URI in neither recognised form alone and still dedups it exactly", func() {
|
||||
existing := &ExistingIndex{
|
||||
ByName: map[string]int{"mirrored-model": 0},
|
||||
ByURI: map[string]string{
|
||||
"https://mirror.example.com/weights/Model-Q8_0.gguf": "mirrored-model",
|
||||
},
|
||||
}
|
||||
gen := []GalleryEntry{
|
||||
{
|
||||
Name: "mirrored-apex",
|
||||
Files: []EntryFile{{URI: "https://mirror.example.com/weights/Model-Q8_0.gguf"}},
|
||||
},
|
||||
{
|
||||
Name: "elsewhere-apex",
|
||||
Files: []EntryFile{{URI: "https://mirror.example.com/weights/Other-Q8_0.gguf"}},
|
||||
},
|
||||
}
|
||||
|
||||
add, reused := Merge(existing, gen)
|
||||
|
||||
Expect(add).To(HaveLen(1))
|
||||
Expect(add[0].Name).To(Equal("elsewhere-apex"))
|
||||
Expect(reused).To(HaveKeyWithValue("mirrored-apex", "mirrored-model"))
|
||||
})
|
||||
})
|
||||
175
.github/ci/apexentries/render.go
vendored
Normal file
175
.github/ci/apexentries/render.go
vendored
Normal file
@@ -0,0 +1,175 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// EntryFile is one downloadable file of a gallery entry.
|
||||
type EntryFile struct {
|
||||
Filename string `yaml:"filename"`
|
||||
SHA256 string `yaml:"sha256"`
|
||||
URI string `yaml:"uri"`
|
||||
}
|
||||
|
||||
// GalleryEntry is the subset of a gallery entry this generator writes.
|
||||
//
|
||||
// Named GalleryEntry rather than Entry because the test files dot-import
|
||||
// Ginkgo, whose table DSL exports an Entry that a package-level Entry would
|
||||
// collide with. The yaml tags are what the gallery index sees, so the Go
|
||||
// identifier is free to differ.
|
||||
type GalleryEntry struct {
|
||||
Name string `yaml:"name"`
|
||||
URL string `yaml:"url"`
|
||||
Description string `yaml:"description,omitempty"`
|
||||
Tags []string `yaml:"tags,omitempty"`
|
||||
Overrides map[string]any `yaml:"overrides,omitempty"`
|
||||
Files []EntryFile `yaml:"files,omitempty"`
|
||||
Variants []VariantRef `yaml:"variants,omitempty"`
|
||||
}
|
||||
|
||||
// VariantRef mirrors the gallery's variant reference: a name and nothing else.
|
||||
type VariantRef struct {
|
||||
Model string `yaml:"model"`
|
||||
}
|
||||
|
||||
// ChildInput is everything needed to render one non-parent entry.
|
||||
type ChildInput struct {
|
||||
Name string
|
||||
Repo string
|
||||
// DraftRepo is the repo publishing the drafter, when it is not the repo
|
||||
// publishing the weights. Speculative pairings routinely cross repos, so
|
||||
// the drafter cannot be assumed to sit next to the weights. Empty means
|
||||
// same-repo, which is how the *-APEX-MTP-GGUF repos ship.
|
||||
DraftRepo string
|
||||
Template string
|
||||
Weights []GGUFFile
|
||||
MMProj *GGUFFile
|
||||
SpecType string
|
||||
DraftFile *GGUFFile
|
||||
BaseTags []string
|
||||
}
|
||||
|
||||
// specTuning is the acceptance-window tuning each spec type ships with, copied
|
||||
// from the hand-written entries that already run these two mechanisms rather
|
||||
// than invented here. The two differ because the drafters differ: self-drafted
|
||||
// MTP heads produce a short, high-confidence proposal (15+ hand-written entries
|
||||
// use 6 with a 0.75 floor), while a separate DFlash drafter is cheap enough to
|
||||
// run far ahead unconditionally (the five hand-written dflash entries use 15 and
|
||||
// set no floor).
|
||||
var specTuning = map[string][]string{
|
||||
"draft-mtp": {"spec_n_max:6", "spec_p_min:0.75"},
|
||||
"draft-dflash": {"spec_n_max:15"},
|
||||
}
|
||||
|
||||
func hfURI(repo, file string) string {
|
||||
return fmt.Sprintf("https://huggingface.co/%s/resolve/main/%s", repo, file)
|
||||
}
|
||||
|
||||
// localPath is where a downloaded file lands.
|
||||
//
|
||||
// The hand-written entries namespace by the repo's BARE name
|
||||
// (llama-cpp/models/<repo>/<file>), which is not unique. LiquidAI/LFM2.5-8B-A1B-GGUF
|
||||
// and unsloth/LFM2.5-8B-A1B-GGUF share a basename, so both claim
|
||||
// llama-cpp/models/LFM2.5-8B-A1B-GGUF/, and installing the second after the first
|
||||
// either overwrites weights whose recorded sha256 belongs to the other file or is
|
||||
// skipped as already present. Two owners publishing the same model name is the
|
||||
// normal case for quantizers, not an edge case, so the owner has to be in the path.
|
||||
//
|
||||
// The owner becomes its own path segment rather than being folded into the
|
||||
// directory name: owner/repo is unique on HuggingFace and "/" cannot occur inside
|
||||
// either half, so this is the only form that is collision-proof by construction.
|
||||
// It still reads as the hand-written convention with the owner restored, and the
|
||||
// extra depth is already present in the index for sharded builds.
|
||||
func localPath(kind, repo, file string) string {
|
||||
// path.Dir yields "." for a repo named without an owner, which path.Join
|
||||
// drops, so such a caller keeps the historical two-segment layout.
|
||||
return path.Join("llama-cpp", kind, path.Dir(repo), path.Base(repo), file)
|
||||
}
|
||||
|
||||
// RenderChild builds one child entry.
|
||||
//
|
||||
// The dflash/mtp tag is added if and only if this entry sets a spec_type,
|
||||
// because variant ranking reads tags and nothing else, and a tag that does not
|
||||
// match what the entry configures either promotes a build that is no faster or
|
||||
// hides one that is.
|
||||
func RenderChild(in ChildInput) GalleryEntry {
|
||||
e := GalleryEntry{
|
||||
Name: in.Name,
|
||||
URL: fmt.Sprintf("github:mudler/LocalAI/gallery/%s@master", in.Template),
|
||||
Tags: append([]string{}, in.BaseTags...),
|
||||
Overrides: map[string]any{},
|
||||
}
|
||||
|
||||
// gallery/virtual.yaml carries no backend, so nothing else would name an
|
||||
// engine for these entries. Matching the hand-written entries on
|
||||
// known_usecases too: LocalAI would fall back to the backend defaults, but
|
||||
// generated entries should not read differently from their neighbours.
|
||||
e.Overrides["backend"] = "llama-cpp"
|
||||
e.Overrides["known_usecases"] = []string{"chat"}
|
||||
|
||||
options := []string{"use_jinja:true"}
|
||||
|
||||
for _, w := range in.Weights {
|
||||
e.Files = append(e.Files, EntryFile{
|
||||
Filename: localPath("models", in.Repo, w.Name),
|
||||
SHA256: w.SHA256,
|
||||
URI: hfURI(in.Repo, w.Name),
|
||||
})
|
||||
}
|
||||
e.Overrides["parameters"] = map[string]any{
|
||||
"model": localPath("models", in.Repo, in.Weights[0].Name),
|
||||
}
|
||||
|
||||
if in.MMProj != nil {
|
||||
// An explicit known_usecases SUPPRESSES the backend-default fallback in
|
||||
// core/gallery/models_types.go, so a multimodal entry left at chat-only
|
||||
// never matches FilterGalleryModelsByUsecase(FLAG_VISION) or
|
||||
// FilterGalleryModelsByMultimodal and vanishes from the UI's vision and
|
||||
// multimodal filters. 19 of the 45 APEX repos ship an mmproj.
|
||||
e.Overrides["known_usecases"] = []string{"chat", "vision"}
|
||||
e.Overrides["mmproj"] = localPath("mmproj", in.Repo, in.MMProj.Name)
|
||||
e.Files = append(e.Files, EntryFile{
|
||||
Filename: localPath("mmproj", in.Repo, in.MMProj.Name),
|
||||
SHA256: in.MMProj.SHA256,
|
||||
URI: hfURI(in.Repo, in.MMProj.Name),
|
||||
})
|
||||
}
|
||||
|
||||
// A spec type is configured independently of a drafter FILE. Weights that
|
||||
// carry their own MTP heads need no second download, and requiring one left
|
||||
// the *-APEX-MTP-GGUF builds shipping the larger heads-bearing weights with
|
||||
// the heads switched off: a strictly bigger download at the same speed,
|
||||
// ranked identically to the plain rung at the same tier.
|
||||
if in.SpecType != "" {
|
||||
options = append(options, "spec_type:"+in.SpecType)
|
||||
options = append(options, specTuning[in.SpecType]...)
|
||||
// The tag is derived from the spec type this entry sets and from nothing
|
||||
// else. Variant ranking reads tags only, so a tag taken from a repo or
|
||||
// entry NAME would promote a build that is no faster whenever the name
|
||||
// and the configuration disagree.
|
||||
e.Tags = append(e.Tags, strings.TrimPrefix(in.SpecType, "draft-"))
|
||||
}
|
||||
|
||||
if in.SpecType != "" && in.DraftFile != nil {
|
||||
// Fall back to the weights repo so pairings that publish the drafter
|
||||
// alongside the weights keep working without restating the repo.
|
||||
draftRepo := in.DraftRepo
|
||||
if draftRepo == "" {
|
||||
draftRepo = in.Repo
|
||||
}
|
||||
draftPath := localPath("models", draftRepo, in.DraftFile.Name)
|
||||
|
||||
e.Overrides["draft_model"] = draftPath
|
||||
e.Overrides["flash_attention"] = "on"
|
||||
e.Files = append(e.Files, EntryFile{
|
||||
Filename: draftPath,
|
||||
SHA256: in.DraftFile.SHA256,
|
||||
URI: hfURI(draftRepo, in.DraftFile.Name),
|
||||
})
|
||||
}
|
||||
|
||||
e.Overrides["options"] = options
|
||||
return e
|
||||
}
|
||||
249
.github/ci/apexentries/render_test.go
vendored
Normal file
249
.github/ci/apexentries/render_test.go
vendored
Normal file
@@ -0,0 +1,249 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("RenderChild", func() {
|
||||
It("tags an entry that configures draft-dflash", func() {
|
||||
e := RenderChild(ChildInput{
|
||||
Name: "qwen3.5-9b-dflash",
|
||||
Repo: "mudler/Example-APEX-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
|
||||
SpecType: "draft-dflash",
|
||||
DraftFile: &GGUFFile{Name: "Example-DFlash.Q8_0.gguf", SHA256: "b"},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
|
||||
Expect(e.Tags).To(ContainElement("dflash"))
|
||||
Expect(e.Tags).ToNot(ContainElement("mtp"))
|
||||
Expect(e.Overrides["options"]).To(ContainElement("spec_type:draft-dflash"))
|
||||
Expect(e.Overrides["draft_model"]).ToNot(BeNil())
|
||||
})
|
||||
|
||||
It("does not tag an MTP-named repo that configures no speculation", func() {
|
||||
// mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF ships MTP-bearing weights. Weights
|
||||
// that carry the heads are not an entry that enables them, and tagging it
|
||||
// would win the feature axis without being any faster.
|
||||
e := RenderChild(ChildInput{
|
||||
Name: "qwen3.6-35b-a3b-apex-mtp-i-quality",
|
||||
Repo: "mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{{Name: "Qwen3.6-35B-A3B-APEX-MTP-I-Quality.gguf", SHA256: "a"}},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
|
||||
Expect(e.Tags).ToNot(ContainElement("mtp"))
|
||||
Expect(e.Tags).ToNot(ContainElement("dflash"))
|
||||
Expect(e.Overrides).ToNot(HaveKey("draft_model"))
|
||||
})
|
||||
|
||||
It("lists every shard of a sharded build and points the model at the first", func() {
|
||||
e := RenderChild(ChildInput{
|
||||
Name: "step-3.7-flash-ud-q4-k-m",
|
||||
Repo: "unsloth/Step-3.7-Flash-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{
|
||||
{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00001-of-00002.gguf", SHA256: "a"},
|
||||
{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00002-of-00002.gguf", SHA256: "b"},
|
||||
},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
|
||||
Expect(e.Files).To(HaveLen(2))
|
||||
params, ok := e.Overrides["parameters"].(map[string]any)
|
||||
Expect(ok).To(BeTrue())
|
||||
Expect(params["model"]).To(HaveSuffix("00001-of-00002.gguf"))
|
||||
Expect(e.Files[0].URI).To(Equal(
|
||||
"https://huggingface.co/unsloth/Step-3.7-Flash-GGUF/resolve/main/UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00001-of-00002.gguf"))
|
||||
})
|
||||
|
||||
It("wires mmproj when the repo publishes one", func() {
|
||||
e := RenderChild(ChildInput{
|
||||
Name: "example-i-mini",
|
||||
Repo: "mudler/Example-APEX-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{{Name: "Example-APEX-I-Mini.gguf", SHA256: "a"}},
|
||||
MMProj: &GGUFFile{Name: "mmproj-F16.gguf", SHA256: "c"},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
|
||||
Expect(e.Overrides["mmproj"]).ToNot(BeNil())
|
||||
Expect(e.Files).To(HaveLen(2))
|
||||
})
|
||||
|
||||
It("names the engine and the usecases the hand-written entries name", func() {
|
||||
// gallery/virtual.yaml supplies no backend, so an entry that omits one
|
||||
// names no engine at all and cannot load.
|
||||
e := RenderChild(ChildInput{
|
||||
Name: "example-i-mini",
|
||||
Repo: "mudler/Example-APEX-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{{Name: "Example-APEX-I-Mini.gguf", SHA256: "a"}},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
|
||||
Expect(e.Overrides["backend"]).To(Equal("llama-cpp"))
|
||||
Expect(e.Overrides["known_usecases"]).To(ContainElement("chat"))
|
||||
})
|
||||
|
||||
It("draws the drafter from DraftRepo when the pairing spans two repos", func() {
|
||||
// unsloth/Qwen3-4B-GGUF pairs with a drafter published separately by
|
||||
// AtomicChat, so a drafter URI built from the weights repo 404s.
|
||||
e := RenderChild(ChildInput{
|
||||
Name: "qwen3-4b-dflash",
|
||||
Repo: "unsloth/Qwen3-4B-GGUF",
|
||||
DraftRepo: "AtomicChat/Qwen3-4B-DFlash-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{{Name: "Qwen3-4B-Q4_K_M.gguf", SHA256: "a"}},
|
||||
SpecType: "draft-dflash",
|
||||
DraftFile: &GGUFFile{Name: "Qwen3-4B-DFlash.Q8_0.gguf", SHA256: "b"},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
|
||||
Expect(e.Files[0].URI).To(Equal(
|
||||
"https://huggingface.co/unsloth/Qwen3-4B-GGUF/resolve/main/Qwen3-4B-Q4_K_M.gguf"))
|
||||
Expect(e.Files[1].URI).To(Equal(
|
||||
"https://huggingface.co/AtomicChat/Qwen3-4B-DFlash-GGUF/resolve/main/Qwen3-4B-DFlash.Q8_0.gguf"))
|
||||
Expect(e.Files[1].Filename).To(Equal(
|
||||
"llama-cpp/models/AtomicChat/Qwen3-4B-DFlash-GGUF/Qwen3-4B-DFlash.Q8_0.gguf"))
|
||||
Expect(e.Overrides["draft_model"]).To(Equal(
|
||||
"llama-cpp/models/AtomicChat/Qwen3-4B-DFlash-GGUF/Qwen3-4B-DFlash.Q8_0.gguf"))
|
||||
})
|
||||
|
||||
It("falls back to the weights repo for the drafter when DraftRepo is empty", func() {
|
||||
// The *-APEX-MTP-GGUF repos ship the drafter alongside the weights.
|
||||
e := RenderChild(ChildInput{
|
||||
Name: "example-apex-dflash",
|
||||
Repo: "mudler/Example-APEX-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
|
||||
SpecType: "draft-dflash",
|
||||
DraftFile: &GGUFFile{Name: "Example-DFlash.Q8_0.gguf", SHA256: "b"},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
|
||||
Expect(e.Files[1].URI).To(Equal(
|
||||
"https://huggingface.co/mudler/Example-APEX-GGUF/resolve/main/Example-DFlash.Q8_0.gguf"))
|
||||
Expect(e.Files[1].Filename).To(Equal(
|
||||
"llama-cpp/models/mudler/Example-APEX-GGUF/Example-DFlash.Q8_0.gguf"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("RenderChild known_usecases", func() {
|
||||
It("declares vision alongside chat when the entry carries an mmproj", func() {
|
||||
// An explicit known_usecases suppresses the backend-default fallback, so a
|
||||
// chat-only multimodal entry disappears from the UI's vision filter.
|
||||
e := RenderChild(ChildInput{
|
||||
Name: "example-i-quality",
|
||||
Repo: "mudler/Example-APEX-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
|
||||
MMProj: &GGUFFile{Name: "mmproj-F16.gguf", SHA256: "c"},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
|
||||
Expect(e.Overrides["known_usecases"]).To(ConsistOf("chat", "vision"))
|
||||
})
|
||||
|
||||
It("leaves a text-only entry at chat", func() {
|
||||
e := RenderChild(ChildInput{
|
||||
Name: "example-i-quality",
|
||||
Repo: "mudler/Example-APEX-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
|
||||
Expect(e.Overrides["known_usecases"]).To(ConsistOf("chat"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("localPath", func() {
|
||||
It("keeps two repos with the same basename but different owners apart", func() {
|
||||
// LiquidAI and unsloth both publish LFM2.5-8B-A1B-GGUF. A path built from
|
||||
// the bare repo name gives both the same local file, so installing the
|
||||
// second overwrites or skips the first and one of them then serves bytes
|
||||
// that do not match its recorded sha256.
|
||||
liquid := RenderChild(ChildInput{
|
||||
Name: "lfm2.5-8b-a1b-i-quality",
|
||||
Repo: "LiquidAI/LFM2.5-8B-A1B-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{{Name: "LFM2.5-8B-A1B-Q8_0.gguf", SHA256: "33ab3b8c"}},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
unsloth := RenderChild(ChildInput{
|
||||
Name: "lfm2.5-8b-a1b-q8-0",
|
||||
Repo: "unsloth/LFM2.5-8B-A1B-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{{Name: "LFM2.5-8B-A1B-Q8_0.gguf", SHA256: "ec11666b"}},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
|
||||
Expect(liquid.Files[0].Filename).ToNot(Equal(unsloth.Files[0].Filename))
|
||||
Expect(unsloth.Files[0].Filename).To(Equal(
|
||||
"llama-cpp/models/unsloth/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q8_0.gguf"))
|
||||
})
|
||||
|
||||
It("namespaces the mmproj by owner too", func() {
|
||||
e := RenderChild(ChildInput{
|
||||
Name: "example-i-quality",
|
||||
Repo: "mudler/Example-APEX-GGUF",
|
||||
Template: "virtual.yaml",
|
||||
Weights: []GGUFFile{{Name: "Example-APEX-I-Quality.gguf", SHA256: "a"}},
|
||||
MMProj: &GGUFFile{Name: "mmproj-F16.gguf", SHA256: "c"},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
|
||||
Expect(e.Overrides["mmproj"]).To(Equal(
|
||||
"llama-cpp/mmproj/mudler/Example-APEX-GGUF/mmproj-F16.gguf"))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("MTP builds", func() {
|
||||
renderTier := func(repo string) GalleryEntry {
|
||||
return RenderChild(ChildInput{
|
||||
Name: "example-i-quality",
|
||||
Repo: repo,
|
||||
Template: "virtual.yaml",
|
||||
SpecType: SpecTypeForRepo(repo),
|
||||
Weights: []GGUFFile{{Name: "Example-I-Quality.gguf", SHA256: "a"}},
|
||||
BaseTags: []string{"llm", "gguf"},
|
||||
})
|
||||
}
|
||||
|
||||
It("turns MTP on for a build off an APEX-MTP repo", func() {
|
||||
// These weights retain the model's own MTP heads, so shipping them with
|
||||
// speculation off is a strictly larger download at the same speed,
|
||||
// ranked identically to the plain rung at the same tier.
|
||||
e := renderTier("mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF")
|
||||
|
||||
Expect(e.Overrides["options"]).To(ContainElements(
|
||||
"spec_type:draft-mtp", "spec_n_max:6", "spec_p_min:0.75"))
|
||||
Expect(e.Tags).To(ContainElement("mtp"))
|
||||
})
|
||||
|
||||
It("needs no drafter file, because the heads travel with the weights", func() {
|
||||
e := renderTier("mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF")
|
||||
|
||||
Expect(e.Overrides).ToNot(HaveKey("draft_model"))
|
||||
Expect(e.Files).To(HaveLen(1))
|
||||
})
|
||||
|
||||
It("leaves a build off a plain APEX repo alone", func() {
|
||||
e := renderTier("mudler/Qwen3.6-35B-A3B-APEX-GGUF")
|
||||
|
||||
Expect(e.Tags).ToNot(ContainElement("mtp"))
|
||||
Expect(e.Overrides["options"]).To(ConsistOf("use_jinja:true"))
|
||||
})
|
||||
|
||||
It("leaves an unsloth counterpart rung alone", func() {
|
||||
// The counterpart quantizes the plain weights; nothing there carries heads.
|
||||
e := renderTier("unsloth/Qwen3.6-35B-A3B-GGUF")
|
||||
|
||||
Expect(e.Tags).ToNot(ContainElement("mtp"))
|
||||
Expect(e.Overrides["options"]).To(ConsistOf("use_jinja:true"))
|
||||
})
|
||||
})
|
||||
71
.github/ci/apexentries/unsloth.go
vendored
Normal file
71
.github/ci/apexentries/unsloth.go
vendored
Normal file
@@ -0,0 +1,71 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// WantedQuants is the fixed unsloth subset this generator emits. It is a
|
||||
// deliberate subset: unsloth publishes north of 20 quants per repo, and the
|
||||
// selector needs useful fitness points rather than every rung.
|
||||
var WantedQuants = []string{"UD-Q4_K_M", "UD-Q5_K_M", "UD-Q6_K", "Q8_0"}
|
||||
|
||||
var shardRE = regexp.MustCompile(`-(\d{5})-of-(\d{5})\.gguf$`)
|
||||
|
||||
// QuantBuild is one unsloth quantization, which may be a single file or an
|
||||
// ordered set of shards.
|
||||
type QuantBuild struct {
|
||||
Quant string
|
||||
Files []GGUFFile
|
||||
Sharded bool
|
||||
}
|
||||
|
||||
// CounterpartCandidates returns the unsloth repo base names worth probing, most
|
||||
// likely first. Both derivations are needed: the repo name finds
|
||||
// unsloth/gemma-4-26B-A4B-it-GGUF, while the file stem is what matches for
|
||||
// repos whose stem is the canonical model name.
|
||||
func CounterpartCandidates(repoName, fileStem string) []string {
|
||||
clean := func(s string) string {
|
||||
s = strings.TrimSuffix(s, "-GGUF")
|
||||
s = regexp.MustCompile(`-(MTP|TQ)$`).ReplaceAllString(s, "")
|
||||
s = strings.TrimSuffix(s, "-APEX")
|
||||
return regexp.MustCompile(`-(MTP|TQ)$`).ReplaceAllString(s, "")
|
||||
}
|
||||
|
||||
out := []string{clean(repoName)}
|
||||
if stem := clean(fileStem); stem != out[0] {
|
||||
out = append(out, stem)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// DiscoverUnslothQuants returns the wanted quants a repo publishes, handling
|
||||
// both the flat single-file layout and the sharded layout where a quant lives
|
||||
// in its own subdirectory.
|
||||
func DiscoverUnslothQuants(files []GGUFFile) []QuantBuild {
|
||||
var out []QuantBuild
|
||||
|
||||
for _, q := range WantedQuants {
|
||||
var flat []GGUFFile
|
||||
var shards []GGUFFile
|
||||
|
||||
for _, f := range files {
|
||||
switch {
|
||||
case !strings.Contains(f.Name, "/") && strings.HasSuffix(f.Name, "-"+q+".gguf"):
|
||||
flat = append(flat, f)
|
||||
case strings.HasPrefix(f.Name, q+"/") && shardRE.MatchString(f.Name):
|
||||
shards = append(shards, f)
|
||||
}
|
||||
}
|
||||
|
||||
switch {
|
||||
case len(flat) > 0:
|
||||
out = append(out, QuantBuild{Quant: q, Files: flat})
|
||||
case len(shards) > 0:
|
||||
sort.Slice(shards, func(i, j int) bool { return shards[i].Name < shards[j].Name })
|
||||
out = append(out, QuantBuild{Quant: q, Files: shards, Sharded: true})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
75
.github/ci/apexentries/unsloth_test.go
vendored
Normal file
75
.github/ci/apexentries/unsloth_test.go
vendored
Normal file
@@ -0,0 +1,75 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("CounterpartCandidates", func() {
|
||||
It("offers both the repo-derived and stem-derived names", func() {
|
||||
// mudler/gemma-4-26B-A4B-it-APEX-GGUF ships gemma-4-26B-A4B-APEX-*.gguf,
|
||||
// and only the repo-derived name finds unsloth/gemma-4-26B-A4B-it-GGUF.
|
||||
got := CounterpartCandidates("gemma-4-26B-A4B-it-APEX-GGUF", "gemma-4-26B-A4B-APEX")
|
||||
|
||||
Expect(got).To(Equal([]string{"gemma-4-26B-A4B-it", "gemma-4-26B-A4B"}))
|
||||
})
|
||||
|
||||
It("strips the MTP marker", func() {
|
||||
got := CounterpartCandidates("Qwopus3.6-35B-A3B-v1-APEX-MTP-GGUF", "Qwopus3.6-35B-A3B-v1-APEX-MTP")
|
||||
|
||||
Expect(got[0]).To(Equal("Qwopus3.6-35B-A3B-v1"))
|
||||
})
|
||||
|
||||
It("strips the TQ marker", func() {
|
||||
// This is the branch that folds mudler/Qwen3.5-35B-A3B-APEX-TQ-GGUF into
|
||||
// the qwen3.5-35b-a3b hub. Without it the probe is
|
||||
// unsloth/Qwen3.5-35B-A3B-TQ-GGUF, which does not exist, so the family
|
||||
// silently loses every unsloth rung.
|
||||
got := CounterpartCandidates("Qwen3.5-35B-A3B-APEX-TQ-GGUF", "Qwen3.5-35B-A3B-APEX-TQ")
|
||||
|
||||
Expect(got).To(Equal([]string{"Qwen3.5-35B-A3B"}))
|
||||
})
|
||||
|
||||
It("does not repeat a candidate when both derivations agree", func() {
|
||||
got := CounterpartCandidates("Qwen3.6-35B-A3B-APEX-GGUF", "Qwen3.6-35B-A3B-APEX")
|
||||
|
||||
Expect(got).To(Equal([]string{"Qwen3.6-35B-A3B"}))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("DiscoverUnslothQuants", func() {
|
||||
It("finds flat single-file quants", func() {
|
||||
files := []GGUFFile{
|
||||
{Name: "Qwen3.6-35B-A3B-UD-Q4_K_M.gguf", SHA256: "a"},
|
||||
{Name: "Qwen3.6-35B-A3B-UD-IQ1_M.gguf", SHA256: "b"},
|
||||
}
|
||||
|
||||
got := DiscoverUnslothQuants(files)
|
||||
|
||||
Expect(got).To(HaveLen(1))
|
||||
Expect(got[0].Quant).To(Equal("UD-Q4_K_M"))
|
||||
Expect(got[0].Sharded).To(BeFalse())
|
||||
Expect(got[0].Files).To(HaveLen(1))
|
||||
})
|
||||
|
||||
It("collects a sharded quant from its subdirectory in shard order", func() {
|
||||
files := []GGUFFile{
|
||||
{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00002-of-00002.gguf", SHA256: "b"},
|
||||
{Name: "UD-Q4_K_M/Step-3.7-Flash-UD-Q4_K_M-00001-of-00002.gguf", SHA256: "a"},
|
||||
}
|
||||
|
||||
got := DiscoverUnslothQuants(files)
|
||||
|
||||
Expect(got).To(HaveLen(1))
|
||||
Expect(got[0].Quant).To(Equal("UD-Q4_K_M"))
|
||||
Expect(got[0].Sharded).To(BeTrue())
|
||||
Expect(got[0].Files).To(HaveLen(2))
|
||||
Expect(got[0].Files[0].Name).To(HaveSuffix("00001-of-00002.gguf"))
|
||||
})
|
||||
|
||||
It("ignores quants outside the wanted subset", func() {
|
||||
files := []GGUFFile{{Name: "Model-UD-IQ2_XXS.gguf", SHA256: "a"}}
|
||||
|
||||
Expect(DiscoverUnslothQuants(files)).To(BeEmpty())
|
||||
})
|
||||
})
|
||||
312
.github/ci/apexentries/verify.go
vendored
Normal file
312
.github/ci/apexentries/verify.go
vendored
Normal file
@@ -0,0 +1,312 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
type verifyEntry struct {
|
||||
Name string `yaml:"name"`
|
||||
Tags []string `yaml:"tags"`
|
||||
Variants []VariantRef `yaml:"variants"`
|
||||
Overrides struct {
|
||||
// Backend scopes the checks that only hold for one engine. An entry that
|
||||
// declares none takes its configuration from the referenced url: template,
|
||||
// which this verifier never reads, so it cannot be judged either way.
|
||||
Backend string `yaml:"backend"`
|
||||
Options []string `yaml:"options"`
|
||||
// MMProj and DraftModel name the files that are not weights. They are
|
||||
// the only signal for it: a drafter lands in the same models/ prefix as
|
||||
// the weights, so the path alone cannot tell them apart.
|
||||
MMProj string `yaml:"mmproj"`
|
||||
DraftModel string `yaml:"draft_model"`
|
||||
} `yaml:"overrides"`
|
||||
Files []struct {
|
||||
Filename string `yaml:"filename"`
|
||||
SHA256 string `yaml:"sha256"`
|
||||
URI string `yaml:"uri"`
|
||||
} `yaml:"files"`
|
||||
}
|
||||
|
||||
// Verify checks the invariants the variants schema and the tagging rule
|
||||
// require. It returns every problem rather than the first, so one run tells the
|
||||
// author everything that needs fixing.
|
||||
func Verify(path string) []string {
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return []string{fmt.Sprintf("reading %s: %v", path, err)}
|
||||
}
|
||||
|
||||
var entries []verifyEntry
|
||||
if err := yaml.Unmarshal(raw, &entries); err != nil {
|
||||
return []string{fmt.Sprintf("parsing %s: %v", path, err)}
|
||||
}
|
||||
|
||||
var problems []string
|
||||
|
||||
byName := map[string]verifyEntry{}
|
||||
for _, e := range entries {
|
||||
if _, seen := byName[e.Name]; seen {
|
||||
problems = append(problems, fmt.Sprintf("duplicate entry name: %s", e.Name))
|
||||
continue
|
||||
}
|
||||
byName[e.Name] = e
|
||||
}
|
||||
|
||||
for _, e := range entries {
|
||||
for _, v := range e.Variants {
|
||||
target, ok := byName[v.Model]
|
||||
if !ok {
|
||||
problems = append(problems, fmt.Sprintf("%s: variant %q does not exist", e.Name, v.Model))
|
||||
continue
|
||||
}
|
||||
if len(target.Variants) > 0 {
|
||||
problems = append(problems, fmt.Sprintf("%s: variant %q declares variants of its own", e.Name, v.Model))
|
||||
}
|
||||
}
|
||||
|
||||
for _, f := range e.Files {
|
||||
if requiresSHA256(f.Filename) && f.SHA256 == "" {
|
||||
problems = append(problems, fmt.Sprintf("%s: file %s has no sha256", e.Name, f.Filename))
|
||||
}
|
||||
}
|
||||
|
||||
problems = append(problems, checkWeightCount(e)...)
|
||||
problems = append(problems, checkFeatureTag(e, "dflash")...)
|
||||
problems = append(problems, checkFeatureTag(e, "mtp")...)
|
||||
}
|
||||
|
||||
problems = append(problems, checkPathCollisions(entries)...)
|
||||
|
||||
return problems
|
||||
}
|
||||
|
||||
// checkPathCollisions catches two different upstream files claiming one local
|
||||
// path. The install layer keys on the local filename, so whichever entry is
|
||||
// installed second either overwrites weights the first entry recorded a
|
||||
// different sha256 for or is skipped as already present. Either way some entry
|
||||
// afterwards serves bytes that do not match its own checksum, and nothing at
|
||||
// install time says so.
|
||||
//
|
||||
// This is an index-wide invariant rather than a per-entry one: neither entry is
|
||||
// wrong on its own and the collision exists only in their pairing. The usual
|
||||
// source is a path scheme built from the repo's BARE name, because two owners
|
||||
// publishing the same model name is routine for quantizers.
|
||||
//
|
||||
// Sharing a path is fine when the uri is the same, which is how several entries
|
||||
// legitimately reuse one projector. Files with no uri are skipped: there is
|
||||
// nothing to compare.
|
||||
func checkPathCollisions(entries []verifyEntry) []string {
|
||||
type source struct{ uri, entry string }
|
||||
|
||||
first := map[string]source{}
|
||||
reported := map[string]bool{}
|
||||
|
||||
var problems []string
|
||||
for _, e := range entries {
|
||||
for _, f := range e.Files {
|
||||
if f.Filename == "" || f.URI == "" {
|
||||
continue
|
||||
}
|
||||
prev, seen := first[f.Filename]
|
||||
if !seen {
|
||||
first[f.Filename] = source{uri: f.URI, entry: e.Name}
|
||||
continue
|
||||
}
|
||||
if prev.uri == f.URI || reported[f.Filename] {
|
||||
continue
|
||||
}
|
||||
// Reported once per path however many entries pile onto it, so one
|
||||
// heavily reused filename cannot bury the rest of the report.
|
||||
reported[f.Filename] = true
|
||||
problems = append(problems, fmt.Sprintf(
|
||||
"local path %s is claimed by two different uris: %s (%s) and %s (%s)",
|
||||
f.Filename, prev.uri, prev.entry, f.URI, e.Name))
|
||||
}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// auxiliaryExtensions are the metadata formats an entry ships beside its
|
||||
// weights, where an unverified download is a nuisance rather than a hole.
|
||||
//
|
||||
// The exclusion is stated as a list of metadata formats on purpose. Requiring
|
||||
// the checksum only on a blessed list of weight formats would silently exempt
|
||||
// every format nobody has shipped yet, and it already exempted safetensors
|
||||
// weights, which are downloaded and loaded exactly like GGUF ones.
|
||||
var auxiliaryExtensions = []string{".json", ".txt", ".md"}
|
||||
|
||||
// requiresSHA256 reports whether an unverified download of this file would be
|
||||
// a supply-chain hole rather than a cosmetic gap.
|
||||
func requiresSHA256(filename string) bool {
|
||||
for _, ext := range auxiliaryExtensions {
|
||||
if strings.HasSuffix(filename, ext) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// checkWeightCount catches an entry carrying two whole models. The flat-match
|
||||
// branch in DiscoverUnslothQuants appends every match, so a quant label that is
|
||||
// a suffix of another one (Q8_0 of UD-Q8_0) collects both files into one build
|
||||
// while the rendered model: points at only the first. The result downloads
|
||||
// twice the bytes and serves whichever file sorted first, silently.
|
||||
//
|
||||
// Shards are exempt because a sharded build is legitimately many files.
|
||||
//
|
||||
// The collision is a property of llama-cpp quant discovery, so the check is
|
||||
// scoped to that backend. Multi-component TTS, ASR and diffusion engines ship an
|
||||
// encoder, a decoder and a vocoder as one model, and there the second GGUF is
|
||||
// the design rather than a bug.
|
||||
func checkWeightCount(e verifyEntry) []string {
|
||||
if e.Overrides.Backend != "llama-cpp" {
|
||||
return nil
|
||||
}
|
||||
|
||||
var weights []string
|
||||
for _, f := range e.Files {
|
||||
switch {
|
||||
case !strings.HasSuffix(f.Filename, ".gguf"):
|
||||
case shardRE.MatchString(f.Filename):
|
||||
case f.Filename == e.Overrides.MMProj:
|
||||
case f.Filename == e.Overrides.DraftModel:
|
||||
default:
|
||||
weights = append(weights, f.Filename)
|
||||
}
|
||||
}
|
||||
|
||||
if len(weights) > 1 {
|
||||
return []string{fmt.Sprintf("%s: more than one weight file: %s", e.Name, strings.Join(weights, ", "))}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkFeatureTag enforces the rule in both directions. A tag without the
|
||||
// configuration promotes a build that is no faster; configuration without the
|
||||
// tag leaves a genuinely faster build ranked as plain.
|
||||
//
|
||||
// It only speaks about backends whose declaration it can actually read, because
|
||||
// a rule applied where the evidence is invisible reports noise rather than bugs.
|
||||
func checkFeatureTag(e verifyEntry, feature string) []string {
|
||||
decl, configured, judgeable := featureDeclaration(e, feature)
|
||||
if !judgeable {
|
||||
return nil
|
||||
}
|
||||
|
||||
tagged := false
|
||||
for _, t := range e.Tags {
|
||||
if t == feature {
|
||||
tagged = true
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
switch {
|
||||
case tagged && !configured:
|
||||
return []string{fmt.Sprintf("%s: tagged %s but sets no %s", e.Name, feature, decl)}
|
||||
case configured && !tagged:
|
||||
return []string{fmt.Sprintf("%s: sets %s but is not tagged %s", e.Name, decl, feature)}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// featureDeclaration implements the per-backend table in
|
||||
// .agents/adding-gallery-models.md. It returns the declaration the backend uses
|
||||
// to configure the feature, whether the entry carries it, and whether this
|
||||
// verifier is in a position to answer at all.
|
||||
func featureDeclaration(e verifyEntry, feature string) (decl string, configured, judgeable bool) {
|
||||
switch e.Overrides.Backend {
|
||||
case "llama-cpp":
|
||||
decl = "spec_type:draft-" + feature
|
||||
for _, o := range e.Overrides.Options {
|
||||
if strings.TrimSpace(o) == decl {
|
||||
return decl, true, true
|
||||
}
|
||||
}
|
||||
return decl, false, true
|
||||
|
||||
case "ds4":
|
||||
// ds4 carries the MTP heads in the weights and turns them on with
|
||||
// mtp_path / mtp_draft. It has no dflash counterpart, so dflash is not a
|
||||
// question that can be asked of a ds4 entry.
|
||||
if feature != "mtp" {
|
||||
return "", false, false
|
||||
}
|
||||
decl = "mtp_path:"
|
||||
for _, o := range e.Overrides.Options {
|
||||
o = strings.TrimSpace(o)
|
||||
if strings.HasPrefix(o, "mtp_path:") || strings.HasPrefix(o, "mtp_draft:") {
|
||||
return decl, true, true
|
||||
}
|
||||
}
|
||||
return decl, false, true
|
||||
|
||||
default:
|
||||
// sglang configures the feature with speculative_algorithm: in the
|
||||
// referenced gallery/*.yaml, and an entry that declares no backend takes
|
||||
// its whole configuration from its url: template. Verify reads one index
|
||||
// file and follows neither, so it must not judge these in either
|
||||
// direction.
|
||||
return "", false, false
|
||||
}
|
||||
}
|
||||
|
||||
// UnaccountedQuants reports a wanted quant the repo demonstrably publishes but
|
||||
// that discovery produced no build for. The layout that triggers it today is
|
||||
// root-level shards, which match neither branch of DiscoverUnslothQuants; no
|
||||
// counterpart ships that way yet, but a batch generator must not drop a build
|
||||
// with nothing said about it.
|
||||
func UnaccountedQuants(files []GGUFFile, builds []QuantBuild) []string {
|
||||
built := map[string]bool{}
|
||||
for _, b := range builds {
|
||||
built[b.Quant] = true
|
||||
}
|
||||
|
||||
var problems []string
|
||||
for _, q := range WantedQuants {
|
||||
if built[q] {
|
||||
continue
|
||||
}
|
||||
for _, f := range files {
|
||||
if filePublishesQuant(f.Name, q) {
|
||||
problems = append(problems, fmt.Sprintf("quant %s is published upstream (%s) but produced no build", q, f.Name))
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
// filePublishesQuant reports whether an upstream file is a publication of
|
||||
// quant q. It anchors on the quant label the way DiscoverUnslothQuants does,
|
||||
// as the trailing token of the base name or as the sharding subdirectory, so
|
||||
// the diagnostic and the discovery it audits cannot disagree about what a file
|
||||
// is.
|
||||
//
|
||||
// An unanchored match would reproduce the very collision this diagnostic warns
|
||||
// about: Q8_0 is a substring of UD-Q8_0, so a repo publishing only UD-Q8_0
|
||||
// would be reported as publishing an unbuilt Q8_0, which it does not, and
|
||||
// UD-Q8_0 is not a wanted quant at all.
|
||||
func filePublishesQuant(name, q string) bool {
|
||||
if strings.HasPrefix(name, q+"/") {
|
||||
return true
|
||||
}
|
||||
|
||||
base := name[strings.LastIndex(name, "/")+1:]
|
||||
// Shard numbering sits between the quant label and the extension, so it has
|
||||
// to come off before the label can be read as the trailing token. Root-level
|
||||
// shards are the layout that matches neither branch of
|
||||
// DiscoverUnslothQuants, and so the layout this diagnostic mainly catches.
|
||||
base = shardRE.ReplaceAllString(base, ".gguf")
|
||||
|
||||
if !strings.HasSuffix(base, "-"+q+".gguf") {
|
||||
return false
|
||||
}
|
||||
// UD- is unsloth's dynamic-quant modifier, and UD-<q> is a distinct quant
|
||||
// label rather than a publication of <q>.
|
||||
return !strings.HasSuffix(base, "-UD-"+q+".gguf")
|
||||
}
|
||||
480
.github/ci/apexentries/verify_test.go
vendored
Normal file
480
.github/ci/apexentries/verify_test.go
vendored
Normal file
@@ -0,0 +1,480 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
var _ = Describe("Verify", func() {
|
||||
write := func(body string) string {
|
||||
dir := GinkgoT().TempDir()
|
||||
p := filepath.Join(dir, "index.yaml")
|
||||
Expect(os.WriteFile(p, []byte(body), 0o600)).To(Succeed())
|
||||
return p
|
||||
}
|
||||
|
||||
It("passes a sound index", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: parent
|
||||
variants:
|
||||
- model: child
|
||||
files:
|
||||
- filename: a.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
- name: child
|
||||
files:
|
||||
- filename: b.gguf
|
||||
sha256: bb
|
||||
uri: https://example.com/b.gguf
|
||||
`))).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("reports a variant pointing at a missing entry", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: parent
|
||||
variants:
|
||||
- model: ghost
|
||||
files:
|
||||
- filename: a.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
`))).To(ContainElement(ContainSubstring("ghost")))
|
||||
})
|
||||
|
||||
It("reports a variant that itself declares variants", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: parent
|
||||
variants:
|
||||
- model: child
|
||||
files:
|
||||
- filename: a.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
- name: child
|
||||
variants:
|
||||
- model: grandchild
|
||||
files:
|
||||
- filename: b.gguf
|
||||
sha256: bb
|
||||
uri: https://example.com/b.gguf
|
||||
- name: grandchild
|
||||
files:
|
||||
- filename: c.gguf
|
||||
sha256: cc
|
||||
uri: https://example.com/c.gguf
|
||||
`))).To(ContainElement(ContainSubstring("declares variants of its own")))
|
||||
})
|
||||
|
||||
It("reports duplicate entry names", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: dup
|
||||
files:
|
||||
- filename: a.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
- name: dup
|
||||
files:
|
||||
- filename: b.gguf
|
||||
sha256: bb
|
||||
uri: https://example.com/b.gguf
|
||||
`))).To(ContainElement(ContainSubstring("duplicate entry name")))
|
||||
})
|
||||
|
||||
It("reports a file with no sha256", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: one
|
||||
files:
|
||||
- filename: a.gguf
|
||||
uri: https://example.com/a.gguf
|
||||
`))).To(ContainElement(ContainSubstring("no sha256")))
|
||||
})
|
||||
|
||||
It("reports an entry tagged dflash without a matching spec_type", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: liar
|
||||
tags:
|
||||
- dflash
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
options:
|
||||
- use_jinja:true
|
||||
files:
|
||||
- filename: a.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
`))).To(ContainElement(ContainSubstring("tagged dflash")))
|
||||
})
|
||||
|
||||
It("reports an entry configuring spec_type without the tag", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: shy
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
options:
|
||||
- spec_type:draft-mtp
|
||||
files:
|
||||
- filename: a.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
`))).To(ContainElement(ContainSubstring("not tagged mtp")))
|
||||
})
|
||||
|
||||
// ds4 carries the MTP heads in the weights and names them with mtp_path, so
|
||||
// the rule holds there in a different vocabulary rather than not at all.
|
||||
It("reports a ds4 entry configuring mtp_path without the tag", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: ds4-shy
|
||||
overrides:
|
||||
backend: ds4
|
||||
options:
|
||||
- mtp_path:model-mtp.gguf
|
||||
- mtp_draft:2
|
||||
files:
|
||||
- filename: a.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
`))).To(ContainElement(ContainSubstring("not tagged mtp")))
|
||||
})
|
||||
|
||||
It("reports a ds4 entry tagged mtp that configures no mtp_path", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: ds4-liar
|
||||
tags:
|
||||
- mtp
|
||||
overrides:
|
||||
backend: ds4
|
||||
options:
|
||||
- context_size:4096
|
||||
files:
|
||||
- filename: a.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
`))).To(ContainElement(ContainSubstring("tagged mtp")))
|
||||
})
|
||||
|
||||
It("accepts a ds4 entry that both configures mtp_path and carries the tag", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: ds4-honest
|
||||
tags:
|
||||
- mtp
|
||||
overrides:
|
||||
backend: ds4
|
||||
options:
|
||||
- mtp_path:model-mtp.gguf
|
||||
files:
|
||||
- filename: a.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
`))).To(BeEmpty())
|
||||
})
|
||||
|
||||
// sglang declares speculative_algorithm in the referenced gallery/*.yaml,
|
||||
// which Verify never reads, so it may not judge such an entry either way.
|
||||
It("says nothing about an sglang entry tagged mtp", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: sglang-mtp
|
||||
tags:
|
||||
- mtp
|
||||
overrides:
|
||||
backend: sglang
|
||||
files: []
|
||||
`))).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("says nothing about the tag on an entry with no declared backend", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: templated
|
||||
tags:
|
||||
- mtp
|
||||
files:
|
||||
- filename: a.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
`))).To(BeEmpty())
|
||||
})
|
||||
|
||||
// The flat-match branch in unsloth.go appends every match, so a repo
|
||||
// publishing both a plain and a UD Q8_0 renders one entry holding two full
|
||||
// models while model: points at only the first.
|
||||
It("reports an entry holding more than one non-shard weight file", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: greedy
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
options:
|
||||
- use_jinja:true
|
||||
parameters:
|
||||
model: llama-cpp/models/repo/Model-Q8_0.gguf
|
||||
files:
|
||||
- filename: llama-cpp/models/repo/Model-Q8_0.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
- filename: llama-cpp/models/repo/Model-UD-Q8_0.gguf
|
||||
sha256: bb
|
||||
uri: https://example.com/b.gguf
|
||||
`))).To(ContainElement(ContainSubstring("more than one weight file")))
|
||||
})
|
||||
|
||||
It("accepts many shards alongside an mmproj and a drafter", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: sharded
|
||||
tags:
|
||||
- mtp
|
||||
overrides:
|
||||
backend: llama-cpp
|
||||
options:
|
||||
- spec_type:draft-mtp
|
||||
mmproj: llama-cpp/mmproj/repo/mm.gguf
|
||||
draft_model: llama-cpp/models/repo/Model-draft.gguf
|
||||
files:
|
||||
- filename: llama-cpp/models/repo/Model-00001-of-00002.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
- filename: llama-cpp/models/repo/Model-00002-of-00002.gguf
|
||||
sha256: bb
|
||||
uri: https://example.com/b.gguf
|
||||
- filename: llama-cpp/mmproj/repo/mm.gguf
|
||||
sha256: cc
|
||||
uri: https://example.com/c.gguf
|
||||
- filename: llama-cpp/models/repo/Model-draft.gguf
|
||||
sha256: dd
|
||||
uri: https://example.com/d.gguf
|
||||
`))).To(BeEmpty())
|
||||
})
|
||||
|
||||
// Multi-component TTS and ASR engines legitimately ship an encoder, a
|
||||
// tokenizer, a vocoder and so on as one model, so the collision the weight
|
||||
// count catches does not exist for them.
|
||||
It("accepts a multi-component non-llama-cpp entry declaring five weights", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: multi
|
||||
overrides:
|
||||
backend: qwen3-tts-cpp
|
||||
files:
|
||||
- filename: talker.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
- filename: tokenizer.gguf
|
||||
sha256: bb
|
||||
uri: https://example.com/b.gguf
|
||||
- filename: vocoder.gguf
|
||||
sha256: cc
|
||||
uri: https://example.com/c.gguf
|
||||
- filename: encoder.gguf
|
||||
sha256: dd
|
||||
uri: https://example.com/d.gguf
|
||||
- filename: vae.gguf
|
||||
sha256: ee
|
||||
uri: https://example.com/e.gguf
|
||||
`))).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("says nothing about the weight count of an entry with no declared backend", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: templated-weights
|
||||
files:
|
||||
- filename: model-Q4_K_M.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
- filename: model-mmproj-f16.gguf
|
||||
sha256: bb
|
||||
uri: https://example.com/b.gguf
|
||||
`))).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("says nothing about an auxiliary metadata file carrying no sha256", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: aux
|
||||
files:
|
||||
- filename: a.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
- filename: params.json
|
||||
sha256: ""
|
||||
uri: https://example.com/params.json
|
||||
`))).To(BeEmpty())
|
||||
})
|
||||
|
||||
// safetensors weights are downloaded and loaded exactly like GGUF weights,
|
||||
// so an unverified one is the same supply-chain hole.
|
||||
It("reports a safetensors weight carrying no sha256", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: vae
|
||||
files:
|
||||
- filename: wan_2.1_vae.safetensors
|
||||
sha256: ""
|
||||
uri: https://example.com/vae.safetensors
|
||||
`))).To(ContainElement(ContainSubstring("no sha256")))
|
||||
})
|
||||
|
||||
It("says nothing about a txt or md file carrying no sha256", func() {
|
||||
Expect(Verify(write(`
|
||||
- name: docs
|
||||
files:
|
||||
- filename: notes.txt
|
||||
sha256: ""
|
||||
uri: https://example.com/notes.txt
|
||||
- filename: README.md
|
||||
sha256: ""
|
||||
uri: https://example.com/README.md
|
||||
`))).To(BeEmpty())
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("UnaccountedQuants", func() {
|
||||
// A quant published only as root-level shards matches neither branch in
|
||||
// DiscoverUnslothQuants, so without this diagnostic the build would vanish
|
||||
// from a batch run with nothing said about it.
|
||||
It("reports a wanted quant upstream publishes but discovery dropped", func() {
|
||||
files := []GGUFFile{
|
||||
{Name: "Model-UD-Q4_K_M-00001-of-00003.gguf", SHA256: "aa"},
|
||||
{Name: "Model-UD-Q4_K_M-00002-of-00003.gguf", SHA256: "bb"},
|
||||
{Name: "Model-UD-Q4_K_M-00003-of-00003.gguf", SHA256: "cc"},
|
||||
}
|
||||
|
||||
Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).
|
||||
To(ContainElement(ContainSubstring("UD-Q4_K_M")))
|
||||
})
|
||||
|
||||
It("says nothing when every published wanted quant produced a build", func() {
|
||||
files := []GGUFFile{
|
||||
{Name: "Model-UD-Q4_K_M.gguf", SHA256: "aa"},
|
||||
{Name: "UD-Q6_K/Model-UD-Q6_K-00001-of-00002.gguf", SHA256: "bb"},
|
||||
{Name: "UD-Q6_K/Model-UD-Q6_K-00002-of-00002.gguf", SHA256: "cc"},
|
||||
}
|
||||
|
||||
Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("says nothing about a wanted quant the repo does not publish at all", func() {
|
||||
files := []GGUFFile{{Name: "Model-UD-Q4_K_M.gguf", SHA256: "aa"}}
|
||||
|
||||
Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).To(BeEmpty())
|
||||
})
|
||||
|
||||
// UD-Q8_0 is its own quant label and is not a wanted one. Reading it as a
|
||||
// publication of Q8_0 is the substring collision this diagnostic exists to
|
||||
// warn about, and subdirectory-sharded UD quants are the normal unsloth
|
||||
// layout for large repos, so the false positive would fire on every batch.
|
||||
It("does not read a subdirectory-sharded UD-Q8_0 as a published Q8_0", func() {
|
||||
files := []GGUFFile{
|
||||
{Name: "UD-Q8_0/Model-UD-Q8_0-00001-of-00002.gguf", SHA256: "aa"},
|
||||
{Name: "UD-Q8_0/Model-UD-Q8_0-00002-of-00002.gguf", SHA256: "bb"},
|
||||
}
|
||||
|
||||
Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).To(BeEmpty())
|
||||
})
|
||||
|
||||
// A quant in its own subdirectory but not shard-numbered matches neither
|
||||
// branch of DiscoverUnslothQuants, so it is genuinely published and
|
||||
// genuinely undiscovered.
|
||||
It("reports a wanted quant published in its own subdirectory without shard numbering", func() {
|
||||
files := []GGUFFile{{Name: "Q8_0/Model-Q8_0.gguf", SHA256: "aa"}}
|
||||
|
||||
Expect(UnaccountedQuants(files, DiscoverUnslothQuants(files))).
|
||||
To(ContainElement(ContainSubstring("quant Q8_0 is published upstream")))
|
||||
})
|
||||
|
||||
// builds is empty on purpose: it isolates the file-to-quant match from
|
||||
// whatever DiscoverUnslothQuants would have made of the same file.
|
||||
It("matches the flat single-file layout", func() {
|
||||
files := []GGUFFile{{Name: "Model-Q8_0.gguf", SHA256: "aa"}}
|
||||
|
||||
Expect(UnaccountedQuants(files, nil)).
|
||||
To(ConsistOf(ContainSubstring("quant Q8_0 is published upstream")))
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("Verify local path collisions", func() {
|
||||
write := func(body string) string {
|
||||
dir := GinkgoT().TempDir()
|
||||
p := filepath.Join(dir, "index.yaml")
|
||||
Expect(os.WriteFile(p, []byte(body), 0o600)).To(Succeed())
|
||||
return p
|
||||
}
|
||||
|
||||
It("reports one local path claimed by two different uris", func() {
|
||||
// The shape that shipped: LiquidAI and unsloth both publish
|
||||
// LFM2.5-8B-A1B-GGUF, so a path built from the bare repo name gives both
|
||||
// entries the same local file under two different checksums.
|
||||
Expect(Verify(write(`
|
||||
- name: lfm2.5-8b-a1b
|
||||
files:
|
||||
- filename: llama-cpp/models/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q8_0.gguf
|
||||
sha256: 33ab3b8c
|
||||
uri: https://huggingface.co/LiquidAI/LFM2.5-8B-A1B-GGUF/resolve/main/LFM2.5-8B-A1B-Q8_0.gguf
|
||||
- name: lfm2.5-8b-a1b-q8-0
|
||||
files:
|
||||
- filename: llama-cpp/models/LFM2.5-8B-A1B-GGUF/LFM2.5-8B-A1B-Q8_0.gguf
|
||||
sha256: ec11666b
|
||||
uri: https://huggingface.co/unsloth/LFM2.5-8B-A1B-GGUF/resolve/main/LFM2.5-8B-A1B-Q8_0.gguf
|
||||
`))).To(ContainElement(SatisfyAll(
|
||||
ContainSubstring("claimed by two different uris"),
|
||||
ContainSubstring("lfm2.5-8b-a1b-q8-0"),
|
||||
)))
|
||||
})
|
||||
|
||||
It("accepts two entries reusing one file from the same uri", func() {
|
||||
// Sibling builds of one repo legitimately share a projector.
|
||||
Expect(Verify(write(`
|
||||
- name: a
|
||||
files:
|
||||
- filename: llama-cpp/mmproj/mudler/Example-GGUF/mmproj-F16.gguf
|
||||
sha256: cc
|
||||
uri: https://huggingface.co/mudler/Example-GGUF/resolve/main/mmproj-F16.gguf
|
||||
- name: b
|
||||
files:
|
||||
- filename: llama-cpp/mmproj/mudler/Example-GGUF/mmproj-F16.gguf
|
||||
sha256: cc
|
||||
uri: https://huggingface.co/mudler/Example-GGUF/resolve/main/mmproj-F16.gguf
|
||||
`))).To(BeEmpty())
|
||||
})
|
||||
|
||||
It("reports a collision once however many entries pile onto the path", func() {
|
||||
problems := Verify(write(`
|
||||
- name: a
|
||||
files:
|
||||
- filename: shared.gguf
|
||||
sha256: aa
|
||||
uri: https://example.com/a.gguf
|
||||
- name: b
|
||||
files:
|
||||
- filename: shared.gguf
|
||||
sha256: bb
|
||||
uri: https://example.com/b.gguf
|
||||
- name: c
|
||||
files:
|
||||
- filename: shared.gguf
|
||||
sha256: cc
|
||||
uri: https://example.com/c.gguf
|
||||
`))
|
||||
|
||||
var collisions int
|
||||
for _, p := range problems {
|
||||
if strings.Contains(p, "claimed by two different uris") {
|
||||
collisions++
|
||||
}
|
||||
}
|
||||
Expect(collisions).To(Equal(1))
|
||||
})
|
||||
|
||||
It("says nothing about files that carry no uri", func() {
|
||||
// A hand-written entry may record only a checksum. There is no upstream
|
||||
// to compare, so the check cannot conclude anything either way.
|
||||
Expect(Verify(write(`
|
||||
- name: a
|
||||
files:
|
||||
- filename: shared.gguf
|
||||
sha256: aa
|
||||
- name: b
|
||||
files:
|
||||
- filename: shared.gguf
|
||||
sha256: bb
|
||||
`))).To(BeEmpty())
|
||||
})
|
||||
})
|
||||
152
.github/ci/galleryedit/edit.go
vendored
Normal file
152
.github/ci/galleryedit/edit.go
vendored
Normal file
@@ -0,0 +1,152 @@
|
||||
// Package galleryedit splices variant references into the LocalAI gallery index
|
||||
// as TEXT.
|
||||
//
|
||||
// Re-serialising the index through a YAML marshaller would reflow 40,000 lines,
|
||||
// drop the anchors and merge keys the gallery relies on, and produce a diff no
|
||||
// reviewer could read, which makes a pull request worthless even when the
|
||||
// content inside it is right. Every generator that adds variants to an entry the
|
||||
// gallery already ships therefore edits lines, and they share this package so
|
||||
// that two of them cannot drift apart on where a variants block belongs.
|
||||
package galleryedit
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var (
|
||||
entryStart = regexp.MustCompile(`^-(?: |$)`)
|
||||
inlineName = regexp.MustCompile(`^- (?:&\S+ )?name:`)
|
||||
keyName = regexp.MustCompile(`^ name:`)
|
||||
keyVariants = regexp.MustCompile(`^ variants:\s*(.*)$`)
|
||||
variantItem = regexp.MustCompile(`^ - `)
|
||||
unsafeInName = regexp.MustCompile(`[:#{}\[\],&*?|>'"%@` + "`" + `]|^\s|\s$`)
|
||||
)
|
||||
|
||||
// Entry is the positional view of one gallery entry: what it is called and
|
||||
// which lines it occupies. Nothing about what the entry MEANS belongs here, so
|
||||
// each caller keeps its own semantic decode and only hands over the coordinates.
|
||||
type Entry struct {
|
||||
Name string
|
||||
// StartLine and EndLine bound the entry, zero based and half open.
|
||||
StartLine int
|
||||
EndLine int
|
||||
}
|
||||
|
||||
// Insert is one entry's pending variants addition. The caller owns the contents
|
||||
// of Variants: this package neither orders nor deduplicates them, because the
|
||||
// right order and the right dedup rule differ between generators.
|
||||
type Insert struct {
|
||||
Entry Entry
|
||||
Variants []string
|
||||
}
|
||||
|
||||
// Scan splits index text into lines and reports the line each top level list
|
||||
// item begins on.
|
||||
func Scan(text string) (lines []string, starts []int) {
|
||||
lines = strings.Split(text, "\n")
|
||||
for i, line := range lines {
|
||||
if entryStart.MatchString(line) {
|
||||
starts = append(starts, i)
|
||||
}
|
||||
}
|
||||
return lines, starts
|
||||
}
|
||||
|
||||
// Apply splices every insert into the index lines and returns the new text.
|
||||
func Apply(lines []string, inserts []Insert) ([]string, error) {
|
||||
type edit struct {
|
||||
at int
|
||||
remove int
|
||||
insert []string
|
||||
}
|
||||
var edits []edit
|
||||
|
||||
for _, in := range inserts {
|
||||
if len(in.Variants) == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
items := make([]string, 0, len(in.Variants))
|
||||
for _, v := range in.Variants {
|
||||
items = append(items, " - model: "+QuoteName(v))
|
||||
}
|
||||
|
||||
at, remove, err := insertionPoint(lines, in.Entry)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
block := items
|
||||
if remove > 0 || !hasVariantsKey(lines, in.Entry) {
|
||||
block = append([]string{" variants:"}, items...)
|
||||
}
|
||||
edits = append(edits, edit{at: at, remove: remove, insert: block})
|
||||
}
|
||||
|
||||
// Applying from the bottom up keeps every line number computed against the
|
||||
// original text valid while earlier edits are still pending.
|
||||
sort.Slice(edits, func(i, j int) bool { return edits[i].at > edits[j].at })
|
||||
|
||||
out := append([]string(nil), lines...)
|
||||
for _, e := range edits {
|
||||
tail := append([]string(nil), out[e.at+e.remove:]...)
|
||||
out = append(out[:e.at], append(append([]string(nil), e.insert...), tail...)...)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func hasVariantsKey(lines []string, e Entry) bool {
|
||||
for i := e.StartLine; i < e.EndLine; i++ {
|
||||
if keyVariants.MatchString(lines[i]) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// insertionPoint reports where new variant items belong, and how many existing
|
||||
// lines the insertion replaces.
|
||||
//
|
||||
// An entry with no variants key gets one right after its name, which is where
|
||||
// the hand-written families put it. An entry with an empty "variants: []" has
|
||||
// that line replaced by a block. An entry with a block gets its items appended.
|
||||
func insertionPoint(lines []string, e Entry) (at int, remove int, err error) {
|
||||
for i := e.StartLine; i < e.EndLine; i++ {
|
||||
m := keyVariants.FindStringSubmatch(lines[i])
|
||||
if m == nil {
|
||||
continue
|
||||
}
|
||||
if strings.TrimSpace(m[1]) == "[]" {
|
||||
return i, 1, nil
|
||||
}
|
||||
if strings.TrimSpace(m[1]) != "" {
|
||||
return 0, 0, fmt.Errorf("entry %q writes its variants inline (%q); this job only edits block lists", e.Name, strings.TrimSpace(m[1]))
|
||||
}
|
||||
last := i
|
||||
for j := i + 1; j < e.EndLine && variantItem.MatchString(lines[j]); j++ {
|
||||
last = j
|
||||
}
|
||||
return last + 1, 0, nil
|
||||
}
|
||||
|
||||
if inlineName.MatchString(lines[e.StartLine]) {
|
||||
return e.StartLine + 1, 0, nil
|
||||
}
|
||||
for i := e.StartLine; i < e.EndLine; i++ {
|
||||
if keyName.MatchString(lines[i]) {
|
||||
return i + 1, 0, nil
|
||||
}
|
||||
}
|
||||
return 0, 0, fmt.Errorf("entry %q has no name line to anchor the insertion to", e.Name)
|
||||
}
|
||||
|
||||
// QuoteName quotes a variant reference when the name would otherwise change
|
||||
// meaning as bare YAML. Config-suffixed names carry a ":" and always need it.
|
||||
func QuoteName(name string) string {
|
||||
if unsafeInName.MatchString(name) {
|
||||
return `"` + strings.ReplaceAll(name, `"`, `\"`) + `"`
|
||||
}
|
||||
return name
|
||||
}
|
||||
114
.github/ci/variantproposals/edit.go
vendored
114
.github/ci/variantproposals/edit.go
vendored
@@ -2,119 +2,41 @@ package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var (
|
||||
inlineName = regexp.MustCompile(`^- (?:&\S+ )?name:`)
|
||||
keyName = regexp.MustCompile(`^ name:`)
|
||||
keyVariants = regexp.MustCompile(`^ variants:\s*(.*)$`)
|
||||
variantItem = regexp.MustCompile(`^ - `)
|
||||
unsafeInName = regexp.MustCompile(`[:#{}\[\],&*?|>'"%@` + "`" + `]|^\s|\s$`)
|
||||
"github.com/mudler/LocalAI/.github/ci/galleryedit"
|
||||
)
|
||||
|
||||
// ApplyFamilies writes the proposed variant lists into the index text.
|
||||
//
|
||||
// The edit is textual on purpose. Re-serialising the index through a YAML
|
||||
// marshaller would reflow 40,000 lines, drop the anchors and merge keys the
|
||||
// gallery relies on, and produce a diff no reviewer could read, which would
|
||||
// make the pull request worthless even when the proposals inside it are right.
|
||||
// The line editing itself lives in galleryedit, shared with the apexentries
|
||||
// generator. Both jobs add variants to entries the gallery already ships, and a
|
||||
// second answer to "where does a variants block go" would drift from this one;
|
||||
// see that package for why the edit is textual rather than a YAML round trip.
|
||||
func ApplyFamilies(ix *Index, families []Family) ([]string, error) {
|
||||
byName, _ := ix.ByName()
|
||||
|
||||
type edit struct {
|
||||
at int
|
||||
remove int
|
||||
insert []string
|
||||
ordinal int
|
||||
}
|
||||
var edits []edit
|
||||
|
||||
var inserts []galleryedit.Insert
|
||||
for _, f := range families {
|
||||
entry, ok := byName[strings.ToLower(f.Parent)]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("parent %q is not in the index", f.Parent)
|
||||
}
|
||||
items := make([]string, 0, len(f.Proposals))
|
||||
|
||||
variants := make([]string, 0, len(f.Proposals))
|
||||
for _, p := range f.Proposals {
|
||||
items = append(items, " - model: "+quoteName(p.Variant))
|
||||
variants = append(variants, p.Variant)
|
||||
}
|
||||
|
||||
at, remove, err := insertionPoint(ix, entry)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
insert := items
|
||||
if remove > 0 || !hasVariantsKey(ix, entry) {
|
||||
insert = append([]string{" variants:"}, items...)
|
||||
}
|
||||
edits = append(edits, edit{at: at, remove: remove, insert: insert, ordinal: entry.Index})
|
||||
inserts = append(inserts, galleryedit.Insert{
|
||||
Entry: galleryedit.Entry{
|
||||
Name: entry.Name,
|
||||
StartLine: entry.StartLine,
|
||||
EndLine: entry.EndLine,
|
||||
},
|
||||
Variants: variants,
|
||||
})
|
||||
}
|
||||
|
||||
// Applying from the bottom up keeps every line number computed against the
|
||||
// original text valid while earlier edits are still pending.
|
||||
sort.Slice(edits, func(i, j int) bool { return edits[i].at > edits[j].at })
|
||||
|
||||
lines := append([]string(nil), ix.Lines...)
|
||||
for _, e := range edits {
|
||||
tail := append([]string(nil), lines[e.at+e.remove:]...)
|
||||
lines = append(lines[:e.at], append(append([]string(nil), e.insert...), tail...)...)
|
||||
}
|
||||
return lines, nil
|
||||
}
|
||||
|
||||
func hasVariantsKey(ix *Index, e *GalleryEntry) bool {
|
||||
for i := e.StartLine; i < e.EndLine; i++ {
|
||||
if keyVariants.MatchString(ix.Lines[i]) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// insertionPoint reports where new variant items belong, and how many existing
|
||||
// lines the insertion replaces.
|
||||
//
|
||||
// An entry with no variants key gets one right after its name, which is where
|
||||
// the hand-written families put it. An entry with an empty "variants: []" has
|
||||
// that line replaced by a block. An entry with a block gets its items appended.
|
||||
func insertionPoint(ix *Index, e *GalleryEntry) (at int, remove int, err error) {
|
||||
for i := e.StartLine; i < e.EndLine; i++ {
|
||||
m := keyVariants.FindStringSubmatch(ix.Lines[i])
|
||||
if m == nil {
|
||||
continue
|
||||
}
|
||||
if strings.TrimSpace(m[1]) == "[]" {
|
||||
return i, 1, nil
|
||||
}
|
||||
if strings.TrimSpace(m[1]) != "" {
|
||||
return 0, 0, fmt.Errorf("entry %q writes its variants inline (%q); this job only edits block lists", e.Name, strings.TrimSpace(m[1]))
|
||||
}
|
||||
last := i
|
||||
for j := i + 1; j < e.EndLine && variantItem.MatchString(ix.Lines[j]); j++ {
|
||||
last = j
|
||||
}
|
||||
return last + 1, 0, nil
|
||||
}
|
||||
|
||||
if inlineName.MatchString(ix.Lines[e.StartLine]) {
|
||||
return e.StartLine + 1, 0, nil
|
||||
}
|
||||
for i := e.StartLine; i < e.EndLine; i++ {
|
||||
if keyName.MatchString(ix.Lines[i]) {
|
||||
return i + 1, 0, nil
|
||||
}
|
||||
}
|
||||
return 0, 0, fmt.Errorf("entry %q has no name line to anchor the insertion to", e.Name)
|
||||
}
|
||||
|
||||
// quoteName quotes a variant reference when the name would otherwise change
|
||||
// meaning as bare YAML. Config-suffixed names carry a ":" and always need it.
|
||||
func quoteName(name string) string {
|
||||
if unsafeInName.MatchString(name) {
|
||||
return `"` + strings.ReplaceAll(name, `"`, `\"`) + `"`
|
||||
}
|
||||
return name
|
||||
return galleryedit.Apply(ix.Lines, inserts)
|
||||
}
|
||||
|
||||
11
.github/ci/variantproposals/index.go
vendored
11
.github/ci/variantproposals/index.go
vendored
@@ -8,6 +8,8 @@ import (
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
|
||||
"github.com/mudler/LocalAI/.github/ci/galleryedit"
|
||||
)
|
||||
|
||||
// File is the subset of a gallery file entry the proposer reads.
|
||||
@@ -57,7 +59,6 @@ type Index struct {
|
||||
}
|
||||
|
||||
var (
|
||||
entryStart = regexp.MustCompile(`^-(?: |$)`)
|
||||
anchorStart = regexp.MustCompile(`^- &(\S+)`)
|
||||
mergeStart = regexp.MustCompile(`^- !!merge <<: \*(\S+)`)
|
||||
)
|
||||
@@ -82,13 +83,7 @@ func ParseIndex(text string) (*Index, error) {
|
||||
return nil, fmt.Errorf("decoding gallery index: %w", err)
|
||||
}
|
||||
|
||||
lines := strings.Split(text, "\n")
|
||||
var starts []int
|
||||
for i, line := range lines {
|
||||
if entryStart.MatchString(line) {
|
||||
starts = append(starts, i)
|
||||
}
|
||||
}
|
||||
lines, starts := galleryedit.Scan(text)
|
||||
if len(starts) != len(entries) {
|
||||
return nil, fmt.Errorf("gallery index has %d decoded entries but %d top level list items; refusing to edit by line number", len(entries), len(starts))
|
||||
}
|
||||
|
||||
39
.github/workflows/ci-tools-tests.yaml
vendored
Normal file
39
.github/workflows/ci-tools-tests.yaml
vendored
Normal file
@@ -0,0 +1,39 @@
|
||||
---
|
||||
# The packages under .github/ci/ are invisible to `go list ./...`, so neither
|
||||
# `make lint` nor the repository test run ever touches them. Their specs are
|
||||
# dead weight until a workflow names each package explicitly.
|
||||
name: 'CI tool tests'
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- '.github/ci/**'
|
||||
- '.github/workflows/ci-tools-tests.yaml'
|
||||
push:
|
||||
branches:
|
||||
- master
|
||||
paths:
|
||||
- '.github/ci/**'
|
||||
jobs:
|
||||
ci-tools:
|
||||
name: 'Test the .github/ci generators'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version-file: go.mod
|
||||
cache: false
|
||||
|
||||
# The discovery heuristics are the risky part of these tools. A regression
|
||||
# produces confident, wrong gallery entries, which is worse than no tool.
|
||||
- name: 'Test the APEX entry generator'
|
||||
run: go test ./.github/ci/apexentries/
|
||||
|
||||
- name: 'Test the variant proposer'
|
||||
run: go test ./.github/ci/variantproposals/
|
||||
|
||||
# Shared by both generators above. Its behaviour is exercised through their
|
||||
# specs; this step exists so a break in the shared package fails under its
|
||||
# own name rather than as a puzzling failure in whichever caller ran first.
|
||||
- name: 'Test the shared gallery editor'
|
||||
run: go test ./.github/ci/galleryedit/
|
||||
7
.gitignore
vendored
7
.gitignore
vendored
@@ -111,3 +111,10 @@ core/http/react-ui/test-results/
|
||||
# the realtime-conformance gate; only the .fizz sources are authoritative.
|
||||
formal-verification/*.json
|
||||
formal-verification/out/
|
||||
|
||||
# `go build ./.github/ci/apexentries` drops a binary of the package name into
|
||||
# whatever directory it runs in, one `git add -A` away from being committed.
|
||||
# Both paths are anchored: an unanchored `apexentries` would also match the
|
||||
# package directory itself and untrack the source.
|
||||
/apexentries
|
||||
/.github/ci/apexentries/apexentries
|
||||
|
||||
6057
gallery/index.yaml
6057
gallery/index.yaml
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user