mirror of
https://github.com/mudler/LocalAI.git
synced 2026-10-28 17:18:49 -04:00
mudler-agentandEttore Di Giacinto
3b179ed142
* ci(gallery): have the checksum bot write the model library The checksum bot edited gallery/index.yaml, which is a generated copy of gallery/models/index.yaml. The model library check fails any PR where the copy differs from the generated one, and an edit made to the copy never reaches the source. Point the checker at the library, read the files under .model.files as the library keeps them, and run modelcatalog after the checker so the PR carries both files. Assisted-by: Claude:claude-sonnet-5-5 [Claude Code] Signed-off-by: Ettore Di Giacinto <mudler@localai.io> * ci(gallery): have the other index bots write the model library The gallery agent and the variant proposer still read and wrote the generated gallery copy, so their pull requests failed the model library check and their changes never reached the source. The gallery agent now appends library entries to gallery/models/index.yaml, converting its entries with the gallery package and skipping ids the library already has. Its duplicate check reads the same file. The variant proposer decodes the library shape (the body under model:) and splices variants into the model body through a library layout in galleryedit. The anchor refusal goes: the library has no anchors. Its tests use library entries. Both workflows run modelcatalog after the bot and before the pull request step, and use the Go version from go.mod. Signed-off-by: Ettore Di Giacinto <mudler@localai.io> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
221 lines
11 KiB
YAML
221 lines
11 KiB
YAML
name: Gallery Agent
|
|
on:
|
|
|
|
schedule:
|
|
- cron: '0 */12 * * *' # Run every 4 hours
|
|
workflow_dispatch:
|
|
inputs:
|
|
search_term:
|
|
description: 'Search term for models'
|
|
required: false
|
|
default: 'GGUF'
|
|
type: string
|
|
limit:
|
|
description: 'Maximum number of models to process'
|
|
required: false
|
|
default: '15'
|
|
type: string
|
|
quantization:
|
|
description: 'Preferred quantization format'
|
|
required: false
|
|
default: 'Q4_K_M'
|
|
type: string
|
|
max_models:
|
|
description: 'Maximum number of models to add to the gallery'
|
|
required: false
|
|
default: '1'
|
|
type: string
|
|
jobs:
|
|
gallery-agent:
|
|
if: github.repository == 'mudler/LocalAI'
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v7
|
|
with:
|
|
token: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v5
|
|
with:
|
|
go-version-file: go.mod
|
|
- name: Proto Dependencies
|
|
run: |
|
|
# Install protoc
|
|
curl -L -s https://github.com/protocolbuffers/protobuf/releases/download/v26.1/protoc-26.1-linux-x86_64.zip -o protoc.zip && \
|
|
unzip -j -d /usr/local/bin protoc.zip bin/protoc && \
|
|
rm protoc.zip
|
|
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
|
|
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
|
|
PATH="$PATH:$HOME/go/bin" make protogen-go
|
|
- name: Process gallery-agent PR commands
|
|
env:
|
|
GH_TOKEN: ${{ secrets.UPDATE_BOT_TOKEN }}
|
|
REPO: ${{ github.repository }}
|
|
SEARCH: 'gallery agent in:title'
|
|
run: |
|
|
# Walk gallery-agent PRs and act on maintainer comments:
|
|
# /gallery-agent blacklist → label `gallery-agent/blacklisted` + close (never repropose)
|
|
# /gallery-agent recreate → close without label (next run may repropose)
|
|
# Only comments from OWNER / MEMBER / COLLABORATOR are honored so
|
|
# random users can't drive the bot.
|
|
#
|
|
# We scan both open PRs AND recently-closed PRs that don't already
|
|
# carry the blacklist label. This covers the common flow where a
|
|
# maintainer writes /gallery-agent blacklist and immediately clicks
|
|
# Close — without this, the next scheduled run wouldn't see the
|
|
# command (PR is already closed) and would repropose the model.
|
|
gh label create gallery-agent/blacklisted \
|
|
--repo "$REPO" --color ededed \
|
|
--description "gallery-agent must not repropose this model" 2>/dev/null || true
|
|
|
|
prs_open=$(gh pr list --repo "$REPO" --state open --search "$SEARCH" \
|
|
--json number --jq '.[].number')
|
|
# Closed PRs from the last 14 days that don't yet have the blacklist label.
|
|
# Bounded window keeps the scan cheap while covering late-applied commands.
|
|
since=$(date -u -d '14 days ago' +%Y-%m-%d)
|
|
prs_closed=$(gh pr list --repo "$REPO" --state closed \
|
|
--search "$SEARCH closed:>=$since -label:gallery-agent/blacklisted" \
|
|
--json number --jq '.[].number')
|
|
prs=$(printf '%s\n%s\n' "$prs_open" "$prs_closed" | sort -u | sed '/^$/d')
|
|
for pr in $prs; do
|
|
state=$(gh pr view "$pr" --repo "$REPO" --json state --jq '.state')
|
|
cmds=$(gh pr view "$pr" --repo "$REPO" --json comments \
|
|
--jq '.comments[] | select(.authorAssociation=="OWNER" or .authorAssociation=="MEMBER" or .authorAssociation=="COLLABORATOR") | .body')
|
|
if echo "$cmds" | grep -qE '(^|[[:space:]])/gallery-agent[[:space:]]+blacklist([[:space:]]|$)'; then
|
|
echo "PR #$pr: blacklist command found (state=$state)"
|
|
gh pr edit "$pr" --repo "$REPO" --add-label gallery-agent/blacklisted || true
|
|
if [ "$state" = "OPEN" ]; then
|
|
gh pr close "$pr" --repo "$REPO" --comment "Blacklisted via \`/gallery-agent blacklist\`. This model will not be reproposed." || true
|
|
fi
|
|
elif [ "$state" = "OPEN" ] && echo "$cmds" | grep -qE '(^|[[:space:]])/gallery-agent[[:space:]]+recreate([[:space:]]|$)'; then
|
|
echo "PR #$pr: recreate command found"
|
|
gh pr close "$pr" --repo "$REPO" --comment "Closed via \`/gallery-agent recreate\`. The next scheduled run will propose this model again." || true
|
|
fi
|
|
done
|
|
|
|
- name: Collect skip URLs for the gallery agent
|
|
id: open_prs
|
|
env:
|
|
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
REPO: ${{ github.repository }}
|
|
SEARCH: 'gallery agent in:title'
|
|
run: |
|
|
# Skip set =
|
|
# URLs from any open gallery-agent PR (avoid duplicate PRs for the same model while one is pending)
|
|
# + URLs from closed PRs carrying the `gallery-agent/blacklisted` label (hard blacklist)
|
|
# Plain-closed PRs without the label are ignored — closing a PR is
|
|
# not by itself a "never propose again" signal; maintainers must
|
|
# opt in via the /gallery-agent blacklist comment command.
|
|
urls_open=$(gh pr list --repo "$REPO" --state open --search "$SEARCH" \
|
|
--json body --jq '[.[].body] | join("\n")' \
|
|
| grep -oE 'https://huggingface\.co/[^ )]+' || true)
|
|
urls_blacklist=$(gh pr list --repo "$REPO" --state closed --search "$SEARCH" \
|
|
--label gallery-agent/blacklisted \
|
|
--json body --jq '[.[].body] | join("\n")' \
|
|
| grep -oE 'https://huggingface\.co/[^ )]+' || true)
|
|
urls=$(printf '%s\n%s\n' "$urls_open" "$urls_blacklist" | sort -u | sed '/^$/d')
|
|
echo "Skip URLs:"
|
|
echo "$urls"
|
|
{
|
|
echo "urls<<EOF"
|
|
echo "$urls"
|
|
echo "EOF"
|
|
} >> "$GITHUB_OUTPUT"
|
|
|
|
- name: Run gallery agent
|
|
env:
|
|
SEARCH_TERM: ${{ github.event.inputs.search_term || 'GGUF' }}
|
|
LIMIT: ${{ github.event.inputs.limit || '15' }}
|
|
QUANTIZATION: ${{ github.event.inputs.quantization || 'Q4_K_M' }}
|
|
MAX_MODELS: ${{ github.event.inputs.max_models || '1' }}
|
|
EXTRA_SKIP_URLS: ${{ steps.open_prs.outputs.urls }}
|
|
run: |
|
|
# The agent reads and appends to the model library, the source.
|
|
export GALLERY_INDEX_PATH=$PWD/gallery/models/index.yaml
|
|
go run ./.github/gallery-agent
|
|
|
|
- name: Regenerate the gallery copy from the library
|
|
# gallery/index.yaml is generated from the library; the model library
|
|
# check fails the PR when they disagree.
|
|
run: go run ./scripts/models/cmd/modelcatalog --library gallery/models/index.yaml --out gallery/index.yaml
|
|
|
|
- name: Check for changes
|
|
id: check_changes
|
|
run: |
|
|
if git diff --quiet gallery/models/index.yaml gallery/index.yaml; then
|
|
echo "changes=false" >> $GITHUB_OUTPUT
|
|
echo "No changes detected in the model library"
|
|
else
|
|
echo "changes=true" >> $GITHUB_OUTPUT
|
|
echo "Changes detected in the model library"
|
|
git diff gallery/models/index.yaml gallery/index.yaml
|
|
fi
|
|
|
|
- name: Read gallery agent summary
|
|
id: read_summary
|
|
if: steps.check_changes.outputs.changes == 'true'
|
|
run: |
|
|
if [ -f "./gallery-agent-summary.json" ]; then
|
|
echo "summary_exists=true" >> $GITHUB_OUTPUT
|
|
# Extract summary data using jq
|
|
echo "search_term=$(jq -r '.search_term' ./gallery-agent-summary.json)" >> $GITHUB_OUTPUT
|
|
echo "total_found=$(jq -r '.total_found' ./gallery-agent-summary.json)" >> $GITHUB_OUTPUT
|
|
echo "models_added=$(jq -r '.models_added' ./gallery-agent-summary.json)" >> $GITHUB_OUTPUT
|
|
echo "quantization=$(jq -r '.quantization' ./gallery-agent-summary.json)" >> $GITHUB_OUTPUT
|
|
echo "processing_time=$(jq -r '.processing_time' ./gallery-agent-summary.json)" >> $GITHUB_OUTPUT
|
|
|
|
# Create a formatted list of added models with URLs
|
|
added_models=$(jq -r 'range(0; .added_model_ids | length) as $i | "- [\(.added_model_ids[$i])](\(.added_model_urls[$i]))"' ./gallery-agent-summary.json | tr '\n' '\n')
|
|
echo "added_models<<EOF" >> $GITHUB_OUTPUT
|
|
echo "$added_models" >> $GITHUB_OUTPUT
|
|
echo "EOF" >> $GITHUB_OUTPUT
|
|
rm -f ./gallery-agent-summary.json
|
|
else
|
|
echo "summary_exists=false" >> $GITHUB_OUTPUT
|
|
fi
|
|
|
|
- name: Create Pull Request
|
|
if: steps.check_changes.outputs.changes == 'true'
|
|
uses: peter-evans/create-pull-request@v8
|
|
with:
|
|
token: ${{ secrets.UPDATE_BOT_TOKEN }}
|
|
push-to-fork: ci-forks/LocalAI
|
|
commit-message: 'chore(model gallery): :robot: add new models via gallery agent'
|
|
title: 'chore(model gallery): :robot: add ${{ steps.read_summary.outputs.models_added || 0 }} new models via gallery agent'
|
|
# Branch has to be unique so PRs are not overriding each other
|
|
branch-suffix: timestamp
|
|
body: |
|
|
This PR was automatically created by the gallery agent workflow.
|
|
|
|
**Summary:**
|
|
- **Search Term:** ${{ steps.read_summary.outputs.search_term || github.event.inputs.search_term || 'GGUF' }}
|
|
- **Models Found:** ${{ steps.read_summary.outputs.total_found || 'N/A' }}
|
|
- **Models Added:** ${{ steps.read_summary.outputs.models_added || '0' }}
|
|
- **Quantization:** ${{ steps.read_summary.outputs.quantization || github.event.inputs.quantization || 'Q4_K_M' }}
|
|
- **Processing Time:** ${{ steps.read_summary.outputs.processing_time || 'N/A' }}
|
|
|
|
**Added Models:**
|
|
${{ steps.read_summary.outputs.added_models || '- No models added' }}
|
|
|
|
### Bot commands
|
|
|
|
Maintainers (owner / member / collaborator) can control this PR
|
|
by leaving a comment with one of:
|
|
|
|
- `/gallery-agent recreate` — close this PR; the next scheduled
|
|
run will propose this model again (useful if the entry needs
|
|
to be regenerated with fresh metadata).
|
|
- `/gallery-agent blacklist` — close this PR and permanently
|
|
prevent the gallery agent from ever reproposing this model.
|
|
|
|
Plain "Close" (without a command) is treated as a no-op: the
|
|
model may be reproposed by a future run.
|
|
|
|
**Workflow Details:**
|
|
- Triggered by: `${{ github.event_name }}`
|
|
- Run ID: `${{ github.run_id }}`
|
|
- Commit: `${{ github.sha }}`
|
|
signoff: true
|
|
delete-branch: true
|