mirror of
https://github.com/mudler/LocalAI.git
synced 2026-07-30 18:09:05 -04:00
47c0e06198
The "Bump Backend dependencies" workflow has failed every night for the last two weeks. #11012 fixed one cause (repos renamed under localai-org); what is left is rate limiting. bump_deps.sh fans out to ~25 parallel matrix jobs that each query api.github.com anonymously. Anonymous calls are capped at 60/hour per source IP and GitHub-hosted runners egress through shared NAT addresses, so a random handful of jobs draw HTTP 403 and die at curl exit 22 with an empty response. Last night that hit ggml-org/whisper.cpp and mudler/depth-anything.cpp -- both public and resolvable, nothing wrong with either pin. Route every bump script through a shared gh_curl helper that sends GITHUB_TOKEN when present (1000/hour instead of 60) and retries transient failures, including the 403s that plain --retry ignores. The helper suppresses xtrace around the call so the Authorization header cannot land in a public job log. bump_docs.sh had a sharper version of the same bug: it piped an unchecked response into `jq -r .tag_name`, so a throttled request resolved to the string "null" and would have been published as the docs version. It now refuses to write anything it cannot resolve to a tag. Verified locally by running all four scripts end to end against their real upstreams: correct SHAs/tags written, exit 0; a nonexistent repo now fails with a named diagnostic instead of a bare exit 22 and leaves the pinned file untouched; the token is absent from the xtrace output; and the scripts still work unauthenticated. Assisted-by: Claude:opus-4.8 [Claude Code] Co-authored-by: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com>
40 lines
1.4 KiB
Bash
Executable File
40 lines
1.4 KiB
Bash
Executable File
#!/bin/bash
|
|
# Shared curl wrapper for the nightly dependency-bump scripts.
|
|
#
|
|
# The bump workflow fans out to ~25 parallel matrix jobs, each querying
|
|
# api.github.com. Anonymous API calls are capped at 60/hour per source IP and
|
|
# GitHub-hosted runners egress through shared NAT addresses, so a random handful
|
|
# of jobs were getting rate-limited (HTTP 403 -> curl exit 22, empty response)
|
|
# every single night. Authenticating with GITHUB_TOKEN lifts the ceiling to
|
|
# 1000/hour; the retries absorb whatever transient blips remain.
|
|
|
|
# Wraps curl with GitHub auth (when a token is present) plus retry/timeout
|
|
# hardening. Callers pass their own headers and the URL.
|
|
gh_curl() {
|
|
# The bump scripts run under `set -x`; without this the Authorization header
|
|
# would be echoed into the job log on every call.
|
|
local had_xtrace=0
|
|
case "$-" in
|
|
*x*) had_xtrace=1; set +x ;;
|
|
esac
|
|
|
|
local args=(
|
|
--silent --show-error --location --fail
|
|
# --retry-all-errors so 403 rate-limit responses are retried too; plain
|
|
# --retry only covers 408/429/5xx. curl honours Retry-After when sent.
|
|
--retry 5 --retry-delay 3 --retry-all-errors
|
|
--connect-timeout 15 --max-time 60
|
|
)
|
|
if [ -n "${GITHUB_TOKEN:-}" ]; then
|
|
args+=(--header "Authorization: Bearer ${GITHUB_TOKEN}")
|
|
fi
|
|
|
|
curl "${args[@]}" "$@"
|
|
local rc=$?
|
|
|
|
if [ "$had_xtrace" -eq 1 ]; then
|
|
set -x
|
|
fi
|
|
return $rc
|
|
}
|