Files
LocalAI/pkg/oci/cosignverify/verify.go
mudler-agentandEttore Di Giacinto 543fb4bd24 fix(gallery): verification follow-ups for oci:// galleries (#12243)
* fix(gallery): verification follow-ups for oci:// galleries

Follow-ups from the post-merge review of #12238 and #12239.

Only a policy decision is a refusal now. cosignverify wraps
ErrPolicyRejected around a failed signature check, an identity or
source-repository mismatch, a not_before cutoff and a missing or
unparseable bundle. A TUF, registry or network failure during
verification, or a timeout, is an outage: the gallery falls back to the
copy verified under the current policy, as it does when the registry is
down.

An oci:// gallery with a verification block, or any oci:// gallery under
strict integrity, is no longer answered by an https://, github: or
file:// mirror. Such a mirror is ignored with a warning, because nothing
can check its signature. The index of an HTTP gallery, whose policy only
covers its backend images, is cached under the URL-only name again, so no
unchecked body is stored under a policy-keyed name.

The in-memory index cache key now includes the policy. After a runtime
policy change the index is fetched again, and entries with a relative url
install again.

The registry digest lookups after install and upgrade, and in the
upgrade check, run only for real registry references (new
URI.LooksLikeRegistryOCI), not for ollama:// or ocifile://.

The refusal message names strict integrity when that is the cause, and
the gallery name is no longer repeated.

Specs pin the URL-only cache name for galleries without a policy, a fixed
key for a fixed policy, and that every GalleryVerification field changes
the key. The docs describe refusal, outage, mirrors and strict integrity.

Assisted-by: Claude:claude-opus-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(gallery): reset listings on gallery changes, classify referrer outages

Review follow-ups for this PR.

The React UI lists from AvailableGalleryModelsCached, which is keyed by
nothing. A gallery change through the settings API or a
runtime_settings.json edit now drops that listing when the model or
backend gallery configuration differs. Before, the UI kept the old list,
with local paths into the old policy's tree, until the next background
refresh, or for good when the new policy refused the gallery.

In cosignverify, a referrer the registry fails to serve now makes the
lookup an outage whatever other referrers failed and in any order, since
the unread one may be the valid signature. An invalid policy (Validate in
NewVerifier, an unparseable not_before) is ErrPolicyRejected, because no
fetch can make it usable.

The docs say that only an oci:// gallery with a verification block skips
non-OCI mirrors, and list an unusable policy as a refusal.

Assisted-by: Claude:claude-opus-5-5 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
2026-09-24 21:20:21 +02:00

373 lines
13 KiB
Go

// Package cosignverify verifies cosign-signed OCI images using sigstore-go.
//
// LocalAI uses this to gate backend installs on a keyless-cosign signature
// from a trusted GitHub Actions OIDC identity, so a registry/tag compromise
// alone is not sufficient to ship a tampered backend image.
//
// Producer side: CI signs each pushed backend image with cosign 2.2+ and
// the `--new-bundle-format --registry-referrers-mode=oci-1-1` flags. The
// signature is then a standalone Sigstore bundle stored as an OCI 1.1
// referrer of the image manifest.
//
// Consumer side (this package): bundle.go discovers the bundle via the
// referrers API and hands it directly to sigstore-go's verifier. There is
// no legacy-cosign-annotation fallback — we own both ends.
package cosignverify
import (
"context"
"encoding/hex"
"errors"
"fmt"
"net/http"
"net/url"
"strings"
"sync"
"time"
registrytypes "github.com/docker/docker/api/types/registry"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
"github.com/mudler/LocalAI/internal"
"github.com/mudler/LocalAI/pkg/credentials"
"github.com/sigstore/sigstore-go/pkg/fulcio/certificate"
"github.com/sigstore/sigstore-go/pkg/root"
"github.com/sigstore/sigstore-go/pkg/tuf"
"github.com/sigstore/sigstore-go/pkg/verify"
)
// ErrPolicyRejected marks a verification that reached a decision: the image
// carries no signature, or its signature does not satisfy the policy.
//
// Callers need the distinction because every other failure here (the TUF
// root or the registry being unreachable, a timeout, a 5xx) says nothing
// about the image, and a caller that keeps a copy verified earlier may serve
// it through an outage but must never serve it over a refusal.
var ErrPolicyRejected = errors.New("rejected by the signature policy")
// Policy is the verification policy a backend image must satisfy.
//
// At least one of Issuer / IssuerRegex must be set, and at least one of
// Identity / IdentityRegex. The (Issuer, Identity) pair pins which OIDC
// principal Fulcio issued the signing cert to — for GitHub Actions keyless
// signing this is typically:
//
// Issuer: "https://token.actions.githubusercontent.com"
// IdentityRegex: "^https://github.com/<org>/<repo>/\\.github/workflows/<file>@refs/.*"
//
// A registry compromise alone cannot satisfy this; the attacker would also
// need to compromise the GitHub Actions OIDC identity to obtain a Fulcio
// cert with a matching SAN.
type Policy struct {
Issuer string
IssuerRegex string
Identity string
IdentityRegex string
// SourceRepository, when set, must equal the signing certificate's
// source-repository extension exactly (for GitHub Actions,
// https://github.com/<owner>/<repo>). When a reusable signing
// workflow is shared by several repositories, the SAN names that
// shared workflow, so the SAN alone accepts a signature made for any
// of its callers; the source repository is what pins the caller.
SourceRepository string
// TUFRootURL overrides the default sigstore public-good TUF mirror
// (tuf-repo-cdn.sigstore.dev). Leave empty for the public good.
TUFRootURL string
// TUFCachePath overrides the on-disk cache directory for the TUF
// metadata. Leave empty for the sigstore-go default.
TUFCachePath string
// RequireTLog requires an inclusion proof from the Rekor transparency
// log. Defaults to true; only disable for testing.
RequireTLog *bool
// RequireSCT requires the signing certificate to embed a Signed
// Certificate Timestamp from the certificate-transparency log.
// Defaults to true.
RequireSCT *bool
// NotBefore rejects signatures whose Rekor integrated time is older
// than this. This is the revocation lever: keyless cosign certs are
// ephemeral so there is no CA-side revocation, but advancing NotBefore
// in the gallery YAML invalidates any signature produced before a
// known compromise window. Zero value means no time-based cutoff.
NotBefore time.Time
}
func boolOrTrue(b *bool) bool {
if b == nil {
return true
}
return *b
}
// Validate returns an error if the policy is missing required fields.
func (p Policy) Validate() error {
if p.Issuer == "" && p.IssuerRegex == "" {
return errors.New("cosignverify: policy must set Issuer or IssuerRegex")
}
if p.Identity == "" && p.IdentityRegex == "" {
return errors.New("cosignverify: policy must set Identity or IdentityRegex")
}
if p.SourceRepository != "" {
u, err := url.Parse(p.SourceRepository)
if err != nil || u.Scheme != "https" || u.Host == "" || strings.TrimSpace(p.SourceRepository) != p.SourceRepository {
return errors.New("cosignverify: source repository must be an https URL, such as https://github.com/<owner>/<repo>")
}
}
return nil
}
// certificateIdentity is the identity a signature's certificate must match.
// Without a source repository it is exactly the short identity used before
// the field existed.
func (p Policy) certificateIdentity() (verify.CertificateIdentity, error) {
if p.SourceRepository == "" {
return verify.NewShortCertificateIdentity(p.Issuer, p.IssuerRegex, p.Identity, p.IdentityRegex)
}
san, err := verify.NewSANMatcher(p.Identity, p.IdentityRegex)
if err != nil {
return verify.CertificateIdentity{}, err
}
issuer, err := verify.NewIssuerMatcher(p.Issuer, p.IssuerRegex)
if err != nil {
return verify.CertificateIdentity{}, err
}
return verify.NewCertificateIdentity(san, issuer, certificate.Extensions{SourceRepositoryURI: p.SourceRepository})
}
// Verifier verifies cosign-signed OCI images against a fixed Policy.
//
// Cheap to construct, safe for concurrent use. The TUF trusted root is
// fetched once per (root URL, cache path) tuple across all Verifiers in
// the process — installing N backends from the same gallery does one TUF
// fetch, not N.
type Verifier struct {
policy Policy
// Registry plumbing — reused from the existing pkg/oci surface so we
// honor the same auth / transport conventions.
auth *registrytypes.AuthConfig
transport http.RoundTripper
}
// NewVerifier constructs a Verifier. The trusted root is not fetched yet;
// it is loaded on the first call to VerifyImage. auth and t may be nil.
func NewVerifier(p Policy, auth *registrytypes.AuthConfig, t http.RoundTripper) (*Verifier, error) {
if err := p.Validate(); err != nil {
// A policy that cannot be used admits nothing, so a caller must
// treat it as a refusal and not as an outage a cached copy covers.
return nil, fmt.Errorf("%w: %w", ErrPolicyRejected, err)
}
return &Verifier{policy: p, auth: auth, transport: t}, nil
}
// trustedMaterialCacheKey identifies which TUF mirror + on-disk cache a
// Verifier wants. Two Verifiers with identical keys share trusted material.
type trustedMaterialCacheKey struct {
URL string
Path string
}
type trustedMaterialEntry struct {
once sync.Once
material root.TrustedMaterialCollection
err error
}
var trustedMaterialCache sync.Map // map[trustedMaterialCacheKey]*trustedMaterialEntry
func (v *Verifier) loadTrustedMaterial() (root.TrustedMaterialCollection, error) {
key := trustedMaterialCacheKey{URL: v.policy.TUFRootURL, Path: v.policy.TUFCachePath}
val, _ := trustedMaterialCache.LoadOrStore(key, &trustedMaterialEntry{})
entry := val.(*trustedMaterialEntry)
entry.once.Do(func() {
opts := tuf.DefaultOptions()
if v.policy.TUFRootURL != "" {
opts.RepositoryBaseURL = v.policy.TUFRootURL
}
if v.policy.TUFCachePath != "" {
opts.CachePath = v.policy.TUFCachePath
}
client, err := tuf.New(opts)
if err != nil {
entry.err = fmt.Errorf("cosignverify: initialising TUF client: %w", err)
return
}
trustedRootJSON, err := client.GetTarget("trusted_root.json")
if err != nil {
entry.err = fmt.Errorf("cosignverify: fetching trusted_root.json: %w", err)
return
}
tr, err := root.NewTrustedRootFromJSON(trustedRootJSON)
if err != nil {
entry.err = fmt.Errorf("cosignverify: parsing trusted root: %w", err)
return
}
entry.material = root.TrustedMaterialCollection{tr}
})
return entry.material, entry.err
}
// VerifyImage resolves imageRef to its manifest digest, fetches the cosign
// signature attachment (the conventional `:sha256-<hex>.sig` tag), assembles
// a Sigstore bundle from the cosign annotations, and verifies that bundle
// against the configured Policy.
//
// Returns nil on the first signature in the attachment that satisfies the
// policy. Returns an error if none do, or if any part of the fetch fails.
func (v *Verifier) VerifyImage(ctx context.Context, imageRef string) error {
if err := ctx.Err(); err != nil {
return err
}
trusted, err := v.loadTrustedMaterial()
if err != nil {
return err
}
ref, err := name.ParseReference(imageRef)
if err != nil {
return fmt.Errorf("cosignverify: parse image ref %q: %w", imageRef, err)
}
opts := v.remoteOptions(ctx)
// Resolve the image to its manifest digest. With the new-bundle-format
// flow the cosign signature is taken over the manifest digest directly,
// so this is also the artifact we ask the verifier to bind against.
// Skip the HEAD when the ref is already digest-pinned (the typical
// path from pkg/downloader, which resolves the digest before calling
// us): name.ParseReference returns a name.Digest in that case.
var digest v1.Hash
if d, ok := ref.(name.Digest); ok {
h, herr := v1.NewHash(d.DigestStr())
if herr != nil {
return fmt.Errorf("cosignverify: parsing pinned digest %q: %w", d.DigestStr(), herr)
}
digest = h
} else {
desc, herr := remote.Head(ref, opts...)
if herr != nil {
return fmt.Errorf("cosignverify: resolving image descriptor: %w", herr)
}
digest = desc.Digest
}
bun, err := bundleFromOCISignature(ref, digest, opts)
if err != nil {
return err
}
verifierOpts := []verify.VerifierOption{}
if boolOrTrue(v.policy.RequireSCT) {
verifierOpts = append(verifierOpts, verify.WithSignedCertificateTimestamps(1))
}
if boolOrTrue(v.policy.RequireTLog) {
verifierOpts = append(verifierOpts, verify.WithTransparencyLog(1))
verifierOpts = append(verifierOpts, verify.WithObserverTimestamps(1))
}
certID, err := v.policy.certificateIdentity()
if err != nil {
// A policy that cannot be built admits nothing, whatever the
// network does, so this is a decision rather than an outage.
return fmt.Errorf("cosignverify: building identity policy: %w: %w", ErrPolicyRejected, err)
}
sev, err := verify.NewVerifier(trusted, verifierOpts...)
if err != nil {
return fmt.Errorf("cosignverify: constructing verifier: %w", err)
}
artifactDigest, err := hex.DecodeString(digest.Hex)
if err != nil {
return fmt.Errorf("cosignverify: decoding image digest: %w", err)
}
artifactPolicy := verify.WithArtifactDigest(digest.Algorithm, artifactDigest)
result, err := sev.Verify(bun, verify.NewPolicy(artifactPolicy, verify.WithCertificateIdentity(certID)))
if err != nil {
return fmt.Errorf("cosignverify: verification failed for %s: %w: %w", imageRef, ErrPolicyRejected, err)
}
if !v.policy.NotBefore.IsZero() {
if err := enforceNotBefore(result, v.policy.NotBefore); err != nil {
return fmt.Errorf("cosignverify: %s: %w", imageRef, err)
}
}
return nil
}
// enforceNotBefore rejects a verification result whose earliest verified
// timestamp predates cutoff. Used as a revocation lever — see Policy.NotBefore.
func enforceNotBefore(result *verify.VerificationResult, cutoff time.Time) error {
if result == nil || len(result.VerifiedTimestamps) == 0 {
// Defensive: with RequireTLog=true (the default) sigstore-go will
// have already failed verification if there was no verifiable
// timestamp, so this branch is only reachable if a caller set
// RequireTLog=false. Treat as a hard error: if you opted into
// NotBefore, you implicitly opted into needing a timestamp.
return fmt.Errorf("%w: signature has no verified timestamp; cannot enforce NotBefore", ErrPolicyRejected)
}
earliest := result.VerifiedTimestamps[0].Timestamp
for _, ts := range result.VerifiedTimestamps[1:] {
if ts.Timestamp.Before(earliest) {
earliest = ts.Timestamp
}
}
if earliest.Before(cutoff) {
return fmt.Errorf("%w: signature integrated time %s is before NotBefore cutoff %s",
ErrPolicyRejected, earliest.Format(time.RFC3339), cutoff.Format(time.RFC3339))
}
return nil
}
func (v *Verifier) remoteOptions(ctx context.Context) []remote.Option {
t := v.transport
if t == nil {
t = http.DefaultTransport
}
// Match the retry policy used elsewhere in pkg/oci so transient
// registry hiccups don't fail verification.
t = transport.NewRetry(t)
opts := []remote.Option{
remote.WithContext(ctx),
remote.WithTransport(t),
remote.WithUserAgent(internal.UserAgent()),
}
if v.auth != nil {
opts = append(opts, remote.WithAuth(staticAuth{auth: v.auth}))
} else {
opts = append(opts, remote.WithAuthFromKeychain(credentials.Keychain()))
}
return opts
}
// staticAuth mirrors pkg/oci's adapter so callers can pass the same
// docker auth config they use everywhere else.
type staticAuth struct {
auth *registrytypes.AuthConfig
}
func (s staticAuth) Authorization() (*authn.AuthConfig, error) {
if s.auth == nil {
return nil, nil
}
return &authn.AuthConfig{
Username: s.auth.Username,
Password: s.auth.Password,
Auth: s.auth.Auth,
IdentityToken: s.auth.IdentityToken,
RegistryToken: s.auth.RegistryToken,
}, nil
}