Files
caddy/modules/caddyhttp/reverseproxy
6584c75999 reverseproxy: isolate active health-check state per distinct check config (#7916)
* reverseproxy: isolate active health-check state per distinct check config

Multiple reverse_proxy handlers configured with different active health
checks (health_uri, health_headers, ...) against the same upstream dial
address currently share a single Host in the global pool, so one
handler's failing probes mark the address unhealthy for every other
handler. Key the pool by dial address plus a stable fingerprint of the
active health-check config, so distinct checks get independent health
state.

The fingerprint is strictly internal to pool identity: the Prometheus
upstreams_healthy label and the /reverse_proxy/upstreams admin endpoint
continue to report the plain dial address, unchanged.

Dynamic upstreams are intentionally out of scope here: they resolve
through a separate per-lookup path (dynamicHosts) and collapsing there
has different lifetime semantics; noted for a follow-up.

Fixes #7870

* reverseproxy: use strings.Cut in hostKeyAddress

Satisfies the modernize linter; behaviour is unchanged, since Cut returns
the whole string when the separator is absent.

* reverseproxy: expose the health-check fingerprint as a public discriminator

Health state is now kept per (dial address, active health check config),
but both user-visible surfaces still reported address alone:

- caddy_reverse_proxy_upstreams_healthy was labeled only by upstream, so
  every handler sharing an address wrote the same series concurrently and
  the reported value was whichever updater ran last. The metric gains a
  health_check label carrying the config fingerprint ("" when no active
  checks), so each health target owns its series; aggregate across checks
  with sum/min by (upstream).

- /reverse_proxy/upstreams reported one entry per pool key but with only
  the plain address, so consumers indexing by address silently discarded
  all but one entry. Entries now carry health_check (omitted when empty),
  and the endpoint documents that (address, health_check) is the entry's
  identity — one entry per health target, deliberately not aggregated,
  since any aggregation here would be lossy and undocumented.

Tests: two handlers on one address with different checks must produce two
metric series reflecting their own state (fails if the fingerprint is
dropped from the label), and two admin entries distinguished by non-empty
fingerprints.

* reverseproxy: narrow the fix to per-Upstream active health counters

Move the consecutive active pass/fail counters from Host onto Upstream,
alongside the active unhealthy state that already lives there, instead of
re-keying the global host pool.

Host is keyed by dial address alone, but an active health check is
configured per handler, so two handlers dialing the same address with
different health_uri or health_headers share those counters and can push
each other over their own thresholds. Upstream is already per-handler and
already carries the active unhealthy flag, so the counters belong next to
it and the pool keeps its plain dial-address keys.

This drops the host key fingerprint and its exposure in the metric label
and the admin upstreams endpoint; the metric series identity is left for
separate consideration.

---------

Co-authored-by: SillyZir <269283839+SillyZir@users.noreply.github.com>
Co-authored-by: Zen Dodd <mail@steadytao.com>
2026-08-12 15:20:15 +10:00
..
2026-08-11 08:00:15 -06:00
2026-06-12 12:39:01 -06:00