* reverseproxy: extract DNS SRV+A discovery into a reusable package
Move the SRV and A/AAAA resolution + caching logic out of reverseproxy
into a new, transport-neutral package (dynamicupstreams) that returns
neutral targets instead of *reverseproxy.Upstream. reverseproxy's
SRVUpstreams and AUpstreams now delegate to it and build their own
upstreams from the targets; behavior is unchanged.
This lets other proxies (e.g. third-party layer4 proxies) reuse the same
DNS discovery + caching instead of copying it, reducing duplication and
maintenance burden. reverseproxy/upstreams.go shrinks substantially.
RFC for the de-duplication discussed in caddyserver/caddy-l4#429.
* test(dynamicupstreams): cover ResetAllSRV, filtered records, cache bound, concurrent refresh
Brings the package to 96.8% statement coverage. Adds:
- TestResetAllSRV: full-cache drop
- TestSRVFilteredRecords: LookupSRV partial-error semantics (usable
records returned, error downgraded to a warning)
- TestSRVCacheBound / TestACacheBound: eviction keeps the cache bounded
at maxCacheEntries
- TestSRVConcurrentRefreshDeduplicates / TestAConcurrentRefreshDeduplicates:
two goroutines missing the read-lock fast path trigger only one lookup
(verified under -race)
The grace-period and filtered-records logs now use a level-enabled logger
so the log-write branches execute. The only uncovered statements are the
two double-checked-locking re-check returns, which are reachable solely in
a narrow race window (the same untested idiom as reverseproxy's SRV/A).
* reverseproxy: move DNS SRV+A discovery into modules/internal/network
Per review feedback, the shared discovery code does not belong in a
top-level package: it is internal helper code, not part of Caddy's
runtime surface. Move it from the top-level dynamicupstreams package into
the existing modules/internal/network package and update reverse_proxy's
upstreams.go to consume it from there.
Being under modules/internal means third-party modules (e.g. layer4)
cannot import it yet; they keep their own copy until layer4 is part of
Caddy's standard distribution, at which point they can switch over.
* test(dynamicupstreams): correct the concurrent-refresh test's claim
The two concurrent-refresh tests documented themselves as covering the
write-lock re-check branch ("the second re-checks under the write lock"),
but they do not: while the first goroutine holds the write lock the second
cannot acquire the read lock either, so by the time it acquires srvMu the
cache is already populated and it is served by the read-lock fast path. The
50ms sleep was presented as letting the second goroutine "queue on the write
lock" but proved nothing.
Make the comments and failure messages honest: these tests assert
deduplication (two concurrent callers for one uncached key share a single
lookup) and, under -race, race-safety on the shared cache -- not a specific
locking branch. Ordering is enforced by the channels, so the sleep is removed.
No production code change.
* reverseproxy: preserve DNS discovery logging and grace behavior
Keep refresh and discovery logs in the DNS lookup path, retaining the original SRV field types. Preserve error suppression when an SRV lookup fails with grace enabled before any result has been cached.
Add regression tests for refresh-only logging and cold-cache grace handling.
AI-assisted-by: OpenAI Codex (GPT-6)
---------
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* network_proxy: reject proxy URLs that resolve to a port with no host
The host check after placeholder replacement had its arguments swapped:
strings.Split("", pUrl.Host)[0] == ":"
This splits the empty string using pUrl.Host as the separator, so it
returns [""] and element 0 is always "", never ":". The comparison was
dead for every possible host value, which meant the "http://:80" case
named in the comment directly below it was never rejected. Such a URL
was handed back as the proxy, and the failure surfaced later as a
confusing dial error instead of the intended message.
Only the pUrl.Host == "" half of the condition ever did anything, so
"/some/path" was still caught.
Use url.URL.Hostname(), which returns the host with any port stripped
and is "" for exactly the two cases the comment describes -- ":80" and
"" -- while leaving IPv6 literals such as "[::1]:80" and userinfo forms
intact. That collapses both clauses into one expression.
Also adds tests for this function; the package previously had none.
* network_proxy: cover userinfo-only and IPv6 hosts in the tests
Differential-tested the old predicate against the new one across 36 URL
forms. Every behavioural change is in the same direction -- previously
accepted, now rejected -- and all of them are host-less. Nothing that was
rejected before is accepted now, and no legitimate host form changes.
Two of those forms were worth pinning down in the test:
- "http://user:pass@:8080" parses with Host ":8080", so it is just as
host-less as "http://:80". The comment in the source doesn't name
this variant, but it was accepted before and is rejected now.
- IPv6 literals are full of colons, so a repair that split Host on ":"
rather than using Hostname() could plausibly reject them. Added
"[::1]:8080" and "[2001:db8::1]" as regression guards; both are
accepted before and after, which is the point.
On unfixed master the port-only and userinfo cases both fail; the IPv6
cases pass on both sides.
* core: add modular `network_proxy` support
Co-authored-by: @ImpostorKeanu
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* move modules around
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* add caddyfile implementation
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* address feedbcak
* Apply suggestions from code review
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
* adapt ForwardProxyURL to use the NetworkProxyRaw
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* remove redundant `url` in log
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
* code review
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* remove `.source` from the module ID
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
---------
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
