* reverseproxy: Keep own headers after 1xx responses
Since #6656 the 1xx handler clears the whole header map after writing the
informational response, which also drops headers set before proxying (for
example by the header handler). Restore those after clearing, as #5091 did.
* reverseproxy: Use maps.Copy to restore own headers
* reverseproxy: Check the 103 reaches the client in the 1xx headers test
Wrap backend dial failures as reverseproxy.DialError so non-GET requests can be retried when no connection was made.
AI-assisted implementation and test; reviewed by zemamba.
Signed-off-by: zemamba <zemamba@gmail.com>
* templates: Delete file headers on any template error
A template that fails to parse or execute (not only {{httpError}}) left
the Content-Length, Etag and related headers set by file_server on the
error response, so clients received a 500 that declared the file's size
but had no body.
* templates: Also delete Content-Encoding and Content-Range on errors
Precompressed and ranged file responses could leave these headers on the
replacement error response.
A handler setting a write deadline through http.ResponseController
reached the connection directly, and the next Write or Flush through
IdleTimeoutWriter replaced it with now+write_idle_timeout. Handlers
bounding a single write, like Mercure's dispatch_timeout, which drops
SSE subscribers that can't receive an update within 5s, were silently
extended to the 1 minute default. On HTTP/2, clearing between writes
also wiped the handler deadline.
IdleTimeoutWriter now implements SetWriteDeadline, keeps the handler
deadline as a ceiling for idle resets, and restores it instead of
HardDeadline alone when clearing between writes. A handler can
tighten the idle timeout but not loosen it.
directoryListing built each symlink's lookup path from the escaped
request path. In a directory whose name needs escaping, such as one with
a space, the stat failed, so directory symlinks were listed as files,
file symlinks showed the link's own size, and reveal_symlinks showed no
target.
Use the decoded path, which the function already computes for the page
title. The escaped path is still used for the page path and item URLs.
read_idle_timeout and write_idle_timeout default to 1 minute when zero,
and the wrappers are only installed for positive values, so a negative
value already disables them. Document it and cover it with tests, since
it is the only way to opt out, for instance behind a reverse proxy that
already protects from slow clients.
* caddyhttp: only bound HTTP/2 body reads in flight with the idle timeout
On HTTP/2, the read deadline is a timer that fails the request body
when it fires, even with no read in flight. IdleTimeoutReader left the
idle deadline armed between reads, so a handler that stopped reading
for longer than read_body_idle (1 minute by default) lost the body,
although the client was only held back by flow control. For example,
reverse_proxy uploading to an upstream that's slow to accept the body
failed with a 502.
This is the read side of #8118. Like IdleTimeoutWriter does since
read on HTTP/2, so the idle deadline only bounds reads in flight. A
client that stops sending is still cut off, since the deadline is set
before each read. HTTP/1 and HTTP/3 only check the deadline during a
read, so they're unchanged.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* caddyhttp: track a released hard read deadline as armed
From review: with HardDeadline set, releasing the idle deadline between
reads left an HTTP/2 timer armed (at HardDeadline) while deadlineSet
recorded that none was, so clearDeadlineLocked would skip clearing it.
Track which deadline is armed (none, idle, or only the hard one) instead
of a bool, and have each check ask the question it means: re-arming and
releasing only apply to an armed idle deadline, while clearing applies
to any.
The writer's deadlineSet already only means "idle deadline armed",
since nothing clears the hard deadline it leaves; document that.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* caddyhttp: clear released hard deadline at handler end, skip no-op sets
From review:
- After a partial read releases the idle deadline to HardDeadline, a
handler that returns without reading again left that timer armed past
HandlerDone, which only cleared after a terminal error. HandlerDone
now also clears a released hard deadline.
- When the idle deadline is capped to HardDeadline, the reader and
writer set the same deadline again when releasing it, and the writer
once more in HandlerDone. On HTTP/2 each is a message to the serve
loop. Both now remember the deadline they armed and skip setting it
again. Within a request, only these wrappers set the client's
deadlines.
Replaces the earlier test that faked a terminal error with the partial
read followed directly by HandlerDone.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* caddyhttp: only skip re-setting an armed hard deadline
Skipping any deadline equal to the one already armed also skipped idle
deadlines on Windows, where two calls to time.Now in the same clock tick
return the same time, so successive idle deadlines can be equal. That
broke tests counting deadline calls there. Only skip HardDeadline, the
case from review where the idle deadline is capped to it; other
deadlines are always set.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* caddyhttp: restore deadlines without trusting cached transport state
Always set the requested read and write deadlines because another handler can change them through its own response controller. Remove the armedAt shortcut and cover intervening deadline changes for reads, writes, ReadFrom, flushes, and HandlerDone.
AI-assisted with OpenAI Codex (GPT-6); tested under human direction.
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Since #7017, every *fs.PathError from stat becomes fs.ErrInvalid, and
that includes ENOTDIR. So a request like /index.php/foo got 400 instead
of 404, and pass_thru no longer fell through to the next handler.
Map ENOTDIR to fs.ErrNotExist before that check. Other path errors, such
as NUL bytes, overly long names, or transient I/O failures, keep their
current handling.
* reverseproxy: Optionally detach stream (websockets) from config lifecycle
* lint
* Improved logging facilities
* Adjustments from Weidi's review
* record bytes read and written for response writers unless detached
* simplify streaming handling
* correctly close detached streams
* make handleUpgradeTunnel a standalone func
* fix tests
* fix tests
* change the log level if hijacking without writing a status code first
* only clean up connections when stopped
* Move type and const down to the bottom
* Rename to tunnelTracker, reflow some comments
* Add note about capturing h
* Rename to stream_detached
* More comments
* reverseproxy: Preserve half-close behavior after rebase
Keep master half-close propagation and its unit coverage in the refactored tunnel lifecycle. Adapt response finalization tests to the selected upstream argument.
AI-assisted-by: Codex (GPT-6)
* reverseproxy: Fix detached stream cleanup and recorder chains
Snapshot the tracker registry before closing connections, and unregister empty trackers outside their mutex. Mark trackers stopped at cleanup entry so upstream removal closes detached streams even with stream_close_delay, while attached streams retain their delay.
Detach every supported response writer in the middleware chain so inner recorders do not retain state or count detached traffic. Add regressions for all four failures and an access-log/encode/intercept reload integration test.
AI-assisted-by: Codex (GPT-6)
---------
Co-authored-by: WeidiDeng <weidi_deng@icloud.com>
* timeouts: take precedence over server-wide and enclosing idle timeouts
The timeouts handler wrapped the request body and response writer again,
on top of the server's own idle timeout wrappers. The wrapper closest to
the connection sets the deadline last, so the server-wide read_body_idle
and write_idle (on by default since 2.11.6) silently overrode the
route's settings. The same happened between nested timeouts handlers,
where the outer, less specific one won.
The server now makes its idle timeout wrappers available through the
request context, and the timeouts handler overrides their settings while
it runs, restoring them when it returns. It only wraps when there's
nothing to override, and then registers its own wrappers so that nested
handlers override those. The server-wide read_timeout and write_timeout
still cap the whole transfer. A handler that doesn't set
max_write_chunk keeps the server-wide one.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* timeouts: re-arm deadlines on override and suspend outer min-rate window
From review:
- On HTTP/2, a read or write deadline is a timer that fires even with
no read or write in flight. A deadline armed before a timeouts handler
started kept the outer timeout and could fail the body (or reset the
stream) inside a longer inner scope before the next read reset it.
Override and restore now move an armed deadline to the settings that
apply.
- Time spent under an override counted against the outer MinRate
window, so the restored deadline could already have expired. The
outer window is now suspended while overridden: its start moves
forward by the time spent overridden, and bytes transferred in the
meantime don't count towards it.
Adds an HTTP/2 read-before-inner-scope regression test and nested
minimum-rate coverage.
Written with Claude Code (Claude Opus 5.5); the human submitter is
responsible for reviewing it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
CertMagic assembles an event's data before emit() can look at OnEvent,
because Go evaluates arguments first, so tls_get_certificate costs a map
and a ClientHello copy on every handshake even when nothing will read
them. caddyserver/certmagic#406 added Config.ShouldEmitFunc so an
embedder can answer that question first.
Export App.ShouldEmit and pass it along with OnEvent. The predicate has
to account for every observer, not just subscribed handlers: Emit writes
a debug line for each event, so an event is observable whenever debug
logging is on. Emit now asks the same method it exports, so the two
cannot drift.
Benchmarks are included; measurements are in the pull request.
With the Go 1.26 floor, golangci-lint's modernize analyzer (errorsastype)
flags the three errors.As calls that look for RequestBodyLimitError, which
turned the Lint workflow red on master after #8056. Replace them with the
generic errors.AsType; behavior is unchanged.
Assistance disclosure: this change was made by an AI agent (Claude Sonnet
5.5, model ID claude-sonnet-5-5) acting as Matthew Holt's agent and under
his approval. Build, go vet and the caddyhttp and caddyhttp/templates tests
pass locally; golangci-lint itself was not run locally.
Co-authored-by: Claude Code <noreply@anthropic.com>
Track pending HTTP server shutdowns across configuration reloads so
termination waits for responses still served by an older configuration.
Bound that wait by the terminating app's grace period and retain each
reload's grace context until its servers finish shutting down.
Cover HTTP/1.1, HTTP/2, and finite termination grace periods with
regression tests.
Provision previously only checked for a single opening brace at the
start of a destination, so values like {result}suffix, {result, and
{result}} passed validation. strings.Trim(dest, "{}") then silently
stored a wrong placeholder name (e.g. result}suffix), making the
intended value unavailable.
Require the destination to be exactly one placeholder: a single
opening brace at the start, a single closing brace at the end, and a
non-empty name.
Fixes#8073
The basic auth provider expanded account usernames and passwords with
repl.ReplaceAll, which blanks any {...} the replacer does not recognize
Credentials are config data, so a username like alice-{bogus} was stored
as alice- and a password like ab{cd}ef was quietly rewritten into
different valid base64, turning a config mistake into a working
password the admin never set
Use ReplaceKnown, matching the respond headers fix in #8014 and the
header handler fix in #4880, so unknown braces survive and real
placeholders like {env.*} still expand
The set body is user-supplied and commonly JSON, but it was expanded with
ReplaceAll, which blanks any {...} the replacer does not recognize, so a
plain JSON body was sent empty and a body mixing JSON with a real
placeholder was mangled
Use ReplaceKnown, matching the respond body, so unrecognized braces are
left intact and real placeholders still expand. Regression test pins both
directions
Co-authored-by: hktitof <hktitof@users.noreply.github.com>
* caddyhttp: fix randString sameCase dictionary to match its docs
The doc comment says randString excludes confusing characters like
I, l, 1, 0, O. When sameCase is true, uppercase letters and the
characters l and o should be excluded. But the sameCase dictionary
still contained '0'. Drop it.
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* Update modules/caddyhttp/errors.go
Signed-off-by: Matt Holt <mholt@users.noreply.github.com>
* elaborate the doc, and expand tests
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* Update modules/caddyhttp/errors_randstring_test.go
Co-authored-by: Zen Dodd <mail@steadytao.com>
Signed-off-by: Mohammed Al Sahaf <mohammed@caffeinatedwonders.com>
---------
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
Signed-off-by: Matt Holt <mholt@users.noreply.github.com>
Signed-off-by: Mohammed Al Sahaf <mohammed@caffeinatedwonders.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Zen Dodd <mail@steadytao.com>
when a request exceeds the request_body max_size limit, the request_body handler wraps the http.MaxBytesError into a caddyhttp.HandlerError carrying status 413, but the {http.request.body} and {http.request.body_base64} placeholders ran io.Copy with the error ignored, so they silently returned the truncated prefix as though it were the complete body. docs promise a 413 for reads past max_size, and silently truncating is a bad failure mode in templates and vars_regexp where the body value drives decisions. see #7691 and the narrow follow-up prescribed when #7692 was closed
reading the body now returns a dedicated RequestBodyLimitError marker instead of a generic HandlerError, and only when the read failure is actually the max_size limit. the consumers (template placeholder function, vars and vars_regexp matchers) recognize exactly that marker and wrap it in a status carrying handler error so the oversized request fails with 413, while every unrelated error value keeps its old behavior: templates still render it as text and the vars matchers still match on its error text. the surfaced error is stripped of its generated id and stack trace so a placeholder stringified into a response body cannot leak the call stack
negative regressions prove an unrelated HandlerError in templates, vars and vars_regexp does not start controlling request handling, plus integration tests for the template and vars_regexp 413 paths
ai assisted (GLM agent) under Abdel's direction and local verification
Signed-off-by: Abdel <hktitof@gmail.com>
* Move websocket header normalization later
A regression since #6621, these headers can be rewritten by cloneHeaders
and by .Add from user and transport ops. Moving this closer to where
it's round-tripped feels like the nicest solution.
I've put the harness I used to reproduce the issue at
https://codeberg.org/nfreya/caddy-websocket-headers-repro
* Integration test for normalized websocket headers
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
A request whose body has no length of its own — Transfer-Encoding: chunked, or
HTTP/2 and HTTP/3 requests sent without the header — cannot be forwarded over
FastCGI until it has been buffered in full, because CGI/1.1 requires
CONTENT_LENGTH and php-fpm hangs when it is absent or wrong and the body is not
empty. When request_buffers is too small to hold the whole body, the length
stays unknown and the request is refused with 411.
The refusal said none of that. RoundTrip passes r.ContentLength to Post, which
is -1 for such a request, so the operator got a bare "411 Length Required" with
either no error at all or strconv's "invalid syntax" on a value they never
wrote. On #7386 that sent two people looking for the fault in their backend.
The 411 now carries the reason and the remedy, and the unknown-length case is
told apart from a genuinely malformed value. ParseUint becomes ParseInt plus an
explicit negative check, which is strictly tighter: values above MaxInt64 used
to be accepted and are unreachable anyway, since CONTENT_LENGTH is always
FormatInt of an int64 by the time Do sees it.
No status code changes. Every request that was refused before is still refused;
raising request_buffers is still what makes a large chunked body work.
Tests: the reachable path through Post, the unusable CONTENT_LENGTH values Do
itself guards against, an integration test driving a chunked body through a
fastcgi reverse_proxy over a unix socket at each side of the buffer boundary
(including request_buffers -1, which succeeds at any size and is the remedy),
and a boundary test recording that a body exactly the size of the buffer counts
as partial.
That last one is deliberately not changed here. Telling "exactly the limit"
apart from "more to come" costs a read that a paused stream may never answer,
and bufferedBody also backs response_buffers: measured against a body that
delivers exactly the limit and stops, the current code hands the buffered
prefix on immediately, while peeking one byte first delivers nothing at all.
Co-authored-by: Aditya <205600203+Rohilalala@users.noreply.github.com>
Co-authored-by: Zen Dodd <mail@steadytao.com>
* reverseproxy: build TLS client config once so HTTP/3 keeps tls_trust_pool
reuse the config built for the HTTP/1.1+2 transport and give the HTTP/3
transport a clone of it to honour the tls_trust_pool
Signed-off-by: Timo Stark <tippexs91@googlemail.com>
Co-authored-by: Faiyaz Rahman <faiyazrahman03@gmail.com>
* Adding issue reference
---------
Signed-off-by: Timo Stark <tippexs91@googlemail.com>
Co-authored-by: Faiyaz Rahman <faiyazrahman03@gmail.com>
* reverseproxy: propagate TCP half-close on upgraded streams
handleUpgradeResponse starts one copy goroutine per direction but returns as
soon as the first one reports a result, and returning runs two unconditional
deferred closes (the client connection and the backend connection). A client
that ends its send direction with a TCP half-close therefore causes the whole
tunnel to be torn down: the backend connection is reset mid-conversation and
whatever it was about to send is never delivered.
Treat a clean EOF as the end of one direction rather than the end of the
tunnel. When a copy finishes without error, propagate the half-close to the
destination with CloseWrite where the connection supports it, and keep waiting
for the other direction so pending bytes can still drain. Real copy errors, the
stream timeout and request cancellation/shutdown still tear down immediately.
If the destination does not support CloseWrite, the peer cannot be told that
the direction ended, so the previous behavior is kept and the tunnel is torn
down rather than held open until the stream timeout.
This is the same class that was fixed upstream in net/http/httputil
(https://go.dev/issue/35892); Caddy has its own upgraded-stream copier, so the
standard library fix does not apply here.
Fixes#8026
* reverseproxy: follow net/http/httputil shape for the half-close
Replace the copyResult struct and the closeWriter/closeWrite helpers with
upstream's plain error channel and errCopyDone sentinel, propagating
CloseWrite inside the copier, as requested in review.
Behaviour is unchanged. The wait is repeated twice inside the existing
select because handleUpgradeResponse also selects on the stream timeout
and sizes the channel at 2 so both goroutines can exit when it fires
(#7418), which net/http/httputil has no equivalent of.
* reverseproxy: explain the repeated wait next to it
Move the reasoning for why the wait is repeated inside the select, rather
than written as upstream's two-line form, into a comment beside the loop
so a future reader does not have to find the review discussion.
* reverseproxy: add an upgraded-stream half-close integration test
Drive a real upgraded stream through the handler and check that closing one
direction is propagated to the other side instead of tearing the tunnel down.
Mirrors net/http/httputil's TestReverseProxyWebSocketHalfTCP, which covers the
same class upstream (https://go.dev/issue/35892): a backend that hijacks and
hands back a *net.TCPConn, the handler in front of it, and a client that dials
it directly, then the four close-read / close-write combinations.
Against the unfixed handler the two half-close cases fail - the surviving
direction never delivers its pending bytes - while the two close-read cases
pass, so the test isolates exactly the reported behaviour.
* reverseproxy: give the half-close test reads a deadline
Without one, a regression blocks in ReadFull until the package timeout
instead of failing the test, which is the flakiness an end-to-end test of
this kind is usually accused of. Ten seconds is far above any real latency
here and turns a hang into a clear failure.
* reverseproxy: hijack the test backend via http.NewResponseController
Matches how the rest of the tree reaches Hijack (responsewriter.go,
caddyauth.go, streaming.go itself) and keeps working if the writer is
wrapped, where a direct http.Hijacker assertion would not.
The respond directive expanded header field names and values with
repl.ReplaceAll, which blanks any {...} the replacer does not recognize
Header values are config data, often JSON or carrying literal braces, so a
value like {key:value} was sent empty and a-{b}-c came out a--c
Use ReplaceKnown, matching the header handler fix in #4880 (same class as
#4418) and the respond body expansion a few lines below, so unknown braces
survive and real placeholders still expand
Opening the bbolt database only reported timeout, which sounds like a
broken config. Catch bolt.ErrTimeout with errors.Is and say another
process holds the lock, and that caddy reload is the right command when
Caddy is already running. The error stays fatal.
The active health check body is user-supplied and commonly JSON, but it
was expanded with ReplaceAll, which blanks any `{...}` the replacer does
not recognize. Only globals are available here, so a JSON
health_request_body was sent as an empty body.
This is not limited to bodies without placeholders. A body that does use
one is mangled too: `{"token":"{env.SOME_VAR}"}` is sent as `"}`, since
the opening `{"token":"` is consumed as an unrecognized placeholder
before the real one is reached.
Use ReplaceKnown, matching the header values a few lines below, so
unrecognized braces are left intact and real placeholders still expand.
Only the body is changed. The Host header on the same path keeps
ReplaceAll deliberately: Go's HTTP server blanks a Host containing
braces, so leaving `{...}` intact there would send a Host the upstream
discards, where blanking it lets Go fall back to the URL host.
App.Emit does a fair amount of work before it can discover that no handler
is bound: it derives three loggers, one of which formats the event's UUID
even when debug logging is off, and registers a replacer callback. Only
then does it reach "shortcut if event not bound at all".
Some events are emitted on every TLS handshake -- CertMagic emits
tls_get_certificate as the first statement of GetCertificateWithContext --
so on a server with no events configuration that work runs per handshake
and is discarded every time.
Return early when neither the event's name nor the catch-all is bound and
debug logging is off, which are exactly the conditions under which nothing
can observe the event. caddy.NewEvent still runs, so the returned Event is
unchanged for callers.
Benchmarks are included; measurements are in the pull request.
* caddytls: synchronize storage cleaner with TLS.Stop via context and WaitGroup
* Unify the storage cleanup synchronization to both tls and ech.
* Cannot embed the sync.WaitGroup directly as the TLS struct is copied.
* caddytls: propagate cancellable context to ECH rotation and add sync tests
Pass cancellable context to ECH key rotation so in-flight storage locks
and operations unblock when TLS.Stop is invoked. Add comprehensive tests
verifying TLS.Stop cleanly unblocks and waits for storage cleaner and
ECH workers.
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
Jay Hemnani