derp/derpserver: drop per-client writer goroutine, start on demand

Browse Source

Each client connection ran two goroutines for its lifetime: the reader
in sclient.run and a sendLoop blocked in a select over its send
queues, pong, peer gone, mesh update, and keepalive channels. Almost
all clients are idle at any moment, so the second goroutine mostly
pinned memory: a 4 KiB stack, a g struct, a sudog per select case,
three channels, and a context and errgroup. At 100k idle connections
that was about 8 KB of a client's 22 KB RSS.

Instead, start up the sendLoop only as needed, letting the goroutine
go away otherwise, like Go 1.28-dev's http2 code
(golang/go@5c51011e82) with similar parking to
https://go.dev/cl/834084 but DERP's producers are all non-blocking, so
a kick bit replaces that http2 code's send count.

Measured with 100k idle TLS connections, server RSS per client went
from 22.3 KB to 14.5 KB (22.9 KB to 15.3 KB after each connection had
carried a packet), goroutines dropped from 2 to 1, and with no change
in BenchmarkSendRecv throughput or allocations and no change in the
time to do 100k serial round trips, each of which parks and wakes the
writer. (it's super cheap to start goroutines)

Updates #21064

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I7c3e9a51d4b8f2607a1e5c3d9f8b2a4e6c0d1f3b

...

This commit is contained in:

500 Internal Server Error

Gitea Version: 1.28.0+dev-477-g8b6ad49a5f