mirror of
https://github.com/tailscale/tailscale.git
synced 2026-09-22 03:25:16 -04:00
Brad Fitzpatrick
f87a1b1a82
Every packet the server relayed allocated a fresh []byte for its payload in recvPacket or recvForwardPacket and dropped it once the destination's sendLoop had written it. On one busy server, this was observed allocating about 160 MB/sec of short-lived garbage, and GC plus malloc were about 5% of the process CPU profile. Instead, take payload buffers from a size-classed sync.Pool on the Server, with power-of-two classes from 1 KiB up to derp.MaxPacketSize, and return them once the packet has been written, forwarded, or dropped. sync.Pool holds nothing per connection and is trimmed by the GC, so idle clients pin no memory; only packets actually in flight hold a buffer. A compile-time assertion ties the largest size class to derp.MaxPacketSize, and the get and put helpers panic on sizes outside the pool's classes rather than indexing past it. Because the memory is now reused, PacketForwarder implementations must not retain the payload after ForwardPacket returns. Make that explicit in the signature: the payload is passed as a new derp.LoanedBytes value, which exposes only Len, WriteTo, and Clone, so an implementation has to copy to keep it. derp.Client and derphttp.Client, the real implementations, already wrote it out synchronously; the test-only channelFwd now clones. BenchmarkSendRecv shows one fewer allocation per relayed packet and, for 1000-byte packets, B/op down from 1278 to 263. ns/op on the loopback benchmarks is dominated by syscalls and is unchanged within noise. Updates #21064 Change-Id: Ie40c82388ddb5d22f75fa828749b53fcaba9adde Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>