mirror of
https://github.com/tailscale/tailscale.git
synced 2026-09-22 03:25:16 -04:00
Brad Fitzpatrick
f87a1b1a82
Every packet the server relayed allocated a fresh []byte for its payload in recvPacket or recvForwardPacket and dropped it once the destination's sendLoop had written it. On one busy server, this was observed allocating about 160 MB/sec of short-lived garbage, and GC plus malloc were about 5% of the process CPU profile. Instead, take payload buffers from a size-classed sync.Pool on the Server, with power-of-two classes from 1 KiB up to derp.MaxPacketSize, and return them once the packet has been written, forwarded, or dropped. sync.Pool holds nothing per connection and is trimmed by the GC, so idle clients pin no memory; only packets actually in flight hold a buffer. A compile-time assertion ties the largest size class to derp.MaxPacketSize, and the get and put helpers panic on sizes outside the pool's classes rather than indexing past it. Because the memory is now reused, PacketForwarder implementations must not retain the payload after ForwardPacket returns. Make that explicit in the signature: the payload is passed as a new derp.LoanedBytes value, which exposes only Len, WriteTo, and Clone, so an implementation has to copy to keep it. derp.Client and derphttp.Client, the real implementations, already wrote it out synchronously; the test-only channelFwd now clones. BenchmarkSendRecv shows one fewer allocation per relayed packet and, for 1000-byte packets, B/op down from 1278 to 263. ns/op on the loopback benchmarks is dominated by syscalls and is unchanged within noise. Updates #21064 Change-Id: Ie40c82388ddb5d22f75fa828749b53fcaba9adde Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
34 lines
926 B
Go
34 lines
926 B
Go
// Copyright (c) Tailscale Inc & contributors
|
|
// SPDX-License-Identifier: BSD-3-Clause
|
|
|
|
package derp
|
|
|
|
import (
|
|
"bytes"
|
|
"io"
|
|
)
|
|
|
|
// LoanedBytes is a packet payload lent to a callee for the duration of
|
|
// a call. The lender reuses the memory once the call returns, so the
|
|
// callee must not retain it; Clone gives it a copy to keep.
|
|
//
|
|
// It deliberately has no accessor for the underlying slice.
|
|
type LoanedBytes struct {
|
|
bs []byte
|
|
}
|
|
|
|
// LoanBytes lends b for the duration of the call it's passed to.
|
|
func LoanBytes(b []byte) LoanedBytes { return LoanedBytes{bs: b} }
|
|
|
|
// Len returns the number of bytes.
|
|
func (b LoanedBytes) Len() int { return len(b.bs) }
|
|
|
|
// WriteTo writes the bytes to w.
|
|
func (b LoanedBytes) WriteTo(w io.Writer) (int64, error) {
|
|
n, err := w.Write(b.bs)
|
|
return int64(n), err
|
|
}
|
|
|
|
// Clone returns a copy of the bytes that the caller owns.
|
|
func (b LoanedBytes) Clone() []byte { return bytes.Clone(b.bs) }
|