mirror of
https://github.com/tailscale/tailscale.git
synced 2026-09-13 06:19:11 -04:00
Joe Tsai
9f4fe8b5f2
This adds a new ring buffer implementation that aims to replace logtail.Buffer and the on-disk implementation in filch.Filch. There are several problems with filch.Filch: * Filching stderr should not be done at the buffer layer. This makes structured representation within the buffer difficult as arbitrary stderr data may unexpectedly appear, which hinders attempts at more structured data. * Log messages are assumed to be discreet lines rather than arbitrary bytes. This makes it harder to switch the structured representation (e.g., using CBOR instead). * Data that appears asynchronously through stderr never triggers a wake-up within logtail. Consequently logs may never be uploaded. * Relatedly, there is no mechanism for notifying that data has newly arrived in the buffer. * There is no two-stage exfiltration. The TryReadLine method may or may not persist the fact that the data was read. It arbitrarily depends on whether we cross a magical file boundary in the dual-file approach. A failed upload followed by a restart results in dropped logs. A successful upload followed by a restart results in duplicated logs. The new Buffer interface and VolatileBuffer implementation are a step in the direction to resolving these problems. * In the future, filching will output to a separate pipe that we explicitly process the data for, before putting it into the log buffer. By processing the data, we can protect against stderr garbage being inserted into the buffer unexpectedly breaking any structure. * The Buffer.Peek and Buffer.DiscardUntil methods provide a way to exfiltrate in a two-step manner. When uploading, we peek at a chunk of data to upload. When successful, we discard the data, ensuring that the buffer knows not to provide that data again. The Len method can be used to suggest to the logging service the amount of back pressure that exists. Updates tailscale/corp#21363 Signed-off-by: Joe Tsai <joetsai@digital-static.net>