Package cobs implements Consistent Overhead Byte Stuffing (COBS),
a technique for reliable packet framing over serial byte streams.
This has future utility for storing a sequence of arbitrary log entries
on disk without needing to depend on intrinsic framing within
the log entries themselves (e.g., JSON or CBOR).
While more complicated, COBS is superior to offset-based framing
mechanisms as the null byte can be trivially used to demarcate
the boundaries of a frame. This makes COBS more resistant
against bit-corruption where a single corrupted offset
can make everything else in the file unreadable.
COBS makes it possible to resynchronize framing after a
corrupted section by simply searching for the next null.
Performance:
Benchmark/EncodeForward/Zeros-32 16341 76312 ns/op 13740.68 MB/s 0 B/op 0 allocs/op
Benchmark/EncodeReverse/Zeros-32 6326 188261 ns/op 5569.79 MB/s 0 B/op 0 allocs/op
Benchmark/DecodeForward/Zeros-32 16461 72140 ns/op 14535.28 MB/s 0 B/op 0 allocs/op
Benchmark/EncodeForward/NonZeros-32 41797 29155 ns/op 35965.56 MB/s 0 B/op 0 allocs/op
Benchmark/EncodeReverse/NonZeros-32 4792 248788 ns/op 4214.74 MB/s 0 B/op 0 allocs/op
Benchmark/DecodeForward/NonZeros-32 35790 34584 ns/op 30319.92 MB/s 0 B/op 0 allocs/op
Benchmark/EncodeForward/Random-32 23042 53727 ns/op 19516.64 MB/s 0 B/op 0 allocs/op
Benchmark/EncodeReverse/Random-32 3164 374590 ns/op 2799.26 MB/s 0 B/op 0 allocs/op
Benchmark/DecodeForward/Random-32 27241 58506 ns/op 17922.41 MB/s 0 B/op 0 allocs/op
EncodeReverse performance is notably slower than EncodeForward
because modern CPU architectures are not as optimized for
reading from memory in reverse.
However, reverse encoding is necessary if appending into
a dst buffer that is identical to the src buffer.
In such a case, the CPU performance hit is worth the benefit
of avoiding an intermediate allocation.
Speeds of GB/s is still plenty fast enough and
magnitudes faster than JSON or CBOR encoding.
Updates #17242
Updates tailscale/corp#21363
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
