mirror of
https://github.com/syncthing/syncthing.git
synced 2025-12-23 22:18:14 -05:00
a259a009c8
The benchstat tool allows custom grouping when comparing with what it calls "sub-name configuration keys": https://pkg.go.dev/golang.org/x/perf@v0.0.0-20250813145418-2f7363a06fe1/cmd/benchstat#hdr-Configuring_comparisons That's quite useful for these benchmarks, as we basically have two independent configs: The type of benchmark and the size. Real example usage for the prepared named statements PR (results are rubbish for unrelated reasons): ``` $ benchstat -row ".name /n" bench-main.out bench-prepared.out goos: linux goarch: amd64 pkg: github.com/syncthing/syncthing/internal/db/sqlite cpu: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz │ bench-main-20250823_014059.out │ bench-prepared-20250823_022849.out │ │ sec/op │ sec/op vs base │ Update Insert100Loc 248.5m ± 8% ¹ 157.7m ± 7% ¹ -36.54% (p=0.000 n=50) Update RepBlocks100 253.7m ± 4% ¹ 163.6m ± 7% ¹ -35.49% (p=0.000 n=50) Update RepSame100 130.42m ± 3% ¹ 60.26m ± 2% ¹ -53.80% (p=0.000 n=50) Update Insert100Rem 38.54m ± 5% ¹ 21.94m ± 1% ¹ -43.07% (p=0.000 n=50) Update GetGlobal100 10.897m ± 4% ¹ 4.231m ± 1% ¹ -61.17% (p=0.000 n=50) Update LocalSequenced 7.560m ± 5% ¹ 3.124m ± 2% ¹ -58.68% (p=0.000 n=50) Update GetDeviceSequenceLoc 17.554µ ± 6% ¹ 8.400µ ± 1% ¹ -52.15% (n=50) Update GetDeviceSequenceRem 17.727µ ± 4% ¹ 8.237µ ± 2% ¹ -53.54% (p=0.000 n=50) Update RemoteNeed 4.147 ± 77% ¹ 1.903 ± 78% ¹ -54.11% (p=0.000 n=50) Update LocalNeed100Largest 21.516m ± 22% ¹ 9.312m ± 47% ¹ -56.72% (p=0.000 n=50) geomean 15.35m 7.486m -51.22% ¹ benchmarks vary in .fullname ```