mirror of
https://github.com/tailscale/tailscale.git
synced 2026-09-20 02:28:38 -04:00
jonathan/missing-exit-node
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 James Tucker
|
6608b9a387 |
tsweb/compserve, client/web: add zstd, remove brotli for precompressed assets
Adds tsweb/compserve: content negotiation for precompressed static variants, with a transcode-to-identity fallback for clients that do not accept an encoding (including when the raw file is absent), and CompressWriter, which live-compresses dynamic responses with zstd in its fastest mode, streamed incrementally with no buffering. Negotiation is q-value and wildcard aware (gzip;q=0 previously matched gzip). client/web serves its prebuilt embedded assets through compserve, replacing brotli with zstd; the embedded FS is wrapped in tsweb/vcstime for conditional-request mod times. tsweb/compserve/gzip.go keeps transitional serving of gzip variants from pre-zstd file systems (such as the currently published web-client-prebuilt module): passthrough to gzip-accepting clients, transcoded to identity otherwise; it becomes inert once a zstd-only module is published. util/zstdframe gains pooled GetDecoder and GetStreamingEncoder (concurrency=1). util/precompress is now a build-time tool, generating zstd variants only. cmd/tsconnect and cmd/build-webclient consume the new precompress/compserve split. tsweb.AcceptsEncoding and tsweb/tswebutil are removed; negotiation lives in compserve and the deprecated shim had no callers. go.mod bumps web-client-prebuilt. Also fixes a transcoding bug where http.ServeContent's size probe via the promoted zstd.Decoder.WriteTo could report a zero length, serving empty bodies. Updates tailscale/corp#20099 Signed-off-by: James Tucker <james@tailscale.com> |
||
|
Brad Fitzpatrick
|
3bf0149b29 |
util/zstdframe, util/testenv: don't pool coders within synctest bubbles
The zstd Encoder and Decoder types use channels internally, created
when the coder is constructed. A coder constructed by a goroutine
inside a testing/synctest bubble therefore owns bubble-associated
channels, and if it lands in zstdframe's process-wide pools and is
later reused outside that bubble, the Go runtime kills the process:
fatal error: receive on synctest channel from outside bubble
This has been crashing test binaries that mix synctest-based tests
with regular tests exercising zstd compression in parallel, taking
out every other test in the package with it.
Add testenv.InSynctestBubble and use it in zstdframe to construct a
fresh coder per call within a bubble instead of using the pools.
Pooling behavior outside of bubbles (including in benchmarks) is
unchanged.
As of Go 1.26 there is no public API to query bubble membership, so
InSynctestBubble looks for the "synctest bubble N" annotation that
the runtime renders in the current goroutine's runtime.Stack header.
That annotation is not covered by the Go compatibility promise, so
tests fail loudly (in util/testenv directly, and in util/zstdframe by
reintroducing the pooled-coder crash) if a future Go release changes
it. The check costs ~2us and runs only in test binaries, detected by
an uncached flag.Lookup("test.v") rather than testenv.InTest: this
path is reachable from package init functions (before testing has
registered its flags), where InTest would permanently latch a false
result into its cache, breaking later InTest and AssertInTest calls.
Fixes tailscale/corp#45861
Change-Id: I7d89e2d0de51e30098ceda25c12d27918acc46e8
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
|
||
|
Will Norris
|
3ec5be3f51 |
all: remove AUTHORS file and references to it
This file was never truly necessary and has never actually been used in the history of Tailscale's open source releases. A Brief History of AUTHORS files --- The AUTHORS file was a pattern developed at Google, originally for Chromium, then adopted by Go and a bunch of other projects. The problem was that Chromium originally had a copyright line only recognizing Google as the copyright holder. Because Google (and most open source projects) do not require copyright assignemnt for contributions, each contributor maintains their copyright. Some large corporate contributors then tried to add their own name to the copyright line in the LICENSE file or in file headers. This quickly becomes unwieldy, and puts a tremendous burden on anyone building on top of Chromium, since the license requires that they keep all copyright lines intact. The compromise was to create an AUTHORS file that would list all of the copyright holders. The LICENSE file and source file headers would then include that list by reference, listing the copyright holder as "The Chromium Authors". This also become cumbersome to simply keep the file up to date with a high rate of new contributors. Plus it's not always obvious who the copyright holder is. Sometimes it is the individual making the contribution, but many times it may be their employer. There is no way for the proejct maintainer to know. Eventually, Google changed their policy to no longer recommend trying to keep the AUTHORS file up to date proactively, and instead to only add to it when requested: https://opensource.google/docs/releasing/authors. They are also clear that: > Adding contributors to the AUTHORS file is entirely within the > project's discretion and has no implications for copyright ownership. It was primarily added to appease a small number of large contributors that insisted that they be recognized as copyright holders (which was entirely their right to do). But it's not truly necessary, and not even the most accurate way of identifying contributors and/or copyright holders. In practice, we've never added anyone to our AUTHORS file. It only lists Tailscale, so it's not really serving any purpose. It also causes confusion because Tailscalars put the "Tailscale Inc & AUTHORS" header in other open source repos which don't actually have an AUTHORS file, so it's ambiguous what that means. Instead, we just acknowledge that the contributors to Tailscale (whoever they are) are copyright holders for their individual contributions. We also have the benefit of using the DCO (developercertificate.org) which provides some additional certification of their right to make the contribution. The source file changes were purely mechanical with: git ls-files | xargs sed -i -e 's/\(Tailscale Inc &\) AUTHORS/\1 contributors/g' Updates #cleanup Change-Id: Ia101a4a3005adb9118051b3416f5a64a4a45987d Signed-off-by: Will Norris <will@tailscale.com> |
||
|
Joe Tsai
|
1a38d2a3b4 |
util/zstdframe: support specifying a MaxWindowSize (#11595)
Specifying a smaller window size during compression provides a knob to tweak the tradeoff between memory usage and the compression ratio. Updates tailscale/corp#18514 Signed-off-by: Joe Tsai <joetsai@digital-static.net> |
||
|
Joe Tsai
|
d4bfe34ba7 |
util/zstdframe: add package for stateless zstd compression (#11481)
The Go zstd package is not friendly for stateless zstd compression. Passing around multiple zstd.Encoder just for stateless compression is a waste of memory since the memory is never freed and seldom used if no compression operations are happening. For performance, we pool the relevant Encoder/Decoder with the specific options set. Functionally, this package is a wrapper over the Go zstd package with a more ergonomic API for stateless operations. This package can be used to cleanup various pre-existing zstd.Encoder pools or one-off handlers spread throughout our codebases. Performance: BenchmarkEncode/Best 1690 610926 ns/op 25.78 MB/s 1 B/op 0 allocs/op zstd_test.go:137: memory: 50.336 MiB zstd_test.go:138: ratio: 3.269x BenchmarkEncode/Better 10000 100939 ns/op 156.04 MB/s 0 B/op 0 allocs/op zstd_test.go:137: memory: 20.399 MiB zstd_test.go:138: ratio: 3.131x BenchmarkEncode/Default 15775 74976 ns/op 210.08 MB/s 105 B/op 0 allocs/op zstd_test.go:137: memory: 1.586 MiB zstd_test.go:138: ratio: 3.064x BenchmarkEncode/Fastest 23222 53977 ns/op 291.81 MB/s 26 B/op 0 allocs/op zstd_test.go:137: memory: 599.458 KiB zstd_test.go:138: ratio: 2.898x BenchmarkEncode/FastestLowMemory 23361 50789 ns/op 310.13 MB/s 15 B/op 0 allocs/op zstd_test.go:137: memory: 334.458 KiB zstd_test.go:138: ratio: 2.898x BenchmarkEncode/FastestNoChecksum 23086 50253 ns/op 313.44 MB/s 26 B/op 0 allocs/op zstd_test.go:137: memory: 599.458 KiB zstd_test.go:138: ratio: 2.900x BenchmarkDecode/Checksum 70794 17082 ns/op 300.96 MB/s 4 B/op 0 allocs/op zstd_test.go:163: memory: 316.438 KiB BenchmarkDecode/NoChecksum 74935 15990 ns/op 321.51 MB/s 4 B/op 0 allocs/op zstd_test.go:163: memory: 316.438 KiB BenchmarkDecode/LowMemory 71043 16739 ns/op 307.13 MB/s 0 B/op 0 allocs/op zstd_test.go:163: memory: 79.347 KiB We can see that the options are taking effect where compression ratio improves with higher levels and compression speed diminishes. We can also see that LowMemory takes effect where the pooled coder object references less memory than other cases. We can see that the pooling is taking effect as there are 0 amortized allocations. Additional performance: BenchmarkEncodeParallel/zstd-24 1857 619264 ns/op 1796 B/op 49 allocs/op BenchmarkEncodeParallel/zstdframe-24 1954 532023 ns/op 4293 B/op 49 allocs/op BenchmarkDecodeParallel/zstd-24 5288 197281 ns/op 2516 B/op 49 allocs/op BenchmarkDecodeParallel/zstdframe-24 6441 196254 ns/op 2513 B/op 49 allocs/op In concurrent usage, handling the pooling in this package has a marginal benefit over the zstd package, which relies on a Go channel as the pooling mechanism. In particular, coders can be freed by the GC when not in use. Coders can be shared throughout the program if they use this package instead of multiple independent pools doing the same thing. The allocations are unrelated to pooling as they're caused by the spawning of goroutines. Updates #cleanup Updates tailscale/corp#18514 Updates tailscale/corp#17653 Updates tailscale/corp#18005 Signed-off-by: Joe Tsai <joetsai@digital-static.net> |