GET / in the client/web assets handler normalized to an empty fs
path. The empty name is invalid, and file systems wrapped in fs.Sub
reject it with fs.ErrInvalid instead of fs.ErrNotExist. The
index.html fallback only triggers on fs.ErrNotExist, so it never ran
and the request failed with a 500 "internal error". This broke the
corp smoke test that curls http://100.100.100.100/.
Normalize the empty path to index.html before serving. Also make
compserve.ServeFile return fs.ErrNotExist for invalid paths, per its
documented contract, instead of leaking fs.ErrInvalid.
Both layers have regression tests verified to fail without the fix.
Updates tailscale/corp#20099
Updates #12170
Signed-off-by: James Tucker <james@tailscale.com>
http.ResponseController.Flush prefers a FlushError method at each level
of the ResponseWriter wrapper chain and falls back to Flusher.Flush,
which can't report an error. loggingResponseWriter only had Flush, so
handlers using ResponseController to flush never saw the underlying
writer's flush errors, such as a stream reset by an expired write
deadline. Add FlushError that delegates via ResponseController so
errors from the wrapped writer propagate.
Updates tailscale/corp#48531
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I259c5de234435f632387261165fe15dcaf34ff45
The varz handler got its memstats_* metrics from the expvar package's
"memstats" func, which calls runtime.ReadMemStats and so stopped the
world on every Prometheus scrape. Keep the names but compute them from
runtime/metrics, and never call that func, even from
WritePrometheusExpvar.
While there, export the /tailscale/ metrics from our Go fork (stack
size histogram, stack copy counters, timer zombie counts and lifetime
histogram), which nothing could see before, plus a few upstream ones
with no MemStats equivalent: scheduling latency and GC pause
histograms, live heap, GC and total CPU seconds, thread count, and
mutex wait time. The last replaces derper's hand-rolled version.
Everything read is cheap and a scrape allocates nothing after the
first. Names use a go_runtime_ namespace rather than go_ so they can't
collide with the Prometheus Go client's collector in promvarz binaries.
The runtime's 162-bucket time histograms are reduced to one bucket per
factor of four from 256ns to 1s.
Updates #21300
Updates tailscale/go#189
Updates golang/go#75935
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I7e3c9a41f2b85d6e0c4a9b1d3f8e7c2a5b6d4e19
Commit 6608b9a38 removed tsweb.AcceptsEncoding, saying it had no
callers, but it has many callers in the tailscale.io repo. Restore the
function and its test unchanged so those builds work again.
Updates #12170
Updates tailscale/corp#48447 (broken by this)
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I7c3e2a9f5b1d4e8a6c0f2b3d9e1a7c5f4b8d2e6a
Adds tsweb/compserve: content negotiation for precompressed static
variants, with a transcode-to-identity fallback for clients that do not
accept an encoding (including when the raw file is absent), and
CompressWriter, which live-compresses dynamic responses with zstd in its
fastest mode, streamed incrementally with no buffering. Negotiation is
q-value and wildcard aware (gzip;q=0 previously matched gzip).
client/web serves its prebuilt embedded assets through compserve,
replacing brotli with zstd; the embedded FS is wrapped in
tsweb/vcstime for conditional-request mod times. tsweb/compserve/gzip.go
keeps transitional serving of gzip variants from pre-zstd file systems
(such as the currently published web-client-prebuilt module):
passthrough to gzip-accepting clients, transcoded to identity otherwise;
it becomes inert once a zstd-only module is published.
util/zstdframe gains pooled GetDecoder and GetStreamingEncoder
(concurrency=1). util/precompress is now a build-time tool, generating
zstd variants only. cmd/tsconnect and cmd/build-webclient consume the
new precompress/compserve split. tsweb.AcceptsEncoding and
tsweb/tswebutil are removed; negotiation lives in compserve and the
deprecated shim had no callers. go.mod bumps web-client-prebuilt.
Also fixes a transcoding bug where http.ServeContent's size probe via
the promoted zstd.Decoder.WriteTo could report a zero length, serving
empty bodies.
Updates tailscale/corp#20099
Signed-off-by: James Tucker <james@tailscale.com>
embed.FS stores and serves no ModTime: files report a zero ModTime,
which disables HTTP caching semantics when served with net/http: no
Last-Modified header is sent and If-Modified-Since requests are never
answered with a 304.
vcstime.FS wraps an fs.FS (typically an embed.FS) and reports the
vcs.time commit time from the binary's build info as the modification
time of files that have no real timestamp of their own, enabling
http.FS to provide working cache headers for clean builds. Files that
already have a real timestamp are passed through untouched, and
binaries built without VCS stamping degrade to the file system's own
behavior.
Updates tailscale/corp#48172
Signed-off-by: James Tucker <james@tailscale.com>
The tsweb DebugHandler already links to expvar, Prometheus varz, and
pprof, but the only runtime visibility beyond pprof was the handful of
runtime.MemStats fields that varz special-cases. The runtime/metrics
package has far more (GC CPU classes, scheduler latencies, stop-the-world
pause histograms, heap breakdowns, and so on) and is the runtime's
preferred, cheaper interface.
Add a /debug/runtime-metrics handler. By default it serves only an index
of metric names, kinds, and descriptions, which are static, so viewing
the page does not read any values. The index is an HTML table for
browsers (Accept: text/html) and plain text otherwise; format=html or
format=text overrides the sniffing.
Values are read only on request, always as JSON:
- /debug/runtime-metrics/gc/heap/allocs:bytes returns that metric's
bare value, handy for scripts and curl.
- /debug/runtime-metrics?name=NAME (repeatable) returns a JSON object
keyed by metric name. A trailing * matches by prefix, so name=/gc/*
returns the GC metrics and name=* returns everything.
Histograms are objects with counts and buckets arrays; infinite bucket
boundaries, which JSON cannot represent, are the strings "-Inf" and
"+Inf".
In the HTML index, exact metric names link to the bare value form, and
each run of metrics sharing a directory is headed by a row linking every
ancestor directory to its wildcard query (/gc/*, /gc/heap/*, ...).
Responses inherit the debug handler's nosniff, framing, and CSP headers,
and application/json is not a script MIME type, so the values cannot be
pulled in cross-origin as a script by a malicious page.
Like the pprof handlers, this is excluded from js/wasm builds to keep
them small.
Updates #21300
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I7c2e9f4a1b8d3e6f0a5c9b2d4e7f1a3c6b8d0e2f
To decide whether we can bump GOAMD64 for our production binaries, we
first need to know what x86-64 microarchitecture levels the fleet's
CPUs actually support. Add two gauges to the default metric set on
linux/amd64: goamd64_capable, the maximum GOAMD64 level the host CPU
supports per /proc/cpuinfo, and goamd64_compiled, the level the
running binary was built with per its build info. Both are computed
once at init and report 0 on error.
The per-level feature lists match the checks in Go's
runtime/asm_amd64.s.
Updates tailscale/corp#47099
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I3b4e5431ae1501f0b7d584c12d9af805e9ab6125
Export the machine's boot time (the btime line from Linux's
/proc/stat) as node_boot_time_seconds, named to match what
Prometheus's node exporter uses for the same value. Combined with
process_start_unix_time, this can be used to distinguish process
restarts from whole node restarts.
The value is parsed once per process lifetime, not per scrape, and
the metric is only published when a value is available, so non-Linux
systems don't export a bogus zero.
Updates tailscale/corp#44743
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I5f53186b97bb1482bd1a5387c0910b0ae26544ff
* ipn/localapi,ipnlocal,feature/acme,client/local: honour Retry-After on cert rate-limit
serveCert now responds with 429 + Retry-After when the underlying ACME
error is a rate limit, instead of a generic 500. client/local surfaces
this as a typed RateLimitedError with the parsed hint so callers can
back off intelligently.
Updates tailscale/corp#42164
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
* tsweb,feature/acme,ipn/localapi,ipnlocal: generalise cert error → HTTP mapping via tsweb.HTTPStatuser
Introduces a tsweb.HTTPStatuser interface, any error can implement
to describe its intended HTTP response (code, message, headers).
Moves CertRateLimitedError from ipnlocal to feature/acme where it's
constructed, and it now uses HTTPStatuser to return 429 + Retry-After.
serveCert now checks for tsweb.HTTPStatuser rather than the specific
error type, so it no longer needs to know about the ACME rate-limit
type.
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
---------
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
When running `tailscale netcheck`, the reported timestamp used to be
in UTC and formatted according to RFC 3339 with a `T` to separate the
date from the time:
sfllaw@h2co3:~$ tailscale netcheck | head -n3
Report:
* Time: 2026-06-01T21:12:32.252620138Z
This is machine-readable time leaking out to the user interface. Times
in normal commands are formatted for humans to read:
sfllaw@h2co3:~$ date
Mon 01 Jun 2026 02:39:14 PM PDT
sfllaw@h2co3:~$ journalctl -t tailscaled | tail -n1
Jun 01 14:35:21 h2co3 tailscaled[3328921]: wgengine: sending TSMP disco key advertisement to 100.90.144.102
sfllaw@h2co3:~$ timedatectl show
Timezone=America/Los_Angeles
LocalRTC=no
CanNTP=yes
NTP=yes
NTPSynchronized=yes
TimeUSec=Mon 2026-06-01 14:38:32 PDT
RTCTimeUSec=Mon 2026-06-01 14:38:32 PDT
sfllaw@h2co3:~$ uptime --since
2026-05-15 07:37:45
This PR makes the times printed by the CLI commands consistent:
- For `tailscale routecheck`, it now prints local time as
`2026-05-15 07:37:45-07:00`.
- For `netlogfmt`, it has always printed local time with a space,
but now includes the time zone.
- All machine-readable outputs continue to be standard RFC 3339 in
UTC, i.e. `--format=json`.
As part of a general cleanup, this PR also adds standard common
time.Format layouts as tstime constants.
Fixes#19928
Signed-off-by: Simon Law <sfllaw@tailscale.com>
Avery found a bunch of tests that fail with -count=2.
Updates tailscale/corp#40176 (tracks making our CI detect them)
Change-Id: Ie3e4398070dd92e4fe0146badddf1254749cca20
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Co-authored-by: Avery Pennarun <apenwarr@tailscale.com>
Add a new envknob that allows connections from trusted CIDR ranges
to access debug endpoints without Tailscale authentication. This is
useful for in-cluster scrapers like Prometheus that are not on a
tailnet, do not have static IP addresses and cannot use debug keys.
Fixes#19282
Signed-off-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Add a new vet analyzer that checks t.Run subtest names don't contain
characters requiring quoting when re-running via "go test -run". This
enforces the style guide rule: don't use spaces or punctuation in
subtest names.
The analyzer flags:
- Direct t.Run calls with string literal names containing spaces,
regex metacharacters, quotes, or other problematic characters
- Table-driven t.Run(tt.name, ...) calls where tt ranges over a
slice/map literal with bad name field values
Also fix all 978 existing violations across 81 test files, replacing
spaces with hyphens and shortening long sentence-like names to concise
hyphenated forms.
Updates #19242
Change-Id: Ib0ad96a111bd8e764582d1d4902fe2599454ab65
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I omitted a lot of the min/max modernizers because they didn't
result in more clear code.
Some of it's older "for x := range 123".
Also: errors.AsType, any, fmt.Appendf, etc.
Updates #18682
Change-Id: I83a451577f33877f962766a5b65ce86f7696471c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Currently the expvar exporter attempts to write expvar.String, which
breaks the Prometheus metric page.
Updates tailscale/corp#36552
Signed-off-by: Anton Tolchanov <anton@tailscale.com>
This file was never truly necessary and has never actually been used in
the history of Tailscale's open source releases.
A Brief History of AUTHORS files
---
The AUTHORS file was a pattern developed at Google, originally for
Chromium, then adopted by Go and a bunch of other projects. The problem
was that Chromium originally had a copyright line only recognizing
Google as the copyright holder. Because Google (and most open source
projects) do not require copyright assignemnt for contributions, each
contributor maintains their copyright. Some large corporate contributors
then tried to add their own name to the copyright line in the LICENSE
file or in file headers. This quickly becomes unwieldy, and puts a
tremendous burden on anyone building on top of Chromium, since the
license requires that they keep all copyright lines intact.
The compromise was to create an AUTHORS file that would list all of the
copyright holders. The LICENSE file and source file headers would then
include that list by reference, listing the copyright holder as "The
Chromium Authors".
This also become cumbersome to simply keep the file up to date with a
high rate of new contributors. Plus it's not always obvious who the
copyright holder is. Sometimes it is the individual making the
contribution, but many times it may be their employer. There is no way
for the proejct maintainer to know.
Eventually, Google changed their policy to no longer recommend trying to
keep the AUTHORS file up to date proactively, and instead to only add to
it when requested: https://opensource.google/docs/releasing/authors.
They are also clear that:
> Adding contributors to the AUTHORS file is entirely within the
> project's discretion and has no implications for copyright ownership.
It was primarily added to appease a small number of large contributors
that insisted that they be recognized as copyright holders (which was
entirely their right to do). But it's not truly necessary, and not even
the most accurate way of identifying contributors and/or copyright
holders.
In practice, we've never added anyone to our AUTHORS file. It only lists
Tailscale, so it's not really serving any purpose. It also causes
confusion because Tailscalars put the "Tailscale Inc & AUTHORS" header
in other open source repos which don't actually have an AUTHORS file, so
it's ambiguous what that means.
Instead, we just acknowledge that the contributors to Tailscale (whoever
they are) are copyright holders for their individual contributions. We
also have the benefit of using the DCO (developercertificate.org) which
provides some additional certification of their right to make the
contribution.
The source file changes were purely mechanical with:
git ls-files | xargs sed -i -e 's/\(Tailscale Inc &\) AUTHORS/\1 contributors/g'
Updates #cleanup
Change-Id: Ia101a4a3005adb9118051b3416f5a64a4a45987d
Signed-off-by: Will Norris <will@tailscale.com>
We were missing this metric, but it can be important for some workloads.
Varz memstats output allocation cost reduced from 30 allocs per
invocation to 1 alloc per invocation.
Updates tailscale/corp#28033
Signed-off-by: James Tucker <james@tailscale.com>
Co-authored-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Allows the use of tsweb without pulling in all of the heavy prometheus
client libraries, protobuf and so on.
Updates #15160
Signed-off-by: David Anderson <dave@tailscale.com>
Allow customizing the title on the debug index page. Also add methods
for registering http.HandlerFunc to make it a little easier on callers.
Updates tailscale/corp#27058
Change-Id: Ia101a4a3005adb9118051b3416f5a64a4a45987d
Signed-off-by: Will Norris <will@tailscale.com>
So we can locate them in logs more easily.
Updates tailscale/corp#24721
Change-Id: Ia766c75608050dde7edc99835979a6e9bb328df2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Allows the use of tsweb.LogHandler exclusively for callbacks describing the
handler HTTP requests.
Fixes#12837
Signed-off-by: Paul Scott <paul@tailscale.com>
With this change, the error handling and request logging are all done in defers
after calling inner.ServeHTTP. This ensures that any recovered values which we
want to re-panic with retain a useful stacktrace. However, we now only
re-panic from errorHandler when there's no outside logHandler. Which if you're
using StdHandler there always is. We prefer this to ensure that we are able to
write a 500 Internal Server Error to the client. If a panic hits http.Server
then the response is not sent back.
Updates #12784
Signed-off-by: Paul Scott <paul@tailscale.com>
StdHandler/retHandler would previously emit one log line for each request.
If there were multiple StdHandler in the chain, there would be one log line
per instance of retHandler.
With this change, only the outermost StdHandler/logHandler actually logs the
request or invokes OnStart or OnCompletion callbacks. The error-rendering part
of retHandler lives on in errorHandler, and errorHandler passes those errors up
the stack to logHandler through a callback that logHandler places in the
request.Context().
Updates tailscale/corp#19999
Signed-off-by: Paul Scott <paul@tailscale.com>
To complement the existing `onCompletion` callback, which is called
after request handler.
Updates tailscale/corp#17075
Signed-off-by: Anton Tolchanov <anton@tailscale.com>
This commit introduces a userspace program for managing an experimental
eBPF XDP STUN server program. derp/xdp contains the eBPF pseudo-C along
with a Go pkg for loading it and exporting its metrics.
cmd/xdpderper is a package main user of derp/xdp.
Updates tailscale/corp#20689
Signed-off-by: Jordan Whited <jordan@tailscale.com>
This change makes our access log record more consistent with the
new log/tslog package formatting of "time". Note that we can
change slog itself to call "time" "when" but we're chosing
to make this breaking change to be consistent with the std lib's
defaults.
Updates tailscale/corp#17071
Signed-off-by: Marwan Sulaiman <marwan@tailscale.com>
The inflight request tracker only starts recording a new bucket after
the first non-error request. Unfortunately, it's written in such a way
that ONLY successful requests are ever marked as being finished. Once a
bucket has had at least one successful request and begun to be tracked,
all subsequent error cases are never marked finished and always appear
as in-flight.
This change ensures that if a request is recorded has having been
started, we also mark it as finished at the end.
Updates tailscale/corp#19767
Signed-off-by: Will Norris <will@tailscale.com>
James Tucker