updates tailscale/corp#33007
When a selected exit node can't carry internet traffic due to a misconfiguration (wrong ID, node
deleted, routes removed, etc), blackhole default routes are installed and all internet traffic is dropped.
That is the correct behavior -- better to drop than leak to the local network -- but it was entirely silent:
the stable-ID lookup in nodeBackend.updateRouteManagerPrefs misses without a log line, and Status
leaves ExitNodeStatus nil. The user sees a healthy Tailscale with no internet. If an admin fat-fingers the
exit node name in an IT policy, for example, it's easy to break every node with zero feedback.
This adds an exit-node-unavailable health warning covering every way the selection can fail to carry
traffic (short of reachability which is a separate concern), reported via exitnodehealth.ArgExitNodeReason.
The warning names the exit node, caching its display name while it is still a peer so the name survives
its departure, and falls back to the stable ID or IP. When the selection is mandated by the ExitNodeID
or ExitNodeIP policy settings, the message tells the user to contact their network administrator instead
of suggesting they pick another exit node.
To test this, set a forced exit node policy with some random ip or node id. The warning has a 5 second
threshold. It should clear as soon as you change to a proper exit node.
Signed-off-by: Jonathan Nobels <jonathan@tailscale.com>
Every packet the server relayed allocated a fresh []byte for its
payload in recvPacket or recvForwardPacket and dropped it once the
destination's sendLoop had written it. On one busy server, this was
observed allocating about 160 MB/sec of short-lived garbage, and GC
plus malloc were about 5% of the process CPU profile.
Instead, take payload buffers from a size-classed sync.Pool on the
Server, with power-of-two classes from 1 KiB up to derp.MaxPacketSize,
and return them once the packet has been written, forwarded, or
dropped. sync.Pool holds nothing per connection and is trimmed by the
GC, so idle clients pin no memory; only packets actually in flight
hold a buffer. A compile-time assertion ties the largest size class to
derp.MaxPacketSize, and the get and put helpers panic on sizes outside
the pool's classes rather than indexing past it.
Because the memory is now reused, PacketForwarder implementations must
not retain the payload after ForwardPacket returns. Make that explicit
in the signature: the payload is passed as a new derp.LoanedBytes
value, which exposes only Len, WriteTo, and Clone, so an implementation
has to copy to keep it. derp.Client and derphttp.Client, the real
implementations, already wrote it out synchronously; the test-only
channelFwd now clones.
BenchmarkSendRecv shows one fewer allocation per relayed packet and,
for 1000-byte packets, B/op down from 1278 to 263. ns/op on the
loopback benchmarks is dominated by syscalls and is unchanged within
noise.
Updates #21064
Change-Id: Ie40c82388ddb5d22f75fa828749b53fcaba9adde
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
We banned use of multierr in various dep tests, so this makes the
situation more obvious if someone stumbles across it.
Change-Id: I17e80880e57e5005fcaea864a365ba264411e802
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
sclient.debugLogf and Server.debugLogf check a debug flag before
logging, but Go evaluates and boxes their arguments before the call.
The per-packet call sites in run, handleFrameSendPacket,
handleFrameForwardPacket, sendPkt, recordDrop, and sendPacket's
deferred stats func were therefore calling key.NodePublic.ShortString
and boxing frame headers on every relayed packet, all for messages
that were then discarded.
On one busy server's heap profile, those discarded arguments were
about half of all objects allocated by the process. Guard each hot
call site with the debug flag so nothing is built unless it will be
logged, and document that requirement on both debugLogf methods.
While here, give the sender cardinality sketch its key bytes from a
stack array rather than an AppendTo(nil) allocation per packet.
BenchmarkSendRecv drops from 10 or 11 allocations per relayed packet
to 3, and BenchmarkConcurrentStreams from 11 to 4.
Updates #21064
Change-Id: Ibb7c4fbff546b6f1b40ce0a21d41ae0976705941
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
testwrapper ran a separate, sequential "go test" invocation for each
package pattern on its command line. That is fine for a single "./..."
argument but not for callers that pass an explicit package list: CI
jobs in the corp repo passing ~200 packages ran ~200 serial go test
processes with no cross-package parallelism and a fixed set of
never-cacheable lookups per process, and spent several times longer
on process startup, package loading, cache lookups, and serial test
binary links than on running tests. See tailscale/corp#48453 for the
details.
Locally, on 203 packages with a fully warm build and test cache, so
measuring only the per-invocation overhead:
old (203 go test processes): 26.4s
new (1 go test process): 3.6s (7.3x faster)
Our own Windows CI job hits the same path: its "sharded:N/M" mode
expands to an explicit list of that shard's packages via listpkgs, so
each shard ran one go test process per package, and Windows process
startup is slower still. Each shard now runs as one invocation.
Fixestailscale/corp#48453
Updates tailscale/corp#47035
Change-Id: I3f796ff1724af40f93be9f918a7ddfde3bb45a91
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Commit 6608b9a38 removed tsweb.AcceptsEncoding, saying it had no
callers, but it has many callers in the tailscale.io repo. Restore the
function and its test unchanged so those builds work again.
Updates #12170
Updates tailscale/corp#48447 (broken by this)
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I7c3e2a9f5b1d4e8a6c0f2b3d9e1a7c5f4b8d2e6a
Moving to v2 because v1 references repositories that have been
deleted from GitHub, breaking GOPROXY=direct.
The lint config (.golangci.yml) was already in v2 format; this updates
the tool dependency used by 'make lint' to golangci-lint/v2 v2.13.2,
drops the now-obsolete blank import in internal/tooldeps in favor of a
Go 'tool' directive, and bumps the CI workflow binary to match.
Updates #cleanup
Signed-off-by: James Tucker <jftucker@gmail.com>
Adds tsweb/compserve: content negotiation for precompressed static
variants, with a transcode-to-identity fallback for clients that do not
accept an encoding (including when the raw file is absent), and
CompressWriter, which live-compresses dynamic responses with zstd in its
fastest mode, streamed incrementally with no buffering. Negotiation is
q-value and wildcard aware (gzip;q=0 previously matched gzip).
client/web serves its prebuilt embedded assets through compserve,
replacing brotli with zstd; the embedded FS is wrapped in
tsweb/vcstime for conditional-request mod times. tsweb/compserve/gzip.go
keeps transitional serving of gzip variants from pre-zstd file systems
(such as the currently published web-client-prebuilt module):
passthrough to gzip-accepting clients, transcoded to identity otherwise;
it becomes inert once a zstd-only module is published.
util/zstdframe gains pooled GetDecoder and GetStreamingEncoder
(concurrency=1). util/precompress is now a build-time tool, generating
zstd variants only. cmd/tsconnect and cmd/build-webclient consume the
new precompress/compserve split. tsweb.AcceptsEncoding and
tsweb/tswebutil are removed; negotiation lives in compserve and the
deprecated shim had no callers. go.mod bumps web-client-prebuilt.
Also fixes a transcoding bug where http.ServeContent's size probe via
the promoted zstd.Decoder.WriteTo could report a zero length, serving
empty bodies.
Updates tailscale/corp#20099
Signed-off-by: James Tucker <james@tailscale.com>
The update/install localapi handler had no PermitWrite check, so any
local user who could reach the localapi socket could make the root
daemon self-update and restart itself. This has been the case since the
endpoint was added in November 2023.
Gate the handler behind PermitWrite so that only root or the operator
user can trigger a self-update, matching the other mutating handlers.
Updates tailscale/corp#48187
Change-Id: Iadfef939f5dab684652cd220e77de63b54bfca2f
Reported-by: Ben Carman <benthecarman@live.com>
Signed-off-by: Patrick O'Doherty <patrick@tailscale.com>
Similar to #16462, when we are acting as a TCP proxy, we need to pass
through half-closes correctly since clients and servers will sometimes
close one direction of the connection and still rely on the other
direction working.
Fixes#20883.
Signed-off-by: Naman Sood <mail@nsood.in>
We have multiple situations where we have a `net.Conn` representing a
TCP connection for a proxy and we need access to the underlying
`CloseWrite()` and `CloseRead()` functions to properly pass through
half-closes (see #16462, #20883). Add an interface we can cast to in
order to get access to these functions.
Updates #20883.
Signed-off-by: Naman Sood <mail@nsood.in>
The Windows GitHub runners' provisioning daemon, provjobd.exe, opens a
handle to each freshly-written tailscaled.exe. tb.TempDir's own cleanup
retries for a fixed 2s and gives up, failing whichever test happens to tear
down while the handle is held.
Updates #21099
Signed-off-by: Yaruk Asghar <yaruk@tailscale.com>
`decode4` assigned `q.subofs` before validating it against the declared IP total length. A packet with an IHL past the end of the buffer was rejected but left `subofs` dangling there, so a later Transport call would panic.
This change only store `subofs` once validated. As defense in depth an additional bounds check is added in Transport.
Fuzzing was expanded and improved to get better coverage in `packet.go`.
Credit to @Dev-next-gen for finding and reporting.
Fixestailscale/corp#48322
Updates tailscale/corp#46608
Change-Id: I198d06b921add9b188daad79d4ca473aed1e3b66
Signed-off-by: Mike Jensen <mikej@tailscale.com>
This method has been deprecated for four months; replace all uses with
the Peers or NoPeers equivalents.
Updates #12542
Change-Id: I7b5800d0d92775839dbfb6021751a5dfacc53d2f
Signed-off-by: Alex Chan <alexc@tailscale.com>
TestNATPing called SetMasqueradeAddresses on the test control server
and then immediately read both nodes' status, expecting the new
masqueraded peer addresses to already be there. But the change reaches
the nodes asynchronously via their streaming map responses, so under
load the status check ran before the new map response arrived and the
test failed with "n1 sees n2 as 100.64.0.2; want 100.64.2.1" and the
like. This was the dominant failure mode on the flakes dashboard (11 of
the 18 most recent CI failures) and the only one found in a six hour
Antithesis run (run 30b4de27d89d8d7a651e7b43b3f1ec3f-61-9, 54 failures
in 16,599 runs).
Wait for each node's status to report the expected peer address
instead. Also retry the "tailscale ping" invocations, since a ping can
fail transiently right after a map response changes a peer's addresses
and before the engine is reconfigured; the second most common failure
mode was pings exiting with status 1. Failed pings now include the CLI
output in the error rather than a bare exit status.
Locally, flakestress (32 workers) reproduced the failure in 29 of 210
runs before this change (13.8%) and in 0 of 1,173 runs after.
The remaining Windows-only failure mode on the dashboard, TempDir
cleanup failing because tailscaled.exe is still open, is a
harness-wide issue that affects every integration test and is not
specific to this test.
Fixes#12169
Updates tailscale/corp#47865
Change-Id: I7c3e2b8a5d914f0e6a2b1c9d8e7f6a5b4c3d2e1f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This is the output of the new misc/bumpdeps tool (#21325) run with
--exclude-newer-than-days=7, which asks proxy.golang.org for the newest
version of every direct dependency, ignoring releases younger than a
week in favor of the newest older one, and runs a single go get.
gvisor tracks its "go" branch, wireguard-go its "tailscale" branch,
and golang-x-crypto its "main" branch (the proxy's @latest for it is
a stray v0.91.0 tag from 2024 that predates our acme fork changes).
Indirect deps only moved as far as MVS pulled them.
The week-long cooldown held back gvisor, the gokrazy modules,
chromedp/cdproto, and hashicorp/raft-boltdb/v2, whose only newer
versions are days old; they'll come along next time.
Several upstream changes needed small fixes: nfpm's PrepareForPackager
takes a modification time now (a zero time keeps the old behavior of
using the source file's mtime), esbuild's ServeOptions.Port became an
int while ServeResult.Host became a Hosts slice, client-go's
EventRecorder.Eventf is now recognized by vet as a printf wrapper (so
the k8s-operator calls that passed a preformatted message switch to
Event), google/nftables v0.3.0 reads back the kernel's
NF_NAT_RANGE_PROTO_SPECIFIED flag into a new expr.NAT.Specified field
(so the port map DNAT rule now sets it too or findRule never matches
the rule it just added), and staticcheck v0.8.1 knows encoding/json/v2's
embed tag option, so the two SA5008 suppressions for it are gone.
Two tests assumed old library behavior. client-go's fake clientset now
replays existing objects when a watch starts, as a real apiserver does,
so the k8s-proxy config test must tolerate the loader ignoring that
no-op event before the real reload arrives. fyne.io/systray moved its
dbusmenu object path and answers the first GetLayout with depth 1, so
the systray test now finds the menu via the item's Menu property and
polls until the submenu entries appear.
Then make tidy, make updatedeps, and make kube-generate-all (the
controller-gen bump to v0.22.0 changes doc strings, stops listing
top-level metadata as required, and crd-ref-docs now marks optional
fields).
Updates #8043
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I3f9a2c6e8b1d4705a9e2c7b8d1f4e6a0c2b5d8e3
embed.FS stores and serves no ModTime: files report a zero ModTime,
which disables HTTP caching semantics when served with net/http: no
Last-Modified header is sent and If-Modified-Since requests are never
answered with a 304.
vcstime.FS wraps an fs.FS (typically an embed.FS) and reports the
vcs.time commit time from the binary's build info as the modification
time of files that have no real timestamp of their own, enabling
http.FS to provide working cache headers for clean builds. Files that
already have a real timestamp are passed through untouched, and
binaries built without VCS stamping degrade to the file system's own
behavior.
Updates tailscale/corp#48172
Signed-off-by: James Tucker <james@tailscale.com>
The bumpdep workflow ran every go get with GOPROXY=direct, copying the
corp update-oss workflow. For a module like github.com/gokrazy/kernel.amd64
that means cloning a repo full of kernel image blobs, which hung the
first real run of the workflow indefinitely.
Fetch generic modules through the default GOPROXY (proxy.golang.org),
which serves branch names like @main just fine. Keep GOPROXY=direct only
for wireguard-go, which is our own small repo where seeing a just-pushed
commit matters more than the proxy's cache lag. Also give the job a
45 minute timeout so a future hang doesn't hold a runner for six hours.
Updates tailscale/corp#48312
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I3e7d1a9c5b2f48e06d7a1c9e4b3f2a8d6c5e7f10
This is the OSS analog of the corp repo's update-oss workflow. It's
manually dispatched with a comma-separated list of dependencies and the
URL of the issue motivating the bump, updates each dependency, runs
"make tidy" and "make updatedeps", and opens a pull request from the
tailscale-code-updater app assigned to whoever triggered it.
Three names are special-cased: "go" runs ./pull-toolchain.sh,
"wireguard-go" tracks github.com/tailscale/wireguard-go@tailscale, and
"gvisor" tracks gvisor.dev/gvisor@go. Anything else is treated as a Go
module path and bumped to @latest (or to an explicit path@version). The
issue URL becomes the "Updates" line of the generated commit message.
Updates tailscale/corp#48312
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I6b2f9c1e8d4a7350b2e9f1c4d8a6e2b7f3c5d9a1
Bumping deps one "go get foo@latest" at a time is slow with the ~500
modules in go.mod. This tool parses go.mod with x/mod/modfile, asks
proxy.golang.org concurrently for the newest version of each direct
dependency (about a second for the whole file), and then runs a single
"go get" with the modules that actually have something newer. Arguments
narrow it to modules whose path contains one of the given substrings,
so "bumpdeps gvisor" does the obvious thing.
gvisor.dev/gvisor, github.com/tailscale/wireguard-go, and
github.com/tailscale/golang-x-crypto follow branches ("go", "tailscale",
and "main" respectively) rather than tags, so those are resolved via the
proxy's @v/<branch>.info endpoint instead of @latest.
The --exclude-newer-than-days flag is a cooldown in the sense of
https://nesbitt.io/2026/03/04/package-managers-need-to-cool-down.html:
releases younger than that are ignored in favor of the newest one old
enough, so a compromised upstream has to go unnoticed that long before
we'd pick it up. It defaults to off. Branch-tracked modules have nothing
older to fall back to, so they're held until their head has aged.
The tool never downgrades, either by semver (the proxy's @latest can be
older than a pseudo-version we're already on) or by commit time (forks
can carry stray tags that sort above their real development branch, as
golang-x-crypto's v0.91.0 from 2024 does). It skips replaced modules and
modules whose latest release declares a different module path, since go
get rejects those and they need their import paths changed by hand.
Indirect deps are left to MVS by default; -indirect bumps them too, but
that tends to break the build when their importers haven't caught up.
Updates #8043
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I7c2e9a41d5f0b83e6a1c4d9f2b7e8a0c3d5f6b1e
Add Node.StableTailnetID for control to send the current tailnet's
stable ID on the self node. Expose it as CurrentTailnet.StableID in
LocalAPI status and `tailscale status --json`, and display it in
`tailscale whoami`.
Bump CurrentCapabilityVersion to 148.
Updates #14375
RELNOTE: Show the current tailnet's stable ID in status JSON and whoami.
Change-Id: I0525ff735de8113c8d124045a94e00c19a5a02e2
Signed-off-by: Andrew Dunham <andrew@tailscale.com>
Requesting a UDP mapping is the right thing to do since an "all
protocols" (and "all ports") mapping would be akin to requesting
to be a DMZ on that network.
Updates #cleanup
Change-Id: Icc83d4fe14dbec0eb844b093d2d92756d6c05228
Signed-off-by: Francois Marier <francois@tailscale.com>
The profile switch log lines passed cp.UserProfile().LoginName without
calling it, so they printed "%!q(func() string=0x7ff7141cd060)" in place
of the login name.
Updates #cleanup
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I4e6a8c0b2d5f7a9c1e3b5d7f9a1c3e5b7d9f1a3c
tsp is a protocol library and should not care about the system's routing
table, interfaces, or how to reach DERP for bootstrap DNS. Those are
tailscaled concerns. Without an explicit DNSCache, ts2021 built a resolver
whose LookupIPFallback did bootstrap DNS over DERP whenever the first dial
to control failed, and that path also crashed on tsp's nil netmon.Monitor.
Provide a plain dnscache.Resolver with no fallback so a failed dial is
just a failed dial.
Updates tailscale/corp#47865
Change-Id: If62ddb90139d5590ca3e0e3f54235bbbc0d8aec2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
MakeLookupFunc documents its netMon parameter as optional, but
bootstrapDNSMap passed it straight to netns.NewDialer, which has panicked
on nil since 3672f29a4. Any caller that omitted the monitor and then had a
first dial fail (which is when dnscache consults LookupIPFallback) crashed
with "netns.NewDialer called with nil netMon". control/tsp clients hit this
under Antithesis fault injection.
Use a plain net.Dialer when no monitor is provided and add a regression
test.
Updates tailscale/corp#47865
Change-Id: Ie8255033602dd1f8243c1ced8f453fb7ab9be74d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The TLS probe fetched and parsed the leaf certificate's CRL on every
run. That is fine when the CRL is small, but some CAs publish CRLs of
several megabytes: the one for the AWS ACM R2M04 intermediate is about
2.5MB, which at the default 15s interval is a continuous 170kB/s per
probed node.
Cache parsed CRLs by distribution point URL and reuse each for up to
an hour, or until its NextUpdate if that comes first. An hour is the
HTTP max-age Let's Encrypt serves on its root CRL. A CRL is cached
only after its signature verifies, every use still re-verifies it
against the probing leaf's issuer (the cache is keyed by URL alone),
and a CRL without a NextUpdate is never cached since it declares no
validity window.
Concurrent probes fetch through singleflight.DoChanContext to avoid
re-fetching a single CRL, with each waiter keeping its own deadline. A
caller that missed the cache re-checks it inside the singleflight
closure, since singleflight dedupes only calls that overlap.
Leaf certificates whose issuer is missing from the presented chain now
fail before any fetch. Previously the probe downloaded the CRL and then
panicked in CheckSignatureFrom, which the prober recovered and recorded
as a probe failure.
Also update the TLS probe's doc comments, which said OCSP where the
code checks a CRL.
Fixes#21310
Signed-off-by: Thomas Desrosiers <git@hive.pw>
The test control server always listened on 127.0.0.1:9911, which is
useless for a node in a VM on the same machine. --addr picks the listen
address; the DERP and STUN servers follow it.
--ssh-policy loads a tailcfg.SSHPolicy from a JSON or HuJSON file and
sends it to every node, which also grants them the SSH node capability so
that "tailscale up --ssh" is accepted. That is what testcontrol.Server
already supported for tests; this exposes it for manual testing of the
Tailscale SSH server.
Updates tailscale/corp#47865
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I2b7c4e9f1a3d5c6e8b0f2a4d6c8e1b3f5a7d9c0e
The main server file mixed portable session handling with Unix details:
sending SIGHUP to end a session, decoding exec.ExitError, the euid check
for whether the process can switch users, agent forwarding's chown of the
socket, and reading /etc/ssh host keys as root. Those now sit behind small
functions (hangupProcess, waitProcess, canSwitchToLocalUser,
handleSSHAgentForwarding, systemHostKeyFile, isRootUser) in the new
process_unix.go, along with the incubator's forwarded-environment pipe
helpers, and the session's *exec.Cmd moves into an embedded osSessionState
struct defined there, so that the portable code no longer refers to the
process representation at all.
user.go keeps only the portable userMeta and userLookup; the login shell
and default PATH logic moves to user_unix.go. The SFTP child entrypoint and
its stdio adapter, which incubator.go and incubator_plan9.go each had a
copy of, move to sftp.go. The c2n usernames handler gains a hook for
platforms that list users some other way than /etc/passwd.
The agent socket's uid and gid are parsed as 31-bit rather than 32-bit
unsigned values so that the conversion to int for os.Chown cannot
overflow on 32-bit platforms, which is the pattern CodeQL flags.
Updates #cleanup
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I4c7e2b9a0d3f5e1c8b6a4d2f0e9c7b5a3d1f8e6c
dnsQueryForName builds the query for GET /dns-query?q=<name>, the peerAPI's
interactive debug mode. The name comes from the peer's query string and the
only thing done to it is appending a trailing dot, but the query is built
with dnsmessage.MustNewName, which panics as soon as the name is longer than
255 bytes.
The panic happens before the query reaches the resolver, so the nameAllowed
filter never gets a say and only sourceAllowed has to be true: any of the
user's own untagged devices, and any peer an extension hook lets through,
such as a client using this node as an exit node with DNS proxying allowed.
http.Server recovers it, so tailscaled survives, but the peer's connection is
dropped instead of answered and the node logs a panic trace per request.
Just under the limit the name was mishandled too: a 255-byte name is accepted
by NewName but rejected by Question when it packs the name, and that error
was dropped, so Finish returned a well-formed query with no question in it
which was then handed to the resolver.
Have dnsQueryForName use NewName, check the error from Question, and return
the error from Finish. handleDNSQuery turns a name it cannot build a query
for into the 400 it already uses for the other malformed-request cases.
Fixes#21307
Change-Id: I7c1a4b7e4f9a1d2c3b5e8f0a6d4c2b9e1f3a7d50
Signed-off-by: leoca <leo.camus23@gmail.com>
Update gVisor to include its fix for RACK loss detection with coarse
monotonic clocks. Configure netstack with the 500 microsecond clock
resolution used on Windows so RACK accounts for timestamp quantization.
Remove the TCP recovery override that disabled RACK, enabling gVisor's
default RACK behavior on all platforms.
Switch netstack to cubic congestion control. The int overflow in CUBIC
sender cwnd arithmetic that required pinning reno has since been reworked
upstream into float arithmetic with RFC 9438 target clamping.
Align the natlab vnet stack with netstack: enable cubic, and drop the
now-redundant explicit SACK and receive-buffer moderation sets, both of
which are gVisor defaults.
Fixes#9707
Signed-off-by: James Tucker <jftucker@gmail.com>
Payload guards a truncated packet by comparing both length and dataofs
against len(b), but the slice it returns is b[dataofs:length], so what
actually has to hold is dataofs <= length. Those are independent:
length comes from the IPv4 total length header field, while dataofs is
derived from the sub-protocol header, and decode4 never checks that the
declared total length covers the transport header.
A 28-byte IPv4/UDP packet declaring a total length of 20 decodes to
length=20, dataofs=28, len(b)=28, and Payload then evaluates b[28:20].
ICMPv4 and TCP reach the same state with 24- and 40-byte packets.
I found this by fuzzing Decode and then calling the accessors on the
result. I did not find a caller that can be driven into it from the
network: wireguard-go truncates decrypted packets to the declared IP
length, so on the inbound path dataofs > length implies dataofs >
len(b) and the existing guard already catches it.
Add a FuzzParsedPayload target that decodes and then calls Payload,
seeded with valid IPv4/IPv6 packets and with invalid ones, including
the three short total length packets above, and build it in
fuzz/oss-fuzz.sh.
Fixes#21231
Change-Id: Ie5d2100464b79750626b1bfefbe4020c4a42ca91
Signed-off-by: leoca <leo.camus23@gmail.com>
The nonce used in PCP identifies a particular port mapping. It needs
to be generated randomly for the initial mapping, and then saved so
that it can be used to renew or delete a mapping later. This is how
the router checks that we're authorized to change existing mappings.
If we receive a response for a different nonce, we should ignore it
since it's not meant for us (or the router is misbehaving).
Fixes#21127
Change-Id: Ic7874d376a74791807953f2f559c8a14a8eb75e9
Signed-off-by: Francois Marier <francois@tailscale.com>
The tsweb DebugHandler already links to expvar, Prometheus varz, and
pprof, but the only runtime visibility beyond pprof was the handful of
runtime.MemStats fields that varz special-cases. The runtime/metrics
package has far more (GC CPU classes, scheduler latencies, stop-the-world
pause histograms, heap breakdowns, and so on) and is the runtime's
preferred, cheaper interface.
Add a /debug/runtime-metrics handler. By default it serves only an index
of metric names, kinds, and descriptions, which are static, so viewing
the page does not read any values. The index is an HTML table for
browsers (Accept: text/html) and plain text otherwise; format=html or
format=text overrides the sniffing.
Values are read only on request, always as JSON:
- /debug/runtime-metrics/gc/heap/allocs:bytes returns that metric's
bare value, handy for scripts and curl.
- /debug/runtime-metrics?name=NAME (repeatable) returns a JSON object
keyed by metric name. A trailing * matches by prefix, so name=/gc/*
returns the GC metrics and name=* returns everything.
Histograms are objects with counts and buckets arrays; infinite bucket
boundaries, which JSON cannot represent, are the strings "-Inf" and
"+Inf".
In the HTML index, exact metric names link to the bare value form, and
each run of metrics sharing a directory is headed by a row linking every
ancestor directory to its wildcard query (/gc/*, /gc/heap/*, ...).
Responses inherit the debug handler's nosniff, framing, and CSP headers,
and application/json is not a script MIME type, so the values cannot be
pulled in cross-origin as a script by a malicious page.
Like the pprof handlers, this is excluded from js/wasm builds to keep
them small.
Updates #21300
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I7c2e9f4a1b8d3e6f0a5c9b2d4e7f1a3c6b8d0e2f
conn25 node attribute type
This node attribute configuration knob, when enabled, was bypassing two
peercap (tailscale.com/cap/conn25/<app_name>) checks while they were
under development and testing:
- PeerAPI DoH on the connector verifying the peercap includes the app in
query parameter of the request.
- PeerAPI ConnectorTransitIPRequest verifying the peercap includes the
app in the request.
With this change, we always perform those checks, and delete bypass
knob.
Updates tailscale/corp#40076
Updates tailscale/corp#48041
Signed-off-by: Michael Ben-Ami <mzb@tailscale.com>
Each connected client held two buffered channels of 32 pkts each (its
regular and disco send queues), about 5.5 KB per client for the
lifetime of the connection, even though almost all clients are idle at
any given moment.
Replace the channels with pktQueue, a mutex-guarded ring buffer whose
backing array comes from a per-server sync.Pool on first enqueue and
goes back when the queue drains empty, so an idle client holds no
queue memory at all. Enqueuers wake the client's sendLoop through a
one-slot channel.
Updates #21064
Change-Id: I2d7e8f4a1b6c3950e2a7d4b8f1c5e3a9d6b0c2e4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The forwarder's upstream UDP socket is unconnected, so any host that
could send a datagram to its ephemeral port reached the single
ReadFromUDPAddrPort call that decided the query: a datagram with the
right 16-bit transaction ID was returned to the client as the
resolver's answer no matter what address it came from, and a datagram
with the wrong ID failed the query outright, handing it to the TCP
fallback (or to nothing at all, with TCP retries disabled).
sendUDP now keeps reading until a datagram arrives that could be a
reply to the query it sent: one from the resolver's address and port
carrying the transaction ID of the request. Other datagrams are
dropped and counted, so neither a spoofed reply nor a single stray
datagram can answer or end the query. This is the same source check
the kernel applies to a connected socket; the socket stays unconnected
because its ListenPacket path is what binds the query to a link
(IP_BOUND_IF on macOS, ForwardLinkSelector elsewhere). The read loop
still ends when the query context expires and closeOnCtxDone closes
the conn, as before.
On Windows an oversized datagram is reported as a truncation error
without a source address, so there the transaction ID remains the only
check, as before.
Thanks to Ben Carman for the report!
Updates tailscale/corp#48187
Reported-by: Ben Carman
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: Ibabbec5c82739b66caa6fb943faa376c39adbf64
TestOpenresolvDNS pins the case where Tailscale owns the only resolvconf
snippet, so on its own it would also pass if tailscaled never read an OS
base config at all. Register a snippet the way a DHCP client would, then
toggle accept-dns to force a reapply, and assert its nameserver becomes
quad-100's upstream.
SetAcceptDNS is the toggle: tailscaled has no reason to re-read the OS
config on its own, and this avoids rebooting the guest to change what the
OS resolver looks like mid-test.
Updates #20825
Updates tailscale/corp#44793
Signed-off-by: Brendan Creane <bcreane@gmail.com>
TestUserMetricsByteCounters is currently our top flake at http://flakes/
The test waited for a direct path between its two localhost nodes by
polling s1's status until CurAddr was set, then asserted that the
transferred bytes appeared in the path="direct_ipv4" counters. Both
steps were flaky:
- CurAddr is only reported while magicsock has a confirmed, currently
trusted UDP address for the peer, which depends on a heartbeat ping
and CallMeMaybe cycle that can lag arbitrarily under CI load. In the
CI failure logs the transfer itself went direct and the byte counter
assertions passed; the only failure was the polling helper's
t.Error.
- On a machine where localhost has both IPv4 and IPv6, magicsock can
pick ::1 (it prefers IPv6 on latency ties), so the transfer lands in
the direct_ipv6 counters instead.
- If no direct path is established yet, the transfer goes over DERP, whose
wire-byte total (relay framing, plus TCP retransmissions inside the
tunnel under load) can exceed any fixed tolerance; one flakestress
run counted 33% more than the payload sent.
Drop the direct-path wait and the hardcoded path label, and assert the
one invariant the test actually cares about: at least the number of
bytes sent must be counted across all path counters combined. There is
deliberately no upper bound, since the counters legitimately include
overhead, retransmissions, and the duplicate copies magicsock sends on
several paths while a direct path is being confirmed.
Verified with flakestress: the old test failed within 7 runs, while the
fixed test passed 33,521 runs with no failures.
See
http://flakes/analyze-test?name=tailscale.com%2ftsnet.TestUserMetricsByteCounters
Updates #deflake
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I4d8a9bfb9549ea9d8d33d3a3b55b4d03f9b9be78
On Windows, I could run tailscaled in the terminal over SSH only when
built with ts_omit_syspolicy, but that's kinda a sad workaround. This
is the alternative.
syspolicy registers a Registry policy store for the current user
whenever tailscaled isn't running as LocalSystem, and the first read
of that store takes the Group Policy read lock via
EnterCriticalPolicySection(FALSE). That call fails with
ERROR_ACCESS_DENIED when the user has no interactive logon session,
which is the case for a tailscaled.exe started over SSH (or WinRM,
psexec, a non-interactive scheduled task). The store's Lock method
returned the error, the reader failed to load, and LocalBackend
treated that as fatal:
ipnlocal.NewLocalBackend: syspolicy: LocalBackend failed to register
policy change callback: failed to get a store reader: Access is denied.
The GP lock is only there to keep reads of several settings consistent
while Group Policy is being applied; the store already documents that
reading the Registry without it is safe, and already skips it when
gp.ErrLockRestricted is returned during service start. Treat
ERROR_ACCESS_DENIED the same way: log it and read unlocked. Other errors
from the lock still propagate.
The optionalPolicyLock wrapper now holds the lockableCloser interface
instead of *gp.PolicyLock so the test can inject a failing lock.
Tested on a Windows Server 2022 VM: the test passes from an SSH session,
and a foreground tailscaled started from that session now runs and logs in
instead of exiting at startup.
Updates #21290
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I395f356730092299cf020a6787c56ab3b9ed6fe5
The ts_omit_<name> build tags omit a feature at build time; there has
been no way to do the same at runtime. Some users (either proactively
or in response to a security announcement) might like a way to disable
a feature that's linked-in in their binaries that they're not using.
Then a mitigation announcement can say "set this env var" without
asking users to rebuild or wait for a new release.
This adds env var TS_DISABLE_FEATURE, a comma-separated list of
feature names to disable, and the listed set is reported by the
debug-optional-features LocalAPI endpoint next to the registered set.
The legacy per-feature knobs such as TS_DISABLE_SSH_SERVER and
TS_DISABLE_TAILDROP keep working independently.
A disabled feature behaves as if it had not been linked: it is absent
from feature.IsRegistered, its hooks are unset, and its extensions and
handlers are not registered. Three pieces make that happen:
* feature.Register now returns bool, false when disabled, and
feature packages gate their registration init on it. It was added
to the feature packages that never called it (including taildrop
and ssh), which also completes the picture reported by
debug-optional-features. taildrop, routecheck, favorites, and
serviceclientprefs had registration split across several inits and
now register from one gated init.
* ipnext.RegisterExtension ignores a disabled feature's extension.
* feature.Hook.Set and feature.Hooks.Add walk the call stack and
silently skip when the calling package under feature/<name> is
disabled. This covers sub-packages such as
feature/captiveportal/netcheckhook, which cannot call Register
themselves without colliding with their parent, and future
packages whose authors forget the gate.
ssh/tailssh's registrations moved from its inits into tailssh.Register,
called from feature/ssh's gated init. The aws and kube state stores and
syspolicy's Windows store registration are gated too.
feature/register_disable_test.go runs this test binary as a child
process (it links condregister, as tailscaled does) with
TS_DISABLE_FEATURE set to every registered feature at once, and fails
if any of them register anyway, so a feature that ignores the variable
cannot land.
Updates #12614
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I720af6ccab844ae060a9dfd1539fee577fd483e3
Jonathan Nobels