Adds an opt-in, in-memory aggregator of recent connection-rejection
events (TSMP rejects received from peers, outbound TSMP rejects we emit
on ACL-blocked inbound flows, and pendopen timeouts) keyed by
(direction, proto, peer-address, reason). The aggregated data is exposed
over a new debug-rejects LocalAPI endpoint and a GET /debug/rejects c2n
endpoint, intended for future GUI/CLI consumption when diagnosing why a
connection failed.
Architecture:
- net/connreject holds the data types and a per-LocalBackend
Aggregator (LRU-bounded, default 256 entries on desktop / 32 on
mobile, per direction).
- feature/connreject is a self-registering ipnext.Extension that owns
one Aggregator per LocalBackend, installs note callbacks on the
tundev and engine, subscribes to OnSelfChange to flip the runtime
gate, and serves the LocalAPI/c2n endpoints.
- wgengine.Engine and *tstun.Wrapper each gain a SetConnRejectNote
setter; data-plane sites use a single atomic.Pointer load + nil
check, so the cost when no consumer is installed is one MOV.
Gating:
- Compile-time: ts_omit_connreject build tag (standard
feature/buildfeatures + condregister plumbing). Trims ~41 KB.
- Runtime: nodecap.ConnReject node attribute, off by default
at the control plane. May be removed once the feature is enabled
by default.
Updates CapabilityVersion to 146 (clients understand nodecap.ConnReject
and can serve GET /debug/rejects).
Adds Proto/Src/Dst accessors on flowtrack.Tuple (used by pendopen to
construct events without exposing the tuple's internals to the
aggregator).
Updates #1094
Updates #14802
Change-Id: I83e8f24a7e66fa2d158d128bd25fbe851134941b
Signed-off-by: James Tucker <james@tailscale.com>
Add a new modular dnsresolvecache feature that records every successful
DNS resolution from net/dnscache as a JSON file per hostname in
$statedir/dns-cache/, so a later boot with misconfigured DNS can still
find last-known-good IPs for critical hostnames like the control plane.
Files are rewritten only when their contents change, so a file's
modification time records when the answer last changed.
When regular DNS resolution fails, the disk cache is now consulted
before the DERP-based bootstrap DNS in net/dnsfallback. This is the
first step toward removing the DERP-based mechanism: new clientmetrics
(dnscache_disk_fallback_hit, dnscache_disk_fallback_miss,
dnscache_derp_fallback_ok, dnscache_derp_fallback_dial_ok) will tell us
when the DERP path no longer fires in the fleet and can be deleted.
The feature is linked into tailscaled by default (omittable with
ts_omit_dnsresolvecache) and is not included in tsnet.
Updates #21028
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I59228cbf68e3b48dfb1215cdd12bd8166ab14034
Android denies app UIDs both NETLINK_ROUTE (golang/go#40569, #2293)
and /proc/net, so net.Interfaces always fails and netmon.New errors
out before a standalone binary can do anything. The Android app solves
this from Java via netmon.RegisterInterfaceGetter, but raw binaries
run under Termux or a rooted shell have no Java to lean on. This is
the second half of #21129, following the androiddns feature.
I added a netmon fallback hook, consulted only when no interface
getter was registered and net.Interfaces failed, and a new androidbin
feature (ts_omit_androidbin) that implements it: report a single
synthetic interface whose v4 and v6 addresses come from asking the
kernel to route an outbound UDP socket, which sends no packets and is
permitted in the app sandbox. That's enough for magicsock to discover
local endpoints. The fallback also requires runtime evidence of
Android (GOOS=android, or /dev/__properties__ existing for GOOS=linux
binaries running under an Android kernel), so it's inert on regular
Linux.
The androidbin feature also blank imports androiddns, and fixes a
third gap I found while testing: GOOS=linux binaries on Android have
an empty CA root pool, because Go's unix root loader doesn't know
Android's /system/etc/security/cacerts (the GOOS=android loader
does), so all TLS verification fails. On Android it points
SSL_CERT_DIR there unless the user already set it, as Termux's
ca-certificates package does.
The feature is on by default in tailscaled builds on Linux and
Android via condregister, and deliberately not linked into tsnet by
default; tsnet apps and other programs opt in with a blank import of
tailscale.com/feature/androidbin.
I verified on an Android 13 emulator with SELinux enforcing, running
GOOS=linux static binaries under the app UID (run-as), where
net.Interfaces fails with the exact netlinkrib permission denial from
the issue: netmon.New succeeds with the synthetic interface, and a
tailcat binary importing this feature does DNS via dnsproxyd, fetches
its DERP map over TLS using the Android cert store, completes STUN,
selects a DERP region, and prints its address.
Updates #21129
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: If7dbcbb825ecd24bcf6d9d64b1e334dd00700d51
Previously the DERP handler served each connection for its lifetime
on its hijacked connection's net/http handler goroutine. That
goroutine's conn.serve stack frame kept the dead HTTP/1 server state
reachable for the whole DERP connection: the http.conn and its 4KB
bufio.Writer (hijack hands over c.bufw but conn.serve still references
it, so derpserver returning it to its flush pool never made it
collectable), the 4KB bufio.Reader, the upgrade *http.Request with its
parsed headers, and the request context chain. The goroutine also kept
the stack growth from the TLS handshake and HTTP request parsing.
Instead, hand the connection off to a new goroutine and return from
the handler (ala tailscale/corp@dc09e27aef), letting all the HTTP
upgrade state be collected. Give Accept a smaller 1KB frame reader,
draining and releasing the hijacked reader if it contains buffered
bytes from a fast-start client, and a nil bufio.Writer so writes go
through pooled buffers held only for the duration of a write instead
of a per-connection buffer.
Because the handler now returns at handoff time, cmd/derper's
gauge_derper_tls_active_version decrement can no longer be deferred
to handler return: intercept Hijack in the TLS metrics wrapper and,
for hijacked connections (DERP, its WebSocket flavor, and CONNECT),
decrement the gauge once when the hijacked connection closes,
restoring the gauge's connection-lifetime semantics. Teach
derpserver's TCP RTT stats to unwrap the close-hook conn so they
still find the underlying *net.TCPConn.
Also soften the UntypedHexString deprecation notices in types/key to
warnings: the untyped hex string format is the DERP wire protocol's
key encoding, so these call sites are legitimate and permanent, and
a Deprecated marker just makes them light up in editors and linters.
The cautionary text about the format's risks remains.
Measured with 100,000 idle TLS DERP connections on linux/amd64:
standing memory drops from 55.6KB to 32.6KB per connection (-41%).
Updates #21064
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: If05a0c6ea79134807e9e8872861db216
The PeerRelay resource hardcoded UDP port 41641 in the generated
Service, the tailscaled config, status.endpoints and the advertised
static endpoints. That breaks deployments where the load balancer
address or port is not what peers can actually reach, e.g. behind NAT.
Add spec.service.port to choose the UDP port, and spec.staticEndpoints
to advertise extra address:port pairs alongside the discovered load
balancer endpoints. A replica whose only endpoint is a static one
counts as addressed, and a static entry's port wins when it names an
address the load balancer already provides.
The e2e tests now supply static endpoints, which lets them assert the
PeerRelayReady condition goes True in kind, where no cloud controller
ever gives the LoadBalancer Services an address.
Fixes#20821
Change-Id: I463012cd447c81c2849fa653fa18eb82662aaf4f
Signed-off-by: David Bond <davidsbond93@gmail.com>
Move the container image build logic out of the e2e test setup into
cmd/k8s-operator/e2e/internal/build, with a thin CLI wrapper at
cmd/k8s-operator/e2e/build, so CI can publish images once per commit and
fan out into multiple test jobs that consume them. The command skips
images that already exist in the registry, so a retried run doesn't
fail if the registry is immutable.
Also adds support for the test harness leveraging workload identity
federation credentials so we can use GitHub's token issuer for the test
code itself, and AWS' OIDC provider for the operator, and avoid using
any secrets in CI.
Updates tailscale/corp#46577
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
Android doesn't have /etc/resolv.conf. This causes problems for people
running GOOS=linux binaries (or GOOS=android binaries without cgo, so
they don't use Android's bionic libc) in Termux, adb shell, etc.
(Go binaries built with cgo use bionic on Android: golang/go#10714)
16 years ago when I was on the Android team I added a system-wide DNS
cache (dnsproxyd) and made bionic query that, so each Android app
wasn't doing its own DNS resolution. That interface was never meant to
be stable, and I thought that code would be surely dead by now 16
years later, but apparently it lives on, and is more stable now: both
empirically (time, ossification?), and because of how Android's split system
updates work nowadays, the dnsproxyd lives on the other side of bionic,
so they seem to keep it pretty stable. The old bionic<->dnsproxyd APIs
I added 16 years ago are still there, but 8 years ago it got some additional
APIs to query by a DNS packet instead.
So use it! If we find ourselves on Android and without libc access
(and because we don't want to pull in ebitengine/purego with all its
side effects), just query the DNS server like bionic does.
This can be disabled in Linux binaries with ts_omit_androiddns.
Old links:
LineageOS/android_system_netd@007e987feehttps://android.googlesource.com/platform/system/netd/+/007e987fee7e815e0c4bc820f434a632b7a69a9d
("DNS proxy thread in netd.")
aosp-mirror/platform_bionic@a1dbf0b453https://android.googlesource.com/platform/bionic/+/a1dbf0b453801620565e5911f354f82706b0200d
("DNS proxy: the start. proxies getaddrinfo calls.")
Back then I found it cleaner to proxy at the getaddrinfo level rather
than speak in terms of DNS packets. The raw-packet resnsend command I
use here came eight years later, added in November 2018 for Android
10's android_res_nsend NDK API:
LineageOS/android_system_netd@c0c818f448https://android.googlesource.com/platform/system/netd/+/c0c818f448efa90ab1f9b1733fb86c5e22fb894c
("Add resNetworkSend cmd in DnsProxyListener")
Android 10 (codename Q, API level 29, released September 2019) is
therefore the minimum OS version for this to work.
I verified this against the DnsResolver module on an Android 13
emulator with SELinux enforcing, from the shell UID, with both a pure
Go GOOS=android binary and a static GOOS=linux binary: raw queries,
NXDOMAIN handling, the runtime Android detection, and a tailcat binary
reaching DERP with lookups visible in the daemon's logcat output, some
served from my 2010 DNS cache.
Updates #21129
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I0d63763e255a077e4e5745b3e64ba0d78dab6d69
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This commit bumps the wireguard-go dependency to incorporate changes to
the packet memory model and the tun.Device.Read and conn.ReceiveFunc I/O
interfaces. It updates their implementations accordingly.
These changes improve throughput in all measured benchmarks and reduce
peak RSS in six of eight cases. The two regressions will be addressed in
a follow-up commit that reduces peak RSS below the baseline measured at
1e69418. That work is kept separate to simplify review.
The following throughput and peak RSS benchmarks were performed with
iperf3 between two Intel i5-12400 nodes running Ubuntu 24.04 (Linux 6.8).
The UDP benchmarks did not use UDP GSO on the sender, so they were
roughly equivalent to single packet I/O through wireguard-go.
TCP/1 signifies one TCP stream; TCP/128 signifies 128 parallel TCP
streams.
Throughput (Mb/s)
Test 1e69418 After Change
TCP/1 10,371 11,354 +9.5%
TCP/128 7,886 8,404 +6.6%
UDP/1 2,111 2,853 +35.1%
UDP/128 1,747 2,235 +28.0%
Peak memory (VmHWM, kB)
Test Side 1e69418 After Change
TCP/1 TX 98,240 52,596 -46.5%
RX 287,748 73,384 -74.5%
TCP/128 TX 101,196 52,812 -47.8%
RX 290,420 63,620 -78.1%
UDP/1 TX 58,864 160,840 +173.2%
RX 137,516 49,900 -63.7%
UDP/128 TX 66,148 116,096 +75.5%
RX 154,384 56,556 -63.4%
Updates tailscale/corp#46716
Updates tailscale/corp#22467
Updates tailscale/corp#36989
Updates tailscale/corp#37878
Signed-off-by: Jordan Whited <jordan@tailscale.com>
Once the pkg-types script has generated pkg.d.ts and checked them
against the tsconfig.json and node_modules in cmd/tsconnect, copy
them into the pkgDir so they get bundled into the package.
Updates #19707
Signed-off-by: Gesa Stupperich <gesa@tailscale.com>
Replace the allowed-IP-only peer callback result with wgcfg.PeerConfig.
It carries allowed IPs and an optional pre-shared key through lazy peer
creation and active peer synchronization.
Update wireguard-go for the new peer PSK APIs.
Updates tailscale/tailcat#84
Change-Id: Iacd9d2c74b0b64d690f3cbdf93918686ac6076d7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Previously, the new disco keys entering the client via TSMP was routed
into controlClient to allow for deduplication and filtering of disco
keys and avoid control overwriting an active TSMP learned key with a
stale key.
A system supporting multiple disco keys was introduced to allow these
keys living side by side, along side a system for selecting an active
egress key with a bias towards keys learned via TSMP. On ingress any
known key is accepted.
This PR makes the switch to key routing, by sending new TSMP learned
disco keys directly into the userspace engine and in turn magicsock,
removing the need for a full layer of deduplication in the
controlClient.
Additionally, the system that previously fully reconfigured clients on
disco key updates coming from control is now using an optimistic
handshake througha newly introduced wireguard-go method,
ScheduleHandshakeOnUserSend implemented in:
https://github.com/tailscale/wireguard-go/pull/81
What this PR does not do is revert the mapSession back to being single
writer. Bringing the mapSession back to this state is desired, however
to ensure the plumbing is done right and easier to reason about, that
will be done in a separate PR.
Updates #20590
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
When the wasmbuild is run from external workflows, it can't derive
the version stamps from its build context. Allow setting the
ProdLDFlags version.longStamp and version.shortStamp from the
VERSION_LONG and VERSION_SHORT environment variables when present.
This way, if the wasmbuild caller already knows them (e.g. from
mkversion), it can pass them in.
This avoids "x.y.z-ERR-BuildInfo" showing up in the admin console's
machine version column.
Updates #19707
Signed-off-by: Gesa Stupperich <gesa@tailscale.com>
When receiving disco traffic from a node, mark that node as having been
seen. Should that disco key not be the active use disco key, switch to
that one as being the active key. Additionally, clear states on the
magicsock connection and prepare for sending a new WG handshake whenever
user data is transmitted.
Sets up for:
- Routing TSMP keys directly into magicsock
- Switching the active connection reset mechanism to the optimistic
handshake
- Cleaning up paths into controlClient
Updates #20494
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
Moving FreeBSD subnet routing from netstack to the kernel changes the
behavior of every existing FreeBSD subnet router, and the kernel path is
not yet ready to be the default:
- The pf NAT rule we install never matches. On a production FreeBSD
15.0 subnet router, "pfctl -vsn -a tailscale" reports 97071
evaluations with 0 packets and 0 translations, and "pfctl -s info"
reports translate: 0. So --snat-subnet-routes=true, the default,
silently performs no source NAT at all.
- Inserting the pf anchor at runtime requires reloading the main
ruleset, which cannot preserve the contents of any table that
ruleset references. See the comment in osrouter.loadPFMainRuleset.
Netstack does its own SNAT in userspace, touches no system state, and is
what FreeBSD has always used, so keep it as the default. The kernel path
can be opted into with TS_DEBUG_NETSTACK_SUBNETS=false; only that path
can serve --snat-subnet-routes=false, which netstack cannot do because
it must rewrite the source address.
With this, TestSubnetRouterFreeBSD passes.
Updates #5573
Signed-off-by: Martin Minkus <martin.minkus@sonic.com>
Use net.JoinHostPort when expanding proxy targets so IPv6 loopback addresses retain the required brackets. Accept ::1 as an HTTP and TCP destination in both current and legacy serve implementations.
Fixes#8702
Signed-off-by: James Tucker <jftucker@gmail.com>
Promote the toolchain from Go 1.26.6 to Go 1.27.0, matching what
go.toolchain.next.rev has been testing. Besides the toolchain files
themselves (updated by pull-toolchain.sh), this bumps the go.mod go
directive, the Dockerfile golang base image, and the README, and
regenerates the depaware.txt files and the gzip assets in
tempfork/spf13/cobra and util/eventbus, whose bytes change with
Go 1.27's rewritten compress/flate.
Also bump golangci-lint to v2.13.1, the first release line built
with Go 1.27; the prebuilt v2.10.1 binary refuses to target a Go
version newer than the one it was built with.
Also bump golang.org/x/net to v0.58.0 (plus the sibling x/ module
upgrades it requires) to pick up upstream commit 8d10596d2624
(http2: avoid deadlocks in wrapped ClientConn state callback),
which we hit during Go 1.27 rc testing.
Also add docs/go-bump-checklist.md for next time.
Updates #20220
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: Ia3e4c9effafbc91227eed39efb52f1fba1b8d89c
Extend e2e test setup to work with a remote registry and real cluster.
Previously only kind was supported.
Detect cluster node architecture and build images only for the given
architecture to reduce build time. Build for all platforms as the fallback
option. This assumes only one node architecture per cluster, which is
reasonable for our test cases.
By default, --build loads the built images into a kind cluster. Use with
--registry to instead specify and push to a remote registry.
Fixestailscale/corp#46577
Signed-off-by: Becky Pauley <becky@tailscale.com>
The functions used to be used in ipn/ipnlocal but we changed that, so
the functions can be moved into feature/conn25 now.
Updates tailscale/corp#47250
Signed-off-by: Fran Bull <fran@tailscale.com>
Clients can advertise an opaque app name in their ClientInfo but the
server previously did nothing with it.
Constrain app names to at most 32 bytes of printable ASCII, enforced
both in derp.NewClient and by the server when it parses the ClientInfo.
Extend the peerPresent frame, following its existing pattern of
appending optional fields, with a length-prefixed app name after the
flags byte, so trusted mesh watchers (other DERP nodes and stats
tools) can attribute connections by app. Old clients ignore the extra
bytes; old servers send frames without them.
Also add a derper --disallow-app-names flag taking a comma-separated
list of app names whose connections are refused, except for trusted
mesh peers.
Updates tailscale/corp#24454
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I6e721258675145833aafa1355fabf7fc05a5a204
The set-config loop uses uint16 endpoints. When Last is 65535, the increment wraps to zero and file or Unix targets continue indefinitely.
Break after applying Last so every closed range terminates without changing ordinary range behavior.
Fixes#20873
Signed-off-by: Bonobo <github@in9.at>
The service-pg-reconciler reconciles Services annotated for an ingress
ProxyGroup, but its ProxyGroup watch reused ingressProxyGroupFilter,
which is ingressesFromIngressProxyGroup. That handler lists Ingresses
and returns Ingress keys, so when a ProxyGroup became Available the
requests it produced never matched a Service and the reconciler's Get
just came back NotFound.
This fixes this by adding servicesFromIngressProxyGroup, which lists the
Services indexed for the ProxyGroup and returns their keys, matching what
the egress path already does with egressSvcsFromEgressProxyGroup.
Fixes#20944
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Also reports when shard results are missing, unreadable, duplicated, or
empty, so the pass count and percentage can't overstate what actually ran.
Updates #20931
Signed-off-by: Yaruk Asghar <yaruk@tailscale.com>
Guard optional annotations maps in generated CEL expressions before
looking up expose or proxy-group annotations. This lets ordinary
annotationless Services and Ingresses pass admission while preserving
deny-all and allowlist behavior for present proxy-group annotations.
RELNOTE: Kubernetes ProxyGroupPolicy now permits annotationless resources.
Fixes#20906
Change-Id: I8b9475d34c003ca6d233246c021ac656e0530fb1
Signed-off-by: joshrzemien <joshua.k.rzemien@gmail.com>
Historically, when DERP regions were switched away from strings to
numeric identifiers in PR #14641, tailcfg.Node.HomeDERP was declared
as an int instead of its own type.
This PR declares a new tailcfg.DERPRegionID type, represented by an
int64, and converts the following fields to use this type:
- netcheck.Report.PreferredDERP
- netcheck.Report.RegionLatency
- netcheck.Report.RegionV4Latency
- netcheck.Report.RegionV6Latency
- tailcfg.DERPHomeParams.RegionScore
- tailcfg.DERPMap.Regions
- tailcfg.DERPNode.RegionID
- tailcfg.DERPRegion.RegionID
- tailcfg.NetInfo.PreferredDERP
- tailcfg.Node.HomeDERP
- tailcfg.PeerChange.DERPRegion
- tailcfg.PingResponse.DERPRegionID
Note that the original field was an int, while the new field is backed
by an int64. This change makes DERPRegionID the same size on both
32-bit and 64-bit architectures.
Fixes: #20165
Change-Id: Ic6f795a6d791dd16f756f246d5a02085443e212f
Signed-off-by: Simon Law <sfllaw@tailscale.com>
Let clients pin favorite devices, exit nodes, and services so GUIs can
surface & change them. Pins are stored per login profile in the new
favorites feature module, keyed per category; devices and exit nodes by
StableNodeID, services by ServiceName.
The item types live in a leaf package feature/favorites/pintype,
keeping them out of the core ipn hierarchy. Each category has
its own type (pintype.Device, pintype.ExitNode, pintype.Service).
Exposed over LocalAPI at GET/POST /localapi/v0/pins, where POST replaces
only the categories named in the request so a client can update one
category without clobbering the others. Pins are local to the device and
are not synced across a user's devices.
updates tailscale/corp#44836
Signed-off-by: Will Hannah <willh@tailscale.com>
Co-authored-by: Will Hannah <wph@Wills-Virtual-Machine.local>
containerboot waits up to 60s for the initial map response before
failing. Slow map responses (>60s seen in production) leaves
containerboot to timeout and fail.
Add a TS_BOOT_TIMEOUT env var to override the default. Falls back to
60s when unset.
Fixes#20912
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
In cases where cigocacher is using a proxy that handles the auth, it's
possible to fetch stats without knowing our own access token. Don't fail
early if the access token isn't passed. If there's no proxy, it will
fail with the error from the gocached server.
Updates tailscale/corp#45427
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
Deploy k8s nameserver during e2e test setup, and point the cluster resolver
(CoreDNS or kube-dns) to it so tests can resolve MagicDNS names inside the
cluster.
Add a test to verify singleton L7 Ingress is reachable from inside the
cluster using its MagicDNS Name.
Update existing egress tests to use a dedicated tailnet target per test (to
avoid conflicts). Egress tests now also verify that an egress target is
reachable from within the cluster using both its Service and MagicDNS name.
To successfully curl using the target's MagicDNS name, publish test CAs as a
ConfigMap to the cluster, and mount these for each curl pod.
Fixestailscale/corp#38027
Signed-off-by: Becky Pauley <becky@tailscale.com>
kubeconfigPath dereferences FileInfo for non-ENOENT stat errors even though os.Stat returns nil FileInfo. Preserve the failing list entry so checkKubeconfigWritable reports the existing access error instead of panicking.
Updates #11604
Signed-off-by: Bonobo <github@in9.at>
Static size checks (iossize) catch binary dirty-page growth but nothing
covered runtime heap cost, which is what actually consumes the iOS
Network Extension's 50 MiB jetsam budget. Bring up a tsnet backend
(with the full condregister feature set, matching shipping clients)
against an in-process testcontrol server and assert live post-GC heap
budgets for (a) backend startup with zero peers and (b) marginal cost
per netmap peer.
The startup test measures 1.3 MiB today and fails loudly on the
conn25 flow-table pre-allocation regression (17 MiB) that jetsam-killed
the iOS extension on large tailnets.
Budgets are deliberately generous (6-12x current measurements) to stay
flake-free while still catching the multi-MiB regressions that matter
for mobile.
A new debugknob enables us to constrain the GSO/GRO batch size to 1 for
these tests so as to avoid the memory allocation associated with those
buffers, which are a known issue with their own work stream.
Updates tailscale/corp#46408
Updates tailscale/corp#18514
Signed-off-by: James Tucker <james@tailscale.com>
This commit modifies the "generate" tool we use on the kubernetes operator
that produces helm chart and static manifest assets for CRDs.
Previously, this required always remembering to add new constants to
a `main.go` and did not have any mechanism to fail in CI if you forgot
to. Now this tool will iterate over all the CRDs and ensures that they're
in the places they're expected to be, with a test that will fail if they
are not.
This removes the requirement for remembering to add these constants
every time you have a new CRD.
Closes: #20594
Signed-off-by: David Bond <davidsbond93@gmail.com>
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
This commit changes how PeerRelay services are exposed on AWS. A Network
Load Balancer only forwards to targets in an availability zone enabled
on it, and spec.aws.elasticIPs pins each service to a single subnet,
which enables just one zone. A replica scheduled anywhere else silently
receives nothing while still reporting PeerRelayReady with an address in
status.endpoints.
Without spec.aws we now leave the subnet unpinned, so the AWS Load
Balancer Controller spreads the load balancer over every zone it finds,
and cross-zone load balancing is on by default so any of its addresses
reach the pod. Hostname resolution is no longer gated on the
eip-allocations annotation, which had left these unpinned services in
EndpointsPending forever, and a failure to resolve now logs at debug
since it is expected while a load balancer provisions.
Such a load balancer has an address per zone, and AWS bills for each, so
every one of them is now advertised rather than only the lowest sorted.
That also lets a peer reach the relay when one zone is unreachable.
status.endpoints gains address as a second list map key so a replica can
hold an entry per address; no field changes, so existing readers of
endpoints[].address keep working. Readiness counts the replicas that
have an endpoint rather than the entries, so a replica with several
addresses cannot mask one that has none.
The pods now serve containerboot's health check endpoint and the load
balancer is pointed at it over HTTP. A peer relay listens only on UDP,
so the default TCP check against the port the load balancer forwards
could never succeed and every target sat unhealthy while relaying
perfectly well. /healthz reports 200 once the device has tailnet
addresses, which is the condition that actually matters.
The CRD docs now describe spec.aws as the exception, note that it also
needs a ProxyClass pinning pods to the zone of the subnets it names, and
drop the claim that an Elastic IP has an availability zone of its own.
Fixes: https://github.com/tailscale/tailscale/issues/20833
Signed-off-by: David Bond <davidsbond93@gmail.com>
The egressPodsReconciler (added in #14792) only sets the
tailscale.com/egress-services readiness condition on egress ProxyGroup
replica Pods that declare the corresponding readiness gate. However, the
gate was never actually added to the egress Pod template, so the
reconciler always hit its early-return and the readiness condition was
never set. This commit adds said readiness gate to the egress Pod template.
Updates #14326
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
ProxyGroup, Recorder, and PeerRelay each carried a near-identical copy of
the auth key re-issuance state machine (in-flight tracking, per-parent rate
limiting, stale-device cleanup). A bug fix had to land in three places and
could silently drift.
Extract it into a shared tailscaled.Reissuer, alongside the other tailscaled
workload helpers (NewAuthKey, AuthKeyFromConfigSecret, DeviceIDFromStateSecret)
that the callers already use. It owns its own mutex, tracks in-flight reissues
per replica keyed by parent, and rate-limits re-issuance per parent; the three
reconcilers drive it via EnsureState/RemoveState/ShouldReissue. The device
deletion helper is shared too, so the reissue state machine and its tests now
live in one place.
Updates #20544
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
This test was the repo's slowest at 60 seconds of wall time, all of it
spent sleeping: three of its subtests reach the unconditional 20 second
failover wait in kube/services.EnsureServicesAdvertised, despite using
a pure in-memory FakeLocalClient with no real control or I/O.
Run each subtest in a testing/synctest bubble so the wait elapses on
the fake clock instead. The test now completes in milliseconds.
Fixes#20792
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I3442815f7efcf6de740f893197ee0461ab049bb2
Fix the small number of existing violations of this check, and enable it for
future runs. The fixes needed were:
- Clean up a few misspelled package names (probably renames).
- Clean up a few lexical nits ("Package x" instead of "The x package").
- Add lint directives to some files affected by build tag variance.
- Add a missing package comment and re-generate the k8s docs.
The lint overrides are a little ugly, but there are only a few places where we
need them, and it's probably worthwhile to enable the check on the rest of the
repo. Rather than replicate the docs around the build tag, I made the lint
diagnotics reference the "correct" file.
Updates #cleanup
Change-Id: I0d97f2f468542af456a0396cf9a023f04f23e436
Signed-off-by: M. J. Fromberger <fromberger@tailscale.com>
The json/v2 prototype used to support a `format` tag option,
which has been removed for the initial release of json/v2 in Go 1.27.
The wrapper types in this package provide a way to avoid using
the `format` tag option for all existing use-cases.
The types are written to cooperate with other tag options
such as `string`, which may stringify JSON numbers.
We adjust cmd/vet/jsontags accordingly.
Updates #20220
Updates tailscale/corp#45953
Change-Id: Ie1fcea41dc30983e9acc43085f42a6e8ee49d26e
Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Package tailcfg defines the types and constants used by the Tailscale
protocol, but since everything is all in one package, it’s difficult
to sift through the docs: https://pkg.go.dev/tailscale.com/tailcfg
We define and enumerate capabilities as string constants for
tailcfg.NodeCapability and tailcfg.PeerCapability. This PR extracts
them into their own packages:
- tailcfg.CapabilityFileSharing becomes nodecap.FileSharing
- tailcfg.NodeAttrOnlyTCP443 becomes nodecap.OnlyTCP443
- tailcfg.PeerCapabilityTaildrive becomes peercap.Taildrive
We originally intended for CapabilityFoo to grant an entitlement or
permission for Foo, and for NodeAttrBar to configure Bar in the
nodeAttrs section of the policy file. However, there was no technical
enforcement of this convention, so new capabilities have used the
NodeAttr prefix regardless of meaning. Therefore, this PR unifies
tailcfg.CapabilityFoo and tailcfg.NodeAttrBar into a single package as
nodecap.Foo and nodecap.Bar.
Ran `go fix -inline ./...` and committed the changes that replaced
uses of the tailcfg aliases with the authoritative ones.
Updates #20259
Change-Id: Ieb7e7e6c8247c39faf42fdf15c68cdc7c621c730
Signed-off-by: Simon Law <sfllaw@tailscale.com>
The sysctler init container shells out to sysctl to turn on IP
forwarding for non-userspace proxies. That binary ships in the
procps-ng package, which Alpine has but Red Hat's UBI does not, so
on UBI the init container exits 127 and every proxy Pod is stuck in
PodInitializing and never registers a device.
Updates: https://github.com/tailscale/corp/issues/45981
Updates: https://github.com/tailscale/corp/issues/44443
Signed-off-by: David Bond <davidsbond93@gmail.com>
This commit adds a new `--base-image` flag to the e2e test suite
so that tests can build the operator and operator accessories
with a different base docker image. We want this so that we can
try things out with red hat's UBI as part of getting the operator
up and running on openshift clusters.
We can then modify the e2e test runner to use a matrix for normal
alpine base and redhat's.
Updates: https://github.com/tailscale/corp/issues/45981
Updates: https://github.com/tailscale/corp/issues/44443
Signed-off-by: David Bond <davidsbond93@gmail.com>
The Helm chart hardcoded the operator's Deployment, ServiceAccount,
Role, and RoleBinding names to "operator".
Adds a standard _helpers.tpl with name/fullname template functions and
use them for the operator's resources, including the oauth Secret and
the cluster-scoped ClusterRole/ClusterRoleBinding. When neither
nameOverride nor fullnameOverride is set, the fullname resolves to the
historical hardcoded names rather than the release name, so existing
installations upgrade with no resource renames; rendering with default
values is byte-identical to before.
Fixestailscale/tailscale#18232
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
The nameserver Deployment created for a DNSConfig was the only
operator-managed workload with no way to configure imagePullSecrets, so its
pods could not pull the nameserver image from a private registry.
Adds imagePullSecrets to NameserverPod and thread it through to the nameserver
Deployment pod spec.
Updates #16772
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
The Helm chart only applied imagePullSecrets to the operator Deployment's
pod spec, so proxy pods (which use a different ServiceAccount) never
received them and could not pull images from private registries.
Apply imagePullSecrets to the operator, proxies, and kube-apiserver-auth-proxy
ServiceAccounts.
Updates #16772
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
This commit adds a new end-to-end test for the `PeerRelay` custom
resource.
This test is currently quite limited due to the fact that we are running
our tests within a kind cluster within github actions. This means it's
not really possible to give the peer relays a proper public IP address
via the `LoadBalancer` type services that we spin up.
That being said, we intend to expand our e2e test suite with actual
real clusters in future so this can be expanded upon at a later date.
For now, this test spins up a single and multi-replica deployment of
a peer relay and confirms that it has been registered with control
and is configured to act as a peer relay.
This test also caught a small bug where the server URL was not being
passed into the peer relay's configuration, which has been fixed here.
Closes: https://github.com/tailscale/corp/issues/45731
Closes: https://github.com/tailscale/corp/issues/45737
Signed-off-by: David Bond <davidsbond93@gmail.com>
PR #20388 (commit ca9f6971e) bumped k8s.io/* from v0.34 to v0.35 as a
transitive effect of pulling in helm v3.21 (for containerd CVE fixes).
The old controller-runtime v0.19.4 is aligned with k8s 1.31 APIs and
regresses at runtime against v0.35 client-go (informer reflectors stall
on Watch responses), leaving the operator unable to reconcile.
v0.23.3 is the release aligned with k8s.io/* v0.35 / Kubernetes 1.35.
The operator uses a narrow slice of controller-runtime (manager +
builder + client + reconcile + handler + source), so this bump is a
lift-and-shift: no reconciler wiring changes needed.
Test scaffolding fix: controller-runtime v0.20+ populates TypeMeta
(Kind/APIVersion) on objects returned by the fake client. The shared
expectEqual helper now strips TypeMeta before diffing so existing
tests continue to work without needing every 'want' object updated.
Tested: deployed to live clusters across Kubernetes 1.33, 1.34, 1.35,
and 1.36 (identical build per cluster). Verified the operator starts and
reconciles cleanly on all versions with no RBAC/forbidden errors, CRDs
establish and enforce validation, and the field indexers resolve
correctly under annotation churn (no stale-index misrouting). Exercised
the Service, ProxyClass, Connector, egress (simple + ProxyGroup, incl.
the EndpointSlice reconciler), and HA ingress-for-pg reconcilers.
Confirmed informer/watch recovery after an API server restart forces a
re-list/re-watch (the exact regression this bump fixes).
Updates tailscale/corp#44898
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Go 1.27 requires this new v0.8.0-rc.1.
But staticcheck 0.8's SA4023 gets stricter and points out that
modifiedExternallyError and handleListenersAccept always return
non-nil errors, and that MonitorHealth's callers don't need a separate
nil check before errors.Is. Simplify all three call sites; no behavior
change.
But then a handful of other places that SA4023 is angry about are
wrong (because it's not considering build tags) and can't be addressed
by ignore directives (again not considering build tags), so we just
disable SA4023 for now.
Updates #20220
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I2fefe3b986b5798c2e01624a0e9820839d21a569
James Tucker