If we don't close the connection between SSH server and recorder
explicitly once it's idle after the upload stream is closed, the
connection stays open and holds on to a port on the server. This
leads to port exhaustion on the server in the medium to long run.
To avoid this, close the idle connections explicitly. As an extra
step of precaution, set an idleConnTimeout of 30 seconds on both
the HTTP1 and HTTP2 recorder clients.
Updates tailscale/corp#43742
Signed-off-by: Gesa Stupperich <gesa@tailscale.com>
In direct mode we write resolv.conf via a temp file and rename(2), which
preserves the source's generic etc_t label instead of net_conf_t, causing
AVC denials when NetworkManager later manages the file. Run restorecon
after the rename (Linux, SELinux-enforcing, best effort) to restore the
policy-default label.
Fixes#20149
Signed-off-by: Brendan Creane <bcreane@gmail.com>
This patch adds support for the fmt.Stringer interface to the
ipn.NotifyWatchOpt enum. This is useful when debugging these bitmasks.
For example:
fmt.Printf("%s", ipn.NotifyPeerChanges | ipn.NotifyNoNetMap)
// Output: (ipn.NotifyPeerChanges | ipn.NotifyNoNetMap)
Fixes#20066
Signed-off-by: Simon Law <sfllaw@tailscale.com>
Add support for configuring egress to destinations reachable via 4via6
subnet routes, using either the synthesized 4via6 address or the MagicDNS
name (in the form <IPv4-with-hyphens>-via-<siteID>[.*]).
Also update the Connector to validate and advertise 4via6 subnet routes.
Export net/netutil.ValidateViaPrefix so it can be reused by the Connector
validation logic.
This change only affects standalone egress proxies — ProxyGroup egress
requires IPv6 support before it can use 4via6.
Updates #19334
Change-Id: I6faecd6eb61ab55fc0cd97fe417af6b6a12fe7fc
Signed-off-by: Becky Pauley <becky@tailscale.com>
This patch adds:
- Set.All which returns an iter.Seq to complement Set.Slice.
- Set.AddSeq which adds an iter.Seq.
- Set.DeleteSeq which deletes an iter.Seq to complement Set.AddSeq
and provide the missing method for deleting multiple elements.
- Set.DeleteSlice and Set.DeleteSet to complement AddSlice and AddSet.
Updates #cleanup
Signed-off-by: Simon Law <sfllaw@tailscale.com>
The Logger previously took a *netmap.NetworkMap at Startup and on every
ReconfigNetworkMap call, denormalizing it into per-IP and self lookup
maps. That denormalization is O(n) over all peers and ran on every
netmap update, contributing to the broader quadratic behavior we want
to eliminate when a single peer is added or removed.
Instead, this makes netlog ask LocalBackend (well, nodeBackend) for
the info it needs, letting us remove the netmap.NetworkMap type
entirely from the netlog package.
This is a dependency to removing the netmap.NetworkMap type from
upstream callers, like wgengine.Engine in general.
Updates #12542
Change-Id: Ib5f2de96e788a667332c0a6f7ac833b3d0053b5c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
In PR #17809, @bradfitz tried to fix tsnet_test.TestConn by making the
second tailscaled start after the first was fully set up. On slow
runners, the Ping for connectivity to the second server would race
against that server establishing a connection with its DERP home. If
the Ping arrived too soon, the DERP server would respond with
PeerGoneNotHome and the Ping would wait for its full timeout before
failing the test.
This patch introduces waitForHomeDERPConnected and makes startServer
block until the server’s home DERP has established its connection.
This patch also reduces the Ping timeout to 10 seconds for the tsnet
tests, which should be enough that a hung Ping is fast enough for
interactive debugging, but with enough headroom for a RekeyTimeout.
Fixes#12766
Signed-off-by: Simon Law <sfllaw@tailscale.com>
Mappings from transit IPs to real IPs are stored ephemerally in the
connector, so they're lost on restart. When we send a packet to the
connector with a transit IP it does not recognize, it sends us a TSMP
message saying so (see #19883). If we (the client) know of such a
mapping, we now re-send it to the connector so that a connection can
proceed.
Fixestailscale/corp#34256.
Signed-off-by: Naman Sood <mail@nsood.in>
Add support for the still pending encoding.ScalarMarshaler and
encoding.ScalarUnmarshaler interfaces, approved in golang/go#56235.
This patch deprecates geo.Point.MarshalUint64 in favour of
geo.Point.MarshalScalar and also adds an inline directive for go fix.
The same applies for the UnmarshalUint64 and UnmarshalScalar methods.
Updates #16583
Signed-off-by: Simon Law <sfllaw@tailscale.com>
Follow-up cleanups to the IPv6 fragment extension header support added in
the previous commit:
- Document that minFragBlks is sized for IPv4 but intentionally reused by
decode6 for IPv6 fragments, where it is conservative (IPv6 fragments
carry no per-fragment IP header) and only ever rejects more later
fragments as Unknown, never fewer.
- Add a TestDecode case for a first fragment reachable only through a
chained extension header (base Next Header = Hop-by-Hop Options, which
chains to Fragment). decode6 only parses the Fragment header when it is
the base header's immediate Next Header, so this must classify as
Unknown. The test locks in that scoping decision.
Updates #20083
Updates #20140
Change-Id: Ibece03c6baf2385b0cc399f179819b08cbe921cc
Signed-off-by: James Tucker <james@tailscale.com>
util/def: add def.Bool and def.Duration default parse helpers
Replace multiple instances of def.Bool and def.Duration with a new util/def
package.
Updates #20018
Co-authored-by: Bobby <boby@codelabs.co.id>
Co-authored-by: Simon Law <sfllaw@tailscale.com>
Signed-off-by: Bobby <boby@codelabs.co.id>
Signed-off-by: Simon Law <sfllaw@tailscale.com>
Add UploadLogs, a stateless alternative to NewLogger for callers that
want to push a batch of log entries without the background uploader,
ring buffer, stderr echoing, or network-up gating that a Logger
provides. Entries are encoded, batched up to the server's maximum
upload size, and POSTed synchronously; unlike Logger it does not retry.
The Logger construction is split into a new unexported newLogger so the
connection/encode/upload machinery is shared without starting the
background goroutine.
Log entries are modeled as a generic LogEntry[T] whose Value is inlined
(via go-json-experiment) alongside the reserved "logtail" metadata
member. T may be a struct (or pointer), a map with a string key, or a
jsontext.Value; use jsontext.Value to mix differently-shaped payloads in
a single upload. UploadLogs fills in client_time/proc_id/proc_seq from
the Config where the caller leaves them zero.
Updates tailscale/corp#40908
Change-Id: Idbf23cd0eb8233082fbdb9abed0f6f153b9225ba
Signed-off-by: James Scott <jim@tailscale.com>
ipnlocal.LocalBackend.populatePeerStatusLocked assumed that Hostinfo
was always valid, but that’s not always true, especially in tests.
ipnlocal.peerAPIPorts suffered from a similar assumption.
This patch checks for NodeView.Valid and Hostinfo.Valid; assuming the
zero value as a safe default.
Updates #8948
Updates #12542
Signed-off-by: Simon Law <sfllaw@tailscale.com>
The earlier aa5da2e5f2 made peer adds and removes through a netmap
delta path that mutates only nodeBackend, on the assumption that
PeerForIP, lookupPeerByIP, the engine's wireguard config
(e.lastCfgFull), the engine BART, wgdev's PeerLookupFunc closure, and
the engine's cached netmap (e.netMap) would all stay correct without
further updates. They don't. I'd totally forgotten that
Engine.PeerForIP has its own alternate IP-to-peer lookup codepath.
Concretely, all of these failed for a peer that arrived via
[tailcfg.MapResponse.PeersChanged] (and never via a full
[tailcfg.MapResponse.Peers] list):
- [wgengine.Engine.PeerForIP] read from e.netMap and e.lastCfgFull
(neither updated on the delta path) and so missed the new
peer. The rando non-data-plane callers (Ping, TSMP, pendopen,
debug endpoints, tsdial.Dialer.UseNetstackForIP for tsnet and
onlyNetstack tailscaled) all returned "no matching peer".
- The engine BART (built from e.lastCfgFull) missed the new peer's
subnet routes / exit-node default routes.
- wgdev's [device.PeerLookupFunc] closure (rebuilt only inside
wgcfg.ReconfigDevice) didn't have the new peer's noise key, so
outbound encryption to the new peer dropped the packet even when
SetPeerByIPPacketFunc returned the right NodePublic.
- And nothing in the delta path triggered NodeMutationRemove to
flow through to authReconfig either, so the same stale state
pointed at removed peers indefinitely.
So just (functionally) revert it for now, to have something easily
cherry-pickable to the 1.100 release branch. Proper fixes can come later
for the next release.
This also adds three new tests:
- TestPingPeerLearnedViaDelta runs disco and TSMP subtests over a
delta-added peer with only self addresses. disco exercises the
cold PeerForIP path (magicsock); TSMP exercises the full data path
through wgdev encryption. Both fail without this fix.
- TestPingSubnetRouteOfDeltaPeer exercises a subnet-router peer
arriving via delta. With s1 in --accept-routes mode, an IP
inside the advertised CIDR must resolve to s2 and a TSMP ping
must round-trip. Hits the BART + lastCfgFull + wgdev staleness
in one go.
- TestPingSelfReturnsIsLocalIP is a regression guard for the
IsSelf early-out in Engine.Ping. Passes on main today; included
here so future refactors of PeerForIP can't regress self
handling without test breakage.
Updates tailscale/corp#43394
Change-Id: I7a049271359bd73e7147ae9e2554e85614c2b8d2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
decode6 didn't parse the IPv6 Fragment extension header (Next Header 44),
so any source-fragmented IPv6 packet was classified as an unknown protocol
and matched no ACL rule. The filter then silently dropped it and counted it
as an "acl" drop, even on allow-all tailnets, blackholing large UDP (DNS,
WebRTC, etc.) over a tailnet's IPv6 addresses. IPv4 fragments were already
handled by decode4.
Parse the fragment header the same way: read the first fragment's transport
ports so the filter matches it like an unfragmented packet, pass later
fragments through as ipproto.Fragment, and reject overlapping-fragment
offsets (RFC 1858) and first fragments too short to hold the transport
header as unknown.
Fixes#20083
Signed-off-by: Steve Avery <hello@stevenavery.com>
Added in #20111, but it is too noisy under real load to be useful.
Updates #12542
Change-Id: Ib99a8966ade0bfa4281fccc057249819cdcdfe83
Signed-off-by: M. J. Fromberger <fromberger@tailscale.com>
`go run` builds a manifest-less .exe, so Windows applies installer-
detection heuristics and requests admin privileges to programs that
contains "install", "setup", or "update". Rename to dodge that.
Updates #20133
Change-Id: I144d3fcb076d7a02e4a3eb9fd079ee022a035c76
Signed-off-by: Fernando Serboncini <fserb@tailscale.com>
Add a workflow that requests review from @tailscale/k8s-devs on PRs
touching Kubernetes operator, kube libraries, container build, etc.
Also cleans up check out code on k8s and dataplane workflow.
Updates #cleanup
Change-Id: I6fd7cacf71e1299f7e8f546ef52c4063fbf6bab8
Signed-off-by: Fernando Serboncini <fserb@tailscale.com>
tailscale serve set-config now also accepts the legacy raw ipn.ServeConfig
format (as emitted by `tailscale serve status --json` and consumed via
TS_SERVE_CONFIG, which has no "version" field), so the common
serve-status-edit-set workflow stops failing. Only the services-oriented
content is applied; any node-level fields are skipped with a warning to
stderr pointing users at get-config to migrate.
Fixestailscale/corp#39793
Signed-off-by: Brendan Creane <bcreane@gmail.com>
Bumps wireguard-go pin to include the roaming endpoints fix, and
two internal enhancements.
Pulls stock wireguard-go for non-tailscale simulation in tests,
to use its endpoint discovery mechanism.
Updates #20082
Change-Id: I2ff282cb7fe4ab099ce5e780a1d40ae86a6a6964
Signed-off-by: Alex Valiushko <alexvaliushko@tailscale.com>
Package features/conn25 wires up the hooks directly on the tun wrapper
without needing to go through the userspace engine, so this codepath is
unused and not needed.
Updates #cleanup
Signed-off-by: Michael Ben-Ami <mzb@tailscale.com>
Add an UpdatePeers method to the cache. This allows us to support netmap peer deltas,
by allowing just the peers to be updated in an existing cache. As a safety check, reject
an update if there was no base netmap data to apply a change to.
Then, when processing peer mutations in the backend, capture any changes that should
be applied to the cache and update it, if one is enabled.
Updates #12542
Change-Id: I2f8790a8fdc5e85fce6700ba4821a8cb10dddffa
Signed-off-by: M. J. Fromberger <fromberger@tailscale.com>
Since deltas are only (at present) received from the control plane, processing
a delta signifies we are no longer operating on a netmap fully loaded from
cache, even if most of the netmap is still in the same configuration.
Updates #12542
Change-Id: I84132c4bf2dde6e5c1c57144645edb986b051dca
Signed-off-by: M. J. Fromberger <fromberger@tailscale.com>
Flakeytest seems to not work on vmtest. We have a few PRs that will fix
the problem on these tests, so skip to unblock.
Updates #19843
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
The hook fires when a flow is removed for any reason (LRU capacity eviction,
tuple-collision displacement, or idle-time expiry). The hook is invoked
exactly once per flow, after the flow table mutex is released, so callbacks
may safely acquire other locks.
We rename the IPMapper interface to Conn25Datapath, and add
ClientFlowCreated/ClientFlowRemoved methods so *Conn25 can keep client-side
address assignments alive while traffic is in flight. Those methods are
currently stubbed for future work.
Connector flows do not currently call these methods.
Updates tailscale/corp#38630
Updates tailscale/corp#43180
Signed-off-by: Michael Ben-Ami <mzb@tailscale.com>
The returned error in the signature is left over from previous
implementations and was only returning nil.
If we know NewFlow will succeed we can fire a create hook (implemented
in a future commit) before NewFlow, which will prevent a remove hook for
a flow from firing before the create hook for the same flow.
Updates tailscale/corp#38630
Signed-off-by: Michael Ben-Ami <mzb@tailscale.com>
Adds tailscaled_serve_{inbound,outbound}_bytes_total, labeled by Tailscale
Service name, by wrapping the peer-facing conn in tcpHandlerForVIPService.
Per-service counters persist for the process lifetime rather than being
evicted on serve-config changes.
Fixes#19572
Signed-off-by: Raj Singh <raj@tailscale.com>
Co-authored-by: Ethan Smith <ethan.smith@grafana.com>
When running under the macOS sandbox, "tailscale configure kubeconfig"
refused outright whenever $KUBECONFIG was set, assuming the path would
not be writable. Yet when $KUBECONFIG was unset it happily relied on the
home-relative-path entitlement to write to ~/.kube/config, so the two
paths made inconsistent assumptions about what the sandbox can reach.
Resolve the kubeconfig path first, then check whether the target file
(or the nearest existing parent directory) is actually writable. Only
report an error if it is not, and include macOS sandbox guidance in that
error since a path outside the home directory is the likely cause. This
lets a $KUBECONFIG that does point under the home directory work, rather
than being rejected unconditionally.
Fixes#20007
Change-Id: I9880363c38b981efaed7e97367851ddacf647be1
Signed-off-by: James Tucker <james@tailscale.com>
This adds testcontrol support for expiring individual node keys,
in order to enable test scenarios involving to key-expiry and
extension.
Updates #19326
Signed-off-by: Gesa Stupperich <gesa@tailscale.com>
authRoutine snapshots c.loginGoal, runs TryLogin without the lock,
then writes back loggedIn/loginGoal under the lock. If a concurrent
Login() or Logout() changes the goal during the in-flight request,
the write-back overwrites the new intent: the more recent login goal
is silently dropped, or a logout is reverted to logged-in.
Gate both the URL-followup and success commits on c.loginGoal still
matching the goal we were processing. Stale results are ignored and
the next iteration runs with the current goal.
Updates #19326
Signed-off-by: Gesa Stupperich <gesa@tailscale.com>
When a client's node key expires and the user clicks "Login" (or runs
`tailscale up`), the Login() method was cancelling the map poll context.
This caused key extension notifications from the server to be lost,
leaving clients stuck in NeedsLogin state even after an admin extended
their key.
The fix has three parts:
1. Login(): Don't cancel mapCtx if we have valid credentials (loggedIn=true)
or a valid node key. This allows the map poll to continue receiving
server notifications while the auth flow proceeds in parallel.
2. mapRoutine(): Poll when we have a node key, even if !loggedIn. This
handles the tsnet restart scenario where control returns an AuthURL
(so loggedIn=false) but we still have a valid node key that can
receive map updates.
3. sendStatus()/UpdateFullNetmap(): Forward netmaps when we have a node
key, not just when loggedIn. This ensures the backend sees key expiry
changes even when the auth flow hasn't completed.
"First successful flow wins": if a key extension arrives via map poll,
the client recovers automatically. If the auth flow completes first,
that works too. Either way, the client is no longer stuck.
This aligns with the SeamlessKeyRenewal philosophy: maintain connectivity
paths while authentication proceeds, allowing server-initiated recovery.
Fixes#19326
Change-Id: I26dbbc1fa7c1159ba075362e44d02814355d6b44
Signed-off-by: Avery Pennarun <apenwarr@tailscale.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* cmd/k8s-operator: rework [unexpected] log lines
This commit modifies several places in the operator logs where we
prepend `[unexpected]` to instead use an appropriate logging level.
The `[unexpected]` prefix is intended to be used when the program
violates some internal invariant (or for example, a database has
become corrupted). Many of these cases were simply log lines that
then fell back to a default value/behaviour. These have been releveled
to warnings.
Some of these log lines also seemed extraeneous as for the example of
service reconcilers logging when there is no proxy group annotation. As
far as I can tell we've never had any predicates for limiting the
services reconciled to ones with that annotation, so they can just
be removed to reduce log spam.
Fixes: #cleanup
Signed-off-by: David Bond <davidsbond93@gmail.com>
* Update cmd/k8s-operator/egress-services-readiness.go
Co-authored-by: BeckyPauley <64131207+BeckyPauley@users.noreply.github.com>
Signed-off-by: David Bond <davidsbond@users.noreply.github.com>
* Update cmd/k8s-operator/operator.go
Co-authored-by: BeckyPauley <64131207+BeckyPauley@users.noreply.github.com>
Signed-off-by: David Bond <davidsbond@users.noreply.github.com>
---------
Signed-off-by: David Bond <davidsbond93@gmail.com>
Signed-off-by: David Bond <davidsbond@users.noreply.github.com>
Co-authored-by: BeckyPauley <64131207+BeckyPauley@users.noreply.github.com>
Prevent tailscale ssh from automatically adding a username when
connecting to a server, only forward one if provided. The previous
behaviour prevented username overrides in the ssh configuration, since
the provided username takes precedence to the configured one.
This also keeps the tailscale ssh a thin wrapper around ssh by not
adding any extra arguments unless required.
Fixes#19357
Signed-off-by: Örjan Fors <o@42mm.org>
I previously (in #20096) had only considered the tailscaled deps
and forgot about the CLI deps. This does the CLI ones too.
containerboot and k8s-operator aren't applicable because they build
from oss already.
Updates tailscale/corp#43243
Updates #20067
Change-Id: I66790f822b5d040e7fcf90feabca24669f69cf61
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
We aren't supposed to be using CODEOWNERS as blocking
reviews, blocking global cleanups.
(This is why we want to move to go/policybot)
Updates tailscale/corp#13972
Change-Id: I380258e2d4ffd0720d57d891adab06c8ca388617
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The 1 minute timeout was hitting timers inside wireguard-go, leading
stale connections hanging forever. Increasing the timeout to 2 minutes
makes a small subset of cached connections establish direct connections
slightly slower.
Updates to wireguard-go will allow a better hook for when to send these
messages in the future. This change only makes fixes the error mode but
if we have better triggers coming in wireguard-go, we should be using
those.
Updates #20081
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
9be21088f4 changed sending disco pings so
a callMeMaybe would be not be gated by endpoints existing if the node
was running off of a cached netmap.
This commit partly reverts that change, but keeps in a few bug fixes in
that commit and the tests that was introduced and now skipped.
The behaviour prior to 9be21088f4 is
retained.
Updates #20085
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
This commit modifies the reconciler for the `Tailnet` custom resource
to allow referenced secrets to specify an `audience` field. If a
referenced secret contains both an `audience` and `client_id` we assume
the user's intention is to use workload identity.
In that case, we configure the tailscale API client to authenticate
using the Kubernetes token request API against the operator's service
account. This requires the operator to be aware of its own service
account name.
A small change has also been made to the messages added to the `Tailnet`
CRD's status field in the even that it is missing scopes to make it
clearer that certain scopes may not be applied.
Closes: #19090
Updates: #19471
Signed-off-by: David Bond <davidsbond93@gmail.com>
