bufioWriterPool was sized at 2KiB, this limited the number
of packets per TLS record during a push, limiting throughput
and increasing syscalls
Increase the buffer to 16KiB to pass extra DERP frames per
flush and TLS record write.
Update the websocket path to also use the pooled buffer
Updates tailscale/corp#50628
Signed-off-by: Mike O'Driscoll <mikeo@tailscale.com>
In singleton egress mode (TS_TAILNET_TARGET_FQDN), a failed resolution
of the target FQDN logged the error but then broke out of the main
runLoop instead of continuing to the next netmap update. From then on
containerboot blocked in wg.Wait() while the pod still looked healthy,
and the egress proxy kept forwarding to stale addresses until restart.
A missing target is typically transient (peer not yet in the netmap,
ACL change in flight), so log and continue to let the next netmap
update retry the resolution and reinstall the forwarding rules.
Fixes#22102
Signed-off-by: Santhi Prakash <b.santhiprakash@gmail.com>
This is a follow-up to #21777, which added this functionality but didn't
test it yet.
Updates tailscale/corp#39984.
Signed-off-by: Naman Sood <mail@nsood.in>
When tailscaled denies a serve config, the CLI advises configuring an
operator to avoid root. That advice is wrong for path and Unix socket
targets, which require root regardless. Replace the access-denied
error for those targets with one that says to use sudo.
Updates #21204
Signed-off-by: Brendan Creane <bcreane@gmail.com>
The operator user could serve a path or Unix socket if tailscaled
found a sudo rule for them, but the check used the GECOS display name
and so failed for most users with a 401. The check cannot be repaired
safely: listing another user's sudo rules never prompts, so it proves
a rule exists, not that the user has authenticated. An operator who
had not entered their sudo password could serve any root-readable file
or root-only socket. Require root instead and say so in the error.
Fixes#21204Fixes#21985Fixes#21988
Signed-off-by: Brendan Creane <bcreane@gmail.com>
portmapper.go was introduced for UPnP in commit
8ed43f1f2a.
This also adds a unit test which exercises the incomplete
natlab server implementation of these portmapping protocols.
Updates #21713
Change-Id: I33b61ccbe8c1d8d99d56aa357ae24777be3f5739
Signed-off-by: Francois Marier <francois@tailscale.com>
This type was previously logically distinct from AppConnectorAttr
despite containing the same fields. It is now logically also the same,
so doesn't need to exist separately.
Fixestailscale/corp#50897.
Signed-off-by: Naman Sood <mail@nsood.in>
Wire up wireguard-go's new NewPeerConfig.Hybrid field. (no current users)
Updates tailscale/corp#50068
Co-authored-by: David Anderson <danderson@tailscale.com>
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I80d7af011b43273e6c949214c6d862f9247a82e1
The funnel command checked Funnel access with a hardcoded port 443, so
any requested port passed and the CLI reported it as available on the
internet. Check the port the user asked for instead. Turning Funnel
off skips the check so existing configs on other ports can still be
removed.
Fixes#10742Fixes#21989
Signed-off-by: Brendan Creane <bcreane@gmail.com>
* cmd/k8s-operator: grant dnsconfigs/finalizers so the nameserver provisions on OpenShift
The nameserver ServiceAccount, Deployment, Service and ConfigMap are created
with an ownerReference to the cluster-scoped DNSConfig and
blockOwnerDeletion: true. Clusters running the
OwnerReferencesPermissionEnforcement admission plugin - which OpenShift enables
by default and upstream Kubernetes does not - require the creating identity to
hold update on the owner finalizers subresource before accepting such a child.
The operator ClusterRole grants dnsconfigs and dnsconfigs/status but no rule on
any finalizers subresource, so every nameserver reconcile is rejected:
serviceaccounts "nameserver" is forbidden: cannot set blockOwnerDeletion if
an ownerReference refers to a resource you cannot set finalizers on
It retries forever, the DNSConfig never gets a status, and dns-records-reconciler
waits indefinitely. Granting the rule lets the reconcile succeed on the next
attempt; it is inert where the admission plugin is not enabled.
Static manifest regenerated with ./tool/go run ./cmd/k8s-operator/generate staticmanifests
Fixes#21275
Signed-off-by: Hamza Khokhar <hamza.khokhar660@gmail.com>
* cmd/k8s-operator: grant proxygroups, recorders and services finalizers
Extend the OwnerReferencesPermissionEnforcement fix to the other owners whose
children are created with blockOwnerDeletion: true: ProxyGroup and Recorder
(tailscale.com) and Service (core). Without update on their finalizers
subresource, creates of the owned resources are rejected on clusters that
enable the admission plugin, such as OpenShift.
Static manifest regenerated with ./tool/go run ./cmd/k8s-operator/generate staticmanifests
Updates #21275
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Hamza Khokhar <hamza.khokhar660@gmail.com>
---------
Signed-off-by: Hamza Khokhar <hamza.khokhar660@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Restructure the L3, L3 HA and L7 ingress e2e tests into IP family subtests
to cover IPv4 and IPv6, plus DualStack for ProxyGroup.
Updates tailscale/corp#38029
Signed-off-by: Becky Pauley <becky@tailscale.com>
The root device fetch timed out after one second, but some router firmware only starts answering after just over a second, so UPnP never worked behind it. Allow three seconds.
Updates #21860
Signed-off-by: Aniket <aniketkomailkro@gmail.com>
Serve deletes client-supplied `Tailscale-User-Login` (and similar) headers
before setting its own. However, `http.Header.Del` only matches the exact
canonical name. A client could send Tailscale_User_Login instead, which
was passed through to the backend.
For Go (and some other) backends, that's not an issue, but backends that
map headers to CGI-style variables (HTTP_TAILSCALE_USER_LOGIN) treat
both names the same. Thus, for Funnel requests (or tagged nodes), where
serve sets no identity headers, the backend could trust the spoofed
value. The same applies to Tailscale-App-Capabilities.
This fixes the issue by also deleting any header whose name matches
case-insensitively after replacing '_' with '-'.
This is similar to how other projects do this:
- Django drops them: https://www.djangoproject.com/weblog/2015/jan/13/security/
- nginx does not allow them by default: https://nginx.org/en/docs/http/ngx_http_core_module.html#underscores_in_headers
- Puma had a CVE and now drops them: https://nvd.nist.gov/vuln/detail/cve-2024-45614Fixestailscale/corp#50821
Signed-off-by: Andrew Dunham <andrew@tailscale.com>
Similar to b6fc02d for private destinations, a destination inside one of
the configured magic or transit IP pools is never somewhere an app can be
reached: the magic range is installed as a route towards Tailscale on
every configured node, and the transit range belongs to conn25's own
peer-to-connector mappings. Forwarding to one routes back into the
mapping layer instead of out to an app.
Updates https://github.com/tailscale/corp/issues/50118
Updates https://github.com/tailscale/corp/issues/46169
Change-Id: I8116543d4b7a02e1f4c3c6b882ee3ddec1ac4f97
Signed-off-by: Mazdak Nasab <mazdak.nasab@gmail.com>
derp.Client.ForwardPacket flushed on every call, so each packet a
derper forwards to a mesh peer cost its own TLS record and write
syscall, serialized behind the client's write mutex with the syscall
inside the critical section.
ForwardPacket now appends the frame to one of two 16 KiB buffers and
returns. A flusher goroutine, started if none is running, swaps the
buffer for the spare under the mutex and writes it with the mutex
released, so callers keep filling the spare while a write is in
progress and frames from concurrent callers share a write. A caller
that finds the spare full waits for the write to complete, preserving
backpressure. The buffers are allocated on first forward, so only mesh
clients carry them.
Forward buffers are written directly to the connection rather than
through the client's bufio.Writer. A new mutex serializes those writes
with the client's other senders, since not every Conn tolerates
concurrent writes and bufio can split one frame over several.
A failed write is logged with the number of frames dropped, closes the
connection, wakes waiting callers with the error, and is returned by
every later call, so derphttp reconnects as before.
Adds BenchmarkForwardPacket, which also reports percentiles of
ForwardPacket's own latency, unit tests for batching, backpressure,
oversized frames, write failure, and write serialization, and an
end-to-end test that concurrently forwarded packets all reach the
destination.
Updates tailscale/corp#50628
Signed-off-by: Mike O'Driscoll <mikeo@tailscale.com>
Go's HTTP/1 server closes the request body once the handler starts
writing the response. The proxy's Transport may still be reading that
body, and when its final read fails it closes the backend connection
under the response it is copying. The client gets a truncated response.
Fixes#21894
Signed-off-by: Brendan Creane <bcreane@gmail.com>
Co-authored-by: Sam Jean <samj@tailscale.com>
Add an ipn.Prefs.Tailnet field, passed through controlclient.Options
into tailcfg.RegisterRequest.Tailnet, so a client can recommend or
require a tailnet at registration (e.g. to skip the tailnet picker when
the intent is already known) without impersonating MDM.
Previously this value could only come from the "Tailnet" syspolicy
setting. That setting still takes precedence as an administrative
override; the prefs value is used only when no policy is set.
Updates #22093
Signed-off-by: Adriano Sela Aviles <adriano@tailscale.com>
Adds an e2e test for proxy cleanup when the state Secret has no
device_id, as the unit tests use a fake client and can't show that
control honours the machineKey filter. It fills the tailnet with other
devices, checks the filter returns only the proxy's device, then removes
device_id from the state Secret and unexposes the Service. It checks
that the proxy's device is deleted and the other devices are left alone.
Updates #18922
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Adds tests that ProxyGroup scale-down and cleanup, Recorder cleanup and
PeerRelay cleanup delete the device matching the machine key in a state
Secret that has no device ID. The Recorder test puts that Secret on
replica 0, so it also checks that cleanup carries on to the other
replicas.
Updates #18922
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Adds a test that proxy cleanup deletes the device matching the machine
key in a state Secret that has no device_id. It also checks that
device_id is still used when present, that Secrets with a missing or
invalid machine key are skipped, and that Connector status does not
list a proxy before it reports its device ID.
Updates #18922
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
ProxyGroup, Recorder and PeerRelay cleanup and replica scale-down only
delete a replica's device if its state Secret holds the device's ID. A
replica that is shut down after it registered with control but before it
learnt its ID leaves its device in the tailnet.
This fixes this by deleting the device by the machine key in the state
Secret in each of these places when there is no ID.
Updates #18922
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Recorder cleanup deletes the device of each replica in turn. When a
replica had no state Secret or no device ID in it, cleanup stopped there
and reported itself done, so the devices of later replicas were left in
the tailnet.
This fixes this by moving on to the next replica instead.
Updates #18922
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Service, Ingress and Connector proxy cleanup finds the devices to delete
through the device_id key in each proxy's state Secret, which
containerboot only writes after the proxy's first netmap. If the proxy is
shut down after it registered with control but before that (for example,
a Service exposed and then unexposed in quick succession), cleanup finds
no device, removes the state Secret and the finalizer, and the device is
left in the tailnet. Connector replica scale-down has the same problem.
This fixes this by deleting the device by the machine key in the state
Secret when there is no device_id. Service, Ingress and Connector status
still only list devices that have an ID.
Updates #18922
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
tailscaled only learns its device ID, and so only writes it to its state
Secret, once it gets its first netmap. If it is shut down after it
registered with control but before that, there is no ID to delete its
device by. tailscaled does write its machine key to the state Secret
before it registers.
This adds EnsureUnreportedDeviceDeleted, which looks up the device by
that machine key and deletes it, for reconcilers to use when a state
Secret has no device ID.
Updates #18922
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
TransitIPRequest to connector
Parse the tokens from the HTTP headers in the response. Determine which
node responded to the DNS request from the RequestURL passed in as
metadata to (*extension).mapDNSResponse. Store the token and connector
node alongside the TransitIP in the addrs struct. Then populate the new
Token field in TransitIPRequest with the stored token.
Updates tailscale/corp#46169
Signed-off-by: Michael Ben-Ami <mzb@tailscale.com>
A retained legacy cert may have been issued by a different account even when no pre-adoption local key is available. Require a matching fingerprint before claiming a shared-account renewal replaces it.
Fixes#22041
Signed-off-by: Jaron Kramer <info@jkramertech.com>
Monitor teardown SIGHUPed and SIGKILLed every remembered child group
when the transport closed. sshd signals nothing at a non-PTY session
end; background jobs with redirected stdio survive and reparent to
init. Clients rely on that: VS Code Remote-SSH starts its remote
server as a plain background job in the session's group and expects
it to outlive connection drops, and tailscale upgrades run as
non-PTY execs whose work should survive a daemon restart.
Use an explicit graceful monitor close to preserve session jobs on
ordinary disconnect and daemon shutdown. Close PTY masters normally,
letting the terminal driver signal the foreground group, but do not
signal non-PTY jobs. Remove Linux Pdeathsig from session children;
forwarding children retain it and always die with the connection.
Commit duration and recording-failure termination to the monitor
before publishing cancellation or writing the termination message.
The monitor owns per-session HUP-to-KILL escalation even after the
leader exits or the connection closes. Revocation arms connection-wide
cleanup. Pin every termination target's leader until after SIGKILL,
including during teardown, so group signaling cannot hit reused PIDs.
Select teardown targets explicitly and leave graceful survivors alone.
Unexpected RPC loss fails closed. If a termination mandate cannot be
delivered, close the RPC rather than subsequently requesting graceful
teardown; this may also kill sibling jobs on a broken channel. Keep
both the escalation and monitor shutdown waits bounded.
Add coverage for default-HUP graceful survivors, actual duration and
recording-failure termination followed by immediate disconnect, sibling
survival, HUP-exiting leaders with immune group members, broken RPCs,
and leader pinning through teardown's SIGKILL pass.
Fixes#22076
Signed-off-by: James Tucker <james@tailscale.com>
Change-Id: I70b4ef40035330dd542d256ba79d889a12426a74
Retry UDP reads returning EHOSTUNREACH or ENETUNREACH, including their
Winsock equivalents, without returning those errors to wireguard-go.
Count them in magicsock_recv_udp_error_reachability. Other read errors are unchanged.
Updates #20616
Updates #19504
Signed-off-by: James Tucker <james@tailscale.com>
Several reports including a severe internal issue have demonstrated use
cases where the current RACK implementation still fails too poorly to be
our enabled default.
Updates #21946
Updates #21651
Updates #18307
Signed-off-by: James Tucker <james@tailscale.com>
This test encodes the current behaviour of the Tailscale client.
It only covers the creation of new mappings.
Updates #21713
Change-Id: Ia44877036569660f4129484d5059ed22ffbf9cda
Signed-off-by: Francois Marier <francois@tailscale.com>
Similar to 7304dd4 for host-scoped destinations, conn25 needs to
block private destinations: a connector forwards to app destinations,
and the packet filter only ever sees the transit IP, never the
destination, so a private destination would let a client reach into the
connector's own network.
Updates https://github.com/tailscale/corp/issues/50118
Updates https://github.com/tailscale/corp/issues/46169
Change-Id: I700b0b5a270723f5a77907e35c52d729f88c44b6
Signed-off-by: Mazdak Nasab <mazdak.nasab@gmail.com>
Logging is quite aggressive and most of the logs can be hidden behind
component logging. A few logs are instead gated on if they changed or if
the original state was zero.
Updates #cleanup
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
Bound the server shutdown wait and the output drain after a session
process exits. Descendants can keep stdio open after the session
process exits, so neither wait can depend on EOF alone. Abort the SSH
channel on drain timeout without sending EOF concurrently with an
outstanding output write.
Close all PTY master descriptors on cancellation and let the terminal
driver send SIGHUP. Duplicate the masters with CLOEXEC and O_NONBLOCK
before wrapping them with os.NewFile so the runtime poller can
interrupt pending I/O on Close. Keep the PTY child's blocking mode intact.
Use RawConn.Control for duplication and window changes to synchronize
with Close and avoid restoring blocking mode on the original master
through File.Fd.
Non-PTY sessions receive SIGHUP explicitly. Keep SIGKILL escalation
for monitor sessions and add it for directly launched sessions. Use
os.Process.Signal on the direct path to synchronize signaling with
reaping. Make the escalation delay configurable alongside the
shutdown and I/O drain timeouts.
Use a process-completion channel to stop the cancellation worker on
normal exit, and join the worker before sending exit status so any
cancellation message is written before channel teardown. Cover normal
completion and cancellation through live monitored SSH sessions.
Fixes tailscale/corp#50380
Fixes#22059
Updates #20628
Updates #21827
Signed-off-by: James Tucker <james@tailscale.com>
Change-Id: I2a66741ba156a607dc3df70a79e140f19737b7a5
Resolve Server.Dial and Server.HTTPClient names using the tailnet's DNS
configuration, including MagicDNS, search domains, split DNS routes, and
global resolvers. Preserve the dialer's local MagicDNS fast path.
Add Server.LookupIP for direct lookups returning netip.Addr values. Keep
the resolver implementation private to tsnet. Search expansion uses
ndots=1, includes the MagicDNS suffix when enabled, and respects trailing
dots. Each lookup uses the current DNS configuration.
Use the internal resolver before considering host fallback. Configured
upstream failures and authoritative local denials must not fall back to
the host resolver; the host's hosts file is consulted only during fallback.
Distinguish configured routes from ambient cloud-host fallback resolvers.
Cover search expansion, split and global DNS, MagicDNS-disabled operation,
configuration updates, hosts-file precedence, and dialing in tests.
Fixes#5840
Updates #7984
Updates #15260
Updates #4677
Change-Id: I832bc4c68658d2d25c5dc29b79acf611fc11e59e
Signed-off-by: James Tucker <james@tailscale.com>
Since 1.104.0 (#21303), netstack's TCP uses gVisor's RACK-TLP loss
detection again. In the gVisor that go.mod pins, RACK's loss marks do
not reduce the SACK recovery pipe, and after an RTO, RACK starts fast
recovery for the same flight, with ssthresh 2 and cwnd 5. After a
burst loss, the sender can then retransmit about two segments per RTO
for seconds to tens of seconds, and for minutes when the client does
not offer TCP timestamps (Windows' default), because the RTO then
keeps backing off. That stalls TCP that netstack terminates, such as
tailscale serve, Tailscale SSH and tsnet, while kernel TCP over the
same path does not stall.
Add TS_NETSTACK_ENABLE_RACK as an explicit opt-out without changing
the default. Unset keeps gVisor's default, RACK on; true keeps it on
explicitly. Setting it to false brings back the RFC 6675 SACK loss
recovery that netstack used from 1.82 through 1.102, until go.mod has
a gVisor with the fixes (google/gvisor#15432 and google/gvisor#16081).
Keep RACK enabled by default because disabling it is several times
slower on paths that reorder packets (#20893).
Test the default, explicitly enabled, and explicitly disabled cases.
Updates #21946Fixes#22045
Signed-off-by: Aditya Merchant <adityamerchant08@gmail.com>
Signed-off-by: James Tucker <james@tailscale.com>
Co-authored-by: James Tucker <james@tailscale.com>
Change-Id: I7d8012beed8a5722fb026203625299b5597fdadc
Also includes a minor change to a packet identification helper in magicsock,
to make it correctly identify retried hybrid handshake packets.
Updates tailscale/corp#50068
Signed-off-by: David Anderson <danderson@tailscale.com>
Change-Id: I6204b8f6b60342fe3c0b7b63c4b90c8c6a6a6964
Similar to #20866 for 4via6, conn25 needs to block host-scoped
destinations. Due to the similarity between the two situations, the
matching is no longer in a 4via6-specific location. Since there is no
backwards compatibility to satisfy, there is no exception list for
conn25.
ipn/ipnlocal (for 4via6) and feature/conn25 now both use the new
package.
Updates tailscale/corp#50118
Updates tailscale/corp#46169
Change-Id: Ic3f9cccd2f5823a2ca48bd78607792b0fd6d7db9
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
termWriter converted each chunk of the SSH output to a JS string on its
own which incorrectly handles multi-byte characters across chunk
boundaries. Move it to a separate cmd/tsconnect/termwriter package with
tests, and hold back incomplete sequences for the next write.
Fixes#22007
Signed-off-by: Aarnav Tale <git@tale.me>
Addresses handed out by ippool.next() are marked in use, and the only path
back to the pool is the popExpired loop in reserveAddresses, which can
only see an addrs once it's in the client's assignments. So a reservation
that fails after taking an address leaks it from the pool
for the lifetime of the process.
The leak is the result of interaction between the magic and transit pools.
First an address from magic pool is picked. Then another address from
the transit pool is picked. If an error happens during picking an address
from transit pool, the address from the magic pool needs to be returned,
otherwise that address is leaked, as the address in not in the client
assignments and won't be released.
Fixes https://github.com/tailscale/tailscale/pull/22019
Change-Id: I3d98282762576cc57c135761859902a1d0a63c31
Signed-off-by: Mazdak Nasab <mazdak.nasab@gmail.com>
Allocate a fresh 16-bit ID for each QueryDNS request instead of using the
constant ID 1.
Updates tailscale/corp#48187
Change-Id: I9b47a3fbbef70f0462f1da4f3b29eb77cf31f9f0
Signed-off-by: James Tucker <james@tailscale.com>
Accept a caller context in LocalBackend.QueryDNS and pass it to the
internal resolver. Use the HTTP request context for localapi DNS queries
so cancellation and deadlines bound the lookup.
Updates #8338
Change-Id: I1bb8ec0b779c883ec40d52063adf25ff36cf43e3
Signed-off-by: James Tucker <james@tailscale.com>
Claus Lensbøl