Guard optional annotations maps in generated CEL expressions before
looking up expose or proxy-group annotations. This lets ordinary
annotationless Services and Ingresses pass admission while preserving
deny-all and allowlist behavior for present proxy-group annotations.
RELNOTE: Kubernetes ProxyGroupPolicy now permits annotationless resources.
Fixes#20906
Change-Id: I8b9475d34c003ca6d233246c021ac656e0530fb1
Signed-off-by: joshrzemien <joshua.k.rzemien@gmail.com>
This commit changes how PeerRelay services are exposed on AWS. A Network
Load Balancer only forwards to targets in an availability zone enabled
on it, and spec.aws.elasticIPs pins each service to a single subnet,
which enables just one zone. A replica scheduled anywhere else silently
receives nothing while still reporting PeerRelayReady with an address in
status.endpoints.
Without spec.aws we now leave the subnet unpinned, so the AWS Load
Balancer Controller spreads the load balancer over every zone it finds,
and cross-zone load balancing is on by default so any of its addresses
reach the pod. Hostname resolution is no longer gated on the
eip-allocations annotation, which had left these unpinned services in
EndpointsPending forever, and a failure to resolve now logs at debug
since it is expected while a load balancer provisions.
Such a load balancer has an address per zone, and AWS bills for each, so
every one of them is now advertised rather than only the lowest sorted.
That also lets a peer reach the relay when one zone is unreachable.
status.endpoints gains address as a second list map key so a replica can
hold an entry per address; no field changes, so existing readers of
endpoints[].address keep working. Readiness counts the replicas that
have an endpoint rather than the entries, so a replica with several
addresses cannot mask one that has none.
The pods now serve containerboot's health check endpoint and the load
balancer is pointed at it over HTTP. A peer relay listens only on UDP,
so the default TCP check against the port the load balancer forwards
could never succeed and every target sat unhealthy while relaying
perfectly well. /healthz reports 200 once the device has tailnet
addresses, which is the condition that actually matters.
The CRD docs now describe spec.aws as the exception, note that it also
needs a ProxyClass pinning pods to the zone of the subnets it names, and
drop the claim that an Elastic IP has an availability zone of its own.
Fixes: https://github.com/tailscale/tailscale/issues/20833
Signed-off-by: David Bond <davidsbond93@gmail.com>
authkey.go references tsclient.Client in getAuthKey but never imported
the package, breaking main.
Updates #20544
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
ProxyGroup, Recorder, and PeerRelay each carried a near-identical copy of
the auth key re-issuance state machine (in-flight tracking, per-parent rate
limiting, stale-device cleanup). A bug fix had to land in three places and
could silently drift.
Extract it into a shared tailscaled.Reissuer, alongside the other tailscaled
workload helpers (NewAuthKey, AuthKeyFromConfigSecret, DeviceIDFromStateSecret)
that the callers already use. It owns its own mutex, tracks in-flight reissues
per replica keyed by parent, and rate-limits re-issuance per parent; the three
reconcilers drive it via EnsureState/RemoveState/ShouldReissue. The device
deletion helper is shared too, so the reissue state machine and its tests now
live in one place.
Updates #20544
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Fix the small number of existing violations of this check, and enable it for
future runs. The fixes needed were:
- Clean up a few misspelled package names (probably renames).
- Clean up a few lexical nits ("Package x" instead of "The x package").
- Add lint directives to some files affected by build tag variance.
- Add a missing package comment and re-generate the k8s docs.
The lint overrides are a little ugly, but there are only a few places where we
need them, and it's probably worthwhile to enable the check on the rest of the
repo. Rather than replicate the docs around the build tag, I made the lint
diagnotics reference the "correct" file.
Updates #cleanup
Change-Id: I0d97f2f468542af456a0396cf9a023f04f23e436
Signed-off-by: M. J. Fromberger <fromberger@tailscale.com>
Package tailcfg defines the types and constants used by the Tailscale
protocol, but since everything is all in one package, it’s difficult
to sift through the docs: https://pkg.go.dev/tailscale.com/tailcfg
We define and enumerate capabilities as string constants for
tailcfg.NodeCapability and tailcfg.PeerCapability. This PR extracts
them into their own packages:
- tailcfg.CapabilityFileSharing becomes nodecap.FileSharing
- tailcfg.NodeAttrOnlyTCP443 becomes nodecap.OnlyTCP443
- tailcfg.PeerCapabilityTaildrive becomes peercap.Taildrive
We originally intended for CapabilityFoo to grant an entitlement or
permission for Foo, and for NodeAttrBar to configure Bar in the
nodeAttrs section of the policy file. However, there was no technical
enforcement of this convention, so new capabilities have used the
NodeAttr prefix regardless of meaning. Therefore, this PR unifies
tailcfg.CapabilityFoo and tailcfg.NodeAttrBar into a single package as
nodecap.Foo and nodecap.Bar.
Ran `go fix -inline ./...` and committed the changes that replaced
uses of the tailcfg aliases with the authoritative ones.
Updates #20259
Change-Id: Ieb7e7e6c8247c39faf42fdf15c68cdc7c621c730
Signed-off-by: Simon Law <sfllaw@tailscale.com>
The nameserver Deployment created for a DNSConfig was the only
operator-managed workload with no way to configure imagePullSecrets, so its
pods could not pull the nameserver image from a private registry.
Adds imagePullSecrets to NameserverPod and thread it through to the nameserver
Deployment pod spec.
Updates #16772
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
This commit adds a new end-to-end test for the `PeerRelay` custom
resource.
This test is currently quite limited due to the fact that we are running
our tests within a kind cluster within github actions. This means it's
not really possible to give the peer relays a proper public IP address
via the `LoadBalancer` type services that we spin up.
That being said, we intend to expand our e2e test suite with actual
real clusters in future so this can be expanded upon at a later date.
For now, this test spins up a single and multi-replica deployment of
a peer relay and confirms that it has been registered with control
and is configured to act as a peer relay.
This test also caught a small bug where the server URL was not being
passed into the peer relay's configuration, which has been fixed here.
Closes: https://github.com/tailscale/corp/issues/45731
Closes: https://github.com/tailscale/corp/issues/45737
Signed-off-by: David Bond <davidsbond93@gmail.com>
PeerRelay replicas got a single-use auth key minted on config Secret
creation and never again, so a replica that lost its login (device
deleted, state Secret lost, key expiry) could never re-auth despite
containerboot writing the reissue_authkey signal.
Port the shouldReissueAuthKey pattern already used by ProxyGroup and
Recorder: track in-flight reissues per replica, rate-limit re-issuance
per PeerRelay, and delete the stale device before minting a new key.
Updates #20544
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
This commit moves the proxyclass reconciliation logic from its original
home in the rather large main package of the k8s-operator to a dedicated
package within `k8s-operator/reconciler/proxyclass`.
Closes: https://github.com/tailscale/corp/issues/37085
Signed-off-by: David Bond <davidsbond93@gmail.com>
The session recording WebSocket parser rejected frames whose extended
payload length field had not fully arrived yet, tearing down 'kubectl
exec' sessions whenever a read from the network delivered only the
first two or three bytes of a frame header. Every data frame carrying
126 or more bytes of payload uses an extended length field, so bulk
stdin (such as piping a file into a pod through the API server proxy)
failed with near-certainty on large transfers, while small interactive
sessions were rarely affected.
Teach fragmentDimensions to report an incomplete frame header
distinctly, so that both the data and control frame paths buffer and
wait for more bytes instead of erroring out and killing the session.
Add tests covering both extended length forms and control frames being
split at every boundary within the frame header, into two and three
segments, on both the read (client to server) and write (server to
client) paths, plus a bulk stdin passthrough test approximating how
'kubectl exec' frames and chunks a large stdin stream.
Updates tailscale/corp#39583
Change-Id: I9c28a17bc6bee898abfa2af2519c99fff9144217
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This commit contains the Kubernetes implementation of peer relays via the new `PeerRelay` CRD. It's a mega branch consisting of the commits of other PRs gone into this work:
1. https://github.com/tailscale/tailscale/pull/20211
2. https://github.com/tailscale/tailscale/pull/20329
3. https://github.com/tailscale/tailscale/pull/20423
4. https://github.com/tailscale/tailscale/pull/20503
An instance of the `PeerRelay` CRD deploys a `StatefulSet` of containerboot instances configured to advertise themselves as peer relays using the IP addresses configured via `LoadBalancer` services on each cloud provider (with some AWS specifics as it's less automatic than its competing cloud providers).
Per replica, a `LoadBalancer` type `Service` resource is provisioned and its IP address is used to configure the respective relay.
This has been tested with success in AWS, GCP & Azure and provides additional modification to `Service` resources via the CRD for any other kinds of deployment environments. It also contains some work that may appear to be duplication of what already exists within `cmd/k8s-operator` so we can start building an appropriate migration path for `Connector`, `ProxyGroup` etc into respective `k8s-operator/reconciler/*` packages.
Closes https://github.com/tailscale/corp/issues/34524
Add support for configuring egress to destinations reachable via 4via6
subnet routes, using either the synthesized 4via6 address or the MagicDNS
name (in the form <IPv4-with-hyphens>-via-<siteID>[.*]).
Also update the Connector to validate and advertise 4via6 subnet routes.
Export net/netutil.ValidateViaPrefix so it can be reused by the Connector
validation logic.
This change only affects standalone egress proxies — ProxyGroup egress
requires IPv6 support before it can use 4via6.
Updates #19334
Change-Id: I6faecd6eb61ab55fc0cd97fe417af6b6a12fe7fc
Signed-off-by: Becky Pauley <becky@tailscale.com>
This commit modifies the reconciler for the `Tailnet` custom resource
to allow referenced secrets to specify an `audience` field. If a
referenced secret contains both an `audience` and `client_id` we assume
the user's intention is to use workload identity.
In that case, we configure the tailscale API client to authenticate
using the Kubernetes token request API against the operator's service
account. This requires the operator to be aware of its own service
account name.
A small change has also been made to the messages added to the `Tailnet`
CRD's status field in the even that it is missing scopes to make it
clearer that certain scopes may not be applied.
Closes: #19090
Updates: #19471
Signed-off-by: David Bond <davidsbond93@gmail.com>
Several packages built their HTTP transports with
http.DefaultTransport.(*http.Transport).Clone()
The standard library only documents http.DefaultTransport as an
http.RoundTripper, so an application is free to replace it with a
RoundTripper that is not a *http.Transport (e.g. an instrumented or
tracing wrapper). When such an application embeds tsnet.Server, the
unchecked type assertion panics as soon as tsnet brings up its control
connection, DNS bootstrap, or log uploader.
Add netutil.NewDefaultTransport, which returns a clone of the global
when it is still the standard *http.Transport (preserving existing
behavior) and otherwise returns a fresh transport mirroring the stdlib
defaults. Route every clone site through it.
Updates #19937
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Achille Roussel <achille.roussel@gmail.com>
* cmd/k8s-operator: truncate long label values in metrics resources
Kubernetes label values have a 63-character limit, but resource names
can be up to 253 characters. When a Service or Ingress with a long
name is exposed via Tailscale, the operator fails to reconcile because
it uses the parent resource name directly as label values on metrics
Services.
Truncate label values that may exceed the limit by keeping the first
54 characters and appending a SHA256-based hash suffix to preserve
uniqueness.
Fixes#18894
Signed-off-by: Daniel Pañeda <daniel.paneda@clickhouse.com>
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
* cmd/k8s-operator: move TruncateLabelValue to shared k8s-operator package
Move the label truncation helper to k8s-operator/utils.go so it can be
reused by other components that need to produce valid Kubernetes labels.
Signed-off-by: Daniel Pañeda <daniel.paneda@clickhouse.com>
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
* cmd/k8s-operator: truncate long domain label values in cert resources
Applies TruncateLabelValue to certResourceLabels in order to prevent API
server validation failures. This covers both the HA Ingress and kube-apiserver
proxy reconcilers, as both flow through certResourceLabels.
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
* cmd/k8s-operator: remove empty metrics_resources_test.go, use hyphens in test names to satisfy go vet
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
---------
Signed-off-by: Daniel Pañeda <daniel.paneda@clickhouse.com>
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Co-authored-by: chaosinthecrd <tom@tmlabs.co.uk>
This commit modifies the kubernetes operator to use the `tailscale-client-go-v2`
package instead of the internal tailscale client it was previously using. This
now gives us the ability to expand out custom resources and features as they
become available via the API module.
The tailnet reconciler has also been modified to manage clients as tailnets
are created and removed, providing each subsequent reconciler with a single
`ClientProvider` that obtains a tailscale client for the respective tailnet
by name, or the operator's default when presented with a blank string.
Fixes: https://github.com/tailscale/corp/issues/38418
Signed-off-by: David Bond <davidsbond93@gmail.com>
Consolidate the duplicated WebSocket frame-parsing logic from Read
and Write into a shared processFrames loop, fixing several bugs in
the process:
- Mixed control and data frames in a single Read/Write call buffer
were not handled: a control frame would cause merged data frames
to be skipped.
- Multiple data frames into one Write call weren't being correctly
parsed: only the first frame was processed, ignoring the rest in
the buffer.
- msg.isFinalized was being set before confirming the fragment was
complete, so an incomplete msg fragment, could've been sometimes
marked as finalized.
- Continuation frames without any payload were being treated as if
they didn't have stream ID, even thought the id is already known
from the initial fragment.
Fixestailscale/corp#39583
Signed-off-by: Fernando Serboncini <fserb@tailscale.com>
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Co-authored-by: chaosinthecrd <tom@tmlabs.co.uk>
Add a new vet analyzer that checks t.Run subtest names don't contain
characters requiring quoting when re-running via "go test -run". This
enforces the style guide rule: don't use spaces or punctuation in
subtest names.
The analyzer flags:
- Direct t.Run calls with string literal names containing spaces,
regex metacharacters, quotes, or other problematic characters
- Table-driven t.Run(tt.name, ...) calls where tt ranges over a
slice/map literal with bad name field values
Also fix all 978 existing violations across 81 test files, replacing
spaces with hyphens and shortening long sentence-like names to concise
hyphenated forms.
Updates #19242
Change-Id: Ib0ad96a111bd8e764582d1d4902fe2599454ab65
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I omitted a lot of the min/max modernizers because they didn't
result in more clear code.
Some of it's older "for x := range 123".
Also: errors.AsType, any, fmt.Appendf, etc.
Updates #18682
Change-Id: I83a451577f33877f962766a5b65ce86f7696471c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
considerable latency was seen when using k8s-proxy with ProxyGroup
in the kubernetes operator. Switching to L4 TCPForward solves this.
Fixes tailscale#18171
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Co-authored-by: chaosinthecrd <tom@tmlabs.co.uk>
Remove the TS_EXPERIMENTAL_KUBE_API_EVENTS env var from the operator and its
helm chart. This has already been marked as deprecated, and has been
scheduled to be removed in release 1.96.
Add a check in helm chart to fail if the removed variable is set to true,
prompting users to move to ACLs instead.
Fixes: #18875
Signed-off-by: Becky Pauley <becky@tailscale.com>
This commit implements a reconciler for the new `ProxyGroupPolicy`
custom resource. When created, all `ProxyGroupPolicy` resources
within the same namespace are merged into two `ValidatingAdmissionPolicy`
resources, one for egress and one for ingress.
These policies use CEL expressions to limit the usage of the
"tailscale.com/proxy-group" annotation on `Service` and `Ingress`
resources on create & update.
Included here is also a new e2e test that ensures that resources that
violate the policy return an error on creation, and that once the
policy is changed to allow them they can be created.
Closes: https://github.com/tailscale/corp/issues/36830
Signed-off-by: David Bond <davidsbond93@gmail.com>
This commit adds a new custom resource definition to the kubernetes
operator named `ProxyGroupPolicy`. This resource is namespace scoped
and is used as an allow list for which `ProxyGroup` resources can be
used within its namespace.
The `spec` contains two fields, `ingress` and `egress`. These should
contain the names of `ProxyGroup` resources to denote which can be
used as values in the `tailscale.com/proxy-group` annotation within
`Service` and `Ingress` resources.
The intention is for these policies to be merged within a namespace and
produce a `ValidatingAdmissionPolicy` and `ValidatingAdmissionPolicyBinding`
for both ingress and egress that prevents users from using names of
`ProxyGroup` resources in those annotations.
Closes: https://github.com/tailscale/corp/issues/36829
Signed-off-by: David Bond <davidsbond93@gmail.com>
This file was never truly necessary and has never actually been used in
the history of Tailscale's open source releases.
A Brief History of AUTHORS files
---
The AUTHORS file was a pattern developed at Google, originally for
Chromium, then adopted by Go and a bunch of other projects. The problem
was that Chromium originally had a copyright line only recognizing
Google as the copyright holder. Because Google (and most open source
projects) do not require copyright assignemnt for contributions, each
contributor maintains their copyright. Some large corporate contributors
then tried to add their own name to the copyright line in the LICENSE
file or in file headers. This quickly becomes unwieldy, and puts a
tremendous burden on anyone building on top of Chromium, since the
license requires that they keep all copyright lines intact.
The compromise was to create an AUTHORS file that would list all of the
copyright holders. The LICENSE file and source file headers would then
include that list by reference, listing the copyright holder as "The
Chromium Authors".
This also become cumbersome to simply keep the file up to date with a
high rate of new contributors. Plus it's not always obvious who the
copyright holder is. Sometimes it is the individual making the
contribution, but many times it may be their employer. There is no way
for the proejct maintainer to know.
Eventually, Google changed their policy to no longer recommend trying to
keep the AUTHORS file up to date proactively, and instead to only add to
it when requested: https://opensource.google/docs/releasing/authors.
They are also clear that:
> Adding contributors to the AUTHORS file is entirely within the
> project's discretion and has no implications for copyright ownership.
It was primarily added to appease a small number of large contributors
that insisted that they be recognized as copyright holders (which was
entirely their right to do). But it's not truly necessary, and not even
the most accurate way of identifying contributors and/or copyright
holders.
In practice, we've never added anyone to our AUTHORS file. It only lists
Tailscale, so it's not really serving any purpose. It also causes
confusion because Tailscalars put the "Tailscale Inc & AUTHORS" header
in other open source repos which don't actually have an AUTHORS file, so
it's ambiguous what that means.
Instead, we just acknowledge that the contributors to Tailscale (whoever
they are) are copyright holders for their individual contributions. We
also have the benefit of using the DCO (developercertificate.org) which
provides some additional certification of their right to make the
contribution.
The source file changes were purely mechanical with:
git ls-files | xargs sed -i -e 's/\(Tailscale Inc &\) AUTHORS/\1 contributors/g'
Updates #cleanup
Change-Id: Ia101a4a3005adb9118051b3416f5a64a4a45987d
Signed-off-by: Will Norris <will@tailscale.com>
This commit contains the implementation of multi-tailnet support within the Kubernetes Operator
Each of our custom resources now expose the `spec.tailnet` field. This field is a string that must match the name of an existing `Tailnet` resource. A `Tailnet` resource looks like this:
```yaml
apiVersion: tailscale.com/v1alpha1
kind: Tailnet
metadata:
name: example # This is the name that must be referenced by other resources
spec:
credentials:
secretName: example-oauth
```
Each `Tailnet` references a `Secret` resource that contains a set of oauth credentials. This secret must be created in the same namespace as the operator:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: example-oauth # This is the name that's referenced by the Tailnet resource.
namespace: tailscale
stringData:
client_id: "client-id"
client_secret: "client-secret"
```
When created, the operator performs a basic check that the oauth client has access to all required scopes. This is done using read actions on devices, keys & services. While this doesn't capture a missing "write" permission, it catches completely missing permissions. Once this check passes, the `Tailnet` moves into a ready state and can be referenced. Attempting to use a `Tailnet` in a non-ready state will stall the deployment of `Connector`s, `ProxyGroup`s and `Recorder`s until the `Tailnet` becomes ready.
The `spec.tailnet` field informs the operator that a `Connector`, `ProxyGroup`, or `Recorder` must be given an auth key generated using the specified oauth client. For backwards compatibility, the set of credentials the operator is configured with are considered the default. That is, where `spec.tailnet` is not set, the resource will be deployed in the same tailnet as the operator.
Updates https://github.com/tailscale/corp/issues/34561
* k8s-operator,kube: removing enableSessionRecordings option. It seems
like it is going to create a confusing user experience and it's going to
be a very niche use case, so we have decided to defer this for now.
Updates tailscale/corp#35796
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
* k8s-operator: adding metric for env var deprecation
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
---------
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
This commit adds the `spec.replicas` field to the `Recorder` custom
resource that allows for a highly available deployment of `tsrecorder`
within a kubernetes cluster.
Many changes were required here as the code hard-coded the assumption
of a single replica. This has required a few loops, similar to what we
do for the `Connector` resource to create auth and state secrets. It
was also required to add a check to remove dangling state and auth
secrets should the recorder be scaled down.
Updates: https://github.com/tailscale/tailscale/issues/17965
Signed-off-by: David Bond <davidsbond93@gmail.com>
This commit modifies the k8s-operator's api proxy implementation to only
enable forwarding of api requests to tsrecorder when an environment
variable is set.
This new environment variable is named `TS_EXPERIMENTAL_KUBE_API_EVENTS`.
Updates https://github.com/tailscale/corp/issues/32448
Signed-off-by: David Bond <davidsbond93@gmail.com>
The hijacker on k8s-proxy's reverse proxy is used to stream recordings
to tsrecorder as they pass through the proxy to the kubernetes api
server. The connection to the recorder was using the client's
(e.g., kubectl) context, rather than a dedicated one. This was causing
the recording stream to get cut off in scenarios where the client
cancelled the context before streaming could be completed.
By using a dedicated context, we can continue streaming even if the
client cancels the context (for example if the client request
completes).
Fixes#17404
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
DNS configuration support to ProxyClass, allowing users to customize DNS resolution for Tailscale proxy pods.
Fixes#16886
Signed-off-by: Raj Singh <raj@tailscale.com>
This commit modifies the `DNSConfig` custom resource to allow specifying
a replica count when deploying a nameserver. This allows deploying
nameservers in a HA configuration.
Updates https://github.com/tailscale/corp/issues/32589
Signed-off-by: David Bond <davidsbond93@gmail.com>
This change adds full IPv6 support to the Kubernetes operator's DNS functionality,
enabling dual-stack and IPv6-only cluster support.
Fixes#16633
Signed-off-by: Raj Singh <raj@tailscale.com>
This commit adds a `replicas` field to the `Connector` custom resource that
allows users to specify the number of desired replicas deployed for their
connectors.
This allows users to deploy exit nodes, subnet routers and app connectors
in a highly available fashion.
Fixes#14020
Signed-off-by: David Bond <davidsbond93@gmail.com>
The serve code leaves it up to the system's DNS resolver and netstack to
figure out how to reach the proxy destination. Combined with k8s-proxy
running in userspace mode, this means we can't rely on MagicDNS being
available or tailnet IPs being routable. I'd like to implement that as a
feature for serve in userspace mode, but for now the safer fix to get
kube-apiserver ProxyGroups consistently working in all environments is to
switch to using localhost as the proxy target instead.
This has a small knock-on in the code that does WhoIs lookups, which now
needs to check the X-Forwarded-For header that serve populates to get
the correct tailnet IP to look up, because the request's remote address
will be loopback.
Fixes#16920
Change-Id: I869ddcaf93102da50e66071bb00114cc1acc1288
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
This occasionally panics waiting on a nil ctx, but was missed in the
previous PR because it's quite a rare flake as it needs to progress to a
specific point in the parser.
Updates #16678
Change-Id: Ifd36dfc915b153aede36b8ee39eff83750031f95
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
When kubectl starts an interactive attach session, it sends 2 resize
messages in quick succession. It seems that particularly in HTTP mode,
we often receive both of these WebSocket frames from the underlying
connection in a single read. However, our parser currently assumes 0-1
frames per read, and leaves the second frame in the read buffer until
the next read from the underlying connection. It doesn't take long after
that before we end up failing to skip a control message as we normally
should, and then we parse a control message as though it will have a
stream ID (part of the Kubernetes protocol) and error out.
Instead, we should keep parsing frames from the read buffer for as long
as we're able to parse complete frames, so this commit refactors the
messages parsing logic into a loop based on the contents of the read
buffer being non-empty.
k/k staging/src/k8s.io/kubectl/pkg/cmd/attach/attach.go for full
details of the resize messages.
There are at least a couple more multiple-frame read edge cases we
should handle, but this commit is very conservatively fixing a single
observed issue to make it a low-risk candidate for cherry picking.
Updates #13358
Change-Id: Iafb91ad1cbeed9c5231a1525d4563164fc1f002f
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
joshrzemien