outbound: synthesize client policies on Unimplemented#2396
Merged
Conversation
If the policy controller is from a Linkerd version earlier than 2.13.x, it will return the `Unimplemented` gRPC status code for requests to the `OutboundPolicies` API. The proxy's outbound policy client will currently retry this error code, rather than synthesizing a default policy. Since 2.13.x proxies require an `OutboundPolicy` to be discovered before handling outbound traffic, this means that 2.13.x proxies cannot handle outbound connections when the control plane is on an earlier version. Therefore, installing Linkerd 2.13 and then downgrading to 2.12 can potentially break the data plane's ability to route traffic. In order to support downgrade scenarios, the proxy should also synthesize a default policy when receiving an `Unimplemented` gRPC status code from the policy controller. This branch changes the proxy to do that. A warning is logged which indicates that the control plane version is older than the proxy's.
olix0r
reviewed
Apr 24, 2023
Member
olix0r
left a comment
There was a problem hiding this comment.
I think warnings probably aren't worth it here. During legitimate downgrades this could generate a ton of log spam for services like prometheus.
olix0r
approved these changes
Apr 24, 2023
6b88667 to
8fb01a6
Compare
hawkw
added a commit
to linkerd/linkerd2
that referenced
this pull request
Apr 25, 2023
This proxy release fixes an issue where outbound proxies would attempt protocol detection on ports that are marked as opaque. It also adds support for ranges (such as `10-20`) in the `LINKERD2_PROXY_INBOUND_PORTS_DISABLE_PROTOCOL_DETECTION` and `LINKERD2_PROXY_INBOUND_PORTS` environment variables. Finally, it changes the proxy to synthesize default client policies when the policy controller returns an `Unimplemented` gRPC status code, allowing 2.13 proxies to coexist with 2.12 control planes in downgrade scenarios. --- * outbound: determine protocol based on `OutboundPolicy` (linkerd/linkerd2-proxy#2397) * set default `trust_dns` log level to `ERROR` (linkerd/linkerd2-proxy#2393) * outbound: test load balancer behavior with failure accrual (linkerd/linkerd2-proxy#2375) * outbound: add missing `meta` field in test policy (linkerd/linkerd2-proxy#2400) * inbound: determine default policies using the opaque ports env var (linkerd/linkerd2-proxy#2395) * outbound: synthesize client policies on `Unimplemented` (linkerd/linkerd2-proxy#2396) * build(deps): bump io-lifetimes from 1.0.4 to 1.0.10 (linkerd/linkerd2-proxy#2379) * chore: allow `syn` v1 and v2 to coexist peacefully (linkerd/linkerd2-proxy#2401) * build(deps): bump futures from 0.3.26 to 0.3.28 (linkerd/linkerd2-proxy#2370) * build(deps): bump async-trait from 0.1.66 to 0.1.68 (linkerd/linkerd2-proxy#2368) Signed-off-by: Eliza Weisman <eliza@buoyant.io>
hawkw
added a commit
to linkerd/linkerd2
that referenced
this pull request
Apr 26, 2023
This proxy release fixes an issue where outbound proxies would attempt protocol detection on ports that are marked as opaque. It also adds support for ranges (such as `10-20`) in the `LINKERD2_PROXY_INBOUND_PORTS_DISABLE_PROTOCOL_DETECTION` and `LINKERD2_PROXY_INBOUND_PORTS` environment variables. Finally, it changes the proxy to synthesize default client policies when the policy controller returns an `Unimplemented` gRPC status code, allowing 2.13 proxies to coexist with 2.12 control planes in downgrade scenarios. --- * outbound: determine protocol based on `OutboundPolicy` (linkerd/linkerd2-proxy#2397) * set default `trust_dns` log level to `ERROR` (linkerd/linkerd2-proxy#2393) * outbound: test load balancer behavior with failure accrual (linkerd/linkerd2-proxy#2375) * outbound: add missing `meta` field in test policy (linkerd/linkerd2-proxy#2400) * inbound: determine default policies using the opaque ports env var (linkerd/linkerd2-proxy#2395) * outbound: synthesize client policies on `Unimplemented` (linkerd/linkerd2-proxy#2396) * build(deps): bump io-lifetimes from 1.0.4 to 1.0.10 (linkerd/linkerd2-proxy#2379) * chore: allow `syn` v1 and v2 to coexist peacefully (linkerd/linkerd2-proxy#2401) * build(deps): bump futures from 0.3.26 to 0.3.28 (linkerd/linkerd2-proxy#2370) * build(deps): bump async-trait from 0.1.66 to 0.1.68 (linkerd/linkerd2-proxy#2368) Signed-off-by: Eliza Weisman <eliza@buoyant.io>
risingspiral
pushed a commit
to linkerd/linkerd2
that referenced
this pull request
May 4, 2023
This proxy release fixes an issue where outbound proxies would attempt protocol detection on ports that are marked as opaque. It also adds support for ranges (such as `10-20`) in the `LINKERD2_PROXY_INBOUND_PORTS_DISABLE_PROTOCOL_DETECTION` and `LINKERD2_PROXY_INBOUND_PORTS` environment variables. Finally, it changes the proxy to synthesize default client policies when the policy controller returns an `Unimplemented` gRPC status code, allowing 2.13 proxies to coexist with 2.12 control planes in downgrade scenarios. --- * outbound: determine protocol based on `OutboundPolicy` (linkerd/linkerd2-proxy#2397) * set default `trust_dns` log level to `ERROR` (linkerd/linkerd2-proxy#2393) * outbound: test load balancer behavior with failure accrual (linkerd/linkerd2-proxy#2375) * outbound: add missing `meta` field in test policy (linkerd/linkerd2-proxy#2400) * inbound: determine default policies using the opaque ports env var (linkerd/linkerd2-proxy#2395) * outbound: synthesize client policies on `Unimplemented` (linkerd/linkerd2-proxy#2396) * build(deps): bump io-lifetimes from 1.0.4 to 1.0.10 (linkerd/linkerd2-proxy#2379) * chore: allow `syn` v1 and v2 to coexist peacefully (linkerd/linkerd2-proxy#2401) * build(deps): bump futures from 0.3.26 to 0.3.28 (linkerd/linkerd2-proxy#2370) * build(deps): bump async-trait from 0.1.66 to 0.1.68 (linkerd/linkerd2-proxy#2368) Signed-off-by: Eliza Weisman <eliza@buoyant.io>
risingspiral
pushed a commit
to linkerd/linkerd2
that referenced
this pull request
May 5, 2023
This proxy release fixes an issue where outbound proxies would attempt protocol detection on ports that are marked as opaque. It also adds support for ranges (such as `10-20`) in the `LINKERD2_PROXY_INBOUND_PORTS_DISABLE_PROTOCOL_DETECTION` and `LINKERD2_PROXY_INBOUND_PORTS` environment variables. Finally, it changes the proxy to synthesize default client policies when the policy controller returns an `Unimplemented` gRPC status code, allowing 2.13 proxies to coexist with 2.12 control planes in downgrade scenarios. --- * outbound: determine protocol based on `OutboundPolicy` (linkerd/linkerd2-proxy#2397) * set default `trust_dns` log level to `ERROR` (linkerd/linkerd2-proxy#2393) * outbound: test load balancer behavior with failure accrual (linkerd/linkerd2-proxy#2375) * outbound: add missing `meta` field in test policy (linkerd/linkerd2-proxy#2400) * inbound: determine default policies using the opaque ports env var (linkerd/linkerd2-proxy#2395) * outbound: synthesize client policies on `Unimplemented` (linkerd/linkerd2-proxy#2396) * build(deps): bump io-lifetimes from 1.0.4 to 1.0.10 (linkerd/linkerd2-proxy#2379) * chore: allow `syn` v1 and v2 to coexist peacefully (linkerd/linkerd2-proxy#2401) * build(deps): bump futures from 0.3.26 to 0.3.28 (linkerd/linkerd2-proxy#2370) * build(deps): bump async-trait from 0.1.66 to 0.1.68 (linkerd/linkerd2-proxy#2368) Signed-off-by: Eliza Weisman <eliza@buoyant.io> Signed-off-by: Eric Anderson <eric@buoyant.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
If the policy controller is from a Linkerd version earlier than 2.13.x, it will return the
UnimplementedgRPC status code for requests to theOutboundPoliciesAPI. The proxy's outbound policy client will currently retry this error code, rather than synthesizing a default policy. Since 2.13.x proxies require anOutboundPolicyto be discovered before handling outbound traffic, this means that 2.13.x proxies cannot handle outbound connections when the control plane is on an earlier version. Therefore, installing Linkerd 2.13 and then downgrading to 2.12 can potentially break the data plane's ability to route traffic.In order to support downgrade scenarios, the proxy should also synthesize a default policy when receiving an
UnimplementedgRPC status code from the policy controller. This branch changes the proxy to do that. A warning is logged which indicates that the control plane version is older than the proxy's.