xds: use V4_PREFERRED dnsLookupFamily by default by zirain · Pull Request #4745 · envoyproxy/gateway

zirain · 2024-11-20T19:56:53Z

fixes: #4744

zirain · 2024-11-20T19:57:24Z

cc @zetaab can you verify this on your env? hope it will work.

codecov · 2024-11-20T20:03:50Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 65.60%. Comparing base (78da42c) to head (c17c635).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4745      +/-   ##
==========================================
- Coverage   65.61%   65.60%   -0.02%     
==========================================
  Files         211      211              
  Lines       31961    31961              
==========================================
- Hits        20972    20968       -4     
- Misses       9751     9753       +2     
- Partials     1238     1240       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests

zhaohuabing

@zirain @arkodg I think V4_PREFERRED won't work for IPv6 env where the envoy pod only has an IPv6 address.

If V4_PREFERRED is specified, the DNS resolver will first perform a lookup for addresses in the IPv4 family and fallback to a lookup for addresses in the IPv6 family.

zirain · 2024-11-21T02:42:51Z

@zirain @arkodg I think V4_PREFERRED won't work for IPv6 env where the envoy pod only has an IPv6 address.

If V4_PREFERRED is specified, the DNS resolver will first perform a lookup for addresses in the IPv4 family and fallback to a lookup for addresses in the IPv6 family.

tested passed on an IPv6 only cluster.

3b26516

zhaohuabing · 2024-11-21T02:52:58Z

3b26516

I mean "real" IPv6 only where the pod just has an IPv6 address and has no IPv4 address.

Does the pod in this test has an IPv4 address? If it only has an IPv6 address, it shouldn't be able to establish an connection to an IPv4 server.

zirain · 2024-11-21T02:59:50Z

3b26516

I mean "real" IPv6 only where the pod just has an IPv6 address and has no IPv4 address.

Does the pod in this test has an IPv4 address? If it only has an IPv6 address, it shouldn't be able to establish an connection to an IPv4 server.

it's IPv6 only cluster, not IPv6 first.

zhaohuabing · 2024-11-21T04:44:27Z

#4744

I guess if AUTO(IPv6_prefered) didn't work for pod without IPv6 address ( like in #4744), then IPv4_prefered may not work for pod without IPv4 address as well. I may be wrong, will test it myself.

zhaohuabing · 2024-11-21T06:37:54Z

This test verfied that V4_PREFERRED doesn't work with IPv6 only.

#4745SP with a JWK configuration:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
  name: demo-api-jwt
spec:
  jwt:
    providers:
    - name: test
      remoteJWKS:
        uri: https://www.zhaohuabing.com/misc/jwks.json
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: backend

DNS records for IPv4 and IPv6

➜  gateway git:(use/V4_PREFERRED) dig www.zhaohuabing.com     

;; ANSWER SECTION:
www.zhaohuabing.com.	247	IN	A	104.21.60.99
www.zhaohuabing.com.	247	IN	A	172.67.195.133

➜  gateway git:(use/V4_PREFERRED) dig AAAA www.zhaohuabing.com

;; ANSWER SECTION:
www.zhaohuabing.com.	244	IN	AAAA	2606:4700:3037::ac43:c385
www.zhaohuabing.com.	244	IN	AAAA	2606:4700:3034::6815:3c63

Test with IPv6 only, failed to fetch jwks

export IP_FAMILY=ipv6; export IMAGE_PULL_POLICY=IfNotPresent; make create-cluster kube-install-image kube-deploy

Envoy log

[2024-11-21 06:37:38.446][1][warning][jwt] [source/extensions/filters/http/jwt_authn/jwks_async_fetcher.cc:115] Jwks async fetching url=https://www.zhaohuabing.com/misc/jwks.json: failed

The jwks can be downloaded via curl --ipv6

curl --ipv6 -v https://www.zhaohuabing.com/misc/jwks.json
* Host www.zhaohuabing.com:443 was resolved.
* IPv6: 2606:4700:3034::6815:3c63, 2606:4700:3037::ac43:c385
* IPv4: (none)
*   Trying [2606:4700:3034::6815:3c63]:443...
* Connected to www.zhaohuabing.com (2606:4700:3034::6815:3c63) port 443
....
{ "keys":[ {"e":"AQAB","kid":"DHFbpoIUqrY8t2zpA2qXfCmr5VO5ZEr4RzHU_-envvQ","kty":"RSA","n":"xAE7eB6qugXyCAG3yhh7pkDkT65pHymX-P7KfIupjf59vsdo91bSP9C8H07pSAGQO1MV_xFj9VswgsCg4R6otmg5PV2He95lZdHtOcU5DXIg_pbhLdKXbi66GlVeK6ABZOUW3WYtnNHD-91gVuoeJT_DwtGGcp4ignkgXfkiEm4sw-4sfb4qdt5oLbyVpmW6x9cfa7vs2WTfURiCrBoUqgBo_-4WTiULmmHSGZHOjzwa8WtrtOQGsAFjIbno85jp6MnGGGZPYZbDAa_b3y5u-YpW7ypZrvD8BgtKVjgtQgZhLAGezMt0ua3DRrWnKqTZ0BJ_EyxOGuHJrLsn00fnMQ"}]}

Test with IPv4 only, succeeded

export IP_FAMILY=ipv4; export IMAGE_PULL_POLICY=IfNotPresent; make create-cluster kube-install-image kube-deploy

[2024-11-21 06:32:39.285][1][debug][filter] [source/extensions/filters/http/common/jwks_fetcher.cc:92] onSuccess: fetch pubkey [uri = https://www.zhaohuabing.com/misc/jwks.json]: succeeded

zirain · 2024-11-21T07:03:04Z

can you passed the test with IPv4_only on a IPv6 only cluster?

zhaohuabing

LGTM Thanks.

Sorry I didn't notice that the DNSLookupfamily will be changed based on the IPFamily setting.

It would be useful to include a few lines to the IPFamily API docs explaining how IPFamily affects the Envoy DNS resolver.

Signed-off-by: zirain <zirain2009@gmail.com>

api/v1alpha1/envoyproxy_types.go

zhaohuabing · 2024-11-22T04:16:14Z

envoyProxy resource IPFamily is only meant for listener and Envoy Proxy fleet Service spec

BackendRef IPFamily determines cluster setting

Are we planning to introduce another "BackendRef IPFamily" configuration knob? Even though they have different meaning, typically the value of listener IP famaily and the Backend IP family for in-cluster services should be the same.

* use Cluster_V4_PREFERRED Signed-off-by: zirain <zirain2009@gmail.com> * release notes Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com>

* fix: tcp listener is rejected when no route attached (#4681) * fix: tcp listener is rejected when no route attached Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * change cluter name Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix listener connection limit test Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix listener connetcp keepalive test Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix tcp endpoint stats test Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix tcp-route-enable-req-resp-sizes-stats Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix extensionpolicy-tcp-udp-http test Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix lint Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> (cherry picked from commit f99c36c) Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix: remove backendrefs validation (#4705) * remove backendrefs validation Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * add tests Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * add tests Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> Co-authored-by: zirain <zirain2009@gmail.com> (cherry picked from commit 5068698) Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix: translator reports errors for existing clusters and secretes (#4707) * fix: existing clusters and secretes Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix cluster index for SP Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * minor change Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * minor change Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * minor change Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * minor change Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix lint Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * add comment Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * remove index Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix lint Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * xds: always use `::` and `IPv4Compact` for dynamic listener (#4743) * enable IPv4Compact Signed-off-by: zirain <zirain2009@gmail.com> * fix xds test Signed-off-by: zirain <zirain2009@gmail.com> * release-notes Signed-off-by: zirain <zirain2009@gmail.com> * nit Signed-off-by: zirain <zirain2009@gmail.com> * gen Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> (cherry picked from commit 78da42c) Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * Fix: frequent 503 errors when connecting to a Service experiencing high Pod churn (#4754) * Revert "fix: some status updates are discarded by the status updater (#4337)" This reverts commit 14830c7. Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * store update events and process it later Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * rename method Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * add release note Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * xds: use V4_PREFERRED dnsLookupFamily by default (#4745) * use Cluster_V4_PREFERRED Signed-off-by: zirain <zirain2009@gmail.com> * release notes Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> Signed-off-by: zirain <zirain2009@gmail.com> Co-authored-by: zirain <zirain2009@gmail.com>

* use Cluster_V4_PREFERRED Signed-off-by: zirain <zirain2009@gmail.com> * release notes Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com>

) * fix: tcp listener is rejected when no route attached (envoyproxy#4681) * fix: tcp listener is rejected when no route attached Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * change cluter name Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix listener connection limit test Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix listener connetcp keepalive test Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix tcp endpoint stats test Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix tcp-route-enable-req-resp-sizes-stats Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix extensionpolicy-tcp-udp-http test Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix lint Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> (cherry picked from commit f99c36c) Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix: remove backendrefs validation (envoyproxy#4705) * remove backendrefs validation Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * add tests Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * add tests Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> Co-authored-by: zirain <zirain2009@gmail.com> (cherry picked from commit 5068698) Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix: translator reports errors for existing clusters and secretes (envoyproxy#4707) * fix: existing clusters and secretes Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix cluster index for SP Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * minor change Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * minor change Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * minor change Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * minor change Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix lint Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * add comment Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * remove index Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * fix lint Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * xds: always use `::` and `IPv4Compact` for dynamic listener (envoyproxy#4743) * enable IPv4Compact Signed-off-by: zirain <zirain2009@gmail.com> * fix xds test Signed-off-by: zirain <zirain2009@gmail.com> * release-notes Signed-off-by: zirain <zirain2009@gmail.com> * nit Signed-off-by: zirain <zirain2009@gmail.com> * gen Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> (cherry picked from commit 78da42c) Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * Fix: frequent 503 errors when connecting to a Service experiencing high Pod churn (envoyproxy#4754) * Revert "fix: some status updates are discarded by the status updater (envoyproxy#4337)" This reverts commit 14830c7. Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * store update events and process it later Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * rename method Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * add release note Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> * xds: use V4_PREFERRED dnsLookupFamily by default (envoyproxy#4745) * use Cluster_V4_PREFERRED Signed-off-by: zirain <zirain2009@gmail.com> * release notes Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: Huabing Zhao <zhaohuabing@gmail.com> Signed-off-by: zirain <zirain2009@gmail.com> Co-authored-by: zirain <zirain2009@gmail.com>

zirain requested a review from a team as a code owner November 20, 2024 19:56

arkodg previously approved these changes Nov 20, 2024

View reviewed changes

arkodg requested review from a team November 20, 2024 20:07

zhaohuabing requested changes Nov 21, 2024

View reviewed changes

zhaohuabing previously approved these changes Nov 21, 2024

View reviewed changes

zhaohuabing added the cherrypick/release-v1.2.2 label Nov 21, 2024

zirain dismissed stale reviews from zhaohuabing and arkodg via b4bbdef November 21, 2024 07:27

zhaohuabing previously approved these changes Nov 21, 2024

View reviewed changes

zirain dismissed zhaohuabing’s stale review via 0593c9c November 21, 2024 07:38

zhaohuabing previously approved these changes Nov 21, 2024

View reviewed changes

zirain requested a review from arkodg November 21, 2024 08:05

zirain added 2 commits November 21, 2024 17:13

use Cluster_V4_PREFERRED

94d8535

Signed-off-by: zirain <zirain2009@gmail.com>

release notes

1dfd21e

Signed-off-by: zirain <zirain2009@gmail.com>

zirain dismissed zhaohuabing’s stale review via c17c635 November 21, 2024 09:13

zirain force-pushed the use/V4_PREFERRED branch from 0593c9c to c17c635 Compare November 21, 2024 09:13

zhaohuabing previously approved these changes Nov 21, 2024

View reviewed changes

arkodg reviewed Nov 21, 2024

View reviewed changes

api/v1alpha1/envoyproxy_types.go Outdated Show resolved Hide resolved

zirain mentioned this pull request Nov 22, 2024

dnslookupfamily returns ipv6 addresses for external clusters (oidc) #4744

Closed

zirain dismissed zhaohuabing’s stale review via 1dfd21e November 22, 2024 02:06

zirain force-pushed the use/V4_PREFERRED branch from c17c635 to 1dfd21e Compare November 22, 2024 02:06

zirain requested review from arkodg and zhaohuabing November 22, 2024 02:13

zhaohuabing approved these changes Nov 22, 2024

View reviewed changes

arkodg approved these changes Nov 26, 2024

View reviewed changes

arkodg merged commit 0130b77 into envoyproxy:main Nov 26, 2024

zirain deleted the use/V4_PREFERRED branch November 27, 2024 00:19

zhaohuabing mentioned this pull request Nov 27, 2024

[release/v1.2] cherry pick bug fixes and IPv6 to v1.2.2 #4765

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

xds: use V4_PREFERRED dnsLookupFamily by default#4745

xds: use V4_PREFERRED dnsLookupFamily by default#4745
arkodg merged 2 commits intoenvoyproxy:mainfrom
zirain:use/V4_PREFERRED

zirain commented Nov 20, 2024

Uh oh!

zirain commented Nov 20, 2024

Uh oh!

codecov bot commented Nov 20, 2024 •

edited

Loading

Uh oh!

zhaohuabing left a comment

Uh oh!

zirain commented Nov 21, 2024 •

edited

Loading

Uh oh!

zhaohuabing commented Nov 21, 2024

Uh oh!

zirain commented Nov 21, 2024

Uh oh!

zhaohuabing commented Nov 21, 2024 •

edited

Loading

Uh oh!

zhaohuabing commented Nov 21, 2024 •

edited

Loading

Uh oh!

zirain commented Nov 21, 2024

Uh oh!

zhaohuabing left a comment •

edited

Loading

Uh oh!

Uh oh!

zhaohuabing commented Nov 22, 2024 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

zirain commented Nov 20, 2024

Uh oh!

zirain commented Nov 20, 2024

Uh oh!

codecov bot commented Nov 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

zhaohuabing left a comment

Choose a reason for hiding this comment

Uh oh!

zirain commented Nov 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zhaohuabing commented Nov 21, 2024

Uh oh!

zirain commented Nov 21, 2024

Uh oh!

zhaohuabing commented Nov 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zhaohuabing commented Nov 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zirain commented Nov 21, 2024

Uh oh!

zhaohuabing left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

zhaohuabing commented Nov 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Nov 20, 2024 •

edited

Loading

zirain commented Nov 21, 2024 •

edited

Loading

zhaohuabing commented Nov 21, 2024 •

edited

Loading

zhaohuabing commented Nov 21, 2024 •

edited

Loading

zhaohuabing left a comment •

edited

Loading

zhaohuabing commented Nov 22, 2024 •

edited

Loading