Codecov Report

Attention: Patch coverage is 69.54545% with 67 lines in your changes missing coverage. Please review.

Project coverage is 65.33%. Comparing base (096cb8d) to head (b677236).
Report is 43 commits behind head on main.

@@            Coverage Diff             @@
##             main    #5352      +/-   ##
==========================================
+ Coverage   65.19%   65.33%   +0.13%     
==========================================
  Files         214      220       +6     
  Lines       34321    35180     +859     
==========================================
+ Hits        22377    22986     +609     
- Misses      10591    10777     +186     
- Partials     1353     1417      +64     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Still WIP but this example "works". A challenge is the way that Envoy internally tries to keep an overall equal request level between other Proxy instances so I think we'll need to add the EnvoyProxy service as an EDS cluster in bootstrap and then have the endpoints populated afterwards.

@jukie Hi! I am considering using this feature for my edge-proxy envoy setup and I was wondering if you could give me some clarifications on how this feature's intent and perhaps some thoughts on your implementation.
As far as I understand envoy developers intent - this feature is meant to be tracking ratio of both the callee and caller service (clusters) local workload to the total workload. And the result would be somewhat equal request rate to the all of upstream service instances in all of the localities. Your implementation considers Envoy itself as a originating cluster with the static size of 1. Isn't that the reason that this setup doesn't achieve the goal of leaving of requests in the local zone? Because from envoy's point of view local zone is the only zone requests are coming from and that's why it would try to spread it equally between all of the zones. You've mentioned adding the EnvoyProxy service as an EDS cluster in bootstrap. Would you mind elaborating on how that would tackle the described problem? And thank you for you work!

Because for now my thoughts are that zone-aware routing does not quite align with the edge proxy setup :)

@AlexKrudu , @jukie is working in upstream to change the behavior for ingress envoyproxy/envoy#38756, which can be leveraged here

Thanks @arkodg! But with that force_locality_direct_routing feature enabled, wouldn't the zone-aware rouing feature equivalent to just setting priority P(0) for the endpoints in the same zone as the envoy? And P(1) for the rest for instance

@AlexKrudu we're trying to achieve zone precedence/priority w/o explicitly using the priority feature, and adding another level of precedence within priority (0)

you could achieve the same with priority, but you'd have to build a different CDS + EDS for every envoy in every zone which is non trivial + requires more resources like memory

/retest

Is there any preference for handling EnvoyProxy RBAC? Will need permissions to get nodes in the cluster for zone discovery

• A: Add Role/RoleBinding management permissions to EG
• B: All EnvoyProxy deployments use the same predefined ServiceAccount and RBAC is setup via helm
• C: Predefined Role/RoleBinding via helm chart with a ref target of some-sa-prefix-*

hey @jukie vote for B
we already create an SA per fleet

Hey @arkodg, this is ready for review now!

/retest

/retest

Because I've added multiple nodes to the E2E tests the UDS pods need to include affinity for the proxy pods

/retest

/retest

/retest

/retest