Skip to content

feat: Add EnvoyProxyTemplate for global Envoy Proxy defaults#7698

Merged
arkodg merged 3 commits intoenvoyproxy:mainfrom
PulseInnovations:allow-provide-default-proxy-image
Jan 28, 2026
Merged

feat: Add EnvoyProxyTemplate for global Envoy Proxy defaults#7698
arkodg merged 3 commits intoenvoyproxy:mainfrom
PulseInnovations:allow-provide-default-proxy-image

Conversation

@mgs255
Copy link
Copy Markdown
Contributor

@mgs255 mgs255 commented Dec 8, 2025

Introduce EnvoyProxy default configuration

Add a new envoyProxy field to the EnvoyGateway spec to support cluster-wide shared defaults. This defines a three-level hierarchy for EnvoyProxy configuration (highest to lowest priority):

  1. Gateway-level EnvoyProxy (via Gateway.spec.infrastructure.parametersRef)
  2. GatewayClass-level EnvoyProxy (via GatewayClass.spec.parametersRef)
  3. EnvoyProxy defaults on EnvoyGateway

Currently uses replace semantics where the most specific configuration wins completely. The default spec is almost completely redundant in the sense that it gets completely replaced by any more specific configuration - but this establishes the foundation for a future PR that will add a MergeType to enable merging configurations across levels.

Towards #4764

Signed-off-by: Michael Sommerville msommerville@gmail.com

@mgs255 mgs255 force-pushed the allow-provide-default-proxy-image branch 2 times, most recently from 7da24bf to 02beef5 Compare December 8, 2025 10:56
@mgs255 mgs255 force-pushed the allow-provide-default-proxy-image branch 3 times, most recently from fd2a520 to 0ae8e1d Compare December 10, 2025 17:21
@mgs255 mgs255 marked this pull request as ready for review December 10, 2025 17:26
@mgs255 mgs255 requested a review from a team as a code owner December 10, 2025 17:26
@codecov
Copy link
Copy Markdown

codecov bot commented Dec 10, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.78%. Comparing base (626a08a) to head (fa0a115).
⚠️ Report is 29 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7698      +/-   ##
==========================================
+ Coverage   73.52%   73.78%   +0.26%     
==========================================
  Files         237      237              
  Lines       35653    35767     +114     
==========================================
+ Hits        26214    26392     +178     
+ Misses       7576     7518      -58     
+ Partials     1863     1857       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mgs255 mgs255 force-pushed the allow-provide-default-proxy-image branch from 0ae8e1d to 804ff4b Compare December 15, 2025 07:45
@mgs255 mgs255 marked this pull request as draft December 15, 2025 07:46
@mgs255 mgs255 force-pushed the allow-provide-default-proxy-image branch 4 times, most recently from 23b261c to ff220ab Compare December 16, 2025 18:16
@mgs255 mgs255 marked this pull request as ready for review December 16, 2025 18:22
@mgs255 mgs255 force-pushed the allow-provide-default-proxy-image branch 3 times, most recently from e5283b6 to c198efe Compare December 17, 2025 11:51
@mgs255 mgs255 requested review from arkodg, jukie and zirain December 18, 2025 13:27
@@ -180,8 +180,10 @@ type KubernetesPodSpec struct {
// Volumes that can be mounted by containers belonging to the pod.
// More info: https://kubernetes.io/docs/concepts/storage/volumes
//
// +patchMergeKey=name
// +patchStrategy=merge
Copy link
Copy Markdown
Contributor

@arkodg arkodg Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, can you also add these for ImagePullSecrets , TopologySpreadConstraints, InitContainers and all other slices in EnvoyProxy

"github.com/envoyproxy/gateway/internal/utils"
)

// MergeEnvoyProxyConfigs merges EnvoyProxy configurations using a 3-level hierarchy:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this changes the spec
3. completely replaces 2. today, there is no merge

so 1. would either merge with 3. if it exists or merge with 2. if 3. is absent

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes you are right. The original version only did the merge if the template was present. See below.

// (e.g., env vars, volumes) are combined from both sources.
//
// +optional
EnvoyProxyTemplate *EnvoyProxySpec `json:"envoyProxyTemplate,omitempty"`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we may want create a new struct type and embed EnvoyProxySpec which allows us to add a mergeType or something similar to highlight how this will get merged

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To preserve backward compatibility, added a new EnvoyProxyTemplateSpec struct with a mergeType option - where Replace is the default.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arkodg Would you be able to give this another once over? It would be great to get this in.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for the delay, reviewing rn

@mgs255 mgs255 force-pushed the allow-provide-default-proxy-image branch from dd0a4dc to 67f0b55 Compare December 23, 2025 10:23
zirain
zirain previously approved these changes Dec 30, 2025
@netlify
Copy link
Copy Markdown

netlify bot commented Jan 8, 2026

Deploy Preview for cerulean-figolla-1f9435 canceled.

Name Link
🔨 Latest commit fa0a115
🔍 Latest deploy log https://app.netlify.com/projects/cerulean-figolla-1f9435/deploys/69777b652382420008adb41e

// with or replace template definitions based on the mergeType.
//
// +optional
EnvoyProxyTemplate *EnvoyProxyTemplateSpec `json:"envoyProxyTemplate,omitempty"`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be EnvoyProxyTemplate or EnvoyProxy
@envoyproxy/gateway-maintainers

(lets time box this to a week to avoid more delay for this PR)

//
// +optional
// +kubebuilder:default=Replace
MergeType *EnvoyProxyTemplateMergeType `json:"mergeType,omitempty"`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the BackendTrafficPolicy case, the child policy specifies the MergeType and defaults to Replace, right now in this case the parent has this setting, should we move this to the EnvoyProxy resource ?
@envoyproxy/gateway-maintainers

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's some dicussion in the past here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We chatted about this in the community meeting. I'm a +1 on moving MergeType into EnvoyProxy and using that in EnvoyGateway too so we can define the merge behavior in all layers.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah piggybacking based on what Isaac said, we can split this work into 2 parts/PRs

  • Introduce EnvoyProxy *EnvoyProxySpec in EnvoyGateway and use the same replace semantics we have for deciding what EnvoyProxy wins
  • Later introduce MergeType into EnvoyProxySpec and deal with multi level merge

sorry for the churn @mgs255, hope this adds more clarity

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arkodg Yes, sounds reasonable but does that work in practice?

The way I've installed this in our environments, we have all of the configuration defined centrally in the helm values file. There is large envoyProxyTemplate e.g:

        # Lots above....
        rateLimitHpa:
          minReplicas: 2
          maxReplicas: 10
          metrics:
            - type: Resource
              resource:
                name: cpu
                target:
                  type: Utilization
                  averageUtilization: 60
        envoyProxyTemplate:
          mergeType: StrategicMerge
          spec:
            mergeGateways: true
            logging:
              level:
                default: info
                config: info
                upstream: info
                jwt: info
                oauth2: info
                http: info
                filter: info
                router: info
                connection: info
            provider:
              kubernetes:
                envoyHpa:
                  minReplicas: 2
                  maxReplicas: 10
                  metrics:
                    - type: Resource
                      resource:
                        name: cpu
                        target:
                          type: Utilization
                          averageUtilization: 60
       # Lots more below....
           ...

We also a minimal EnvoyProxy object in each target cluster using the following template in a different helm chart:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: {{ .Values.proxyName }}
  namespace: {{ .Release.Namespace }}
  labels: {{- include "pulse-envoy-aws-gateway.labels" . | nindent 4 }}
# Ideally we would not set any properties here and just let the template override everything - unfortunately
# a complete empty spec (even {} or null) appears invalid
spec:
  logging:
    level:
      default: {{ .Values.logging.level.proxy }}

If we only supported the replace option initially then the most specific EnvoyProxy would completely replace the values from the template, right?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah just suggesting the 2 step approach to increase the velocity of code review and getting something in, but agree when merge lands, thats when your use cases and more powerful use cases will be unlocked

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @mgs255 would be great if we can get part 1 into v1.7.0-rc.1 (deadline is 01/28)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok - re-done based on those comments. I will work on some further changes to also add the mergeType and start testing this in our environments.

@arkodg
Copy link
Copy Markdown
Contributor

arkodg commented Jan 11, 2026

hey have some open questions here @mgs255, will raise these in the community meeting next week and get them resolved

@mgs255 mgs255 force-pushed the allow-provide-default-proxy-image branch from f709f2e to 6adea4a Compare January 22, 2026 18:14
Add a new envoyProxy field to the EnvoyGateway spec to support
cluster-wide shared defaults. This defines a three-level hierarchy
for EnvoyProxy configuration (highest to lowest priority):

  1. Gateway-level EnvoyProxy (via Gateway.spec.infrastructure.parametersRef)
  2. GatewayClass-level EnvoyProxy (via GatewayClass.spec.parametersRef)
  3. EnvoyProxy defaults on EnvoyGateway

Currently uses replace semantics where the most specific configuration
wins completely. The default spec is almost completely redundant in the
sense that it gets completely replaced by any more specific configuration
- but this establishes the foundation for a future PR that will add
a MergeType to enable merging configurations across levels.

Towards envoyproxy#4764

Signed-off-by: Michael Sommerville <msommerville@gmail.com>
@mgs255 mgs255 force-pushed the allow-provide-default-proxy-image branch from 6adea4a to 5d64148 Compare January 22, 2026 18:16
@mgs255 mgs255 requested review from arkodg and zirain January 22, 2026 18:20
Signed-off-by: Michael Sommerville <msommerville@gmail.com>
zirain
zirain previously approved these changes Jan 24, 2026
Copy link
Copy Markdown
Member

@zirain zirain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, it would be good to add some e2e for this with following PR.

e.g. add a dummy/valid EnvoyProxy in test/config/envoy-gateaway-config/default.yaml and the one referenced by GatewayClass will override it, e2e should all pass.

@arkodg
Copy link
Copy Markdown
Contributor

arkodg commented Jan 24, 2026

thanks, LGTM, +1 to zirain's comments around add some tests

Signed-off-by: Michael Sommerville <msommerville@gmail.com>
@mgs255 mgs255 force-pushed the allow-provide-default-proxy-image branch from e246274 to fa0a115 Compare January 26, 2026 14:34
Copy link
Copy Markdown
Member

@zirain zirain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@zirain
Copy link
Copy Markdown
Member

zirain commented Jan 28, 2026

One thing could be imrpove in the next is adding another test case by adding a new GC and it shouldn’t be provision correctly because of the invalid image.

@arkodg arkodg merged commit cf54ff9 into envoyproxy:main Jan 28, 2026
51 of 60 checks passed
@zirain
Copy link
Copy Markdown
Member

zirain commented Jan 28, 2026

no sure what happen, but this cause e2e always failed on dual+default.

@mgs255
Copy link
Copy Markdown
Contributor Author

mgs255 commented Jan 28, 2026

That is really odd. It must be related to the addition in contexts.go. I'll open a new branch removing that section.

I'm having a real problem trying to run the e2e tests locally on Mac though so it is a pain to debug this.

@arkodg @zirain I've pushed a branch which should completely disable that defaulting behaviour for now.

#8103

SadmiB pushed a commit to SadmiB/gateway that referenced this pull request Jan 30, 2026
…oxy#7698)

* Introduce EnvoyProxy default configuration

Add a new envoyProxy field to the EnvoyGateway spec to support
cluster-wide shared defaults. This defines a three-level hierarchy
for EnvoyProxy configuration (highest to lowest priority):

  1. Gateway-level EnvoyProxy (via Gateway.spec.infrastructure.parametersRef)
  2. GatewayClass-level EnvoyProxy (via GatewayClass.spec.parametersRef)
  3. EnvoyProxy defaults on EnvoyGateway

Currently uses replace semantics where the most specific configuration
wins completely. The default spec is almost completely redundant in the
sense that it gets completely replaced by any more specific configuration
- but this establishes the foundation for a future PR that will add
a MergeType to enable merging configurations across levels.

Towards envoyproxy#4764

Signed-off-by: Michael Sommerville <msommerville@gmail.com>
Signed-off-by: Sadmi Bouhafs <sadmibouhafs@gmail.com>
@ion-elgreco
Copy link
Copy Markdown
Contributor

Just wanted to say thanks for adding this! :) It simplified all my deployments a lot 🫡

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants