Skip to content

BackendTrafficPolicy with ProxyProtocolV2 enabled with TLS Backend now fails to configure the envoy cluster. #7438

@EM-Savard

Description

@EM-Savard

Description:
When upgrading from envoy-gateway v1.5.4 to v1.6.0-rc.1, traffic sent to backends through a BackendTrafficPolicy configured with PROXY Protocol v2 fails. The issue occurs both when using BackendTLSPolicy and when testing the auto_sni feature.
In v1.5.4, the combination of BackendTrafficPolicy (with PROXY Protocol v2) and BackendTLSPolicy worked correctly. In v1.6.0-rc.1, this same configuration results in:

  1. No cluster being defined for the backend service in Envoy's configuration
  2. PROXY Protocol headers not being correctly included in traffic sent to the backend

Expected behavior: Traffic should be sent with proper PROXY Protocol v2 headers as it did in v1.5.4, and the backend cluster should be properly registered in Envoy's configuration.

Repro steps:
Environment: envoy-gateway v1.5.4 (working configuration)

Configure ClientTrafficPolicy (enable proxy protocol)

  enableProxyProtocol: true

Configure BackendTrafficPolicy with PROXY Protocol v2:

  proxyProtocol:
    version: V2

Configure BackendTLSPolicy:

apiVersion: gateway.networking.k8s.io/v1alpha3
kind: BackendTLSPolicy
metadata:
  name: echo-toolbox-tls-policy
spec:
  targetRefs:
    - group: gateway.envoyproxy.io
      kind: Backend
      name: active-haproxy
    - group: gateway.envoyproxy.io
      kind: Backend
      name: passive-haproxy
  validation:
    caCertificateRefs:
    - name: internal-domain-root-ca
      group: ''
      kind: ConfigMap
    hostname: echo-toolbox.internal.domain.com
  1. Configure HAProxy frontend with:
bind *:10443 accept-proxy

Result: ✅ Everything works. Connection passes through and Hostname/IP is sent correctly.


Environment: envoy-gateway v1.6.0-rc.1 (failing configuration)

Test 1: Same configuration as v1.5.4

  • Same ClientTrafficPolicy, BackendTrafficPolicy, and BackendTLSPolicy configuration
  • Result: ❌ Getting 503 errors
  • egctl config envoy-proxy cluster shows no cluster for echo-toolbox service
  • Envoy logs show error:
[2025-11-05 17:08:31.878][1][warning][config] [source/extensions/config_subscription/grpc/delta_subscription_state.cc:285] delta config for type.googleapis.com/envoy.config.cluster.v3.Cluster rejected: Error adding/updating cluster(s) httproute/envoy-gateway-system/echo-toolbox/rule/0: ALPN configured for cluster httproute/envoy-gateway-system/echo-toolbox/rule/0 which has a non-ALPN transport socket: name: "httproute/envoy-gateway-system/echo-toolbox/rule/0"
  • The cluster for the service is not registered

Test 2: Removing BTLSP, keeping PROXY Protocol v2

  • Result: ❌ Same error. The cluster for the service is still not registered

Test 3: Removing BTLSP AND removing PROXY Protocol v2 from BackendTrafficPolicy

  • Result: ❌ Cluster is now registered, but getting error: upstream connect error or disconnect/reset before headers. retried and the latest reset reason: remote connection failure, transport failure reason: TLS_error:|33554536:system library:OPENSSL_internal:Connection reset by peer:TLS_error_end

Test 4: Removing BTLSP, removing PROXY Protocol v2, AND disabling HAProxy accept-proxy

  • Result: ✅ Cluster registered. Traffic routes correctly. Host IP and Hostname header is forwarded to the backend.

Conclusion: The regression appears when combining BackendTrafficPolicy with PROXY Protocol v2 and BackendTLSPolicy or auto_sni in v1.6.0-rc.1. The same configuration (with BTLSP) works in v1.5.4.

Environment:

  • envoy-gateway version: v1.6.0-rc.1 (regression from v1.5.4)
  • Features used: BackendTrafficPolicy (PROXY Protocol v2), BackendTLSPolicy, ClientTrafficPolicy and auto_sni

Notes:
This was first reported through new feature auto_host_sni issue #6901

And a potential solution was proposed to fix this regression: #6901 (comment)

Ref:
manifests.yaml
Image
envoy-cluster-empty-configuration.json

@AL-Mathieu

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions