-
Notifications
You must be signed in to change notification settings - Fork 709
Description
Description:
When upgrading from envoy-gateway v1.5.4 to v1.6.0-rc.1, traffic sent to backends through a BackendTrafficPolicy configured with PROXY Protocol v2 fails. The issue occurs both when using BackendTLSPolicy and when testing the auto_sni feature.
In v1.5.4, the combination of BackendTrafficPolicy (with PROXY Protocol v2) and BackendTLSPolicy worked correctly. In v1.6.0-rc.1, this same configuration results in:
- No cluster being defined for the backend service in Envoy's configuration
- PROXY Protocol headers not being correctly included in traffic sent to the backend
Expected behavior: Traffic should be sent with proper PROXY Protocol v2 headers as it did in v1.5.4, and the backend cluster should be properly registered in Envoy's configuration.
Repro steps:
Environment: envoy-gateway v1.5.4 (working configuration)
Configure ClientTrafficPolicy (enable proxy protocol)
enableProxyProtocol: trueConfigure BackendTrafficPolicy with PROXY Protocol v2:
proxyProtocol:
version: V2Configure BackendTLSPolicy:
apiVersion: gateway.networking.k8s.io/v1alpha3
kind: BackendTLSPolicy
metadata:
name: echo-toolbox-tls-policy
spec:
targetRefs:
- group: gateway.envoyproxy.io
kind: Backend
name: active-haproxy
- group: gateway.envoyproxy.io
kind: Backend
name: passive-haproxy
validation:
caCertificateRefs:
- name: internal-domain-root-ca
group: ''
kind: ConfigMap
hostname: echo-toolbox.internal.domain.com- Configure HAProxy frontend with:
bind *:10443 accept-proxy
Result: ✅ Everything works. Connection passes through and Hostname/IP is sent correctly.
Environment: envoy-gateway v1.6.0-rc.1 (failing configuration)
Test 1: Same configuration as v1.5.4
- Same
ClientTrafficPolicy,BackendTrafficPolicy, andBackendTLSPolicyconfiguration - Result: ❌ Getting 503 errors
egctl config envoy-proxy clustershows no cluster forecho-toolboxservice- Envoy logs show error:
[2025-11-05 17:08:31.878][1][warning][config] [source/extensions/config_subscription/grpc/delta_subscription_state.cc:285] delta config for type.googleapis.com/envoy.config.cluster.v3.Cluster rejected: Error adding/updating cluster(s) httproute/envoy-gateway-system/echo-toolbox/rule/0: ALPN configured for cluster httproute/envoy-gateway-system/echo-toolbox/rule/0 which has a non-ALPN transport socket: name: "httproute/envoy-gateway-system/echo-toolbox/rule/0"
- The cluster for the service is not registered
Test 2: Removing BTLSP, keeping PROXY Protocol v2
- Result: ❌ Same error. The cluster for the service is still not registered
Test 3: Removing BTLSP AND removing PROXY Protocol v2 from BackendTrafficPolicy
- Result: ❌ Cluster is now registered, but getting error:
upstream connect error or disconnect/reset before headers. retried and the latest reset reason: remote connection failure, transport failure reason: TLS_error:|33554536:system library:OPENSSL_internal:Connection reset by peer:TLS_error_end
Test 4: Removing BTLSP, removing PROXY Protocol v2, AND disabling HAProxy accept-proxy
- Result: ✅ Cluster registered. Traffic routes correctly. Host IP and Hostname header is forwarded to the backend.
Conclusion: The regression appears when combining BackendTrafficPolicy with PROXY Protocol v2 and BackendTLSPolicy or auto_sni in v1.6.0-rc.1. The same configuration (with BTLSP) works in v1.5.4.
Environment:
- envoy-gateway version: v1.6.0-rc.1 (regression from v1.5.4)
- Features used:
BackendTrafficPolicy(PROXY Protocol v2),BackendTLSPolicy,ClientTrafficPolicyandauto_sni
Notes:
This was first reported through new feature auto_host_sni issue #6901
And a potential solution was proposed to fix this regression: #6901 (comment)
