-
Notifications
You must be signed in to change notification settings - Fork 708
Description
Description:
We are seeing conflict between namespaced BackendTrafficPolicies due to them targeting (incorrectly) HTTPRoutes across namespaces. We are using ReferenceGrants allowing targeting/using services in our primary namespace from the beta namespaces for HTTPRoutes and SecurityPolicy, however the BackendTrafficPolicy, and SecurityPolicy both use targetSelectors on HTTPRoutes which should only allow targeting the same namespace.
https://gateway.envoyproxy.io/docs/concepts/gateway_api_extensions/backend-traffic-policy/#targets
https://gateway.envoyproxy.io/docs/concepts/gateway_api_extensions/security-policy/#targets
Important: A SecurityPolicy can only target resources in the same namespace as the policy itself.
Repro steps:
# namespace: example-web-beta-242
> kubectl get backendtrafficpolicies www-proxy-replacement-beta-dynamic-retry -oyaml
NAME AGE
www-proxy-replacement-beta-dynamic-retry 19m
> kubectl get backendtrafficpolicies www-proxy-replacement-beta-dynamic-retry -oyaml
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
annotations:
argocd.argoproj.io/tracking-id: example-web-beta-242:gateway.envoyproxy.io/BackendTrafficPolicy:example-web-beta-242/www-proxy-replacement-beta-dynamic-retry
name: www-proxy-replacement-beta-dynamic-retry
namespace: example-web-beta-242
spec:
healthCheck:
passive:
baseEjectionTime: 15s
consecutive5XxErrors: 5
consecutiveGatewayErrors: 5
consecutiveLocalOriginFailures: 5
interval: 3s
maxEjectionPercent: 10
splitExternalLocalOriginErrors: false
retry:
numRetries: 2
perRetry:
backOff:
baseInterval: 100ms
maxInterval: 10s
retryOn:
triggers:
- connect-failure
- gateway-error
- refused-stream
- reset
targetSelectors:
- group: gateway.networking.k8s.io
kind: HTTPRoute
matchLabels:
envoy-retry: www-proxy-replacement-beta-dynamic
timeout:
http:
requestTimeout: 30s
tcp:
connectTimeout: 2s
status:
ancestors:
- ancestorRef:
group: gateway.networking.k8s.io
kind: Gateway
name: external-iap
namespace: envoy-gateway-system
conditions:
- lastTransitionTime: "2025-09-04T08:58:59Z"
message: Unable to target HTTPRoute www-proxy-replacement-beta-dynamic-marketing-api-external-iap,
another BackendTrafficPolicy has already attached to it
observedGeneration: 1
reason: Conflicted
status: "False"
type: Accepted
controllerName: gateway.envoyproxy.io/gatewayclass-controller
> kubectl get referencegrant -A
NAMESPACE NAME AGE
example-production example-web-beta-104 9d
example-production example-web-beta-108 9d
example-production example-web-beta-110 9d
example-production example-web-beta-111 9d
example-production example-web-beta-122 9d
example-production example-web-beta-123 9d
example-production example-web-beta-139 9d
example-production example-web-beta-140 9d
example-production example-web-beta-145 9d
example-production example-web-beta-146 9d
example-production example-web-beta-154 9d
example-production example-web-beta-174 9d
example-production example-web-beta-185 8d
example-production example-web-beta-191 7d19h
example-production example-web-beta-193 7d1h
example-production example-web-beta-195 7d
example-production example-web-beta-207 2d21h
example-production example-web-beta-214 47h
example-production example-web-beta-215 47h
example-production example-web-beta-216 47h
example-production example-web-beta-222 42h
example-production example-web-beta-224 25h
example-production example-web-beta-227 24h
example-production example-web-beta-231 23h
example-production example-web-beta-232 23h
example-production example-web-beta-233 23h
example-production example-web-beta-235 22h
example-production example-web-beta-237 20h
example-production example-web-beta-240 18h
example-production example-web-beta-242 115m
example-production example-web-beta-243 53m
example-production example-web-beta-63 9d
example-production example-web-beta-64 9d
example-production example-web-beta-94 9d
example-production example-web-beta-99 9d
> kubectl get referencegrant -n example-production example-web-beta-242 -oyaml
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
annotations:
argocd.argoproj.io/tracking-id: example-web-beta-242:gateway.networking.k8s.io/ReferenceGrant:example-production/example-web-beta-242
name: example-web-beta-242
namespace: example-production
spec:
from:
- group: gateway.envoyproxy.io
kind: SecurityPolicy
namespace: example-web-beta-242
- group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: example-web-beta-242
to:
- group: ""
kind: Service
❯ egctl x status backendtrafficpolicy -A
NAMESPACE NAME ANCESTOR REFERENCE TYPE STATUS REASON
asimov asimov gateway/external-iap Accepted True Accepted
mail-template-editor mail-template-editor gateway/external-iap Accepted True Accepted
example-production fraudulent-listings-api gateway/cluster-internal Accepted True Accepted
example-production kibana-myexample gateway/external-iap Accepted True Accepted
example-production logviewer gateway/external-iap Accepted True Accepted
example-production similar-items-recs-api gateway/cluster-internal Accepted True Accepted
example-production webapi gateway/cluster-internal Accepted True Accepted
example-production www-proxy-replacement-beta-a-retry gateway/cluster-internal Accepted True Accepted
example-production www-proxy-replacement-beta-b-retry gateway/cluster-internal Accepted True Accepted
example-production www-proxy-replacement-beta-c-retry gateway/cluster-internal Accepted True Accepted
example-production www-proxy-replacement-beta-verify-retry gateway/cluster-internal Accepted True Accepted
example-production www-proxy-replacement-canary-retry gateway/cluster-internal Accepted True Accepted
example-production www-proxy-replacement-retry gateway/cluster-internal Accepted True Accepted
example-production you-might-like-recs-api-cluster gateway/cluster-internal Accepted True Accepted
example-web-beta-104 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-108 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-110 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-111 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-122 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-123 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-139 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-140 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-145 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-146 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-154 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-174 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-185 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-191 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-193 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-195 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-207 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-214 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-215 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-216 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-222 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-224 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-227 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-231 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-232 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-233 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-235 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-237 www-proxy-replacement-beta-dynamic-retry gateway/external-iap Accepted False Conflicted
example-web-beta-240 www-proxy-replacement-beta-dynamic-retry gateway/external-iap Accepted False Conflicted
example-web-beta-242 www-proxy-replacement-beta-dynamic-retry gateway/external-iap Accepted False Conflicted
example-web-beta-243 www-proxy-replacement-beta-dynamic-retry gateway/external-iap Accepted False Conflicted
example-web-beta-63 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-64 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-94 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
example-web-beta-99 www-proxy-replacement-beta-dynamic-retry gateway/cluster-internal Accepted False Conflicted
Environment:
Include the environment like gateway version, envoy version and so on.
❯ kubectl version
Client Version: v1.32.2
Kustomize Version: v5.5.0
Server Version: v1.32.6-gke.1060000
❯ egctl version
client: v1.5.0
server:
- Name: envoy-gateway-56f7549fb-52l8m
Namespace: envoy-gateway-system
envoyGatewayVersion: v1.5.0
envoyProxyVersion: distroless-v1.35.0
gatewayAPIVersion: v1.3.1-0.20250527223622-54df0a899c1c
gitCommitID: c2e5b2e118a4d81db698d22d4ff4ebb0211ec8a2
golangVersion: go1.24.6
- Name: envoy-gateway-56f7549fb-thwqw
Namespace: envoy-gateway-system
envoyGatewayVersion: v1.5.0
envoyProxyVersion: distroless-v1.35.0
gatewayAPIVersion: v1.3.1-0.20250527223622-54df0a899c1c
gitCommitID: c2e5b2e118a4d81db698d22d4ff4ebb0211ec8a2
golangVersion: go1.24.6
- Name: envoy-gateway-56f7549fb-zk45c
Namespace: envoy-gateway-system
envoyGatewayVersion: v1.5.0
envoyProxyVersion: distroless-v1.35.0
gatewayAPIVersion: v1.3.1-0.20250527223622-54df0a899c1c
gitCommitID: c2e5b2e118a4d81db698d22d4ff4ebb0211ec8a2
golangVersion: go1.24.6