When there are multiple Gateways having listeners with same port and `mergeGateway` enabled, using CORS for each `Gateway` separately, only the first Gateway works.

*Description*:
When there are multiple Gateways having listeners with same port and `mergeGateway` enabled, using CORS for each `Gateway` separately, only to the first Gateway will work.

*Repro steps*:
Install the Gateway API CRDs and Envoy Gateway
```bash
helm install eg oci://docker.io/envoyproxy/gateway-helm --version v0.0.0-latest -n envoy-gateway-system --create-namespace
```
Use aill-in-one yaml described below:
```bash
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: custom-proxy-config
    namespace: envoy-gateway-system

---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: custom-proxy-config
  namespace: envoy-gateway-system
spec:
  mergeGateways: true 
---
apiVersion: v1
kind: Namespace
metadata:
  name: sealos-system
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: eg
  namespace: sealos-system
spec:
  gatewayClassName: eg
  listeners:
    - hostname: "*.192.168.0.15.nip.io"
      name: http
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: All
---
apiVersion: v1
kind: Namespace
metadata:
  name: ns-oehe
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: bdkzlmibsivuiqav
  namespace: ns-oehe
spec:
  replicas: 1
  selector:
    matchLabels:
      app: bdkzlmibsivuiqav
  template:
    metadata:
      labels:
        app: bdkzlmibsivuiqav
    spec:
      containers:
      - image: nginx:1.25
        name: nginx
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        resources:
          limits:
            cpu: 100m
            memory: 64Mi
          requests:
            cpu: 10m
            memory: 16Mi
---
apiVersion: v1
kind: Service
metadata:
  name: bdkzlmibsivuiqav
  namespace: ns-oehe
spec:
  clusterIP: "10.96.0.2"
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: bdkzlmibsivuiqav
  type: ClusterIP
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: bdkzlmibsivuiqav
  namespace: ns-oehe
spec:
  hostnames:
  - ntjxuedx.192.168.0.15.nip.io
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: eg
    namespace: sealos-system
    sectionName: http
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: bdkzlmibsivuiqav
      port: 80
      weight: 1
    matches:
    - path:
        type: PathPrefix
        value: /
---
apiVersion: v1
kind: Namespace
metadata:
  name: ns-vrpg
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mfqjpuycbgjrtdww
  namespace: ns-vrpg
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mfqjpuycbgjrtdww
  template:
    metadata:
      labels:
        app: mfqjpuycbgjrtdww
    spec:
      containers:
      - image: nginx:1.25
        name: nginx
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        resources:
          limits:
            cpu: 100m
            memory: 64Mi
          requests:
            cpu: 10m
            memory: 16Mi
---
apiVersion: v1
kind: Service
metadata:
  name: mfqjpuycbgjrtdww
  namespace: ns-vrpg
spec:
  clusterIP: "10.96.0.3"
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: mfqjpuycbgjrtdww
  type: ClusterIP
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: mfqjpuycbgjrtdww
  namespace: ns-vrpg
spec:
  gatewayClassName: eg
  listeners:
  - hostname: qccbahgo.qccbahgo
    name: http
    port: 80
    protocol: HTTP
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: mfqjpuycbgjrtdww
  namespace: ns-vrpg
spec:
  hostnames:
  - qccbahgo.qccbahgo
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: mfqjpuycbgjrtdww
    namespace: ns-vrpg
    sectionName: http
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: mfqjpuycbgjrtdww
      port: 80
      weight: 1
    matches:
    - path:
        type: PathPrefix
        value: /
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
  name: cors-example
  namespace: ns-vrpg
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: mfqjpuycbgjrtdww
  cors:
    allowOrigins:
    - "http://*.foo.com"
    allowMethods:
    - PUT
    - GET
    - POST
    - DELETE
    - PATCH
    - OPTIONS
    maxAge: 600s
    allowCredentials: true
```

Get the name of the Envoy service created the by the Gateway, because mergeGateway is enabled, the way to obtain the Envoy service differs from the quickstart. 
```bash
export ENVOY_SERVICE=$(kubectl get svc -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gatewayclass=eg -o jsonpath='{.items[0].metadata.name}')
```
Port forward to the Envoy service
```bash
kubectl -n envoy-gateway-system port-forward service/${ENVOY_SERVICE} 8888:80 &
```
Verify that the CORS headers are present in the response of the OPTIONS request from http://www.foo.com, and the host is qccbahgo.qccbahgo

```bash
curl -H "Origin: http://www.foo.com" \
  -H "Host: qccbahgo.qccbahgo" \
  -H "Access-Control-Request-Method: GET" \
  -X OPTIONS -v -s \
  http://localhost:8888/
```
The following is the response, indicating that the request was not allowed, which is not reasonable.

```bash
...
< HTTP/1.1 405 Method Not Allowed
< server: nginx/1.25.4
...
```
In this case, I applied the SecurityPolicy to the second Gateway, but the SecurityPolicy did not work, although when I checked the status of the SecurityPolicy, it showed as Accepted.

Furthermore, when I applied the SecurityPolicy only to the first Gateway, it works. The steps are as follows: 

1. Delete the existing SecurityPolicy
```bash
kubectl delete SecurityPolicy cors-example -n ns-vrpg
```
2. Apply the new SecurityPolicy to the first Gateway
```bash
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
  name: cors-example
  namespace: sealos-system
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: eg
  cors:
    allowOrigins:
    - "http://*.foo.com"
    allowMethods:
    - PUT
    - GET
    - POST
    - DELETE
    - PATCH
    - OPTIONS
    maxAge: 600s
    allowCredentials: true
```
3. Verify that the CORS headers are present in the response of the OPTIONS request from http://www.foo.com, and the host is ntjxuedx.192.168.0.15.nip.io

```bash
curl -H "Origin: http://www.foo.com" \
  -H "Host: ntjxuedx.192.168.0.15.nip.io" \
  -H "Access-Control-Request-Method: GET" \
  -X OPTIONS -v -s \
  http://localhost:8888/
```
The following is the response,  indicating that the SecurityPolicy works.

```bash
< HTTP/1.1 200 OK
< access-control-allow-origin: http://www.foo.com
< access-control-allow-credentials: true
< access-control-allow-methods: PUT, GET, POST, DELETE, PATCH, OPTIONS
< access-control-max-age: 600
```
---

So why does the SecurityPolicy behave differently when applied to different gateways? I tried to find out the reason. I used egctl to analyze the differences in applying the SecurityPolicy to the first Gateway and the second Gateway separately. 

- When applying the SecurityPolicy to the first one, compared to the xds without the SecurityPolicy, as shown in the diagram, it adds an HTTP filter to the listener and adds the corresponding filter to the RouteConfiguration.

  - to Listener
  <div align=right>
  <img width="910" alt="image" src="https://github.com/envoyproxy/gateway/assets/60498292/13a3abdd-5e1f-4760-8ae2-01f2275d8107">
  </div>

  - to RouteConfiguration
   <div align=right>
  <img width="938" alt="image" src="https://github.com/envoyproxy/gateway/assets/60498292/402d26a7-ba20-437f-9585-60eb0ffea760">
  </div>
- When applying the SecurityPolicy to the second one, compared to the xds without the SecurityPolicy, as shown in the diagram, it only adds the corresponding filter to the RouteConfiguration.

  - to RouteConfiguration

  <img width="939" alt="image" src="https://github.com/envoyproxy/gateway/assets/60498292/094eb9bb-3ea7-420e-b57e-ce271a2839cf">

So, I think this is the reason why the SecurityPolicy doesn't work when applied to the second Gateway. It lacks the HTTPFilter that should be added to the Listener. 

---

So, why does it lack?  Is it because the listener to which the HTTP filter should be added cannot be found or something? I think this is the key to solving the problem.





Provide feedback

Saved searches

Use saved searches to filter your results more quickly

When there are multiple Gateways having listeners with same port and `mergeGateway` enabled, using CORS for each `Gateway` separately, only the first Gateway works. #2742

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

When there are multiple Gateways having listeners with same port and mergeGateway enabled, using CORS for each Gateway separately, only the first Gateway works. #2742

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

When there are multiple Gateways having listeners with same port and `mergeGateway` enabled, using CORS for each `Gateway` separately, only the first Gateway works. #2742