Skip to content

Duplicated xRoutes in gatewayapi.Resources when attaching xRoute to multiple Gateways #3165

@shuji-2019

Description

@shuji-2019

Description:

When attaching single HTTPRoute to N Gateways, HTTPRoute is added to gatewayapi.Resources N times. It'll lead problems:

  • HTTP Listener 's attachedRoutes is N, not 1.
  • Duplicated ir.HTTPRoutes are added to ir.HTTPListener.
  • Duplicated Endpoints are added to ir.HTTPRoute.Destination. (Envoy will drop duplicated endpoint when handling EDS resource)

When attaching single TCPRoute to N. Gateway, TCPRoute is added to gatewayapi.Resources N times. It'll lead problems:

  • TCP Listener's attachedRoutes is N, not 1, then Listener will be treated as unaccepted. (EG only allow TCP Listener whose attachedRoutes is 1)

Repro steps:

  1. Do quickstart.
  2. Duplicate the only Gateway in quickstart and increase port by 1.
  3. Attach the new Gateway to the only HTTPRoute in quickstart.
  4. Apply the new Gateway and the new HTTPRoute.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: eg-81
spec:
  gatewayClassName: eg
  listeners:
    - name: http
      protocol: HTTP
      port: 81
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: backend
spec:
  parentRefs:
    - name: eg
    - name: eg-81
  hostnames:
    - "www.example.com"
  rules:
    - backendRefs:
        - group: ""
          kind: Service
          name: backend
          port: 3000
          weight: 1
      matches:
        - path:
            type: PathPrefix
            value: /

Environment:

EG: v1.0.1

Logs:

status:
  conditions:
  - lastTransitionTime: "2024-04-10T09:08:19Z"
    message: The Gateway has been scheduled by Envoy Gateway
    observedGeneration: 1
    reason: Accepted
    status: "True"
    type: Accepted
  - lastTransitionTime: "2024-04-10T09:08:19Z"
    message: No addresses have been assigned to the Gateway
    observedGeneration: 1
    reason: AddressNotAssigned
    status: "False"
    type: Programmed
  listeners:
  - attachedRoutes: 2
    conditions:
    - lastTransitionTime: "2024-04-10T09:08:19Z"
      message: Sending translated listener configuration to the data plane
      observedGeneration: 1
      reason: Programmed
      status: "True"
      type: Programmed
    - lastTransitionTime: "2024-04-10T09:08:19Z"
      message: Listener has been successfully translated
      observedGeneration: 1
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2024-04-10T09:08:19Z"
      message: Listener references have been resolved
      observedGeneration: 1
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    name: http
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
    - group: gateway.networking.k8s.io
      kind: GRPCRoute
status:
  conditions:
  - lastTransitionTime: "2024-04-10T09:08:19Z"
    message: The Gateway has been scheduled by Envoy Gateway
    observedGeneration: 1
    reason: Accepted
    status: "True"
    type: Accepted
  - lastTransitionTime: "2024-04-10T09:08:19Z"
    message: No addresses have been assigned to the Gateway
    observedGeneration: 1
    reason: AddressNotAssigned
    status: "False"
    type: Programmed
  listeners:
  - attachedRoutes: 2
    conditions:
    - lastTransitionTime: "2024-04-10T09:08:19Z"
      message: Sending translated listener configuration to the data plane
      observedGeneration: 1
      reason: Programmed
      status: "True"
      type: Programmed
    - lastTransitionTime: "2024-04-10T09:08:19Z"
      message: Listener has been successfully translated
      observedGeneration: 1
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2024-04-10T09:08:19Z"
      message: Listener references have been resolved
      observedGeneration: 1
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    name: http
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
    - group: gateway.networking.k8s.io
      kind: GRPCRoute
2024-04-10T09:08:21.351Z	INFO	gateway-api	runner/runner.go:56	received an update	{"runner": "gateway-api"}
2024-04-10T09:08:21.351Z	INFO	gateway-api	runner/runner.go:103	proxy:
  listeners:
  - address: null
    name: default/eg-81/http
    ports:
    - containerPort: 10081
      name: http
      protocol: HTTP
      servicePort: 81
  metadata:
    labels:
      gateway.envoyproxy.io/owning-gateway-name: eg-81
      gateway.envoyproxy.io/owning-gateway-namespace: default
  name: default/eg-81
	{"runner": "gateway-api", "infra-ir": "default/eg-81"}
2024-04-10T09:08:21.352Z	INFO	gateway-api	runner/runner.go:103	proxy:
  listeners:
  - address: null
    name: default/eg/http
    ports:
    - containerPort: 10080
      name: http
      protocol: HTTP
      servicePort: 80
  metadata:
    labels:
      gateway.envoyproxy.io/owning-gateway-name: eg
      gateway.envoyproxy.io/owning-gateway-namespace: default
  name: default/eg
	{"runner": "gateway-api", "infra-ir": "default/eg"}
2024-04-10T09:08:21.352Z	INFO	gateway-api	runner/runner.go:114	accessLog:
  text:
  - path: /dev/stdout
http:
- address: 0.0.0.0
  hostnames:
  - '*'
  isHTTP2: false
  name: default/eg/http
  path:
    escapedSlashesAction: UnescapeAndRedirect
    mergeSlashes: true
  port: 10080
  routes:
  - backendWeights:
      invalid: 0
      valid: 0
    destination:
      name: httproute/default/backend/rule/0
      settings:
      - addressType: IP
        endpoints:
        - host: 10.250.83.54
          port: 3000
        - host: 10.250.83.54
          port: 3000
        protocol: HTTP
        weight: 1
    hostname: www.example.com
    isHTTP2: false
    name: httproute/default/backend/rule/0/match/0/www_example_com
    pathMatch:
      distinct: false
      name: ""
      prefix: /
  - backendWeights:
      invalid: 0
      valid: 0
    destination:
      name: httproute/default/backend/rule/0
      settings:
      - addressType: IP
        endpoints:
        - host: 10.250.83.54
          port: 3000
        - host: 10.250.83.54
          port: 3000
        protocol: HTTP
        weight: 1
    hostname: www.example.com
    isHTTP2: false
    name: httproute/default/backend/rule/0/match/0/www_example_com
    pathMatch:
      distinct: false
      name: ""
      prefix: /
	{"runner": "gateway-api", "xds-ir": "default/eg"}
2024-04-10T09:08:21.352Z	INFO	gateway-api	runner/runner.go:114	accessLog:
  text:
  - path: /dev/stdout
http:
- address: 0.0.0.0
  hostnames:
  - '*'
  isHTTP2: false
  name: default/eg-81/http
  path:
    escapedSlashesAction: UnescapeAndRedirect
    mergeSlashes: true
  port: 10081
  routes:
  - backendWeights:
      invalid: 0
      valid: 0
    destination:
      name: httproute/default/backend/rule/0
      settings:
      - addressType: IP
        endpoints:
        - host: 10.250.83.54
          port: 3000
        - host: 10.250.83.54
          port: 3000
        protocol: HTTP
        weight: 1
    hostname: www.example.com
    isHTTP2: false
    name: httproute/default/backend/rule/0/match/0/www_example_com
    pathMatch:
      distinct: false
      name: ""
      prefix: /
  - backendWeights:
      invalid: 0
      valid: 0
    destination:
      name: httproute/default/backend/rule/0
      settings:
      - addressType: IP
        endpoints:
        - host: 10.250.83.54
          port: 3000
        - host: 10.250.83.54
          port: 3000
        protocol: HTTP
        weight: 1
    hostname: www.example.com
    isHTTP2: false
    name: httproute/default/backend/rule/0/match/0/www_example_com
    pathMatch:
      distinct: false
      name: ""
      prefix: /
	{"runner": "gateway-api", "xds-ir": "default/eg-81"}

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions