Skip to content

cilium-operator accepts, but fails to create clusterip service for gateway #38852

@bittrance

Description

@bittrance

Is there an existing issue for this?

  • I have searched the existing issues

Version

equal or higher than v1.17.2 and lower than v1.18.0

What happened?

In a Kind cluster, using Cilium 1.18.0.pre.0 for CNI, a Gateway with a CiliumGatewayClassConfig with spec.service.type: ClusterIP is accepted, but it fails to create the corresponding service. Operator error:

time=2025-04-09T19:32:26Z level=error msg="Reconciler error" module=operator.operator-controlplane.leader-lifecycle.controller-runtime controller=gateway controllerGroup=gateway.networking.k8s.io controllerKind=Gateway Gateway.name=hello-upstream Gateway.namespace=waypoint-test namespace=waypoint-test name=hello-upstream reconcileID=57a45317-c807-4282-b73c-ba45dd259af7 error="Service "cilium-gateway-hello-upstream" is invalid: spec.externalTrafficPolicy: Invalid value: "Cluster": may only be set for externally-accessible services"

I would have expected the same behavior as a Service resource: I set type: ClusterIP and avoid setting externalTrafficPolicy to win. The way I read the Cilium code, there is currently no way to get that state. Alternatively, if such a setup is not supported, I would have expected the operator to refuse resource creation.

(I am aware that this is a bit bleeding edge given that "internal" Gateway resources are not really finalized. See e.g. kubernetes-sigs/gateway-api#3608.)

How can we reproduce the issue?

Kind config:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
  - role: worker
  - role: worker
  - role: worker
networking:
  disableDefaultCNI: true
  kubeProxyMode: none

Installation:

kind create cluster --config ./gateway-api-cilium/kind-cluster-no-cni.yaml
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/experimental-install.yaml
helm upgrade --install cilium cilium/cilium \
  --version 1.18.0.pre.0 \
  --namespace kube-system \
  --set image.pullPolicy=IfNotPresent \
  --set ipam.mode=kubernetes \
  --set gatewayAPI.enabled=true \
  --set nodePort.enabled=true \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=kind-control-plane \
  --set k8sServicePort=6443

Example manifest:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: cilium-internal
spec:
  controllerName: io.cilium/gateway-controller
  description: The default Cilium GatewayClass
  parametersRef:
    group: cilium.io
    kind: CiliumGatewayClassConfig
    name: cilium-internal
---
apiVersion: cilium.io/v2alpha1
kind: CiliumGatewayClassConfig
metadata:
  name: cilium-internal
spec:
  service:
    type: ClusterIP
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: cilium-internal
spec:
  gatewayClassName: cilium-internal
  listeners:
    - name: http
      protocol: HTTP
      port: 8082

Cilium Version

cilium-cli: v0.18.2 compiled with go1.24.0 on linux/amd64
cilium image (default): v1.17.0
cilium image (stable): v1.17.2
cilium image (running): 1.18.0-pre.0

Kernel Version

Linux xxx 6.8.0-53-generic #55-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan 17 15:37:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Kubernetes Version

Client Version: v1.31.2
Kustomize Version: v5.4.2
Server Version: v1.31.0
Kind: 1.24

Regression

Config classes were added in 1.18.0, so brand new.

Sysdump

No response

Relevant log output

Anything else?

No response

Cilium Users Document

  • Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/agentCilium agent related.feature/k8s-gateway-apikind/bugThis is a bug in the Cilium logic.kind/community-reportThis was reported by a user in the Cilium community, eg via Slack.staleThe stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions