Skip to content

Helm job clustermesh-apiserver-generate-certs never created #42045

@FlareSensei

Description

@FlareSensei

Is there an existing issue for this?

  • I have searched the existing issues

Version

equal or higher than v1.18.2 and lower than v1.19.0

What happened?

The job clustermesh-apiserver-generate-certs is never created because of the helm-hooks post-install because the clustermesh-apiserver PODs only get healthy after the certificates are created. Chicken-and-egg problem.

$ cat ./templates/clustermesh-apiserver/tls-cronjob/job.yaml

{{- if and (and .Values.clustermesh.useAPIServer (eq .Values.clustermesh.apiserver.kvstoremesh.kvstoreMode "internal")) .Values.clustermesh.apiserver.tls.auto.enabled (eq .Values.clustermesh.apiserver.tls.auto.method "cronJob") }}
---
apiVersion: batch/v1
kind: Job
metadata:
  name: clustermesh-apiserver-generate-certs
  namespace: {{ include "cilium.namespace" . }}
  labels:
    k8s-app: clustermesh-apiserver-generate-certs
    {{- with .Values.commonLabels }}
    {{- toYaml . | nindent 4 }}
    {{- end }}
    app.kubernetes.io/part-of: cilium
  annotations:
    "helm.sh/hook": post-install,post-upgrade
    {{- with .Values.certgen.annotations.job }}
    {{- toYaml . | nindent 4 }}
    {{- end }}
    {{- with .Values.clustermesh.annotations }}
      {{- toYaml . | nindent 4 }}
    {{- end }}
{{ include "clustermesh-apiserver-generate-certs.job.spec" . }}
{{- end }}

Workaround is to manually create the job:

kubectl -n runtime create job --from=cronjob/clustermesh-apiserver-generate-certs clustermesh-apiserver-generate-certs-manual

How can we reproduce the issue?

Create fresh clustermesh with tls.auto.method=cronJob

Cilium Version

Cilium 1.18.2

Kernel Version

[ec2-user@ip-10-1-182-52 ~]$ uname -a
Linux ip-10-1-182-52.eu-central-1.compute.internal 6.1.150-174.273.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Sep 9 12:21:26 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Kubernetes Version

EKS 1.32

Regression

No response

Sysdump

No response

Relevant log output

Anything else?

No response

Cilium Users Document

  • Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/agentCilium agent related.area/clustermeshRelates to multi-cluster routing functionality in Cilium.area/helmImpacts helm charts and user deployment experiencekind/bugThis is a bug in the Cilium logic.kind/community-reportThis was reported by a user in the Cilium community, eg via Slack.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions