Skip to content

Prometheus operator fails to emit events after upgrade to v0.86.0 (events.k8s.io forbidden) #8076

@Pedro-Sant0s

Description

@Pedro-Sant0s

Is there an existing issue for this?

  • I have searched the existing issues

What happened?

Description

After upgrading to prometheus-operator v0.86.0 (via kube-prometheus-stack), the operator fails to emit Kubernetes Events when reconciling invalid resources such as AlertmanagerConfig.
It seems that the operator now uses the events.k8s.io/v1 API to emit events, but the default RBAC configuration still only includes permissions for the legacy core API apiGroups: [""], as documented here: https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/platform/rbac.md
As a result, the operator logs the following error:

level=error caller=/go/pkg/mod/k8s.io/client-go@v0.34.1/tools/events/event_broadcaster.go:270
msg="Server rejected event (will not retry!)"
err="events.events.k8s.io is forbidden: User \"system:serviceaccount:monitoring-system:kps-operator\" cannot create resource

Steps to Reproduce

Expected Result

Actual Result

Prometheus Operator Version

v0.86.0

Kubernetes Version

v1.33.4

Kubernetes Cluster Type

kind

How did you deploy Prometheus-Operator?

helm chart:prometheus-community/kube-prometheus-stack

Manifests

prometheus-operator log output

evel=error caller=/go/pkg/mod/k8s.io/client-go@v0.34.1/tools/events/event_broadcaster.go:270
msg="Server rejected event (will not retry!)"
err="events.events.k8s.io is forbidden: User \"system:serviceaccount:monitoring-system:kps-operator\" cannot create resource

Anything else?

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions