-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Closed
Labels
Description
Is there an existing issue for this?
- I have searched the existing issues
What happened?
Description
Recently the TestGatedFeatures/ServiceMonitorStatusWithMultipleWorkloads test has been failing from time to time.
From a first investigation, there's a race condition when a configuration resource is selected by 2 workloads and their reconciliation loops try to add their respective binding at the same time:
prometheus-operator/pkg/operator/config_resource.go
Lines 289 to 298 in 772eefb
| // Initialize the workload bindings. | |
| return json.Marshal(patch{ | |
| patchOperation{ | |
| Op: "add", | |
| Path: "/status", | |
| Value: monitoringv1.ConfigResourceStatus{ | |
| Bindings: []monitoringv1.WorkloadBinding{binding}, | |
| }, | |
| }, | |
| }) |
If the patch operations happen simultaneously, the last one wins.
Prometheus Operator Version
mainKubernetes Version
N/AKubernetes Cluster Type
kind
How did you deploy Prometheus-Operator?
yaml manifests
Manifests
prometheus-operator log output
N/AAnything else?
No response
Reactions are currently unavailable