Is there an existing issue for this?
What happened?
Description
The operator rejects a labelmap relabel action that was OK in the past. Afaik the labelmap is valid.
Steps to Reproduce
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: xxx
spec:
endpoints:
- interval: 2m
metricRelabelings:
- action: labelmap
regex: ^cluster$
replacement: exported_${1}
path: /metrics
port: http-9255
relabelings: []
jobLabel: app.kubernetes.io/name
Expected Result
To be valid
Actual Result
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning InvalidConfiguration 51m prometheus-controller "xxx" was rejected due to invalid configuration: endpoints[0]: metricRelabelConfigs: [0]: "exported_${1}" is invalid 'replacement' for labelmap action
### Prometheus Operator Version
```shell
Labels: app.kubernetes.io/component=controller
app.kubernetes.io/name=prometheus-operator
app.kubernetes.io/version=0.88.0
Kubernetes Version
serverVersion:
buildDate: "2025-10-13T04:22:26Z"
compiler: gc
emulationMajor: "1"
emulationMinor: "33"
gitCommit: a53c2ee3f2e9859ad8413e216edd44aed40e011d
gitTreeState: clean
gitVersion: v1.33.5-gke.1308000
goVersion: go1.24.6 X:boringcrypto
Kubernetes Cluster Type
GKE
How did you deploy Prometheus-Operator?
prometheus-operator/kube-prometheus
Manifests
prometheus-operator log output
ts=2026-01-27T08:33:18.435109607Z level=warn caller=/workspace/pkg/prometheus/resource_selector.go:152 msg="skipping object" component=prometheus-controller key=metrics/p01-stg-ew4-003 kind=ServiceMonitor error="endpoints[0]: metricRelabelConfigs: [0]: \"exported_${1}\" is invalid 'replacement' for labelmap action" object=xxxAnything else?
The issue seems to be in this:
if action == string(relabel.LabelMap) && (rc.Replacement != nil) && !lcv.isValidLabelName(*rc.Replacement) {
return fmt.Errorf("%q is invalid 'replacement' for %s action", *rc.Replacement, rc.Action)
}
$1 is not a valid label name until after regex substitution.
I wrote a simple test case (using this example):
diff --git a/pkg/prometheus/validation/validator_test.go b/pkg/prometheus/validation/validator_test.go
index 84c71dad1..d533b2e4c 100644
--- a/pkg/prometheus/validation/validator_test.go
+++ b/pkg/prometheus/validation/validator_test.go
@@ -164,6 +164,16 @@ func TestValidateRelabelConfig(t *testing.T) {
},
prometheus: defaultPrometheusSpec,
},
+ // Test valid labelmap relabel config with regex replacement specified
+ {
+ scenario: "valid labelmap config with replacement",
+ relabelConfig: monitoringv1.RelabelConfig{
+ Action: "labelmap",
+ Regex: "__meta_kubernetes_(.*)",
+ Replacement: ptr.To("k8s_${1}"),
+ },
+ prometheus: defaultPrometheusSpec,
+ },
$ go test pkg/prometheus/validation/*
--- FAIL: TestValidateRelabelConfig (0.00s)
--- FAIL: TestValidateRelabelConfig/valid_labelmap_config_with_replacement#01 (0.00s)
validator_test.go:408:
Error Trace: /Users/bsmit/repos/github/prometheus-operator/prometheus-operator/pkg/prometheus/validation/validator_test.go:408
Error: Received unexpected error:
"k8s_${1}" is invalid 'replacement' for labelmap action
Test: TestValidateRelabelConfig/valid_labelmap_config_with_replacement#01
FAIL
prometheus/prometheus#17370 this issue has a similar issue, but on Prometheus' side. I think the fix is the same here, but applying to the code doesn't seem like a simple fix
Is there an existing issue for this?
What happened?
Description
The operator rejects a labelmap relabel action that was OK in the past. Afaik the labelmap is valid.
Steps to Reproduce
Expected Result
To be valid
Actual Result
Kubernetes Version
Kubernetes Cluster Type
GKE
How did you deploy Prometheus-Operator?
prometheus-operator/kube-prometheus
Manifests
prometheus-operator log output
Anything else?
The issue seems to be in this:
$1is not a valid label name until after regex substitution.I wrote a simple test case (using this example):
prometheus/prometheus#17370 this issue has a similar issue, but on Prometheus' side. I think the fix is the same here, but applying to the code doesn't seem like a simple fix