-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Description
Is there an existing issue for this?
- I have searched the existing issues
What happened?
Description
In prometheus-operator v0.88 an additional validation was introduced for Slack action URLs. After upgrading, an existing AlertmanagerConfig that previously worked now fails validation due to the new URL regex check.
Steps to Reproduce
I have a slackConfigs action that uses Go templating to build a Silence link dynamically depending on alert status and labels:
slackConfigs:
apiURL:
- actions:
- type: '{{ if eq .Status "firing" }}button{{ end }}'
text: '{{ if eq .Status "firing" }}Silence{{ end }}'
url: '{{ if eq .Status "firing" }}{{ .ExternalURL }}/#/silences/new?filter=%7B{{ range .CommonLabels.SortedPairs }}{{ if ne .Name "alertname" }}{{ .Name }}%3D"{{ .Value }}"%2C%20{{ end }}{{ end }}alertname%3D"{{ .CommonLabels.alertname }}"%7D{{ end }}'
After upgrading prometheus-operator, applying this config now fails with:
spec.receivers[3].slackConfigs[0].actions[0].url: Invalid value: "{{ if eq .Status \"firing\" }}{{ .ExternalURL }}/#/silences/new?filter=%7B{{ range .CommonLabels.SortedPairs }}{{ if ne .Name \"alertname\" }}{{ .Name }}%3D\"{{ .Value }}\"%2C%20{{ end }}{{ end }}alertname%3D\"{{ .CommonLabels.alertname }}\"%7D{{ end }}": spec.receivers[3].slackConfigs[0].actions[0].url in body should match '^https?://.+$'
It looks like the CRD validation is applied to the raw, unrendered template string, so the value doesn’t start with http:// or https:// yet and therefore fails the regex.
Expected Result
I expected that templated URLs (e.g. using {{ .ExternalURL }} or other Go templates) would still be allowed, just like before the upgrade, since Alertmanager itself will render those templates at runtime into a valid https://… URL.
In other words, I expected the operator to accept templated URL strings that will become valid URLs when rendered.
Actual Result
• The operator validates spec.receivers[].slackConfigs[].actions[*].url against ^https?://.+$ before template rendering.
• Any templated URL that does not literally start with http/https in the raw string is rejected, even though it is a valid URL at runtime.
Why this is a problem
• It breaks existing configurations that rely on Go templating to build URLs dynamically (e.g. Silence links or links using .ExternalURL).
• It forces users to hardcode full URLs instead of using Alertmanager’s templating capabilities.
Prometheus Operator Version
prometheus-operator version: 0.88.0Kubernetes Version
clientVersion:
buildDate: "2025-12-17T12:32:06Z"
compiler: gc
gitCommit: 66452049f3d692768c39c797b21b793dce80314e
gitTreeState: clean
gitVersion: v1.35.0
goVersion: go1.25.5
major: "1"
minor: "35"
platform: darwin/arm64
kustomizeVersion: v5.7.1
serverVersion:
buildDate: "2025-11-19T17:44:12Z"
compiler: gc
emulationMajor: "1"
emulationMinor: "34"
gitCommit: 4092d31cfceb915285d457d4438eac72c5d43043
gitTreeState: clean
gitVersion: v1.34.1-gke.3355004
goVersion: go1.24.6 X:boringcrypto
major: "1"
minCompatibilityMajor: "1"
minCompatibilityMinor: "33"
minor: "34"
platform: linux/amd64Kubernetes Cluster Type
GKE
How did you deploy Prometheus-Operator?
helm chart:prometheus-community/kube-prometheus-stack
prometheus-operator log output
spec.receivers[3].slackConfigs[0].actions[0].url: Invalid value: "{{ if eq .Status \"firing\" }}{{ .ExternalURL }}/#/silences/new?filter=%7B{{ range .CommonLabels.SortedPairs }}{{ if ne .Name \"alertname\" }}{{ .Name }}%3D\"{{ .Value }}\"%2C%20{{ end }}{{ end }}alertname%3D\"{{ .CommonLabels.alertname }}\"%7D{{ end }}": spec.receivers[3].slackConfigs[0].actions[0].url in body should match '^https?://.+$'
ts=2026-01-17T15:56:30.321823459Z level=info caller=/workspace/pkg/alertmanager/operator.go:603 msg="sync alertmanager" component=alertmanager-controller key=monitoring/kps
ts=2026-01-17T15:56:30.345363254Z level=error caller=/workspace/pkg/operator/resource_reconciler.go:686 msg="Unhandled Error" logger=UnhandledError err="sync \"monitoring/kps\" failed: provision alertmanager configuration: failed to initialize from global AlertmanagerConfig: failed to validate 'slackConfig' - receiver slack: [0]: invalid 'action'[0]: url: validate url from string failed for {{ if eq .Status \"firing\" }}{{ .ExternalURL }}/#/silences/new?filter=%7B{{ range .CommonLabels.SortedPairs }}{{ if ne .Name \"alertname\" }}{{ .Name }}%3D\"{{ .Value }}\"%2C%20{{ end }}{{ end }}alertname%3D\"{{ .CommonLabels.alertname }}\"%7D{{ end }}: invalid character 'f' after top-level value"