-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Closed
Labels
Description
What did you do?
- Configured Alertmanager with OpsGenie receiver
- Created an alert in Grafana UI that will always fire
What did you expect to see?
Expected Alertmanager to create a new alert in OpsGenie
What did you see instead? Under which circumstances?
No alert created in OpsGenie, looking at the logs I get this error:
Notify for alerts failed" num_alerts=1 err="devops-opsgenie/opsgenie[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 405: {\"message\":\"Request method 'POST' not supported\",\"took\":0.001,\"requestId\":\"d50741e2-e518-4d31-b562-7b1d0ac21f4d\"}
Environment
- System information:
Linux 5.4.219-126.411.amzn2.x86_64 x86_64
- Alertmanager version:
alertmanager, version 0.24.0 (branch: HEAD, revision: f484b17fa3c583ed1b2c8bbcec20ba1db2aa5f11)
build user: root@265f14f5c6fc
build date: 20220325-09:31:33
go version: go1.17.8
platform: linux/amd64
- Prometheus version:
prometheus, version 2.40.5 (branch: HEAD, revision: 44af4716c86138869aa621737139e6dacf0e2550)
build user: root@70f803b28803
build date: 20221201-12:50:06
go version: go1.19.3
platform: linux/amd64
- Alertmanager configuration file:
global:
resolve_timeout: 5m
opsgenie_api_url: https://api.opsgenie.com/v2/alerts
receivers:
- name: "null"
- name: devops-opsgenie
opsgenie_configs:
- api_key_file: /opt/opsgenie/opsgenie_api_key_devops
description: test
message: test
priority: '{{ range .Alerts }}{{ if eq .Labels.severity "critical"}}P1{{else}}P3{{end}}{{end}}'
responders:
- name: DevOps
type: team
send_resolved: true
tags: '{{ range .Alerts }}{{ .Labels.Values | join "," }}{{end}}'
route:
group_by:
- grafana_folder
- alertname
group_interval: 5m
group_wait: 30s
receiver: "null"
repeat_interval: 12h
routes:
- group_by:
- '...'
group_interval: 30s
group_wait: 1s
matchers:
- team = "devops"
receiver: devops-opsgenie
repeat_interval: 30m
templates:
- /etc/alertmanager/config/*.tmpl
Now, I was able to solve this by simply removing the key global.opsgenie_api_url, and upon reading the docs again I noticed the strange wording used:
# The host to send OpsGenie API requests to.
[ api_url: [<string>](https://prometheus.io/docs/alerting/latest/configuration/#string) | default = global.opsgenie_api_url ]
So which is it? host or url? this is a big distinction and the above just makes it confusing.
The default value configured at global.opsgenie_api_url is https://api.opsgenie.com/, this is a URL.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
Done