Skip to content

Inconsistent notification metrics for not used receivers #3237

@monoxono

Description

@monoxono

What did you do?

  • run alertmanager with the official example config
  • run amtool to add one alert
  • check notifications related metrics, those start with "alertmanager_notification_"

What did you expect to see?
For not used receivers, either metrics exist for all of them or none of them.
(I'm inclined to no metrics for not used receivers.)

What did you see instead? Under which circumstances?
For most receivers, when they're not used, notifications related metrics still exist.
For example, opsgenie is not used in that config, but the time series alertmanager_notification_latency_seconds_sum{integration="opsgenie"} exists and the value is always 0.

This is not true for two new recently added receivers, webex and discord.

related code where webex and discord are missing:

for _, integration := range []string{
"email",
"pagerduty",
"wechat",
"pushover",
"slack",
"opsgenie",
"webhook",
"victorops",
"sns",
"telegram",
} {
m.numNotifications.WithLabelValues(integration)
m.numTotalFailedNotifications.WithLabelValues(integration)
m.numNotificationRequestsTotal.WithLabelValues(integration)
m.numNotificationRequestsFailedTotal.WithLabelValues(integration)
m.notificationLatencySeconds.WithLabelValues(integration)
}

I don't think this issue has real user impact because it happens on not used receivers.

Environment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions