Skip to content

Update alerts api to display that notifications are muted for certain receivers. #2832

@dubyte

Description

@dubyte

The is a new feature about mute_time_intervals and if approved to active_time_intervals. The notifications are not sent when they are muted but there is no way using the API what notification will be delivered or not.

Example imagine that we have an alert and we have two receivers but for devs receivers the notification is muted:

Current state:

GET /api/v2/alerts

[
  {
    "annotations": {
      "description": "randomina.lan:9100 of job node has been down for more than 10 minutes.",
      "summary": "Endpoint randomina.lan:9100 down"
    },
    "endsAt": "2022-02-14T01:07:46.140Z",
    "fingerprint": "0593f1c55738ce93",
    "receivers": [
      {
        "name": "devs"
      },
      {
        "name": "monitoring"
      }
    ],
    "startsAt": "2022-02-12T09:01:16.140Z",
    "status": {
      "inhibitedBy": [],
      "silencedBy": [],
      "state": "active"
    },
    "updatedAt": "2022-02-13T20:03:46.134-05:00",
    "generatorURL": "http://azul:9090/graph?g0.expr=up+%3D%3D+0&g0.tab=1",
    "labels": {
      "alertname": "InstanceDown",
      "instance": "randomina.lan:9100",
      "job": "node",
      "severity": "medium"
    }
  }
]

So the alert is active and is not inhibited or silenced but it wont be delivered to devs reciever so I propose the next

GET /api/v2/alerts

[
  {
    "annotations": {
      "description": "<HOST2>:9100 of job node has been down for more than 10 minutes.",
      "summary": "Endpoint <HOST2>:9100 down"
    },
    "endsAt": "<DATETIME>",
    "fingerprint": "0593f1c55738ce93",
    "receivers": [
      {
        "name": "devs"
      },
      {
        "name": "monitoring"
      }
    ],
    "startsAt": "<DATETIME>",
    "status": {
      "inhibitedBy": [],
      "notificationMutedFor": ["dev"],  <------------------------ This is new
      "silencedBy": [],
      "state": "<DATETIME>"
    },
    "updatedAt": "<DATETIME>",
    "generatorURL": "http://<HOST>:9090/graph?g0.expr=up+%3D%3D+0&g0.tab=1",
    "labels": {
      "alertname": "InstanceDown",
      "instance": "<HOST2>:9100",
      "job": "node",
      "severity": "medium"
    }
  }
]

In this case we tell that even when the alert is active the notification itself is muted for the dev reciever.

Please provide feedback if this make sense I can send a PR.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    To triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions