Skip to content

[alerting] sticky "red" status when getting a 50x error calculating alerting health status #93062

@pmuellr

Description

@pmuellr

Kibana version: 7.11.0

Elasticsearch version: 7.11.0

Server OS version: cloud

Describe the bug:

It appears that if alerting encounters a 50x status while it's calculating it's health status, that error state will be sticky, and won't ever get cleared.

Steps to reproduce:

  1. Seems to happen fairly often in the cloud, but not deterministic
  2. check Kibana API /api/status?v8format=true
  3. you may see errors like the ones shown below
  4. once you get that error indication, it will now never change
  5. all plugins dependent on alerting will also be "red"
  6. everything seems to be working normally, but there's a lot of "red" plugins in the status

Expected behavior:

  • never see a "red" indication caused by transient 50x responses

Screenshots (if relevant):

image

Metadata

Metadata

Assignees

Labels

Team:ResponseOpsPlatform ResponseOps team (formerly the Cases and Alerting teams) t//bugFixes for quality problems that affect the customer experience

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions