Skip to content

fix(notify): flaky tests#17899

Merged
bwplotka merged 1 commit intoprometheus:mainfrom
siavashs:fix/notify-flaky-tests
Jan 22, 2026
Merged

fix(notify): flaky tests#17899
bwplotka merged 1 commit intoprometheus:mainfrom
siavashs:fix/notify-flaky-tests

Conversation

@siavashs
Copy link
Contributor

@siavashs siavashs commented Jan 20, 2026

Which issue(s) does the PR fix:

Add a helper function to set up AlertmanagerSets.
Fix all flaky tests.

Does this PR introduce a user-facing change?

NONE

@siavashs
Copy link
Contributor Author

Asked maintainers to re-run the jobs few more times to be 100% sure if tests are fixed.

Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it's not about timeouts here 🤔

Image

@siavashs siavashs force-pushed the fix/notify-flaky-tests branch from d1216d5 to 16929c7 Compare January 20, 2026 14:22
@siavashs
Copy link
Contributor Author

Looks like it's not about timeouts here 🤔

Yes, there was a bug were tests were running with sendloops but no AMs in the sets.
I pushed a fix, let's see if that resolves the issue 🤞

@siavashs siavashs force-pushed the fix/notify-flaky-tests branch from 16929c7 to f09f3a6 Compare January 20, 2026 14:47
@siavashs siavashs force-pushed the fix/notify-flaky-tests branch from f09f3a6 to 2188a11 Compare January 20, 2026 15:12
Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try this, thanks!

@siavashs
Copy link
Contributor Author

Still failing on Windows...

@siavashs
Copy link
Contributor Author

Let's run notifier tests 1000 times to capture all flaky tests on CI...

@siavashs siavashs force-pushed the fix/notify-flaky-tests branch 11 times, most recently from 47214c1 to fc36ad6 Compare January 21, 2026 11:10
@siavashs siavashs requested a review from machine424 January 21, 2026 11:10
Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work and thank you!

Added some idea to avoid that inconsistent check in prod code, as per @machine424 comment.

Copy link
Member

@machine424 machine424 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some suggestions and questions

@siavashs siavashs requested a review from machine424 January 21, 2026 14:29
@siavashs siavashs force-pushed the fix/notify-flaky-tests branch from db2f0e7 to 48b328a Compare January 21, 2026 15:29
Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!!!

@bwplotka bwplotka enabled auto-merge (squash) January 21, 2026 15:41
auto-merge was automatically disabled January 22, 2026 08:26

Head branch was pushed to by a user without write access

@siavashs siavashs force-pushed the fix/notify-flaky-tests branch 4 times, most recently from 7ff0898 to 5e9c8fd Compare January 22, 2026 09:46
Add a helper function to set up AlertmanagerSets.
Fix all flaky tests.

Signed-off-by: Siavash Safi <siavash@cloudflare.com>
@siavashs siavashs force-pushed the fix/notify-flaky-tests branch 2 times, most recently from 0550b65 to 749cc22 Compare January 22, 2026 10:39
@bwplotka bwplotka merged commit d9ccd70 into prometheus:main Jan 22, 2026
32 checks passed
@siavashs siavashs deleted the fix/notify-flaky-tests branch January 22, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants