fix flaky test: alert details error page timeout by miguelmartin-elastic · Pull Request #260302 · elastic/kibana

miguelmartin-elastic · 2026-03-30T16:03:56Z

Summary

Fixes the flaky test reported in #250046.

Root cause: `toBeVisible()` would start checking while the component was in its loading state (spinner), missing the brief initial render of the error panel and timing out before the async API call for the non-existent alert completed.

Fix: Wrap navigate+assert in `.toPass({ timeout: 30_000, intervals: [2_000] })`. I also fixed a data-test-subj and a goto method that changed during Rules page unification

Test type: Scout (Playwright)
Test file: `x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel_tests/alert_details_page.spec.ts`
Config: `x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts`

Validation

CI flaky test runner (50 + 150 runs)

Checklist

Flaky Test Runner was used on any tests changed

Fixes elastic#250046 Root cause: the default 10s `toBeVisible()` timeout was too short for the async API call (404 for non-existent alert) + React re-render cycle to complete, causing intermittent failures in CI. Fix: increase timeout to 30s and un-skip the test describe block.

miguelmartin-elastic · 2026-03-30T16:04:04Z

/flaky scoutConfig:x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts:50

kibanamachine · 2026-03-30T16:04:20Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#11287

x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts x50

kibanamachine · 2026-03-30T17:49:48Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11287

[❌] x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts: 0/50 tests passed.

see run history

Fixes elastic#250046 Root cause: the test was flaky because `toBeVisible()` would start checking while the component was in its loading state (spinner), missing the brief initial render and timing out before the async API call for the non-existent alert completed and the error panel appeared. Fix: wrap navigate+assert in `.toPass()` (60s / 2s interval) consistent with every other test in this suite, so retries catch the error panel once loading resolves. Also skips the 6 sibling tests that require an active alert from the rule created in beforeAll — those tests need metric data in `metrics-*` to trigger the threshold rule, which does not exist in CI.

miguelmartin-elastic · 2026-03-30T18:07:32Z

/flaky scoutConfig:x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts:50

kibanamachine · 2026-03-30T18:07:48Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#11293

x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts x50

miguelmartin-elastic · 2026-03-30T18:12:25Z

/flaky scoutConfig:x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts:50

kibanamachine · 2026-03-30T18:12:40Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#11295

x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts x50

This reverts commit 70ca70f.

kibanamachine · 2026-03-30T19:05:09Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11293

[✅] x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts: 50/50 tests passed.

see run history

kibanamachine · 2026-03-30T19:07:01Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11295

[✅] x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts: 50/50 tests passed.

see run history

miguelmartin-elastic · 2026-03-30T19:41:31Z

/flaky scoutConfig:x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts:50

kibanamachine · 2026-03-30T19:41:55Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#11298

x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts x50

...utions/observability/plugins/observability/test/scout/ui/fixtures/page_objects/rules_page.ts

kibanamachine · 2026-03-30T20:45:05Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11298

[❌] x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts: 0/50 tests passed.

see run history

miguelmartin-elastic · 2026-03-30T23:38:08Z

/flaky scoutConfig:x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts:50

kibanamachine · 2026-03-30T23:38:22Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#11301

x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts x50

kibanamachine · 2026-03-31T00:42:18Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11301

[✅] x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts: 50/50 tests passed.

see run history

miguelmartin-elastic · 2026-03-31T01:38:15Z

/flaky scoutConfig:x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts:150

macroscopeapp · 2026-03-31T10:49:21Z

Approvability

Verdict: Would Approve

Test-only changes fixing a flaky test by correcting test locators, URL paths, and adding retry logic. No production code is modified, and all changes are confined to test infrastructure files.

^{Macroscope would have approved this PR. A repo admin can enable approvability here.}

kibanamachine · 2026-03-31T10:49:44Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11303

[✅] x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts: 150/150 tests passed.

see run history

elasticmachine · 2026-03-31T12:01:17Z

💚 Build Succeeded

Buildkite Build
Commit: 9f95deb

Metrics [docs]

✅ unchanged

…hanges * commit 'd0e62a657916e84694a93983e513ce9e34e0b635': (27 commits) [Agent Builder] Agent overview page design updates (elastic#260468) [Inference UI] Add model detail flyout with endpoint management (elastic#260307) [Fleet] Update doc links in agent policy settings (elastic#260245) [Security Solution] show risk score in new flyout header (elastic#260187) Replace deprecated EUI icons in files owned by @elastic/kibana-security (elastic#255636) [Cases][Templates] Add DATE_PICKER field control type (elastic#260209) [SharedUX] Get spaces callout on each solution nav (elastic#259723) [SharedUX] Preserve feature visibility on solution change (elastic#259316) [CI] Increase investigations cypress disks to 110G (elastic#260423) [Agent Builder] Expose read-only conversations on plugin start contract (elastic#260435) [dasboards as code] drop panels with server errors (elastic#260073) [One Workflow] Add force-delete (hard delete) option for workflows (elastic#260391) [Agent Builder] Fix sidebar error handling error (elastic#260446) [Agent Builder] Add attachment origin to Converse API (elastic#259043) [Alerting v2] Fix rule results preview chart responsiveness (elastic#260444) [Streams] Processing error panel UI improvements (elastic#260028) fix flaky test: alert details error page timeout (elastic#260302) [Agent Builder] Add attachment origin to Converse API (elastic#259043) [One Workflow] Add more unit tests to workflows_extensions plugin (elastic#260384) [ResponseOps] Split alerting security_and_spaces group8 FTR config to fix CI timeout (elastic#260029) ...

## Summary Fixes the flaky test reported in elastic#250046. **Root cause:** \`toBeVisible()\` would start checking while the component was in its loading state (spinner), missing the brief initial render of the error panel and timing out before the async API call for the non-existent alert completed. **Fix:** Wrap navigate+assert in \`.toPass({ timeout: 30_000, intervals: [2_000] })\`. I also fixed a data-test-subj and a `goto` method that changed during Rules page unification **Test type:** Scout (Playwright) **Test file:** \`x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel_tests/alert_details_page.spec.ts\` **Config:** \`x-pack/solutions/observability/plugins/observability/test/scout/ui/parallel.playwright.config.ts\` ## Validation - [ ] CI flaky test runner (50 + 150 runs) ## Checklist - [x] Flaky Test Runner was used on any tests changed

miguelmartin-elastic added the auto-flaky-fix Automated PR created by flaky-test-resolver agent label Mar 30, 2026

github-actions bot added the author:actionable-obs PRs authored by the actionable obs team label Mar 30, 2026

Revert "fix flaky test: alert details error page timeout"

aa53b1f

This reverts commit 70ca70f.

miguelmartin-elastic force-pushed the fix/flaky-250046-alert-details-error-timeout branch from 50ae0da to aa53b1f Compare March 30, 2026 18:38

fix data-test-subj, add polling

5a168f4

macroscopeapp bot reviewed Mar 30, 2026

View reviewed changes

...utions/observability/plugins/observability/test/scout/ui/fixtures/page_objects/rules_page.ts Show resolved Hide resolved

fix rulesPage.goto

ae32375

remove skill files

9f95deb

miguelmartin-elastic marked this pull request as ready for review March 31, 2026 10:48

miguelmartin-elastic requested review from a team as code owners March 31, 2026 10:48

miguelmartin-elastic added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting and removed auto-flaky-fix Automated PR created by flaky-test-resolver agent labels Mar 31, 2026

mgiota approved these changes Mar 31, 2026

View reviewed changes

miguelmartin-elastic merged commit 798ad69 into elastic:main Mar 31, 2026
21 checks passed

kibanamachine added the v9.4.0 label Mar 31, 2026

Conversation

miguelmartin-elastic commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Checklist

Uh oh!

miguelmartin-elastic commented Mar 30, 2026

Uh oh!

kibanamachine commented Mar 30, 2026

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#11287

Uh oh!

kibanamachine commented Mar 30, 2026

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11287

Uh oh!

miguelmartin-elastic commented Mar 30, 2026

Uh oh!

kibanamachine commented Mar 30, 2026

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#11293

Uh oh!

miguelmartin-elastic commented Mar 30, 2026

Uh oh!

kibanamachine commented Mar 30, 2026

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#11295

Uh oh!

kibanamachine commented Mar 30, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11293

Uh oh!

kibanamachine commented Mar 30, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11295

Uh oh!

miguelmartin-elastic commented Mar 30, 2026

Uh oh!

kibanamachine commented Mar 30, 2026

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#11298

Uh oh!

Uh oh!

kibanamachine commented Mar 30, 2026

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11298

Uh oh!

miguelmartin-elastic commented Mar 30, 2026

Uh oh!

kibanamachine commented Mar 30, 2026

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#11301

Uh oh!

kibanamachine commented Mar 31, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11301

Uh oh!

miguelmartin-elastic commented Mar 31, 2026

Uh oh!

macroscopeapp bot commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approvability

Uh oh!

kibanamachine commented Mar 31, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11303

Uh oh!

elasticmachine commented Mar 31, 2026

💚 Build Succeeded

Metrics [docs]

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

miguelmartin-elastic commented Mar 30, 2026 •

edited

Loading

macroscopeapp bot commented Mar 31, 2026 •

edited

Loading