[Discover][Metrics] Fix e2e metrics flaky test by lucaslopezf · Pull Request #254786 · elastic/kibana

lucaslopezf · 2026-02-24T17:31:20Z

Summary

Fixes flaky Metrics Experience E2E tests by reorganizing the Scout test structure to comply with Scout's config naming requirements.

Root Cause

Scout expects specific config names (parallel.playwright.config.ts or playwright.config.ts) to properly assign CI cluster resources. Our custom names (metrics_experience.parallel.playwright.config.ts, traces_experience.parallel.playwright.config.ts) caused Scout to fall back to a default cluster with fewer resources, leading to intermittent timeouts.

Changes

Unified config: Single parallel.playwright.config.ts pointing to ./parallel_tests
Unified global setup: Single globalSetupHook handling both Metrics and Traces data setup

This PR is necessary to avoid CI cluster misassignment and test flakiness. More details in this comment.

Thanks a lot @dmlemeshko !!

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

See some risk examples
...

lucaslopezf · 2026-02-24T17:31:29Z

/ci

lucaslopezf · 2026-02-24T17:33:23Z

/flaky scoutConfig:src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts:30

kibanamachine · 2026-02-24T17:33:37Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10843

src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts x30

kibanamachine · 2026-02-24T18:33:59Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#10843

[❌] src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts: 0/30 tests passed.

see run history

lucaslopezf · 2026-02-24T18:53:03Z

/ci

lucaslopezf · 2026-02-24T18:53:06Z

/flaky scoutConfig:src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts:30

kibanamachine · 2026-02-24T18:53:36Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10845

src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts x30

kibanamachine · 2026-02-24T19:53:24Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10845

[✅] src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts: 30/30 tests passed.

see run history

lucaslopezf · 2026-02-25T06:38:30Z

/ci

lucaslopezf · 2026-02-25T06:40:54Z

/flaky scoutConfig:src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts:30

kibanamachine · 2026-02-25T06:41:11Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10846

src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts x30

kibanamachine · 2026-02-25T07:32:28Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10846

[✅] src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts: 30/30 tests passed.

see run history

lucaslopezf · 2026-02-25T08:07:42Z

/flaky scoutConfig:src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts:30

kibanamachine · 2026-02-25T08:07:57Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10847

src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts x30

kibanamachine · 2026-02-25T09:02:02Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10847

[✅] src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts: 30/30 tests passed.

see run history

lucaslopezf · 2026-02-25T13:34:59Z

/ci

lucaslopezf · 2026-02-25T15:24:40Z

/ci

lucaslopezf · 2026-02-25T15:25:00Z

/flaky scoutConfig:src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts:35

kibanamachine · 2026-02-25T15:25:18Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10858

src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts x35

kibanamachine · 2026-02-25T16:28:13Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10858

[✅] src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts: 35/35 tests passed.

see run history

lucaslopezf · 2026-02-25T16:36:26Z

/flaky scoutConfig:src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts:40

kibanamachine · 2026-02-25T16:36:48Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10859

src/platform/plugins/shared/discover/test/scout/ui/metrics_experience_parallel.playwright.config.ts x40

kibanamachine · 2026-03-02T10:42:35Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10917

[✅] src/platform/plugins/shared/discover/test/scout/ui/parallel.playwright.config.ts: 50/50 tests passed.

see run history

lucaslopezf · 2026-03-02T10:44:09Z

/flaky scoutConfig:src/platform/plugins/shared/discover/test/scout/ui/parallel.playwright.config.ts:50

kibanamachine · 2026-03-02T10:44:22Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10921

src/platform/plugins/shared/discover/test/scout/ui/parallel.playwright.config.ts x50

lucaslopezf · 2026-03-02T10:59:00Z

/ci

kibanamachine · 2026-03-02T11:40:29Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10921

[✅] src/platform/plugins/shared/discover/test/scout/ui/parallel.playwright.config.ts: 50/50 tests passed.

see run history

elasticmachine · 2026-03-02T12:15:23Z

💚 Build Succeeded

Buildkite Build
Commit: af940aa

Metrics [docs]

✅ unchanged

History

💛 Build #402744 was flaky 0e45904
💔 Build #402515 failed 7bd3cc4
💚 Build #402024 succeeded d327aa3
💛 Build #401179 was flaky c48c7fb
💚 Build #400845 succeeded 49413a2
💚 Build #400778 succeeded 49413a2

elasticmachine · 2026-03-02T12:16:57Z

Pinging @elastic/obs-exploration-team (Team:obs-exploration)

iblancof

Worth mentioning for other reviewers that this is just a temporary workaround until the appex team adds support for separating tests with namespaces (which will be happening soon in appex-qa-team/762) .

There’s already an issue (observability-dev/5290) created on our side to separate the configs/ownerships again once that support is merged.

akowalska622

Data Discovery changes LGTM

lucaslopezf · 2026-03-02T18:08:33Z

/flaky scoutConfig:src/platform/plugins/shared/discover/test/scout/ui/parallel.playwright.config.ts:100

kibanamachine · 2026-03-02T18:08:52Z

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10927

src/platform/plugins/shared/discover/test/scout/ui/parallel.playwright.config.ts x100

kibanamachine · 2026-03-02T19:29:30Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10927

[✅] src/platform/plugins/shared/discover/test/scout/ui/parallel.playwright.config.ts: 100/100 tests passed.

see run history

@dmlemeshko

Closes: elastic#254759, elastic#254760, elastic#254752 ## Summary Fixes flaky Metrics Experience E2E tests by reorganizing the Scout test structure to comply with Scout's config naming requirements. ### Root Cause Scout expects specific config names (`parallel.playwright.config.ts` or `playwright.config.ts`) to properly assign CI cluster resources. Our custom names (`metrics_experience.parallel.playwright.config.ts`, `traces_experience.parallel.playwright.config.ts`) caused Scout to fall back to a default cluster with fewer resources, leading to intermittent timeouts. ### Changes - **Unified config**: Single `parallel.playwright.config.ts` pointing to `./parallel_tests` - **Unified global setup**: Single `globalSetupHook` handling both Metrics and Traces data setup This PR is necessary to avoid CI cluster misassignment and test flakiness. More details in [this comment](elastic#254759 (comment)). Thanks a lot @dmlemeshko !! ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

@dmlemeshko

Closes: elastic#254759, elastic#254760, elastic#254752 ## Summary Fixes flaky Metrics Experience E2E tests by reorganizing the Scout test structure to comply with Scout's config naming requirements. ### Root Cause Scout expects specific config names (`parallel.playwright.config.ts` or `playwright.config.ts`) to properly assign CI cluster resources. Our custom names (`metrics_experience.parallel.playwright.config.ts`, `traces_experience.parallel.playwright.config.ts`) caused Scout to fall back to a default cluster with fewer resources, leading to intermittent timeouts. ### Changes - **Unified config**: Single `parallel.playwright.config.ts` pointing to `./parallel_tests` - **Unified global setup**: Single `globalSetupHook` handling both Metrics and Traces data setup This PR is necessary to avoid CI cluster misassignment and test flakiness. More details in [this comment](elastic#254759 (comment)). Thanks a lot @dmlemeshko !! ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

Wait for discoverLayout before checking data view switch

ccd2523

test workers

709c69b

lucaslopezf and others added 2 commits February 25, 2026 07:35

Merge branch 'main' into 254752-fix-e2e-metrics-flaky-test

ce8221c

remove skip

5cff8a7

Add gotoWithExtendedTimeout

2c8b258

Merge branch 'main' into 254752-fix-e2e-metrics-flaky-test

49413a2

lucaslopezf changed the title ~~Wait for discoverLayout before checking data view switch~~ [Discover][Metrics] Fix e2e metrics flaky test Feb 25, 2026

lucaslopezf mentioned this pull request Feb 25, 2026

Failing test: Metrics in Discover - Grid Navigation - should paginate through metrics #254759

Closed

lucaslopezf added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:obs-exploration Observability Exploration team v9.4.0 labels Mar 2, 2026

lucaslopezf added the Feature:Metrics in Discover label Mar 2, 2026

lucaslopezf marked this pull request as ready for review March 2, 2026 12:16

lucaslopezf requested review from a team as code owners March 2, 2026 12:16

Merge branch 'main' into 254752-fix-e2e-metrics-flaky-test

15d1d9d

iblancof approved these changes Mar 2, 2026

View reviewed changes

steliosmavro mentioned this pull request Mar 2, 2026

Enforce standard Playwright config naming in scout_test_file_naming rule #255535

Merged

akowalska622 approved these changes Mar 2, 2026

View reviewed changes

tfcmarques approved these changes Mar 2, 2026

View reviewed changes

lucaslopezf merged commit ab46925 into elastic:main Mar 3, 2026
16 checks passed

This was linked to issues Mar 4, 2026

Failing test: Metrics in Discover - Grid - should render grid with WHERE filter #254760

Closed

Failing test: Metrics in Discover - Grid - should render metrics grid with cards #254752

Closed

This was referenced Mar 4, 2026

Failing test: Metrics in Discover - Grid - should render grid with WHERE filter #254760

Closed

Failing test: Metrics in Discover - Grid - should render metrics grid with cards #254752

Closed

Conversation

lucaslopezf commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Root Cause

Changes

Checklist

Identify risks

Uh oh!

lucaslopezf commented Feb 24, 2026

Uh oh!

lucaslopezf commented Feb 24, 2026

Uh oh!

kibanamachine commented Feb 24, 2026

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10843

Uh oh!

kibanamachine commented Feb 24, 2026

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#10843

Uh oh!

lucaslopezf commented Feb 24, 2026

Uh oh!

lucaslopezf commented Feb 24, 2026

Uh oh!

kibanamachine commented Feb 24, 2026

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10845

Uh oh!

kibanamachine commented Feb 24, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10845

Uh oh!

lucaslopezf commented Feb 25, 2026

Uh oh!

lucaslopezf commented Feb 25, 2026

Uh oh!

kibanamachine commented Feb 25, 2026

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10846

Uh oh!

kibanamachine commented Feb 25, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10846

Uh oh!

lucaslopezf commented Feb 25, 2026

Uh oh!

kibanamachine commented Feb 25, 2026

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10847

Uh oh!

kibanamachine commented Feb 25, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10847

Uh oh!

lucaslopezf commented Feb 25, 2026

Uh oh!

lucaslopezf commented Feb 25, 2026

Uh oh!

lucaslopezf commented Feb 25, 2026

Uh oh!

kibanamachine commented Feb 25, 2026

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10858

Uh oh!

kibanamachine commented Feb 25, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10858

Uh oh!

lucaslopezf commented Feb 25, 2026

Uh oh!

kibanamachine commented Feb 25, 2026

Flaky Test Runner

✅ Build triggered - kibana-flaky-test-suite-runner#10859

Uh oh!

kibanamachine commented Mar 2, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#10917

Uh oh!

lucaslopezf commented Mar 2, 2026

lucaslopezf commented Feb 24, 2026 •

edited

Loading