[Flyout System] Automated functional tests using Scout framework by tsullivan · Pull Request #252171 · elastic/kibana

tsullivan · 2026-02-06T19:39:06Z

Summary

Closes https://github.com/elastic/kibana-team/issues/2566
Includes [Flyouts Example Plugin] Use historyKey for flyout demos #260557 (for history group support in the example plugin app)
Hold until after Update EUI to v114.0.0 #259497

Changes

data-test-subj attributes — Added to flyout buttons, switches, and close controls in both the EuiFlyout component and Overlays API implementations to enable reliable test selectors.

Scout UI tests — Two tests covering:

Open a main flyout and a child flyout, verify cascade close behavior
Open a push flyout, open a second overlay flyout, verify Back button navigation

Minor Flyout System Examples plugin app refactoring

See [Flyout System] Automated functional tests using Scout framework #252171 (comment)

To run locally

node scripts/scout.js start-server \
  --arch stateful \
  --domain classic \
  --serverConfigSet examples

# terminal two: run the tests
npx playwright test --config examples/flyout_system/test/scout_examples/ui/playwright.config.ts \
  --project local \
  --grep @local-stateful-classic \
  --headed

tsullivan · 2026-02-07T17:17:42Z

changed to draft since the new tests are not running in CI

dmlemeshko · 2026-02-09T10:53:29Z

@tsullivan We currently don't have automatic discovery for scout tests under examples/ directory, but it is a trivial change. Without discovery tests won't be run automatically in CI, you need to explicitly add a buildkite step in pipeline.

The question is how you would like to run those tests in CI, together with other functional tests or is there any specific goals?

Dosant · 2026-02-09T11:21:21Z

@tsullivan We currently don't have automatic discovery for scout tests under examples/ directory, but it is a trivial change. Without discovery tests won't be run automatically in CI, you need to explicitly add a buildkite step in pipeline.

The question is how you would like to run those tests in CI, together with other functional tests or is there any specific goals?

@dmlemeshko, similar to the existing example plugin tests, we would like to run these tests as part of the regular CI run. With these tests, we want to make sure that our core and Chrome APIs are stable without relying on testing specific user concerns of the apps that we don't own.

tsullivan · 2026-02-09T17:56:12Z

@dmlemeshko would you be able to help with ensuring this test can run in CI? Would that work have to be separate or can a change be made in this PR that would enable it?

dmlemeshko · 2026-02-10T09:35:02Z

@dmlemeshko would you be able to help with ensuring this test can run in CI? Would that work have to be separate or can a change be made in this PR that would enable it?

We would have to do it in the separate PR, checking details with the Team

...hared/kbn-scout/src/servers/configs/config_sets/examples/stateful/classic.stateful.config.ts

tsullivan · 2026-02-18T15:47:15Z

Blocked on https://github.com/elastic/appex-qa-team/issues/708

tsullivan · 2026-03-26T22:31:17Z

examples/flyout_system/test/scout_examples/ui/tests/flyout_with_component.spec.ts


+  test.afterAll(() => {
+    throw new Error(`Test error to verify CI run`);
+  });


This will ensure that CI is running Scout tests for example plugins, and is to prove useful after #259170

elasticmachine · 2026-03-30T14:45:39Z

Pinging @elastic/appex-sharedux (Team:SharedUX)

examples/flyout_system/public/components/_flyout_with_component.tsx

examples/flyout_system/public/components/_flyout_with_overlays.tsx

tsullivan · 2026-03-31T19:50:25Z

I used flyout_system_examples as the plugin name for the Scout config and CI failed with:


Discover Playwright Configs and upload to Buildkite artifacts (selective testing)
--
.peggy require hook already registered, skipping
info Selective testing: 1 affected module(s), 49 rest, 0 unmapped
ERROR The following plugin(s)/package(s) are not registered in Scout CI config '.buildkite/scout_ci_config.yml':
- flyoutSystemExamples (plugin)
Read more: src/platform/packages/shared/kbn-scout/README.md
🚨 Error: The command exited with status 1

Switched to camelCase name (taking from plugin.id) in 31b2cf3

tsullivan · 2026-03-31T20:05:28Z

I wish to wait to merge this until after #259497, because the "scoped history" changes coming in will affect the test flow (we will need to add historyKey to the test flyouts)

…g/flyout-system

examples/flyout_system/public/components/app.tsx

tsullivan · 2026-04-01T18:28:51Z

This brings in #260557, to add support for history groups, which functionally changed after Kibana upgraded to EUI 114.

Description from that PR:

Summary

EUI v114 brought changes to the way that flyouts can have a shared history: each flyout needs to share a historyKey object with the flyouts that it wants to share history with.

Screenshots

This PR also cleans up the example page code by putting the "Non-Session" flyouts in their own section, and removing the unnecessary filler text at the bottom.

Before

After

elasticmachine · 2026-04-02T00:02:33Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: c645ba0

Failed CI Steps

Test Failures

[job] [logs] Scout: [ platform / workflows_management ] plugin / local-serverless-security_complete - Workflow execution - Alert triggers - should trigger workflow from alert
[job] [logs] FTR Configs #34 / serverless search UI Rule details Edit rule with deleted connector should show and update deleted connectors when there are existing connectors of the same type

Metrics [docs]

✅ unchanged

History

💛 Build #420821 was flaky d1de8f6
💔 Build #420040 failed 31b2cf3
💔 Build #419991 failed dd57aa9
💔 Build #419270 failed cff83d0
💔 Build #418916 failed 4be2bfd
💔 Build #418264 failed 665651c

…stic#252171) ## Summary - Closes elastic/kibana-team#2566 - Includes elastic#260557 (for history group support in the example plugin app) - [x] Hold until after elastic#259497 ### Changes **`data-test-subj` attributes** — Added to flyout buttons, switches, and close controls in both the EuiFlyout component and Overlays API implementations to enable reliable test selectors. **Scout UI tests** — Two tests covering: - Open a main flyout and a child flyout, verify cascade close behavior - Open a push flyout, open a second overlay flyout, verify Back button navigation **Minor Flyout System Examples plugin app refactoring** - See elastic#252171 (comment) ### To run locally ```bash node scripts/scout.js start-server \ --arch stateful \ --domain classic \ --serverConfigSet examples # terminal two: run the tests npx playwright test --config examples/flyout_system/test/scout_examples/ui/playwright.config.ts \ --project local \ --grep @local-stateful-classic \ --headed ``` --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

tsullivan requested review from a team as code owners February 6, 2026 19:39

tsullivan added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting labels Feb 6, 2026

tsullivan marked this pull request as draft February 7, 2026 17:17

dmlemeshko reviewed Feb 10, 2026

View reviewed changes

...hared/kbn-scout/src/servers/configs/config_sets/examples/stateful/classic.stateful.config.ts Show resolved Hide resolved

tsullivan force-pushed the scout-testing/flyout-system branch from cf72006 to abe3625 Compare February 12, 2026 18:28

tsullivan added the blocked label Feb 18, 2026

tsullivan force-pushed the scout-testing/flyout-system branch from abe3625 to 6373eff Compare March 26, 2026 21:32

tsullivan commented Mar 26, 2026

View reviewed changes

scout: support tests in example plugins

24a1c71

tsullivan mentioned this pull request Mar 27, 2026

[scout] support example plugins #259170

Merged

tsullivan added 2 commits March 27, 2026 09:57

[Flyout System] Automated functional tests using Scout framework

4d2cb77

add test error

c978c85

tsullivan force-pushed the scout-testing/flyout-system branch from bf0379c to c978c85 Compare March 27, 2026 16:57

kibanamachine and others added 2 commits March 27, 2026 17:24

Changes from node scripts/eslint_all_files --no-cache --fix

665651c

Merge branch 'main' into scout-testing/flyout-system

4be2bfd

tsullivan marked this pull request as ready for review March 30, 2026 14:45

tsullivan added the Team:SharedUX Platform AppEx-SharedUX (formerly Global Experience) t// label Mar 30, 2026

tsullivan added 2 commits March 30, 2026 15:14

Merge branch 'main' into scout-testing/flyout-system

4efc75f

update .buildkite/scout_ci_config.yml

cff83d0

tsullivan requested a review from a team as a code owner March 30, 2026 22:27

jbudz approved these changes Mar 30, 2026

View reviewed changes

kowalczyk-krzysztof approved these changes Mar 31, 2026

View reviewed changes

tsullivan added 3 commits March 31, 2026 11:34

Merge branch 'main' into scout-testing/flyout-system

6a55d63

fix tsx comment placement

dd57aa9

use manifest.plugin.id from kibana.jsonc for scout ci config

31b2cf3

tsullivan added 8 commits April 1, 2026 10:18

Merge branch 'main' into scout-testing/flyout-system

5c9f07a

remove test failure

3e2f87a

add historyKey

aa41dc6

refactor non-session flyouts

e86ba50

clean up filler text

2274820

page sections tweak

fa6da37

Merge branch 'main' into flyouts/example-plugin-history-fixes

6e396a7

Merge branch 'flyouts/example-plugin-history-fixes' into scout-testin…

d1de8f6

…g/flyout-system

tsullivan mentioned this pull request Apr 1, 2026

[Flyouts Example Plugin] Use historyKey for flyout demos #260557

Closed

11 tasks

macroscopeapp bot reviewed Apr 1, 2026

View reviewed changes

examples/flyout_system/public/components/app.tsx Outdated Show resolved Hide resolved

wrap the historyKey in useMemo

c645ba0

tsullivan merged commit 2d8ebee into elastic:main Apr 2, 2026
20 checks passed

kibanamachine added the v9.4.0 label Apr 2, 2026

Conversation

tsullivan commented Feb 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

To run locally

Uh oh!

tsullivan commented Feb 7, 2026

Uh oh!

dmlemeshko commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Dosant commented Feb 9, 2026

Uh oh!

tsullivan commented Feb 9, 2026

Uh oh!

dmlemeshko commented Feb 10, 2026

Uh oh!

Uh oh!

tsullivan commented Feb 18, 2026

Uh oh!

tsullivan Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Mar 30, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tsullivan commented Mar 31, 2026

Uh oh!

tsullivan commented Mar 31, 2026

Uh oh!

Uh oh!

tsullivan commented Apr 1, 2026

Summary

Screenshots

Uh oh!

elasticmachine commented Apr 2, 2026

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

tsullivan commented Feb 6, 2026 •

edited

Loading

dmlemeshko commented Feb 9, 2026 •

edited

Loading

tsullivan Mar 26, 2026 •

edited

Loading