Skip to content

[SLO] Migrate synthetics to scout !!#225035

Merged
shahzad31 merged 35 commits intoelastic:mainfrom
shahzad31:slo-scout
Jul 14, 2025
Merged

[SLO] Migrate synthetics to scout !!#225035
shahzad31 merged 35 commits intoelastic:mainfrom
shahzad31:slo-scout

Conversation

@shahzad31
Copy link
Copy Markdown
Contributor

@shahzad31 shahzad31 commented Jun 24, 2025 •
edited by mgiota
Loading

Summary

Migrate synthetics based SLO plugin tests to scout !!

How to test locally

If you want to try it out locally here are the commands:

Start the server

// ESS
node scripts/scout.js start-server --stateful

// Serverless
node scripts/scout.js start-server --serverless=[es|oblt|security]

Then run in another terminal:

// ESS
npx playwright test --config x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts --project=local --grep @ess

// Serverless
npx playwright test --config x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts --project=local --grep @svlOblt

@shahzad31 shahzad31 requested review from a team as code owners June 24, 2025 09:39
@botelastic botelastic bot added the Team:actionable-obs Formerly "obs-ux-management", responsible for SLO, o11y alerting, significant events, & synthetics. label Jun 24, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jun 24, 2025

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

@github-actions github-actions bot added the author:obs-ux-management PRs authored by the obs ux management team label Jun 24, 2025
@mgiota mgiota self-requested a review June 24, 2025 10:52
@shahzad31 shahzad31 added the ci:scout-ui-tests Run Scout UI tests label Jun 27, 2025
@shahzad31 shahzad31 requested a review from a team as a code owner June 27, 2025 09:48
@shahzad31 shahzad31 added release_note:skip Skip the PR/issue when compiling release notes backport:version Backport to applied version labels labels Jul 2, 2025
@cesco-f cesco-f self-assigned this Jul 7, 2025
@dmlemeshko dmlemeshko self-requested a review July 8, 2025 10:17
kdelemme
kdelemme reviewed Jul 8, 2025
View reviewed changes
x-pack/solutions/observability/plugins/slo/ui_tests/services/slo_data_service.ts Outdated
@@ -81,8 +69,8 @@ export class SLoDataService {
groupBy: ['user.id'],
Copy link
Copy Markdown
Contributor

@kdelemme kdelemme Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add this settings to the created slo payload, it might help with the time it takes to generate the SLO

settings: {
 preventInitialBackfill: true
}

Copy link
Copy Markdown
Contributor

@cesco-f cesco-f Jul 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

dmlemeshko
dmlemeshko reviewed Jul 8, 2025
View reviewed changes
x-pack/solutions/observability/plugins/slo/ui_tests/fixtures/page_objects/index.ts Outdated
}

export function extendPageObjects(pageObjects: ObltPageObjects, page: ScoutPage): SLOPageObjects {
page.setDefaultTimeout(60_000); // Set a default timeout for all page actions
Copy link
Copy Markdown
Contributor

@dmlemeshko dmlemeshko Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it too much? Every testing tool is encouraging to use explicit waiting over implicit, could you share if there was issue relying on defaults?

Copy link
Copy Markdown
Contributor

@cesco-f cesco-f Jul 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed the set default timeout but kept the timeout for the tests in slos_overview.spec.ts to 3 minutes because even with @kdelemme's suggestion it takes a long time to generate SLOs.

Screenshot 2025-07-09 at 10 32 45

dmlemeshko
dmlemeshko reviewed Jul 8, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@dmlemeshko dmlemeshko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left few questions + you need to update structure to follow a recent change #226411 (sorry for extra work)

mgiota
mgiota requested changes Jul 8, 2025
View reviewed changes
x-pack/solutions/observability/plugins/slo/ui_tests/README.md Outdated

```bash
// ESS
npx playwright test --config x-pack/solutions/observability/plugins/slo/ui_tests/playwright.config.ts --project=local --grep @ess
Copy link
Copy Markdown
Contributor

@mgiota mgiota Jul 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was a recent restructuring of scout test directory in this PR. Which basically means we should move stuff under slo/test/scout/ui folder instead of slo/ui_tests

Copy link
Copy Markdown
Contributor

@mgiota mgiota Jul 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I addressed restructuring in this commit and updated README here

mgiota
mgiota reviewed Jul 8, 2025
View reviewed changes
.buildkite/scout_ci_config.yml Outdated
ui_tests:
enabled:
- slo
disabled:
Copy link
Copy Markdown
Contributor

@mgiota mgiota Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmlemeshko This is just for testing? Let's not forget to re-enable the plugins.

Copy link
Copy Markdown
Contributor

@dmlemeshko dmlemeshko Jul 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

totally, this is a workaround before we add flaky-test-runner support for Scout

@cesco-f cesco-f requested a review from dmlemeshko July 14, 2025 08:17
dmlemeshko
dmlemeshko reviewed Jul 14, 2025
View reviewed changes
x-pack/solutions/observability/plugins/slo/test/scout/ui/tests/slos_overview.spec.ts Outdated
Comment on lines +27 to +44
test('Go to slos overview', async ({ page }) => {
// Already navigated in beforeEach
// This test ensures the page loads
expect(page).toBeDefined();
});

test('validate data retention tab', async ({ page }) => {
await expect(async () => {
await page.getByTestId('querySubmitButton').click();

await expect
.poll(() => page.locator('text=Test Stack SLO').count(), { timeout: 1000 })
.toBeGreaterThan(5);
}).toPass({
intervals: [10000],
timeout: TEST_TIMEOUT,
});
});
Copy link
Copy Markdown
Contributor

@dmlemeshko dmlemeshko Jul 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it makes sense to combine it into a single test block? It does look like a single flow and 2nd scenario won't work if page is not loaded anyways

Copy link
Copy Markdown
Contributor

@cesco-f cesco-f Jul 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

dmlemeshko
dmlemeshko approved these changes Jul 14, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@dmlemeshko dmlemeshko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left a small nit

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jul 14, 2025

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/scout-oblt 76 77 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
observability 1.3MB 1.3MB +69.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
@kbn/scout-oblt 3 4 +1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
observability 95.8KB 95.8KB +39.0B
Unknown metric groups

API count

id before after diff
@kbn/scout-oblt 370 371 +1

ESLint disabled line counts

id before after diff
slo 16 15 -1

Total ESLint disabled count

id before after diff
slo 20 19 -1

History

cc @cesco-f

@shahzad31 shahzad31 merged commit 726042c into elastic:main Jul 14, 2025
11 of 12 checks passed
@shahzad31 shahzad31 deleted the slo-scout branch July 14, 2025 12:09
@kibanamachine kibanamachine added v9.2.0 backport missing Added to PRs automatically when the are determined to be missing a backport. labels Jul 14, 2025
@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Jul 15, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 225035 locally
cc: @cesco-f

@shahzad31 shahzad31 added backport:skip This PR does not require backporting and removed backport:version Backport to applied version labels labels Jul 15, 2025
@kibanamachine kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Jul 15, 2025
Bluefinger pushed a commit to Bluefinger/kibana that referenced this pull request Jul 22, 2025
@shahzad31 @kibanamachine
@cesco-f @dmlemeshko @mgiota @Bluefinger
[SLO] Migrate synthetics to scout !! (elastic#225035)
ace73c0 
## Summary

Migrate synthetics based SLO plugin tests to scout  !!

### How to test locally

If you want to try it out locally here are the commands:

Start the server

```bash
// ESS
node scripts/scout.js start-server --stateful

// Serverless
node scripts/scout.js start-server --serverless=[es|oblt|security]
```

Then run in another terminal:

```bash
// ESS
npx playwright test --config x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts --project=local --grep @ess

// Serverless
npx playwright test --config x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts --project=local --grep @svlOblt
```

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Francesco Fagnani <francesco.fagnani@elastic.co>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Co-authored-by: Panagiota Mitsopoulou <giota85@gmail.com>
Co-authored-by: Francesco Fagnani <fagnani.francesco@gmail.com>
kertal pushed a commit to kertal/kibana that referenced this pull request Jul 25, 2025
@shahzad31 @kibanamachine
@cesco-f @dmlemeshko @mgiota @kertal
[SLO] Migrate synthetics to scout !! (elastic#225035)
6aaa30d 
## Summary

Migrate synthetics based SLO plugin tests to scout  !!

### How to test locally

If you want to try it out locally here are the commands:

Start the server

```bash
// ESS
node scripts/scout.js start-server --stateful

// Serverless
node scripts/scout.js start-server --serverless=[es|oblt|security]
```

Then run in another terminal:

```bash
// ESS
npx playwright test --config x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts --project=local --grep @ess

// Serverless
npx playwright test --config x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts --project=local --grep @svlOblt
```

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Francesco Fagnani <francesco.fagnani@elastic.co>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Co-authored-by: Panagiota Mitsopoulou <giota85@gmail.com>
Co-authored-by: Francesco Fagnani <fagnani.francesco@gmail.com>
dmlemeshko pushed a commit to dmlemeshko/kibana that referenced this pull request Mar 9, 2026
@shahzad31
[SLO] Migrate synthetics to scout !! (elastic#225035)
5958f35 
## Summary

Migrate synthetics based SLO plugin tests to scout  !!

### How to test locally

If you want to try it out locally here are the commands:

Start the server

```bash
// ESS
node scripts/scout.js start-server --stateful

// Serverless
node scripts/scout.js start-server --serverless=[es|oblt|security]
```

Then run in another terminal:

```bash
// ESS
npx playwright test --config x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts --project=local --grep @ess

// Serverless
npx playwright test --config x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts --project=local --grep @svlOblt
```

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Francesco Fagnani <francesco.fagnani@elastic.co>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Co-authored-by: Panagiota Mitsopoulou <giota85@gmail.com>
Co-authored-by: Francesco Fagnani <fagnani.francesco@gmail.com>
(cherry picked from commit 726042c)

# Conflicts:
#	.buildkite/pipelines/pull_request/slo_plugin_e2e.yml
#	.buildkite/scout_ci_config.yml
#	.buildkite/scripts/steps/functional/slo_plugin_e2e.sh
#	x-pack/solutions/observability/packages/kbn-scout-oblt/src/playwright/fixtures/parallel_run_fixtures.ts
#	x-pack/solutions/observability/packages/kbn-scout-oblt/src/playwright/fixtures/single_thread_fixtures.ts
#	x-pack/solutions/observability/packages/kbn-scout-oblt/src/playwright/fixtures/types.ts
#	x-pack/solutions/observability/packages/kbn-scout-oblt/src/playwright/fixtures/worker/index.ts
#	x-pack/solutions/observability/plugins/slo/e2e/services/slo_data_service.ts
#	x-pack/solutions/observability/plugins/slo/e2e/tsconfig.json
dmlemeshko pushed a commit to dmlemeshko/kibana that referenced this pull request Mar 9, 2026
@shahzad31
[SLO] Migrate synthetics to scout !! (elastic#225035)
82f98d9 
## Summary

Migrate synthetics based SLO plugin tests to scout  !!

### How to test locally

If you want to try it out locally here are the commands:

Start the server

```bash
// ESS
node scripts/scout.js start-server --stateful

// Serverless
node scripts/scout.js start-server --serverless=[es|oblt|security]
```

Then run in another terminal:

```bash
// ESS
npx playwright test --config x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts --project=local --grep @ess

// Serverless
npx playwright test --config x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts --project=local --grep @svlOblt
```

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Francesco Fagnani <francesco.fagnani@elastic.co>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Co-authored-by: Panagiota Mitsopoulou <giota85@gmail.com>
Co-authored-by: Francesco Fagnani <fagnani.francesco@gmail.com>
(cherry picked from commit 726042c)

# Conflicts:
#	.buildkite/pipelines/pull_request/slo_plugin_e2e.yml
#	.buildkite/scout_ci_config.yml
#	.buildkite/scripts/steps/functional/slo_plugin_e2e.sh
#	x-pack/solutions/observability/packages/kbn-scout-oblt/src/playwright/fixtures/parallel_run_fixtures.ts
#	x-pack/solutions/observability/packages/kbn-scout-oblt/src/playwright/fixtures/single_thread_fixtures.ts
#	x-pack/solutions/observability/packages/kbn-scout-oblt/src/playwright/fixtures/types.ts
#	x-pack/solutions/observability/packages/kbn-scout-oblt/src/playwright/fixtures/worker/index.ts
#	x-pack/solutions/observability/plugins/slo/e2e/services/slo_data_service.ts
#	x-pack/solutions/observability/plugins/slo/e2e/tsconfig.json
@dmlemeshko dmlemeshko mentioned this pull request Mar 9, 2026
Merged
@dmlemeshko
Copy link
Copy Markdown
Contributor

dmlemeshko commented Mar 9, 2026

💚 All backports created successfully

Status Branch Result
✅ 8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

dmlemeshko added a commit that referenced this pull request Mar 10, 2026
@dmlemeshko @shahzad31 @kibanamachine
[8.19] [SLO] Migrate synthetics to scout !! (#225035) (#256684)
5d25580 
# Backport

This will backport the following commits from `main` to `8.19`:
- [[SLO] Migrate synthetics to scout !!
(#225035)](#225035)

<!--- Backport version: 10.2.0 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT
[{"author":{"name":"Shahzad","email":"shahzad31comp@gmail.com"},"sourceCommit":{"committedDate":"2025-07-14T12:09:21Z","message":"[SLO]
Migrate synthetics to scout !! (#225035)\n\n## Summary\n\nMigrate
synthetics based SLO plugin tests to scout !!\n\n### How to test
locally\n\nIf you want to try it out locally here are the
commands:\n\nStart the server\n\n```bash\n// ESS\nnode scripts/scout.js
start-server --stateful\n\n// Serverless\nnode scripts/scout.js
start-server --serverless=[es|oblt|security]\n```\n\nThen run in another
terminal:\n\n```bash\n// ESS\nnpx playwright test --config
x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts
--project=local --grep @ess\n\n// Serverless\nnpx playwright test
--config
x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts
--project=local --grep @svlOblt\n```\n\n---------\n\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by:
Francesco Fagnani <francesco.fagnani@elastic.co>\nCo-authored-by:
Dzmitry Lemechko <dzmitry.lemechko@elastic.co>\nCo-authored-by:
Panagiota Mitsopoulou <giota85@gmail.com>\nCo-authored-by: Francesco
Fagnani
<fagnani.francesco@gmail.com>","sha":"726042c9a91729511bbb4a7866890536f15b4010","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:skip","Team:actionable-obs","author:obs-ux-management","ci:scout-ui-tests","v9.2.0"],"title":"[SLO]
Migrate synthetics to scout
!!","number":225035,"url":"https://github.com/elastic/kibana/pull/225035","mergeCommit":{"message":"[SLO]
Migrate synthetics to scout !! (#225035)\n\n## Summary\n\nMigrate
synthetics based SLO plugin tests to scout !!\n\n### How to test
locally\n\nIf you want to try it out locally here are the
commands:\n\nStart the server\n\n```bash\n// ESS\nnode scripts/scout.js
start-server --stateful\n\n// Serverless\nnode scripts/scout.js
start-server --serverless=[es|oblt|security]\n```\n\nThen run in another
terminal:\n\n```bash\n// ESS\nnpx playwright test --config
x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts
--project=local --grep @ess\n\n// Serverless\nnpx playwright test
--config
x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts
--project=local --grep @svlOblt\n```\n\n---------\n\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by:
Francesco Fagnani <francesco.fagnani@elastic.co>\nCo-authored-by:
Dzmitry Lemechko <dzmitry.lemechko@elastic.co>\nCo-authored-by:
Panagiota Mitsopoulou <giota85@gmail.com>\nCo-authored-by: Francesco
Fagnani
<fagnani.francesco@gmail.com>","sha":"726042c9a91729511bbb4a7866890536f15b4010"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/225035","number":225035,"mergeCommit":{"message":"[SLO]
Migrate synthetics to scout !! (#225035)\n\n## Summary\n\nMigrate
synthetics based SLO plugin tests to scout !!\n\n### How to test
locally\n\nIf you want to try it out locally here are the
commands:\n\nStart the server\n\n```bash\n// ESS\nnode scripts/scout.js
start-server --stateful\n\n// Serverless\nnode scripts/scout.js
start-server --serverless=[es|oblt|security]\n```\n\nThen run in another
terminal:\n\n```bash\n// ESS\nnpx playwright test --config
x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts
--project=local --grep @ess\n\n// Serverless\nnpx playwright test
--config
x-pack/solutions/observability/plugins/slo/test/scout/ui/playwright.config.ts
--project=local --grep @svlOblt\n```\n\n---------\n\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by:
Francesco Fagnani <francesco.fagnani@elastic.co>\nCo-authored-by:
Dzmitry Lemechko <dzmitry.lemechko@elastic.co>\nCo-authored-by:
Panagiota Mitsopoulou <giota85@gmail.com>\nCo-authored-by: Francesco
Fagnani
<fagnani.francesco@gmail.com>","sha":"726042c9a91729511bbb4a7866890536f15b4010"}}]}]
BACKPORT-->

---------

Co-authored-by: Shahzad <shahzad31comp@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kdelemme kdelemme kdelemme left review comments

@cesco-f cesco-f cesco-f left review comments

@mgiota mgiota mgiota approved these changes

@jbudz jbudz jbudz approved these changes

@dmlemeshko dmlemeshko dmlemeshko approved these changes

Assignees

@cesco-f cesco-f

Labels

author:obs-ux-management PRs authored by the obs ux management team backport:skip This PR does not require backporting ci:scout-ui-tests Run Scout UI tests release_note:skip Skip the PR/issue when compiling release notes Team:actionable-obs Formerly "obs-ux-management", responsible for SLO, o11y alerting, significant events, & synthetics. v8.19.13 v9.2.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@shahzad31 @elasticmachine @mgiota @dmlemeshko @kibanamachine @kdelemme @jbudz @cesco-f