[One Workflow] Add Scout API test scaffold and execution tests by skynetigor · Pull Request #256300 · elastic/kibana

skynetigor · 2026-03-05T17:36:17Z

Motivation

The workflows_management plugin had Scout UI tests (test/scout_workflows_ui/ui/) but lacked any API-level tests. Workflow execution behaviour — concurrency control strategies and scheduled execution — is critical to verify at the API layer, where timing-sensitive scenarios can be tested without the overhead of browser-based UI interactions.

This PR introduces a Scout API test project alongside the existing UI test project in the same scout_workflows_ui root, and adds the first set of API tests covering workflow execution concurrency control and scheduled workflows.

What was added

Scout API test scaffold

api/playwright.config.ts — Playwright config pointing at ./tests
api/fixtures/constants.ts — shared headers (kbn-xsrf, x-elastic-internal-origin) and WORKFLOWS_API_BASE path
api/fixtures/workflows_api_service.ts — WorkflowsApiService class wrapping apiClient with methods: create, run, update, getExecution, getExecutions, waitForTermination, bulkDelete, deleteAll
api/fixtures/index.ts — extended apiTest fixture exposing getWorkflowsApi factory (accepts RoleApiCredentials)

Concurrency control tests (`concurency_control.spec.ts`)

cancel-in-progress strategy — triggers 3 concurrent executions with the same concurrency key; asserts that all previous executions are cancelled and only the latest completes with all 4 steps
drop strategy — triggers 3 concurrent executions; asserts that only the first execution completes while subsequent ones are skipped with 0 steps

Scheduled workflow tests (`scheduled_workflow.spec.ts`)

Enabling a scheduled workflow triggers executions automatically — creates a workflow with every: 5s, enables it, waits ~16s, verifies 2+ executions fired and all completed successfully with correct inter-execution timing
Disabling a scheduled workflow stops new executions — enables the workflow, waits for executions, disables it, waits again, asserts at most 1 extra in-flight execution appeared after disable
Scheduled executions do not overlap — uses a long-running workflow (6s wait step > 5s interval) to prove the scheduler waits for the previous run to finish before starting the next, by asserting consecutive start times are spaced >10s apart

- Introduced Playwright configuration file for testing workflows. - Created constants for common headers and API base URL. - Implemented WorkflowsApiService to handle API interactions, including methods for creating, running, and managing workflow executions. - Added tests for workflow execution concurrency control and scheduled workflows to ensure proper functionality and behavior. This commit lays the groundwork for comprehensive testing of the workflows management feature.

- Updated the method for evaluating expressions from `evaluateExpression` to `render` in the ConcurrencyManager class. - This change enhances clarity and aligns with the updated templating engine API. No functional changes are expected; this is primarily a refactor for improved code maintainability.

- Updated import statements to use `type` for type-only imports in `concurency_control.spec.ts` and `scheduled_workflow.spec.ts`. - Reformatted code for better clarity and consistency, including adjustments to promise handling and array filtering. - Removed unnecessary comments and whitespace to enhance code cleanliness. These changes aim to improve the maintainability and readability of the test files without altering their functionality.

src/platform/plugins/shared/workflows_management/test/scout_workflows_ui/api/.eslintrc.json

...s_management/test/scout_workflows_ui/api/tests/workflow_execution/concurency_control.spec.ts

- Modified test cases in `concurrency_manager.test.ts` to ensure that null, undefined, and empty string evaluations return null instead of the original key. - Adjusted the `evaluateConcurrencyKey` method in `concurrency_manager.ts` to return null for empty strings and to streamline the rendering logic, enhancing clarity and consistency. These changes improve the accuracy of concurrency key evaluations and ensure that the tests reflect the intended behavior.

..._management/test/scout_workflows_ui/api/tests/workflow_execution/concurrency_control.spec.ts

...ns/shared/workflows_management/test/scout_workflows_ui/api/fixtures/workflows_api_service.ts

This commit introduces a new ESLint configuration file (.eslintrc.js) for the Workflows API tests, disabling the rule that requires the API client in API tests. The previous .eslintrc.json file has been removed as part of this update.

This commit updates the Workflows API service by converting it from a plain object to a class-based implementation. The methods for creating, updating, bulk creating, deleting, and retrieving workflows have been refactored to use async/await syntax for better readability and error handling. Additionally, the import statements have been adjusted to reflect the new structure.

This commit updates the Workflows API fixtures by transitioning from a role-based API client to a more structured approach using the KbnClient. The `spaceTest` function has been introduced to extend the API services, allowing for better integration with the Workflows API Service. Import statements have also been adjusted to reflect these changes.

This commit refactors the Workflows API tests to utilize the `spaceTest` context, enhancing the structure and organization of the tests. The tests have been updated to accommodate the new space-based API service interactions, including adjustments to the import statements and the workflow creation and deletion processes. Additionally, the concurrency control and scheduled workflow tests have been modified to ensure proper handling of space-specific workflows.

skynetigor · 2026-03-09T11:23:27Z

...s_management/test/scout_workflows_ui/api/tests/workflow_execution/scheduled_workflow.spec.ts

+`;
+
+// FLAKY: https://github.com/elastic/security-team/issues/16272
+spaceTest.describe.skip('Scheduled workflow execution', { tag: tags.deploymentAgnostic }, () => {


Skipped due to inconsistent interval between scheduled executions

This commit further refines the Workflows API tests by removing the explicit use of space IDs in workflow creation, deletion, and other API interactions. The tests have been updated to leverage the context provided by the `spaceTest`, enhancing readability and maintainability. Additionally, adjustments have been made to ensure that the API service methods align with the new structure, promoting a cleaner and more efficient testing framework.

…igor/kibana into scout_tests_for_workflows

elasticmachine · 2026-03-09T18:09:53Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: e42bed5

Failed CI Steps

Test Failures

[job] [logs] Jest Tests #3 / discover responsive sidebar should render buttons in data view picker correctly
[job] [logs] Scout: [ platform / spaces ] plugin / local-serverless-observability_complete - Spaces selection - as Admin - displays the space selection menu in header

Metrics [docs]

✅ unchanged

History

💔 Build #406850 failed 80eb667
💔 Build #406842 failed b9b6cb0
💔 Build #406802 failed a91acdd
💔 Build #406720 failed 07ceb1b
💔 Build #406652 failed 5b31e91
💔 Build #406284 failed 9972b95

yngrdyn

LGTM

…e_fix * commit '565f7545c422192218b803874fbdf93e8d8f08ee': (27 commits) [Lens API] ESQL schema for XY separately for Agent and some small token optimizations (elastic#256885) Fix "Accessing resource attributes before async attributes settled" telemetry error (elastic#256880) [Security Solution][Attacks/Alerts][Attacks page][Table section] Preserver "Sort by" state on Attacks page (elastic#256717) (elastic#256795) [APM] Improve redirect with default date range guard (elastic#256887) [Security Solution][Attacks/Alerts][Attacks page][Table section] Add assignees avatars to the group component (elastic#250126) (elastic#256901) [Docs] add xpack.alerting.rules.maxScheduledPerMinute setting description (elastic#257041) [SO] Fix non-deterministic ordering in nested find API integration tests (elastic#256447) [Write-restricted dashboards] Update user profile retrieval for getShouldAddAccessControl (elastic#255065) [One Workflow] Add Scout API test scaffold and execution tests (elastic#256300) [Fleet] add use_apm if dynamic_signal_types are enabled (elastic#256429) [Fleet] ignore data streams starting with `.` in Fleet API (elastic#256625) [ES|QL] METRICS_INFO support: columns_after & summary (elastic#256758) [Agent Builder] Agent plugins: initial installation support (elastic#256478) [Streams] Add field descriptions and documentation-only field overrides (elastic#255136) [api-docs] 2026-03-11 Daily api_docs build (elastic#257023) [Security Solution] fix alerts page infinite loading state due to data view error (elastic#256983) [Logging] Add `service.*` global fields (elastic#256878) [Canvas] Apply embeddable transforms to embeddable elements (elastic#252191) [table_list_view_table] stabilize jest test (elastic#254991) [Obs AI] get_index_info: add unit tests (elastic#256802) ...

## Motivation The `workflows_management` plugin had Scout UI tests (`test/scout_workflows_ui/ui/`) but lacked any API-level tests. Workflow execution behaviour — concurrency control strategies and scheduled execution — is critical to verify at the API layer, where timing-sensitive scenarios can be tested without the overhead of browser-based UI interactions. This PR introduces a Scout API test project alongside the existing UI test project in the same `scout_workflows_ui` root, and adds the first set of API tests covering workflow execution concurrency control and scheduled workflows. ## What was added ### Scout API test scaffold - `api/playwright.config.ts` — Playwright config pointing at `./tests` - `api/fixtures/constants.ts` — shared headers (`kbn-xsrf`, `x-elastic-internal-origin`) and `WORKFLOWS_API_BASE` path - `api/fixtures/workflows_api_service.ts` — `WorkflowsApiService` class wrapping `apiClient` with methods: `create`, `run`, `update`, `getExecution`, `getExecutions`, `waitForTermination`, `bulkDelete`, `deleteAll` - `api/fixtures/index.ts` — extended `apiTest` fixture exposing `getWorkflowsApi` factory (accepts `RoleApiCredentials`) ### Concurrency control tests (`concurency_control.spec.ts`) - **cancel-in-progress strategy** — triggers 3 concurrent executions with the same concurrency key; asserts that all previous executions are cancelled and only the latest completes with all 4 steps - **drop strategy** — triggers 3 concurrent executions; asserts that only the first execution completes while subsequent ones are skipped with 0 steps ### Scheduled workflow tests (`scheduled_workflow.spec.ts`) - **Enabling a scheduled workflow triggers executions automatically** — creates a workflow with `every: 5s`, enables it, waits ~16s, verifies 2+ executions fired and all completed successfully with correct inter-execution timing - **Disabling a scheduled workflow stops new executions** — enables the workflow, waits for executions, disables it, waits again, asserts at most 1 extra in-flight execution appeared after disable - **Scheduled executions do not overlap** — uses a long-running workflow (6s wait step > 5s interval) to prove the scheduler waits for the previous run to finish before starting the next, by asserting consecutive start times are spaced >10s apart --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

skynetigor added 2 commits March 5, 2026 18:30

skynetigor added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting labels Mar 5, 2026

skynetigor marked this pull request as ready for review March 5, 2026 17:38

skynetigor requested a review from a team as a code owner March 5, 2026 17:38

botelastic bot added the Team:One Workflow Team label for One Workflow (Workflow automation) label Mar 5, 2026

skynetigor commented Mar 5, 2026

View reviewed changes

src/platform/plugins/shared/workflows_management/test/scout_workflows_ui/api/.eslintrc.json Outdated Show resolved Hide resolved

Kiryous reviewed Mar 5, 2026

View reviewed changes

...s_management/test/scout_workflows_ui/api/tests/workflow_execution/concurency_control.spec.ts Outdated Show resolved Hide resolved

yngrdyn reviewed Mar 6, 2026

View reviewed changes

..._management/test/scout_workflows_ui/api/tests/workflow_execution/concurrency_control.spec.ts Show resolved Hide resolved

yngrdyn reviewed Mar 6, 2026

View reviewed changes

...ns/shared/workflows_management/test/scout_workflows_ui/api/fixtures/workflows_api_service.ts Outdated Show resolved Hide resolved

skynetigor added 11 commits March 6, 2026 17:46

Merge branch 'main' into scout_tests_for_workflows

ead18b4

refactor tests

d9cd79a

Update scheduled_workflow.spec.ts

9972b95

Add ESLint configuration for Workflows API tests

0a306dd

This commit introduces a new ESLint configuration file (.eslintrc.js) for the Workflows API tests, disabling the rule that requires the API client in API tests. The previous .eslintrc.json file has been removed as part of this update.

move

121e67b

move

5b6dbaa

change

f65f5ba

Update scheduled_workflow.spec.ts

8de2e0f

skynetigor commented Mar 9, 2026

View reviewed changes

skynetigor added 5 commits March 9, 2026 13:18

fix typo

8578af4

Merge branch 'main' into scout_tests_for_workflows

84106e5

Update scheduled workflow test timeouts and execution expectations

f498363

Merge branch 'scout_tests_for_workflows' of https://github.com/skynet…

07ceb1b

…igor/kibana into scout_tests_for_workflows

skynetigor and others added 6 commits March 9, 2026 14:57

add tests for workflow composition

a91acdd

Update index.ts

c76fdf0

Merge branch 'main' into scout_tests_for_workflows

b9b6cb0

Changes from node scripts/lint.js --fix

80eb667

Changes from node scripts/eslint_all_files --no-cache --fix

a45af7e

Merge branch 'main' into scout_tests_for_workflows

e42bed5

yngrdyn approved these changes Mar 11, 2026

View reviewed changes

skynetigor merged commit f7dda77 into elastic:main Mar 11, 2026
18 checks passed

kibanamachine added the v9.4.0 label Mar 11, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[One Workflow] Add Scout API test scaffold and execution tests#256300

[One Workflow] Add Scout API test scaffold and execution tests#256300
skynetigor merged 26 commits intoelastic:mainfrom
skynetigor:scout_tests_for_workflows

skynetigor commented Mar 5, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

skynetigor Mar 9, 2026

Uh oh!

elasticmachine commented Mar 9, 2026

Uh oh!

yngrdyn left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

skynetigor commented Mar 5, 2026

Motivation

What was added

Scout API test scaffold

Concurrency control tests (concurency_control.spec.ts)

Scheduled workflow tests (scheduled_workflow.spec.ts)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

skynetigor Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Mar 9, 2026

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

Uh oh!

yngrdyn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Concurrency control tests (`concurency_control.spec.ts`)

Scheduled workflow tests (`scheduled_workflow.spec.ts`)