[EDR Workflows] Fix and unskip Unable to create Package Policy via Fleet! FTR api tests#250075
Conversation
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#10572[❌] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/artifacts/trial_license_complete_tier/configs/endpoint_exceptions_moved_ff.ess.config.ts: 0/25 tests passed. |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#10576[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/artifacts/trial_license_complete_tier/configs/endpoint_exceptions_moved_ff.ess.config.ts: 25/25 tests passed. |
|
Pinging @elastic/security-defend-workflows (Team:Defend Workflows) |
💚 Build Succeeded
Metrics [docs]
History
|
|
Starting backport for target branches: 8.19, 9.1, 9.2, 9.3 |
…leet!` FTR api tests (elastic#250075) ## Summary This PR fixes the flakiness around creating an Endpoint package policy using `endpointPolicyTestResources.createPolicy()`. When an Endpoint package policy is created, the default rules are installed in the background, which process can be flaky, or, together with other functionalities, simply too slow (i.e. taking more than 2 minutes, causing `socket hang up` error). Therefore, there's a workaround by the Detections team to install a mock `security_detection_engine` package. This fix workaround is applied in the EDR API tests as well. For more info, see the original workaround PR: - elastic#227689 Flaky runner: 25/25 ✅ for all 6 test groups: - artifacts ESS + serverless - artifacts with Endpoint exceptions moved FF enabled ESS + serverless - spaces ESS + serverless https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/10576/steps/canvas fix elastic#249780 fix elastic#249463 fix elastic#249468 fix elastic#248923 fix elastic#248759 fix elastic#248578 fix elastic#248554 fix elastic#247416 fix elastic#247211 fix elastic#246564 fix elastic#246341 fix elastic#246450 fix elastic#246356 fix elastic#246207 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed (cherry picked from commit d36d440)
💔 Some backports could not be created
Note: Successful backport PRs will be merged automatically after passing CI. Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
… via Fleet!` FTR api tests (#250075) (#250450) # Backport This will backport the following commits from `main` to `9.3`: - [[EDR Workflows] Fix and unskip `Unable to create Package Policy via Fleet!` FTR api tests (#250075)](#250075) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Gergő Ábrahám","email":"gergo.abraham@elastic.co"},"sourceCommit":{"committedDate":"2026-01-26T15:46:47Z","message":"[EDR Workflows] Fix and unskip `Unable to create Package Policy via Fleet!` FTR api tests (#250075)\n\n## Summary\n\nThis PR fixes the flakiness around creating an Endpoint package policy\nusing `endpointPolicyTestResources.createPolicy()`.\n\nWhen an Endpoint package policy is created, the default rules are\ninstalled in the background, which process can be flaky, or, together\nwith other functionalities, simply too slow (i.e. taking more than 2\nminutes, causing `socket hang up` error). Therefore, there's a\nworkaround by the Detections team to install a mock\n`security_detection_engine` package. This fix workaround is applied in\nthe EDR API tests as well.\n\nFor more info, see the original workaround PR:\n- https://github.com/elastic/kibana/pull/227689\n\nFlaky runner: 25/25 ✅ for all 6 test groups:\n- artifacts ESS + serverless\n- artifacts with Endpoint exceptions moved FF enabled ESS + serverless\n- spaces ESS + serverless\n\n\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/10576/steps/canvas\n\nfix #249780\nfix #249463\nfix #249468\nfix #248923\nfix #248759\nfix #248578\nfix #248554\nfix #247416\nfix #247211\nfix #246564\nfix #246341\nfix #246450\nfix #246356\nfix #246207\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed","sha":"d36d440e1dff84f0ca2306c97dd08fed16259c22","branchLabelMapping":{"^v9.4.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Defend Workflows","backport:all-open","v9.4.0"],"title":"[EDR Workflows] Fix and unskip `Unable to create Package Policy via Fleet!` FTR api tests","number":250075,"url":"https://github.com/elastic/kibana/pull/250075","mergeCommit":{"message":"[EDR Workflows] Fix and unskip `Unable to create Package Policy via Fleet!` FTR api tests (#250075)\n\n## Summary\n\nThis PR fixes the flakiness around creating an Endpoint package policy\nusing `endpointPolicyTestResources.createPolicy()`.\n\nWhen an Endpoint package policy is created, the default rules are\ninstalled in the background, which process can be flaky, or, together\nwith other functionalities, simply too slow (i.e. taking more than 2\nminutes, causing `socket hang up` error). Therefore, there's a\nworkaround by the Detections team to install a mock\n`security_detection_engine` package. This fix workaround is applied in\nthe EDR API tests as well.\n\nFor more info, see the original workaround PR:\n- https://github.com/elastic/kibana/pull/227689\n\nFlaky runner: 25/25 ✅ for all 6 test groups:\n- artifacts ESS + serverless\n- artifacts with Endpoint exceptions moved FF enabled ESS + serverless\n- spaces ESS + serverless\n\n\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/10576/steps/canvas\n\nfix #249780\nfix #249463\nfix #249468\nfix #248923\nfix #248759\nfix #248578\nfix #248554\nfix #247416\nfix #247211\nfix #246564\nfix #246341\nfix #246450\nfix #246356\nfix #246207\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed","sha":"d36d440e1dff84f0ca2306c97dd08fed16259c22"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.4.0","branchLabelMappingKey":"^v9.4.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/250075","number":250075,"mergeCommit":{"message":"[EDR Workflows] Fix and unskip `Unable to create Package Policy via Fleet!` FTR api tests (#250075)\n\n## Summary\n\nThis PR fixes the flakiness around creating an Endpoint package policy\nusing `endpointPolicyTestResources.createPolicy()`.\n\nWhen an Endpoint package policy is created, the default rules are\ninstalled in the background, which process can be flaky, or, together\nwith other functionalities, simply too slow (i.e. taking more than 2\nminutes, causing `socket hang up` error). Therefore, there's a\nworkaround by the Detections team to install a mock\n`security_detection_engine` package. This fix workaround is applied in\nthe EDR API tests as well.\n\nFor more info, see the original workaround PR:\n- https://github.com/elastic/kibana/pull/227689\n\nFlaky runner: 25/25 ✅ for all 6 test groups:\n- artifacts ESS + serverless\n- artifacts with Endpoint exceptions moved FF enabled ESS + serverless\n- spaces ESS + serverless\n\n\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/10576/steps/canvas\n\nfix #249780\nfix #249463\nfix #249468\nfix #248923\nfix #248759\nfix #248578\nfix #248554\nfix #247416\nfix #247211\nfix #246564\nfix #246341\nfix #246450\nfix #246356\nfix #246207\n\n### Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers should verify this PR satisfies this list as well.\n\n- [x] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed","sha":"d36d440e1dff84f0ca2306c97dd08fed16259c22"}}]}] BACKPORT--> Co-authored-by: Gergő Ábrahám <gergo.abraham@elastic.co>
Note all tests are enabled and non-flaky on 8.19/9.2, so no need to backport to those versions ✅ |
Summary
This PR fixes the flakiness around creating an Endpoint package policy using
endpointPolicyTestResources.createPolicy().When an Endpoint package policy is created, the default rules are installed in the background, which process can be flaky, or, together with other functionalities, simply too slow (i.e. taking more than 2 minutes, causing
socket hang uperror). Therefore, there's a workaround by the Detections team to install a mocksecurity_detection_enginepackage. This fix workaround is applied in the EDR API tests as well.For more info, see the original workaround PR:
Flaky runner: 25/25 ✅ for all 6 test groups:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/10576/steps/canvas
fix #249780
fix #249463
fix #249468
fix #248923
fix #248759
fix #248578
fix #248554
fix #247416
fix #247211
fix #246564
fix #246341
fix #246450
fix #246356
fix #246207
Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.