Skip to content

[EDR Workflows] Unskip FTR tests that failed due to transient Fleet service unavailability#260519

Merged
szwarckonrad merged 8 commits intoelastic:mainfrom
szwarckonrad:unskip-edr-workflows-flaky-tests
Apr 1, 2026
Merged

[EDR Workflows] Unskip FTR tests that failed due to transient Fleet service unavailability#260519
szwarckonrad merged 8 commits intoelastic:mainfrom
szwarckonrad:unskip-edr-workflows-flaky-tests

Conversation

@szwarckonrad
Copy link
Copy Markdown
Contributor

Summary

Unskips 6 EDR Workflows FTR test suites that were skipped due to transient installFleetPackageByUpload failures (socket hang up / timeout) in CI. These were not caused by code defects — the Fleet package upload service was temporarily unavailable during the kibana-on-merge runs in late January 2026.

All 6 issues share the identical error:

Error: retry.tryWithRetries reached timeout 20000 ms waiting for 'installFleetPackageByUpload'
Error: socket hang up

The failures occurred in the before all hook (test setup), not in the test logic itself. The underlying service has since stabilized.

Test suites unskipped

Test Suite Issue
Policy Response Integration Tests - ESS #250458
Policy Response Integration Tests - Serverless #250698
Resolver Integration Tests - Serverless #250782
Resolver Integration Tests - ESS #250805
Role Migration Tests - Serverless #250957
Policy Tests - Serverless #250973

Files changed

All under x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/:

  • policy/trial_license_complete_tier/index.ts
  • policy_response/trial_license_complete_tier/index.ts
  • resolver/trial_license_complete_tier/index.ts
  • role_migrations/trial_license_complete_tier/index.ts
  • scripts_library/trial_license_complete_tier/index.ts

Closes #250458
Closes #250698
Closes #250782
Closes #250805
Closes #250957
Closes #250973

@szwarckonrad szwarckonrad added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:Defend Workflows “EDR Workflows” sub-team of Security Solution labels Mar 31, 2026
@szwarckonrad szwarckonrad self-assigned this Mar 31, 2026
@kibanamachine
Copy link
Copy Markdown
Contributor

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11314

[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/policy/trial_license_complete_tier/configs/ess.config.ts: 20/20 tests passed.
[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/policy/trial_license_complete_tier/configs/serverless.config.ts: 20/20 tests passed.
[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/policy_response/trial_license_complete_tier/configs/ess.config.ts: 20/20 tests passed.
[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/policy_response/trial_license_complete_tier/configs/serverless.config.ts: 20/20 tests passed.
[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/resolver/trial_license_complete_tier/configs/ess.config.ts: 20/20 tests passed.
[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/resolver/trial_license_complete_tier/configs/serverless.config.ts: 20/20 tests passed.
[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/role_migrations/trial_license_complete_tier/configs/ess.config.ts: 20/20 tests passed.
[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/role_migrations/trial_license_complete_tier/configs/serverless.config.ts: 20/20 tests passed.
[❌] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/scripts_library/trial_license_complete_tier/configs/ess.config.ts: 0/20 tests passed.
[❌] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/scripts_library/trial_license_complete_tier/configs/serverless.config.ts: 0/20 tests passed.

see run history

@kibanamachine
Copy link
Copy Markdown
Contributor

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11321

[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/scripts_library/trial_license_complete_tier/configs/ess.config.ts: 25/25 tests passed.
[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/scripts_library/trial_license_complete_tier/configs/serverless.config.ts: 25/25 tests passed.

see run history

@szwarckonrad szwarckonrad marked this pull request as ready for review March 31, 2026 19:37
@szwarckonrad szwarckonrad requested a review from a team as a code owner March 31, 2026 19:37
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

@szwarckonrad szwarckonrad enabled auto-merge (squash) April 1, 2026 16:09
@elasticmachine
Copy link
Copy Markdown
Contributor

⏳ Build in-progress, with failures

Failed CI Steps

Test Failures

  • [job] [logs] Scout: [ platform / streams_app-stateful-classic ] plugin / local-stateful-classic - Query streams - Create query stream - should properly handle errors for invalid query streams
  • [job] [logs] Scout: [ platform / streams_app-stateful-classic ] plugin / local-stateful-classic - Stream data routing - AI suggestions button - should show button when AI features are enabled

History

cc @szwarckonrad

@szwarckonrad szwarckonrad merged commit 44b1667 into elastic:main Apr 1, 2026
19 checks passed
mbondyra added a commit to mbondyra/kibana that referenced this pull request Apr 1, 2026
…heck

* commit '6f040b29a5220ce12886a9731f656613e50aff06': (34 commits)
  [Entity Analytics] Add entity resolution UI to service flyout (elastic#260504)
  [Dashboard] Fix setState in embeddables (elastic#260082)
  [EDR Workflows] Unskip FTR tests that failed due to transient Fleet service unavailability (elastic#260519)
  [Observability:Streams] Fix query streams error handling test (elastic#260777)
  [Alerting v2] Dispatcher grouping modes, throttle strategies, and matcher autosuggestion (elastic#260249)
  [Dashboard] State extraction as a consistent override (elastic#259839)
  [Alerting v2] [Rule authoring] Fix rule name validation and error visibility in create/edit flow (elastic#260337)
  [Fix] re-introduce sln breadcrumbs to unified rules (elastic#260289)
  [Security Solution][Endpoint] Updated kibana docs to include `xpack.securitySolution.maxEndpointScriptFileSize` as configurable in cloud (elastic#260568)
  [Alerting v2] updated the alerting-v2-constants package with artifacts constants, fix to the runbook max characters (elastic#260342)
  [Automatic Import V2] Provide user tooltips (elastic#260725)
  [One Workflow] Deduplicate step types by base type in workflow list (elastic#260763)
  [Security Solution] Execution results UI: Enable the feature flag (elastic#260711)
  [Metrics][Discover] internal/search/esql_async returns 200 but METRICS_INFO responds with error (elastic#260746)
  Collapse redundant anyOf/oneOf array unions in OAS query params (elastic#260585)
  [Unified rules] Hide stack rules from global search (elastic#260088)
  [Agent Builder] Sidebar navigation updates (elastic#260728)
  [* As Code] Use PUT for upserts (elastic#260318)
  Update EUI to v114.0.0 (elastic#259497)
  [Entity Resolution] Add contextual-security-apps as co-owner of resolution paths (elastic#260659)
  ...

# Conflicts:
#	src/platform/plugins/shared/dashboard/public/index.ts
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this pull request Apr 2, 2026
…ervice unavailability (elastic#260519)

## Summary

Unskips 6 EDR Workflows FTR test suites that were skipped due to
transient `installFleetPackageByUpload` failures (socket hang up /
timeout) in CI. These were not caused by code defects — the Fleet
package upload service was temporarily unavailable during the
`kibana-on-merge` runs in late January 2026.

All 6 issues share the identical error:
```
Error: retry.tryWithRetries reached timeout 20000 ms waiting for 'installFleetPackageByUpload'
Error: socket hang up
```

The failures occurred in the `before all` hook (test setup), not in the
test logic itself. The underlying service has since stabilized.

### Test suites unskipped

| Test Suite | Issue |
|------------|-------|
| Policy Response Integration Tests - ESS |
elastic#250458 |
| Policy Response Integration Tests - Serverless |
elastic#250698 |
| Resolver Integration Tests - Serverless |
elastic#250782 |
| Resolver Integration Tests - ESS |
elastic#250805 |
| Role Migration Tests - Serverless |
elastic#250957 |
| Policy Tests - Serverless |
elastic#250973 |

### Files changed

All under
`x-pack/solutions/security/test/security_solution_api_integration/test_suites/edr_workflows/`:
- `policy/trial_license_complete_tier/index.ts`
- `policy_response/trial_license_complete_tier/index.ts`
- `resolver/trial_license_complete_tier/index.ts`
- `role_migrations/trial_license_complete_tier/index.ts`
- `scripts_library/trial_license_complete_tier/index.ts`

Closes elastic#250458
Closes elastic#250698
Closes elastic#250782
Closes elastic#250805
Closes elastic#250957
Closes elastic#250973
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment