Skip to content

[8.19] [Security Solution] Fix real prebuilt rules package related flakiness in integration tests (#227689)#228777

Merged
kibanamachine merged 1 commit intoelastic:8.19from
kibanamachine:backport/8.19/pr-227689
Jul 21, 2025
Merged

[8.19] [Security Solution] Fix real prebuilt rules package related flakiness in integration tests (#227689)#228777
kibanamachine merged 1 commit intoelastic:8.19from
kibanamachine:backport/8.19/pr-227689

Conversation

@kibanamachine
Copy link
Copy Markdown
Contributor

Backport

This will backport the following commits from main to 8.19:

Questions ?

Please refer to the Backport tool documentation

… in integration tests (elastic#227689)

**Resolves: elastic#220333
**Resolves: elastic#227571
**Resolves: elastic#219407

## Summary

This PR fixes flakiness caused by the real prebuilt rules package installation.

## Details

Multiple Rule Management API endpoints install the prebuilt rules package under the hood when it's not installed. In normal case this installation happens from the EPR. The problem is that EPR might be unavailable for some period of time or there network issues blocking EPR access. On top of that real package is heavy and requires some time and resources to be installed. While the major part of our functionality doesn't depend on the real prebuilt rules package and the concrete rules there. On top of that API endpoints depending on the real prebuilt rules package fail with HTTP 500 error when EPR is unavailable.

This PR extends the Functional Test Runner with `rootHooks.beforeAll` configuration option which is used to install a lightweight mock prebuilt rules package. This action prevents real package installation make the tests much more predictable.

## Flaky test runner

- ✅ [ECH import custom rules integration tests (200 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8627)
- ✅ [Serverless import custom rules integration tests (200 runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8642)

(cherry picked from commit 76908c5)
@kibanamachine kibanamachine added the backport This PR is a backport of another PR label Jul 21, 2025
@kibanamachine kibanamachine enabled auto-merge (squash) July 21, 2025 12:49
@kibanamachine kibanamachine merged commit dc37476 into elastic:8.19 Jul 21, 2025
11 checks passed
@elasticmachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

The CI Stats report is too large to be displayed here, check out the CI build annotation for this information.

cc @maximpn

maximpn added a commit that referenced this pull request Jul 23, 2025
…akiness in integration tests (#227689) (#228814)

# Backport

This will backport the following commits from `main` to `8.18`:
- [[Security Solution] Fix real prebuilt rules package related flakiness
in integration tests
(#227689)](#227689)

<!--- Backport version: 10.0.1 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Maxim
Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-07-21T12:42:34Z","message":"[Security
Solution] Fix real prebuilt rules package related flakiness in
integration tests (#227689)\n\n**Resolves:
https://github.com/elastic/kibana/issues/220333**\n**Resolves:
https://github.com/elastic/kibana/issues/227571**\n**Resolves:
https://github.com/elastic/kibana/issues/219407**\n\n## Summary\n\nThis
PR fixes flakiness caused by the real prebuilt rules package
installation.\n\n## Details\n\nMultiple Rule Management API endpoints
install the prebuilt rules package under the hood when it's not
installed. In normal case this installation happens from the EPR. The
problem is that EPR might be unavailable for some period of time or
there network issues blocking EPR access. On top of that real package is
heavy and requires some time and resources to be installed. While the
major part of our functionality doesn't depend on the real prebuilt
rules package and the concrete rules there. On top of that API endpoints
depending on the real prebuilt rules package fail with HTTP 500 error
when EPR is unavailable.\n\nThis PR extends the Functional Test Runner
with `rootHooks.beforeAll` configuration option which is used to install
a lightweight mock prebuilt rules package. This action prevents real
package installation make the tests much more predictable.\n\n## Flaky
test runner\n\n- ✅ [ECH import custom rules integration tests (200
runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8627)\n-
✅ [Serverless import custom rules integration tests (200
runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8642)","sha":"76908c5fa09ecdca5b7df5458e2a4cc1fba330c9","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","backport:version","v9.1.0","v8.19.0","v9.2.0","v8.18.4","v9.0.4"],"title":"[Security
Solution] Fix real prebuilt rules package related flakiness in
integration
tests","number":227689,"url":"https://github.com/elastic/kibana/pull/227689","mergeCommit":{"message":"[Security
Solution] Fix real prebuilt rules package related flakiness in
integration tests (#227689)\n\n**Resolves:
https://github.com/elastic/kibana/issues/220333**\n**Resolves:
https://github.com/elastic/kibana/issues/227571**\n**Resolves:
https://github.com/elastic/kibana/issues/219407**\n\n## Summary\n\nThis
PR fixes flakiness caused by the real prebuilt rules package
installation.\n\n## Details\n\nMultiple Rule Management API endpoints
install the prebuilt rules package under the hood when it's not
installed. In normal case this installation happens from the EPR. The
problem is that EPR might be unavailable for some period of time or
there network issues blocking EPR access. On top of that real package is
heavy and requires some time and resources to be installed. While the
major part of our functionality doesn't depend on the real prebuilt
rules package and the concrete rules there. On top of that API endpoints
depending on the real prebuilt rules package fail with HTTP 500 error
when EPR is unavailable.\n\nThis PR extends the Functional Test Runner
with `rootHooks.beforeAll` configuration option which is used to install
a lightweight mock prebuilt rules package. This action prevents real
package installation make the tests much more predictable.\n\n## Flaky
test runner\n\n- ✅ [ECH import custom rules integration tests (200
runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8627)\n-
✅ [Serverless import custom rules integration tests (200
runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8642)","sha":"76908c5fa09ecdca5b7df5458e2a4cc1fba330c9"}},"sourceBranch":"main","suggestedTargetBranches":["8.18"],"targetPullRequestStates":[{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/228779","number":228779,"state":"MERGED","mergeCommit":{"sha":"67927486cfb64e83f106c715d1e8c9ac33df477a","message":"[9.1]
[Security Solution] Fix real prebuilt rules package related flakiness in
integration tests (#227689) (#228779)\n\n# Backport\n\nThis will
backport the following commits from `main` to `9.1`:\n- [[Security
Solution] Fix real prebuilt rules package related flakiness\nin
integration
tests\n(#227689)](https://github.com/elastic/kibana/pull/227689)\n\n\n\n###
Questions ?\nPlease refer to the [Backport
tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by:
Maxim Palenov
<maxim.palenov@elastic.co>"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/228777","number":228777,"state":"MERGED","mergeCommit":{"sha":"dc37476f1470baad79170f51edee9a432bb6123c","message":"[8.19]
[Security Solution] Fix real prebuilt rules package related flakiness in
integration tests (#227689) (#228777)\n\n# Backport\n\nThis will
backport the following commits from `main` to `8.19`:\n- [[Security
Solution] Fix real prebuilt rules package related flakiness\nin
integration
tests\n(#227689)](https://github.com/elastic/kibana/pull/227689)\n\n\n\n###
Questions ?\nPlease refer to the [Backport
tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by:
Maxim Palenov
<maxim.palenov@elastic.co>"}},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/227689","number":227689,"mergeCommit":{"message":"[Security
Solution] Fix real prebuilt rules package related flakiness in
integration tests (#227689)\n\n**Resolves:
https://github.com/elastic/kibana/issues/220333**\n**Resolves:
https://github.com/elastic/kibana/issues/227571**\n**Resolves:
https://github.com/elastic/kibana/issues/219407**\n\n## Summary\n\nThis
PR fixes flakiness caused by the real prebuilt rules package
installation.\n\n## Details\n\nMultiple Rule Management API endpoints
install the prebuilt rules package under the hood when it's not
installed. In normal case this installation happens from the EPR. The
problem is that EPR might be unavailable for some period of time or
there network issues blocking EPR access. On top of that real package is
heavy and requires some time and resources to be installed. While the
major part of our functionality doesn't depend on the real prebuilt
rules package and the concrete rules there. On top of that API endpoints
depending on the real prebuilt rules package fail with HTTP 500 error
when EPR is unavailable.\n\nThis PR extends the Functional Test Runner
with `rootHooks.beforeAll` configuration option which is used to install
a lightweight mock prebuilt rules package. This action prevents real
package installation make the tests much more predictable.\n\n## Flaky
test runner\n\n- ✅ [ECH import custom rules integration tests (200
runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8627)\n-
✅ [Serverless import custom rules integration tests (200
runs)](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8642)","sha":"76908c5fa09ecdca5b7df5458e2a4cc1fba330c9"}},{"branch":"8.18","label":"v8.18.4","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.0","label":"v9.0.4","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/228778","number":228778,"state":"OPEN"}]}]
BACKPORT-->
maximpn added a commit that referenced this pull request Dec 3, 2025
…uilt rules OOM tests in 8.19 (#245087)

**Partially addresses:** #188090

## Summary

Fleet doesn't have #202435 backported to 8.19. It blocks package removal for packages installed by direct upload. With #228777 we have a We have a mock prebuilt rules package installed via upload at the global hook.

This PR fixes the issue by skipping mock prebuilt rules package installation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants