[Security Solution] Rectify prebuilt rules export test plan#222796
Merged
maximpn merged 3 commits intoelastic:mainfrom Jun 24, 2025
Merged
[Security Solution] Rectify prebuilt rules export test plan#222796maximpn merged 3 commits intoelastic:mainfrom
maximpn merged 3 commits intoelastic:mainfrom
Conversation
Contributor
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
Contributor
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
Contributor
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
Contributor
💚 Build Succeeded
cc @maximpn |
f253a85 to
24c7178
Compare
nikitaindik
reviewed
Jun 18, 2025
Contributor
nikitaindik
left a comment
There was a problem hiding this comment.
Thanks for refactoring this, @maximpn! I've reviewed and left a few questions.
| - [Scenarios](#scenarios) | ||
| - [Core Functionality](#core-functionality) | ||
| - [Scenario: Exporting prebuilt rule individually from rule details page](#scenario-exporting-prebuilt-rule-individually-from-rule-details-page) | ||
| - [Scenario: Exporting custom rule individually from rule details page](#scenario-exporting-custom-rule-individually-from-rule-details-page) |
Contributor
There was a problem hiding this comment.
Are custom rules related scenarios removed now? I can't find them in other test plans. If yes, what do you think about creating a custom_rule_export.md with these?
Contributor
Author
There was a problem hiding this comment.
Right. It looks kind of strange to have test scenarios purely for custom rules while these test scenarios should be concerned by prebuilt rules. We have separate tests for custom rules import. I'll move removed test scenarios to a separate plan.
| And the custom rules' "ruleSource" "type" should be "internal" | ||
| And the customized prebuilt rules' "isCustomized" value should be true | ||
| And the exported prebuilt rules should include an "immutable" field having true value | ||
| And the exported prebuilt rules "ruleSource.type" should be "external" |
Contributor
There was a problem hiding this comment.
Let's make "ruleSource" -> "rule_source" to match other scenarios
|
|
||
| ### Licensing | ||
|
|
||
| #### Scenario: Exporting a mixture of prebuilt and custom rules via export API under insufficient license** |
Contributor
There was a problem hiding this comment.
Why remove this? Don't we want to cover the exports API specifically?
Contributor
Author
There was a problem hiding this comment.
Please check Useful information section. I made a note that described test scenarios are expected to work under both low- and high-tier licenses. It looks sufficient to run the tests in prebuilt_rules/common folder under low-tier license with the assumption it works under high-tier licensees as well. Though we might have to test under different licenses. Please check out prebuilt rules import test plan implementation PR for more details.
This is also one important thing to note regarding low-tier license and Prebuilt Rules Customization for tests setup. Despite the licensing our public CRUD APIs allow to customize prebuilt rules. Since we can't break the public API's behavior this fact is used in the tests implementation.
| ### Error Handling | ||
|
|
||
| #### Scenario: Exporting beyond the export limit | ||
| #### **Scenario: Exporting beyond the export limit** |
Contributor
There was a problem hiding this comment.
Do you think this should be more specific, like whether it's in UI or via bulk API / exports API?
Contributor
Author
There was a problem hiding this comment.
Technically it should be all of them. You may see Automation is missing here. So I'll specify it there.
Anyway I wasn't able to implement this test scenario in this PR as it described in the PR description. Since it has less impact I'd have a look at this later on.
24c7178 to
280eed7
Compare
Contributor
Author
|
@nikitaindik Thank you for reviewing my PR 🙏 I've addressed your comments. Could you have a look? |
nikitaindik
approved these changes
Jun 24, 2025
Contributor
nikitaindik
left a comment
There was a problem hiding this comment.
Thanks for addressing my comments! 👍 Let's merge!
Contributor
|
Starting backport for target branches: 8.19 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jun 24, 2025
…222796) **Addresses:** elastic#202079 ## Summary This PR aligns prebuilt rules export test plan with the rest test plans. It doesn't include functional changes. (cherry picked from commit a3e4c2e)
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Jun 24, 2025
…222796) (#225025) # Backport This will backport the following commits from `main` to `8.19`: - [[Security Solution] Rectify prebuilt rules export test plan (#222796)](#222796) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-06-24T08:58:08Z","message":"[Security Solution] Rectify prebuilt rules export test plan (#222796)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202079\n\n## Summary\n\nThis PR aligns prebuilt rules export test plan with the rest test plans. It doesn't include functional changes.","sha":"a3e4c2e770bf551a52525e37d65fa84774d924b0","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v9.1.0","v8.19.0"],"title":"[Security Solution] Rectify prebuilt rules export test plan","number":222796,"url":"https://github.com/elastic/kibana/pull/222796","mergeCommit":{"message":"[Security Solution] Rectify prebuilt rules export test plan (#222796)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202079\n\n## Summary\n\nThis PR aligns prebuilt rules export test plan with the rest test plans. It doesn't include functional changes.","sha":"a3e4c2e770bf551a52525e37d65fa84774d924b0"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/222796","number":222796,"mergeCommit":{"message":"[Security Solution] Rectify prebuilt rules export test plan (#222796)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202079\n\n## Summary\n\nThis PR aligns prebuilt rules export test plan with the rest test plans. It doesn't include functional changes.","sha":"a3e4c2e770bf551a52525e37d65fa84774d924b0"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
akowalska622
pushed a commit
to akowalska622/kibana
that referenced
this pull request
Jun 25, 2025
…222796) **Addresses:** elastic#202079 ## Summary This PR aligns prebuilt rules export test plan with the rest test plans. It doesn't include functional changes.
18 tasks
maximpn
added a commit
that referenced
this pull request
Jul 4, 2025
**Addresses:** #202079 **Relates to:** #222796 ## Summary This PR implements Prebuilt Rules export [test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md). Existing tests were adjusted and extended to match the test plan. ## Caveats The test plan describes a scenario to verify prebuilt rules export fails for 10K+ rules but this scenario is tricky to implement. Due to ES limitations on filtering more than 10K the majority of utility functions like `deleteAllRules()` fail. The proper implementation requires proper setup and cleanup to make sure the test doesn't block the testing workflow. As the result of the mentioned complexities implementation of the test scenario for 10K+ rules is skipped in this PR.
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jul 4, 2025
…c#224611) **Addresses:** elastic#202079 **Relates to:** elastic#222796 ## Summary This PR implements Prebuilt Rules export [test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md). Existing tests were adjusted and extended to match the test plan. ## Caveats The test plan describes a scenario to verify prebuilt rules export fails for 10K+ rules but this scenario is tricky to implement. Due to ES limitations on filtering more than 10K the majority of utility functions like `deleteAllRules()` fail. The proper implementation requires proper setup and cleanup to make sure the test doesn't block the testing workflow. As the result of the mentioned complexities implementation of the test scenario for 10K+ rules is skipped in this PR. (cherry picked from commit 6120cae)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jul 4, 2025
…c#224611) **Addresses:** elastic#202079 **Relates to:** elastic#222796 ## Summary This PR implements Prebuilt Rules export [test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md). Existing tests were adjusted and extended to match the test plan. ## Caveats The test plan describes a scenario to verify prebuilt rules export fails for 10K+ rules but this scenario is tricky to implement. Due to ES limitations on filtering more than 10K the majority of utility functions like `deleteAllRules()` fail. The proper implementation requires proper setup and cleanup to make sure the test doesn't block the testing workflow. As the result of the mentioned complexities implementation of the test scenario for 10K+ rules is skipped in this PR. (cherry picked from commit 6120cae)
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Jul 4, 2025
…c#224611) **Addresses:** elastic#202079 **Relates to:** elastic#222796 ## Summary This PR implements Prebuilt Rules export [test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md). Existing tests were adjusted and extended to match the test plan. ## Caveats The test plan describes a scenario to verify prebuilt rules export fails for 10K+ rules but this scenario is tricky to implement. Due to ES limitations on filtering more than 10K the majority of utility functions like `deleteAllRules()` fail. The proper implementation requires proper setup and cleanup to make sure the test doesn't block the testing workflow. As the result of the mentioned complexities implementation of the test scenario for 10K+ rules is skipped in this PR. (cherry picked from commit 6120cae) # Conflicts: # x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md # x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/common/configs/ess_air_gapped.config.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/common/index.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/index.ts
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Jul 5, 2025
…c#224611) **Addresses:** elastic#202079 **Relates to:** elastic#222796 ## Summary This PR implements Prebuilt Rules export [test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md). Existing tests were adjusted and extended to match the test plan. ## Caveats The test plan describes a scenario to verify prebuilt rules export fails for 10K+ rules but this scenario is tricky to implement. Due to ES limitations on filtering more than 10K the majority of utility functions like `deleteAllRules()` fail. The proper implementation requires proper setup and cleanup to make sure the test doesn't block the testing workflow. As the result of the mentioned complexities implementation of the test scenario for 10K+ rules is skipped in this PR. (cherry picked from commit 6120cae) # Conflicts: # x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Jul 5, 2025
…c#224611) **Addresses:** elastic#202079 **Relates to:** elastic#222796 ## Summary This PR implements Prebuilt Rules export [test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md). Existing tests were adjusted and extended to match the test plan. ## Caveats The test plan describes a scenario to verify prebuilt rules export fails for 10K+ rules but this scenario is tricky to implement. Due to ES limitations on filtering more than 10K the majority of utility functions like `deleteAllRules()` fail. The proper implementation requires proper setup and cleanup to make sure the test doesn't block the testing workflow. As the result of the mentioned complexities implementation of the test scenario for 10K+ rules is skipped in this PR. (cherry picked from commit 6120cae) # Conflicts: # x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md
adcoelho
pushed a commit
to adcoelho/kibana
that referenced
this pull request
Jul 7, 2025
…c#224611) **Addresses:** elastic#202079 **Relates to:** elastic#222796 ## Summary This PR implements Prebuilt Rules export [test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md). Existing tests were adjusted and extended to match the test plan. ## Caveats The test plan describes a scenario to verify prebuilt rules export fails for 10K+ rules but this scenario is tricky to implement. Due to ES limitations on filtering more than 10K the majority of utility functions like `deleteAllRules()` fail. The proper implementation requires proper setup and cleanup to make sure the test doesn't block the testing workflow. As the result of the mentioned complexities implementation of the test scenario for 10K+ rules is skipped in this PR.
Contributor
Author
|
Backports to |
kertal
pushed a commit
to kertal/kibana
that referenced
this pull request
Jul 25, 2025
…c#224611) **Addresses:** elastic#202079 **Relates to:** elastic#222796 ## Summary This PR implements Prebuilt Rules export [test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_export.md). Existing tests were adjusted and extended to match the test plan. ## Caveats The test plan describes a scenario to verify prebuilt rules export fails for 10K+ rules but this scenario is tricky to implement. Due to ES limitations on filtering more than 10K the majority of utility functions like `deleteAllRules()` fail. The proper implementation requires proper setup and cleanup to make sure the test doesn't block the testing workflow. As the result of the mentioned complexities implementation of the test scenario for 10K+ rules is skipped in this PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses: #202079
Summary
This PR aligns prebuilt rules export test plan with the rest test plans. It doesn't include functional changes.