[Security Solution] Actualize prebuilt rule upgrade test plans#222606
Merged
maximpn merged 11 commits intoelastic:mainfrom Jun 24, 2025
Merged
[Security Solution] Actualize prebuilt rule upgrade test plans#222606maximpn merged 11 commits intoelastic:mainfrom
maximpn merged 11 commits intoelastic:mainfrom
Conversation
9fab9b8 to
ee954f4
Compare
Contributor
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
Contributor
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
Contributor
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
Contributor
💚 Build Succeeded
cc @maximpn |
nikitaindik
reviewed
Jun 18, 2025
Contributor
nikitaindik
left a comment
There was a problem hiding this comment.
Thanks for expanding and refactoring of test plans! Looks very good. I've left some questions/suggestions. Please take a look.
| When user opens the Prebuilt Rules installation page | ||
| Then the available prebuilt rules should be shown | ||
| When user picks the desired number of <rows_per_page> | ||
| Then the <rows_per_page> of the available prebuilt rules should be shown on the page |
Contributor
There was a problem hiding this comment.
nit
Suggested change
| Then the <rows_per_page> of the available prebuilt rules should be shown on the page | |
| Then up to <rows_per_page> of the available prebuilt rules should be shown on the page |
Contributor
Author
There was a problem hiding this comment.
Up to doesn't sound too strict for the test. For example users may see 25 rules per page by default and decide to switch the number of rules per page to 100. Showing the same 25 rules per page might be a bug if the total number of rules satisfying the filter is greater than 25.
If the test is implemented with 3 rules only we can't be sure the functionality works correctly.
| Then it should disappear | ||
| ``` | ||
|
|
||
| #### **Scenario: User can switch between prebuilt rules previews having upgrades** |
Contributor
There was a problem hiding this comment.
This is no longer possible since the flyout now has a backdrop that prevents clicking on another rule. More info here: #191978 (comment)
We can remove this scenario.
| - [Common assumptions](./prebuilt_rules_common_info.md#common-assumptions). | ||
| - Package with prebuilt rules is already installed, and rule assets from it are stored in Elasticsearch. | ||
| - It's expected the Prebuilt Rules upgrade workflow works seamlessly even if some or all prebuilt rules may have their **base versions** missing. | ||
| - \<Upgrade Prebuilt Rules CTA\> combines two bulk upgrade options |
Contributor
There was a problem hiding this comment.
Does that mean that we want to write tests for both "all" and selected rules cases for every scenario that mentions ?
Contributor
Author
There was a problem hiding this comment.
Yes. It covers different set of input API endpoint parameters. And we already have such tests in bulk_upgrade_all_prebuilt_rules.ts and bulk_upgrade_selected_prebuilt_rules.ts.
| - If user chooses to preview a prebuilt rule to be installed, we currently show this preview in a flyout. | ||
| - In the prebuilt rule preview a tab that doesn't have any sections should not be displayed and a section that doesn't have any properties also should not be displayed. | ||
|
|
||
| Examples of rule properties we show in the prebuilt rule preview flyout: |
Contributor
There was a problem hiding this comment.
I think this table needs to stay because it's referenced in scenario "User can see correct rule information in preview before installing".
Or we can somehow merge it with the table in the "common info" file so we can have a single table (and a single place to update if new fields are added). WDYT?
Contributor
Author
There was a problem hiding this comment.
prebuilt_rules_common_info.md file has Customizable rule fields section. It was created earlier based on the removed table. Do you still think we should explicitly specify that flyout shows fields listed in Customizable rule fields section?
45c702b to
6e9138d
Compare
Contributor
Author
|
@nikitaindik Thanks for reviewing my PR 🙏 I've addressed your comments. Could you have a look? |
nikitaindik
approved these changes
Jun 24, 2025
Contributor
nikitaindik
left a comment
There was a problem hiding this comment.
Thx for the updates! LGTM now 🚀
Contributor
|
Starting backport for target branches: 8.19 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jun 24, 2025
…ic#222606) **Addresses:** elastic#202078 **Resolves:** elastic#166215 ## Summary This PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state. ## Details The changes are summarized in the following items - Missing scenarios were added - elastic#166215 was addressed - The wording was fixed to make the test scenarios shorter and focused - Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses. (cherry picked from commit 1a59438)
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Jun 24, 2025
…#222606) (#225040) # Backport This will backport the following commits from `main` to `8.19`: - [[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)](#222606) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-06-24T10:09:52Z","message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v9.1.0","v8.19.0"],"title":"[Security Solution] Actualize prebuilt rule upgrade test plans","number":222606,"url":"https://github.com/elastic/kibana/pull/222606","mergeCommit":{"message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/222606","number":222606,"mergeCommit":{"message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
akowalska622
pushed a commit
to akowalska622/kibana
that referenced
this pull request
Jun 25, 2025
…ic#222606) **Addresses:** elastic#202078 **Resolves:** elastic#166215 ## Summary This PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state. ## Details The changes are summarized in the following items - Missing scenarios were added - elastic#166215 was addressed - The wording was fixed to make the test scenarios shorter and focused - Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Jul 7, 2025
…ic#222606) **Addresses:** elastic#202078 **Resolves:** elastic#166215 ## Summary This PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state. ## Details The changes are summarized in the following items - Missing scenarios were added - elastic#166215 was addressed - The wording was fixed to make the test scenarios shorter and focused - Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses. (cherry picked from commit 1a59438)
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Jul 7, 2025
…ic#222606) **Addresses:** elastic#202078 **Resolves:** elastic#166215 ## Summary This PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state. ## Details The changes are summarized in the following items - Missing scenarios were added - elastic#166215 was addressed - The wording was fixed to make the test scenarios shorter and focused - Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses. (cherry picked from commit 1a59438)
Contributor
Author
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
maximpn
added a commit
that referenced
this pull request
Jul 7, 2025
…#222606) (#226785) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)](#222606) <!--- Backport version: 10.0.1 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-06-24T10:09:52Z","message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v9.1.0","v8.19.0"],"title":"[Security Solution] Actualize prebuilt rule upgrade test plans","number":222606,"url":"https://github.com/elastic/kibana/pull/222606","mergeCommit":{"message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/222606","number":222606,"mergeCommit":{"message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/225040","number":225040,"state":"MERGED","mergeCommit":{"sha":"8fdd25d5af7b5d8726539d945519da0ab4c55106","message":"[8.19] [Security Solution] Actualize prebuilt rule upgrade test plans (#222606) (#225040)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.19`:\n- [[Security Solution] Actualize prebuilt rule upgrade test plans\n(#222606)](https://github.com/elastic/kibana/pull/222606)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}}]}] BACKPORT-->
maximpn
added a commit
that referenced
this pull request
Jul 7, 2025
…222606) (#226784) # Backport This will backport the following commits from `main` to `9.0`: - [[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)](#222606) <!--- Backport version: 10.0.1 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-06-24T10:09:52Z","message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v9.1.0","v8.19.0"],"title":"[Security Solution] Actualize prebuilt rule upgrade test plans","number":222606,"url":"https://github.com/elastic/kibana/pull/222606","mergeCommit":{"message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/222606","number":222606,"mergeCommit":{"message":"[Security Solution] Actualize prebuilt rule upgrade test plans (#222606)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Resolves:** https://github.com/elastic/kibana/issues/166215\n\n## Summary\n\nThis PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.\n\n## Details\n\nThe changes are summarized in the following items\n\n- Missing scenarios were added\n- #166215 was addressed\n- The wording was fixed to make the test scenarios shorter and focused\n- Prebuilt Rules Customization Milestone 2 test scenarios were migrated to a separate test plan `prebuilt_rule_json_diff.md`. The functionality is still relevant and used for rule type changes and under low-tier licenses.","sha":"1a59438b125e45418d01cab29cc06ca0838740a2"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/225040","number":225040,"state":"MERGED","mergeCommit":{"sha":"8fdd25d5af7b5d8726539d945519da0ab4c55106","message":"[8.19] [Security Solution] Actualize prebuilt rule upgrade test plans (#222606) (#225040)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.19`:\n- [[Security Solution] Actualize prebuilt rule upgrade test plans\n(#222606)](https://github.com/elastic/kibana/pull/222606)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}}]}] BACKPORT-->
Contributor
|
Starting backport for target branches: 8.18, 8.19, 9.1 |
Contributor
|
Starting backport for target branches: 8.18, 8.19, 9.0, 9.1 |
Contributor
|
Starting backport for target branches: 8.18, 8.19, 9.0, 9.1 |
Contributor
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
Contributor
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
1 similar comment
Contributor
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses: #202078
Resolves: #166215
Summary
This PR actualizes prebuilt rule upgrade test plans to correspond to the current feature state.
Details
The changes are summarized in the following items
prebuilt_rule_json_diff.md. The functionality is still relevant and used for rule type changes and under low-tier licenses.