Summary

Write test plans (one or a few) and create a comprehensive suite of automated tests for the workflow of upgrading prebuilt rules.

Features to cover:

• Reviewing rules for upgrading @jpdjere
• Upgrading prebuilt rules one-by-one without preview (Rule Updates table) @jpdjere
• Upgrading prebuilt rules in bulk (Rule Updates table) @jpdjere
• Upgrading rule fields - per each rule field - via API and UI @jpdjere
  • name
  • description
  • etc - I feel that we need to cover literally every rule field
• Upgrading prebuilt rules one-by-one with preview (Rule Upgrade flyout) @maximpn
  • Non-conflicts, solvable conflicts, non-solvable conflicts
  • Diffs and diff options
  • Suggesting merged versions
  • Editing and accepting changes to the Final version
  • Callouts
  • Validation
  • State persistence
  • Upgrading and saving changes
  • Anything else?
• Specific and edge cases
  • Type changes on upgrade
  • Missing base versions
  • Race conditions and Optimistic Concurrency Control

Please cover both the features under the feature flag turned ON and OFF.

Related tickets

Related functional tickets to cover with tests:

• API
  • [Security Solution] Extend the POST /upgrade/_review API endpoint's contract and functionality #180153
  • [Security Solution] Handle specific fields in /upgrade/_review upgrade workflow #180393
  • [Security Solution] Extend the POST /upgrade/_perform API endpoint's contract and functionality #166376
  • [Security Solution] Handle specific fields in /upgrade/_perform endpoint upgrade workflow #186544
  • [Security Solution] Add InvestigationFields and AlertSuppression fields to the upgrade workflow #190597
  • [Security Solution] Remove exceptions_list, author and license from Diffable Rule #196213
• UI
  • [Security Solution] Handle conflicting rules in the Rule Upgrade table #180589
  • [Security Solution] Rework Update flyout to display all field updates and build Three-Way-Diff field component #171520
  • [Security Solution] Handle specific fields in the upgrade workflow's UI #188065
  • [Security Solution] Implement UI for updating prebuilt rule to a new rule type (MVP) #180395

Related bugs to cover with tests:

• [Security Solution] Replace PATCH logic with PUT when upgrading rules #180195
• [Security Solution] Unable to upgrade rules with missing base version #200904
• [Security Solution] Prebuilt rule customization is lost on upgrade when Base version is missing #201500
• [Security Solution] Inconsistent conflict state messaging in Rule Upgrade flyout #200811
• [Security Solution] Incorrect field count in Rule Upgrade flyout #200286
• [Security Solution] Implement rule upgrade concurrency control #200134
• Anything else from Milestone 3 bugs under the corresponding section

Related PRs

Test plans for diff algorithms

• [Security Solution] Simple diff algorithm test plan #184484
• [Security Solution] Array of scalar values diff algorithm test plan #186325
• [Security Solution] Multi-line string fields diff algorithm test plan #188323
• [Security Solution] data_source field diff algorithm test plan #189669
• [Security Solution] Test plan for query fields diff algorithm #192529
• [Security Solution] Test plan for rule type field diff algorithm #193372

Test plans for prebuilt rule upgrade workflow

• [Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v1 #203331
• [Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 #205922
• [Security Solution] Test plan for upgrading prebuilt rules with preview (Rule Upgrade flyout) v1 #205645
• [Security Solution] Refactor prebuilt rule customization test plans, part 1 #211300
• [Security Solution] Refactor prebuilt rule customization test plans, part 2 #211472
• [Security Solution] Refactor prebuilt rule customization test plans, part 3 #212420
• [Security Solution] Test plan for prebuilt rule upgrade notifications #212817
• [Security Solution] Actualize prebuilt rule upgrade test plans #222606
  • All backports merged

Test coverage for prebuilt rule upgrade workflow

• [Security Solution] Enhance test coverage for Prebuilt Rules upgrade workflows #204677
• [Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature #209260
• [Security Solution] Implement prebuilt rule upgrade notifications test plan #225912
• [Security Solution] Implement prebuilt rules upgrade test plan #228620