…1439)

Fixes: elastic#166376 (main ticket)
Fixes: elastic#186544 (handling of
specific fields)
Fixes: elastic#180195 (replace PATCH
with PUT logic on rule upgrade)

## Summary

- Enhances the `/upgrade/_perform` endpoint to upgrade rules in a way
that works with prebuilt rules customized by users and resolve conflicts
between user customizations and updates from Elastic.
- Handles special fields under the hood (see below)
- Replaces the update prebuilt rule logic to work with PUT instead of
PATCH.

### Rough implementation plan
- For each `upgradeableRule`, we attempt to build the payload necessary
to pass to `upgradePrebuiltRules()`, which is of type
`PrebuiltRuleAsset`. So we retrieve the field names from
`FIELDS_PAYLOAD_BY_RULE_TYPE` and loop through them.
- If any of those `field`s are non-upgreadable, (i.e. its value needs to
be handled under the hood) we do so in `determineFieldUpgradeStatus`.
- Otherwise, we continue to build a `FieldUpgradeSpecifier` for each
field, which will help us determine if that field needs to be set to the
base, current, target version, OR if it needs to be calculated as a
MERGED value, or it is passed in the request payload as a RESOLVED
value.
- Notice that we are iterating over "flat" (non-grouped) fields which
are part of the `PrebuiltRuleAsset` schema. This means that mapping is
necessary between these flat fields and the diffable (grouped) fields
that are used in the API contract, part of `DiffableRule`. For example,
if we try to determine the value for the `query` field, we will need to
look up for its value in the `eql_query` field if the target rule is
`eql` or in `esql_query` if the target rule is `esql`. All these
mappings can be found in `diffable_rule_fields_mappings.ts`.
- Once a `FieldUpgradeSpecifier` has been retrieved for each field of
the payload we are building, retrieve its actual value: either fetching
it from the base, current or target versions of the rule, from the three
way diff calculation, or retrieving it from the request payload if it
resolved.
- Do this for all upgreadable rules, and the pass the payload array into
`upgradePrebuiltRules()`.
- **IMPORTANT:** The upgrade prebuilt rules logic has been changed from
PATCH to PUT. That means that if the next version of a rule removes a
field, and the user updates to that target version, those fields will be
undefined in the resulting rule. **Additional example:** a installs a
rule, and creates a `timeline_id` for it rule by modifying it. If
neither the next version (target version) still does not have a
`timeline_id` field for it, and the user updates to that target version
fully (without resolving the conflict), that field will not exist
anymore in the resulting rule.

## Acceptance criteria

- [x] Extend the contract of the API endpoint according to the
[POC](elastic#144060):
- [x] Add the ability to pick the `MERGED` version for rule upgrades. If
the `MERGED` version is selected, the diffs are recalculated and the
rule fields are updated to the result of the diff calculation. This is
only possible if all field diffs return a `conflict` value of either
`NO`. If any fields returns a value of `NON_SOLVABLE` or `SOLVABLE`,
reject the request with an error specifying that there are conflicts,
and that they must be resolved on a per-field basis.
- [x] Calculate diffs inside this endpoint, when the value of
`pick_version` is `MERGED`.
- [x] Add the ability to specify rule field versions, to update specific
fields to different `pick_versions`: `BASE' | 'CURRENT' | 'TARGET' |
'MERGED' | 'RESOLVED'` (See `FieldUpgradeRequest` in
[PoC](elastic#144060) for details)

## Handling of special fields

Specific fields are handled under the hood based on
elastic#186544

See implementation in
`x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/api/perform_rule_upgrade/determine_field_upgrade_status.ts`,
which imports fields to handle under the hood:
- `DiffableFieldsToOmit`
- `FieldsToUpdateToCurrentVersion`

## Edge cases

- [x] If target version of rule has a **rule type change**, check that
all `pick_version`, at all levels, match `TARGET`. Otherwise, create new
error and add to ruleErrors array.
- [x] if a rule has a specific `targetVersion.type` (for example, EQL)
and the user includes in its `fields` object of the request payload any
fields which do not match that rule type (in this case, for example,
sending in `machine_learning_job_id` as part of `fields`), throw an
error for that rule.
- [x] Calculation of field diffs: what happens if some fields have a
conflict value of `NON_SOLVABLE`:
- [x] If the whole rule is being updated to `MERGED`, and **ANY** fields
return with a `NON_SOLVABLE` conflict, reject the whole update for that
rule: create new error and add to ruleErrors array.
- [x] **EXCEPTION** for case above: the whole rule is being updated to
`MERGED`, and one or more of the fields return with a `NON_SOLVABLE`
conflict, BUT those same fields have a specific `pick_version` for them
in the `fields` object which **ARE NOT** `MERGED`. No error should be
reported in this case.
- [x] The whole rule is being updated to any `pick_version` other than
MERGED, but any specific field in the `fields` object is set to upgrade
to `MERGED`, and the diff for that fields returns a `NON_SOLVABLE`
conflict. In that case, create new error and add to ruleErrors array.

### TODO

- [[Security Solution] Add InvestigationFields and AlertSuppression
fields to the upgrade workflow
[elastic#190597]](elastic#190597):
InvestigationFields is already working, but AlertSuppression is still
currently handled under the hood to update to current version.

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
(cherry picked from commit 7c38873)