[RAM] Adds Bulk Edit API to rulesClient by vitaliidm · Pull Request #126904 · elastic/kibana

vitaliidm · 2022-03-04T12:46:57Z

Addresses

[RAM] Bulk update on rules #124715

Summary

adds bulkEdit method to rulesClient
adds multi_terms bucket aggregations to savedObjectClient
adds createPointInTimeFinderAsInternalUser to encryptedSavedObjectClient
adds alerting API for bulkEdit

curl --location --request POST 'http://localhost:5601/kbn/internal/alerting/rules/_bulk_edit' \
--header 'kbn-xsrf: reporting' \
--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==' \
--header 'Content-Type: application/json' \
--data-raw '{
    "ids": ["4cb80374-b5c7-11ec-8f1e-adaa7d7d57e5"],
    "operations":  [{
        "operation": "add",
        "field": "tags",
        "value": ["foo"]
    }]
}'

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios

Release note

Adds new bulkEdit method to alerting rulesClient and internal _bulk_edit API, that allow bulk editing of rules.

…ithub.com/vitaliidm/kibana into security-solution/bulk-update-rulesClient

vitaliidm · 2022-03-17T11:48:36Z

@elasticmachine merge upstream

# Conflicts: # x-pack/plugins/alerting/server/routes/index.ts # x-pack/plugins/alerting/server/rules_client.mock.ts # x-pack/plugins/alerting/server/rules_client/audit_events.ts # x-pack/plugins/alerting/server/rules_client/rules_client.ts

…dex.ts Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>

…ypted_saved_objects_api.ts Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>

…dex.ts Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>

x-pack/plugins/alerting/server/rules_client/rules_client.ts

ymao1 · 2022-04-27T18:47:37Z

x-pack/plugins/alerting/server/rules_client/rules_client.ts

+      async ({ key: [ruleType, consumer] }) => {
+        this.ruleTypeRegistry.ensureRuleTypeEnabled(ruleType);
+
+        await this.authorization.ensureAuthorized({


We audit these attempts on individual update. Should we also be auditing this action?

Good catch! Thank you, I've added audit of the error

ymao1 · 2022-04-27T18:55:31Z

x-pack/plugins/alerting/server/rules_client/rules_client.ts

+            // validate schedule interval
+            if (attributes.schedule.interval) {
+              const intervalInMs = parseDuration(attributes.schedule.interval as string);
+              if (intervalInMs < this.minimumScheduleIntervalInMs) {


We've updated this logic to add an enforce option, so we throw if enforce = true and log a warning if it's false

if (intervalInMs < this.minimumScheduleIntervalInMs && this.minimumScheduleInterval.enforce) { throw Boom.badRequest( `Error creating rule: the interval is less than the allowed minimum interval of ${this.minimumScheduleInterval.value}` ); }

if (intervalInMs < this.minimumScheduleIntervalInMs && !this.minimumScheduleInterval.enforce) { this.logger.warn( `Rule schedule interval (${data.schedule.interval}) for "${createdAlert.attributes.alertTypeId}" rule type with ID "${createdAlert.id}" is less than the minimum value (${this.minimumScheduleInterval.value}). Running rules at this interval may impact alerting performance. Set "xpack.alerting.rules.minimumScheduleInterval.enforce" to true to prevent creation of these rules.` ); }

thanks for letting know about recent changes in interval validation. I've updated it

x-pack/plugins/alerting/server/rules_client/rules_client.ts

x-pack/plugins/alerting/tsconfig.json

x-pack/test/alerting_api_integration/spaces_only/tests/alerting/bulk_edit.ts

mikecote

Looking good! I finished my first pass to get this PR reviewed. I'll review tests, pull down, etc next time but wanted to get my questions / some feedback out first.

x-pack/plugins/alerting/tsconfig.json

...k/plugins/alerting/server/invalidate_pending_api_keys/bulk_mark_api_keys_for_invalidation.ts

x-pack/plugins/alerting/server/routes/bulk_edit_rules.ts

x-pack/plugins/alerting/server/rules_client/rules_client.ts

# Conflicts: # x-pack/test/alerting_api_integration/security_and_spaces/alerting/bulk_edit.ts # x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts

vitaliidm · 2022-05-11T13:04:04Z

@elasticmachine merge upstream

mikecote

Tested locally, changes LGTM!

vitaliidm · 2022-05-11T16:36:43Z

Tested locally, changes LGTM!

Thanks, @mikecote
One question from me: I've revisited a bit logic for API keys invalidation

To invalidate newly generated API key even if bulkUpdate operation fails
Not to invalidate old key, if rule wasn't updated due to conflict for example

Here is a commit. Please let me know if it makes sense.
0a34da4

mikecote · 2022-05-11T16:51:07Z

Tested locally, changes LGTM!

Thanks, @mikecote
One question from me: I've revisited a bit logic for API keys invalidation

To invalidate newly generated API key even if bulkUpdate operation fails

Not to invalidate old key, if rule wasn't updated due to conflict for example

Here is a commit. Please let me know if it makes sense.
0a34da4

Yup the change makes sense and looks good 👍 , thanks!

ymao1

LGTM!

kibana-ci · 2022-05-11T18:27:52Z

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`alerting`	317	330	+13

Unknown metric groups

API count

id	before	after	diff
`alerting`	326	339	+13
`encryptedSavedObjects`	48	51	+3
total			+16

History

💔 Build #44133 failed 0a34da4
💚 Build #44038 succeeded 2ca3afc
💚 Build #44012 succeeded 9faef0b
💚 Build #43757 succeeded 00020ed
💔 Build #43696 failed 38428e1

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @vitaliidm

vitaliidm and others added 15 commits January 11, 2022 09:31

POC

db16543

Merge branch 'main' into security-solution/bulk-update-rulesClient

23cc54a

POC update

4c6bac3

Merge branch 'main' into security-solution/bulk-update-rulesClient

6bb3489

add bulkmark API keys invalidate

d116acc

Merge branch 'security-solution/bulk-update-rulesClient' of https://g…

4fdc48d

…ithub.com/vitaliidm/kibana into security-solution/bulk-update-rulesClient

actions refactoring

7a151d2

Merge branch 'main' into security-solution/bulk-update-rulesClient

ba9a4ab

Merge branch 'main' into security-solution/bulk-update-rulesClient

b020e11

move actions to bulk actions

42db655

encrypted + multi_terms

82aeb49

add validated mutated params

5d9580d

revert changes

b61ca0f

updates

8de2546

Merge branch 'main' into security-solution/bulk-update-rulesClient

20250ca

kibanamachine and others added 14 commits March 17, 2022 07:48

Merge branch 'main' into security-solution/bulk-update-rulesClient

d7edcb5

POC route

6351fba

few updates

c1ca8f3

add alering route

3f7dcf6

comment taskRunner actions

6245f23

FEW UPDATES

22699a3

add tests for alerting routes

be2f05b

fix typo

3d3a39d

unterface update

c9debba

original params for validatedParams

da313f1

remove unnecessary encrypted method

c180049

aggr validation

a48a93c

add bulk_edit tests

07f1e42

vitaliidm and others added 5 commits April 26, 2022 14:43

Update x-pack/plugins/encrypted_saved_objects/server/saved_objects/in…

cfb3a56

…dex.ts Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>

Update x-pack/plugins/encrypted_saved_objects/server/saved_objects/in…

61f307b

…dex.ts Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>

Update x-pack/test/encrypted_saved_objects_api_integration/tests/encr…

2cab36b

…ypted_saved_objects_api.ts Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>

Update x-pack/plugins/encrypted_saved_objects/server/saved_objects/in…

ebec27a

…dex.ts Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>

fix tests

722adbf

ymao1 reviewed Apr 27, 2022

View reviewed changes

mikecote reviewed Apr 27, 2022

View reviewed changes

vitaliidm added 10 commits May 9, 2022 13:20

Merge branch 'main' into security-solution/bulk-update-rulesClient

712ff08

# Conflicts: # x-pack/test/alerting_api_integration/security_and_spaces/alerting/bulk_edit.ts # x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts

remove typo(?)

19ca219

fix merge conflict issue

2b4a992

CR: add audit of authorization

d9138c2

CR: interval and operations values

4e74487

CR: mapped params

2e6d918

CR: remove mark_api_key_for_invalidation

38428e1

CR: fix failed test

00020ed

CR: move applyBulkOPeration to a separate file

0b53e91

CR: add missing _bulk_edit route error handlers

9faef0b

Merge branch 'main' into security-solution/bulk-update-rulesClient

2ca3afc

mikecote approved these changes May 11, 2022

View reviewed changes

revisist of API keys invalidation

0a34da4

ymao1 approved these changes May 11, 2022

View reviewed changes

fix test

5045d45

vitaliidm enabled auto-merge (squash) May 11, 2022 17:31

vitaliidm merged commit e3c47ec into elastic:main May 11, 2022

vitaliidm mentioned this pull request May 12, 2022

[RAM] Bulk update on rules #124715

Closed

vitaliidm deleted the security-solution/bulk-update-rulesClient branch March 4, 2024 17:31

Conversation

vitaliidm commented Mar 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Release note

Uh oh!

vitaliidm commented Mar 17, 2022

Uh oh!

Uh oh!

ymao1 Apr 27, 2022

Choose a reason for hiding this comment

Uh oh!

vitaliidm May 9, 2022

Choose a reason for hiding this comment

Uh oh!

ymao1 Apr 27, 2022

Choose a reason for hiding this comment

Uh oh!

vitaliidm May 10, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mikecote left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vitaliidm commented May 11, 2022

Uh oh!

mikecote left a comment

Choose a reason for hiding this comment

Uh oh!

vitaliidm commented May 11, 2022

Uh oh!

mikecote commented May 11, 2022

Uh oh!

ymao1 left a comment

Choose a reason for hiding this comment

Uh oh!

kibana-ci commented May 11, 2022

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

API count

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

vitaliidm commented Mar 4, 2022 •

edited

Loading