[Alerting] Allow rule types to extract/inject saved object references on rule CRU by ymao1 · Pull Request #101896 · elastic/kibana

ymao1 · 2021-06-10T13:26:38Z

Resolves #87992

Summary

This PR provides a framework level hook to allow individual rule types to register functions for extracting/injecting saved object references from and into a rule's parameters.

The AlertType interface has been updated to include a new ExtractedParams template type and optional useSavedObjectReferences functions. Both useSavedObjectReferences.extractReferences and useSavedObjectReferences.injectReferences must be defined if one is defined.

Updated `AlertType` interface

export interface AlertType<
  Params extends AlertTypeParams = never,
  ExtractedParams extends AlertTypeParams = never -----> this can be the same as Params if the rule type wants
  State extends AlertTypeState = never,
  InstanceState extends AlertInstanceState = never,
  InstanceContext extends AlertInstanceContext = never,
  ActionGroupIds extends string = never,
  RecoveryActionGroupId extends string = never
> {
  id: string;
  name: string;
  actionGroups: Array<ActionGroup<ActionGroupIds>>;
  defaultActionGroupId: ActionGroup<ActionGroupIds>['id'];
  recoveryActionGroup?: ActionGroup<RecoveryActionGroupId>;
  executor: ExecutorType;
  producer: string;
  actionVariables?: {};
  minimumLicenseRequired: LicenseType;
  /**
   * These are the new hooks
   **/
  useSavedObjectReferences?: {
    extractReferences: (params: Params) => RuleParamsAndRefs<ExtractedParams>;
    injectReferences: (params: SavedObjectAttributes, references: SavedObjectReference[]) => Params;
  };
}

In the alerts_client, the extractReferences hook would be called (if defined) on create and update and the injectReferences hook would be called (if defined) on get and find. The framework will prefix any extracted SO reference names with a framework level prefix (currently param:) to delineate between framework level SO references and references extracted by the rule type.

Example hooks implementation:

useSavedObjectReferences: {
  extractReferences: (params: Params): RuleParamsAndRefs<ExtractedParams> => {
    const { testSavedObjectId, ...otherParams } = params;

    const testSavedObjectRef = 'testRef_0';
    const references = [
      {
        name: `testRef_0`,
        id: testSavedObjectId,
        type: 'index-pattern',
      },
    ];
    return { params: { ...otherParams, testSavedObjectRef }, references };
  },
  injectReferences: (params: SavedObjectAttributes, references: SavedObjectReference[]) => {
    const { testSavedObjectRef, ...otherParams } = params;
    const reference = references.find((ref) => ref.name === testSavedObjectRef);
    if (!reference) {
      throw new Error(`Test reference "${testSavedObjectRef}"`);
    }
    return { ...otherParams, testSavedObjectId: reference.id } as Params;
  },
},

Checklist

Delete any items that are not applicable to this PR.

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios

…n rule create

…ding extractReferences to create and update workflow

…ing/saved-object-references

…/ymao1/kibana into alerting/saved-object-references

…r create and update

…ing/saved-object-references

ymao1 · 2021-06-15T11:48:40Z

@elasticmachine merge upstream

…/ymao1/kibana into alerting/saved-object-references

…ing/saved-object-references

FrankHassanabad

LGTM

I did this:

Reading through the description and the example.
Making sure I understood the example.
In the example I saw that you can remove params as well as add them back if you only want to store them in the saved references which is 👍 great.
Read through the two changes in files changed

I did not do this:

I did not try to create a function that exercises this functionality and ensure it works. Assuming someone from kibana-alerting will do this part and/or a e2e test at some point from kibana-alerting.

I assume once approved we will then implement this for exception lists and then write a SO migration to upgrade existing exception lists. These two PR's could be separate from our end.

mikecote

Changes LGTM! I tested locally with the index threshold rule type, I was able to see it work as expected 👍

mikecote · 2021-07-08T13:09:16Z

x-pack/plugins/alerting/server/types.ts

  minimumLicenseRequired: LicenseType;
+  useSavedObjectReferences?: {
+    extractReferences: (params: Params) => RuleParamsAndRefs<ExtractedParams>;
+    injectReferences: (params: SavedObjectAttributes, references: SavedObjectReference[]) => Params;


I'm thinking there's a reason for it to be SavedObjectAttributes but I figured I'd ask, should this be ExtractedParams?

Suggested change

injectReferences: (params: SavedObjectAttributes, references: SavedObjectReference[]) => Params;

injectReferences: (params: ExtractedParams, references: SavedObjectReference[]) => Params;

mikecote · 2021-07-08T14:15:04Z

x-pack/plugins/alerting/server/alerts_client/alerts_client.ts

  dateStart?: string;
 }

+const extractedSavedObjectParamReferenceNamePrefix = 'param:';


optional nit: should we add a comment about modifying this value will require a migration for existing SOs with references?

ymao1 · 2021-07-12T11:42:21Z

@elasticmachine merge upstream

kindsun

Maps team owned file updates lgtm!

ymao1 · 2021-07-16T18:22:20Z

@elasticmachine merge upstream

…/ymao1/kibana into alerting/saved-object-references

ymao1 · 2021-07-20T17:46:45Z

@elasticmachine merge upstream

kibanamachine · 2021-07-20T20:04:09Z

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`alerting`	230	234	+4

Unknown metric groups

API count

id	before	after	diff
`alerting`	238	242	+4

History

💚 Build #139335 succeeded 1921d4f
💛 Build #139165 was flaky 37fe7b2
💚 Build #138431 succeeded 9d1c536
💚 Build #138025 succeeded b3570f5
💚 Build #137799 succeeded 5d04948

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @ymao1

YulNaumenko

LGTM! Awesome new capabilities for the rule types. Tested this manually on the extending Index Threshold with the new hooks - it works!

… on rule CRU (elastic#101896) * Adding function hooks into rule type definition and call extract fn on rule create * Adding hooks for extracting and injecting saved object references. Adding extractReferences to create and update workflow * Adding type template for extracted params * Adding type template for extracted params * Adding type template for extracted params * Adding type template for extracted params * Calling injectReferences function if defined. Finishing unit tests for create and update * Adding tests for get * Adding tests for find * Cleanup * Fixing types check * Fixing functional tests * Fixing functional tests * Fixing tests * Updating README * Throwing boom error instead of normal error * Adding framework level prefix to extracted saved object reference names * Fixing types * Fixing types * PR fixes Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

kibanamachine · 2021-07-20T21:12:00Z

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.

… on rule CRU (#101896) (#106333) * Adding function hooks into rule type definition and call extract fn on rule create * Adding hooks for extracting and injecting saved object references. Adding extractReferences to create and update workflow * Adding type template for extracted params * Adding type template for extracted params * Adding type template for extracted params * Adding type template for extracted params * Calling injectReferences function if defined. Finishing unit tests for create and update * Adding tests for get * Adding tests for find * Cleanup * Fixing types check * Fixing functional tests * Fixing functional tests * Fixing tests * Updating README * Throwing boom error instead of normal error * Adding framework level prefix to extracted saved object reference names * Fixing types * Fixing types * PR fixes Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: ymao1 <ying.mao@elastic.co>

…y-show-migrate-to-authzd-users * 'master' of github.com:elastic/kibana: (48 commits) [Canvas] Expression shape (elastic#103219) [FTR] Skips Vega tests [Sample data] Use Lens in ecommerce data (elastic#106039) [APM] Backends inventory & overview page routes (elastic#106223) [TSVB] Add more functional tests for Gauge and TopN (elastic#105361) Add toggle to enable/disable rule install from SOs (elastic#106189) Improve unit test coverage of FS API calls (elastic#106242) Remove recursive plugin status in meta field (elastic#106286) [Ingest pipelines] add community id processor (elastic#103863) [XY axis] Fixes the values inside bar charts (elastic#106198) [data.search] Set default expiration to 1m if search sessions are disabled (elastic#105329) set the doc title when navigating to reporting and unset when navigating away (elastic#106253) [Lens] Display legend inside chart (elastic#105571) [RAC] [TGrid] Migrate the TGrid's rendering to `EuiDataGrid` (elastic#106199) [Security Solutions] Removes the elastic legacy client from lists and security_solution plugins (elastic#106130) [Enterprise Search] Require security plugin in 8.0 (elastic#106307) [DOCS] Updates screenshots in Dev Tools docs (elastic#105859) [DOCS] Updates text and screenshots in tags doc (elastic#105853) [Alerting] Allow rule types to extract/inject saved object references on rule CRU (elastic#101896) Jest and Storybook fixes (elastic#104991) ... # Conflicts: # x-pack/plugins/reporting/public/plugin.ts

… on rule CRU (elastic#101896) * Adding function hooks into rule type definition and call extract fn on rule create * Adding hooks for extracting and injecting saved object references. Adding extractReferences to create and update workflow * Adding type template for extracted params * Adding type template for extracted params * Adding type template for extracted params * Adding type template for extracted params * Calling injectReferences function if defined. Finishing unit tests for create and update * Adding tests for get * Adding tests for find * Cleanup * Fixing types check * Fixing functional tests * Fixing functional tests * Fixing tests * Updating README * Throwing boom error instead of normal error * Adding framework level prefix to extracted saved object reference names * Fixing types * Fixing types * PR fixes Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

ymao1 added 18 commits June 8, 2021 14:16

Adding function hooks into rule type definition and call extract fn o…

646a41f

…n rule create

Adding hooks for extracting and injecting saved object references. Ad…

9cb4e3b

…ding extractReferences to create and update workflow

Adding type template for extracted params

e166343

Merge branch 'master' of https://github.com/elastic/kibana into alert…

789e267

…ing/saved-object-references

Adding type template for extracted params

fbcb2cc

Adding type template for extracted params

6412a22

Adding type template for extracted params

ae10948

Merge branch 'alerting/saved-object-references' of https://github.com…

b202231

…/ymao1/kibana into alerting/saved-object-references

Calling injectReferences function if defined. Finishing unit tests fo…

3cce73b

…r create and update

Adding tests for get

878209a

Merge branch 'master' of https://github.com/elastic/kibana into alert…

fd7ce92

…ing/saved-object-references

Adding tests for find

dc97b5b

Cleanup

b710b39

Fixing types check

3ec8fd4

Fixing functional tests

4980b8a

Fixing functional tests

75e75fe

Fixing tests

4cec840

Merge branch 'master' of https://github.com/elastic/kibana into alert…

ac1e835

…ing/saved-object-references

kibanamachine and others added 5 commits June 15, 2021 07:48

Merge branch 'master' into alerting/saved-object-references

ddaf000

Updating README

188eb10

Merge branch 'alerting/saved-object-references' of https://github.com…

266d7d9

…/ymao1/kibana into alerting/saved-object-references

Throwing boom error instead of normal error

8a0c98a

Merge branch 'master' of https://github.com/elastic/kibana into alert…

3bde24f

…ing/saved-object-references

ymao1 changed the title ~~Alerting/saved object references~~ [Alerting] Allow rule types to extract/inject saved object references on rule CRU Jun 16, 2021

ymao1 added 3 commits June 17, 2021 08:00

Merge branch 'master' of https://github.com/elastic/kibana into alert…

ec936a7

…ing/saved-object-references

Merge branch 'master' of https://github.com/elastic/kibana into alert…

330912f

…ing/saved-object-references

Adding framework level prefix to extracted saved object reference names

40c5344

ymao1 self-assigned this Jun 21, 2021

ymao1 added the Feature:Alerting label Jun 21, 2021

FrankHassanabad approved these changes Jul 7, 2021

View reviewed changes

mikecote self-requested a review July 8, 2021 13:07

mikecote approved these changes Jul 8, 2021

View reviewed changes

ymao1 removed the request for review from spong July 12, 2021 11:42

Merge branch 'master' into alerting/saved-object-references

5d04948

kindsun approved these changes Jul 12, 2021

View reviewed changes

ymao1 added 2 commits July 12, 2021 15:23

PR fixes

b3570f5

Merging in master

9d1c536

kibanamachine and others added 3 commits July 16, 2021 14:22

Merge branch 'master' into alerting/saved-object-references

37fe7b2

Merging in master

d786bbf

Merge branch 'alerting/saved-object-references' of https://github.com…

1921d4f

…/ymao1/kibana into alerting/saved-object-references

Merge branch 'master' into alerting/saved-object-references

9fdfea3

YulNaumenko approved these changes Jul 20, 2021

View reviewed changes

ymao1 added the auto-backport Deprecated - use backport:version if exact versions are needed label Jul 20, 2021

ymao1 merged commit e2aacdc into elastic:master Jul 20, 2021

kibanamachine mentioned this pull request Jul 20, 2021

[7.x] [Alerting] Allow rule types to extract/inject saved object references on rule CRU (#101896) #106333

Merged

ymao1 mentioned this pull request Jul 20, 2021

[Security Solution][Detections] Migrate to using Saved Object References array to reference Exception Lists from Rules #101975

Closed

ymao1 mentioned this pull request Jul 23, 2021

[7.14] [Alerting] Allow rule types to extract/inject saved object references on rule CRU (#101896) #106642

Closed

This was referenced Jul 26, 2021

[Draft] Followup issues for alerting saved object sharing #106767

Closed

[Maps][Alerting] Migrate to using Saved Object References array to reference index patterns from Rules #107066

Closed

ymao1 mentioned this pull request May 17, 2022

[Discover][Alerting] Implement editing of dataView, query & filters #131688

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Alerting] Allow rule types to extract/inject saved object references on rule CRU#101896

[Alerting] Allow rule types to extract/inject saved object references on rule CRU#101896
ymao1 merged 37 commits intoelastic:masterfrom
ymao1:alerting/saved-object-references

ymao1 commented Jun 10, 2021 •

edited

Loading

Uh oh!

ymao1 commented Jun 15, 2021

Uh oh!

FrankHassanabad left a comment •

edited

Loading

Uh oh!

mikecote left a comment

Uh oh!

mikecote Jul 8, 2021

Uh oh!

mikecote Jul 8, 2021

Uh oh!

ymao1 commented Jul 12, 2021

Uh oh!

kindsun left a comment •

edited

Loading

Uh oh!

ymao1 commented Jul 16, 2021

Uh oh!

ymao1 commented Jul 20, 2021

Uh oh!

kibanamachine commented Jul 20, 2021

API count

Uh oh!

YulNaumenko left a comment

Uh oh!

kibanamachine commented Jul 20, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

	injectReferences: (params: SavedObjectAttributes, references: SavedObjectReference[]) => Params;
	injectReferences: (params: ExtractedParams, references: SavedObjectReference[]) => Params;

Conversation

ymao1 commented Jun 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Uh oh!

ymao1 commented Jun 15, 2021

Uh oh!

FrankHassanabad left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mikecote left a comment

Choose a reason for hiding this comment

Uh oh!

mikecote Jul 8, 2021

Choose a reason for hiding this comment

Uh oh!

mikecote Jul 8, 2021

Choose a reason for hiding this comment

Uh oh!

ymao1 commented Jul 12, 2021

Uh oh!

kindsun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ymao1 commented Jul 16, 2021

Uh oh!

ymao1 commented Jul 20, 2021

Uh oh!

kibanamachine commented Jul 20, 2021

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

API count

History

Uh oh!

YulNaumenko left a comment

Choose a reason for hiding this comment

Uh oh!

kibanamachine commented Jul 20, 2021

💚 Backport successful

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

ymao1 commented Jun 10, 2021 •

edited

Loading

FrankHassanabad left a comment •

edited

Loading

kindsun left a comment •

edited

Loading