Skip to content

Rule execution log support backfill rule run types#183898

Merged
nkhristinin merged 34 commits intoelastic:mainfrom
nkhristinin:event-log
May 28, 2024
Merged

Rule execution log support backfill rule run types#183898
nkhristinin merged 34 commits intoelastic:mainfrom
nkhristinin:event-log

Conversation

@nkhristinin
Copy link
Copy Markdown
Contributor

@nkhristinin nkhristinin commented May 21, 2024

Rule execution log support backfill rule run types

Screen.Recording.2024-05-22.at.13.43.13.mov

Feature flag

manualRuleRunEnabled

Description

  • Add new column for table with rule run type "Manual" / "Scheduled"
  • Add new switch to show column with source event time range for backfill run
  • event execution log api support run_type_filters filters as parameter with values like "standard" and "backfill"
  • event execution log result will return new field for backfill runs - backfill

How to test

1 . Enable feature flag - manualRuleRunEnabled
2. For you rule call schedule api /internal/alerting/rules/backfill/_schedule POST
With this body (put your values for rule id and date range):

[{"rule_id":"58b4b926-6348-4c23-be1f-870a461fa342","start":"2024-05-21T13:00:00.000Z","end":"2024-05-21T14:05:00.000Z"}]

@nkhristinin
Copy link
Copy Markdown
Contributor Author

/ci

@nkhristinin
Copy link
Copy Markdown
Contributor Author

/ci

@nkhristinin
Copy link
Copy Markdown
Contributor Author

/ci

@nkhristinin
Copy link
Copy Markdown
Contributor Author

/ci

@nkhristinin nkhristinin changed the title Event log Rule execution log support backfill rule run types May 22, 2024
@nkhristinin nkhristinin marked this pull request as ready for review May 22, 2024 11:54
@nkhristinin nkhristinin requested review from a team as code owners May 22, 2024 11:54
@nkhristinin nkhristinin requested a review from xcrzx May 22, 2024 11:54
@nkhristinin
Copy link
Copy Markdown
Contributor Author

@elasticmachine merge upstream

Copy link
Copy Markdown
Contributor

@e40pud e40pud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Also checked locally and I can see manual rule runs. Left a nit comment.

/>
),
field: 'backfill',
render: (backfill: { to: string; from: string }) => {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like FoemattedDate handles this, but anyway according to schema defined above, both to and from are optional and can be undefined.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I upated schema, I keep backfill optional, but if it's present, then from and to also present

Copy link
Copy Markdown
Contributor

@xcrzx xcrzx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested the PR locally and found no major issues. There is a local timezone issue that is not straightforward and might need some clarification, but other than that, it looks good to me. Thank you for the PR, @nkhristinin!

if (runTypeFilters.length > 0 && runTypeFilters.length < 2) {
const ruleRunEventActions = runTypeFilters.map((runType) => {
if (runType === RuleRunTypeEnum.standard) {
return 'execute';
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use EVENT_LOG_ACTIONS.execute here instead of the hardcoded string?

if (runType === RuleRunTypeEnum.standard) {
return 'execute';
} else {
return 'execute-backfill';
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. Can we use EVENT_LOG_ACTIONS.executeBackfill here instead of the hardcoded string?

page,
perPage,
sort,
runTypeFilters,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runTypeFilters is unused, please remove it from the interface

export const COLUMN_SOURCE_EVENT_TIME_RANGE_TOOLTIP = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.sourceEventTimeRangeTooltip',
{
defaultMessage: "Only for manual rule runs. Don't inlcude additional lookback time.",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand what "Don't inlcude additional lookback time." means. Does it refer to the displayed interval not including lookback time, or does it mean the rule was executed without lookback?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, there's a typo in inlcude -> include

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will check with docs, thanks!

Comment on lines +86 to +98
{
name: i18n.COLUMN_TYPE,
field: 'type',
sortable: false,
width: '10%',
render: (value, record) => {
return (
<EuiText size="s">
{record.backfill ? RULE_EXECUTION_TYPE_BACKFILL : RULE_EXECUTION_TYPE_STANDARD}
</EuiText>
);
},
},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we hide this column behind the feature flag?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably it's fine, as it will be scheduled rule type, which is true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to show our users a column that only has a single value, 'Scheduled', for all rows? It would probably be best to hide it to avoid confusion for those who haven't enabled the feature flag.

Comment on lines +459 to +462
if (showSourceEventTimeRange) {
columns.push(...getSourceEventTimeRangeColumns());
messageColumnWidth = 30;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to check that the feature flag is enabled here as well.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

showSourceEventTimeRange - this state controlled by switch which is hidden by feature flag

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still need to cover the situation when the feature flag gets enabled and then disabled for any reason. When I disabled the feature flag locally, the table continued showing this column.

@xcrzx
Copy link
Copy Markdown
Contributor

xcrzx commented May 27, 2024

@nkhristinin Have you aligned with the docs team regarding the changes in this PR? Do we want to document them?
cc @joepeeples

@nkhristinin
Copy link
Copy Markdown
Contributor Author

@nkhristinin Have you aligned with the docs team regarding the changes in this PR? Do we want to document them? cc @joepeeples

I plan to have additional Ticket for this and next PR, to get feedback for UI texts and docs creation.

I want to merge this PR before that, as it needed for all next PR related to this feature

@nkhristinin nkhristinin requested a review from xcrzx May 27, 2024 14:54
@nkhristinin
Copy link
Copy Markdown
Contributor Author

@elasticmachine merge upstream

Copy link
Copy Markdown
Contributor

@xcrzx xcrzx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you, Nikita, for addressing my comment 👍

@kibana-ci
Copy link
Copy Markdown

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 5483 5487 +4

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 15.1MB 15.1MB +4.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
securitySolution 83.6KB 84.0KB +439.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@nkhristinin nkhristinin merged commit 69b28f3 into elastic:main May 28, 2024
@kibanamachine kibanamachine added v8.15.0 backport:skip This PR does not require backporting labels May 28, 2024
@banderror banderror added Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Feature:Rule Monitoring Security Solution Detection Rule Monitoring area Team:Detection Engine Security Solution Detection Engine Area labels May 28, 2024
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-detection-engine (Team:Detection Engine)

@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting Feature:Rule Monitoring Security Solution Detection Rule Monitoring area release_note:feature Makes this part of the condensed release notes Team:Detection Engine Security Solution Detection Engine Area Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.15.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants