Skip to content

Fix powershell error on events 40961 and 40962#10792

Merged
bjmcnic merged 4 commits intoelastic:mainfrom
bjmcnic:fix_powershell_operational_ingest
Sep 12, 2024
Merged

Fix powershell error on events 40961 and 40962#10792
bjmcnic merged 4 commits intoelastic:mainfrom
bjmcnic:fix_powershell_operational_ingest

Conversation

@bjmcnic
Copy link
Copy Markdown
Contributor

@bjmcnic bjmcnic commented Aug 14, 2024

Proposed commit message

Fix pipeline_error for powershell_operational events 40961 and 40962.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@bjmcnic bjmcnic added Integration:windows Windows Team:Security-Windows Platform Security Windows Platform team [elastic/sec-windows-platform] labels Aug 14, 2024
@bjmcnic bjmcnic self-assigned this Aug 14, 2024
@bjmcnic bjmcnic requested a review from a team as a code owner August 14, 2024 17:31
@elasticmachine
Copy link
Copy Markdown

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)

@bjmcnic bjmcnic requested a review from a team as a code owner August 14, 2024 17:36
@bjmcnic bjmcnic requested review from AndersonQ and mauri870 August 14, 2024 17:36
Copy link
Copy Markdown
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing changes to manifest.yml that match the change in changelog.yml

description: Remove all empty values from event_data.
lang: painless
source: ctx?.winlog?.event_data?.entrySet().removeIf(entry -> entry.getValue() == null || entry.getValue().equals(""));
if: ctx?.winlog?.event_data != null
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For sure we need to add this to:

forwarded/elasticsearch/ingest_pipeline/powershell_operational.yml

but it probably applies to:

  • powershell/elasticsearch/ingest_pipeline/default.yml
  • sysmon_operational/elasticsearch/ingest_pipeline/default.yml
  • forwarded/elasticsearch/ingest_pipeline/powershell.yml
  • forwarded/elasticsearch/ingest_pipeline/sysmon_operational.yml

Do you happen to have any sample events you could add to powershell_operational/_dev/test/pipeline/test-events.json ? That way the tests will catch this if we break it in the future.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leehinman Thanks for the tips. I was able to find sample events for powershell_operational and added them to tests. I also made the analogous fix powershell and sysmon_operational as you'd suggested, but was unable to create real events that would trigger the issue for those.

@andrewkroh andrewkroh added the bugfix Pull request that fixes a bug issue label Aug 19, 2024
@jamiehynds
Copy link
Copy Markdown

Hi @bjmcnic - a customer has been in touch looking for an update on this PR. Their main focus is 40961 and 40962 event support within the powershell_operational pipeline? Is it possible to prioritise that pipeline first, and handle the forwarded events (and other) pipelines separately?

@bjmcnic
Copy link
Copy Markdown
Contributor Author

bjmcnic commented Sep 10, 2024

@jamiehynds I added the tests for powershell_operational and the forwarded events for that. I was unable to naturally generate erring events for powershell or sysmon_operational, but I put the fixes in anyway. We'll see what reviewers think.

@elasticmachine
Copy link
Copy Markdown

🚀 Benchmarks report

Package windows 👍(4) 💚(2) 💔(3)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
applocker_msi_and_script 9433.96 7462.69 -1971.27 (-20.9%) 💔
applocker_packaged_app_deployment 14705.88 11904.76 -2801.12 (-19.05%) 💔
powershell_operational 4347.83 2976.19 -1371.64 (-31.55%) 💔

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link
Copy Markdown

💚 Build Succeeded

History

cc @bjmcnic

@elastic-sonarqube
Copy link
Copy Markdown

Copy link
Copy Markdown
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for adding the test cases.

@bjmcnic bjmcnic merged commit 8711ef8 into elastic:main Sep 12, 2024
@elasticmachine
Copy link
Copy Markdown

Package windows - 2.0.2 containing this change is available at https://epr.elastic.co/search?package=windows

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
Fix pipeline_error for powershell_operational events 40961 and 40962.
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
Fix pipeline_error for powershell_operational events 40961 and 40962.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes a bug issue Integration:windows Windows Team:Security-Windows Platform Security Windows Platform team [elastic/sec-windows-platform]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants