[windows] Fix parsing of event data fields for event 600#9490
[windows] Fix parsing of event data fields for event 600#9490marc-gr merged 4 commits intoelastic:mainfrom
Conversation
69c4dc4 to
fdf18ee
Compare
|
Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform) |
fdf18ee to
15e2b90
Compare
|
fyi I was checking our "SEI Demo" cluster and I see that error for 600, 403, and 400. 
|
leehinman
left a comment
There was a problem hiding this comment.
LGTM. Some of the args contain "\n", not sure if that is expected or not.
packages/windows/data_stream/powershell/_dev/test/pipeline/test-events.json-expected.json
Show resolved
Hide resolved
🚀 Benchmarks reportPackage
|
| Data stream | Previous EPS | New EPS | Diff (%) | Result |
|---|---|---|---|---|
applocker_msi_and_script |
9523.81 | 7352.94 | -2170.87 (-22.79%) | 💔 |
powershell |
2392.34 | 1506.02 | -886.32 (-37.05%) | 💔 |
To see the full report comment with /test benchmark fullreport
| Split Events 4xx and 600 event data fields. | ||
| Some events can contain multiline values containing also '\n', '\s', and '=' characters, | ||
| for this reason a simple KV processor is not reliable enough and we need a more specific parsing. | ||
| lang: painless |
There was a problem hiding this comment.
packages/windows/data_stream/powershell/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
| source: |- | ||
| def p = ctx.winlog?.event_data[params["field"]]; | ||
| // Define the pattern that will match all keys | ||
| def pat = /(^|(^[\n]?))?\t([^\s\W]+)=/m; |
There was a problem hiding this comment.
I think this regex assumes that the powershell script will be pretty print with indentation.
The loop will skip over such line form the event example
\n\t\t\t\t\t\t\t[Parameter(ValueFromPipeline=\"\", ValueFromPipelineByPropertyName=\"\",
But if the script was not print with identation, could this became
\n\t[Parameter(ValueFromPipeline=\"\", ValueFromPipelineByPropertyName=\"\",
and break the parser?
There was a problem hiding this comment.
I assume that the idea was to put the whole function body under one key
[HostApplication] = [C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe function...]
intxgo
left a comment
There was a problem hiding this comment.
LGTM, the KV parsing might be misaligned in some corner cases, but it'll always parse the content without error, which is an improvement.
💚 Build Succeeded
History
|
|
100.0% Coverage on New Code
0.0% Duplication on New Code|
Package windows - 1.44.5 containing this change is available at https://epr.elastic.co/search?package=windows |
Proposed commit message
Some 600 powershell events can contain multiline values, meaning the current KV split is not enough to handle them. This adds specific logic to handle these.
Checklist
changelog.ymlfile.Closes #9469