Validate Source IP "LOCAL" or "Unknown" in Windows Security Logs by MakoWish · Pull Request #34295 · elastic/beats

MakoWish · 2023-01-17T21:28:54Z

What does this PR do?

Some security events contain a source IP address of "LOCAL" or "Unknown" which are not valid IP addresses. This PR will correct the processing of events containing one of those values.

Why is it important?

This bug causes mapping exceptions and prevents these events from being ingested.

Checklist

I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

-fixes #34263

mergify · 2023-01-17T21:29:45Z

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @MakoWish? 🙏.
For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

elasticmachine · 2023-01-17T21:34:32Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-01-19T00:54:51.954+0000
Duration: 39 min 51 sec

Test stats 🧪

Test	Results
Failed	0
Passed	336
Skipped	0
Total	336

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2023-01-17T22:24:17Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

efd6 · 2023-01-17T22:25:00Z

/test

efd6

Thanks.

Are you able to provide events to exercise this in tests?

MakoWish · 2023-01-17T22:35:33Z

I unfortunately cannot provide EVTX files for this one, as the only record of this happening so far is from servers, and I do not have access to log into those servers. Since I have already made the changes to our Ingest Pipeline, I could provide the JSON of some of these documents in Elasticsearch for these events.

MakoWish · 2023-01-17T22:39:37Z

Scratch that. I was able to find a client with some of these. As with the other PR, these events do contain sensitive information. Please let me know how/where I can send them to you, and I will do that.

MakoWish · 2023-01-17T23:14:56Z

Attaching five scrubbed events of each where the source IP is either "LOCAL" or "Unknown".
source_ip_invalid.zip

MakoWish · 2023-01-17T23:57:00Z

Can we back-port this?

Tests cases mechanically derived from user-provided XML scrubbed event data.

efd6 · 2023-01-18T01:25:35Z

I've added the tests temporarily in the collection testdata. I want to flesh this out with a way to just drop XML files into inputs, but this is a bigger change, so I will do that separately.

efd6 · 2023-01-18T02:56:34Z

/test

efd6 · 2023-01-19T00:54:38Z

/test

efd6

Thanks

) Tests cases mechanically derived from user-provided XML scrubbed event data. Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co> (cherry picked from commit 4a1e56f) # Conflicts: # x-pack/winlogbeat/module/security/ingest/security.yml # x-pack/winlogbeat/module/security/test/testdata/4778.evtx.golden.json # x-pack/winlogbeat/module/security/test/testdata/4778.golden.json # x-pack/winlogbeat/module/security/test/testdata/4779.evtx.golden.json # x-pack/winlogbeat/module/security/test/testdata/4779.golden.json

) Tests cases mechanically derived from user-provided XML scrubbed event data. Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co> (cherry picked from commit 4a1e56f)

) (#34309) Tests cases mechanically derived from user-provided XML scrubbed event data. Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co> (cherry picked from commit 4a1e56f) Co-authored-by: Eric <26614684+MakoWish@users.noreply.github.com>

) Tests cases mechanically derived from user-provided XML scrubbed event data. Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>

Validate Source IP "LOCAL" or "Unknown" in Windows Security Logs

ac29052

MakoWish requested a review from a team as a code owner January 17, 2023 21:28

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 17, 2023

mergify bot assigned MakoWish Jan 17, 2023

MakoWish mentioned this pull request Jan 17, 2023

Validate Source IP "LOCAL" or "Unknown" in Windows Security Logs #34284

Closed

1 task

efd6 added enhancement Winlogbeat Team:Security-External Integrations backport-skip Skip notification from the automated backport with mergify 8.7-candidate labels Jan 17, 2023

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 17, 2023

efd6 reviewed Jan 17, 2023

View reviewed changes

efd6 added bug backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.6.0 Automated backport with mergify and removed backport-skip Skip notification from the automated backport with mergify enhancement labels Jan 18, 2023

add tests cases for unknown and local address

876f944

Tests cases mechanically derived from user-provided XML scrubbed event data.

Merge branch 'elastic:main' into fix_source_ip_local

17fc53f

efd6 approved these changes Jan 19, 2023

View reviewed changes

efd6 merged commit 4a1e56f into elastic:main Jan 19, 2023

mergify bot mentioned this pull request Jan 19, 2023

[7.17](backport #34295) Validate Source IP "LOCAL" or "Unknown" in Windows Security Logs #34308

Closed

mergify bot mentioned this pull request Jan 19, 2023

[8.6](backport #34295) Validate Source IP "LOCAL" or "Unknown" in Windows Security Logs #34309

Merged

efd6 mentioned this pull request Mar 20, 2023

Regression to #19627 for Security Events with Source IP "LOCAL" #34263

Closed

MakoWish deleted the fix_source_ip_local branch March 25, 2023 18:18

chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023

Validate Source IP "LOCAL" or "Unknown" in Windows Security Logs (#34295

b37800f

) Tests cases mechanically derived from user-provided XML scrubbed event data. Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>

chemamartinez mentioned this pull request Aug 3, 2023

[System] Validate ClientAddress IP for Windows Security events elastic/integrations#7237

Merged

4 tasks

Conversation

MakoWish commented Jan 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

Related issues

Uh oh!

mergify bot commented Jan 17, 2023

Uh oh!

elasticmachine commented Jan 17, 2023 • edited by jenkins-beats-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

Uh oh!

elasticmachine commented Jan 17, 2023

Uh oh!

efd6 commented Jan 17, 2023

Uh oh!

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

MakoWish commented Jan 17, 2023

Uh oh!

MakoWish commented Jan 17, 2023

Uh oh!

MakoWish commented Jan 17, 2023

Uh oh!

MakoWish commented Jan 17, 2023

Uh oh!

efd6 commented Jan 18, 2023

Uh oh!

efd6 commented Jan 18, 2023

Uh oh!

efd6 commented Jan 19, 2023

Uh oh!

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MakoWish commented Jan 17, 2023 •

edited

Loading

elasticmachine commented Jan 17, 2023 •

edited by jenkins-beats-ci bot

Loading