Skip to content

[auditbeat] New ETW backend for FIM#45887

Merged
marc-gr merged 48 commits intoelastic:mainfrom
marc-gr:auditbeat/etw-fim
Sep 19, 2025
Merged

[auditbeat] New ETW backend for FIM#45887
marc-gr merged 48 commits intoelastic:mainfrom
marc-gr:auditbeat/etw-fim

Conversation

@marc-gr
Copy link
Copy Markdown
Contributor

@marc-gr marc-gr commented Aug 11, 2025

Proposed commit message

Introduces a new backend for File Integrity Monitoring (FIM) on Windows, built on Event Tracing for Windows (ETW).

Motivation

  • Rich Context: Kernel-level tracing provides detailed context for every operation, including the specific process responsible for the change.

Implementation Details

  1. ETW Consumer: A new consumer subscribes to the Kernel-FileIo ETW provider to receive a raw stream of file system activity events.

  2. Event Correlator: Raw ETW events are extremely granular (e.g., a simple file modification can generate fileCreate, fileWrite, fileSetInformation, and fileClose events). The correlator:

    • Groups these low-level events by file handle and process.
    • Analyzes the events within a group to determine a single, meaningful, user-facing action (created, updated, deleted, moved, etc.).
    • Handles timeouts for operations that are never formally closed.
    • This turns a noisy, high-volume stream of kernel events into concise, actionable FIM data.
  3. Rich Metadata Collection: The implementation collects rich metadata for each event, including:

    • File ownership and permissions.
    • Support for Alternate Data Streams (ADS) and Extended Attributes (EA).

Testing

Added a test to generate samples for both Windows backends for the sake of comparison. This tests can't be enabled to run on CI since their output may vary due to system load, etw buffer flushing, etc. but is still useful for reference and to see how both implementations behave.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

Related issues

@marc-gr marc-gr added enhancement Auditbeat backport-skip Skip notification from the automated backport with mergify Team:Security-Windows Platform Windows Platform Team in Security Solution labels Aug 11, 2025
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Aug 11, 2025
@github-actions
Copy link
Copy Markdown
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 11, 2025

@marc-gr marc-gr marked this pull request as ready for review August 11, 2025 10:35
@marc-gr marc-gr requested review from a team as code owners August 11, 2025 10:35
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)

Copy link
Copy Markdown
Contributor

@belimawr belimawr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the changes in the files that belong to the data-plane.

Copy link
Copy Markdown
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition. There's a lot here to review. I think this needs another reviewer.

Can you ensure that buildkits run the auditbeat tests on Windows with RACE_DETECTOR=true set in the environment.

@marc-gr marc-gr requested a review from andrewkroh September 16, 2025 11:35
Copy link
Copy Markdown
Contributor

@chemamartinez chemamartinez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, mostly focused on the part related with the ETW interface.

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
Copy link
Copy Markdown
Contributor

@colleenmcginnis colleenmcginnis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving to unblock, but I left the tiniest suggestion below. I believe you will also need to rerun make update and commit any changes to get check-docs to pass.

@marc-gr marc-gr enabled auto-merge (squash) September 19, 2025 07:57
@marc-gr marc-gr disabled auto-merge September 19, 2025 13:02
@marc-gr marc-gr enabled auto-merge (squash) September 19, 2025 13:02
@marc-gr marc-gr merged commit bd713ba into elastic:main Sep 19, 2025
249 of 252 checks passed
@marc-gr marc-gr deleted the auditbeat/etw-fim branch September 19, 2025 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Auditbeat backport-skip Skip notification from the automated backport with mergify enhancement Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team Team:Security-Windows Platform Windows Platform Team in Security Solution

Projects

None yet

Development

Successfully merging this pull request may close these issues.

File Integrity Monitoring | User Information - Windows

8 participants