[Auditbeat] fim(ebpf): enrich file events with process data by mmat11 · Pull Request #38199 · elastic/beats

mmat11 · 2024-03-06T15:08:40Z

Proposed commit message

fim(ebpf): enrich file events with process data

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

Make sure entity ID is calculated consistently everywhere // copied from https://github.com/elastic/beats/blob/main/x-pack/auditbeat/module/system/process/process.go#L139

Related issues

elastic/integrations#7401

Screenshot

mergify · 2024-03-06T15:09:50Z

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @mmat11? 🙏.
For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

elasticmachine · 2024-03-06T15:15:54Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Duration: 180 min 13 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-03-08T12:45:04Z

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

auditbeat/_meta/fields.common.yml

auditbeat/docs/fields.asciidoc

auditbeat/module/file_integrity/event.go

nicholasberlin

LGTM FWIW

elasticmachine · 2024-03-11T16:33:47Z

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

leehinman

Overall looks good.

Just a couple of questions. process.pid matching the ECS type is the biggest question.

auditbeat/_meta/fields.common.yml

auditbeat/module/file_integrity/event.go

auditbeat/module/file_integrity/event_linux.go

…user-data

pkoutsovasilis · 2024-04-05T09:34:00Z

run docs-build rebuild

alexsapran · 2024-04-05T09:36:38Z

run docs-build

pierrehilbert

Thx for your work and investigations!

dliappis · 2024-04-05T10:20:10Z

metricbeat/Dockerfile

@@ -1,4 +1,5 @@
 FROM golang:1.21.8
+COPY --from=docker:26.0.0-alpine3.19 /usr/local/bin/docker  /usr/local/bin/


Can we leave a comment (with a TODO perhaps that it will be addressed via #38678) ?

* fim(ebpf): enrich file events with process data * apply review suggestions * apply review suggestions * fix(fim/ebpf): move process fields to event root and insert them so keys do not contain dots * fix(fim/ebpf): refactor HostID to utilise sync.OnceValue and expose boot time * fix(fim/ebpf): refactor TicksPerSecond to utilise sync.OnceValue * fix(fim/ebpf): remove empty slice allocation * chore: go mod tidy * fix: explicitly set go 1.21.8 in go.mod * fix(fim/ebpf): nil slice of errors in TestNewEventFromEbpfEvent * fix(fim/ebpf): remove re-declaration of already ecs included fields * fix(fim/ebpf): utilise OnceValues to declutter the code * fix(fim/ebpf): remove x-pack import from OSS package * fix(fim/ebpf): propagate process fields changes to integration tests * chore: go mod tidy * ci: temporary solution to outdated docker compose python library * ci: transition to a fixed tag for docker image instead of a rolling one --------- Co-authored-by: Panos Koutsovasilis <panos.koutsovasilis@elastic.co> Co-authored-by: Pierre HILBERT <pierre.hilbert@elastic.co> (cherry picked from commit dbdaac3) # Conflicts: # go.mod # go.sum

…38743) * Manual port of docker CI fix from #38199 * Fix order in requirements.txt.

…38743) * Manual port of docker CI fix from #38199 * Fix order in requirements.txt. (cherry picked from commit 33b776a) # Conflicts: # libbeat/tests/system/requirements.txt # libbeat/tests/system/requirements_aix.txt # metricbeat/Dockerfile

…ocker-compose package (#38746) * [7.17] Fix Python systems tests with forked docker-compose package (#38743) * Manual port of docker CI fix from #38199 * Fix order in requirements.txt. (cherry picked from commit 33b776a) # Conflicts: # libbeat/tests/system/requirements.txt # libbeat/tests/system/requirements_aix.txt # metricbeat/Dockerfile * Resolve conflicts * Restore uintentionally removed packages * Remove duplicate package. * Add dropped docker copy --------- Co-authored-by: Craig MacKenzie <craig.mackenzie@elastic.co>

…38199) * fim(ebpf): enrich file events with process data * apply review suggestions * apply review suggestions * fix(fim/ebpf): move process fields to event root and insert them so keys do not contain dots * fix(fim/ebpf): refactor HostID to utilise sync.OnceValue and expose boot time * fix(fim/ebpf): refactor TicksPerSecond to utilise sync.OnceValue * fix(fim/ebpf): remove empty slice allocation * chore: go mod tidy * fix: explicitly set go 1.21.8 in go.mod * fix(fim/ebpf): nil slice of errors in TestNewEventFromEbpfEvent * fix(fim/ebpf): remove re-declaration of already ecs included fields * fix(fim/ebpf): utilise OnceValues to declutter the code * fix(fim/ebpf): remove x-pack import from OSS package * fix(fim/ebpf): propagate process fields changes to integration tests * chore: go mod tidy * ci: temporary solution to outdated docker compose python library * ci: transition to a fixed tag for docker image instead of a rolling one --------- Co-authored-by: Panos Koutsovasilis <panos.koutsovasilis@elastic.co> Co-authored-by: Pierre HILBERT <pierre.hilbert@elastic.co>

…38743) * Manual port of docker CI fix from #38199 * Fix order in requirements.txt. (cherry picked from commit 33b776a) # Conflicts: # libbeat/tests/system/requirements.txt # libbeat/tests/system/requirements_aix.txt # metricbeat/Dockerfile

…h process data (#38742) * [Auditbeat] fim(ebpf): enrich file events with process data (#38199) * fim(ebpf): enrich file events with process data * apply review suggestions * apply review suggestions * fix(fim/ebpf): move process fields to event root and insert them so keys do not contain dots * fix(fim/ebpf): refactor HostID to utilise sync.OnceValue and expose boot time * fix(fim/ebpf): refactor TicksPerSecond to utilise sync.OnceValue * fix(fim/ebpf): remove empty slice allocation * chore: go mod tidy * fix: explicitly set go 1.21.8 in go.mod * fix(fim/ebpf): nil slice of errors in TestNewEventFromEbpfEvent * fix(fim/ebpf): remove re-declaration of already ecs included fields * fix(fim/ebpf): utilise OnceValues to declutter the code * fix(fim/ebpf): remove x-pack import from OSS package * fix(fim/ebpf): propagate process fields changes to integration tests * chore: go mod tidy * ci: temporary solution to outdated docker compose python library * ci: transition to a fixed tag for docker image instead of a rolling one --------- Co-authored-by: Panos Koutsovasilis <panos.koutsovasilis@elastic.co> Co-authored-by: Pierre HILBERT <pierre.hilbert@elastic.co> (cherry picked from commit dbdaac3) # Conflicts: # go.mod # go.sum * fix: resolve conflicts --------- Co-authored-by: Mattia Meleleo <melmat@tuta.io> Co-authored-by: Panos Koutsovasilis <panos.koutsovasilis@elastic.co>

mmat11 requested review from a team as code owners March 6, 2024 15:08

mmat11 requested review from fearful-symmetry and leehinman March 6, 2024 15:08

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 6, 2024

mergify bot assigned mmat11 Mar 6, 2024

mmat11 force-pushed the matt/fim-user-data branch from 83d9ec6 to 36c6ecd Compare March 6, 2024 15:12

mmat11 force-pushed the matt/fim-user-data branch 8 times, most recently from ae5dd9b to f11bcbd Compare March 7, 2024 18:39

mmat11 added the Team:Security-Linux Platform Linux Platform Team in Security Solution label Mar 8, 2024

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Mar 8, 2024

mmat11 requested a review from andrewkroh March 8, 2024 12:45

nicholasberlin reviewed Mar 11, 2024

View reviewed changes

auditbeat/_meta/fields.common.yml Outdated Show resolved Hide resolved

auditbeat/docs/fields.asciidoc Outdated Show resolved Hide resolved

auditbeat/module/file_integrity/event.go Show resolved Hide resolved

auditbeat/module/file_integrity/event.go Show resolved Hide resolved

mmat11 force-pushed the matt/fim-user-data branch from afe8971 to 7ada154 Compare March 11, 2024 15:01

nicholasberlin approved these changes Mar 11, 2024

View reviewed changes

pierrehilbert added the Team:Elastic-Agent Label for the Agent team label Mar 11, 2024

leehinman requested changes Mar 11, 2024

View reviewed changes

auditbeat/_meta/fields.common.yml Outdated Show resolved Hide resolved

auditbeat/module/file_integrity/event.go Outdated Show resolved Hide resolved

auditbeat/module/file_integrity/event_linux.go Show resolved Hide resolved

mmat11 force-pushed the matt/fim-user-data branch 2 times, most recently from c98f693 to 9bc78d4 Compare March 12, 2024 13:35

pkoutsovasilis force-pushed the matt/fim-user-data branch from eae0bda to 5a87cee Compare April 3, 2024 21:17

pkoutsovasilis and others added 2 commits April 4, 2024 00:17

chore: go mod tidy

4bd33cf

Merge branch 'main' into matt/fim-user-data

2941987

pkoutsovasilis force-pushed the matt/fim-user-data branch 2 times, most recently from 58cd307 to bd4cf98 Compare April 4, 2024 18:47

ci: temporary solution to outdated docker compose python library

e37a6d3

pkoutsovasilis force-pushed the matt/fim-user-data branch from bd4cf98 to e37a6d3 Compare April 4, 2024 18:53

pkoutsovasilis added 2 commits April 5, 2024 02:58

ci: transition to a fixed tag for docker image instead of a rolling one

1ca5f8b

Merge remote-tracking branch 'refs/remotes/beats/main' into matt/fim-…

4c385e3

…user-data

pierrehilbert approved these changes Apr 5, 2024

View reviewed changes

pkoutsovasilis merged commit dbdaac3 into main Apr 5, 2024

pkoutsovasilis deleted the matt/fim-user-data branch April 5, 2024 10:19

dliappis reviewed Apr 5, 2024

View reviewed changes

faec added the backport-v8.13.0 Automated backport with mergify label Apr 5, 2024

mergify bot mentioned this pull request Apr 5, 2024

[8.13](backport #38199) [Auditbeat] fim(ebpf): enrich file events with process data #38742

Merged

7 tasks

cmacknz added a commit to cmacknz/beats that referenced this pull request Apr 5, 2024

Manual port of docker CI fix from elastic#38199

7ab0e0a

cmacknz mentioned this pull request Apr 5, 2024

[7.17] Fix Python systems tests with forked docker-compose package #38743

Merged

cmacknz added a commit that referenced this pull request Apr 5, 2024

[7.17] Fix Python systems tests with forked docker-compose package (#…

33b776a

…38743) * Manual port of docker CI fix from #38199 * Fix order in requirements.txt.

mergify bot mentioned this pull request Apr 5, 2024

[8.13](backport #38743) [7.17] Fix Python systems tests with forked docker-compose package #38746

Merged

pkoutsovasilis mentioned this pull request Apr 8, 2024

File Integrity Monitoring | User Information - Linux | KProbe Backend elastic/integrations#8692

Closed

This was referenced Apr 8, 2024

[8.12](backport #38743) [7.17] Fix Python systems tests with forked docker-compose package #38766

Closed

[8.13](backport #38328) [Auditbeat] fim(ebpf): enrich file events with container id #38775

Merged

		@@ -1,4 +1,5 @@
		FROM golang:1.21.8
		COPY --from=docker:26.0.0-alpine3.19 /usr/local/bin/docker /usr/local/bin/

Conversation

mmat11 commented Mar 6, 2024 • edited by pkoutsovasilis Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Author's Checklist

Related issues

Screenshot

Uh oh!

mergify bot commented Mar 6, 2024

Uh oh!

elasticmachine commented Mar 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

Uh oh!

elasticmachine commented Mar 8, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nicholasberlin left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Mar 11, 2024

Uh oh!

leehinman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pkoutsovasilis commented Apr 5, 2024

Uh oh!

alexsapran commented Apr 5, 2024

Uh oh!

pierrehilbert left a comment

Choose a reason for hiding this comment

Uh oh!

dliappis Apr 5, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

mmat11 commented Mar 6, 2024 •

edited by pkoutsovasilis

Loading

elasticmachine commented Mar 6, 2024 •

edited

Loading