Skip to content

[Filebeat] [auditd]: Support EXECVE events with truncated argument list#30382

Merged
adriansr merged 3 commits intoelastic:mainfrom
adriansr:fix_fb_auditd_exec_truncated
Feb 17, 2022
Merged

[Filebeat] [auditd]: Support EXECVE events with truncated argument list#30382
adriansr merged 3 commits intoelastic:mainfrom
adriansr:fix_fb_auditd_exec_truncated

Conversation

@adriansr
Copy link
Copy Markdown
Contributor

@adriansr adriansr commented Feb 14, 2022 •
edited
Loading

What does this PR do?

This modifies Filebeat's auditd ingest pipeline to support parsing of EXECVE events with truncated argument lists.

In a normal EXECVE event, the auditd fields argc(=N) and a0 to aN-1 are present. The pipeline would store the arguments in the process.args array, as well as set process.args_count to N and process.executable to process.args[0].

A truncated EXECVE event usually lacks the argc field, and contains only the last few aNN fields.

In that case, this PR will add the arguments into process.args with a leading warning [... N truncated arguments ...] and will not populate process.executable.

Why is it important?

This PR avoids ingesting an arbitrarily large number of fields in the form aNN, aNN_len and aNN[M], to prevent a mapping explosion leading to large indices:

Could not index event to Elasticsearch: "status"=>400,
"error"=>{
"type"=>"illegal_argument_exception",
"reason"=>"Limit of total fields [10000] has been exceeded"}}

This was partially fixed by #29601, but after it was merged, we observed truncated EXECVE records that were still causing issues.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@adriansr
Auditd: Support EXECVE records with truncated argument list
87302b4 
This modifies Filebeat's auditd pipeline to support parsing of EXECVE
records with truncated argument lists.

When such a log is found, the arguments will be appended to process.args
with a leading entry informing about the truncation.

This is to prevent a mapping explosion in Filebeat when a lot of these
logs are ingested with the previous pipeline version.
@adriansr
Changelog entry
3fbb6df
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 14, 2022
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Feb 14, 2022

This pull request does not have a backport label. Could you fix it @adriansr? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Feb 14, 2022
@adriansr adriansr added backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.0.0 Automated backport with mergify backport-v8.1.0 Automated backport with mergify bug review Team:Security-External Integrations labels Feb 14, 2022
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Feb 14, 2022

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 14, 2022
@mergify mergify bot removed the backport-skip Skip notification from the automated backport with mergify label Feb 14, 2022
@adriansr adriansr added the needs_integration_sync Changes in this PR need synced to elastic/integrations. label Feb 14, 2022
@adriansr adriansr requested review from a team and andrewkroh February 14, 2022 19:03
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Feb 14, 2022 •
edited by jenkins-beats-ci bot
Loading

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-02-17T14:01:37.132+0000

  • Duration: 89 min 13 sec

Test stats 🧪

Test Results
Failed 1
Passed 8498
Skipped 911
Total 9410

Test errors 1

Expand to view the tests failures

Extended / filebeat-windows-2016-windows-2016 / TestNewModuleRegistry – github.com/elastic/beats/v7/filebeat/fileset
    Expand to view the error details

     Failed

    Expand to view the stacktrace

     === RUN   TestNewModuleRegistry
{"log.level":"info","@timestamp":"2022-02-17T15:17:55.901Z","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":108},"message":"Enabled modules/filesets: nginx (error), nginx (access), mysql (slowlog), mysql (error), system (syslog), system (auth), auditd (log)","ecs.version":"1.6.0"}
    modules_test.go:104: 
        	Error Trace:	modules_test.go:104
        	Error:      	Not equal: 
        	            	expected: []string{"error", "access"}
        	            	actual  : []string{"access", "error"}
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1,4 +1,4 @@
        	            	 ([]string) (len=2) {
        	            	- (string) (len=5) "error",
        	            	- (string) (len=6) "access"
        	            	+ (string) (len=6) "access",
        	            	+ (string) (len=5) "error"
        	            	 }
        	Test:       	TestNewModuleRegistry
--- FAIL: TestNewModuleRegistry (0.02s)

Steps errors 7

Expand to view the steps failures

filebeat-goIntegTest - mage goIntegTest
  • Took 5 min 10 sec . View more details here
  • Description: mage goIntegTest
filebeat-windows-2016-windows-2016 - mage build unitTest
  • Took 2 min 25 sec . View more details here
  • Description: mage build unitTest
filebeat-windows-2016-windows-2016 - mage build unitTest
  • Took 1 min 33 sec . View more details here
  • Description: mage build unitTest
filebeat-windows-2016-windows-2016 - mage build unitTest
  • Took 1 min 33 sec . View more details here
  • Description: mage build unitTest
gsutil -m -q cp -a public-read test-build-artifacts-filebeat-windows-2016-windows-2016-tgz gs://beat
  • Took 0 min 2 sec . View more details here
  • Description: @echo off gsutil -m -q cp -a public-read test-build-artifacts-filebeat-windows-2016-windows-2016-tgz gs://beats-ci-temp/Beats/beats/PR-30382-2
filebeat-windows-10-windows-10 - mage build unitTest
  • Took 3 min 12 sec . View more details here
  • Description: mage build unitTest
Error signal
  • Took 0 min 0 sec . View more details here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors 1

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Extended / filebeat-windows-2016-windows-2016 / TestNewModuleRegistry – github.com/elastic/beats/v7/filebeat/fileset

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@adriansr
Copy link
Copy Markdown
Contributor Author

adriansr commented Feb 17, 2022

Tests failed due to flaky test, fixed here #30453

@adriansr adriansr merged commit 79229e7 into elastic:main Feb 17, 2022
@adriansr adriansr deleted the fix_fb_auditd_exec_truncated branch February 17, 2022 17:18
mergify bot pushed a commit that referenced this pull request Feb 17, 2022
@adriansr
[Filebeat] [auditd]: Support EXECVE events with truncated argument li…
c45304a 
…st (#30382)

This modifies Filebeat's auditd pipeline to support parsing of EXECVE
records with truncated argument lists.

When such a log is found, the arguments will be appended to process.args
with a leading entry informing about the truncation.

This is to prevent a mapping explosion in Filebeat when a lot of these
logs are ingested with the previous pipeline version.

(cherry picked from commit 79229e7)

# Conflicts:
#	filebeat/module/auditd/log/ingest/pipeline.yml
@mergify mergify bot mentioned this pull request Feb 17, 2022
Closed
mergify bot pushed a commit that referenced this pull request Feb 17, 2022
@adriansr
[Filebeat] [auditd]: Support EXECVE events with truncated argument li…
27c8a0f 
…st (#30382)

This modifies Filebeat's auditd pipeline to support parsing of EXECVE
records with truncated argument lists.

When such a log is found, the arguments will be appended to process.args
with a leading entry informing about the truncation.

This is to prevent a mapping explosion in Filebeat when a lot of these
logs are ingested with the previous pipeline version.

(cherry picked from commit 79229e7)
@mergify mergify bot mentioned this pull request Feb 17, 2022
Merged
mergify bot pushed a commit that referenced this pull request Feb 17, 2022
@adriansr
[Filebeat] [auditd]: Support EXECVE events with truncated argument li…
2e5cf52 
…st (#30382)

This modifies Filebeat's auditd pipeline to support parsing of EXECVE
records with truncated argument lists.

When such a log is found, the arguments will be appended to process.args
with a leading entry informing about the truncation.

This is to prevent a mapping explosion in Filebeat when a lot of these
logs are ingested with the previous pipeline version.

(cherry picked from commit 79229e7)
@mergify mergify bot mentioned this pull request Feb 17, 2022
Merged
adriansr added a commit that referenced this pull request Feb 17, 2022
@mergify @adriansr
[8.1](backport #30382) [Filebeat] [auditd]: Support EXECVE events wit…
a7bcd67 
…h truncated argument list (#30457)

* [Filebeat] [auditd]: Support EXECVE events with truncated argument list (#30382)

This modifies Filebeat's auditd pipeline to support parsing of EXECVE
records with truncated argument lists.

When such a log is found, the arguments will be appended to process.args
with a leading entry informing about the truncation.

This is to prevent a mapping explosion in Filebeat when a lot of these
logs are ingested with the previous pipeline version.

(cherry picked from commit 79229e7)

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>
adriansr added a commit that referenced this pull request Feb 17, 2022
@mergify @adriansr
[8.0](backport #30382) [Filebeat] [auditd]: Support EXECVE events wit…
8e4e1fa 
…h truncated argument list (#30456)

* [Filebeat] [auditd]: Support EXECVE events with truncated argument list (#30382)

This modifies Filebeat's auditd pipeline to support parsing of EXECVE
records with truncated argument lists.

When such a log is found, the arguments will be appended to process.args
with a leading entry informing about the truncation.

This is to prevent a mapping explosion in Filebeat when a lot of these
logs are ingested with the previous pipeline version.

(cherry picked from commit 79229e7)

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>
v1v added a commit to v1v/beats that referenced this pull request Feb 21, 2022
@v1v
Merge remote-tracking branch 'upstream/main' into feature/use-with-ki…
6c6cce5 
…nd-k8s-env

* upstream/main:
  fix typos and improve sentences (elastic#30432)
  Add drop and explicit tests to avoid duplicate ingest of elasticsearch logs (elastic#30440)
  {,x-pack/}auditbeat: replace uses of github.com/pkg/errors with stdlib equivalents (elastic#30321)
  Spelling fix (elastic#30439)
  packetbeat/beater: make sure Npcap installation runs before interfaces are needed in all cases (elastic#30438)
  Add BC about Homebrew no longer being available in 8.0 (elastic#30419)
  Install gawk as a replacement for mawk in Docker containers. (elastic#30452)
  Clean up python-related system tests (elastic#30415)
  Fix TestNewModuleRegistry flakiness (elastic#30453)
  [Filebeat] [auditd]: Support EXECVE events with truncated argument list (elastic#30382)
  Set `log.offset` to the start of the reported line in filestream (elastic#30445)
  clarify SelectedPackageTypes meaning and improve its usage (elastic#30142)
  [elasticsearch module] serialize shards properties (elastic#30408)
  Add docs about hints and templates autodiscovery priority (elastic#30343)
v1v added a commit to v1v/beats that referenced this pull request Feb 22, 2022
@v1v
Merge remote-tracking branch 'upstream/main' into feature/refactor-pa…
29471a7 
…ckaging-docker

* upstream/main: (26 commits)
  Update docker/distribution to 2.8.0 (elastic#30462)
  Add `parsers` examples to `filestream` reference configuration (elastic#30529)
  extend documentation about setting orchestrator.cluster fields (elastic#30518)
  Forward-port 8.0.1 changelog to main (elastic#30522)
  Switch skip to use `CI` (elastic#30512)
  packetbeat/beater: don't attempt to install npcap when already installed (elastic#30509)
  Fix Docker module: rename fields on dashboards (elastic#30500)
  fix typos and improve sentences (elastic#30432)
  Add drop and explicit tests to avoid duplicate ingest of elasticsearch logs (elastic#30440)
  {,x-pack/}auditbeat: replace uses of github.com/pkg/errors with stdlib equivalents (elastic#30321)
  Spelling fix (elastic#30439)
  packetbeat/beater: make sure Npcap installation runs before interfaces are needed in all cases (elastic#30438)
  Add BC about Homebrew no longer being available in 8.0 (elastic#30419)
  Install gawk as a replacement for mawk in Docker containers. (elastic#30452)
  Clean up python-related system tests (elastic#30415)
  Fix TestNewModuleRegistry flakiness (elastic#30453)
  [Filebeat] [auditd]: Support EXECVE events with truncated argument list (elastic#30382)
  Set `log.offset` to the start of the reported line in filestream (elastic#30445)
  clarify SelectedPackageTypes meaning and improve its usage (elastic#30142)
  [elasticsearch module] serialize shards properties (elastic#30408)
  ...
@adriansr adriansr mentioned this pull request Feb 22, 2022
Merged
4 tasks
adriansr added a commit to elastic/integrations that referenced this pull request Feb 23, 2022
@adriansr
[Auditd] Populate process.args array with process arguments (#2730)
c91d581 
Prevents the indices exceeding the 10,000 field limit due to an
arbitrarily large number of aNN fields.

This is a combination of the following Filebeat module fixes:
 - elastic/beats#29601
 - elastic/beats#30382

Updates version to 2.1.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewkroh andrewkroh andrewkroh approved these changes

Assignees

@adriansr adriansr

Labels

backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.0.0 Automated backport with mergify backport-v8.1.0 Automated backport with mergify bug needs_integration_sync Changes in this PR need synced to elastic/integrations. review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@adriansr @elasticmachine @andrewkroh