System auth log fixes by Tacklebox · Pull Request #14456 · elastic/integrations

Tacklebox · 2025-07-08T17:54:19Z

Proposed commit message

This adds a grok pattern and a few replacements to handle previously unmatched fields in some auth log messages.

Checklist

I have added an entry to my package's changelog.yml file.

nicholasberlin · 2025-07-08T18:27:48Z

packages/system/data_stream/auth/elasticsearch/ingest_pipeline/log.yml

+      target_field: source.address
+      ignore_missing: true
+      ignore_failure: true
+  - rename:


this rename looks like a repeat of line 82's .. intentional? if so, why?

Unintentional, thanks!

elasticmachine · 2025-07-08T20:00:30Z

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

nicholasberlin

LGTM once CI agrees

belimawr

LGTM. Once CI is green, request a re-review and I'll approve it.

andrewkroh · 2025-07-23T14:52:25Z

packages/system/data_stream/auth/_dev/test/pipeline/test-secure-rhel7.log-expected.json

            "input": {
                "type": "log"
            },
-            "message": "pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=89.160.20.156  user=root",


Why is the message going away? The message should remain IMO to allow for searches on the string (it's indexed as a text-family data type).

It looks like this was already the behavior based on the some other samples, but I think deleting message is the wrong thing to do.

I have opened a fix to address this and consolidate the processing logic for journald and logfile inputs. That will make incorporating these PAM parsing enhancements easier for you.

#16542

elastic-vault-github-plugin-prod · 2025-07-23T15:27:51Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

elastic-sonarqube · 2025-09-15T15:51:27Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

botelastic · 2025-10-15T16:26:39Z

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

… in ecs fields

The changelog was updated but not the manifest.

…erns are successfully parsed

elasticmachine · 2025-10-29T15:53:05Z

💔 Build Failed

Buildkite Build
Commit: 6a6bdce

Failed CI Steps

:pipeline:⬆️ Upload Pipeline: .buildkite/pipeline.yml

History

💔 Build #31365 failed 68cb0e5
💔 Build #30445 failed 926a2fe
💔 Build #30444 failed 7f25953
💚 Build #29186 succeeded fc23255
💚 Build #28989 succeeded 283cf9b

cc @Tacklebox

andrewkroh · 2025-11-07T18:09:41Z

packages/system/data_stream/auth/_dev/test/pipeline/test-auth.log-expected.json

+                "address": "",
+                "domain": ""


Let's remove these if when the values are empty.

botelastic · 2025-12-07T18:48:40Z

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

botelastic · 2026-01-11T21:48:47Z

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

botelastic · 2026-02-10T22:42:16Z

Hi! This PR has been stale for a while and we're going to close it as part of our cleanup procedure. We appreciate your contribution and would like to apologize if we have not been able to review it, due to the current heavy load of the team. Feel free to re-open this PR if you think it should stay open and is worth rebasing. Thank you for your contribution!

Tacklebox self-assigned this Jul 8, 2025

Tacklebox requested a review from a team as a code owner July 8, 2025 17:54

Tacklebox added the bugfix Pull request that fixes a bug issue label Jul 8, 2025

Tacklebox requested review from a team as code owners July 8, 2025 17:58

Tacklebox requested review from belimawr and efd6 July 8, 2025 17:59

nicholasberlin reviewed Jul 8, 2025

View reviewed changes

andrewkroh added Integration:system System Team:Security-Linux Platform Linux Platform Security team [elastic/sec-linux-platform] labels Jul 8, 2025

nicholasberlin approved these changes Jul 9, 2025

View reviewed changes

belimawr reviewed Jul 9, 2025

View reviewed changes

andrewkroh reviewed Jul 23, 2025

View reviewed changes

Tacklebox force-pushed the mborden/system_auth_log_fixes branch from 926a2fe to 68cb0e5 Compare September 15, 2025 15:15

botelastic bot added the Stalled label Oct 15, 2025

Tacklebox and others added 7 commits October 29, 2025 16:45

Moved removal of pam kv entries till after they are used to fill data…

e6a2f87

… in ecs fields

Added test for previously broken log pattern

9244376

Updated expected values with extra data extracted by the new pattern

20f5ef7

Add changelog entry

5c28893

remove duplicate rename

753c87b

fix(packages/system/manifest.yml): apply version 2.5.4

18ae69e

The changelog was updated but not the manifest.

Restore the the message field even if message-type specific grok patt…

6a6bdce

…erns are successfully parsed

Tacklebox force-pushed the mborden/system_auth_log_fixes branch from 68cb0e5 to 6a6bdce Compare October 29, 2025 15:51

botelastic bot removed the Stalled label Oct 29, 2025

andrewkroh reviewed Nov 7, 2025

View reviewed changes

botelastic bot added the Stalled label Dec 7, 2025

andrewkroh mentioned this pull request Dec 12, 2025

refactor(system.auth): consolidate auth message parsing #16542

Merged

5 tasks

botelastic bot removed the Stalled label Dec 12, 2025

botelastic bot added the Stalled label Jan 11, 2026

botelastic bot closed this Feb 10, 2026

Conversation

Tacklebox commented Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Uh oh!

nicholasberlin Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

Tacklebox Jul 9, 2025

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jul 8, 2025

Uh oh!

nicholasberlin left a comment

Choose a reason for hiding this comment

Uh oh!

belimawr left a comment

Choose a reason for hiding this comment

Uh oh!

andrewkroh Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

andrewkroh Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

elastic-vault-github-plugin-prod bot commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

Uh oh!

elastic-sonarqube bot commented Sep 15, 2025

Quality Gate passed

Uh oh!

botelastic bot commented Oct 15, 2025

Uh oh!

elasticmachine commented Oct 29, 2025

💔 Build Failed

Failed CI Steps

History

Uh oh!

andrewkroh Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

botelastic bot commented Dec 7, 2025

Uh oh!

botelastic bot commented Jan 11, 2026

Uh oh!

botelastic bot commented Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Tacklebox commented Jul 8, 2025 •

edited

Loading

elastic-vault-github-plugin-prod bot commented Jul 23, 2025 •

edited

Loading