Skip to content

refactor(system.auth): consolidate auth message parsing#16542

Merged
andrewkroh merged 3 commits intoelastic:mainfrom
andrewkroh:system/refactor/auth-deduplication
Jan 20, 2026
Merged

refactor(system.auth): consolidate auth message parsing#16542
andrewkroh merged 3 commits intoelastic:mainfrom
andrewkroh:system/refactor/auth-deduplication

Conversation

@andrewkroh
Copy link
Copy Markdown
Member

Proposed commit message

Extract duplicated message parsing logic from log.yml and journald.yml
into a shared message.yml pipeline. This consolidates processing for
SSH events, sudo commands, PAM messages, user/group management, and
event categorization into a single location.

The shared pipeline operates on an input contract where callers
set the 'message' field before invocation. This eliminates the
_temp.message intermediate field.

The log pipeline parses the syslog header and extracts the message body
directly into the message field (when the beats syslog processor hasn't 
already done so).

The journald pipeline does some minor cleanup to fields produced by the
journald input to align with ECS and eliminate noise.

The message field is now preserved for all events regardless
of input type, unifying behavior between log and journald inputs.

Add event.kind: pipeline_error to the default.yml on_failure handler
per best practices. Fix mustache template escaping in message.yml
error messages to use triple braces for proper value interpolation.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

This will make it easier to incorporate the enhanced PAM log processing logic from #14456.

Screenshots

Extract duplicated message parsing logic from log.yml and journald.yml
into a shared message.yml pipeline. This consolidates processing for
SSH events, sudo commands, PAM messages, user/group management, and
event categorization into a single location.

The shared pipeline operates on an input contract where callers
set the 'message' field before invocation. This eliminates the
_temp.message intermediate field.

The log pipeline parses the syslog header and extracts the message body
directly into the message field (when the beats syslog processor hasn't
already done so).

The journald pipeline does some minor cleanup to fields produced by the
journald input to align with ECS and eliminate noise.

The message field is now preserved for all events regardless
of input type, unifying behavior between log and journald inputs.

Add event.kind: pipeline_error to the default.yml on_failure handler
per best practices. Fix mustache template escaping in message.yml
error messages to use triple braces for proper value interpolation.
@andrewkroh andrewkroh force-pushed the system/refactor/auth-deduplication branch from 239f83c to f7a2aa4 Compare December 12, 2025 20:59
@andrewkroh andrewkroh added Integration:system System enhancement New feature or request labels Dec 12, 2025
@andrewkroh andrewkroh marked this pull request as ready for review December 12, 2025 21:01
@andrewkroh andrewkroh requested review from a team as code owners December 12, 2025 21:01
@andrewkroh andrewkroh mentioned this pull request Dec 12, 2025
1 task
@andrewkroh andrewkroh added the Team:Security-Linux Platform Linux Platform Security team [elastic/sec-linux-platform] label Dec 12, 2025
@elasticmachine
Copy link
Copy Markdown

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

@elastic-vault-github-plugin-prod
Copy link
Copy Markdown

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@botelastic
Copy link
Copy Markdown

botelastic bot commented Jan 11, 2026

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Jan 11, 2026
@botelastic botelastic bot removed the Stalled label Jan 13, 2026
Copy link
Copy Markdown
Contributor

@Tacklebox Tacklebox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The auth log pipelines have a single source of truth and can't get out of sync now 👍

@andrewkroh andrewkroh enabled auto-merge (squash) January 20, 2026 22:40
@elasticmachine
Copy link
Copy Markdown

💚 Build Succeeded

History

@andrewkroh andrewkroh merged commit 0723475 into elastic:main Jan 20, 2026
8 checks passed
@elastic-vault-github-plugin-prod
Copy link
Copy Markdown

Package system - 2.11.0 containing this change is available at https://epr.elastic.co/package/system/2.11.0/

jakubgalecki0 pushed a commit to jakubgalecki0/integrations that referenced this pull request Feb 19, 2026
Extract duplicated message parsing logic from log.yml and journald.yml
into a shared message.yml pipeline. This consolidates processing for
SSH events, sudo commands, PAM messages, user/group management, and
event categorization into a single location.

The shared pipeline operates on an input contract where callers
set the 'message' field before invocation. This eliminates the
_temp.message intermediate field.

The log pipeline parses the syslog header and extracts the message body
directly into the message field (when the beats syslog processor hasn't
already done so).

The journald pipeline does some minor cleanup to fields produced by the
journald input to align with ECS and eliminate noise.

The message field is now preserved for all events regardless
of input type, unifying behavior between log and journald inputs.

Add event.kind: pipeline_error to the default.yml on_failure handler
per best practices. Fix mustache template escaping in message.yml
error messages to use triple braces for proper value interpolation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:system System Team:Security-Linux Platform Linux Platform Security team [elastic/sec-linux-platform]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants