Skip to content

Adding preserve original event option to System integration Syslog datastream#6528

Merged
kgeller merged 12 commits intoelastic:mainfrom
kgeller:system-syslog-preserve-original
Jun 30, 2023
Merged

Adding preserve original event option to System integration Syslog datastream#6528
kgeller merged 12 commits intoelastic:mainfrom
kgeller:system-syslog-preserve-original

Conversation

@kgeller
Copy link
Copy Markdown
Member

@kgeller kgeller commented Jun 9, 2023 •
edited
Loading

What does this PR do?

This adds the option to the syslog datastream to ensure users can preserve the original message if required.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Closes #6524

Screenshots

Screenshot 2023-06-09 at 2 47 36 PM

@kgeller kgeller added the enhancement New feature or request label Jun 9, 2023
@kgeller kgeller self-assigned this Jun 9, 2023
@kgeller kgeller changed the title Adding preserve event.original to System integration Syslog datastream Adding preserve original event option to System integration Syslog datastream Jun 9, 2023
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Jun 9, 2023 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-06-30T12:53:18.338+0000

  • Duration: 16 min 32 sec

Test stats 🧪

Test Results
Failed 0
Passed 146
Skipped 0
Total 146

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Jun 9, 2023 •
edited
Loading

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (3/3) 💚
Files 100.0% (4/4) 💚 2.198
Classes 100.0% (4/4) 💚 2.198
Methods 60.759% (48/79) 👎 -34.478
Lines 100.0% (2811/2811) 💚 8.06
Conditionals 100.0% (0/0) 💚

@kgeller kgeller marked this pull request as ready for review June 9, 2023 18:45
@kgeller kgeller requested a review from a team as a code owner June 9, 2023 18:45
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Jun 9, 2023

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

andrewkroh
andrewkroh reviewed Jun 9, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@andrewkroh andrewkroh left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other changes are needed to implement this. Take a look at other packages for an example config. The ingest pipeline needs updated.

@kgeller
Copy link
Copy Markdown
Member Author

kgeller commented Jun 9, 2023

Other changes are needed to implement this. Take a look at other packages for an example config. The ingest pipeline needs updated.

Oops, I was totally looking at the wrong pipeline and thought it was already there. Should be all there now.

andrewkroh
andrewkroh reviewed Jun 9, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a new pipeline test that enables the preserve_original_event.

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Jun 13, 2023

💚 Build Succeeded

cc @kgeller

@LaZyDK LaZyDK mentioned this pull request Jun 19, 2023
Merged
4 tasks
@kgeller
Copy link
Copy Markdown
Member Author

kgeller commented Jun 20, 2023

@elastic/obs-infraobs-integrations Would someone mind reviewing since the team is the code owner?

@andrewkroh andrewkroh added the Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] label Jun 22, 2023
@kgeller
Copy link
Copy Markdown
Member Author

kgeller commented Jun 22, 2023

/test

1 similar comment
@kgeller
Copy link
Copy Markdown
Member Author

kgeller commented Jun 23, 2023

/test

@ishleenk17
Copy link
Copy Markdown
Member

ishleenk17 commented Jun 29, 2023

@elastic/obs-infraobs-integrations Would someone mind reviewing since the team is the code owner?

This is under Review. @kgeller

ishleenk17
ishleenk17 reviewed Jun 29, 2023
View reviewed changes
packages/system/data_stream/process/sample_event.json Outdated
@@ -0,0 +1,112 @@
{
Copy link
Copy Markdown
Member

@ishleenk17 ishleenk17 Jun 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks to be a completely newly generated sample event. Any reason why this was not present before ?

Copy link
Copy Markdown
Member Author

@kgeller kgeller Jun 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not. Will delete since adding it was not intentional. I was simply attempting to regenerate things to see if I could get the ci green.

@kgeller kgeller requested a review from ishleenk17 June 29, 2023 16:54
@ishleenk17 ishleenk17 requested a review from a team June 30, 2023 06:28
ishleenk17
ishleenk17 approved these changes Jun 30, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@ishleenk17 ishleenk17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@kgeller kgeller merged commit 188de0f into elastic:main Jun 30, 2023
@kgeller kgeller deleted the system-syslog-preserve-original branch June 30, 2023 14:03
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Jun 30, 2023

Package system - 1.35.0 containing this change is available at https://epr.elastic.co/search?package=system

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewkroh andrewkroh andrewkroh approved these changes

@ishleenk17 ishleenk17 ishleenk17 approved these changes

Assignees

@kgeller kgeller

Labels

enhancement New feature or request Integration:system System Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations]

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[System] Adding preserve event.original to System logs

4 participants

@kgeller @elasticmachine @ishleenk17 @andrewkroh