Skip to content

[System] Fix indentation of tags inside syslog datastream#8345

Merged
kcreddy merged 3 commits intoelastic:mainfrom
kcreddy:system_syslog_tags
Oct 31, 2023
Merged

[System] Fix indentation of tags inside syslog datastream#8345
kcreddy merged 3 commits intoelastic:mainfrom
kcreddy:system_syslog_tags

Conversation

@kcreddy
Copy link
Copy Markdown
Contributor

@kcreddy kcreddy commented Oct 31, 2023 •
edited
Loading

Proposed commit message

  • Fix indentation of tags inside system.syslog input configuration.
  • Add system tests for system.syslog datastream
  • Add missing fields "input.type", "log.file.path", and "log.offset" into syslog datastream.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

$ elastic-package stack down && elastic-package build && elastic-package stack up --version=8.10.3 -d -v && eval "$(elastic-package stack shellinit)" && elastic-package test system --generate --data-streams syslog -v 

--- Test results for package: system - START ---
╭─────────┬─────────────┬───────────┬───────────┬────────┬──────────────╮
│ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │
├─────────┼─────────────┼───────────┼───────────┼────────┼──────────────┤
│ system  │ syslog      │ system    │ default   │ PASS   │  44.8591515s │
╰─────────┴─────────────┴───────────┴───────────┴────────┴──────────────╯
--- Test results for package: system - END   ---
Done

Related issues

Screenshots

@kcreddy kcreddy added Integration:system System bugfix Pull request that fixes a bug issue labels Oct 31, 2023
@kcreddy kcreddy self-assigned this Oct 31, 2023
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Oct 31, 2023 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-10-31T04:55:35.878+0000

  • Duration: 19 min 38 sec

Test stats 🧪

Test Results
Failed 0
Passed 156
Skipped 0
Total 156

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Oct 31, 2023

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (3/3) 💚
Files 100.0% (4/4) 💚
Classes 100.0% (4/4) 💚
Methods 67.857% (57/84) 👎 -24.451
Lines 98.415% (2981/3029) 👍 6.834
Conditionals 100.0% (0/0) 💚

@kcreddy kcreddy marked this pull request as ready for review October 31, 2023 05:12
@kcreddy kcreddy requested review from a team as code owners October 31, 2023 05:12
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Oct 31, 2023

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

ishleenk17
ishleenk17 approved these changes Oct 31, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@ishleenk17 ishleenk17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

muthu-mps
muthu-mps reviewed Oct 31, 2023
View reviewed changes
packages/system/data_stream/syslog/sample_event.json
"name": "thermald"
},
"system": {
"syslog": {}
Copy link
Copy Markdown
Contributor

@muthu-mps muthu-mps Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just trying to understand the data that we receive in the system.syslog field. The test logs and the sample events doesn't log any data into this field and the ingest GROK pattern too doesn't process data in the system.syslog field. Can we consider removing the field definition in fields.yml if we are not going to log data in system.syslog field?

Copy link
Copy Markdown
Contributor Author

@kcreddy kcreddy Oct 31, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@muthu-mps Maybe the field is there to let users have their custom fields into this group using the @custom pipeline.
But if its not needed, can you create another issue/PR? I only added some fields because the system tests were failing otherwise.

Copy link
Copy Markdown
Member

@andrewkroh andrewkroh Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This empty object is result of deleting all of the keys from within the syslog map. We should be removing it via the ingest pipeline if it is entirely empty.

It would probably be simpler to write the output of the grok into _temp rather than system.syslog if we will always be renaming everything under system.syslog.* such that it becomes empty. Then we can unconditionally remove _temp at the end.

@kcreddy kcreddy merged commit a43c895 into elastic:main Oct 31, 2023
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Oct 31, 2023

Package system - 1.47.1 containing this change is available at https://epr.elastic.co/search?package=system

2 similar comments
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Oct 31, 2023

Package system - 1.47.1 containing this change is available at https://epr.elastic.co/search?package=system

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Oct 31, 2023

Package system - 1.47.1 containing this change is available at https://epr.elastic.co/search?package=system

@kcreddy kcreddy deleted the system_syslog_tags branch February 7, 2025 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewkroh andrewkroh andrewkroh left review comments

@muthu-mps muthu-mps muthu-mps left review comments

@efd6 efd6 efd6 approved these changes

@ishleenk17 ishleenk17 ishleenk17 approved these changes

Assignees

@kcreddy kcreddy

Labels

bugfix Pull request that fixes a bug issue Integration:system System

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@kcreddy @elasticmachine @andrewkroh @efd6 @muthu-mps @ishleenk17