Skip to content

Move s3 input and cloudwatch input into the same generic data stream#2710

Merged
kaiyan-sheng merged 8 commits intoelastic:mainfrom
kaiyan-sheng:aws_logs
Mar 8, 2022
Merged

Move s3 input and cloudwatch input into the same generic data stream#2710
kaiyan-sheng merged 8 commits intoelastic:mainfrom
kaiyan-sheng:aws_logs

Conversation

@kaiyan-sheng
Copy link
Copy Markdown

@kaiyan-sheng kaiyan-sheng commented Feb 17, 2022 •
edited
Loading

What does this PR do?

This PR is to adjust the new aws_logs package to have one generic data stream. Inside the generic data stream, we includes both s3 input and cloudwatch input as options for users to ingest custom logs from AWS.

With this change, no matter which input users decide to use, the documents will all be stored into aws_logs.generic dataset and same index.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Screenshots

Screen Shot 2022-03-01 at 10 51 29 AM

After changing data stream name:
Screen Shot 2022-03-07 at 5 05 30 PM

@kaiyan-sheng kaiyan-sheng self-assigned this Feb 17, 2022
@kaiyan-sheng kaiyan-sheng added the Team:Cloud-Monitoring Label for the Cloud Monitoring team label Feb 17, 2022
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Feb 17, 2022 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview [preview](https://ci-stats.elastic.co/app/apm/services/beats-ci/transactions/view?rangeFrom=2022-03-08T17:54:47.577Z&rangeTo=2022-03-08T18:14:47.577Z&transactionName=BUILD Ingest-manager/integrations/PR-{number}&transactionType=job&latencyAggregationType=avg&traceId=cfdd79526d087ef0dca7e116c4d95b5f&transactionId=4547f1e96c342a06)

Expand to view the summary

Build stats

  • Start Time: 2022-03-08T18:04:47.577+0000

  • Duration: 11 min 40 sec

Test stats 🧪

Test Results
Failed 0
Passed 2
Skipped 0
Total 2

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@kaiyan-sheng kaiyan-sheng marked this pull request as draft February 23, 2022 12:06
@kaiyan-sheng kaiyan-sheng mentioned this pull request Feb 23, 2022
Closed
@kaiyan-sheng kaiyan-sheng marked this pull request as ready for review March 8, 2022 00:06
@kaiyan-sheng kaiyan-sheng requested a review from a team as a code owner March 8, 2022 00:06
@kaiyan-sheng kaiyan-sheng merged commit c2a89e1 into elastic:main Mar 8, 2022
@kaiyan-sheng kaiyan-sheng deleted the aws_logs branch March 8, 2022 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@endorama endorama endorama approved these changes

Assignees

@kaiyan-sheng kaiyan-sheng

Labels

Team:Cloud-Monitoring Label for the Cloud Monitoring team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@kaiyan-sheng @elasticmachine @endorama