Skip to content

Cleanup: organizing code in awss3/input.go#38958

Merged
faec merged 4 commits intoelastic:mainfrom
faec:s3input-cleanup
Apr 30, 2024
Merged

Cleanup: organizing code in awss3/input.go#38958
faec merged 4 commits intoelastic:mainfrom
faec:s3input-cleanup

Conversation

@faec
Copy link
Copy Markdown
Contributor

@faec faec commented Apr 15, 2024

Cleanups in x-pack/filebeat/input/awss3/input.go.

  • Split up the two main configuration cases, SQS queues versus bare S3 buckets, into two explicit helper functions (s3Input.runQueueReader and s3Input.runS3Poller) instead of handling them inline in s3Input.Run.
  • Simplify region-detection logic in getRegionFromQueueURL (regionMismatchError is no longer needed)
  • Rename createS3Lister to createS3Poller (since it creates an s3Poller)

This is only a cleanup / reorganization, it does not change any behavior.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@faec faec added the Team:Elastic-Agent Label for the Agent team label Apr 15, 2024
@faec faec self-assigned this Apr 15, 2024
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Apr 15, 2024
@faec faec requested a review from kaiyan-sheng April 15, 2024 20:53
@faec faec added the cleanup label Apr 15, 2024
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Apr 15, 2024

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @faec? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

@faec faec mentioned this pull request Apr 15, 2024
Open
@faec faec marked this pull request as ready for review April 15, 2024 20:54
@faec faec requested a review from a team as a code owner April 15, 2024 20:54
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Apr 15, 2024

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Apr 15, 2024 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Duration: 137 min 24 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

zmoog
zmoog approved these changes Apr 23, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@zmoog zmoog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@faec
Copy link
Copy Markdown
Contributor Author

faec commented Apr 23, 2024

Thanks! I'm going to hold off on merging this til #39131 is in -- this one isn't a bug fix so it shouldn't be backported after feature freeze, and I don't want to have to fix the merge conflicts twice when backporting the real fix to 8.13.

@faec faec merged commit 2fa1123 into elastic:main Apr 30, 2024
@faec faec deleted the s3input-cleanup branch April 30, 2024 16:39
@faec faec mentioned this pull request Apr 30, 2024
Merged
6 tasks
faec added a commit that referenced this pull request May 3, 2024
@faec
Fix sqs region selection (#39327)
4e6d762 
Fix an error in region selection that was introduced in a previous cleanup (#38958). When the configured region disagrees with the region detected from the queue URL, the configured region is supposed to take precedence. Due to a misreading, my code instead chose the URL region when there is a conflict.

I've broken region selection out into another helper function to make this logic easier to test, and added several unit test cases that would have caught this mistake.
This was referenced May 24, 2024
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zmoog zmoog zmoog approved these changes

@kaiyan-sheng kaiyan-sheng Awaiting requested review from kaiyan-sheng

Assignees

@faec faec

Labels

cleanup Team:Elastic-Agent Label for the Agent team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@faec @elasticmachine @zmoog