Skip to content

x-pack/filebeat/input/httpjson: allow elision of set and append failure logging#39929

Merged
efd6 merged 1 commit intoelastic:mainfrom
efd6:34544-httpjson
Jun 21, 2024
Merged

x-pack/filebeat/input/httpjson: allow elision of set and append failure logging#39929
efd6 merged 1 commit intoelastic:mainfrom
efd6:34544-httpjson

Conversation

@efd6
Copy link
Contributor

@efd6 efd6 commented Jun 18, 2024

Proposed commit message

See title.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

Author's Checklist

  • [ ]

How to test this PR locally

In the httpjson package run the following commands. The first should show a set of messages at DEBUG level and none at ERROR level. The second should show no output.

go test -run 'TestInput/.*_not_log_fail' -v | grep ignored

go test -run 'TestInput/.*[^_][^n][^o][^t][^_][^l][^o][^g][^_][^f][^a][^i][^l]$' -v | grep ignored # ¯\_(ツ)_/¯

Related issues

Use cases

Screenshots

Logs

@efd6 efd6 added enhancement Filebeat Filebeat backport-skip Skip notification from the automated backport with mergify Team:Security-Service Integrations Security Service Integrations Team labels Jun 18, 2024
@efd6 efd6 self-assigned this Jun 18, 2024
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 18, 2024
@efd6 efd6 force-pushed the 34544-httpjson branch from 708e505 to ff0e8ce Compare June 18, 2024 00:05
@efd6 efd6 marked this pull request as ready for review June 18, 2024 02:26
@efd6 efd6 requested a review from a team as a code owner June 18, 2024 02:26
@elasticmachine
Copy link
Contributor

elasticmachine commented Jun 18, 2024

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@chrisberkhout chrisberkhout self-requested a review June 19, 2024 09:29
chrisberkhout
chrisberkhout approved these changes Jun 19, 2024
View reviewed changes
Copy link
Contributor

@chrisberkhout chrisberkhout left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a fine improvement for the noisy logs problem.

At some point it would be good to stop requiring failure to terminate pagination. Perhaps there could be a pagination_until option:

response.pagination_until: '[[not (index .last_response.body.meta "cursor")]]'
response.pagination:
- set:
    target: url.params.cursor
    value: '[[.last_response.body.meta.cursor]]'

@efd6 efd6 requested a review from chrisberkhout June 20, 2024 22:10
@mergify
Copy link
Contributor

mergify bot commented Jun 20, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 34544-httpjson upstream/34544-httpjson
git merge upstream/main
git push upstream 34544-httpjson

@efd6 efd6 force-pushed the 34544-httpjson branch from ff0e8ce to 912cc26 Compare June 20, 2024 23:29
chrisberkhout
chrisberkhout approved these changes Jun 21, 2024
View reviewed changes
Copy link
Contributor

@chrisberkhout chrisberkhout left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@efd6 efd6 merged commit b0dcd95 into elastic:main Jun 21, 2024
@agmic agmic mentioned this pull request Jul 18, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chrisberkhout chrisberkhout chrisberkhout approved these changes

Assignees

@efd6 efd6

Labels

backport-skip Skip notification from the automated backport with mergify enhancement Filebeat Filebeat Team:Security-Service Integrations Security Service Integrations Team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Elastic Agent shouldn't log httpjson pagination completion as an Agent Error [httpjson] Improve handling of end-of-pagination

3 participants

@efd6 @elasticmachine @chrisberkhout