Skip to content

Cherry-pick #19552 to 6.8: Add max bytes in line limit#20087

Merged
kvch merged 1 commit intoelastic:6.8from
kvch:backport_19552_6.8
Jul 23, 2020
Merged

Cherry-pick #19552 to 6.8: Add max bytes in line limit#20087
kvch merged 1 commit intoelastic:6.8from
kvch:backport_19552_6.8

Conversation

@kvch
Copy link
Copy Markdown
Contributor

@kvch kvch commented Jul 21, 2020

Cherry-pick of PR #19552 to 6.8 branch. Original message:

What does this PR do?

Handles OOM on long lines gracefully, writes the error into log.

Why is it important?

Handle bad data gracefully without OOM.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have added tests that prove my fix is effective or that my feature works

Author's Checklist

  • I just recently started looking at the beats code so might not have enough knowledge of any possible side-effects of this change. Please review with that in mind.

How to test this PR locally

Refer to #19500 for test case.
The filebeat doesn't get killed due to OOM with this change and the error is logged (see the screenshot attached).

Related issues

Screenshots

Screen Shot 2020-07-01 at 9 16 06 AM

Logs

2020-07-01T13:15:17.630Z	INFO	[crawler]	beater/crawler.go:108	Loading and starting Inputs completed. Enabled inputs: 1
2020-07-01T13:15:17.633Z	INFO	log/harvester.go:297	Harvester started for file: /test/readme.log
2020-07-01T13:15:17.635Z	ERROR	log/harvester.go:330	Read line error: exceeded max bytes in line limit; File: /test/readme.log

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 21, 2020
@kvch kvch force-pushed the backport_19552_6.8 branch from 2dfdfe9 to 96df3d4 Compare July 21, 2020 10:34
@kvch kvch added [zube]: In Review Team:Services (Deprecated) Label for the former Integrations-Services team and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jul 21, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/integrations-services (Team:Services)

@urso
Copy link
Copy Markdown

urso commented Jul 21, 2020

Please check the lint output. Looks like Filebeat does not compile with the backport.

@kvch kvch force-pushed the backport_19552_6.8 branch from 96df3d4 to 25b3ebf Compare July 22, 2020 15:58
@kvch
Copy link
Copy Markdown
Contributor Author

kvch commented Jul 23, 2020

Failing tests are unrelated.

@kvch kvch added the v6.8.12 label Jul 23, 2020
@kvch kvch merged commit f431ae6 into elastic:6.8 Jul 23, 2020
@zube zube bot removed the [zube]: Done label Oct 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport review Team:Services (Deprecated) Label for the former Integrations-Services team v6.8.12

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants