Increase max compilation rate for Filebeat modules tests#9777
Merged
ruflin merged 3 commits intoelastic:masterfrom Dec 27, 2018
Merged
Increase max compilation rate for Filebeat modules tests#9777ruflin merged 3 commits intoelastic:masterfrom
ruflin merged 3 commits intoelastic:masterfrom
Conversation
Contributor
Author
|
@jakelandis A few questions about the above:
The pipeline for the above can be found here: https://github.com/elastic/beats/blob/master/filebeat/module/logstash/slowlog/ingest/pipeline-plain.json It does not contain an scripts but quite a few grok patterns. |
Contributor
Author
|
@jakelandis An interesting addition here. The docs state the limit is 15/min, but the error above says 75/5min which is similar but no the same ... |
ycombinator
reviewed
Dec 26, 2018
Contributor
There was a problem hiding this comment.
CI is failing with:
Too many dynamic script compilations within, max: [100/1m]
So looks like even 100/1m is not enough 😬😬😬
Contributor
Author
There was a problem hiding this comment.
I set it now to 1000/1m and at least locally it works. This is mainly to get test back to green. We should figure out about the why afterwards.
Module tests started to fail with the following message: ``` Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate] setting ``` The reason for this is (I think) that we started to use scripts more heavily in our pipelines so we hit this limit. For the tests we can easily increase the limit to solve this. In production this could still be an issue when all pipelines are loaded at the same time and we need to find a solution here. In parallel we should also investigate if there are some non parameterised scripts which get recompiled every time.
kaiyan-sheng
approved these changes
Dec 27, 2018
Contributor
kaiyan-sheng
left a comment
There was a problem hiding this comment.
Thanks for updating postgresql-9.6-new-timestamp.log-expected.json!
ycombinator
added a commit
that referenced
this pull request
Jan 29, 2019
Partial backport of ##9777. This PR ncreases the dynamic script compilations for Filebeat system tests to `1000/1m`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Module tests started to fail with the following message:
The reason for this is (I think) that we started to use scripts more heavily in our pipelines so we hit this limit. For the tests we can easily increase the limit to solve this. In production this could still be an issue when all pipelines are loaded at the same time and we need to find a solution here.
In parallel we should also investigate if there are some non parameterised scripts which get recompiled every time.