Only request wildcard expansion for hidden indices if supported#20938
Merged
ycombinator merged 9 commits intoelastic:masterfrom Sep 3, 2020
ycombinator:indices-stats-hidden-bc
Merged
Only request wildcard expansion for hidden indices if supported#20938ycombinator merged 9 commits intoelastic:masterfrom ycombinator:indices-stats-hidden-bc
ycombinator merged 9 commits intoelastic:masterfrom
ycombinator:indices-stats-hidden-bc
Conversation
Contributor
|
Pinging @elastic/integrations-services (Team:Services) |
Contributor
|
Pinging @elastic/stack-monitoring (Stack monitoring) |
Contributor
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
|
jsoriano
approved these changes
Sep 2, 2020
Member
jsoriano
left a comment
There was a problem hiding this comment.
LGTM, comments are not blockers.
| var BulkStatsAvailableVersion = common.MustNewVersion("8.0.0") | ||
| //ExpandWildcardsHiddenAvailableVersion is the version since when the "expand_wildcards" query parameter to | ||
| // the Indices Stats API can accept "hidden" as a value. | ||
| ExpandWildcardsHiddenAvailableVersion = common.MustNewVersion("7.9.0") |
Member
There was a problem hiding this comment.
In #18703 an integration test was added and it was being executed against Elasticsearch 7.7.0, why did this test didn't fail?
Should we add some other test for the expand wildcards case?
Contributor
Author
There was a problem hiding this comment.
Comment on lines
+133
to
+134
| ew := u.Query().Get(expandWildcardsParam) | ||
| if !strings.HasSuffix(ew, hiddenSuffix) { |
Member
There was a problem hiding this comment.
The path comes from a constant, do we need to do these checks to see if some suffix is already present?
Contributor
Author
There was a problem hiding this comment.
Yeah, I guess you're right. I was thinking of a different problem (#16044) but that's not applicable here. I will remove the checks.
Contributor
Author
|
@jsoriano Thanks for the excellent review. I have addressed all your feedback. Please re-review when you get a chance. Thanks! |
jsoriano
approved these changes
Sep 3, 2020
Member
|
Failing test seems related. |
v1v
added a commit
to v1v/beats
that referenced
this pull request
Sep 3, 2020
…ne-2.0 * upstream/master: [Metricbeat][test] Disable ec2 flaky test (elastic#20959) Check if tracer is active before starting a transaction (elastic#20852) [Elastic Agent] Add support for variable replacement from providers (elastic#20839) Only request wildcard expansion for hidden indices if supported (elastic#20938) [Ingest Manager] New agent structure (symlinks) (elastic#20400) [Ingest Manager] Print a message confirming shutdown (elastic#20948) Skip flaky test on unix input (elastic#20942) [Ingest Manager] Align introspect-inspect naming in code (elastic#20952) [Filebeat][zeek] Map new x509 fields for ssl module (elastic#20927) [CI] fix regression with variable name (elastic#20930) [Autodiscover] Handle input-not-finished errors in config reload (elastic#20915) [Ingest Manager] Remove Success from fleet contract (elastic#20449)
v1v
added a commit
to v1v/beats
that referenced
this pull request
Sep 3, 2020
…-faster * upstream/master: [Metricbeat][test] Disable ec2 flaky test (elastic#20959) Check if tracer is active before starting a transaction (elastic#20852) [Elastic Agent] Add support for variable replacement from providers (elastic#20839) Only request wildcard expansion for hidden indices if supported (elastic#20938) [Ingest Manager] New agent structure (symlinks) (elastic#20400) [Ingest Manager] Print a message confirming shutdown (elastic#20948) Skip flaky test on unix input (elastic#20942) [Ingest Manager] Align introspect-inspect naming in code (elastic#20952) [Filebeat][zeek] Map new x509 fields for ssl module (elastic#20927)
ycombinator
added a commit
that referenced
this pull request
Sep 8, 2020
…s if supported (#20938) (#20962) * Only request wildcard expansion for hidden indices if supported (#20938) * Refactoring: inverting logic to make room for another case * Expand hidden indices wildcards if monitored ES supports that option * Adding CHANGELOG entry * Fixing formatting * Avoid unnecessary setting * Removing unnecessary suffix existence checks * Fixing feature version * Add test cases to unit test * Updating test # Conflicts: # metricbeat/module/elasticsearch/index/index_test.go * Fixing CHANGELOG
ycombinator
added a commit
that referenced
this pull request
Sep 8, 2020
…) (#20963) * Refactoring: inverting logic to make room for another case * Expand hidden indices wildcards if monitored ES supports that option * Adding CHANGELOG entry * Fixing formatting * Avoid unnecessary setting * Removing unnecessary suffix existence checks * Fixing feature version * Add test cases to unit test * Updating test # Conflicts: # metricbeat/module/elasticsearch/index/index_test.go
melchiormoulin
pushed a commit
to melchiormoulin/beats
that referenced
this pull request
Oct 14, 2020
…tic#20938) * Refactoring: inverting logic to make room for another case * Expand hidden indices wildcards if monitored ES supports that option * Adding CHANGELOG entry * Fixing formatting * Avoid unnecessary setting * Removing unnecessary suffix existence checks * Fixing feature version * Add test cases to unit test * Updating test
leweafan
pushed a commit
to leweafan/beats
that referenced
this pull request
Apr 28, 2023
…s if supported (elastic#20938) (elastic#20962) * Only request wildcard expansion for hidden indices if supported (elastic#20938) * Refactoring: inverting logic to make room for another case * Expand hidden indices wildcards if monitored ES supports that option * Adding CHANGELOG entry * Fixing formatting * Avoid unnecessary setting * Removing unnecessary suffix existence checks * Fixing feature version * Add test cases to unit test * Updating test # Conflicts: # metricbeat/module/elasticsearch/index/index_test.go * Fixing CHANGELOG
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR teaches the
elasticsearch/indexmetricset to check the version of the Elasticsearch cluster it's monitoring and, depending on that, ask the cluster to expand wildcards for hidden indices or not.Why is it important?
In #18703 we enhanced the
elasticsearch/indexmetricset to ask the monitored Elasticsearch cluster to expand wildcards for hidden indices. This enhancement was released in Metricbeat 7.9.0.Unfortunately this enhancement a backwards-incompatible change, as wildcard expansion for hidden indices is only supported starting Elasticsearch 7.9.0. So if a Metricbeat >= 7.90 tries to monitor an Elasticsearch cluster < 7.9.0, the
elasticsearch/indexmetricset is not able to collect metrics and throws the following error:Checklist
I have made corresponding changes to the documentationI have made corresponding change to the default configuration filesCHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.