Skip to content

Cache Expensive Index Stats in TransportClusterStatsAction#63943

Merged
original-brownbear merged 3 commits intoelastic:masterfrom
original-brownbear:make-monitoring-less-crazy-expensive
Oct 27, 2020
Merged

Cache Expensive Index Stats in TransportClusterStatsAction#63943
original-brownbear merged 3 commits intoelastic:masterfrom
original-brownbear:make-monitoring-less-crazy-expensive

Conversation

@original-brownbear
Copy link
Copy Markdown
Contributor

Follow up to #62753, which moved this expensive method off of the transport threads.
Often times monitoring will hit this endpoint every few seconds while the metadata
will likely change at a much slower interval.
Given that in practice the computation of the stats might take up to a minute for large
cluster states, caching them on a best-effort basis seems like a reasonable improvement.

Follow up to #62753, which moved this expensive method off of the transport threads.
Often times monitoring will hit this endpoint every few seconds while the metadata
will likely change at a much slower interval.
Given that in practice the computation of the stats might take up to a minute for large
cluster states, caching them on a best-effort basis seems like a reasonable improvement.
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Monitoring)

@elasticmachine elasticmachine added the Team:Data Management (obsolete) DO NOT USE. This team no longer exists. label Oct 20, 2020
// guards #mappingStats, #analysisStats and #metaVersion
private final Object statsMutex = new Object();

private MappingStats mappingStats;
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously this isn't sticky but in practice this will mostly be called on master by monitoring and the overhead of this is small. And even if this is less helpful for the stats REST API it's going to often help anyway statistically speaking I'd expect it to "often" help still.


private MappingStats mappingStats;
private AnalysisStats analysisStats;
private long metaVersion = -1L;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm absolutely out of practice here, so please do correct me if I'm wrong, but should metaVersion be volatile?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for that. We only ever touch that field under the statsMutex so we are guaranteed to always have a consistent view of it.

@danhermann danhermann self-requested a review October 22, 2020 15:02
Comment on lines +93 to +94
MappingStats mappingStats = null;
AnalysisStats analysisStats = null;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a drive-by comment (I'll leave the full review to Dan), but can you rename these variables? I think shadowing class variables (outside of constructors) can lead to bugs for people who reference it not realizing which one they're getting

Copy link
Copy Markdown
Contributor

@danhermann danhermann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @original-brownbear! This LGTM once @dakrone's comment has been addressed.

@original-brownbear
Copy link
Copy Markdown
Contributor Author

Jenkins run elasticsearch-ci/1 (some unrelated reindex thingy)

@original-brownbear
Copy link
Copy Markdown
Contributor Author

Thanks everyone!

@original-brownbear original-brownbear merged commit 0d39cb5 into elastic:master Oct 27, 2020
@original-brownbear original-brownbear deleted the make-monitoring-less-crazy-expensive branch October 27, 2020 09:28
original-brownbear added a commit that referenced this pull request Oct 27, 2020
…64195)

Follow up to #62753, which moved this expensive method off of the transport threads.
Often times monitoring will hit this endpoint every few seconds while the metadata
will likely change at a much slower interval.
Given that in practice the computation of the stats might take up to a minute for large
cluster states, caching them on a best-effort basis seems like a reasonable improvement.
@original-brownbear original-brownbear restored the make-monitoring-less-crazy-expensive branch December 6, 2020 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants