Remove unnecessary iteration in request cache clear#19263
Remove unnecessary iteration in request cache clear#19263jainankitk merged 23 commits intoopensearch-project:mainfrom
Conversation
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
|
❌ Gradle check result for f650cb4: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #19263 +/- ##
============================================
+ Coverage 72.88% 72.89% +0.01%
- Complexity 70219 70246 +27
============================================
Files 5702 5702
Lines 322202 322216 +14
Branches 46625 46627 +2
============================================
+ Hits 234832 234892 +60
+ Misses 68418 68354 -64
- Partials 18952 18970 +18 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Codecov is saying the methods for cache clear are not covered, this is not true as they are covered in the ITs (confirmed with IntelliJ debugger) |
Signed-off-by: Peter Alfonsi <peter.alfonsi@gmail.com>
|
❕ Gradle check result for 7d3cd10: UNSTABLE Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
...n/java/org/opensearch/action/admin/indices/cache/clear/TransportClearIndicesCacheAction.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Peter Alfonsi <peter.alfonsi@gmail.com>
|
❌ Gradle check result for 37486b8: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
|
❌ Gradle check result for fa15ef0: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
|
❌ Gradle check result for 5e334a9: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
#17678 again |
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
|
❌ Gradle check result for d3081ee: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
...c/main/java/org/opensearch/action/support/broadcast/node/TransportBroadcastByNodeAction.java
Show resolved
Hide resolved
|
❌ Gradle check result for 10aa089: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
016b1ee to
668b8ea
Compare
|
❌ Gradle check result for 668b8ea: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for fb5fe89: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
...n/java/org/opensearch/action/admin/indices/cache/clear/TransportClearIndicesCacheAction.java
Outdated
Show resolved
Hide resolved
...n/java/org/opensearch/action/admin/indices/cache/clear/TransportClearIndicesCacheAction.java
Outdated
Show resolved
Hide resolved
|
❌ Gradle check result for 598cdf1: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
598cdf1 to
10aa089
Compare
Signed-off-by: Peter Alfonsi <peter.alfonsi@gmail.com>
|
❌ Gradle check result for b43d86b: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Ankit Jain <jainankitk@apache.org>
|
❌ Gradle check result for 288fe1c: null Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
|
❕ Gradle check result for 350f34e: UNSTABLE Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
…ct#19263) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: Peter Alfonsi <peter.alfonsi@gmail.com> Signed-off-by: Ankit Jain <jainankitk@apache.org> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Ankit Jain <jainankitk@apache.org>
Description
Adds a node-level hook to TransportBroadcastByNodeAction which runs after all shard-level operations on that node have finished.
Also reworks TransportClearIndicesCacheAction to use this new hook, to prevent needlessly iterating through all cache keys once per shard, which caused high cache clear latency especially if the disk cache is used. I bundled the two together into one PR since both changes are pretty small.
I got cache clear numbers from cleaning up 2M request cache keys in a disk tier, spread across 10 indices (1 shard each), on a single node hosted on a c5.2xl instance.
Related Issues
Resolves:
#19183
#19118
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.