Skip to content

Implement parallel shard refresh behind cluster settings#17782

Merged
sachinpkale merged 4 commits intoopensearch-project:mainfrom
ashking94:shard-level-refresh
Apr 20, 2025
Merged

Implement parallel shard refresh behind cluster settings#17782
sachinpkale merged 4 commits intoopensearch-project:mainfrom
ashking94:shard-level-refresh

Conversation

@ashking94
Copy link
Copy Markdown
Member

@ashking94 ashking94 commented Apr 3, 2025 •
edited
Loading

Description

This PR implements parallel shard-level refresh capability that can be enabled/disabled via cluster settings. Key changes include:

  • Introduces new cluster setting cluster.index.refresh.shard_level.enabled (default: false) to control refresh behavior
  • When enabled, refresh tasks run at individual shard level instead of index level
  • Maintains backward compatibility by defaulting to index-level refresh when setting is disabled
  • Supports dynamic updates to refresh intervals and refresh modes

Implementation Details

  • Added new AsyncShardRefreshTask class to handle shard-level refresh operations
  • Modified IndexService and IndexShard to manage refresh task lifecycle
  • Added synchronization mechanisms to safely handle transitions between index and shard level refresh modes
  • Implemented proper cleanup of refresh tasks during mode transitions
  • Added comprehensive test coverage for the new functionality

Benefits

  • Enables more granular refresh control at shard level
  • Allows for better resource utilization through parallel refresh operations
  • Maintains existing refresh behavior by default for backward compatibility

The feature is marked as experimental with @ExperimentalApi annotation.

Related Issues

Meta issue: #17776

Check List

  • Functionality includes testing.
  • [ ] API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@ashking94 ashking94 force-pushed the shard-level-refresh branch from b5886d8 to d814114 Compare April 3, 2025 19:05
@ashking94 ashking94 mentioned this pull request Apr 3, 2025
Open
2 tasks
@ashking94 ashking94 force-pushed the shard-level-refresh branch from d814114 to 74c406f Compare April 3, 2025 19:09
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2025

❕ Gradle check result for 74c406f: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 3, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 81.05263% with 18 lines in your changes missing coverage. Please review.

Project coverage is 72.54%. Comparing base (cbaddd3) to head (11e7831).
Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
...in/java/org/opensearch/index/shard/IndexShard.java 68.96% 2 Missing and 7 partials ⚠️
...c/main/java/org/opensearch/index/IndexService.java 84.31% 1 Missing and 7 partials ⚠️
...in/java/org/opensearch/indices/IndicesService.java 92.85% 0 Missing and 1 partial ⚠️
Additional details and impacted files 
@@             Coverage Diff              @@
##               main   #17782      +/-   ##
============================================
+ Coverage     72.51%   72.54%   +0.02%     
- Complexity    67108    67128      +20     
============================================
  Files          5475     5478       +3     
  Lines        309916   310125     +209     
  Branches      45060    45080      +20     
============================================
+ Hits         224725   224966     +241     
+ Misses        66895    66820      -75     
- Partials      18296    18339      +43

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ashking94 ashking94 marked this pull request as ready for review April 4, 2025 04:04
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 15, 2025

❌ Gradle check result for 4349e25: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 15, 2025

❌ Gradle check result for 4349e25: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@opensearch-ci-bot opensearch-ci-bot mentioned this pull request Apr 15, 2025
Open
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2025

❌ Gradle check result for 4349e25: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2025

❌ Gradle check result for 4349e25: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@ashking94 ashking94 force-pushed the shard-level-refresh branch from 4349e25 to c428af3 Compare April 16, 2025 16:42
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 16, 2025

❌ Gradle check result for c428af3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@opensearch-ci-bot opensearch-ci-bot mentioned this pull request Apr 16, 2025
Open
@ashking94
Implement parallel shard refresh behind cluster settings
ae52998 
Signed-off-by: Ashish Singh <ssashish@amazon.com>
@ashking94
Incorporate PR review comments
92c3b1d 
Signed-off-by: Ashish Singh <ssashish@amazon.com>
@ashking94
Incorporate PR review comments
855d9bf 
Signed-off-by: Ashish Singh <ssashish@amazon.com>
@ashking94
Fix compilation failure
11e7831 
Signed-off-by: Ashish Singh <ssashish@amazon.com>
@ashking94 ashking94 force-pushed the shard-level-refresh branch from c428af3 to 11e7831 Compare April 19, 2025 09:00
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 19, 2025

❌ Gradle check result for 11e7831: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 19, 2025

❌ Gradle check result for 11e7831: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 19, 2025

✅ Gradle check result for 11e7831: SUCCESS

@sachinpkale sachinpkale merged commit 96481cc into opensearch-project:main Apr 20, 2025
31 checks passed
Bukhtawar
Bukhtawar reviewed Apr 22, 2025
View reviewed changes
server/src/main/java/org/opensearch/index/shard/IndexShard.java
Comment on lines +5568 to +5569
refreshTask.close();
refreshTask = null;
Copy link
Copy Markdown
Contributor

@Bukhtawar Bukhtawar Apr 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally the close should de-reference the refreshTask

x-INFiN1TY-x pushed a commit to x-INFiN1TY-x/OpenSearch_Local that referenced this pull request Apr 24, 2025
@ashking94 @tqranjan
Implement parallel shard refresh behind cluster settings (opensearch-…
fd22ca2 
…project#17782)

* Implement parallel shard refresh behind cluster settings

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Incorporate PR review comments

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Incorporate PR review comments

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Fix compilation failure

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Tanishq Ranjan <tqranjan@amazon.com>
Harsh-87 pushed a commit to Harsh-87/OpenSearch that referenced this pull request May 7, 2025
@ashking94
Implement parallel shard refresh behind cluster settings (opensearch-…
0e10208 
…project#17782)

* Implement parallel shard refresh behind cluster settings

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Incorporate PR review comments

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Incorporate PR review comments

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Fix compilation failure

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
Harsh-87 pushed a commit to Harsh-87/OpenSearch that referenced this pull request May 7, 2025
@ashking94
Implement parallel shard refresh behind cluster settings (opensearch-…
55f1a08 
…project#17782)

* Implement parallel shard refresh behind cluster settings

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Incorporate PR review comments

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Incorporate PR review comments

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Fix compilation failure

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
@opensearch-ci-bot opensearch-ci-bot mentioned this pull request May 23, 2025
Open
@BrewTestBot BrewTestBot mentioned this pull request Jun 24, 2025
Merged
@Jorgesnchz Jorgesnchz mentioned this pull request Jun 27, 2025
Closed
6 tasks
This was referenced Aug 9, 2025
Open
Open
@kkewwei kkewwei mentioned this pull request Nov 12, 2025
Open
This was referenced Jan 10, 2026
Merged
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Bukhtawar Bukhtawar Bukhtawar left review comments

@sachinpkale sachinpkale sachinpkale approved these changes

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@andrross andrross Awaiting requested review from andrross andrross is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@cwperks cwperks Awaiting requested review from cwperks cwperks is a code owner

@dblock dblock Awaiting requested review from dblock

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@jed326 jed326 Awaiting requested review from jed326 jed326 is a code owner

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@mch2 mch2 Awaiting requested review from mch2 mch2 is a code owner

@msfroh msfroh Awaiting requested review from msfroh msfroh is a code owner

@nknize nknize Awaiting requested review from nknize

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@reta reta Awaiting requested review from reta reta is a code owner

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@shwetathareja shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

@VachaShah VachaShah Awaiting requested review from VachaShah

@bugmakerrrrrr bugmakerrrrrr Awaiting requested review from bugmakerrrrrr

@jainankitk jainankitk Awaiting requested review from jainankitk

@linuxpi linuxpi Awaiting requested review from linuxpi

+1 more reviewer

@shourya035 shourya035 shourya035 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ashking94 @sachinpkale @Bukhtawar @shourya035