Skip to content

[Segment Replication] Update PrimaryShardAllocator to prefer replicas with higher replication checkpoint (#4041)#4252

Merged
dreamer-89 merged 1 commit intoopensearch-project:2.xfrom
dreamer-89:segrep_failover_primary_selection_2x
Aug 18, 2022
Merged

[Segment Replication] Update PrimaryShardAllocator to prefer replicas with higher replication checkpoint (#4041)#4252
dreamer-89 merged 1 commit intoopensearch-project:2.xfrom
dreamer-89:segrep_failover_primary_selection_2x

Conversation

@dreamer-89
Copy link
Copy Markdown
Member

Backport of d308a29

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

… with higher replication checkpoint (opensearch-project#4041)

* [Segment Replication] Update PrimaryShardAllocator to prefer replicas having higher replication checkpoint

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Use empty replication checkpoint to avoid NPE

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Update NodeGatewayStartedShards to optionally wire in/out ReplicationCheckpoint field

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Use default replication checkpoint causing EOF errors on empty checkpoint

* Add indexSettings to GatewayAllocator to allow ReplicationCheckpoint comparator only for segrep enabled indices

* Add unit tests for primary term first replica promotion & comparator fix

* Fix NPE on empty IndexMetadata

* Remove settings from AllocationService and directly inject in GatewayAllocator

* Add more unit tests and minor code clean up

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Address review comments & integration test

Signed-off-by: Suraj Singh <surajrider@gmail.com>

* Fix comparator on null ReplicationCheckpoint

Signed-off-by: Suraj Singh <surajrider@gmail.com>

Signed-off-by: Suraj Singh <surajrider@gmail.com>
@dreamer-89 dreamer-89 requested review from a team and reta as code owners August 18, 2022 00:37
@dreamer-89 dreamer-89 requested a review from mch2 August 18, 2022 00:37
@github-actions
Copy link
Copy Markdown
Contributor

Gradle Check (Jenkins) Run Completed with:

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

Merging #4252 (11c9417) into 2.x (8c2f50f) will decrease coverage by 0.05%.
The diff coverage is 57.14%.

@@             Coverage Diff              @@
##                2.x    #4252      +/-   ##
============================================
- Coverage     70.63%   70.57%   -0.06%     
+ Complexity    56943    56883      -60     
============================================
  Files          4570     4570              
  Lines        273499   273527      +28     
  Branches      40103    40110       +7     
============================================
- Hits         193180   193037     -143     
- Misses        64206    64315     +109     
- Partials      16113    16175      +62     
Impacted Files Coverage Δ
...ateway/TransportNodesListGatewayStartedShards.java 37.58% <38.09%> (-1.06%) ⬇️
.../opensearch/test/gateway/TestGatewayAllocator.java 94.00% <71.42%> (-3.73%) ⬇️
.../org/opensearch/gateway/PrimaryShardAllocator.java 79.20% <100.00%> (+0.42%) ⬆️
.../replication/checkpoint/ReplicationCheckpoint.java 56.81% <100.00%> (+3.32%) ⬆️
...java/org/opensearch/client/indices/DataStream.java 0.00% <0.00%> (-76.09%) ⬇️
.../opensearch/client/indices/CloseIndexResponse.java 17.50% <0.00%> (-51.25%) ⬇️
...r/src/main/java/org/opensearch/http/HttpUtils.java 16.66% <0.00%> (-50.00%) ⬇️
.../java/org/opensearch/node/NodeClosedException.java 50.00% <0.00%> (-50.00%) ⬇️
...ch/transport/ReceiveTimeoutTransportException.java 50.00% <0.00%> (-50.00%) ⬇️
.../index/shard/IndexShardNotRecoveringException.java 0.00% <0.00%> (-50.00%) ⬇️
... and 484 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@dreamer-89 dreamer-89 merged commit 9df9ab3 into opensearch-project:2.x Aug 18, 2022
@dreamer-89 dreamer-89 deleted the segrep_failover_primary_selection_2x branch August 18, 2022 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants