Inject Unfollow before Rollover and Shrink#37625
Merged
AthenaEryma merged 13 commits intoelastic:masterfrom Jan 28, 2019
Merged
Inject Unfollow before Rollover and Shrink#37625AthenaEryma merged 13 commits intoelastic:masterfrom
AthenaEryma merged 13 commits intoelastic:masterfrom
Conversation
We inject an Unfollow action before Shrink because the Shrink action cannot be safely used on a following index, as it may not be fully caught up with the leader index before the "original" following index is deleted and replaced with a non-following Shrunken index. The Unfollow action will verify that 1) the index is marked as "complete", and 2) all operations up to this point have been replicated from the leader to the follower before explicitly disconnecting the follower from the leader. Injecting an Unfollow action before the Rollover action is done mainly as a convenience: This allow users to use the same lifecycle policy on both the leader and follower cluster without having to explictly modify the policy to unfollow the index, while doing what we expect users to want in most cases.
Collaborator
|
Pinging @elastic/es-core-features |
3 tasks
dakrone
approved these changes
Jan 22, 2019
.../core/src/main/java/org/elasticsearch/xpack/core/indexlifecycle/TimeseriesLifecycleType.java
Show resolved
Hide resolved
Contributor
Author
|
This has docs now, which should be reviewed as well. I had to fix a couple HLRC tests (21a2548 and 7a42638), which I may not have done in the best way. I ended up setting the poll interval in the client tests, as the tests which assume ILM starts waiting for Rollover immediately break when there's an injected Unfollow action. Without the fast poll interval set, it will often, but not always, wait on (@talevy the above is the thing I mentioned on slack) |
dakrone
approved these changes
Jan 23, 2019
Member
dakrone
left a comment
There was a problem hiding this comment.
LGTM, left one super minor comment about a typo
Contributor
Author
|
@elasticmachine run elasticsearch-ci/default-distro |
Contributor
Author
|
Looks like CI choked. @elasticmachine run elasticsearch-ci/1 |
AthenaEryma
added a commit
to AthenaEryma/elasticsearch
that referenced
this pull request
Jan 29, 2019
We inject an Unfollow action before Shrink because the Shrink action cannot be safely used on a following index, as it may not be fully caught up with the leader index before the "original" following index is deleted and replaced with a non-following Shrunken index. The Unfollow action will verify that 1) the index is marked as "complete", and 2) all operations up to this point have been replicated from the leader to the follower before explicitly disconnecting the follower from the leader. Injecting an Unfollow action before the Rollover action is done mainly as a convenience: This allow users to use the same lifecycle policy on both the leader and follower cluster without having to explictly modify the policy to unfollow the index, while doing what we expect users to want in most cases.
AthenaEryma
added a commit
that referenced
this pull request
Jan 30, 2019
We inject an Unfollow action before Shrink because the Shrink action cannot be safely used on a following index, as it may not be fully caught up with the leader index before the "original" following index is deleted and replaced with a non-following Shrunken index. The Unfollow action will verify that 1) the index is marked as "complete", and 2) all operations up to this point have been replicated from the leader to the follower before explicitly disconnecting the follower from the leader. Injecting an Unfollow action before the Rollover action is done mainly as a convenience: This allow users to use the same lifecycle policy on both the leader and follower cluster without having to explictly modify the policy to unfollow the index, while doing what we expect users to want in most cases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We inject an Unfollow action before Shrink because the Shrink action
cannot be safely used on a following index, as it may not be fully
caught up with the leader index before the "original" following index is
deleted and replaced with a non-following Shrunken index. The Unfollow
action will verify that 1) the index is marked as "complete", and 2) all
operations up to this point have been replicated from the leader to the
follower before explicitly disconnecting the follower from the leader.
Injecting an Unfollow action before the Rollover action is done mainly
as a convenience: This allow users to use the same lifecycle policy on
both the leader and follower cluster without having to explictly modify
the policy to unfollow the index, while doing what we expect users to
want in most cases.
Relates to #34648
Follow-on to #36970
This is marked as WIP because it still needs docs, but the code and tests are ready for review.