Fix race condition in Feature Migration Status API#80572
Merged
AthenaEryma merged 1 commit intoelastic:masterfrom Nov 10, 2021
Merged
Fix race condition in Feature Migration Status API#80572AthenaEryma merged 1 commit intoelastic:masterfrom
AthenaEryma merged 1 commit intoelastic:masterfrom
Conversation
Prior to this commit, there is a race condition in the Feature Migration Status API where the returned status can be `MIGRATION_NEEDED`, even if a migration is already in progress (and therefore the returned value should have been `IN_PROGRESS`). This commit adds a test for this case which reliably fails without the fix, and fixes the bug. The fix is straightforward: While we already examine the persistent task metadata to determine progress, the part of that metadata that we examined did was not updated until the task's been running for a bit. However, if we check for the *existence* of the task metadata, that is guaranteed to be in the cluster state by the time the request to start the migration completes (and is removed immediately after the task finishes - that's why we have separate metadata for the migration results instead of just using the task state).
Collaborator
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
williamrandolph
approved these changes
Nov 10, 2021
Contributor
williamrandolph
left a comment
There was a problem hiding this comment.
The fix makes sense, and the code LGTM! Thanks for figuring this out. I verified it on the command line too, using the setup I used previously to replicate the issue.
Contributor
Author
|
Thanks, both for the review and independently verifying that this fixes things! |
AthenaEryma
added a commit
to AthenaEryma/elasticsearch
that referenced
this pull request
Nov 10, 2021
Prior to this commit, there is a race condition in the Feature Migration Status API where the returned status can be `MIGRATION_NEEDED`, even if a migration is already in progress (and therefore the returned value should have been `IN_PROGRESS`). This commit adds a test for this case which reliably fails without the fix, and fixes the bug. The fix is straightforward: While we already examine the persistent task metadata to determine progress, the part of that metadata that we examined did was not updated until the task's been running for a bit. However, if we check for the *existence* of the task metadata, that is guaranteed to be in the cluster state by the time the request to start the migration completes (and is removed immediately after the task finishes - that's why we have separate metadata for the migration results instead of just using the task state).
AthenaEryma
added a commit
to AthenaEryma/elasticsearch
that referenced
this pull request
Nov 10, 2021
Prior to this commit, there is a race condition in the Feature Migration Status API where the returned status can be `MIGRATION_NEEDED`, even if a migration is already in progress (and therefore the returned value should have been `IN_PROGRESS`). This commit adds a test for this case which reliably fails without the fix, and fixes the bug. The fix is straightforward: While we already examine the persistent task metadata to determine progress, the part of that metadata that we examined did was not updated until the task's been running for a bit. However, if we check for the *existence* of the task metadata, that is guaranteed to be in the cluster state by the time the request to start the migration completes (and is removed immediately after the task finishes - that's why we have separate metadata for the migration results instead of just using the task state).
elasticsearchmachine
pushed a commit
that referenced
this pull request
Nov 10, 2021
Prior to this commit, there is a race condition in the Feature Migration Status API where the returned status can be `MIGRATION_NEEDED`, even if a migration is already in progress (and therefore the returned value should have been `IN_PROGRESS`). This commit adds a test for this case which reliably fails without the fix, and fixes the bug. The fix is straightforward: While we already examine the persistent task metadata to determine progress, the part of that metadata that we examined did was not updated until the task's been running for a bit. However, if we check for the *existence* of the task metadata, that is guaranteed to be in the cluster state by the time the request to start the migration completes (and is removed immediately after the task finishes - that's why we have separate metadata for the migration results instead of just using the task state).
elasticsearchmachine
pushed a commit
that referenced
this pull request
Nov 10, 2021
Prior to this commit, there is a race condition in the Feature Migration Status API where the returned status can be `MIGRATION_NEEDED`, even if a migration is already in progress (and therefore the returned value should have been `IN_PROGRESS`). This commit adds a test for this case which reliably fails without the fix, and fixes the bug. The fix is straightforward: While we already examine the persistent task metadata to determine progress, the part of that metadata that we examined did was not updated until the task's been running for a bit. However, if we check for the *existence* of the task metadata, that is guaranteed to be in the cluster state by the time the request to start the migration completes (and is removed immediately after the task finishes - that's why we have separate metadata for the migration results instead of just using the task state).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prior to this commit, there is a race condition in the Feature Migration
Status API where the returned status can be
MIGRATION_NEEDED, even ifa migration is already in progress (and therefore the returned value
should have been
IN_PROGRESS). This commit adds a test for this casewhich reliably fails without the fix, and fixes the bug.
The fix is straightforward: While we already examine the persistent task
metadata to determine progress, the part of that metadata that we
examined did was not updated until the task's been running for a bit.
However, if we check for the existence of the task metadata, that is
guaranteed to be in the cluster state by the time the request to start the
migration completes (and is removed immediately after the task finishes
of just using the task state).
Fixes #79680