Skip to content

WIP - Make recovery source async#37174

Closed
dnhatn wants to merge 22 commits intoelastic:masterfrom
dnhatn:async-recovery
Closed

WIP - Make recovery source async#37174
dnhatn wants to merge 22 commits intoelastic:masterfrom
dnhatn:async-recovery

Conversation

@dnhatn
Copy link
Copy Markdown
Member

@dnhatn dnhatn commented Jan 7, 2019

I made this draft, so we have something to evaluate the async option. Since this PR is on top of #32512, please check out and review it locally or have a look at the last commit. If we agree to proceed, I will break down this into smaller PRs.

@dnhatn dnhatn added WIP :Distributed/Recovery Anything around constructing a new shard, either from a local or a remote source. v7.0.0 team-discuss labels Jan 7, 2019
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-distributed

Copy link
Copy Markdown
Member

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a nice approach. As in #37076 I'm unclear about whether we still need CancellableThreads here, but that doesn't need resolving right now.

@s1monw
Copy link
Copy Markdown
Contributor

s1monw commented Jan 7, 2019

I think we should start step by step making things async. I don't necessarily understand why we depend this on the work on CompleteableFuture. I don't think we should use futures unless absolutely necessary and should use callbacks / listeners instead, consistently across the codebase. There are many things that I personally dislike about futures, the main issue here is that you need to be explicit on the return value and it's not obvious that the method is async. I would really like to prevent this and rather get rid of futures where we can.

Copy link
Copy Markdown
Contributor

@s1monw s1monw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see my comment above

@dnhatn
Copy link
Copy Markdown
Member Author

dnhatn commented Jan 7, 2019

I think we should start step by step making things async.

Yes, that's my plan if we agree to make recovery async.

I don't necessarily understand why we depend this on the work on CompleteableFuture. I don't think we should use futures unless absolutely necessary and should use callbacks / listeners instead, consistently across the codebase.

Sure, I will use listeners for this. With CompleteableFuture, the flow of the async code is somehow similar to the sequential flow. That's the only reason I prefer CompleteableFuture over callback.

dnhatn added a commit that referenced this pull request Jan 8, 2019
Today we create a global instance of RecoveryResponse then mutate it
when executing each recovery step. This is okay for the current
sequential recovery flow but  not suitable for an asynchronous recovery
which we are targeting. With this commit, we return the result of each
step separately, then construct a RecoveryResponse at the end.

Relates #37174
dnhatn added a commit that referenced this pull request Jan 9, 2019
Today we create a global instance of RecoveryResponse then mutate it
when executing each recovery step. This is okay for the current
sequential recovery flow but  not suitable for an asynchronous recovery
which we are targeting. With this commit, we return the result of each
step separately, then construct a RecoveryResponse at the end.

Relates #37174
@dnhatn dnhatn closed this Jan 10, 2019
@dnhatn dnhatn deleted the async-recovery branch January 10, 2019 09:14
dnhatn added a commit to dnhatn/elasticsearch that referenced this pull request Jan 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Distributed/Recovery Anything around constructing a new shard, either from a local or a remote source. WIP

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants