Can one of the admins verify this patch?

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/23406/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/23408/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/23409/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/23589/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/23785/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/23786/
Test FAILed.

@raulchen, @zhijunfu , @edoakes
I definitely realize that the repeated if local_mode checks are not super maintainable, but it is an improvement over the existing separate python execution path. It is at least in the same location as the main logic.

I think that making more abstractions for CoreWorker is a great idea, but most likely the scope for future work.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/23799/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/23801/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/23802/
Test FAILed.

@zhijunfu @raulchen I agree that all of the if-else blocks aren't ideal, but it has a key benefit in that it minimizes the differences in the local mode and normal execution paths. The main issue with the existing Python implementation is that it's too abstracted from the normal path, so it ends up being very out of sync in behavior and users see issues when trying to go from local mode to normal mode (which is the whole point of supporting local mode). We could probably achieve this with the right abstractions but just completely subbing out a full local mode worker implementation is probably not the best idea.

@edoakes yeah, I agree that it's not a good idea to completely sub out local mode. I was suggesting that we should abstract the parts where local and cluster mode are different. Take task execution for example, we can have an abstract ExecuteTask method in the base class, and the local mode and cluster mode sub classes can implement this method in different ways. For the parts where local mode and cluster mode have the same logic, we should re-use the same code as much as possible.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/24064/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/24066/
Test FAILed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/24075/
Test PASSed.