[Core] support out of order execution for threaded/async actors #19822
Description
The following will timeout
import ray
ray.init()
@ray.remote
class A:
def echo(self, inp):
print(inp)
return inp
@ray.remote
def never_return():
import time
time.sleep(10000)
inp_ref_1 = never_return.remote()
inp_ref_2 = ray.put(2)
a = A.options(max_concurrency=2).remote()
out_ref_1 = a.echo.remote(inp_ref_1)
out_ref_2 = a.echo.remote(inp_ref_2)
assert ray.get(out_ref_2, timeout=5) == 2 # currently timeout!I went all the way back to Ray 0.8.0 and it is the same behavior, so this is not a regression. For concurrent actors and async actors, however, I think the snippet should work and we should allow later tasks to execute first when their dependencies ready. This is a semantics change.
Context: user bug report for Ray Serve https://discuss.ray.io/t/concurrent-queries-blocking-following-queries/3949/2