-
Notifications
You must be signed in to change notification settings - Fork 7.4k
[Serve] Deadlock when awaiting DeploymentResponse #54201
Copy link
Copy link
Closed
Labels
P1Issue that should be fixed within a few weeksIssue that should be fixed within a few weeksbugSomething that is supposed to be working; but isn'tSomething that is supposed to be working; but isn'tserveRay Serve Related IssueRay Serve Related Issuestability
Description
What happened + What you expected to happen
The following code will hang indefinitely
from ray import serve
import asyncio
@serve.deployment
class DeploymentA:
async def __call__(self, number: int) -> int:
await asyncio.sleep(1)
return number * 2
@serve.deployment
class DeploymentB:
async def __call__(self, number: int) -> int:
await asyncio.sleep(1)
return number * 3
@serve.deployment
class DeploymentC:
async def __call__(self, number: int) -> int:
await asyncio.sleep(1)
return number * 4
@serve.deployment
class ComposedDeployment:
def __init__(self, deployment_a, deployment_b, deployment_c):
self.deployment_a = deployment_a
self.deployment_b = deployment_b
self.deployment_c = deployment_c
async def __call__(self, number: int) -> int:
a = self.deployment_a.remote(number)
b = self.deployment_b.remote(a)
c = self.deployment_c.remote(b)
true_b = await b
true_c = await c
return true_b * true_c
composed_deployment = ComposedDeployment.bind(
DeploymentA.bind(), DeploymentB.bind(), DeploymentC.bind()
)
handle = serve.run(composed_deployment)
print("Deployment done")
response = handle.remote(5)
print(response.result())Changing the order of the await will solve the problem (await c before await b)
async def __call__(self, number: int) -> int:
a = self.deployment_a.remote(number)
b = self.deployment_b.remote(a)
c = self.deployment_c.remote(b)
true_c = await c
true_b = await b
return true_b * true_cVersions / Dependencies
2.47.1
Reproduction script
Run the above code
Issue Severity
Medium: It is a significant difficulty but I can work around it.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
P1Issue that should be fixed within a few weeksIssue that should be fixed within a few weeksbugSomething that is supposed to be working; but isn'tSomething that is supposed to be working; but isn'tserveRay Serve Related IssueRay Serve Related Issuestability