-
Notifications
You must be signed in to change notification settings - Fork 7.4k
Closed
Labels
P3Issue moderate in impact or severityIssue moderate in impact or severitybugSomething that is supposed to be working; but isn'tSomething that is supposed to be working; but isn'tcoreIssues that should be addressed in Ray CoreIssues that should be addressed in Ray Corepending-cleanupThis issue is pending cleanup. It will be removed in 2 weeks after being assigned.This issue is pending cleanup. It will be removed in 2 weeks after being assigned.
Description
What is the problem?
object store thrashes when ray.get is called in a non-main thread because ray.get wouldn't unsubscribe the get dependency because the RPC to unsubscribe the get dependency is not called (NotifyDirectCallUnblock), resulting in that the give object is pulled / evicted until the get dependency is unsubscribed by the task termination.
Reproduction (REQUIRED)
Please provide a short code snippet (less than 50 lines if possible) that can be copy-pasted to reproduce the issue. The snippet should have no external library dependencies (i.e., use fake or mock data / environments):
@rkooo567 will prepare the simple reproduction script
- I have verified my script runs in a clean environment and reproduces the issue.
- I have verified the issue also occurs with the latest wheels.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
P3Issue moderate in impact or severityIssue moderate in impact or severitybugSomething that is supposed to be working; but isn'tSomething that is supposed to be working; but isn'tcoreIssues that should be addressed in Ray CoreIssues that should be addressed in Ray Corepending-cleanupThis issue is pending cleanup. It will be removed in 2 weeks after being assigned.This issue is pending cleanup. It will be removed in 2 weeks after being assigned.