PoolActor now uses num_cpus=0 to avoid any deadlock#22048
Merged
pcmoritz merged 6 commits intoray-project:masterfrom Feb 8, 2022
Merged
PoolActor now uses num_cpus=0 to avoid any deadlock#22048pcmoritz merged 6 commits intoray-project:masterfrom
pcmoritz merged 6 commits intoray-project:masterfrom
Conversation
Contributor
Author
wuisawesome
requested changes
Feb 2, 2022
Contributor
wuisawesome
left a comment
There was a problem hiding this comment.
Can we add the repro from #21488 as a test case?
wuisawesome
reviewed
Feb 3, 2022
| with Pool(ray_address="auto") as pool: | ||
| return list(pool.map(poolit_a, range(2, 4, 1))) | ||
|
|
||
| ray.init(num_cpus=1) |
Contributor
There was a problem hiding this comment.
nit: can we initialize ray with a fixture if possible? Ideally we can use ray_start_shared_local_modes?
wuisawesome
approved these changes
Feb 3, 2022
simonsays1980
pushed a commit
to simonsays1980/ray
that referenced
this pull request
Feb 27, 2022
ray-project#21488 (comment) : > We discussed this issue in a bit more detail and came to the conclusion that we should set the CPU resource requirement for each actor in the actor pool to 0, to make the Ray Pool compatible/same behavior as the Python multiprocessing pool. Would that work for you @yogeveran ? (very similar to solution 4 mentioned above, but with 0.0 instead of 0.1, so it works in all cases).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
#21488 (comment) :
Related issue number
Closes #21488
Checks
scripts/format.shto lint the changes in this PR.