-
Notifications
You must be signed in to change notification settings - Fork 11.1k
Open
Description
What version of gRPC and what language are you using?
grpc 1.51.1
What operating system (Linux, Windows,...) and version?
macOS Catalina 10.15
What runtime / compiler are you using (e.g. python version or version of gcc)
python 3.8.15
What did you do?
Please provide either 1) A unit test for reproducing the bug or 2) Specific steps for us to follow to reproduce the bug. If there’s not enough information to debug the problem, gRPC team may close the issue at their discretion. You’re welcome to re-open the issue once you have a reproduction.
- With a ray installation 2.1.0
pip install ray==2.1.0
- Running a simple workload that starts a ray cluster and launches tasks to 2 threads.
import ray
def run():
ray.init(num_cpus=2)
@ray.remote
def g():
pass
ray.get(g.remote())
ray.get(g.remote())
ray.shutdown()
for _ in range(10000):
run()
What did you expect to see?
Script runs ok.
What did you see instead?
Script hanging with
E1213 01:47:43.538549000 4544900544 thread_pool.cc:253] Waiting for thread pool to idle before forking
E1213 01:47:46.545305000 4544900544 thread_pool.cc:253] Waiting for thread pool to idle before forking
E1213 01:47:49.551290000 4544900544 thread_pool.cc:253] Waiting for thread pool to idle before forking
E1213 01:47:52.554189000 4544900544 thread_pool.cc:253] Waiting for thread pool to idle before forking
E1213 01:47:55.556100000 4544900544 thread_pool.cc:253] Waiting for thread pool to idle before forking
....
Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).
See TROUBLESHOOTING.md for how to diagnose problems better.
Anything else we should know about your project / environment?
- There are reports of similar issues from other runtimes/language as well: gRPC extension prevents PHP execution #31772
Reactions are currently unavailable