-
-
Notifications
You must be signed in to change notification settings - Fork 16.3k
Event loop group shutdown with IOUring slow on Java 25 #16174
Description
Environment: Netty version: 4.2.9.Final
Hi, we hit a problem in an integration test suite that rapidly creates and destroys some netty based test clients, where they preferred IOUring as transport, and they are running on java 25. The tests were running out of resources and on debugging we found that the netty event loop threads were unexpectedly growing throughout our tests and eventually exhausting resources somewhere. In our tests we were closing the EventLoopGroup but not blocking on the shutdown future completing. Changing the clients to use epoll fixed our test issue. Also interesting is that our test suite running on java 21 is not affected.
With some debugging I found that the EventLoopGroups were taking a very long time to be destroyed.
Here's a reproducer demonstrating that there is a difference between how long it takes to shutdown IOUring ELGs on java 21 vs java 25. It appears that it is related to how many ELGs you are trying to destroy in parallel, as well as how many threads you have in each ELG.
https://github.com/robobario/iouring-reproducer/tree/main
There are some details in the reproducer about my environment and a flamegraph attached where the vast majority of the time is spent in jdk.internal.misc.ScopedMemoryAccess.closeScope0(MemorySessionImpl, ScopedMemoryAccess$ScopedAccessError), which was caused by io.netty.channel.uring.IoUringIoHandler.destroy().
Maybe this is related to #15582, another problem with IOUring shutdown taking a long time.