Skip to content

io.netty.channel.epoll.Native.epollWait0 high cpu usage #5896

@luis-guideti

Description

@luis-guideti

I'm experiencing very high cpu load while using the native transport, the epollWait method is taking as much cpu as a while (true) { } loop

This is not much of an issue if you only have one netty server in the box, but while running multiple instances the cpu load can go as high as 100.0 while without using native transport it never reaches 10.0

Thread dump with native transport:
`
"Netty IO Thread #0" - Thread t@41
java.lang.Thread.State: RUNNABLE
at io.netty.channel.epoll.Native.epollWait0(Native Method)
at io.netty.channel.epoll.Native.epollWait(Native.java:117)
at io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:231)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:255)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
at java.lang.Thread.run(Unknown Source)

Locked ownable synchronizers:
- None
`

Profiler with native transport, 100% cpu time (just like an infinite loop, which increases cpu usage, this is just with 1 connection)
image

Server specs

java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
CentOS Linux release 7.2.1511 (Core)
Netty 4.1.5.final
CPU Xeon 1245v2 4/8c

Now the profiler without native transport, notice there's basically no cpu time with 1 connection
image

Thread dump without native transport
"Netty IO Thread #0" - Thread t@46
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(Unknown Source)
at sun.nio.ch.EPollSelectorImpl.doSelect(Unknown Source)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source)
- locked <111a62cd> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <5846f671> (a java.util.Collections$UnmodifiableSet)
- locked <6480e209> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(Unknown Source)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:731)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:391)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
at java.lang.Thread.run(Unknown Source)

Locked ownable synchronizers:
- None

This can be reproduced with the message test server on my linux box, just send something like 10 messages per second, 5 bytes each message.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions