Oh, the same uv_backend_timeout is use by uv_run so this will cause uv__io_poll to be called with timeout=0, will definitely cause a performance regression. Probably need to split either uv_backend_timeout or uv__io_poll.

The problem description makes sense to me, although I have not thought about it closely enough to suggest a solution. It sounds like you are on the right track though. Perhaps uv_backend_timeout should take responsibility for processing the watcher_queue, so that external callers can be sure the uv_backend_fd is up-to-date after checking that function (without needing a second full run through uv_run to do that)?

Updated to split uv_backend_timeout.

Perhaps uv_backend_timeout should take responsibility for processing the watcher_queue

Sounds interesting, basically that means to split some work from uv__io_poll into uv_backend_timeout. Of course this changes iteration order somehow when using UV_RUN_NOWAIT; I can construct a use case that's able to observe this difference, but the case shows more problems. Let's suppose

• user has a prepare handle that registers a new FD (e.g. by uv_poll_start)
• and later unregisters it in a check handle

When using UV_RUN_DEFAULT everything works, the FD is always polled by kernel when blocking; but with either UV_RUN_ONCE or UV_RUN_NOWAIT, that FD would have been unregistered after uv_backend_timeout returns, but prepare handles hadn't been executed, no FD given to kernel. If we return 0 for non-empty watcher_queue then this case becomes a busy polling loop, neither satisfactory.

Arguably this case looks niche, but seems the only way to fix the whole embedding problem is to alter UV_RUN_NOWAIT to return from uv_run right before the final call to epoll_wait; that's the only point where everyone expects to block. I'm unsure how "breaking" this is; maybe push it to v2? Or we can add another uv_run_mode...

EDIT, not the only way, we can also change uv__io_start and uv__io_stop to do (un)registration immediately, instead of queueing.

IIUC, this will fix #3101

@vtjnash this is only a partial fix. As described above, the "start poll in prepare --- stop in check" pattern will become a busy loop with this change; maybe we can call it "working", but barely useful. Still better than nothing at all, though.

I suspect that pattern may be rare, because it incurs unnecessary syscalls at every iteration, so performance sensitive users probably have avoided it; but I have no statistics to support my guess.

Summing up options discussed so far:

• just this patch: fixes only part of the problem, but simple; edge case degrades to busy wait
• update fd in uv_backend_timeout: fixes only part of the problem, but same edge case will ignore events; also bulkier
• change behavior of UV_RUN_NOWAIT: super bulky change, fixes everything, but... breaking change?
• a new run mode: do we want to introduce a new mode just to tell users the old mode doesn't work?

Maybe we can go with 1 for now and postpone 3 for v2. I'm ready to get back and add tests for this if you are happy with this change; afraid I don't have enough time to carry out option 3 however.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Fixed by #3466