I was using watchdog extensively in a quite large application with many threads. It would occasionally result in ValueError: filedescriptor out of range in select() exceptions.
After some searching, I believe that the culprit is indeed the select.select() call. On Linux the man page says
WARNING: select() can monitor only file descriptors numbers that are less than FD_SETSIZE (1024)—an unreasonably low limit for many modern applications—and this limitation will not change. All modern applications should instead use poll(2) or epoll(7), which do not suffer this limitation.
I tried to move to select.poll() in #1078. Maybe there is a better approach than manually checking hasattr(select, 'poll'). Let me know if this works or any other options are preferred. In order to avoid regressions, it might be wise to add a test that opens >1024 file descriptors and then uses watchdog or something along those lines. If this is required, I could take a stab at it.
I was using watchdog extensively in a quite large application with many threads. It would occasionally result in
ValueError: filedescriptor out of range in select()exceptions.After some searching, I believe that the culprit is indeed the
select.select()call. On Linux the man page saysI tried to move to
select.poll()in #1078. Maybe there is a better approach than manually checkinghasattr(select, 'poll'). Let me know if this works or any other options are preferred. In order to avoid regressions, it might be wise to add a test that opens >1024 file descriptors and then uses watchdog or something along those lines. If this is required, I could take a stab at it.