Skip to content

Possible race condition leading to the main loop dying? #374

@oakkitten

Description

@oakkitten

I just might be wrong because if this indeed a race condition it should be breaking more things. Anyway, I got this exception (line numbers might be wrong due to debug statements):

Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/.../lib/python3.8/site-packages/aqt/mediasrv.py", line 89, in run
    self.server.run()
  File "/.../lib/python3.8/site-packages/waitress/server.py", line 323, in run
    self.asyncore.loop(
  File "/.../lib/python3.8/site-packages/waitress/wasyncore.py", line 285, in loop
    poll_fun(timeout, map)
  File "/.../lib/python3.8/site-packages/waitress/wasyncore.py", line 211, in poll
    r, w, e = select.select(r, w, e, timeout)
OSError: [Errno 9] Bad file descriptor

This error was extremely rare but since I was getting it while running tests I could just run a lot of them until one failed, which I did, and I think the problem is a follows.

  1. First, thread Thread-1 that the app I'm testing is launching, one that runs waitress server, assembles the descriptor lists for select:

    r = []
    w = []
    e = []
    for fd, obj in list(map.items()): # list() call FBO py3
    is_r = obj.readable()
    is_w = obj.writable()
    if is_r:
    r.append(fd)
    # accepting sockets should not be writable
    if is_w and not obj.accepting:
    w.append(fd)
    if is_r or is_w:
    e.append(fd)

  2. Then, thread waitress-0 deletes one of the channels, in my case it was <waitress.channel.HTTPChannel 127.0.0.1:54044 at 0x7f10ec052400>, and immediately closes the socket:

    def close(self):
    self.connected = False
    self.accepting = False
    self.connecting = False
    self.del_channel()
    if self.socket is not None:
    try:
    self.socket.close()
    except OSError as why:
    if why.args[0] not in (ENOTCONN, EBADF):
    raise

    Stack of waitiress-0 at the moment:

      File "/usr/lib/python3.8/threading.py", line 890, in _bootstrap
        self._bootstrap_inner()
      File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
        self.run()
      File "/usr/lib/python3.8/threading.py", line 870, in run
        self._target(*self._args, **self._kwargs)
      File "/.../lib/python3.8/site-packages/waitress/task.py", line 84, in handler_thread
        task.service()
      File "/.../lib/python3.8/site-packages/waitress/channel.py", line 426, in service
        task.service()
      File "/.../lib/python3.8/site-packages/waitress/task.py", line 168, in service
        self.execute()
      File "/.../lib/python3.8/site-packages/waitress/task.py", line 451, in execute
        self.channel.write_soon(app_iter)
      File "/.../lib/python3.8/site-packages/waitress/channel.py", line 377, in write_soon
        (flushed, exception) = self._flush_exception(self._flush_some)
      File "/.../lib/python3.8/site-packages/waitress/channel.py", line 132, in _flush_exception
        return (flush(), False)
      File "/.../lib/python3.8/site-packages/waitress/channel.py", line 270, in _flush_some
        num_sent = self.send(chunk)
      File "/.../lib/python3.8/site-packages/waitress/wasyncore.py", line 479, in send
        self.handle_close()
      File "/.../lib/python3.8/site-packages/waitress/channel.py", line 317, in handle_close
        wasyncore.dispatcher.close(self)
      File "/.../lib/python3.8/site-packages/waitress/wasyncore.py", line 519, in close
        traceback.print_stack()
  3. Then, thread Thread-1 is trying to see if the file descriptor of the socked closed above is writable, which leads to the the exception above:

    try:
    r, w, e = select.select(r, w, e, timeout)
    except OSError as err:
    if err.args[0] != EINTR:
    raise
    else:
    return

Python 3.8.10, waitress 2.1.1, Ubuntu 20.04 LTS focal @ WSL2

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions