Skip to content

monkey patching of existing RLock is broken on CPython3 #546

@antocuni

Description

@antocuni

eventlet.patcher._green_existing_locks tries to monkey-patch locks which have been created before the call to monkey patch. To do so, it relies on gc.get_objects() to find them.

This works well on CPython2.7, but it is broken on CPython3.x, as the following example demonstrate (it prints Found! on 2.7 and Not found on 3.6):

import threading
import gc
lock = threading.RLock()
for obj in gc.get_objects():
    if obj is lock:
        print('Found!')
        break
else:
    print('Not found')

This happens because in 3.6 threading.RLock() returns an object of type _thread._RLock, which is implemented in C, inside _threadmodule.c.
In particular, RLockType does not have the flag Py_TPFLAGS_HAVE_GC, and as such it is never returned by gc.get_objects().

Another more direct demonstration of the problem:

import threading
lock = threading.RLock()
print('my lock [1]:', type(lock), lock, id(lock))

import eventlet
eventlet.monkey_patch()
print('my lock [2]:', type(lock), lock, id(lock))

On my machine, with the latest eventlet HEAD (commit a915bb6), running it on python3.6 shows that id(lock) is still the same even after the monkey_patch().

The fact that it went unnoticed until now probably means that this specific functionality lacks a test.

However, I don't know how to fix the problem: it seems to me that there is not reasonable way to implement this functionality in Python3 (not counting the fact that even if we find the existing locks, monkey-patching them is hard: _fix_py3_lock looks very fragile and there are probably cases in which it doesn't work at all).

A possible, brutal "solution" would be to fail and complain loudly if you try to call monkey_patch after threading has already been imported. It probably breaks a lot of existing code, but it's still better than leaving it half-broken as it is now

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions