Skip to content

Conditional with gil and threading #3554

@rth

Description

@rth

A nogil block with a loop that uses an unlikely with gil statement e.g. approximately,

cpdef float test_gil(float[:] x):
    cdef int i = 0
    cdef int size = x1.shape[0]
    cdef float out = 0
    with nogil:
        while i < size:
            out_sum += x[i]
            if i > size +1:
                with gil:
                    raise ValueError
   return out 

appears work similarly to the while function not releasing gil, when using in multi-threaded code.

See more complete minimal example in scikit-learn/scikit-learn#17038 (comment)

Naively I though that since the with gil is never executed, performance wise this would be equivalent to the same function without the with gil block (or roughly that threading performance would be directly impacted by the fraction of run time where the gil is released). However it does not seem to be the case.

Is this behavior expected? If so maybe the documentation section https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html#acquiring-and-releasing-the-gil could be updated to mention conditional acquiring of GIL.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions