Skip to content

RecursionError in scalarmath hypothesis tests #18749

@rgommers

Description

@rgommers

test_operator_object_left and test_operator_object_right are failing with:

RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low
Details
___________________________________________________________ test_operator_object_left ________________________________________________________________________________________________________________ test_operator_object_left ____________________________________________________________

    @given(sampled_from(objecty_things),
>          sampled_from(reasonable_operators_for_scalars),
           sampled_from(types))
    @settings(verbosity=Verbosity.verbose)

f          = <function given.<locals>.run_test_as_given.<locals>.wrapped_test at 0x7f9e6c8c49d0>

../numpy/core/tests/test_scalarmath.py:735: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/site-packages/hypothesis/core.py:646: in execute_once
        data       = ConjectureData        data       = ConjectureData(VALID, 3 bytes, frozen)
        exception  = RecursionError('cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low')
        expected_failure = (RecursionError('cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low'), 'Traceback (...sionlimit(n)\nRecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low\n')
        is_final   = True
        print_example = True
        result     = None
        run        = <function StateForActualGivenExecution.execute_once.<locals>.run at 0x7f9de5a1f1f0>
        self       = <hypothesis.core.StateForActualGivenExecution object at 0x7f9e02b057c0>
        test       = <function test_operator_object_left at 0x7f9e6c8c4820>
        text_repr  = ['o=<object at 0x7f9e6cb56c00>, op=<function _operator.le>, type_=bool_']
        traceback  = 'Traceback (most recent call last):\n  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_...rsionlimit(n)\nRecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low\n'
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <hypothesis.core.StateForActualGivenExecution object at 0x7f9e02b057c0>
message = 'Hypothesis test_operator_object_left(o=<object at 0x7f9e6cb56c00>, op=<function _operator.le>, type_=bool_) produces unreliable results: Falsified on the first call but did not on a subsequent one'

    def __flaky(self, message):
        if len(self.falsifying_examples) <= 1:
>           raise Flaky(message)
E           hypothesis.errors.Flaky: Hypothesis test_operator_object_left(o=<object at 0x7f9e6cb56c00>, op=<function _operator.le>, type_=bool_) produces unreliable results: Falsified on the first call but did not on a subsequent one

message    = 'Hypothesis test_operator_object_left(o=<object at 0x7f9e6cb56c00>, op=<function _operator.le>, type_=bool_) produces unreliable results: Falsified on the first call but did not on a subsequent one'
self       = <hypothesis.core.StateForActualGivenExecution object at 0x7f9e02b057c0>

/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/site-packages/hypothesis/core.py:874: Flaky
------------------------------------------------------------------- Hypothesis -------------------------------------------------------------------
Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.lt>,
    type_=numpy.bool_,
)
Trying example: test_operator_object_left(
    o=None, op=<function _operator.lt>, type_=numpy.bool_,
)
Trying example: test_operator_object_left(
    o=None, op=<function _operator.add>, type_=numpy.int64,
)
Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.mod>,
    type_=numpy.bool_,
)
Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.mod>,
    type_=numpy.complex256,
)
Trying example: test_operator_object_left(
    o=None, op=<function _operator.mul>, type_=numpy.bool_,
)
Trying example: test_operator_object_left(
    o=None, op=<function _operator.mul>, type_=numpy.int16,
)
Trying example: test_operator_object_left(
    o=None, op=<function _operator.add>, type_=numpy.bool_,
)
Trying example: test_operator_object_left(
    o=None, op=<function _operator.matmul>, type_=numpy.bool_,
)
Trying example: test_operator_object_left(
    o=None, op=<function _operator.matmul>, type_=numpy.uint16,
)
Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.mul>,
    type_=numpy.bool_,
)
Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.mul>,
    type_=numpy.float128,
)
Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.truediv>,
    type_=numpy.float128,
)
Traceback (most recent call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low

Trying example: test_operator_object_left(
    o=None, op=<function _operator.truediv>, type_=numpy.float128,
)
Traceback (most recent call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low

Trying example: test_operator_object_left(
    o=None, op=<function _operator.truediv>, type_=numpy.int8,
)
Traceback (most recent call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low

Trying example: test_operator_object_left(
    o=None, op=<function _operator.le>, type_=numpy.int8,
)
Traceback (most recent call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low

Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.matmul>,
    type_=numpy.float64,
)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
t call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low

Trying example: test_operator_object_left(
    o=None, op=<function _operator.sub>, type_=numpy.ulonglong,
)
Traceback (most recent call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low

Trying example: test_operator_object_left(
    o=None, op=<function _operator.le>, type_=numpy.complex128,
)
Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.add>,
    type_=numpy.int8,
)
Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>, op=<function _operator.le>, type_=numpy.int8,
)
Traceback (most recent call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low

Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.le>,
    type_=numpy.bool_,
)
Traceback (most recent call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low

Trying example: test_operator_object_left(
    o=None, op=<function _operator.le>, type_=numpy.bool_,
)
Traceback (most recent call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low

Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.add>,
    type_=numpy.uint8,
)
Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.add>,
    type_=numpy.bool_,
)
Traceback (most recent call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low

Trying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.le>,
    type_=numpy.bool_,
)
Falsifying example: test_operator_object_left(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.le>,
    type_=numpy.bool_,
)
Failed to reproduce exception. Expected: 
Traceback (most recent call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 740, in test_operator_object_left
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low


You can reproduce this example by temporarily adding @reproduce_failure('6.8.9', b'AAABAA==') as a decorator on your test case
___________________________________________________________ test_operator_object_right ___________________________________________________________

>   ???

f          = <function given.<locals>.run_test_as_given.<locals>.wrapped_test at 0x7f9e6c8c4ca0>
../numpy/core/tests/test_scalarmath.py:747: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/site-packages/hypothesis/core.py:646: in execute_once
    self.__flaky(
        data       = ConjectureData(VALID, 3 bytes, frozen)
        exception  = RecursionError('cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low')
        expected_failure = (RecursionError('cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low'), 'Traceback (...sionlimit(n)\nRecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low\n')
        is_final   = True
        print_example = True
        result     = None
        run        = <function StateForActualGivenExecution.execute_once.<locals>.run at 0x7f9de4698dc0>
        self       = <hypothesis.core.StateForActualGivenExecution object at 0x7f9de4689be0>
        test       = <function test_operator_object_right at 0x7f9e6c8c4940>
        text_repr  = ['o=<object at 0x7f9e6cb56c00>, op=<function _operator.lt>, type_=complex256']
        traceback  = 'Traceback (most recent call last):\n  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 751, in test_...rsionlimit(n)\nRecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low\n'
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <hypothesis.core.StateForActualGivenExecution object at 0x7f9de4689be0>
message = 'Hypothesis test_operator_object_right(o=<object at 0x7f9e6cb56c00>, op=<function _operator.lt>, type_=complex256) produces unreliable results: Falsified on the first call but did not on a subsequent one'

    def __flaky(self, message):
        if len(self.falsifying_examples) <= 1:
>           raise Flaky(message)
E           hypothesis.errors.Flaky: Hypothesis test_operator_object_right(o=<object at 0x7f9e6cb56c00>, op=<function _operator.lt>, type_=complex256) produces unreliable results: Falsified on the first call but did not on a subsequent one

message    = 'Hypothesis test_operator_object_right(o=<object at 0x7f9e6cb56c00>, op=<function _operator.lt>, type_=complex256) produces unreliable results: Falsified on the first call but did not on a subsequent one'
self       = <hypothesis.core.StateForActualGivenExecution object at 0x7f9de4689be0>

/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/site-packages/hypothesis/core.py:874: Flaky
------------------------------------------------------------------- Hypothesis -------------------------------------------------------------------
Falsifying example: test_operator_object_right(
    o=<object at 0x7f9e6cb56c00>,
    op=<function _operator.lt>,
    type_=numpy.complex256,
)
Failed to reproduce exception. Expected: 
Traceback (most recent call last):
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 751, in test_operator_object_right
    with recursionlimit(100):
  File "/home/gitpod/mambaforge3/envs/numpy-dev/lib/python3.9/contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "/workspace/numpy/numpy/core/tests/test_scalarmath.py", line 719, in recursionlimit
    sys.setrecursionlimit(n)
RecursionError: cannot set the recursion limit to 100 at the recursion depth 75: the limit is too low

I observed this in the Gitpod dev environment, to reproduce:

  1. Go to https://www.gitpod.io/#https://github.com/numpy/numpy and wait till the environment opens (takes 1-2 minutes)
  2. cd doc && ipython followed by import numpy as np; np.test()"

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions