Skip to content

TST, MAINT: a few test fails on Apple M2 max/recent compilers + recent OpenBLAS #18308

@tylerjereddy

Description

@tylerjereddy

Noticed a few testing failures with the following setup on SciPy main (hash: c5b8335)

  • Apple M2 Max
  • OS 13.3.1
  • Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  • GNU Fortran (Homebrew GCC 12.2.0) 12.2.0
  • OpenBLAS 0.3.23 (Homebrew)
  • Python 3.11.3

Quick summary of the 7 failures, decent number of which we could probably just bump tol slightly I think:

  1. test_maxiter_worsening[gmres] -- searching for this shows 4 open issues on our tracker
  2. TestHyp2f1.test_region5[hyp2f1_test_case3] -- searching for this shows a compiler sensitivity in Test failures with Intel compilers #17075 and some closed issues tuning the tolerance in the past
  3. TestHyp2f1.test_region4[hyp2f1_test_case47] -- same category as above probably, rtol 1.19608052e-10 vs. 1e-10 enforced (let's just bump it?)
  4. TestHyp1f1.test_a_negative_integer[-150-1-1.5--0.17336479551542044] -- rtol 1.29680401e-14 vs. 1e-14 enforced (let's just bump it?)
  5. TestHyp2f1.test_a_b_neg_int_after_euler_hypergeometric_transformation[hyp2f1_test_case0] -- rtol 2.89302401e-12 vs. 1e-12 enforced
  6. test_dst4_definition_ortho[1-float32] -- relative diff 1.06287481e-05 not equal to 6 decimals
  7. test_uint64_max -- it already has an xfail for a subset of scenarios and a 5-line comment explaining the mess

Full test failure report/tracebacks:

Details
============================================================================================================== FAILURES ==============================================================================================================
__________________________________________________________________________________________________________ test_uint64_max ___________________________________________________________________________________________________________
[gw4] darwin -- Python 3.11.3 /Users/treddy/python_venvs/py_311_scipy_dev/bin/python
scipy/ndimage/tests/test_datatypes.py:62: in test_uint64_max
    assert_(x[2] == int(float(big)))
E   AssertionError
        arr        = array([18446744073709550591, 18446744073709550591, 18446744073709550591],
      dtype=uint64)
        big        = 18446744073709550591
        inds       = array([[-0.1,  0.9,  1.9]])
        x          = array([                   0, 18446744073709549568, 18446744073709551615],
      dtype=uint64)
_______________________________________________________________________________________________ test_dst4_definition_ortho[1-float32] ________________________________________________________________________________________________
[gw1] darwin -- Python 3.11.3 /Users/treddy/python_venvs/py_311_scipy_dev/bin/python
scipy/fft/_pocketfft/tests/test_real_transforms.py:382: in test_dst4_definition_ortho
    assert_array_almost_equal(y, y2, decimal=dec)
        dec        = 6
        dt         = dtype('float32')
        mdata_x    = array([ 0.        ,  0.71428571,  1.42857143,  2.14285714,  2.85714286,
        3.57142857,  4.28571429,  5.        ,  5.71428571,  6.42857143,
        8.14285714,  7.85714286,  8.57142857,  9.28571429, 10.        ])
        rdt        = <class 'numpy.float32'>
        x          = array([ 0.        ,  0.71428573,  1.4285715 ,  2.142857  ,  2.857143  ,
        3.5714285 ,  4.285714  ,  5.        ,  5.714286  ,  6.428571  ,
        7.142857  ,  7.857143  ,  8.571428  ,  9.285714  , 10.        ],
      dtype=float32)
        y          = array([ 2.2527182e+01, -3.0485203e+00,  6.8829376e-01, -6.5594608e-01,
        1.3825622e-01, -3.0407611e-01,  2.43382... -1.2823799e-01, -3.5893470e-02, -9.6654691e-02,
       -4.9417440e-02, -7.6473922e-02, -6.1873209e-02], dtype=float32)
        y2         = array([ 2.25271838e+01, -3.04852029e+00,  6.88293833e-01, -6.55946133e-01,
        1.38256458e-01, -3.04076803e-01,  2...4780e-02, -1.28238488e-01, -3.58937516e-02, -9.66546037e-02,
       -4.94173596e-02, -7.64740413e-02, -6.18725511e-02])
/opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py:81: in inner
    return func(*args, **kwds)
        args       = (array([ 2.2527182e+01, -3.0485203e+00,  6.8829376e-01, -6.5594608e-01,
        1.3825622e-01, -3.0407611e-01,  2.4338...780e-02, -1.28238488e-01, -3.58937516e-02, -9.66546037e-02,
       -4.94173596e-02, -7.64740413e-02, -6.18725511e-02]))
        func       = <function assert_array_almost_equal at 0x111251e40>
        kwds       = {'decimal': 6}
        self       = <contextlib._GeneratorContextManager object at 0x11126b190>
/opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py:81: in inner
    return func(*args, **kwds)
E   AssertionError: 
E   Arrays are not almost equal to 6 decimals
E   
E   Mismatched elements: 1 / 15 (6.67%)
E   Max absolute difference: 2.20676786e-06
E   Max relative difference: 1.06287481e-05
E    x: array([ 2.252718e+01, -3.048520e+00,  6.882938e-01, -6.559461e-01,
E           1.382562e-01, -3.040761e-01,  2.433828e-02, -1.844277e-01,
E          -1.595955e-02, -1.282380e-01, -3.589347e-02, -9.665469e-02,
E          -4.941744e-02, -7.647392e-02, -6.187321e-02], dtype=float32)
E    y: array([ 2.252718e+01, -3.048520e+00,  6.882938e-01, -6.559461e-01,
E           1.382565e-01, -3.040768e-01,  2.433815e-02, -1.844278e-01,
E          -1.595948e-02, -1.282385e-01, -3.589375e-02, -9.665460e-02,
E          -4.941736e-02, -7.647404e-02, -6.187255e-02])
        args       = (<function assert_array_almost_equal.<locals>.compare at 0x2b2b4e840>, array([ 2.2527182e+01, -3.0485203e+00,  6.88293...780e-02, -1.28238488e-01, -3.58937516e-02, -9.66546037e-02,
       -4.94173596e-02, -7.64740413e-02, -6.18725511e-02]))
        func       = <function assert_array_compare at 0x111251c60>
        kwds       = {'err_msg': '', 'header': 'Arrays are not almost equal to 6 decimals', 'precision': 6, 'verbose': True}
        self       = <contextlib._GeneratorContextManager object at 0x11126b010>
______________________________________________________________________ TestHyp2f1.test_a_b_neg_int_after_euler_hypergeometric_transformation[hyp2f1_test_case0] ______________________________________________________________________
[gw6] darwin -- Python 3.11.3 /Users/treddy/python_venvs/py_311_scipy_dev/bin/python
scipy/special/tests/test_hyp2f1.py:530: in test_a_b_neg_int_after_euler_hypergeometric_transformation
    assert_allclose(hyp2f1(a, b, c, z), expected, rtol=rtol)
        a          = -0.5
        b          = -0.9629749245209605
        c          = -15.5
        expected   = (0.9778506962676361+0.044083801141231616j)
        hyp2f1_test_case = Hyp2f1TestCase(a=-0.5, b=-0.9629749245209605, c=-15.5, z=(1.1578947368421053-1.1578947368421053j), expected=(0.9778506962676361+0.044083801141231616j), rtol=1e-12)
        rtol       = 1e-12
        self       = <scipy.special.tests.test_hyp2f1.TestHyp2f1 object at 0x2b029aed0>
        z          = (1.1578947368421053-1.1578947368421053j)
/opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py:81: in inner
    return func(*args, **kwds)
E   AssertionError: 
E   Not equal to tolerance rtol=1e-12, atol=0
E   
E   Mismatched elements: 1 / 1 (100%)
E   Max absolute difference: 2.83181888e-12
E   Max relative difference: 2.89302401e-12
E    x: array(0.977851+0.044084j)
E    y: array(0.977851+0.044084j)
        args       = (<function assert_allclose.<locals>.compare at 0x2b85c9bc0>, array(0.9778507+0.0440838j), array(0.9778507+0.0440838j))
        func       = <function assert_array_compare at 0x113a89c60>
        kwds       = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=1e-12, atol=0', 'verbose': True}
        self       = <contextlib._GeneratorContextManager object at 0x113aa2f50>
________________________________________________________________________________ TestHyp1f1.test_a_negative_integer[-150-1-1.5--0.17336479551542044] _________________________________________________________________________________
[gw11] darwin -- Python 3.11.3 /Users/treddy/python_venvs/py_311_scipy_dev/bin/python
scipy/special/tests/test_hypergeometric.py:78: in test_a_negative_integer
    assert_allclose(sc.hyp1f1(a, b, x), result, atol=0, rtol=1e-14)
        a          = -150
        b          = 1
        result     = -0.17336479551542044
        self       = <scipy.special.tests.test_hypergeometric.TestHyp1f1 object at 0x2ae3d1010>
        x          = 1.5
/opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py:81: in inner
    return func(*args, **kwds)
E   AssertionError: 
E   Not equal to tolerance rtol=1e-14, atol=0
E   
E   Mismatched elements: 1 / 1 (100%)
E   Max absolute difference: 2.24820162e-15
E   Max relative difference: 1.29680401e-14
E    x: array(-0.173365)
E    y: array(-0.173365)
        args       = (<function assert_allclose.<locals>.compare at 0x2b6afa200>, array(-0.1733648), array(-0.1733648))
        func       = <function assert_array_compare at 0x111355c60>
        kwds       = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=1e-14, atol=0', 'verbose': True}
        self       = <contextlib._GeneratorContextManager object at 0x11136f0d0>
____________________________________________________________________________________________ TestHyp2f1.test_region4[hyp2f1_test_case47] _____________________________________________________________________________________________
[gw6] darwin -- Python 3.11.3 /Users/treddy/python_venvs/py_311_scipy_dev/bin/python
scipy/special/tests/test_hyp2f1.py:1394: in test_region4
    assert_allclose(hyp2f1(a, b, c, z), expected, rtol=rtol)
        a          = -0.5
        b          = -15.5
        c          = -7.949900487447654
        expected   = (-2258.1590330318213+8860.193389158803j)
        hyp2f1_test_case = Hyp2f1TestCase(a=-0.5, b=-15.5, c=-7.949900487447654, z=(0.4172413793103451-0.8724137931034484j), expected=(-2258.1590330318213+8860.193389158803j), rtol=1e-10)
        rtol       = 1e-10
        self       = <scipy.special.tests.test_hyp2f1.TestHyp2f1 object at 0x2b02de450>
        z          = (0.4172413793103451-0.8724137931034484j)
/opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py:81: in inner
    return func(*args, **kwds)
E   AssertionError: 
E   Not equal to tolerance rtol=1e-10, atol=0
E   
E   Mismatched elements: 1 / 1 (100%)
E   Max absolute difference: 1.09362783e-06
E   Max relative difference: 1.19608052e-10
E    x: array(-2258.159032+8860.19339j)
E    y: array(-2258.159033+8860.193389j)
        args       = (<function assert_allclose.<locals>.compare at 0x2b859c680>, array(-2258.159032+8860.19338952j), array(-2258.15903303+8860.19338916j))
        func       = <function assert_array_compare at 0x113a89c60>
        kwds       = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=1e-10, atol=0', 'verbose': True}
        self       = <contextlib._GeneratorContextManager object at 0x113aa2f50>
_____________________________________________________________________________________________ TestHyp2f1.test_region5[hyp2f1_test_case3] _____________________________________________________________________________________________
[gw6] darwin -- Python 3.11.3 /Users/treddy/python_venvs/py_311_scipy_dev/bin/python
scipy/special/tests/test_hyp2f1.py:1891: in test_region5
    assert_allclose(hyp2f1(a, b, c, z), expected, rtol=rtol)
        a          = 16.087593263474208
        b          = 1.0
        c          = -15.964218273004214
        expected   = (85592537010.05054-8061416766688.324j)
        hyp2f1_test_case = Hyp2f1TestCase(a=16.087593263474208, b=1.0, c=-15.964218273004214, z=(0.6448275862068968+0.8724137931034486j), expected=(85592537010.05054-8061416766688.324j), rtol=1e-14)
        rtol       = 1e-14
        self       = <scipy.special.tests.test_hyp2f1.TestHyp2f1 object at 0x2b02f0a90>
        z          = (0.6448275862068968+0.8724137931034486j)
/opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py:81: in inner
    return func(*args, **kwds)
E   AssertionError: 
E   Not equal to tolerance rtol=1e-14, atol=0
E   
E   Mismatched elements: 1 / 1 (100%)
E   Max absolute difference: 0.13428089
E   Max relative difference: 1.66562939e-14
E    x: array(8.559254e+10-8.061417e+12j)
E    y: array(8.559254e+10-8.061417e+12j)
        args       = (<function assert_allclose.<locals>.compare at 0x2b85c9bc0>, array(8.5592537e+10-8.06141677e+12j), array(8.5592537e+10-8.06141677e+12j))
        func       = <function assert_array_compare at 0x113a89c60>
        kwds       = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=1e-14, atol=0', 'verbose': True}
        self       = <contextlib._GeneratorContextManager object at 0x113aa2f50>
___________________________________________________________________________________________________ test_maxiter_worsening[gmres] ____________________________________________________________________________________________________
[gw4] darwin -- Python 3.11.3 /Users/treddy/python_venvs/py_311_scipy_dev/bin/python
scipy/sparse/linalg/_isolve/tests/test_iterative.py:491: in test_maxiter_worsening
    assert_(error <= tol*best_error)
E   AssertionError
        A          = array([[-0.11127953+0.j        ,  0.        +0.j        ,
         0.        +0.j        ,  0.16127953+0.j        ],
 ...,
       [ 0.11127953+0.j        ,  0.        +0.j        ,
         0.        +0.j        , -0.16127953+0.j        ]])
        best_error = 0.01622706276063635
        error      = 0.08845091625842058
        info       = 5
        maxiter    = 5
        solver     = <function gmres at 0x107dbefc0>
        tol        = 5
        v          = array([1., 1., 1., 1.])
        x          = array([-1.18262948e+15+1.81939327e+11j, -3.76592219e-03-1.59022831e-01j,
       -3.05910974e-03+1.58937041e-01j, -8.15989803e+14+1.25534361e+11j])
====================================================================================================== short test summary info =======================================================================================================

Metadata

Metadata

Assignees

No one assigned

    Labels

    maintenanceItems related to regular maintenance tasks

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions