Skip to content

ma.TestPolynomial fails using Accelerate on macOS #14644

@shoeffner

Description

@shoeffner

Running python runtests.py -v -s ma fails two tests in numpy/ma/tests/test_extras.py:

  • TestPolynomial.test_polyfit
  • TestPolynomial.test_polyfit_with_masked_NaNs

Reproduce

pipenv install $(head -n4 test_requirements.txt)
pipenv install nose pickle5  
pipenv run python setup.py develop
pipenv install -e .
pipenv run python runtests.py -v -t numpy/ma/tests/test_extras.py::TestPolynomial

Error message:

This is cropped to the essential errors for brevity and peppered with the (probably) corresponding code lines. Note that due to many calls to np.random without seeding the random generator, your output will vary.

Building, see build.log...
Build OK
NumPy version 1.18.0.dev0+ec24179
NumPy relaxed strides checking option: True
========================================================= test session starts =========================================================
platform darwin -- Python 3.7.4, pytest-5.2.0, py-1.8.0, pluggy-0.13.0
rootdir: /Users/shoeffner/Projects/numpy, inifile: pytest.ini
plugins: cov-2.7.1
collected 1903 items                                                                                                                  

numpy/ma/tests/test_extras.py ........................................................FF.....................                   [ 95%]

RankWarning

def test_polyfit(self):
# Tests polyfit
# On ndarrays
x = np.random.rand(10)
y = np.random.rand(20).reshape(-1, 2)
assert_almost_equal(polyfit(x, y, 3), np.polyfit(x, y, 3))

        # warn on rank reduction, which indicates an ill conditioned matrix
        if rank != order and not full:
            msg = "Polyfit may be poorly conditioned"
>           warnings.warn(msg, RankWarning, stacklevel=4)
E           numpy.RankWarning: Polyfit may be poorly conditioned

c          = array([[-8.47984230e+153, -4.08168177e-232],
       [ 9.88131292e-324,  0.00000000e+000],
       [ 1.19279642e-314,  1.20013560e-314],
       [ 1.28763057e-314,  3.21995092e-309]])
deg        = array(3)
full       = False
lhs        = array([[ 1.        ,  1.        ,  1.        ,  1.        ,  1.        ,
         1.        ,  1.        ,  1.        ...5635,  0.37790953,  0.01539077, -0.75740865,
         0.03506115, -1.        ,  0.01098361,  0.05984203,  1.        ]])
lmax       = array(3)
msg        = 'The fit may be poorly conditioned'
order      = 4
rank       = 0
rcond      = 2.220446049250313e-15
resids     = array([], dtype=float64)
rhs        = array([[0.96723583, 0.25115589, 0.07591921, 0.47335804, 0.83546376,
        0.21461214, 0.48720295, 0.10292772, 0.0602..., 0.16955958, 0.29051655, 0.3764026 , 0.53784416,
        0.57034868, 0.59231832, 0.01481114, 0.47632407, 0.57894689]])
s          = array([10.        ,  4.51858015,  3.4355091 ,  2.9844917 ])
scl        = array([3.16227766, 2.12569522, 1.85351264, 1.72756815])
van        = array([[ 1.        ,  0.39509622,  0.15610103,  0.06167493],
       [ 1.        , -0.79841161,  0.6374611 , -0.5089563...    [ 1.        ,  0.3911429 ,  0.15299277,  0.05984203],
       [ 1.        ,  1.        ,  1.        ,  1.        ]])
vander_f   = <function polyvander at 0x10b50aef0>
w          = None
x          = array([ 0.39509622, -0.79841161,  0.72298499,  0.24874447, -0.91154215,
        0.32729701, -1.        ,  0.22228748,  0.3911429 ,  1.        ])
y          = array([[0.96723583, 0.23103402],
       [0.25115589, 0.16955958],
       [0.07591921, 0.29051655],
       [0.47335804,...20295, 0.59231832],
       [0.10292772, 0.01481114],
       [0.06028824, 0.47632407],
       [0.61201684, 0.57894689]])

The actual warning comes from the following part, from the (left) call to polyfit, where the rank is always 0 but the order is 4.

c, resids, rank, s = np.linalg.lstsq(lhs.T/scl, rhs.T, rcond)
c = (c.T/scl).T
# Expand c to include non-fitted coefficients which are set to zero
if deg.ndim > 0:
if c.ndim == 2:
cc = np.zeros((lmax+1, c.shape[1]), dtype=c.dtype)
else:
cc = np.zeros(lmax+1, dtype=c.dtype)
cc[deg] = c
c = cc
# warn on rank reduction
if rank != order and not full:
msg = "The fit may be poorly conditioned"
warnings.warn(msg, RankWarning, stacklevel=2)

AssertionError

def test_polyfit_with_masked_NaNs(self):
x = np.random.rand(10)
y = np.random.rand(20).reshape(-1, 2)
x[0] = np.nan
y[-1,-1] = np.nan
x = x.view(MaskedArray)
y = y.view(MaskedArray)
x[0] = masked
y[-1,-1] = masked
(C, R, K, S, D) = polyfit(x, y, 3, full=True)
(c, r, k, s, d) = np.polyfit(x[1:-1], y[1:-1,:], 3, full=True)
for (a, a_) in zip((C, R, K, S, D), (c, r, k, s, d)):
assert_almost_equal(a, a_)

E       AssertionError: 
E       Arrays are not almost equal
E       
E       Mismatched elements: 8 / 8 (100%)
E       Max absolute difference: 1.2015506
E       Max relative difference: 2.18816199
E        x: array([[-0.292842,  0.256105],
E              [-0.077662,  0.100475],
E              [-0.094572, -0.042055],
E              [ 0.0439  ,  0.097076]])
E        y: array([[0.908708, 0.948916],
E              [0.251341, 0.068777],
E              [0.539048, 0.456381],
E              [0.094167, 0.030449]])

And for both I get some malloc errors captured to stderr (here from the last):

-------------------------------------------------------- Captured stderr call ---------------------------------------------------------
python(8744,0x10d91a5c0) malloc: can't allocate region
*** mach_vm_map(size=18446744072424140800) failed (error code=3)
python(8744,0x10d91a5c0) malloc: *** set a breakpoint in malloc_error_break to debug
init_dgelsd failed init
python(8744,0x10d91a5c0) malloc: can't allocate region
*** mach_vm_map(size=18446744072424140800) failed (error code=3)
python(8744,0x10d91a5c0) malloc: *** set a breakpoint in malloc_error_break to debug
init_dgelsd failed init
======================================== 2 failed, 1898 passed, 1 skipped, 2 xfailed in 7.23s =========================================

Numpy/Python version information:

Output from import sys, numpy; print(numpy.__version__, sys.version):

1.18.0.dev0+ec24179 3.7.4 (default, Jul  9 2019, 18:13:23) 
[Clang 10.0.1 (clang-1001.0.46.4)]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions