Could you fix for the interface compatibility?

Is it possible to add other return values with no or very few overheads?

@asi1024

Could you fix for the interface compatibility?

Is this the return other values?

Is it possible to add other return values with no or very few overheads?

I added the other return values, and now cupy.linalg.lstsq is 100% compatible with numpy.linalg.lstsq. This does add overhead, but now matches numpy's syntax.

The new overhead comes from calculating the sum of square of the residuals (L-2 norm):

    if rank != n or m <= n:
        resids = cupy.array([], dtype=a.dtype)
    elif b.ndim > 1:
        # note that this should be the same (and faster?) than the next couple of lines
        # e = b - core.dot(a, x)
        # resids = cupy.diagonal(core.dot(e.T, e))
        k = b.shape[1]
        resids = cupy.zeros(k, dtype=a.dtype)
        for i in range(k):
            e = b[:, i] - core.dot(a, x[:, i])
            resids[i] = core.dot(e.T, e)
    else:
        e = b - core.dot(a, x)
        resids = core.dot(e.T, e).reshape(-1)

Why @condition.retry(10) at L157 is needed? One of the result value is often distant from the expected value?

Sometimes test fail, in particular cupy.testing.assert_allclose(x_cpu, x_gpu, atol=1e-3) which maybe fails every 2/100 runs. The obvious case is when a is singular. However the following case is not as simple

import numpy as np
a = np.array([[2., 5., 1., 2.],
              [2., 2., 3., 8.],
              [4., 8., 1., 8.],
              [1., 3., 5., 1.]], dtype=np.float32)
b = np.array((9., 6., 0., 3.), dtype=np.float32)

where lstsq is solved in the following manner

x, resid, rank, s = np.linalg.lstsq(a, b)

produces (x_cpu == numpy.linalg.lstsq solution, x_gpu == cupy.linalg.lstsq solution)

AssertionError:
Not equal to tolerance rtol=1e-07, atol=0.001

Mismatch: 50%
Max absolute difference: 0.0032959
Max relative difference: 1.6645674e-05
 x_cpu: array([198.      , -64.71429 ,   6.857143, -35.142857], dtype=float32)
 x_gpu: array([198.0033  , -64.71534 ,   6.857244, -35.14343 ], dtype=float32)

Should I make the tests deterministic to avoid such failures? this could be done by supplying a random seed for each configuration of a and b. or is it okay to retry when a test fails?

Yes, It seems preferred to make the tests deterministic.
This implementation has a condition with rank != n or m <= n, so the tests should have both regular and singular cases.
Additionally, when the given matrix is singular, the first element of the return value certainly does not have to be equal (or close) to the NumPy's one, but the other elements have to be.

@asi1024 Thanks for your recommendations.

Yes, It seems preferred to make the tests deterministic.

I remove the retry condition, and use fixed random seeds to generate the arrays now.

This implementation has a condition with rank != n or m <= n, so the tests should have both regular and singular cases.
Additionally, when the given matrix is singular, the first element of the return value certainly does not have to be equal (or close) to the NumPy's one, but the other elements have to be.

The tests have been reworked to generate a singular matrix. If a singular matrix was generated, the tests no longer check if the first return element is close to NumPy solution. All other return elements are still checked.

Jenkins, test this please.

Successfully created a job for commit cc5012c:

• Dashboard for commit cc5012c

Jenkins CI test (for commit cc5012c, target branch master) succeeded!

LGTM. Thank you for the PR!