why do we have different batch limit for complex and non-complex dtypes? can you link me to where this is documented?

I don't know whether it's documented somewhere, I determined this value via experiments.

CUDA limits kernel launches to y and z grid dimension to 65535. Maybe for non-complex dtypes batching is implemented differently allowing 262140 batches.

synced with @ngimel offline. We should check the magma manual, and better document this difference in the batch_limit since the original comments are uninformative.

It's not documented in magma.
CUDA limits kernel launch configurations of y and z grid dimensions to 65535.
https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#features-and-technical-specifications

I haven't checked the source code for how batching is done for non-complex dtypes, but apparently, complex variants use z-dimension of a grid of thread blocks for batching.

I spent a little time looking at the complex path and didn't figure it out, but I did see this:

    if ( n > 2048 ) {   
        #ifndef MAGMA_NOWARNING
        printf("=========================================================================================\n"
               "   WARNING batched routines are designed for small sizes. It might be better to use the\n"
               "   Native/Hybrid classical routines if you want good performance.\n"
               "=========================================================================================\n");
        #endif
    }           
            

in magma_cpotrf_lg_batched

Yeah we should use cusolver for those, if we don't already.

cc @heitorschueroff, @xwang233 Can you guys please create a tracking issue which linalg functions under which conditions use magma or cusolver or cublas, and which functions still need to be weaned off magma and switched to cusolver?

Thanks, I'll create a tracking issue.

@xwang233's issue is here: #47953

@IvanYashchuk please move the autograd tests to common_methods_invocations.py

I think common_methods_invocations.py does not allow specifying the input function to be tested, it allows specifying only the postprocessing function.
Finite differencing doesn't work correctly for torch.cholesky_solve directly, therefore

def func(A, b, upper):
    if upper:
        A = A.triu()
    else:
        A = A.tril()
    return torch.cholesky_solve(b, A, upper)

is tested instead.

I see. I also synced with @mruberry offline and we came to the conclusion it's ok to add autograd tests in test_linalg.py.