Oops, I am so sorry about this. I thought gradcheck converted the tensors to requires_grad=True.

@vishwakftw nah, it's mostly my fault. having worked with gradcheck for 8 months and I still forget about this.

Thanks for the reminder! Fixed onnx.

cc @fritzo @fehiepsi @apaszke on test_distributions changes.

In this patch, I added an error if none of the input args is a tensor that requires grad. In TestDistributions, we are grad checking some constructors on argument lists with no tensors. So I had to add samples as a grad checked input. Hope that this change is okay with you guys.

@pytorchbot retest this please

cc @t-vi gradcheck on einsum wasn't enabled. After fixing in this commit, there are three cases that fail the check. Do you happen to have some idea why it is happening? I will look into this in a couple days.

@ssnl Ha, thanks for catching this! I think this is as simple as having double:
With

                gradcheck_inps = tuple(t.detach().requires_grad_() for t in test[1:])

passses all three cases that were failing before.
An alternative is, of course, to use just add dtype=torch.double for the generated args.
Do you want me to send a diff or include it in yours?

@t-vi I remember that they fail with a 2x Jacobian difference, but I'll have to try again.

The following get's me true 3x while without the double I get false 3x:

x = torch.randn(5)
y = torch.randn(7)
A = torch.randn(3, 5)
#             if test[0] not in {"i,i->", "i,i->i", "ij,ij->ij"}:
for test in [
    ("i,i->", x, x),            # dot
    ("i,i->i", x, x),           # vector element-wise mul
    ("ij,ij->ij", A, A), # matrix element-wise mul
    ]:
    def do_einsum(*args):
         return torch.einsum(test[0], args)
    gradcheck_inps = tuple(t.detach().double().requires_grad_() for t in test[1:])
    print (test[0], torch.autograd.gradcheck(do_einsum, gradcheck_inps, raise_exception=False))

I just tried again. I believe that the random input are already in double precision (the numpy allclose check fails otherwise). The gradcheck still fails with 2x difference for me:

# i,i->

RuntimeError: Jacobian mismatch for output 0 with respect to input 0,    
numerical:tensor([[ 0.6672],                                             
        [ 2.9573],                                                       
        [-1.9153],                                                       
        [ 1.4615],                                                       
        [-1.3014]], dtype=torch.float64)                                 
analytical:tensor([[ 0.3336],                                            
        [ 1.4787],                                                       
        [-0.9577],                                                       
        [ 0.7307],                                                       
        [-0.6507]], dtype=torch.float64)                                 

oh okay I know why. it's not a bug. it's just that these tests use multiples of same tensor, so numerical perturbation changes not one, but two tensors at the same time!

i'll fix that

Ha. Does the detach() solve that, too?

No.. even if detach() is called, they still share storage.

Ah, but the double() created a separate instance for me...

@t-vi This line is probably the culprit:

:P