@pytorchbot test this please

@pytorchbot test this please

Are there tests for double backprop that could be easily added?

per-op double backprop tests are unnecessary. We only use first-order jacobian vector product functions to compute grads of any order, so as long as first-order is correct it should be all good (assuming autograd code is correct, but we have separate tests for that).

That's not true for code that has a different behavior if the grad is volatile, as in this PR.

i think we should add gradgradcheck, just like gradcheck. We dont know if new-style functions have been written correctly for grad of grad out of the box (for example, user may have rewrapped a Variable somewhere and thought it was okay)

gradcheck runs tests only on volatile grads, so this case is covered

I think that instead of computing a full hessian of each op (these tests would be soooo slooow) we could just add some simple clauses to gradcheck that make sure that there exists a path from grad_input.grad_fn to grad_output's grad accumulator. This should be enough.