@pbelevich Thanks for the tests! Curious did you resolve why there is a difference in behavior between Python and C++ for the following code?

# Python

In [1]: import torch

In [2]: x = torch.ones(1, dtype=float, requires_grad=True)

In [3]: x.backward()

In [4]: x.grad
Out[4]: tensor([1.], dtype=torch.float64)

// C++
auto tensor = torch::ones(1, at::TensorOptions().dtype(at::kFloat).requires_grad(true));
tensor.backward();
std::cout << tensor.grad() << std::endl;
// [ Tensor (undefined) ]

From our earlier discussion, Python tensor.backward() is defined in torch/tensor.py, which has a different implementation than Variable::backward(), and it would be ideal to let the C++ implementation also call torch::autograd::backward to have feature parity with Python. Curious do you see any roadblocks when making the change?

This appears to have broken the XLA build cc @ailzhang

Thanks @suo! pytorch/xla#1065 will be merged soon to fix it! :D

@pbelevich merged this pull request in 98ccae0.

where is the actual note saying what is BC breaking?

ok, it looks like the BC issue is that for non-1-element tensors you no longer create the grad_output to pass through backwards.