resize_output to resize instead of resize_as. (pow also needs to be updated, but it doesn't need to be updated in this PR.)

Progressing in the separate PR #46830

.copy_?

Would you please fix this for pow, too, actually. Sample code demonstrating the error:

torch.tensor((1 + 1j, 2 + 2j)) ** complex(1, 0)
: RuntimeError: fill_ only supports 0-dimension value tensor but got tensor with 1 dimensions.

Progressing as a separate PR #46830

Do these need explicit dispatches? I think the system will work out which functions to call. Explicitly defining the dispatch will, perhaps surprisingly, prevent float_power from deriving its derivative if it's implemented as a composite operation (as suggested above).

pow also has an inplace variant, pow_:

Those entries should probably be moved next to the other pow entries, too.

Once I remove the explicit dispatches, the following error comes out:

AssertionError: There's a formula for float_power(or its functional variant) in derivatives.yaml. It's required to add a dispatch section for it with explicit supported backends e.g CPU/CUDA or DefaultBackend in native_functions.yaml. Please see https://github.com/pytorch/pytorch/tree/master/aten/src/ATen/native#choosing-the-right-dispatch-keyword for instructions to choose the right dispatch keyword.

So, I guess we need the explicit dispatches here for autograd.

In addition, The improvement for pow_ is now progressing in the separate PR #46830.

The way dispatch happens has actually changed. I think the correct dispatch for these is now Math: instead of CPU, CUDA since float_power is implemented using pow.

It seems that when using Math: as dispatch, both autograd test and XLA test will suffer from the precision lack. Even if I directly call pow without any dtype casting like this below, the problem still exists.

Tensor float_power(const Tensor& base, const Tensor& exp) {
  return at::pow(base, exp);
}

Since using CPU, CUDA can avoid from precision lack, maybe we should revert to it?
I am not familiar with autograd yet, maybe I have missed something... 😕

Sorry, can you elaborate on the lack of precision you're seeing, and how changing the dispatch can help with it?

Sorry for the late reply, @mruberry
When I use Math as dispatch, an assertEqual error occurs in test_autograd, saying that the gradients calculated by in-place variant is inconsistent with the general, just like:

>>> a=torch.tensor([1.,2.,3.], dtype=torch.double, requires_grad=True)
>>> b=torch.tensor([1.,2.,3.], dtype=torch.double, requires_grad=True)
>>> grad=torch.randn_like(a).double()
>>> a.float_power(2.).backward(grad)
>>> a.grad
tensor([-4.0256, -1.6108,  1.2338], dtype=torch.float64)
>>> b.float_power_(2.).backward(grad)
>>> b.grad
tensor([-6.0385, -2.0134,  1.4394], dtype=torch.float64)

But in fact, the in-place variants usually not allow to calculating gradients, the original pow is also doing as so.

>>> a.pow_(2.).backward(grad)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.

And when I changed dispatch from Math to CPU, CUDA, making a define in tools/autograd/derivatives.yaml (as we do in the previous version), The above abnormal phenomenon was eliminated.
It seems that there still not have any in-place variant used Math as dispatch so far, so I doubt it may related with this phenomenon...

Thank you for this explanation, this is extremely interesting. cc @ailzhang and @albanD to take a look, too.

RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.

This error is unrelated to the pow formula, it only happens because you modify your leaf inplace. Doing a.clone().pow_(2.) should work just fine.

saying that the gradients calculated by in-place variant is inconsistent with the general

If you don't provide a formula directly in derivatives.yaml, you need to make sure to only ever call functions that do from your generic aten implementation. In particular, always call the at:: version and not native:: version of ops.

Instead of just continuing here, attempt to perform the operation and verify it throws an error using self.assertRaisesRegex.

Addressed!

As above, verify this throws an error instead of continuing

Addressed!

I would remove this paragraph. It's correct, but a little misleading since this PR also allows :attr:`input` to be a scalar. The type support can be mentioned below, too, instead of here.

Addressed!

This mathematical portion is also correct but can be removed. The description above it is already very clear.

Addressed!

"{input}" -> "input (Tensor or Number): the base value(s)"

Addressed!

"value" -> "values"

Addressed!