The array API specification stipulates that for binary operations involving a tensor and a Python scalar, the scalar needs to be converted to the same dtype as the tensor before the operation is performed. PyTorch casts the scalar to a tensor based on its dtype and afterwards performs the default type promotion for the operator.

This can lead to overflows if the tensor dtype can hold the values of the scalar, but the automatically determined dtype cannot.

>>> torch.tensor(0, dtype=torch.float32) + 2.0 ** 63
tensor(9.2234e+18)
>>> torch.tensor(0, dtype=torch.float32) + 2 ** 63
RuntimeError: Overflow when unpacking long

cc @mruberry @rgommers @pmeier @asmeurer @leofang @nairbv