Issue #36029 requests several functions be implemented for complex tensors, including torch.isclose. This issue separates the isclose discussion since it's nuanced and we have a choice of behaviors to implement.

@ngimel and I think there are two candidates for torch.isclose behavior.

The first is to be like NumPy's isclose. This function appears to be broken when dealing with non-finite values, however. See this issue. One possible fix is to handle non-finite values like Python's cmath.isclose, where if a non-finite value is present two complex numbers are "close" only if they are identical. In particular, this approach would:

• When the imaginary and real parts of the complex numbers are finite, the typical isclose() algorithm of abs(a - b) <= (atol + rtol * abs(b)) is used. (Consistent with current NumPy on finite values, divergent from cmath which uses a symmetric comparison. Torch is already consistent with NumPy for floats.)
• If any part is -inf or inf, the numbers must be identical.
• If any part is NaN, the numbers are never close, unless equal_nan=True, in which case NaN is treated like an infinity and the numbers must be identical.

Requiring the real and imaginary part of a complex number be identical is a stringent test, and likely not what we want in our test suite. A second option, then, is to compare the complex numbers' parts. That is, torch.isclose(Complex) = torch.isclose(Real) and torch.isclose(Imag). This would make our testing easier, and we think isclose is principally for testing. It would diverge our behavior from Python and NumPy, however.

Since NumPy's isclose for complex is broken on non-finite values, and this error hasn't been noticed, that may suggest that NumPy isclose is little-used on complex arrays and its behavior may be able to change, too.

cc @ezyang @anjali411 @dylanbespalko
cc @rgommers