cc @ngimel. This check triggers on torch.add(bool_tensor, bool_tensor, 0), which works fine on eager. My understanding is that refs should work in all cases that eager works - do you happen to know the purpose of this check?

We need to modify these checks to account for PyTorch allowing integer numbers with boolean tensors. Typically a number cannot be a "higher" Python type than the Python type corresponding to the tensors' type. For example,

a = torch.randint(0, 5, (2, 2))
b = torch.randint(0, 5, (2, 2))
torch.add(a, b, alpha=.5)
: RuntimeError: For integral input tensors, argument alpha must not be a floating point number.

The exception to this is that boolean tensors will interoperate with integers, and I think the integer is always cast to a bool before the operation. The references just don't implement this particular oddity yet.

Note that this IS an oddity, as it's inconsistent with what we'd expect from type promotion if we wrote torch.add as an actual composite operation:

a = torch.randint(0, 5, (2, 2)).bool()
b = torch.randint(0, 5, (2, 2)).bool()
torch.add(a, b, alpha=1)
: tensor([[True, True],
       [True, True]])

torch.add(a, b) * 1
: tensor([[1, 1],
       [1, 1]])

PyTorch is pretty cool that way.

thanks for the overview! I'll update this check to pass in the bool case.

What do you mean?

@albanD This is the same issue as commented out check on like 909. addmm(bool_tensor1, bool_tensor2, bool_tensor3, beta=1, alpha=1) fails this check, but works fine in eager mode.

Do you know why this check was added? Are there some checks in C++ that we're trying (incorrectly) to emulate?

I am not sure tbh. But I would expect that yes. I'm sure Natalia would know.

Why do we even support addmm for bools? I'm pretty sure we don't support it on cuda. Edit: oh sorry it's not addmm, github was hiding weird things. Then I don't know, let's ask @lezcano. In other ops (like add) we require that alpha is lesser type, but these requirements are inconsistent across ops (add is different from addcmul etc)

This is the check, I believe:

I am currently putting together a more thorough test for decompositions. This test that has uncovered a few issues in some operations, in particular for bools. No idea what's going on there just yet. I plan to fix all the issues the test has uncovered before putting up a PR. Hopefully that'll be today or tomorrow.

In the meantime, I'm leaving here this fantastic behaviour it uncovered:

import torch msk = torch.tensor([False])
b = torch.tensor([False])
print(torch.where(msk, True, b).dtype)
print(torch.ops.aten.where.ScalarSelf(msk, True, b).dtype)
print(torch._refs.where(msk, True, b).dtype)

outputs

torch.bool
torch.int64
torch.bool

This happens just for boolean tensors, all the other types work just fine.