get_gpu_type is only used at one other place, so it would be awesome if you could update that too and get rid of get_gpu_type method.
https://github.com/pytorch/pytorch/blob/master/test/test_cuda.py#L770

also nit - let's just change this to t = type_map.get(obj.type(), obj.type()) and change the line below to res = obj.clone().type(t).cuda()

Might as well get rid of this test in that case
Since

    def test_is_tensor(self):
        for t in types:
            tensor = get_gpu_type(t)()
            self.assertTrue(torch.is_tensor(tensor))
        self.assertTrue(torch.is_tensor(torch.cuda.HalfTensor()))

becomes something like

for t in types:
  tensor = torch.tensor(data, dtype=t).cuda()

yeah right makes sense! let's do that

For C to R functions, input's dtype is not equal to output's dtype. In general, we'd like gradoutput to be the same dtype as output anyway

hmm what if the output is a tuple? I think we should add a similar check for output as input:
output_tuple = output if isinstance(output, tuple) else (output,)

Can output be a tuple though? Input might be a tuple only because when we backward, we might want to populate the grads of multiple inputs. I'm curious which functions return tuples.

not sure about torch.nn module functions, but some torch functions that come to mind are triangular_solve, qr

@soulitzer did you look into this?

There are functions like nn.AdaptiveMaxPool2d that do return a tuple, so I ended up adding the check for the tuple case.

looks good!

is the target in this case supposed to be complex or real? The math makes it look like it should be complex, but the target created is real?

target_fn is only used by test_jit, which basically just tries to see if scripted module behaves the same as the python module. I don't see it handling check_bfloat16 or check_half either.

does this cause warnings? Because we usually warn when the result is resized (not from something 0-sized).

Yep

Should be fixed in the latest update. When the shape of result matches the post-reduce shape a warning should no longer appear.

@gchanan do you want to take another look at this

this should go in an else branch:

Tensor result;
if (input.is_complex()) {
...
} else {
    result = at::empty({0}, input.options());
}

oh wait just saw you have a return statement in the if condition. I still think it might be cleaner to change it to an if else statement with a common return

Hmm, then you'd have to declare result before the if else. Otherwise it would go out of scope by the time you try to return it.

is that necessarily cleaner? :P I feel like it could be good either way.

wait if you 'declare' it before, you are technically doing an extra default initialization then copy assigning instead of simply copy initializing

A nice trick for this that @ezyang showed me is to use a lambda:

const auto float_type = [&]() {
  if (input.is_complex()) {
    return c10::toValueType(input.scalar_type());
  } else {
    return input.scalar_type();
  }
}();
Tensor result = at::empty({0}, input.options().dtype(float_type));
return at::l1_loss_out(result, input, target, reduction);

But beyond that, what happens if you call c10::toValueType on a non complex dtype? Is it just returned as-is? If so, you don't need branching in this function at all!

Ahh you're right c10::toValueType does handle the non-complex dtype by just returning as-is.