Adam pointed out that when matching the .is_cuda property of a new tensor to an existing tensor, we should also take care to match the device:

old_tensor = ...
new_tensor = torch.arange(10)
if old_tensor.is_cuda:
    new_tensor = new_tensor.cuda(old_tensor.get_device())