@apaszke you mean assert that target.is_same_size(input)?

Yeah I guess that if it's an element-wise loss then they should match exactly.

This was my first thought too (as the docstring also says they should be the same size - the same for BCELoss).

cc @martinarjovsky who had some thoughts on this.

(btw I think there's still an issue in my weight sizing in the test - I'll fix that once we reach a conclusion on how we should fix this issue generally)

@apaszke if we add a check in binary_cross_entropy_with_logits to raise if target.size() != input.size(), then we should also add this to binary_cross_entropy as their interfaces should match.

However, this will break backwards compatibility as I think many people use it in the following way:

target = Variable(torch.rand(64))
output = Variable(torch.rand(64,1) - 0.5)
nn.BCELoss()(sigmoid(output), target)

Looking at the interface of BCELoss a bit more, it is also quite inconsistent i.e.

target = Variable(torch.rand(5))
output = Variable(torch.rand(5, 1) - 0.5)
nn.BCELoss()(nn.Sigmoid()(output), target) # this works fine
weight = torch.ones(1)
nn.BCELoss(weight)(nn.Sigmoid()(output), target) #this fails as target.size() != input.size()
nn.BCELoss(weight)(nn.Sigmoid()(output), target.unsqueeze(1)) #this works

I'm in favour of asserting target.size() == input.size() as it leads to simpler code, happy to ammend this PR if you agree @apaszke (but it will break backwards compatibility of BCELoss with single dim targets)

How about adding strict checks to with_logits version and deprecation warnings to the ones that we had earlier?

Sounds sensible - I'll update this PR with that then

@apaszke I've made the changes and added the warning in functional.binary_cross_entropy and in torch/nn/_functions/thnn/loss.BCELoss. Should I just add the warning in BCECriterion.c instead?

Hello! Adding checks for sizes would be good. The thing I think it's really necessary is that behaviour in the 'with_logits' version is exactly the same as the one on probabilities [for BCE and non-binary CE]. The main thing that was worrisome to me was that a shape mismatch affected one and not the other. Cheers :)

thanks Alykhan!