Skip to content

torch.nn.CrossEntropyLoss output shape specification does not match behaviour #79531

@azhx

Description

@azhx

📚 The doc issue

https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html#torch.nn.CrossEntropyLoss

Just slightly changing the provided example we can see

loss = nn.CrossEntropyLoss(reduction = "none")
# Example of target with class probabilities
input = torch.randn(3, 5, requires_grad=True)
target = torch.randn(3, 5).softmax(dim=1)
output = loss(input, target)

target.shape
# torch.Size([3, 5])
output.shape
# torch.Size([3])

This does not match the provided shape specification that

Output: If reduction is ‘none’, same shape as the target. Otherwise, scalar.

Suggest a potential alternative/fix

Output: If target contains probabilities and reduction is ‘none’, shape (N). If target contains class indices and reduction is 'none', same shape as target. Otherwise scalar.

cc @svekars @holly1238 @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are

Metadata

Metadata

Assignees

No one assigned

    Labels

    actionablemodule: docsRelated to our documentation, both in docs/ and docblocksmodule: nnRelated to torch.nntriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions