This I think we should revert. Why wouldn't this work?

@patrickvonplaten

I had a previous conversation with @NielsRogge on Slack.

He was using celine98/canine-s-finetuned-sst2, which has "problem_type": "single_label_classification", set in the config.

Due to setting, in the following block,

the block for the condition self.config.problem_type is None is not run, and it continue to be single_label_classification, therefore the output is not compatible with the provided labels (which is to work with multiple labels here).

For Roberta, cardiffnlp/twitter-roberta-base-emotion is used, and the problem_type is not set in the config. Therefore, the model code is able to set it to multi_label_classification.

Happy with "force-passing" single_label_classification as a flag here to overwrite default configs

You mean multi_label_classification? It's ok to keep this change?

I think it is in the reversed direction:

• celine98/canine-s-finetuned-sst2 has config is set to single_label_classification)

• but this block in PT_SEQUENCE_CLASSIFICATION_SAMPLE
  

  transformers/src/transformers/utils/doc.py

  Lines 269 to 282 in d77680e

  	```python
  	>>> # To train a model on `num_labels` classes, you can pass `num_labels=num_labels` to `.from_pretrained(...)`
  	>>> num_labels = len(model.config.id2label)
  	>>> model = {model_class}.from_pretrained(
  	... "{checkpoint}", num_labels=num_labels, problem_type="multi_label_classification"
  	... )
  	>>> labels = torch.nn.functional.one_hot(torch.tensor([predicted_class_id]), num_classes=num_labels).to(
  	... torch.float
  	... )
  	>>> loss = model(**inputs, labels=labels).loss
  	>>> loss.backward() # doctest: +IGNORE_RESULT
  	```
  	"""

  is meant to be multi_label_classification. See labels = torch.nn.functional.one_hot.

• That's why @NielsRogge needs to add problem_type="multi_label_classification" in the call to from_pretrained.

I will wait @NielsRogge joining this discussion, since he knows better the reason behind his change.

Well, he is faster than my response ...

Ah I see sorry yes you're right - ok to keep the change for me then!

Nono all good this makes perfect sense, I misunderstood here.