[Fix]Fix bug of classwise weight in bce loss by Ezra-Yu · Pull Request #5776 · open-mmlab/mmdetection

Ezra-Yu · 2021-08-03T09:36:56Z

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

fix bug of classwise_weight in bce loss, and add unit test

Modification

this line has a bug. the loss only scale in the location where the label is 1. The class_weight has no effect on the location where the label is 0.

my exp:

CLAassistant · 2021-08-03T09:37:01Z

All committers have signed the CLA.

mzr1996 · 2021-08-04T05:56:44Z

I think here we need @xvjiarui, refers to ElectronicElephant@f526be0

xvjiarui

LGTM
Just to make sure there is no BC breaking.

mzr1996 · 2021-08-05T02:42:09Z

LGTM
Just to make sure there is no BC breaking.

Does Yolov3 need to use pos_weight? I notice it's modified in YOLOv3-mmdetection

AronLin · 2021-08-05T05:40:48Z

mmdet/models/losses/cross_entropy_loss.py

        weight = weight.float()
    loss = F.binary_cross_entropy_with_logits(
-        pred, label.float(), pos_weight=class_weight, reduction='none')
+        pred, label.float(), weight=class_weight, reduction='none')


Please take a look at the docstring of F.binary_cross_entropy_with_logits.
weight should be a tensor that matches the input tensor shape. It is Not the class-aware weight.
pos_weight should be a vector with a length equal to the number of classes.

it will broadcasts to (....., C)
Of course, the premise is to convert pred tensor into shape of (..., C).

pos_weight is a vector with a length equal to the number of classes， but it is different with the classwise weight. Using pos_weight will pos items and neg items loss unblanced.

AronLin · 2021-08-05T05:41:57Z

tests/test_metrics/test_losses.py

    assert torch.allclose(loss_cls(fake_pred, fake_label), torch.tensor(200.))

+    # test bce_loss
+    cls_score = torch.Tensor([[-200, 100], [500, -1000], [300, -300]])


bce_loss now only supports the input tensor with shape (n, 1).

yolov3 cls head uses bce loss, it's pred input tensor has shpae (batchsize, number_booxes, number_class)

codecov · 2021-08-10T13:50:54Z

Codecov Report

Merging #5776 (4d326d0) into master (87eda06) will decrease coverage by 0.37%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5776      +/-   ##
==========================================
- Coverage   66.47%   66.10%   -0.38%     
==========================================
  Files         281      281              
  Lines       22019    21870     -149     
  Branches     3659     3659              
==========================================
- Hits        14638    14458     -180     
- Misses       6631     6651      +20     
- Partials      750      761      +11

Flag	Coverage Δ
unittests	`66.10% <ø> (-0.35%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmdet/models/losses/cross_entropy_loss.py	`100.00% <ø> (ø)`
mmdet/utils/contextmanagers.py	`0.00% <0.00%> (-18.58%)`	⬇️
mmdet/models/dense_heads/dense_test_mixins.py	`38.46% <0.00%> (-5.13%)`	⬇️
mmdet/models/utils/normed_predictor.py	`83.33% <0.00%> (-4.77%)`	⬇️
mmdet/models/roi_heads/test_mixins.py	`70.21% <0.00%> (-4.26%)`	⬇️
mmdet/models/detectors/base.py	`53.65% <0.00%> (-2.50%)`	⬇️
mmdet/models/roi_heads/base_roi_head.py	`85.29% <0.00%> (-2.21%)`	⬇️
mmdet/datasets/pipelines/formating.py	`64.22% <0.00%> (-2.16%)`	⬇️
mmdet/core/bbox/coder/yolo_bbox_coder.py	`58.97% <0.00%> (-2.01%)`	⬇️
mmdet/core/anchor/point_generator.py	`51.57% <0.00%> (-1.96%)`	⬇️
... and 49 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 87eda06...4d326d0. Read the comment docs.

AronLin · 2021-08-12T12:21:59Z

pred = torch.Tensor([[2, 4],
                     [-2, 1], 
                     [-1, 3]])  # (N, C)
label = torch.Tensor([[0, 1], 
                     [1, 0], 
                     [1, 1]])  # (N, C)
class_weight = torch.tensor([0.1, 0.9])  # (C, )

# loss without class_weight
loss_cls_cfg = dict(
    type='CrossEntropyLoss',
    use_sigmoid=True,
    loss_weight=1.0,
    reduction='none'
)
loss_cls = build_loss(loss_cls_cfg)
origin_loss = loss_cls(pred, label)
weight_between_classes = loss_cls(pred, label, weight=class_weight)

# loss with class_weight
loss_cls_cfg = dict(
    type='CrossEntropyLoss',
    use_sigmoid=True,
    loss_weight=1.0,
    class_weight = torch.tensor([0.1, 0.9]),
    reduction='none'
)
loss_cls = build_loss(loss_cls_cfg)
weight_between_samples = loss_cls(pred, label)

# output
origin_loss: tensor([[2.1269, 0.0181],
                     [2.1269, 1.3133],
                     [1.3133, 0.0486]])
weight_between_classes: tensor([[0.2127, 0.0163],
                                [0.2127, 1.1819],
                                [0.1313, 0.0437]])
weight_between_samples: tensor([[2.1269, 0.0163],
                                [0.2127, 1.3133],
                                [0.1313, 0.0437]])

After discussion, we think it is not a bug and we do not need to change the interface of class_weight, here are the reasons:
(1). In the original design, class_weight is used to adjust the weight between the positive and negative samples of each class. So the class_weight is applied to positive samples.
(2). If we want to adjust the importance between classes, we can use the interface 'weight' in the forward function.
(3). If we approve this PR, it will be hard to implement the function in (1). But the function in (2) is easy to implement without this PR.

Please check whether the above items are in reason, I will close this PR if you agree with me.
Kindly ping @ZwwWayne @xvjiarui @mzr1996 @Ezra-Yu .

Ezra-Yu · 2021-08-12T12:38:19Z

OK，understand that.

Ezra-Yu added 3 commits August 3, 2021 16:50

fix bug of classwise weight in bce loss

cd0a42b

add unit test

424e6b6

update unit test

e8b3d59

jshilong changed the title ~~[bug] fix bug of classwise weight in bce loss~~ [Bug] fix bug of classwise weight in bce loss Aug 3, 2021

jshilong changed the title ~~[Bug] fix bug of classwise weight in bce loss~~ [Bug] Fix bug of classwise weight in bce loss Aug 3, 2021

ZwwWayne requested review from AronLin and jshilong August 4, 2021 02:17

Merge branch 'open-mmlab:master' into classwise-weight-bce

71ef123

mzr1996 mentioned this pull request Aug 4, 2021

[Feature] Support classwise weight in losses open-mmlab/mmpretrain#388

Merged

xvjiarui approved these changes Aug 5, 2021

View reviewed changes

Ezra-Yu added 2 commits August 5, 2021 11:16

Merge branch 'open-mmlab:master' into classwise-weight-bce

fd5eabf

update label a one dim

6e59bdc

Ezra-Yu changed the title ~~[Bug] Fix bug of classwise weight in bce loss~~ [Bug] Fix bug of classwise weight in bce loss [WIP] Aug 5, 2021

Ezra-Yu changed the title ~~[Bug] Fix bug of classwise weight in bce loss [WIP]~~ [WIP]Fix bug of classwise weight in bce loss Aug 5, 2021

AronLin reviewed Aug 5, 2021

View reviewed changes

Ezra-Yu changed the title ~~[WIP]Fix bug of classwise weight in bce loss~~ [Fix]Fix bug of classwise weight in bce loss Aug 6, 2021

Ezra-Yu added 2 commits August 10, 2021 16:47

Merge branch 'open-mmlab:master' into classwise-weight-bce

6521e83

add unit tests

4d326d0

ZwwWayne closed this Aug 12, 2021

Ezra-Yu mentioned this pull request Sep 7, 2021

[Bug] ”weight=class_weight“ should be modified for ”pos_weight=class_weight“ open-mmlab/mmpretrain#434

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fix]Fix bug of classwise weight in bce loss #5776

[Fix]Fix bug of classwise weight in bce loss #5776
Ezra-Yu wants to merge 8 commits intoopen-mmlab:masterfrom
Ezra-Yu:classwise-weight-bce

Ezra-Yu commented Aug 3, 2021

Uh oh!

CLAassistant commented Aug 3, 2021 •

edited

Loading

Uh oh!

mzr1996 commented Aug 4, 2021 •

edited

Loading

Uh oh!

xvjiarui left a comment

Uh oh!

mzr1996 commented Aug 5, 2021

Uh oh!

AronLin Aug 5, 2021 •

edited

Loading

Uh oh!

Ezra-Yu Aug 5, 2021 •

edited

Loading

Uh oh!

Ezra-Yu Aug 10, 2021

Uh oh!

AronLin Aug 5, 2021

Uh oh!

Ezra-Yu Aug 5, 2021

Uh oh!

codecov bot commented Aug 10, 2021 •

edited

Loading

Uh oh!

AronLin commented Aug 12, 2021 •

edited

Loading

Uh oh!

Ezra-Yu commented Aug 12, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

Ezra-Yu commented Aug 3, 2021

Motivation

Modification

Uh oh!

CLAassistant commented Aug 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mzr1996 commented Aug 4, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xvjiarui left a comment

Choose a reason for hiding this comment

Uh oh!

mzr1996 commented Aug 5, 2021

Uh oh!

AronLin Aug 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ezra-Yu Aug 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ezra-Yu Aug 10, 2021

Choose a reason for hiding this comment

Uh oh!

AronLin Aug 5, 2021

Choose a reason for hiding this comment

Uh oh!

Ezra-Yu Aug 5, 2021

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Aug 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

AronLin commented Aug 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Ezra-Yu commented Aug 12, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

CLAassistant commented Aug 3, 2021 •

edited

Loading

mzr1996 commented Aug 4, 2021 •

edited

Loading

AronLin Aug 5, 2021 •

edited

Loading

Ezra-Yu Aug 5, 2021 •

edited

Loading

codecov bot commented Aug 10, 2021 •

edited

Loading

AronLin commented Aug 12, 2021 •

edited

Loading