Add Overview of Benchmark and Model Zoo by Keiku · Pull Request #6985 · open-mmlab/mmdetection

Keiku · 2022-01-11T08:53:11Z

Motivation

According to issue #6868, I created a table that makes the list of model zoo easy to understand. I hope this makes it easier for everyone to understand the models that mmdetection supports. For similar documentation, I referred to the Overview of Kit Structures in Paddle Detection. At a minimum, separating Object Detection and Instance Segmentation makes it easier for users to understand the models supported by mmdetection.

Modification

The list of Benchmark and model zoo in README.md is changed to table.

BC-breaking (Optional)

The Chinese version of the document can also be changed. We should also change the list of model zoos in the link (docs/en/model_zoo.md).

Use cases (Optional)

Only the models that exist in config are listed. Not all features of mmdetection are listed.

Checklist

I left a comment # Not officially listed as a list that hasn't been officially listed yet. Please check if it matches.
Check if the table division is correct.
Check if the listed items correspond to the table category.
The order of the list is the submission date of the paper and the year of adoption at the conference, but please make sure that they match. Reference information is provided here.
This table is created based on v2.19.1, so please add the update.

codecov · 2022-01-11T09:14:05Z

Codecov Report

Merging #6985 (b306b77) into dev (ff9bc39) will decrease coverage by 0.00%.
The diff coverage is n/a.

❗ Current head b306b77 differs from pull request most recent head 3e8da65. Consider uploading reports for the commit 3e8da65 to get more accurate results

@@            Coverage Diff             @@
##              dev    #6985      +/-   ##
==========================================
- Coverage   62.35%   62.35%   -0.01%     
==========================================
  Files         327      327              
  Lines       26129    26129              
  Branches     4424     4424              
==========================================
- Hits        16293    16292       -1     
  Misses       8969     8969              
- Partials      867      868       +1

Flag	Coverage Δ
unittests	`62.33% <ø> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmdet/core/bbox/assigners/max_iou_assigner.py	`72.36% <0.00%> (-1.32%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ff9bc39...3e8da65. Read the comment docs.

RangiLyu · 2022-01-14T08:19:11Z

Thanks for your contribution!

How about switching to a format like this?

Architectures

Object Detection	Instance Segmentation	Panoptic Segmentation	Distillation
Fast R-CNN (ICCV'2015) Faster R-CNN (NeurIPS'2015) SSD (ECCV'2016) RetinaNet (ICCV'2017) Cascade R-CNN (CVPR'2018) YOLOv3 (ArXiv'2018) CornerNet (ECCV'2018) Grid R-CNN (CVPR'2019) CenterNet (CVPR'2019) Libra R-CNN (CVPR'2019)	Mask R-CNN (ICCV'2017) Cascade Mask R-CNN (CVPR'2018) Mask Scoring R-CNN (CVPR'2019) Hybrid Task Cascade (CVPR'2019) YOLACT (ICCV'2019) InstaBoost (ICCV'2019) SOLO (ECCV'2020)	Panoptic FPN	Localization Distillation (ArXiv'2021) Label Assignment Distillation (WACV'2022)

Components

Backbones	Necks	Heads	Loss
VGG (ICLR'2015) ResNet (CVPR'2016) ResNeXt (CVPR'2017) MobileNetV2 (CVPR'2018) HRNet (CVPR'2019) Generalized Attention (ICCV'2019) GCNet (ICCVW'2019) Res2Net (TPAMI'2020) RegNet (CVPR'2020) ResNeSt (ArXiv'2020) PVT (ICCV'2021) Swin (CVPR'2021) PVTv2 (ArXiv'2021)

Keiku · 2022-01-19T04:51:55Z

@RangiLyu Thank you for your instructions. I refactored the table at your instructions.
@hhaAndroid @ZwwWayne Is there anything you would like to change?

RangiLyu · 2022-01-19T09:19:58Z

Looks great! And I have some suggestions:

Rename "Architectures" to "Algorithms"
The comments like # Not officially listed etc. can be deleted.
And about the component section, we can remove the Heads and move those papers to the "Algorithms" table.

RangiLyu · 2022-01-19T09:20:37Z

@ZwwWayne @hhaAndroid Any suggestions?

ZwwWayne · 2022-01-19T12:13:25Z

LGTM. Can be merged after cleaning '# Not officially listed'

Keiku · 2022-01-20T10:23:59Z

@RangiLyu Updated for reviews.

The Algorithms are as follows.

It seems that the centering of Components has failed, but I'm not sure.

@RangiLyu When I cover mmdetection's configs, it looks like a list like the link below. Some items (configs) are missing in the above table, is that okay?

https://github.com/Keiku/mmdet-overview-of-kit-structures

README.md

RangiLyu · 2022-01-21T02:54:57Z

When I cover mmdetection's configs, it looks like a list like the link below. Some items (configs) are missing in the above table, is that okay?

It's okay.

Keiku · 2022-01-21T02:59:17Z

@RangiLyu Excuse me. Let me ask you a question.

What is the difference between Faster R-CNN (NeurIPS'2015) and RPN (NeurIPS'2015)? Is there any problem with RPN in the Object Detection category?

What about the following categories?

"Distillation" may be set to "Other", and "Distillation" and "Contrastive Learning" may be included in "Other".
It may be possible to put "Common" ((OHEM (CVPR'2016), Group Normalization (ECCV'2018), DCNv2 (CVPR'2019), Weight Standardization (ArXiv'2019), Prime Sample Attention (CVPR'2020), Large-Scale Jitter)) in Components.

There was also a new README.md PR recently. Probably conflict.

RangiLyu · 2022-01-21T08:30:17Z

What is the difference between Faster R-CNN (NeurIPS'2015) and RPN (NeurIPS'2015)? Is there any problem with RPN in the Object Detection category?

RPN is the first stage in Faster R-CNN, It's Okay to put it in Object Detection category.

"Distillation" may be set to "Other", and "Distillation" and "Contrastive Learning" may be included in "Other".

It may be possible to put "Common" ((OHEM (CVPR'2016), Group Normalization (ECCV'2018), DCNv2 (CVPR'2019), Weight Standardization (ArXiv'2019), Prime Sample Attention (CVPR'2020), Large-Scale Jitter)) in Components.

Yes, you can add "Other" both to Algorithms and Components.

RangiLyu · 2022-01-25T12:35:50Z

@Keiku Hi, any update? We hope this PR can be merged in the coming version of MMDetection.

Keiku · 2022-01-25T12:51:46Z

@RangiLyu I'll work on it immediately.

Keiku · 2022-01-25T13:35:42Z

@RangiLyu Please check for updates.

Since the original material of Strong Baselines is Copy-Paste paper, I chose CVPR'2021. Please refer to the following link. New, improved Detectron2 Mask R-CNN baselines

ZwwWayne · 2022-01-25T13:52:59Z

LGTM. Thank you for the kind modification @Keiku !

RangiLyu

LGTM

RangiLyu · 2022-01-27T08:04:34Z

This PR resolves #7030

* add overview of benchmark and model zoo * Changed tree/master link to configs folder reference * Changed the format of the table * Moved Heads item to Object Detection table and delete the Heads table * Delete unnecessary comments * Add TOOD (ICCV'2021) * Rename Architectures to Algorithms * Fix centering for Algorithms and Components * Create "Other", and add "Common"

add overview of benchmark and model zoo

b9a9a7e

Changed tree/master link to configs folder reference

b306b77

RangiLyu added the Doc label Jan 14, 2022

RangiLyu requested review from RangiLyu, ZwwWayne and hhaAndroid January 14, 2022 08:31

RangiLyu mentioned this pull request Jan 19, 2022

list of object detection and instance segmentation model list #7030

Closed

Changed the format of the table

c350713

Keiku added 4 commits January 20, 2022 19:02

Moved Heads item to Object Detection table and delete the Heads table

f56ed85

Delete unnecessary comments

953692e

Add TOOD (ICCV'2021)

0b799a5

Rename Architectures to Algorithms

a6878ba

RangiLyu reviewed Jan 21, 2022

View reviewed changes

README.md Outdated Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

Fix centering for Algorithms and Components

415762e

RangiLyu mentioned this pull request Jan 21, 2022

Add some methods and backbones index in README & Add Contents. #7052

Closed

Create "Other", and add "Common"

3e8da65

RangiLyu approved these changes Jan 26, 2022

View reviewed changes

ZwwWayne approved these changes Jan 26, 2022

View reviewed changes

ZwwWayne merged commit c37da19 into open-mmlab:dev Jan 26, 2022

Conversation

Keiku commented Jan 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

Uh oh!

codecov bot commented Jan 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

RangiLyu commented Jan 14, 2022

Uh oh!

Keiku commented Jan 19, 2022

Uh oh!

RangiLyu commented Jan 19, 2022

Uh oh!

RangiLyu commented Jan 19, 2022

Uh oh!

ZwwWayne commented Jan 19, 2022

Uh oh!

Keiku commented Jan 20, 2022

Uh oh!

Uh oh!

Uh oh!

RangiLyu commented Jan 21, 2022

Uh oh!

Keiku commented Jan 21, 2022

Uh oh!

RangiLyu commented Jan 21, 2022

Uh oh!

RangiLyu commented Jan 25, 2022

Uh oh!

Keiku commented Jan 25, 2022

Uh oh!

Keiku commented Jan 25, 2022

Uh oh!

ZwwWayne commented Jan 25, 2022

Uh oh!

RangiLyu left a comment

Choose a reason for hiding this comment

Uh oh!

RangiLyu commented Jan 27, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Keiku commented Jan 11, 2022 •

edited

Loading

codecov bot commented Jan 11, 2022 •

edited

Loading