Skip to content

[Feature] Multi task head#675

Closed
piercus wants to merge 6 commits intoopen-mmlab:devfrom
piercus:multi_task_head
Closed

[Feature] Multi task head#675
piercus wants to merge 6 commits intoopen-mmlab:devfrom
piercus:multi_task_head

Conversation

@piercus
Copy link
Copy Markdown
Contributor

@piercus piercus commented Jan 26, 2022

Motivation

I've read the #628 issue and implemented a proposal for MultiTaskClsHead in order to mix multiple classification head to create a multi task classification tool.

Modification

Please briefly describe what modification is made in this PR.

  • Created a unit test for MultiTaskClsHead
  • the implementation of MultiTaskClsHead

NB : Current implementation is made to be generic and the different head can output different shapes (it will give the ability to mix head from mmcls with heads from mmpose or other projects). The drawback of this is that the output of the simple_test method cannot be a tensor, i choose to deactivate the post_process=False mode for the simple_test method. Any suggestion would be welcome

BC-breaking (Optional)

No

Use cases (Optional)

The basic use case here it to classify multiple classes using the same neural network.
In order to make it work, it is needed to implement a compatible dataset.

I have made the following :

  • Starting from the ImageNet dataset
  • in load_annotations, info['gt_label'] is a tuple containing one gt_label per head
  • evaluate method need to be reworked

Checklist

Before PR:

  • Pre-commit or other linting tools are used to fix the potential lint issues.
  • Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
  • The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

  • If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
  • CLA has been signed and all committers have signed the CLA in this PR.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Jan 26, 2022

CLA assistant check
All committers have signed the CLA.

@mzr1996 mzr1996 changed the base branch from master to dev January 27, 2022 01:44
@mzr1996
Copy link
Copy Markdown
Member

mzr1996 commented Jan 27, 2022

Sounds like a good feature. We will review it recently.
In addition, MMCLS is now developed based on the dev branch, and I have switched this branch's base to dev. More details can be found in Develop based on MMClassification

@codecov
Copy link
Copy Markdown

codecov bot commented Jan 27, 2022

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.07%. Comparing base (702c196) to head (df68f03).
⚠️ Report is 425 commits behind head on dev.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #675      +/-   ##
==========================================
+ Coverage   86.68%   87.07%   +0.38%     
==========================================
  Files         128      131       +3     
  Lines        8255     8571     +316     
  Branches     1422     1476      +54     
==========================================
+ Hits         7156     7463     +307     
- Misses        885      888       +3     
- Partials      214      220       +6     
Flag Coverage Δ
unittests 86.99% <100.00%> (+0.36%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Ezra-Yu
Copy link
Copy Markdown
Collaborator

Ezra-Yu commented Jan 27, 2022

Thank you for your contribution, please follow this pre-commit guide to fix the lint.

@piercus piercus changed the title Multi task head [Feature] Multi task head Jan 27, 2022
@piercus
Copy link
Copy Markdown
Contributor Author

piercus commented Jan 27, 2022

@mzr1996 @Ezra-Yu Thanks for your support !

With the switch of the base, i feel this PR was carrying a lot of non-multi_task_head-related changes (maybe some change to merge from master to dev)

I have cleaned up everything from the dev branch, cherry-picking my changes, so the PR contains only relevant changes
I also ran pre-commit in order to fix the linting.

Now the PR should be more clean and easier to review.

@mzr1996 mzr1996 mentioned this pull request Jan 29, 2022
27 tasks
@ck37
Copy link
Copy Markdown

ck37 commented Feb 7, 2022

Thanks for the work on this! I need to combine an image classification head with an mmdet object detection model as well as mmocr, so would really appreciate this being possible in a single combined model.

@YilanWang
Copy link
Copy Markdown

I think you need to give some examples in Class commit, thanks for your work! It is very helpful! @piercus

@mzr1996
Copy link
Copy Markdown
Member

mzr1996 commented Apr 29, 2022

Hello, I have modified some APIs to support the entire multi-task design. More details can be found in #808
In the design, we use a dict instead of a tuple to save the ground-truth labels passed to MultiTaskClsHead.
And I added the support of post_process=False in the simple_test.

@piercus piercus mentioned this pull request Nov 25, 2022
7 tasks
@Ezra-Yu
Copy link
Copy Markdown
Collaborator

Ezra-Yu commented Apr 7, 2023

this feature has been supported in the new main branch, so close it here.

@Ezra-Yu Ezra-Yu closed this Apr 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants