Adjust vision transformer backbone architectures#524
Merged
xvjiarui merged 10 commits intoopen-mmlab:masterfrom Apr 30, 2021
Merged
Adjust vision transformer backbone architectures#524xvjiarui merged 10 commits intoopen-mmlab:masterfrom
xvjiarui merged 10 commits intoopen-mmlab:masterfrom
Conversation
added 3 commits
April 28, 2021 19:31
* Add DropPath, trunc_normal_ for VisionTransformer implementation; * Add class token buring intermediate period and remove it during final period;
Codecov Report
@@ Coverage Diff @@
## master #524 +/- ##
==========================================
+ Coverage 86.60% 86.68% +0.07%
==========================================
Files 99 101 +2
Lines 5160 5234 +74
Branches 834 848 +14
==========================================
+ Hits 4469 4537 +68
- Misses 533 535 +2
- Partials 158 162 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
* Remove class token and reshape entire token feature from NLC to NCHW;
added 2 commits
April 29, 2021 14:56
* Add related unit test;
xvjiarui
reviewed
Apr 30, 2021
mmseg/models/backbones/vit.py
Outdated
| final feature map. Default: False. | ||
| interpolate_mode (str): Select the interpolate mode for position | ||
| embeding vector resize. Default: bilinear. | ||
| input_cls_token (bool): If concatenating class token into image tokens |
Collaborator
There was a problem hiding this comment.
Suggested change
| input_cls_token (bool): If concatenating class token into image tokens | |
| with_cls_token (bool): If concatenating class token into image tokens |
xvjiarui
reviewed
Apr 30, 2021
mmseg/models/backbones/vit.py
Outdated
|
|
||
| self.blocks = nn.Sequential(*[ | ||
| self.num_stages = depth | ||
| self.out_indices = tuple(range(self.num_stages)) |
Collaborator
There was a problem hiding this comment.
We may make this an argument, default to (12, )
xvjiarui
reviewed
Apr 30, 2021
| self.drop_prob = drop_prob | ||
| self.keep_prob = 1 - drop_prob | ||
|
|
||
| def forward(self, x): |
Collaborator
There was a problem hiding this comment.
tests/test_models/test_utils/test_drop.py
xvjiarui
reviewed
Apr 30, 2021
mmseg/models/backbones/vit.py
Outdated
| x = x[:, 1:] | ||
|
|
||
| outs = [] | ||
| block_len = len(self.blocks) |
Collaborator
There was a problem hiding this comment.
This variable is not needed.
xvjiarui
reviewed
Apr 30, 2021
mmseg/models/backbones/vit.py
Outdated
| block_len = len(self.blocks) | ||
| for i, blk in enumerate(self.blocks): | ||
| x = blk(x) | ||
| if i == block_len - 1: |
Collaborator
There was a problem hiding this comment.
Suggested change
| if i == block_len - 1: | |
| if i == len(self.blocks) - 1: |
* Add unit test for DropPath;
xvjiarui
reviewed
Apr 30, 2021
xvjiarui
reviewed
Apr 30, 2021
xvjiarui
approved these changes
Apr 30, 2021
bowenroom
pushed a commit
to bowenroom/mmsegmentation
that referenced
this pull request
Feb 25, 2022
* Adjust vision transformer backbone architectures; * Add DropPath, trunc_normal_ for VisionTransformer implementation; * Add class token buring intermediate period and remove it during final period; * Fix some parameters loss bug; * * Store intermediate token features and impose no processes on them; * Remove class token and reshape entire token feature from NLC to NCHW; * Fix some doc error * Add a arg for VisionTransformer backbone to control if input class token into transformer; * Add stochastic depth decay rule for DropPath; * * Fix output bug when input_cls_token=False; * Add related unit test; * * Add arg: out_indices to control model output; * Add unit test for DropPath; * Apply suggestions from code review Co-authored-by: Jerry Jiarui XU <xvjiarui0826@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add DropPath, trunc_normal_ for VisionTransformer implementation;
Add arg for VisionTransformer backbone to control if input class token into transformer;
Class token will be removed during output stage;