Skip to content

[WIP][feature] Support Autoformer supernet training and search#288

Closed
aptsunny wants to merge 9 commits intoopen-mmlab:dev-1.xfrom
aptsunny:autoformer_algo
Closed

[WIP][feature] Support Autoformer supernet training and search#288
aptsunny wants to merge 9 commits intoopen-mmlab:dev-1.xfrom
aptsunny:autoformer_algo

Conversation

@aptsunny
Copy link
Copy Markdown
Contributor

@aptsunny aptsunny commented Sep 13, 2022

Description

Modified based on open-mmlab/mmrazor/dev-1.x and pprp/mmrazor/autoformer_dev branch.

Add subnet and supernet weights from gml for autoformer. Add interface for search config.

Modification

  1. Amend dataloader sample adapt randaug.
  2. Support load autoformer supernet checkpoint in search loop.
  3. Continued improvements of candidate.

Issues

  1. Training the supernet need 32 gpus. cost too much time.
  2. Convert gml ckpt to mmrazor, can see the search.log achieve the 83.39 with the params 65.285M.
  3. Training the supernet is easy to overfitting, so we early stopping when get to 300th epoch almost.
2022/09/10 16:04:15 - mmengine - INFO - Epoch(train) [384][300/626]  lr: 3.5416e-04  eta: 19:09:28  time: 0.9363  data_time: 0.0374  memory: 14791  grad_norm: 0.9041  loss: 2.2315
2022/09/10 16:05:49 - mmengine - INFO - Epoch(train) [384][400/626]  lr: 3.5341e-04  eta: 19:07:53  time: 0.9818  data_time: 0.0458  memory: 14748  grad_norm: 0.8975  loss: 2.1782
2022/09/10 16:07:23 - mmengine - INFO - Epoch(train) [384][500/626]  lr: 3.5265e-04  eta: 19:06:19  time: 0.9532  data_time: 0.0280  memory: 14875  grad_norm: 0.8923  loss: 2.2506
2022/09/10 16:08:56 - mmengine - INFO - Epoch(train) [384][600/626]  lr: 3.5190e-04  eta: 19:04:44  time: 0.8982  data_time: 0.0234  memory: 14963  grad_norm: 0.9125  loss: 2.4638
2022/09/10 16:09:20 - mmengine - INFO - Exp name: autoformer_supernet_32xb256_in1k_20220907_220356
2022/09/10 16:09:20 - mmengine - INFO - Saving checkpoint at 384 epochs
2022/09/10 16:09:42 - mmengine - INFO - Epoch(val) [384][7/7]  accuracy/top1: 82.5160  accuracy/top5: 96.0540

2022/09/12 11:09:54 - mmengine - INFO - top k scores before update: [82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969]
2022/09/12 11:09:54 - mmengine - INFO - top k scores after update: [82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969, 82.48600769042969]
2022/09/12 11:10:06 - mmengine - INFO - Epoch:[20/20], top1_score: 82.48600769042969

TODO

  • Add Autoformer readme .
  • Upload ckpt and log.

@aptsunny aptsunny mentioned this pull request Oct 10, 2022
Closed
1 task
@aptsunny aptsunny closed this Oct 11, 2022
This was referenced Oct 11, 2022
Merged
Merged
humu789 pushed a commit to humu789/mmrazor that referenced this pull request Feb 13, 2023
@lvhan028
Update README (open-mmlab#288)
1ed55f8 
* update README and add three blank docs that are going to present SDK

* check in mmdeploy's logo image

* remove comments in README

* update acknowledgement

* development->deployment

* add Human3D link

* use captical word in citation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sunnyxiaohu sunnyxiaohu Awaiting requested review from sunnyxiaohu

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@aptsunny