Skip to content

feat(src/layer): add vision_transformer benchmark#3730

Merged
nihui merged 21 commits intoTencent:masterfrom
tpoisonooo:add-vision-transformer
May 17, 2022
Merged

feat(src/layer): add vision_transformer benchmark#3730
nihui merged 21 commits intoTencent:masterfrom
tpoisonooo:add-vision-transformer

Conversation

@tpoisonooo
Copy link
Copy Markdown
Contributor

@tpoisonooo tpoisonooo commented Apr 24, 2022 •
edited
Loading

支持 vision_transformer。

  • onnx2ncnn.cpp 增加了转换 GELU 和 Crop
  • MultiHeadAttenction opr 加了句注释，冷门 opr 不说来源真的扛不住...
  • benchmark 增加了转好的 ViT.param 。完整的模型已经测过数值对的

速度测试 on 8 核 i5 CPU，330MB 大模型。

vision_transformer min = 3236.20 max = 3541.62 avg = 3403.61

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 24, 2022 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.35%. Comparing base (bf1109c) to head (6b5b942).
Report is 789 commits behind head on master.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #3730      +/-   ##
==========================================
+ Coverage   93.33%   93.35%   +0.01%     
==========================================
  Files         672      674       +2     
  Lines      171577   171980     +403     
==========================================
+ Hits       160146   160556     +410     
+ Misses      11431    11424       -7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tpoisonooo tpoisonooo changed the title [WIP] feat(src/layer): add vision_transformer benchmark and relative layer feat(src/layer): add vision_transformer benchmark and relative layer Apr 25, 2022
@lgtm-com
Copy link
Copy Markdown

lgtm-com Bot commented Apr 25, 2022

This pull request introduces 1 alert when merging eb7d4b2 into f14388a - view on LGTM.com

new alerts:

  • 1 for Multiplication result converted to larger type

@tpoisonooo tpoisonooo changed the title feat(src/layer): add vision_transformer benchmark and relative layer [WIP] feat(src/layer): add vision_transformer benchmark and relative layer Apr 25, 2022
@tpoisonooo tpoisonooo mentioned this pull request Apr 26, 2022
Merged
@tpoisonooo tpoisonooo changed the title [WIP] feat(src/layer): add vision_transformer benchmark and relative layer feat(src/layer): add vision_transformer benchmark and relative layer Apr 26, 2022
@tpoisonooo
Copy link
Copy Markdown
Contributor Author

tpoisonooo commented Apr 26, 2022

@nihui test_squeezenet 的错，是要我重造模型？ 我也没改涉及 sqz 的东西。

@tpoisonooo
Copy link
Copy Markdown
Contributor Author

tpoisonooo commented Apr 26, 2022

@nihui 我没有要改的了。

@tpoisonooo
Copy link
Copy Markdown
Contributor Author

tpoisonooo commented Apr 29, 2022

@nihui 好人给 review 一下呗。

@nihui
Copy link
Copy Markdown
Member

nihui commented May 6, 2022 

InnerProduct     MatMul_28                1 1 onnx::MatMul_180 onnx::Transpose_182 0=3072 1=0 2=2359296
Permute          Transpose_29             1 1 onnx::Transpose_182 onnx::Reshape_183 0=1
Reshape          Reshape_31               1 1 onnx::Reshape_183 onnx::Add_185 0=1 1=-1 2=3072
BinaryOp         Add_32                   2 1 onnx::Add_185 onnx::Add_653 onnx::Reshape_188 0=0
Reshape          Reshape_34               1 1 onnx::Reshape_188 onnx::Transpose_190 0=-1 1=3072
Permute          Transpose_35             1 1 onnx::Transpose_190 mmdeploy::Gelu_191 0=1

这套东西，其实就是一个 nn.Linear ，被onnx拆散了 😆

@tpoisonooo tpoisonooo mentioned this pull request May 7, 2022
Closed
@tpoisonooo
Copy link
Copy Markdown
Contributor Author

tpoisonooo commented May 7, 2022

@nihui 改好啦，扔掉了 gather； ViT.param 也小了很多。

@tpoisonooo tpoisonooo changed the title feat(src/layer): add vision_transformer benchmark and relative layer feat(src/layer): add vision_transformer benchmark May 16, 2022
@tpoisonooo tpoisonooo mentioned this pull request May 16, 2022
Merged
@nihui nihui merged commit 6fd801b into Tencent:master May 17, 2022
@nihui
Copy link
Copy Markdown
Member

nihui commented May 17, 2022

Thanks for your contribution !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tpoisonooo @codecov-commenter @nihui