Skip to content

ONNX: Add support for Style trasfer models, ONNX model zoo#16709

Merged
opencv-pushbot merged 1 commit intoopencv:3.4from
ashishkrshrivastava:cvonnx
Mar 3, 2020
Merged

ONNX: Add support for Style trasfer models, ONNX model zoo#16709
opencv-pushbot merged 1 commit intoopencv:3.4from
ashishkrshrivastava:cvonnx

Conversation

@ashishkrshrivastava
Copy link
Copy Markdown
Contributor

@ashishkrshrivastava ashishkrshrivastava commented Mar 1, 2020

resolves #16707

Merge with extra: opencv/opencv_extra#719

opencv_extra=cvonnx

Pull Request Readiness Checklist

  • I agree to contribute to the project under OpenCV (BSD) License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
  • The PR is proposed to proper branch
  • There is reference to original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

This PR adds :

  1. Gather-Cast subgraph fusion
  2. Mul-Cast subgraph fusion

Detailed Description

Different Pytorch versions create different subgraphs, according to what I have observed till now, the difference is only because of presence or absence of cast layer after Gather and Mul layer. The idea is to fuse Gather-Cast and Mul-Cast . Hence presence or absence of Cast layer wouldn't make any affect on loading of model.

Below attached, three different subgraphs obtained using same code but different pytorch versions. Producer of Fast style transfer models is Pytorch 1.1, which is middle subgraph in attached image . PR #16573 added support for only first subgraph. This PR adds support for all three subgraphs.

Screenshot from 2020-03-01 20-28-02

Merging these changes

would allow loading of all Style trasfer models of ONNX model Zoo which includes:

  • Mosaic
  • Candy
  • Rain Princess
  • Udnie
  • Pointilism
force_builders=Custom,Custom Win,Custom Mac
build_image:Custom=ubuntu-openvino-2020.1.0:16.04
build_image:Custom Win=openvino-2020.1.0
build_image:Custom Mac=openvino-2020.1.0

test_modules:Custom=dnn,python2,python3,java
test_modules:Custom Win=dnn,python2,python3,java
test_modules:Custom Mac=dnn,python2,python3,java

buildworker:Custom=linux-1
# disabled due high memory usage: test_opencl:Custom=ON
test_opencl:Custom=OFF
test_bigdata:Custom=1
test_filter:Custom=*

@ashishkrshrivastava
Copy link
Copy Markdown
Contributor Author

@dkurt, So you want me to remove Gather-Cast related changes and only keep Mul-Cast related changes ?

@dkurt
Copy link
Copy Markdown
Member

dkurt commented Mar 3, 2020

@ashishkrshrivastava, no, please keep both Gather-Cast and Mul-Cast but remove i -= matchedNodesIds.size() - 1; from the core of graphs fusion logic.

@dkurt
Copy link
Copy Markdown
Member

dkurt commented Mar 3, 2020

Please resolve merge conflicts (rebase with updated 3.4)

Copy link
Copy Markdown
Member

@dkurt dkurt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Thank you!

@ashishkrshrivastava
Copy link
Copy Markdown
Contributor Author

Thank you!

Thank you so much 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unable to load Style transfer models from ONNX Model Zoo

4 participants