Skip to content

dnn : Fix convolution asymmetric padding bug in onnx importer#20394

Merged
opencv-pushbot merged 1 commit intoopencv:3.4from
jebastin-nadar:conv-asymmetric-pads
Jul 16, 2021
Merged

dnn : Fix convolution asymmetric padding bug in onnx importer#20394
opencv-pushbot merged 1 commit intoopencv:3.4from
jebastin-nadar:conv-asymmetric-pads

Conversation

@jebastin-nadar
Copy link
Copy Markdown
Contributor

@jebastin-nadar jebastin-nadar commented Jul 12, 2021

Merge with extra : opencv/opencv_extra#881

relates #19316 #19499 #19696 #19957

Using tools like tf2onnx and keras2onnx, to convert tensorflow networks to onnx format, fuses zero-padding layer with convolution layer but does not check if the padding is symmetric when importing such converted networks.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
  • The PR is proposed to proper branch
  • There is reference to original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake
force_builders=Custom,Custom Win,Custom Mac
build_image:Custom=ubuntu-openvino-2021.4.0:20.04
build_image:Custom Win=openvino-2021.4.0
build_image:Custom Mac=openvino-2021.4.0

test_modules:Custom=dnn,python2,python3,java
test_modules:Custom Win=dnn,python2,python3,java
test_modules:Custom Mac=dnn,python2,python3,java

buildworker:Custom=linux-1
# disabled due high memory usage: test_opencl:Custom=ON
test_opencl:Custom=ON
test_bigdata:Custom=1
test_filter:Custom=*Conv*

@asmorkalov asmorkalov requested a review from rogday July 12, 2021 16:17
Copy link
Copy Markdown
Member

@alalek alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for contribution!

TEST_P(Test_ONNX_layers, Convolution)
{
testONNXModels("convolution");
testONNXModels("conv_asymmetric_pads");
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to provide ONNX test data generator code (generate_onnx_models.py in opencv_extra PR) ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I don't think its possible. torch.onnx.export() does not fuses padding with conv layer which is required to test this change.
As mentioned in description, tools like tf2onnx and keras2onnx use this type of fusion (onnx/tensorflow-onnx#334). The onnx model in opencv_extra has been generated using tf2onnx.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The onnx model in opencv_extra has been generated using tf2onnx.

tf2onnx has python interface, would it be possible to use that?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tf2onnx has python interface

Yes it can be used, but the original network is built using tensorflow. generate_onnx_models.py uses Pytorch.
Code used to generate onnx model

import tensorflow as tf
import tf2onnx
tf.keras.backend.set_image_data_format('channels_first')

inp = tf.keras.layers.Input(batch_shape = (1, 2, 3, 4))
pad = tf.keras.layers.ZeroPadding2D(padding = ((2, 1), (2, 1)))(inp)
out = tf.keras.layers.Conv2D(3, kernel_size = 3)(pad)
model = tf.keras.Model(inp, out)
onnx_model, _ = tf2onnx.convert.from_keras(model, output_path="/conv_asymmetric_pads.onnx")

Copy link
Copy Markdown
Member

@alalek alalek Jul 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it makes sense to create new file like generate_onnx_tf.py (or tf2 to mention TensorFlow 2.x).

BTW, Old file can be renamed to generate_onnx_pytorch.py separately.

Update: removed _testdata from names.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Should I add these 2 changes in opencv_extra PR?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to create a new file in opencv_extra.
It would be nice if you can prepare requirements-tf.txt file too (list required pip packages there, pip freeze may help with that, example)

Copy link
Copy Markdown
Member

@alalek alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 👍
Thank you for contribution!

@jebastin-nadar
Copy link
Copy Markdown
Contributor Author

Squashed commits into one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants