Skip to content

OpenCV does not support ONNX models with dynamic input/output shape #18072

@wenqingzhang-gordon

Description

@wenqingzhang-gordon
System information (version)
  • OpenCV => 4.4.0-pre
  • Operating System / Platform => Ubuntu 18.04
  • Compiler => qmake (Qt5)
Detailed description
  1. The current version of OpenCV does not support ONNX models with dynamic input/output shape

  2. Bug Report:
    Suppose that an ONNX model has a fixed input shape ( H: 736, W: 1280 )
    Then, I input an image with a wrong shape ( H: 1280, W: 736 )
    However, OpenCV would not throw an exception, and the output is not right.

Steps to reproduce
  1. I provide an ONNX model with a dynamic input shape:
    https://drive.google.com/file/d/1ihzycC2R_HzUAH941P_ghxA1eDjezy6c/view?usp=sharing
    It is generated by the following codes:
    output_onnx = 'DB_dynamic_input.onnx'
    print("==> Exporting model to ONNX format at '{}'".format(output_onnx))
    input_names = ["input"]
    output_names = ["out"]
    inputs = torch.randn((1, 3, 576, 800), dtype=torch.float32)
    dynamic_axes = {'input': {0: 'batch_size', 2: 'height', 3: 'width'}, 'out': {0: 'batch', 1: 'height', 2: 'width'}}
    torch_out = torch.onnx.export(model, inputs, output_onnx, export_params=True, verbose=True,
                                  input_names=input_names, output_names=output_names,
                                  opset_version=11,
                                  dynamic_axes=dynamic_axes)

Then, I test it with onnxruntime

    onnx_path = 'DB_dynamic_input.onnx'
    session = onnxruntime.InferenceSession(onnx_path)
    session.get_modelmeta()
    input_name = session.get_inputs()[0].name
    output_name = session.get_outputs()[0].name
    inputs = torch.randn((4, 3, 736, 1280), dtype=torch.float32).numpy()
    preds = session.run([output_name], {input_name: inputs})
    print(preds[0].shape)

and get an output with a right shape

(4, 736, 1280)

When I try to import it in OpenCV, I will get the following Exception:

terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.4.0-pre) /home/hannibal/Files/Tools/opencv/modules/dnn/src/layers/pooling_layer.cpp:1186: error: (-215:Assertion failed) (outShape[2 + i] - 1) * strides[i] < inpShape[i] + pads_end[i] in function 'getMemoryShapes'
  1. This bug can be reproduced by using the DB_IC15_resnet50.onnx provided in my tutorial:
    https://github.com/HannibalAPE/opencv/blob/text_det_recog_demo/doc/tutorials/dnn/dnn_text_spotting/dnn_text_spotting.markdown
    The input shape of it is fixed, when it is converted to ONNX format.
    The right input shape should be (1, 3, 736, 1280), but you can input a blob with a wrong shape (1, 3, 1280, 736).
Issue submission checklist
  • I report the issue, it's not a question
  • I checked the problem with documentation, FAQ, open issues,
    answers.opencv.org, Stack Overflow, etc and have not found solution
  • I updated to latest OpenCV version and the issue is still there
  • There is reproducer code and related data files: videos, images, onnx, etc

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions