Add model maxim for dehaze by yunongLiu1 · Pull Request #82 · opencv/opencv_zoo

yunongLiu1 · 2022-08-15T21:48:20Z

Hi~ below is the info for this pull request:

Contributor: Yunong Liu
Model: MAXIM: Multi-Axis MLP for Image Processing (CVPR 2022 Oral)

Checklist:

replace with ONNX model

fengyuentau

Thank you for the contribution! Please take a look at the comments below.

fengyuentau · 2022-08-19T13:55:31Z

models/dehaze_maxim/demo.py

+from maxim import MAXIM
+import cv2 as cv
+import numpy as np
+from PIL import Image


Please, use opencv for all image processing and display

Thanks for your reply! I have replace all commands that used PIL.Image with opencv function :)

fengyuentau · 2022-08-19T13:59:42Z

models/dehaze_maxim/maxim.py

+
+    def _create_maxim_model(self):
+        print("Loading model....")
+        self.model = ort.InferenceSession(self._modelPath)


Did you try loading this model and inference with opencv? This zoo is intended to provide models work with opencv.

Thanks so much for pointing this out. However, when I tried use cv2.dnn.readNetFromONNX() or cv2.dnn.readNet(). It raise an error below:

cv2.error: OpenCV(4.6.0) /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/onnx/onnx_importer.cpp:1040: error: (-2:Unspecified error) in function 'handleNode'

> Node [Gather@ai.onnx]:(onnx_node!jax2tf_predict_/MAXIM/jit__resize_/GatherV2_2) parse error: OpenCV(4.6.0) /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/onnx/onnx_importer.cpp:2907: error: (-215:Assertion failed) indexMat.total() == 1 in function 'parseGather'
`

Could you be so kind to give me any advice on how to solve it?

Many thanks!

Simplify the model using onnxsim, try to load using OpenCV

If problem persists, have a try with last OpenCV: pip install opencv-python-rolling.

If problem still persists, we will need a fix in the opencv source code.

Thanks a lot for your very fast reply!

The ONNX model I tried before is already simplified. However, in case it's not, I re-run onnxsim for it.

and install the last OpenCV pip install opencv-python-rolling. make sure get the successful message below:

Successfully installed opencv-python-rolling-4.6.0.20220820

After that the same error msg still persists. In case it helps, I attached the full error msg below.

Many thanks again!

[ERROR:0@16.068] global /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/onnx/onnx_importer.cpp (1044) handleNode DNN/ONNX: ERROR during processing node with 2 inputs and 1 outputs: [Gather]:(onnx_node!jax2tf_predict_/MAXIM/jit__resize_/GatherV2_2) from domain='ai.onnx'
Traceback (most recent call last):
File "/Users/lyn/Github/opencv_zoo/models/dehaze_maxim/demo.py", line 21, in
model = MAXIM(modelPath=args.model,
File "/Users/lyn/Github/opencv_zoo/models/dehaze_maxim/maxim.py", line 85, in init
self._create_maxim_model()
File "/Users/lyn/Github/opencv_zoo/models/dehaze_maxim/maxim.py", line 101, in create_maxim_model
self.model = cv.dnn.readNetFromONNX(self.modelPath)
cv2.error: OpenCV(4.6.0-dev) /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/onnx/onnx_importer.cpp:1063: error: (-2:Unspecified error) in function 'handleNode'
Node [Gather@ai.onnx]:(onnx_node!jax2tf_predict/MAXIM/jit__resize/GatherV2_2) parse error: OpenCV(4.6.0-dev) /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/onnx/onnx_importer.cpp:2645: error: (-215:Assertion failed) indexMat.total() == 1 in function 'parseGather'

parse error: OpenCV(4.6.0-dev) /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/onnx/onnx_importer.cpp:2645: error: (-215:Assertion failed) indexMat.total() == 1 in function 'parseGather'

Looks like we have a problem parsing the gather op in opencv. You can try solving it or wait a bit for my fix if possible.

Thanks a lot! And apologize for the late reply, I was a bit busy after term start. I tried to install opencv-python-rolling-5.0.0.20220924 . Unfortunately, the same error persists.

Updates are not forwarded to 5.x yet. So please have a try with latest 4.6: pip install “opencv-python-rolling<5.0”

Thanks a lot for your help! I tried with 4.6. The error mentioned above was solved then.
But, there's a new error appears, which seems like opencv does not support the op 'GreaterOrEqual' currently.

`[ERROR:0@18.681] global /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/onnx/onnx_importer.cpp (1019) handleNode DNN/ONNX: ERROR during processing node with 2 inputs and 1 outputs: [GreaterOrEqual]:(onnx_node!jax2tf_predict_/MAXIM/stage_0_encoder_block_0/channel_attention_block_10/jit_leaky_relu_/GreaterEqual) from domain='ai.onnx'
Traceback (most recent call last):
File "demo.py", line 21, in
model = MAXIM(modelPath=args.model,
File "/Users/lyn/Library/Mobile Documents/com~~apple~~CloudDocs/.Trash/opencv_zoo 3/models/dehaze_maxim/maxim.py", line 85, in init
self._create_maxim_model()
File "/Users/lyn/Library/Mobile Documents/com~~apple~~CloudDocs/.Trash/opencv_zoo 3/models/dehaze_maxim/maxim.py", line 101, in _create_maxim_model
self.model = cv.dnn.readNetFromONNX(self._modelPath)
cv2.error: OpenCV(4.6.0-dev) /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/onnx/onnx_importer.cpp:1038: error: (-2:Unspecified error) in function 'handleNode'

Node [GreaterOrEqual@ai.onnx]:(onnx_node!jax2tf_predict_/MAXIM/stage_0_encoder_block_0/channel_attention_block_10/jit_leaky_relu_/GreaterEqual) parse error: OpenCV(4.6.0-dev) /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/layer_internals.hpp:110: error: (-2:Unspecified error) Can't create layer "onnx_node!jax2tf_predict_/MAXIM/stage_0_encoder_block_0/channel_attention_block_10/jit_leaky_relu_/GreaterEqual" of type "GreaterOrEqual" in function 'getLayerInstance'
`

opencv/opencv#22611 for supporting GreaterOrEqual has been merged. Please have a try after this Saturday. Install with pip install "opencv-python-rolling<5.0", the suffix should be 20221015.

Thanks for reminder :) The new version did solve the previous problem. I encountered another problem then.

[ERROR:0@15.250] global /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/onnx/onnx_importer.cpp (1050) handleNode DNN/ONNX: ERROR during processing node with 2 inputs and 1 outputs: [Gemm]:(onnx_node!Gemm__9265) from domain='ai.onnx'
Traceback (most recent call last):
File "demo.py", line 21, in
model = MAXIM(modelPath=args.model,
File "/Users/lyn/Documents/opencv_zoo_20221022/models/dehaze_maxim/maxim.py", line 85, in init
self._create_maxim_model()
File "/Users/lyn/Documents/opencv_zoo_20221022/models/dehaze_maxim/maxim.py", line 101, in _create_maxim_model
self.model = cv.dnn.readNetFromONNX(self._modelPath)
cv2.error: OpenCV(4.6.0-dev) /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/onnx/onnx_importer.cpp:1069: error: (-2:Unspecified error) in function 'handleNode'
Node [Gemm@ai.onnx]:(onnx_node!Gemm__9265) parse error: OpenCV(4.6.0-dev) /Users/runner/work/opencv-python/opencv-python/opencv/modules/dnn/src/onnx/onnx_importer.cpp:587: error: (-5:Bad argument) Blob Reshape__9264:0 not found in const blobs in function 'getBlob'

i left it until today since it seems not like a problem in the source code and I was distracted by my university courses. Apologize for the delay! Any suggestions at this stage would be helpful!

fengyuentau · 2022-11-28T02:10:32Z

@yunongLiu1 friendly reminder

yunongLiu1 and others added 15 commits August 15, 2022 18:58

Create demo.py

999495c

example images

e70152b

sample output

bff19fc

sample outputs

b72a070

Add LICENSE

baeebc3

add TFLite model

715c72b

renam TFLite model

6d0855e

Update demo.py

9b46f3f

Delete dehazed_maxim_2022Aug.tflite

b963e5f

replace with ONNX model

upload ONNX model

3eaeba0

upload simplified ONNX model

386086c

Update images

0e277e1

Update demo.py maxim.py and README

f8dd597

restruct demo.py

b09151c

rename folder

0fca1a7

yunongLiu1 marked this pull request as ready for review August 15, 2022 21:49

yunongLiu1 added 2 commits August 15, 2022 23:39

Update visualization method

3043205

re-allocate codes for calculate psnr

9bde487

fengyuentau reviewed Aug 19, 2022

View reviewed changes

yunongLiu1 added 2 commits August 20, 2022 15:43

delete PIL.Image

f23b4b6

Revert change in tools/quantize/transformer.py

189e835

fengyuentau self-assigned this Aug 21, 2022

fengyuentau added the add model request to add a new model label Aug 25, 2022

fengyuentau mentioned this pull request Aug 27, 2022

add Gather implementation opencv/opencv#21738

Merged

6 tasks

fengyuentau mentioned this pull request Oct 8, 2022

DNN: need support of the GreaterOrEqual operator from ONNX opencv/opencv#22610

Closed

fengyuentau mentioned this pull request Nov 29, 2022

DNN: gemm support transA and transB, and first input is constance. opencv/opencv#22882

Merged

6 tasks

fengyuentau added the stale Issues or PRs marked as stale will be closed in 7 days label Mar 14, 2023

fengyuentau closed this Jun 6, 2023

Conversation

yunongLiu1 commented Aug 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fengyuentau left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yunongLiu1 Aug 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fengyuentau Oct 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yunongLiu1 Nov 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fengyuentau commented Nov 28, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yunongLiu1 commented Aug 15, 2022 •

edited

Loading

yunongLiu1 Aug 20, 2022 •

edited

Loading

fengyuentau Oct 14, 2022 •

edited

Loading

yunongLiu1 Nov 28, 2022 •

edited

Loading