Issue when converting Cascade Mask R-CNN to ONNX #768
Description
I am facing an error when I try to convert my mmdet Cascade Mask R-CNN model to ONNX. My Cascade R-CNN model is trained on my own dataset of 26 classes (I modified the mmdet config file accordingly). It looks like the issue happens with the creation of batch_index in the mmdeploy/codebase/mmdet/models/roi_heads/cascade_roi_head.py code, there is a issue in the tensor shape.
Could you please help me ? Thank you for your work !
The command that I ran :
python ./mmdeploy/tools/deploy.py \
./mmdeploy/configs/mmdet/instance-seg/instance-seg_onnxruntime_dynamic.py \
./mmdetection/configs/cascade_rcnn/cascade_mask_rcnn_inference.py \
./mmdetection/out_dir/cascade_mask_rcnn//best_bbox_mAP_epoch_9.pth \
./img.jpg \
--work-dir ./mmdeploy_models/cascade/ \
--device cuda:0 \
--dump-info
Environment:
2022-07-19 11:31:43,024 - mmdeploy - INFO -
2022-07-19 11:31:43,024 - mmdeploy - INFO - **********Environmental information**********
2022-07-19 11:31:43,811 - mmdeploy - INFO - sys.platform: linux
2022-07-19 11:31:43,812 - mmdeploy - INFO - Python: 3.8.8 (default, Feb 24 2021, 21:46:12) [GCC 7.3.0]
2022-07-19 11:31:43,812 - mmdeploy - INFO - CUDA available: True
2022-07-19 11:31:43,812 - mmdeploy - INFO - GPU 0: NVIDIA TITAN RTX
2022-07-19 11:31:43,812 - mmdeploy - INFO - GPU 1: NVIDIA GeForce GTX 1080 Ti
2022-07-19 11:31:43,812 - mmdeploy - INFO - CUDA_HOME: /usr/local/cuda
2022-07-19 11:31:43,812 - mmdeploy - INFO - NVCC: Build cuda_11.1.TC455_06.29190527_0
2022-07-19 11:31:43,812 - mmdeploy - INFO - GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
2022-07-19 11:31:43,812 - mmdeploy - INFO - PyTorch: 1.8.0
2022-07-19 11:31:43,812 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:
- GCC 7.3
- C++ Version: 201402
- Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.1
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
- CuDNN 8.0.5
- Magma 2.5.2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,
2022-07-19 11:31:43,812 - mmdeploy - INFO - TorchVision: 0.9.0
2022-07-19 11:31:43,812 - mmdeploy - INFO - OpenCV: 4.6.0
2022-07-19 11:31:43,812 - mmdeploy - INFO - MMCV: 1.4.0
2022-07-19 11:31:43,812 - mmdeploy - INFO - MMCV Compiler: GCC 7.3
2022-07-19 11:31:43,812 - mmdeploy - INFO - MMCV CUDA Compiler: 11.1
2022-07-19 11:31:43,812 - mmdeploy - INFO - MMDeploy: 0.6.0+unknown
2022-07-19 11:31:43,812 - mmdeploy - INFO -
2022-07-19 11:31:43,812 - mmdeploy - INFO - **********Backend information**********
2022-07-19 11:31:44,101 - mmdeploy - INFO - onnxruntime: 1.8.1 ops_is_avaliable : True
2022-07-19 11:31:44,103 - mmdeploy - INFO - tensorrt: None ops_is_avaliable : False
2022-07-19 11:31:44,112 - mmdeploy - INFO - ncnn: None ops_is_avaliable : False
2022-07-19 11:31:44,112 - mmdeploy - INFO - pplnn_is_avaliable: False
2022-07-19 11:31:44,113 - mmdeploy - INFO - openvino_is_avaliable: False
2022-07-19 11:31:44,113 - mmdeploy - INFO -
2022-07-19 11:31:44,113 - mmdeploy - INFO - **********Codebase information**********
2022-07-19 11:31:44,114 - mmdeploy - INFO - mmdet: 2.24.1
2022-07-19 11:31:44,114 - mmdeploy - INFO - mmseg: None
2022-07-19 11:31:44,114 - mmdeploy - INFO - mmcls: None
2022-07-19 11:31:44,114 - mmdeploy - INFO - mmocr: None
2022-07-19 11:31:44,114 - mmdeploy - INFO - mmedit: None
2022-07-19 11:31:44,114 - mmdeploy - INFO - mmdet3d: None
2022-07-19 11:31:44,114 - mmdeploy - INFO - mmpose: None
2022-07-19 11:31:44,114 - mmdeploy - INFO - mmrotate: None
Error traceback :
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
ret = func(*args, **kwargs)
File "/workspace/mmdeploy/mmdeploy/apis/pytorch2onnx.py", line 92, in torch2onnx
export(
File "/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 356, in _wrap
return self.call_function(func_name_, *args, **kwargs)
File "/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 326, in call_function
return self.call_function_local(func_name, *args, **kwargs)
File "/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 275, in call_function_local
return pipe_caller(*args, **kwargs)
File "/workspace/mmdeploy/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
ret = func(*args, **kwargs)
File "/workspace/mmdeploy/mmdeploy/apis/onnx/export.py", line 122, in export
torch.onnx.export(
File "/opt/conda/lib/python3.8/site-packages/torch/onnx/__init__.py", line 271, in export
return utils.export(model, args, f, export_params, verbose, training,
File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 88, in export
_export(model, args, f, export_params, verbose, training, input_names, output_names,
File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 691, in _export
_model_to_graph(model, args, verbose, input_names,
File "/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper
return self.func(self, *args, **kwargs)
File "/workspace/mmdeploy/mmdeploy/apis/onnx/optimizer.py", line 10, in model_to_graph__custom_optimizer
graph, params_dict, torch_out = ctx.origin_func(*args, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 454, in _model_to_graph
graph, params, torch_out, module = _create_jit_graph(model, args,
File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 417, in _create_jit_graph
graph, torch_out = _trace_and_get_graph_from_model(model, args)
File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 377, in _trace_and_get_graph_from_model
torch.jit._get_trace_graph(model, args, strict=False, _force_outplace=False, _return_inputs_states=True)
File "/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py", line 1139, in _get_trace_graph
outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py", line 125, in forward
graph, out = torch._C._create_graph_by_tracing(
File "/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py", line 116, in wrapper
outs.append(self.inner(*trace_inputs))
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 887, in _call_impl
result = self._slow_forward(*input, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 860, in _slow_forward
result = self.forward(*input, **kwargs)
File "/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper
return self.func(self, *args, **kwargs)
File "/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base.py", line 70, in base_detector__forward
return __forward_impl(ctx, self, img, img_metas=img_metas, **kwargs)
File "/workspace/mmdeploy/mmdeploy/core/optimizers/function_marker.py", line 261, in g
rets = f(*args, **kwargs)
File "/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/base.py", line 26, in __forward_impl
return self.simple_test(img, img_metas, **kwargs)
File "/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper
return self.func(self, *args, **kwargs)
File "/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/two_stage.py", line 59, in two_stage_detector__simple_test
return self.roi_head.simple_test(x, proposals, img_metas, rescale=False)
File "/workspace/mmdeploy/mmdeploy/core/rewriters/rewriter_utils.py", line 371, in wrapper
return self.func(self, *args, **kwargs)
File "/workspace/mmdeploy/mmdeploy/codebase/mmdet/models/roi_heads/cascade_roi_head.py", line 90, in cascade_roi_head__simple_test
mask_rois = torch.cat([batch_index, rois], dim=-1)
RuntimeError: Sizes of tensors must match except in dimension 1. Got 25 and 1 (The offending index is 0)
2022-07-19 11:41:58,842 - mmdeploy - ERROR - `mmdeploy.apis.pytorch2onnx.torch2onnx` with Call id: 0 failed. exit.
Metadata
Metadata
Assignees
Labels
No labels