inference with onnx and opencv gives different results #24044
Description
System Information
General configuration for OpenCV 4.8.0-dev =====================================
Version control: 4.8.0-64-g1f7025f028-dirty
Extra modules:
Location (extra): C:/lib/opencv_contrib/modules
Version control (extra): 4.8.0-4-gd89b2b9b
Platform:
Timestamp: 2023-06-29T16:39:13Z
Host: Windows 10.0.22621 AMD64
CMake: 3.26.1
CMake generator: Visual Studio 17 2022
CMake build tool: C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe
MSVC: 1935
Configuration: Debug Release
CPU/HW features:
Baseline: SSE SSE2 SSE3
requested: SSE3
Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
requested: SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
SSE4_1 (18 files): + SSSE3 SSE4_1
SSE4_2 (2 files): + SSSE3 SSE4_1 POPCNT SSE4_2
FP16 (1 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
AVX (8 files): + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
AVX2 (37 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
AVX512_SKX (8 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX
C/C++:
Built as dynamic libs?: YES
C++ standard: 11
C++ Compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe (ver 19.35.32215.0)
C++ flags (Release): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP /MD /O2 /Ob2 /DNDEBUG
C++ flags (Debug): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /wd4819 /MP /MDd /Zi /Ob0 /Od /RTC1
C Compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.35.32215/bin/Hostx64/x64/cl.exe
C flags (Release): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /MP /MD /O2 /Ob2 /DNDEBUG
C flags (Debug): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:precise /MP /MDd /Zi /Ob0 /Od /RTC1
Linker flags (Release): /machine:x64 /INCREMENTAL:NO
Linker flags (Debug): /machine:x64 /debug /INCREMENTAL
ccache: NO
Precompiled headers: YES
Extra dependencies: cudart_static.lib nppc.lib nppial.lib nppicc.lib nppidei.lib nppif.lib nppig.lib nppim.lib nppist.lib nppisu.lib nppitc.lib npps.lib cublas.lib cudnn.lib cufft.lib -LIBPATH:C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.1/lib/x64
3rdparty dependencies:
OpenCV modules:
To be built: alphamat aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform java line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python3 quality rapid reg rgbd saliency sfm shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab viz wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
Disabled: world
Disabled by dependency: -
Unavailable: cvv freetype hdf julia matlab ovis python2
Applications: tests perf_tests examples apps
Documentation: doxygen python javadoc
Non-free algorithms: YES
Windows RT support: NO
GUI: WIN32UI
Win32 UI: YES
OpenGL support: YES (opengl32 glu32)
VTK support: YES (ver 9.2.5)
Media I/O:
ZLib: optimized C:/install/zlib/lib/zlib.lib debug C:/install/zlib/lib/zlibd.lib (ver 1.2.13)
JPEG: build-libjpeg-turbo (ver 2.1.3-62)
SIMD Support Request: YES
SIMD Support: NO
WEBP: build (ver encoder: 0x020f)
PNG: optimized C:/install/libpng/lib/libpng16.lib debug C:/install/libpng/lib/libpng16d.lib (ver 1.6.40)
TIFF: build (ver 42 - 4.2.0)
JPEG 2000: build (ver 2.5.0)
OpenEXR: build (ver 2.3.0)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES
Video I/O:
DC1394: NO
FFMPEG: YES (prebuilt binaries)
avcodec: YES (58.134.100)
avformat: YES (58.76.100)
avutil: YES (56.70.100)
swscale: YES (5.9.100)
avresample: YES (4.0.0)
GStreamer: NO
DirectShow: YES
Media Foundation: YES
DXVA: YES
Parallel framework: Concurrency
Other third-party libraries:
Intel IPP: 2021.8 [2021.8.0]
at: C:/lib/build/opencv/3rdparty/ippicv/ippicv_win/icv
Intel IPP IW: sources (2021.8.0)
at: C:/lib/build/opencv/3rdparty/ippicv/ippicv_win/iw
Lapack: YES (C:/Program Files (x86)/Intel/oneAPI/mkl/2023.0.0/lib/intel64/mkl_intel_lp64.lib C:/Program Files (x86)/Intel/oneAPI/mkl/2023.0.0/lib/intel64/mkl_sequential.lib C:/Program Files (x86)/Intel/oneAPI/mkl/2023.0.0/lib/intel64/mkl_core.lib)
OpenVINO: YES (2022.3.0)
Eigen: YES (ver ..)
Custom HAL: NO
Protobuf: build (3.19.1)
Flatbuffers: builtin/3rdparty (23.5.9)
NVIDIA CUDA: YES (ver 12.1, CUFFT CUBLAS)
NVIDIA GPU arch: 86
NVIDIA PTX archs:
cuDNN: YES (ver 8.8.0)
OpenCL: YES (NVD3D11)
Include path: C:/lib/opencv/3rdparty/include/opencl/1.2
Link libraries: Dynamic load
Python 3:
Interpreter: C:/Program Files/Python310/python.exe (ver 3.10.10)
Libraries: optimized C:/Program Files/Python310/libs/python310.lib debug C:/Program Files/Python310/libs/python310_d.lib (ver 3.10.10)
numpy: C:/Users/laurent/AppData/Roaming/Python/Python310/site-packages/numpy/core/include (ver 1.23.5)
install path: C:/Users/laurent/AppData/Roaming/Python/Python310/site-packages/cv2/python-3.10
Python (for build): C:/Program Files/Python310/python.exe
Java:
ant: C:/apache-ant-1.10.13/bin/ant.bat (ver 1.10.13)
Java: NO
JNI: C:/Program Files/Java/jdk-19/include C:/Program Files/Java/jdk-19/include/win32 C:/Program Files/Java/jdk-19/include
Java wrappers: YES (ANT)
Java tests: YES
Install to: C:/install/opencv
-----------------------------------------------------------------
Detailed description
pytorch and onnx gives same results
opencv does not give good results
Results
ONNX RESULT
[[4.446731 4.4490666 4.46463 4.4546375 4.4510665 4.456948 ]
[4.4421244 4.4491835 4.4703193 4.460532 4.4576974 4.462741 ]
[4.440228 4.4505563 4.4774194 4.4691973 4.4677935 4.470848 ]
[4.435331 4.4522853 4.484742 4.480783 4.480365 4.4803877]
[4.4323177 4.4522853 4.4912124 4.4923916 4.4934096 4.4903293]]
OPENCV
[[5.1145616 5.1145616 5.1145616 5.1145616 5.1145616 5.1145616]
[5.115294 5.115294 5.115294 5.115294 5.115294 5.115294 ]
[5.116197 5.116197 5.116197 5.116197 5.116197 5.116197 ]
[5.1162567 5.1162567 5.1162567 5.1162567 5.1162567 5.1162567]
[5.115356 5.115356 5.115356 5.115356 5.115356 5.115356 ]]
0.84839463
7.712089
Steps to reproduce
simplified model can be loaded here
image is here
import onnx
import onnxruntime as rt
import numpy as np
import cv2 as cv
onnx_name = "AdaBins_kitti_sim.onnx"
image = cv.imread(cv.samples.findFile("classroom__rgb_00283.jpg"))
# DATA for ONNX and OPENCV
blob = np.transpose(image.astype(np.float32), [2, 0, 1])/255
blob = blob.reshape((1,3, 480, 640))
# ONNX inference
sess = rt.InferenceSession(onnx_name)
input_name = sess.get_inputs()[0].name
output_name = sess.get_outputs()[0].name
predonnx = sess.run([sess.get_outputs()[0].name,
sess.get_outputs()[1].name,
],
{input_name: blob})
print("ONNX RESULT")
disparity_onnx = predonnx[1][0, 0, : ,:]
bins_onnx = predonnx[0]
print(disparity_onnx[100:105,25:43:3])
# OPENCV inference
net = cv.dnn.readNet(onnx_name)
net.setInput(blob)
pred_opencv = net.forward([sess.get_outputs()[0].name,
sess.get_outputs()[1].name,
])
print("OPENCV")
disparity_opencv = pred_opencv[1][0, 0, : ,:]
bins_opencv = pred_opencv[0]
print(disparity_opencv[100:105,25:43:3])
print(np.mean((disparity_onnx-disparity_opencv)**2))
print(np.max((disparity_onnx-disparity_opencv)**2))
Issue submission checklist
- I report the issue, it's not a question
- I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
- I updated to the latest OpenCV version and the issue is still there
- There is reproducer code and related data files (videos, images, onnx, etc)