Crash in net.forward() using inference engine on Raspberry Pi 4 and Intel NCS2 #17954
Description
System Information
- OpenCV: 4.4.0-openvino
- Openvino: 2020.4
- Operating System: Raspberry Pi OS May 2020
- GCC: gcc (Raspbian 8.3.0-6+rpi1) 8.3.0
Background
I am getting issues running a Darknet Yolov3-Tiny model with the OpenCV inference engine enhanced by the Intel Neural Compute Stick 2. I use OpenCV in CPU only mode it runs fine but once I turn on the inference engine backend it crashes.
The same application code and model ran fine on the following combination of versions:
- OpenCV 4.3.0-Openvino-2020.3.0
- Openvino 2020.3
The reason I want to upgrade is to leverage some of the Yolo optimisations in OpenCV 4.4.
Issue
My application is written in Python 3.7 and when calling the net.forward() I get the following error:
terminate called after throwing an instance of 'ngraph::NodeValidationFailure'
what(): Check 'is_valid_permutation(permutation, arg_shape.rank())' failed at /home/pi/openvino/ngraph/src/ngraph/op/transpose.cpp:65:
While validating node 'v1::Transpose Transpose_202(Add_200[0]:f32{1,72,13,13}, Constant_201[0]:i64{4}) -> (dynamic?)':
Permutation AxisVector{0, 3, 336, 0} is not valid for input shape {1,72,13,13}
or
terminate called after throwing an instance of 'ngraph::NodeValidationFailure'
what(): Check 'is_valid_permutation(permutation, arg_shape.rank())' failed at /home/pi/openvino/ngraph/src/ngraph/op/transpose.cpp:65:
While validating node 'v1::Transpose Transpose_202(Add_200[0]:f32{1,72,13,13}, Constant_201[0]:i64{4}) -> (dynamic?)':
Permutation AxisVector{0, 3, 104, 1} is not valid for input shape {1,72,13,13}
Reproduce
This is being built on target. i.e. on the Raspberry Pi 4.
- Checkout versions of repos from first section of this issue. Assume both
opencvandopenvinoare in$HOME apt-getall prerequisites- Increase swap partition to 2048
- Build OpenVino with following cmake config:
-D CMAKE_BUILD_TYPE=Release
-D ENABLE_SSE42=OFF
-D THREADING=SEQ
-D ENABLE_GNA=OFF
-D ENABLE_OPENCV=OFF
-D ENABLE_PYTHON=ON
-D CMAKE_CXX_FLAGS="-march=armv7-a -latomic"
-D PYTHON_EXECUTABLE=/usr/bin/python3
-D PYTHON_LIBRARY=/usr/lib/arm-linux-gnueabihf/libpython3.7m.so
-D ENABLE_MKL_DNN=OFF
-D ENABLE_CLDNN=OFF
-D PYTHON_INCLUDE_DIR=/usr/include/python3.7m- "Install" openvino by setting paths (this is so OpenCV can find the inference engine when building)
export LD_LIBRARY_PATH=/usr/local/lib:$HOME/openvino/bin/armv7l/Release/lib
export INTEL_OPENVINO_DIR=$HOME/openvino
export INTEL_CVSDK_DIR=$HOME/openvino
export InferenceEngine_DIR=$HOME/openvino/build
export ngraph_DIR=$HOME/openvino/build/ngraph
- Build OpenCV (with contrib modules) using the following cmake config:
# Default from https://github.com/opencv/opencv/wiki/Intel's-Deep-Learning-Inference-Engine-backend
-D CMAKE_BUILD_TYPE=RELEASE
-D CMAKE_INSTALL_PREFIX=/usr/local
-D ENABLE_NEON=ON
-D CPU_BASELINE="NEON"
-D ENABLE_VFPV3=ON
-D ENABLE_CXX11=ON
# Match flags from Openvino
-D CMAKE_EXE_LINKER_FLAGS="-latomic"
-D CMAKE_CXX_FLAGS="-march=armv7-a -latomic"
# Disable unused examples and tests
-D BUILD_EXAMPLES=OFF
-D INSTALL_PYTHON_EXAMPLES=OFF
-D BUILD_PERF_TESTS=OFF
-D BUILD_TESTS=OFF
-D WITH_IPP=OFF
# Additional Modules
-D OPENCV_EXTRA_MODULES_PATH="$HOME/opencv/opencv_contrib/modules"
-D OPENCV_ENABLE_NONFREE=ON
# Python Paths
-D PYTHON2_INCLUDE_PATH=/usr/include/python2.7
-D PYTHON2_NUMPY_INCLUDE_DIRS="$HOME/.local/lib/python2.7/site-packages/numpy/core/include"
-D PYTHON3_INCLUDE_PATH=/usr/include/python3.7m
-D PYTHON3_NUMPY_INCLUDE_DIRS="$HOME/.local/lib/python3.7/dist-packages/numpy/core/include"
-D PYTHON3_CVPY_SUFFIX=.cpython-37m-arm-linux-gnueabihf.so
-D BUILD_opencv_python3=yes
# Inference Engine Paths
-D WITH_INF_ENGINE=ON
-D INF_ENGINE_RELEASE="2020040000"
-D INF_ENGINE_LIB_DIRS="$HOME/openvino/bin/armv7l/Release/lib"
-D INF_ENGINE_INCLUDE_DIRS="$HOME/openvino/inference-engine/include"
-D CMAKE_FIND_ROOT_PATH="$HOME/openvino/"
-D ENABLE_PRECOMPILED_HEADERS=OFF # https://github.com/opencv/opencv/issues/15278#issuecomment-522219507- Make, make install and ldconfig
- Set up USB rules: https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_raspbian.html
The python script simplified is
dnn_net = cv2.dnn.readNet(darknet_cfg_file, darknet_weights_file)
dnn_net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE)
dnn_net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD)
# As per OpenCV example defaults
blob = cv2.dnn.blobFromImage( cv2.resize(frame, (416, 416)),
1 / 255, (416, 416), (10, 20, 30), swapRB=True, crop=False)
self.dnn_net.setInput(blob)
out = self.dnn_net.forward()
# Crash here