Added OpenVINO ARM target by l-bat · Pull Request #19632 · opencv/opencv

l-bat · 2021-02-26T08:28:06Z

force_builders=Custom,Custom Win,Custom Mac
build_image:Custom=ubuntu-openvino-2021.2.0:20.04
build_image:Custom Win=openvino-2021.1.0
build_image:Custom Mac=openvino-2021.2.0

test_modules:Custom=dnn,python2,python3,java
test_modules:Custom Win=dnn,python2,python3,java
test_modules:Custom Mac=dnn,python2,python3,java

buildworker:Custom=linux-1
# disabled due high memory usage: test_opencl:Custom=ON
test_opencl:Custom=OFF
test_bigdata:Custom=1
test_filter:Custom=*

alalek · 2021-02-26T08:49:28Z

modules/dnn/include/opencv2/dnn/dnn.hpp

        DNN_TARGET_MYRIAD,
-        DNN_TARGET_FPGA  //!< FPGA device with CPU fallbacks using Inference Engine's Heterogeneous plugin.
+        DNN_TARGET_FPGA,  //!< FPGA device with CPU fallbacks using Inference Engine's Heterogeneous plugin.
+        DNN_TARGET_ARM,


I believe DNN_TARGET_CPU should be reused instead.

Can we collect some extra information through InferenceEngine Query API to detect ARM CPU plugin?

For ARM plugin, we need to skip some tests and layers. If we we are going to use DNN_TARGET_CPU, we need to add extra information (conditions) to do this.

We have such conditions for NCS / NCS2: getInferenceEngineVPUType()

Added getInferenceEngineCPUType()

alalek · 2021-03-17T21:35:16Z

modules/dnn/src/op_inf_engine.cpp

+    AutoLock lock(getInitializationMutex());
+    InferenceEngine::Core& ie = getCore("CPU");
+    const std::vector<std::string> devices = ie.GetAvailableDevices();
+    return std::find(devices.begin(), devices.end(), std::string("CPU")) != devices.end();


@ilya-lavrenov Is there some robust approach to detect plugin type through InferenceEngine QueryAPI?

AFAIK, for example, GNA has software and hardware-accelerated implementations.

@l-bat Please use FULL_DEVICE_NAME and check for "arm_compute::NEON":

https://github.com/openvinotoolkit/openvino_contrib/blob/9acb635162d0ff0cf35cad780f668a9e2b4cdece/modules/arm_plugin/src/arm_plugin.cpp#L187-L188

FULL_DEVICE_NAME for ARM plugin has "arm_compute::NEON". Maybe we can rely on it..

alalek · 2021-03-18T08:14:27Z

modules/dnn/src/op_inf_engine.cpp

+    AutoLock lock(getInitializationMutex());
+    InferenceEngine::Core& ie = getCore("CPU");
+    const std::vector<std::string> devices = ie.GetAvailableDevices();
+    return std::find(devices.begin(), devices.end(), std::string("CPU")) != devices.end();


@l-bat Please use FULL_DEVICE_NAME and check for "arm_compute::NEON":

https://github.com/openvinotoolkit/openvino_contrib/blob/9acb635162d0ff0cf35cad780f668a9e2b4cdece/modules/arm_plugin/src/arm_plugin.cpp#L187-L188

alalek · 2021-03-18T08:18:27Z

modules/dnn/src/op_inf_engine.cpp


+bool isArmPlugin()
+{
+    static bool armPlugin = getInferenceEngineCPUType() == "ARM";


Lets make this check robust.
We do not really need to check for "ARM" CPU type here (we know that in compile time).
We should check for "ARM COMPUTE" plugin instead.

isArmPlugin => isArmComputePlugin
"ARM" => "ARM_COMPUTE"

alalek · 2021-03-18T08:22:01Z

modules/dnn/test/test_onnx_importer.cpp

         backend == DNN_BACKEND_INFERENCE_ENGINE_NN_BUILDER_2019) && target == DNN_TARGET_MYRIAD)
        applyTestTag(CV_TEST_TAG_DNN_SKIP_IE_MYRIAD, CV_TEST_TAG_DNN_SKIP_IE_NN_BUILDER, CV_TEST_TAG_DNN_SKIP_IE_NGRAPH);

+    if (backend == DNN_BACKEND_INFERENCE_ENGINE_NGRAPH && target == DNN_TARGET_CPU && getInferenceEngineCPUType() == "ARM")


== "ARM"

Please use macro defines instead of raw strings:

it is very easy to make a typo in the string which is hard to find

compilers raise error for typos in macro definition names

Is it enough to add the macro 'CV_DNN_INFERENCE_ENGINE_CPU_TYPE_ARM_COMPUTE' or should we add a macro for CPU plugin too to use it in 'getInferenceEngineCPUType()? Do we need a public function 'getInferenceEngineCPUType() or can we create a public isArmComputePlugin()?

Similar to VPU part.

isArmComputePlugin()

Lets keep it private. getInferenceEngineCPUType() is enough

alalek · 2021-03-18T08:23:25Z

modules/dnn/src/layers/convolution_layer.cpp

        auto& ieInpNode = nodes[0].dynamicCast<InfEngineNgraphNode>()->node;
        std::vector<size_t> dims = ieInpNode->get_shape();
-        CV_Assert(dims.size() == 4 || dims.size() == 5);
+        CV_Assert(dims.size() >= 3 && dims.size() <= 5);


Lets emit clear error diagnostic messages:

CV_Check(dims.size(), dims.size() >= 3 && dims.size() <= 5, "");

alalek · 2021-03-18T08:23:45Z

modules/dnn/src/layers/normalize_bbox_layer.cpp

+#if INF_ENGINE_VER_MAJOR_GT(INF_ENGINE_RELEASE_2021_2)
+        auto mul = std::make_shared<ngraph::op::v1::Multiply>(norm, weight, ngraph::op::AutoBroadcastType::NUMPY);
+#else
        auto mul = std::make_shared<ngraph::op::v0::Multiply>(norm, weight, ngraph::op::AutoBroadcastType::NUMPY);
-        return Ptr<BackendNode>(new InfEngineNgraphNode(mul));
+#endif
+            return Ptr<BackendNode>(new InfEngineNgraphNode(mul));


Please fix indentation

alalek

Well done! Thank you 👍

l-bat added the category: dnn label Feb 26, 2021

l-bat added this to the 3.4.14 milestone Feb 26, 2021

alalek reviewed Feb 26, 2021

View reviewed changes

l-bat force-pushed the lb/ie_arm_target branch from 2550252 to 35685e0 Compare February 26, 2021 09:29

l-bat force-pushed the lb/ie_arm_target branch 3 times, most recently from b544505 to a71bda4 Compare March 9, 2021 12:56

l-bat marked this pull request as ready for review March 9, 2021 12:57

l-bat force-pushed the lb/ie_arm_target branch 5 times, most recently from d392728 to 3ab0685 Compare March 10, 2021 09:10

l-bat added 4 commits March 15, 2021 13:08

Added IE ARM target

39193de

Added OpenVINO ARM target

7f5e4a4

Delete ARM target

4cb6358

Detect ARM platform

da04d02

l-bat force-pushed the lb/ie_arm_target branch 4 times, most recently from f25a3bb to ff77c65 Compare March 16, 2021 08:47

Changed device name in ArmPlugin

ff77c65

l-bat requested a review from alalek March 17, 2021 13:24

alalek reviewed Mar 17, 2021

View reviewed changes

alalek reviewed Mar 18, 2021

View reviewed changes

Change ARM detection

c1375dd

alalek approved these changes Mar 20, 2021

View reviewed changes

alalek merged commit c0dd82f into opencv:3.4 Mar 20, 2021

alalek mentioned this pull request Mar 22, 2021

(4.x) Merge 3.4 #19764

Merged

alalek mentioned this pull request Apr 9, 2021

(5.x) Merge 4.x #19885

Merged

Uh oh!

Conversation

l-bat commented Feb 26, 2021 • edited by alalek Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alalek left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

l-bat commented Feb 26, 2021 •

edited by alalek

Loading