System Information

OpenCV version: 4.8.0
Operating System / Platform: Windows 11
Compiler & compiler version: VS 2022 (17.6.3)
Python version: 3.10.11

Detailed description

OpenCV supports OpenVINO as a backend to speedup DNNs at inference time, which requires to compile with OpenVINO support. Excerpt from the build config:

General configuration for OpenCV 4.8.0 =====================================
  ...
  Parallel framework:            TBB (ver 2021.9 interface 12090)
  ...
  Other third-party libraries:
    OpenVINO:                    YES (2022.3.0)

So OpenVINO is available. Thereafter, the documentation states

If OpenCV is compiled with Intel's Inference Engine library, DNN_BACKEND_DEFAULT
means DNN_BACKEND_INFERENCE_ENGINE. Otherwise it equals to DNN_BACKEND_OPENCV.

To us, this reads as that by default OpenVINO / DNN_BACKEND_INFERENCE_ENGINE should be used by default in this build configuration. However, performing some tests using a ResNet model from the model zoo shows a different image:

# test script:
import os
import time
import numpy as np
import cv2 as cv

# download link: https://github.com/opencv/opencv_zoo/tree/main/models/image_classification_ppresnet
model = cv.dnn.readNet(os.path.join(os.path.dirname(__file__), 'image_classification_ppresnet50_2022jan.onnx'))

# optionally select a backend
#model.setPreferableBackend(cv.dnn.DNN_BACKEND_INFERENCE_ENGINE)
#model.setPreferableBackend(cv.dnn.DNN_BACKEND_OPENCV)
#model.setPreferableBackend(cv.dnn.DNN_BACKEND_DEFAULT)

# test prediction time
model.setInput(cv.dnn.blobFromImage(np.ones((224,224,3), dtype=np.float32)))
for i in range(5): # warmup
    out = model.forward()
# test 100 predictions
t1 = time.time()
for i in range(100):
    out = model.forward()

t2 = time.time()
print(t2-t1)
input()

Running this script this way (i.e. no explicit backend) or setting either cv.dnn.DNN_BACKEND_OPENCV or cv.dnn.DNN_BACKEND_DEFAULT requires about 15-16 seconds on the testing machine (i7-10510U CPU) but only about 9 seconds with setting cv.dnn.DNN_BACKEND_INFERENCE_ENGINE. Thus, our conclusion is that OpenVINO is actually not used by default and requires an explicit setting of the respective backend.

Furthermore, checking the integer values of the backends shows

>>> cv.dnn.DNN_BACKEND_DEFAULT
0
>>> cv.dnn.DNN_BACKEND_OPENCV
3
>>> cv.dnn.DNN_BACKEND_INFERENCE_ENGINE
2

which means that the default values do not align with the ones explained in documentation.

Steps to reproduce

Running the provided code with enabling OpenVINO, i.e. adding the line

model.setPreferableBackend(cv.dnn.DNN_BACKEND_INFERENCE_ENGINE)

is significantly faster than any of the other options.

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
• I updated to the latest OpenCV version and the issue is still there
• There is reproducer code and related data files (videos, images, onnx, etc)