System Information

OpenCV version: 4.9.0
Operating System / Platform: Windows 11
Compiler & compiler version: Visual Studio 2022

Detailed description

We found two related, but generally independent issues. In the 4.9.0 release, the extended CMake configuration options to select a compile time default DNN backend are included, cf. #24218 and #24252. However, if getParam_DNN_BACKEND_DEFAULT() returns DNN_BACKEND_INFERENCE_ENGINE instead of DNN_BACKEND_OPENCV - i.e. either by using a compiled-in default or an explicit configuration option returned by getConfigurationParameterSizeT - calling forward() results in the following runtime error:

# compiled with OPENCV_DNN_BACKEND_DEFAULT = DNN_BACKEND_INFERENCE_ENGINE
import cv2 as cv;
model = cv.dnn.readNet('path/to/model.onnx')
model.setInput(...) # use valid input here, e.g. np.zeros((...), dtype=np.float32)
model.forward()

error: OpenCV(4.9.0) opencv\modules\dnn\src\legacy_backend.cpp:124: error: (-213:The function/feature is not implemented) Unknown backend identifier in function 'cv::dnn::dnn4_v20231225::detail::wrapMat'

Explicitly setting the backend before forwarding would fix the issue, however breaks the aim of the default value. We think the problem is that inside the constructor of Net::Impl, the following code

// net_impl.cpp
Net::Impl::Impl()
{
    // ...
    preferableBackend = (Backend)getParam_DNN_BACKEND_DEFAULT();
    // ...
}

only defines the backend, but skips the required initialization implemented in setPreferableBackend(). Clearly, this is only noticeable if getParam_DNN_BACKEND_DEFAULT() returns something different than DNN_BACKEND_OPENCV.
Instead, the initialization should be

// net_impl.cpp
preferableBackend = DNN_BACKEND_OPENCV;

and a call to setPreferableBackend(getParam_DNN_BACKEND_DEFAULT()) is required. As the latter depends on the Net instance and not just on the implementation class, this cannot be fixed in net_impl.cpp.

To fix this, we tested to add the following code in net.hpp

Net::Net()
    : impl(makePtr<Net::Impl>())
{
    setPreferableBackend(getParam_DNN_BACKEND_DEFAULT());
}

This works, but we don't know whether this is the best option to fix it. We can create a PR including these two modifications, please let us know whether this is desired.

Furthermore, we noted a related issue appearing after re-changing the backend. It can be reproduced with all builds, defining OPENCV_DNN_BACKEND_DEFAULT is not required. Thus, it is independent of the previous issue. We only noticed it during the same tests. It seems that after loading a model, the first forward() call initializes something that is required to change the backend thereafter. Here, something is broken if the first forward() call uses DNN_BACKEND_INFERENCE_ENGINE. For example, the following code

import cv2 as cv;
model = cv.dnn.readNet('path/to/model.onnx')
model.setPreferableBackend(cv.dnn.DNN_BACKEND_INFERENCE_ENGINE) # don't call forward() before this call
model.setInput(...); model.forward()                            # works
model.setPreferableBackend(cv.dnn.DNN_BACKEND_OPENCV)           # works as well
model.setInput(...); model.forward()                            # error

yields the following error message:

error: OpenCV(4.9.0) opencv\modules\dnn\src\net_impl.cpp:570: error: (-215:Assertion failed) inp.total() in function 'cv::dnn::dnn4_v20231225::Net::Impl::allocateLayers'

It works if another forward() call is added before a different backend is selected. This is particularly relevant if a non-default backend is used.

Steps to reproduce

See description.

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
• I updated to the latest OpenCV version and the issue is still there
• There is reproducer code and related data files (videos, images, onnx, etc)