-
-
Notifications
You must be signed in to change notification settings - Fork 56.5k
error: (-215:Assertion failed) Passed input shapes do not match with parsed input shapes! in function 'cv::dnn::LayerEinsumImpl::getMemoryShapes' #25077
Copy link
Copy link
Closed
Labels
Milestone
Description
System Information
General configuration for OpenCV 4.9.0-dev =====================================
Version control: 4.9.0-189-g2d204243af
Extra modules:
Location (extra): C:/lib/opencv_contrib/modules
Version control (extra): 4.9.0-21-gd870648f
Platform:
Timestamp: 2024-02-22T10:48:57Z
Host: Windows 10.0.22631 AMD64
CMake: 3.26.1
CMake generator: Visual Studio 17 2022
CMake build tool: C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuil
Detailed description
Model is loaded. Error is thrown in inference process
I check result with numpy and onnx.
Opencv thrown an error :
[ERROR:0@6387.898] global net_impl.cpp:1165 cv::dnn::dnn4_v20231225::Net::Impl::getLayerShapesRecursively OPENCV/DNN: [Einsum]:(onnx_node_output_0!out1): getMemoryShapes() throws exception. inputs=3 outputs=0/1 blobs=0
[ERROR:0@6387.899] global net_impl.cpp:1168 cv::dnn::dnn4_v20231225::Net::Impl::getLayerShapesRecursively input[0] = [ 1 300 8 32 ]
[ERROR:0@6387.899] global net_impl.cpp:1168 cv::dnn::dnn4_v20231225::Net::Impl::getLayerShapesRecursively input[1] = [ 1 8 32 1 ]
[ERROR:0@6387.899] global net_impl.cpp:1168 cv::dnn::dnn4_v20231225::Net::Impl::getLayerShapesRecursively input[2] = [ 1 300 ]
[ERROR:0@6387.899] global net_impl.cpp:1178 cv::dnn::dnn4_v20231225::Net::Impl::getLayerShapesRecursively Exception message: OpenCV(4.9.0-dev) C:\lib\opencv\modules\dnn\src\layers\einsum_layer.cpp:440: error: (-215:Assertion failed) Passed input shapes do not match with parsed input shapes! in function 'cv::dnn::LayerEinsumImpl::getMemoryShapes'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 47, in <module>
cv2.error: OpenCV(4.9.0-dev) C:\lib\opencv\modules\dnn\src\layers\einsum_layer.cpp:440: error: (-215:Assertion failed) Passed input shapes do not match with parsed input shapes! in function 'cv::dnn::LayerEinsumImpl::getMemoryShapes'
Steps to reproduce
Source
import numpy as np
import onnx
import onnxsim
import onnx.reference
import cv2 as cv
eqn = "nlhd, nhdv, nlh -> nlhv"
init_Q = np.ones(shape=(1, 300, 8, 32))
init_KV = np.ones(shape=(1, 8, 32, 1))
init_onnx = np.ones(shape=(1, 300, 8))
numpy_res=np.einsum(eqn,init_Q, init_KV, init_onnx)
out1 = onnx.helper.make_tensor_value_info('out1', onnx.TensorProto.FLOAT, [None, None, None, None])
inp_init_Q = onnx.helper.make_tensor_value_info('inp_init_Q', onnx.TensorProto.FLOAT, init_Q.shape)
inp_init_KV = onnx.helper.make_tensor_value_info('inp_init_KV', onnx.TensorProto.FLOAT, init_KV.shape)
inp_init_onnx = onnx.helper.make_tensor_value_info('inp_init_onnx', onnx.TensorProto.FLOAT, init_onnx.shape)
node1 = onnx.helper.make_node( "Einsum", inputs=["inp_init_Q","inp_init_KV", "inp_init_onnx"], outputs=["out1"], equation=eqn)
graph = onnx.helper.make_graph([node1], 'test_einsum', [inp_init_Q, inp_init_KV, inp_init_onnx], [out1])
onnx_model = onnx.helper.make_model(graph)
feeds = {'inp_init_Q': init_Q, 'inp_init_KV': init_KV, 'inp_init_onnx': init_onnx}
sess = onnx.reference.ReferenceEvaluator(onnx_model)
res_onnx = sess.run(["out1"], feeds)
print("ONNX result")
print("Max diff onnx numpy " , abs(res_onnx[0]-numpy_res).max())
onnx_name = eqn.replace('>', '')
print("Writting model")
with open(onnx_name + ".onnx", "wb") as f:
f.write(onnx_model.SerializeToString())
print("Reading model")
net = cv.dnn.readNet(onnx_name+'.onnx')
net.enableWinograd(True)
net.setInput(init_Q.astype(np.float32), 'inp_init_Q')
net.setInput(init_KV.astype(np.float32), 'inp_init_KV')
net.setInput(init_onnx.astype(np.float32), 'inp_init_onnx')
res_ocv = net.forward()
in c++ with debug
vector<int> szInitQ = { 1, 300, 8, 32 };
vector<int> szInitKV = { 1, 8, 32, 1 };
vector<int> szInitOnnx = { 1, 300, 8 };
Mat inpInitQ(szInitQ, CV_32FC1, Scalar::all(1));
Mat inpInitKV(szInitKV, CV_32FC1, Scalar::all(1));
Mat inpInitOnnx(szInitOnnx, CV_32FC1, Scalar::all(1));
Net netEinsum = readNet("nlhd, nhdv, nlh - nlhv.onnx");
netEinsum.setInput(inpInitQ, "inp_init_Q");
netEinsum.setInput(inpInitKV, "inp_init_KV");
netEinsum.setInput(inpInitOnnx, "inp_init_onnx");
//netEinsum.setInput(b, "inp1");
Mat c = netEinsum.forward();
and error is
[ INFO:0@3.599] global onnx_importer.cpp:806 cv::dnn::dnn4_v20231225::ONNXImporter::populateNet DNN/ONNX: loading ONNX v9 model produced by ''. Number of nodes = 1, initializers = 0, inputs = 3, outputs = 1
[ INFO:0@3.599] global onnx_importer.cpp:699 cv::dnn::dnn4_v20231225::ONNXImporter::parseOperatorSet DNN/ONNX: ONNX opset version = 19
[ INFO:0@3.601] global onnx_importer.cpp:977 cv::dnn::dnn4_v20231225::ONNXImporter::handleNode DNN/ONNX: processing node with 3 inputs and 1 outputs: [Einsum]:(onnx_node_output_0!out1) from domain='ai.onnx'
OpenCV(4.9.0-dev) Error: Unspecified error (> Real output can not be shaped in to _**requred**_ output (expected: 'reqProd == realProd'), where
> 'reqProd' is 2400
> must be equal to
> 'realProd' is 19200
) in void __cdecl cv::dnn::LayerEinsumImpl::forward(const class cv::debug_build_guard::_InputArray &,const class cv::debug_build_guard::_OutputArray &,const class cv::debug_build_guard::_OutputArray &), file C:\lib\opencv\modules\dnn\src\layers\einsum_layer.cpp, line 550
Issue submission checklist
- I report the issue, it's not a question
- I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
- I updated to the latest OpenCV version and the issue is still there
- There is reproducer code and related data files (videos, images, onnx, etc)
Reactions are currently unavailable
