Load TensorFlow 2 model (SSD, Object Detection API) with OpenCV #19257
Description
System information (version)
- OpenCV => 4.5.1 (python)
- TensorFlow => 2.3.1 (python)
- Operating System / Platform => Windows 10 (64 Bit)
- Compiler => Visual Studio 2019
Detailed description
After training a SSD MobileNet v2 using the TensorFlow Object Detection API with TensorFlow2, I converted the SavedModel into a FrozenGraph in order to make it compatible with OpenCV. I am using following code to load the model:
import numpy as np
import cv2 as cv
net = cv.dnn.readNet('frozen_graph.pb') This gives following error message:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
cv2.error: OpenCV(4.5.1) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-oduouqig\opencv\modules\dnn\src\tensorflow\tf_importer.cpp:578: error: (-2:Unspecified error) More than one input is Const op in function 'cv::dnn::dnn4_v20201117::`anonymous-namespace'::TFImporter::getConstBlob'
The script opencv/tf_text_graph_ssd.py runs into an assertion:
Traceback (most recent call last):
File "tf_text_graph_ssd_original.py", line 405, in <module>
createSSDGraph(args.input, args.config, args.output)
File "tf_text_graph_ssd_original.py", line 292, in createSSDGraph
assert(num_matched_layers == num_layers)
AssertionError
After fixing the assertion by adapting the layer names in the script, the text graph could be created. After loading the model with
net = cv.dnn.readNet('frozen_graph.pb', 'graph.pbtxt') there is a different error message
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
cv2.error: OpenCV(4.5.1) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-oduouqig\opencv\modules\dnn\src\tensorflow\tf_graph_simplifier.cpp:1019: error: (-215:Assertion failed) permIds.size() == net.node_size() in function 'cv::dnn::dnn4_v20201117::sortByExecutionOrder'
I attach my frozen_graph.pb and my pipeline.config.
model.zip
Steps to reproduce
1.) Train a SSD MobileNet v2 using the TensorFlow Object Detection API and export it to a SavedModel
# Train the network
python model_main_tf2.py --model_dir="training" --pipeline_config_path="training/pipeline.config"
# Export the network
python exporter_main_v2.py --input_type=image_tensor --pipeline_config_path="training/pipeline.config" --trained_checkpoint_dir="training" --output_directory="training/model"
2.) Convert the SavedModel to a FrozenGraph
import tensorflow as tf
from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2
loaded = tf.saved_model.load('training/model/saved_model')
infer = loaded.signatures['serving_default']
f = tf.function(infer).get_concrete_function(input_tensor=tf.TensorSpec(shape=[1, None, None, 3], dtype=tf.uint8))
f2 = convert_variables_to_constants_v2(f)
graph_def = f2.graph.as_graph_def()
# Export frozen graph
with tf.io.gfile.GFile('frozen_graph.pb', 'wb') as f:
f.write(graph_def.SerializeToString())3.) Try to load the FrozenGraph with OpenCV as described above: failed
Anyone can reproduce the issue and help me to fix it?