Skip to content

Faster-RCNN object detection models from TensorFlow#11255

Merged
opencv-pushbot merged 1 commit intoopencv:3.4from
dkurt:dnn_tf_faster_rcnn
May 31, 2018
Merged

Faster-RCNN object detection models from TensorFlow#11255
opencv-pushbot merged 1 commit intoopencv:3.4from
dkurt:dnn_tf_faster_rcnn

Conversation

@dkurt
Copy link
Copy Markdown
Member

@dkurt dkurt commented Apr 6, 2018

@teaglin
Copy link
Copy Markdown

teaglin commented Apr 18, 2018

Hi, I tried out this build and it works with the coco model in the TensorFlow model zoo. But it does not work with a custom trained model. I get this error. Any ideas?

Error: Unspecified error (Const input blob for weights not found) in getConstBlob, file /Volumes/HD3/opencv/modules/dnn/src/tensorflow/tf_importer.cpp, line 464

@dkurt
Copy link
Copy Markdown
Member Author

dkurt commented Apr 18, 2018

@teaglin, please specify what config file did you use and what changes in it did you make? If you can share the model we can solve it faster. Thanks!

@teaglin
Copy link
Copy Markdown

teaglin commented Apr 18, 2018

@dkurt I used faster_rcnn_inception_v2_pets.config as a starting point. The only change I made was to the class count. I only have 1 class. The model can be downloaded here. Let me know if you need anything else.

https://www.dropbox.com/s/d5oh7lp8iutjdtd/frozen_inference_graph.pb?dl=0

@dkurt
Copy link
Copy Markdown
Member Author

dkurt commented Apr 19, 2018

@teaglin, Thank you! Could you please specify what kind of objects is detected to test an accuracy?

@dkurt dkurt force-pushed the dnn_tf_faster_rcnn branch from a08effc to a4790c0 Compare April 19, 2018 08:10
@teaglin
Copy link
Copy Markdown

teaglin commented Apr 19, 2018

@dkurt The model is used for detecting sports balls.

@dkurt
Copy link
Copy Markdown
Member Author

dkurt commented Apr 19, 2018

@teaglin, Please regenerate a .pbtxt file using an updated script from this PR. I've added extra graph transformations to remove Identity nodes.

python tf_text_graph_faster_rcnn.py --input frozen_inference_graph.pb --output graph.pbtxt --num_classes 1

Then you can use object_detection.py:

python object_detection.py --input footballer.jpg --model frozen_inference_graph.pb --config graph.pbtxt --mean 127.5 127.5 127.5 --scale 0.00784 --rgb

res

@teaglin
Copy link
Copy Markdown

teaglin commented Apr 19, 2018

@dkurt Awesome, I really appreciate the help. I will try this out.

On a side note do any of the other configs of rcnn work? I tried the coco resnet50 and resnet101 from tensor flow model zoo. They loaded, but I got really low confidence values. Are there different mean and scale values that have to be used?

@dkurt
Copy link
Copy Markdown
Member Author

dkurt commented Apr 19, 2018

@teaglin, Thank you!

On a side note do any of the other configs of rcnn work? I tried the coco resnet50 and resnet101 from tensor flow model zoo. They loaded, but I got really low confidence values. Are there different mean and scale values that have to be used?

Definitely yes, you may check it at README (see PR's files). All the ResNet-based Faster-RCNN networks work with input images with subtracted mean values 103.939 116.779 123.68 (blue, green, red channels correspondingly) and scale = 1.0 (no scale).

@dkurt dkurt force-pushed the dnn_tf_faster_rcnn branch from a4790c0 to bf87a43 Compare May 30, 2018 14:14
@vpisarev vpisarev self-assigned this May 31, 2018
@vpisarev
Copy link
Copy Markdown
Contributor

👍

@opencv-pushbot opencv-pushbot merged commit bf87a43 into opencv:3.4 May 31, 2018
@alalek alalek mentioned this pull request Jun 4, 2018
@lbv-h
Copy link
Copy Markdown

lbv-h commented Jun 6, 2018

Hi, I created graph.pbtxt with tf_text_graph_faster_rcnn.py from frozen_inference_graph.pb, ran cv.dnn.readNetFromTensorflow('frozen_inference_graph.pb', 'graph.pbtxt') and I got next error:

CropAndResize:CropAndResize(FirstStageFeatureExtractor/InceptionV2/InceptionV2/Mixed_4e/concat)(CropAndResize/crop_size)(detection_out)
method:[ ]
extrapolation_value:[ ]
OpenCV(3.4.1) Error: Unspecified error (Unknown layer type CropAndResize in op CropAndResize) in cv::dnn::experimental_dnn_v4::`anonymous-namespace'::TFImporter::populateNet, file C:\projects\opencv-python\opencv\modules\dnn\src\tensorflow\tf_importer.cpp, line 1582
Traceback (most recent call last):
  File "E:/Диплом/tensorflow network/test.py", line 3, in <module>
    cvNet = cv.dnn.readNetFromTensorflow('frozen_inference_graph.pb', 'graph.pbtxt')
cv2.error: OpenCV(3.4.1) C:\projects\opencv-python\opencv\modules\dnn\src\tensorflow\tf_importer.cpp:1582: error: (-2) Unknown layer type CropAndResize in op CropAndResize in function cv::dnn::experimental_dnn_v4::`anonymous-namespace'::TFImporter::populateNet

I thought that I need to manually update the file opencv/samples/dnn/tf_text_graph_faster_rcnn.py and others but I can't find the file on my disk (there is no directory C:\projects).
I have this error on both Windows 10 and Ubuntu 16.4.
I installed opencv-python via pip on Windows and via conda-forge on Ubuntu.
Please, can you give me any advices?

@dkurt
Copy link
Copy Markdown
Member Author

dkurt commented Jun 6, 2018

@UserLiubov, this patch has been merged a week ago. I don't think that opencv installed from pip is up to date. Try to build it from source https://github.com/opencv/opencv/tree/3.4.
Duplicate of #11691.

@Luca3424
Copy link
Copy Markdown

Hi, I'm trying to load a custom trained TensorFlow faster_rcnn_inception_v2 model with faster_rcnn_inception_v2_pets.config as a starting point.
I'm using OpenCV 3.4.2 and I updated the graph.pbtxt file with the tf_text_graph_faster_rcnn.py script.
While using a ssd_inception_v2 custom model everything works fine, but as soon as I'm trying the F-RCNN it crashes with the following error:

\opencv\modules\dnn\src\tensorflow\tf_importer.cpp:532: error: (-2:Unspecified error) Const kernel input not found in function 'cv::dnn::experimental_dnn_v5::`anonymous-namespace'::TFImporter::getConstBlob'

It seems like it's crashing while loading the model and config in the readNetFromTensorflow function.

Does anyone of you guys know this problem?
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants