Skip to content

DNN/ONNX(netvlad_tf): ReduceL2, Where and Equal layers are missing. Slice with negative starts input is not supported. #20428

@Richard-coder

Description

@Richard-coder
System information (version)
  • OpenCV => 4.5.3
  • Operating System / Platform => ubuntu18
  • Compiler => gcc&g++ 7.5
  • Tensorflow => 1.8 cpu
  • onnx => 1.9
  • onnxruntime => 1.8
  • tf2onnx => 1.8.5
Detailed description

I convert this uzh-rpg/netvlad_tf_open to onnx, and try to use opencv dnn to load onnx like this

cv::dnn::Net net = cv::dnn::readNetFromONNX ("path to onnx model");

but I meet error like this

[ERROR:0] global opencv-4.5.3/modules/dnn/src/onnx/onnx_importer.cpp (2127) handleNode DNN/ONNX: ERROR during processing node with 2 inputs and 1 outputs: [ReduceSum]:(vgg16_netvlad_pca/l2_normalize/Sum:0)
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.5.3) opencv-4.5.3/modules/dnn/src/onnx/onnx_importer.cpp:2146: error: (-2:Unspecified error) in function 'handleNode'
> Node [ReduceSum]:(vgg16_netvlad_pca/l2_normalize/Sum:0) parse error: OpenCV(4.5.3) opencv-4.5.3/modules/dnn/src/onnx/onnx_importer.cpp:624: error: (-215:Assertion failed) node_proto.input_size() == 1 in function 'handleNode'
> 

the ReduceSum part of onnx is
2021-07-19 17-57-02 的屏幕截图

this ReduceSum is part of tf.nn.l2_normalize() in tensorflow model

        # NetVLAD
        x = tf.nn.l2_normalize(x, dim=-1)
        x = layers.netVLAD(x, 64)
Steps to reproduce

first I use following code to conver tensorflow model from checkpoint to pb

import tensorflow as tf

import os
import cv2
from pathlib import Path
import h5py
from tqdm import tqdm
import numpy as np

import netvlad_tf.nets as nets

import tensorflow as tf
from tensorflow.python.saved_model import tag_constants
tf.contrib.resampler

model_dir = "./checkpoints/vd16_pitts30k_conv5_3_vlad_preL2_intra_white"
output_frozen_path = "./output/hf_frozen.pb"


resize_max=1024

tf.reset_default_graph()

image_batch = tf.placeholder(
    dtype=tf.float32, shape=[1, 1024, 1024, 3])
#tf.identity(image_batch, name="prInput")
print("test image_batch")
print(image_batch.name)
net_out = nets.vgg16NetvladPca(image_batch)

saver = tf.train.Saver()
sess = tf.Session()
ckpt = nets.defaultCheckpoint()
saver.restore(sess, ckpt)

path = "netvlad_tf_open-master/demo/query/query1.jpg"

img = cv2.imread(str(path))
h, w = img.shape[:2]
if max(h, w) > resize_max:
    scale = resize_max / max(h, w)
    h_new, w_new = int(round(h*scale)), int(round(w*scale))
    img = cv2.resize(img, (w_new, h_new), interpolation=cv2.INTER_LINEAR)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

desc = sess.run(net_out, feed_dict={image_batch: img[None]})[0]

graph = tf.get_default_graph()

#frozen_graph = tf.graph_util.convert_variables_to_constants(
#    sess, graph.as_graph_def(),
#    output_node_names=['Placeholder', 'vgg16_netvlad_pca/l2_normalize_1']
#)
#
#frozen_graph = tf.graph_util.remove_training_nodes(frozen_graph, ['Placeholder', 'vgg16_netvlad_pca/l2_normalize_1'])
frozen_graph = tf.graph_util.convert_variables_to_constants(
    sess, graph.as_graph_def(), output_node_names=['vgg16_netvlad_pca/l2_normalize_1'])
with tf.gfile.GFile(output_frozen_path, mode='wb') as f:
    f.write(frozen_graph.SerializeToString())

summary_write = tf.summary.FileWriter("./output/log" , graph)

print("run success")

then , conver pb to onnx

import tensorflow as tf
import numpy as np
from tensorflow.python.saved_model import tag_constants
tf.contrib.resampler
import tf2onnx

frozen_model = "./output/hf_frozen.pb"

with tf.Session() as sess:
    with tf.gfile.GFile(frozen_model, mode="rb") as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())
        sess.graph.as_default()
        tf.import_graph_def(graph_def, name='')

        #onnx_graph = tf2onnx.tfonnx.process_tf_graph(sess.graph, opset = 7, input_names=["Placeholder:0"], inputs_as_nchw=["Placeholder:0"],  output_names=["vgg16_netvlad_pca/l2_normalize_1:0"])
        onnx_graph = tf2onnx.tfonnx.process_tf_graph(sess.graph, opset = 13,  input_names=["Placeholder:0"],  output_names=["vgg16_netvlad_pca/l2_normalize_1:0"])
        model_proto = onnx_graph.make_model("test")
        with open("path to onnx", "wb") as f:
            f.write(model_proto.SerializeToString())
        print("done")

then use onnxruntime to verify onnx

import numpy as np
import onnxruntime as ort

import os
import cv2
from pathlib import Path
import h5py
from tqdm import tqdm
import numpy as np

#img = np.load("./assets/image.npz").reshape([1, 784])  
sess_ort = ort.InferenceSession("path to onnx")


path = "netvlad_tf_open-master/demo/query/query1.jpg"

resize_max=1024

img = cv2.imread(str(path))
h, w = img.shape[:2]
if max(h, w) > resize_max:
    scale = resize_max / max(h, w)
    h_new, w_new = int(round(h*scale)), int(round(w*scale))
    img = cv2.resize(img, (w_new, h_new), interpolation=cv2.INTER_LINEAR)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

#img = img.astype(np.float32)/255

data = np.zeros((1, 1024, 1024, 3), np.float32) 
data[0, :, :, :] = img

#data = data.transpose(0,3,1,2) # Batch, Chanel, Height, Width

res = sess_ort.run(output_names=["vgg16_netvlad_pca/l2_normalize_1:0"], input_feed={"Placeholder:0": data})
print(res)
print("done")

it can run successfully and result is the same as tensorflow.

but when i try to use opencv dnn to read onnx, i meet error

cv::dnn::Net net = cv::dnn::readNetFromONNX ("path to onnx");

onnxfile is big to upload, 596.0Mb

Issue submission checklist
  • I report the issue, it's not a question
  • I checked the problem with documentation, FAQ, open issues,
    forum.opencv.org, Stack Overflow, etc and have not found solution
  • I updated to latest OpenCV version and the issue is still there
  • There is reproducer code and related data files: videos, images, onnx, etc

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions