Reading Darknet from cv::FileStorage by tani · Pull Request #11104 · opencv/opencv

tani · 2018-03-18T07:18:19Z

This pullrequest changes

I added new constructor for DarknetImporter to load from std::istream.
Current constructor gets to two pathnames which are raw strings. Without std::filesystem of C++17, some pathname has problem for cross-platform development. This changes makes abstractly to load files.

std::ifstream cfg("path/to/cfg");
std::ifstream model("path/to/model");
Net net = readNetFromDarknet(cfg, model);

And more, by this changes we will be able to include resources into binary, like followings.

std::istringstream cfg("some configuration text");
std::ifstream model("path/to/model");
Net net = readNetFromDarknet(cfg, model);

allow_multiple_commits=1

dkurt · 2018-03-18T08:32:32Z

+    *  @returns Network object that ready to do forward, throw an exception in failure cases.
+    *  @returns Net object.
+    */
+    CV_EXPORTS_W Net readNetFromDarknet(std::istream &cfgStream, std::istream &darknetModelStream);


Due std::istream has no wrapping policy we need to use CV_EXPORTS here and below.

OK, now I will to replace std::istream to IStream of 3rdparty. Can I add 3rdparty/openexr/IlmImf/ImfIO.h ? Or what do I should use?

I got your saying. I will change std::istream to the other buffered structure like #9994.

dkurt · 2018-03-18T08:36:35Z


-
-            bool ReadDarknetFromWeightsFile(const char *darknetModel, NetParameter *net)
+            bool ReadDarknetFromWeightsStream(std::istream &darknetModelStream, NetParameter *net)


Do you mind just rename darknetModelStream to ifile here?

Of course. I renamed.

dkurt · 2018-03-18T08:39:48Z

 {
-    DarknetImporter darknetImporter(cfgFile.c_str(), darknetModel.c_str());
+    std::ifstream cfgStream(cfgFile.c_str());
+    CV_Assert(cfgStream.is_open());


There were assertions include paths to files,

CV_Error(cv::Error::StsParseError, "Failed to parse NetParameter file: " + std::string(cfgFile));

I think it's useful to keep some kind of error message that prints actual paths.

I think so too. I added it.

tani · 2018-03-19T05:46:01Z

@dkurt I added readNetFromDarknet(FileNode &cfgFile, FileNode &darknetModel) instread of the first request. This changes solves the topic of this request.

The parameters in the file which has specific pathname, can pass into FileStorage with std::string which is read via std::ifstream.
FileStorage can have the data in memory. So we will be able to include resources into binary, as const char*.

dkurt · 2018-04-27T06:36:19Z

+    FileStorage ofs(".xml", FileStorage::WRITE | FileStorage::MEMORY);
+    ofs.write("cfgFile", buffer.str());
+    FileStorage ifs(ofs.releaseAndGetString(), FileStorage::READ | FileStorage::MEMORY | FileStorage::FORMAT_XML);
+    Net net = readNetFromDarknet(ifs["cfgFile"]);


@Asciian, please remind me why you decided to replace std::istream to cv::FileNode? The last one seems to be more complicated here.

@dkurt Because std::istream couldn't become the arguments of exporting functions and knew no way to implements the topic of this issue without cv::FileNode. If you know that, Please tell me.

@dkurt Ah, I know another way. With raw string, we can implements that. But the way requires adding the function two arguments which are used for telling the length of content.

dkurt · 2018-04-27T06:39:58Z

+    ofs.write("cfgFile", buffer.str());
+    FileStorage ifs(ofs.releaseAndGetString(), FileStorage::READ | FileStorage::MEMORY | FileStorage::FORMAT_XML);
+    Net net = readNetFromDarknet(ifs["cfgFile"]);
+    ASSERT_FALSE(net.empty());


I think we need to check an accuracy too. Could you make some of existing accuracy tests to be parametric similar to one of our tests for loading models from a buffer?

opencv/modules/dnn/test/test_caffe_importer.cpp

Lines 85 to 116 in 1031dfe

TEST_P(Reproducibility_AlexNet, Accuracy)

{

bool readFromMemory = get<0>(GetParam());

Net net;

{

const string proto = findDataFile("dnn/bvlc_alexnet.prototxt", false);

const string model = findDataFile("dnn/bvlc_alexnet.caffemodel", false);

if (readFromMemory)

{

string dataProto;

ASSERT_TRUE(readFileInMemory(proto, dataProto));

string dataModel;

ASSERT_TRUE(readFileInMemory(model, dataModel));

net = readNetFromCaffe(dataProto.c_str(), dataProto.size(),

dataModel.c_str(), dataModel.size());

}

else

net = readNetFromCaffe(proto, model);

ASSERT_FALSE(net.empty());

}

net.setPreferableTarget(get<1>(GetParam()));

Mat sample = imread(_tf("grace_hopper_227.png"));

ASSERT_TRUE(!sample.empty());

net.setInput(blobFromImage(sample, 1.0f, Size(227, 227), Scalar(), false), "data");

Mat out = net.forward("prob");

Mat ref = blobFromNPY(_tf("caffe_alexnet_prob.npy"));

normAssert(ref, out);

}

dkurt · 2018-07-04T15:22:52Z

@Asciian, I've reverted this changes back to std::istream because we cannot write a raw binary data (weights) into a text .xml file because it breaks a markup.

tani · 2018-07-04T16:10:04Z

I got it. BTW, I guess std::istream isn’t exported for Python and Java. Do you have any idea? We have the only way via FileNode, don’t you?

dkurt · 2018-07-05T06:56:57Z

@Asciian, I see your point. Please let me experiment a bit with a FileStorage and a binary data.

vpisarev · 2018-07-06T13:11:54Z

@dkurt, filestorage supports base64-encoded data; I suggest to use that. Maybe embedded a Darknet model (or any deep learning model) into a FileStorage is not very good idea in general and we or users can implement so alternative options. E.g. extend our Darknet parser to read the net from memory, if it was the original intention.

Remove some assertions Replace std::ifstream to std::istream Add test for new importer Remove constructor to load file Rename cfgStream and darknetModelStream to ifile Add error notification to inform pathname to user Use FileStorage instead of std::istream Use FileNode instead of FileStorage Fix typo

dkurt · 2018-07-10T09:38:40Z

@Asciian, I've added methods which receive bytes buffers via std::vector<char>. In python you may use it in the following way:

import cv2 as cv
import numpy as np

with open('yolov3.cfg', 'rt') as f:
    cfgFile = bytearray(f.read())
with open('yolov3.weights', 'rb') as f:
    weightsFile = bytearray(f.read())

inp = np.random.standard_normal([1, 3, 416, 416]).astype(np.float32)

net = cv.dnn.readNetFromDarknet(cfgFile, weightsFile)
net.setInput(inp)
out = net.forward()

tani · 2018-07-10T23:02:28Z

It looks good. However, as my comments,

cloud you tell me which I should use in JavaAPI? I'd like to try it in JavaAPI.
Now we have three way to read these networks, with string as filename, with bytearrays as content which requires 2 arguments, with string as buffers which requires 4 arguments. It very confuses users because there are almost string but they have different semantics. Would you like to redefine the interface for the development? If you don't mind, I'll open the new issue.

dkurt · 2018-07-11T09:54:09Z

@Asciian, Thank you. I've fixed that. Please check dnn/misc/java/test/DnnTensorFlowTest.java file for details:

File modelFile = new File(modelFileName);
byte[] modelBuffer = new byte[ (int)modelFile.length() ];

try {
    FileInputStream fis = new FileInputStream(modelFile);
    fis.read(modelBuffer);
    fis.close();
} catch (IOException e) {
    System.out.println(e.getMessage());
}
net = Dnn.readNetFromTensorflow(new MatOfByte(modelBuffer));

tani · 2018-07-11T11:38:23Z

That's a bytearray. Thank you. I said that std::vector<char> looks like string. But std::vector<uchar> a completely different from it. Please don't mind the second comments.

alalek · 2018-07-11T11:49:35Z

+        }
+        catch (Exception e) {
+            System.out.println(e.getMessage());
+            throw e;


In tests we can use this:

fail("DNN forward() failed: " + e.getMessage());

Thank you! Fixed that.

…dnn importers

tani · 2018-07-18T00:14:28Z

@alalek and @dkurt I'm glad to see that this request is merged. Thank you so much.

dkurt reviewed Mar 18, 2018

View reviewed changes

tani changed the title ~~Reading Darknet from std::istream~~ Reading Darknet from std::FileStorage Mar 19, 2018

tani changed the title ~~Reading Darknet from std::FileStorage~~ Reading Darknet from cv::FileStorage Mar 19, 2018

jancikpetr1981 approved these changes Apr 4, 2018

View reviewed changes

dkurt reviewed Apr 27, 2018

View reviewed changes

Asciian and others added 2 commits July 9, 2018 10:02

Add readNet* functions which parse models from byte arrays

d57e540

alalek reviewed Jul 11, 2018

View reviewed changes

Replace std::vector<char> to std::vector<uchar> for Java bindings of …

8b5f061

…dnn importers

alalek approved these changes Jul 17, 2018

View reviewed changes

opencv-pushbot merged commit 8b5f061 into opencv:3.4 Jul 17, 2018

opencv-pushbot pushed a commit that referenced this pull request Jul 17, 2018

Merge pull request #11104 from asciian:reading_from_stream

6c4f618

alalek mentioned this pull request Jul 17, 2018

Merge 3.4 #11996

Merged

dkurt mentioned this pull request Dec 3, 2019

Load IR model from buffer #16034

Merged



		bool ReadDarknetFromWeightsFile(const char darknetModel, NetParameter net)
		bool ReadDarknetFromWeightsStream(std::istream &darknetModelStream, NetParameter *net)

	TEST_P(Reproducibility_AlexNet, Accuracy)
	{
	bool readFromMemory = get<0>(GetParam());
	Net net;
	{
	const string proto = findDataFile("dnn/bvlc_alexnet.prototxt", false);
	const string model = findDataFile("dnn/bvlc_alexnet.caffemodel", false);
	if (readFromMemory)
	{
	string dataProto;
	ASSERT_TRUE(readFileInMemory(proto, dataProto));
	string dataModel;
	ASSERT_TRUE(readFileInMemory(model, dataModel));

	net = readNetFromCaffe(dataProto.c_str(), dataProto.size(),
	dataModel.c_str(), dataModel.size());
	}
	else
	net = readNetFromCaffe(proto, model);
	ASSERT_FALSE(net.empty());
	}

	net.setPreferableTarget(get<1>(GetParam()));

	Mat sample = imread(_tf("grace_hopper_227.png"));
	ASSERT_TRUE(!sample.empty());

	net.setInput(blobFromImage(sample, 1.0f, Size(227, 227), Scalar(), false), "data");
	Mat out = net.forward("prob");
	Mat ref = blobFromNPY(_tf("caffe_alexnet_prob.npy"));
	normAssert(ref, out);
	}

Uh oh!

Conversation

tani commented Mar 18, 2018 • edited by alalek Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

This pullrequest changes

Uh oh!

dkurt Mar 18, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tani Mar 18, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tani commented Mar 19, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dkurt commented Jul 4, 2018

Uh oh!

tani commented Jul 4, 2018

Uh oh!

dkurt commented Jul 5, 2018

Uh oh!

vpisarev commented Jul 6, 2018

Uh oh!

dkurt commented Jul 10, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tani commented Jul 10, 2018

Uh oh!

dkurt commented Jul 11, 2018

Uh oh!

tani commented Jul 11, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tani commented Jul 18, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

tani commented Mar 18, 2018 •

edited by alalek

Loading

dkurt Mar 18, 2018 •

edited

Loading

tani Mar 18, 2018 •

edited

Loading

dkurt commented Jul 10, 2018 •

edited

Loading