Fixes #22747. Support [crop] configuration for DarkNet by Dhanwanth1803 · Pull Request #24384 · opencv/opencv

Dhanwanth1803 · 2023-10-10T18:57:09Z

Request for comments. This is my first PR.

Merge with extra: opencv/opencv_extra#1112

resolves #22747

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

opencv_extra=feat-crop1

opencv-alalek · 2023-10-11T09:18:18Z

It makes sense to add test to avoid regressions in the future

see test code for Test_Darknet_layers
related test data is here: https://github.com/opencv/opencv_extra/tree/4.x/testdata/dnn/darknet

Dhanwanth1803 · 2023-10-11T10:39:13Z

It makes sense to add test to avoid regressions in the future

see test code for Test_Darknet_layers

related test data is here: https://github.com/opencv/opencv_extra/tree/4.x/testdata/dnn/darknet

Yes sure @opencv-alalek. I will add the tests very soon. Can you please assign this issue to me.

dkurt · 2023-10-11T12:53:13Z

modules/dnn/src/darknet/darknet_io.cpp

+                        int crop_width = getParam<int>(layer_params, "crop_width", 224);
+                        int flip = getParam<int>(layer_params, "flip", 1);
+                        int exposure = getParam<int>(layer_params, "exposure", 1);
+                        int saturation = getParam<int>(layer_params, "saturation", 1);


Please use boolean parameters for true/false flags:

bool flip = getParam<bool>(layer_params, "flip", true);

Yes. thankyou @dkurt. I will make the changes asap.

Dhanwanth1803 · 2023-10-14T03:48:29Z

test_backends.txt

@dkurt @opencv-alalek Hi guys, I am trying to add the test layers. Its actually a lot of test layers and I am finding it hard to understand the parameters and add it accordingly. I added a couple until now idk if they are correct. Can you guys help me with any resources or tell me how can I go about this problem. It would be a great help. thanks :) (request for comments)

dkurt · 2023-10-18T09:38:44Z

@Dhanwanth1803, you can create a single layer network similar to models from https://github.com/opencv/opencv_extra/tree/4.x/testdata/dnn/darknet. Then, use a script generate_darknet_models.py to create test data files. As Crop layer has no weights, it's enough to write only a .cfg file. Take a look at average pooling as a reference: https://github.com/opencv/opencv_extra/blob/4.x/testdata/dnn/darknet/avgpool_softmax.cfg

After that, add all the generated files to a new branch to opencv_extra with the same name: [feat-crop](https://github.com/Dhanwanth1803/opencv/tree/feat-crop) and open a PR.

Dhanwanth1803 · 2023-10-22T05:55:52Z

@Dhanwanth1803, you can create a single layer network similar to models from https://github.com/opencv/opencv_extra/tree/4.x/testdata/dnn/darknet. Then, use a script generate_darknet_models.py to create test data files. As Crop layer has no weights, it's enough to write only a .cfg file. Take a look at average pooling as a reference: https://github.com/opencv/opencv_extra/blob/4.x/testdata/dnn/darknet/avgpool_softmax.cfg

After that, add all the generated files to a new branch to opencv_extra with the same name: [feat-crop](https://github.com/Dhanwanth1803/opencv/tree/feat-crop) and open a PR.

Thank you @dkurt. I will be working on it.

Dhanwanth1803 · 2023-10-23T13:52:12Z

@dkurt Like you said, i tried to generate test data files using generate_darknet_models.py . But it gave me the following error.
AttributeError: module 'darknet' has no attribute 'load_net'
So I checked for load_net function in the darknet repo as well. Turns out there is no such function there. There is a load_network function but i tried using it already. Its not helping.

dkurt · 2023-10-23T14:02:37Z

@Dhanwanth1803, check https://github.com/pjreddie/darknet/blob/f6afaabcdf85f77e7aff2ec55c020c0e297c77f9/python/darknet.py#L85

Dhanwanth1803 · 2023-10-24T12:30:57Z

@dkurt I have installed darknet and set the path using export and also i have given read and write permissions to the config files in testdata/dnn/darknet. when the run the generate_darknet_models.py I am getting the following error message.

Couldn't open file: shortcut.cfg

This is happening for every function call trying to read a .cfg file.

dkurt · 2023-10-24T12:45:56Z

@Dhanwanth1803, Run the script from opencv_extra/testdata/dnn/darknet folder

Dhanwanth1803 · 2023-10-24T14:46:03Z

@Dhanwanth1803, you can create a single layer network similar to models from https://github.com/opencv/opencv_extra/tree/4.x/testdata/dnn/darknet. Then, use a script generate_darknet_models.py to create test data files. As Crop layer has no weights, it's enough to write only a .cfg file. Take a look at average pooling as a reference: https://github.com/opencv/opencv_extra/blob/4.x/testdata/dnn/darknet/avgpool_softmax.cfg

After that, add all the generated files to a new branch to opencv_extra with the same name: [feat-crop](https://github.com/Dhanwanth1803/opencv/tree/feat-crop) and open a PR.

@dkurt I have put up PR in opencv_extra. Please do check it out and let me know if any changes are required. Thank you.

Dhanwanth1803 · 2023-10-26T13:30:16Z

@Dhanwanth1803, you can create a single layer network similar to models from https://github.com/opencv/opencv_extra/tree/4.x/testdata/dnn/darknet. Then, use a script generate_darknet_models.py to create test data files. As Crop layer has no weights, it's enough to write only a .cfg file. Take a look at average pooling as a reference: https://github.com/opencv/opencv_extra/blob/4.x/testdata/dnn/darknet/avgpool_softmax.cfg
After that, add all the generated files to a new branch to opencv_extra with the same name: [feat-crop](https://github.com/Dhanwanth1803/opencv/tree/feat-crop) and open a PR.

@dkurt I have put up PR in opencv_extra. Please do check it out and let me know if any changes are required. Thank you.

I have put up a PR again @dkurt . Please check it out.

dkurt · 2023-10-26T13:42:42Z

modules/dnn/src/darknet/darknet_io.cpp

                    fused_layer_names.push_back(last_layer);
                }

+                void setCrop(int crop_height, int crop_width, int flip, int exposure, int saturation, int angle)


Unused arguments. How layer can work without them?

dkurt · 2023-10-27T06:37:43Z

@Dhanwanth1803, add a test to https://github.com/opencv/opencv/blob/4.x/modules/dnn/test/test_darknet_importer.cpp

Dhanwanth1803 · 2023-10-27T06:41:56Z

@Dhanwanth1803, add a test to https://github.com/opencv/opencv/blob/4.x/modules/dnn/test/test_darknet_importer.cpp

Okay. Will do.

asmorkalov · 2023-11-02T10:24:04Z

@Dhanwanth1803 Friendly reminder.

Dhanwanth1803 · 2023-11-02T10:57:07Z

@Dhanwanth1803 Friendly reminder.

have exams currently in my school right now. I continue on 8th November. Is that fine?

Dhanwanth1803 · 2023-11-10T17:44:31Z

@Dhanwanth1803, add a test to https://github.com/opencv/opencv/blob/4.x/modules/dnn/test/test_darknet_importer.cpp

@dkurt Should I add a test to

@Dhanwanth1803, add a test to https://github.com/opencv/opencv/blob/4.x/modules/dnn/test/test_darknet_importer.cpp

@dkurt I have made changes for addition of crop layer in https://github.com/opencv/opencv/blob/4.x/modules/dnn/src/darknet/darknet_io.cpp and test in https://github.com/opencv/opencv/blob/4.x/modules/dnn/test/test_darknet_importer.cpp. Should I be making changes to any new file in addition to these two?

asmorkalov · 2023-11-11T06:11:39Z

@Dhanwanth1803 the PR still includes changes for single file. Looks like you forgot to add test changes to the commit.

dkurt · 2023-11-12T08:04:02Z

Please remove an old branch https://github.com/Dhanwanth1803/opencv_extra/tree/feat-crop

Dhanwanth1803 · 2023-11-12T08:19:43Z

Please remove an old branch https://github.com/Dhanwanth1803/opencv_extra/tree/feat-crop

Yes Sir deleted just now/

dkurt · 2023-11-13T05:45:39Z

@Dhanwanth1803, please make test pass locally. Build OpenCV with ts module and run opencv_test_dnn --gtest_filter=Test_Darknet_layers.crop/0

For now problem is not solved:

[ RUN      ] Test_Darknet_layers.crop/0, where GetParam() = OCV/CPU
unknown file: error: C++ exception with description "OpenCV(4.8.0-dev) c:\build\precommit_windows64\4.x\opencv\modules\dnn\src\net_impl.hpp:108: error: (-2:Unspecified error) Can't create layer "crop_0" of type "CropLayer" in function 'cv::dnn::dnn4_v20230620::Net::Impl::getLayerInstance'
" thrown in the test body.
[  FAILED  ] Test_Darknet_layers.crop/0, where GetParam() = OCV/CPU (1 ms)

dkurt · 2023-11-13T05:46:59Z

modules/dnn/src/darknet/darknet_io.cpp

+                    crop_param.set<int>("crop_height", crop_height);
+                    crop_param.set<int>("crop_width", crop_width);
+                    crop_param.name = "CropLayer-name";
+                    crop_param.type = "CropLayer";


There is no "CropLayer" in OpenCV - just "Crop"

Moreover, there is no crop_height and crop_width attributes for Crop. You need to utilize Slice with it's begin, end or size (https://github.com/opencv/opencv/blob/4.x/modules/dnn/src/layers/slice_layer.cpp):

std::vector<int> begin = {0, 0, 0, 0}; std::vector<int> sizes = {-1, -1, crop_height, crop_width}; crop_param.set("begin", DictValue::arrayInt(&begin[0], begin.size())); crop_param.set("size", DictValue::arrayInt(&sizes[0], sizes.size())); crop_param.type = "Slice";

if crop_height and crop_width are not attributes for crop. Why did we generate test data using those parameters in opencv_extra

These are native Darknet names and this is ok for test data. Importer should remap this parameters to expected ones by OpenCV

Okay. I will replace it with the code snippet you gave.

modules/dnn/src/darknet/darknet_io.cpp

asmorkalov · 2023-12-01T07:20:45Z

@Dhanwanth1803 Friendly reminder

Dhanwanth1803 · 2023-12-03T17:28:33Z

@Dhanwanth1803 Friendly reminder

Yes sir still working on it @asmorkalov

asmorkalov · 2023-12-20T12:09:13Z

@dkurt

[ RUN      ] Test_Darknet_layers.crop/0, where GetParam() = OCV/CPU
unknown file: Failure
C++ exception with description "OpenCV(4.8.0-dev) /home/ci/opencv/modules/ts/src/ts.cpp:1071: error: (-2:Unspecified error) OpenCV tests: Can't find required data file: dnn/darknet/crop_in.npy in function 'findData'
" thrown in the test body.
[  FAILED  ] Test_Darknet_layers.crop/0, where GetParam() = OCV/CPU (0 ms)

dkurt · 2023-12-20T18:12:43Z

@asmorkalov, GitHub Actions CI ignores Merge with extra which points to a different branch name. Buildbot CI passes.

asmorkalov · 2023-12-21T07:17:07Z

@dkurt Extra CI options are handled by buildbot only. Please ignore Github actions for now.

dkurt · 2023-12-22T12:35:44Z

CUDA test failed: https://github.com/opencv/opencv/actions/runs/7224612370/job/19892629635?pr=24704

Dhanwanth1803 · 2023-12-23T20:37:46Z

@dkurt Thank you for making the changes. I was not able to work for personal reasons I will continue to contribute soon.

Fixes opencv#22747. Support [crop] configuration for DarkNet opencv#24384 Request for comments. This is my first PR. **Merge with extra**: opencv/opencv_extra#1112 resolves opencv#22747 - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake

opencv-alalek added feature category: dnn labels Oct 11, 2023

opencv-alalek added the pr: needs test New functionality requires minimal tests set label Oct 11, 2023

dkurt reviewed Oct 11, 2023

View reviewed changes

asmorkalov assigned dkurt Oct 20, 2023

asmorkalov added this to the 4.9.0 milestone Oct 20, 2023

dkurt mentioned this pull request Oct 25, 2023

added testdata for crop layer #24384 opencv/opencv_extra#1111

Closed

dkurt reviewed Oct 26, 2023

View reviewed changes

fixes-22747 supp crop config

4f209a0

Dhanwanth1803 force-pushed the feat-crop branch from 2a33772 to 4f209a0 Compare November 11, 2023 18:17

dkurt reviewed Nov 13, 2023

View reviewed changes

modules/dnn/src/darknet/darknet_io.cpp Outdated Show resolved Hide resolved

Consider noadjust and use Slice layer

aaa11a8

asmorkalov removed the pr: needs test New functionality requires minimal tests set label Dec 20, 2023

dkurt approved these changes Dec 22, 2023

View reviewed changes

asmorkalov merged commit 027aee8 into opencv:4.x Dec 22, 2023

dkurt mentioned this pull request Dec 22, 2023

[CUDA] Hotfix Scale with 1 parameter #24751

Merged

6 tasks

asmorkalov mentioned this pull request Jan 19, 2024

5.x merge 4.x #24862

Merged

Uh oh!

Conversation

Dhanwanth1803 commented Oct 10, 2023 • edited by opencv-alalek Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

opencv-alalek commented Oct 11, 2023

Uh oh!

Dhanwanth1803 commented Oct 11, 2023

Uh oh!

dkurt Oct 11, 2023

Choose a reason for hiding this comment

Uh oh!

Dhanwanth1803 Oct 11, 2023

Choose a reason for hiding this comment

Uh oh!

Dhanwanth1803 commented Oct 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkurt commented Oct 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Dhanwanth1803 commented Oct 22, 2023

Uh oh!

Dhanwanth1803 commented Oct 23, 2023

Uh oh!

dkurt commented Oct 23, 2023

Uh oh!

Dhanwanth1803 commented Oct 24, 2023

Uh oh!

dkurt commented Oct 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Dhanwanth1803 commented Oct 24, 2023

Uh oh!

Dhanwanth1803 commented Oct 26, 2023

Uh oh!

dkurt Oct 26, 2023

Choose a reason for hiding this comment

Uh oh!

dkurt commented Oct 27, 2023

Uh oh!

Dhanwanth1803 commented Oct 27, 2023

Uh oh!

asmorkalov commented Nov 2, 2023

Uh oh!

Dhanwanth1803 commented Nov 2, 2023

Uh oh!

Dhanwanth1803 commented Nov 10, 2023

Uh oh!

asmorkalov commented Nov 11, 2023

Uh oh!

dkurt commented Nov 12, 2023

Uh oh!

Dhanwanth1803 commented Nov 12, 2023

Uh oh!

dkurt commented Nov 13, 2023

Uh oh!

dkurt Nov 13, 2023

Choose a reason for hiding this comment

Uh oh!

dkurt Nov 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dhanwanth1803 Nov 20, 2023

Choose a reason for hiding this comment

Uh oh!

dkurt Nov 20, 2023

Choose a reason for hiding this comment

Uh oh!

Dhanwanth1803 Nov 20, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asmorkalov commented Dec 1, 2023

Uh oh!

Dhanwanth1803 commented Dec 3, 2023

Uh oh!

asmorkalov commented Dec 20, 2023

Uh oh!

Dhanwanth1803 commented Oct 10, 2023 •

edited by opencv-alalek

Loading

Dhanwanth1803 commented Oct 14, 2023 •

edited

Loading

dkurt commented Oct 18, 2023 •

edited

Loading

dkurt commented Oct 24, 2023 •

edited

Loading

dkurt Nov 14, 2023 •

edited

Loading