Skip to content

Add support for custom padding in DNN preprocessing#24569

Merged
asmorkalov merged 6 commits intoopencv:4.xfrom
Abdurrahheem:ash/padding_value_fix
Nov 28, 2023
Merged

Add support for custom padding in DNN preprocessing#24569
asmorkalov merged 6 commits intoopencv:4.xfrom
Abdurrahheem:ash/padding_value_fix

Conversation

@Abdurrahheem
Copy link
Copy Markdown
Contributor

@Abdurrahheem Abdurrahheem commented Nov 21, 2023

This PR add functionality for specifying value in padding.
It is required in many preprocessing pipelines in DNNs such as Yolox object detection model

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@Abdurrahheem Abdurrahheem self-assigned this Nov 21, 2023
@asmorkalov asmorkalov changed the title Add support for padding value and tests Add support for custom padding in DNN preprocessing Nov 21, 2023
Copy link
Copy Markdown
Member

@fengyuentau fengyuentau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does YOLO family use different padding values?

@Abdurrahheem
Copy link
Copy Markdown
Contributor Author

Does YOLO family use different padding values?

It uses 114. We do not have such support out of the box

@dkurt
Copy link
Copy Markdown
Member

dkurt commented Nov 22, 2023

It uses 114. We do not have such support out of the box

It seems like not a padding but mean value. So after subtraction of mean 114 final padding is zero

@fengyuentau
Copy link
Copy Markdown
Member

It uses 114. We do not have such support out of the box.

Ok. More questions:

  1. 114 is needed by which YOLO version?
  2. Do other YOLO versions use different value?

It seems like not a padding but mean value. So after subtraction of mean 114 final padding is zero

IIRC, the letterbox tranformation pads the image to a certain scale, for example, h:w=1:1; then when it is resized, it does not make distortion on the potential objects.

@LaurentBerger
Copy link
Copy Markdown
Contributor

It uses 114. We do not have such support out of the box

Agree I think values are necessary like pytorch https://pytorch.org/docs/stable/generated/torch.nn.functional.pad.html#torch.nn.functional.pad or tensorflow https://www.tensorflow.org/api_docs/python/tf/pad

@Abdurrahheem
Copy link
Copy Markdown
Contributor Author

It uses 114. We do not have such support out of the box.

Ok. More questions:

  1. 114 is needed by which YOLO version?
  2. Do other YOLO versions use different value?

It seems like not a padding but mean value. So after subtraction of mean 114 final padding is zero

IIRC, the letterbox tranformation pads the image to a certain scale, for example, h:w=1:1; then when it is resized, it does not make distortion on the potential objects.

Answer to the questions

  1. Yes, yolox explicitly uses 144 for as border value
  2. Yolov8 also uses 144 AFAIK, but not sure

Regardless what yolo family uses, our blobFromImagesWithParamsImpl could not get border value as an input before this past. Now it can

@asmorkalov asmorkalov added this to the 4.9.0 milestone Nov 23, 2023
@Abdurrahheem Abdurrahheem mentioned this pull request Nov 24, 2023
5 tasks
@Abdurrahheem Abdurrahheem mentioned this pull request Nov 24, 2023
6 tasks
@Abdurrahheem Abdurrahheem marked this pull request as ready for review November 24, 2023 17:43
@Abdurrahheem Abdurrahheem mentioned this pull request Nov 26, 2023
6 tasks
Copy link
Copy Markdown
Member

@fengyuentau fengyuentau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@asmorkalov asmorkalov merged commit 5278560 into opencv:4.x Nov 28, 2023
cv::copyMakeBorder(
targetImg, targetImg, 0, 0,
targetSize.width / 2,
targetSize.width / 2,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

targetSize vs borderSize - that is a mess.

Test should not depend on exact values of its input parameters.

int left = (size.width - rw)/2;
int right = size.width - left - rw;
copyMakeBorder(images[i], images[i], top, bottom, left, right, BORDER_CONSTANT);
copyMakeBorder(images[i], images[i], top, bottom, left, right, BORDER_CONSTANT, param.borderValue);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

images[i], images[i]

Operation is not in-place.
We should not use the same argument for src and dst - this is anti-pattern and bad practice.

@LaurentBerger LaurentBerger mentioned this pull request Nov 30, 2023
6 tasks
IskXCr pushed a commit to Haosonn/opencv that referenced this pull request Dec 20, 2023
Add support for custom padding in DNN preprocessing opencv#24569

This PR add functionality for specifying value in padding.
It is required in many preprocessing pipelines in DNNs such as Yolox object detection model

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
@asmorkalov asmorkalov mentioned this pull request Jan 19, 2024
thewoz pushed a commit to thewoz/opencv that referenced this pull request May 29, 2024
Add support for custom padding in DNN preprocessing opencv#24569

This PR add functionality for specifying value in padding.
It is required in many preprocessing pipelines in DNNs such as Yolox object detection model

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants