Skip to content

Correct width/height returned by cudacodec::VideoReader::FormatInfo()#3001

Merged
alalek merged 2 commits intoopencv:masterfrom
cudawarped:fix_cudacodec_wh
Jul 10, 2021
Merged

Correct width/height returned by cudacodec::VideoReader::FormatInfo()#3001
alalek merged 2 commits intoopencv:masterfrom
cudawarped:fix_cudacodec_wh

Conversation

@cudawarped
Copy link
Copy Markdown
Contributor

@cudawarped cudawarped commented Jul 9, 2021

When decoding certain resolutions (e.g. 1080p), the size of the frame returned from nextFrame() (1920x1088) can be larger than the usable area (1920x1080) due to the coded sizes being multiples of n bytes, for efficient coding/decoding.

This causes a problem in cudacodec because currently the values of width and height returned by cudaCodec::Format() are different depending on which video source is used. The two possibilities are FFmpeg and cuvid, with FFmpeg returning the usable width and height (1920x1080) and cuvid returning the coded width and height (1920x1088).

I think it would make sense for both video sources to return the coded width and height if possible because that corresponds to the dimensions of the frame returned by nextFrame(). Unfortunately in the case of the FFmpeg backend this won't be known for certain until after the first call to nextFrame() where the decoder may or may not reconfigure itself based on the coded bit stream. To avoid changing the internals of cudacodec too much (it needs a re-write to mirror the existing Nvidia sample code anyway) I have included an ugly hack using a flag to signal if the dimensions of width and height can be relied upon.

The accuracy test case has been updated and should fail without this fix.

Additionally I have included the display area so that the frame can be cropped to get the usable area (e.g. 1920x1080 from 1920x1088).

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
  • The PR is proposed to proper branch
  • There is reference to original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake
force_builders=Custom
buildworker:Custom=linux-4,linux-6
build_image:Custom=ubuntu-cuda:18.04

Copy link
Copy Markdown
Member

@alalek alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for contribution 👍

Comment on lines +132 to +134
format_.height = cap.get(CAP_PROP_FRAME_HEIGHT);
format_.width = cap.get(CAP_PROP_FRAME_WIDTH);
format_.displayArea = Rect(0, 0, cap.get(CAP_PROP_FRAME_WIDTH), cap.get(CAP_PROP_FRAME_HEIGHT));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes sense to reuse already fetched .get() results instead of querying a new one.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed 👍

Copy link
Copy Markdown
Member

@alalek alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants