Add V4L2_PIX_FMT_Y16_BE pixel format by firebladed · Pull Request #18498 · opencv/opencv

firebladed · 2020-10-02T17:41:42Z

Add V4L2_PIX_FMT_Y16_BE pixel format
#18495

swaps byte order then treats as V4L2_PIX_FMT_Y16
relates to #23944

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
The PR is proposed to proper branch
There is reference to original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake
Test using Melexis MLX90640

Add V4L2_PIX_FMT_Y16_BE pixel format swaps byte order then treats as V4L2_PIX_FMT_Y16

change local variable "frame" to "temp2"

alalek

Thank you for contribution!

Please make builds green.

alalek · 2020-10-16T07:32:27Z

modules/videoio/src/cap_v4l.cpp

+        };        
+        temp2.ForEach(<uint16_t>(Operator()); 
+
+        temp2.convertTo(temp, CV_8U, 1.0 / 256);


temp2.ForEach

Extraction through cv::split() from CV_8UC2 should perform better.

asmorkalov · 2020-11-24T09:09:50Z

@firebladed Do you have chance to finish the PR? Also it'll be great if you add list of know cameras that are enabled by the patch to the PR description. It becomes merge commit message and is very useful for git history.

firebladed · 2020-11-24T17:30:09Z

@asmorkalov sorry, had issue an get it to work and been busy with other things recently, will try to look at again a soon as i can.

asmorkalov · 2021-03-03T09:26:03Z

@firebladed Do you have chance to finish the patch? There are changes in GStreamer support that introduces conflicts.

remove whitespace

firebladed · 2021-03-06T17:20:09Z

I have a version that appears to work however i need to do some more tests and i don't know why the OpenCV CN Windows 10 x64 build failed

alalek · 2021-03-06T17:39:05Z

OpenCV CN

Ignore, there builders are optional for now.

firebladed · 2021-03-26T13:12:23Z

Ive tested visually using the MLX90640 on ubuntu on a raspberypi and it appears to give the same output as
ffmpeg -f rawvideo -pixel_format y16be -video_size "32x26" -i /dev/video0 -vf scale=288:234 -f fbdev -pix_fmt bgra /dev/fb0

there appears to be some delay between running python script and getting an image which i cant currently account for

Id like to test using a stream that doesn't use the camera but so far i haven't found anything that can produce a raw y16be stream, ffmpeg doesn't appear to convert to y16be, that i can send to the videoloopback

For anyone wanting to use the MLX90640 ive also discovered that this method of obtaining video is of limited use, as the MLX90640 image correction requires data embedded in the video stream and implicit scaling to 8bit corrupts this data.

have to set fourcc and disable rgb conversion before doing image correction and then convert to desired format

vs.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc('Y','1','6',' '))
vs.set(cv2.CAP_PROP_CONVERT_RGB, 0)

asenyaev · 2021-04-08T22:48:38Z

jenkins cn please retry a build

asmorkalov · 2021-10-05T10:14:31Z

Thanks for the patch! OpenCV runs on Big-Endian platforms too (like PPC), so the patch with bytes swap does not work there. please add endianess check.

firebladed · 2022-02-14T14:32:25Z

Thanks for the patch! OpenCV runs on Big-Endian platforms too (like PPC), so the patch with bytes swap does not work there. please add endianess check.

sorry for slow reply

im not sure exactly what you mean, im also not familiar with PPC

unless im missing something

V4L2_PIX_FMT_Y16 and V4L2_PIX_FMT_Y16_BE

are the same except that the MSB and LSB are swapped for each pixel

so to convert between them i just swap the MSB and LSB, processor endianess is irrelevent

what should i be checking for endianess?

does

       cv::split(temp,channels);
        cv::addWeighted(channels[0],256,channels[1],1,0,temp2,CV_16UC1);

not work on big endian platforms?

are you saying that i need to use this on big endian platforms?

       cv::split(temp,channels);
        cv::addWeighted(channels[1],256,channels[0],1,0,temp2,CV_16UC1);

mshabunin · 2023-03-20T09:04:08Z

To check current platform you can copy this function to cap_v4l.cpp:

opencv/modules/imgcodecs/src/utils.hpp

Lines 136 to 139 in c4226f0

    
           CV_INLINE bool  isBigEndian( void ) 
        
           { 
        
               return (((const int*)"\0\x1\x2\x3\x4\x5\x6\x7")[0] & 255) != 0; 
        
           }

On BE platform we don't need to swap bytes of V4L2_PIX_FMT_Y16_BE image, while V4L2_PIX_FMT_Y16 image should be transformed. So, perhaps your code should be extracted to a function and applied to either of the formats depending on the platform.

- https://www.kernel.org/doc/html/v4.10/media/uapi/v4l/pixfmt-y16-be.html - https://www.kernel.org/doc/html/v4.10/media/uapi/v4l/pixfmt-y16.html - 10-bits precision is not supported

opencv-alalek · 2023-07-12T17:13:27Z

According to documentation:

Implementation has been updated for using cv::extractChannel() call (no need to perform LE/BE checks for arithmetic operations)

mshabunin · 2023-07-12T19:25:42Z

Maybe we should also add a new test case to the videoio_v4l2.formats with Y16_BE type?

mshabunin · 2023-07-13T18:17:03Z

I've added this case to the test, it passes for me.

asmorkalov

👍
Tested manually with Ubuntu 18.04 + vivid driver

opencv-alalek · 2023-07-19T11:44:32Z

modules/videoio/test/test_precomp.hpp

    std::string res = fourccToString(fourcc);
-    std::replace_if(res.begin(), res.end(), [](uint8_t c){ return c < '0' || c > 'z'; }, '_');
+    // TODO: return hex values for invalid characters
+    std::transform(res.begin(), res.end(), res.begin(), [](uint8_t c) { return (c >= '0' && c <= 'z') ? c : (c == ' ' ? '_' : 'x'); });


There is build regression with MSVS2019/2017: https://pullrequest.opencv.org/buildbot/builders/4_x-win64-vc16/builds/100191

Related CI issue: opencv/ci-gha-workflow#108

Created PR #24087

Add V4L2_PIX_FMT_Y16_BE pixel format opencv#18498 Address opencv#18495 relates to opencv#23944 ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV - [ ] The PR is proposed to proper branch - [x] There is reference to original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake - [ ] Test using Melexis MLX90640

firebladed added 2 commits October 2, 2020 18:38

Add V4L2_PIX_FMT_Y16_BE pixel format

2997d25

Add V4L2_PIX_FMT_Y16_BE pixel format swaps byte order then treats as V4L2_PIX_FMT_Y16

Update cap_v4l.cpp

38072d8

change local variable "frame" to "temp2"

alalek reviewed Oct 16, 2020

View reviewed changes

asmorkalov added category: videoio category: videoio(camera) and removed category: videoio labels Nov 12, 2020

firebladed closed this Mar 4, 2021

firebladed deleted the patch-1 branch March 4, 2021 13:36

firebladed restored the patch-1 branch March 4, 2021 13:38

firebladed reopened this Mar 4, 2021

firebladed added 2 commits March 4, 2021 18:50

Update cap_v4l.cpp

ee598b9

Update cap_v4l.cpp

4e1c676

remove whitespace

firebladed marked this pull request as ready for review March 26, 2021 13:12

asmorkalov mentioned this pull request Sep 7, 2022

Add support for V4L2_PIX_FMT_Y16_BE #22455

Closed

mshabunin mentioned this pull request Jul 7, 2023

videoio: test for V4L using virtual device #23944

Merged

alalek added 2 commits July 12, 2023 17:05

videoio: use cv::extractChannel() for V4L2_PIX_FMT_Y16[_BE] modes

68bf72d

- https://www.kernel.org/doc/html/v4.10/media/uapi/v4l/pixfmt-y16-be.html - https://www.kernel.org/doc/html/v4.10/media/uapi/v4l/pixfmt-y16.html - 10-bits precision is not supported

Merge remote-tracking branch 'upstream/4.x'

c047720

opencv-pushbot force-pushed the patch-1 branch from a772380 to c047720 Compare July 12, 2023 17:11

opencv-alalek removed the incomplete label Jul 12, 2023

opencv-alalek added the feature label Jul 12, 2023

videoio/v4l2: test for Y16 and Y16BE

02dd4c9

asmorkalov requested a review from opencv-alalek July 14, 2023 07:08

asmorkalov approved these changes Jul 14, 2023

View reviewed changes

asmorkalov added the platform: linux label Jul 14, 2023

asmorkalov added this to the 4.9.0 milestone Jul 14, 2023

asmorkalov self-assigned this Jul 14, 2023

asmorkalov merged commit 7819ec7 into opencv:4.x Jul 14, 2023

opencv-alalek reviewed Jul 19, 2023

View reviewed changes

asmorkalov mentioned this pull request Jul 27, 2023

(5.x) Merge 4.x #24070

Merged

Uh oh!

Conversation

firebladed commented Oct 2, 2020 • edited by asmorkalov Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Uh oh!

alalek left a comment

Choose a reason for hiding this comment

Uh oh!

alalek Oct 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asmorkalov commented Nov 24, 2020

Uh oh!

firebladed commented Nov 24, 2020

Uh oh!

asmorkalov commented Mar 3, 2021

Uh oh!

firebladed commented Mar 6, 2021

Uh oh!

alalek commented Mar 6, 2021

Uh oh!

firebladed commented Mar 26, 2021

Uh oh!

asenyaev commented Apr 8, 2021

Uh oh!

asmorkalov commented Oct 5, 2021

Uh oh!

firebladed commented Feb 14, 2022

Uh oh!

mshabunin commented Mar 20, 2023

Uh oh!

opencv-alalek commented Jul 12, 2023

Uh oh!

mshabunin commented Jul 12, 2023

Uh oh!

mshabunin commented Jul 13, 2023

Uh oh!

asmorkalov left a comment

Choose a reason for hiding this comment

Uh oh!

opencv-alalek Jul 19, 2023

Choose a reason for hiding this comment

Uh oh!

opencv-alalek Jul 20, 2023

Choose a reason for hiding this comment

Uh oh!

mshabunin Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

firebladed commented Oct 2, 2020 •

edited by asmorkalov

Loading

alalek Oct 16, 2020 •

edited

Loading