Gstreamer backend ignores default CONVERT_RGB parameter for most formats #14084
Description
Detailed description
The default behaviour for capturing images in OpenCV is that cameras should return 8-bit 3-channel BGR images by default (i.e. CAP_PROP_CONVERT_RGB should be default-true for all capture backends).
Currently, the Gstreamer backend does not do this for single-channel formats, e.g. capturing a GRAY8 image will return a single channel image. I noticed this while I was writing some tests for #14035
Steps to reproduce
This test case will fail:
TEST(Videoio_Gstreamer, 8_bit_convert)
{
Size frame_size = Size(640,480);
int count_frames = 10;
string format = "video/x-raw, format=GRAY8;
std::ostringstream pipeline;
pipeline << "videotestsrc pattern=ball num-buffers=" << count_frames << " ! " << format;
pipeline << ", width=" << frame_size.width << ", height=" << frame_size.height << " ! appsink";
VideoCapture cap;
ASSERT_NO_THROW(cap.open(pipeline.str(), CAP_GSTREAMER));
ASSERT_TRUE(cap.isOpened());
ASSERT_TRUE(cap.get(CAP_PROP_CONVERT_RGB));
// Check initial result is 8-bit
Mat frame;
cap >> frame;
EXPECT_EQ(frame.size(), frame_size);
EXPECT_EQ(frame.depth(), CV_8U);
EXPECT_EQ(frame.channels(), 3);
cap.set(CAP_PROP_CONVERT_RGB, false);
Mat frame_mono;
cap >> frame_mono;
EXPECT_EQ(frame_mono.size(), frame_size);
EXPECT_EQ(frame_mono.depth(), CV_8U);
EXPECT_EQ(frame_mono.channels(), 1);
cap.release();
ASSERT_FALSE(cap.isOpened());
}Proposed fix
Add a variable to the class to hold the appropriate COLOR_X code, and make the conversion if appropriate. For example:
else if(strcasecmp(format, "NV12") == 0))
{
channels = 1;
convertCode = COLOR_YUV2RGB_NV12;
sz.height = sz.height * 3 / 2;
}Colour conversion would then be handled in GStreamerCapture::retrieveFrame.
I'm happy to make this change, but I'll wait until my PR for 16-bit has been approved since it already has some of the logic, something like:
Mat src;
if (isOutputByteBuffer){
src = Mat(Size(info.size, 1), CV_8UC1, info.data);
}else if(channels == 1 && shouldConvertRGB){
src = Mat(sz, CV_MAKETYPE(CV_8U, 3));
if(depth == 16){
unsigned char *src_ptr = src.data;
unsigned short *src_ptr_16 = reinterpret_cast<unsigned short*>(info.data)
for (int i = 0; i < width*height; i++) {
// Crush to 8-bit
unsigned char pixel_8 = static_cast<unsigned char>(src_ptr_16[i] >> 8);
;
*src_ptr = pixel_8;
src_ptr++;
*src_ptr = pixel_8;
src_ptr++;
*src_ptr = pixel_8;
src_ptr++;
}
}else{
Mat src_mono(sz, CV_MAKETYPE(CV_8U, 1), info.data);
cvtColor(src_mono, src, convertCode);
}
}else{
// Raw frame
if(depth == 16){
src = Mat(sz, CV_MAKETYPE(CV_16U, channels), info.data);
}else{
src = Mat(sz, CV_MAKETYPE(CV_8U, channels), info.data);
}
}