System information (version)

• OpenCV 4.2 tested, but issue probably present in all versions of OpenCV
• cv2 4.5.1 & 3.4.7 tested, but issue probably present in all versions of cv2

Detailed description

Cameras often produce images in one of the Bayer CFA (Color Filter Array) patterns: BayerRG (aka RGGB), BayerBG (BGGR), BayerGB (GBRG), and BayerGR (GRBG).

Every single reference and software (other than OpenCV) I've seen out there uses the top left 2x2 sub-matrix of the CFA to name the Bayer pattern. For example,

• MATLAB: https://www.mathworks.com/help/images/ref/demosaic.html
• Camera manufacturer Basler: https://www.baslerweb.com/en/sales-support/knowledge-base/frequently-asked-questions/bayer-to-rgb-and-missed-pixel-value-at-last-row-and-line/15245/

OpenCV, on the other hand, uses the 2x2 sub-matrix that starts at the 2nd row and 2nd column of the CFA:
https://docs.opencv.org/master/de/d25/imgproc_color_conversions.html

Because of this, what the rest of the world calls a BayerRG pattern, OpenCV calls a BayerBG pattern.

So, if I configure a camera to produce BayerRG images, and if I use the COLOR_BayerRG2RGB code to convert the images into an RGB layout, I will actually get a BGR layout. I will need to use the COLOR_BayerBG2RGB code to get the RGB layout.

I believe this naming inconsistency is the source of all the confusion expressed in several other issues/threads, including:

• https://forum.opencv.org/t/bayerrg-rgb-conversion/1466
• readbayer color channels are out of order danforthcenter/plantcv#642
• cvtcolor bayer RGB conversions incorrectl aliases for BGR #4857

OpenCV demosaicing seems to need an update to align with the rest of the world.

Steps to reproduce behavior in cv2

import cv2
import numpy as np
import matplotlib.pyplot as plt

# Simple 12x6 array 
I = np.uint8([
    255, 0, 255, 0, 255, 0, 
    0, 0, 0, 0, 0, 0,
    255, 0, 255, 0, 255, 0,
    0, 0, 0, 0, 0, 0,
    255, 0, 255, 0, 255, 0,
    0, 0, 0, 0, 0, 0,
    255, 0, 255, 0, 255, 0,
    0, 0, 0, 0, 0, 0,
    255, 0, 255, 0, 255, 0,
    0, 0, 0, 0, 0, 0,
    255, 0, 255, 0, 255, 0,
    0, 0, 0, 0, 0, 0
          ]).reshape(12, 6)

# Interpret the array as a BayerRG image and convert BayerRG --> RGB
# As per common definition of BayerRG, the array represents an all-red image
# NOTE: We know OpenCV often uses the BGR layout by default, but here I'm asking for RGB layout explicitly
J = cv2.cvtColor(I, cv2.COLOR_BayerRG2RGB)

# I expected an all-red image, but this display will show an all-blue image
plt.imshow(J)

# To confirm, this will print [0 0 255] for all RGB pixels
print(J)

Steps to compare the above results with MATLAB's

Feed the same example to MATLAB with the following code, by going to https://www.mathworks.com/help/images/ref/demosaic.html and clicking on “Try This Example” to open interactive coding window. MATLAB will display an all-red image as expected.

I = uint8([
255, 0, 255, 0, 255, 0; ...
0, 0, 0, 0, 0, 0; ...
255, 0, 255, 0, 255, 0; ...
0, 0, 0, 0, 0, 0; ...
255, 0, 255, 0, 255, 0; ...
0, 0, 0, 0, 0, 0; ...
255, 0, 255, 0, 255, 0; ...
0, 0, 0, 0, 0, 0; ...
255, 0, 255, 0, 255, 0; ...
0, 0, 0, 0, 0, 0; ...
255, 0, 255, 0, 255, 0; ...
0, 0, 0, 0, 0, 0
]);

J = demosaic(I,'rggb');

fig=figure, imshow(J); truesize(fig, [48, 24]);