System information (version)

• OpenCV => 3.2.0
• Operating System / Platform => Windows 10 64-bit / Python
• Compiler => MSC v.1900 64 bit (AMD64)

Detailed description

Using the VNG demosaicing in cvtColor( ) results in edge artifacts for edges running at orientations of 315 degrees (roughly NW -> SE). You can see these artifacts in the image below created with the OpenCV VNG algorithm (top row), but not in an image created using a reference implementation of the VNG algorithm (in attached code vng.py) (bottom row). Some color artifacts along edges are expected due to the nature of the interpolation, but the artifacts shown here on edges running from upper left to lower right are more severe than expected, and worse than the reference output. Displaying a single channel (such as just the red channel shown on the right) makes it easier to see that pixels far from the edge are being modified in the OpenCV version.

For edges running along different orientations, such as 45 degrees (roughly SW -> NE), the OpenCV VNG implementation does not show artifacts, and appears to match the reference VNG implementation, as shown below.

Since the VNG algorithm computes gradients in the eight compass directions and there are separate blocks of code for processing each gradient direction, is it possible that there is a pointer error in the code for one of the gradient directions ?

Steps to reproduce

The examples above were generated by calling OpenCV from Python inside a Jupyter Notebook.
The input and output images for the examples above are attached, and can be duplicated by running the attached notebook "VNG_bug.ipynb"

VNG_bug.zip