System information (version)
- OpenCV => 3.4.1
- Operating System / Platform => any
- Compiler => any
Detailed description
Several bugs were introduced as a fix of the issue #6336. The issue reporter provided an erroneous example in which a camera matrix of a scaled image was computed by simply multiplying the matrix with the scaling factor that was used to scale the image (see my comment in the #6336). OpenCV code for the mentioned functions (and all that depend upon) was correct up until that point and is now buggy and inconsistent.
The following changes, a86a5c7#diff-e0620279a9a42557449008a88b6ed816 and fa42a1c#diff-e0620279a9a42557449008a88b6ed816 should be reverted. If anything else was changed because of #6336 - I only checked the calibration.cpp - revert it as well.
The bugs were partly caused by the lack of the documentation. To reduce false issue reports in the future, the origin of the image coordinate system should be noted in the calib3d documentation.