Created opencv_stitching_tool App (based on Python API)#20758
Created opencv_stitching_tool App (based on Python API)#20758lukasalexanderweber wants to merge 13 commits intoopencv:4.xfrom lukasalexanderweber:master
Conversation
* seperated stitching interface (stitching_detailed.py) from functionality (stitching_detailed package) * Added unit test * FeatureDetection functionality in class * FeatureMatching functionality in class * Detected and described bug in image and matches subsetting * Fixed and extracted subsetting of features/matches in class * Excluded matches graph dot file saving into Subsetter class. * Camera functionality in classes * Created Downscaler class * Image Resizing functionality in class * Warper functionality in class * ExposureCompensation functionality into class * Seam Finding functionality in class * Blender functionality in class * Timelapser functionality in class * Created stitcher class * Added possibility to set nr. of keypoints/features * Created performance test * Resize small images can be based on medium images to improve processing time / RAM usage * Created generators in image composition to follow stitching pipeline * Excluded panorama dimension estimation into module
* added keypoint drawing functionality * added matches drawing functionality * added seam drawing functionality
Wondering if Microsoft ICE software would be able to stitch these many images? If you want to give it a try, you can download it here. |
|
@catree thanks for the tip! However, the performance of the opencv algorithm is very good and I think there is no need to switch to another software. My work is part of a research project and I am happy to have complete control over the stitching process. Also, my institute focuses on working with and improving open source software. I just want to share the (for me it seems like) improvements I did :) |
|
👍 Any effort to improve OpenCV is really great. I am using ICE to stitch my personal photos since it is really easy to use and provide good results for me. I was just wondering if ICE would be able to deal with 10000 images, independently of the performance of OpenCV. |
|
I can't test it since I have some noise elements which were put on the plans during scanning (e.g. Barcodes). I resize the scans to medium resolution through the OpenCV Stitcher API, then remove noise elements using object detection, and then match the images (since the Barcodes disturb a correct matching). So I'm quite sure ICE wouldn't work as I desire. |
* created python image stitching tutorial (jupyter notebook) * updated timescaler and blender returns (int8) * added line size functionality for seam line drawing
|
@lukasalexanderweber Thank you for contribution! Currently OpenCV samples are small enough applications that demonstrate OpenCV API. OpenCV Dev team need to discuss how to properly place this contribution and promote to users. I hope we will back with proposal in 2 weeks. /cc @LaurentBerger |
|
Nice contribution. About App in opencv it's easy to include in opencv but how to maintain? |
|
@LaurentBerger Thanks! Nice to know that it's a process of translating cpp directly in py. On the long run it might be desirable to bring the cpp example to a similar structure as the here proposed (If I would know how to programm in C++ I would have done it directly..). I know that having background code on a single sample structured in modules/package is not exactly brilliant. However I think a clear structure (instead of 900 lines in one file) can rather help a new user than
If this would happen, the proposed Tutorial could be converted into an official opencv tutorial as SVMs where the user can switch between the programming languages. It would as well solve the maintainace problem. I'm open for any short-term suggestions where to put the code, if at all. |
|
Hello @lukasalexanderweber. Thanks for the hard work. As alalek mentioned OpenCV samples are designed to be relatively small and simple to understand how OpenCV works and how to use it for particular use cases. We discussed the PR on OpenCV team meeting and what to recommend you to restore original stitching_detialed.py in samples/python location and move your version to apps folder together with the tutorial. It's great tool for experiments and research. |
|
Also consider renaming application to |
* restored functionality in app folder * aligned stitcher parameters with single class parameters, updated seam drawing, removed privacy of functions * updated tutorial
|
@alalek @asmorkalov I restored the work into the app folder Please note opencv/opencv_extra#915 |
|
@alalek I changed the tutoral image sizes from opencv/opencv_extra#915 to opencv/opencv_extra#919 |
The patent expired 2020: #17119
|
@alalek at the moment I noticed that SIFT is no longer patented and already in the main repository: #17119 In stitching_detailed.py however it still tries to import it from xfeatures (cv.xfeatures2d_SIFT.create) instead of using cv.SIFT_create. In the new app I changed it already. Can I also change it in stitching_detailed.py or should I open a new pull request for that? |
|
@lukasalexanderweber You are right! Please open a new PR for this fix. |
|
@alalek can you say if and when the App will be merged? I have a new Feature and would like to present it in a new Pull Request |
|
Thank you for update! Should we re-open opencv_extra PR: opencv/opencv_extra#915 ? BTW, opencv_extra usually contains test data for tests.
This file is large >10Mb. |
I submitted a new PR with reduced image sizes: opencv/opencv_extra#919
I moved the Tutorial into opencv_stitching_tutorial and created a small Readme with a link |
|
Looks good! Could you please move images from opencv_extra's PR to this tutorial repository? (probably in original non-compressed size) Please take a look on whitespace issues here. |
|
Please see #21020 |
relates #20744 #20723
tutorial needs data from opencv/opencv_extra#915
Introduction
The advanced stitching example stitching_detailed.py is a big script with awesome functionality. It is, however, very long and a bit unstructured to grasp what's going on. It's by far the most complex script in the python samples folder:
There is, however, a very clear pipeline description.
Refactoring
I changed the stitching_detailed.py so that it only contains the argument parser. At the end, the functionality of the stitcher class is called. The stitcher class follows precisely the stitching pipeline of the image. Unlike in the original program, the intermediate results can be obtained and analysed easily.
I didn't changed much of the functionality, the most work was done in structuring the functionality.
I added the possibility to specify the number of features (default 500) in the Feature Detection step, since 500 features are not enough for my REALY big plans (see this issue/question).
The final results (from blender or timelapser) are returned in 8-bit numpy arrays so they can be saved on disk and viewed using cv.imshow(). The conversion is done using cv::convertScaleAbs
Problems during the stitching pipeline raise a
StitchingErrorAdditionally, I added functionality to subset the matches and cameras, which was not done in the original file. If there are noise images at the beginning, this leads to significant artefacts:
python stitching_detailed.py s1.jpg s2.jpg boat5.jpg boat2.jpg boat3.jpg boat4.jpg boat1.jpg boat6.jpg --compose_megapix 3 --output boat_default_with_noise_original.jpg --save_graph boat_default_with_noise_original.txtWith the proposed changes, this is not an issue anymore. The performance speed is the same as before the restructuring, the RAM needed reduced for the boat example from 130MB to 95MB. See test_performance.py. The tests can be found here. With a view to the tutorial, I tested different settings for the boat example as well as the affine example of the budapest map within test_stitcher.py
Drawing Functionality
For intermediate results I added some drawing functionality, e.g. the match inliers or the seam masks upon the final panorama:
Tutorial
For the refactored functioality and the new drawing functionality I created a Jupyter Notebook Tutorial.
Background
I need to stitch >10000 plans of waterway-infrastructure buildings. During digitalization, the scanner was too small for the whole plan. There are overlaps that allow the use of the stitching algorithm and I'm thankful that OpenCV contains the stitcher module. For documentation purposes for my work I wanted to structure the code and now hope to discuss if the changes might could merged into the opencv master branch.
Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.