Skip to content

Adding OpenCV's GPU GFTT/ OpticalFlow and CudaSift support#1330

Merged
matlabbe merged 15 commits intomasterfrom
gftt_sift_cuda
Sep 13, 2024
Merged

Adding OpenCV's GPU GFTT/ OpticalFlow and CudaSift support#1330
matlabbe merged 15 commits intomasterfrom
gftt_sift_cuda

Conversation

@matlabbe
Copy link
Copy Markdown
Member

@matlabbe matlabbe commented Aug 24, 2024

These new features are available if built with OpenCV CUDA.

New features:

  • Faster GFTT: with GFTT/Gpu=true
  • Faster optical flow based F2F Odometry: with Vis/CorFlowGpu=true
  • Faster stereo correspondences: with Stereo/Gpu=true
  • Faster SIFT features extraction (with new CudaSift dependency): with SIFT/Gpu=true
  • Faster F2F Odometry approach using parameters above like this: --Odom/Strategy 1 --Vis/CorFlowGpu true --Vis/CoreType 1 --Stereo/Gpu true --GFTT/Gpu true --Vis/BundleAdjustment 0

Use this fork to build CudaSift: https://github.com/matlabbe/CudaSift

@matlabbe matlabbe self-assigned this Aug 24, 2024
@matlabbe matlabbe marked this pull request as ready for review August 27, 2024 21:50
@matlabbe matlabbe changed the title Adding GFTT and SIFT Cuda support Adding GFTT and CudaSift support Aug 29, 2024
@matlabbe matlabbe changed the title Adding GFTT and CudaSift support Adding GFTT cuda and CudaSift support Aug 29, 2024
@matlabbe matlabbe changed the title Adding GFTT cuda and CudaSift support Adding OpenCV's GFTT-GPU and CudaSift support Aug 29, 2024
@matlabbe
Copy link
Copy Markdown
Member Author

matlabbe commented Aug 29, 2024

When building rtabmap_ros against rtabmap with this PR (on Ubuntu 20.04 ROS Noetic with local version of OpenCV 4.2+Cuda shadowing OpenCV system binaries), we can get this error:

/usr/bin/ld: /usr/local/lib/libopencv_cudafeatures2d.so.4.2.0: undefined reference to `cv::cuda::device::ThrustAllocator::getAllocator()'
/usr/bin/ld: /usr/local/lib/libopencv_cudafeatures2d.so.4.2.0: undefined reference to `cv::cuda::StreamAccessor::getStream(cv::cuda::Stream const&)'

It seems to be because during linking (see with make VERBOSE=1), the binary /usr/lib/x86_64-linux-gnu/libopencv_core.so.4.2.0 is in the list (because of ros dependency cv_bridge) before /usr/local/lib/libopencv_core.so.4.2.0, so cuda core functions are not found. Well, a workaround if we don't want to rebuild cv_bridge is to make a symlink from /usr/lib/x86_64-linux-gnu/libopencv_core.so.4.2.0 to /usr/local/lib/libopencv_core.so.4.2.0 (backup original binary file to revert afterwards) like this:

/usr/lib/x86_64-linux-gnu$ ls -l libopencv_core*
-rw-r--r-- 1 root root 9124066 Feb 17  2020 libopencv_core.a
lrwxrwxrwx 1 root root      21 Feb 17  2020 libopencv_core.so -> libopencv_core.so.4.2
lrwxrwxrwx 1 root root      38 Aug 29 16:50 libopencv_core.so.4.2 -> /usr/local/lib/libopencv_core.so.4.2.0
lrwxrwxrwx 1 root root      38 Aug 29 16:51 libopencv_core.so.4.2.0 -> /usr/local/lib/libopencv_core.so.4.2.0
-rw-r--r-- 1 root root 3840856 Feb 17  2020 libopencv_core.so.4.2.0.back

Note that this would only work if you rebuild exactly the same OpenCV version than the system one to avoid any conflicts.

@matlabbe
Copy link
Copy Markdown
Member Author

matlabbe commented Aug 31, 2024

Loop closure performance comparison with CudaSift (pretty similar to OpenCV's SURF and SIFT):
image

@matlabbe matlabbe changed the title Adding OpenCV's GFTT-GPU and CudaSift support Adding OpenCV's GPU GFTT/ OpticalFlow and CudaSift support Sep 1, 2024
@matlabbe matlabbe merged commit 69ac21f into master Sep 13, 2024
@matlabbe matlabbe deleted the gftt_sift_cuda branch September 13, 2024 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant