NVIDIA_OPTICAL_FLOW_2_0_INTEGRATION by vchiluka5 · Pull Request #2807 · opencv/opencv_contrib

vchiluka5 · 2021-01-06T04:40:55Z

This PR integrates NVIDIA'S Optical Flow SDK 2.0.

[*] I agree to contribute to the project under Apache 2 License.
[*] To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
[*] The PR is proposed to proper branch
[*] There is reference to original bug report and related work
[*] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
[*] The feature is well documented and sample code can be built with the project CMake

force_builders=Custom
buildworker:Custom=linux-4
build_image:Custom=ubuntu-cuda:18.04

vchiluka5 · 2021-01-08T15:59:49Z

I see python bindings error. @alalek Can you please suggest what i can change to fix those?

alalek · 2021-01-09T04:02:00Z

You can disable bindings generation for this class:

remove _W from CV_EXPORTS_W
remove CV_WRAP

vchiluka5 · 2021-01-09T09:26:16Z

You can disable bindings generation for this class:

remove _W from CV_EXPORTS_W

remove CV_WRAP

we will need python bindings since some of our users use python version of above class.
is there any other way to solve this error?

alalek

we will need python bindings

Consider adding simple test for Python API (don't check accuracy, only parameters passing/return types/sizes).
Use this as an example: https://github.com/opencv/opencv_contrib/blob/4.5.1/modules/cudaobjdetect/misc/python/test/test_cudaobjdetect.py

modules/cudaoptflow/include/opencv2/cudaoptflow.hpp

alalek

Thank you for update!

modules/cudaoptflow/include/opencv2/cudaoptflow.hpp

alalek

Well done!

modules/cudaoptflow/misc/python/test/test_nvidiaopticalflow.py

modules/cudaoptflow/include/opencv2/cudaoptflow.hpp

modules/cudaoptflow/src/nvidiaOpticalFlow.cpp

tomoaki0705 · 2021-01-17T21:45:26Z

Hi @vchiluka5
Since this PR was merged, I see build failure on Jetson TX1 and TX2.
The build passes on Xavier, NX and Nano, which has CUDA 10.0 or later, but my TX1 and TX2 still has CUDA 8.0.

The error message is as following.

[ 57%] Building NVCC (Device) object modules/cudaoptflow/CMakeFiles/cuda_compile_1.dir/src/cuda/cuda_compile_1_generated_nvidiaOpticalFlow.cu.o
/opencv_contrib/modules/cudaoptflow/src/cuda/nvidiaOpticalFlow.cu(76): error: no instance of overloaded function "surf2Dwrite" matches the argument list
            argument types are: (short2, cudaSurfaceObject_t, unsigned long, int, cudaSurfaceBoundaryMode)

1 error detected in the compilation of "/tmp/tmpxft_00001b35_00000000-7_nvidiaOpticalFlow.cpp1.ii".
CMake Error at cuda_compile_1_generated_nvidiaOpticalFlow.cu.o.Release.cmake:279 (message):
  Error generating file
  /opencv/build/modules/cudaoptflow/CMakeFiles/cuda_compile_1.dir/src/cuda/./cuda_compile_1_generated_nvidiaOpticalFlow.cu.o


modules/cudaoptflow/CMakeFiles/opencv_cudaoptflow.dir/build.make:70: recipe for target 'modules/cudaoptflow/CMakeFiles/cuda_compile_1.dir/src/cuda/cuda_compile_1_generated_nvidiaOpticalFlow.cu.o' failed
make[2]: *** [modules/cudaoptflow/CMakeFiles/cuda_compile_1.dir/src/cuda/cuda_compile_1_generated_nvidiaOpticalFlow.cu.o] Error 1
CMakeFiles/Makefile2:18502: recipe for target 'modules/cudaoptflow/CMakeFiles/opencv_cudaoptflow.dir/all' failed
make[1]: *** [modules/cudaoptflow/CMakeFiles/opencv_cudaoptflow.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

I guess you didn't intend to break this but it seems that this code has no longer compatibility with CUDA 8.0.
Could you have a look on this issue please ? (I guess that's quicker to fix(

Best

vchiluka5 · 2021-01-18T09:04:23Z

Hi @vchiluka5
Since this PR was merged, I see build failure on Jetson TX1 and TX2.
The build passes on Xavier, NX and Nano, which has CUDA 10.0 or later, but my TX1 and TX2 still has CUDA 8.0.

The error message is as following.

[ 57%] Building NVCC (Device) object modules/cudaoptflow/CMakeFiles/cuda_compile_1.dir/src/cuda/cuda_compile_1_generated_nvidiaOpticalFlow.cu.o
/opencv_contrib/modules/cudaoptflow/src/cuda/nvidiaOpticalFlow.cu(76): error: no instance of overloaded function "surf2Dwrite" matches the argument list
            argument types are: (short2, cudaSurfaceObject_t, unsigned long, int, cudaSurfaceBoundaryMode)

1 error detected in the compilation of "/tmp/tmpxft_00001b35_00000000-7_nvidiaOpticalFlow.cpp1.ii".
CMake Error at cuda_compile_1_generated_nvidiaOpticalFlow.cu.o.Release.cmake:279 (message):
  Error generating file
  /opencv/build/modules/cudaoptflow/CMakeFiles/cuda_compile_1.dir/src/cuda/./cuda_compile_1_generated_nvidiaOpticalFlow.cu.o


modules/cudaoptflow/CMakeFiles/opencv_cudaoptflow.dir/build.make:70: recipe for target 'modules/cudaoptflow/CMakeFiles/cuda_compile_1.dir/src/cuda/cuda_compile_1_generated_nvidiaOpticalFlow.cu.o' failed
make[2]: *** [modules/cudaoptflow/CMakeFiles/cuda_compile_1.dir/src/cuda/cuda_compile_1_generated_nvidiaOpticalFlow.cu.o] Error 1
CMakeFiles/Makefile2:18502: recipe for target 'modules/cudaoptflow/CMakeFiles/opencv_cudaoptflow.dir/all' failed
make[1]: *** [modules/cudaoptflow/CMakeFiles/opencv_cudaoptflow.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

I guess you didn't intend to break this but it seems that this code has no longer compatibility with CUDA 8.0.
Could you have a look on this issue please ? (I guess that's quicker to fix(

Best

Optical flow API supports from CUDA 10 and above. You can add an exception which doesnt include nvidia_optical_flow using
-DHAVE_NVIDIA_OPTFLOW=0.

vchiluka5 force-pushed the NVIDIA_OPTICAL_FLOW_SDK_2.0_INTEGRATION branch 2 times, most recently from 7133697 to 4698d90 Compare January 8, 2021 14:15

vchiluka5 force-pushed the NVIDIA_OPTICAL_FLOW_SDK_2.0_INTEGRATION branch from 4698d90 to b82f091 Compare January 12, 2021 09:13

alalek reviewed Jan 12, 2021

View reviewed changes

modules/cudaoptflow/include/opencv2/cudaoptflow.hpp Outdated Show resolved Hide resolved

modules/cudaoptflow/include/opencv2/cudaoptflow.hpp Outdated Show resolved Hide resolved

alalek reviewed Jan 12, 2021

View reviewed changes

modules/cudaoptflow/include/opencv2/cudaoptflow.hpp Outdated Show resolved Hide resolved

modules/cudaoptflow/include/opencv2/cudaoptflow.hpp Outdated Show resolved Hide resolved

vchiluka5 force-pushed the NVIDIA_OPTICAL_FLOW_SDK_2.0_INTEGRATION branch 2 times, most recently from 20d72fe to 4a60183 Compare January 15, 2021 07:46

alalek reviewed Jan 15, 2021

View reviewed changes

modules/cudaoptflow/include/opencv2/cudaoptflow.hpp Outdated Show resolved Hide resolved

vchiluka5 force-pushed the NVIDIA_OPTICAL_FLOW_SDK_2.0_INTEGRATION branch 2 times, most recently from 55bd5de to 57d7cb1 Compare January 15, 2021 08:43

alalek reviewed Jan 15, 2021

View reviewed changes

modules/cudaoptflow/misc/python/test/test_nvidiaopticalflow.py Outdated Show resolved Hide resolved

modules/cudaoptflow/include/opencv2/cudaoptflow.hpp Outdated Show resolved Hide resolved

vchiluka5 force-pushed the NVIDIA_OPTICAL_FLOW_SDK_2.0_INTEGRATION branch from 57d7cb1 to a6a9d49 Compare January 16, 2021 04:43

alalek reviewed Jan 16, 2021

View reviewed changes

modules/cudaoptflow/src/nvidiaOpticalFlow.cpp Show resolved Hide resolved

NVIDIA_OPTICAL_FLOW_2_0_INTEGRATION

582fe44

vchiluka5 force-pushed the NVIDIA_OPTICAL_FLOW_SDK_2.0_INTEGRATION branch from a6a9d49 to 582fe44 Compare January 16, 2021 08:01

alalek approved these changes Jan 16, 2021

View reviewed changes

opencv-pushbot merged commit 1c957be into opencv:master Jan 16, 2021

vchiluka5 deleted the NVIDIA_OPTICAL_FLOW_SDK_2.0_INTEGRATION branch January 17, 2021 09:43

tomoaki0705 mentioned this pull request Jan 18, 2021

cudaoptflow: disable Optical Flow SDK when CUDA version is insufficient #2824

Merged

6 tasks

This was referenced Jan 19, 2021

Implement cv::cuda::inRange (Fixes OpenCV #6295) #2803

Merged

add cuda::Stream constructor with cuda stream flags opencv/opencv#19286

Merged

alalek mentioned this pull request Apr 9, 2021

(5.x) Merge 4.x #2919

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NVIDIA_OPTICAL_FLOW_2_0_INTEGRATION#2807

NVIDIA_OPTICAL_FLOW_2_0_INTEGRATION#2807
opencv-pushbot merged 1 commit intoopencv:masterfrom
vchiluka5:NVIDIA_OPTICAL_FLOW_SDK_2.0_INTEGRATION

vchiluka5 commented Jan 6, 2021 •

edited by alalek

Loading

Uh oh!

vchiluka5 commented Jan 8, 2021

Uh oh!

alalek commented Jan 9, 2021

Uh oh!

vchiluka5 commented Jan 9, 2021

Uh oh!

alalek left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alalek left a comment

Uh oh!

Uh oh!

alalek left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tomoaki0705 commented Jan 17, 2021

Uh oh!

vchiluka5 commented Jan 18, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

vchiluka5 commented Jan 6, 2021 • edited by alalek Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vchiluka5 commented Jan 8, 2021

Uh oh!

alalek commented Jan 9, 2021

Uh oh!

vchiluka5 commented Jan 9, 2021

Uh oh!

alalek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alalek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alalek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tomoaki0705 commented Jan 17, 2021

Uh oh!

vchiluka5 commented Jan 18, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vchiluka5 commented Jan 6, 2021 •

edited by alalek

Loading