Add CUDA backend for LSTM layer by JulieBar · Pull Request #20938 · opencv/opencv

JulieBar · 2021-10-25T11:25:36Z

Add CUDA backend for LSTM layer

force_builders=Custom
buildworker:Custom=linux-4,linux-6
Xbuild_image:Custom=ubuntu-cuda:18.04
build_image:Custom=ubuntu-cuda-cc52:18.04
test_modules:Custom=dnn

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
The PR is proposed to proper branch
There is reference to original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

alalek · 2021-10-28T12:31:13Z

BTW, Warning from https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html#torch.nn.LSTM :

There are known non-determinism issues for RNN functions on some versions of cuDNN and CUDA. You can enforce deterministic behavior by setting the following environment variables:

On CUDA 10.1, set environment variable CUDA_LAUNCH_BLOCKING=1. This may affect performance.

On CUDA 10.2 or later, set environment variable (note the leading colon symbol) CUBLAS_WORKSPACE_CONFIG=:16:8 or CUBLAS_WORKSPACE_CONFIG=:4096:2.

See the cuDNN 8 Release Notes for more information.

modules/dnn/src/cuda4dnn/csl/tensor_ops.hpp

modules/dnn/src/cuda4dnn/primitives/recurrent_cells.hpp

modules/dnn/src/layers/recurrent_layers.cpp

modules/dnn/src/cuda4dnn/csl/cudnn/cudnn.hpp

YashasSamaga · 2021-10-29T05:46:11Z

Does the CI build with both cuDNN 7 and cuDNN 8?

(Sorry, didn't notice this was a draft)

JulieBar · 2021-10-29T17:06:21Z

Does the CI build with both cuDNN 7 and cuDNN 8?

(Sorry, didn't notice this was a draft)

I'm going to ask for your review anyway :) Thank you!
I'll address your comments, correct what's left, and ping you again later if you don't mind.

rogday

LGTM! 👍

rogday · 2022-03-30T14:56:19Z

@YashasSamaga, could you please take a look?

Co-authored-by: Julia Bareeva <jbareeva@gmail.com>