Skip to content

Fix LSTM support in ONNX#21522

Merged
alalek merged 8 commits intoopencv:3.4from
rogday:lstm
Mar 15, 2022
Merged

Fix LSTM support in ONNX#21522
alalek merged 8 commits intoopencv:3.4from
rogday:lstm

Conversation

@rogday
Copy link
Copy Markdown
Member

@rogday rogday commented Jan 26, 2022

Merge after #21542
Merge with extra: opencv/opencv_extra#959

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@rogday rogday added bug pr: needs test New functionality requires minimal tests set category: dnn (onnx) ONNX suport issues in DNN module labels Jan 26, 2022
@rogday rogday force-pushed the lstm branch 2 times, most recently from fc993d8 to 215bf8d Compare January 28, 2022 16:42
@rogday rogday changed the base branch from 4.x to 3.4 February 16, 2022 12:35
@rogday rogday force-pushed the lstm branch 4 times, most recently from def4fc1 to 9686f5c Compare February 17, 2022 17:29
@rogday rogday added feature test and removed pr: needs test New functionality requires minimal tests set labels Feb 17, 2022
@rogday rogday marked this pull request as ready for review February 17, 2022 18:19
@asmorkalov
Copy link
Copy Markdown
Contributor

@Daniil-Osokin could you take a look and comment?

@rogday rogday marked this pull request as draft February 25, 2022 12:30
@rogday rogday force-pushed the lstm branch 2 times, most recently from 3681772 to 3baf334 Compare March 1, 2022 23:47
@rogday rogday marked this pull request as ready for review March 2, 2022 02:31

// CUDA needs input blobs to be rearranged in a specific way, but some transformations
// in ONNXImporter are destructive, so we keep a copy.
std::vector<Mat> cudaWorkaround;
Copy link
Copy Markdown
Member

@alalek alalek Mar 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR is targeted to 3.4 branch, so there is no CUDA in DNN.
Is there related PR targeted to 4.x?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there is. In 4.x branch the contents of this vector will be used, but here its purpose is to tell if we need any onnx-related transformations after forward.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid confusion we need to rename this field (there is no CUDA code in DNN module for 3.4 branch).
What is about originalBlobs or other more clear name?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, fixed.

@Daniil-Osokin
Copy link
Copy Markdown

Math looks ok, so just style commentaries 👍.

@alalek alalek merged commit 93353ae into opencv:3.4 Mar 15, 2022
@rogday rogday deleted the lstm branch March 15, 2022 07:56
@alalek alalek mentioned this pull request Mar 26, 2022
yash112-lang pushed a commit to yash112-lang/opencv that referenced this pull request Mar 31, 2022
Fix LSTM support in ONNX

* fix LSTM and add peephole support

* disable old tests

* turn lambdas into functions

* more hacks for  c++98

* add assertions

* slice fixes

* backport of cuda-related fixes

* address review comments
@opencv-pushbot opencv-pushbot mentioned this pull request Apr 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug category: dnn (onnx) ONNX suport issues in DNN module feature test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants