Skip to content

Support for Unflatten operation requred by Attention layer - 4.x#25861

Merged
asmorkalov merged 3 commits intoopencv:4.xfrom
Abdurrahheem:ash/torch-attention-export-fix-4x
Jul 4, 2024
Merged

Support for Unflatten operation requred by Attention layer - 4.x#25861
asmorkalov merged 3 commits intoopencv:4.xfrom
Abdurrahheem:ash/torch-attention-export-fix-4x

Conversation

@Abdurrahheem
Copy link
Copy Markdown
Contributor

Pull Request Readiness Checklist

All test data and models for PR are located #1190

This PR fixes issue reised when importing batched vanilla Attention layer from PyTorch via ONNX. Currently batched version of Attention layer in PyTorch has unflatten operation inside. unflatten operation causes issue in reshape layer (see the Reshape_2 in the graph below) due to incorrect output of slice layer. This PR particularly fixes slice and concat layers to handle unflatten operation.

image

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@Abdurrahheem Abdurrahheem added this to the 4.11.0 milestone Jul 3, 2024
@Abdurrahheem Abdurrahheem requested review from asmorkalov and dkurt July 3, 2024 11:33
@Abdurrahheem Abdurrahheem self-assigned this Jul 3, 2024
@Abdurrahheem Abdurrahheem changed the title fix for unflatten operation (required by bached attention from pytorch) Support for Unflatten operation requred by Attention layer - 4.x Jul 3, 2024
@asmorkalov asmorkalov marked this pull request as ready for review July 4, 2024 05:53
@asmorkalov
Copy link
Copy Markdown
Contributor

@dkurt please take a look again.

@asmorkalov asmorkalov assigned dkurt and unassigned Abdurrahheem Jul 4, 2024
@asmorkalov asmorkalov added bug category: dnn category: dnn (onnx) ONNX suport issues in DNN module labels Jul 4, 2024
asmorkalov pushed a commit to opencv/opencv_extra that referenced this pull request Jul 4, 2024
…-fix-4x

Unflatten operation test data & models #1190

This PR creates models and test required by [#25861](opencv/opencv#25861)
@asmorkalov asmorkalov merged commit efbc9f0 into opencv:4.x Jul 4, 2024
@asmorkalov asmorkalov mentioned this pull request Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug category: dnn (onnx) ONNX suport issues in DNN module category: dnn

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants