Skip to content

Fixed OpenVINO gemm layer#25518

Merged
asmorkalov merged 6 commits intoopencv:4.xfrom
alexlyulkov:al/fixed-gemm-openvino
May 14, 2024
Merged

Fixed OpenVINO gemm layer#25518
asmorkalov merged 6 commits intoopencv:4.xfrom
alexlyulkov:al/fixed-gemm-openvino

Conversation

@alexlyulkov
Copy link
Copy Markdown
Contributor

@alexlyulkov alexlyulkov commented May 1, 2024

Fixed OpenVINO gemm layer
The problem was that our layer didn't properly handle all the possible gemm options in OpenVINO mode
Fixes #25472

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@alexlyulkov alexlyulkov requested review from asmorkalov and dkurt May 1, 2024 10:01
@asmorkalov asmorkalov added this to the 4.10.0 milestone May 1, 2024
@asmorkalov
Copy link
Copy Markdown
Contributor

asmorkalov commented May 1, 2024

The related ONNX conformance test was skipped in Abduragim's PR. Please enable it back.

@dkurt
Copy link
Copy Markdown
Member

dkurt commented May 2, 2024

Please change a target branch to 4.x

@asmorkalov asmorkalov added the pr: needs rebase Rebase patch (and squash fixup commits) on the top of target branch label May 2, 2024
@alexlyulkov alexlyulkov force-pushed the al/fixed-gemm-openvino branch from e00f234 to 9a477eb Compare May 3, 2024 07:09
@alexlyulkov alexlyulkov changed the base branch from 5.x to 4.x May 3, 2024 07:10
Alexander Lyulkov added 2 commits May 3, 2024 13:15
@asmorkalov asmorkalov removed the pr: needs rebase Rebase patch (and squash fixup commits) on the top of target branch label May 3, 2024
@asmorkalov
Copy link
Copy Markdown
Contributor

asmorkalov commented May 3, 2024

The PR brings several test regressions with OpenVINO 2022.1.1:

[  FAILED  ] 26 tests, listed below:
[  FAILED  ] Test_Model.Keypoints_face/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_Model.TextRecognitionWithCTCPrefixBeamSearch/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Linear/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Multiplication/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Constant/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.LinearWithConstant/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.MatmulWithTwoInputs/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Alexnet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Googlenet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.CaffeNet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.RCNN_ILSVRC13/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ZFNet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ResNet18v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ResNet50v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.MobileNet_v2_FP16/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.LResNet100E_IR/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Inception_v2/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Inception_v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Shufflenet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Resnet34_kinetics/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ViT_B_32/0, where GetParam() = NGRAPH/CPU

@asmorkalov
Copy link
Copy Markdown
Contributor

asmorkalov commented May 3, 2024

Regressions with OpenVINO 2024.0:

[  FAILED  ] Test_Int8_nets.Inception_v2/1, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_Int8_nets.Shufflenet/1, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_Model.Keypoints_face/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_Model.TextRecognition/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_Model.TextRecognitionWithCTCPrefixBeamSearch/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Linear/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Multiplication/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.Constant/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.LinearWithConstant/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_layers.MatmulWithTwoInputs/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Alexnet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Googlenet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.CaffeNet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.RCNN_ILSVRC13/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ZFNet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ResNet18v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ResNet50v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.MobileNet_v2_FP16/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.LResNet100E_IR/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Inception_v2/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Inception_v1/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Shufflenet/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.Resnet34_kinetics/0, where GetParam() = NGRAPH/CPU
[  FAILED  ] Test_ONNX_nets.ViT_B_32/0, where GetParam() = NGRAPH/CPU

@asmorkalov
Copy link
Copy Markdown
Contributor

asmorkalov commented May 3, 2024

Failure example:

[----------] 1 test from Test_ONNX_nets
[ RUN      ] Test_ONNX_nets.Alexnet/0, where GetParam() = NGRAPH/CPU
unknown file: Failure
C++ exception with description "Check 'TRShape::broadcast_merge_into(output_shape, input_shapes[1], autob)' failed at src/core/shape_inference/include/eltwise_shape_inference.hpp:28:
While validating node 'opset1::Add Add_51 (opset1::MatMul MatMul_49[0]:f32[4096,1000], opset1::Constant Constant_50[0]:f32[1000,1]) -> (dynamic[...])' with friendly_name 'Add_51':
Argument shapes are inconsistent.
" thrown in the test body.
[  FAILED  ] Test_ONNX_nets.Alexnet/0, where GetParam() = NGRAPH/CPU (705 ms)
[----------] 1 test from Test_ONNX_nets (705 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (705 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] Test_ONNX_nets.Alexnet/0, where GetParam() = NGRAPH/CPU

fengyuentau
fengyuentau reviewed May 5, 2024
View reviewed changes
asmorkalov
asmorkalov reviewed May 13, 2024
View reviewed changes
modules/dnn/test/test_onnx_conformance_layer_parser_denylist.inl.hpp
Comment on lines -124 to -125
"test_gemm_default_matrix_bias",
"test_gemm_default_no_bias",
Copy link
Copy Markdown
Contributor

@asmorkalov asmorkalov May 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove.

asmorkalov
asmorkalov approved these changes May 13, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@asmorkalov asmorkalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@asmorkalov asmorkalov self-assigned this May 14, 2024
@asmorkalov asmorkalov merged commit 03507e0 into opencv:4.x May 14, 2024
klatism pushed a commit to klatism/opencv that referenced this pull request May 17, 2024
@alexlyulkov @klatism
Merge pull request opencv#25518 from alexlyulkov:al/fixed-gemm-openvino
1746e55 
Fixed OpenVINO gemm layer opencv#25518

Fixed OpenVINO gemm layer
The problem was that our layer didn't properly handle all the possible gemm options in OpenVINO mode
Fixes opencv#25472

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
@mshabunin mshabunin mentioned this pull request Jun 14, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fengyuentau fengyuentau fengyuentau left review comments

@asmorkalov asmorkalov asmorkalov approved these changes

@dkurt dkurt Awaiting requested review from dkurt

Assignees

@asmorkalov asmorkalov

Labels

category: dnn feature

Projects

None yet

Milestone

4.10.0

Development

Successfully merging this pull request may close these issues.

Test_ONNX_conformance.Layer_Test/test_gemm_default_matrix_bias_NGRAPH_CPU crashes

4 participants

@alexlyulkov @asmorkalov @dkurt @fengyuentau