G-API: Fix incorrect OpaqueKind for Kernel outputs by TolyaTalamanov · Pull Request #23843 · opencv/opencv

TolyaTalamanov · 2023-06-21T18:24:42Z

Pull Request Readiness Checklist

Overview

The PR is going to fix several problems:

Major: GKernel doesn't hold kind for its outputs. Since GModelBuilder traverse graph from outputs to inputs once it reaches any output of the operation it will use its kind to create Data meta for all operation outputs. Since it essential for python to know GTypeInfo (which is shape and kind) it will be confused.

Consider this operation:

 @cv.gapi.op('custom.square_mean', in_types=[cv.GArray.Int], out_types=[cv.GOpaque.Float, cv.GArray.Int])
    class GSquareMean:
        @staticmethod
        def outMeta(desc):
            return cv.empty_gopaque_desc(), cv.empty_array_desc()

Even though GOpaque is Float, corresponding metadata might have Int kind because it might be taken from cv.GArray.Int
so it will be a problem if one of the outputs of these operation is graph output because python will cast it to the wrong type based on Data meta.

Minor: Some of the OpenVINO IR's doesn't any layout information for input. It's usually true only for IRv10 but since OpenVINO 2.0 need this information to correctly configure resize we need to put default layout if there no such assigned in ov::Model.

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

TolyaTalamanov · 2023-06-21T18:29:27Z

modules/gapi/misc/python/test/test_gapi_sample_pipelines.py

+            pkg = cv.gapi.kernels(GSquareMeanImpl)
+            mean, squares = comp.apply(cv.gin([1,2,3]), args=cv.gapi.compile_args(pkg))
+
+            self.assertEqual([1,4,9], list(squares))


@asmorkalov @opencv-alalek I'm a little bit confused why we convert std::vector<T> to python tuple instead of list?
https://github.com/opencv/opencv/blob/4.x/modules/python/src2/cv2_convert.hpp#L359-L362

TolyaTalamanov · 2023-06-21T19:00:15Z

@dmatveev Could you have a look, please?

TolyaTalamanov · 2023-06-21T19:00:23Z

@smirnov-alexey

dmatveev

👍 Fantastic, thanks for the quick turn-around!!!
Probably a short problem description would still make sense - for historic reasons..

modules/gapi/include/opencv2/gapi/gkernel.hpp

dmatveev · 2023-06-21T20:01:58Z

modules/gapi/include/opencv2/gapi/streaming/desync.hpp

        , {cv::detail::GTypeTraits<G>::shape}               // output Shape
        , {cv::detail::GTypeTraits<G>::op_kind}             // input data kinds
        , {cv::detail::GObtainCtor<G>::get()}               // output template ctors
+        , {cv::detail::GTypeTraits<G>::op_kind}             // output data kinds


Suggested change

, {cv::detail::GTypeTraits<G>::op_kind} // output data kinds

, {cv::detail::GTypeTraits<G>::op_kind} // output data kind

modules/gapi/include/opencv2/gapi/streaming/meta.hpp

Co-authored-by: Dmitry Matveev <dmm1989@gmail.com>

@staticmethod

…aque-kind-for-kernel G-API: Fix incorrect OpaqueKind for Kernel outputs opencv#23843 ### Pull Request Readiness Checklist #### Overview The PR is going to fix several problems: 1. Major: `GKernel` doesn't hold `kind` for its outputs. Since `GModelBuilder` traverse graph from outputs to inputs once it reaches any output of the operation it will use its `kind` to create `Data` meta for all operation outputs. Since it essential for `python` to know `GTypeInfo` (which is `shape` and `kind`) it will be confused. Consider this operation: ``` @cv.gapi.op('custom.square_mean', in_types=[cv.GArray.Int], out_types=[cv.GOpaque.Float, cv.GArray.Int]) class GSquareMean: @staticmethod def outMeta(desc): return cv.empty_gopaque_desc(), cv.empty_array_desc() ``` Even though `GOpaque` is `Float`, corresponding metadata might have `Int` kind because it might be taken from `cv.GArray.Int` so it will be a problem if one of the outputs of these operation is graph output because python will cast it to the wrong type based on `Data` meta. 2. Minor: Some of the OpenVINO `IR`'s doesn't any layout information for input. It's usually true only for `IRv10` but since `OpenVINO 2.0` need this information to correctly configure resize we need to put default layout if there no such assigned in `ov::Model`. See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [ ] I agree to contribute to the project under Apache 2 License. - [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [ ] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake

@staticmethod

…aque-kind-for-kernel G-API: Fix incorrect OpaqueKind for Kernel outputs opencv#23843 ### Pull Request Readiness Checklist #### Overview The PR is going to fix several problems: 1. Major: `GKernel` doesn't hold `kind` for its outputs. Since `GModelBuilder` traverse graph from outputs to inputs once it reaches any output of the operation it will use its `kind` to create `Data` meta for all operation outputs. Since it essential for `python` to know `GTypeInfo` (which is `shape` and `kind`) it will be confused. Consider this operation: ``` @cv.gapi.op('custom.square_mean', in_types=[cv.GArray.Int], out_types=[cv.GOpaque.Float, cv.GArray.Int]) class GSquareMean: @staticmethod def outMeta(desc): return cv.empty_gopaque_desc(), cv.empty_array_desc() ``` Even though `GOpaque` is `Float`, corresponding metadata might have `Int` kind because it might be taken from `cv.GArray.Int` so it will be a problem if one of the outputs of these operation is graph output because python will cast it to the wrong type based on `Data` meta. 2. Minor: Some of the OpenVINO `IR`'s doesn't any layout information for input. It's usually true only for `IRv10` but since `OpenVINO 2.0` need this information to correctly configure resize we need to put default layout if there no such assigned in `ov::Model`. See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [ ] I agree to contribute to the project under Apache 2 License. - [ ] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [ ] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake

Fix bug with missed OpaqueKind

3c3508b

TolyaTalamanov added this to the 4.8.0 milestone Jun 21, 2023

TolyaTalamanov commented Jun 21, 2023

View reviewed changes

Set default layout for 4D tensor OV backend

f70139d

dmatveev self-assigned this Jun 21, 2023

dmatveev added category: g-api / gapi bug labels Jun 21, 2023

dmatveev approved these changes Jun 21, 2023

View reviewed changes

TolyaTalamanov and others added 2 commits June 22, 2023 07:14

Update modules/gapi/include/opencv2/gapi/gkernel.hpp

ed42961

Co-authored-by: Dmitry Matveev <dmm1989@gmail.com>

Update modules/gapi/include/opencv2/gapi/streaming/meta.hpp

fd955e0

Co-authored-by: Dmitry Matveev <dmm1989@gmail.com>

asmorkalov merged commit 6084851 into opencv:4.x Jun 22, 2023

asmorkalov mentioned this pull request Jul 27, 2023

(5.x) Merge 4.x #24070

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

G-API: Fix incorrect OpaqueKind for Kernel outputs#23843

G-API: Fix incorrect OpaqueKind for Kernel outputs#23843
asmorkalov merged 4 commits intoopencv:4.xfrom
TolyaTalamanov:at/fix-missing-opaque-kind-for-kernel

TolyaTalamanov commented Jun 21, 2023 •

edited

Loading

Uh oh!

TolyaTalamanov Jun 21, 2023

Uh oh!

TolyaTalamanov commented Jun 21, 2023

Uh oh!

TolyaTalamanov commented Jun 21, 2023

Uh oh!

dmatveev left a comment

Uh oh!

Uh oh!

dmatveev Jun 21, 2023

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	, {cv::detail::GTypeTraits<G>::op_kind} // output data kinds
	, {cv::detail::GTypeTraits<G>::op_kind} // output data kind

Uh oh!

Conversation

TolyaTalamanov commented Jun 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Readiness Checklist

Overview

Uh oh!

TolyaTalamanov Jun 21, 2023

Choose a reason for hiding this comment

Uh oh!

TolyaTalamanov commented Jun 21, 2023

Uh oh!

TolyaTalamanov commented Jun 21, 2023

Uh oh!

dmatveev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dmatveev Jun 21, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

TolyaTalamanov commented Jun 21, 2023 •

edited

Loading