G-API: Support remote inference by smirnov-alexey · Pull Request #20156 · opencv/opencv

smirnov-alexey · 2021-05-25T14:36:01Z

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
The PR is proposed to proper branch
There is reference to original bug report and related work
There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
The feature is well documented and sample code can be built with the project CMake

force_builders=Custom,Custom Win,Custom Mac
build_gapi_standalone:Linux x64=ade-0.1.1f
build_gapi_standalone:Win64=ade-0.1.1f
build_gapi_standalone:Mac=ade-0.1.1f
build_gapi_standalone:Linux x64 Debug=ade-0.1.1f

Xbuild_image:Custom=centos:7
Xbuildworker:Custom=linux-1
build_gapi_standalone:Custom=ade-0.1.1f

build_image:Custom=ubuntu-openvino-2021.3.0:20.04
build_image:Custom Win=openvino-2021.3.0
build_image:Custom Mac=openvino-2021.3.0

test_modules:Custom=gapi,python2,python3,java
test_modules:Custom Win=gapi,python2,python3,java
test_modules:Custom Mac=gapi,python2,python3,java

buildworker:Custom=linux-1
# disabled due high memory usage: test_opencl:Custom=ON
test_opencl:Custom=OFF
test_bigdata:Custom=1
test_filter:Custom=*

smirnov-alexey · 2021-05-25T14:36:33Z

This PR: #20151 has to be merged first

smirnov-alexey · 2021-05-25T14:37:03Z

@dmatveev @TolyaTalamanov @rgarnov @AsyaPronina please, review

dmatveev

The rest of the diff seems to be mediaframe extension-related

modules/gapi/include/opencv2/gapi/infer/ie.hpp

modules/gapi/src/backends/ie/giebackend.cpp

modules/gapi/src/backends/ie/giebackend/giewrapper.hpp

smirnov-alexey · 2021-05-31T13:23:43Z

@dmatveev please, take a look at the changes

dmatveev

@TolyaTalamanov I expect your review on the blob management part in the iebackend.

modules/gapi/include/opencv2/gapi/infer/ie.hpp

modules/gapi/src/api/media.cpp

modules/gapi/src/backends/ie/giebackend.cpp

…exey/opencv into as/gapi_remote_infer

smirnov-alexey · 2021-06-07T21:54:47Z

@dmatveev please, take a look once more

modules/gapi/src/backends/ie/giebackend.cpp

…i_remote_infer

smirnov-alexey · 2021-06-21T16:13:35Z

@dmatveev please, take a look at the latest version

dmatveev · 2021-06-22T22:19:08Z

@TolyaTalamanov any review from your end here?

modules/gapi/include/opencv2/gapi/infer/ie.hpp

dmatveev · 2021-06-22T22:25:30Z

modules/gapi/src/backends/ie/giebackend.cpp

    explicit IEUnit(const cv::gapi::ie::detail::ParamDesc &pp)
        : params(pp) {
+        InferenceEngine::ParamMap* ctx_params =
+                            cv::util::any_cast<InferenceEngine::ParamMap>(&params.context_config);


so before making this cast at all, can you please check if any is not null?

I don't think we can check if it's null besides casting it to something

Under null I mean empty, but I see our any don't have has_value as defined in std

Please add a follow-up task

Added a task

dmatveev · 2021-06-22T22:25:52Z

modules/gapi/src/backends/ie/giebackend.cpp

+        if (ctx_params != nullptr) {
+            auto ie_core = cv::gimpl::ie::wrap::getCore();
+            rctx = ie_core.CreateContext(params.device_id, *ctx_params);
+        }


and if it is not null and couldn't cast, we should probably print out a warning.

I don't think we can check if it's null besides casting it to something

modules/gapi/src/backends/ie/giebackend.cpp

smirnov-alexey · 2021-06-23T14:03:08Z

@dmatveev @TolyaTalamanov please, take a look and lets merge this as a first approximation

dmatveev

Let's merge it, thanks.

dmatveev · 2021-06-25T14:01:40Z

modules/gapi/include/opencv2/gapi/infer/ie.hpp

+    /** @brief Specifies configuration for RemoteContext in InferenceEngine.
+
+    When RemoteContext is configured the backend imports the networks using the context.
+    It also expects cv::MediaFrames to be actually remote, to operate with blobs via the context.


cv::MediaFrames -> input data.

G-API: Support remote inference * Extend MediaFrame to be able to extract additional info besides access * Add API for remote inference * Add default implementation for blobParams() * Add default implementation for blobParams() * Address review comments * Fix any_cast usage * Add comment on the default blobParams() * Address review comments * Add missing rctx * Minor fix * Fix indentation and comment * Address review comments * Add documentation

smirnov-alexey added 2 commits May 25, 2021 01:54

Extend MediaFrame to be able to extract additional info besides access

6700fe0

Add API for remote inference

d9af5a7

dmatveev self-assigned this May 31, 2021

dmatveev added this to the 4.5.3 milestone May 31, 2021

smirnov-alexey added 2 commits May 31, 2021 15:27

Add default implementation for blobParams()

48fc58e

Add default implementation for blobParams()

29b5e0e

dmatveev reviewed May 31, 2021

View reviewed changes

Address review comments

c074284

Fix any_cast usage

797de81

dmatveev reviewed Jun 3, 2021

View reviewed changes

dmatveev requested a review from TolyaTalamanov June 3, 2021 08:07

smirnov-alexey added 4 commits June 7, 2021 17:04

Add comment on the default blobParams()

713a257

Merge branch 'as/extend_media_frame' of https://github.com/smirnov-al…

8e50fd4

…exey/opencv into as/gapi_remote_infer

Address review comments

9302f4b

Add missing rctx

9a8bc29

dmatveev reviewed Jun 7, 2021

View reviewed changes

smirnov-alexey added 3 commits June 21, 2021 18:37

Minor fix

e374478

Merge branch 'master' of https://github.com/opencv/opencv into as/gap…

dad63b3

…i_remote_infer

Fix indentation and comment

2430ec5

dmatveev reviewed Jun 22, 2021

View reviewed changes

Address review comments

0e65633

Add documentation

78a6465

dmatveev approved these changes Jun 25, 2021

View reviewed changes

alalek merged commit c95a564 into opencv:master Jun 25, 2021

alalek mentioned this pull request Oct 15, 2021

(5.x) Merge 4.x #20886

Merged

Uh oh!

Conversation

smirnov-alexey commented May 25, 2021

Pull Request Readiness Checklist

Uh oh!

smirnov-alexey commented May 25, 2021

Uh oh!

smirnov-alexey commented May 25, 2021

Uh oh!

dmatveev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

smirnov-alexey commented May 31, 2021

Uh oh!

dmatveev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

smirnov-alexey commented Jun 7, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

smirnov-alexey commented Jun 21, 2021

Uh oh!

dmatveev commented Jun 22, 2021

Uh oh!

Uh oh!

dmatveev Jun 22, 2021

Choose a reason for hiding this comment

Uh oh!

smirnov-alexey Jun 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmatveev Jun 25, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smirnov-alexey Jun 28, 2021

Choose a reason for hiding this comment

Uh oh!

dmatveev Jun 22, 2021

Choose a reason for hiding this comment

Uh oh!

smirnov-alexey Jun 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmatveev Jun 25, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

smirnov-alexey commented Jun 23, 2021

Uh oh!

dmatveev left a comment

Choose a reason for hiding this comment

Uh oh!

dmatveev Jun 25, 2021

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

smirnov-alexey Jun 23, 2021 •

edited

Loading

dmatveev Jun 25, 2021 •

edited

Loading

smirnov-alexey Jun 23, 2021 •

edited

Loading