G-API-NG/API: Introduced inference API and IE-based backend by dmatveev · Pull Request #15090 · opencv/opencv

dmatveev · 2019-07-18T20:46:07Z

Inference in G-API

This PR introduces basic inference support to G-API.

Changes overview

Core G-API

own::Mat has been extended with N-dimensions support. Continues
mimicing to cv::Mat in this behavior. Added tests.
GMatDesc now also supports dimensions in addition to a regular 2D
size. A new constructor has been added to.
cv::descr_of() overload for vector of Mats has been renamed to
cv::descrs_of() (and I am still wondering if we actually need this
function). A vector of Mats now is a valid host form for GArray<GMat>.
Operations are extended with tags. By default, all our old regular
kernels have empty tags (and are identified solely by kernel ID).
Engine has been extended with backend-defined metadata resolution
callbacks (see below).

On backend-specific `outMeta`

In our regular interface, an outMeta (defining formats operations's
of outputs given the inputs) is defined with the operation itself --
so it describes API contract and not a particular implementation.

Sometimes an operation can't fully describe its meta when it is
defined, and so backends can put their own methods to the graph once
there's a clarity.

Example: when we define a network as API entry, we're not certain
about its input/output formats (data type, dimensions, etc) until we
actually load the network. This loading happens on graph compilation
stage, not on the graph construction stage, and is defined by the
underlying engines which do the actual work.

New entities

Inference API

Inference API is represented by just two files:

include/opencv2/gapi/infer.hpp
src/api/ginfer.cpp

Header file is the most importand one, it defines two generic
operations: Infer and InferList, and an API to express networks we
use (G_API_NET() macro).

Once a user declares his network, it can be used to construct graphs.
Network's name (a macro parameter) is used as an operation tag (see
above) to distinguish between different infer calls in graph (note -
the operation itself is always the same).

Similar to kernels, scoring engines are specified in the form of
backend-specific network parameters which are then passed to G-API
as compilation arguments (as a Network package).

Since every inference backend's parameters may differ, but network
parameters are also tightly coupled with networks itselves,
backend-specific parameters are usually derivatives from the network
APIs. Networks (as types) are specified as template arguments to
universal backend-specific parameter structures so knowledge about
network and its backend combines in a single object.

A code is worth a thousand words so please review a test in this PR.

Inference backend (DLDT)

The above API is (as usual) just an interface which can be implemented
(multiple times). This PR introduces a basic DLDT/IE-based
implementation of the inference interface.

This backend implements the two universal inference kernels (Infer
and InferList) and provies a configuration API (see
gapi/infer/ie.hpp). It already supports automatic preprocessing with
Crop/Scale/CSC pipeline out-of-the-box. And yes, in this case G-API is
working from within a G-API. :)

Tests

So far tested with a DLDT Age/Gender model only. Model is taken from
DNN module's own test suite.

force_builders=Custom,Custom Win,Custom Mac
build_image:Custom=ubuntu-openvino-2019r2.0:16.04
build_image:Custom Win=openvino-2019r1
build_image:Custom Mac=openvino-2019r1

test_modules:Custom=gapi
test_modules:Custom Win=gapi
test_modules:Custom Mac=gapi

buildworker:Custom=linux-1
test_opencl:Custom=ON
test_bigdata:Custom=1
test_filter:Custom=*

build_gapi_standalone:Linux x64=ade-0.1.1d
build_gapi_standalone:Win64=ade-0.1.1d
build_gapi_standalone:Mac=ade-0.1.1d
build_gapi_standalone:Linux x64 Debug=ade-0.1.1d

dmatveev

19 new FIXMEs introduced, needs to be reduced

modules/gapi/CMakeLists.txt

dmatveev · 2019-07-19T06:23:08Z

@alalek thanks for the magic script :)

dmatveev

Added few-more self-comments

modules/gapi/CMakeLists.txt

modules/gapi/include/opencv2/gapi/gmat.hpp

modules/gapi/include/opencv2/gapi/infer.hpp

modules/gapi/include/opencv2/gapi/own/mat.hpp

modules/gapi/src/backends/ie/giebackend.cpp

modules/gapi/src/compiler/gmodel.hpp

modules/gapi/src/compiler/passes/kernels.cpp

modules/gapi/test/infer/gapi_infer_ie_test.cpp

dmatveev · 2019-07-23T13:09:30Z

@dkurt @AsyaPronina @rgarnov @andrey-golubev can you please have a look?

andrey-golubev

Overall looks good, I'd suggest to make minor changes in some places

modules/gapi/include/opencv2/gapi/gkernel.hpp

andrey-golubev · 2019-07-23T15:26:06Z

modules/gapi/include/opencv2/gapi/gmetaarg.hpp

 } // namespace detail

+// Note: descr_of(std::vector<..>) returns a GArrayDesc, while
+//       descrs_of(std::vector<..>) returns an array of Meta args!


Sounds confusing. May be descr_of and metas_of/meta_args_of are better?

Please check if IE is not using it and I will delete it at all.

It is used in IE

OK, will rename to metas_of if any new change to this PR will be requested.

modules/gapi/include/opencv2/gapi/infer.hpp

modules/gapi/include/opencv2/gapi/infer/ie.hpp

modules/gapi/include/opencv2/gapi/infer/ie/util.hpp

modules/gapi/src/backends/ie/giebackend.cpp

modules/gapi/src/backends/ie/giebackend.hpp

modules/gapi/src/compiler/gcompiler.cpp

modules/gapi/src/backends/ie/giebackend.hpp

dmatveev · 2019-07-26T11:14:07Z

@andrey-golubev addressed the majority if your comments, thanks a lot for such a torough review!

@alalek builds seem to be green (except ABI change which was intentional - to fix a problem), I believe this can be merged when time permits and things become safe to merge (post-4.1.1)

andrey-golubev

Looks good to me

dmatveev · 2019-07-31T17:55:39Z

@alalek do we wait for anything else to merge?

alalek · 2019-08-01T16:04:00Z

modules/gapi/test/infer/gapi_infer_ie_test.cpp

+
+    const std::string path = "Retail/object_attributes/age_gender/dldt/age-gender-recognition-retail-0013";
+    const auto topology_path = findDataFile(path + ".xml");
+    const auto weights_path  = findDataFile(path + ".bin");


Please use false with second parameter for findDataFile() (for files which are not from opencv_extra)

Done, thanks!

modules/gapi/test/infer/gapi_infer_ie_test.cpp

- Very quick-n-dirty implementation - OpenCV's own DNN module is not used - No tests so far

- Added tests on multi-dimensional own::Mat - Added tests on GMatDesc with dimensions - Documentation on infer.hpp - Fixed more warnings + added a ROI list test - Fix descr_of clash for vector<Mat> & standalone mode - Fix build issue with gcc-4.8x - Addressed review comments

- Pass `false` to findDataFile() - Add deprecation warning suppression macros for IE

dmatveev · 2019-08-05T14:47:58Z

@alalek anything else?

dmatveev · 2019-08-06T05:26:31Z

🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 🍾 THanks!

…nce-api * G-API-NG/API: Introduced inference API and IE-based backend - Very quick-n-dirty implementation - OpenCV's own DNN module is not used - No tests so far * G-API-NG/IE: Refined IE backend, added more tests * G-API-NG/IE: Fixed various CI warnings & build issues + tests - Added tests on multi-dimensional own::Mat - Added tests on GMatDesc with dimensions - Documentation on infer.hpp - Fixed more warnings + added a ROI list test - Fix descr_of clash for vector<Mat> & standalone mode - Fix build issue with gcc-4.8x - Addressed review comments * G-API-NG/IE: Addressed review comments - Pass `false` to findDataFile() - Add deprecation warning suppression macros for IE

dmatveev commented Jul 18, 2019

View reviewed changes

modules/gapi/CMakeLists.txt Show resolved Hide resolved

modules/gapi/CMakeLists.txt Show resolved Hide resolved

dmatveev commented Jul 22, 2019

View reviewed changes

dmatveev force-pushed the dm/ng-0001-g-api-inference-api branch 2 times, most recently from 69a70f4 to ba6c699 Compare July 22, 2019 23:26

dmatveev requested review from andrey-golubev and rgarnov July 23, 2019 13:08

dmatveev added the category: g-api / gapi label Jul 23, 2019

dmatveev requested review from AsyaPronina and dkurt July 23, 2019 13:08

andrey-golubev reviewed Jul 24, 2019

View reviewed changes

dmatveev force-pushed the dm/ng-0001-g-api-inference-api branch from 960697a to 08949dd Compare July 25, 2019 23:31

andrey-golubev reviewed Jul 26, 2019

View reviewed changes

modules/gapi/src/backends/ie/giebackend.hpp Show resolved Hide resolved

andrey-golubev approved these changes Jul 26, 2019

View reviewed changes

dmatveev mentioned this pull request Jul 26, 2019

The facial landmarks via IE sample is added for dnn module #15137

Closed

alalek reviewed Aug 1, 2019

View reviewed changes

dmatveev added 4 commits August 3, 2019 12:54

G-API-NG/API: Introduced inference API and IE-based backend

7a72aec

- Very quick-n-dirty implementation - OpenCV's own DNN module is not used - No tests so far

G-API-NG/IE: Refined IE backend, added more tests

92f1613

G-API-NG/IE: Addressed review comments

d405ad1

- Pass `false` to findDataFile() - Add deprecation warning suppression macros for IE

dmatveev force-pushed the dm/ng-0001-g-api-inference-api branch from 014d18b to d405ad1 Compare August 3, 2019 10:16

alalek assigned andrey-golubev Aug 5, 2019

alalek merged commit 0757a51 into opencv:master Aug 5, 2019

alalek mentioned this pull request Aug 13, 2019

G-API: Valgrind issues "OwnMatConversion.WithND" (2019-08-10) #15293

Closed

Uh oh!

Conversation

dmatveev commented Jul 18, 2019 • edited by alalek Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Inference in G-API

Changes overview

Core G-API

On backend-specific outMeta

New entities

Inference API

Inference backend (DLDT)

Tests

Uh oh!

dmatveev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dmatveev commented Jul 19, 2019

Uh oh!

dmatveev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dmatveev commented Jul 23, 2019

Uh oh!

andrey-golubev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

andrey-golubev Jul 23, 2019

Choose a reason for hiding this comment

Uh oh!

dmatveev Jul 25, 2019

Choose a reason for hiding this comment

Uh oh!

andrey-golubev Jul 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmatveev Jul 26, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dmatveev commented Jul 26, 2019

Uh oh!

andrey-golubev left a comment

Choose a reason for hiding this comment

Uh oh!

dmatveev commented Jul 31, 2019

Uh oh!

alalek Aug 1, 2019

Choose a reason for hiding this comment

Uh oh!

dmatveev Aug 3, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dmatveev commented Aug 5, 2019

Uh oh!

dmatveev commented Aug 6, 2019

Uh oh!

Reviewers

Assignees

dmatveev commented Jul 18, 2019 •

edited by alalek

Loading

On backend-specific `outMeta`

andrey-golubev Jul 26, 2019 •

edited

Loading