Design proposal

Introduction

OpenVINO provides API for specifying input/output nn model characteristics such as: Precision, Layout, particularly for input it's also possible to specify if some preprocessing should be applied. This characteristics are individual per input or output and configurable via: IE::Data.

Motivation

To make user's life easier G-API configure OpenVINO with following behavior:

1. Precision for every input layer set automatically to type that came as graph input meta. https://github.com/opencv/opencv/blob/4.x/modules/gapi/src/backends/ie/giebackend.cpp#L1076
2. Layout for every input layer calculated based on this:
   2.1 Check if input either IMAGE or TENSOR:
   https://github.com/opencv/opencv/blob/4.x/modules/gapi/src/backends/ie/giebackend.cpp#L138
   2.2 In case IMAGE it will be always NHWC.
   2.3 In case TENSOR it will be decided based on number of dims: https://github.com/opencv/opencv/blob/4.x/modules/gapi/src/backends/ie/giebackend.cpp#L96
3. Output Precision/Layout is impossible to configure at all. They're left by default.

Based on this behavior there are several flaws:

1. Multi-dim (dim == 4) cv::Mat is always treated as NCHW and there is no any chance to specify NHWC instead.
2. Multi-dim (dim == 2) cv::Mat is always treated as NC and there is no any chance to specify HW instead.
3. Precision/Layout for output layers impossible to configure. It might slightly improve performance if user let's say wanna receive FP16 output data without conversion performed on OpenVINO side.

Proposal

Actually proposal is simple, introduce following handles inside cv::gapi::ie::Params structure:

API Example:

cv::gapi::ie::Params(...).cfgInputLayout('input0', 'NCHW')
                         .cfgOutputPrecision('output0', 'FP16')
                         .cfgOutputLayout('output0', 'NCHW')
                         .cfgOutputPrecision('output1', 'FP16')
                         .cfgOutputLayout('output1', 'NCHW')

Another idea is use CV_* enums to specify precision:

cv::gapi::ie::Params(...).cfgOutputPrecision(CV_16F) // set all output layers precision to ie::FP16

I'd suggest using string as type for Layout (probably for Precision)

1. OV2.0 uses strings: https://docs.openvino.ai/latest/openvino_docs_OV_UG_Layout_Overview.html
2. It doesn't require extra enum's on G-API side.
3. Easy adopt to python