1. I'd move it to cv::gapi::wip
2. I'd rename it to benchmark_mode to illustrate the purpose.

Should it be cv::gapi::wip or cv::gapi::wip::ov?

the latter, of course.

Done

probably you can omit it from there - as it is not the mandatory parameter.

it can be false internally by default.

This is the place where inference_only from GOVExecutable is propogated to the specific inference kernel (e.g Infer, InferList, etc):
https://github.com/TolyaTalamanov/opencv/blob/at/implement-inference-only-mode-for-ov-backend/modules/gapi/src/backends/ov/govbackend.cpp#L1501-L1502

I guess there is no sense to do something like that:

auto ctx = std::make_shared<OVCallContext>(uu, out, op.args, op.outs,
            std::move(stub_meta), std::move(input_objs), std::move(output_objs));
if (m_inference_only) {
    ctx->enableInferenceOnly();
}

It's easier to pass this straight to context ctor.

It should remain in the class but it shouldn't be a mandatory constructor argument

Done

probably you don't want to track all the individual fine-tuning knobs here, so I'd propose a configuration-like structure instead..

Do you mean something like this?

namespace cv { namespace gapi { namespace wip { namespace { ov {
struct execution_config {
    bool enable_benchmark_mode = false;
};

} // ov
} // wip
} // gapi
} // cv

// Example:

auto compile_args = cv::compile_args(cv::gapi::wip::ov::execution_config{ true /* benchmark mode */ };

Not at the API level. Inside.

Done

..and pass it as a whole thing here.

This method was quite generic, but now it knows explicitly about inference_only mode - purely an abstraction leak.

This method still doesn't know about inference only implementation details it just propagates "some" configuration into the kernel. But your point is correct, need to decide who exactly should be responsible for handling this mode:

There are a few options:

1. Kernel - This is the current approach. inference_only is propagated as the following:
   compileArgs -> GOVExecutable -> OVCallContext -> Kernel (e.g Infer, InferList, etc)

   The benefit is that IInferExecutor that used as proxy to submit inference tasks don't know about this option at all, the kernel hides this under set_input_data and read_output_data callbacks so it may continue work in sync/async modes without knowledges about this mode.

2. RequestPool / IInferExecutor - In this apporach GOVExecutable may configure RequestPool the way it knows about benchmark mode.

• Prons:
  • Kernels don't know about this mode so they continue submit their tasks through RequestPool API.
• Cons:
  • inference_only mode will be enabled for all Kernel's (now it's only relevant for Infer (InferList, InferROI, InferList2 throw exception if mode is enabled).
  • Mode should be handled for both SyncInferExecutor and AsyncInferExecutor. This is a little bit tricky since currently read_output_data is also responsible for posting outputs to maintain contract with streaming executor.
    https://github.com/TolyaTalamanov/opencv/blob/at/implement-inference-only-mode-for-ov-backend/modules/gapi/src/backends/ov/govbackend.cpp#L471

Current approach looked less invasive.

Yes and my point is to make some configuration really a some configuration but not a specific field.

Done