Skip to content

[IE CLDNN] Build time optimization for OCL kernels#4725

Merged
sshlyapn merged 11 commits intoopenvinotoolkit:masterfrom
yeonbok:taylor_async_batch_WW11_rebase_both_tbb_threadpool_default_with_threadpool
Mar 31, 2021
Merged

[IE CLDNN] Build time optimization for OCL kernels#4725
sshlyapn merged 11 commits intoopenvinotoolkit:masterfrom
yeonbok:taylor_async_batch_WW11_rebase_both_tbb_threadpool_default_with_threadpool

Conversation

@yeonbok
Copy link
Copy Markdown
Contributor

@yeonbok yeonbok commented Mar 11, 2021 •
edited
Loading

Details:

  • Build OCL kernel batches in parallel
  • Two implementations for threading : tbb & threadpool

Tickets:

@yeonbok yeonbok added the category: GPU OpenVINO GPU plugin label Mar 11, 2021
@yeonbok yeonbok force-pushed the taylor_async_batch_WW11_rebase_both_tbb_threadpool_default_with_threadpool branch 7 times, most recently from d2b8612 to 2dec624 Compare March 12, 2021 05:17
@yeonbok yeonbok added this to the 2021.4 milestone Mar 12, 2021
@yeonbok yeonbok force-pushed the taylor_async_batch_WW11_rebase_both_tbb_threadpool_default_with_threadpool branch from 2dec624 to b59d461 Compare March 12, 2021 05:42
@yeonbok
Copy link
Copy Markdown
Contributor Author

yeonbok commented Mar 12, 2021 •
edited
Loading

@ababushk Hi Andrey, this patch is adding tbb thread on cldnn code, but seems that cldnn unittest job is not loading tbb library. Could you please add tbb path to LD_LIBRARY_PATH for the environment of that job?
https://openvino-ci.intel.com/job/private-ci/job/ie/job/ie-tests-linux-ubuntu18-cldnn_unit/28298/console

@yeonbok yeonbok marked this pull request as ready for review March 12, 2021 07:39
@yeonbok yeonbok requested review from a team as code owners March 12, 2021 07:39
@yeonbok yeonbok force-pushed the taylor_async_batch_WW11_rebase_both_tbb_threadpool_default_with_threadpool branch from 3507246 to d737a08 Compare March 12, 2021 09:40
@ilya-lavrenov ilya-lavrenov requested a review from myshevts March 12, 2021 09:51
@yeonbok yeonbok force-pushed the taylor_async_batch_WW11_rebase_both_tbb_threadpool_default_with_threadpool branch 2 times, most recently from 7b15775 to 44e4ca9 Compare March 15, 2021 06:22
@yeonbok
Copy link
Copy Markdown
Contributor Author

yeonbok commented Mar 17, 2021

@vladimir-paramuzov Having two versions of threads : thread pool + tbb, we have a concern of management. How do you think remove threadpool path and go only with tbb?

@ababushk
Copy link
Copy Markdown
Contributor

ababushk commented Mar 17, 2021

@yeonbok CI environment fix was merged to master, please make some new commits to your PR for changes to be propagated

yeonbok and others added 7 commits March 25, 2021 22:43
@yeonbok yeonbok force-pushed the taylor_async_batch_WW11_rebase_both_tbb_threadpool_default_with_threadpool branch from a40c6cf to f67409f Compare March 25, 2021 13:59
@yeonbok yeonbok requested a review from andrew-k-park March 25, 2021 14:01
@yeonbok yeonbok force-pushed the taylor_async_batch_WW11_rebase_both_tbb_threadpool_default_with_threadpool branch 4 times, most recently from 2e49ebc to a8c5e7d Compare March 25, 2021 15:33
lznamens
lznamens reviewed Mar 25, 2021
View reviewed changes
inference-engine/thirdparty/clDNN/api/engine.hpp Outdated
///< (switched off for older drivers then NEO).
uint16_t n_streams; ///< Number of queues executed in parallel
const std::string kernels_cache_path; ///< Path to compiled kernels cache
uint16_t n_threads; ///< Number of threads used to build program
Copy link
Copy Markdown
Contributor

@lznamens lznamens Mar 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose it makes sense to align this comment as others.

Copy link
Copy Markdown
Contributor Author

@yeonbok yeonbok Mar 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied

inference-engine/thirdparty/clDNN/src/gpu/kernels_cache.h Outdated
while (true) {
std::unique_lock<std::mutex> lock(this->_q_m);
_cv.wait(lock, [this]() { return (!this->_tasks.empty()) || (_stop_pool); });
if ( (_stop_pool) && (this->_tasks.empty())) return;
Copy link
Copy Markdown
Contributor

@lznamens lznamens Mar 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove extra space before break.

Copy link
Copy Markdown
Contributor Author

@yeonbok yeonbok Mar 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied

@yeonbok yeonbok force-pushed the taylor_async_batch_WW11_rebase_both_tbb_threadpool_default_with_threadpool branch 6 times, most recently from 007cf68 to 53ccea8 Compare March 26, 2021 07:56
@yeonbok
[IE CLDNN] Update stress test's reference memory consumption w.r.t th…
6909d7d 
…e updated default n_threads (prev:2 => now:max)
@yeonbok yeonbok force-pushed the taylor_async_batch_WW11_rebase_both_tbb_threadpool_default_with_threadpool branch from 53ccea8 to 6909d7d Compare March 29, 2021 01:48
@yeonbok yeonbok requested a review from sshlyapn March 29, 2021 08:45
@sshlyapn sshlyapn merged commit b58c648 into openvinotoolkit:master Mar 31, 2021
luo-cheng2021 pushed a commit to luo-cheng2021/openvino that referenced this pull request Apr 7, 2021
@yeonbok @donghyeo @luo-cheng2021
[IE CLDNN] Build time optimization for OCL kernels (openvinotoolkit#4725
afc5957 
)

Build OCL kernel batches of different buckets in parallel

Co-authored-by: Donghyeon Jeong <donghyeon.jeong@intel.com>
@vladimir-paramuzov vladimir-paramuzov mentioned this pull request Apr 19, 2021
Closed
mryzhov pushed a commit to mryzhov/openvino that referenced this pull request Apr 23, 2021
@yeonbok @donghyeo @mryzhov
[IE CLDNN] Build time optimization for OCL kernels (openvinotoolkit#4725
dd7e62e 
)

Build OCL kernel batches of different buckets in parallel

Co-authored-by: Donghyeon Jeong <donghyeon.jeong@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@geunhwan geunhwan Awaiting requested review from geunhwan

@isanghao isanghao Awaiting requested review from isanghao

@kelvinchoi-intel kelvinchoi-intel Awaiting requested review from kelvinchoi-intel

@ahnyoung-paul ahnyoung-paul Awaiting requested review from ahnyoung-paul

@donghyeo donghyeo Awaiting requested review from donghyeo

@myshevts myshevts Awaiting requested review from myshevts

@andrew-k-park andrew-k-park Awaiting requested review from andrew-k-park

4 more reviewers

@ilya-lavrenov ilya-lavrenov ilya-lavrenov left review comments

@vladimir-paramuzov vladimir-paramuzov vladimir-paramuzov left review comments

@lznamens lznamens lznamens left review comments

@sshlyapn sshlyapn sshlyapn approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@sshlyapn sshlyapn

Labels

category: GPU OpenVINO GPU plugin

Projects

None yet

Milestone

2021.4

Development

Successfully merging this pull request may close these issues.

7 participants

@yeonbok @ababushk @ilya-lavrenov @sshlyapn @vladimir-paramuzov @lznamens @donghyeo