Skip to content

save allocation of memory for fake image#19536

Merged
opencv-pushbot merged 1 commit intoopencv:3.4from
WeiChungChang:ReduceMemory
Feb 22, 2021
Merged

save allocation of memory for fake image#19536
opencv-pushbot merged 1 commit intoopencv:3.4from
WeiChungChang:ReduceMemory

Conversation

@WeiChungChang
Copy link
Copy Markdown
Contributor

@WeiChungChang WeiChungChang commented Feb 16, 2021

In ProposalLayerImpl layer, there is a fake image. For now, we actually allocate memory for it.

// We've chosen the smallest data type because we need just a shape from it.
 fakeImageBlob.create(shape(1, 1, imInfo.at<float>(0), imInfo.at<float>(1)), CV_8UC1);

For example, if the input image has [N, C, H, W] = [1, 3, 1024, 1024], it will create a memory space of 1024*1024 = 1MB.
For edge devices, allocate 1MB unnecessary memory indeed consumes too much system resource.
Also notice that fakeImageBlob is a class member variable so its life scope is the same as dnn net.

However, this fake mat is used at:

  • prior box layer and
  • detection out layer.

In both of the layers, we don't access the content but merely request the shape info.
Instead, in the PR, we make fakeImageBlob as local variable. Also, it carries correct shape info but point to nulptr data(since it will NOT be used at all). So we can save, ex 1MB when doing inference for a model with ProposalLayer.

The figure below shows the result of memory usage probe (by valgrind --tool=massif) for ProposalLayerImpl fake image. in this case the input image is of [N, C, H, W] = [1, 3, 650, 650].
Originally the peak usage of memory is about 486.8 KB (650 * 650 ~ 422,5K plus alignment and other system memory ).

    KB
486.8^                                    #                                   
     |                                    #:::::::::::::::::::::::::::::      
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |                                    #                                   
     |     @::::::::::::::::::::::::::::::#                             :     
     |     @                              #                             ::::: 
     |     @                              #                             :     
   0 +----------------------------------------------------------------------->KB
     0                                                                   974.

With this PR the peak usage of memory is only~ 72.04KB (only system memory ).

    KB
74.02^                                     :                                  
     |                                  @@#:::::::::::::::::::::::::::::::::: 
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
     |                                  @@#:                                  
   0 +----------------------------------------------------------------------->KB
     0                                                                   149.0

Notice that for HAVE_OPENCL, we may apply the same optimization also.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
  • The PR is proposed to proper branch
  • There is reference to original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@asmorkalov asmorkalov self-requested a review February 16, 2021 10:56
asmorkalov
asmorkalov approved these changes Feb 16, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@asmorkalov asmorkalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Good catch! Thanks!

@alalek
Copy link
Copy Markdown
Member

alalek commented Feb 20, 2021

This patch should go into 3.4 branch first.
We will merge changes from 3.4 into master regularly (weekly/bi-weekly).

Please:

  • change "base" branch of this PR: master => 3.4 (use "Edit" button near PR title)
  • rebase your commits from master onto 3.4 branch. For example:
    git rebase -i --onto upstream/3.4 upstream/master
    (check list of your commits, save and quit (Esc + "wq" + Enter)
    where upstream is configured by following this GitHub guide and fetched (git fetch upstream).
  • push rebased commits into source branch of your fork (with --force option)

Note: no needs to re-open PR, apply changes "inplace".

@WeiChungChang WeiChungChang changed the base branch from master to 3.4 February 21, 2021 23:27
@opencv-pushbot opencv-pushbot merged commit 513f5a8 into opencv:3.4 Feb 22, 2021
@tomoaki0705 tomoaki0705 mentioned this pull request Feb 23, 2021
Closed
4 tasks
This was referenced Feb 23, 2021
Merged
Merged
@alalek alalek mentioned this pull request Apr 9, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@asmorkalov asmorkalov asmorkalov approved these changes

@LupusSanctus LupusSanctus Awaiting requested review from LupusSanctus

Assignees

@asmorkalov asmorkalov

Labels

category: dnn optimization

Projects

None yet

Milestone

3.4.14

Development

Successfully merging this pull request may close these issues.

4 participants

@WeiChungChang @alalek @asmorkalov @opencv-pushbot