Skip to content

Improves FLANN's heap allocations by a memory pool#20460

Merged
opencv-pushbot merged 1 commit intoopencv:masterfrom
cv3d:flannHeapPool
Aug 15, 2021
Merged

Improves FLANN's heap allocations by a memory pool#20460
opencv-pushbot merged 1 commit intoopencv:masterfrom
cv3d:flannHeapPool

Conversation

@cv3d
Copy link
Copy Markdown
Contributor

@cv3d cv3d commented Jul 26, 2021

FLANN allocates huge amounts of memory with every call to findNeighbors due to its private heap allocations. This affects memory throughput and overall speed of user applications, especially the ones that frequently query for neighbors. After all, although malloc does cache on its own, it also does a lot of smart things besides that, which costs some time penalty. To solve the problem, this PR introduces a static, thread-safe, and bounded memory pool for heap allocations. It have been tested thoroughly for very long hours, under extensive parallelization, and with strict leakage profiling. The tests showed more than 70x memory throughput gain. For roughly the same amount of calls, the memory footprint of findNeighbors using master branch was around 18.2 GB/sec, which got reduced by this PR to just 250.5 MB/sec (see the allocations throughput table below). Additionally, this PR cleans up some unnecessary variables in the cvflann::Heap class.

Master This PR
master PR

Pull Request Readiness Checklist

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or other license that is incompatible with OpenCV
  • The PR is proposed to proper branch
  • N/A: There is reference to original bug report and related work

Already existing:

  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake
force_builders=linux32

@cv3d cv3d changed the base branch from master to 3.4 July 26, 2021 12:03
@cv3d cv3d changed the base branch from 3.4 to master July 26, 2021 12:04
@cv3d cv3d marked this pull request as draft July 26, 2021 12:09
@cv3d cv3d force-pushed the flannHeapPool branch 4 times, most recently from b479984 to 2ac046c Compare July 26, 2021 19:14
@cv3d cv3d marked this pull request as ready for review July 26, 2021 19:34
@cv3d
Copy link
Copy Markdown
Contributor Author

cv3d commented Jul 26, 2021

Hi @alalek,
How are you doing?

I believe this PR is ready for your review.
Many thanks~

Copy link
Copy Markdown
Member

@alalek alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution!

Please take a look on the comments below.

@cv3d
Copy link
Copy Markdown
Contributor Author

cv3d commented Aug 5, 2021

@alalek I revised according to your valuable comments.

Please let me know if you still think there are some remaining improvements that can be made.
Thanks~

Copy link
Copy Markdown
Member

@alalek alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for update!

@cv3d
Copy link
Copy Markdown
Contributor Author

cv3d commented Aug 9, 2021

@alalek Appreciate your followup.
Maybe you want to go through the resolutions I made, and double check they are satisfactory. Thanks~

@alalek
Copy link
Copy Markdown
Member

alalek commented Aug 10, 2021

Please take a look on warnings from 32-bit compilation on Linux platforms.

@cv3d
Copy link
Copy Markdown
Contributor Author

cv3d commented Aug 11, 2021

@alalek Now Win64 OpenCL is failing to build, although I believe it is not related to this PR.
Can you please have a look? Thanks~

Copy link
Copy Markdown
Member

@alalek alalek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Thank you for contribution 👍

@opencv-pushbot opencv-pushbot merged commit 4bbe28b into opencv:master Aug 15, 2021
@alalek alalek mentioned this pull request Oct 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants