Skip to content

C++ code coverage support#905

Merged
rapids-bot[bot] merged 6 commits intorapidsai:branch-21.12from
harrism:fea-gcov
Nov 9, 2021
Merged

C++ code coverage support#905
rapids-bot[bot] merged 6 commits intorapidsai:branch-21.12from
harrism:fea-gcov

Conversation

@harrism
Copy link
Member

@harrism harrism commented Nov 4, 2021

Adds a cmake option (CODE_COVERAGE) to compile a build with relevant options to GCC and NVCC to generate code coverage profiling.

This can then be used with coverage tools like gcovr or coveralls to generate a detailed coverage summary (e.g. in HTML, JSON, XML, etc.).

Also adds a gcovr.cfg with basic options and a conda dependency on gcovr. Finally, it adds some explicit template instantiations in test files to ensure all template class methods are included in coverage analysis (so coverage % isn't artificially high). (reserved for future PR)

Instructions for generating code coverage

mkdir -p <RMM_ROOT>/build/debug && cd <RMM_ROOT>build/debug
cmake .. -DCMAKE_BUILD_TYPE=Debug -DCODE_COVERAGE=ON
make
ctest ## or ninja
cd <RMM_ROOT>
covr -r . -j 12 -o build/debug/coverage/ build/debug/tests/CMakeFiles/

Note that parallel building should not be used, because --keep is used with .cu files.

With rapids-compose, just set build type to Debug in .env, then

build-rmm-cpp -DCODE_COVERAGE=ON
test-rmm-cpp
covr -r . -j 12 -o build/debug/coverage/ build/debug/tests/CMakeFiles/

Then open build/debug/coverage/coverage_details.html. It looks like this:

image

Summary of changes required to get good coverage analysis:

  • Compile with --coverage,-fprofile-abs-path,-fkeep-inline-functions,-fno-elide-constructors and link with --coverage.
  • Compile CUDA code --keep (otherwise gcov gets errors finding NVCC's deleted intermediate-stage files)
  • Explicitly instantiate template classes at least once in test source files. This ensures any unused methods in these classes get shown as not covered by tests. Reserved for a future PR to improve code coverage.

@harrism harrism requested review from a team as code owners November 4, 2021 01:57
@harrism harrism requested review from cwharris and rongou November 4, 2021 01:57
@github-actions github-actions bot added CMake conda cpp Pertains to C++ code labels Nov 4, 2021
@harrism harrism added non-breaking Non-breaking change improvement Improvement / enhancement to an existing function labels Nov 4, 2021
@harrism harrism self-assigned this Nov 4, 2021
@harrism harrism requested a review from robertmaynard November 4, 2021 02:02
@harrism harrism changed the title Add C++ code coverage support C++ code coverage support Nov 9, 2021
@harrism
Copy link
Member Author

harrism commented Nov 9, 2021

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 59d6f34 into rapidsai:branch-21.12 Nov 9, 2021
rapids-bot bot pushed a commit that referenced this pull request Dec 1, 2021
Now that #905 added C++ code coverage support, this PR improves test coverage. Much of the improvement comes from adding a new `adaptor_test.cpp` which generically tests the common functions of all 7 adaptor types. Small test additions improve coverage of many other files as well. At least one typo bug was uncovered and fixed.

If you build with `-DCODE_COVERAGE=ON -DRMM_LOGGING_LEVEL=TRACE`, overall code coverage is now 99.5% and all but one file has 100% coverage. That one file, arena.hpp is undergoing work concurrent to this PR, and improvement to 100% requires additional testing that might be best undertaken by @rongou, so I will leave it.

![image](https://user-images.githubusercontent.com/783069/143160783-aecda008-2d7d-4ceb-8982-34bb1a35174c.png)

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Robert Maynard (https://github.com/robertmaynard)
  - Rong Ou (https://github.com/rongou)
  - Jake Hemstad (https://github.com/jrhemstad)

URL: #920
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CMake conda cpp Pertains to C++ code improvement Improvement / enhancement to an existing function non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants