Option to build tests#727
Conversation
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
Signed-off-by: Alan Jowett <alan.jowett@microsoft.com>
036dd88 to
866964d
Compare
WalkthroughThe changes introduce a new build matrix configuration in the Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Outside diff range comments (3)
CMakeLists.txt (3)
Line range hint
98-107: LGTM: Conditional compilation of test filesThe conditional compilation of test files is a good practice. It reduces compilation time and binary size when tests are not needed.
One suggestion for improvement:
While using
file(GLOB ...)for test files is generally acceptable, consider explicitly listing the test files instead. This approach provides better control over which files are included and makes it easier to spot when a new test file is added or removed. For example:if(VERIFIER_ENABLE_TESTS) set(ALL_TEST "./src/test/test_conformance.cpp" "./src/test/test_marshal.cpp" "./src/test/test_print.cpp" "./src/test/test_verify.cpp" "./src/test/test_wto.cpp" "./src/test/test_yaml.cpp" ) endif()This approach is more explicit and less prone to accidentally including or excluding files.
Line range hint
133-151: LGTM: Conditional setting of test executable propertiesThe conditional setting of target properties for test executables is well implemented. This ensures that these properties are only set when tests are enabled, maintaining a clean build configuration.
One suggestion for improvement:
Consider using a variable for the RUNTIME_OUTPUT_DIRECTORY instead of repeating "${CMAKE_BINARY_DIR}/.." for each executable. This would make it easier to change the output directory for all test executables in the future if needed. For example:
if(VERIFIER_ENABLE_TESTS) set(TEST_OUTPUT_DIR "${CMAKE_BINARY_DIR}/..") set_target_properties(check tests run_yaml conformance_check PROPERTIES RUNTIME_OUTPUT_DIRECTORY "${TEST_OUTPUT_DIR}") add_dependencies(tests conformance_check) endif()This approach reduces repetition and makes the configuration more maintainable.
Line range hint
179-200: LGTM: Conditional compilation and linking of test executablesThe conditional setting of compilation flags and library linkage for test executables is well implemented. This ensures that these settings are only applied when tests are enabled, maintaining a clean build configuration.
One suggestion for improvement:
Consider grouping the compile options for each executable together, followed by the link libraries. This can improve readability and make it easier to manage the settings for each executable. For example:
if(VERIFIER_ENABLE_TESTS) # Compile options foreach(target check tests) target_compile_options(${target} PRIVATE ${COMMON_FLAGS}) target_compile_options(${target} PUBLIC "$<$<CONFIG:DEBUG>:${DEBUG_FLAGS}>") target_compile_options(${target} PUBLIC "$<$<CONFIG:RELEASE>:${RELEASE_FLAGS}>") target_compile_options(${target} PUBLIC "$<$<CONFIG:SANITIZE>:${SANITIZE_FLAGS}>") endforeach() # Link libraries target_link_libraries(check PRIVATE ebpfverifier) target_link_libraries(tests PRIVATE ebpfverifier bpf_conformance ${CMAKE_DL_LIBS} Threads::Threads Catch2::Catch2WithMain ) target_link_libraries(run_yaml PRIVATE ebpfverifier) target_link_libraries(conformance_check PRIVATE ebpfverifier) endif()This approach reduces repetition and groups related settings together, making the configuration more maintainable.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
📒 Files selected for processing (2)
- .github/workflows/build.yml (4 hunks)
- CMakeLists.txt (8 hunks)
🧰 Additional context used
🔇 Additional comments (6)
.github/workflows/build.yml (3)
21-21: Excellent addition ofbuild_testsmatrix!The introduction of the
build_testsparameter with[true, false]options in the matrix strategy is a great improvement. This change allows for more flexible CI configurations, potentially reducing build times when tests are not required.
58-58: Excellent consistency in Windows build configuration!The addition of
build_tests: [true, false]to the Windows build matrix mirrors the change made in the Ubuntu build job. This ensures consistency across different OS builds and provides the same flexibility for conditional test execution in Windows environments.
21-21: Overall, excellent improvements to the CI workflow!The changes introduced in this PR significantly enhance the flexibility of the CI pipeline by allowing conditional building and execution of tests. The implementation is consistent across both Ubuntu and Windows jobs, which is commendable.
Key improvements:
- Introduction of
build_testsmatrix parameter.- Conditional CMake configuration for tests.
- Conditional execution of test steps.
These changes will likely lead to more efficient CI runs when tests are not required. The code is well-structured and the changes are implemented consistently.
A few minor suggestions have been made for improved readability and consistency, but these are not critical. Great work on enhancing the CI workflow!
Also applies to: 41-41, 45-45, 49-49, 58-58, 73-73, 77-77
CMakeLists.txt (3)
8-9: LGTM: New option for enabling testsThe addition of the
VERIFIER_ENABLE_TESTSoption is a good practice. It allows for flexibility in the build process, enabling or disabling test compilation as needed. The default value ofONmaintains backwards compatibility with existing build processes.
124-129: LGTM: Conditional addition of test executablesThe conditional addition of test executables (check, tests, run_yaml, conformance_check) is well implemented. This approach ensures that these executables are only built when tests are enabled, which helps to reduce compilation time and disk usage when tests are not needed.
The variety of test executables suggests a comprehensive testing strategy, covering different aspects of the project's functionality and conformance.
Line range hint
1-201: Overall assessment: Well-implemented conditional test compilationThe changes introduced in this CMakeLists.txt file significantly improve the flexibility and efficiency of the build system by introducing conditional compilation for test-related components. The new
VERIFIER_ENABLE_TESTSoption allows users to easily control whether tests are built, which can lead to faster build times and reduced resource usage when tests are not needed.Key improvements:
- Conditional inclusion of the Catch2 testing framework
- Conditional compilation of test source files
- Conditional addition and configuration of test executables
- Conditional inclusion of the bpf_conformance subdirectory
- Conditional setting of compilation flags and library linkage for test executables
These changes are consistently applied throughout the file and follow CMake best practices. The default setting of
ONforVERIFIER_ENABLE_TESTSensures backwards compatibility with existing build processes.While the implementation is generally very good, there are a few minor suggestions for improvement in the individual comments above, mainly focused on reducing repetition and improving maintainability.
Great job on implementing this feature! It significantly enhances the project's build system flexibility.
|
@Alan-Jowett feel free to dismiss coderabbit comments you deem irrelevant or wrong |
|
Two comments:
|
Signed-off-by: Alan Jowett <alan.jowett@microsoft.com>
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Outside diff range comments (3)
CMakeLists.txt (3)
Line range hint
98-107: LGTM with suggestion: Conditional inclusion of test filesThe conditional inclusion of test files based on
VERIFIER_ENABLE_TESTSis good. It ensures that test-related files are only processed when tests are enabled.Consider replacing
file(GLOB ...)with explicit file listings. While GLOB is convenient, it can lead to build system issues if new files are added without re-running CMake. Here's an example of how you could refactor this:if(VERIFIER_ENABLE_TESTS) set(ALL_TEST src/test/test_conformance.cpp src/test/test_marshal.cpp src/test/test_print.cpp src/test/test_verify.cpp src/test/test_wto.cpp src/test/test_yaml.cpp ) endif()This approach provides better control over which files are included in the build and ensures that CMake always picks up changes to the file list.
Line range hint
133-151: LGTM: Setting properties for test executablesThe addition of runtime output directory settings for test executables and the dependency between 'tests' and 'conformance_check' are good practices.
For consistency and easier maintenance, consider using a loop or a function to set the runtime output directory for all test executables. For example:
if(VERIFIER_ENABLE_TESTS) foreach(test_exec check tests run_yaml conformance_check) set_target_properties(${test_exec} PROPERTIES RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/..") endforeach() add_dependencies(tests conformance_check) endif()This approach reduces repetition and makes it easier to apply the same settings to all test executables consistently.
Line range hint
179-200: LGTM: Conditional compilation and linking for test executablesThe conditional setting of compile options and linking of libraries for test executables is well-implemented. It ensures that these settings are only applied when tests are enabled, which is consistent with the
VERIFIER_ENABLE_TESTSoption.To improve readability and maintainability, consider creating a function to apply common settings to all test executables. For example:
if(VERIFIER_ENABLE_TESTS) function(configure_test_executable target) target_compile_options(${target} PRIVATE ${COMMON_FLAGS}) target_compile_options(${target} PUBLIC "$<$<CONFIG:DEBUG>:${DEBUG_FLAGS}>") target_compile_options(${target} PUBLIC "$<$<CONFIG:RELEASE>:${RELEASE_FLAGS}>") target_compile_options(${target} PUBLIC "$<$<CONFIG:SANITIZE>:${SANITIZE_FLAGS}>") target_link_libraries(${target} PRIVATE ebpfverifier) endfunction() configure_test_executable(check) configure_test_executable(tests) target_link_libraries(tests PRIVATE bpf_conformance ${CMAKE_DL_LIBS} Threads::Threads Catch2::Catch2WithMain) configure_test_executable(run_yaml) configure_test_executable(conformance_check) endif()This approach centralizes the common configuration and makes it easier to maintain consistent settings across all test executables.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
📒 Files selected for processing (2)
- .github/workflows/build.yml (4 hunks)
- CMakeLists.txt (8 hunks)
🧰 Additional context used
📓 Learnings (1)
CMakeLists.txt (1)
Learnt from: Alan-Jowett PR: vbpf/ebpf-verifier#727 File: CMakeLists.txt:18-25 Timestamp: 2024-10-14T15:06:59.489Z Learning: In the ebpf-verifier project, focus code reviews on changes introduced in the PR and avoid commenting on existing code that was not modified.
🔇 Additional comments (4)
.github/workflows/build.yml (3)
21-21: LGTM! Great addition for flexible CI configurations.The introduction of the
targetparameter with valuestestsandlibraryin the matrix strategy is a good approach. This allows for more granular control over the CI process, potentially speeding up builds by separating library and test compilations.
49-49: LGTM! Proper conditional execution of tests.The addition of the condition
if: ${{matrix.target == 'tests'}}to the "Run unit tests" step is correct and aligns well with the new matrix configuration. This ensures that tests are only executed when explicitly targeted, potentially saving CI time for library-only builds.
53-53: LGTM! Consistent conditional execution of memory leak tests.The addition of the condition
if: ${{matrix.target == 'tests'}}to the "Test for memory leaks" step is correct and consistent with the previous test execution step. This ensures that memory leak tests are only run when necessary, further optimizing the CI process for different build targets.CMakeLists.txt (1)
Line range hint
1-200: Summary: Well-implemented conditional test compilationOverall, the changes to implement conditional test compilation based on the
VERIFIER_ENABLE_TESTSoption are well-done and consistent throughout the CMakeLists.txt file. The modifications allow for more flexible build configurations, which can be particularly useful in CI/CD pipelines or deployments where tests are not needed.Key points:
- The new
VERIFIER_ENABLE_TESTSoption is clear and set to ON by default, maintaining backwards compatibility.- Test-related components (Catch2, test files, executables, and compile/link settings) are consistently wrapped in conditional blocks.
- The changes follow CMake best practices, with a few suggestions for minor improvements in code organization and clarity.
One area that requires verification is the conditional inclusion of the bpf_conformance subdirectory. Please ensure that bpf_conformance is not required by the main project outside of testing contexts.
This pull request introduces a new option to conditionally build and run tests in the CI workflow and modifies the CMake configuration to support this option. The changes are primarily focused on enhancing the build process and improving the flexibility of the CI pipeline.
CI Workflow Enhancements:
.github/workflows/build.yml: Added abuild_testsmatrix to conditionally build and run tests. Updated the build and test steps to respect this new matrix parameter. [1] [2] [3] [4]CMake Configuration Updates:
CMakeLists.txt: Introduced a new optionVERIFIER_ENABLE_TESTSto control the inclusion of test-related targets and dependencies. Wrapped test-related configuration and target definitions in conditional statements based on this option. [1] [2] [3] [4] [5] [6] [7] [8]Summary by CodeRabbit
New Features
Bug Fixes
Documentation