Merged
Conversation
ef3b9ee to
9f5bc59
Compare
ligurio
added a commit
to ligurio/nanodata
that referenced
this pull request
Jul 17, 2024
The proposed patch a bit ugly: fuzzing tests will be run sequentionally after functional testing and this will make a job longer. This will be reworked after introducing CTest support [1], these tests will be executed in parallel with other tests. 1. tarantool#10216 NO_CHANGELOG=ci NO_DOC=ci NO_TEST=ci
ligurio
added a commit
to ligurio/nanodata
that referenced
this pull request
Jul 17, 2024
The proposed patch a bit ugly: fuzzing tests will be run sequentionally after functional testing and this will make a job longer. This will be reworked after introducing CTest support [1], these tests will be executed in parallel with other tests. 1. tarantool#10216 NO_CHANGELOG=ci NO_DOC=ci NO_TEST=ci
ligurio
added a commit
to ligurio/nanodata
that referenced
this pull request
Jul 17, 2024
The proposed patch a bit ugly: fuzzing tests will be run sequentionally after functional testing and this will make a job longer. This will be reworked after introducing CTest support [1], these tests will be executed in parallel with other tests. 1. tarantool#10216 NO_CHANGELOG=ci NO_DOC=ci NO_TEST=ci
ligurio
added a commit
to ligurio/nanodata
that referenced
this pull request
Jul 17, 2024
The proposed patch a bit ugly: fuzzing tests will be run sequentionally after functional testing and this will make a job longer. This will be reworked after introducing CTest support [1], these tests will be executed in parallel with other tests. 1. tarantool#10216 NO_CHANGELOG=ci NO_DOC=ci NO_TEST=ci
ligurio
added a commit
to ligurio/nanodata
that referenced
this pull request
Jul 17, 2024
The proposed patch a bit ugly: fuzzing tests will be run sequentionally after functional testing and this will make time of jobs execution longer. This will be reworked after introducing CTest support [1], these tests will be executed in parallel with other tests. The proposed test was added to workflow with debug build, because this workflow runs in a short CI, and to workflow with enabled ASAN because AddressSanitizer can help to find additional problems related to incorrect work with memory. 1. tarantool#10216 NO_CHANGELOG=ci NO_DOC=ci NO_TEST=ci
ligurio
added a commit
to ligurio/nanodata
that referenced
this pull request
Jul 17, 2024
The proposed patch a bit ugly: fuzzing tests will be run sequentionally after functional testing and this will make time of jobs execution longer. This will be reworked after introducing CTest support [1], these tests will be executed in parallel with other tests. The proposed test was added to workflow with debug build, because this workflow runs in a short CI, and to workflow with enabled ASAN because AddressSanitizer can help to find additional problems related to incorrect work with memory. 1. tarantool#10216 NO_CHANGELOG=ci NO_DOC=ci NO_TEST=ci
ligurio
added a commit
to ligurio/nanodata
that referenced
this pull request
Jul 17, 2024
The proposed patch a bit ugly: fuzzing tests will be run sequentionally after functional testing and this will make time of jobs execution longer. This will be reworked after introducing CTest support [1], these tests will be executed in parallel with other tests. The proposed test was added to workflow with debug build, because this workflow runs in a short CI, and to workflow with enabled ASAN because AddressSanitizer can help to find additional problems related to incorrect work with memory. 1. tarantool#10216 NO_CHANGELOG=ci NO_DOC=ci NO_TEST=ci
81afeda to
30ba8eb
Compare
4672e1b to
8b883dd
Compare
ligurio
added a commit
to tarantool/small
that referenced
this pull request
Aug 20, 2024
The patch introduces a CMake function create_test and replaces add_test() with create_test(). Needed for assigning test labels in the following commit. Needed for tarantool/tarantool#10216
ligurio
added a commit
to tarantool/small
that referenced
this pull request
Aug 20, 2024
locker
pushed a commit
to tarantool/small
that referenced
this pull request
Aug 20, 2024
The patch introduces a CMake function create_test and replaces add_test() with create_test(). Needed for assigning test labels in the following commit. Needed for tarantool/tarantool#10216
locker
pushed a commit
to tarantool/small
that referenced
this pull request
Aug 20, 2024
d783924 to
b37b2b6
Compare
Member
Author
|
Rebased to master branch after merging commits in PR #10655. |
b37b2b6 to
289a74e
Compare
289a74e to
54b41eb
Compare
Buristan
reviewed
Nov 19, 2024
Buristan
reviewed
Nov 19, 2024
Buristan
reviewed
Nov 19, 2024
Buristan
reviewed
Nov 19, 2024
Member
Author
|
(rebased to master with merged #10725) |
This was referenced Nov 20, 2024
Member
Author
|
(rebased to master with merged #10778) |
Member
Author
|
(rebased to draft PR #10824) |
Buristan
approved these changes
Dec 18, 2024
Collaborator
Buristan
left a comment
There was a problem hiding this comment.
Sergey,
Thanks for the fixes!
LGTM!
Totktonada
approved these changes
Dec 18, 2024
Contributor
Totktonada
left a comment
There was a problem hiding this comment.
Code owner review request is due to the test/config-luatest/CMakeLists.txt file. I don't mind to add it.
It is not possible to get a number of parallel jobs in CMake passed by the user with option "--parallel", but CMake allows to pass a number of parallel jobs with environment variable CMAKE_BUILD_PARALLEL_LEVEL [1] on configuration phase. The value set by that environment variable is used as a number of CPU threads when it was not specified by user. Number of CPU threads detected using builtin CMake function [2]. NOTE: CMAKE_BUILD_PARALLEL_LEVEL has been added in a version 3.12. In earlier CMake versions environment variable is ignored. 1. https://cmake.org/cmake/help/latest/envvar/CMAKE_BUILD_PARALLEL_LEVEL.html 2. https://cmake.org/cmake/help/latest/module/ProcessorCount.html NO_CHANGELOG=build NO_DOC=build NO_TEST=build
Needed for the following commits. NO_CHANGELOG=build NO_DOC=build NO_TEST=build
Needed for the following commit. NO_CHANGELOG=build NO_DOC=build NO_TEST=build
The patch introduces CTest support, see [1] and [2]: the CMake
target `test` was replaced by builtin CMake target that executes
ctest-based tests. The target `test` that runs `test-run.py` was
renamed to `test-all`. Build and test steps remains the same
as before:
$ cmake -S . -B build
$ cmake --build build --parallel
$ cmake --build build --target test
One could use `ctest` tool instead:
$ ctest --print-labels
Test project <snipped>
All Labels:
LuaJIT-tests
PUC-Rio-Lua-5.1-tests
app
app-luatest
app-tap
box
box-luatest
box-py
<snipped>
$ ctest # Run all tests.
$ ctest -L app-luatest # Run tests by label.
$ ctest --rerun-fail # Run only the tests
# that failed previously.
$ ctest --verbose # Print details to stdout.
$ ctest --output-on-failure # Print details to stdout
# on test failure only.
The possible benefits of using CTest are:
- test execution triggers rebuilding of dependencies automatically
- close integration with a build system options
(ASAN/UBSAN/Debug/Release etc.) and without setting builddir
manually
- versatile tool for running different kind of tests (functional,
performance, fuzzing tests) made using different test frameworks
and libraries: TAP, luatest, diff-based tests, Google Benchmark,
LuaJIT-based and PUC Rio Lua-based tests etc.
- no more stderr and stdout suppressing
- easy access to a raw command line for running tests manually
for debug purposes (see `ctest --verbose`)
Note that CMake targets test-unit, test-unit-force, test-func,
test-func-force, test-all, 1mops_write_perftest,
box_select_perftest, tarantoolgh-7089-vclock-copy_perftest,
uri_escape_unescape_perftest, column_scan_perftest,
tuple_perftest, bps_tree_perftest, light_perftest, small_perftest,
memtx_perftest are still operating for backward compatibility.
Note that it is not possible to add dependencies to `add_test()`
in CMake, see [3]. CMake 3.7 introduces FIXTURES_REQUIRED [4] and
FIXTURES_SETUP [5], but these test properties cannot be used - the
currently supported CMake version is lower. This is workarounded
by the introduced macro `add_test_suite_target` that adds
a CMake-test for each testsuite that executes a target that builds
test dependencies.
1. https://cmake.org/cmake/help/latest/manual/ctest.1.html
2. https://cmake.org/cmake/help/book/mastering-cmake/chapter/Testing%20With%20CMake%20and%20CTest.html
3. https://gitlab.kitware.com/cmake/cmake/-/issues/8774
4. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_REQUIRED.html
5. https://cmake.org/cmake/help/latest/prop_test/FIXTURES_SETUP.html
6. tarantool/luajit@9f1137b
Related to tarantool#5000.
NO_CHANGELOG=testing
NO_DOC=testing
The patch add CTest support for performance tests. Now it is possible to execute these tests using `ctest` tool with labels `performance-lua`, `performance-c` and `performance`. NO_CHANGELOG=build NO_DOC=build NO_TEST=build
The patch add CTest support for fuzzing tests. Now it is possible to execute these tests using `ctest` tool with labels `fuzzing-c` and `fuzzing`. NO_CHANGELOG=build NO_DOC=build NO_TEST=build
The patch add CTest support for fuzzing tests written in Lua. Now it is possible to execute these tests using `ctest` tool with labels `fuzzing-lua` and `fuzzing`. Follows up commit 33670ea ("test/fuzz: add an engine fuzzing test"). NO_CHANGELOG=build NO_DOC=build NO_TEST=build
The patch add support of luatest's tests by CTest. The environment variable `VARDIR`, used by luatest [1], is set to `/tmp/t`, the same value used by test-run.py. The CMake target `tarantool` is not used a pseudotest "*-deps", because target `box_generate_lua_sources`, required for building tarantool and that builds Lua source files, being run in parallel can corrupt generated files. The tests in luatest suites can be executed by `ctest` using labels app-luatest, box-luatest, engine-luatest, metrics-luatest, replication-luatest, sql-luatest, vinyl-luatest. 1. test-run/lib/luatest/luatest/server.lua. NO_CHANGELOG=build NO_DOC=build NO_TEST=build
The patch introduce a target `test-force-ctest` that runs functional regression tests and this target is used in GHA workflow `static_build_cmake_linux`. NO_CHANGELOG=build NO_DOC=build NO_TEST=build
Member
Author
|
(rebased to a master branch) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on:
Related:
ENABLE_ASAN) Memleak of FiberIsCancelled exception #10733Follow ups: