test: get rid of long tests#11043
Merged
Buristan merged 7 commits intotarantool:masterfrom Mar 3, 2025
Merged
Conversation
a01cf28 to
297e521
Compare
Buristan
approved these changes
Jan 23, 2025
Collaborator
Buristan
left a comment
There was a problem hiding this comment.
Hi, Sergey!
There are no objections from my side, so LGTM.
However, I'm a little bit confused by changes in <test/engine/box.lua>. Are they needed for the long test setup? If so, please mention this in the commit message.
- There is no precise criteria to distinguish long tests from
Typo: s/is/are/
- The mark "long" will not be likely be removed, even if it is
Typo: s/likely be/likely to be/
The patch removes
longmarkers in tests and merge test suite
Typo: s/merge/merges/
engine_longwith "long" tests together withenginesuite.
Typo: s/engine suite/the engine suite/
Totktonada
reviewed
Jan 23, 2025
Totktonada
reviewed
Jan 23, 2025
Totktonada
approved these changes
Jan 23, 2025
Contributor
Totktonada
left a comment
There was a problem hiding this comment.
Good movement! LGTM after resolving two small questions above.
locker
approved these changes
Jan 24, 2025
Totktonada
approved these changes
Jan 24, 2025
Contributor
|
The following long tests are failed in CI (with ASAN):
It seems they need a special glance. |
Totktonada
reviewed
Jan 24, 2025
Totktonada
reviewed
Jan 24, 2025
Totktonada
reviewed
Jan 24, 2025
This was referenced Jan 24, 2025
297e521 to
d479a1d
Compare
d479a1d to
48551a0
Compare
d888ecf to
4936d86
Compare
ImeevMA
approved these changes
Feb 21, 2025
Collaborator
ImeevMA
left a comment
There was a problem hiding this comment.
I looked at the SQL related fixes and they LGTM!
The patch fix testcase names in `sql-luatest/ghs_119_too_long_mem_values_test.lua`. NO_CHANGELOG=testing NO_DOC=testing
The test `replication/prune.test.lua` is failed when running by tarantool built with enabled UndefinedBehaviourSanitizer: NO_WRAP [002] /home/sergeyb/sources/MRG/tarantool/src/box/xrow.c:2333:26: runtime error: left shift of 1 by 31 places cannot be represented in type 'int' [002] SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/sergeyb/sources/MRG/tarantool/src/box/xrow.c:2333:26 [001] [ fail ] NO_WRAP The patch fixes that. Fixes tarantool#11045 Part of tarantool#10840 NO_CHANGELOG=fix UB NO_DOC=fix UB NO_TEST=not required
The patch cast variable `N` to int64 to fix signed integer overflow. The issue was found by regression test `sql-luatest/ghs_122_allocations_in_printf_test.lua`, so an additional test is not required. Fixes tarantool#11176 Part of tarantool#10840 NO_CHANGELOG=fix UB NO_DOC=fix UB NO_TEST=not required
The patch speeding up test ghs_119_too_long_mem_values_test in about 2-2.5 times (16 sec vs. 7 sec). Part of tarantool#10840 NO_CHANGELOG=testing NO_DOC=testing
93c4969 to
9f7878f
Compare
locker
approved these changes
Feb 28, 2025
The patch makes possible running test previously marked as "long" with timeouts suitable for other tests even with ASAN build. The test `sql-luatest/ghs_119_too_long_mem_values_test.lua` generates data for checking limits and appropriate error message in SQL. The testcases `replace` and `concat` are the slowest and with enabled ASAN requires more than 310 sec for execution, so these testcases are skipped when build instrumented by ASAN. Also, the tag `parallel` was added, `test-run.py` will run testcases in parallel. Part of tarantool#10840 NO_CHANGELOG=testing NO_DOC=testing
There are two tests that marked as "long": delete_insert.test.lua and delete_replace_update.test.lua. Both tests cannot be executed on AArch64 because execution time is exceeded timeout (320 sec). Both tests runs a function many times, so these tests were designed as a load tests, not a functional. Mixing load and functional test is a bad idea, to allow executing these tests on AArch64 the patch reduces a number of iterations and overall times is reduced in 10 times. As a mitigation we have performance tests that used as a load and performance tests. Needed for the following patch. Part of tarantool#10840 NO_CHANGELOG=testing NO_DOC=testing
See details in tarantool#10840, in short: - There are no precise criteria to distinguish long tests from other tests. Often it's a matter of taste: it feels like some tests take longer than others. - The mark "long" will not be likely to be removed, even if it is no longer such. - Splitting tests into long and short makes testing more difficult and reduces the frequency of running some tests. The patch removes `long` markers in tests and merges test suite `engine_long` with "long" tests together with the `engine` suite. Note, the test `vinyl/select_consistency.test.lua` was marked as "long", it is gone in commit 153463d ("test: drop vinyl/select_consistency.test.lua"). Closes tarantool#10840 NO_CHANGELOG=testing NO_DOC=testing
9f7878f to
cf8d76b
Compare
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Blocked by:
See details in #10840, in short:
The patch removes
longmarkers in tests and merge test suiteengine_longwith "long" tests together withenginesuite.Closes #10840
NO_CHANGELOG=testing
NO_DOC=testing
NO_TEST=testing