Skip to content

Introduce a test for fuzzing engines#10264

Merged
locker merged 2 commits intotarantool:masterfrom
ligurio:ligurio/engine-fuzzing
Jul 22, 2024
Merged

Introduce a test for fuzzing engines#10264
locker merged 2 commits intotarantool:masterfrom
ligurio:ligurio/engine-fuzzing

Conversation

@ligurio
Copy link
Member

@ligurio ligurio commented Jul 17, 2024

No description provided.

@ligurio ligurio requested a review from a team as a code owner July 17, 2024 11:43
@ligurio ligurio force-pushed the ligurio/engine-fuzzing branch 3 times, most recently from f211fc2 to 30fc96f Compare July 17, 2024 12:01
@coveralls
Copy link

coveralls commented Jul 17, 2024

Coverage Status

coverage: 87.096% (+0.007%) from 87.089%
when pulling 72c657d on ligurio:ligurio/engine-fuzzing
into a3ef8fb
on tarantool:master
.

@ligurio ligurio force-pushed the ligurio/engine-fuzzing branch 2 times, most recently from 512462e to 6c196f2 Compare July 17, 2024 13:02
@ligurio
Copy link
Member Author

ligurio commented Jul 17, 2024

@ligurio ligurio added the do not merge Not ready to be merged label Jul 17, 2024
@ligurio ligurio force-pushed the ligurio/engine-fuzzing branch from 6c196f2 to a54b333 Compare July 18, 2024 10:37
@ligurio ligurio added full-ci Enables all tests for a pull request and removed do not merge Not ready to be merged labels Jul 18, 2024
@ligurio ligurio requested a review from locker July 18, 2024 11:15
@sergepetrenko sergepetrenko requested a review from Gumix July 18, 2024 11:25
NO_CHANGELOG=typo
NO_DOC=typo
NO_TEST=typo
@ligurio ligurio force-pushed the ligurio/engine-fuzzing branch from a54b333 to 50cfe55 Compare July 19, 2024 13:29
@ligurio ligurio requested review from Gumix and locker July 19, 2024 13:33
@ligurio ligurio force-pushed the ligurio/engine-fuzzing branch from 50cfe55 to 030e648 Compare July 19, 2024 13:34
@ligurio ligurio assigned locker and unassigned ligurio Jul 19, 2024
@locker locker assigned Gumix and unassigned locker Jul 19, 2024
@Gumix Gumix assigned ligurio and unassigned Gumix Jul 19, 2024
The test for Tarantool allows you to randomly generate DDL and DML
operations, apply these operations to vinyl and memtx spaces and
toggle random error injections simultaneously. All random
things generated by the test depends on random seed, that can
be passed via command-line argument.

Bugs found by the test:

- tarantool#9995
- tarantool#10026
- tarantool#10033
- tarantool#10082
- tarantool#10090
- tarantool#10094
- tarantool#10095
- tarantool#10096
- tarantool#10096
- tarantool#10097
- tarantool#10099
- tarantool#10100
- tarantool#10109
- tarantool#10123
- tarantool#10128
- tarantool#10134
- tarantool#10147
- tarantool#10148
- tarantool#10153
- tarantool#10233
- tarantool#10235
- tarantool#10236
- tarantool#10237
- tarantool#10245
- tarantool#10262
- tarantool#10265
- tarantool#10267
- tarantool#10277
- tarantool#10278
- tarantool#10283

Part of tarantool#4349
Closes tarantool#5076

NO_CHANGELOG=fuzzing
NO_DOC=fuzzing
NO_TEST=fuzzing
@ligurio
Copy link
Member Author

ligurio commented Jul 21, 2024

There was a problem when error injections (ERRINJ_WAL_FALLOCATE and ERRINJ_BUILD_INDEX) were not applied due to wrong arguments. The patch below fixes arguments in err injections (index name was replaced with index id), fixes comments for err injections and adds errinj_f fiber object to a table with workers. All errors reported by errinj fiber will be printed at the end of the test.

The patch below:

diff --git a/test/fuzz/lua/test_engine.lua b/test/fuzz/lua/test_engine.lua
index e1c5bbac9b..f193ce4c38 100644
--- a/test/fuzz/lua/test_engine.lua
+++ b/test/fuzz/lua/test_engine.lua
@@ -1068,7 +1068,7 @@ local errinj_set = {
     -- alter.
     ERRINJ_BUILD_INDEX = {
         enable = function(space)
-            return oneof(keys(space.index))
+            return math.random(#(keys(space.index)))
         end,
         disable = function(_space)
             return -1
@@ -1176,7 +1176,7 @@ local errinj_set = {
         enable = enable_errinj_boolean,
         disable = disable_errinj_boolean,
     },
-    -- Set to true to fail vinyl index dump.
+    -- Set to timeout to fail vinyl index dump.
     ERRINJ_VY_INDEX_DUMP = {
         enable = enable_errinj_timeout,
         disable = function() return -1 end,
@@ -1262,7 +1262,7 @@ local errinj_set = {
         enable = enable_errinj_timeout,
         disable = disable_errinj_timeout,
     },
-    -- Set to true to fail allocation of vinyl tuple.
+    -- Set to timeout to fail allocation of vinyl tuple.
     ERRINJ_VY_STMT_ALLOC = {
         enable = enable_errinj_timeout,
         disable = disable_errinj_timeout,
@@ -1283,11 +1283,11 @@ local errinj_set = {
         enable = enable_errinj_boolean,
         disable = disable_errinj_boolean,
     },
-    -- Set to true to fail WAL write due to error allocating disk
+    -- Set to timeout to fail WAL write due to error allocating disk
     -- space.
     ERRINJ_WAL_FALLOCATE = {
-        enable = enable_errinj_boolean,
-        disable = disable_errinj_boolean,
+        enable = enable_errinj_timeout,
+        disable = disable_errinj_timeout,
     },
     -- Set to true to fail WAL write.
     ERRINJ_WAL_IO = {
@@ -1319,7 +1319,7 @@ local errinj_set = {
         enable = enable_errinj_boolean,
         disable = disable_errinj_boolean,
     },
-    -- Set to true to fail write to xlog file.
+    -- Set to timeout to fail write to xlog file.
     ERRINJ_WAL_WRITE_PARTIAL = {
         enable = enable_errinj_timeout,
         disable = disable_errinj_timeout,
@@ -1329,7 +1329,7 @@ local errinj_set = {
         enable = enable_errinj_boolean,
         disable = disable_errinj_boolean,
     },
-    -- Set to true to fail xlog data read.
+    -- Set to timeout to fail xlog data read.
     ERRINJ_XLOG_READ = {
         enable = enable_errinj_timeout,
         disable = disable_errinj_timeout,
@@ -1402,7 +1402,7 @@ local function run_test(num_workers, test_duration, test_dir,
     end, arg_test_duration)
     errinj_f:set_joinable(true)
     errinj_f:name('ERRINJ')
-    errinj_f:join()
+    table.insert(workers, errinj_f)
 
     local error_messages = {}
     for _, fb in ipairs(workers) do

@ligurio ligurio force-pushed the ligurio/engine-fuzzing branch from 030e648 to 72c657d Compare July 21, 2024 15:46
@locker locker merged commit 33670ea into tarantool:master Jul 22, 2024
@ligurio ligurio deleted the ligurio/engine-fuzzing branch July 22, 2024 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-ci Enables all tests for a pull request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants