The current implementation collects stats in a std::vector. As the vector grows it needs to be resized, during long runs this resize can take significant amounts of time and appear to the user as though the server is not responding. This is both inacurate and confusing to users using the tool.

The chart below clearly demonstrates the problem during a long run against redis. You can see big dropouts around 208, 520, 2080, 4056 seconds, profiler shows this time is spent in memmove during a vector resize.

The Fix

During the benchmark its critical that updating stats takes a constant time. Otherwise the benchmark results will be skewed. Because of this a std::vector with its O(n) insert is not the correct datastructure. The fix changes the code to a std::list which can guarantee O(1) inserts resulting in a more accurate benchmark.

The run with the fix achieved 159022 sets/s, while the original code produced 158304 sets/s

memtier command:
-t 50 -c 8 -n 2000000 -d 256 --key-minimum=1 --key-maximum=800000000 --ratio 1:0 --key-pattern=P:P --hide-histogram --json-out-file=redis.json