Note: Please use Issues only for bug reports. For questions, discussions, feature requests, etc. post to dev group: https://www.facebook.com/groups/rocksdb.dev

When running super heavy file ingestion stress test, verifying the state of the database sometimes fails. This might be a problem with the stress test, or an actual problem with file ingestion, but either way it seems worth figuring out.

Expected behavior

We'd like to see this message:

Crash-recovery verification passed :)

Actual behavior

But instead sometimes we see a message like this:

Verification failed for column family 0 key 7341: Length of value read is not equal
Crash-recovery verification failed :(
2019/09/24-19:30:12  Starting database operations
2019/09/24-19:30:12  Starting verification
Verification failed :(

Steps to reproduce the behavior

build db_stress, preferably with DEBUG_LEVEL=0, then run the below command. On my machine it fails in ~10 seconds.

$ python tools/db_crashtest.py blackbox --simple --compression_type=none --interval=3 --max_key=10000 --ingest_external_file_one_in=100 --nooverwritepercent=0 --ingest_external_file_width=10