-
Notifications
You must be signed in to change notification settings - Fork 4.1k
roachtest,storage: use lenient pebble.Cleaner in some roachtests #146991
Description
Describe the problem
When durability-related errors occur, it is important to be able to retrace the history of operations on the storage engine in as much detail as possible. However, removal of old SSTs and reuse of WAL files can wipe out history.
As of #146990, we keep roachtest clusters that show certain storage durability errors around for investigation. But we could go further and also slow down removal of old data, at least in certain tests for which performance/low disk usage is not critical, to have more history in these artifacts.
Pebble has the concept of a Cleaner which is tasked with disposing of files that are no longer needed. The default cleaner simply removes the file, but pebble also has an ArchiveCleaner that instead moves the files to an archival directory. This wouldn't be quite right for our testing since it fills up the disks too aggressively. But we could add a cleaner that is sensitive to disk usage and evicts well ahead of the disks filling up1, or keeps a fixed-time or fixed-size history of files around.
Jira issue: CRDB-50845
Footnotes
-
though we'd need to make sure we're not throwing off the allocator through the large amount of ancillary disk usage! ↩