Based on the analysis done in #64 and discussion with users in #68, we remove cometbft-db and refactor CometBFT to use only one database backend (pebble).

DoD:

• CometBFT uses a single DB backend #4486
• Benchmarked the performance against the baseline established in Storage evaluation baseline: measure and report current storage related behaviour of CometBFT #67
• A test showcasing how the how the DBProvider can be used now to use another DB instead of Pebble (in case people don't want to migrate). docs(guides): explain how to use a custom DB #4952
• Add migration script

Current Status for Future Reference

All the work was (and should be in the future) pushed to the feature/remove-cometbftdb branch. #4514 tracks the work already merged in the branch and has links to the PRs that implemented the work.

Blocker

The biggest blocker is #4486, which does the heavy lifting of removing cometbft-db from the code. The issue has a comprehensive ToDo list pointing to PRs that did work and listing what still needs to be done. All the PRs that contributed to closing one of the ToDo list items of this issue merged their work to feature/remove-cometbftdb, therefore they will also appear in the list in #4514.

At the time of writing one ToDo item of #4486 is under development (marked as WIP in the ToDo list). You can see the work being done under #4601. In the spirit of consistency with everything else, #4601 has its own ToDo list of items to be completed with links to their PRs. You can also browse its commits to see what's already done.

Additional Work

As the rest of the ToDo list above says:

• we must thoroughly test these changes.
• we must update the docs. They should make clear the following:
  • we are not using cometbft-db anymore
  • if users want to use their own database, they must change the implementation of DefaultDBProvider to make it return a database implementation satisfying the DB, Batch, and Iterator instances. Then compile cometbft with their changes.
• We must write a migration script to permit users to port their data from whatever database they are using to pebble. We haven't started this work yet, but we were opting for providing an offline migration. We reached out to the community and there was no opposition to this choice.