Skip to content

feat(storage)!: remove cometbft-db#4868

Merged
melekes merged 19 commits intomainfrom
feature/remove-cometbftdb
Feb 10, 2025
Merged

feat(storage)!: remove cometbft-db#4868
melekes merged 19 commits intomainfrom
feature/remove-cometbftdb

Conversation

@melekes
Copy link
Collaborator

@melekes melekes commented Jan 23, 2025

Closes #4486.

Alessandro Sforzin and others added 11 commits November 19, 2024 14:30
Partially addresses #4486.

### Context
CometBFT will stop importing
[cometbft-db](https://github.com/cometbft/cometbft-db) as a dependency
to support multiple database backends. Instead, it will use
[pebble](https://github.com/cockroachdb/pebble) by default.

### Changes
This PR adds package `storage` to the `internal` packages: 
- it exports the `DB`, `Batch`, and `Iterator` interfaces that database
backends must implement to work with CometBFT. Previously, they were
offered by cometbft-db.
- it exports the pebbledb implementation of the above interfaces.

Users can provide their own implementation of these interfaces using a
different database under the hood. However, we will support and maintain
only the pebble implementation that CometBFT will use.

**Note**: this PR ports the code currently provided by cometbft-db. It
does not add new functionalities to either the interfaces or their
implementation using pebble. What it does:
- improve docs
- clearer error messages
- minor refactoring of nested `if` statements for readability

---

#### PR checklist

- ~[] Tests written/updated~
- ~[ ] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)~
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
…face. (#4515)

Partially addresses #4486.

### Context
CometBFT will stop importing
[cometbft-db](https://github.com/cometbft/cometbft-db) as a dependency
to support multiple database backends. Instead, it will use
[pebble](https://github.com/cockroachdb/pebble) by default.

### Changes
This PR adds unit tests to the pebble implementation of the `DB`
interface that CometBFT will use once we remove cometbft-db.
…terface. (#4522)

Partially addresses #4486.

### Context
CometBFT will stop importing
[cometbft-db](https://github.com/cometbft/cometbft-db) as a dependency
to support multiple database backends. Instead, it will use
[pebble](https://github.com/cockroachdb/pebble) by default.

### Changes
This PR adds unit tests to the pebble implementation of the `Batch`
interface that CometBFT will use once we remove cometbft-db.
… interface. (#4530)

Partially addresses #4486.

### Context
CometBFT will stop importing
[cometbft-db](https://github.com/cometbft/cometbft-db) as a dependency
to support multiple database backends. Instead, it will use
[pebble](https://github.com/cockroachdb/pebble) by default.

### Changes
This PR adds unit tests to the pebble implementation of the `Iterator`
interface that CometBFT will use once we remove cometbft-db.
Partially addresses #4486.

### Context
CometBFT will stop importing
[cometbft-db](https://github.com/cometbft/cometbft-db) as a dependency
to support multiple database backends. Instead, it will use
[pebble](https://github.com/cockroachdb/pebble) by default.

### Changes
This PR adds type `PrefixDB` to `internal/storage` from cometbft-db. 
`PrefixDB` is a database wrapper that implements the`DB`, `Batch`, and
`Iterator` interfaces.

**Note**: this PR makes some changes to the implementation of
`PrefixDB`. Namely:
- `PrefixDB` no longer uses a mutex, because we think that it is
sufficient to rely on the wrapped database's concurrency control
mechanism. That is, concurrent calls to `PrefixDB`'s methods will
already "lock" on the calls to the wrapped DB methods.
- `NewPrefixDB` now returns an error if the given prefix is empty. We
think allowing the creation of a `PrefixDB` with no prefix was an
oversight in the design.
- `prefixDBBatch` methods now want a pointer receiver. 
- `PrefixDB.Compact` now prepends the prefix to `start` and `end` keys
defining the iterator's range. The previous implementation did not do
so, and this was likely an oversight. We believed this behavior
contradicted the definition of a `PrefixDB` as a logical database that
prepends the prefix to all keys before operating on the underlying
database.
Partially addresses #4486.

This PR removes cometbft-db from cometbft build process:
- Dockerfiles
- Makefiles
- CI
Partially addresses #4486.

Context
CometBFT will stop importing cometbft-db as a dependency to support multiple database backends. Instead, it will use pebble by default.

Changes
This PR removes cometbft-db as a dependency and changes all the code to use the new internal/storage pkg instead.

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
@melekes melekes requested a review from a team as a code owner January 23, 2025 07:01
@melekes melekes requested a review from a team January 23, 2025 07:01
@melekes melekes requested a review from a team as a code owner January 23, 2025 07:01
@melekes melekes self-assigned this Jan 23, 2025
@mergify mergify bot mentioned this pull request Jan 23, 2025
@mergify mergify bot mentioned this pull request Jan 23, 2025
@melekes

This comment was marked as resolved.

@melekes
Copy link
Collaborator Author

melekes commented Feb 3, 2025

#4915

@zrbecker
Copy link

zrbecker commented Feb 7, 2025

This looks good to me. It doesn't modify the public components of CometBFT as much as I thought it would, mostly stuff around config and CLI.

@faddat
Copy link
Contributor

faddat commented Feb 7, 2025

Thank you very much! Now we have one full of fewer repository to ever think about again

@melekes melekes added this pull request to the merge queue Feb 10, 2025
Merged via the queue into main with commit dceff8b Feb 10, 2025
65 checks passed
@melekes melekes deleted the feature/remove-cometbftdb branch February 10, 2025 16:07
zrbecker pushed a commit to zrbecker/cometbft that referenced this pull request May 31, 2025
zrbecker pushed a commit to zrbecker/cometbft that referenced this pull request Jun 3, 2025
github-merge-queue bot pushed a commit that referenced this pull request Jun 3, 2025
---
Closes: #5148

Preparing main branch for a release cut, and want to pull back the db
backend removal for now. Original PR
(#4868)

Also restoring a Sha256 hash function that the SDK uses in a number of
places for easier upgradability.

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CometBFT uses a single DB backend

3 participants