Skip to content

build(deps): CometBFT no longer imports cometbft-db#4601

Merged
melekes merged 64 commits intofeature/remove-cometbftdbfrom
alesforz/code-uses-storage-pkg
Jan 23, 2025
Merged

build(deps): CometBFT no longer imports cometbft-db#4601
melekes merged 64 commits intofeature/remove-cometbftdbfrom
alesforz/code-uses-storage-pkg

Conversation

@alesforz
Copy link
Collaborator

@alesforz alesforz commented Dec 3, 2024

Partially addresses #4486.

Context

CometBFT will stop importing cometbft-db as a dependency to support multiple database backends. Instead, it will use pebble by default.

Changes

This PR removes cometbft-db as a dependency and changes all the code to use the new internal/storage pkg instead.

TODO


PR checklist

  • Tests written/updated
  • [ ] Changelog entry added in .changelog (we use unclog to manage our changelog)
  • Updated relevant documentation (docs/ or spec/) and code comments

Alessandro added 30 commits December 3, 2024 11:26
- `NewMemDB` to create an in-memory instance of pebble (for testing).
- `PrefixIterator` to create an iterator over keys that begin with a given prefix.
- `incrementBigEndian` to increase the value of a byte slice (read as a big-endian
   unsigned integer) by one.
github-merge-queue bot pushed a commit that referenced this pull request Dec 10, 2024
…ronment` (#4639)

### Context
Because `rpc.Environment` does not store a `GenesisDoc` in memory
anymore, (see #1290), we don't need to create it as a singleton. The
risk of storing multiple copies of the genesis in memory isn't there
anymore, because we now load it from disk.

### This Change
This PR removes the `sync.Once` construct that we put in place.

### Additional Note
The change also ensures that
[`TestProvider`](https://github.com/cometbft/cometbft/blob/b16c6fc2c8b2fc5a468fead32a8fe9057d6cce2f/light/provider/http/http_test.go#L36)
in the `light/provider/http` package behaves as expected. In `main`,
this test passes because it's set up using a
[`MemDB`](https://github.com/cometbft/cometbft-db/blob/4cf60c715fe8daccb9dce3b24295575bd461d5d8/memdb.go#L52),
which is a dummy in-memory store rather than a real database. Thus,
database `Get` operations always succeed.

However, in the context of the [work to remove
cometbft-db](#4601), we now use
a "real" database, i.e., one that a `Node` closes when it shuts down.
Since the `Environment` object was treated as a singleton, each
iteration of `TestProvider` created a new `Node` using the same
underlying database. This database would be closed at the end of the
first iteration when that iteration's `Node` shut down. Subsequent
iterations then attempted to call `Get` on a closed database, causing a
panic.

This change fixes that issue.

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- ~[ ] Updated relevant documentation (`docs/` or `spec/`) and code
comments~
github-merge-queue bot pushed a commit that referenced this pull request Dec 11, 2024
…4642)

### Context
The
[`TxIndexer`](https://github.com/cometbft/cometbft/blob/2b1db1c16bf2db16b81b49fef3581e79679fbed6/state/txindex/indexer.go#L17)
interface defines how to index and search transactions. Its implementers
need to interact with a database, which callers are typically expected
to close when done. However, `TxIndexer` does not provide a `Close`
method. This prevents closing the database used by the transaction
indexer, causing goroutines associated with that database to leak.

Two
[tests](https://github.com/cometbft/cometbft/blob/2b1db1c16bf2db16b81b49fef3581e79679fbed6/internal/inspect/inspect_test.go#L29),
`TestInspectConstructor` and `TestInspectRun`, show this problem
indirectly. These tests check whether an
[`Inspector`](https://github.com/cometbft/cometbft/blob/2b1db1c16bf2db16b81b49fef3581e79679fbed6/internal/inspect/inspect.go#L32)
leaks goroutines. The `Inspector` sets up three databases: a block
database, a state database, and a transaction indexer database. It
closes only the block and state databases because it cannot close the
transaction indexer’s database (because of the missing `Close` method).
Therefore, the transaction indexer database's goroutines leak, and the
two tests above detect that.

In `main`, both tests pass because they use a
[`MemDB`](https://github.com/cometbft/cometbft-db/blob/4cf60c715fe8daccb9dce3b24295575bd461d5d8/memdb.go#L52),
an in-memory store that does not spawn goroutines. However, in the [work
to remove cometbft-db](#4601),
the tests switch to a “real” database that does spawn goroutines.
Without a proper `Close` method in `TxIndexer`, these goroutines remain
active, causing `TestInspectConstructor` and `TestInspectRun` to fail.

### This Change 
This PR adds a `Close` method to the `TxIndexer` interface and updates
the existing implementation to properly close the underlying database.
It also modifies the code to call `TxIndexer.Close` where needed,
ensuring that database resources are released and preventing goroutine
leaks.

---

#### PR checklist

- [x] Tests written/updated
- [x] Changelog entry added in `.changelog` (we use
[unclog](https://github.com/informalsystems/unclog) to manage our
changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code
comments
Alessandro Sforzin added 2 commits December 11, 2024 16:38
The test unfolds too quickly for the node's indexer to finish its operations.
Therefore, the goroutine below that calls n.Stop() will close the indexer's
database right when the indexer is trying to write to it at line
state/txindex/indexer_service.go:109
thus causing a panic.  This small sleep allows the indexer to finish its
write operation before the node is shut down.
Unfortunately, there is no graceful shutdown available for the database layer,
i.e., make the database wait for all the active connections before closing.

The test does not fail in `main` because the code in `main` does not close
the indexer's database when shutting down a node (which causes its goroutines
to leak. See #4642).
@alesforz alesforz marked this pull request as ready for review December 11, 2024 17:35
@alesforz alesforz requested a review from a team as a code owner December 11, 2024 17:35
@alesforz alesforz requested a review from a team December 11, 2024 17:35
Copy link
Collaborator

@melekes melekes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work @alesforz 👍

### Context
In reviewing #4601, we discussed about exporting the `DB`, `Batch`, and
`Iterator` interfaces instead of leaving them in an `internal` package.
After some thinking we decided to export them in a new non-internal
package `cmtdb`. However, our implementation of those interfaces remain
unexported.

### Changes
This PR adds pkg `cmtdb` exporting the `DB`, `Batch`, and `Iterator`
interfaces. Additionally, it changes the code to use this new package
rather than the now obsolete `internal/storage` (which this PR removes).
Copy link
Collaborator

@melekes melekes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale For use by stalebot label Dec 31, 2024
@cason cason added wip Work in progress and removed stale For use by stalebot labels Dec 31, 2024
….DefaultDBProvider()` (#4697)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
@zrbecker zrbecker marked this pull request as draft January 22, 2025 19:04
@zrbecker
Copy link

@melekes I noticed this was marked with wip, but had an approval.

Looks like this PR has been stale about a month. Should this work still be done?

@melekes
Copy link
Collaborator

melekes commented Jan 23, 2025

Should this work still be done?

It would be beneficial for CometBFT.

@melekes melekes marked this pull request as ready for review January 23, 2025 05:09
@melekes melekes removed the wip Work in progress label Jan 23, 2025
@melekes melekes merged commit 56ffb50 into feature/remove-cometbftdb Jan 23, 2025
@melekes melekes deleted the alesforz/code-uses-storage-pkg branch January 23, 2025 06:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Dependency updates P:tech-debt Priority: Technical debt that needs to be paid off to enable us to move faster, reliably storage

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants