cmd: add tool for compaction of goleveldb#8564
Conversation
creachadair
left a comment
There was a problem hiding this comment.
Mechanically this seems fine, just one small concurrency thing I noticed (plus @tychoish's comment).
| Use: "experimental-compact-db", | ||
| Short: "force compacts the tendermint storage engine (only GoLevelDB supported)", | ||
| Long: ` | ||
| This is a temporary utility command that performs a force compaction on the state |
There was a problem hiding this comment.
If this is temporary, can we avoid having it live in the command-line tool entirely? It seems like this could be a separate program that the operator can run if this affects them.
If we keep wanting to do this sort of thing longer-term, I wonder if we shouldn't make an experimental subsection of the CLI (e.g., tendermint experimental screw-up-my-settings-because-i-like-it-that-way --force).
There was a problem hiding this comment.
How I've been seeing the tendermint cli (and this can be of course subject to change) is as a utility tool predominantly for node operators to assist them in their operations. In this regard, I'm happy to bloat out the binary with various commands so long as the cli remains intuitive to use and structured.
I think there's merit in the point that operators running out of process nodes perhaps don't want all this extra stuff hanging on to their binary
There was a problem hiding this comment.
I think the view that the tendermint binary doesn't matter makes sense if all nodes are expected to be wrapped in the SDK. And I guess that's the most common case. But I don't think that means we should just say "anything goes" for folks who want to run a non-SDK chain.
Perhaps what we need is a second designated tmutils tool, where the kitchen sink can go.
(cherry picked from commit 30bfe51) # Conflicts: # cmd/tendermint/main.go # go.mod # go.sum
(cherry picked from commit 30bfe51) # Conflicts: # cmd/tendermint/main.go # go.mod
goleveldb has been reported to continue growing in disk size even after pruning. This is due to goleveldb's poor compaction. To manage disk space, validators periodically use state sync to reset and restart their nodes.
There are some tools such as https://github.com/binaryholdings/cosmprund which provide a cli for force compacting the db. This PR ports this across to the tendermint cli for more convenient access for validators.
Note: this directly imports goleveldb as there are issues with updating tm-db (see #8014)