Bandwidth optimization #30
Description
Inspired by tendermint/tendermint#9880 but expanded here.
High-level tracking issue for general bandwidth optimization efforts.
CometBFT-based networks consume large amounts of bandwidth.
This results in high operational costs for the network operators (typically, the validators).
We need to reduce the bandwidth consumption of the solution while maintaining its core properties, in terms of correctness, progress and fault-tolerance.
Several strategies to reduce the bandwidth usage in CometBFT have been proposed by CometBFT developers and community members.
While some of these ideas are simple to implement, others require much larger efforts, implying impactful modifications on the protocol.
This results in a dilemma when looking for a solution. We also note that there is currently no general strategy to validate a solution, whether it would be through local tests, or by collecting information from the operators.
A third concern comes from the fact that while there are clear quick wins, it seems also necessary to prepare a long term effort.
The subject of improving bandwidth usage in peer-to-peer systems is vast, with decades of efforts in research and practical solutions.
This issue tracks our efforts in addressing a large spectrum of approaches to reduce bandwidth usage in CometBFT.
We prioritize the work based on their impact on use cases, on the effort needed to implement it, and on how it helps pave the way for future improvements.
The main problem is subdivided as listed next. Note that some of the problems/solutions might overlap.
### Preliminary work
- [ ] #1048
- [ ] #1076
- [ ] #1081
### Short term goals
- [ ] #1058
- [ ] #1060
- [ ] #1059
- [ ] #1061
### Long term goals
- [ ] #11
- [ ] #2027
- [x] Feasibility study to replace the current gossip protocol
- [ ] #3297
Previous efforts
- Investigate what kind of data is consuming the most bandwidth
- rfc 27: report on bandwidth usage within Tendermint tendermint/tendermint#9706 (Rendered RFC)
- Pinpoint inefficiencies in block, vote and transaction propagation #26
- mempool: handling incoming txs while catching up #734
-Changes from consensus: correctly save reference to previous height precommits tendermint/tendermint#9760, released in v0.34.25, testing for backwards compatibility in e2e: Enable multi-version testing in nightly QA and for releases tendermint/tendermint#9928
Related, but not covered by this issue.
- Investigate protocol change options for more substantial bandwidth optimization
References
- Bandwidth Improvements tendermint/tendermint#9575 (efforts toward bandwidth improvements in Q4-2022)