I have been recently thinking about possible improvements to the way BlockID's are used and how blocks are propogated. This issue sets out to encompass these thoughts.

NOTE: There are parts here that may be seen as spec level and other parts that may be seen as implementation level. I apologise for jumbling them together.

Motivation

• Reformat Block ID to be purely the hash of the header and to not include PartSetHeader. This therefore can reduce header/block sizes.
• Use a PartSet method that doesn't duplicate Commit's in the BlockStore
• Leverage incremental verification of block

Current Flow

For those that have a good understanding of the flow of consensus, you can probably just skip this section. For those that would like a quick summary of the particular flow in consensus I am addressing I will try to provide a brief overview.

In the first round, the proposer produces a block and a proposal for that block. It gossips this proposal and then breaks the block into parts where it can broadcast each part to its peers.

A node that receives the proposal will verify that it is signed by the correct validator and then set the proposal and PartSet as follows:

cs.Proposal = proposal
cs.ProposalBlockParts = types.NewPartSetFromHeader(proposal.BlockID.PartSetHeader)

As parts come in they are validated (using merkle proofs) and added to the PartSet. At the same time nodes can be receiving and tallying votes. Only once all the parts are received and the block is complete does the node advance to the prevote state, verify the block and then send out a prevote for it. It then becomes set here:

cs.ValidBlock = block

On the other side of the coin, nodes that have a PartSet will gossip both the header and part set to peers to help them catch up and enable them to be able to vote.

Proposal

I will break my proposal into two phases

Phase 1

In regards to the current flow, there doesn't seem to be any need for PartSetHeader outside of consensus. Therefore the primary change would be to remove this from BlockID, thus changing from:

type BlockID struct {
	Hash          tmbytes.HexBytes `json:"hash"`
	PartSetHeader PartSetHeader    `json:"part_set_header"`
}

to

type BlockID tmbytes.HexBytes `json:"hash"`

Vote's would continue to be for BlockID's (or the hash of the header) but Proposal's would look slightly different:

type Proposal struct {
	Height       int64             `json:"height"`
	Round        int32             `json:"round"`
	POLRound     int32             `json:"pol_round"` // -1 if null.
  // Hash of the block parts header
	BlockPartsID tmbytes.HexBytes  `json:"block_parts_id"`
	Timestamp    time.Time         `json:"timestamp"`
	Signature    []byte            `json:"signature"`
}

Proposers would broadcast this message, alongside the PartSetHeader and block Part's. In the gossip data routine, peers would send data to peers in the following order:

1. Proposal
2. PartSetHeader
3. Special block Part's (to be discussed later)
4. Normal block Part's'

Nodes who received PartSetHeader would hash it and compare with cd.Proposal.BlockPartsID before locking it.

cs.Proposal = proposal
cs.ProposalBlockParts = partSetHeader

Phase 2

Now I will cover changes to the PartSetHeader and how we can incrementally verify the block. This will look as follows:

type PartSetHeader struct {
  Header		Header			`json:"header"`
  LastCommit		Commit			`json:"last_commit"`
  EvidenceParts	        uint32			`json:"evidence_parts"`
  TxParts		uint32			`json:"tx_parts"`
  Hash			tmbytes.HexBytes	`json:"hash"`
}

After confirming this matches the part set header in the proposal, a node will begin to validate both the header and the last commit. If any of these fail verification, the node can prevote<nil> early. After verifying this, it will only need to (a) verify evidence (if there is any) and (b) check that the DataHash does in fact match the hash of all the data.

99.99% of the time EvidenceParts will be 0, which means there is no evidence to verify. In the case that there is evidence, EvidenceParts represents the first n amount of parts that can be combined to form the EvidenceData struct. These first parts are what I denoted as special block Parts and will generally get sent first. When all evidence parts are collected, the node can then verify the evidence in the block. The node then must only wait for the remainding TxParts before entering the prevote step. Part verification remains the same, using a merkle proof, the hash of the PartSetHeader and the bytes of the part to prove their validity.

The last piece of the proposal, the blocks themselves, are stored in the BlockStore by being split up into the Header (or BlockMeta), the commit, and a set of block Part's that represent the evidence and txs. This avoids duplication of the header and commit which we currently have.

Reference

• BlockID: Consider ways to remove PartsHeader spec#78
• Bucky's comment (specifically see the Other section) Vote & commitsig redundancy  #4835 (comment)

Notes

• If we have a height with multiple rounds, proposers at later rounds will first check if there is already a ValidBlock and repropose that. This uses the existing block and thus the existing header which has the original proposer address. Therefore we might see a circumstance where a block is committed in round 1 with proposer B but the block has proposer A from round 0 in the header. Is this a concern?

• When reproposing an existing block, a proposer will gossip the existing block parts, now with an updated round. Is there necessity in regossiping a block that has already been gossiped. Is it not sufficient enough that the new proposer sends a new proposal but now links to the ID of the existing block.

• I have opted to having the evidence (which 99% of the time is empty), come in a separate part. If we take the current part size of 65536 Bytes , it may be reasonable to include evidence also as part of the PartSetHeader

• Whilst looking through the code, I asked myself whether block validity was a prerequisite to prevoting for a block or was it just precommitting. Given that after receiving the PartSetHeader nodes will have verified the header, commit and potentially evidence (if there is none). In fact the only remaining check for validity is that the block is complete and that the txs hash to the data hash. Hence, I was wondering whether it would be possible after validating the PartSetHeader for a node to prevote for that block. Then in the rara case that it never received the rest of the parts before timeoutPrecommit or the data hash didn't match then the node would afterwards precommit nil and thus never actually locked on to the block. I am not sure if this breaches Tendermints safety and correctness gaurantees.