Skip to content

[blockchain] v2 riri Schedule composit data structure#3848

Merged
tac0turtle merged 9 commits intomasterfrom
brapse/blockchain-riri-schedule
Aug 5, 2019
Merged

[blockchain] v2 riri Schedule composit data structure#3848
tac0turtle merged 9 commits intomasterfrom
brapse/blockchain-riri-schedule

Conversation

@brapse
Copy link
Contributor

@brapse brapse commented Jul 30, 2019

(This PR trumps #3821)

The blockchain reactor is responsible for fetching and applying blocks from peers in order to catch up a node. #3777 specifies a new architecture architecture which includes a scheduler component which:

  • Decouples scheduling from IO
  • Has strictly deterministic business logic
  • Uses explicit time instead of timers

This PR include an implementation of the schedule data structure. The schedule is a data structure used to determine the optimal schedule of requests to the optimal set of peers in order to perform fast-sync as fast and efficiently as possible.

Blocks transition from:
unknown -> new -> pending -> received -> processed
and peers transition from:
new -> ready -> removed
based on a synchronous methods calls.

This PR aims to solve #2897 by modeling a large part of the protocol as synchronous methods which can tested/verified directly.

  • Referenced an issue explaining the need for the change
  • Updated all relevant documentation in docs
  • Updated all code comments where relevant
  • Wrote tests
  • Updated CHANGELOG_PENDING.md

    + The schedule is a data structure used to determine the optimal
      schedule of requests to the optimal set of peers in order to perform
      fast-sync as fast and efficiently as possible.
@brapse brapse marked this pull request as ready for review July 30, 2019 16:56
@brapse brapse requested review from ebuchman, melekes and xla as code owners July 30, 2019 16:56
@brapse brapse requested a review from ancazamfir July 30, 2019 16:56
@codecov-io
Copy link

codecov-io commented Jul 30, 2019

Codecov Report

Merging #3848 into master will decrease coverage by 1.05%.
The diff coverage is 67.01%.

@@            Coverage Diff             @@
##           master    #3848      +/-   ##
==========================================
- Coverage   65.99%   64.94%   -1.06%     
==========================================
  Files         237      217      -20     
  Lines       20388    18210    -2178     
==========================================
- Hits        13456    11827    -1629     
+ Misses       5879     5465     -414     
+ Partials     1053      918     -135
Impacted Files Coverage Δ
blockchain/v2/schedule.go 67.01% <67.01%> (ø)
rpc/lib/server/http_params.go
types/encoding_helper.go
types/params.go
types/validator_set.go
proxy/multi_app_conn.go
types/block.go
types/evidence.go
types/proposal.go
crypto/secp256k1/secp256k1.go
... and 39 more

@tac0turtle
Copy link
Contributor

@ancazamfir @brapse I believe this should be merged and then we can start on the stage of this process, thoughts?

brapse and others added 5 commits August 5, 2019 11:10
    + The schedule is a data structure used to determine the optimal
      schedule of requests to the optimal set of peers in order to perform
      fast-sync as fast and efficiently as possible.
Copy link
Contributor

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed on tm call, will merge this PR ⚡️

@tac0turtle tac0turtle merged commit 0cf8812 into master Aug 5, 2019
@tac0turtle tac0turtle deleted the brapse/blockchain-riri-schedule branch August 5, 2019 15:26
"github.com/tendermint/tendermint/p2p"
)

type Event interface{}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

godoc?

case blockStateProcessed:
return "Processed"
default:
return fmt.Sprintf("unknown blockState: %d", e)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't it be better to panic in this case?

case peerStateRemoved:
return "Removed"
default:
return fmt.Sprintf("unknown peerState: %d", e)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same question


func (sc *schedule) addPeer(peerID p2p.ID) error {
if _, ok := sc.peers[peerID]; ok {
return fmt.Errorf("Cannot add duplicate peer %s", peerID)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return fmt.Errorf("Cannot add duplicate peer %s", peerID)
return fmt.Errorf("cannot add duplicate peer %s", peerID)

errors must not start with a capital letter

func (sc *schedule) markPending(peerID p2p.ID, height int64, time time.Time) error {
peer, ok := sc.peers[peerID]
if !ok {
return fmt.Errorf("Can't find peer %s", peerID)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var ErrNoPeer = fmt.Errorf("can't find peer %s", peerID)

}

state := sc.getStateAtHeight(height)
if state != blockStateNew {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't select be more appropriate here

switch {
case state != blockStateNew:
  return ...
case peer.state != peerStateReady:
  ...

return heights
}

func (sc *schedule) selectPeer(peers []*scPeer) *scPeer {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

randomPeer?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants