Skip to content

ValidatorSet change delayed by 1 block, and lite refactor (#2)#1815

Merged
ebuchman merged 32 commits intodevelopfrom
jae/literefactor4
Aug 5, 2018
Merged

ValidatorSet change delayed by 1 block, and lite refactor (#2)#1815
ebuchman merged 32 commits intodevelopfrom
jae/literefactor4

Conversation

@jaekwon
Copy link
Contributor

@jaekwon jaekwon commented Jun 26, 2018

Rebase of #1756. Addresses #1313

@jaekwon jaekwon requested review from ebuchman, melekes and xla as code owners June 26, 2018 01:17
@ebuchman
Copy link
Contributor

LGTM

Not sure why the test_p2p is failing here but not develop - looks like it has to do with ABCI getting merged in but it's trying to install it from an external repo! @xla @melekes

Need to update the changelog - this is breaking for:

  • Header (breaks blockchain)
  • State (breaks on disk representation of state)
  • State execution - ie. when validator set changes take effect
  • lite - lots of changes
  • rpc - /commit result has signed_header field (instead of its elements being top level)

@melekes
Copy link
Contributor

melekes commented Jun 26, 2018

    --- FAIL: TestWALCrash/many_non-empty_blocks (110.74s)
        replay_test.go:137: ====== LOOP 1
        replay_test.go:172: WAL paniced: failed to write consensus.timeoutInfo to WAL (fileline: /go/src/github.com/tendermint/tendermint/consensus/state.go:593)
        replay_test.go:64: ====== WAL:
                ^MB16ED404000000160A0E099FE8315B000000001502722E391204C7309B41B89F99210000002C0A0E099FE8315B00000000155AF7C62B121A9DB37AAE08041A12526F756E64537465704E6577486569676874CD5C66830000002A0A0E09A0E8315B00000000157EFCE50012189DB37AAE08041A10526F756E645374657050726F706F7365
        replay_test.go:137: ====== LOOP 2
        replay_test.go:172: WAL paniced: failed to write consensus.timeoutInfo to WAL (fileline: /go/src/github.com/tendermint/tendermint/consensus/state.go:593)
        replay_test.go:64: ====== WAL:
                ^M39C82B3F000000160A0E09A1E8315B00000000159021E0241204C7309B41
        replay_test.go:137: ====== LOOP 3
        replay_test.go:172: WAL paniced: failed to write types.EventDataRoundState to WAL (fileline: /go/src/github.com/tendermint/tendermint/consensus/state.go:545)
        replay_test.go:64: ====== WAL:
                ^MADDF7164000000160A0E09A2E8315B0000000015F6171C141204C7309B41
        replay_test.go:137: ====== LOOP 4
        replay_test.go:172: WAL paniced: failed to write consensus.msgInfo to WAL (fileline: /go/src/github.com/tendermint/tendermint/consensus/replay_test.go:247)
        replay_test.go:64: ====== WAL:
                ^MD93C119F000000160A0E09A3E8315B00000000155CCF0E161204C7309B41
        replay_test.go:137: ====== LOOP 5
        replay_test.go:172: WAL paniced: failed to write consensus.msgInfo to WAL (fileline: /go/src/github.com/tendermint/tendermint/consensus/replay_test.go:247)
        replay_test.go:64: ====== WAL:
                ^M58B0F040000000160A0E09A3E8315B0000000015F89575361204C7309B41
        replay_test.go:137: ====== LOOP 6
        replay_test.go:172: WAL paniced: failed to write types.EventDataRoundState to WAL (fileline: /go/src/github.com/tendermint/tendermint/consensus/state.go:545)

wal tests are failing. error messages can definitely be improved

@jaekwon jaekwon force-pushed the jae/literefactor4 branch 2 times, most recently from 1e1bfb2 to f5d0b88 Compare June 27, 2018 22:38
@codecov-io
Copy link

codecov-io commented Jun 27, 2018

Codecov Report

❗ No coverage uploaded for pull request base (develop@6e3c5e8). Click here to learn what that means.
The diff coverage is 52.2%.

@@            Coverage Diff             @@
##             develop    #1815   +/-   ##
==========================================
  Coverage           ?   60.89%           
==========================================
  Files              ?      193           
  Lines              ?    15802           
  Branches           ?        0           
==========================================
  Hits               ?     9623           
  Misses             ?     5370           
  Partials           ?      809
Impacted Files Coverage Δ
rpc/core/pipe.go 31.7% <ø> (ø)
libs/pubsub/pubsub.go 83.89% <ø> (ø)
lite/proxy/query.go 0% <0%> (ø)
lite/proxy/verifier.go 0% <0%> (ø)
rpc/core/consensus.go 0% <0%> (ø)
state/validation.go 36.36% <0%> (ø)
lite/proxy/wrapper.go 0% <0%> (ø)
cmd/tendermint/commands/lite.go 13.15% <0%> (ø)
rpc/core/blocks.go 18.75% <0%> (ø)
rpc/core/status.go 0% <0%> (ø)
... and 13 more

@jaekwon jaekwon force-pushed the jae/literefactor4 branch from f5d0b88 to b51ed13 Compare June 27, 2018 23:24
@jaekwon jaekwon mentioned this pull request Jun 29, 2018
7 tasks
@ebuchman
Copy link
Contributor

ebuchman commented Jul 2, 2018

Merged in develop post v0.22.0 launch. Will merge this back to develop today and prepare for v0.23.0. Whoo!

err := ic.useClosestTrust(fc.Height())
if err != nil {
// Get the next validator set.
nextValset, err := ic.source.ValidatorSet(ic.chainID, shdr.Height+1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't understand the purpose of this line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once you've certified a header, you want to also fetch the next validator set. the next validator set (hash) is in the block header, which we can do because we delay validator set changes by 1 block (this PR).

// best match trusted full commit, and if good, persist to ic.trusted.
// Returns ErrTooMuchChange when >2/3 of tfc did not sign sfc.
// Panics if tfc.Height() >= sfc.Height().
func (ic *InquiringCertifier) verifyAndSave(tfc, sfc FullCommit) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use longer names for parameters.

FOR_LOOP:
for {
// Fetch latest full commit from trusted.
tfc, err := ic.trusted.LatestFullCommit(ic.chainID, 1, h)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Latest trusted full commit is known at this point as it has already been fetched.
Why we don't pass it as a parameter to this function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO make issue and assign to Jae

}

// Try to update to full commit with checks.
err = ic.verifyAndSave(tfc, sfc)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I am not wrong this method does not return ErrTooMuchChange.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm wonder why the tests didn't catch it. I'll fix and test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO make issue and assign to Jae

// we can handle IsTooMuchChangeErr specially
if !liteErr.IsTooMuchChangeErr(err) {
return err
FOR_LOOP:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This loop can probably be rewritten to reduce number of LastCommit fetches.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saving optimizations for later. This one isn't too bad, still has the O() property we want, though the whole scheme could improve w/ Tendermint streaming memoized valset changes to the lite client (rather than any binary search).

)

func ValidateBlockMeta(meta *types.BlockMeta, check lite.Commit) error {
func ValidateBlockMeta(meta *types.BlockMeta, sh types.SignedHeader) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proxy package does not seem to be in finished state.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO make issue

Copy link
Contributor

@milosevic milosevic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have finished review of lite package. There are lot of good code, concepts are cleaner than before, but there are still few open questions and possibility for code improvement. I recommend to extract parts of clean code (in smaller) chunks from this code and to go with normal development process with it.

@jaekwon jaekwon mentioned this pull request Aug 2, 2018
4 tasks
@ebuchman
Copy link
Contributor

ebuchman commented Aug 2, 2018

Ok - merged develop and fixed conflicts. Thanks @jaekwon for addressing some review and removing the consensus changes.

Let's get this merged once the tests pass and punt all the other comments to future issue: #2137

@ebuchman
Copy link
Contributor

ebuchman commented Aug 2, 2018

Linter failed because amino is pointing to a branch rather than a release ...

@ebuchman ebuchman merged commit 06a157a into develop Aug 5, 2018
@zramsay zramsay deleted the jae/literefactor4 branch November 8, 2018 19:02
cboh4 pushed a commit to scrtlabs/tendermint that referenced this pull request Apr 7, 2025
…LightTrusting` (tendermint#1806) (tendermint#1815)

* Introduce `countAllSignatures` in `VerifyCommitLight` & `VerifyCommitLightTrusting`

* Revert unneded change

* Addressed @insumity's comments

---------

Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants