feat: election of ValidatorSet based on VRF#74
Conversation
|
This is why the |
| ChainId: h.genDoc.ChainID, | ||
| ConsensusParams: csParams, | ||
| Validators: nextVals, | ||
| Validators: nextVals, // ValidatorOrVoter: validator |
There was a problem hiding this comment.
Could this property be 'Validators' or 'Voters'?
When does it become 'Validators' and when does it become 'Voters'?
types/voter_set.go
Outdated
|
|
||
| } | ||
|
|
||
| func (voters *VoterSet) SelectProposer(proofHash []byte, height int64, round int) *Validator { |
There was a problem hiding this comment.
Why did you add the ability to select proposers to this 'VoterSet' struct?
As I know the VoterSet store the selected voters, so I don't think it's appropriate.
circleci failure is fixed at tendermint/tendermint@960c740 |
state/validation.go
Outdated
| // a legit address and a known validator. | ||
| if len(block.ProposerAddress) != crypto.AddressSize || | ||
| !state.Validators.HasAddress(block.ProposerAddress) { | ||
| !state.Voters.HasAddress(block.ProposerAddress) { |
There was a problem hiding this comment.
It must be Validators
state/validation.go
Outdated
| // validate vrf proof | ||
| message := state.MakeHashMessage(block.Round) | ||
| _, val := state.Validators.GetByAddress(block.ProposerAddress) | ||
| _, val := state.Voters.GetByAddress(block.ProposerAddress) |
| NextValidatorsHash tmbytes.HexBytes `json:"next_validators_hash"` // validators for the next block | ||
| ConsensusHash tmbytes.HexBytes `json:"consensus_hash"` // consensus params for current block | ||
| AppHash tmbytes.HexBytes `json:"app_hash"` // state after txs from the previous block | ||
| VotersHash tmbytes.HexBytes `json:"voters_hash"` // voters for the current block |
There was a problem hiding this comment.
Since the VoterSet is obtained from the ValidatorSet, I will not include Validators.hash() verification because I believe that the Voters.hash() verification alone can serve as Validators.hash() verification.
zemyblue
left a comment
There was a problem hiding this comment.
This PR is very big.
I couldn't check the detail of code.
Next time, please, separate appropriate size. 🙏
consensus/reactor.go
Outdated
| cs := conR.conS | ||
| cs.mtx.RLock() | ||
| height, valSize, lastCommitSize := cs.Height, cs.Validators.Size(), cs.LastCommit.Size() | ||
| height, valSize, lastCommitSize := cs.Height, cs.Voters.Size(), cs.LastCommit.Size() |
There was a problem hiding this comment.
How about changing the valSize to voterSize?
| return nil, err | ||
| } | ||
| state.Validators = types.NewValidatorSet(vals) | ||
| state.Voters = types.ToVoterAll(state.Validators) |
There was a problem hiding this comment.
When reset the block data, There may be a lot of validator. So, I think we need to select voterSet even at 0 block. And this function can be big logic. So, please add another PR.
evidence/pool.go
Outdated
| // TODO: something better ? | ||
| valset, _ := sm.LoadValidators(evpool.stateDB, evidence.Height()) | ||
| _, val := valset.GetByAddress(evidence.Address()) | ||
| _, voterSet, _ := sm.LoadValidators(evpool.stateDB, evidence.Height()) |
There was a problem hiding this comment.
Please check again.
Because the node to do violation may not be exist in voterSet.
There was a problem hiding this comment.
You're right. Non-voter can broadcast an evidence for a byzantine vote.
| // However, if the likely candidates are less than the `limitCandidates`, | ||
| // the number of voters may be less than the `limitCandidates`. | ||
| // This is to prevent falling into an infinite loop. | ||
| func RandomSamplingToMax( |
There was a problem hiding this comment.
This logic of function is difference with #22's algorithm.
So, it need to proof the logic.
Please, check it.
There was a problem hiding this comment.
It is the same as #22 algorithm(implemented as RandomSamplingWithPriority) except to limit the number of loops as limitCandidates. Let's continue to discuss MaxVoters problem at the next meeting.
libs/rand/sampling_test.go
Outdated
| e.Win = uint64(float64(e.Win) * times) | ||
| return e.Win | ||
| } | ||
|
|
There was a problem hiding this comment.
Please, add test case of RandomSamplingToMax function
rpc/client/mock/client.go
Outdated
|
|
||
| func (c Client) Validators(height *int64, page, perPage int) (*ctypes.ResultValidators, error) { | ||
| return core.Validators(&rpctypes.Context{}, height, page, perPage) | ||
| func (c Client) Validators(height *int64, page, perPage int) (*ctypes.ResultVoters, error) { |
There was a problem hiding this comment.
I think this Validators function doesn't need.
rpc/core/consensus.go
Outdated
| return validators(ctx, heightPtr, page, perPage, sm.LoadValidators) | ||
| } | ||
|
|
||
| func validators(ctx *rpctypes.Context, heightPtr *int64, page, perPage int, |
There was a problem hiding this comment.
How about changing this function name to voters?
| // Extra +1 due to nextValSet delay. | ||
| NextValidators *types.ValidatorSet | ||
| Validators *types.ValidatorSet | ||
| LastValidators *types.ValidatorSet |
There was a problem hiding this comment.
The Validators remains, but the LastValidators is removed.
If you think the Validators need to remain, please, check the existence of LastValidators
There was a problem hiding this comment.
Validators is used for selecting proposer, but LastValidators is used nowhere.
There was a problem hiding this comment.
OK, I understand.
Thank you.
types/voter_set.go
Outdated
| result := &VoterSet{Voters: copyValidatorListShallow(validators.Validators), totalVotingPower: 0} | ||
| result.updateTotalVotingPower() |
There was a problem hiding this comment.
Isn't this the same as NewVoterSet(validators.Validators)?
There was a problem hiding this comment.
You're right. I will fix it.
| @@ -0,0 +1,112 @@ | |||
| package types | |||
There was a problem hiding this comment.
I feel there seem to be too few test cases for the features in types/voter_set.go, but was the original Tendermint also this much about?
There was a problem hiding this comment.
This PR has become so big that I will post the tests as a separate PR.
| func (c *Client) Validators(height *int64, page, perPage int) (*ctypes.ResultVoters, error) { | ||
| return c.next.Voters(height, page, perPage) |
There was a problem hiding this comment.
Is it ok the return value is ResultVoters against the method name Validators()? And if the process is exactly the same as Voters(), I think it's better to replace return c.Voters(...).
libs/rand/sampling.go
Outdated
| Priority() uint64 | ||
| LessThan(other Candidate) bool | ||
| IncreaseWin() | ||
| MultiplyWin(times float64) uint64 |
There was a problem hiding this comment.
times will probably be a natural number, but why float64?
There was a problem hiding this comment.
Oh, it's not used now. I will remove it. This was made to coordinate rewards, which was intended to be corrected by a floating number of times for more accuracy. We should further experiment and discuss and coordinate compensation.
feat: add voteset to state feat: implement VoterSet fix: test failure fix: change validators to voters more feat: implement select voters feat: implement RandomSamplingToMax feat: add test case feat: more test fix: fmt check failure fix: circleci failure fix: randValidator may create a validator having 0 voting power fix: elect a proposer among validators not among voters fix: apply comment; proposer must be found in validators fix: apply comments fix: contracts_test failure fix: contracts_test failure fix: apply comments
feat: add voteset to state feat: implement VoterSet fix: test failure fix: change validators to voters more feat: implement select voters feat: implement RandomSamplingToMax feat: add test case feat: more test fix: fmt check failure fix: circleci failure fix: randValidator may create a validator having 0 voting power fix: elect a proposer among validators not among voters fix: apply comment; proposer must be found in validators fix: apply comments fix: contracts_test failure fix: contracts_test failure fix: apply comments
feat: add voteset to state feat: implement VoterSet fix: test failure fix: change validators to voters more feat: implement select voters feat: implement RandomSamplingToMax feat: add test case feat: more test fix: fmt check failure fix: circleci failure fix: randValidator may create a validator having 0 voting power fix: elect a proposer among validators not among voters fix: apply comment; proposer must be found in validators fix: apply comments fix: contracts_test failure fix: contracts_test failure fix: apply comments
feat: add voteset to state feat: implement VoterSet fix: test failure fix: change validators to voters more feat: implement select voters feat: implement RandomSamplingToMax feat: add test case feat: more test fix: fmt check failure fix: circleci failure fix: randValidator may create a validator having 0 voting power fix: elect a proposer among validators not among voters fix: apply comment; proposer must be found in validators fix: apply comments fix: contracts_test failure fix: contracts_test failure fix: apply comments
feat: add voteset to state feat: implement VoterSet fix: test failure fix: change validators to voters more feat: implement select voters feat: implement RandomSamplingToMax feat: add test case feat: more test fix: fmt check failure fix: circleci failure fix: randValidator may create a validator having 0 voting power fix: elect a proposer among validators not among voters fix: apply comment; proposer must be found in validators fix: apply comments fix: contracts_test failure fix: contracts_test failure fix: apply comments
* Fix skipped `TestReactorWithTimeoutCommit` test * Fix `TestByzantine` * Modify to find proposer directly on `TestReactorRecordsVotesAndBlockParts` * fix: reactor test failure * set resource for the circleci machine * fix: skipped tests * fix: race bug * fix: apply comment * chore: remove blank line * fix: refine forceProposer * fix: remove hard code * fix: refactoring forceProposer * fixed skipped testcase (but in the end, skipped them that were difficult to apply VRF) * fixed all FIXME, pass all tests * Fix race detection error of `SelectProposer` (unit tests uncompleted) * Fix `TestReactorHelperMode` error in the `blockchain/v2/reactor_test.go` * fix lint warnings. * Modify the file name of `CHANGELOG.md` to `CHANGELOG_OF_TENDERMINT.md` * changelog and version * Add LINE version in Tendermint version * Disable checking the markdown links. - Error links are all origin Tendermint links * Fix swagger format error. * Fix dredd skip paths. * Fix `/genesis` dredd test - Change `max_age` to `max_age_num_blocks` of GenesisResponse in the swagger - Add `max_age_duration` of GenesisResponse in the swagger * Fix dredd fail case. - `broadcast_tx_sync` - `broadcast_tx_async` - `broadcast_tx_commit` - `tx_search` * Fix dredd test case(2) - /block_results - /tx_search - /dump_consensus_state - /consensus_params * Add fixed dredd test to circleCI * Add changelog * Remove unused RemoveProperty of unmarshal.go * feat: election of ValidatorSet based on VRF #74 feat: add voteset to state feat: implement VoterSet fix: test failure fix: change validators to voters more feat: implement select voters feat: implement RandomSamplingToMax feat: add test case feat: more test fix: fmt check failure fix: circleci failure fix: randValidator may create a validator having 0 voting power fix: elect a proposer among validators not among voters fix: apply comment; proposer must be found in validators fix: apply comments fix: contracts_test failure fix: contracts_test failure fix: apply comments * Apply changelog of Tendermint about v0.33.4 (It's a fix commit that has already been fixed in the past.) * fix lint error * fix p2p test of circleCI ``` The following packages have unmet dependencies: libc6-dev : Breaks: libgcc-8-dev (< 8.4.0-2~) but 8.3.0-6 is to be installed E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages. ``` * fix unit test of `helpers_test` * fix lint error * Update protobuf implementations (This commit is empty because it re-make *.pb.go with 3, which was accidentally generated with protobuf 2 in the past commit.) * Apply reviews (It's a fix commit that has already been fixed in the past.) * fix some bugs and apply formatting, for a merged branch zemyblue/apply_v0.33.4 * fix: blockchain/v1/peer_test failure (The changes in v0.34 take precedence.) * fix: consensus timed out failure * fix: diable proto-checking of circle-ci * fix: raise resource class to pass test suits * fix: restore test code * Revert "fix: restore test code" This reverts commit dec47e5. * refactor: rename VotingPower to StakingPower * fix: separate StakingPower and VotingPower * feat: implement RandomSamplingWithoutReplacement * fix: lint error * feat: implement assigning voting power * fix: lint error * fix: lint error * fix: lite2 test failure * fix: proto generated file * fix: diable proto-checking of circle-ci * fix: apply comment; use VotingPower on adding vote * fix: apply comment; remove totalStakingPower from VoterSet * fix: apply comment; fix NewVoterSet * fix: apply comment; rename validatorSet to voterSet and fix compile errors * fix: apply comment; use VotingPower on consensus * fix: lint error * fix: lint error * fix: lite test compile error * fix: remove unused function * fix: modify validator to voter in comments * fix: total voting power overflow * fix: update total voting power if 0 * docs: change log * fix: apply comments * fix: lint error * fix: rewrite randomThreshold; remove priorityRateThreshold; some test cases * fix: lint error * test: add test for randomThreshold * test: add testing for verifying idempotence of randomThreshold * fix: lint error * fix: improve voting power polacy * fix: compile error * fix: lint error * fix: test case * test: add comment * fix: remove unused function * fix: define MaxTotalVotingPower * fix: remove useless test case, and leave todo * fix some bugs and apply formatting, for a merged branch feature/voting_power * Add `CalNumOfVoterToElect` function to calculate the number of voter to elect * re-add removed libraries in the before commit. * Modify don't using floating-point to calculate `CalNumOfVoterToElect`. * Add `CalNumOfVoterToElect` function to calculate the number of voter to elect * re-add removed libraries in the before commit. * feat: add voter params to genesis doc * feat: add voter params to state * fix: add go.sum * test: add TestCalVotersNum2 * docs: change log * fix: test failure * test: add consensus test with voter election * test: refine the test more precisely * fix: apply comments * fix: lint error, test failure * fix: consensus logic, race error * fix: rename a parameter; modify limit value * fix: add comment, refactoring test code * fix: rename AccuracyPrecision to ElectionPrecision * fix: apply comment; debugging log conflict * fix some fails and apply formatting, for a merged branch feature/num_of_voters * Update changelog of v0.33.4-0.2 * Change LINE Core version to 0.2 * fix golang lint warning * introduce BLS12-381 signature key (build incompleted) * introduce BLS12-381 signature key * Fix the floating-point problem of sampling * fix integer overflow problem of `winPoint`. * fix lint warning * fix lint warning * add `GenerateVRFProof` function for `RetrySignerClient` * fix lint warning. * Remove `CGO_ENABLED` in Makefile. - BLS library always need the CGO_ENABLED=1 option * fix contract_test error in circleCI * fix contract_test error in circleCI * fix contract_test error in circleCI * types: return an error if voting power overflows in VerifyCommitTrusting - apply missing codes of tendermint/tendermint#4896 * Add `VoterSet` protobuf and set to State message * Add `VoterParams` protobuf and set to State message * Fix type error * Apply the PR review feedback - remove `LastValidators` - add `LastProofHash` * feat: remove NextVoters from state * fix: modify LoadVoters/SaveVoters logic * fix: disable skipping verification * test: add test case for load/save voters * test: apply voter sampling to lite verification * test: add TestVerifyAdjacentHeadersWithVoterSampling * fix: fmt errors * fix: dredd error * fix: evidence test failure * fix: lint error * fix: test failure * fix: modify private function name * fix: apply goimports * fix: apply comments * fix: lint error * fix: apply comment * optimize imports * add voting power in `VotingInfo` of abci * update changelog. * modify from `VoterInfo.power` to `VoterInfo.voting_power` for distinguishing * test: add test case; non-voter votes * test: verifying voter sampling under validator set changed * fix: add comments * test: add test case load/save voters * fix: lint error * fix: race problem * fix: test failure * fix: minus voting power * fix: lint error * fix: remove useless test code * fix: apply comment; add test case of max total staking * add comment * fix: test failure * fix: use voting power in MedianTime * Update changelogs and the version about 0.33.5-0.3 * introduced a composite-key that delegates processing to each key-function * fix the points made in the review * add public-key type in sending ValidatorUpdate Tx (there is little difference because alternate way to get the type of public key has been implemented) * Ready for demo (#106) * Add a script to generate new validator and add new validator to the chain * Fix: `add_validator.sh` script error * Add `Validators` in metric of consensus. * Add the setting of `addr_book_strict` of config change to false * Add `ValidatorsPower` in consensus metrics * fix: set consensus_voter_power to 0 if not elected as voter * fix: citest failure * fix: nil pointer reference * fix: set voter power metric with label * fix: check nil * fix: race * feat: add an option for selecting priv key type * fix: test cases * fix: golang ci error * fix: apply priv-key-type option to testnet command * fix: bls compile problem * fix: contract-tests failure * fix: modify change log pending * fix: typo * feat: Make voting satisfy finality * fix: Make voter sorting working well * Move sampling logic to types package * Change the voter that don't use winpoint to validator * Change condition to sort voters * fix: Fix error during test run * feat: log execution time * feat: add consensus duration metric * fix: end time overwrite bug * feat: modify metric type to histogram * fix: duration max float * feat: add proposal creating metric and missing proposal metric * fix: time duration error * fix: change log * fix: fmt error * fix: rename a configure * test: Write a test case to verify a new voter sampling * test: Write a test case to verify a new voter sampling * Change a condition when determining if is voters byzantine change a staking power to voting power in condition to determine if is voters byzantine change a type of winpoint to big int. beacause, it is using for sampling only * test: Use a seed to generate random value in test * fix: overflow, integer division truncated, more test cases * fix: fmt error * feat: add test sample, verifying sort order * fix: priv test failure * fix: apply comment; log seed for random * feat: async reactor receiving * fix: add VotingPower to abci.Evidence * ci: remove `add-path` (#5674) * fix: github action failure * fix: github action failure * fix: skip test cases that require skipping mode in verification, and a few FAIL cases * fix: calculate MaxDataBytes accurately by pv key type * fix: lint error * fix: apply review comments * fix: merge conflict * fix: golangci error * fix: golangci error * fix: golangci error * change Evidence size calculation to be based on Signature size, and remove PubKey from Evidence * added signature aggregation and verify to block * fixed to restore aggregate signature when restoring Commit to VoteSet * fixed a trivial typo * fix: rollback needless modification * fix: Changed to ubuntu20.04 due to end of support for ubuntu19.0 * fix: typo about variable names that select the type of private key * codecov: Restore a codecov job in circleci * codecov: Add a codecov token in config * codecov: Report all of modules to codecov * codecov: Allow a undercoverd source can pass CI * test: don't use foo-bar.net in TestHTTPClientMakeHTTPDialer (#5997) This test relied on connecting to the external site `foo-bar.net`, and (predictably) the site went down and broke all of our CI runs. This changes it to use local HTTP servers instead. (cherry picked from commit f54f80b) * ci: increase the test_cover timeout * apply 0.33.8 * change golang version to 1.15 * fix: A bug fix * feat: add an option for selecting priv key type to `unsafe_reset_all`, `unsafe_reset_priv_validator`, and `gen_validator` * fix: use the proposer address selected by VRF when creating new proposal block * revert: it's correct to create the Proposal Block iff privValidator is Proposer * Merge pull request #190 from line/fix/set_proposer_selected_in_vrf_to_block Add a test case to confirm that the selected Proposer matches the VRF-selected one * test: add tests for each validator type (#189) * test: add test GenFilePV() with the addition of new privKeyType * test: add aggregate signature test * use fmt.Sprint for string to int conversion * bump version and update changelog * privval: increase read/write timeout to 5s and calculate ping interval based on it (#5638) Partially closes #5550 * consensus: only call privValidator.GetPubKey once per block (#5143) Closes #4865 * only retrieve pubkey once for all validators (partially fixes #4865) (#4895) in consensus/state.go, when calulating metrics, retrieve address (ergo, pubkey) once prior to iterating over validatorset to ensure we do not make excessive calls to signer. Partially closes: #4865 * fix: replace privValidatorPubKey on the same time of privValidator to ensure the state become correct * remove redundant section mistakenly added in the merge * remove redundant empty line * fix: Address to the problem that the signature of bls voter cannot be verified (#202) * fix: Address to the problem that the signature of bls voter cannot be verified * fix: fix error handling * fix: fix error handling omission * fix: Specify the capacity of the slice `blsPubKeys` and `msgs` in advance * feat: Modify MakeCommit and AggregateSignature to one operation at once (#210) * feat: Modify `MakeCommit` and `AggregateSignature` to atomic operation * fix: fix panic error message with `MakeCommit` and add this test * fix: Correct variable name * fix: fix flag location * refactor: move `isEqualVoteWithoutSignature` to vote_test.go and separate `MakeCommit` panic test * fix: add assert.Fail and log when ed25519 only * chore: remove hard coded codecov token (#220) * fix: Specified RecvBufSize for pex_reactor_test (91e1df7) * remove Skip() of test cases that are currently successful * add a TODO comment after investigating the fail of TestWALCrash since it was still present in Tendermint Co-authored-by: zemyblue <zemyblue@gmail.com> Co-authored-by: Woosang Son <woosang.son@linecorp.com> Co-authored-by: hongsup.so <hongsup.so@linecorp.com> Co-authored-by: shiki.takahashi <shiki.takahashi@linecorp.com> Co-authored-by: kukugi <wonkuk_seo@linecorp.com> Co-authored-by: egonspace <kumdory@hanmail.net> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: mariko <frogandfrog009@gmail.com> Co-authored-by: Erik Grinaker <erik@interchain.berlin> Co-authored-by: Sangyeop.lee <sangyeop.lee@linecorp.com> Co-authored-by: kokeshiM0chi <52264064+kokeshiM0chi@users.noreply.github.com> Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> Co-authored-by: Joe Bowman <joe@chorus.one> Co-authored-by: tnasu <toshimasa_nasu@hotmail.com>
Closes: #33
Description
Implement election of validators based on VRF
ValidatorSettoValidatorSetandVoterSetA diagram for understanding
TODO in next PRs
MaxVoters < nValidatorsFor contributor use:
docs/) and code commentsFiles changedin the Github PR explorer