abci: Vote Extension 1#6646
Conversation
|
This pull request introduces 3 alerts when merging 546b7dc into 0cb0dd7 - view on LGTM.com new alerts:
|
|
Signing and encoding tests are done. Rest will be done at a separate PR. |
…atech/tendermint into mconcat/abci-vote-extension
…ndermint into mconcat/abci-vote-extension
types/block.go
Outdated
| ValidatorAddress Address `json:"validator_address"` | ||
| Timestamp time.Time `json:"timestamp"` | ||
| Signature []byte `json:"signature"` | ||
| VoteExtension VoteExtensionSigned `json:"vote_extension"` |
There was a problem hiding this comment.
off topic question: how would this work with aggregation of signatures?
There was a problem hiding this comment.
Signature aggregation for consensus votes should be managed by Tendermint.
If BLS was being used, and your state machine wanted VoteExtensions, then you would have to pay in the cost of verifying BLS signatures across many distinct messages. If VoteExtensionSigned was left blank, then all messages would still be the same.
Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
…o mconcat/abci-vote-extension
| } | ||
|
|
||
| func (cli *grpcClient) PrepareProposalAsync( | ||
| // NOTE: call is synchronous, use ctx to break early if needed |
There was a problem hiding this comment.
| // NOTE: call is synchronous, use ctx to break early if needed | |
| // ExtendVoteAsync ... |
Also, if it's async then why is is synchronous?
There was a problem hiding this comment.
I honestly don't know why this comments says "call is synchronous", I just copied it from other methods.
| ) | ||
| } | ||
|
|
||
| // NOTE: call is synchronous, use ctx to break early if needed |
There was a problem hiding this comment.
| // NOTE: call is synchronous, use ctx to break early if needed | |
| // VerifyVoteExtensionAsync ... |
Same comment as above.
proto/tendermint/abci/types.proto
Outdated
| } | ||
|
|
||
| message ResponseVerifyVoteExtension { | ||
| // XXX |
There was a problem hiding this comment.
Maybe this was for flagging the fields for review.
Do we anticipate the data field to ever be used, or is it just included for consistency with ResponseDeliverTx & ResponseCheckTx?
To be honest, I don't understand what the data or info fields are used for in other responses either. I have preference for leaving these four fields, and then in a later pass of ABCI++ assess the utility of data and info everywhere.
There was a problem hiding this comment.
My mistake, I've changed the response fields to enum Result in the second PR and not merged back. I will apply it here.
There was a problem hiding this comment.
I introduced these first for the consistency with the Response*Tx, but I have found that the recent methods have introduced simple enum style result type, and I thought it would be better for our case. The voteext result enum has three result types, each called ACCEPT, REJECT, ABORT, where ACCEPT and RESULT act as validation/invalidation for a vote extension, and ABORT acts as a malicious vote extension(such as wrong oracle price data). I think handling on those ABORT type could be done inside the state machine itself, without returning any additional information to Tendermint side.
(Please suggest a better name for ABORT if you have any)
| // that is additionally included in the vote | ||
| type VoteExtension struct { | ||
| AppDataToSign []byte `json:"app_data_to_sign"` | ||
| AppDataSelfAuthenticating []byte `json:"app_data_self_authenticating"` |
There was a problem hiding this comment.
I'm still not quite clear on what this field is or what it is to be used for? Is it a signature or something?
There was a problem hiding this comment.
Both AppData* are custom data returned by the app, but only the AppDataToSign is actually get signed by the validator. AppDataSelfAuthenticating could be used for aggregated signatures(or other signature extensions).
There was a problem hiding this comment.
For threshold decryption, AppDataSelfAuthenticating is going to be each validators decryption shares for txs in the block it is precommitting over.
These should not be signed over, as they will not get included in the final block data.
Every node can check this self-authenticating data against the validators public key shares in the state machine, and be convinced of the decryption share validity.
The next block proposer then does not include these in the final block, it only includes a "combined" decryption share, that every full node can check, and then decrypt the txs with.
| ResponseExtendVote extend_vote = 17; | ||
| ResponseVerifyVoteExtension verify_vote_extension = 18; |
There was a problem hiding this comment.
Does the formatter have to be re-ran to make the equals signs align, or is this fine per golint?
There was a problem hiding this comment.
the developer needs to install some extra tooling. We can leave as is and fix the little things like this later on
|
@mconcat there are some cosmetic changes requested, could you apply them, this way we can merge this. |
Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
…ndermint into mconcat/abci-vote-extension
* add proto, add boilerplates * add canonical * fix tests * add vote signing test * Update internal/consensus/msgs_test.go * modify state execution in progress * add extension signing * VoteExtension -> ExtendVote * apply review * update data structures * Add comments * Apply suggestions from code review Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * *Signed -> *ToSign * add Vote to RequestExtendVote * apply reviews * Apply suggestions from code review Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * fix typo, modify proto Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
* add proto, add boilerplates * add canonical * fix tests * add vote signing test * Update internal/consensus/msgs_test.go * modify state execution in progress * add extension signing * VoteExtension -> ExtendVote * apply review * update data structures * Add comments * Apply suggestions from code review Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * *Signed -> *ToSign * add Vote to RequestExtendVote * apply reviews * Apply suggestions from code review Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * fix typo, modify proto Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
* add proto, add boilerplates * add canonical * fix tests * add vote signing test * Update internal/consensus/msgs_test.go * modify state execution in progress * add extension signing * VoteExtension -> ExtendVote * apply review * update data structures * Add comments * Apply suggestions from code review Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * *Signed -> *ToSign * add Vote to RequestExtendVote * apply reviews * Apply suggestions from code review Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * fix typo, modify proto Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
* add proto, add boilerplates * add canonical * fix tests * add vote signing test * Update internal/consensus/msgs_test.go * modify state execution in progress * add extension signing * VoteExtension -> ExtendVote * apply review * update data structures * Add comments * Apply suggestions from code review Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * *Signed -> *ToSign * add Vote to RequestExtendVote * apply reviews * Apply suggestions from code review Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * fix typo, modify proto Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
* add proto, add boilerplates * add canonical * fix tests * add vote signing test * Update internal/consensus/msgs_test.go * modify state execution in progress * add extension signing * VoteExtension -> ExtendVote * apply review * update data structures * Add comments * Apply suggestions from code review Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * *Signed -> *ToSign * add Vote to RequestExtendVote * apply reviews * Apply suggestions from code review Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * fix typo, modify proto Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
* add proto, add boilerplates * add canonical * fix tests * add vote signing test * Update internal/consensus/msgs_test.go * modify state execution in progress * add extension signing * VoteExtension -> ExtendVote * apply review * update data structures * Add comments * Apply suggestions from code review Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * *Signed -> *ToSign * add Vote to RequestExtendVote * apply reviews * Apply suggestions from code review Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * fix typo, modify proto Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com>
* [cherry-picked] abci: Vote Extension 1 (#6646) * add proto, add boilerplates * add canonical * fix tests * add vote signing test * Update internal/consensus/msgs_test.go * modify state execution in progress * add extension signing * VoteExtension -> ExtendVote * apply review * update data structures * Add comments * Apply suggestions from code review Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * *Signed -> *ToSign * add Vote to RequestExtendVote * apply reviews * Apply suggestions from code review Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * fix typo, modify proto Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * [cherry-picked] ABCI Vote Extension 2 (#6885) * add proto, add boilerplates * add canonical * fix tests * add vote signing test * Update internal/consensus/msgs_test.go * modify state execution in progress * add extension signing * add extension signing * VoteExtension -> ExtendVote * modify state execution in progress * add extension signing * verify in progress * modify CommitSig * fix test * apply review * update data structures * Apply suggestions from code review * Add comments * fix test * VoteExtensionSigned => VoteExtensionToSigned * Apply suggestions from code review Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * *Signed -> *ToSign * add Vote to RequestExtendVote * add example VoteExtension * apply reviews * fix vote * Apply suggestions from code review Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * fix typo, modify proto * add abcipp_kvstore.go * add extension test * fix test * fix test * fix test * fit lint * uncomment test * refactor test in progress * gofmt * apply review * fix lint Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> * [cheryy-picked] abci: PrepareProposal-VoteExtension integration [2nd try] (#7821) * PrepareProposal-VoteExtension integration (#6915) * make proto-gen * Fix protobuf crash in e2e nightly tests * Update types/vote.go Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Addressed @creachadair's comments Co-authored-by: mconcat <monoidconcat@gmail.com> Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * [cherry-picked] Vote extensions: new design (#8031) * Changed the spec text to agreed VoteExtension solution * Revert "Removed protobufs related to vote extensions" This reverts commit 4566f1e. * Changes to ABCI protocol buffers * Update spec/core/data_structures.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update spec/core/data_structures.md Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Fix dangling link in ABCI++ readme * Addressed comments Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * [cherry-picked] abci++: Sync implementation and spec for vote extensions (#8141) * Refactor so building and linting works This is the first step towards implementing vote extensions: generating the relevant proto stubs and getting the build and linter to pass. Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fix typo Signed-off-by: Thane Thomson <connect@thanethomson.com> * Better describe method given vote extensions Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fix types tests Signed-off-by: Thane Thomson <connect@thanethomson.com> * Move CanonicalVoteExtension to canonical types proto defs Signed-off-by: Thane Thomson <connect@thanethomson.com> * Regenerate protos including latest PBTS synchrony params update Signed-off-by: Thane Thomson <connect@thanethomson.com> * Inject vote extensions into proposal Signed-off-by: Thane Thomson <connect@thanethomson.com> * Thread vote extensions through code and fix tests Signed-off-by: Thane Thomson <connect@thanethomson.com> * Remove extraneous empty value initialization Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fix lint Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fix missing VerifyVoteExtension request data Signed-off-by: Thane Thomson <connect@thanethomson.com> * Explicitly ensure length > 0 to sign vote extension Signed-off-by: Thane Thomson <connect@thanethomson.com> * Explicitly ensure length > 0 to sign vote extension Signed-off-by: Thane Thomson <connect@thanethomson.com> * Remove extraneous comment Signed-off-by: Thane Thomson <connect@thanethomson.com> * Update privval/file.go Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Update types/vote_test.go Co-authored-by: M. J. Fromberger <fromberger@interchain.io> * Format Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fix ABCI proto generation scripts for Linux Signed-off-by: Thane Thomson <connect@thanethomson.com> * Sync intermediate and goal protos Signed-off-by: Thane Thomson <connect@thanethomson.com> * Update internal/consensus/common_test.go Co-authored-by: Sergio Mena <sergio@informal.systems> * Use dummy value with clearer meaning Signed-off-by: Thane Thomson <connect@thanethomson.com> * Rewrite loop for clarity Signed-off-by: Thane Thomson <connect@thanethomson.com> * Panic on ABCI++ method call failure Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add strong correctness guarantees when constructing extended commit info for ABCI++ Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add strong guarantee in extendedCommitInfo that the number of votes corresponds Signed-off-by: Thane Thomson <connect@thanethomson.com> * Make extendedCommitInfo function more robust At first extendedCommitInfo expected votes to be in the same order as their corresponding validators in the supplied CommitInfo struct, but this proved to be rather difficult since when a validator set's loaded from state it's first sorted by voting power and then by address. Instead of sorting the votes in the same way, this approach simply maps votes to their corresponding validator's address prior to constructing the extended commit info. This way it's easy to look up the corresponding vote and we don't need to care about vote order. Signed-off-by: Thane Thomson <connect@thanethomson.com> * Remove extraneous validator address assignment Signed-off-by: Thane Thomson <connect@thanethomson.com> * Sign over canonical vote extension Signed-off-by: Thane Thomson <connect@thanethomson.com> * Validate vote extension signature against canonical vote extension Signed-off-by: Thane Thomson <connect@thanethomson.com> * Update privval tests for more meaningful dummy value Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add vote extension capability to E2E test app Signed-off-by: Thane Thomson <connect@thanethomson.com> * Disable lint for weak RNG usage for test app Signed-off-by: Thane Thomson <connect@thanethomson.com> * Use parseVoteExtension instead of custom parsing in PrepareProposal Signed-off-by: Thane Thomson <connect@thanethomson.com> * Only include extension if we have received txs It's unclear at this point why this is necessary to ensure that the application's local app_hash matches that committed in the previous block. Signed-off-by: Thane Thomson <connect@thanethomson.com> * Require app_hash from app to match that from last block Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add contrived (possibly flaky) test to check that vote extensions code works Signed-off-by: Thane Thomson <connect@thanethomson.com> * Remove workaround for problem now solved by #8229 Signed-off-by: Thane Thomson <connect@thanethomson.com> * add tests for vote extension cases * Fix spelling mistake to appease linter Signed-off-by: Thane Thomson <connect@thanethomson.com> * Collapse redundant if statement Signed-off-by: Thane Thomson <connect@thanethomson.com> * Formatting Signed-off-by: Thane Thomson <connect@thanethomson.com> * Always expect an extension signature, regardless of whether an extension is present Signed-off-by: Thane Thomson <connect@thanethomson.com> * Votes constructed from commits cannot include extensions or signatures Signed-off-by: Thane Thomson <connect@thanethomson.com> * Pass through vote extension in test helpers Signed-off-by: Thane Thomson <connect@thanethomson.com> * Temporarily disable vote extension signature requirement Signed-off-by: Thane Thomson <connect@thanethomson.com> * Expand on vote equality test errors for clarity Signed-off-by: Thane Thomson <connect@thanethomson.com> * Expand on vote matching error messages in testing Signed-off-by: Thane Thomson <connect@thanethomson.com> * Allow for selective subscription by vote type This is an attempt to fix the intermittently failing `TestPrepareProposalReceivesVoteExtensions` test in the internal consensus package. Occasionally we get prevote messages via the subscription channel, and we're not interested in those. This change allows us to specify what types of votes we're interested in (i.e. precommits) and discard the rest. Signed-off-by: Thane Thomson <connect@thanethomson.com> * Read lock consensus state mutex in test helper to avoid data race Signed-off-by: Thane Thomson <connect@thanethomson.com> * Revert BlockIDFlag parameter in node test Signed-off-by: Thane Thomson <connect@thanethomson.com> * Perform additional check in ProcessProposal for special txs generated by vote extensions Signed-off-by: Thane Thomson <connect@thanethomson.com> * e2e: check that our added tx does not cause all txs to exceed req.MaxTxBytes Signed-off-by: Thane Thomson <connect@thanethomson.com> * Only set vote extension signatures when signing is successful Signed-off-by: Thane Thomson <connect@thanethomson.com> * Remove channel capacity constraint in test helper to avoid missing messages Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add TODO to always require extension signatures in vote validation Signed-off-by: Thane Thomson <connect@thanethomson.com> * e2e: reject vote extensions if the request height does not match what we expect Signed-off-by: Thane Thomson <connect@thanethomson.com> * types: remove extraneous call to voteWithoutExtension in test Signed-off-by: Thane Thomson <connect@thanethomson.com> * Remove unnecessary address parameter from CanonicalVoteExtension Signed-off-by: Thane Thomson <connect@thanethomson.com> * privval: change test vote type to precommit since we use an extension Signed-off-by: Thane Thomson <connect@thanethomson.com> * privval: update signing logic to cater for vote extensions Signed-off-by: Thane Thomson <connect@thanethomson.com> * proto: update field descriptions for vote message Signed-off-by: Thane Thomson <connect@thanethomson.com> * proto: update field description for vote extension sig in vote message Signed-off-by: Thane Thomson <connect@thanethomson.com> * proto/types: use fixed-length 64-bit integers for rounds in CanonicalVoteExtension Signed-off-by: Thane Thomson <connect@thanethomson.com> * consensus: fix flaky TestPrepareProposalReceivesVoteExtensions Signed-off-by: Thane Thomson <connect@thanethomson.com> * consensus: remove previously added test helper functionality Signed-off-by: Thane Thomson <connect@thanethomson.com> * e2e: add error logs when we get an unexpected height in ExtendVote or VerifyVoteExtension requests Signed-off-by: Thane Thomson <connect@thanethomson.com> * node_test: get validator addresses from privvals Signed-off-by: Thane Thomson <connect@thanethomson.com> * privval/file_test: optimize filepv creation in tests Signed-off-by: Thane Thomson <connect@thanethomson.com> * privval: add test to check that vote extensions are always signed Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add a script to check documentation for ToC entries. (#8356) This script verifies that each document in the docs and architecture directory has a corresponding table-of-contents entry in its README file. It can be run manually from the command line. - Hook up this script to run in CI (optional workflow). - Update ADR ToC to include missing entries this script found. * build(deps): Bump async from 2.6.3 to 2.6.4 in /docs (#8357) Bumps [async](https://github.com/caolan/async) from 2.6.3 to 2.6.4. - [Release notes](https://github.com/caolan/async/releases) - [Changelog](https://github.com/caolan/async/blob/v2.6.4/CHANGELOG.md) - [Commits](caolan/async@v2.6.3...v2.6.4) --- updated-dependencies: - dependency-name: async dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * privval/file_test: reset vote ext sig before signing Signed-off-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: M. J. Fromberger <fromberger@interchain.io> Co-authored-by: Sergio Mena <sergio@informal.systems> Co-authored-by: William Banfield <wbanfield@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix cherry-picks * make proto-gen * make mockery * fix build * All units tests passing * linter error * Update consensus/state_test.go Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com> * Addressed @williambanfield's comments * Go, not C! Co-authored-by: mconcat <monoidconcat@gmail.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Dev Ojha <ValarDragon@users.noreply.github.com> Co-authored-by: M. J. Fromberger <fromberger@interchain.io> Co-authored-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: William Banfield <wbanfield@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Vote extension proto struct definitions.
This PR implements mostly the boilerplate of adding ABCI methods, and updating the vote proto definitions for vote extensions. This is done according to the flow of #{above issue number}
Namely it adds the ABCI methods for VerifyVoteExtension and ExtendVote. It also contains tests to ensure that the relevant datum, if added, get gossiped around correctly.