Skip to content

privval: migrate to protobuf#4813

Merged
tac0turtle merged 29 commits intoproto-breakagefrom
marko/canonical
May 27, 2020
Merged

privval: migrate to protobuf#4813
tac0turtle merged 29 commits intoproto-breakagefrom
marko/canonical

Conversation

@tac0turtle
Copy link
Contributor

Description

define canonical types in protobuf

Closes: #XXX

@auto-comment
Copy link

auto-comment bot commented May 11, 2020

👋 Thanks for creating a PR!

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

  • Wrote tests
  • Updated CHANGELOG_PENDING.md
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
  • Updated relevant documentation (docs/) and code comments
  • Re-reviewed Files changed in the Github PR explorer
  • Applied Appropriate Labels

Thank you for your contribution to Tendermint! 🚀

@tac0turtle tac0turtle self-assigned this May 11, 2020
@tac0turtle tac0turtle added the T:encoding Type: Amino, ProtoBuf label May 11, 2020
@tac0turtle tac0turtle changed the title types: define canonical types in proto privval: migrate to protobuf May 11, 2020
"type": "tendermint/PrivKeyEd25519",
"value": "%s"
}
"pubKey": {"ed25519":"%s"},
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the preferred way of handling this transition for validators? should we maintain backwards compatibility or have a migration path.

cc @melekes @erikgrinaker @tessr

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious about this too - it has implications for the RPC API as well, and a few other places. If we're going to break this, we might as well do it now since we're breaking everything else with Protobuf, but it's a lot of breakage to deal with.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having given this some thought, I think the new scheme makes more sense (using a key per variant instead of a type field), since it can represent an actual well-defined schema instead of having a random assortment of fields depending on the type field. This also simplifies decoding, since we don't have to do it in two passes (first to get the type field, then to decode the contents). My vote is 👍.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be fine as long as there's an upgrade path

@tac0turtle
Copy link
Contributor Author

lint is failing because of things on master. working on fixing those now

@tac0turtle tac0turtle marked this pull request as ready for review May 20, 2020 13:41
@tac0turtle tac0turtle requested a review from tessr as a code owner May 20, 2020 13:41
// MarshalJSONIndent provides an auxiliary function to return Proto3 indented
// JSON encoded bytes of a message.
func MarshalJSONIndent(msg proto.Message) ([]byte, error) {
jm := &jsonpb.Marshaler{EmitDefaults: false, OrigName: false, Indent: " "}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make this a global variable to avoid recreating it every time we want to encode something?

privval/file.go Outdated
err = cdc.UnmarshalJSON(keyJSONBytes, &pvKey)
if err != nil {
pvK := privvalproto.FilePVKey{}
if err := jsonpb.Unmarshal(strings.NewReader(string(keyJSONBytes)), &pvK); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if err := jsonpb.Unmarshal(strings.NewReader(string(keyJSONBytes)), &pvK); err != nil {
if err := jsonpb.Unmarshal(bytes.NewReader(keyJSONBytes), &pvK); err != nil {

privval/file.go Outdated
}
err = cdc.UnmarshalJSON(stateJSONBytes, &pvState)
if err != nil {
if err := jsonpb.Unmarshal(strings.NewReader(string(stateJSONBytes)), &pvS); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

return
}

const maxRemoteSignerMsgSize = 1024 * 10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}

enum Errors {
ERRORS_UNKNOWN = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drop the ERRORS prefix?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

following the protobuf style guide, this will have to be done else where as well (ABCI).

var err error
vote.Signature, err = val.Key.PrivKey.Sign(vote.SignBytes(chainID))

fmt.Println(vote, vote2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
fmt.Println(vote, vote2)
t.Log(vote, vote2)

_ = voteVal.SignVote("mock-chain-id", &vote)

v := vote.ToProto()
_ = voteVal.SignVote("mock-chain-id", v)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

panic if err

@codecov-commenter
Copy link

codecov-commenter commented May 26, 2020

Codecov Report

Merging #4813 into proto-breakage will decrease coverage by 0.06%.
The diff coverage is 75.00%.

@@                Coverage Diff                 @@
##           proto-breakage    #4813      +/-   ##
==================================================
- Coverage           62.16%   62.09%   -0.07%     
==================================================
  Files                 165      165              
  Lines               15238    15242       +4     
==================================================
- Hits                 9473     9465       -8     
- Misses               4931     4940       +9     
- Partials              834      837       +3     
Impacted Files Coverage Δ
config/toml.go 65.95% <ø> (ø)
crypto/ed25519/ed25519.go 39.21% <ø> (ø)
tools/tm-signer-harness/internal/test_harness.go 62.06% <75.00%> (+0.66%) ⬆️
libs/events/events.go 93.20% <0.00%> (-4.86%) ⬇️
blockchain/v0/reactor.go 59.91% <0.00%> (-0.89%) ⬇️
p2p/pex/addrbook.go 71.89% <0.00%> (-0.88%) ⬇️
p2p/pex/pex_reactor.go 82.22% <0.00%> (-0.54%) ⬇️
blockchain/v0/pool.go 78.66% <0.00%> (+0.31%) ⬆️

Copy link
Contributor

@melekes melekes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@tac0turtle tac0turtle added S:automerge Automatically merge PR when requirements pass and removed S:automerge Automatically merge PR when requirements pass labels May 27, 2020
@tac0turtle tac0turtle merged commit d8bc822 into proto-breakage May 27, 2020
@tac0turtle tac0turtle deleted the marko/canonical branch May 27, 2020 11:10
tac0turtle added a commit that referenced this pull request Jun 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T:encoding Type: Amino, ProtoBuf

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants