rpc: add chunked rpc interface by tychoish · Pull Request #6445 · tendermint/tendermint

tychoish · 2021-05-10T22:08:35Z

I think there may be additional testing that would be useful, and I
think we should also update documentation and specs, maybe, but I'm
not sure the full scope or ordering of that work relative to this PR.

Resolves #6354

codecov · 2021-05-10T22:14:59Z

Codecov Report

Merging #6445 (5d9d10b) into master (0781ca3) will decrease coverage by 0.20%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #6445      +/-   ##
==========================================
- Coverage   60.91%   60.71%   -0.21%     
==========================================
  Files         288      289       +1     
  Lines       27070    27156      +86     
==========================================
- Hits        16490    16488       -2     
- Misses       8888     8962      +74     
- Partials     1692     1706      +14

Impacted Files	Coverage Δ
light/rpc/client.go	`9.70% <0.00%> (-0.04%)`	⬇️
node/node.go	`49.76% <0.00%> (-0.16%)`	⬇️
rpc/core/env.go	`47.05% <0.00%> (-14.48%)`	⬇️
rpc/core/net.go	`39.34% <0.00%> (-11.72%)`	⬇️
rpc/core/routes.go	`0.00% <0.00%> (ø)`
rpc/core/types/responses.go	`33.33% <ø> (ø)`
privval/signer_listener_endpoint.go	`80.00% <0.00%> (-9.42%)`	⬇️
types/tx.go	`82.97% <0.00%> (-4.26%)`	⬇️
privval/socket_listeners.go	`78.72% <0.00%> (-4.26%)`	⬇️
privval/signer_endpoint.go	`75.75% <0.00%> (-3.04%)`	⬇️
... and 15 more

cmwaters

This all looks good to me. I think the first thing you should do is open a PR in the spec repo looking to add this rpc endpoint. Then we can merge this one / add more tests if you feel like some edge cases are missing. After that we can update the docs - which I think is mainly updating the swagger.yml file.

light/proxy/routes.go

cmwaters · 2021-05-11T09:25:27Z

rpc/client/rpc_test.go

+		require.NoError(t, err)
+
+		decoded := make([]string, 0, first.TotalChunks)
+		for i := first.ChunkNumber; i < first.TotalChunks+1; i++ {


How big is this test genesis file? Does it even loop more than once?

yeah we don't loop in the test. On the plus side we don't actually handle the test case differently.

Do we want to create a test where there are more than one chunk?

cmwaters · 2021-05-11T09:27:28Z

rpc/client/rpc_test.go

+		doc := []byte(strings.Join(decoded, ""))
+
+		var out types.GenesisDoc
+		require.NoError(t, tmjson.Unmarshal(doc, &out),


Being able to unmarshal basically ensures that the chunked genesis is the same as the one before right? Although technically they could be different

yeah, ensures that that we didn't break the JSON encoding.

alexanderbez

Looks great @tychoish. I just a few comments 👍

rpc/core/env.go

rpc/core/types/responses.go

tychoish · 2021-05-11T15:41:21Z

@cmwaters (et al) I added this to the spec: tendermint/spec#299

rpc/core/env.go

rpc/core/net.go

tac0turtle

utACK.

Could you add a changelog and upgrading.md entry.

alexanderbez · 2021-05-24T12:11:01Z

@tychoish what's stopping this from being merged?

tac0turtle · 2021-07-14T07:50:13Z

@Mergifyio backport release/v0.34.x

mergify · 2021-07-14T07:50:49Z

Command backport release/v0.34.x: failure

No backport have been created

Backport to branch release/v0.34.x failed: Branch not found

tac0turtle · 2021-07-14T07:51:41Z

@Mergifyio backport v0.34.x

(cherry picked from commit d913406) # Conflicts: # light/proxy/routes.go # node/node.go # rpc/core/net.go # rpc/core/routes.go

mergify · 2021-07-14T07:52:47Z

Command backport v0.34.x: success

Backports have been created

#6717 rpc: add chunked rpc interface (backport #6445) has been created for branch v0.34.x

* rpc: add chunked rpc interface (#6445) (cherry picked from commit d913406) # Conflicts: # light/proxy/routes.go # node/node.go # rpc/core/net.go # rpc/core/routes.go * fix conflicts Co-authored-by: Sam Kleinman <garen@tychoish.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com>

cmwaters · 2021-07-14T09:28:55Z

I don't see the /genesis_chunked endpoint in the RPC documentation (https://docs.tendermint.com/master/rpc/). Is there something we're missing?

tac0turtle · 2021-07-14T09:30:58Z

nope, it wasn't added in this PR.

will make an issue

* rpc: add chunked rpc interface (#6445) (cherry picked from commit d913406) * fix conflicts Co-authored-by: Sam Kleinman <garen@tychoish.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> (cherry picked from commit da9eefd)

tychoish added 2 commits May 10, 2021 17:36

rpc: add chunked rpc interface

21d2801

add tests

38dd809

tychoish requested review from alexanderbez, cmwaters, ebuchman and tessr as code owners May 10, 2021 22:08

Merge remote-tracking branch 'origin/master' into rpc-genesis-chunks

e0a414d

cmwaters reviewed May 11, 2021

View reviewed changes

alexanderbez approved these changes May 11, 2021

View reviewed changes

rpc/core/env.go Outdated Show resolved Hide resolved

rpc/core/env.go Show resolved Hide resolved

rpc/core/env.go Outdated Show resolved Hide resolved

rpc/core/env.go Outdated Show resolved Hide resolved

rpc/core/types/responses.go Outdated Show resolved Hide resolved

alexanderbez added the C:rpc Component: JSON RPC, gRPC label May 11, 2021

tychoish added 3 commits May 11, 2021 10:57

cr feedback

de518ea

Merge remote-tracking branch 'origin/master' into rpc-genesis-chunks

cb81445

fix lint

1583510

tychoish requested a review from cmwaters May 11, 2021 15:11

tychoish mentioned this pull request May 11, 2021

rpc: add chunked genesis endpoint tendermint/spec#299

Merged

cmwaters approved these changes May 12, 2021

View reviewed changes

rpc/core/env.go Outdated Show resolved Hide resolved

rpc/core/net.go Outdated Show resolved Hide resolved

tychoish added 5 commits May 12, 2021 11:28

spec feedback: change parameter name and make zero indexed

38fd151

add error to old end point

39e4653

errors and documentation

dd9673b

Merge branch 'master' into rpc-genesis-chunks

6f42a17

fix test

895b9c9

tac0turtle approved these changes May 17, 2021

View reviewed changes

Merge branch 'master' into rpc-genesis-chunks

5d9d10b

tychoish merged commit d913406 into tendermint:master May 24, 2021

cmwaters mentioned this pull request Jul 13, 2021

Large genesis makes node crash #6706

Closed

mergify bot mentioned this pull request Jul 14, 2021

rpc: add chunked rpc interface (backport #6445) #6717

Merged

mergify bot pushed a commit that referenced this pull request Jul 14, 2021

rpc: add chunked rpc interface (#6445)

79083e6

(cherry picked from commit d913406) # Conflicts: # light/proxy/routes.go # node/node.go # rpc/core/net.go # rpc/core/routes.go

cmwaters mentioned this pull request Feb 22, 2022

docs: remove spec section from v0.34 docs #7940

Merged

This was referenced Apr 22, 2022

feature/GRAPH 306 v0.34.13 dm graphprotocol/tendermint#15

Merged

feature/GRAPH 306 v0.34.14 dm graphprotocol/tendermint#16

Merged

williambanfield pushed a commit that referenced this pull request Nov 1, 2022

rpc: add chunked rpc interface (#6445)

4bcbfd6

Conversation

tychoish commented May 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented May 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cmwaters left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cmwaters May 11, 2021

Choose a reason for hiding this comment

Uh oh!

tychoish May 11, 2021

Choose a reason for hiding this comment

Uh oh!

cmwaters May 12, 2021

Choose a reason for hiding this comment

Uh oh!

cmwaters May 11, 2021

Choose a reason for hiding this comment

Uh oh!

tychoish May 11, 2021

Choose a reason for hiding this comment

Uh oh!

alexanderbez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tychoish commented May 11, 2021

Uh oh!

Uh oh!

Uh oh!

tac0turtle left a comment

Choose a reason for hiding this comment

Uh oh!

alexanderbez commented May 24, 2021

Uh oh!

tac0turtle commented Jul 14, 2021

Uh oh!

mergify bot commented Jul 14, 2021

Uh oh!

tac0turtle commented Jul 14, 2021

Uh oh!

mergify bot commented Jul 14, 2021

Uh oh!

cmwaters commented Jul 14, 2021

Uh oh!

tac0turtle commented Jul 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tychoish commented May 10, 2021 •

edited

Loading

codecov bot commented May 10, 2021 •

edited

Loading

tac0turtle commented Jul 14, 2021 •

edited

Loading