e2e: add contexts and e2e generator tests#9426
Conversation
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
| func Benchmark(testnet *e2e.Testnet, benchmarkLength int64) error { | ||
| block, _, err := waitForHeight(testnet, 0) | ||
| func Benchmark(ctx context.Context, testnet *e2e.Testnet, benchmarkLength int64) error { | ||
| block, _, err := waitForHeight(ctx, testnet, 0) |
There was a problem hiding this comment.
Small point here. This is not counted towards the test duration time. If I understand correctly, this will wait for all nodes to report they are at height 0 akka they have started etc.) Does this include other operations that are relevant to measure - creating connections between nodes for example?
There was a problem hiding this comment.
No this just simply waits until all nodes have started before beginning the actual benchmark
sergio-mena
left a comment
There was a problem hiding this comment.
There are some involved chunks I didn't review (yet).
Before reviewing (potentially) already reviewed code, is this PR a result of a purely backport? Or are some things reworked?
If a backport, what PRs were used as source? (or what "git diff" command?)
I'm asking because that's the most efficient way to review a backport.
|
|
||
| ## Conceptual Overview | ||
|
|
||
| End-to-end testnets are used to test Tendermint functionality as a user would use it, by spinning up a set of nodes with various configurations and making sure the nodes and network behave correctly. The background for the E2E test suite is outlined in [RFC-001](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-066-e2e-testing.md). |
There was a problem hiding this comment.
RFC001 seems to refer to storage, not e2e. Am I missing something?
There was a problem hiding this comment.
Yeah I guess this was missed in the earlier PR
|
This PR isn't based on cherry-picked commits. There were probably a handful of PRs. I've just directly looked at the diff between v0.36 and main and copied the relevant items across. This seems to be adding contexts to e2e methods, a test for the generator, more documentation and addition of extra data in the benchmark command |
What was the exact "git diff" you ran? (IOW, which two hashes were you comparing?) |
main and v0.36.x. (do you want the exact hashes?) |
A priori, not necessary, I'll "git diff" using the tip of both branches. If I run into ambiguities, I'll let you know. |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
thanethomson
left a comment
There was a problem hiding this comment.
It seems like all this change does is plumb contexts through the E2E tests?
If so, the issue title and description don't really match up with what's being done here. Are there other changes from v0.36 that will eventually be ported across?
|
Sorry for taking some to time come back to this. |
That's the crux of it. There were some more documenting changes; a test was added to ensure that all generated testnets are valid; modifications to how we wait for nodes to reach a certain height was adjusted (a lot of small stuff)
Yeah, there are still a chunk of changes that can only happen once finalize block and vote extensions have actually been implemented. This PR is supposed to address the "preliminary work" within e2e as outlined by Sergio. There will definitely need to be a second PR later on. I'm fine with renaming it to something more scoped. |
| PrivValServer string `toml:"privval_server"` | ||
| PrivValKey string `toml:"privval_key"` | ||
| PrivValState string `toml:"privval_state"` | ||
| Misbehaviors map[string]string `toml:"misbehaviors"` |
There was a problem hiding this comment.
So this doesn't currently support misbehavior? (Doesn't look like this was used anywhere)
There was a problem hiding this comment.
Yeah I think this was vestige from v0.34 back when the maverick node was around. It's unused.
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
This PR tackles some of the preliminary work in regards to the e2e outlined in #9396