Skip to content

test/e2e: add random testnet generator#5479

Merged
mergify[bot] merged 4 commits intomasterfrom
erik/e2e-generator
Oct 9, 2020
Merged

test/e2e: add random testnet generator#5479
mergify[bot] merged 4 commits intomasterfrom
erik/e2e-generator

Conversation

@erikgrinaker
Copy link
Contributor

Closes #5291. Adds a randomized testnet generator. Nightly CI job will be submitted separately. A few of the testnets can be a bit flaky, even after disabling known-faulty behavior and making minor tweaks, and the larger networks may be too resource-intensive to run in CI - this will be optimized separately.

@erikgrinaker erikgrinaker added the T:test Type: Tests that need love label Oct 8, 2020
@erikgrinaker erikgrinaker self-assigned this Oct 8, 2020
@codecov
Copy link

codecov bot commented Oct 8, 2020

Codecov Report

Merging #5479 into master will decrease coverage by 0.23%.
The diff coverage is 7.69%.

@@            Coverage Diff             @@
##           master    #5479      +/-   ##
==========================================
- Coverage   61.32%   61.08%   -0.24%     
==========================================
  Files         259      262       +3     
  Lines       23461    23630     +169     
==========================================
+ Hits        14388    14435      +47     
- Misses       7606     7732     +126     
+ Partials     1467     1463       -4     
Impacted Files Coverage Δ
test/e2e/generator/generate.go 0.00% <0.00%> (ø)
test/e2e/generator/main.go 0.00% <0.00%> (ø)
test/e2e/generator/random.go 39.39% <39.39%> (ø)
crypto/sr25519/pubkey.go 43.47% <0.00%> (-8.70%) ⬇️
blockchain/v0/reactor.go 60.59% <0.00%> (-0.99%) ⬇️
mempool/reactor.go 84.09% <0.00%> (-0.76%) ⬇️
privval/signer_endpoint.go 75.75% <0.00%> (ø)
... and 7 more

Copy link
Contributor

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK.

For the larger networks we could look into the possibility of spreading them across machines/jobs. Not sure how github handles the p2p aspect of it, we will need to read into this

@erikgrinaker
Copy link
Contributor Author

For the larger networks we could look into the possibility of spreading them across machines/jobs.

Nah, the runner only supports a single Docker daemon, so it can only run on a single machine. Implementing additional runners (e.g. Kubernetes, VMs, etc) is explicitly out-of-scope for this iteration, since it gets real complex real fast.

@erikgrinaker erikgrinaker added the S:automerge Automatically merge PR when requirements pass label Oct 9, 2020
@mergify mergify bot merged commit 260cc5d into master Oct 9, 2020
@mergify mergify bot deleted the erik/e2e-generator branch October 9, 2020 12:33
erikgrinaker added a commit that referenced this pull request Oct 20, 2020
Closes #5291. Adds a randomized testnet generator. Nightly CI job will be submitted separately. A few of the testnets can be a bit flaky, even after disabling known-faulty behavior and making minor tweaks, and the larger networks may be too resource-intensive to run in CI - this will be optimized separately.
erikgrinaker added a commit that referenced this pull request Oct 22, 2020
Closes #5291. Adds a randomized testnet generator. Nightly CI job will be submitted separately. A few of the testnets can be a bit flaky, even after disabling known-faulty behavior and making minor tweaks, and the larger networks may be too resource-intensive to run in CI - this will be optimized separately.
erikgrinaker added a commit that referenced this pull request Oct 22, 2020
Closes #5291. Adds a randomized testnet generator. Nightly CI job will be submitted separately. A few of the testnets can be a bit flaky, even after disabling known-faulty behavior and making minor tweaks, and the larger networks may be too resource-intensive to run in CI - this will be optimized separately.
erikgrinaker added a commit that referenced this pull request Oct 22, 2020
Closes #5291. Adds a randomized testnet generator. Nightly CI job will be submitted separately. A few of the testnets can be a bit flaky, even after disabling known-faulty behavior and making minor tweaks, and the larger networks may be too resource-intensive to run in CI - this will be optimized separately.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S:automerge Automatically merge PR when requirements pass T:test Type: Tests that need love

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: new end-to-end test suite

2 participants