Skip to content

test: add GitHub action for end-to-end tests#5452

Merged
erikgrinaker merged 11 commits intomasterfrom
erik/e2e-ci
Oct 5, 2020
Merged

test: add GitHub action for end-to-end tests#5452
erikgrinaker merged 11 commits intomasterfrom
erik/e2e-ci

Conversation

@erikgrinaker
Copy link
Contributor

@erikgrinaker erikgrinaker commented Oct 3, 2020

Partial fix for #5291.

@erikgrinaker erikgrinaker added the T:test Type: Tests that need love label Oct 3, 2020
@erikgrinaker erikgrinaker self-assigned this Oct 3, 2020
@erikgrinaker erikgrinaker changed the base branch from master to erik/e2e-test-cases October 3, 2020 14:55
@codecov
Copy link

codecov bot commented Oct 3, 2020

Codecov Report

Merging #5452 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5452      +/-   ##
==========================================
- Coverage   61.42%   61.41%   -0.02%     
==========================================
  Files         259      259              
  Lines       23456    23456              
==========================================
- Hits        14409    14405       -4     
- Misses       7590     7600      +10     
+ Partials     1457     1451       -6     
Impacted Files Coverage Δ
blockchain/v2/routine.go 69.09% <0.00%> (-9.10%) ⬇️
types/tx.go 88.63% <0.00%> (-4.55%) ⬇️
privval/signer_endpoint.go 78.78% <0.00%> (-3.04%) ⬇️
statesync/syncer.go 78.48% <0.00%> (-1.69%) ⬇️
statesync/snapshots.go 91.59% <0.00%> (-1.69%) ⬇️
consensus/reactor.go 75.09% <0.00%> (-1.51%) ⬇️
consensus/state.go 68.90% <0.00%> (+0.18%) ⬆️
blockchain/v0/reactor.go 62.56% <0.00%> (+0.98%) ⬆️
mempool/reactor.go 86.36% <0.00%> (+2.27%) ⬆️
blockchain/v0/pool.go 79.33% <0.00%> (+3.32%) ⬆️
... and 2 more

@erikgrinaker erikgrinaker marked this pull request as ready for review October 3, 2020 15:29
Copy link
Contributor

@melekes melekes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. If this action becomes required paths will have to be replaced

@erikgrinaker
Copy link
Contributor Author

Yeah, it should be required, will update GitHub after merging.

@tac0turtle
Copy link
Contributor

Yeah, it should be required, will update GitHub after merging.

With the current flow the action will sit as pending if none of the paths are touched. I can't recall if this blocks merging or not though. We will want to mimic the flow in the tests.yml file.

     - uses: technote-space/get-diff-action@v3
        with:
          SUFFIX_FILTER: |
            .go
            .mod
            .sum
          SET_ENV_NAME_INSERTIONS: 1
          SET_ENV_NAME_LINES: 1

@erikgrinaker
Copy link
Contributor Author

erikgrinaker commented Oct 5, 2020

Oh, right - yeah, I was wondering why we implemented this in two different ways. I just picked the simplest one. 🙂 Will update.

Base automatically changed from erik/e2e-test-cases to master October 5, 2020 10:23
@erikgrinaker erikgrinaker added the S:automerge Automatically merge PR when requirements pass label Oct 5, 2020
@erikgrinaker erikgrinaker merged commit 7e27e9b into master Oct 5, 2020
@erikgrinaker erikgrinaker deleted the erik/e2e-ci branch October 5, 2020 10:44
lovincyrus added a commit that referenced this pull request Oct 5, 2020
* docs: specify TM version in go tutorials (#5427)

Closes #5425

* privval: allow passing options to NewSignerDialerEndpoint (#5434)

Required for #5291 to set timeouts for remote signers.

* config: set statesync.rpc_servers when generating config file (#5433)

Required for #5291, to generate configuration files with state sync RPC servers.

* consensus: check block parts don't exceed maximum block bytes (#5431)

* ci: docker remvoe circleci and add github action (#5420)

* privval: fix ping message encoding (#5441)

Fixes #5371.

* docs: revise ADR 56, documenting short term decision around amnesia evidence  (#5440)

* light: expand on errors and docs (#5443)

* test: add end-to-end testing framework (#5435)

Partial fix for #5291. For details, see [README.md](https://github.com/tendermint/tendermint/blob/erik/e2e-tests/test/e2e/README.md) and [RFC-001](https://github.com/tendermint/tendermint/blob/master/docs/rfc/rfc-001-end-to-end-testing.md).

This only includes a single test case under `test/e2e/tests/`, as a proof of concept - additional test cases will be submitted separately. A randomized testnet generator will also be submitted separately, there a currently just a handful of static testnets under `test/e2e/networks/`. This will eventually replace the current P2P tests and run in CI.

* changelog: add missing date to v0.33.5 release, fix indentation (#5454)

I forgot to add the date when we cut 0.33.5. This fixes that. It also fixes a header indentation issue for 0.33.8.

* test: add basic end-to-end test cases (#5450)

Partial fix for #5291.

This adds a basic set of test cases for core network invariants. Although small, it is sufficient to replace and extend the current set of P2P tests. Further test cases can be added later.

* test: add GitHub action for end-to-end tests (#5452)

Partial fix for #5291.

* fix RPC blockresults reutrn (#5459)

## Description

In blocks_results we use the proto definition of abciResponses: https://github.com/tendermint/tendermint/blob/2672b91ab099b8b02f3afabae4a0a745acd93c3f/rpc/core/blocks.go#L152-L155, this leads to the use of the proto definition of the pubkey which is an interface in go (oneof). The interface must be registered with the JSON encoder to have it work correctly.

A clearer divide between proto types and native types is needed.

Closes: #XXX

* circleci: remove Gitian reproducible_builds job (#5462)

* docs: fix links to adr 56 (#5464)

## Description

fix broken link from a previous change

* test: remove P2P tests (#5453)

Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
Co-authored-by: Erik Grinaker <erik@interchain.berlin>
Co-authored-by: Callum Waters <cmwaters19@gmail.com>
Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: Tess Rinearson <tess.rinearson@gmail.com>
melekes pushed a commit that referenced this pull request Oct 9, 2020
erikgrinaker added a commit that referenced this pull request Oct 20, 2020
erikgrinaker added a commit that referenced this pull request Oct 22, 2020
erikgrinaker added a commit that referenced this pull request Oct 22, 2020
erikgrinaker added a commit that referenced this pull request Oct 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S:automerge Automatically merge PR when requirements pass T:test Type: Tests that need love

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants