de-mystify tests & run them in parallel by zramsay · Pull Request #1031 · tendermint/tendermint

zramsay · 2017-12-29T01:08:37Z

reduces the reliance on bash scripts (see tendermint/coding#45) by putting various test pathways as make targets. This let's us use circle's parallelism feature to shorten test time and prepares for move to jenkins

zramsay · 2017-12-29T02:16:34Z

Makefile

+	# run the go unit tests with coverage (in docker)
+	docker run --name run_test -t tester bash test/test_cover.sh
+
+test_cover_ci:


the *_ci suffix is temporary and replaces some logic in the test.sh file. The won't be necessary when jenkins is setup

zramsay · 2017-12-29T02:17:41Z

Makefile

 	@echo "--> Installing tools"
 	go get -u -v $(GOTOOLS)
-	@gometalinter.v2 --install
+	#@gometalinter.v2 --install


punt the linter until jenkins

zramsay · 2017-12-29T02:20:21Z

circle.yml

 test:
  override:
-    - cd "$PROJECT_PATH" && set -o pipefail && make test_integrations 2>&1 | tee test_integrations.log:
+   - case ${CIRCLE_NODE_INDEX} in 0) cd "$PROJECT_PATH" && make build_docker_test_image && make test_cover_ci;; 1) cd "$PROJECT_PATH" && make build_docker_test_image && make test_apps_ci;; 2) cd "$PROJECT_PATH" && make build_docker_test_image && make test_persistence_ci;; 3) cd "$PROJECT_PATH" && make build_docker_test_image && make test_p2p;; esac:


ugly but works. build_docker... could be added as pre-run to the relevant make targets but that's gonna get re-worked with jenkins anyway

zramsay · 2017-12-29T02:20:58Z

circle.yml

+    #- cd "$PROJECT_PATH" && mv test_integrations.log "${CIRCLE_ARTIFACTS}"
+    #- cd "$PROJECT_PATH" && bash <(curl -s https://codecov.io/bash) -f coverage.txt
+    #- cd "$PROJECT_PATH" && mv coverage.txt "${CIRCLE_ARTIFACTS}"
+    #- cd "$PROJECT_PATH" && cp test/logs/messages "${CIRCLE_ARTIFACTS}/docker_logs.txt"


not sure what/how to do with this with the parallelism

this is ditching the coverage and the logs. We need to keep both ... I don't know how worth it it is to get this stuff done on Circle if we're only going to move to Jenkins ...

zramsay · 2017-12-29T02:22:06Z

test/net/test.sh

-bash test/net/setup.sh
-
-# start the testnet
-bash test/net/start.sh


all the mintnet stuff is deprecated. nothing useful in this dir

zramsay · 2017-12-29T02:22:39Z

test/persist/test_failure_indices.sh


    # before starting tendermint, remove the rpc socket
-    rm $RPC_ADDR
+    rm -f $RPC_ADDR


throws a warning on first run

zramsay · 2017-12-29T02:44:17Z

test/run_test.sh

-	echo "* branch $BRANCH; testing libs"
-	# checkout every github.com/tendermint dir and run its tests
-	bash test/test_libs.sh
-fi


all moved into Makefile, to be consumed by Jenkins and users

zramsay · 2017-12-29T02:44:37Z

test/test.sh

-	echo "* [$(date +"%T")] building docker image"
-	bash "$DIR/docker/build.sh"
-fi
+pwd


copied over from somewhere ... ?

lets remove this too

zramsay · 2017-12-29T02:45:03Z

test/test.sh

-fi
+#TODO
+## copy the coverage results out of docker container
+#docker cp run_test:/go/src/github.com/tendermint/tendermint/coverage.txt .


also to be dealt with in circle.yml - or just do it right in jenkins once

zramsay · 2017-12-29T02:52:38Z

test/test.sh

-if [[ "$CIRCLECI" == true ]]; then
-	docker run --name run_test -e CIRCLECI=true -t tester bash test/run_test.sh
-else
-	docker run --name run_test -t tester bash test/run_test.sh


moved and split to Makefile

zramsay · 2017-12-29T02:52:47Z

test/test.sh

-if [[ "$SKIP_BUILD" == "" ]]; then
-	echo
-	echo "* [$(date +"%T")] building docker image"
-	bash "$DIR/docker/build.sh"


in makefile

melekes · 2017-12-29T21:30:55Z

Makefile

 	vagrant ssh -c 'make test_integrations'

+### go tests without docker
+test_unit:


why not just test?

test would imply all the tests

melekes · 2017-12-29T21:31:21Z

Makefile

+### coverage, app, persistence, and libs tests
+test_cover:
+	# cleanup
+	bash ./test/test.sh


lets rename to ./test/cleanup.sh

zramsay · 2018-01-08T21:52:51Z

merging to jenkins branch

* de-mystify tests & run them in parallel (#1031) * test optimization for jenkins (#1093) * makefile cleanup * tests: split fast and slow go tests, closes #1055 * pr comments * restore circle conditions * fix need_abci * ... * docker run: no :Z for circle? * Remove cmd breaking comment

* unclog build * Bump version

zramsay requested review from ebuchman and melekes as code owners December 29, 2017 01:08

zramsay commented Dec 29, 2017

View reviewed changes

zramsay changed the title ~~WIP: run tests in parallel~~ run tests in parallel Dec 29, 2017

zramsay changed the title ~~run tests in parallel~~ de-mystify tests & run them in parallel Dec 29, 2017

zramsay force-pushed the tests-in-parallel branch from 6d8b8b5 to d55c12c Compare December 29, 2017 04:01

melekes reviewed Dec 29, 2017

View reviewed changes

zramsay added 11 commits January 3, 2018 19:06

run unit tests in parallel

65c1aae

use makefile to build docker test image

7e5c73e

script?

3c2c14a

script?

3a38c20

add warning to dockerfile

6554c69

test_libs.sh release logic for jenkins

7c46081

tests: demystify

289e26e

tests: remove deprecated net tests

dbda5d1

let's see what happens

e7d6296

circle

697d243

tests: add p2p to make & circle

17b5a9c

zramsay force-pushed the tests-in-parallel branch 2 times, most recently from 07c218d to 46272a0 Compare January 3, 2018 19:16

get ride of volumes

086ee4f

zramsay force-pushed the tests-in-parallel branch from f299b78 to 086ee4f Compare January 4, 2018 03:19

try the jenkins hack

7b0f4a0

zramsay mentioned this pull request Jan 4, 2018

Split fast go tests from slow go tests #1055

Closed

tests: ignore some packages; passes locally

2eb2b6c

zramsay closed this Jan 4, 2018

zramsay reopened this Jan 4, 2018

zramsay closed this Jan 4, 2018

zramsay reopened this Jan 4, 2018

zramsay closed this Jan 4, 2018

zramsay reopened this Jan 4, 2018

zramsay added 5 commits January 4, 2018 23:06

Merge branch 'develop' into tests-in-parallel

76afcd0

Merge branch 'develop' into tests-in-parallel

50db5b4

comment

a3fda2c

one less bash script

c49dcd1

less docker

3c78d75

zramsay force-pushed the tests-in-parallel branch from 3ba54fe to 3c78d75 Compare January 8, 2018 17:20

zramsay added 3 commits January 8, 2018 18:47

file perms for jenkins

7a9dd4d

...

d52fd37

Merge branch 'develop' into tests-in-parallel

d70e926

zramsay changed the base branch from develop to jenkins January 8, 2018 21:51

zramsay merged commit ef94221 into jenkins Jan 8, 2018

zramsay deleted the tests-in-parallel branch January 8, 2018 21:52

zramsay added a commit that referenced this pull request Jan 30, 2018

de-mystify tests & run them in parallel (#1031)

38fc7aa

ebuchman pushed a commit that referenced this pull request Feb 1, 2018

de-mystify tests & run them in parallel (#1031)

ebebbf8

xla pushed a commit that referenced this pull request Mar 6, 2018

de-mystify tests & run them in parallel (#1031)

38d2927

xla pushed a commit that referenced this pull request Mar 6, 2018

de-mystify tests & run them in parallel (#1031)

cd827d7

firelizzard18 pushed a commit to AccumulateNetwork/tendermint that referenced this pull request Nov 24, 2023

Release v0.38.0-rc2 (tendermint#1031)

8a7ab5a

* unclog build * Bump version

Conversation

zramsay commented Dec 29, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zramsay commented Jan 8, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zramsay commented Dec 29, 2017 •

edited

Loading