E2E: Add support for fast prototyping by otrack · Pull Request #1139 · cometbft/cometbft

otrack · 2023-07-17T16:07:11Z

This PR addresses issue #1059.

It is built atop #1043.

Summary:
This PR adds a support for fast prototyping to the E2E test suite. Please read the REAME.md located here to start using this code.

Progress:
To date, the following goals are in full (or partially) done:

Speed-up docker image construction
Add new small docker image (new make target)
Include Prometheus server support to docker compose
Provide manifests for medium-scale experiments (Provide manifests for medium-scale experiments [3d] #1082)
(partial) Allow customizing manifests to choose reactors (Allow customizing manifests to choose reactors [3d] #1083)
(partial) Provide scripts to collect and process metrics (Provide scripts to collect and process metrics #1084)
(partial) Add full support for e2e tests in fast-prototyping mode (Add full support for e2e tests in fast-prototyping mode #1085))

PR checklist

Tests written/updated
Changelog entry added in .changelog (we use unclog to manage our changelog)
Updated relevant documentation (docs/ or spec/) and code comments

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Revert "config: add bootstrap peers (#9680)" This reverts commit f12588a. * docs/p2p: bootstrap_peers config flag removed

* Revert "Remove unused code (cometbft#286)" This reverts commit a2d9915. Signed-off-by: Thane Thomson <connect@thanethomson.com> * node: Remove access to consensus state Consensus state should only ever be accessible via the consensus reactor. Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add changelog entry Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fix mistake in changelog entry Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com>

Signed-off-by: Thane Thomson <connect@thanethomson.com>

* Remove UPnP functionality Signed-off-by: Thane Thomson <connect@thanethomson.com> * Update documentation and specs to reflect UPnP removal Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add changelog entry Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com>

* ADR 107: Rename proto versions to pre-v1 betas * ADR 107: fix hyperlinks to ADR 103 * ADR 107: authorship of the revision Co-authored-by: Thane Thomson <connect@thanethomson.com> * ADR 107: change status to Accepted --------- Co-authored-by: Thane Thomson <connect@thanethomson.com>

* Add first draft Signed-off-by: Thane Thomson <connect@thanethomson.com> * Expand comment on actor receive method Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fix grammar Signed-off-by: Thane Thomson <connect@thanethomson.com> * Emphasize/clarify conclusions Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com>

…tbft#1131) Bumps [github.com/bufbuild/buf](https://github.com/bufbuild/buf) from 1.23.1 to 1.24.0. - [Release notes](https://github.com/bufbuild/buf/releases) - [Changelog](https://github.com/bufbuild/buf/blob/main/CHANGELOG.md) - [Commits](bufbuild/buf@v1.23.1...v1.24.0) --- updated-dependencies: - dependency-name: github.com/bufbuild/buf dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…ometbft#1132) Bumps [github.com/vektra/mockery/v2](https://github.com/vektra/mockery) from 2.31.1 to 2.32.0. - [Release notes](https://github.com/vektra/mockery/releases) - [Changelog](https://github.com/vektra/mockery/blob/master/docs/changelog.md) - [Commits](vektra/mockery@v2.31.1...v2.32.0) --- updated-dependencies: - dependency-name: github.com/vektra/mockery/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…etbft#1133) Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2.9.0 to 2.9.1. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](docker/setup-buildx-action@v2.9.0...v2.9.1) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…metbft#1134) Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.23.1 to 1.24.0. - [Release notes](https://github.com/bufbuild/buf-setup-action/releases) - [Commits](bufbuild/buf-setup-action@v1.23.1...v1.24.0) --- updated-dependencies: - dependency-name: bufbuild/buf-setup-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

- Add new small docker image (new make target)

- Add one example of custom mempool reactor (gossip w. tunnable propagation rate) - Provide (initial) support for e2e tests in fast-prototyping mode - Add new (customizable) benchmark

- Provide scripts to collect and process metrics

lasarojc

While there are improvements to be done, we could do them in the cometbft repo.

mempool/mempool.go

lasarojc · 2023-07-28T11:17:26Z

test/e2e/app/app.go

+	// Inject custom reactors (see node/main.go#startNode for a list of possibilities)
+	ExperimentalCustomReactors map[string]string `toml:"experimental_custom_reactors"`
+
+	// Experimental gossip "propagation rate" and "send once" features


Not sure if "experimental" should be part of the variable name or just in the comments of both the code and template config file.
On the good side of doing so, it will be impossible for someone to use the flag without knowing it is experimental. On the bad side, once it is graduated from experimental, there is a lot more updating to do, including the config files.

I believe this helps to understand that the feature is not available in the base reactor. The change is not that difficult when merging.

Another approach would be to add a set of experimental parameters. My feeling is that it is a bit heavy weighted.

@cason was proposing also that instead of having experimental reactors under fast-prototyping/reactors, to have them in different branches. Such a solution is also fully do-able.

test/e2e/fast-prototyping/experiments.sh

test/e2e/fast-prototyping/README.md

lasarojc · 2023-07-28T12:23:35Z

test/e2e/fast-prototyping/reactors/mempool/gossip/metrics.go

@@ -0,0 +1,20 @@
+package gossip


I'd suggest renaming this package/reactor to simpleGossip as gossip is very generic and will likely be component of every other reactor variant.

test/e2e/runner/rpc.go

test/e2e/runner/start.go

test/e2e/runner/stats.go

lasarojc · 2023-07-31T13:47:57Z

Awesome work.
Althought there are a few things to improve, this is already very useful and IMHO we should bring this into a feature branch in the CometBFT repo and finish the work there.

otrack · 2023-07-31T15:11:48Z

Awesome work. Althought there are a few things to improve, this is already very useful and IMHO we should bring this into a feature branch in the CometBFT repo and finish the work there.

Thanks a lot :-)

I plan to commit the PR today, and will let you create a feature branch based on it.

…pierre/fast-prototyping-1059

- remove false.csv (added by mistake)

Co-authored-by: lasaro <lasaro@gmail.com>

…ack/cometbft into pierre/fast-prototyping-1059

Co-authored-by: lasaro <lasaro@gmail.com>

…ack/cometbft into pierre/fast-prototyping-1059

Co-authored-by: lasaro <lasaro@gmail.com>

- remove not needed configuration file

…ack/cometbft into pierre/fast-prototyping-1059

- load: change info reporting when failing to sync with a node

thanethomson and others added 24 commits July 13, 2023 11:05

ci: Trigger workflows on merge group (cometbft#1118)

b3bc721

Signed-off-by: Thane Thomson <connect@thanethomson.com>

Rename removeTxOnReactor

3fb0325

Comment

fb22d97

Rename removeFromCache

52bd1fb

Comment

06c0d03

Remove test line forgotten in cometbft#934.

abf02b7

Comment

f3b6a7f

Revert "config: add bootstrap peers (#9680)" (cometbft#1109)

2d3ee28

* Revert "config: add bootstrap peers (#9680)" This reverts commit f12588a. * docs/p2p: bootstrap_peers config flag removed

ci: Disable CodeQL check in merge queues (cometbft#1123)

f2f4379

Signed-off-by: Thane Thomson <connect@thanethomson.com>

- Speed-up docker image construction

1086d9b

- Add new small docker image (new make target)

- change visibility (public) of two methods in mempool.Entry

9d52a43

- remove log in p2p/mock/reactor.go (not used and was failing)

d02eca8

revert mempool test size

0e23129

- Include Prometheus server support to docker compose

31e6fc5

- Allow customizing manifests to choose reactors

6a58ea8

- Add one example of custom mempool reactor (gossip w. tunnable propagation rate) - Provide (initial) support for e2e tests in fast-prototyping mode - Add new (customizable) benchmark

- Add (initial) scripts to run experiments

566ad5b

- Provide scripts to collect and process metrics

otrack requested a review from a team as a code owner July 17, 2023 16:07

otrack requested a review from a team July 17, 2023 16:07

otrack mentioned this pull request Jul 17, 2023

E2E: Add support for fast prototyping #1115

Closed

3 tasks

otrack added wip Work in progress P:bandwidth-optimization Priority: Optimize bandwidth usage labels Jul 17, 2023

otrack marked this pull request as draft July 17, 2023 16:09

otrack added 5 commits July 28, 2023 16:11

update

0d9cbfd

update

f4e76c8

fix prometheus e2e label

ba09a0b

update

17657f6

assign to Prometheus the last network addr.

7308f70

lasarojc approved these changes Jul 28, 2023

View reviewed changes

report (physical or logical) latency

1ab6f32

otrack and others added 15 commits July 31, 2023 17:25

Merge remote-tracking branch 'upstream/hvanz/mempool-interface' into …

458c398

…pierre/fast-prototyping-1059

- follow comemnts/remarks on the PR

c65c8fc

- remove false.csv (added by mistake)

remove log event that was added for debugging purposes

7ea06b7

Update test/e2e/fast-prototyping/render.sh

ca0a1ba

Co-authored-by: lasaro <lasaro@gmail.com>

Merge branch 'pierre/fast-prototyping-1059' of https://github.com/otr…

14c17bb

…ack/cometbft into pierre/fast-prototyping-1059

Update test/e2e/fast-prototyping/README.md

79d564d

Co-authored-by: lasaro <lasaro@gmail.com>

add custom benchmark config file under test/e2e/networks

1e093d8

Merge branch 'pierre/fast-prototyping-1059' of https://github.com/otr…

8d7dc21

…ack/cometbft into pierre/fast-prototyping-1059

Update test/e2e/node/main.go

9caa1eb

Co-authored-by: lasaro <lasaro@gmail.com>

Update test/e2e/runner/start.go

f86ce40

Co-authored-by: lasaro <lasaro@gmail.com>

- remove some dead code

d4fb499

- remove not needed configuration file

Merge branch 'pierre/fast-prototyping-1059' of https://github.com/otr…

173383e

…ack/cometbft into pierre/fast-prototyping-1059

remove results files (should not have)

0eab951

remove exp. config files (not needed)

8e9e891

two small fixes

d7bc83d

otrack marked this pull request as ready for review July 31, 2023 16:57

otrack requested a review from a team as a code owner July 31, 2023 16:57

- testnet: fix Prometheus IP computation for large network mask

0b23ba2

- load: change info reporting when failing to sync with a node

otrack merged commit 62e3137 into cometbft:pierre/fast-prototyping-1059 Jul 31, 2023

This was referenced Aug 1, 2023

Add support for fast prototyping #1059

Closed

Porting fast-docker image from #1089 #1192

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

E2E: Add support for fast prototyping#1139

E2E: Add support for fast prototyping#1139
otrack merged 106 commits intocometbft:pierre/fast-prototyping-1059from
otrack:pierre/fast-prototyping-1059

otrack commented Jul 17, 2023

Uh oh!

lasarojc left a comment

Uh oh!

Uh oh!

lasarojc Jul 28, 2023

Uh oh!

otrack Jul 31, 2023

Uh oh!

Uh oh!

Uh oh!

lasarojc Jul 28, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lasarojc commented Jul 31, 2023

Uh oh!

otrack commented Jul 31, 2023 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

otrack commented Jul 17, 2023

PR checklist

Uh oh!

lasarojc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lasarojc Jul 28, 2023

Choose a reason for hiding this comment

Uh oh!

otrack Jul 31, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lasarojc Jul 28, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lasarojc commented Jul 31, 2023

Uh oh!

otrack commented Jul 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

otrack commented Jul 31, 2023 •

edited

Loading