Skip to content

add separated runs by UUID#9367

Merged
mergify[bot] merged 3 commits intomainfrom
wb/issue-9352
Sep 6, 2022
Merged

add separated runs by UUID#9367
mergify[bot] merged 3 commits intomainfrom
wb/issue-9352

Conversation

@williambanfield
Copy link
Contributor

@williambanfield williambanfield commented Sep 2, 2022

This should be the last piece needed for this tool.
This allows the tool to generate reports on multiple experimental runs that may have been performed against the same chain.

The load tool has been updated to generate a UUID on startup to uniquely identify each experimental run. The report tool separates all of the results it reads by UUID and performs separate calculations for each discovered experiment.

Sample output is as follows

Experiment ID: 6bd7d1e8-d82c-4dbe-a1b3-40ab99e4fa30

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 86.632837ms
        Maximum Latency: 1.151089602s
        Average Latency: 813.759361ms
        Standard Deviation: 225.189977ms

Experiment ID: 453960af-6295-4282-aed6-367fc17c0de0

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 79.312992ms
        Maximum Latency: 1.162446243s
        Average Latency: 422.755139ms
        Standard Deviation: 241.832475ms

Total Invalid Tx: 0

closes: #9352

PR checklist

  • Tests written/updated, or no tests needed
  • CHANGELOG_PENDING.md updated, or no changelog entry needed
  • Updated relevant documentation (docs/) and code comments, or no
    documentation updates needed

@williambanfield williambanfield marked this pull request as ready for review September 2, 2022 21:27
@williambanfield williambanfield requested a review from a team September 2, 2022 21:27
Copy link
Contributor

@thanethomson thanethomson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea!

testConns = 512
testRate = 4
)
testId := [16]byte(uuid.New())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seem to be a couple of minor warnings about the naming convention here.

@williambanfield williambanfield added S:backport-to-v0.34.x Tell mergify to backport the PR to v0.34.x S:backport-to-v0.37.x Tell mergify to backport the PR to v0.37.x S:automerge Automatically merge PR when requirements pass labels Sep 6, 2022
@mergify mergify bot merged commit 1067ba1 into main Sep 6, 2022
@mergify mergify bot deleted the wb/issue-9352 branch September 6, 2022 14:38
mergify bot pushed a commit that referenced this pull request Sep 6, 2022
This _should_ be the last piece needed for this tool.
This allows the tool to generate reports on multiple experimental runs that may have been performed against the same chain.

The `load` tool has been updated to generate a `UUID` on startup to uniquely identify each experimental run. The `report` tool separates all of the results it reads by `UUID` and performs separate calculations for each discovered experiment.

Sample output is as follows

```
Experiment ID: 6bd7d1e8-d82c-4dbe-a1b3-40ab99e4fa30

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 86.632837ms
        Maximum Latency: 1.151089602s
        Average Latency: 813.759361ms
        Standard Deviation: 225.189977ms

Experiment ID: 453960af-6295-4282-aed6-367fc17c0de0

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 79.312992ms
        Maximum Latency: 1.162446243s
        Average Latency: 422.755139ms
        Standard Deviation: 241.832475ms

Total Invalid Tx: 0
```

closes: #9352

#### PR checklist

- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
      documentation updates needed

(cherry picked from commit 1067ba1)

# Conflicts:
#	go.mod
mergify bot pushed a commit that referenced this pull request Sep 6, 2022
This _should_ be the last piece needed for this tool.
This allows the tool to generate reports on multiple experimental runs that may have been performed against the same chain.

The `load` tool has been updated to generate a `UUID` on startup to uniquely identify each experimental run. The `report` tool separates all of the results it reads by `UUID` and performs separate calculations for each discovered experiment.

Sample output is as follows

```
Experiment ID: 6bd7d1e8-d82c-4dbe-a1b3-40ab99e4fa30

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 86.632837ms
        Maximum Latency: 1.151089602s
        Average Latency: 813.759361ms
        Standard Deviation: 225.189977ms

Experiment ID: 453960af-6295-4282-aed6-367fc17c0de0

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 79.312992ms
        Maximum Latency: 1.162446243s
        Average Latency: 422.755139ms
        Standard Deviation: 241.832475ms

Total Invalid Tx: 0
```

closes: #9352

#### PR checklist

- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
      documentation updates needed

(cherry picked from commit 1067ba1)

# Conflicts:
#	go.mod
williambanfield added a commit that referenced this pull request Sep 6, 2022
* add separated runs by UUID (#9367)

This _should_ be the last piece needed for this tool.
This allows the tool to generate reports on multiple experimental runs that may have been performed against the same chain.

The `load` tool has been updated to generate a `UUID` on startup to uniquely identify each experimental run. The `report` tool separates all of the results it reads by `UUID` and performs separate calculations for each discovered experiment.

Sample output is as follows

```
Experiment ID: 6bd7d1e8-d82c-4dbe-a1b3-40ab99e4fa30

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 86.632837ms
        Maximum Latency: 1.151089602s
        Average Latency: 813.759361ms
        Standard Deviation: 225.189977ms

Experiment ID: 453960af-6295-4282-aed6-367fc17c0de0

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 79.312992ms
        Maximum Latency: 1.162446243s
        Average Latency: 422.755139ms
        Standard Deviation: 241.832475ms

Total Invalid Tx: 0
```

closes: #9352

#### PR checklist

- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
      documentation updates needed

(cherry picked from commit 1067ba1)

# Conflicts:
#	go.mod

* fix merge conflict

* fix lint

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: William Banfield <wbanfield@gmail.com>
williambanfield added a commit that referenced this pull request Sep 6, 2022
* add separated runs by UUID (#9367)

This _should_ be the last piece needed for this tool.
This allows the tool to generate reports on multiple experimental runs that may have been performed against the same chain.

The `load` tool has been updated to generate a `UUID` on startup to uniquely identify each experimental run. The `report` tool separates all of the results it reads by `UUID` and performs separate calculations for each discovered experiment.

Sample output is as follows

```
Experiment ID: 6bd7d1e8-d82c-4dbe-a1b3-40ab99e4fa30

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 86.632837ms
        Maximum Latency: 1.151089602s
        Average Latency: 813.759361ms
        Standard Deviation: 225.189977ms

Experiment ID: 453960af-6295-4282-aed6-367fc17c0de0

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 79.312992ms
        Maximum Latency: 1.162446243s
        Average Latency: 422.755139ms
        Standard Deviation: 241.832475ms

Total Invalid Tx: 0
```

closes: #9352

#### PR checklist

- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
      documentation updates needed

(cherry picked from commit 1067ba1)

# Conflicts:
#	go.mod

* fix merge conflict

* fix lint

Co-authored-by: William Banfield <4561443+williambanfield@users.noreply.github.com>
Co-authored-by: William Banfield <wbanfield@gmail.com>
samricotta pushed a commit that referenced this pull request Sep 7, 2022
This _should_ be the last piece needed for this tool.
This allows the tool to generate reports on multiple experimental runs that may have been performed against the same chain.

The `load` tool has been updated to generate a `UUID` on startup to uniquely identify each experimental run. The `report` tool separates all of the results it reads by `UUID` and performs separate calculations for each discovered experiment.

Sample output is as follows

```
Experiment ID: 6bd7d1e8-d82c-4dbe-a1b3-40ab99e4fa30

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 86.632837ms
        Maximum Latency: 1.151089602s
        Average Latency: 813.759361ms
        Standard Deviation: 225.189977ms

Experiment ID: 453960af-6295-4282-aed6-367fc17c0de0

        Connections: 1
        Rate: 1000
        Size: 1024

        Total Valid Tx: 9000
        Total Negative Latencies: 0
        Minimum Latency: 79.312992ms
        Maximum Latency: 1.162446243s
        Average Latency: 422.755139ms
        Standard Deviation: 241.832475ms

Total Invalid Tx: 0
```

closes: #9352 

#### PR checklist

- [ ] Tests written/updated, or no tests needed
- [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed
- [ ] Updated relevant documentation (`docs/`) and code comments, or no
      documentation updates needed
@williambanfield williambanfield restored the wb/issue-9352 branch September 9, 2022 16:10
james-chf added a commit to heliaxdev/tendermint that referenced this pull request Nov 25, 2022
…x-rc1

* release/v0.37.0-rc1:
  QA Process report for v0.37.x (and baseline for v0.34.x) (tendermint#9499) (tendermint#9577)
  Fix TX payload for DO testnets (tendermint#9540) (tendermint#9542)
  blocksync: retry requests after timeout (backport tendermint#9518) (tendermint#9533)
  Extend the load report tool to include transactions' hashes (tendermint#9509) (tendermint#9513)
  build(deps): Bump styfle/cancel-workflow-action from 0.10.0 to 0.10.1 (tendermint#9502)
  build(deps): Bump actions/stale from 5 to 6 (tendermint#9494)
  loadtime: add block time to the data point (tendermint#9484) (tendermint#9489)
  config: Add missing storage section when generating config (tendermint#9483) (tendermint#9487)
  Sync Vote.Verify() in spec with implementation (tendermint#9466) (tendermint#9476)
  fix spec (tendermint#9467) (tendermint#9469)
  metrics: fix panic because of absent prometheus label (tendermint#9455) (tendermint#9474)
  Ensure Dockerfile stages use consistent Go version (backport tendermint#9462) (tendermint#9472)
  build(deps): Bump slackapi/slack-github-action from 1.21.0 to 1.22.0 (tendermint#9432)
  build(deps): Bump bufbuild/buf-setup-action from 1.7.0 to 1.8.0 (tendermint#9453)
  state: restore previous error message (tendermint#9435) (tendermint#9440)
  build(deps): Bump gonum.org/v1/gonum from 0.11.0 to 0.12.0 (tendermint#9411)
  docs: Update ADRs for v0.37 (tendermint#9399) (tendermint#9418)
  build(deps): Bump github.com/spf13/viper from 1.12.0 to 1.13.0 (tendermint#9410)
  build(deps): Bump github.com/lib/pq from 1.10.6 to 1.10.7 (tendermint#9409)
  feat: support HTTPS inside websocket (tendermint#9416) (tendermint#9422)
  Removed unused param (tendermint#9394)
  test: generate uuid on startup for load tool (tendermint#9383) (tendermint#9392)
  add redirect links (tendermint#9385) (tendermint#9389)
  refactor: migrate to cosmos/gogoproto (backport tendermint#9356) (tendermint#9381)
  cmd: print all versions of tendermint and its sub protocols  (tendermint#9329) (tendermint#9386)
  Add missing changes changelog files (backport tendermint#9376) (tendermint#9382)
  add separated runs by UUID (backport tendermint#9367) (tendermint#9379)
  spec: abci++ cleanup for v0.37 (backport tendermint#9288) (tendermint#9374)
  ci: Remove "(WARNING: BETA SOFTWARE)" tagline from all upcoming releases (tendermint#9371) (tendermint#9372)
  Update rpc client header (tendermint#9276) (tendermint#9349)
  ci: Pre-release workflows (backport tendermint#9366) (tendermint#9368)
  test: add the loadtime report tool (tendermint#9351) (tendermint#9364)
  Update Tendermint version to v0.37.0 (tendermint#9354)
  test: add the loadtime tool (tendermint#9342) (tendermint#9357)

# Conflicts:
#	version/version.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S:automerge Automatically merge PR when requirements pass S:backport-to-v0.34.x Tell mergify to backport the PR to v0.34.x S:backport-to-v0.37.x Tell mergify to backport the PR to v0.37.x

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: loadtime report tool performs separate calculation for different experimental runs

4 participants