Overview

To define good default synchronous parameters for PBTS, collecting some relevant metrics is essential.

Original issue: tendermint/tendermint#7202

Currently planned test cases

1. QA/PBTS: Run 200-nodes test with v0.38.x's saturation point #2460 [Auxiliary run - not part of regular QA report]

• Run on v1
• PBTS enable height: 1
• Run QA 200-node TC, but only with v0.38.x saturation point (r=200, c=2)
• Run the experiment (90 secs each) 4-5 times
  • or just run the experiment for 3-4 mins
• Using latency emulation
• output: PBTS team can look at ProposalTimestampDifference to set default values

2. QA/v1: Run 200-nodes test for final report without latency emulation #2461 [Part of regular QA report]

• Run on v1
• PBTS enable height: 1
• Running saturation discovery
• Duration ~40 mins
• NOT Using latency emulation
• output: QA report, 200-node test, with saturation point report section, and metrics report section (for sat point)
• output: compare metrics between (v0.38.x, bft time) and (v1, PBTS)

3. QA/v1: Run 200-nodes test for final report with latency emulation #2513 [Part of regular QA report]

• Run on v1
• PBTS enable height: 1
• Running saturation discovery (mainly because with latencies the saturation point is likely to be different)
• Duration ~40 mins
• Using latency emulation
• output: baseline for future releases (200-node without latency emulation is now deprecated)
• output: compare metrics between (v1, no-lat-emulation) and (v1, lat-emulation)

4. New test case: X nodes [PBTS specific - not part of the regular QA report]

• Run on v1
• PBTS enable height: 1
• Duration ~Y mins
• Using latency emulation
• No Tx load
• clock_skew enable on: close to 1/3 of voting power
• clock_skew value: +5 seconds (i.e. 5 s in the future)
• output: average (or histogram of) number of rounds to decide, and block latency
• This test case is a DRAFT:
  • Play around with e2e on docker (Daniel volunteered 😄)