Skip to content

feat: add benchmark workflow for base-builder with base-reth-node validator#591

Merged
meyer9 merged 9 commits intomainfrom
meyer9/gha-benchmark
Feb 4, 2026
Merged

feat: add benchmark workflow for base-builder with base-reth-node validator#591
meyer9 merged 9 commits intomainfrom
meyer9/gha-benchmark

Conversation

@meyer9
Copy link
Copy Markdown
Contributor

@meyer9 meyer9 commented Jan 28, 2026

Summary

  • Adds a GitHub Actions workflow that benchmarks base-builder as sequencer with base-reth-node (built from this repo) as the validator node

Changes

  • .github/benchmark/rbuilder.yml - Benchmark configuration
  • .github/workflows/benchmark.yml - GitHub Actions workflow

The workflow:

  • Builds base-reth-node with maxperf profile
  • Builds reth and base-builder dependencies (with caching)
  • Runs transfer throughput benchmark (10 blocks, 1B gas limit)
  • Uploads benchmark output and HTML report as artifacts

Test plan

  • Verify workflow runs successfully
  • Check benchmark output artifact
  • Check HTML report artifact

@cb-heimdall
Copy link
Copy Markdown
Collaborator

cb-heimdall commented Jan 28, 2026

✅ Heimdall Review Status

Requirement Status More Info
Reviews 1/1
Denominator calculation
Show calculation
1 if user is bot 0
1 if user is external 0
2 if repo is sensitive 0
From .codeflow.yml 1
Additional review requirements
Show calculation
Max 0
0
From CODEOWNERS 0
Global minimum 0
Max 1
1
1 if commit is unverified 1
Sum 2

@meyer9 meyer9 marked this pull request as draft January 28, 2026 18:19
@meyer9 meyer9 marked this pull request as ready for review January 28, 2026 19:37
@meyer9
Copy link
Copy Markdown
Contributor Author

meyer9 commented Jan 28, 2026

Next steps:

  • add a workflow to compare the output of main to a commit 1 day ago, 1 week ago, and 1 month ago
  • add a workflow to compare the output of every PR to main

@danyalprout danyalprout changed the title feat: add benchmark workflow for rbuilder with base-reth-node validator feat: add benchmark workflow for base-builder with base-reth-node validator Jan 29, 2026
Copy link
Copy Markdown
Collaborator

@danyalprout danyalprout left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@meyer9 this looks good to me, the benchmark job is failing to start reth (assuming this maybe due to the binary change or something), but should be good to go once fixed

@meyer9 meyer9 force-pushed the meyer9/gha-benchmark branch from dd1f34d to aa83b64 Compare February 3, 2026 15:10
@meyer9
Copy link
Copy Markdown
Contributor Author

meyer9 commented Feb 4, 2026

dependent on base/benchmark#162

@meyer9 meyer9 force-pushed the meyer9/gha-benchmark branch from cdc5f98 to 05c292d Compare February 4, 2026 16:52
Adds a GitHub Actions workflow that benchmarks rbuilder as sequencer
with base-reth-node (built from this repo) as the validator node.

The workflow:
- Builds base-reth-node with maxperf profile
- Builds reth and rbuilder dependencies (with caching)
- Runs transfer throughput benchmark (10 blocks, 1B gas limit)
- Uploads benchmark output and HTML report as artifacts
- Build base-reth-node and rbuilder in a single job to share common crates
- Remove reth build since it's not needed for this benchmark
Skip expensive builds when the same commit SHA and rbuilder version
have already been built. All build steps are conditional on cache miss.
Use hash of Cargo.toml, Cargo.lock, crates/, and bin/ directories
instead of commit SHA. This allows iterating on the benchmark
workflow without triggering binary rebuilds.
@meyer9 meyer9 force-pushed the meyer9/gha-benchmark branch from 05c292d to 2d23652 Compare February 4, 2026 17:15
@meyer9 meyer9 requested a review from danyalprout February 4, 2026 18:05
@meyer9 meyer9 enabled auto-merge February 4, 2026 18:06
- name: Build base-builder
if: steps.cache-binaries.outputs.cache-hit != 'true'
run: |
cargo build --bin base-builder --profile maxperf
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My preference would be to have these as justfile targets so there's only one place where the production build target lives, but not a blocker.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will address in next PR!

Copy link
Copy Markdown
Contributor

@refcell refcell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gtg -- comments are non-blocking

@meyer9 meyer9 added this pull request to the merge queue Feb 4, 2026
Merged via the queue into main with commit 01eebbd Feb 4, 2026
17 checks passed
@meyer9 meyer9 deleted the meyer9/gha-benchmark branch February 4, 2026 18:41
refcell pushed a commit that referenced this pull request Feb 18, 2026
refcell pushed a commit that referenced this pull request Feb 18, 2026
haardikk21 pushed a commit that referenced this pull request Mar 17, 2026
…idator (#591)

* feat: add benchmark workflow for rbuilder with base-reth-node validator

Adds a GitHub Actions workflow that benchmarks rbuilder as sequencer
with base-reth-node (built from this repo) as the validator node.

The workflow:
- Builds base-reth-node with maxperf profile
- Builds reth and rbuilder dependencies (with caching)
- Runs transfer throughput benchmark (10 blocks, 1B gas limit)
- Uploads benchmark output and HTML report as artifacts

* refactor: combine binary builds to share compiled dependencies

- Build base-reth-node and rbuilder in a single job to share common crates
- Remove reth build since it's not needed for this benchmark

* feat: cache binaries by commit SHA to avoid rebuilds

Skip expensive builds when the same commit SHA and rbuilder version
have already been built. All build steps are conditional on cache miss.

* perf: cache binaries by source hash instead of commit SHA

Use hash of Cargo.toml, Cargo.lock, crates/, and bin/ directories
instead of commit SHA. This allows iterating on the benchmark
workflow without triggering binary rebuilds.

* Update to base-builder and remove checkout step

* Debug log level

* Use test branch

* switch back

* Use base-builder naming instead of rbuilder
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants