Verify Versioned Hashes During Optimistic Sync by ethDreamer · Pull Request #4832 · sigp/lighthouse

ethDreamer · 2023-10-11T22:00:17Z

Issue Addressed

Validate Versioned Hashes During Optimistic Sync #4831

Proposed Changes

I've done a bit of refactoring which I think makes the code a bit cleaner and eliminates some cloning of the execution payload.

Additional Info

I used alloy-consensus to pull the versioned hashes from the raw transactions. Perhaps we should replace other functions that rely on ethers_core with alloy crates as well? An easy place to start (that could potentially fit in this PR) is calculating the block hash.

michaelsproul · 2023-10-17T00:00:49Z

Please re-target to unstable

realbigsean · 2023-10-18T15:12:33Z

Perhaps we should replace other functions that rely on ethers_core with reth crates as well

I'm down to replace this too. The reth version pinned in this PR imports ethers_core anyways though. But reth is currently migrating from ethers-rs and to alloy-rs. In the next reth release, reth-primitives will no longer import ethers_core and the reth-rlp crate has been deprecated (they're directly using alloy-rlp). In the semi-near term I'm pretty sure they plan to migrate reth-primitives to alloy-primitives as well.

So maybe we can merge this one, and open an issue to migrate to alloy-rs, and also get rid of the ether_core dep generally.

realbigsean · 2023-11-07T16:25:04Z

I'm gonna marked this blocked cause I think we're waiting on reth-primitives to feature gate c-kzg

ethDreamer · 2024-02-06T09:12:22Z

Looks like the reth team was able to update alloy just in time for deneb 🎉! After updating this PR to use alloy we no longer have build conflicts over c-kzg. Looks like the tests that I previously designed to test this code are working as expected with alloy as well. I think this is ready to review now @realbigsean

beacon_node/execution_layer/Cargo.toml

beacon_node/execution_layer/src/engine_api/new_payload_request.rs

beacon_node/execution_layer/src/versioned_hashes.rs

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

michaelsproul · 2024-02-15T07:08:07Z

@Mergifyio requeue

mergify · 2024-02-15T07:08:16Z

requeue

✅ This pull request will be re-embarked automatically

Details

The followup queue command will be automatically executed to re-embark the pull request

mergify · 2024-02-15T07:08:18Z

queue

🛑 The pull request has been removed from the queue `default`

Details

The queue conditions cannot be satisfied due to failing checks.

You can take a look at Queue: Embarked in merge queue check runs for more details.

In case of a failure due to a flaky test, you should first retrigger the CI.
Then, re-embark the pull request into the merge queue by posting the comment
@mergifyio refresh on the pull request.

michaelsproul · 2024-02-15T11:15:15Z

@Mergifyio requeue

mergify · 2024-02-15T11:16:29Z

requeue

✅ This pull request will be re-embarked automatically

Details

The followup queue command will be automatically executed to re-embark the pull request

mergify · 2024-02-15T11:16:31Z

queue

🛑 The pull request has been removed from the queue `default`

Details

Pull request #4832 has been dequeued by a dequeue command.

You can take a look at Queue: Embarked in merge queue check runs for more details.

In case of a failure due to a flaky test, you should first retrigger the CI.
Then, re-embark the pull request into the merge queue by posting the comment
@mergifyio refresh on the pull request.

michaelsproul · 2024-02-15T11:41:57Z

This is blocked on an update to the CI runner image for the MSRV

https://github.com/sigp/lighthouse/actions/runs/7915078252/job/21606147109?pr=5247

michaelsproul · 2024-02-15T11:42:08Z

@Mergifyio unqueue

mergify · 2024-02-15T11:42:23Z

unqueue

✅ The pull request has been removed from the queue `default`

Squashed commit of the following: commit 9cd7386 Author: Mark Mackey <mark@sigmaprime.io> Date: Fri Feb 9 12:57:39 2024 +0800 Update to rust 1.75 & Pin alloy-consensus commit 5d5b08d Author: Mark Mackey <mark@sigmaprime.io> Date: Fri Feb 9 12:39:34 2024 +0800 Faster Versioned Hash Extraction commit 2dddb84 Author: ethDreamer <37123614+ethDreamer@users.noreply.github.com> Date: Fri Feb 9 11:16:28 2024 +0800 Update beacon_node/execution_layer/src/engine_api/new_payload_request.rs Co-authored-by: realbigsean <seananderson33@GMAIL.com> commit 32d3e99 Author: ethDreamer <37123614+ethDreamer@users.noreply.github.com> Date: Tue Feb 6 17:03:28 2024 +0800 Update to use Alloy Instead of Reth Crates (#14) commit 7aadcad Author: Mark Mackey <mark@sigmaprime.io> Date: Wed Oct 18 14:48:05 2023 -0500 Fix Problems Caused By Merge commit 04b5f26 Author: Mark Mackey <mark@sigmaprime.io> Date: Wed Oct 11 16:45:11 2023 -0500 Added Moar Tests commit 3e78411 Author: Mark Mackey <mark@sigmaprime.io> Date: Wed Oct 11 15:14:39 2023 -0500 Added Tests for Version Hash Verification commit dd18fed Author: Mark Mackey <mark@sigmaprime.io> Date: Tue Oct 10 17:19:49 2023 -0500 Verify Versioned Hashes commit faa9ff1 Author: Mark Mackey <mark@sigmaprime.io> Date: Fri Oct 6 18:10:50 2023 -0500 Refactor for Clarity commit 27716c0 Author: Mark Mackey <mark@sigmaprime.io> Date: Fri Oct 6 17:56:16 2023 -0500 Convert NewPayloadRequest to use Reference

realbigsean · 2024-02-18T11:59:55Z

@mergify requeue

mergify · 2024-02-18T11:59:59Z

requeue

✅ This pull request will be re-embarked automatically

Details

The followup queue command will be automatically executed to re-embark the pull request

mergify · 2024-02-18T12:00:00Z

queue

✅ The pull request has been merged automatically

Details

The pull request has been merged automatically at a264afd

ethDreamer added the deneb label Oct 11, 2023

ethDreamer force-pushed the verify_versioned_hashes branch from 2a81cb8 to 05b7dd6 Compare October 12, 2023 20:11

michaelsproul deleted the branch sigp:unstable October 16, 2023 23:58

michaelsproul closed this Oct 16, 2023

michaelsproul reopened this Oct 17, 2023

michaelsproul changed the base branch from deneb-free-blobs to unstable October 17, 2023 00:01

realbigsean added the ready-for-review The code is ready for review label Oct 18, 2023

realbigsean self-requested a review October 18, 2023 15:26

ethDreamer force-pushed the verify_versioned_hashes branch from 05b7dd6 to 54c7913 Compare October 18, 2023 16:37

realbigsean added the blocked label Nov 7, 2023

dapplion removed the ready-for-review The code is ready for review label Jan 29, 2024

ethDreamer added 6 commits February 6, 2024 11:02

Convert NewPayloadRequest to use Reference

27716c0

Refactor for Clarity

faa9ff1

Verify Versioned Hashes

dd18fed

Added Tests for Version Hash Verification

3e78411

Added Moar Tests

04b5f26

Fix Problems Caused By Merge

7aadcad

ethDreamer force-pushed the verify_versioned_hashes branch from 0a029b8 to 7aadcad Compare February 6, 2024 07:18

Update to use Alloy Instead of Reth Crates (#14)

32d3e99

ethDreamer added ready-for-review The code is ready for review and removed blocked labels Feb 6, 2024

realbigsean requested changes Feb 7, 2024

View reviewed changes

beacon_node/execution_layer/Cargo.toml Outdated Show resolved Hide resolved

beacon_node/execution_layer/src/engine_api/new_payload_request.rs Outdated Show resolved Hide resolved

beacon_node/execution_layer/src/versioned_hashes.rs Outdated Show resolved Hide resolved

realbigsean added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Feb 7, 2024

ethDreamer and others added 2 commits February 9, 2024 11:16

Update beacon_node/execution_layer/src/engine_api/new_payload_request.rs

2dddb84

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

Faster Versioned Hash Extraction

5d5b08d

mergify bot mentioned this pull request Feb 15, 2024

merge queue: embarking unstable (7c23625) and #4832 together #5239

Closed

5 tasks

mergify bot added a commit that referenced this pull request Feb 15, 2024

Merge of #4832

bef8793

This was referenced Feb 15, 2024

merge queue: embarking unstable (7c23625) and [#4867 + #4832 + #5233] together #5240

Closed

Schedule Deneb on mainnet #5233

Merged

mergify bot added a commit that referenced this pull request Feb 15, 2024

Merge of #4832

18e1668

mergify bot mentioned this pull request Feb 15, 2024

merge queue: embarking unstable (7c23625) and [#4832 + #5233] together #5243

Closed

5 tasks

mergify bot added a commit that referenced this pull request Feb 15, 2024

Merge of #4832

efe6989

mergify bot mentioned this pull request Feb 15, 2024

merge queue: embarking unstable (7c23625) and #4832 together #5245

Closed

5 tasks

mergify bot added a commit that referenced this pull request Feb 15, 2024

Merge of #4832

e278d54

This was referenced Feb 15, 2024

merge queue: embarking unstable (7c23625) and [#5233 + #4867 + #4832 + #5010 + #5137] together #5247

Closed

Handle unknown head during attestation publishing #5010

Merged

enable doppelganger tests for deneb #5137

Merged

paulhauner mentioned this pull request Feb 18, 2024

Release v5.0.0 #5254

Merged

mergify bot added a commit that referenced this pull request Feb 18, 2024

Merge of #4832

aac2747

mergify bot mentioned this pull request Feb 18, 2024

merge queue: embarking unstable (1711b80) and #4832 together #5255

Closed

5 tasks

mergify bot merged commit a264afd into sigp:unstable Feb 18, 2024

jimmygchen mentioned this pull request Feb 21, 2024

Validate Versioned Hashes During Optimistic Sync #4831

Closed

Conversation

ethDreamer commented Oct 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue Addressed

Proposed Changes

Additional Info

Uh oh!

michaelsproul commented Oct 17, 2023

Uh oh!

realbigsean commented Oct 18, 2023

Uh oh!

realbigsean commented Nov 7, 2023

Uh oh!

ethDreamer commented Feb 6, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

michaelsproul commented Feb 15, 2024

Uh oh!

mergify bot commented Feb 15, 2024

✅ This pull request will be re-embarked automatically

Uh oh!

mergify bot commented Feb 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛑 The pull request has been removed from the queue default

Uh oh!

michaelsproul commented Feb 15, 2024

Uh oh!

mergify bot commented Feb 15, 2024

✅ This pull request will be re-embarked automatically

Uh oh!

mergify bot commented Feb 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛑 The pull request has been removed from the queue default

Uh oh!

michaelsproul commented Feb 15, 2024

Uh oh!

michaelsproul commented Feb 15, 2024

Uh oh!

mergify bot commented Feb 15, 2024

✅ The pull request has been removed from the queue default

Uh oh!

realbigsean commented Feb 18, 2024

Uh oh!

mergify bot commented Feb 18, 2024

✅ This pull request will be re-embarked automatically

Uh oh!

mergify bot commented Feb 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ The pull request has been merged automatically

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ethDreamer commented Oct 11, 2023 •

edited

Loading

mergify bot commented Feb 15, 2024 •

edited

Loading

🛑 The pull request has been removed from the queue `default`

mergify bot commented Feb 15, 2024 •

edited

Loading

🛑 The pull request has been removed from the queue `default`

✅ The pull request has been removed from the queue `default`

mergify bot commented Feb 18, 2024 •

edited

Loading