Persist light client bootstrap#5915
Merged
mergify[bot] merged 77 commits intosigp:unstablefrom Sep 10, 2024
Merged
Conversation
…rsist-light-client-updates
…rsist-light-client-updates
…rsist-light-client-updates
Member
Author
|
This PR is in a good spot for another review. The pruning logic has been simplified, the bootstrap tests are passing and I've done a bit of tweaking to the test infra to make light client testing a bit easier in general. I am planning on writing some specific tests related to the pruning logic. I'm also planning on running a mainnet node with light client enabled and compare I/O + disk usage vs a node w/ light client disabled. If its a negligible difference it'd be cool to just enable light clients by default |
dapplion
reviewed
Aug 26, 2024
dapplion
reviewed
Aug 26, 2024
| } | ||
|
|
||
| if epoch_boundary_blocks.contains(&block_root) { | ||
| non_checkpoint_block_roots.remove(&block_root); |
Collaborator
There was a problem hiding this comment.
Much cleaner <3
The intended logic is:
no skipped slot
- Slot 2/1 root C : Add C to non_checkpoint_block_roots
- Slot 2/0 root B : Add B to epoch_boundary_blocks
- Slot 1/31 root A : Add A to non_checkpoint_block_roots
slot 31 skipped
- Slot 2/1 root C : Add C to non_checkpoint_block_roots
- Slot 2/0 root A : Add A to epoch_boundary_blocks
- Slot 1/31 root A : Add A to non_checkpoint_block_roots, remove A from non_checkpoint_block_roots
slot 0 skipped
- Slot 2/1 root B : Add B to non_checkpoint_block_roots
- Slot 2/0 root B : Add B to epoch_boundary_blocks, remove B from non_checkpoint_block_roots
- Slot 1/31 root A : Add A to non_checkpoint_block_roots
…rsist-light-client-bootstrap
michaelsproul
approved these changes
Sep 9, 2024
Member
|
@mergify queue |
✅ The pull request has been merged automaticallyDetailsThe pull request has been merged automatically at a94b12b |
5 tasks
chong-he
pushed a commit
to chong-he/lighthouse
that referenced
this pull request
Nov 26, 2024
* persist light client updates * update beacon chain to serve light client updates * resolve todos * cache best update * extend cache parts * is better light client update * resolve merge conflict * initial api changes * add lc update db column * fmt * added tests * add sim * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * fix some weird issues with the simulator * tests * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * test changes * merge conflict * testing * started work on ef tests and some code clean up * update tests * linting * noop pre altair, were still failing on electra though * allow for zeroed light client header * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * merge unstable * remove unwraps * remove unwraps * fetch bootstrap without always querying for state * storing bootstrap parts in db * mroe code cleanup * test * prune sync committee branches from dropped chains * Update light_client_update.rs * merge unstable * move functionality to helper methods * refactor is best update fn * refactor is best update fn * improve organization of light client server cache logic * fork diget calc, and only spawn as many blcoks as we need for the lc update test * resovle merge conflict * add electra bootstrap logic, add logic to cache current sync committee * add latest sync committe branch cache * fetch lc update from the cache if it exists * fmt * Fix beacon_chain tests * Add debug code to update ranking_order ef test * Fix compare code * merge conflicts * merge conflict * add better error messaging * resolve merge conflicts * remove lc update from basicsim * rename sync comittte variable and fix persist condition * refactor get_light_client_update logic * add better comments, return helpful error messages over http and rpc * pruning canonical non checkpoint slots * fix test * rerun test * update pruning logic, add tests * fix tests * fix imports * fmt * refactor db code * Refactor db method * Refactor db method * add additional comments * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-bootstrap * fix merge * linting * merge conflict * prevent overflow * enable lc server for http api tests * fix tests * remove prints * remove warning * revert change
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue Addressed
Which issue # does this PR address?
Partially #3651
Proposed Changes
This doc details how a
LightClientBootstrapobject can be stored in chunks so that it can then be served later on without any expensive computations/db querieshttps://hackmd.io/@dapplion/BkxPbc6eQT
To summarize
LightClientHeadercan be constructed by fetching the full block. Blocks are already stored by root and are available forever. Fetching a single block by root isn't that expensive of a db querySyncCommitteeBranchin the db, keyed off the block rootSyncCommitteewill also be stored in the db, keyed off the sync committee period.LightClientBootstrapfrom the current finalized period, we can construct theSyncCommitteefrom the current state, since fetching the current beacon state is cheapSyncCommitteefrom the db.Additional Info
This PR builds off of #5545.
#5545 should be merged before this PR