Skip to content

4444-ish: Cleanup/disable header only forward sync#8209

Merged
asdacap merged 21 commits into
masterfrom
cleanup/disable-header-only-forward-sync
Mar 14, 2025
Merged

4444-ish: Cleanup/disable header only forward sync#8209
asdacap merged 21 commits into
masterfrom
cleanup/disable-header-only-forward-sync

Conversation

@asdacap

@asdacap asdacap commented Feb 14, 2025

Copy link
Copy Markdown
Contributor
  • Post merge block downloader originally does not allow header only forward sync. I've modified it to allow it, which then reveal an undisired behaviour where it does not switch to full sync unless the state sync finished at exactly the best suggested header.
    • This is because, the full sync will start exactly at the block after it the best suggested header (downloaded by the header only forward fast sync), which is also the downloaded state, so it need to be exact. Unless bodies is also downloaded, in which case IBlockchainProcessor will process its parents also.
    • Also state sync should only finish once now.
    • Anyway, I'm inclined to just remove the header only forward sync as per previous post merge behaviour as it significantly clears up which part of the code is PoW header related.
  • Additionally, this ensure that pos SyncPivot, all block is available, an assumption that make it simple to reason about when SyncPivot is movable.
  • Additionally, this make the behaviour of why starting fast sync with --Sync.DownloadBodiesInFastSync false, then restart with true after finished state sync will result in gap of block after the sync pivot to no longer be the case. The same is true with receipts, which I'm also inclined to just always download receipt for block after SyncPivot.
  • Fix on long range state sync (restart on very old node) BlockchainProcessor will load all pending blocks anyway resulting in a very high memory and processing a lot of blocks.

Types of changes

What types of changes does your code introduce?

  • Bugfix?
  • Refactoring

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

Notes on testing

  • Mainnet sync
  • Mainnet full sync
  • Long range state sync catch up still works.
  • Energyweb sync
  • Hive engine tests.
  • Sync github action.

@asdacap asdacap marked this pull request as draft February 14, 2025 14:39
@asdacap asdacap marked this pull request as ready for review February 17, 2025 05:15
Comment thread src/Nethermind/Nethermind.Synchronization/Blocks/FastSyncFeed.cs
@asdacap asdacap force-pushed the cleanup/disable-header-only-forward-sync branch from 8f72a88 to 0e938b9 Compare February 26, 2025 07:20
@asdacap asdacap requested a review from LukaszRozmej March 4, 2025 10:09

@MarekM25 MarekM25 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested it on a pre-merge network? EnergyWeb or Joc?

iterations++;
if (iterations > MaxBranchSize)
{
throw new InvalidOperationException($"Maximum size of branch reached ({MaxBranchSize}). This is unexpected.");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we want to do reorg with more blocks? I know that it is currently impossible, but we should aim to be able reorg to the last finalized block and we might have long non-finality events like in Holesky

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Increase MaxBranchSize?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, assuming that a beacon sync was initiated, this should not get called as block downloader would suggest the block one by one.

@asdacap asdacap changed the title Cleanup/disable header only forward sync 4444-ish: Cleanup/disable header only forward sync Mar 11, 2025
@asdacap asdacap requested a review from MarekM25 March 12, 2025 07:37
}

currentNumber += 1;
_currentNumber += 1;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this get out of sync with actual block numbers being requested?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line 155 reset it and only one of this DownloadBlocks is executed at any time.

@asdacap asdacap merged commit 3bc23ad into master Mar 14, 2025
@asdacap asdacap deleted the cleanup/disable-header-only-forward-sync branch March 14, 2025 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants