feat: Add legacy RPC routing support for historical block data migration#6
Merged
louisliu2048 merged 8 commits intodevfrom Nov 7, 2025
Merged
feat: Add legacy RPC routing support for historical block data migration#6louisliu2048 merged 8 commits intodevfrom
louisliu2048 merged 8 commits intodevfrom
Conversation
This commit implements a complete Legacy RPC routing system to support seamless data migration from Erigon to Reth for XLayer network. Key Features: - Add legacy RPC client for routing historical block requests - Implement LegacyRpc trait for OpEthApi and EthApi - Support hybrid eth_getLogs queries across migration boundary - Add comprehensive integration test script for legacy RPC validation - Remove filter-related legacy code (not suitable for historical blocks) Core Changes: - crates/rpc/rpc-eth-types: Add LegacyRpcClient and configuration - crates/rpc/rpc-eth-api: Add LegacyRpc trait and helpers - crates/rpc/rpc: Implement legacy routing in filter.rs and core.rs - crates/optimism/rpc: Implement LegacyRpc trait for OpEthApi - crates/node/builder: Pass legacy config to EthApiBuilder Testing: - Add test_legacy_rpc.sh script with 10 test phases - Add comprehensive README for testing documentation - Test coverage: block queries, transactions, state, logs, filters, edge cases Docker: - Update DockerfileOp to include curl for debugging - Add build scripts for debug and release images Configuration: - XLayer Mainnet: cutoff block 42810021 - XLayer Testnet: cutoff block 12241701 - Legacy RPC timeout: 30s (configurable) This implementation ensures backward compatibility while maintaining performance for new block data served directly from Reth.
2898591 to
235747e
Compare
- Resolve conflicts in RPC configuration files - Keep both legacy RPC config and send_raw_transaction_sync_timeout - Update dependency: add dyn-clone alongside humantime - Sync with latest dev branch improvements
- Add exec_legacy() helper function for unified logging - Add elapsed_ms tracking for all legacy RPC calls - Add detailed metrics for hybrid eth_getLogs queries - Refactor XLayer-specific code to separate *_xlayer.rs files - Simplify macro implementations for better maintainability Changes: - Legacy RPC calls now log request/response with timing - Hybrid mode shows legacy_logs, local_logs, and total counts - Error cases include elapsed_ms and detailed error messages - All XLayer extensions moved to dedicated files for clean separation
dee8b4c to
4f1a7a4
Compare
- Add rpc_server_xlayer.rs for XLayer-specific extensions - Auto-apply genesis block number as default for legacy_cutoff_block - Only 2 files modified, 10 lines changed - All XLayer logic isolated in dedicated file
bac0b7b to
b0e7ebb
Compare
JimmyShi22
approved these changes
Nov 6, 2025
xzav3r
approved these changes
Nov 7, 2025
Vui-Chee
added a commit
that referenced
this pull request
Nov 11, 2025
* dev: (458 commits) Ignore xlayer-e2e-test during integration testing (#32) feat(genesis): optimize genesis loading with built-in xlayer specs (#23) Mirror default timeout in `clap` args struct (#31) fix constraint (#30) support trace tx log (#16) Avoid cloning tx during payload execution (#29) Feature: import blockchain data into op-reth (#20) perf(cargo): Use alloy 1.1.0 and fix some warning issue (#26) feat: Add E2E Test Suite (#19) fix tx number inconsistencies issue cause by intercept bridge function (#25) fix build error feat: integrate Apollo dynamic configuration (#8) feat: support `eth_transactionPreExec` api (#10) chore: bump version chore: bump revm v31.0.1 (paradigmxyz#19567) Xiong/block bridge okb (#21) feat: Add legacy RPC routing support for historical block data migration (#6) feat(genesis): Support legacyXLayerBlock in genesis.json (#11) chore: Update cursor rules, add rust comprehensive cursor rules (#17) chore: bump v1.9.0 (paradigmxyz#19507) ...
louisliu2048
pushed a commit
that referenced
this pull request
Mar 24, 2026
…ap checklist Complete implementation of the MPT WAL-First gap checklist (10 items) plus additional sei-db alignment fixes identified during review. WAL subsystem (#1): - Segment-based binary WAL (bincode, mptwal02 format) with CRC32 checksums - Record header contains version field for O(1) scan without deserialization - Segment-aware prune/truncate (delete whole segments, rewrite only boundary) - Corrupted tail detection and recovery on append - Single fsync per append (removed redundant meta.json fsync) Snapshot rewrite (#2): - Per-block incremental publish_generation in persist worker (keeps L3 hot) - Periodic full rewrite via separate published-rewrite worker thread - Pre-built segments from materializer's in-memory state (avoids disk reload) - Staged activation: compact/prune before meta activation (atomic boundary) - Retry mechanism for dropped rewrite jobs (pending_rewrite tracking) Backpressure (#3): - max_durable_lag / max_published_lag config fields - wait_for_backpressure() blocks frontend commit when lag exceeds threshold - 60s timeout with warning on backpressure wait Seek-best-snapshot (#4): - seek_best_snapshot_version() finds max(snapshot) <= target - WAL chain validation before replay attempt - Clear error messages with snapshot version and WAL range WAL auto-cleanup (#5): - Auto prune_before after each set_durable_version in both worker paths - Floor = min(manifest.earliest, published.earliest_snapshot) Replay parallelism (#6): - Aggressive parallelism thresholds during replay (storage_tries_min=4) - Batch WAL pre-fetch (64 entries per batch, amortizes lock + IO) Temp cleanup (#7): already done by codex account_root independent (#8): - CommitWalEntry.account_root now explicitly passed, not copied from state_root WAL upgrade field (#9): - CommitWalUpgrade struct (key/value pairs) for schema migrations - CommitWalEntry.upgrades field (empty for normal commits) Snapshot rate limiting (#10): - IoRateLimiter token bucket for background snapshot writes - snapshot_write_rate_mb_per_sec config (default 0 = unlimited) COW Arena: - MutableTrieArena rewritten with frozen base (Arc) + overlay (HashMap) - clone() is O(1) for frozen base, O(overlay) for mutations - freeze() uses Arc::make_mut for in-place patch when strong_count=1 - set_committed_base drops old base before snapshot() to ensure O(overlay) freeze Additional sei-db alignment: - SetInitialVersion API with validation (version==0, fresh DB) - nextVersion jump logic matching sei-db's nextVersionU32 - LoadForOverwriting: pre-open truncation (manifest + WAL + published) - Graceful error recovery: published baseline errors are non-fatal (warn only) - MemoryStats for in-memory node tracking - Lazy published view refresh in apply_bundle_state - Published rewrite timeout (configurable, default 60s) B4.5 profile (1M accounts, 10 blocks × 5K updates): - mpt-db per-block: ~370ms (vs reth ~1290ms) = ~3.5x faster - account_root: ~28ms (COW freeze working correctly) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
XLayer Legacy RPC Implementation
Based on op-geth migration RPC implementation
Parameters
--rpc.legacy-url--rpc.legacy-timeoutAPI Implementation Status
eth_getBlockByNumbereth_getBlockByHasheth_getHeaderByNumbereth_getHeaderByHasheth_getBlockReceiptseth_getBlockTransactionCountByNumbereth_getBlockTransactionCountByHasheth_getTransactionByHasheth_getTransactionReceipteth_getTransactionByBlockHashAndIndexeth_getTransactionByBlockNumberAndIndexeth_getRawTransactionByHasheth_getRawTransactionByBlockHashAndIndexeth_getRawTransactionByBlockNumberAndIndexeth_getBalanceeth_getCodeeth_getStorageAteth_getTransactionCounteth_getLogseth_newFiltereth_getFilterLogseth_getFilterChangeseth_uninstallFiltereth_getBlockInternalTransactionseth_getInternalTransactionsintegration testing
https://github.com/okx/xlayer-toolkit/blob/zjg/reth-legacy/scripts/legacy-api/test_legacy_rpc.sh