feat(op-interop-filter): implement core filtering logic#15
feat(op-interop-filter): implement core filtering logic#15opsuperchain wants to merge 7 commits intodevelopfrom
Conversation
|
@codex pls review |
|
To use Codex here, create a Codex account and connect to github. |
8ef5b52 to
def9152
Compare
c5116cf to
612a4f1
Compare
2c29436 to
a002b93
Compare
a002b93 to
bc68c48
Compare
There was a problem hiding this comment.
This PR is being reviewed by Cursor Bugbot
Details
You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.
To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.
1e36f03 to
47fcce6
Compare
|
Add a note for how to improve the cross validation logic to be faster (not wait for global cross unsafe to advance, instead do on a per chain basis) |
b7ae5a8 to
e1f45f2
Compare
577eabb to
7e33bb3
Compare
Implements the core filtering logic for op-interop-filter, a lightweight interop validation service designed to run alongside sequencers. Key components: - ChainIngester interface with LogsDBChainIngester (RPC + logsdb) and MockChainIngester (in-memory for tests) - LockstepCrossValidator for cross-chain message validation with lockstep timestamp advancement across all chains - Backend coordinator managing chain ingesters and failsafe state - RPC frontends: supervisor_checkAccessList (public) and admin API (JWT) Validation includes: - Message timing (init < execution, expiry window) - Cross-unsafe timestamp tracking - Checksum verification via logsdb Test coverage for validation logic, failsafe behavior, and cross-validator. Future work (see TODOs): - Reorg handling in chain ingester - Specific invalid message flagging - Kurtosis/devstack integration tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
96cd5ff to
be1c1ed
Compare
Review Guide: PR ethereum-optimism#18604 Refactoring CommitThis commit addresses all review feedback from the upstream PR. Below is a comprehensive guide for reviewing the changes. Quick Stats
High-Level Changes1. Validation Logic ConsolidationFiles: Moved all validation logic into the cross-validator. This eliminates the separate 2. Ingestion Loop ImprovementsFiles:
3. Safety: Panic → Error ReturnsFiles: Replaced 3 panics with proper error returns to prevent DoS via crafted timestamps. Example: // Before (panic reachable from RPC)
if expiresAt < access.Timestamp {
panic("overflow in expiry calculation")
}
// After (returns error)
if expiresAt < access.Timestamp {
return fmt.Errorf("overflow in expiry calculation: %w", types.ErrConflict)
}4. Dead Code RemovalFiles: Deleted unused 5. Error Handling ImprovementsFiles:
6. Config IntegrationFiles: Added Test Coverage: Complete BreakdownBackend Tests (
|
| Test | Code Path | What It Verifies |
|---|---|---|
TestBackend_Failsafe_ManualEnabled |
SetFailsafeEnabled(), FailsafeEnabled() |
Manual failsafe toggle works |
TestBackend_Failsafe_ChainError |
FailsafeEnabled() with chain error |
Chain ingester errors trigger failsafe |
TestBackend_Failsafe_CrossValidatorError |
FailsafeEnabled() with validator error |
Validator errors trigger failsafe |
TestBackend_Failsafe_AllClear |
FailsafeEnabled() normal state |
No errors = no failsafe |
TestBackend_Ready |
Ready() |
Backend ready state with/without chains |
TestBackend_CheckAccessList |
CheckAccessList() |
Failsafe, not ready, safety levels, unknown chain, happy path |
Cross-Validator Tests (lockstep_cross_validator_test.go) - 12 tests
| Test | Code Path | What It Verifies |
|---|---|---|
TestCrossValidator_TimeoutExceedsExpiry |
Expiry calculation | Rejects if timeout > expiry window |
TestCrossValidator_CrossUnsafe_Boundary |
CrossUnsafe validation | At boundary accepts, beyond rejects |
TestCrossValidator_KnownChain |
Chain lookup | Succeeds for known chains |
TestCrossValidator_UnknownChain |
Chain lookup | Returns error for unknown chains |
TestCrossValidator_ValidationFailureSetsError |
Error state | Validation failure sets validator error |
TestValidateAccessEntry_TimestampNotIngested |
Min ingested check | Rejects if timestamp not yet ingested |
TestValidateExecMsg_InitBeforeInclusion |
Temporal validation | Rejects when init timestamp not before inclusion |
TestValidateExecMsg_MessageExpired |
Expiry validation | Rejects expired messages |
TestAdvanceValidation_WaitsForChainsReady |
advanceValidation() |
Waits until all chains ready |
TestAdvanceValidation_InitializesToStartTimestamp |
advanceValidation() |
Initializes to start timestamp |
TestAdvanceValidation_AdvancesTimestamp |
advanceValidation() |
Advances crossValidatedTs over time |
TestAdvanceValidation_StopsAtMinIngested |
advanceValidation() |
Stops at slowest chain timestamp |
Chain Ingester Tests (logsdb_chain_ingester_test.go) - 18 tests
| Test | Code Path | What It Verifies |
|---|---|---|
TestLogsDBChainIngester_InitLogsDB |
initLogsDB() |
Creates logsDB and chain directory |
TestLogsDBChainIngester_SealParentBlock |
sealParentBlock() |
Seals parent block from RPC |
TestLogsDBChainIngester_IngestBlock |
ingestBlock() |
Ingests block and stores logs |
TestLogsDBChainIngester_IngestMultipleBlocks |
ingestBlock() x N |
Sequential block ingestion |
TestLogsDBChainIngester_Ready |
Ready() |
Reports ready after reaching start timestamp |
TestLogsDBChainIngester_ErrorState |
SetError(), Error(), ClearError() |
Error state management works |
TestLogsDBChainIngester_Contains |
Contains() |
Queries stored logs correctly |
TestLogsDBChainIngester_ReorgDetection |
Parent hash validation | Detects reorg via parent hash mismatch |
TestLogsDBChainIngester_QueryMethods |
GetExecMsgsAtTimestamp(), BlockHashAt() |
Query methods work correctly |
TestLogsDBChainIngester_Integration_RealLogsDB |
Full flow | End-to-end ingestion with real logsDB |
TestLogsDBChainIngester_InitIngestion_FreshStart |
initIngestion() fresh |
Starts from calculated backfill block |
TestLogsDBChainIngester_InitIngestion_ResumeFromExistingDB |
initIngestion() resume |
Resumes from existing DB state |
TestLogsDBChainIngester_InitIngestion_ErrorGettingHead |
initIngestion() RPC error |
Handles RPC failure gracefully |
TestLogsDBChainIngester_IngestBlock_NonSequentialError |
Block ordering | Rejects non-sequential blocks |
TestLogsDBChainIngester_IngestBlock_RPCError |
RPC failure | Handles missing block gracefully |
TestLogsDBChainIngester_IngestBlock_ReceiptsError |
Receipts failure | Handles missing receipts gracefully |
TestLogsDBChainIngester_IngestBlock_ErrorStateSkipsIngestion |
Error state | Skips ingestion when in error state |
TestLogsDBChainIngester_ErrorTypes |
Error type strings | New error reasons have correct strings |
JWT Auth Tests (jwt_auth_test.go) - 2 tests (5 subtests)
| Test | Code Path | What It Verifies |
|---|---|---|
TestJWTAuthentication |
JWT middleware | Valid JWT allows access, invalid rejected, supervisor API not gated |
TestSupervisorAPIWithoutJWT |
JWT middleware | Supervisor API works without JWT configured |
Testing Approach
Mock Strategy
mockChainIngester - In-memory implementation for unit tests:
- Stores logs in a
map[logKey]BlockSeal - Simple state management (
ready,err,latestBlock) - Thread-safe with
sync.RWMutex - Used by: Backend tests, Cross-validator tests
MockEthClient - RPC mock for integration tests:
- Stores blocks and receipts in maps
- Supports error injection (
SetInfoByLabelErr, etc.) - Used by: LogsDBChainIngester tests
mockCrossValidator - Simple validator mock:
- Returns configurable errors
- Used by: Backend tests
Test Categories
- Unit tests - Test individual methods with mocks
- Integration tests - Test component interactions with real logsDB
- Error injection tests - Verify graceful handling of failures
- Boundary tests - Test edge cases (overflow, expiry, timestamps)
What's NOT Tested (Known Gaps)
Start()/Stop()lifecycle (requires goroutine coordination)runIngestion()loop (tested indirectly viaingestBlock())- Concurrent access (no explicit race tests, but uses atomic/mutex)
Review Checklist
- Validation logic correctly moved from
validation.goto cross-validator - Panics replaced with error returns (3 locations)
- Dead code removed (
findBlockByTimestamp) - New error types used correctly (
ErrDataCorruption,ErrorInvalidExecutingMessage) - Tests cover the documented code paths
- Mock naming is consistent (
mockChainIngester,mockCrossValidator,MockEthClient)
0446784 to
1e37c72
Compare
…eview feedback This commit addresses all review comments from PR ethereum-optimism#18604, incorporating feedback from manual review and automated agent analysis. ## PR Review Comments Addressed ### Validation Logic Consolidation - Inline validation into cross-validator, delete validation.go ethereum-optimism#18604 (comment) - Use uint64 instead of safemath for timestamp checks ethereum-optimism#18604 (comment) - Add panic on overflow (later converted to error return for safety) ethereum-optimism#18604 (comment) - Add min ingested timestamp validation ethereum-optimism#18604 (comment) ### Ingestion Loop Improvements - Use ticker for poll loop instead of time.Sleep ethereum-optimism#18604 (comment) - Check reorg in all cases (validate parent hash on every block) ethereum-optimism#18604 (comment) - Clarify app context cancel handling ethereum-optimism#18604 (comment) ### Service/Config Integration - Get head info by label instead of index ethereum-optimism#18604 (comment) - Pull blocktime from rollup config ethereum-optimism#18604 (comment) - Add --networks and --rollup-configs flags for chain configuration ethereum-optimism#18604 (comment) ### Backend/Interface Cleanup - Add supportedSafetyLevel helper function ethereum-optimism#18604 (comment) - Rename checkExecutingMessage to more descriptive name ethereum-optimism#18604 (comment) ### Test Organization - Rename chain_ingester_test.go to logsdb_chain_ingester_test.go ethereum-optimism#18604 (comment) - Replace binary search test with rollup config test ethereum-optimism#18604 (comment) ## Agent Review Findings (10 agents) ### Critical Fixes - Replace 3 panics with error returns to prevent DoS via crafted timestamps Files: lockstep_cross_validator.go:137-143, 193-196 - Delete dead code: findBlockByTimestamp() function (50 lines) File: logsdb_chain_ingester.go:39-88 - Add failsafe triggers for ErrDataCorruption and ErrInvalidExecutingMessage File: logsdb_chain_ingester.go (ingestBlock error handling) ### Code Quality - Standardize mock naming: MockChainIngester -> mockChainIngester File: mock_test.go - Add constants: progressLogInterval, dataDirPermissions File: logsdb_chain_ingester.go - Remove unreachable code in getMinIngestedTimestamp File: lockstep_cross_validator.go:324 - Add new error types: ErrInvalidLog sentinel, ErrorInvalidExecutingMessage reason File: errors.go ### Test Coverage Expansion - Add 8 tests for initIngestion scenarios (fresh start, resume, errors) - Add error injection tests for ErrDataCorruption, ErrInvalidLog - Add cross-validator test coverage (450+ lines) ## Files Changed - backend.go: Add supportedSafetyLevel helper - backend_test.go: Update mock naming - config.go: Add rollup config loading from flags - errors.go: Add ErrInvalidLog, ErrorInvalidExecutingMessage - interfaces.go: Rename checkExecutingMessage - lockstep_cross_validator.go: Inline validation, fix panics - lockstep_cross_validator_test.go: New comprehensive tests - logsdb_chain_ingester.go: Ticker loop, constants, failsafe handling - logsdb_chain_ingester_test.go: New tests (renamed from chain_ingester_test.go) - mock_test.go: New consolidated mocks (renamed from mock_ingester_test.go) - service.go: Rollup config integration - validation.go: Deleted (inlined into cross-validator) - flags/flags.go: Add --networks, --rollup-configs flags Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…mestamp On restart, the cross-validator was failing with "missing data, earliest search checkpoint" error because: 1. startTimestamp was recalculated as new `now` on each startup 2. findAndSetEarliestBlock found the sealed anchor block instead of the first fully ingested block 3. The anchor block can't be queried via OpenBlock (no predecessor data) Changes: - Calculate startTimestamp once at startup, pass to all components - Chain ingesters calculate backfillTimestamp internally with underflow protection - Cross-validator waits for chains to be Ready() before initializing - findAndSetEarliestBlock: Use OpenBlock instead of FindSealedBlock to find first queryable block - sealParentBlock: Don't set earliestBlockNum (anchor is not queryable) - ingestBlock: Set earliestBlockNum on first successful ingestion Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1e37c72 to
a2c6b07
Compare
Merge related tests into single tests with sequential asserts: - Backend Ready tests (2 → 1) - Backend CheckAccessList tests (5 → 1) - CrossUnsafe boundary tests (2 → 1) - Ingester query tests (2 → 1) Delete redundant tests: - ChecksumMatch, ChecksumNotFound (fake checksums not useful) - TimeoutZero (keep TimeoutExceedsExpiry) - DirectUsage (covered by Integration_RealLogsDB) Net -185 lines while maintaining coverage. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
a2c6b07 to
301eb54
Compare
| if errors.Is(err, ErrInvalidLog) { | ||
| c.SetError(ErrorInvalidExecutingMessage, fmt.Sprintf("invalid log at block %d: %v", blockNum, err)) | ||
| return nil | ||
| } |
| execMsg, err := processors.DecodeExecutingMessageLog(l) | ||
| if err != nil { | ||
| return 0, fmt.Errorf("invalid log %d in block %d: %w: %w", l.Index, blockNum, ErrInvalidLog, err) | ||
| } |
There was a problem hiding this comment.
Remove all logHash or execMsg checks
- Change log levels from Info to Debug for metrics startup messages - Remove unnecessary header/timestamp lookup at startup - Use clock.SystemClock.Now() instead of time.Now() for startTimestamp - Rename earliestBlockNum → earliestIngestedBlock for clarity - Remove startingBlock/startingBlockSet fields, compute on demand via calculateStartingBlock() - Add config validation to reject backfillDuration exceeding current timestamp - Remove unused Timestamp fields from IngesterError and ValidatorError - Inline dataDirPermissions constant Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
d30dd80 to
5770a84
Compare
…sgsAtTimestamp Fix bug where `earliest == 0` was used to check if data was uninitialized, but block 0 is a valid genesis block for most OP chains (Base, Zora, etc.). Now correctly uses the earliestIngestedBlockSet atomic bool flag. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…tion - Extract validateMessageTiming() to consolidate temporal constraint checks - Remove duplicate overflow check for expiresAt calculation - Add genesis block underflow guard in initIngestion - Reduce log verbosity for message expiry window config - Add TestValidateMessageTiming with comprehensive edge cases - Add TestCrossValidator_InitiatingMessageNotFound for RPC coverage Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Closing: base branch targets upstream develop |
…16 (ethereum-optimism#19271) Add missing @param blueprint NatSpec to OpcmContractRef struct (#2). Add comments about pause blocking interop upgrades (#3). Document migrate() scope limitations and re-migration risks (#7, #15). Update PERMIT_ALL_CONTRACTS_INSTRUCTION comment (#12). Document intentional use of chainSystemConfigs[0] for shared contracts (#16). Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
feat(op-interop-filter): implement core filtering logic
Summary
This PR implements the core filtering logic for
op-interop-filter, a lightweight interop validation service designed to run alongside sequencers. The filter validates cross-chain executing messages by:supervisor_checkAccessListRPC endpoint for sequencers to validate transactionsThis implementation focuses on the filter logic and does not include sysgo/devstack integration, which is deferred to a follow-up PR.
Architecture
Files Overview
interfaces.goChainIngesterandCrossValidatorbackend.gologsdb_chain_ingester.goChainIngester: RPC + logsdb + reorg detectionlockstep_cross_validator.goCrossValidator: timestamp advancement + validationfrontend.goservice.goconfig.govalidation.goerrors.gomock_ingester_test.goChainIngesterfor unit testsbackend_test.goReview Guide
Recommended order:
interfaces.go- Core abstractions (ChainIngester,CrossValidator)backend.go- How components connectlockstep_cross_validator.go- Cross-chain validation logiclogsdb_chain_ingester.go- Block ingestion and reorg detectionfrontend.go- RPC layerbackend_test.go- Validate understanding through testsTest Coverage
Future Improvements
lockstep_cross_validator.go:217Test Plan
interop-filter-utils🤖 Generated with Claude Code