Skip to content

feat(op-interop-filter): implement core filtering logic#15

Closed
opsuperchain wants to merge 7 commits intodevelopfrom
karlfloersch/interop-filter-logic
Closed

feat(op-interop-filter): implement core filtering logic#15
opsuperchain wants to merge 7 commits intodevelopfrom
karlfloersch/interop-filter-logic

Conversation

@opsuperchain
Copy link
Copy Markdown
Collaborator

@opsuperchain opsuperchain commented Dec 12, 2025

feat(op-interop-filter): implement core filtering logic

Summary

This PR implements the core filtering logic for op-interop-filter, a lightweight interop validation service designed to run alongside sequencers. The filter validates cross-chain executing messages by:

  • Ingesting blocks from multiple L2 chains into per-chain LogsDB databases
  • Validating message timing constraints (init before execution, expiry windows)
  • Tracking cross-validated timestamps across all chains using a lockstep model
  • Providing a supervisor_checkAccessList RPC endpoint for sequencers to validate transactions

This implementation focuses on the filter logic and does not include sysgo/devstack integration, which is deferred to a follow-up PR.

Architecture

                                 ┌─────────────────────────────────────────┐
                                 │                Service                  │
                                 │  (Lifecycle, config, pprof, metrics)    │
                                 └───────────────────┬─────────────────────┘
                                                     │
                    ┌────────────────────────────────┼────────────────────────────────┐
                    │                                │                                │
                    ▼                                ▼                                ▼
           ┌────────────────┐             ┌──────────────────┐             ┌──────────────────┐
           │  RPC Server    │             │     Backend      │             │  Metrics Server  │
           │  (geth RPC)    │             │  (coordinator)   │             │  (prometheus)    │
           └───────┬────────┘             └────────┬─────────┘             └──────────────────┘
                   │                               │
         ┌─────────┴─────────┐                     │
         │                   │                     │
         ▼                   ▼                     │
 ┌───────────────┐  ┌────────────────┐             │
 │ QueryFrontend │  │ AdminFrontend  │             │
 │ (supervisor)  │  │ (admin, JWT)   │             │
 └───────┬───────┘  └───────┬────────┘             │
         │                  │                      │
         └────────┬─────────┘                      │
                  │                                │
                  ▼                                ▼
           ┌─────────────────────────────────────────────────────────────────┐
           │                            Backend                              │
           │   - Coordinates chain ingesters and cross-validator             │
           │   - Manages failsafe mode (manual + error-based)                │
           │   - Routes CheckAccessList requests                             │
           └──────────────┬───────────────────────────────┬──────────────────┘
                          │                               │
                          ▼                               ▼
           ┌──────────────────────────────┐   ┌──────────────────────────────┐
           │   LockstepCrossValidator     │   │  ChainIngesters (per chain)  │
           │                              │   │                              │
           │  - Validates executing msgs  │◄──┤  - LogsDBChainIngester       │
           │  - Tracks cross-validated    │   │    (RPC + logsdb)            │
           │    timestamp across chains   │   │  - MockChainIngester         │
           │  - Lockstep advancement:     │   │    (in-memory for tests)     │
           │    waits for ALL chains      │   │                              │
           └──────────────────────────────┘   └──────────────┬───────────────┘
                                                             │
                                              ┌──────────────┴──────────────┐
                                              │                             │
                                              ▼                             ▼
                                       ┌────────────┐                ┌────────────┐
                                       │  Chain A   │                │  Chain B   │
                                       │  LogsDB    │                │  LogsDB    │
                                       │  (local)   │                │  (local)   │
                                       └────────────┘                └────────────┘

Files Overview

File Purpose
interfaces.go Core interfaces: ChainIngester and CrossValidator
backend.go Central coordinator managing chain ingesters and failsafe state
logsdb_chain_ingester.go Production ChainIngester: RPC + logsdb + reorg detection
lockstep_cross_validator.go Production CrossValidator: timestamp advancement + validation
frontend.go RPC frontends: public query + JWT-protected admin
service.go Top-level lifecycle orchestration
config.go CLI flag parsing
validation.go Message timing validation
errors.go Ingester error types
mock_ingester_test.go In-memory ChainIngester for unit tests
backend_test.go Comprehensive unit tests

Review Guide

Recommended order:

  1. interfaces.go - Core abstractions (ChainIngester, CrossValidator)
  2. backend.go - How components connect
  3. lockstep_cross_validator.go - Cross-chain validation logic
  4. logsdb_chain_ingester.go - Block ingestion and reorg detection
  5. frontend.go - RPC layer
  6. backend_test.go - Validate understanding through tests

Test Coverage

  • Message Timing: init < inclusion, expiry boundaries
  • Failsafe: manual toggle, automatic on chain errors
  • Ready State: chain readiness tracking
  • CheckAccessList: failsafe, uninitialized, safety levels
  • CrossValidator: timeout expiry, cross-unsafe timestamps, unknown chains, checksum verification, error propagation

Future Improvements

  1. Reorg handling - Recovery logic for chain reorganizations
  2. Specific invalid message flagging - See TODO in lockstep_cross_validator.go:217
  3. Recoverable error states - Recover without full restart
  4. Kurtosis/devstack integration tests - Follow-up PR
  5. Per-chain validation advancement - Remove lockstep bottleneck

Test Plan

  • Unit tests pass
  • Manual testing with interop-filter-utils
  • Kurtosis/devstack integration (follow-up PR)

🤖 Generated with Claude Code

@karlfloersch
Copy link
Copy Markdown
Owner

@codex pls review

@chatgpt-codex-connector
Copy link
Copy Markdown

To use Codex here, create a Codex account and connect to github.

@opsuperchain opsuperchain force-pushed the karlfloersch/interop-filter-logic branch from 8ef5b52 to def9152 Compare December 18, 2025 04:54
@opsuperchain opsuperchain force-pushed the karlfloersch/interop-filter-logic branch from c5116cf to 612a4f1 Compare January 7, 2026 20:12
@opsuperchain opsuperchain force-pushed the karlfloersch/interop-filter-logic branch 6 times, most recently from 2c29436 to a002b93 Compare January 8, 2026 22:19
@karlfloersch karlfloersch force-pushed the karlfloersch/interop-filter-logic branch from a002b93 to bc68c48 Compare January 8, 2026 22:22
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is being reviewed by Cursor Bugbot

Details

You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

@opsuperchain opsuperchain force-pushed the karlfloersch/interop-filter-logic branch 9 times, most recently from 1e36f03 to 47fcce6 Compare January 11, 2026 03:20
@karlfloersch
Copy link
Copy Markdown
Owner

Add a note for how to improve the cross validation logic to be faster (not wait for global cross unsafe to advance, instead do on a per chain basis)

@opsuperchain opsuperchain force-pushed the karlfloersch/interop-filter-logic branch 7 times, most recently from b7ae5a8 to e1f45f2 Compare January 13, 2026 19:47
@opsuperchain opsuperchain force-pushed the karlfloersch/interop-filter-logic branch 5 times, most recently from 577eabb to 7e33bb3 Compare January 13, 2026 20:57
Implements the core filtering logic for op-interop-filter, a lightweight
interop validation service designed to run alongside sequencers.

Key components:
- ChainIngester interface with LogsDBChainIngester (RPC + logsdb) and
  MockChainIngester (in-memory for tests)
- LockstepCrossValidator for cross-chain message validation with
  lockstep timestamp advancement across all chains
- Backend coordinator managing chain ingesters and failsafe state
- RPC frontends: supervisor_checkAccessList (public) and admin API (JWT)

Validation includes:
- Message timing (init < execution, expiry window)
- Cross-unsafe timestamp tracking
- Checksum verification via logsdb

Test coverage for validation logic, failsafe behavior, and cross-validator.

Future work (see TODOs):
- Reorg handling in chain ingester
- Specific invalid message flagging
- Kurtosis/devstack integration tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@opsuperchain opsuperchain force-pushed the karlfloersch/interop-filter-logic branch 6 times, most recently from 96cd5ff to be1c1ed Compare January 14, 2026 23:57
@opsuperchain
Copy link
Copy Markdown
Collaborator Author

opsuperchain commented Jan 15, 2026

Review Guide: PR ethereum-optimism#18604 Refactoring Commit

This commit addresses all review feedback from the upstream PR. Below is a comprehensive guide for reviewing the changes.

Quick Stats

  • Files changed: 15
  • Lines: +2,268 / -675
  • Test functions: 43 total (38 top-level + 5 subtests)

High-Level Changes

1. Validation Logic Consolidation

Files: lockstep_cross_validator.go, validation.go (deleted)

Moved all validation logic into the cross-validator. This eliminates the separate validation.go file and makes the validation path clearer.

2. Ingestion Loop Improvements

Files: logsdb_chain_ingester.go

  • Replaced time.Sleep polling with time.Ticker for cleaner shutdown
  • Added parent hash validation on every block (catches reorgs immediately)
  • Added constants for magic numbers (progressLogInterval, dataDirPermissions)
  • Improved checkReorg() to distinguish expected vs unexpected cases:
    • Head before earliest block → expected, debug log (we started later)
    • Have block but can't get hash → unexpected, error log
    • Hash mismatch → reorg detected, trigger failsafe

3. Safety: Panic → Error Returns

Files: lockstep_cross_validator.go:137-143, 193-196

Replaced 3 panics with proper error returns to prevent DoS via crafted timestamps. Example:

// Before (panic reachable from RPC)
if expiresAt < access.Timestamp {
    panic("overflow in expiry calculation")
}

// After (returns error)
if expiresAt < access.Timestamp {
    return fmt.Errorf("overflow in expiry calculation: %w", types.ErrConflict)
}

4. Dead Code Removal

Files: logsdb_chain_ingester.go

Deleted unused findBlockByTimestamp() binary search function (50 lines).

5. Error Handling Improvements

Files: errors.go, logsdb_chain_ingester.go

  • Added ErrInvalidLog sentinel error
  • Added ErrorInvalidExecutingMessage ingester error reason
  • Added failsafe triggers for ErrDataCorruption and ErrInvalidExecutingMessage

6. Config Integration

Files: config.go, flags/flags.go, service.go

Added --networks and --rollup-configs flags for chain configuration via superchain-registry.


Test Coverage: Complete Breakdown

Backend Tests (backend_test.go) - 6 tests

Test Code Path What It Verifies
TestBackend_Failsafe_ManualEnabled SetFailsafeEnabled(), FailsafeEnabled() Manual failsafe toggle works
TestBackend_Failsafe_ChainError FailsafeEnabled() with chain error Chain ingester errors trigger failsafe
TestBackend_Failsafe_CrossValidatorError FailsafeEnabled() with validator error Validator errors trigger failsafe
TestBackend_Failsafe_AllClear FailsafeEnabled() normal state No errors = no failsafe
TestBackend_Ready Ready() Backend ready state with/without chains
TestBackend_CheckAccessList CheckAccessList() Failsafe, not ready, safety levels, unknown chain, happy path

Cross-Validator Tests (lockstep_cross_validator_test.go) - 12 tests

Test Code Path What It Verifies
TestCrossValidator_TimeoutExceedsExpiry Expiry calculation Rejects if timeout > expiry window
TestCrossValidator_CrossUnsafe_Boundary CrossUnsafe validation At boundary accepts, beyond rejects
TestCrossValidator_KnownChain Chain lookup Succeeds for known chains
TestCrossValidator_UnknownChain Chain lookup Returns error for unknown chains
TestCrossValidator_ValidationFailureSetsError Error state Validation failure sets validator error
TestValidateAccessEntry_TimestampNotIngested Min ingested check Rejects if timestamp not yet ingested
TestValidateExecMsg_InitBeforeInclusion Temporal validation Rejects when init timestamp not before inclusion
TestValidateExecMsg_MessageExpired Expiry validation Rejects expired messages
TestAdvanceValidation_WaitsForChainsReady advanceValidation() Waits until all chains ready
TestAdvanceValidation_InitializesToStartTimestamp advanceValidation() Initializes to start timestamp
TestAdvanceValidation_AdvancesTimestamp advanceValidation() Advances crossValidatedTs over time
TestAdvanceValidation_StopsAtMinIngested advanceValidation() Stops at slowest chain timestamp

Chain Ingester Tests (logsdb_chain_ingester_test.go) - 18 tests

Test Code Path What It Verifies
TestLogsDBChainIngester_InitLogsDB initLogsDB() Creates logsDB and chain directory
TestLogsDBChainIngester_SealParentBlock sealParentBlock() Seals parent block from RPC
TestLogsDBChainIngester_IngestBlock ingestBlock() Ingests block and stores logs
TestLogsDBChainIngester_IngestMultipleBlocks ingestBlock() x N Sequential block ingestion
TestLogsDBChainIngester_Ready Ready() Reports ready after reaching start timestamp
TestLogsDBChainIngester_ErrorState SetError(), Error(), ClearError() Error state management works
TestLogsDBChainIngester_Contains Contains() Queries stored logs correctly
TestLogsDBChainIngester_ReorgDetection Parent hash validation Detects reorg via parent hash mismatch
TestLogsDBChainIngester_QueryMethods GetExecMsgsAtTimestamp(), BlockHashAt() Query methods work correctly
TestLogsDBChainIngester_Integration_RealLogsDB Full flow End-to-end ingestion with real logsDB
TestLogsDBChainIngester_InitIngestion_FreshStart initIngestion() fresh Starts from calculated backfill block
TestLogsDBChainIngester_InitIngestion_ResumeFromExistingDB initIngestion() resume Resumes from existing DB state
TestLogsDBChainIngester_InitIngestion_ErrorGettingHead initIngestion() RPC error Handles RPC failure gracefully
TestLogsDBChainIngester_IngestBlock_NonSequentialError Block ordering Rejects non-sequential blocks
TestLogsDBChainIngester_IngestBlock_RPCError RPC failure Handles missing block gracefully
TestLogsDBChainIngester_IngestBlock_ReceiptsError Receipts failure Handles missing receipts gracefully
TestLogsDBChainIngester_IngestBlock_ErrorStateSkipsIngestion Error state Skips ingestion when in error state
TestLogsDBChainIngester_ErrorTypes Error type strings New error reasons have correct strings

JWT Auth Tests (jwt_auth_test.go) - 2 tests (5 subtests)

Test Code Path What It Verifies
TestJWTAuthentication JWT middleware Valid JWT allows access, invalid rejected, supervisor API not gated
TestSupervisorAPIWithoutJWT JWT middleware Supervisor API works without JWT configured

Testing Approach

Mock Strategy

mockChainIngester - In-memory implementation for unit tests:

  • Stores logs in a map[logKey]BlockSeal
  • Simple state management (ready, err, latestBlock)
  • Thread-safe with sync.RWMutex
  • Used by: Backend tests, Cross-validator tests

MockEthClient - RPC mock for integration tests:

  • Stores blocks and receipts in maps
  • Supports error injection (SetInfoByLabelErr, etc.)
  • Used by: LogsDBChainIngester tests

mockCrossValidator - Simple validator mock:

  • Returns configurable errors
  • Used by: Backend tests

Test Categories

  1. Unit tests - Test individual methods with mocks
  2. Integration tests - Test component interactions with real logsDB
  3. Error injection tests - Verify graceful handling of failures
  4. Boundary tests - Test edge cases (overflow, expiry, timestamps)

What's NOT Tested (Known Gaps)

  • Start()/Stop() lifecycle (requires goroutine coordination)
  • runIngestion() loop (tested indirectly via ingestBlock())
  • Concurrent access (no explicit race tests, but uses atomic/mutex)

Review Checklist

  • Validation logic correctly moved from validation.go to cross-validator
  • Panics replaced with error returns (3 locations)
  • Dead code removed (findBlockByTimestamp)
  • New error types used correctly (ErrDataCorruption, ErrorInvalidExecutingMessage)
  • Tests cover the documented code paths
  • Mock naming is consistent (mockChainIngester, mockCrossValidator, MockEthClient)

@opsuperchain opsuperchain force-pushed the karlfloersch/interop-filter-logic branch 2 times, most recently from 0446784 to 1e37c72 Compare January 15, 2026 21:12
opsuperchain and others added 2 commits January 15, 2026 21:14
…eview feedback

This commit addresses all review comments from PR ethereum-optimism#18604, incorporating
feedback from manual review and automated agent analysis.

## PR Review Comments Addressed

### Validation Logic Consolidation
- Inline validation into cross-validator, delete validation.go
  ethereum-optimism#18604 (comment)
- Use uint64 instead of safemath for timestamp checks
  ethereum-optimism#18604 (comment)
- Add panic on overflow (later converted to error return for safety)
  ethereum-optimism#18604 (comment)
- Add min ingested timestamp validation
  ethereum-optimism#18604 (comment)

### Ingestion Loop Improvements
- Use ticker for poll loop instead of time.Sleep
  ethereum-optimism#18604 (comment)
- Check reorg in all cases (validate parent hash on every block)
  ethereum-optimism#18604 (comment)
- Clarify app context cancel handling
  ethereum-optimism#18604 (comment)

### Service/Config Integration
- Get head info by label instead of index
  ethereum-optimism#18604 (comment)
- Pull blocktime from rollup config
  ethereum-optimism#18604 (comment)
- Add --networks and --rollup-configs flags for chain configuration
  ethereum-optimism#18604 (comment)

### Backend/Interface Cleanup
- Add supportedSafetyLevel helper function
  ethereum-optimism#18604 (comment)
- Rename checkExecutingMessage to more descriptive name
  ethereum-optimism#18604 (comment)

### Test Organization
- Rename chain_ingester_test.go to logsdb_chain_ingester_test.go
  ethereum-optimism#18604 (comment)
- Replace binary search test with rollup config test
  ethereum-optimism#18604 (comment)

## Agent Review Findings (10 agents)

### Critical Fixes
- Replace 3 panics with error returns to prevent DoS via crafted timestamps
  Files: lockstep_cross_validator.go:137-143, 193-196
- Delete dead code: findBlockByTimestamp() function (50 lines)
  File: logsdb_chain_ingester.go:39-88
- Add failsafe triggers for ErrDataCorruption and ErrInvalidExecutingMessage
  File: logsdb_chain_ingester.go (ingestBlock error handling)

### Code Quality
- Standardize mock naming: MockChainIngester -> mockChainIngester
  File: mock_test.go
- Add constants: progressLogInterval, dataDirPermissions
  File: logsdb_chain_ingester.go
- Remove unreachable code in getMinIngestedTimestamp
  File: lockstep_cross_validator.go:324
- Add new error types: ErrInvalidLog sentinel, ErrorInvalidExecutingMessage reason
  File: errors.go

### Test Coverage Expansion
- Add 8 tests for initIngestion scenarios (fresh start, resume, errors)
- Add error injection tests for ErrDataCorruption, ErrInvalidLog
- Add cross-validator test coverage (450+ lines)

## Files Changed
- backend.go: Add supportedSafetyLevel helper
- backend_test.go: Update mock naming
- config.go: Add rollup config loading from flags
- errors.go: Add ErrInvalidLog, ErrorInvalidExecutingMessage
- interfaces.go: Rename checkExecutingMessage
- lockstep_cross_validator.go: Inline validation, fix panics
- lockstep_cross_validator_test.go: New comprehensive tests
- logsdb_chain_ingester.go: Ticker loop, constants, failsafe handling
- logsdb_chain_ingester_test.go: New tests (renamed from chain_ingester_test.go)
- mock_test.go: New consolidated mocks (renamed from mock_ingester_test.go)
- service.go: Rollup config integration
- validation.go: Deleted (inlined into cross-validator)
- flags/flags.go: Add --networks, --rollup-configs flags

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…mestamp

On restart, the cross-validator was failing with "missing data, earliest
search checkpoint" error because:
1. startTimestamp was recalculated as new `now` on each startup
2. findAndSetEarliestBlock found the sealed anchor block instead of the
   first fully ingested block
3. The anchor block can't be queried via OpenBlock (no predecessor data)

Changes:
- Calculate startTimestamp once at startup, pass to all components
- Chain ingesters calculate backfillTimestamp internally with underflow
  protection
- Cross-validator waits for chains to be Ready() before initializing
- findAndSetEarliestBlock: Use OpenBlock instead of FindSealedBlock to
  find first queryable block
- sealParentBlock: Don't set earliestBlockNum (anchor is not queryable)
- ingestBlock: Set earliestBlockNum on first successful ingestion

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@opsuperchain opsuperchain force-pushed the karlfloersch/interop-filter-logic branch from 1e37c72 to a2c6b07 Compare January 15, 2026 21:14
Merge related tests into single tests with sequential asserts:
- Backend Ready tests (2 → 1)
- Backend CheckAccessList tests (5 → 1)
- CrossUnsafe boundary tests (2 → 1)
- Ingester query tests (2 → 1)

Delete redundant tests:
- ChecksumMatch, ChecksumNotFound (fake checksums not useful)
- TimeoutZero (keep TimeoutExceedsExpiry)
- DirectUsage (covered by Integration_RealLogsDB)

Net -185 lines while maintaining coverage.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@opsuperchain opsuperchain force-pushed the karlfloersch/interop-filter-logic branch from a2c6b07 to 301eb54 Compare January 15, 2026 23:12
if errors.Is(err, ErrInvalidLog) {
c.SetError(ErrorInvalidExecutingMessage, fmt.Sprintf("invalid log at block %d: %v", blockNum, err))
return nil
}
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove

execMsg, err := processors.DecodeExecutingMessageLog(l)
if err != nil {
return 0, fmt.Errorf("invalid log %d in block %d: %w: %w", l.Index, blockNum, ErrInvalidLog, err)
}
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove all logHash or execMsg checks

- Change log levels from Info to Debug for metrics startup messages
- Remove unnecessary header/timestamp lookup at startup
- Use clock.SystemClock.Now() instead of time.Now() for startTimestamp
- Rename earliestBlockNum → earliestIngestedBlock for clarity
- Remove startingBlock/startingBlockSet fields, compute on demand via calculateStartingBlock()
- Add config validation to reject backfillDuration exceeding current timestamp
- Remove unused Timestamp fields from IngesterError and ValidatorError
- Inline dataDirPermissions constant

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@opsuperchain opsuperchain force-pushed the karlfloersch/interop-filter-logic branch from d30dd80 to 5770a84 Compare January 21, 2026 16:56
opsuperchain and others added 2 commits January 21, 2026 17:22
…sgsAtTimestamp

Fix bug where `earliest == 0` was used to check if data was uninitialized,
but block 0 is a valid genesis block for most OP chains (Base, Zora, etc.).
Now correctly uses the earliestIngestedBlockSet atomic bool flag.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…tion

- Extract validateMessageTiming() to consolidate temporal constraint checks
- Remove duplicate overflow check for expiresAt calculation
- Add genesis block underflow guard in initIngestion
- Reduce log verbosity for message expiry window config
- Add TestValidateMessageTiming with comprehensive edge cases
- Add TestCrossValidator_InitiatingMessageNotFound for RPC coverage

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@opsuperchain
Copy link
Copy Markdown
Collaborator Author

Closing: base branch targets upstream develop

karlfloersch pushed a commit that referenced this pull request Mar 4, 2026
…16 (ethereum-optimism#19271)

Add missing @param blueprint NatSpec to OpcmContractRef struct (#2).
Add comments about pause blocking interop upgrades (#3). Document
migrate() scope limitations and re-migration risks (#7, #15). Update
PERMIT_ALL_CONTRACTS_INSTRUCTION comment (#12). Document intentional
use of chainSystemConfigs[0] for shared contracts (#16).

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants