Skip to content

Sequencer halting on temporary error #12240

@sebastianst

Description

@sebastianst

From an internal report by @mdehoog:

I'm running an L3 sequencer locally (using Base Sepolia as the "L1") and noticed that block building halts pretty consistently, after the following logs:

t=2024-10-01T23:05:36+0000 lvl=info msg="Started sequencing new block" parent=0xe3411af59e2177377734ab199d2a8698f07db2a4cb11fdf80dbf228c21ef6024:209 l1Origin=0x74f255a508a8219038aa95e56e39de5aaf7b3b5781b9033b404bcd53a7d1c266:16027824
t=2024-10-01T23:05:36+0000 lvl=warn msg="Engine temporary error" err="temp: failed to fetch L1 block info and receipts: querying block: not found"
t=2024-10-01T23:05:36+0000 lvl=debug msg="Engine reported temporary error, but sequencer is not using engine" err="temp: failed to fetch L1 block info and receipts: querying block: not found"

after this temporary error, there are no more "Sequencer action" logs. I'm a little nervous about this happening in prod.
Can anyone explain why we don't schedule another action in this conditional?

d.log.Debug("Engine reported temporary error, but sequencer is not using engine", "err", x.Err)
return

My initial take: I think what happens is that when the sequencer enters startBuildingBlock, the building state in d.latest is cleared because of a previous d.onPayloadSuccess. It then hits a temp error and returns, but it never set any field in the d.latest BuildingState, which only happens at the end right before emitting a BuildStartEvent. This temp error then lands in onEngineTemporaryError where it checks if there's any non-zero BuildState at d.latest to make the decision whether the "sequencer is using the engine", and then returns early because it's still clear, so no future action is scheduled.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-op-nodeArea: op-nodeT-protocolTeam: changes to node components (op-node, op-reth, etc.) implemented by go/rust/etc. devs

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions