Skip to content

Data race on SpanStore cache between concurrent spanById operations #1737

@praetoriansentry

Description

@praetoriansentry

Concurrent read/write access to SpanStore's internal cache at 0x00c0011f1f08. Both operations occur in spanById (span_store.go:164-165) when caching span data during block verification. Race between parallel VerifyHeader and VerifyHeaders goroutines accessing validator snapshots.

8509ba2

INFO [08-28|00:52:54.441] Span check complete                      foundNewSpan=true
INFO [08-28|00:52:55.004] Imported new chain segment               number=387 hash=96800c..6d3f8a blocks=1 txs=0  mgas=0.000 elapsed=1.360ms     mgasps=0.000   snapdiffs=6.53KiB triediffs=41.80KiB triedirty=0.00B
INFO [08-28|00:52:55.397] Whitelisting new milestone from heimdall block=386 hash=0a449e..76d92b
==================
WARNING: DATA RACE
Read at 0x00c0011f1f08 by goroutine 7351:
  github.com/ethereum/go-ethereum/consensus/bor.(*SpanStore).spanById()
      /var/lib/bor/consensus/bor/span_store.go:164 +0x424
  github.com/ethereum/go-ethereum/consensus/bor.(*SpanStore).getFutureSpan()
      /var/lib/bor/consensus/bor/span_store.go:236 +0x10d
  github.com/ethereum/go-ethereum/consensus/bor.(*SpanStore).spanByBlockNumber()
      /var/lib/bor/consensus/bor/span_store.go:208 +0x3a8
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).getVeBlopSnapshot()
      /var/lib/bor/consensus/bor/bor.go:665 +0x16b
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).snapshot()
      /var/lib/bor/consensus/bor/bor.go:547 +0x2a4
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).verifySeal()
      /var/lib/bor/consensus/bor/bor.go:785 +0x129
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).verifyCascadingFields()
      /var/lib/bor/consensus/bor/bor.go:527 +0x635
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).verifyHeader()
      /var/lib/bor/consensus/bor/bor.go:455 +0xb10
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).VerifyHeader()
      /var/lib/bor/consensus/bor/bor.go:341 +0x4c
  github.com/ethereum/go-ethereum/eth.newHandler.func1()
      /var/lib/bor/eth/handler.go:259 +0x1d9
  github.com/ethereum/go-ethereum/eth/fetcher.(*BlockFetcher).importBlocks.func1()
      /var/lib/bor/eth/fetcher/block_fetcher.go:1203 +0x241

Previous write at 0x00c0011f1f08 by goroutine 7354:
  github.com/ethereum/go-ethereum/consensus/bor.(*SpanStore).spanById()
      /var/lib/bor/consensus/bor/span_store.go:165 +0x45b
  github.com/ethereum/go-ethereum/consensus/bor.(*SpanStore).getFutureSpan()
      /var/lib/bor/consensus/bor/span_store.go:236 +0x10d
  github.com/ethereum/go-ethereum/consensus/bor.(*SpanStore).spanByBlockNumber()
      /var/lib/bor/consensus/bor/span_store.go:208 +0x3a8
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).getVeBlopSnapshot()
      /var/lib/bor/consensus/bor/bor.go:665 +0x16b
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).snapshot()
      /var/lib/bor/consensus/bor/bor.go:547 +0x2a4
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).verifySeal()
      /var/lib/bor/consensus/bor/bor.go:785 +0x129
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).verifyCascadingFields()
      /var/lib/bor/consensus/bor/bor.go:527 +0x635
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).verifyHeader()
      /var/lib/bor/consensus/bor/bor.go:455 +0xb10
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).VerifyHeaders.func1()
      /var/lib/bor/consensus/bor/bor.go:361 +0xd1

Goroutine 7351 (running) created at:
  github.com/ethereum/go-ethereum/eth/fetcher.(*BlockFetcher).importBlocks()
      /var/lib/bor/eth/fetcher/block_fetcher.go:1193 +0x634
  github.com/ethereum/go-ethereum/eth/fetcher.(*BlockFetcher).loop()
      /var/lib/bor/eth/fetcher/block_fetcher.go:529 +0x2ca4
  github.com/ethereum/go-ethereum/eth/fetcher.(*BlockFetcher).Start.gowrap1()
      /var/lib/bor/eth/fetcher/block_fetcher.go:298 +0x33

Goroutine 7354 (finished) created at:
  github.com/ethereum/go-ethereum/consensus/bor.(*Bor).VerifyHeaders()
      /var/lib/bor/consensus/bor/bor.go:359 +0x1c9
  github.com/ethereum/go-ethereum/core.(*BlockChain).verifyPendingHeaders()
      /var/lib/bor/core/blockchain.go:3766 +0x899
  github.com/ethereum/go-ethereum/core.(*BlockChain).startHeaderVerificationLoop.func1()
      /var/lib/bor/core/blockchain.go:3720 +0x16d
==================
{
  "IPT_bytes_out": 44103720,
  "output_text": "WARNING: DATA RACE",
  "source": {
    "container": "l2-el-2-bor-heimdall-v2-validator--9cc44e3a7f5941d28a39839f36a39def",
    "name": "l2-el-2-bor-heimdall-v2-validator--9cc44e3a7f5941d28a39839f36a39def",
    "stream": "error"
  },
  "moment": {
    "input_hash": "8161223531707450023",
    "_vtime_ticks": 2665611161737,
    "session_id": "c0ceefcd588ee0d629772f6fd9eae1ec-36-12"
  }
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions