-
Notifications
You must be signed in to change notification settings - Fork 4.1k
kvserver: panic in pkg/kv/kvserver/syncing_write.go #73371
Copy link
Copy link
Closed
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-robotOriginated from a bot.Originated from a bot.T-kvKV TeamKV TeamX-stalebranch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.
Description
ccl/multiregionccl.TestConcurrentAddDropRegions failed with artifacts on master @ 2b8b0f64c63a376784926667f5fbffa4c4fcc60b:
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! /usr/local/go/src/runtime/pprof/runtime.go:40 +0xa3
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc00923a480)
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! /go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:292 +0x58
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !created by github.com/cockroachdb/pebble.(*tableCacheShard).init
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! /go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:284 +0xd1
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !goroutine 170942 [sync.Cond.Wait]:
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !sync.runtime_notifyListWait(0xc00d4aa5e0, 0x0)
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! /usr/local/go/src/runtime/sema.go:513 +0x13d
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !sync.(*Cond).Wait(0x4414e20)
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! /usr/local/go/src/sync/cond.go:56 +0x8c
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !github.com/cockroachdb/pebble/record.(*flusherCond).Wait(...)
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! /go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:193
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !github.com/cockroachdb/pebble/record.(*LogWriter).flushLoop(0xc00d4aa500, {0x447a240, 0x8f93f08})
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! /go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:396 +0x479
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !runtime/pprof.Do({0x622b4e0, 0xc000074108}, {{0xc0004a6540, 0xc016878cf0, 0xc0015a8f80}}, 0xc0015a8fc0)
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! /usr/local/go/src/runtime/pprof/runtime.go:40 +0xa3
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !github.com/cockroachdb/pebble/record.NewLogWriter.func2()
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! /go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:314 +0x5c
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !created by github.com/cockroachdb/pebble/record.NewLogWriter
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! /go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/record/log_writer.go:313 +0x329
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !For more context, check log files in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestConcurrentAddDropRegions2880819291
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !****************************************************************************
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !This node experienced a fatal error (printed above), and as a result the
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !process is terminating.
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !Fatal errors can occur due to faulty hardware (disks, memory, clocks) or a
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !problem in CockroachDB. With your help, the support team at Cockroach Labs
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !will try to determine the root cause, recommend next steps, and we can
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !improve CockroachDB based on your report.
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !Please submit a crash report by following the instructions here:
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! https://github.com/cockroachdb/cockroach/issues/new/choose
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !If you would rather not post publicly, please contact us directly at:
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 ! support@cockroachlabs.com
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !
F211202 07:05:05.624680 182766 kv/kvserver/store_raft.go:348 [n3,s3,r47/6:{-}] 1 !The Cockroach Labs team appreciates your feedback.
ERROR: exit status 7
29 runs completed, 1 failures, over 43m48s
context canceled
Help
See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)Parameters in this failure:
- GOFLAGS=-parallel=4
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-robotOriginated from a bot.Originated from a bot.T-kvKV TeamKV TeamX-stalebranch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.