-
Notifications
You must be signed in to change notification settings - Fork 4.1k
stability: panic on RangeLookup dispatched to correct range #6000
Copy link
Copy link
Closed
Labels
S-1-stabilitySevere stability issues that can be fixed by upgrading, but usually don’t resolve by restartingSevere stability issues that can be fixed by upgrading, but usually don’t resolve by restarting
Milestone
Description
build sha: c382dd7
Beta cluster with photos + block_writer load. Restarted from scratch earlier today.
~1h after a restart, encountered the following:
panic: RangeLookup dispatched to correct range, but no matching RangeDescriptor was found: /Meta2/Table/55/1/8752590996753796817/"88e08dd7-d1b6-4aeb-ad46-1880b351e731"/6611/0
1: running [Created by grpc.(*Server).serveStreams.func1 @ server.go:324]
panic.go:464 panic(#1, #18)
storage store.go:1557 (*Store).Send.func1(0xc833bd1258, 0xc833bd12c8, 0x14449b721ca69e59, 0, 0xc833bd12c0, #5)
panic.go:426 panic(#1, #18)
storage replica_command.go:909 (*Replica).RangeLookup(#12, #19, #9, #22, 0, #4, 0x4, 0x1, 0, 0, ...)
storage replica_command.go:129 (*Replica).executeCmd(#12, #20, #17, 0, 0, 0, #19, #9, 0, #22, ...)
storage replica.go:1784 (*Replica).executeBatch(#12, #20, #17, 0, 0, #19, #9, 0, #22, 0, ...)
storage replica.go:1071 (*Replica).addReadOnlyCmd(#12, #20, #17, #22, 0, #4, 0x4, 0x1, 0, 0, ...)
storage replica.go:765 (*Replica).Send(#12, #20, #17, #22, 0, #4, 0x4, 0x1, 0, 0, ...)
storage store.go:1639 (*Store).Send(#5, #20, #17, #22, 0, #4, 0x4, 0x1, 0, 0, ...)
storage stores.go:177 (*Stores).Send(#6, #20, 0xc82bb06c30, 0, 0, #4, 0x4, 0x1, 0, 0, ...)
server node.go:729 (*Node).Batch.func2()
stop stopper.go:166 (*Stopper).RunTask(#8, 0xc833bd1870, 0)
server node.go:741 (*Node).Batch(#7, #20, #16, 0xc8271b2150, 0, 0, 0)
roachpb api.pb.go:1306 _Internal_Batch_Handler(#2, #7, #20, #16, 0xc833082a00, 0, 0, 0, 0)
grpc server.go:497 (*Server).processUnaryRPC(#11, #21, #14, #15, #10, #3, 0, 0, 0)
grpc server.go:646 (*Server).handleStream(#11, #21, #14, #15, 0)
grpc server.go:323 (*Server).serveStreams.func1.1(#13, #11, #21, #14, #15)
Full log (from ec2-52-91-234-29.compute-1.amazonaws.com):
node4.log.parse.txt
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
S-1-stabilitySevere stability issues that can be fixed by upgrading, but usually don’t resolve by restartingSevere stability issues that can be fixed by upgrading, but usually don’t resolve by restarting