-
Notifications
You must be signed in to change notification settings - Fork 4.1k
roachtest: tpccbench/nodes=6/cpu=16/multi-az failed #72869
Copy link
Copy link
Closed
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.S-1High impact: many users impacted, serious risk of high unavailability or data lossHigh impact: many users impacted, serious risk of high unavailability or data lossT-kvKV TeamKV TeamX-stale
Description
roachtest.tpccbench/nodes=6/cpu=16/multi-az failed with artifacts on release-21.2 @ 774ea54918869b6ba51674942fe5e9f81327d2a7:
The test failed on branch=release-21.2, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/tpccbench/nodes=6/cpu=16/multi-az/run_1
cluster.go:1804,tpcc.go:1177,tpcc.go:1189,search.go:43,search.go:173,tpcc.go:1185,tpcc.go:955,test_runner.go:777: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod start --encrypt=false teamcity-3745090-1637133260-66-n7cpu16-geo:1-6 returned: exit status 7
(1) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod start --encrypt=false teamcity-3745090-1637133260-66-n7cpu16-geo:1-6 returned
| stderr:
|
| stdout:
| <... some data truncated by circular buffer; go to artifacts for details ...>
| nc9(0x19a02c0, 0xc0004e2600, 0x1, 0x2, 0xc00038fa10, 0xc00038fa38)
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 ! /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:457 +0xdf
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !main.wrap.func1(0x19a02c0, 0xc0004e2600, 0x1, 0x2)
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 ! /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:123 +0x6b
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !github.com/spf13/cobra.(*Command).execute(0x19a02c0, 0xc0004e25e0, 0x2, 0x2, 0x19a02c0, 0xc0004e25e0)
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 ! /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:856 +0x2c2
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !github.com/spf13/cobra.(*Command).ExecuteC(0x19a0040, 0x0, 0x0, 0xc0002f6700)
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 ! /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:960 +0x375
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !github.com/spf13/cobra.(*Command).Execute(...)
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 ! /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:897
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !main.main()
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 ! /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1170 +0x26a5
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !****************************************************************************
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !This node experienced a fatal error (printed above), and as a result the
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !process is terminating.
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !Fatal errors can occur due to faulty hardware (disks, memory, clocks) or a
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !problem in CockroachDB. With your help, the support team at Cockroach Labs
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !will try to determine the root cause, recommend next steps, and we can
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !improve CockroachDB based on your report.
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !Please submit a crash report by following the instructions here:
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 ! https://github.com/cockroachdb/cockroach/issues/new/choose
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !If you would rather not post publicly, please contact us directly at:
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 ! support@cockroachlabs.com
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !
| F211117 12:33:10.817461 1 roachprod/install/cluster_synced.go:1777 [-] 2 !The Cockroach Labs team appreciates your feedback.
Wraps: (2) exit status 7
Error types: (1) *cluster.WithCommandDetails (2) *exec.ExitError
Same failure on other branches
- roachtest: tpccbench/nodes=6/cpu=16/multi-az failed [perm denied #72635] #72606 roachtest: tpccbench/nodes=6/cpu=16/multi-az failed [perm denied roachprod: failures due to permission denied on /mnt/data1/cockroach #72635] [C-test-failure O-roachtest O-robot branch-master release-blocker]
This test on roachdash | Improve this report!
Jira issue: CRDB-11328
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.S-1High impact: many users impacted, serious risk of high unavailability or data lossHigh impact: many users impacted, serious risk of high unavailability or data lossT-kvKV TeamKV TeamX-stale