-
Notifications
You must be signed in to change notification settings - Fork 4.1k
roachtest: scaledata/job-coordinator/nodes=3 failed #51822
Copy link
Copy link
Closed
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.branch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.release-blockerIndicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Milestone
Description
(roachtest).scaledata/job-coordinator/nodes=3 failed on master@b8a50cc4d062293915969cdc83e3ec4d057cede5:
The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/scaledata/job-coordinator/nodes=3/run_1
cluster.go:2553,scaledata.go:119,scaledata.go:49,test_runner.go:757: monitor failure: monitor task failed: output in run_060533.883_n4_jobcoordinator_: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2111252-1595484018-14-n4cpu4:4 -- ./job-coordinator --duration_secs=600 --num_workers=16 --cockroach_ip_addresses_csv='10.128.0.61:26257,10.128.0.115:26257,10.128.0.122:26257' --num_jobs_per_worker=8 --job_period_scale_millis=100 returned: exit status 20
(1) attached stack trace
| main.(*monitor).WaitE
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2541
| main.(*monitor).Wait
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2549
| main.runSqlapp
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/scaledata.go:119
| main.registerScaleData.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/scaledata.go:49
| main.(*testRunner).runTest.func2
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:757
Wraps: (2) monitor failure
Wraps: (3) attached stack trace
| main.(*monitor).wait.func2
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2597
Wraps: (4) monitor task failed
Wraps: (5) attached stack trace
| main.(*cluster).RunE
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2201
| main.runSqlapp.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/scaledata.go:115
| main.(*monitor).Go.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2531
| golang.org/x/sync/errgroup.(*Group).Go.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57
| runtime.goexit
| /usr/local/go/src/runtime/asm_amd64.s:1373
Wraps: (6) 2 safe details enclosed
Wraps: (7) output in run_060533.883_n4_jobcoordinator_
Wraps: (8) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2111252-1595484018-14-n4cpu4:4 -- ./job-coordinator --duration_secs=600 --num_workers=16 --cockroach_ip_addresses_csv='10.128.0.61:26257,10.128.0.115:26257,10.128.0.122:26257' --num_jobs_per_worker=8 --job_period_scale_millis=100 returned
| stderr:
| job_id: job_node5_1 instance_id: 600 status: RUNNING nextStatus: SUCCEEDED txResult: 0
| 2020/07/23 06:15:35 Finished run of job instance: &{job_node5_1 600 13 RUNNING 1595484934969957073 1595484935057802264 0xc0008a22f0 1}
| 2020/07/23 06:15:35 Caught error. will sleep: driver: bad connection
| 2020/07/23 06:15:35 Hit timeout while waiting for jobs to succeed. jobSpecs: [{job_node14_0 500 jobConfig0} {job_node14_1 600 jobConfig1} {job_node14_2 700 jobConfig2} {job_node14_3 800 jobConfig3} {job_node14_4 900 jobConfig4} {job_node14_5 500 jobConfig5} {job_node14_6 600 jobConfig6} {job_node14_7 700 jobConfig7}]
| Error: COMMAND_PROBLEM: exit status 1
| (1) COMMAND_PROBLEM
| Wraps: (2) Node 4. Command with error:
| | ```
| | ./job-coordinator --duration_secs=600 --num_workers=16 --cockroach_ip_addresses_csv='10.128.0.61:26257,10.128.0.115:26257,10.128.0.122:26257' --num_jobs_per_worker=8 --job_period_scale_millis=100
| | ```
| Wraps: (3) exit status 1
| Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
|
| stdout:
Wraps: (9) exit status 20
Error types: (1) *withstack.withStack (2) *errutil.withMessage (3) *withstack.withStack (4) *errutil.withMessage (5) *withstack.withStack (6) *safedetails.withSafeDetails (7) *errutil.withMessage (8) *main.withCommandDetails (9) *exec.ExitError
More
Artifacts: /scaledata/job-coordinator/nodes=3
Related:
- roachtest: scaledata/job-coordinator/nodes=3 failed #51700 roachtest: scaledata/job-coordinator/nodes=3 failed C-test-failure O-roachtest O-robot branch-provisional_202007220233_v20.2.0-alpha.2 release-blocker
See this test on roachdash
powered by pkg/cmd/internal/issues
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.branch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.release-blockerIndicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.