-
Notifications
You must be signed in to change notification settings - Fork 4.1k
roachtest: tpccbench/nodes=3/cpu=4 failed #55542
Copy link
Copy link
Closed
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.release-blockerIndicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.
Milestone
Description
(roachtest).tpccbench/nodes=3/cpu=4 failed on release-19.2@3e9adba8b62663b9e3521faad03a67f91fb3fc7a:
| W201014 08:03:47.241164 1 workload/cli/run.go:365 retrying after error while creating load: failed to initialize the load generator: could not scatter ranges: Couldn't exec "ALTER TABLE warehouse SCATTER": dial tcp 10.128.0.41:26257: connect: connection refused
| W201014 08:03:49.220047 1 workload/cli/run.go:365 retrying after error while creating load: failed to initialize the load generator: could not scatter ranges: Couldn't exec "ALTER TABLE warehouse SCATTER": dial tcp 10.128.0.41:26257: connect: connection refused
| W201014 08:03:51.311319 1 workload/cli/run.go:365 retrying after error while creating load: failed to initialize the load generator: could not scatter ranges: Couldn't exec "ALTER TABLE warehouse SCATTER": dial tcp 10.128.0.41:26257: connect: connection refused
| W201014 08:03:53.513543 1 workload/cli/run.go:365 retrying after error while creating load: failed to initialize the load generator: could not scatter ranges: Couldn't exec "ALTER TABLE warehouse SCATTER": dial tcp 10.128.0.41:26257: connect: connection refused
| W201014 08:03:55.387360 1 workload/cli/run.go:365 retrying after error while creating load: failed to initialize the load generator: could not scatter ranges: Couldn't exec "ALTER TABLE warehouse SCATTER": dial tcp 10.128.0.41:26257: connect: connection refused
| W201014 08:03:57.157097 1 workload/cli/run.go:365 retrying after error while creating load: failed to initialize the load generator: could not scatter ranges: Couldn't exec "ALTER TABLE warehouse SCATTER": dial tcp 10.128.0.41:26257: connect: connection refused
| W201014 08:03:59.347102 1 workload/cli/run.go:365 retrying after error while creating load: failed to initialize the load generator: could not scatter ranges: Couldn't exec "ALTER TABLE warehouse SCATTER": dial tcp 10.128.0.41:26257: connect: connection refused
| W201014 08:04:01.466075 1 workload/cli/run.go:365 retrying after error while creating load: failed to initialize the load generator: could not scatter ranges: Couldn't exec "ALTER TABLE warehouse SCATTER": dial tcp 10.128.0.41:26257: connect: connection refused
| W201014 08:04:03.451005 1 workload/cli/run.go:365 retrying after error while creating load: failed to initialize the load generator: could not scatter ranges: Couldn't exec "ALTER TABLE warehouse SCATTER": dial tcp 10.128.0.41:26257: connect: connection refused
| E201014 08:04:03.730917 1 workload/cli/run.go:379 Attempt to create load generator failed. It's been more than 1h0m0s since we started trying to create the load generator so we're giving up. Last failure: failed to initialize the load generator: could not scatter ranges: Couldn't exec "ALTER TABLE warehouse SCATTER": dial tcp 10.128.0.41:26257: connect: connection refused
| Error: failed to initialize the load generator: could not scatter ranges: Couldn't exec "ALTER TABLE warehouse SCATTER": dial tcp 10.128.0.41:26257: connect: connection refused
| Error: COMMAND_PROBLEM: exit status 1
| (1) COMMAND_PROBLEM
| Wraps: (2) Node 4. Command with error:
| | ```
| | ./workload run tpcc --warehouses=1000 --active-warehouses=615 --tolerate-errors --scatter --ramp=5m0s --duration=10m0s {pgurl:1-3} --histograms=perf/warehouses=615/stats.json
| | ```
| Wraps: (3) exit status 1
| Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
|
| stdout:
Wraps: (10) exit status 20
Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.withPrefix (7) *withstack.withStack (8) *errutil.withPrefix (9) *main.withCommandDetails (10) *exec.ExitError
cluster.go:1657,context.go:135,cluster.go:1646,test_runner.go:836: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-2363106-1602654685-06-n4cpu4 --oneshot --ignore-empty-nodes: exit status 1 4: skipped
3: 25462
1: 28491
2: dead
Error: UNCLASSIFIED_PROBLEM: 2: dead
(1) UNCLASSIFIED_PROBLEM
Wraps: (2) attached stack trace
-- stack trace:
| main.glob..func14
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1143
| main.wrap.func1
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:267
| github.com/spf13/cobra.(*Command).execute
| /home/agent/work/.go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:830
| github.com/spf13/cobra.(*Command).ExecuteC
| /home/agent/work/.go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:914
| github.com/spf13/cobra.(*Command).Execute
| /home/agent/work/.go/pkg/mod/github.com/spf13/cobra@v0.0.5/command.go:864
| main.main
| /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1839
| runtime.main
| /usr/local/go/src/runtime/proc.go:203
| runtime.goexit
| /usr/local/go/src/runtime/asm_amd64.s:1357
Wraps: (3) 2: dead
Error types: (1) errors.Unclassified (2) *withstack.withStack (3) *errutil.leafError
More
Artifacts: /tpccbench/nodes=3/cpu=4
See this test on roachdash
powered by pkg/cmd/internal/issues
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-test-failureBroken test (automatically or manually discovered).Broken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.Originated from a bot.release-blockerIndicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.