Skip to content

roachtest: cluster_creation failed #78601

@cockroach-teamcity

Description

@cockroach-teamcity

roachtest.cluster_creation failed with artifacts on master @ 32b45c4bcf1ab41f0ba3abd36cb670eea7f450fd:

The test failed on branch=master, cloud=aws:
test tpccbench/nodes=3/cpu=16 was skipped due to 	test_runner.go:585,test_runner.go:260,stopper.go:494: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c5d.4xlarge --image-id ami-02fc6052104add5ae --key-name teamcity-runner-K9sxDtvNHA4eytNlf-coAmkqWJs= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-0775beaa2f4c74f1c --tag-specifications ResourceType=instance,Tags=[{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-4698278-1648444789-10-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=Created,Value=2022-03-28T12:48:24Z},{Key=Name,Value=teamcity-4698278-1648444789-10-n4cpu16-0002}] --user-data file:///tmp/aws-startup-script3706005379 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping4036371810 --output json: stderr: : exit status 254
		(1) attached stack trace
		  -- stack trace:
		  | github.com/cockroachdb/cockroach/pkg/roachprod/vm.ForProvider
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/vm/vm.go:374
		  | [...repeated from below...]
		Wraps: (2) in provider: aws
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws.(*Provider).runCommand
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws/support.go:194
		  | github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws.(*Provider).runJSONCommand
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws/support.go:204
		  | github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws.(*Provider).runInstance
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws/aws.go:969
		  | github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws.(*Provider).Create.func1
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws/aws.go:443
		  | golang.org/x/sync/errgroup.(*Group).Go.func1
		  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:57
		  | runtime.goexit
		  | 	GOROOT/src/runtime/asm_amd64.s:1581
		Wraps: (4) failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type c5d.4xlarge --image-id ami-02fc6052104add5ae --key-name teamcity-runner-K9sxDtvNHA4eytNlf-coAmkqWJs= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-0775beaa2f4c74f1c --tag-specifications ResourceType=instance,Tags=[{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-4698278-1648444789-10-n4cpu16},{Key=Lifetime,Value=12h0m0s},{Key=Created,Value=2022-03-28T12:48:24Z},{Key=Name,Value=teamcity-4698278-1648444789-10-n4cpu16-0002}] --user-data file:///tmp/aws-startup-script3706005379 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping4036371810 --output json: stderr: 
		  | An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient c5d.4xlarge capacity in the Availability Zone you requested (us-east-2b). Our system will be working on provisioning additional capacity. You can currently get c5d.4xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2c.
		Wraps: (5) exit status 254
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *exec.ExitError
Help

See: roachtest README

See: How To Investigate (internal)

Same failure on other branches

/cc @cockroachdb/dev-inf

This test on roachdash | Improve this report!

Jira issue: CRDB-14193

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions