Skip to content

roachtest: cluster_creation failed #78035

@cockroach-teamcity

Description

@cockroach-teamcity

roachtest.cluster_creation failed with artifacts on release-22.1 @ 2d49a7bd0d1ebeff93277233695411d25b2a5d39:

The test failed on branch=release-22.1, cloud=aws:
test kv0/enc=false/nodes=3/cpu=96 was skipped due to 	test_runner.go:585,test_runner.go:260,stopper.go:494: in provider: aws: failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m5d.24xlarge --image-id ami-02fc6052104add5ae --key-name teamcity-runner-38mJKEnGsabTBGqDXCF5gmtmzp4= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-0775beaa2f4c74f1c --tag-specifications ResourceType=instance,Tags=[{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-4594125-1647494291-11-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=Created,Value=2022-03-17T17:32:28Z},{Key=Name,Value=teamcity-4594125-1647494291-11-n4cpu96-0003}] --user-data file:///tmp/aws-startup-script413225093 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping769912648 --output json: stderr: : exit status 254
		(1) attached stack trace
		  -- stack trace:
		  | github.com/cockroachdb/cockroach/pkg/roachprod/vm.ForProvider
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/vm/vm.go:374
		  | [...repeated from below...]
		Wraps: (2) in provider: aws
		Wraps: (3) attached stack trace
		  -- stack trace:
		  | github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws.(*Provider).runCommand
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws/support.go:194
		  | github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws.(*Provider).runJSONCommand
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws/support.go:204
		  | github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws.(*Provider).runInstance
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws/aws.go:969
		  | github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws.(*Provider).Create.func1
		  | 	github.com/cockroachdb/cockroach/pkg/roachprod/vm/aws/aws.go:443
		  | golang.org/x/sync/errgroup.(*Group).Go.func1
		  | 	golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:57
		  | runtime.goexit
		  | 	GOROOT/src/runtime/asm_amd64.s:1581
		Wraps: (4) failed to run: aws ec2 run-instances --associate-public-ip-address --count 1 --instance-type m5d.24xlarge --image-id ami-02fc6052104add5ae --key-name teamcity-runner-38mJKEnGsabTBGqDXCF5gmtmzp4= --region us-east-2 --security-group-ids sg-0319fc9c9599a6145 --subnet-id subnet-0775beaa2f4c74f1c --tag-specifications ResourceType=instance,Tags=[{Key=Roachprod,Value=true},{Key=Cluster,Value=teamcity-4594125-1647494291-11-n4cpu96},{Key=Lifetime,Value=12h0m0s},{Key=Created,Value=2022-03-17T17:32:28Z},{Key=Name,Value=teamcity-4594125-1647494291-11-n4cpu96-0003}] --user-data file:///tmp/aws-startup-script413225093 --iam-instance-profile Name=roachprod-testing --block-device-mapping file:///tmp/aws-block-device-mapping769912648 --output json: stderr: 
		  | An error occurred (InsufficientInstanceCapacity) when calling the RunInstances operation (reached max retries: 2): We currently do not have sufficient m5d.24xlarge capacity in the Availability Zone you requested (us-east-2b). Our system will be working on provisioning additional capacity. You can currently get m5d.24xlarge capacity by not specifying an Availability Zone in your request or choosing us-east-2a, us-east-2c.
		Wraps: (5) exit status 254
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *exec.ExitError
Help

See: roachtest README

See: How To Investigate (internal)

/cc @cockroachdb/dev-inf

This test on roachdash | Improve this report!

Jira issue: CRDB-13932

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-test-failureBroken test (automatically or manually discovered).O-roachtestO-robotOriginated from a bot.T-testengTestEng Teambranch-release-22.1Used to mark GA and release blockers, technical advisories, and bugs for 22.1

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions