Skip to content

Commit f6433e7

Browse files
authored
fix: Fallback to eu-west-1 if a given buckets LocationConstraint is EU (cloudquery#14476)
#### Summary When using the AWS Plugin we noticed that some of our buckets weren't being indexed by Cloudquery. After some investigation we noticed that those buckets seem to have a LocationConstraint of `EU` instead of an expected region such as `eu-west-1` ```bash ➜ aws git:(aa/EU-Region-Buckets) ✗ aws s3api get-bucket-location --bucket (redacted working bucket) --no-paginate { "LocationConstraint": "eu-west-1" } ``` ```bash ➜ aws git:(aa/EU-Region-Buckets) ✗ aws s3api get-bucket-location --bucket (redacted broken bucket) --no-paginate { "LocationConstraint": "EU" } ``` This PR changes how we decide which region to use fetching s3 bucket data, falling back to `eu-west-1` when a buckets LocationConstraint is set to `EU`. #### Why?! It looks like this fallback to `eu-west-1` was a built in behaviour of the V1 AWS SDK https://github.com/aws/aws-sdk-go/blob/592707ce45b91664d30c270ad0057de033ce4dfe/service/s3/bucket_location.go#L23C6-L32 but was not ported to the V2 API which is used for Cloudquery aws/aws-sdk-go-v2#1473 (comment) In the long run we should switch to the [HeadBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadBucket.html) instead, which has this behaviour built in to the AWS API response. <!-- Explain what problem this PR addresses --> <!--
1 parent efa9e34 commit f6433e7

1 file changed

Lines changed: 3 additions & 0 deletions

File tree

  • plugins/source/aws/resources/services/s3

plugins/source/aws/resources/services/s3/buckets.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,9 @@ func resolveS3BucketsAttributes(ctx context.Context, meta schema.ClientMeta, r *
9797
if output != nil && output.LocationConstraint != "" {
9898
resource.Region = string(output.LocationConstraint)
9999
}
100+
if output != nil && output.LocationConstraint == "EU" {
101+
resource.Region = "eu-west-1"
102+
}
100103
var errAll []error
101104

102105
resolvers := []func(context.Context, schema.ClientMeta, *models.WrappedBucket) error{

0 commit comments

Comments
 (0)