Commit f6433e7
authored
#### Summary
When using the AWS Plugin we noticed that some of our buckets weren't being indexed by Cloudquery. After some investigation we noticed that those buckets seem to have a LocationConstraint of `EU` instead of an expected region such as `eu-west-1`
```bash
➜ aws git:(aa/EU-Region-Buckets) ✗ aws s3api get-bucket-location --bucket (redacted working bucket) --no-paginate
{
"LocationConstraint": "eu-west-1"
}
```
```bash
➜ aws git:(aa/EU-Region-Buckets) ✗ aws s3api get-bucket-location --bucket (redacted broken bucket) --no-paginate
{
"LocationConstraint": "EU"
}
```
This PR changes how we decide which region to use fetching s3 bucket data, falling back to `eu-west-1` when a buckets LocationConstraint is set to `EU`.
#### Why?!
It looks like this fallback to `eu-west-1` was a built in behaviour of the V1 AWS SDK https://github.com/aws/aws-sdk-go/blob/592707ce45b91664d30c270ad0057de033ce4dfe/service/s3/bucket_location.go#L23C6-L32 but was not ported to the V2 API which is used for Cloudquery aws/aws-sdk-go-v2#1473 (comment)
In the long run we should switch to the [HeadBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadBucket.html) instead, which has this behaviour built in to the AWS API response.
<!--
Explain what problem this PR addresses
-->
<!--
1 parent efa9e34 commit f6433e7
1 file changed
Lines changed: 3 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
97 | 97 | | |
98 | 98 | | |
99 | 99 | | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
100 | 103 | | |
101 | 104 | | |
102 | 105 | | |
| |||
0 commit comments