Bug #73626
openrgw/cloud-transition: Transitioning data to AWS non-default region is failing
0%
Description
Description of problem:
Transitioning the ceph data to Amazon s3 is failing. CLOUDTIER is configured to transit the RGW objects to AWS s3 cloud, we verified the region, end_points, access& secret key. But still we see the following errors for corresponding configs.
Bucket exists on aws s3
==============================================================
Config:
"endpoint": "http://s3.amazonaws.com",
"access_key": "xxx",
"secret": "xxx",
"region": "eu-west-1",
"host_style": "path",
"target_storage_class": "",
"target_path": "axdeseccs",
Error:
2025-10-03T10:01:13.433+0000 7fd6ac58b640 0 lifecycle: ERROR: Creating target bucket failed with error: AuthorizationHeaderMalformed
==============================================================
Config:
"endpoint": "http://axdeseccs.s3.amazonaws.com",
"access_key": "xxx",
"secret": "xxx",
"region": "eu-west-1",
"host_style": "path",
"target_storage_class": "",
"target_path": "axdeseccs",
Error:
2025-10-03T10:03:44.979+0000 7f5c1c2f7640 0 lifecycle: ERROR: Creating target bucket failed with error: AccessDenied
==============================================================
Config:
"endpoint": "http://s3.eu-west-1.amazonaws.com",
"access_key": "xxx",
"secret": "xxx",
"region": "eu-west-1",
"host_style": "path",
"target_storage_class": "",
"target_path": "axdeseccs",
Error:
2025-10-03T10:07:37.834+0000 7fbe6b3d7640 0 lifecycle: ERROR: Creating target bucket failed with error: IllegalLocationConstraintException
It works when using virtual hosting URL though -
eg.,
"tier_targets": [
{
"key": "CLOUDTIER",
"val": {
"tier_type": "cloud-s3",
"storage_class": "CLOUDTIER",
"retain_head_object": true,
"s3": {
"endpoint": "http://josgutie-rgw-transition-test-02.s3-eu-west-1.amazonaws.com",
"access_key": "XXXX",
"secret": "XXXXXX",
"region": "eu-west-1",
"host_style": "path",
"target_storage_class": "",
"target_path": "josgutie-rgw-transition-test-02",
"acl_mappings": [],
"multipart_sync_threshold": 44432,
"multipart_min_part_size": 44432
},
"allow_read_through": false,
"read_through_restore_days": 1,
"restore_storage_class": "STANDARD"
}
Updated by Soumya Koduri 7 months ago
AWS expects `LocationConstraint` to be passed as an additional parameter while creating bucket to non-default region which is missing currently in the code,
From https://docs.aws.amazon.com/cli/latest/reference/s3api/create-bucket.html,
"Regions outside of us-east-1 require the appropriate LocationConstraint to be specified in order to create the bucket in the desired region."
Updated by Soumya Koduri 7 months ago
- Status changed from In Progress to Fix Under Review
- Pull request ID set to 66038
Updated by Upkeep Bot 6 months ago
- Status changed from Fix Under Review to Resolved
- Merge Commit set to 7941054bf47aa82da47319e1966152218058b291
- Fixed In set to v20.3.0-4079-g7941054bf4
- Upkeep Timestamp set to 2025-11-11T08:31:42+00:00
Updated by Soumya Koduri 6 months ago
- Status changed from Resolved to Pending Backport
- Backport set to tentacle, squid, reef
Updated by Upkeep Bot 6 months ago
- Copied to Backport #73783: reef: rgw/cloud-transition: Transitioning data to AWS non-default region is failing added
Updated by Upkeep Bot 6 months ago
- Copied to Backport #73784: tentacle: rgw/cloud-transition: Transitioning data to AWS non-default region is failing added
Updated by Upkeep Bot 6 months ago
- Copied to Backport #73785: squid: rgw/cloud-transition: Transitioning data to AWS non-default region is failing added