Project

General

Profile

Actions

Bug #73626

open

rgw/cloud-transition: Transitioning data to AWS non-default region is failing

Added by Soumya Koduri 7 months ago. Updated 8 days ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Backport:
tentacle, squid, reef
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Tags (freeform):
backport_processed
Fixed In:
v20.3.0-4079-g7941054bf4
Released In:
Upkeep Timestamp:
2025-11-11T08:31:42+00:00

Description

Description of problem:

Transitioning the ceph data to Amazon s3 is failing. CLOUDTIER is configured to transit the RGW objects to AWS s3 cloud, we verified the region, end_points, access& secret key. But still we see the following errors for corresponding configs.

Bucket exists on aws s3 ==============================================================
Config:
"endpoint": "http://s3.amazonaws.com",
"access_key": "xxx",
"secret": "xxx",
"region": "eu-west-1",
"host_style": "path",
"target_storage_class": "",
"target_path": "axdeseccs",
Error:

2025-10-03T10:01:13.433+0000 7fd6ac58b640 0 lifecycle: ERROR: Creating target bucket failed with error: AuthorizationHeaderMalformed

==============================================================
Config:
"endpoint": "http://axdeseccs.s3.amazonaws.com",
"access_key": "xxx",
"secret": "xxx",
"region": "eu-west-1",
"host_style": "path",
"target_storage_class": "",
"target_path": "axdeseccs",
Error:

2025-10-03T10:03:44.979+0000 7f5c1c2f7640 0 lifecycle: ERROR: Creating target bucket failed with error: AccessDenied

==============================================================
Config:
"endpoint": "http://s3.eu-west-1.amazonaws.com",
"access_key": "xxx",
"secret": "xxx",
"region": "eu-west-1",
"host_style": "path",
"target_storage_class": "",
"target_path": "axdeseccs",
Error:

2025-10-03T10:07:37.834+0000 7fbe6b3d7640 0 lifecycle: ERROR: Creating target bucket failed with error: IllegalLocationConstraintException

It works when using virtual hosting URL though -

eg.,
"tier_targets": [ {
"key": "CLOUDTIER",
"val": {
"tier_type": "cloud-s3",
"storage_class": "CLOUDTIER",
"retain_head_object": true,
"s3": {
"endpoint": "http://josgutie-rgw-transition-test-02.s3-eu-west-1.amazonaws.com",
"access_key": "XXXX",
"secret": "XXXXXX",
"region": "eu-west-1",
"host_style": "path",
"target_storage_class": "",
"target_path": "josgutie-rgw-transition-test-02",
"acl_mappings": [],
"multipart_sync_threshold": 44432,
"multipart_min_part_size": 44432
},
"allow_read_through": false,
"read_through_restore_days": 1,
"restore_storage_class": "STANDARD"
}


Related issues 3 (3 open0 closed)

Copied to rgw - Backport #73783: reef: rgw/cloud-transition: Transitioning data to AWS non-default region is failingNewSoumya KoduriActions
Copied to rgw - Backport #73784: tentacle: rgw/cloud-transition: Transitioning data to AWS non-default region is failingNewSoumya KoduriActions
Copied to rgw - Backport #73785: squid: rgw/cloud-transition: Transitioning data to AWS non-default region is failingNewSoumya KoduriActions
Actions #1

Updated by Soumya Koduri 7 months ago

AWS expects `LocationConstraint` to be passed as an additional parameter while creating bucket to non-default region which is missing currently in the code,

From https://docs.aws.amazon.com/cli/latest/reference/s3api/create-bucket.html,

"Regions outside of us-east-1 require the appropriate LocationConstraint to be specified in order to create the bucket in the desired region."

Actions #2

Updated by Soumya Koduri 7 months ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 66038
Actions #3

Updated by Upkeep Bot 6 months ago

  • Status changed from Fix Under Review to Resolved
  • Merge Commit set to 7941054bf47aa82da47319e1966152218058b291
  • Fixed In set to v20.3.0-4079-g7941054bf4
  • Upkeep Timestamp set to 2025-11-11T08:31:42+00:00
Actions #4

Updated by Soumya Koduri 6 months ago

  • Status changed from Resolved to Pending Backport
  • Backport set to tentacle, squid, reef
Actions #5

Updated by Upkeep Bot 6 months ago

  • Copied to Backport #73783: reef: rgw/cloud-transition: Transitioning data to AWS non-default region is failing added
Actions #6

Updated by Upkeep Bot 6 months ago

  • Copied to Backport #73784: tentacle: rgw/cloud-transition: Transitioning data to AWS non-default region is failing added
Actions #7

Updated by Upkeep Bot 6 months ago

  • Copied to Backport #73785: squid: rgw/cloud-transition: Transitioning data to AWS non-default region is failing added
Actions #8

Updated by Upkeep Bot 6 months ago

  • Tags (freeform) set to backport_processed
Actions #9

Updated by Soumya Koduri 8 days ago

  • Tags set to cloud-transition, cloud-restore
Actions

Also available in: Atom PDF