Skip to content

rgw/cksum: GetObject omits checksum headers for Range requests#61901

Merged
cbodley merged 1 commit intoceph:mainfrom
cbodley:wip-69936
Feb 28, 2025
Merged

rgw/cksum: GetObject omits checksum headers for Range requests#61901
cbodley merged 1 commit intoceph:mainfrom
cbodley:wip-69936

Conversation

@cbodley
Copy link
Copy Markdown
Contributor

@cbodley cbodley commented Feb 19, 2025

if we send response headers containing full-object checksums, clients will compare them with the returned data and fail with:

botocore.exceptions.FlexibleChecksumError: Expected checksum 4AAr8A== did not match calculated checksum: WbxXZw==

the only thing aws docs say about Range requests refers to multipart uploads:

For completed uploads, you can get an individual part's checksum by using the GetObject or HeadObject operations and specifying a part number or byte range that aligns with a single part.

Fixes: https://tracker.ceph.com/issues/69936

Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

@cbodley cbodley requested a review from a team as a code owner February 19, 2025 15:40
@github-actions github-actions Bot added the rgw label Feb 19, 2025
Copy link
Copy Markdown
Contributor

@mattbenjamin mattbenjamin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for running this down, god be praised

@cbodley
Copy link
Copy Markdown
Contributor Author

cbodley commented Feb 19, 2025

will test against ceph/s3-tests#621 which reenables the relevant s3-tests

if we send response headers containing full-object checksums, clients
will compare them with the returned data and fail with:

> botocore.exceptions.FlexibleChecksumError: Expected checksum 4AAr8A== did not match calculated checksum: WbxXZw==

the only thing aws docs[1] say about Range requests refers to multipart
uploads:

> For completed uploads, you can get an individual part's checksum by using the GetObject or HeadObject operations and specifying a part number or byte range that aligns with a single part.

Fixes: https://tracker.ceph.com/issues/69936

[1] https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html#Part-level-checksums

Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley
Copy link
Copy Markdown
Contributor Author

cbodley commented Feb 25, 2025

qa failed on new test case test_verify_remove_client_id_from_oidc, so i'll need to rerun against the rebase

@cbodley
Copy link
Copy Markdown
Contributor Author

cbodley commented Feb 26, 2025

@cbodley
Copy link
Copy Markdown
Contributor Author

cbodley commented Feb 26, 2025

jenkins test make check arm64

@ivancich
Copy link
Copy Markdown
Member

jenkins test make check arm64

@cbodley cbodley merged commit 7307a3d into ceph:main Feb 28, 2025
@cbodley cbodley deleted the wip-69936 branch February 28, 2025 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants