Default value for S3 checksumValidationEnabled is incompatible with non-AWS implementations #1974
Description
The SDK recently added checksumValidationEnabled to getObject which apparently does some kind of MD5 computation outside the normal Content-MD5 header. This default fails on non-AWS implementations.
Expected Behavior
This is a new feature and the default value should work on non-AWS implementations.
Current Behavior
The current default value fails against S3Proxy: gaul/s3proxy#327
[s3proxy] D 07-29 06:18:10.155 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:287 |::] request: Request(GET /somebucket/somefile.txt)@16a79473
[s3proxy] D 07-29 06:18:10.155 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:312 |::] header: Authorization: AWS4-HMAC-SHA256 Credential=access_key/20200729/ap-northeast-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-retry;host;x-amz-content-sha256;x-amz-date;x-amz-te, Signature=6d547b2ffbd61b017ee70510e42b040b5e69d2273e305e2608fdbb5cc9be8002
[s3proxy] D 07-29 06:18:10.155 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:312 |::] header: x-amz-content-sha256: UNSIGNED-PAYLOAD
[s3proxy] D 07-29 06:18:10.155 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:312 |::] header: X-Amz-Date: 20200729T061810Z
[s3proxy] D 07-29 06:18:10.155 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:312 |::] header: x-amz-te: append-md5
[s3proxy] D 07-29 06:18:10.156 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:312 |::] header: User-Agent: aws-sdk-java/2.10.65 Linux/3.10.0-1062.12.1.el7.x86_64 OpenJDK_64-Bit_Server_VM/11.0.5+10 Java/11.0.5 vendor/Oracle_Corporation io/sync http/Apache
[s3proxy] D 07-29 06:18:10.156 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:312 |::] header: Connection: Keep-Alive
[s3proxy] D 07-29 06:18:10.156 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:312 |::] header: Host: s3proxy-internal
[s3proxy] D 07-29 06:18:10.156 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:312 |::] header: amz-sdk-invocation-id: fb3cd79c-2993-b30d-d95b-7190263c329b
[s3proxy] D 07-29 06:18:10.156 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:312 |::] header: amz-sdk-retry: 0/0/500
[s3proxy] E 07-29 06:18:10.156 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:622 |::] Unknown header x-amz-te with URI /somebucke/somefile.txt
[s3proxy] D 07-29 06:18:10.156 S3Proxy-Jetty-20 o.gaul.s3proxy.S3ProxyHandler:2862 |::] sendSimpleErrorResponse: 501 NotImplemented A header you provided implies functionality that is not implemented. {}
Steps to Reproduce
public static void main(String[] args) throws Exception {
final AwsCredentialsProvider provider = new AwsCredentialsProvider() {
@Override
public AwsCredentials resolveCredentials() {
return new AwsCredentials() {
@Override
public String secretAccessKey() {
return "secret_access_key";
}
@Override
public String accessKeyId() {
return "access_key";
}
};
}
};
S3Client client = S3Client.builder()
.serviceConfiguration(b -> {
b.pathStyleAccessEnabled(true);
})
.credentialsProvider(provider)
.region(Region.of("s3proxy"))
.endpointOverride(new URI("http://s3proxy"))
.build();
ResponseBytes<GetObjectResponse> response =
client.getObject(GetObjectRequest.builder()
.bucket("somebucket")
.key("somefile.txt")
.build(),
ResponseTransformer.toBytes());
Possible Solution
Change the default from enabled to disabled.
Context
Issue getObject against non-AWS S3.
Your Environment
s3client version 2.10