Update aws sdk to 1.10.69 and add use_throttle_retries repository setting by dadoonet · Pull Request #17784 · elastic/elasticsearch

dadoonet · 2016-04-15T12:58:45Z

This PR brings 2 changes:

Upgrade to AWS SDK 1.10.69
Add repositories.s3.use_throttle_retries setting

Upgrade to AWS SDK 1.10.69

Release notes highlights from 1.10.33 for the SDK

Moving from JSON.org to Jackson for request marshallers.
The Java SDK now supports retry throttling to limit the rate of retries during periods of reduced availability. This throttling behavior can be enabled via ClientConfiguration or via the system property "-Dcom.amazonaws.sdk.enableThrottledRetry".
Fixed String case conversion issues when running with non English locales.
AWS SDK for Java introduces a new dynamic endpoint system that can compute endpoints for services in new regions.
Introducing a new AWS region, ap-northeast-2.
Added a new metric, HttpSocketReadTime, that records socket read latency. You can enable this metric by adding enableHttpSocketReadMetric to the system property com.amazonaws.sdk.enableDefaultMetrics. For more information, see Enabling Metrics with the AWS SDK for Java.
New Client Execution timeout feature to set a limit spent across retries, backoffs, ummarshalling, etc. This new timeout can be specified at the client level or per request.
Also included in this release is the ability to specify the existing HTTP Request timeout per request rather than just per client.

Release notes highlights from 1.10.33 for S3

Added support for RequesterPays for all operations.
Ignore the 'Connection' header when generating S3 responses.
Allow users to generate an AmazonS3URI from a string without using URL encoding.
Fixed issue that prevented creating buckets when using a client configured for the s3-external-1 endpoint.
Amazon S3 bucket lifecycle configuration supports two new features: the removal of expired object delete markers and an action to abort incomplete multipart uploads.
Allow TransferManagerConfiguration to accept integer values for multipart upload threshold.
Copy the list of ETags before sorting Copy the list of ETags before sorting aws/aws-sdk-java#589.
Option to disable chunked encoding Option to disable chunked encoding aws/aws-sdk-java#586.
Adding retry on InternalErrors in CompleteMultipartUpload operation. Add Default Retry on InternalError during CompleteMultipartUpload aws/aws-sdk-java#538
Deprecated two APIs : AmazonS3#changeObjectStorageClass and AmazonS3#setObjectRedirectLocation.
Added support for the aws-exec-read canned ACL. Owner gets FULL_CONTROL. Amazon EC2 gets READ access to GET an Amazon Machine Image (AMI) bundle from Amazon S3.

Release notes highlights from 1.10.33 for EC2

Added support for referencing security groups in peered Virtual Private Clouds (VPCs). For more information see the service announcement at https://aws.amazon.com/about-aws/whats-new/2016/03/announcing-support-for-security-group-references-in-a-peered-vpc/ .
Fixed a bug in AWS SDK for Java - Amazon EC2 module that returns NPE for dry run requests.
Regenerated client with new implementation of code generator.
This feature enables support for DNS resolution of public hostnames to private IP addresses when queried over ClassicLink. Additionally, you can now access private hosted zones associated with your VPC from a linked EC2-Classic instance. ClassicLink DNS support makes it easier for EC2-Classic instances to communicate with VPC resources using public DNS hostnames.
You can now use Network Address Translation (NAT) Gateway, a highly available AWS managed service that makes it easy to connect to the Internet from instances within a private subnet in an AWS Virtual Private Cloud (VPC). Previously, you needed to launch a NAT instance to enable NAT for instances in a private subnet. Amazon VPC NAT Gateway is available in the US East (N. Virginia), US West (Oregon), US West (N. California), EU (Ireland), Asia Pacific (Tokyo), Asia Pacific (Singapore), and Asia Pacific (Sydney) regions. To learn more about Amazon VPC NAT, see New - Managed NAT (Network Address Translation) Gateway for AWS
A default read timeout is now applied when querying data from EC2 metadata service.

Add `use_throttle_retries` repository setting

Defaults to false (AWS SDK default value).
If anyone is having trouble, you could enable it with repositories.s3.use_throttle_retries: true in elasticsearch.yml file or set it per repository.

* Moving from JSON.org to Jackson for request marshallers. * The Java SDK now supports retry throttling to limit the rate of retries during periods of reduced availability. This throttling behavior can be enabled via ClientConfiguration or via the system property "-Dcom.amazonaws.sdk.enableThrottledRetry". * Fixed String case conversion issues when running with non English locales. * AWS SDK for Java introduces a new dynamic endpoint system that can compute endpoints for services in new regions. * Introducing a new AWS region, ap-northeast-2. * Added a new metric, HttpSocketReadTime, that records socket read latency. You can enable this metric by adding enableHttpSocketReadMetric to the system property com.amazonaws.sdk.enableDefaultMetrics. For more information, see [Enabling Metrics with the AWS SDK for Java](https://java.awsblog.com/post/Tx3C0RV4NRRBKTG/Enabling-Metrics-with-the-AWS-SDK-for-Java). * New Client Execution timeout feature to set a limit spent across retries, backoffs, ummarshalling, etc. This new timeout can be specified at the client level or per request. Also included in this release is the ability to specify the existing HTTP Request timeout per request rather than just per client. * Added support for RequesterPays for all operations. * Ignore the 'Connection' header when generating S3 responses. * Allow users to generate an AmazonS3URI from a string without using URL encoding. * Fixed issue that prevented creating buckets when using a client configured for the s3-external-1 endpoint. * Amazon S3 bucket lifecycle configuration supports two new features: the removal of expired object delete markers and an action to abort incomplete multipart uploads. * Allow TransferManagerConfiguration to accept integer values for multipart upload threshold. * Copy the list of ETags before sorting aws/aws-sdk-java#589. * Option to disable chunked encoding aws/aws-sdk-java#586. * Adding retry on InternalErrors in CompleteMultipartUpload operation. aws/aws-sdk-java#538 * Deprecated two APIs : AmazonS3#changeObjectStorageClass and AmazonS3#setObjectRedirectLocation. * Added support for the aws-exec-read canned ACL. Owner gets FULL_CONTROL. Amazon EC2 gets READ access to GET an Amazon Machine Image (AMI) bundle from Amazon S3. * Added support for referencing security groups in peered Virtual Private Clouds (VPCs). For more information see the service announcement at https://aws.amazon.com/about-aws/whats-new/2016/03/announcing-support-for-security-group-references-in-a-peered-vpc/ . * Fixed a bug in AWS SDK for Java - Amazon EC2 module that returns NPE for dry run requests. * Regenerated client with new implementation of code generator. * This feature enables support for DNS resolution of public hostnames to private IP addresses when queried over ClassicLink. Additionally, you can now access private hosted zones associated with your VPC from a linked EC2-Classic instance. ClassicLink DNS support makes it easier for EC2-Classic instances to communicate with VPC resources using public DNS hostnames. * You can now use Network Address Translation (NAT) Gateway, a highly available AWS managed service that makes it easy to connect to the Internet from instances within a private subnet in an AWS Virtual Private Cloud (VPC). Previously, you needed to launch a NAT instance to enable NAT for instances in a private subnet. Amazon VPC NAT Gateway is available in the US East (N. Virginia), US West (Oregon), US West (N. California), EU (Ireland), Asia Pacific (Tokyo), Asia Pacific (Singapore), and Asia Pacific (Sydney) regions. To learn more about Amazon VPC NAT, see [New - Managed NAT (Network Address Translation) Gateway for AWS](https://aws.amazon.com/blogs/aws/new-managed-nat-network-address-translation-gateway-for-aws/) * A default read timeout is now applied when querying data from EC2 metadata service.

Defaults to `true`. If anyone is having trouble with this option, you could disable it with `cloud.aws.s3.throttle_retries: false` in `elasticsearch.yml` file.

dadoonet · 2016-04-28T14:04:26Z

@tlrx Could you review this change please?

dadoonet · 2016-04-28T14:24:13Z

plugins/discovery-ec2/src/test/java/org/elasticsearch/discovery/ec2/AmazonEC2Mock.java

    }

+    @Override
+    public DeleteNatGatewayResult deleteNatGateway(DeleteNatGatewayRequest deleteNatGatewayRequest) {


Actually, I should here also send throw new UnsupportedOperationException("Not supported in mock");

I'll update the PR

This PR brings 2 changes: * Upgrade to AWS SDK 1.10.69 * Add cloud.aws.s3.throttle_retries setting # Upgrade to AWS SDK 1.10.69 ## Release notes highlights from 1.10.33 for the SDK * Moving from JSON.org to Jackson for request marshallers. * The Java SDK now supports retry throttling to limit the rate of retries during periods of reduced availability. This throttling behavior can be enabled via ClientConfiguration or via the system property "-Dcom.amazonaws.sdk.enableThrottledRetry". * Fixed String case conversion issues when running with non English locales. * AWS SDK for Java introduces a new dynamic endpoint system that can compute endpoints for services in new regions. * Introducing a new AWS region, ap-northeast-2. * Added a new metric, HttpSocketReadTime, that records socket read latency. You can enable this metric by adding enableHttpSocketReadMetric to the system property com.amazonaws.sdk.enableDefaultMetrics. For more information, see [Enabling Metrics with the AWS SDK for Java](https://java.awsblog.com/post/Tx3C0RV4NRRBKTG/Enabling-Metrics-with-the-AWS-SDK-for-Java). * New Client Execution timeout feature to set a limit spent across retries, backoffs, ummarshalling, etc. This new timeout can be specified at the client level or per request. Also included in this release is the ability to specify the existing HTTP Request timeout per request rather than just per client. ## Release notes highlights from 1.10.33 for S3 * Added support for RequesterPays for all operations. * Ignore the 'Connection' header when generating S3 responses. * Allow users to generate an AmazonS3URI from a string without using URL encoding. * Fixed issue that prevented creating buckets when using a client configured for the s3-external-1 endpoint. * Amazon S3 bucket lifecycle configuration supports two new features: the removal of expired object delete markers and an action to abort incomplete multipart uploads. * Allow TransferManagerConfiguration to accept integer values for multipart upload threshold. * Copy the list of ETags before sorting aws/aws-sdk-java#589. * Option to disable chunked encoding aws/aws-sdk-java#586. * Adding retry on InternalErrors in CompleteMultipartUpload operation. aws/aws-sdk-java#538 * Deprecated two APIs : AmazonS3#changeObjectStorageClass and AmazonS3#setObjectRedirectLocation. * Added support for the aws-exec-read canned ACL. Owner gets FULL_CONTROL. Amazon EC2 gets READ access to GET an Amazon Machine Image (AMI) bundle from Amazon S3. ## Release notes highlights from 1.10.33 for EC2 * Added support for referencing security groups in peered Virtual Private Clouds (VPCs). For more information see the service announcement at https://aws.amazon.com/about-aws/whats-new/2016/03/announcing-support-for-security-group-references-in-a-peered-vpc/ . * Fixed a bug in AWS SDK for Java - Amazon EC2 module that returns NPE for dry run requests. * Regenerated client with new implementation of code generator. * This feature enables support for DNS resolution of public hostnames to private IP addresses when queried over ClassicLink. Additionally, you can now access private hosted zones associated with your VPC from a linked EC2-Classic instance. ClassicLink DNS support makes it easier for EC2-Classic instances to communicate with VPC resources using public DNS hostnames. * You can now use Network Address Translation (NAT) Gateway, a highly available AWS managed service that makes it easy to connect to the Internet from instances within a private subnet in an AWS Virtual Private Cloud (VPC). Previously, you needed to launch a NAT instance to enable NAT for instances in a private subnet. Amazon VPC NAT Gateway is available in the US East (N. Virginia), US West (Oregon), US West (N. California), EU (Ireland), Asia Pacific (Tokyo), Asia Pacific (Singapore), and Asia Pacific (Sydney) regions. To learn more about Amazon VPC NAT, see [New - Managed NAT (Network Address Translation) Gateway for AWS](https://aws.amazon.com/blogs/aws/new-managed-nat-network-address-translation-gateway-for-aws/) * A default read timeout is now applied when querying data from EC2 metadata service. # Add `cloud.aws.s3.throttle_retries` setting Defaults to `true`. If anyone is having trouble with this option, you could disable it with `cloud.aws.s3.throttle_retries: false` in `elasticsearch.yml` file. Backport of elastic#17784 in 2.x branch

tlrx · 2016-05-06T07:35:23Z

plugins/repository-s3/src/main/java/org/elasticsearch/cloud/aws/InternalAwsS3Service.java

        if (maxRetries != null) {
            // If not explicitly set, default to 3 with exponential backoff policy
            clientConfiguration.setMaxErrorRetry(maxRetries);
+            clientConfiguration.setUseThrottleRetries(AwsS3Service.CLOUD_S3.THROTTLE_RETRIES_SETTING.get(settings));


Shouldn't this be set independently from the maxRetries setting? I think the default retry policy is 3 and users might want to throttling this too event if the max_retries is not set in the repository settings.

tlrx · 2016-05-06T07:36:01Z

Left a comment

@tlrx

…tory I initially wrongly put this setting under `cloud.aws.s3.` prefix which does not make sense. It should be placed at the same place as `max_retries`. Also applied @tlrx comments. We should set this even if max_retries is not set (when using default values). Also added some documentation about this setting.

dadoonet · 2016-05-19T14:58:55Z

@tlrx I added new commits. I also changed the title and the description of the PR.

tlrx · 2016-05-25T13:45:18Z

docs/plugins/repository-s3.asciidoc


+Note that you can define S3 repository settings for all S3 repositories in `elasticsearch.yml` configuration file.
+They are all prefixed with `repositories.s3.`.
+


Shouldn't we provide a sample here?

tlrx · 2016-05-25T13:52:56Z

Left some more comments

* changes `throttle_retries` to `use_throttle_retries` * removes registering of all individual repository settings when the plugin starts. Not needed * adds more comment about deprecated method in AWS SDK we need to implement though in a Delegate class within our tests

dadoonet · 2016-05-27T08:16:01Z

@tlrx I added a new commit to address your comments. Thanks a lot!

BTW, I'm changing the description of the PR to conform with those new changes.

dadoonet · 2016-06-13T06:24:27Z

ping @tlrx

tlrx · 2016-06-14T11:28:39Z

LGTM

Will create another issue to change that. Related to this discussion: https://github.com/elastic/elasticsearch/pull/17784/files/f4cd3bd3480bf6c0a51faf408d8ea9c8a8733a6b#r67291936

Follow up for elastic#17784 (comment) Today we are registering repository settings when `S3RepositoryPlugin` starts: ```java settingsModule.registerSetting(S3Repository.Repository.KEY_SETTING); settingsModule.registerSetting(S3Repository.Repository.SECRET_SETTING); settingsModule.registerSetting(S3Repository.Repository.BUCKET_SETTING); settingsModule.registerSetting(S3Repository.Repository.ENDPOINT_SETTING); settingsModule.registerSetting(S3Repository.Repository.PROTOCOL_SETTING); settingsModule.registerSetting(S3Repository.Repository.REGION_SETTING); settingsModule.registerSetting(S3Repository.Repository.SERVER_SIDE_ENCRYPTION_SETTING); settingsModule.registerSetting(S3Repository.Repository.BUFFER_SIZE_SETTING); settingsModule.registerSetting(S3Repository.Repository.MAX_RETRIES_SETTING); settingsModule.registerSetting(S3Repository.Repository.CHUNK_SIZE_SETTING); settingsModule.registerSetting(S3Repository.Repository.COMPRESS_SETTING); settingsModule.registerSetting(S3Repository.Repository.STORAGE_CLASS_SETTING); settingsModule.registerSetting(S3Repository.Repository.CANNED_ACL_SETTING); settingsModule.registerSetting(S3Repository.Repository.BASE_PATH_SETTING); ``` We don't need to register those settings as they are repository level settings and not node level settings. Closes elastic#18945.

# AWS Release Notes From 1.10.69 (see elastic#17784), here are the most important updates: ## Minor 1.10 releases: * Amazon S3 Added support for a new configuration named BucketAccelerateConfiguration which supports faster uploads/downloads to S3 buckets. * Adding several missing throttling error codes for API Gateway and S3. * Amazon S3 Introducing a new version of the ListObjects (ListObjectsV2) API that allows listing objects with a large number of delete markers. ## 1.11: AWS SDK for Java: * Improved URL encoding for REST clients. * Dropped usage of Json.org library in favor of Jackson. * Updated retry policies to include jitter during backoffs. * Generate output POJOs for all operations. * Renamed the aws-java-sdk-flow-build-tools-{sdkversion}.jar to aws-swf-build-tools-1.0.jar. The jar is also available in Maven. AWS SDK for Java - Amazon S3: * Added support to return the part count of an object in object metadata. You can also download a part by setting part number in GetObjectRequest. * TransferManager supports parallel downloads for multipart objects. * Default to Signature Version 4 signing process in all regions. Apache HttpClient upgraded to 4.5.2 # Jackson update Note that [Jackson has been updated at some point](https://github.com/aws/aws-sdk-java/blob/master/pom.xml#L115) but as we don't use dependency management anymore it has never been updated and we did not notice. So I updated: * jackson-databind to 2.6.6. * jackson-annotations to 2.6.0. That being said, I'm unsure if we should better use `${versions.jackson}` instead. # Functional changes For `repository-s3` plugin, this new version of the AWS SDK now sets Throttle Retries (overridable by `use_throttle_retries`) to `true` by default instead of `false` previously. # Security manager issues Tests are failing on `repository-s3` plugin with: ``` [2016-07-26 09:36:40,350][WARN ][repositories ] [AUqzQAb] failed to create repository [s3][test_repo_s3_1] java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessDeclaredMembers") at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) at java.security.AccessController.checkPermission(AccessController.java:884) at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) at java.lang.Class.checkMemberAccess(Class.java:2348) at java.lang.Class.getDeclaredConstructors(Class.java:2019) at com.fasterxml.jackson.databind.introspect.AnnotatedClass.resolveCreators(AnnotatedClass.java:338) at com.fasterxml.jackson.databind.introspect.AnnotatedClass.getStaticMethods(AnnotatedClass.java:245) at com.fasterxml.jackson.databind.introspect.BasicBeanDescription.getFactoryMethods(BasicBeanDescription.java:461) at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._addDeserializerFactoryMethods(BasicDeserializerFactory.java:670) at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._constructDefaultValueInstantiator(BasicDeserializerFactory.java:321) at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findValueInstantiator(BasicDeserializerFactory.java:254) at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.createCollectionDeserializer(BasicDeserializerFactory.java:1027) at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:394) at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:352) at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:264) at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244) at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142) at com.fasterxml.jackson.databind.DeserializationContext.findContextualValueDeserializer(DeserializationContext.java:428) at com.fasterxml.jackson.databind.deser.std.StdDeserializer.findDeserializer(StdDeserializer.java:947) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:439) at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:296) at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244) at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142) at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:461) at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3838) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3732) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2796) at com.amazonaws.partitions.PartitionsLoader.loadPartitionFromStream(PartitionsLoader.java:92) at com.amazonaws.partitions.PartitionsLoader.build(PartitionsLoader.java:84) at com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30) at com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:66) at com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:54) at com.amazonaws.regions.RegionUtils.getRegion(RegionUtils.java:107) at com.amazonaws.services.s3.AmazonS3Client.createSigner(AmazonS3Client.java:3288) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3984) at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1218) at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1175) at org.elasticsearch.cloud.aws.blobstore.S3BlobStore.<init>(S3BlobStore.java:88) at org.elasticsearch.repositories.s3.S3Repository.<init>(S3Repository.java:309) at org.elasticsearch.plugin.repository.s3.S3RepositoryPlugin.lambda$getRepositories$6(S3RepositoryPlugin.java:73) at org.elasticsearch.repositories.RepositoriesService.createRepository(RepositoriesService.java:381) at org.elasticsearch.repositories.RepositoriesService.registerRepository(RepositoriesService.java:354) at org.elasticsearch.repositories.RepositoriesService.access$100(RepositoriesService.java:54) at org.elasticsearch.repositories.RepositoriesService$1.execute(RepositoriesService.java:107) at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45) at org.elasticsearch.cluster.service.ClusterService.runTasksForExecutor(ClusterService.java:553) at org.elasticsearch.cluster.service.ClusterService$UpdateTask.run(ClusterService.java:857) at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:450) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:237) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:200) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ``` Related to elastic#18910.

dadoonet added 2 commits April 15, 2016 14:52

Add cloud.aws.s3.throttle_retries setting

44080a0

Defaults to `true`. If anyone is having trouble with this option, you could disable it with `cloud.aws.s3.throttle_retries: false` in `elasticsearch.yml` file.

dadoonet added review v5.0.0-alpha2 labels Apr 15, 2016

clintongormley added the >upgrade label Apr 18, 2016

clintongormley added v5.0.0-alpha3 and removed v5.0.0-alpha2 labels Apr 26, 2016

dadoonet assigned tlrx Apr 28, 2016

dadoonet reviewed Apr 28, 2016
View reviewed changes

This was referenced Apr 28, 2016

Update aws sdk to 1.10.69 and add throttle_retries repository setting #18048

Merged

Update aws sdk to 1.10.69 and add throttle_retries repository setting elastic/elasticsearch-cloud-aws#273

Merged

Add new UnsupportedException for EC Mock

faa3c6e

dadoonet force-pushed the pr/update-aws-sdk branch from 8b97808 to faa3c6e Compare April 29, 2016 12:42

tlrx reviewed May 6, 2016
View reviewed changes

dadoonet added 2 commits May 19, 2016 16:50

Merge branch 'master' into pr/update-aws-sdk

f4cd3bd

dadoonet changed the title ~~Update aws sdk to 1.10.69 and add cloud.aws.s3.throttle_retries setting~~ Update aws sdk to 1.10.69 and add throttle_retries repository setting May 19, 2016

tlrx reviewed May 25, 2016
View reviewed changes

clintongormley added v5.0.0-alpha4 and removed v5.0.0-alpha3 labels May 26, 2016

dadoonet added 2 commits May 27, 2016 10:13

Fix after review

a445654

* changes `throttle_retries` to `use_throttle_retries` * removes registering of all individual repository settings when the plugin starts. Not needed * adds more comment about deprecated method in AWS SDK we need to implement though in a Delegate class within our tests

Merge branch 'master' into pr/update-aws-sdk

623a5b7

dadoonet changed the title ~~Update aws sdk to 1.10.69 and add throttle_retries repository setting~~ Update aws sdk to 1.10.69 and add use_throttle_retries repository setting May 27, 2016

Fix log use_throttle_retries

63622aa

dadoonet mentioned this pull request May 30, 2016

S3 snapshot failure - files not found #18632

Closed

dadoonet added 2 commits June 15, 2016 14:48

Merge branch 'master' into pr/update-aws-sdk

6322392

Fix typo

b036d23

dadoonet mentioned this pull request Jun 16, 2016

Update aws sdk to 1.11.8 #18910

Closed

Revert change about registering Repository settings

82fee9f

Will create another issue to change that. Related to this discussion: https://github.com/elastic/elasticsearch/pull/17784/files/f4cd3bd3480bf6c0a51faf408d8ea9c8a8733a6b#r67291936

dadoonet mentioned this pull request Jun 17, 2016

Don't register repository settings in S3 plugin #18945

Closed

dadoonet removed the review label Jun 17, 2016

dadoonet self-assigned this Jun 17, 2016

dadoonet merged commit 82fee9f into elastic:master Jun 17, 2016

dadoonet deleted the pr/update-aws-sdk branch June 17, 2016 15:29

dadoonet mentioned this pull request Jul 20, 2016

Don't register repository settings in S3 plugin #19515

Merged

dadoonet mentioned this pull request Jul 26, 2016

Update aws sdk to 1.11.20 #19594

Closed

clintongormley added :Distributed/Discovery-Plugins Anything related to our integration plugins with EC2, GCP and Azure :Distributed/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs and removed :Plugin Discovery EC2 labels Feb 13, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update aws sdk to 1.10.69 and add use_throttle_retries repository setting#17784

Update aws sdk to 1.10.69 and add use_throttle_retries repository setting#17784
dadoonet merged 11 commits intoelastic:masterfrom
dadoonet:pr/update-aws-sdk

dadoonet commented Apr 15, 2016 •

edited

Loading

Uh oh!

dadoonet commented Apr 28, 2016

Uh oh!

dadoonet Apr 28, 2016

Uh oh!

tlrx May 6, 2016

Uh oh!

tlrx commented May 6, 2016

Uh oh!

dadoonet commented May 19, 2016

Uh oh!

tlrx May 25, 2016

Uh oh!

dadoonet May 27, 2016

Uh oh!

tlrx commented May 25, 2016

Uh oh!

dadoonet commented May 27, 2016

Uh oh!

dadoonet commented Jun 13, 2016

Uh oh!

tlrx commented Jun 14, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		Note that you can define S3 repository settings for all S3 repositories in `elasticsearch.yml` configuration file.
		They are all prefixed with `repositories.s3.`.

Conversation

dadoonet commented Apr 15, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Upgrade to AWS SDK 1.10.69

Release notes highlights from 1.10.33 for the SDK

Release notes highlights from 1.10.33 for S3

Release notes highlights from 1.10.33 for EC2

Add use_throttle_retries repository setting

Uh oh!

dadoonet commented Apr 28, 2016

Uh oh!

dadoonet Apr 28, 2016

Choose a reason for hiding this comment

Uh oh!

tlrx May 6, 2016

Choose a reason for hiding this comment

Uh oh!

tlrx commented May 6, 2016

Uh oh!

dadoonet commented May 19, 2016

Uh oh!

tlrx May 25, 2016

Choose a reason for hiding this comment

Uh oh!

dadoonet May 27, 2016

Choose a reason for hiding this comment

Uh oh!

tlrx commented May 25, 2016

Uh oh!

dadoonet commented May 27, 2016

Uh oh!

dadoonet commented Jun 13, 2016

Uh oh!

tlrx commented Jun 14, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dadoonet commented Apr 15, 2016 •

edited

Loading

Add `use_throttle_retries` repository setting