Skip to content

[discovery-ec2] Plugin doesn't retry EC2 describe instance calls #50462

Closed
#50550
Closed
[discovery-ec2] Plugin doesn't retry EC2 describe instance calls#50462
#50550
Assignees
original-brownbear
Labels
:Distributed/Discovery-PluginsAnything related to our integration plugins with EC2, GCP and Azure>bug
@Bukhtawar

Description

@Bukhtawar
Bukhtawar
opened on Dec 23, 2019

Problem Description

Discovery EC2 doesn't retry EC2 describe calls on throttle or exceptions due to NO_RETRY_CONDITION which has the potential to make the throttling worse. Although there is a logic to exponentially backoff in such cases, backoff doesn't kick in.

elasticsearch/plugins/discovery-ec2/src/main/java/org/elasticsearch/discovery/ec2/AwsEc2ServiceImpl.java

Lines 82 to 90 in 7203cee

RetryPolicy.RetryCondition.NO_RETRY_CONDITION,
(originalRequest, exception, retriesAttempted) -> {
// with 10 retries the max delay time is 320s/320000ms (10 * 2^5 * 1 * 1000)
logger.warn("EC2 API request failed, retry again. Reason was:", exception);
return 1000L * (long) (10d * Math.pow(2, retriesAttempted / 2.0d) * (1.0d + rand.nextDouble()));
},
10,
false);
clientConfiguration.setRetryPolicy(retryPolicy);

AWS reference

Metadata

Metadata

Assignees

Labels

:Distributed/Discovery-PluginsAnything related to our integration plugins with EC2, GCP and Azure>bug

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions