Hi everyone,

this PR adds service discovery support for AWS Elastic Container Services. There were discussions around adding support going back three years ago here: #9310

The current solution for ECS integration with Prometheus is to deploy a service that exposes targets via an HTTP endpoint and let Prometheus pull from there. The service discovery itself is then outsourced to a third party component of which there exist:

• https://github.com/teralytics/prometheus-ecs-discovery
• https://github.com/apptality/aws-ecs-cloudmap-prometheus-sd

Needless to say that it would be nicer to not have the additional operational complexity and instead have it natively supported by prometheus. Previously, the integration of ECS service discovery was rejected due to rate limiting concerns:

Some SD mechanisms have rate limits that make them challenging to use. As an example we have unfortunately had to reject Amazon ECS service discovery due to the rate limits being so low that it would not be usable for anything beyond small setups.

For reference, the limits are published here: https://docs.aws.amazon.com/AmazonECS/latest/APIReference/request-throttling.html

This ECS service discovery PR performs the following requests in sequence:

1. ListClusters (unless manually configured) - Burst 50, refill 20, max results 100
2. ListTasks (for each cluster) - Burst 100, refill 20, max results 100
3. DescribeTasks - Counts against the same Cluster resource read actions bucket as the previous ListTasks
4. DescribeTaskDefinition - Burst 50, refill 20, max results 1.

The final step is supposedly the reason why ECS service discovery was thought to only apply to "small setups". However, the response can be cached and reused (which is done in this PR). Then I would argue ECS service discovery would also apply to medium-sized setups (YMMV of course).

If you use the ECS EC2 launch type there would be additional requests (ec2.DescribeInstances) which I didn't include in the example above because the rate limits seem high enough to integrate EC2 service discovery into Prometheus.

I intend to use the changes here for our own setup because we're not operating at such a large scale. However, I would obviously prefer to not live on a fork indefinitely. I'm happy to iterate on the PR if it has a realistic chance of landing. I acknowledge that a decision has already been made against adding ECS service discovery but maybe the information above warrants a reconsideration.

Some additional remarks regarding this PR:

• The name ecs clashes with the aliyun ecs discovery here: Feature/aliyun ecs discovery #15212
• This PR is based on Migrating ec2 service discovery to the AWS sdk v2 #15735 which migrates the AWS SDK from v1 to v2. I figured it makes sense to implement the service discovery directly in v2. This means Migrating ec2 service discovery to the AWS sdk v2 #15735 should be merged first and unfortunately, massively pollutes the diff here.
• I haven't deployed the code into our infrastructure yet but it only works locally. I still wanted to start the discussion.
• No tests yet, but I'm happy to add some if there's a realistic chance of landing upstream