Skip to content

feat: AWS ECS Service Discovery#17105

Closed
matt-gp wants to merge 1 commit intoprometheus:mainfrom
matt-gp:aws-ecs-service-discovery
Closed

feat: AWS ECS Service Discovery#17105
matt-gp wants to merge 1 commit intoprometheus:mainfrom
matt-gp:aws-ecs-service-discovery

Conversation

@matt-gp
Copy link
Collaborator

@matt-gp matt-gp commented Aug 29, 2025

Hi all,

This PR adds service discovery support for AWS Elastic Container Services. Previous discussions for adding this were nearly 4 years ago here 9310. A previous pr was also created 15856 and this is loosely based on this.

Currently if users want service discovery for ECS they have to use a separate service which implements the http service discovery, and query that. ECS is quite a popular way of running containers without having to manage infrastructure so I feel this would be a good addition.

In terms of rate limiting, the request concurrency by default is set to 20 req/sec which aligns with describe/list requests listed https://docs.aws.amazon.com/AmazonECS/latest/APIReference/request-throttling.html.

Tests are also included and I've been testing locally.

Feedback welcome

Does this PR introduce a user-facing change?

NONE

@bboreham
Copy link
Member

bboreham commented Sep 2, 2025

Hi, thanks for this. Are you volunteering to maintain this code long-term?

@matt-gp
Copy link
Collaborator Author

matt-gp commented Sep 3, 2025

Hi, yes I'm willing to maintain this long term.

@matt-gp matt-gp force-pushed the aws-ecs-service-discovery branch from c976b32 to 4a846fb Compare September 29, 2025 08:07
@marcus-crane
Copy link

Exciting 🙂

@matt-gp matt-gp force-pushed the aws-ecs-service-discovery branch 3 times, most recently from ec235c3 to fa007b9 Compare September 30, 2025 19:02
@paulojmdias
Copy link
Contributor

Any news on this PR review? I can also help to maintain this code long-term if needed 🙏

@SuperQ
Copy link
Member

SuperQ commented Oct 26, 2025

I still think we should refactor the AWS discovery into a single aws_sd_configs. Similar to how we do Kubernetes with several roles.

Something like:

- job_name: ecs
  aws_sd_configs:
    role: ecs
    region: us-east-1
    ...

@matt-gp
Copy link
Collaborator Author

matt-gp commented Oct 26, 2025

Yeh that might be a good option to consolidate all of the AWS service discovery options into 1. Also if something new came up in the future we could add more to it quite easily.

I'll look at refactoring if this is the preferred direction.

@SuperQ
Copy link
Member

SuperQ commented Oct 26, 2025

Yes, that would be great if you're willing to work on it.

I would love to see additional roles like rds and elasticache discovery. We currently manually manage a bunch of scrape configs to use the postgres and redis exporters. It would be great to be able to use built-in discovery.

@matt-gp
Copy link
Collaborator Author

matt-gp commented Oct 26, 2025

Cool, I'll make this a more generic AWS SD that we can add more services to as and when (MSK would benefit me as well now that I think about it).

I'll probably put this in a different PR as well as I think the scope has outgrown this one a bit.

@matt-gp matt-gp force-pushed the aws-ecs-service-discovery branch 2 times, most recently from 518658e to 68841a7 Compare October 27, 2025 18:19
Signed-off-by: matt-gp <small_minority@hotmail.com>
@matt-gp
Copy link
Collaborator Author

matt-gp commented Oct 27, 2025

Raised 17406

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants