AWS SD: ECS Discover Standalone Tasks#18029
Merged
sysadmind merged 1 commit intoprometheus:mainfrom Feb 11, 2026
Merged
Conversation
c7754ff to
2b550d2
Compare
a931fff to
7375fc1
Compare
The current ECS role in AWS SD assumes that a task is part of a service. This means that tasks that are started as part of AWS Batch will get missed and not be discovered. This changed fixes this so that standalone tasks can be discovered as well. Signed-off-by: matt-gp <small_minority@hotmail.com>
7375fc1 to
5329355
Compare
Collaborator
Author
|
@SuperQ @sysadmind Would you be able to review? This one is quite big but it does fundamentally change how the ecs discovery works, plus adds quite a lot of optimisations. |
sysadmind
approved these changes
Feb 11, 2026
Contributor
|
Looks like GitHub now supports reviewing individual commits - that would be helpful in a case like this to break down each of these functions into a very easy to review piece. Hopefully the new UI helps with this. |
SuperQ
reviewed
Feb 11, 2026
| clusterMap := make(map[string]types.Cluster) | ||
| errg, ectx := errgroup.WithContext(ctx) | ||
| errg.SetLimit(d.cfg.RequestConcurrency) | ||
| for batch := range slices.Chunk(clusters, 100) { |
Member
There was a problem hiding this comment.
Can you make a const ecsBatchSize = 100 just for cleanliness?
SuperQ
reviewed
Feb 11, 2026
Member
SuperQ
left a comment
There was a problem hiding this comment.
Seems mostly reasonable, just minor nit about the 100 constant.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The current ECS role in AWS SD assumes that a task is part of a service. This means that tasks that are started as part of AWS Batch will get missed and not be discovered.
This change completely reworks/overhauls the ECS role in order to discover ECS Tasks regardless of if they are part of a service or not.
Also this change optimises some of the external calls so that they are done once per cluster instead of once per task and also makes sure that these external calls use pagination and/or batching wherever possible in order to handle situations where there are a lot of resources to discover. Plus removes the custom batching function in favour of the slices.Chunk function from the standard library.
The tests have also been updated also to cover the scenario where a task is standalone and not part of a service.
Which issue(s) does the PR fix:
#17920
Does this PR introduce a user-facing change?