Skip to content

[FEATURE] AWS SD: Add MSK Role#17600

Merged
SuperQ merged 1 commit intoprometheus:mainfrom
matt-gp:aws-msk-service-discovery
Feb 5, 2026
Merged

[FEATURE] AWS SD: Add MSK Role#17600
SuperQ merged 1 commit intoprometheus:mainfrom
matt-gp:aws-msk-service-discovery

Conversation

@matt-gp
Copy link
Collaborator

@matt-gp matt-gp commented Nov 22, 2025

AWS offers a Kafka service called Managed Streaming Kafka (MSK). This allows users to run Kafka clusters without having the overhead of managing the infrastructure themselves.

As part of the monitoring configuration for these clusters, you can enable the JMX Exporter and/or Node Exporter, so that the nodes will expose metrics that users can collect.

There are some caveats with this:

  • Only works with provisioned clusters, eg not serverless.
  • Only Brokers expose Node Exporter metrics.
  • Only Brokers and kRaft Controller nodes expose JMX Exporter metrics, eg not Zookeeper nodes.

Which issue(s) does the PR fix:

Does this PR introduce a user-facing change?

[FEATURE] AWS SD: Add MSK Role #17600

@matt-gp matt-gp force-pushed the aws-msk-service-discovery branch 7 times, most recently from 9d399e9 to d88b2a1 Compare November 22, 2025 15:35
@matt-gp matt-gp changed the title AWS SD: Add MSK Role [FEATURE] AWS SD: Add MSK Role Nov 22, 2025
@krajorama
Copy link
Member

Hello from the bug scrub!

We haven't identified a reviewer at this time. Who would be a good candidate? Also pinging @sysadmind @alanprot possibly?

@matt-gp
Copy link
Collaborator Author

matt-gp commented Nov 25, 2025

@SuperQ Might be a good reviewer as he has reviewed some of the other AWS Service Discovery.

@SuperQ SuperQ self-requested a review November 25, 2025 15:40
@matt-gp matt-gp force-pushed the aws-msk-service-discovery branch from d88b2a1 to f48cb27 Compare November 25, 2025 20:20
@matt-gp matt-gp requested a review from SuperQ November 25, 2025 20:22
@matt-gp matt-gp force-pushed the aws-msk-service-discovery branch 2 times, most recently from a57ffbb to 200045a Compare November 25, 2025 21:36
@matt-gp matt-gp force-pushed the aws-msk-service-discovery branch 4 times, most recently from 1a82c0f to dbfefd7 Compare December 19, 2025 15:29
@matt-gp
Copy link
Collaborator Author

matt-gp commented Jan 2, 2026

We should rebase on top of #17769 and then add a dedicated MSK test to TestMultipleSDConfigsDoNotShareState

@matt-gp matt-gp marked this pull request as draft January 2, 2026 14:10
@matt-gp matt-gp force-pushed the aws-msk-service-discovery branch 4 times, most recently from bb56360 to 18a5794 Compare January 22, 2026 20:09
@matt-gp matt-gp marked this pull request as ready for review January 22, 2026 20:49
@matt-gp matt-gp force-pushed the aws-msk-service-discovery branch from 18a5794 to 636c2b0 Compare January 22, 2026 20:53
@matt-gp matt-gp force-pushed the aws-msk-service-discovery branch from 636c2b0 to b46a8bf Compare January 22, 2026 21:20
@matt-gp
Copy link
Collaborator Author

matt-gp commented Jan 22, 2026

@SuperQ would you be able to take another look when you get a chance?

…timestamp) (prometheus#17411)

Relates to
prometheus#16944 (comment)

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: matt-gp <small_minority@hotmail.com>
@matt-gp matt-gp force-pushed the aws-msk-service-discovery branch from b46a8bf to a65d075 Compare January 28, 2026 18:47
@matt-gp matt-gp requested review from a team and sysadmind as code owners January 28, 2026 18:47
@matt-gp
Copy link
Collaborator Author

matt-gp commented Jan 28, 2026

@sysadmind would you be able to look when you get a chance?

Copy link
Contributor

@sysadmind sysadmind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks okay to me. I don't have much ability to support MSK myself, so this would mostly be on you, @matt-gp to maintain.

@matt-gp
Copy link
Collaborator Author

matt-gp commented Jan 31, 2026

Hi, I have an immediate need for this so am happy to maintain it.

@matt-gp
Copy link
Collaborator Author

matt-gp commented Feb 5, 2026

@SuperQ Sorry to nudge you, are you able to take a look? I have another PR that this is holding up. Thanks

Copy link
Member

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, sorry, this review got lost in my inbox.

@SuperQ SuperQ merged commit cf9d093 into prometheus:main Feb 5, 2026
32 checks passed
@matt-gp matt-gp deleted the aws-msk-service-discovery branch February 5, 2026 14:59
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
…timestamp) (prometheus#17411) (prometheus#17600)

Relates to
prometheus#16944 (comment)

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: matt-gp <small_minority@hotmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Will Bollock <wbollock@linode.com>
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
…timestamp) (prometheus#17411) (prometheus#17600)

Relates to
prometheus#16944 (comment)

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: matt-gp <small_minority@hotmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
…timestamp) (prometheus#17411) (prometheus#17600)

Relates to
prometheus#16944 (comment)

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: matt-gp <small_minority@hotmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
…timestamp) (prometheus#17411) (prometheus#17600)

Relates to
prometheus#16944 (comment)

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: matt-gp <small_minority@hotmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
…timestamp) (prometheus#17411) (prometheus#17600)

Relates to
prometheus#16944 (comment)

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: matt-gp <small_minority@hotmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
wbollock pushed a commit to wbollock/prometheus that referenced this pull request Feb 6, 2026
…timestamp) (prometheus#17411) (prometheus#17600)

Relates to
prometheus#16944 (comment)

Signed-off-by: bwplotka <bwplotka@gmail.com>
Signed-off-by: matt-gp <small_minority@hotmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants