Skip to content

remote_write azure auth : add workload identity support#16788

Merged
bwplotka merged 14 commits intoprometheus:mainfrom
bragi92:kaveesh/wi_support
Aug 26, 2025
Merged

remote_write azure auth : add workload identity support#16788
bwplotka merged 14 commits intoprometheus:mainfrom
bragi92:kaveesh/wi_support

Conversation

@bragi92
Copy link
Contributor

@bragi92 bragi92 commented Jun 26, 2025

Currently, Prometheus supports several authentication mechanisms for remote write including basic auth, authorization headers, sigv4, azuread, and oauth2. However, it lacks support for Azure Workload Identity, which is Microsoft's recommended modern authentication method for workloads running in Azure Kubernetes Service (AKS) and other Azure environments.
Azure Workload Identity provides a secure, token-based authentication mechanism that eliminates the need for storing secrets and credentials in pods. It leverages the relationship between Kubernetes service accounts and Azure managed identities to provide seamless authentication.

Changes
This implementation extends the existing Azure AD authentication support to include Workload Identity authentication for remote write endpoints. The changes allow users to:

Configure Prometheus to use Azure Workload Identity for authenticating remote write requests
Leverage the existing Azure token acquisition mechanisms while supporting the Workload Identity token exchange flow
Maintain backward compatibility with existing Azure AD authentication methods

Benefits

Enhanced Security: Eliminates the need to store Azure credentials as secrets in Kubernetes
Simplified Operations: Reduces credential management overhead for Azure-based deployments
Modern Authentication: Aligns with Microsoft's recommended authentication patterns for Azure workloads
Compliance: Supports organizations' requirements for credential-less authentication in cloud environments

Breaking Changes
None. This is an additive feature that maintains full backward compatibility with existing authentication methods.

Testing
I've added unit tests and also built the prometheus binary locally and deployed it as a container in my AKS cluster with workload identity and see data flowing into my workspace:

image

@bragi92 bragi92 changed the title Kaveesh/wi support remote_write azure auth : add workload identity support Jul 3, 2025
@bragi92 bragi92 marked this pull request as ready for review July 7, 2025 23:29
@bragi92 bragi92 force-pushed the kaveesh/wi_support branch from 8b00adc to 4bcef25 Compare July 22, 2025 21:43
bragi92 added 5 commits July 22, 2025 15:10
Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com>
.
Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com>
Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com>
Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com>
Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com>
@bragi92 bragi92 force-pushed the kaveesh/wi_support branch from 1552995 to 1a8758c Compare July 22, 2025 22:10
@bragi92
Copy link
Contributor Author

bragi92 commented Jul 22, 2025

@bwplotka Can you take a look at this PR when you get a chance? Its similar to #16421 were I added support for system assigned identity. This one adds in support for workload identity and I've been able to validate the changes successfully.

@bragi92
Copy link
Contributor Author

bragi92 commented Jul 29, 2025

Hey @bwplotka, gentle ping on this PR. Let me know what the best next steps would be to get this merged in.

@bwplotka
Copy link
Member

Thanks! I was on holidays, will try to look tomorrow, please ping me on Slack if I don't!

Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, excellent work, code quality wise - thanks for tests!

Generally looks good, but we need to update `documentation, ideally in this PR, to allow users to discover and use the new entries.

I also suggest we don't over explain the setup here. It can change anytime and it will hard to update our code commentary or even any complex instructions in the potential configuration documentation. Let's keep it minimal and refer to Azure docs if possible, WDYT?

bragi92 and others added 8 commits July 31, 2025 12:31
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: bragi92 <kadubey@microsoft.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: bragi92 <kadubey@microsoft.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: bragi92 <kadubey@microsoft.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: bragi92 <kadubey@microsoft.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: bragi92 <kadubey@microsoft.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: bragi92 <kadubey@microsoft.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: bragi92 <kadubey@microsoft.com>
Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com>
@bragi92
Copy link
Contributor Author

bragi92 commented Aug 1, 2025

Thank you for taking a look at this PR @bwplotka. I believe I’ve addressed all your comments — when you have a moment, I’d appreciate another look.

@bragi92
Copy link
Contributor Author

bragi92 commented Aug 6, 2025

Gentle ping on this @bwplotka , Thanks for your help in reviewing this!

Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@bwplotka bwplotka merged commit 20580b6 into prometheus:main Aug 26, 2025
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants