Problem:
“Azure billing metrics” integration requires “Subscription ID” as a required parameter when we should allow the users to collect billing usage details with a scope of subscription OR department OR billing account based on their billing agreement and/or use case.
In an Enterprise Agreement the hierarchy is Billing Account->one or more department->one or more subscription. Users should be able to specify subscription Id, department or billing account Id to define the scope of the usage data. If user has specified department then it will override the subscription scope (as its higher) for usage details. If the user has specified the billing account Id then it will override the subscription or department scope (as its higher) for usage details and that's the highest level.
The code is actually setup to override (https://github.com/elastic/beats/blob/main/x-pack/metricbeat/module/azure/billing/client.go#L52-L57) but the user experience is confusing and our documentation doesn’t explains this clearly as well.
User experience:
This UI behavior of requiring subscription ID as mandatory parameter leads to users believing that we only work at the subscription level and additional parameters that we have exposed in Advanced option for “Billing Scope Department” and “Billing Scope Account ID” does not have any affect. The documentation is not clear causing further confusion on how this is actually supposed to work.
Potential solution:
We will need to discuss and evaluate this further but one possible solution could be:
- In the main integration settings create “Scope: Specify subscription, department or billing account information to define the scope of the usage details collection. One of the value is mandatory.” as a section with 3 optional values “Subscription ID”, Billing Department” & “Billing Account ID”.
- We will need to evaluate what this means for upgrade and what impact it has from backwards compatibility where users already have this integration configured.
- Enhance our documentation to explain the scope and how it works. Include the appropriate links to Microsoft documentation as well.
Some reference links here:
Billing account types and object hierarchy => https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/view-all-accounts
Usage details API => https://docs.microsoft.com/en-us/rest/api/consumption/usage-details/list
Billing role for Enterprise Agreement (EA) => https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/understand-ea-roles
Billing role for Microsoft Online Service program accounts => https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/manage-billing-access
Code where we override the scope => https://github.com/elastic/beats/blob/main/x-pack/metricbeat/module/azure/billing/client.go#L52-L57
Problem:
“Azure billing metrics” integration requires “Subscription ID” as a required parameter when we should allow the users to collect billing usage details with a scope of subscription OR department OR billing account based on their billing agreement and/or use case.
In an Enterprise Agreement the hierarchy is Billing Account->one or more department->one or more subscription. Users should be able to specify subscription Id, department or billing account Id to define the scope of the usage data. If user has specified department then it will override the subscription scope (as its higher) for usage details. If the user has specified the billing account Id then it will override the subscription or department scope (as its higher) for usage details and that's the highest level.
The code is actually setup to override (https://github.com/elastic/beats/blob/main/x-pack/metricbeat/module/azure/billing/client.go#L52-L57) but the user experience is confusing and our documentation doesn’t explains this clearly as well.
User experience:
This UI behavior of requiring subscription ID as mandatory parameter leads to users believing that we only work at the subscription level and additional parameters that we have exposed in Advanced option for “Billing Scope Department” and “Billing Scope Account ID” does not have any affect. The documentation is not clear causing further confusion on how this is actually supposed to work.
Potential solution:
We will need to discuss and evaluate this further but one possible solution could be:
Some reference links here:
Billing account types and object hierarchy => https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/view-all-accounts
Usage details API => https://docs.microsoft.com/en-us/rest/api/consumption/usage-details/list
Billing role for Enterprise Agreement (EA) => https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/understand-ea-roles
Billing role for Microsoft Online Service program accounts => https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/manage-billing-access
Code where we override the scope => https://github.com/elastic/beats/blob/main/x-pack/metricbeat/module/azure/billing/client.go#L52-L57