Skip to content

[AWS] Support collecting dynamic labels for account ID and account name #6232

@kaiyan-sheng

Description

@kaiyan-sheng

Currently we don't support collecting account name/ID from CloudWatch metrics. In AWS CloudWatch, the account ID and account name are stored as a part of the dynamic labels: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/graph-dynamic-labels.html.

When customers use our AWS integrations to collect monitoring metrics, get-metric-data API calls are made. Right now, we are only collecting basic data points for a given timestamp and metric name. We do not support collecting dynamic labels. This being said, adding support for collecting dynamic labels specifically for account ID and name will be very helpful.

Here is the difference between two API calls w/o collecting account ID and name:

Without dynamic labels:

kaiyansheng ~/Documents/SDHs/3367  $ aws cloudwatch get-metric-data  --region eu-central-1 --profile observability --start-time 2023-05-14T00:00:00Z --end-time 2023-05-14T00:15:00Z --metric-data-queries file://./test.json
{
    "MetricDataResults": [
        {
            "Id": "m1",
            "Label": "VolumeIdleTime",
            "Timestamps": [
                "2023-05-14T00:10:00+00:00",
                "2023-05-14T00:05:00+00:00",
                "2023-05-14T00:00:00+00:00"
            ],
            "Values": [
                59.956,
                59.946000000000005,
                59.94
            ],
            "StatusCode": "Complete"
        }
    ],
    "Messages": []
}

The metric-data-queries looks like this:

kaiyansheng ~/Documents/SDHs/3367  $ cat test.json 
[
    {
        "Id": "m1",
        "MetricStat": {
            "Metric": {                 
                "Namespace": "AWS/EBS",
                "MetricName": "VolumeIdleTime",
                "Dimensions": [
                    {
                        "Name": "VolumeId",
                        "Value": "vol-06d94d7e1f2995456"
                    }
                ]
            },
            "Period": 300,
            "Stat": "Average"
        },
       "ReturnData": true 
    }
]

With dynamic label for account ID and name:

kaiyansheng ~/Documents/SDHs/3367  $ aws cloudwatch get-metric-data  --region eu-central-1 --profile observability --start-time 2023-05-14T00:00:00Z --end-time 2023-05-14T00:15:00Z --metric-data-queries file://./test_account.json
{
    "MetricDataResults": [
        {
            "Id": "m1",
            "Label": "MonitoringAccount, 627286350134",
            "Timestamps": [
                "2023-05-14T00:10:00+00:00",
                "2023-05-14T00:05:00+00:00",
                "2023-05-14T00:00:00+00:00"
            ],
            "Values": [
                59.956,
                59.946000000000005,
                59.94
            ],
            "StatusCode": "Complete"
        }
    ],
    "Messages": []
}

The metric-data-queries looks like this:

kaiyansheng ~/Documents/SDHs/3367  $ cat test_account.json 
[
    {
        "Id": "m1",
        "MetricStat": {
            "Metric": {                 
                "Namespace": "AWS/EBS",
                "MetricName": "VolumeIdleTime",
                "Dimensions": [
                    {
                        "Name": "VolumeId",
                        "Value": "vol-06d94d7e1f2995456"
                    }
                ]
            },
            "Period": 300,
            "Stat": "Average"
        },
       "ReturnData": true,
       "Label": "${PROP('AccountLabel')}, ${PROP('AccountId')}"
    }
]

Once we enhance the API call to add collection for account ID and account name through dynamic labels, then we can present them into a separate field in the document that sends to Elasticsearch.

Metadata

Metadata

Assignees

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions