Feature Request
Groupon needed a more flexible histogram aggregation. Support for central re-aggregation is also needed. Central aggregations is currently accomplished outside of telegraf.
Configuration:
## TDigest Compression
## This value corresponds to the number of centroids the histogram will use
## Higher values increase size of data but also precision of calculated percentiles
compression = 30.0
[[aggregators.tdigestagg.bucketing]]
## List of tags that will not be considered for aggregation and not emitted.
## Empty list is valid
exclude_tags=[host]
## "source" is required in output by SLA but typically mapped from another input tag
## If source_tag_key is not set on an input point, a default value will be set and
## a sla_violation tag will be added
source_tag_key=service
## Optional: Default value is "atom"
## "atom" is required for output by SLA. Tag can be submitted with input points or mapped
## from another input tag. If "atom" or configured replacement tag is not set on an input
## point, a default value will be set and a sla_violation tag will be added
atom_replacement_tag_key=az
Local Aggregation Output:
{
"fields": {
"count": 2,
"max": 0.12385434728741589,
"med": 0.0742942278114139,
"min": 0.024734108335411897
},
"name": "cpu_usage_nice",
"tags": {
"atom": "sea1",
"az": "sea1",
"cpu": "cpu-total",
"env": "dev",
"host": "rion-laptop",
"rion": "tdigest-test",
"source": "rion-laptop"
},
"timestamp": 1569347320
}
Central Aggregation Output:
{
"fields": {
"sum._utility": 1230.0,
"centroids": "[{97.97979797979798 1} {97.97979797979798 1} {98 1} {98 1} {98 1} {98 1} {98 1} {98 1} {98.00990099009901 2} {98.01980198019803 2} {98.01980198019803 2} {98.01980198019803 2} {98.98989898989899 1} {98.98989898989899 2} {99 1} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99.00990099009901 2} {99.00990099009901 2} {99.00990099009901 2} {100 2} {100 2} {100 1} {100 1} {100 1} {100 1} {100 1} {100 1} {100 1}]",
"compression": 30
},
"name": "cpu_usage_idle",
"tags": {
"cpu": "cpu1",
"source": "rion-laptop",
"az": "snc1",
"env": "dev",
"service": "awesome",
"aggregates": "max,min,count,p99,p95,avg,med",
"bucket_key": "cpu_usage_idle_awesome_snc1_dev"
},
"timestamp": 1532630290113371000
}
Current behavior:
Statically defined bucket boundaries
Aggregations not generated based on histogram
Desired behavior:
Dynamic histogram buckets
Ability to emit histogram for central aggregation
Arbitrary list of aggregation buckets
Use case
Mathematically accurate percentiles for metrics across multiple sources.
Feature Request
Groupon needed a more flexible histogram aggregation. Support for central re-aggregation is also needed. Central aggregations is currently accomplished outside of telegraf.
Proposal: https://github.com/PhoenixRion/telegraf/tree/master/plugins/aggregators/tdigestagg
Configuration:
Local Aggregation Output:
Central Aggregation Output:
Current behavior:
Statically defined bucket boundaries
Aggregations not generated based on histogram
Desired behavior:
Dynamic histogram buckets
Ability to emit histogram for central aggregation
Arbitrary list of aggregation buckets
Use case
Mathematically accurate percentiles for metrics across multiple sources.