Skip to content

Need a more flexable histogram / tdigest #6440

@PhoenixRion

Description

@PhoenixRion

Feature Request

Groupon needed a more flexible histogram aggregation. Support for central re-aggregation is also needed. Central aggregations is currently accomplished outside of telegraf.

Proposal: https://github.com/PhoenixRion/telegraf/tree/master/plugins/aggregators/tdigestagg

Configuration:

## TDigest Compression
  ## This value corresponds to the number of centroids the histogram will use
  ## Higher values increase size of data but also precision of calculated percentiles
  compression = 30.0

[[aggregators.tdigestagg.bucketing]]
  ## List of tags that will not be considered for aggregation and not emitted.
  ## Empty list is valid
  exclude_tags=[host]

  ## "source" is required in output by SLA but typically mapped from another input tag
  ## If source_tag_key is not set on an input point, a default value will be set and
  ## a sla_violation tag will be added
  source_tag_key=service

  ## Optional: Default value is "atom"
  ## "atom" is required for output by SLA.  Tag can be submitted with input points or mapped
  ## from another input tag.  If "atom" or configured replacement tag is not set on an input
  ## point, a default value will be set and a sla_violation tag will be added
  atom_replacement_tag_key=az

Local Aggregation Output:

{
  "fields": {
    "count": 2,
    "max": 0.12385434728741589,
    "med": 0.0742942278114139,
    "min": 0.024734108335411897
  },
  "name": "cpu_usage_nice",
  "tags": {
    "atom": "sea1",
    "az": "sea1",
    "cpu": "cpu-total",
    "env": "dev",
    "host": "rion-laptop",
    "rion": "tdigest-test",
    "source": "rion-laptop"
  },
  "timestamp": 1569347320
}

Central Aggregation Output:

{
  "fields": {
    "sum._utility": 1230.0,
    "centroids": "[{97.97979797979798 1} {97.97979797979798 1} {98 1} {98 1} {98 1} {98 1} {98 1} {98 1} {98.00990099009901 2} {98.01980198019803 2} {98.01980198019803 2} {98.01980198019803 2} {98.98989898989899 1} {98.98989898989899 2} {99 1} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99 2} {99.00990099009901 2} {99.00990099009901 2} {99.00990099009901 2} {100 2} {100 2} {100 1} {100 1} {100 1} {100 1} {100 1} {100 1} {100 1}]",
    "compression": 30
  },
  "name": "cpu_usage_idle",
  "tags": {
    "cpu": "cpu1",
    "source": "rion-laptop",
    "az": "snc1",
    "env": "dev",
    "service": "awesome",
    "aggregates": "max,min,count,p99,p95,avg,med",
    "bucket_key": "cpu_usage_idle_awesome_snc1_dev"
  },
  "timestamp": 1532630290113371000
}

Current behavior:

Statically defined bucket boundaries
Aggregations not generated based on histogram

Desired behavior:

Dynamic histogram buckets
Ability to emit histogram for central aggregation
Arbitrary list of aggregation buckets

Use case

Mathematically accurate percentiles for metrics across multiple sources.

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature requestRequests for new plugin and for new features to existing plugins

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions