-
Notifications
You must be signed in to change notification settings - Fork 5.8k
aggregators should not re-run processors #7993
Copy link
Copy link
Closed
Labels
area/agentarea/execdIssues related to execd or plugins that would be better suited to be used through execdIssues related to execd or plugins that would be better suited to be used through execdarea/starlarkbugunexpected problem or unintended behaviorunexpected problem or unintended behaviorsize/l1 week or more effort1 week or more effort
Metadata
Metadata
Assignees
Labels
area/agentarea/execdIssues related to execd or plugins that would be better suited to be used through execdIssues related to execd or plugins that would be better suited to be used through execdarea/starlarkbugunexpected problem or unintended behaviorunexpected problem or unintended behaviorsize/l1 week or more effort1 week or more effort
Relevant telegraf.conf:
Steps to reproduce:
run config like above. Run aggregator with any non-idempotent processor
Expected behavior:
processor runs once, outputting
data 10Actual behavior:
processor runs twice, outputting
data 100Additional info:
This is an old feature of aggregators, and it's due to the fact that aggregators re-run all processors. This was probably figured to be a good idea since you may want to modify the output of aggregators, but it's 1. wasteful, and 2. breaks any processors that can't be applied twice to the same field, eg
x * 10where x=1 gives you 10 after one run, but 100 after two runs; it's not idempotent. Adding an aggregator will break any of these processors.The solution here would be to convert aggregators to be processors themselves. This will let you order them across both processors and aggregators, and resolve the problem in a very sensible way. Aggregators are a special case of processors, and with the new streaming processor support, they fit perfectly into the processor model.