Feature Request

Explicitly around the case of supporting multiple disparate Telegraf configurations, which comes in from 1. cloud, 2. community template, and 3. Telegraf configuration service, we need the ability to take multiple desired Telegraf configs merge them into a single Telegraf configuration that Telegraf can run.

An acceptable solution to this could be an external project or library, but that is not necessarily a requirement.

This is important because community template users currently need to manually merge configs. If we don't do this project, it will encourage template authors to work around the issue by merging separate templates into "template stacks" to make template installation easier, at a high cost to flexibility and an explosion in variants (vs what would be small composable templates if we had this).

Use cases:

1. Installing multiple community templates installs multiple Telegraf configurations. It's not unusual for a single server to be collecting data for multiple templates, eg, redis, system, internal, kafka, or any stack you can imagine.
2. This avoids you having to predefine Telegraf "roles" where a role is a set of plugins to use. You could generate the "role" config on the fly from known configurations.

Requirements/Nice-to-Haves:

• [R] Take two configs and produce a single config from it
• [R] Combine multiple outputs pointing to the same destination, respecting buckets
• [N] Generate a default output (eg, for cloud), when none is defined (when desired)
• [N] auto pipelining/namespacing, (or tagpassing?) to ensure configs don't have unintended behavior when merged

Unresolved issues:

• There will be issues around merging multiple configurations with complex workflows, eg, a lot of processors/aggregators, especially with ones using generic field names. Stand-alone configs can get away with not checking metric names, but can cause issues when merged. Config pipelining/namespacing could be a solution here.

Proposal:

• Create a new library that knows how to understand/merge multiple configs, smart enough to remove duplicate configs, with idempotent merge behavior.
• [option 1] extend Telegraf to accept multiple --config arguments
• [option 2] create a stand-alone binary that can merge config files.