Describe the enhancement:
We want to support autodiscover features in the Agent (#19225). For this, it would be nice if we could reuse the existing providers and make Agent handle the runner start/stop logic.
Current status
Right now, autodiscover providers send start/stop events through a bus. The autodiscover manager listens into this bus and manages input/module runners according to these events.
Providers are coupled to config generation (through templates or hints), although all this code is contained in libraries. It is used like this:
|
// Try to match a config |
|
if config := d.templates.GetConfig(event); config != nil { |
|
event["config"] = config |
|
} else { |
|
// If no template matches, try builders: |
|
if config := d.builders.GetConfig(d.generateHints(event)); config != nil { |
|
event["config"] = config |
|
} |
|
} |
I don't see any reason why cannot be done by autodiscover manager (and not the providers) ^
Proposal
To make providers reusable, we need to decouple them from the config generation / runner handling, a few things we would need to do:
- Refactor autodiscover to move the template logic away from providers, into the autodiscover manager.
- Re-define the schema for autodiscover events. For instance, we should include hints into them (instead of the generated config). It should be serializable (cannot carry things like a keystore or meta pointer)
Some caveats:
- We are currently using dynamic fields (a pointer we can update in memory) to update metadata without restarting runners, we may need to rethink this?
- Kubernetes autodiscover supports keystore (again, at the provider level), should this be moved into the autodiscover manager? We can leave it out of Agent for now.
Agent will implement a new template logic that can be hooked into provider events (see the parent issue for more details).
Describe the enhancement:
We want to support autodiscover features in the Agent (#19225). For this, it would be nice if we could reuse the existing providers and make Agent handle the runner start/stop logic.
Current status
Right now, autodiscover providers send start/stop events through a bus. The autodiscover manager listens into this bus and manages input/module runners according to these events.
Providers are coupled to config generation (through templates or hints), although all this code is contained in libraries. It is used like this:
beats/libbeat/autodiscover/providers/docker/docker.go
Lines 307 to 315 in 93a4591
I don't see any reason why cannot be done by autodiscover manager (and not the providers) ^
Proposal
To make providers reusable, we need to decouple them from the config generation / runner handling, a few things we would need to do:
Some caveats:
Agent will implement a new template logic that can be hooked into provider events (see the parent issue for more details).