As a user, I may want to launch Metricbeat modules to monitor services running on AWS EC2 instances. For example:

• to retrieve MySQL metrics from servers tagged with service: mysql.
• to gather Prometheus metrics from all my nodes, running cadvisor.

Autodiscover framework provides the right mechanism to support this kind of configurations, where the user can define module templates to launch based on conditions on the status of an external system.

We could add a new aws_ec2 autodiscover provider to allow autodiscover to react to EC2 instances status, this information could be included in autodiscover events, to be used in autodiscover conditions:

id (instance arn)
host (public or private IP?)
aws.ec2.instance.image.id
aws.ec2.instance.state.name
aws.tags
cloud.availability_zone
cloud.machine.type
cloud.region

Autodiscover is able to enrich events coming out of modules it launches, aws_ec2 should provide the common cloud metadata

Example config:

metricbeat.autodiscover:
  providers:
  - type: aws_ec2
    regions: ["us-east-1"]
    access_key_id: my-access-key
    secret_access_key: my-secret-access-key
    templates:
    - condition:
        equals.aws.tags.service: mysql
      config:
      - module: mysql
        hosts: ["root:secret@tcp(${data.host}:3306)"]