Metricbeat Version used: 6.4.0, 6.6.2

hints-based autodiscover generates incomplete configurations when there are ports defined in the pod spec and hints include hard-coded ports. Some of them must match. If not, errors related to incorrect hosts happen.

2019-03-15T11:28:59.316Z DEBUG [autodiscover] autodiscover/autodiscover.go:195 Check failed for config &{{ } 0xc420681f60}: 3 errors: host parsing failed for etcd-leader: error parsing URL: empty host; host parsing failed for etcd-self: error parsing URL: empty host; host parsing failed for etcd-store: error parsing URL: empty host, won't start runner

raw annotations can be used as workaround as mentioned in this comment.

Original report:

Hi,

I'm trying to get the Prometheus Collector working with Hints based autodiscovery in Kubernetes.

Unfortunately I always get the following Error:

ERROR [autodiscover] cfgfile/list.go:104 Error creating runner from config: 1 error: 1 error: host parsing failed for prometheus-collector: error parsing URL: empty host

I checked the start event for the host field and it's present.

Got a start event: map[host:100.123.3.45 port:1234 kubernetes:{"annotations":{"co":{"elastic":{"metrics/hosts":"${data.host}:10254","metrics/metricsets":"collector","metrics/module":"prometheus","metrics/period":"1m"}}}} config:[0xc42001f4a0] start:true], generated configs: [0xc42001f4a0]

I'm using the following Annotations:

      annotations:
        co.elastic.metrics/hosts: ${data.host}:10254
        co.elastic.metrics/metricsets: collector
        co.elastic.metrics/module: prometheus
        co.elastic.metrics/period: 1m
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"

Any idea why this is happening?