Skip to content

Conditions with dots in label name does not work as expected #1284

@barkbay

Description

@barkbay

For confirmed bugs, please report:

  • Version: 8.4.1
  • Operating System: Agent in GKE + Stack on Elastic Cloud

Autodiscover conditions does not seem to work if there is a / in a K8S Pod's label name used in the condition.

In my use case I want to collect Prometheus metrics from Pods with the app.kubernetes.io/name: global-controllers label:

k get pod -l app.kubernetes.io/name=global-controllers -A -o wide
NAMESPACE        NAME                                 READY   STATUS    RESTARTS   AGE   IP            NODE                                                  NOMINATED NODE   READINESS GATES
elastic-system   global-controllers-fd45757dd-xtrmr   1/1     Running   0          72m   10.20.49.18   gke-michael-dev-cluster--default-pool-d3f94c80-tqlr   <none>           <none>

Here is the standalone config I'm using:

apiVersion: v1
kind: ConfigMap
metadata:
  name: elastic-agent-cluster-metrics-cm
  labels:
    k8s-app: elastic-agent-cluster-metrics
data:
  elastic-agent-cm.yml: |-
    outputs:
      metrics:
        type: elasticsearch
        hosts:
          - >-
            ${ELASTICSEARCH_METRICS_HOST}
        username: ${ELASTICSEARCH_METRICS_USERNAME}
        password: ${ELASTICSEARCH_METRICS_PASSWORD}
    providers:
      kubernetes_leaderelection:
        enabled: true
      kubernetes:
        scope: cluster
        resources:
          pod:
            enabled: true
    inputs:
      - id: prometheus/metrics-prometheus-18fc9e8d-7243-449f-8c54-9407a1972014
        name: prometheus-1
        revision: 1
        type: prometheus/metrics
        use_output: metrics
        meta:
          package:
            name: prometheus
            version: 0.13.0
        data_stream:
          namespace: default
        streams:
          - id: >-
              prometheus/metrics-prometheus.collector-18fc9e8d-7243-449f-8c54-9407a1972014
            data_stream:
              dataset: prometheus.collector
              type: metrics
            metricsets:
              - collector
            hosts:
              - '${kubernetes.pod.ip}:9090'
            metrics_filters.exclude: null
            metrics_filters.include: null
            metrics_path: /metrics
            period: 10s
            condition: ${kubernetes.labels.app.kubernetes.io/name} == 'global-controllers'
            rate_counters: true
            use_types: true

If I use a label name without a / (for example app :foo), and I update the condition to condition: ${kubernetes.labels.app} == 'foo', then Agent collects the metrics as expected.

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions