Skip to content

[Heartbeat][Synthetics/Browser] Unable to run heartbeat 7.14.0 with browser monitor with ECK 1.7 #4794

@BenB196

Description

@BenB196

Bug Report

What did you do?

Deploy a synthetic monitor via ECK

What did you expect to see?

Synthetic to be able to start up correctly.

What did you see instead? Under which circumstances?

Ran into two errors:

{"level":"error","timestamp":"2021-08-25T18:48:13.015Z","caller":"instance/beat.go:989","message":"Exiting: could not create monitor: job err script monitors cannot be run as root! Current UID is 0"}
Exiting: could not create monitor: job err script monitors cannot be run as root! Current UID is 0

Workaround error by removing:

spec:
  deployment:
    podTemplate:
      spec:
        securityContext:
          runAsUser: 0

Get second, different error:
2.

{"level":"error","timestamp":"2021-08-25T18:52:05.809Z","caller":"instance/beat.go:989","message":"Exiting: Beat meta file failed to open: open /usr/share/heartbeat/data/meta.json: permission denied"}
Exiting: Beat meta file failed to open: open /usr/share/heartbeat/data/meta.json: permission denied

Environment

  • ECK version:

    1.7.0

  • Kubernetes information:

    • On premise: yes
    • Kubernetes distribution: Openshift / Rancher / PKS: rke2 v1.20.9+rke2r2
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.5", GitCommit:"6b1d87acf3c8253c123756b9e61dac642678305f", GitTreeState:"clean", BuildDate:"2021-03-18T01:10:43Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.9+rke2r2", GitCommit:"7a576bc3935a6b555e33346fd73ad77c925e9e4a", GitTreeState:"clean", BuildDate:"2021-08-05T23:06:55Z", GoVersion:"go1.15.14b5", Compiler:"gc", Platform:"linux/amd64"}
  • Resource definition:
---
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: heartbeat-synthetics
  namespace: beats
spec:
  config:
    heartbeat:
      monitors:
      - id: app
        name: App
        params:
          okta_url: <snipped_url>
          password: $MONITOR_PASSWORD
          username: $MONITOR_USERNAME
        schedule: '@every 1m'
        source:
          zip_url:
            folder: /
            password: $APP_GIT_PASSWORD
            url: <snipped_git_zip_url>
            username: $APP_GIT_USERNAME
        type: browser
    http:
      enabled: true
      host: 0.0.0.0
      port: 5066
    logging.json: true
  deployment:
    podTemplate:
      metadata:
        annotations:
          co.elastic.metrics/raw: '[{"enabled":true,"module":"beat","hosts":["http://${data.host}:5066"],"metricsets":["stats","state"],"period":"10s","timeout":"3s","xpack":{"enabled":true}}]'
          linkerd.io/inject: enabled
      spec:
        automountServiceAccountToken: true
        containers:
        - env:
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: MONITOR_USERNAME
            valueFrom:
              secretKeyRef:
                key: monitoring_username
                name: heartbeat-synthetics-monitoring
          - name: MONITOR_PASSWORD
            valueFrom:
              secretKeyRef:
                key: monitoring_password
                name: heartbeat-synthetics-monitoring
          - name: APP_GIT_USERNAME
            valueFrom:
              secretKeyRef:
                key: username
                name: heartbeat-synthetics-app-git
          - name: APP_GIT_PASSWORD
            valueFrom:
              secretKeyRef:
                key: password
                name: heartbeat-synthetics-app-git
          name: heartbeat
          resources: {}
        serviceAccount: heartbeat
    replicas: 1
    strategy:
      type: Recreate
  elasticsearchRef:
    name: es-prod
    namespace: elastic-prod
  kibanaRef:
    name: kibana-prod
    namespace: kibana-prod
  type: heartbeat
  version: 7.14.0
  • Logs:
{"level":"error","timestamp":"2021-08-25T18:48:13.015Z","caller":"instance/beat.go:989","message":"Exiting: could not create monitor: job err script monitors cannot be run as root! Current UID is 0"}
Exiting: could not create monitor: job err script monitors cannot be run as root! Current UID is 0

{"level":"error","timestamp":"2021-08-25T18:52:05.809Z","caller":"instance/beat.go:989","message":"Exiting: Beat meta file failed to open: open /usr/share/heartbeat/data/meta.json: permission denied"}
Exiting: Beat meta file failed to open: open /usr/share/heartbeat/data/meta.json: permission denied

Additional Info:

This appears to be a similar issue to: #4562 (comment)

The current example https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.7/config/recipes/beats/heartbeat_es_kb_health.yaml only shows heartbeat being run with runAsUser: 0.

Metadata

Metadata

Assignees

No one assigned

    Labels

    >bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions