Skip to content

Stand-alone agent: it's not possible to get agent documents from elasticsearch #20992

Closed
#21047
Closed
Stand-alone agent: it's not possible to get agent documents from elasticsearch#20992
#21047
Assignees
michalpristas
Labels
bugregression
@mdelapenya

Description

@mdelapenya
mdelapenya
opened on Sep 4, 2020
  • Version: 8.0.0-SNAPSHOT
  • Operating System:
    • Centos, Dockerised, using the official Stand-Alone image
    • Redhat UBI 8, using the official Stand-Alone image for UBI

This behaviour started to appear on Friday 4th. Before that, it worked as expected.

Steps to Reproduce

  1. Start the elastic stack (ES, Kibana and Package Registry) with docker-compose (I attached a compose file reproducing it: bug.zip
  2. Let the agent work for seconds
  3. Write down the hostname of the container representing the agent
  4. In Kibana, query elasticsearch for indexed documents, using the developer tools (apply $YOUR_HOSTNAME here) and the following query:
GET _search
{
  "version": true,
  "size": 500,
  "docvalue_fields": [
    {
      "field": "@timestamp",
      "format": "date_time"
    },
    {
      "field": "system.process.cpu.start_time",
      "format": "date_time"
    },
    {
      "field": "system.service.state_since",
      "format": "date_time"
    }
  ],
  "_source": {
    "excludes": []
  },
  "query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "bool": {
            "filter": [
              {
                "bool": {
                  "should": [
                    {
                      "match_phrase": {
                        "host.name": "${YOUR_HOSTNAME}"
                      }
                    }
                  ],
                  "minimum_should_match": 1
                }
              },
              {
                "bool": {
                  "should": [
                    {
                      "range": {
                        "@timestamp": {
                          "gte": "2020-09-03T14:57:04.948Z"
                        }
                      }
                    }
                  ],
                  "minimum_should_match": 1
                }
              }
            ]
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "2020-09-03T14:57:04.948Z",
              "format": "strict_date_optional_time"
            }
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  }
}

Search Result:

{
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 13,
    "successful" : 13,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

Agent Log

{"log.level":"info","@timestamp":"2020-09-04T19:45:42.447Z","log.origin":{"file.name":"warn/warn.go","file.line":18},"message":"The Elastic Agent is currently in BETA and should not be used in production","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:42.447Z","log.origin":{"file.name":"application/application.go","file.line":47},"message":"Detecting execution mode","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:42.448Z","log.origin":{"file.name":"application/application.go","file.line":56},"message":"Agent is managed locally","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:42.863Z","log.origin":{"file.name":"application/emitter.go","file.line":171},"message":"Supported programs: Endpoint Security, Filebeat, Heartbeat, Metricbeat","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:42.865Z","log.origin":{"file.name":"application/local_mode.go","file.line":132},"message":"Reloading of configuration is on, frequency is set to 10s","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:42.866Z","log.origin":{"file.name":"application/local_mode.go","file.line":143},"message":"Agent is starting","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:42.867Z","log.origin":{"file.name":"application/local_mode.go","file.line":153},"message":"Agent is stopped","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:42.867Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:42.867Z","log.origin":{"file.name":"application/periodic.go","file.line":76},"message":"Configuration changes detected","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:42.867Z","log.origin":{"file.name":"application/periodic.go","file.line":82},"message":"Updated 1 files: /usr/share/elastic-agent/elastic-agent.yml","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:42.871Z","log.origin":{"file.name":"application/emitter.go","file.line":145},"message":"Converting single configuration into specific programs configuration","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:42.873Z","log.origin":{"file.name":"application/router.go","file.line":65},"message":"Creating stream: default","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:42.966Z","log.origin":{"file.name":"application/emitter.go","file.line":145},"message":"Converting single configuration into specific programs configuration","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:42.969Z","log.origin":{"file.name":"application/router.go","file.line":65},"message":"Creating stream: default","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:43.632Z","log.origin":{"file.name":"application/router.go","file.line":80},"message":"Streams default need to run config with ID WXdGDNhI and programs: Metricbeat, FLEET_MONITORING","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:43.633Z","log.origin":{"file.name":"stateresolver/stateresolver.go","file.line":47},"message":"New State ID is WXdGDNhI","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:43.633Z","log.origin":{"file.name":"stateresolver/stateresolver.go","file.line":48},"message":"Converging state requires execution of 2 step(s)","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:43.633Z","log.origin":{"file.name":"operation/operator.go","file.line":250},"message":"operator is looking for metricbeat--8.0.0-SNAPSHOT in app collection: map[]","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:43.633Z","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"metricbeat.8.0.0-SNAPSHOT already exists in /usr/share/elastic-agent/data/elastic-agent-0032c0/downloads/metricbeat-8.0.0-SNAPSHOT-linux-x86_64.tar.gz. Skipping operation operation-fetch","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:43.634Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'retryable block: operation-fetch operation-verify' for metricbeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:43.634Z","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"metricbeat.8.0.0-SNAPSHOT already exists in /usr/share/elastic-agent/data/elastic-agent-0032c0/downloads/metricbeat-8.0.0-SNAPSHOT-linux-x86_64.tar.gz. Skipping operation operation-fetch","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:43.634Z","log.origin":{"file.name":"operation/operation_retryable.go","file.line":83},"message":"running operation 'operation-verify' of the block 'retryable block: operation-fetch operation-verify'","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:43.638Z","log.origin":{"file.name":"application/router.go","file.line":80},"message":"Streams default need to run config with ID WXdGDNhI and programs: Metricbeat, FLEET_MONITORING","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:43.638Z","log.origin":{"file.name":"stateresolver/stateresolver.go","file.line":47},"message":"New State ID is WXdGDNhI","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:43.638Z","log.origin":{"file.name":"stateresolver/stateresolver.go","file.line":48},"message":"Converging state requires execution of 2 step(s)","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:43.638Z","log.origin":{"file.name":"operation/operator.go","file.line":250},"message":"operator is looking for metricbeat--8.0.0-SNAPSHOT in app collection: map[]","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:43.638Z","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"metricbeat.8.0.0-SNAPSHOT already exists in /usr/share/elastic-agent/data/elastic-agent-0032c0/downloads/metricbeat-8.0.0-SNAPSHOT-linux-x86_64.tar.gz. Skipping operation operation-fetch","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:43.638Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'retryable block: operation-fetch operation-verify' for metricbeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:43.638Z","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"metricbeat.8.0.0-SNAPSHOT already exists in /usr/share/elastic-agent/data/elastic-agent-0032c0/downloads/metricbeat-8.0.0-SNAPSHOT-linux-x86_64.tar.gz. Skipping operation operation-fetch","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:43.638Z","log.origin":{"file.name":"operation/operation_retryable.go","file.line":83},"message":"running operation 'operation-verify' of the block 'retryable block: operation-fetch operation-verify'","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:43.788Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'operation-install' for metricbeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:43.797Z","log.origin":{"file.name":"operation/operator.go","file.line":226},"message":"operation 'operation-install' skipped for metricbeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:43.797Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'operation-start' for metricbeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:44.012Z","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2020-09-04T19:45:44Z: type: 'STATE': sub_type: 'STARTING' message: Application: metricbeat--8.0.0-SNAPSHOT[b6d2c1db-fb12-4095-8b1b-9f0d474859c3]: State changed to STARTING: Starting","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:44.013Z","log.origin":{"file.name":"application/periodic.go","file.line":28},"message":"Failed to read configuration, error: could not emit configuration: operator: failed to execute step sc-run, error: failed to start '/usr/share/elastic-agent/data/elastic-agent-0032c0/install/metricbeat-8.0.0-SNAPSHOT-linux-x86_64/metricbeat': fork/exec /usr/share/elastic-agent/data/elastic-agent-0032c0/install/metricbeat-8.0.0-SNAPSHOT-linux-x86_64/metricbeat: text file busy: failed to start '/usr/share/elastic-agent/data/elastic-agent-0032c0/install/metricbeat-8.0.0-SNAPSHOT-linux-x86_64/metricbeat': fork/exec /usr/share/elastic-agent/data/elastic-agent-0032c0/install/metricbeat-8.0.0-SNAPSHOT-linux-x86_64/metricbeat: text file busy","ecs.version":"1.5.0"}
{"log.level":"error","@timestamp":"2020-09-04T19:45:44.013Z","log.origin":{"file.name":"log/reporter.go","file.line":36},"message":"2020-09-04T19:45:44Z: type: 'ERROR': sub_type: 'FAILED' message: Application: metricbeat--8.0.0-SNAPSHOT[b6d2c1db-fb12-4095-8b1b-9f0d474859c3]: State changed to FAILED: failed to start '/usr/share/elastic-agent/data/elastic-agent-0032c0/install/metricbeat-8.0.0-SNAPSHOT-linux-x86_64/metricbeat': fork/exec /usr/share/elastic-agent/data/elastic-agent-0032c0/install/metricbeat-8.0.0-SNAPSHOT-linux-x86_64/metricbeat: text file busy","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:45.231Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'operation-start' for metricbeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:45.441Z","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2020-09-04T19:45:45Z: type: 'STATE': sub_type: 'STARTING' message: Application: metricbeat--8.0.0-SNAPSHOT[b6d2c1db-fb12-4095-8b1b-9f0d474859c3]: State changed to STARTING: Starting","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:45.446Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'operation-config' for metricbeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:45.451Z","log.origin":{"file.name":"operation/monitoring.go","file.line":256},"message":"monitoring configuration generated for filebeat: map[filebeat:map[inputs:[map[index:logs-elastic.agent-default json:map[keys_under_root:true message_key:message overwrite_keys:true] paths:[/usr/share/elastic-agent/data/elastic-agent-0032c0/logs/elastic-agent-json.log] processors:[map[add_fields:map[fields:map[dataset:elastic.agent namespace:default type:logs] target:data_stream]] map[add_fields:map[fields:map[dataset:elastic.agent] target:event]]] type:log] map[index:logs-elastic.agent.metricbeat-default json:map[keys_under_root:true message_key:message overwrite_keys:true] paths:[/usr/share/elastic-agent/data/elastic-agent-0032c0/logs/default/metricbeat-json.log] processors:[map[add_fields:map[fields:map[dataset:elastic.agent.metricbeat namespace:default type:logs] target:data_stream]] map[add_fields:map[fields:map[dataset:elastic.agent.metricbeat] target:event]]] type:log]]] output:map[elasticsearch:map[hosts:http://elasticsearch:9200 password:changeme type:elasticsearch username:elastic]]]","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:45.451Z","log.origin":{"file.name":"operation/monitoring.go","file.line":305},"message":"monitoring configuration generated for metricbeat: map[metricbeat:map[modules:[map[hosts:[http+unix:///tmp/elastic-agent/default/metricbeat/metricbeat.sock] index:metrics-elastic.agent.metricbeat-default metricsets:[stats state] module:beat period:10s processors:[map[add_fields:map[fields:map[dataset:elastic.agent.metricbeat namespace:default type:metrics] target:data_stream]] map[add_fields:map[fields:map[dataset:elastic.agent.metricbeat] target:event]]]]]] output:map[elasticsearch:map[hosts:http://elasticsearch:9200 password:changeme type:elasticsearch username:elastic]]]","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:45.451Z","log.origin":{"file.name":"operation/operator.go","file.line":250},"message":"operator is looking for filebeat--8.0.0-SNAPSHOT--36643631373035623733363936343635 in app collection: map[metricbeat--8.0.0-SNAPSHOT:0xc0003c37a0]","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:45.451Z","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"filebeat.8.0.0-SNAPSHOT already exists in /usr/share/elastic-agent/data/elastic-agent-0032c0/downloads/filebeat-8.0.0-SNAPSHOT-linux-x86_64.tar.gz. Skipping operation operation-fetch","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:45.451Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'retryable block: operation-fetch operation-verify' for filebeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:45.451Z","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"filebeat.8.0.0-SNAPSHOT already exists in /usr/share/elastic-agent/data/elastic-agent-0032c0/downloads/filebeat-8.0.0-SNAPSHOT-linux-x86_64.tar.gz. Skipping operation operation-fetch","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:45.451Z","log.origin":{"file.name":"operation/operation_retryable.go","file.line":83},"message":"running operation 'operation-verify' of the block 'retryable block: operation-fetch operation-verify'","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:45.611Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'operation-install' for filebeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:46.681Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'operation-start' for filebeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:47.073Z","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2020-09-04T19:45:47Z: type: 'STATE': sub_type: 'STARTING' message: Application: filebeat--8.0.0-SNAPSHOT--36643631373035623733363936343635[b6d2c1db-fb12-4095-8b1b-9f0d474859c3]: State changed to STARTING: Starting","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:47.073Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'operation-config' for filebeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:47.074Z","log.origin":{"file.name":"operation/operator.go","file.line":250},"message":"operator is looking for metricbeat--8.0.0-SNAPSHOT--36643631373035623733363936343635 in app collection: map[filebeat--8.0.0-SNAPSHOT--36643631373035623733363936343635:0xc00023bc20 metricbeat--8.0.0-SNAPSHOT:0xc0003c37a0]","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:47.074Z","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"metricbeat.8.0.0-SNAPSHOT already exists in /usr/share/elastic-agent/data/elastic-agent-0032c0/downloads/metricbeat-8.0.0-SNAPSHOT-linux-x86_64.tar.gz. Skipping operation operation-fetch","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:47.074Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'retryable block: operation-fetch operation-verify' for metricbeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:47.074Z","log.origin":{"file.name":"operation/operation_fetch.go","file.line":61},"message":"metricbeat.8.0.0-SNAPSHOT already exists in /usr/share/elastic-agent/data/elastic-agent-0032c0/downloads/metricbeat-8.0.0-SNAPSHOT-linux-x86_64.tar.gz. Skipping operation operation-fetch","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:47.074Z","log.origin":{"file.name":"operation/operation_retryable.go","file.line":83},"message":"running operation 'operation-verify' of the block 'retryable block: operation-fetch operation-verify'","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:47.151Z","log.origin":{"file.name":"operation/operator.go","file.line":226},"message":"operation 'operation-install' skipped for metricbeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:47.151Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'operation-start' for metricbeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:47.405Z","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2020-09-04T19:45:47Z: type: 'STATE': sub_type: 'STARTING' message: Application: metricbeat--8.0.0-SNAPSHOT--36643631373035623733363936343635[b6d2c1db-fb12-4095-8b1b-9f0d474859c3]: State changed to STARTING: Starting","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:47.405Z","log.origin":{"file.name":"operation/operator.go","file.line":230},"message":"running operation 'operation-config' for metricbeat.8.0.0-SNAPSHOT","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:47.406Z","log.origin":{"file.name":"stateresolver/stateresolver.go","file.line":61},"message":"Updating internal state","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:49.683Z","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2020-09-04T19:45:49Z: type: 'STATE': sub_type: 'RUNNING' message: Application: metricbeat--8.0.0-SNAPSHOT[b6d2c1db-fb12-4095-8b1b-9f0d474859c3]: State changed to RUNNING: Running","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:51.162Z","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2020-09-04T19:45:51Z: type: 'STATE': sub_type: 'RUNNING' message: Application: filebeat--8.0.0-SNAPSHOT--36643631373035623733363936343635[b6d2c1db-fb12-4095-8b1b-9f0d474859c3]: State changed to RUNNING: Running","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:51.572Z","log.origin":{"file.name":"log/reporter.go","file.line":40},"message":"2020-09-04T19:45:51Z: type: 'STATE': sub_type: 'RUNNING' message: Application: metricbeat--8.0.0-SNAPSHOT--36643631373035623733363936343635[b6d2c1db-fb12-4095-8b1b-9f0d474859c3]: State changed to RUNNING: Running","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:54.014Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:54.014Z","log.origin":{"file.name":"application/periodic.go","file.line":76},"message":"Configuration changes detected","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:54.014Z","log.origin":{"file.name":"application/periodic.go","file.line":82},"message":"Updated 1 files: /usr/share/elastic-agent/elastic-agent.yml","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:54.017Z","log.origin":{"file.name":"application/emitter.go","file.line":145},"message":"Converting single configuration into specific programs configuration","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:45:54.018Z","log.origin":{"file.name":"application/router.go","file.line":80},"message":"Streams default need to run config with ID WXdGDNhI and programs: Metricbeat, FLEET_MONITORING","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:54.019Z","log.origin":{"file.name":"stateresolver/stateresolver.go","file.line":47},"message":"New State ID is WXdGDNhI","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:54.019Z","log.origin":{"file.name":"stateresolver/stateresolver.go","file.line":48},"message":"Converging state requires execution of 0 step(s)","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:54.019Z","log.origin":{"file.name":"stateresolver/stateresolver.go","file.line":61},"message":"Updating internal state","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:45:54.019Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:46:04.020Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:46:04.020Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:46:14.020Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:46:14.021Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:46:24.023Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:46:24.024Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:46:34.024Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:46:34.024Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:46:44.024Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:46:44.025Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:46:54.026Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:46:54.026Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:47:04.026Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:47:04.026Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:47:14.026Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:47:14.027Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:47:24.028Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:47:24.028Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:47:34.029Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:47:34.029Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
{"log.level":"debug","@timestamp":"2020-09-04T19:47:44.029Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:47:44.030Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}
c{"log.level":"debug","@timestamp":"2020-09-04T19:47:54.032Z","log.origin":{"file.name":"application/periodic.go","file.line":60},"message":"Adding 1 file to watch","ecs.version":"1.5.0"}
{"log.level":"info","@timestamp":"2020-09-04T19:47:54.032Z","log.origin":{"file.name":"application/periodic.go","file.line":98},"message":"No configuration change","ecs.version":"1.5.0"}

where I can see only one error entry:

{"log.level":"error","@timestamp":"2020-09-04T19:45:44.013Z","log.origin":{"file.name":"log/reporter.go","file.line":36},"message":"2020-09-04T19:45:44Z: type: 'ERROR': sub_type: 'FAILED' message: Application: metricbeat--8.0.0-SNAPSHOT[b6d2c1db-fb12-4095-8b1b-9f0d474859c3]: State changed to FAILED: failed to start '/usr/share/elastic-agent/data/elastic-agent-0032c0/install/metricbeat-8.0.0-SNAPSHOT-linux-x86_64/metricbeat': fork/exec /usr/share/elastic-agent/data/elastic-agent-0032c0/install/metricbeat-8.0.0-SNAPSHOT-linux-x86_64/metricbeat: text file busy","ecs.version":"1.5.0"}

cc/ @EricDavisX

Metadata

Metadata

Assignees

Labels

bugregression

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions