V2: mage package will use old, incompatible Beat versions without warning

To reproduce, build the elastic-agent docker container from the head of main and run it. You will immediately see error messages about components failing:

```sh
DEV=true SNAPSHOT=true PLATFORMS=linux/arm64 PACKAGES=docker mage package
docker run docker.elastic.co/beats/elastic-agent-complete:8.6.0-SNAPSHOT
```

You will logs like:

```json
Exiting: error loading config file: stat metricbeat.yml: no such file or directory
{"log.level":"info","@timestamp":"2022-11-10T22:10:59.283Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":325},"message":"Existing component state changed","component":{"id":"http/metrics-monitoring","state":"Failed","message":"Failed: pid '192' exited with code '1'","inputs":[{"id":"http/metrics-monitoring-metrics-monitoring-agent","state":"Failed","message":"Failed: pid '192' exited with code '1'"}],"output":{"id":"http/metrics-monitoring","state":"Failed","message":"Failed: pid '192' exited with code '1'"}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-10T22:11:08.942Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":325},"message":"Existing component state changed","component":{"id":"filestream-monitoring","state":"Starting","message":"Starting: spawned pid '201'","inputs":[{"id":"filestream-monitoring-logs-monitoring-agent","state":"Starting","message":"Starting: spawned pid '201'"}],"output":{"id":"filestream-monitoring","state":"Starting","message":"Starting: spawned pid '201'"}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-10T22:11:08.993Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":325},"message":"Existing component state changed","component":{"id":"system/metrics-default","state":"Starting","message":"Starting: spawned pid '209'","inputs":[{"id":"system/metrics-default-system/metrics","state":"Starting","message":"Starting: spawned pid '209'"}],"output":{"id":"system/metrics-default","state":"Starting","message":"Starting: spawned pid '209'"}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-10T22:11:09.044Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":325},"message":"Existing component state changed","component":{"id":"beat/metrics-monitoring","state":"Starting","message":"Starting: spawned pid '219'","inputs":[{"id":"beat/metrics-monitoring-metrics-monitoring-beats","state":"Starting","message":"Starting: spawned pid '219'"}],"output":{"id":"beat/metrics-monitoring","state":"Starting","message":"Starting: spawned pid '219'"}},"ecs.version":"1.6.0"}
Exiting: error loading config file: stat filebeat.yml: no such file or directory
{"log.level":"info","@timestamp":"2022-11-10T22:11:09.067Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":325},"message":"Existing component state changed","component":{"id":"filestream-monitoring","state":"Failed","message":"Failed: pid '201' exited with code '1'","inputs":[{"id":"filestream-monitoring-logs-monitoring-agent","state":"Failed","message":"Failed: pid '201' exited with code '1'"}],"output":{"id":"filestream-monitoring","state":"Failed","message":"Failed: pid '201' exited with code '1'"}},"ecs.version":"1.6.0"}
Exiting: error loading config file: stat metricbeat.yml: no such file or directory
{"log.level":"info","@timestamp":"2022-11-10T22:11:09.135Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":325},"message":"Existing component state changed","component":{"id":"system/metrics-default","state":"Failed","message":"Failed: pid '209' exited with code '1'","inputs":[{"id":"system/metrics-default-system/metrics","state":"Failed","message":"Failed: pid '209' exited with code '1'"}],"output":{"id":"system/metrics-default","state":"Failed","message":"Failed: pid '209' exited with code '1'"}},"ecs.version":"1.6.0"}
Exiting: error loading config file: stat metricbeat.yml: no such file or directory
{"log.level":"info","@timestamp":"2022-11-10T22:11:09.175Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":325},"message":"Existing component state changed","component":{"id":"beat/metrics-monitoring","state":"Failed","message":"Failed: pid '219' exited with code '1'","inputs":[{"id":"beat/metrics-monitoring-metrics-monitoring-beats","state":"Failed","message":"Failed: pid '219' exited with code '1'"}],"output":{"id":"beat/metrics-monitoring","state":"Failed","message":"Failed: pid '219' exited with code '1'"}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-11-10T22:11:09.289Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":325},"message":"Existing component state changed","component":{"id":"http/metrics-monitoring","state":"Starting","message":"Starting: spawned pid '227'","inputs":[{"id":"http/metrics-monitoring-metrics-monitoring-agent","state":"Starting","message":"Starting: spawned pid '227'"}],"output":{"id":"http/metrics-monitoring","state":"Starting","message":"Starting: spawned pid '227'"}},"ecs.version":"1.6.0"}
Exiting: error loading config file: stat metricbeat.yml: no such file or directory
{"log.level":"info","@timestamp":"2022-11-10T22:11:09.410Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":325},"message":"Existing component state changed","component":{"id":"http/metrics-monitoring","state":"Failed","message":"Failed: pid '227' exited with code '1'","inputs":[{"id":"http/metrics-monitoring-metrics-monitoring-agent","state":"Failed","message":"Failed: pid '227' exited with code '1'"}],"output":{"id":"http/metrics-monitoring","state":"Failed","message":"Failed: pid '227' exited with code '1'"}},"ecs.version":"1.6.0"}
```

Exec-ing into the container to check status reports every component as failed:

```sh
elastic-agent@f38bb03309f1:~$ elastic-agent status
State: DEGRADED
Message: 1 or more components/units in a failed state
Components:
  * system/metrics  (FAILED)
                    Failed: pid '568' exited with code '1'
  * filestream      (FAILED)
                    Failed: pid '561' exited with code '1'
  * beat/metrics    (FAILED)
                    Failed: pid '579' exited with code '1'
  * http/metrics    (FAILED)
                    Failed: pid '580' exited with code '1'
```

I can see that the `metricbeat.yml` file reported as missing is located in the components directory:

```sh
elastic-agent@f38bb03309f1:~$ pwd
/usr/share/elastic-agent
elastic-agent@f38bb03309f1:~$ ls -la data/elastic-agent-e7419d/components/
total 930180
drwxrwx---  1 root root      4096 Nov 10 20:29 .
drwxrwx---  1 root root      4096 Nov 10 20:30 ..
-rw-rw----  1 root root        41 Nov 10 20:28 .build_hash.txt
-rw-rw----  1 root root     13675 Nov 10 20:28 LICENSE.txt
-rw-rw----  1 root root   2566303 Nov 10 20:28 NOTICE.txt
-rw-rw----  1 root root       840 Nov 10 20:28 README.md
drwxrwx---  2 root root      4096 Nov 10 20:28 certs
-rw-r--r--  1 root root      1303 Nov 10 20:28 checksum.yml
-rw-r--r--  1 root root    389399 Nov 10 20:28 fields.yml
-rwxr-xr-x  1 root root 178477751 Nov 10 20:28 filebeat
-rw-r--r--  1 root root    174363 Nov 10 20:28 filebeat.reference.yml
-rw-r--r--  1 root root      4117 Nov 10 20:28 filebeat.spec.yml
-rw-r--r--  1 root root      8622 Nov 10 20:28 filebeat.yml
-rwxr-xr-x  1 root root 161152969 Nov 10 20:28 heartbeat
-rw-r--r--  1 root root     67937 Nov 10 20:28 heartbeat.reference.yml
-rw-r--r--  1 root root      1057 Nov 10 20:28 heartbeat.spec.yml
-rw-r--r--  1 root root      7276 Nov 10 20:28 heartbeat.yml
drwxrwx---  4 root root      4096 Nov 10 20:28 kibana
-rwxr-xr-x  1 root root 239101161 Nov 10 20:28 metricbeat
-rw-r--r--  1 root root    103498 Nov 10 20:28 metricbeat.reference.yml
-rw-r--r--  1 root root      4403 Nov 10 20:28 metricbeat.spec.yml
-rw-r--r--  1 root root      6899 Nov 10 20:28 metricbeat.yml
drwxrwx--- 84 root root      4096 Nov 10 20:29 module
drwxrwx---  2 root root      4096 Nov 10 20:29 modules.d
drwxrwx---  2 root root      4096 Nov 10 20:29 monitors.d
-rw-rw----  1 root root   5526616 Nov 10 20:28 osquery-extension.ext
-rwxr-xr-x  1 root root 146923101 Nov 10 20:28 osquerybeat
-rw-r--r--  1 root root     43600 Nov 10 20:28 osquerybeat.reference.yml
-rw-r--r--  1 root root       584 Nov 10 20:28 osquerybeat.spec.yml
-rw-r--r--  1 root root      6504 Nov 10 20:28 osquerybeat.yml
-rw-rw----  1 root root 217818752 Nov 10 20:28 osqueryd
```

Edit: the restart problem is a separate bug https://github.com/elastic/elastic-agent/issues/1713

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

V2: mage package will use old, incompatible Beat versions without warning #1709

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

V2: mage package will use old, incompatible Beat versions without warning #1709

Description

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions