libbeat: monitor version by graphaelli · Pull Request #26214 · elastic/beats

graphaelli · 2021-06-08T17:16:36Z

What does this PR do?

Expand monitoring to report beat.info.version.

Why is it important?

When monitoring multiple beats version would be a useful dimension to break down metrics.

Checklist

My code follows the style guidelines of this project
~~[ ] I have made corresponding changes to the documentation~~
~~[ ] I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Start a beat and note that verison information is reported:

INFO	[monitoring]	log/log.go:153	Total non-zero metrics	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":40,"time":{"ms":40}},"total":{"ticks":131,"time":{"ms":131},"value":131},"user":{"ticks":91,"time":{"ms":91}}},"info":{"ephemeral_id":"d0703db5-bc9d-45d8-a67b-993e1626339a","uptime":{"ms":5961},"version":"8.0.0"},"memstats":{"gc_next":16014480,"memory_alloc":13199072,"memory_sys":77153288,"memory_total":32009184,"rss":57188352},"runtime":{"goroutines":20}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":1},"output":{"events":{"active":0},"type":"elasticsearch"},"pipeline":{"clients":0,"events":{"active":0},"queue":{"max_events":4096}}},"system":{"cpu":{"cores":16},"load":{"1":11.8486,"15":5.3057,"5":8.9258,"norm":{"1":0.7405,"15":0.3316,"5":0.5579}}}}}}

Expand monitoring to report beat.info.version

elasticmachine · 2021-06-08T17:17:28Z

Pinging @elastic/agent (Team:Agent)

elasticmachine · 2021-06-08T17:20:23Z

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: graphaelli commented: /test
Start Time: 2021-06-09T02:53:59.916+0000
Duration: 39 min 17 sec
Commit: 96fea9a

Trends 🧪

Steps errors

Expand to view the steps failures

`x-pack/elastic-agent-packaging-arm-arm - mage package`

Took 22 min 51 sec . View more details on here
Description: mage package

`Error signal`

Took 0 min 0 sec . View more details on here
Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-06-09T03:31:42.195Z]  Experimental:      true
[2021-06-09T03:31:42.195Z] 
[2021-06-09T03:31:42.195Z] Server: Docker Engine - Community
[2021-06-09T03:31:42.195Z]  Engine:
[2021-06-09T03:31:42.195Z]   Version:          20.10.3
[2021-06-09T03:31:42.195Z]   API version:      1.41 (minimum version 1.12)
[2021-06-09T03:31:42.195Z]   Go version:       go1.13.15
[2021-06-09T03:31:42.195Z]   Git commit:       46229ca
[2021-06-09T03:31:42.195Z]   Built:            Fri Jan 29 14:31:49 2021
[2021-06-09T03:31:42.195Z]   OS/Arch:          linux/arm64
[2021-06-09T03:31:42.195Z]   Experimental:     false
[2021-06-09T03:31:42.195Z]  containerd:
[2021-06-09T03:31:42.195Z]   Version:          1.4.3
[2021-06-09T03:31:42.195Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-06-09T03:31:42.195Z]  runc:
[2021-06-09T03:31:42.195Z]   Version:          1.0.0-rc92
[2021-06-09T03:31:42.195Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-06-09T03:31:42.195Z]  docker-init:
[2021-06-09T03:31:42.195Z]   Version:          0.19.0
[2021-06-09T03:31:42.195Z]   GitCommit:        de40ad0
[2021-06-09T03:31:42.195Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-06-09T03:31:42.195Z] Unable to find image 'arm64v8/alpine:3' locally
[2021-06-09T03:31:43.262Z] 3: Pulling from arm64v8/alpine
[2021-06-09T03:31:43.262Z] 595b0fe564bb: Pulling fs layer
[2021-06-09T03:31:43.771Z] 595b0fe564bb: Verifying Checksum
[2021-06-09T03:31:43.771Z] 595b0fe564bb: Download complete
[2021-06-09T03:31:43.771Z] 595b0fe564bb: Pull complete
[2021-06-09T03:31:43.771Z] Digest: sha256:8f18fae117ec6e5777cc62ba78cbb3be10a8a38639ccfb949521abd95c8301a4
[2021-06-09T03:31:43.771Z] Status: Downloaded newer image for arm64v8/alpine:3
[2021-06-09T03:31:56.754Z] Change permissions with write access of all files inside the specific folder
[2021-06-09T03:31:56.960Z] Running in /var/lib/jenkins/workspace/PR-26214-8-5560b5fc-1d7e-4950-94bb-f8d76f61647d/src/github.com/elastic/beats/build
[2021-06-09T03:31:58.012Z] + rm -rf ve
[2021-06-09T03:31:58.012Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-06-09T03:31:59.086Z] + python .ci/scripts/pre_archive_test.py
[2021-06-09T03:32:04.378Z] Copy ./x-pack/heartbeat/build into build/x-pack/heartbeat/build
[2021-06-09T03:32:04.379Z] Copy ./x-pack/heartbeat/build/golang-crossbuild into build/x-pack/heartbeat/build/golang-crossbuild
[2021-06-09T03:32:04.379Z] Copy ./x-pack/elastic-agent/build into build/x-pack/elastic-agent/build
[2021-06-09T03:32:04.379Z] Copy ./x-pack/elastic-agent/build/golang-crossbuild into build/x-pack/elastic-agent/build/golang-crossbuild
[2021-06-09T03:32:04.379Z] Copy ./x-pack/metricbeat/build into build/x-pack/metricbeat/build
[2021-06-09T03:32:04.379Z] Copy ./x-pack/metricbeat/build/golang-crossbuild into build/x-pack/metricbeat/build/golang-crossbuild
[2021-06-09T03:32:04.379Z] Copy ./x-pack/filebeat/build into build/x-pack/filebeat/build
[2021-06-09T03:32:04.379Z] Copy ./x-pack/filebeat/build/golang-crossbuild into build/x-pack/filebeat/build/golang-crossbuild
[2021-06-09T03:32:04.581Z] Running in /var/lib/jenkins/workspace/PR-26214-8-5560b5fc-1d7e-4950-94bb-f8d76f61647d/src/github.com/elastic/beats/build
[2021-06-09T03:32:04.658Z] Recording test results
[2021-06-09T03:32:08.504Z] None of the test reports contained any result
[2021-06-09T03:32:08.505Z] [Checks API] No suitable checks publisher found.
[2021-06-09T03:32:09.381Z] + tar --version
[2021-06-09T03:32:10.504Z] + tar --exclude=test-build-artifacts-x-pack-elastic-agent-packaging-arm-arm-tgz -czf test-build-artifacts-x-pack-elastic-agent-packaging-arm-arm-tgz .
[2021-06-09T03:32:42.602Z] + gsutil --version
[2021-06-09T03:32:54.081Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-06-09T03:32:55.435Z] + gcloud auth activate-service-account --key-file ****
[2021-06-09T03:32:57.121Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-06-09T03:32:58.252Z] + gsutil -m -q cp -a public-read test-build-artifacts-x-pack-elastic-agent-packaging-arm-arm-tgz gs://beats-ci-temp/Beats/beats/PR-26214-8
[2021-06-09T03:33:04.692Z] + python .ci/scripts/search_system_tests.py
[2021-06-09T03:33:04.968Z] [INFO] system-tests=''. If no empty then let's create a tarball
[2021-06-09T03:33:05.869Z] + go clean -modcache
[2021-06-09T03:33:10.288Z] Cleaning up /var/lib/jenkins/workspace/PR-26214-8-5560b5fc-1d7e-4950-94bb-f8d76f61647d
[2021-06-09T03:33:10.289Z] Client: Docker Engine - Community
[2021-06-09T03:33:10.289Z]  Version:           20.10.3
[2021-06-09T03:33:10.289Z]  API version:       1.41
[2021-06-09T03:33:10.289Z]  Go version:        go1.13.15
[2021-06-09T03:33:10.289Z]  Git commit:        48d30b5
[2021-06-09T03:33:10.289Z]  Built:             Fri Jan 29 14:33:34 2021
[2021-06-09T03:33:10.289Z]  OS/Arch:           linux/arm64
[2021-06-09T03:33:10.289Z]  Context:           default
[2021-06-09T03:33:10.289Z]  Experimental:      true
[2021-06-09T03:33:10.289Z] 
[2021-06-09T03:33:10.289Z] Server: Docker Engine - Community
[2021-06-09T03:33:10.289Z]  Engine:
[2021-06-09T03:33:10.289Z]   Version:          20.10.3
[2021-06-09T03:33:10.289Z]   API version:      1.41 (minimum version 1.12)
[2021-06-09T03:33:10.289Z]   Go version:       go1.13.15
[2021-06-09T03:33:10.289Z]   Git commit:       46229ca
[2021-06-09T03:33:10.289Z]   Built:            Fri Jan 29 14:31:49 2021
[2021-06-09T03:33:10.289Z]   OS/Arch:          linux/arm64
[2021-06-09T03:33:10.289Z]   Experimental:     false
[2021-06-09T03:33:10.289Z]  containerd:
[2021-06-09T03:33:10.289Z]   Version:          1.4.3
[2021-06-09T03:33:10.289Z]   GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
[2021-06-09T03:33:10.289Z]  runc:
[2021-06-09T03:33:10.289Z]   Version:          1.0.0-rc92
[2021-06-09T03:33:10.289Z]   GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
[2021-06-09T03:33:10.289Z]  docker-init:
[2021-06-09T03:33:10.289Z]   Version:          0.19.0
[2021-06-09T03:33:10.289Z]   GitCommit:        de40ad0
[2021-06-09T03:33:10.289Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-06-09T03:33:11.363Z] Change permissions with write access of all files inside the specific folder
[2021-06-09T03:33:11.576Z] Running in /var/lib/jenkins/workspace/PR-26214-8-5560b5fc-1d7e-4950-94bb-f8d76f61647d
[2021-06-09T03:33:15.753Z] Failed in branch x-pack/elastic-agent-packaging-arm-arm
[2021-06-09T03:33:15.814Z] Stage "Packaging-Pipeline" skipped due to earlier failure(s)
[2021-06-09T03:33:15.860Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-26214/src/github.com/elastic/beats
[2021-06-09T03:33:16.152Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-26214
[2021-06-09T03:33:16.193Z] [INFO] getVaultSecret: Getting secrets
[2021-06-09T03:33:16.230Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-06-09T03:33:16.862Z] + chmod 755 generate-build-data.sh
[2021-06-09T03:33:16.862Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-26214/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-26214/runs/8 FAILURE 2356686
[2021-06-09T03:33:16.862Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-26214/runs/8/steps/?limit=10000 -o steps-info.json
[2021-06-09T03:33:18.206Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-26214/runs/8/tests/?status=FAILED -o tests-errors.json
[2021-06-09T03:33:18.206Z] Retry 1/3 exited 22, retrying in 1 seconds...
[2021-06-09T03:33:19.549Z] Retry 2/3 exited 22, retrying in 2 seconds...

❕ Flaky test report

No test was executed to be analysed.

ph

🤦 we didn't do that!

graphaelli · 2021-06-09T02:49:55Z

test failure is unrelated to these changes

graphaelli · 2021-06-09T02:53:44Z

/test

(cherry picked from commit 383f689)

* master: (26 commits) Report total and free CPU for vSphere virtual machines (elastic#26167) [filebeat] Add preserve_original_event option to o365audit input (elastic#26273) Change xml processor names in script processor to match convention (elastic#26263) [Oracle] Fixing default values for paths in config template (elastic#26276) Add more ECS fields to logs (elastic#25998) [Heartbeat] Fix broken invocation of synth package (elastic#26228) rename sqs file name (elastic#26227) Populate the agent action result if there is no matching action handlers (elastic#26152) Add ISO8601 as supported timestamp type (elastic#25564) Move Filebeat azure module to GA (elastic#26168) Filebeat azure module pipeline fixes and changes (elastic#26148) libbeat: monitor version (elastic#26214) Add new parser to filestream input: container (elastic#26115) [Metricbeat] Add state_statefulset replicas.ready (elastic#26088) Disable test processors system test for windows 10 (elastic#26216) Fix startup with failing configuration (elastic#26126) Remove 32 bits version of Elastic Agent. (elastic#25708) Chane fleetmode detection to ony use management.enabled (elastic#26180) Make `filestream` input GA (elastic#26127) libbeat/idxmgmt/ilm: fix alias creation (elastic#26146) ...

libbeat: monitor version

310e2ab

Expand monitoring to report beat.info.version

graphaelli added enhancement v7.14.0 Team:Elastic-Agent Label for the Agent team labels Jun 8, 2021

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 8, 2021

supply pr # in changelog

32a8639

graphaelli marked this pull request as ready for review June 8, 2021 17:17

graphaelli added 2 commits June 8, 2021 13:21

lint

baa5153

add license header

9dfccf4

ph approved these changes Jun 8, 2021

View reviewed changes

make update

fe34604

urso approved these changes Jun 8, 2021

View reviewed changes

graphaelli added 2 commits June 8, 2021 20:06

Merge branch 'master' into monitor-version

6d09203

Merge branch 'master' into monitor-version

96fea9a

graphaelli mentioned this pull request Jun 9, 2021

[Monitoring] Monitor beat version elastic/elasticsearch#73932

Merged

graphaelli added the backport-v7.14.0 Automated backport with mergify label Jun 9, 2021

graphaelli mentioned this pull request Jun 9, 2021

Add new metrics about queue size and output workers to template #25628

Closed

urso merged commit 383f689 into elastic:master Jun 9, 2021

graphaelli deleted the monitor-version branch June 9, 2021 13:05

mergify bot pushed a commit that referenced this pull request Jun 9, 2021

libbeat: monitor version (#26214)

4a850ed

(cherry picked from commit 383f689)

mergify bot mentioned this pull request Jun 9, 2021

[7.x](backport #26214) libbeat: monitor version #26222

Merged

graphaelli pushed a commit that referenced this pull request Jun 9, 2021

libbeat: monitor version (#26214) (#26222)

ac0ef8c

(cherry picked from commit 383f689)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libbeat: monitor version#26214

libbeat: monitor version#26214
urso merged 7 commits intoelastic:masterfrom
graphaelli:monitor-version

graphaelli commented Jun 8, 2021 •

edited

Loading

Uh oh!

elasticmachine commented Jun 8, 2021

Uh oh!

elasticmachine commented Jun 8, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Trends 🧪

`x-pack/elastic-agent-packaging-arm-arm - mage package`

`Error signal`

Uh oh!

ph left a comment

Uh oh!

graphaelli commented Jun 9, 2021

Uh oh!

graphaelli commented Jun 9, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

graphaelli commented Jun 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

How to test this PR locally

Uh oh!

elasticmachine commented Jun 8, 2021

Uh oh!

elasticmachine commented Jun 8, 2021 • edited by jenkins-beats-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Build stats

Trends 🧪

Steps errors

x-pack/elastic-agent-packaging-arm-arm - mage package

Error signal

Log output

❕ Flaky test report

Uh oh!

ph left a comment

Choose a reason for hiding this comment

Uh oh!

graphaelli commented Jun 9, 2021

Uh oh!

graphaelli commented Jun 9, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

graphaelli commented Jun 8, 2021 •

edited

Loading

elasticmachine commented Jun 8, 2021 •

edited by jenkins-beats-ci bot

Loading

`x-pack/elastic-agent-packaging-arm-arm - mage package`

`Error signal`