Skip to content

Cherry-pick #19767 to 7.x: add_process_metadata processor adds container id even if process metadata not accessible#20417

Merged
ChrsMark merged 2 commits intoelastic:7.xfrom
ChrsMark:backport_19767_7.x
Aug 4, 2020
Merged

Cherry-pick #19767 to 7.x: add_process_metadata processor adds container id even if process metadata not accessible#20417
ChrsMark merged 2 commits intoelastic:7.xfrom
ChrsMark:backport_19767_7.x

Conversation

@ChrsMark
Copy link
Copy Markdown
Member

@ChrsMark ChrsMark commented Aug 4, 2020 •
edited by zube bot
Loading

Cherry-pick of PR #19767 to 7.x branch. Original message:

  • Enhancement

What does this PR do?

Changed add_process_metadata processor to get both process metadata and container id and not error out if only one is available.

Why is it important?

If container is non-privileged, then process metadata for external processes is not fully readable (no access to /proc/pid/exe and /proc/pid/cwd) and code errors out before even trying to get container id. Same time container id is still accessible in /proc/pid/cgroup file. Now process metadata is skipped for such processes, but container id is still added.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [] I have made corresponding changes to the documentation
  • [] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • [] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Minor fix, probably no need for doc or changelog.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

@ChrsMark ChrsMark added [zube]: In Review backport Team:Platforms Label for the Integrations - Platforms team labels Aug 4, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Aug 4, 2020

Pinging @elastic/integrations-platforms (Team:Platforms)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Aug 4, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Aug 4, 2020 •
edited by jenkins-beats-ci bot
Loading

💔 Build Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #20417 updated]

  • Start Time: 2020-08-04T08:50:24.044+0000

  • Duration: 83 min 53 sec

Test stats 🧪

Test Results
Failed 0
Passed 14398
Skipped 1773
Total 16171

Steps errors

Expand to view the steps failures

  • Name: General Build Step

    • Description: [2020-08-04T09:15:04.773Z] [Google Cloud Storage Plugin] Found 1 files to download from pattern: gs:

    • Duration: 3 min 58 sec

    • Start Time: 2020-08-04T09:15:04.705+0000

    • log

  • Name: General Build Step

    • Description: [2020-08-04T09:15:00.792Z] [Google Cloud Storage Plugin] Found 1 files to download from pattern: gs:

    • Duration: 3 min 57 sec

    • Start Time: 2020-08-04T09:15:00.735+0000

    • log

  • Name: Install docker-compose 1.21.0

    • Description: .ci/scripts/install-docker-compose.sh

    • Duration: 1 min 57 sec

    • Start Time: 2020-08-04T09:18:35.056+0000

    • log

  • Name: General Build Step

    • Description: [2020-08-04T09:15:08.588Z] [Google Cloud Storage Plugin] Found 1 files to download from pattern: gs:

    • Duration: 3 min 57 sec

    • Start Time: 2020-08-04T09:15:08.506+0000

    • log

  • Name: General Build Step

    • Description: [2020-08-04T09:15:51.590Z] [Google Cloud Storage Plugin] Found 1 files to download from pattern: gs:

    • Duration: 3 min 57 sec

    • Start Time: 2020-08-04T09:15:51.512+0000

    • log

  • Name: General Build Step

    • Description: [2020-08-04T09:15:04.905Z] [Google Cloud Storage Plugin] Found 1 files to download from pattern: gs:

    • Duration: 3 min 58 sec

    • Start Time: 2020-08-04T09:15:04.817+0000

    • log

  • Name: General Build Step

    • Description: [2020-08-04T09:15:52.756Z] [Google Cloud Storage Plugin] Found 1 files to download from pattern: gs:

    • Duration: 3 min 57 sec

    • Start Time: 2020-08-04T09:15:52.702+0000

    • log

  • Name: General Build Step

    • Description: [2020-08-04T09:15:18.660Z] [Google Cloud Storage Plugin] Found 1 files to download from pattern: gs:

    • Duration: 3 min 57 sec

    • Start Time: 2020-08-04T09:15:18.597+0000

    • log

Log output

Expand to view the last 100 lines of log output 

[2020-08-04T10:12:46.429Z] + make mage
[2020-08-04T10:12:46.429Z] Installing mage v1.10.0.
[2020-08-04T10:12:47.447Z] go: downloading github.com/magefile/mage v1.10.0
[2020-08-04T10:12:48.842Z] /var/lib/jenkins/workspace/Beats_beats_PR-20417/.magefile cleaned
[2020-08-04T10:12:49.183Z] + git config --get user.email
[2020-08-04T10:12:49.183Z] + [ -z  ]
[2020-08-04T10:12:49.183Z] + git config user.email beatsmachine@users.noreply.github.com
[2020-08-04T10:12:49.183Z] + git config user.name beatsmachine
[2020-08-04T10:12:49.580Z] + .ci/scripts/terraform-cleanup.sh x-pack/metricbeat
[2020-08-04T10:12:49.580Z] + DIRECTORY=x-pack/metricbeat
[2020-08-04T10:12:49.580Z] + FAILED=0
[2020-08-04T10:12:49.580Z] ++ find x-pack/metricbeat -name terraform.tfstate
[2020-08-04T10:12:49.580Z] + exit 0
[2020-08-04T10:12:50.296Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats
[2020-08-04T10:12:50.617Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-08-04T10:12:50.630Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Lint
[2020-08-04T10:12:50.718Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Winlogbeat-oss
[2020-08-04T10:12:50.799Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-08-04T10:12:50.889Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-08-04T10:12:50.966Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Dockerlogbeat
[2020-08-04T10:12:51.048Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-08-04T10:12:51.138Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-08-04T10:12:51.231Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-08-04T10:12:51.313Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-08-04T10:12:51.406Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Auditbeat-oss-Windows
[2020-08-04T10:12:51.489Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-08-04T10:12:51.572Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-08-04T10:12:51.652Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Auditbeat-x-pack-Windows
[2020-08-04T10:12:51.729Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-08-04T10:12:51.805Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Auditbeat-oss-Linux
[2020-08-04T10:12:51.883Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Filebeat-Windows
[2020-08-04T10:12:51.968Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Libbeat-x-pack
[2020-08-04T10:12:52.053Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Auditbeat-oss-Mac-OS-X
[2020-08-04T10:12:52.128Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-08-04T10:12:52.222Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-08-04T10:12:52.296Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-08-04T10:12:52.378Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-Windows
[2020-08-04T10:12:52.461Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-08-04T10:12:52.548Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-08-04T10:12:52.624Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Auditbeat-x-pack-Mac-OS-X
[2020-08-04T10:12:52.704Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-08-04T10:12:52.782Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-08-04T10:12:52.858Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Filebeat-oss
[2020-08-04T10:12:52.942Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Filebeat-x-pack
[2020-08-04T10:12:53.027Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Libbeat-oss
[2020-08-04T10:12:53.104Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-OSS-Go-Integration-tests
[2020-08-04T10:12:53.182Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Functionbeat-Windows
[2020-08-04T10:12:53.257Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Libbeat-crosscompile
[2020-08-04T10:12:53.336Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-08-04T10:12:53.409Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Libbeat-stress-tests
[2020-08-04T10:12:53.489Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-08-04T10:12:53.565Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-08-04T10:12:53.936Z] + cat
[2020-08-04T10:12:53.936Z] + /usr/local/bin/runbld ./runbld-script
[2020-08-04T10:12:53.936Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-08-04T10:13:00.552Z] runbld>>> runbld started
[2020-08-04T10:13:00.552Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-08-04T10:13:02.471Z] runbld>>> The following profiles matched the job 'Beats/beats/PR-20417' in order of occurrence in the config (last value wins).
[2020-08-04T10:13:03.946Z] runbld>>> Debug logging enabled.
[2020-08-04T10:13:03.946Z] runbld>>> Storing result
[2020-08-04T10:13:04.214Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-08-04T10:13:04.214Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200804101303-24F7AD6F
[2020-08-04T10:13:04.214Z] runbld>>> Adding system facts.
[2020-08-04T10:13:05.183Z] runbld>>> Adding vcs info for the latest commit:  63c0b4c2e9744e0a2706b4def1fe6273a56826c0
[2020-08-04T10:13:05.183Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-08-04T10:13:05.447Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-08-04T10:13:05.447Z] Processing JUnit reports with runbld...
[2020-08-04T10:13:05.447Z] + echo 'Processing JUnit reports with runbld...'
[2020-08-04T10:13:05.725Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-08-04T10:13:05.725Z] runbld>>> DURATION: 41ms
[2020-08-04T10:13:05.725Z] runbld>>> STDOUT: 40 bytes
[2020-08-04T10:13:05.725Z] runbld>>> STDERR: 49 bytes
[2020-08-04T10:13:05.725Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-08-04T10:13:05.725Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats
[2020-08-04T10:13:07.132Z] runbld>>> Storing build metadata: 
[2020-08-04T10:13:07.132Z] runbld>>> Adding test report.
[2020-08-04T10:13:07.132Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats
[2020-08-04T10:13:07.707Z] runbld>>> Found 117 test output files
[2020-08-04T10:13:08.671Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-08-04T10:13:08.671Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-08-04T10:13:08.671Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-08-04T10:13:08.671Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-08-04T10:13:08.671Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-08-04T10:13:08.671Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-OSS-Go-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-08-04T10:13:08.934Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-20417/src/github.com/elastic/beats/Metricbeat-OSS-Go-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-08-04T10:13:10.873Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 16025 Skipped: 1501
[2020-08-04T10:13:10.873Z] runbld>>> Storing result
[2020-08-04T10:13:10.873Z] runbld>>> FAILURES: 0
[2020-08-04T10:13:11.141Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-08-04T10:13:11.141Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200804101303-24F7AD6F
[2020-08-04T10:13:11.141Z] runbld>>> Email notification disabled by environment variable.
[2020-08-04T10:13:11.141Z] runbld>>> Slack notification disabled by environment variable.
[2020-08-04T10:13:16.838Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-20417
[2020-08-04T10:13:16.940Z] [INFO] getVaultSecret: Getting secrets
[2020-08-04T10:13:17.029Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-08-04T10:13:17.793Z] + chmod 755 generate-build-data.sh
[2020-08-04T10:13:17.793Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20417/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20417/runs/2 FAILURE 4973489
[2020-08-04T10:13:17.793Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20417/runs/2/steps/?limit=10000 -o steps-info.json
[2020-08-04T10:13:24.274Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20417/runs/2/tests/?status=FAILED -o tests-errors.json
[2020-08-04T10:13:24.824Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-20417/runs/2/log/ -o pipeline-log.txt

narph
narph reviewed Aug 4, 2020
View reviewed changes
@ChrsMark ChrsMark merged commit 8ba3811 into elastic:7.x Aug 4, 2020
@zube zube bot removed the [zube]: Done label Nov 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@narph narph narph approved these changes

Assignees

No one assigned

Labels

backport Team:Platforms Label for the Integrations - Platforms team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ChrsMark @elasticmachine @narph @jtinkus