Skip to content

Cherry-pick #12905 to 7.x: Allow the Docker image to be run with a random user id#18634

Merged
jsoriano merged 1 commit intoelastic:7.xfrom
jsoriano:backport_12905_7.x
May 19, 2020
Merged

Cherry-pick #12905 to 7.x: Allow the Docker image to be run with a random user id#18634
jsoriano merged 1 commit intoelastic:7.xfrom
jsoriano:backport_12905_7.x

Conversation

@jsoriano
Copy link
Copy Markdown
Member

Cherry-pick of PR #12905 to 7.x branch. Original message:

On secured Kubernetes environments (not only Openshift) the user ID used to run a container can't be known in advance. Consequently the APM server container can't be started on these environments because it expects to run with the user 1000 or 0.

This PR brings some compatibility with such environments, based on the fact that on secured Kubernetes clusters and on Openshift the only thing you know is that the user is always a member of the root group.

You can find more details here: https://docs.openshift.com/container-platform/3.11/creating_images/guidelines.html#openshift-specific-guidelines

See elastic/apm-server#2325 and #12686

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 19, 2020
@jsoriano jsoriano force-pushed the backport_12905_7.x branch from 217192b to 7e5e882 Compare May 19, 2020 10:04
Modify docker images so files required by beats are owned
by group root, this follows Openshifts recommendations to
run containerized applications with custom user ids.

(cherry picked from commit 9dbdc15)

Co-Authored-By: Michael Morello <michael.morello@elastic.co>
@jsoriano jsoriano force-pushed the backport_12905_7.x branch from 7e5e882 to 05d3b2f Compare May 19, 2020 10:05
@jsoriano jsoriano self-assigned this May 19, 2020
@jsoriano jsoriano added the Team:Platforms Label for the Integrations - Platforms team label May 19, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/integrations-platforms (Team:Platforms)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 19, 2020
@jsoriano
Copy link
Copy Markdown
Member Author

run beats-ci/package

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented May 19, 2020

💔 Build Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

Test stats 🧪

Test Results
Failed 0
Passed 6468
Skipped 1018
Total 7486

Steps errors

Expand to view the steps failures

  • Name: Mage build test
    • Description: mage build test

    • Result: FAILURE

    • Duration: 8 min 7 sec

    • Start Time: 2020-05-19T10:32:41.406+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-05-19T11:11:23.114Z] >> go test: Integration-zookeeper Test Passed
[2020-05-19T11:11:23.176Z] Recording test results
[2020-05-19T11:11:29.281Z] Stashed 39 file(s)
[2020-05-19T11:11:29.291Z] Archiving artifacts
[2020-05-19T11:11:30.538Z] + curl -sSLo codecov https://codecov.io/bash
[2020-05-19T11:11:31.114Z] + FILE=auditbeat/build/coverage/full.cov
[2020-05-19T11:11:31.114Z] + [ -f auditbeat/build/coverage/full.cov ]
[2020-05-19T11:11:31.114Z] + FILE=filebeat/build/coverage/full.cov
[2020-05-19T11:11:31.114Z] + [ -f filebeat/build/coverage/full.cov ]
[2020-05-19T11:11:31.114Z] + FILE=heartbeat/build/coverage/full.cov
[2020-05-19T11:11:31.114Z] + [ -f heartbeat/build/coverage/full.cov ]
[2020-05-19T11:11:31.114Z] + FILE=libbeat/build/coverage/full.cov
[2020-05-19T11:11:31.114Z] + [ -f libbeat/build/coverage/full.cov ]
[2020-05-19T11:11:31.114Z] + FILE=metricbeat/build/coverage/full.cov
[2020-05-19T11:11:31.114Z] + [ -f metricbeat/build/coverage/full.cov ]
[2020-05-19T11:11:31.114Z] + FILE=packetbeat/build/coverage/full.cov
[2020-05-19T11:11:31.114Z] + [ -f packetbeat/build/coverage/full.cov ]
[2020-05-19T11:11:31.114Z] + FILE=winlogbeat/build/coverage/full.cov
[2020-05-19T11:11:31.114Z] + [ -f winlogbeat/build/coverage/full.cov ]
[2020-05-19T11:11:31.114Z] + FILE=journalbeat/build/coverage/full.cov
[2020-05-19T11:11:31.114Z] + [ -f journalbeat/build/coverage/full.cov ]
[2020-05-19T11:11:32.519Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats
[2020-05-19T11:11:32.822Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-05-19T11:11:32.835Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Lint
[2020-05-19T11:11:32.910Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Winlogbeat-oss
[2020-05-19T11:11:32.972Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-05-19T11:11:33.036Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Dockerlogbeat
[2020-05-19T11:11:33.113Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Journalbeat-oss
[2020-05-19T11:11:33.180Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-05-19T11:11:33.259Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-05-19T11:11:33.324Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-05-19T11:11:33.409Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-05-19T11:11:33.495Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-05-19T11:11:33.584Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Heartbeat-oss
[2020-05-19T11:11:33.724Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-05-19T11:11:33.843Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-05-19T11:11:33.919Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Libbeat-x-pack
[2020-05-19T11:11:33.997Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Auditbeat-Linux
[2020-05-19T11:11:34.071Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Packetbeat-oss
[2020-05-19T11:11:34.152Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Filebeat-Windows
[2020-05-19T11:11:34.246Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-05-19T11:11:34.338Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-05-19T11:11:34.412Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-05-19T11:11:34.486Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Metricbeat-Windows
[2020-05-19T11:11:34.564Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-05-19T11:11:34.637Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Filebeat-x-pack
[2020-05-19T11:11:34.716Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-05-19T11:11:34.788Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-05-19T11:11:34.862Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Heartbeat-Windows
[2020-05-19T11:11:35.105Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Functionbeat-Windows
[2020-05-19T11:11:35.189Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Filebeat-oss
[2020-05-19T11:11:35.266Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Auditbeat-Windows
[2020-05-19T11:11:35.343Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Libbeat-oss
[2020-05-19T11:11:35.422Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Metricbeat-Python-integration-tests
[2020-05-19T11:11:35.504Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Libbeat-crosscompile
[2020-05-19T11:11:35.594Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Libbeat-stress-tests
[2020-05-19T11:11:35.672Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests
[2020-05-19T11:11:36.030Z] + cat
[2020-05-19T11:11:36.030Z] + /usr/local/bin/runbld ./runbld-script
[2020-05-19T11:11:36.030Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-05-19T11:11:42.619Z] runbld>>> runbld started
[2020-05-19T11:11:42.619Z] runbld>>> 1.6.11/a66728ff8f4356963772e6e6d2069392fa06acbe
[2020-05-19T11:11:44.526Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-18634' in order of occurrence in the config (last value wins).
[2020-05-19T11:11:45.903Z] runbld>>> Debug logging enabled.
[2020-05-19T11:11:45.903Z] runbld>>> Storing result
[2020-05-19T11:11:45.903Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-05-19T11:11:45.903Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200519111145-32A67D1F
[2020-05-19T11:11:45.903Z] runbld>>> Adding system facts.
[2020-05-19T11:11:46.844Z] runbld>>> Adding vcs info for the latest commit:  05d3b2f9c2d4048b60dc488d4cb5993380e60ad9
[2020-05-19T11:11:47.104Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-05-19T11:11:47.104Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-05-19T11:11:47.364Z] + echo 'Processing JUnit reports with runbld...'
[2020-05-19T11:11:47.364Z] Processing JUnit reports with runbld...
[2020-05-19T11:11:47.624Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-05-19T11:11:47.624Z] runbld>>> DURATION: 9ms
[2020-05-19T11:11:47.624Z] runbld>>> STDOUT: 40 bytes
[2020-05-19T11:11:47.624Z] runbld>>> STDERR: 49 bytes
[2020-05-19T11:11:47.624Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-05-19T11:11:47.624Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats
[2020-05-19T11:11:49.007Z] runbld>>> Storing build metadata: 
[2020-05-19T11:11:49.007Z] runbld>>> Adding test report.
[2020-05-19T11:11:49.007Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats
[2020-05-19T11:11:49.942Z] runbld>>> Found 75 test output files
[2020-05-19T11:11:51.327Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-05-19T11:11:51.327Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-05-19T11:11:51.897Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 7336 Skipped: 828
[2020-05-19T11:11:52.156Z] runbld>>> Storing result
[2020-05-19T11:11:52.156Z] runbld>>> FAILURES: 0
[2020-05-19T11:11:52.416Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-05-19T11:11:52.416Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200519111145-32A67D1F
[2020-05-19T11:11:52.416Z] runbld>>> Email notification disabled by environment variable.
[2020-05-19T11:11:52.416Z] runbld>>> Slack notification disabled by environment variable.
[2020-05-19T11:11:58.674Z] Running on worker-395930 in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18634
[2020-05-19T11:11:58.891Z] [INFO] getVaultSecret: Getting secrets
[2020-05-19T11:11:58.941Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-05-19T11:12:00.996Z] + chmod 755 generate-build-data.sh
[2020-05-19T11:12:00.996Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18634/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18634/runs/3 FAILURE 3905359
[2020-05-19T11:12:00.996Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18634/runs/3/steps/?limit=10000 -o steps-info.json
[2020-05-19T11:12:02.469Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18634/runs/3/tests/?status=FAILED -o tests-errors.json
[2020-05-19T11:12:03.946Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18634/runs/3/log/ -o pipeline-log.txt

Copy link
Copy Markdown
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

backport lgtm!

@jsoriano jsoriano merged commit 455f234 into elastic:7.x May 19, 2020
@jsoriano jsoriano deleted the backport_12905_7.x branch May 20, 2020 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport review Team:Platforms Label for the Integrations - Platforms team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants