Build docker images based on Red Hat UBI by jsoriano · Pull Request #20576 · elastic/beats

jsoriano · 2020-08-12T16:54:16Z

What does this PR do?

Add an additional docker build that builds images based on Red Hat UBI, following Red Hat requirements for certified images.

Additional checks have been added to packaging tests for labels and licenses.

Additional changes done to support it also in Elastic Agent images:

Apply Prepare home directories for docker images in a different stage #20356 and Allow the Docker image to be run with a random user id (#12905) #18873 to agent Dockerfile.
Explicitly select a Dockerfile and entry point template, selecting it based on the image name is error prone, as our image names can have suffixes (-oss, -ubi8...).
Add NOTICE.txt file to all agent packages.
Actually run package tests after building packages, added flag to allow root user.
Improved checks on required packages, so they are not always built, what significantly speeds up local builds.

NOTE: Some of the changes for Elastic Agent could be actually moved to separate PRs, but they are quite intertwined and I would like to test them together. I decided to do them on this PR to don't have to conditionally disable package tests, and to be able to test everything together. But happy to move them to other PRs if preferred.

Why is it important?

To build images that can be certified for Red Hat OpenShift.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

Move licenses to /licenses
- Add the proper licenses to x-pack and OSS images
Add labels
Add test to check that licenses are in place
Add test to check that required labels are present
Add build for Elastic Agent

Possibly to be done in a follow up PR

Refactor how files are prepared and copied, so it is not needed to change permissions in Dockerfile
Unify Dockerfiles for Agent and the rest of beats, as they are quite similar
Add checks for the included packages in Agent image

How to test this PR locally

Related issues

elasticmachine · 2020-08-12T18:05:59Z

💚 Build Succeeded

Expand to view the summary

Build stats

Build Cause: [Pull request #20576 updated]
Start Time: 2020-08-17T09:46:57.159+0000
Duration: 74 min 33 sec

Test stats 🧪

Test	Results
Failed	0
Passed	18002
Skipped	1827
Total	19829

Steps errors

Expand to view the steps failures

Name: Install Go 1.14.7
- Description: .ci/scripts/install-go.sh
- Duration: 1 min 47 sec
- Start Time: 2020-08-17T10:11:56.253+0000
- log

urso · 2020-08-13T13:24:28Z

/packaging

jsoriano · 2020-08-13T19:15:04Z

/packaging

elasticmachine · 2020-08-17T09:52:22Z

Pinging @elastic/integrations (Team:Integrations)

elasticmachine · 2020-08-17T09:52:23Z

Pinging @elastic/integrations-platforms (Team:Platforms)

jsoriano · 2020-08-17T09:52:31Z

/packaging

axw

I don't know enough to review the agent bits, but the rest looks good. Thanks!

Just one minor question around the labels, which we might not want in non-UBI images. Probably doesn't matter though.

dev-tools/packaging/templates/docker/Dockerfile.tmpl

elasticmachine · 2020-08-18T02:39:00Z

Pinging @elastic/ingest-management (Team:Ingest Management)

andrewkroh

LGTM.

andrewkroh · 2020-08-19T14:18:46Z

x-pack/elastic-agent/magefile.go

+
+	var requiredPackages []string
+	for _, p := range platformPackages {
+		if _, enabled := devtools.Platforms.Get(p.platform); enabled {


ph · 2020-08-19T21:25:10Z

@blakerouse can you take a look?

michalpristas

Looks good

michalpristas · 2020-08-24T08:33:10Z

dev-tools/mage/dockerbuilder.go

-	if err != nil {
-		// specific missing fallback to generic
-		dockerfile = "Dockerfile.tmpl"
+	dockerfile := "Dockerfile.tmpl"


Add an additional docker build that builds images based on Red Hat UBI, following Red Hat requirements for certified images. Additional checks have been added to packaging tests for labels and licenses. Additional changes done to support it also in Elastic Agent images: * Home directory is prepared in a different stage (elastic#20356). * Allow the docker image to be run with random user ids (elastic#18873). * Explicitly select a Dockerfile and entry point template. * Add NOTICE.txt file to all agent packages. * Actually run package tests after building packages, added flag to allow root user. * Improved checks on required packages, so they are not re-built if they already are. (cherry picked from commit e31794d)

Add an additional docker build that builds images based on Red Hat UBI, following Red Hat requirements for certified images. Additional checks have been added to packaging tests for labels and licenses. Additional changes done to support it also in Elastic Agent images: * Home directory is prepared in a different stage (#20356). * Allow the docker image to be run with random user ids (#18873). * Explicitly select a Dockerfile and entry point template. * Add NOTICE.txt file to all agent packages. * Actually run package tests after building packages, added flag to allow root user. * Improved checks on required packages, so they are not re-built if they already are. (cherry picked from commit e31794d)

Add an additional docker build that builds images based on Red Hat UBI, following Red Hat requirements for certified images. Additional checks have been added to packaging tests for labels and licenses. Additional changes done to support it also in Elastic Agent images: * Home directory is prepared in a different stage (elastic#20356). * Allow the docker image to be run with random user ids (elastic#18873). * Explicitly select a Dockerfile and entry point template. * Add NOTICE.txt file to all agent packages. * Actually run package tests after building packages, added flag to allow root user. * Improved checks on required packages, so they are not re-built if they already are.

elastic#20576

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 12, 2020

jsoriano added Team:Integrations Label for the Integrations team Team:Platforms Label for the Integrations - Platforms team labels Aug 12, 2020

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 12, 2020

jsoriano self-assigned this Aug 12, 2020

jsoriano added :Packaging enhancement labels Aug 12, 2020

jsoriano force-pushed the ubi-docker-images branch from 8d33902 to 6cf2143 Compare August 13, 2020 19:13

jsoriano added 14 commits August 17, 2020 11:33

Recover previous changes for UBI images

32fd76c

Copy licenses

0f03307

Use ubi-minimal

e136dcc

Move licenses to the expected location

655595f

Add labels

ddd4f2f

Use latest ubi-minimal for UBI docker builds

90a1b76

Add more labels

4898800

Copy proper licenses and add test for them

60b413c

Add checks for labels

1c5ee97

Fix package tests for elastic-agent

902f574

Explicitly select Dockerfile and entrypoint

d7933ff

Leverage the use of PLATFORMS when selecting required packages for agent

38085a8

Fix package tests for elastic-agent

f978f4a

Install jq in ubi image for agent

b7ef2b0

jsoriano force-pushed the ubi-docker-images branch from 46ebfa6 to b7ef2b0 Compare August 17, 2020 09:34

Add changelog entry

e1539d1

jsoriano marked this pull request as ready for review August 17, 2020 09:52

jsoriano requested review from axw and urso August 17, 2020 09:53

axw approved these changes Aug 18, 2020

View reviewed changes

dev-tools/packaging/templates/docker/Dockerfile.tmpl Show resolved Hide resolved

botelastic bot added the Team:Ingest Management label Aug 18, 2020

andrewkroh approved these changes Aug 19, 2020

View reviewed changes

jsoriano requested review from a team and removed request for urso August 20, 2020 08:27

michalpristas approved these changes Aug 24, 2020

View reviewed changes

jsoriano merged commit e31794d into elastic:master Aug 24, 2020

jsoriano deleted the ubi-docker-images branch August 24, 2020 09:06

jsoriano mentioned this pull request Aug 24, 2020

Cherry-pick #20576 to 7.x: Build docker images based on Red Hat UBI #20753

Merged

15 tasks

jsoriano added the v7.10.0 label Aug 24, 2020

axw mentioned this pull request Aug 25, 2020

Update to elastic/beats@49e8024953a4 elastic/apm-server#4105

Merged

3 tasks

jsoriano mentioned this pull request Sep 5, 2020

Support running Auditbeat and Elastic Agent docker images without root #20996

Closed

v1v mentioned this pull request Sep 21, 2020

Add option to run UBI8-minimal based stack images elastic/apm-integration-testing#900

Closed

v1v added a commit to v1v/beats that referenced this pull request Sep 29, 2021

Usean extra spec similar to the ubi8 variant

093a855

elastic#20576

github-actions bot mentioned this pull request Mar 27, 2026

[Filebeat] Improve Go integration test target to package Docker image #49754

Open

Conversation

jsoriano commented Aug 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Uh oh!

elasticmachine commented Aug 12, 2020 • edited by jenkins-beats-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

Steps errors

Uh oh!

urso commented Aug 13, 2020

Uh oh!

jsoriano commented Aug 13, 2020

Uh oh!

elasticmachine commented Aug 17, 2020

Uh oh!

elasticmachine commented Aug 17, 2020

Uh oh!

jsoriano commented Aug 17, 2020

Uh oh!

axw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elasticmachine commented Aug 18, 2020

Uh oh!

andrewkroh left a comment

Choose a reason for hiding this comment

Uh oh!

andrewkroh Aug 19, 2020

Choose a reason for hiding this comment

Uh oh!

ph commented Aug 19, 2020

Uh oh!

michalpristas left a comment

Choose a reason for hiding this comment

Uh oh!

michalpristas Aug 24, 2020

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

jsoriano commented Aug 12, 2020 •

edited

Loading

elasticmachine commented Aug 12, 2020 •

edited by jenkins-beats-ci bot

Loading