feat: add CentOS 8 support by mdelapenya · Pull Request #2034 · elastic/e2e-testing

mdelapenya · 2022-01-20T10:35:52Z

chore: rename platforms to include OS
fix: include sles15 in Beats builds
chore: expand coverage for SLES15
chore: run k8s-autodiscover on Debian VMs
chore: run helm test suite on Debian VMs
chore: remove unused platforms
feat: add centos8 support, running fleet test suite

What does this PR do?

This PR adds CentOS 8 as a backend to run the tests, using the AMIs provided by the official Centos website: https://www.centos.org/download/aws-images/

We have also refactored the platforms YAML file to identify the platform with the OS name: i.e. fleet_amd64 > Debian_amd64, also removing duplicated entries.

Finally, we are expanding the feature coverage for SLES15, running all fleet test suites on those boxes.

Why is it important?

Expand OS/Arch coverage

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have run the Unit tests (make unit-test), and they are passing locally
I have run the End-2-End tests for the suite I'm working on, and they are passing locally
I have noticed new Go dependencies (run make notice in the proper directory)

Related issues

Closes Add CentOS 8 support #2033

elasticmachine · 2022-01-20T11:03:38Z

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

Name: Initializing / End-To-End Tests / debian_amd64_apm-server / [empty] – TEST-x86_64-helm-e127680e-2022-01-25-15:12:34.xml
Name: Initializing / End-To-End Tests / debian_amd64_backend_processes / Deploying the agent – Backend Processes
Name: Initializing / End-To-End Tests / debian_amd64_backend_processes / Stopping the agent stops backend processes – Backend Processes
Name: Initializing / End-To-End Tests / debian_amd64_backend_processes / Restarting the installed agent – Backend Processes
Name: Initializing / End-To-End Tests / debian_amd64_backend_processes / Un-enrolling the agent stops backend processes – Backend Processes
Name: Initializing / End-To-End Tests / debian_amd64_backend_processes / Re-enrolling the agent starts the elastic-agent process – Backend Processes
Name: Initializing / End-To-End Tests / debian_amd64_backend_processes / Un-installing the installed agent – Backend Processes
Name: Initializing / End-To-End Tests / debian_amd64_backend_processes / Un-enrolling Elastic Agent stops Elastic Endpoint – Backend Processes
Name: Initializing / End-To-End Tests / debian_amd64_backend_processes / Removing Endpoint from Agent policy stops the connected Endpoint – Backend Processes
Name: Initializing / End-To-End Tests / sles15_backend_processes / Deploying the agent – Backend Processes
Name: Initializing / End-To-End Tests / sles15_backend_processes / Stopping the agent stops backend processes – Backend Processes
Name: Initializing / End-To-End Tests / sles15_backend_processes / Restarting the installed agent – Backend Processes
Name: Initializing / End-To-End Tests / sles15_backend_processes / Un-enrolling the agent stops backend processes – Backend Processes
Name: Initializing / End-To-End Tests / sles15_backend_processes / Re-enrolling the agent starts the elastic-agent process – Backend Processes
Name: Initializing / End-To-End Tests / sles15_backend_processes / Un-installing the installed agent – Backend Processes
Name: Initializing / End-To-End Tests / sles15_backend_processes / Un-enrolling Elastic Agent stops Elastic Endpoint – Backend Processes
Name: Initializing / End-To-End Tests / sles15_backend_processes / Removing Endpoint from Agent policy stops the connected Endpoint – Backend Processes
Name: Initializing / End-To-End Tests / debian_arm64_backend_processes / Deploying the agent – Backend Processes
Name: Initializing / End-To-End Tests / debian_arm64_backend_processes / Stopping the agent stops backend processes – Backend Processes
Name: Initializing / End-To-End Tests / debian_arm64_backend_processes / Restarting the installed agent – Backend Processes
Name: Initializing / End-To-End Tests / debian_arm64_backend_processes / Un-enrolling the agent stops backend processes – Backend Processes
Name: Initializing / End-To-End Tests / debian_arm64_backend_processes / Re-enrolling the agent starts the elastic-agent process – Backend Processes
Name: Initializing / End-To-End Tests / debian_arm64_backend_processes / Un-installing the installed agent – Backend Processes
Name: Initializing / End-To-End Tests / debian_arm64_backend_processes / Un-enrolling Elastic Agent stops Elastic Endpoint – Backend Processes
Name: Initializing / End-To-End Tests / debian_arm64_backend_processes / Removing Endpoint from Agent policy stops the connected Endpoint – Backend Processes
Name: Initializing / End-To-End Tests / debian_amd64_fleet_mode_agent / Deploying the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_amd64_fleet_mode_agent / Restarting the installed agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_amd64_fleet_mode_agent / Un-enrolling the agent deactivates the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_amd64_fleet_mode_agent / Re-enrolling the agent activates the agent in Fleet – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_amd64_fleet_mode_agent / Revoking the enrollment token for the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_amd64_fleet_mode_agent / Un-installing the installed agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / centos8_amd64_fleet_mode_agent / Deploying the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / centos8_amd64_fleet_mode_agent / Restarting the installed agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / centos8_amd64_fleet_mode_agent / Un-enrolling the agent deactivates the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / centos8_amd64_fleet_mode_agent / Re-enrolling the agent activates the agent in Fleet – Fleet Mode Agent
Name: Initializing / End-To-End Tests / centos8_amd64_fleet_mode_agent / Revoking the enrollment token for the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / centos8_amd64_fleet_mode_agent / Un-installing the installed agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / sles15_fleet_mode_agent / Deploying the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / sles15_fleet_mode_agent / Restarting the installed agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / sles15_fleet_mode_agent / Un-enrolling the agent deactivates the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / sles15_fleet_mode_agent / Re-enrolling the agent activates the agent in Fleet – Fleet Mode Agent
Name: Initializing / End-To-End Tests / sles15_fleet_mode_agent / Revoking the enrollment token for the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / sles15_fleet_mode_agent / Un-installing the installed agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_arm64_fleet_mode_agent / Deploying the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_arm64_fleet_mode_agent / Restarting the installed agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_arm64_fleet_mode_agent / Un-enrolling the agent deactivates the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_arm64_fleet_mode_agent / Re-enrolling the agent activates the agent in Fleet – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_arm64_fleet_mode_agent / Revoking the enrollment token for the agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_arm64_fleet_mode_agent / Un-installing the installed agent – Fleet Mode Agent
Name: Initializing / End-To-End Tests / debian_amd64_linux_integration / Adding the Linux Integration to an Agent ... – Linux Integration
Name: Initializing / End-To-End Tests / debian_arm64_linux_integration / Adding the Linux Integration to an Agent ... – Linux Integration
Name: Initializing / End-To-End Tests / sles15_linux_integration / Adding the Linux Integration to an Agent ... – Linux Integration
Name: Initializing / End-To-End Tests / debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #1 – Running on top of Beats
Name: Initializing / End-To-End Tests / debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #1 – Running on top of Beats
Name: Initializing / End-To-End Tests / debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #2 – Running on top of Beats
Name: Initializing / End-To-End Tests / debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #2 – Running on top of Beats
Name: Initializing / End-To-End Tests / debian_arm64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #1 – Running on top of Beats
Name: Initializing / End-To-End Tests / debian_arm64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #1 – Running on top of Beats
Name: Initializing / End-To-End Tests / debian_arm64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #2 – Running on top of Beats
Name: Initializing / End-To-End Tests / debian_arm64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #2 – Running on top of Beats
Name: Initializing / End-To-End Tests / sles15_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #1 – Running on top of Beats
Name: Initializing / End-To-End Tests / sles15_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #1 – Running on top of Beats
Name: Initializing / End-To-End Tests / sles15_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #2 – Running on top of Beats
Name: Initializing / End-To-End Tests / sles15_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #2 – Running on top of Beats

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

It comes without Python installed

.ci/ansible/tasks/setup_test_script.yml

This reverts commit c432675.

user_data us used to execute arbitrary code after a VM is provisioned on AWS

.ci/Jenkinsfile

This reverts commit 90adcdc.

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

.ci/ansible/playbook.yml

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

adam-stokes · 2022-01-24T18:58:37Z

CentOS support is there now, we just need to resolve the artifact version missing

mdelapenya · 2022-01-24T20:37:26Z

CentOS support is there now, we just need to resolve the artifact version missing

We can merge and resolve the bump in another PR

.ci/ansible/files/user-data-centos.sh

* chore: rename platforms to include OS * fix: include sles15 in Beats builds * chore: expand coverage for SLES15 * chore: run k8s-autodiscover on Debian VMs * chore: run helm test suite on Debian VMs * chore: remove unused platforms * feat: add centos8 support, running fleet test suite * fix: install python for CentOS It comes without Python installed * fix: pass suite to Ansible context * fix: use ansible os family * fix: typo * chore: try with platform * Revert "chore: try with platform" This reverts commit c432675. * chore: set user_data for CentOS user_data us used to execute arbitrary code after a VM is provisioned on AWS * fix: use Python interpreter for Ansible in CentOS * Revert "fix: use Python interpreter for Ansible in CentOS" This reverts commit 90adcdc. * Set proper python interpreter based on OS Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * try with auto Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * remove user_data for now Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * chore: remove user data file Co-authored-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> (cherry picked from commit aec2a7a)

* chore: rename platforms to include OS * fix: include sles15 in Beats builds * chore: expand coverage for SLES15 * chore: run k8s-autodiscover on Debian VMs * chore: run helm test suite on Debian VMs * chore: remove unused platforms * feat: add centos8 support, running fleet test suite * fix: install python for CentOS It comes without Python installed * fix: pass suite to Ansible context * fix: use ansible os family * fix: typo * chore: try with platform * Revert "chore: try with platform" This reverts commit c432675. * chore: set user_data for CentOS user_data us used to execute arbitrary code after a VM is provisioned on AWS * fix: use Python interpreter for Ansible in CentOS * Revert "fix: use Python interpreter for Ansible in CentOS" This reverts commit 90adcdc. * Set proper python interpreter based on OS Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * try with auto Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * remove user_data for now Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * chore: remove user data file Co-authored-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> (cherry picked from commit aec2a7a) # Conflicts: # .ci/.e2e-tests-for-elastic-agent.yaml # .ci/.e2e-tests.yaml # .ci/Jenkinsfile # .ci/ansible/playbook.yml # .ci/ansible/tasks/setup_test_script.yml

* chore: rename platforms to include OS * fix: include sles15 in Beats builds * chore: expand coverage for SLES15 * chore: run k8s-autodiscover on Debian VMs * chore: run helm test suite on Debian VMs * chore: remove unused platforms * feat: add centos8 support, running fleet test suite * fix: install python for CentOS It comes without Python installed * fix: pass suite to Ansible context * fix: use ansible os family * fix: typo * chore: try with platform * Revert "chore: try with platform" This reverts commit c432675. * chore: set user_data for CentOS user_data us used to execute arbitrary code after a VM is provisioned on AWS * fix: use Python interpreter for Ansible in CentOS * Revert "fix: use Python interpreter for Ansible in CentOS" This reverts commit 90adcdc. * Set proper python interpreter based on OS Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * try with auto Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * remove user_data for now Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * chore: remove user data file Co-authored-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> (cherry picked from commit aec2a7a) Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

* feat: add CentOS 8 support (#2034) * chore: rename platforms to include OS * fix: include sles15 in Beats builds * chore: expand coverage for SLES15 * chore: run k8s-autodiscover on Debian VMs * chore: run helm test suite on Debian VMs * chore: remove unused platforms * feat: add centos8 support, running fleet test suite * fix: install python for CentOS It comes without Python installed * fix: pass suite to Ansible context * fix: use ansible os family * fix: typo * chore: try with platform * Revert "chore: try with platform" This reverts commit c432675. * chore: set user_data for CentOS user_data us used to execute arbitrary code after a VM is provisioned on AWS * fix: use Python interpreter for Ansible in CentOS * Revert "fix: use Python interpreter for Ansible in CentOS" This reverts commit 90adcdc. * Set proper python interpreter based on OS Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * try with auto Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * remove user_data for now Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> * chore: remove user data file Co-authored-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> (cherry picked from commit aec2a7a) # Conflicts: # .ci/.e2e-tests-for-elastic-agent.yaml # .ci/.e2e-tests.yaml # .ci/Jenkinsfile # .ci/ansible/playbook.yml # .ci/ansible/tasks/setup_test_script.yml * fix: resolve conflicts Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

* main: (45 commits) feat: add CentOS 8 support (elastic#2034) fix: set default region for AWS cli (elastic#2053) chore: use Ansible's built-in replace instead of sed (elastic#2048) chore: split stack configuration and start into two tasks (elastic#2044) feat: enable SSH access to users for debugging cloud instances (elastic#2001) fix: use the right branch for 7.17 backports (elastic#2025) SLES15 enablement (elastic#2007) chore: bump stale agent for main (elastic#2014) Update `fetchBeatsBinary` to be reused in elastic-agent-poc (elastic#1984) chore: add resiliency when provisioning the stack (elastic#1990) chore: bump elastic-package to v0.32.1 (elastic#1959) feat: export Fetch&Download methods in the /pkg directory (elastic#1943) bump stack version 8.1.0-dbc834fd (elastic#1948) bump stack version 8.1.0-76902d39 (elastic#1946) chore: retire 7.15 adding 7.17 (elastic#1938) ci: use withAPMEnv (elastic#1917) Update main branch (elastic#1928) bump stack version 8.1.0-befff95a (elastic#1929) chore: properly evaluate how tests are skipped on CI when checking modified files (elastic#1924) bump stack version 8.1.0-60bffc32 (elastic#1921) ...

mdelapenya added 7 commits January 20, 2022 10:26

chore: rename platforms to include OS

f2bf835

fix: include sles15 in Beats builds

bed7577

chore: expand coverage for SLES15

e2c78a7

chore: run k8s-autodiscover on Debian VMs

045b3ba

chore: run helm test suite on Debian VMs

9e8af01

chore: remove unused platforms

7066eae

feat: add centos8 support, running fleet test suite

68fcba5

mdelapenya added backport-v7.16.0 Automated backport with mergify backport-v7.17.0 Automated backport with mergify backport-v8.0.0 Automated backport with mergify labels Jan 20, 2022

mdelapenya self-assigned this Jan 20, 2022

mdelapenya requested review from a team and jlind23 January 20, 2022 10:35

mdelapenya added area:ci Anything related to the CI Team:Automation Label for the Observability productivity team labels Jan 20, 2022

jlind23 approved these changes Jan 20, 2022

View reviewed changes

mdelapenya added 2 commits January 20, 2022 12:57

fix: install python for CentOS

e11d220

It comes without Python installed

fix: pass suite to Ansible context

0967124

adam-stokes reviewed Jan 20, 2022

View reviewed changes

.ci/ansible/tasks/setup_test_script.yml Show resolved Hide resolved

adam-stokes approved these changes Jan 20, 2022

View reviewed changes

mdelapenya added 7 commits January 20, 2022 13:51

fix: use ansible os family

25f73f8

fix: typo

867bc58

chore: try with platform

c432675

Revert "chore: try with platform"

3dc4831

This reverts commit c432675.

chore: set user_data for CentOS

cdd9651

user_data us used to execute arbitrary code after a VM is provisioned on AWS

Merge branch 'main' into expand-os-coverage

87a7b09

fix: use Python interpreter for Ansible in CentOS

90adcdc

mdelapenya commented Jan 24, 2022

View reviewed changes

.ci/Jenkinsfile Outdated Show resolved Hide resolved

Revert "fix: use Python interpreter for Ansible in CentOS"

d128f46

This reverts commit 90adcdc.

Set proper python interpreter based on OS

9fab26c

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

mdelapenya commented Jan 24, 2022

View reviewed changes

.ci/ansible/playbook.yml Outdated Show resolved Hide resolved

adam-stokes added 2 commits January 24, 2022 11:18

try with auto

f15a8ee

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

remove user_data for now

bd391c6

Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

mdelapenya commented Jan 25, 2022

View reviewed changes

.ci/ansible/files/user-data-centos.sh Outdated Show resolved Hide resolved

mdelapenya and others added 2 commits January 25, 2022 08:00

chore: remove user data file

730c89a

Merge branch 'main' into expand-os-coverage

e4c9370

mdelapenya merged commit aec2a7a into elastic:main Jan 25, 2022

mergify bot mentioned this pull request Jan 25, 2022

[8.0](backport #2034) feat: add CentOS 8 support #2054

Merged

mergify bot mentioned this pull request Jan 25, 2022

[7.17](backport #2034) feat: add CentOS 8 support #2055

Merged

mergify bot mentioned this pull request Jan 25, 2022

[7.16](backport #2034) feat: add CentOS 8 support #2056

Merged

mdelapenya mentioned this pull request Jan 26, 2022

chore: rename Jenkins stage to include suite #2057

Merged

8 tasks

mdelapenya mentioned this pull request Jan 31, 2022

chore: always install tools in runners #2075

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add CentOS 8 support#2034

feat: add CentOS 8 support#2034
mdelapenya merged 22 commits intoelastic:mainfrom
mdelapenya:expand-os-coverage

mdelapenya commented Jan 20, 2022

Uh oh!

elasticmachine commented Jan 20, 2022 •

edited

Loading

Genuine test errors

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adam-stokes commented Jan 24, 2022

Uh oh!

mdelapenya commented Jan 24, 2022

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

mdelapenya commented Jan 20, 2022

What does this PR do?

Why is it important?

Checklist

Related issues

Uh oh!

elasticmachine commented Jan 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🐛 Flaky test report

Genuine test errors

🤖 GitHub comments

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adam-stokes commented Jan 24, 2022

Uh oh!

mdelapenya commented Jan 24, 2022

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

elasticmachine commented Jan 20, 2022 •

edited

Loading