chore: simplify triggering the E2E tests for Beats by mdelapenya · Pull Request #21790 · elastic/beats

mdelapenya · 2020-10-14T11:29:00Z

What does this PR do?

It creates a Jenkins stage right after the package one, in which we will trigger only one job for the E2E tests. To set the test suites to run, we will use the existing BEATS matrix to configure a collection of suites (if/when needed).

We will convert the collection to a Set to remove duplicates, and willl trigger the E2E job with that configuration.

Why is it important?

With previous approach, it triggers up to 4 builds for almost the same test suites, and the E2E job supports passing multiple suites as an comma-separated list of strings. As a direct consequence, only one GH check will be added to the PR, insteadd of multiple, one per modified Beat.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

Groovy file validated with APM Pipeline validator

.ci/packaging.groovy

mdelapenya · 2020-10-14T11:30:30Z

.ci/packaging.groovy

  )

-  def notifyContext = "${env.GITHUB_CHECK_E2E_TESTS_NAME} for ${env.BEATS_FOLDER}"
+  def notifyContext = "${env.GITHUB_CHECK_E2E_TESTS_NAME}"


Only one GH check per PR

mdelapenya · 2020-10-14T11:33:01Z

.ci/packaging.groovy

+  def suites = '' // empty value represents all suites in the E2E tests
+
+  catchError(buildResult: 'UNSTABLE', message: 'Unable to run e2e tests', stageResult: 'FAILURE') {
+    def suitesSet = e2eTestSuites.toSet()


toSet() is allowed by Jenkins sandbox:

https://github.com/jenkinsci/script-security-plugin/blob/master/src/main/resources/org/jenkinsci/plugins/scriptsecurity/sandbox/whitelists/generic-whitelist

mdelapenya · 2020-10-14T11:34:03Z

.ci/packaging.groovy

+
+    if (!suitesSet.contains('ALL')) {
+      suitesSet.each { suite ->
+        suites += "${suite},"


Verified that last comma does not cause any new suite with empty value. See https://beats-ci.elastic.co/job/e2e-tests/job/e2e-testing-mbp/job/master/512/parameters/

elasticmachine · 2020-10-14T11:34:11Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: [Pull request #21790 updated]
Start Time: 2020-10-15T05:22:49.296+0000
Duration: 25 min 30 sec

mdelapenya · 2020-10-14T14:21:57Z

/package

mdelapenya · 2020-10-14T15:27:53Z

/package

mdelapenya · 2020-10-14T16:41:23Z

/package

mdelapenya · 2020-10-15T05:22:35Z

/package

mdelapenya · 2020-10-15T10:07:23Z

Ok, only one check for the E2E is added to the PR. I'd say it's ready to be merged

* chore: pass beat as a method argument (no side-effects) * chore: run tests in a separate stage * fix: use parenthesis * chore: update comment * chore: do not trigger E2E tests if no suite was added * fix: use missing curly brackets * fix: wrong closure wrapping * fix: condition was not set

* feat: add a new step to run the e2e tests for certain parts of Beats (#21100) * feat: add a new step to run the e2e tests for certain parts of Beats We are going to trigger the tests for those parts affected by the elastic-agent, filebeat, or metricbeat, because those are the ones we verify in the e2e-testing suite * chore: do not include heartbeat * feat: trigger the e2e tests * fix: use relative path * chore: use proper target branch name for PRs * chore: use different tag * fix: use proper env variable * chore: pass github checks context to downstream job * chore: revert shared lib version Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: add BASE_DIR env variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove duplicated env * ffix: add param comma separator * fix: wrong copy&paste * chore: move e2e GH check out of the release context * chore: simplify conditional logic * chore: refine execution of test suites * fix: use proper parameter name * chore: set metricbeat version * chore: remove slack notifications on PRs * chore: update parameter * chore: run multiple test suites per beat type Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [E2E Tests] fix: set versions ony for PRs (#21608) * fix: set versions ony for PRs We want to use default versions per branch when running after a merge * fix: add trailing comma Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [CI: Packaging] fix: push ubi8 images too (#21621) * fix: push ubi8 images too * chore: enhance retries Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: use variables in log * chore: add "-oss" images Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * fix: remove extra curly brace in script (#21692) * fix: remove extra curly brace * chore: proper indent * fix: update fleet test suite name (#21738) * chore: create CI artifacts for DEV usage (#21645) It will create the artifacts with some requirements related to integrity * chore: simplify triggering the E2E tests for Beats (#21790) * chore: pass beat as a method argument (no side-effects) * chore: run tests in a separate stage * fix: use parenthesis * chore: update comment * chore: do not trigger E2E tests if no suite was added * fix: use missing curly brackets * fix: wrong closure wrapping * fix: condition was not set * chore: delegate variant pushes to the right method (#21861) * fix: delegate pushes to variants * chore: group conditions for x-pack * chore: simplify with endsWith Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * feat: package aliases for snapshots (#21960) * feat: push aliases for docker images * feat: build alias for snapshots * fix: only update alias on snapshots Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * fix: wrong image name for alias * fix: reuse variable as groovy does not hide variables by scope * chore: extract common logic to a method * Revert "fix: only update alias on snapshots" This reverts commit cff2cef. * Revert "feat: build alias for snapshots" This reverts commit 707e0d7. * chore: do not push aliases for PRs Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * chore: Use third number as x Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* feat: add a new step to run the e2e tests for certain parts of Beats (#21100) * feat: add a new step to run the e2e tests for certain parts of Beats We are going to trigger the tests for those parts affected by the elastic-agent, filebeat, or metricbeat, because those are the ones we verify in the e2e-testing suite * chore: do not include heartbeat * feat: trigger the e2e tests * fix: use relative path * chore: use proper target branch name for PRs * chore: use different tag * fix: use proper env variable * chore: pass github checks context to downstream job * chore: revert shared lib version Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: add BASE_DIR env variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove duplicated env * ffix: add param comma separator * fix: wrong copy&paste * chore: move e2e GH check out of the release context * chore: simplify conditional logic * chore: refine execution of test suites * fix: use proper parameter name * chore: set metricbeat version * chore: remove slack notifications on PRs * chore: update parameter * chore: run multiple test suites per beat type Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [E2E Tests] fix: set versions ony for PRs (#21608) * fix: set versions ony for PRs We want to use default versions per branch when running after a merge * fix: add trailing comma Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [CI: Packaging] fix: push ubi8 images too (#21621) * fix: push ubi8 images too * chore: enhance retries Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: use variables in log * chore: add "-oss" images Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * fix: remove extra curly brace in script (#21692) * fix: remove extra curly brace * chore: proper indent * fix: update fleet test suite name (#21738) * chore: create CI artifacts for DEV usage (#21645) It will create the artifacts with some requirements related to integrity * chore: simplify triggering the E2E tests for Beats (#21790) * chore: pass beat as a method argument (no side-effects) * chore: run tests in a separate stage * fix: use parenthesis * chore: update comment * chore: do not trigger E2E tests if no suite was added * fix: use missing curly brackets * fix: wrong closure wrapping * fix: condition was not set * chore: delegate variant pushes to the right method (#21861) * fix: delegate pushes to variants * chore: group conditions for x-pack * chore: simplify with endsWith Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> # Conflicts: # .ci/packaging.groovy * feat: package aliases for snapshots (#21960) * feat: push aliases for docker images * feat: build alias for snapshots * fix: only update alias on snapshots Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * fix: wrong image name for alias * fix: reuse variable as groovy does not hide variables by scope * chore: extract common logic to a method * Revert "fix: only update alias on snapshots" This reverts commit cff2cef. * Revert "feat: build alias for snapshots" This reverts commit 707e0d7. * chore: do not push aliases for PRs Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * chore: Use third number as x Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* feat: add a new step to run the e2e tests for certain parts of Beats (elastic#21100) * feat: add a new step to run the e2e tests for certain parts of Beats We are going to trigger the tests for those parts affected by the elastic-agent, filebeat, or metricbeat, because those are the ones we verify in the e2e-testing suite * chore: do not include heartbeat * feat: trigger the e2e tests * fix: use relative path * chore: use proper target branch name for PRs * chore: use different tag * fix: use proper env variable * chore: pass github checks context to downstream job * chore: revert shared lib version Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: add BASE_DIR env variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove duplicated env * ffix: add param comma separator * fix: wrong copy&paste * chore: move e2e GH check out of the release context * chore: simplify conditional logic * chore: refine execution of test suites * fix: use proper parameter name * chore: set metricbeat version * chore: remove slack notifications on PRs * chore: update parameter * chore: run multiple test suites per beat type Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [E2E Tests] fix: set versions ony for PRs (elastic#21608) * fix: set versions ony for PRs We want to use default versions per branch when running after a merge * fix: add trailing comma Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [CI: Packaging] fix: push ubi8 images too (elastic#21621) * fix: push ubi8 images too * chore: enhance retries Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: use variables in log * chore: add "-oss" images Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * fix: remove extra curly brace in script (elastic#21692) * fix: remove extra curly brace * chore: proper indent * fix: update fleet test suite name (elastic#21738) * chore: create CI artifacts for DEV usage (elastic#21645) It will create the artifacts with some requirements related to integrity * chore: simplify triggering the E2E tests for Beats (elastic#21790) * chore: pass beat as a method argument (no side-effects) * chore: run tests in a separate stage * fix: use parenthesis * chore: update comment * chore: do not trigger E2E tests if no suite was added * fix: use missing curly brackets * fix: wrong closure wrapping * fix: condition was not set * chore: delegate variant pushes to the right method (elastic#21861) * fix: delegate pushes to variants * chore: group conditions for x-pack * chore: simplify with endsWith Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * feat: package aliases for snapshots (elastic#21960) * feat: push aliases for docker images * feat: build alias for snapshots * fix: only update alias on snapshots Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * fix: wrong image name for alias * fix: reuse variable as groovy does not hide variables by scope * chore: extract common logic to a method * Revert "fix: only update alias on snapshots" This reverts commit cff2cef. * Revert "feat: build alias for snapshots" This reverts commit 707e0d7. * chore: do not push aliases for PRs Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * chore: Use third number as x Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> (cherry picked from commit 624c459)

…23142) Backports the following commits to 7.x: * feat: add a new step to run the e2e tests for certain parts of Beats (#21100) * [E2E Tests] fix: set versions ony for PRs (#21608) * [CI: Packaging] fix: push ubi8 images too (#21621) * fix: remove extra curly brace in script (#21692) * fix: update fleet test suite name (#21738) * chore: create CI artifacts for DEV usage (#21645) * chore: simplify triggering the E2E tests for Beats (#21790) * chore: delegate variant pushes to the right method (#21861) * feat: package aliases for snapshots (#21960) * fix: use proper param name for e2e tests (#22836) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

…ports (elastic#23142) Backports the following commits to 7.x: * feat: add a new step to run the e2e tests for certain parts of Beats (elastic#21100) * [E2E Tests] fix: set versions ony for PRs (elastic#21608) * [CI: Packaging] fix: push ubi8 images too (elastic#21621) * fix: remove extra curly brace in script (elastic#21692) * fix: update fleet test suite name (elastic#21738) * chore: create CI artifacts for DEV usage (elastic#21645) * chore: simplify triggering the E2E tests for Beats (elastic#21790) * chore: delegate variant pushes to the right method (elastic#21861) * feat: package aliases for snapshots (elastic#21960) * fix: use proper param name for e2e tests (elastic#22836) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> (cherry picked from commit 1006bd9)

…23142) (#23396) Backports the following commits to 7.x: * feat: add a new step to run the e2e tests for certain parts of Beats (#21100) * [E2E Tests] fix: set versions ony for PRs (#21608) * [CI: Packaging] fix: push ubi8 images too (#21621) * fix: remove extra curly brace in script (#21692) * fix: update fleet test suite name (#21738) * chore: create CI artifacts for DEV usage (#21645) * chore: simplify triggering the E2E tests for Beats (#21790) * chore: delegate variant pushes to the right method (#21861) * feat: package aliases for snapshots (#21960) * fix: use proper param name for e2e tests (#22836) (cherry picked from commit 1006bd9) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

* feat: add a new step to run the e2e tests for certain parts of Beats (elastic#21100) * feat: add a new step to run the e2e tests for certain parts of Beats We are going to trigger the tests for those parts affected by the elastic-agent, filebeat, or metricbeat, because those are the ones we verify in the e2e-testing suite * chore: do not include heartbeat * feat: trigger the e2e tests * fix: use relative path * chore: use proper target branch name for PRs * chore: use different tag * fix: use proper env variable * chore: pass github checks context to downstream job * chore: revert shared lib version Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: add BASE_DIR env variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove duplicated env * ffix: add param comma separator * fix: wrong copy&paste * chore: move e2e GH check out of the release context * chore: simplify conditional logic * chore: refine execution of test suites * fix: use proper parameter name * chore: set metricbeat version * chore: remove slack notifications on PRs * chore: update parameter * chore: run multiple test suites per beat type Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [E2E Tests] fix: set versions ony for PRs (elastic#21608) * fix: set versions ony for PRs We want to use default versions per branch when running after a merge * fix: add trailing comma Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * [CI: Packaging] fix: push ubi8 images too (elastic#21621) * fix: push ubi8 images too * chore: enhance retries Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: use variables in log * chore: add "-oss" images Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * fix: remove extra curly brace in script (elastic#21692) * fix: remove extra curly brace * chore: proper indent * fix: update fleet test suite name (elastic#21738) * chore: create CI artifacts for DEV usage (elastic#21645) It will create the artifacts with some requirements related to integrity * chore: simplify triggering the E2E tests for Beats (elastic#21790) * chore: pass beat as a method argument (no side-effects) * chore: run tests in a separate stage * fix: use parenthesis * chore: update comment * chore: do not trigger E2E tests if no suite was added * fix: use missing curly brackets * fix: wrong closure wrapping * fix: condition was not set * chore: delegate variant pushes to the right method (elastic#21861) * fix: delegate pushes to variants * chore: group conditions for x-pack * chore: simplify with endsWith Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> # Conflicts: # .ci/packaging.groovy * feat: package aliases for snapshots (elastic#21960) * feat: push aliases for docker images * feat: build alias for snapshots * fix: only update alias on snapshots Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * fix: wrong image name for alias * fix: reuse variable as groovy does not hide variables by scope * chore: extract common logic to a method * Revert "fix: only update alias on snapshots" This reverts commit cff2cef. * Revert "feat: build alias for snapshots" This reverts commit 707e0d7. * chore: do not push aliases for PRs Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> * chore: Use third number as x Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

mdelapenya added 3 commits October 14, 2020 12:02

chore: pass beat as a method argument (no side-effects)

54cdb84

chore: run tests in a separate stage

995df39

fix: use parenthesis

1bd97e6

mdelapenya self-assigned this Oct 14, 2020

mdelapenya added the Team:Automation Label for the Observability productivity team label Oct 14, 2020

mdelapenya requested a review from a team October 14, 2020 11:29

botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Oct 14, 2020

mdelapenya commented Oct 14, 2020

View reviewed changes

.ci/packaging.groovy Outdated Show resolved Hide resolved

mdelapenya commented Oct 14, 2020

View reviewed changes

chore: update comment

76f7b71

mdelapenya commented Oct 14, 2020

View reviewed changes

mdelapenya added 2 commits October 14, 2020 16:41

chore: do not trigger E2E tests if no suite was added

9808a18

fix: use missing curly brackets

ace848e

cachedout approved these changes Oct 14, 2020

View reviewed changes

fix: wrong closure wrapping

3afdcf3

mdelapenya marked this pull request as ready for review October 14, 2020 18:19

fix: condition was not set

92ca4ed

mdelapenya merged commit 8de1a7d into elastic:master Oct 15, 2020

mdelapenya mentioned this pull request Oct 22, 2020

Cherry-pick to 7.10: packaging backports #22076

Merged

mdelapenya mentioned this pull request Oct 22, 2020

Cherry-pick to 7.9: packaging backports #22078

Merged

jsoriano mentioned this pull request Dec 15, 2020

Cherry-pick #22076 to 7.x: Cherry-pick to 7.10: packaging backports #23142

Merged

jsoriano mentioned this pull request Jan 7, 2021

Cherry-pick #23142 to 7.11: Cherry-pick #22076 to 7.x: Cherry-pick to 7.10: packaging backports #23396

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: simplify triggering the E2E tests for Beats#21790

chore: simplify triggering the E2E tests for Beats#21790
mdelapenya merged 8 commits intoelastic:masterfrom
mdelapenya:simplify-e2e-triggers

mdelapenya commented Oct 14, 2020 •

edited

Loading

Uh oh!

Uh oh!

mdelapenya Oct 14, 2020

Uh oh!

mdelapenya Oct 14, 2020

Uh oh!

mdelapenya Oct 14, 2020

Uh oh!

elasticmachine commented Oct 14, 2020 •

edited by jenkins-beats-ci bot

Loading

Build stats

Uh oh!

mdelapenya commented Oct 14, 2020

Uh oh!

mdelapenya commented Oct 14, 2020

Uh oh!

mdelapenya commented Oct 14, 2020

Uh oh!

mdelapenya commented Oct 15, 2020

Uh oh!

mdelapenya commented Oct 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

mdelapenya commented Oct 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

Author's Checklist

Uh oh!

Uh oh!

mdelapenya Oct 14, 2020

Choose a reason for hiding this comment

Uh oh!

mdelapenya Oct 14, 2020

Choose a reason for hiding this comment

Uh oh!

mdelapenya Oct 14, 2020

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Oct 14, 2020 • edited by jenkins-beats-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Uh oh!

mdelapenya commented Oct 14, 2020

Uh oh!

mdelapenya commented Oct 14, 2020

Uh oh!

mdelapenya commented Oct 14, 2020

Uh oh!

mdelapenya commented Oct 15, 2020

Uh oh!

mdelapenya commented Oct 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mdelapenya commented Oct 14, 2020 •

edited

Loading

elasticmachine commented Oct 14, 2020 •

edited by jenkins-beats-ci bot

Loading