This repository was archived by the owner on Sep 17, 2024. It is now read-only.
chore: bump Godog to v0.12.4#2245
Merged
mdelapenya merged 27 commits intoelastic:mainfrom Mar 21, 2022
Merged
Conversation
Contributor
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
💚 Flaky test reportTests succeeded. 🤖 GitHub commentsTo re-run your PR in the CI, just comment with:
|
kuisathaverat
approved these changes
Mar 17, 2022
mdelapenya
commented
Mar 17, 2022
| set -euxo pipefail | ||
|
|
||
| ## original image: ystia/cucumber-html-reporter:latest | ||
| readonly DOCKER_IMAGE="docker.elastic.co/observability-ci/cucumber-html-reporter:latest" |
Contributor
Author
There was a problem hiding this comment.
I already pushed the image under our registry, just in case the image disappears
mdelapenya
commented
Mar 17, 2022
| do | ||
| filename="$(basename ${f})" | ||
| echo "parsing ${filename}" | ||
| docker run --rm \ |
Contributor
Author
There was a problem hiding this comment.
mdelapenya
commented
Mar 18, 2022
Comment on lines
+21
to
+24
| for f in ${FILES} | ||
| do | ||
| filename="$(basename ${f})" | ||
| echo "parsing ${filename}" |
Contributor
Author
There was a problem hiding this comment.
For some reason, the glob expression is not expanded on CI. I'm able to execute this:
$ CUCUMBER_REPORTS_PATH=e2e/_suites/fleet SUITE=fleet ARCHITECTURE=amd TAGS=standalone PLATFORM=Darwin .ci/scripts/generate-cucumber-reports.sh
+ readonly DOCKER_IMAGE=docker.elastic.co/observability-ci/cucumber-html-reporter:latest
+ DOCKER_IMAGE=docker.elastic.co/observability-ci/cucumber-html-reporter:latest
+ CUCUMBER_REPORTS_PATH=e2e/_suites/fleet
+ FILES='e2e/_suites/fleet/TEST*.json'
+ ARCHITECTURE=amd
+ PLATFORM=Darwin
+ SUITE=fleet
+ TAGS=standalone
+ main
+ for f in ${FILES}
++ basename e2e/_suites/fleet/TEST.json
+ filename=TEST.json
+ echo 'parsing TEST.json'
parsing TEST.json
++ pwd
++ pwd
+ docker run --rm -v /Users/mdelapenya/sourcecode/src/github.com/elastic/e2e-testing/e2e/_suites/fleet:/use/src/app/in -v /Users/mdelapenya/sourcecode/src/github.com/elastic/e2e-testing/outputs:/use/src/app/out -e CHR_APP_jsonFile=in/TEST.json -e CHR_APP_output=out/TEST.json.html -e CHR_APP_metadata_arch=amd -e CHR_APP_metadata_platform=Darwin -e CHR_APP_metadata_suite=fleet -e CHR_APP_metadata_tags=standalone docker.elastic.co/observability-ci/cucumber-html-reporter:latest
> cucumber-html-reporter@0.0.1 start /use/src/app
> node main.js
Cucumber HTML report out/TEST.json.html generated successfully.
+ for f in ${FILES}
++ basename e2e/_suites/fleet/TEST2.json
+ filename=TEST2.json
+ echo 'parsing TEST2.json'
parsing TEST2.json
++ pwd
++ pwd
+ docker run --rm -v /Users/mdelapenya/sourcecode/src/github.com/elastic/e2e-testing/e2e/_suites/fleet:/use/src/app/in -v /Users/mdelapenya/sourcecode/src/github.com/elastic/e2e-testing/outputs:/use/src/app/out -e CHR_APP_jsonFile=in/TEST2.json -e CHR_APP_output=out/TEST2.json.html -e CHR_APP_metadata_arch=amd -e CHR_APP_metadata_platform=Darwin -e CHR_APP_metadata_suite=fleet -e CHR_APP_metadata_tags=standalone docker.elastic.co/observability-ci/cucumber-html-reporter:latest
> cucumber-html-reporter@0.0.1 start /use/src/app
> node main.js
Cucumber HTML report out/TEST2.json.html generated successfully.
where there are a few TEST*.json files under the CUCUMBER_REPORTS_PATH.
But on CI, I always get:
[2022-03-18T09:34:29.582Z] + for f in ${FILES}
[2022-03-18T09:34:29.582Z] ++ basename 'src/github.com/elastic/e2e-testing/outputs/18.224.246.92/TEST*.json'
[2022-03-18T09:34:29.582Z] + filename='TEST*.json'
[2022-03-18T09:34:29.582Z] + echo 'parsing TEST*.json'
[2022-03-18T09:34:29.582Z] parsing TEST*.json
Resulting in this error:
[2022-03-18T09:34:36.076Z] Unable to parse cucumberjs output into json: 'in/TEST*.json' { Error: in/TEST*.json: ENOENT: no such file or directory, open 'in/TEST*.json'
@v1v @kuisathaverat any guess?
Member
There was a problem hiding this comment.
I wonder if double quotes are needed in this case
v1v
reviewed
Mar 18, 2022
Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
mdelapenya
commented
Mar 18, 2022
| TAGS="${TAGS:-""}" | ||
|
|
||
| main() { | ||
| for f in "${CUCUMBER_REPORTS_PATH}/TEST*.json" |
Contributor
Author
There was a problem hiding this comment.
v1v
reviewed
Mar 18, 2022
v1v
reviewed
Mar 18, 2022
Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
Contributor
|
|
v1v
reviewed
Mar 18, 2022
mergify bot
pushed a commit
that referenced
this pull request
Mar 21, 2022
* chore: bump godog to v.0.12.4 * chore: update BeforeScenario/AfterScenario to Before/After * chore: update BeforeStep/AfterStep to StepContext().Before/After * chore: log before/after steps * chore: consistent variable name * chore: log step status after execution * chore: support multiple formatters * chore: store cucumber reports in Jenkins UI * chore: add go-license-detector as dependency * chore: notice * chore: store JSON reports as an artifact * fix: remove cucumber until the plugin is installed in the Jenkins instance * chore: update scenario template * fix: wrong log.Debugf signature * chore: generate cucumber reports without the plugin See #120 * chore: enrich cucumber reports metadata * fix: dockerLogin before pulling the new image * chore: update packer-cache images * fix: run under the right dir * fix: right path for cucumber-gen script * chore: simplify files variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: apply suggestions from code review Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove BASE_DIR * chore: update templates to godog v0.12.4 * fix: update stack version in template * chore: update ES template * fix: use output dir from the nodeIP Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> (cherry picked from commit fb6385e)
mergify bot
pushed a commit
that referenced
this pull request
Mar 21, 2022
* chore: bump godog to v.0.12.4 * chore: update BeforeScenario/AfterScenario to Before/After * chore: update BeforeStep/AfterStep to StepContext().Before/After * chore: log before/after steps * chore: consistent variable name * chore: log step status after execution * chore: support multiple formatters * chore: store cucumber reports in Jenkins UI * chore: add go-license-detector as dependency * chore: notice * chore: store JSON reports as an artifact * fix: remove cucumber until the plugin is installed in the Jenkins instance * chore: update scenario template * fix: wrong log.Debugf signature * chore: generate cucumber reports without the plugin See #120 * chore: enrich cucumber reports metadata * fix: dockerLogin before pulling the new image * chore: update packer-cache images * fix: run under the right dir * fix: right path for cucumber-gen script * chore: simplify files variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: apply suggestions from code review Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove BASE_DIR * chore: update templates to godog v0.12.4 * fix: update stack version in template * chore: update ES template * fix: use output dir from the nodeIP Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> (cherry picked from commit fb6385e)
mergify bot
pushed a commit
that referenced
this pull request
Mar 21, 2022
* chore: bump godog to v.0.12.4 * chore: update BeforeScenario/AfterScenario to Before/After * chore: update BeforeStep/AfterStep to StepContext().Before/After * chore: log before/after steps * chore: consistent variable name * chore: log step status after execution * chore: support multiple formatters * chore: store cucumber reports in Jenkins UI * chore: add go-license-detector as dependency * chore: notice * chore: store JSON reports as an artifact * fix: remove cucumber until the plugin is installed in the Jenkins instance * chore: update scenario template * fix: wrong log.Debugf signature * chore: generate cucumber reports without the plugin See #120 * chore: enrich cucumber reports metadata * fix: dockerLogin before pulling the new image * chore: update packer-cache images * fix: run under the right dir * fix: right path for cucumber-gen script * chore: simplify files variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: apply suggestions from code review Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove BASE_DIR * chore: update templates to godog v0.12.4 * fix: update stack version in template * chore: update ES template * fix: use output dir from the nodeIP Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> (cherry picked from commit fb6385e) # Conflicts: # .ci/packer_cache.sh
mdelapenya
added a commit
that referenced
this pull request
Mar 21, 2022
* chore: bump godog to v.0.12.4 * chore: update BeforeScenario/AfterScenario to Before/After * chore: update BeforeStep/AfterStep to StepContext().Before/After * chore: log before/after steps * chore: consistent variable name * chore: log step status after execution * chore: support multiple formatters * chore: store cucumber reports in Jenkins UI * chore: add go-license-detector as dependency * chore: notice * chore: store JSON reports as an artifact * fix: remove cucumber until the plugin is installed in the Jenkins instance * chore: update scenario template * fix: wrong log.Debugf signature * chore: generate cucumber reports without the plugin See #120 * chore: enrich cucumber reports metadata * fix: dockerLogin before pulling the new image * chore: update packer-cache images * fix: run under the right dir * fix: right path for cucumber-gen script * chore: simplify files variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: apply suggestions from code review Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove BASE_DIR * chore: update templates to godog v0.12.4 * fix: update stack version in template * chore: update ES template * fix: use output dir from the nodeIP Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> (cherry picked from commit fb6385e) Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
mdelapenya
added a commit
that referenced
this pull request
Mar 21, 2022
* chore: bump godog to v.0.12.4 * chore: update BeforeScenario/AfterScenario to Before/After * chore: update BeforeStep/AfterStep to StepContext().Before/After * chore: log before/after steps * chore: consistent variable name * chore: log step status after execution * chore: support multiple formatters * chore: store cucumber reports in Jenkins UI * chore: add go-license-detector as dependency * chore: notice * chore: store JSON reports as an artifact * fix: remove cucumber until the plugin is installed in the Jenkins instance * chore: update scenario template * fix: wrong log.Debugf signature * chore: generate cucumber reports without the plugin See #120 * chore: enrich cucumber reports metadata * fix: dockerLogin before pulling the new image * chore: update packer-cache images * fix: run under the right dir * fix: right path for cucumber-gen script * chore: simplify files variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: apply suggestions from code review Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove BASE_DIR * chore: update templates to godog v0.12.4 * fix: update stack version in template * chore: update ES template * fix: use output dir from the nodeIP Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> (cherry picked from commit fb6385e) Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
mdelapenya
added a commit
that referenced
this pull request
Mar 21, 2022
* chore: bump Godog to v0.12.4 (#2245) * chore: bump godog to v.0.12.4 * chore: update BeforeScenario/AfterScenario to Before/After * chore: update BeforeStep/AfterStep to StepContext().Before/After * chore: log before/after steps * chore: consistent variable name * chore: log step status after execution * chore: support multiple formatters * chore: store cucumber reports in Jenkins UI * chore: add go-license-detector as dependency * chore: notice * chore: store JSON reports as an artifact * fix: remove cucumber until the plugin is installed in the Jenkins instance * chore: update scenario template * fix: wrong log.Debugf signature * chore: generate cucumber reports without the plugin See #120 * chore: enrich cucumber reports metadata * fix: dockerLogin before pulling the new image * chore: update packer-cache images * fix: run under the right dir * fix: right path for cucumber-gen script * chore: simplify files variable Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: apply suggestions from code review Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> * chore: remove BASE_DIR * chore: update templates to godog v0.12.4 * fix: update stack version in template * chore: update ES template * fix: use output dir from the nodeIP Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> (cherry picked from commit fb6385e) # Conflicts: # .ci/packer_cache.sh * chore: fix conflicts Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
8 tasks
mdelapenya
added a commit
to mdelapenya/e2e-testing
that referenced
this pull request
Mar 30, 2022
* main: (72 commits) fix: always quote variables in shell scripts (elastic#2284) chore: bring back system integration scenarios (elastic#2233) chore: do not notify build green for fleet nightly builds (elastic#2277) chore: include suite, platform and tags in test reports name (elastic#2273) bump stack version 8.2.0-dcff22d7 (elastic#2270) bump stack version 8.2.0-4509f321 (elastic#2265) chore: bump Godog to v0.12.4 (elastic#2245) bump stack version 8.2.0-5cc993c1 (elastic#2254) bump stack version 8.2.0-82b803a1 (elastic#2250) fix: never process elasticsearch CI artifacts (elastic#2246) bump stack version 8.2.0-ff67d7b8 (elastic#2242) ci: periodic builds for the last two minor versions (elastic#2241) bump stack version 8.2.0-9bac538c (elastic#2240) fix: add end-of-file line in templates (elastic#2237) chore: add Christos to SSH users bump stack version 8.2.0-fee3b8d2 (elastic#2234) Separate ES workload from Agent spec and make it req for the scenarios (elastic#2203) bump stack version 8.2.0-d02c907a (elastic#2228) bump stack version 8.2.0-63265ec9 (elastic#2225) fix: wrong link (elastic#2220) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
It bumps Godog from 0.11 to 0.12.4, which includes multi-formatter support.
For that, we have updated the Go dependencies, and had to update the usages of the lifecycle hooks defined by Godog: BeforeScenario/BeforeStep and AfterScenario/AfterStep are now deprecated.
Finally, we updated the scripts that generate the test report to include 3 formatters: pretty, cucumber and JUnit.
The cucumber format will be stored as a JSON file, downloaded by the Jenkins pipeline, and archived in the Jenkins UI using a script that consumes the Docker image described in #120, generating the HTML report.
Why is it important?
Let's keep dependencies up-to-date, getting the new features it provides, in this case, the multi-formatter support, which will allow us to share nicer reports about the cucumber scenarios.
Checklist
make unit-test), and they are passing locallymake noticein the proper directory)Related issues
Follow-ups
It could be interesting to consolidate all cucumber JSON reports into one, and generate just one HTML report.