Skip to content
This repository was archived by the owner on Sep 17, 2024. It is now read-only.

fix: always quote variables in shell scripts#2284

Merged
mdelapenya merged 7 commits intoelastic:mainfrom
mdelapenya:fix-skip-tags
Mar 29, 2022
Merged

fix: always quote variables in shell scripts#2284
mdelapenya merged 7 commits intoelastic:mainfrom
mdelapenya:fix-skip-tags

Conversation

@mdelapenya
Copy link
Copy Markdown
Contributor

@mdelapenya mdelapenya commented Mar 29, 2022

  • chore: remove @skip:platform tags
  • fix: do not use the @skip:platform tag in pipeline
  • fix: quote variables as much as possible
  • chore: always quote shell variables

What does this PR do?

We are always double-quoting the values for the shell variables that are used to configure the test execution, and that is done here:

  • the shell script used for local development: .ci/scripts/functiona-test.sh
  • the on-the-fly-generated script created by Ansible to be copied into the remote VM

Finally, because we are controlling what scenarios to run on each supported platform using the CI descriptors (see .ci/.e2e-tests.yaml), we do not need the @skip:platform Gherkin annotation, which was used in the past to skip certain scenarios per platform.
With current approach, simply adding/removing the platform from the scenario in that CI descriptor would do it.

Why is it important?

We introduced a regression in #2273, because when sourcing the .env file including a variable with whitespaces on its value, then it does not consider it and directly executes it.

This is the case in:

[2022-03-29T05:29:47.711Z] + source /home/centos/e2e-testing/.env
[2022-03-29T05:29:47.711Z] ++ export OP_LOG_LEVEL=TRACE
[2022-03-29T05:29:47.711Z] ++ OP_LOG_LEVEL=TRACE
[2022-03-29T05:29:47.711Z] ++ export LOG_LEVEL=TRACE
[2022-03-29T05:29:47.712Z] ++ LOG_LEVEL=TRACE
[2022-03-29T05:29:47.712Z] ++ export TIMEOUT_FACTOR=7
[2022-03-29T05:29:47.712Z] ++ TIMEOUT_FACTOR=7
[2022-03-29T05:29:47.712Z] ++ export SUITE=fleet
[2022-03-29T05:29:47.712Z] ++ SUITE=fleet
[2022-03-29T05:29:47.712Z] ++ export PROVIDER=remote
[2022-03-29T05:29:47.712Z] ++ PROVIDER=remote
[2022-03-29T05:29:47.712Z] ++ export ELASTICSEARCH_PASSWORD=changeme
[2022-03-29T05:29:47.712Z] ++ ELASTICSEARCH_PASSWORD=changeme
[2022-03-29T05:29:47.712Z] ++ export KIBANA_PASSWORD=changeme
[2022-03-29T05:29:47.712Z] ++ KIBANA_PASSWORD=changeme
[2022-03-29T05:29:47.712Z] ++ export SKIP_PULL=1
[2022-03-29T05:29:47.712Z] ++ SKIP_PULL=1
[2022-03-29T05:29:47.712Z] ++ export ELASTICSEARCH_URL=http://18.217.140.14:9200
[2022-03-29T05:29:47.712Z] ++ ELASTICSEARCH_URL=http://18.217.140.14:9200
[2022-03-29T05:29:47.712Z] ++ export KIBANA_URL=http://18.217.140.14:5601
[2022-03-29T05:29:47.712Z] ++ KIBANA_URL=http://18.217.140.14:5601
[2022-03-29T05:29:47.712Z] ++ export FLEET_URL=http://18.217.140.14:8220
[2022-03-29T05:29:47.712Z] ++ FLEET_URL=http://18.217.140.14:8220
[2022-03-29T05:29:47.712Z] ++ export STACK_VERSION=7.17.2-cf7cf1eb-SNAPSHOT
[2022-03-29T05:29:47.712Z] ++ STACK_VERSION=7.17.2-cf7cf1eb-SNAPSHOT
[2022-03-29T05:29:47.712Z] ++ export BEAT_VERSION=7.17.2-cf7cf1eb-SNAPSHOT
[2022-03-29T05:29:47.712Z] ++ BEAT_VERSION=7.17.2-cf7cf1eb-SNAPSHOT
[2022-03-29T05:29:47.712Z] ++ export ELASTIC_AGENT_VERSION=7.17.2-cf7cf1eb-SNAPSHOT
[2022-03-29T05:29:47.712Z] ++ ELASTIC_AGENT_VERSION=7.17.2-cf7cf1eb-SNAPSHOT
[2022-03-29T05:29:47.712Z] ++ export GITHUB_CHECK_SHA1=
[2022-03-29T05:29:47.712Z] ++ GITHUB_CHECK_SHA1=
[2022-03-29T05:29:47.712Z] ++ export GITHUB_CHECK_REPO=elastic-agent
[2022-03-29T05:29:47.712Z] ++ GITHUB_CHECK_REPO=elastic-agent
[2022-03-29T05:29:47.712Z] ++ export ELASTIC_APM_GLOBAL_LABELS=branch_name=7.17,build_pr=false,build_id=426,go_arch=amd64,beat_version=7.17.2-cf7cf1eb-SNAPSHOT,elastic_agent_version=7.17.2-cf7cf1eb-SNAPSHOT,stack_version=7.17.2-cf7cf1eb-SNAPSHOT
[2022-03-29T05:29:47.712Z] ++ ELASTIC_APM_GLOBAL_LABELS=branch_name=7.17,build_pr=false,build_id=426,go_arch=amd64,beat_version=7.17.2-cf7cf1eb-SNAPSHOT,elastic_agent_version=7.17.2-cf7cf1eb-SNAPSHOT,stack_version=7.17.2-cf7cf1eb-SNAPSHOT
[2022-03-29T05:29:47.712Z] ++ export REPORT_PREFIX=fleet_centos8_amd64_fleet_mode_agent
[2022-03-29T05:29:47.712Z] ++ REPORT_PREFIX=fleet_centos8_amd64_fleet_mode_agent
[2022-03-29T05:29:47.712Z] ++ '~@skip:amd64'
[2022-03-29T05:29:47.712Z] /home/centos/e2e-testing/.env: line 18: ~@skip:amd64: command not found

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have run the Unit tests (make unit-test), and they are passing locally
  • I have run the End-2-End tests for the suite I'm working on, and they are passing locally
  • I have noticed new Go dependencies (run make notice in the proper directory)

How to test this PR locally

Related issues

@mdelapenya mdelapenya added Team:Automation Label for the Observability productivity team area:ci Anything related to the CI backport-v7.17.0 Automated backport with mergify backport-v8.0.0 Automated backport with mergify backport-v8.1.0 Automated backport with mergify bug Something isn't working impact:critical Immediate priority; high value or cost to the product. priority:blocker Work is on-hold for a product team, business is at risk until resolution of issue labels Mar 29, 2022
@mdelapenya mdelapenya self-assigned this Mar 29, 2022
@mdelapenya mdelapenya requested a review from a team March 29, 2022 06:06
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Mar 29, 2022

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-03-29T10:43:56.709+0000

  • Duration: 53 min 21 sec

Test stats 🧪

Test Results
Failed 1
Passed 314
Skipped 0
Total 315

Test errors 1

Expand to view the tests failures

Initializing / End-To-End Tests / fleet_debian_arm64_system_integration / Adding diskio system/metrics Integration to a Policy – System Integration
    Expand to view the error details

     Step "system/metrics" with "diskio" metrics are present in the datastreams 
    

  • no stacktrace

Steps errors 2

Expand to view the steps failures

Shell Script
  • Took 20 min 56 sec . View more details here
  • Description: ssh -tt -o TCPKeepAlive=yes -o ServerAliveInterval=60 -o ServerAliveCountMax=10 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/jenkins/workspace/PR-2284-4-a3a74ddc-6b7b-40d7-87fe-6873a30d17c8/e2essh admin@3.141.30.219 -- 'sudo bash /home/admin/e2e-testing/.ci/scripts/functional-test.sh "system_integration && ~@nightly" '
Archive the artifacts
  • Took 0 min 1 sec . View more details here
  • Description: [2022-03-29T11:37:11.140Z] Archiving artifacts script returned exit code 2

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors 1

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_system_integration / Adding diskio system/metrics Integration to a Policy – System Integration

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

}

// Skip scenarios per platform
tags += " && ~@skip:${goArch}"
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed because we do not need to skip by arch at the tag level, but at the .e2e-testing.yaml descriptor

@mdelapenya mdelapenya merged commit ed5a86e into elastic:main Mar 29, 2022
mergify bot pushed a commit that referenced this pull request Mar 29, 2022
* chore: remove @Skip:platform tags

* fix: do not use the @Skip:platform tag in pipeline

* fix: quote variables as much as possible

* chore: always quote shell variables

* fix: more double quotes

* fix: more quotes

(cherry picked from commit ed5a86e)
mergify bot pushed a commit that referenced this pull request Mar 29, 2022
* chore: remove @Skip:platform tags

* fix: do not use the @Skip:platform tag in pipeline

* fix: quote variables as much as possible

* chore: always quote shell variables

* fix: more double quotes

* fix: more quotes

(cherry picked from commit ed5a86e)
mergify bot pushed a commit that referenced this pull request Mar 29, 2022
* chore: remove @Skip:platform tags

* fix: do not use the @Skip:platform tag in pipeline

* fix: quote variables as much as possible

* chore: always quote shell variables

* fix: more double quotes

* fix: more quotes

(cherry picked from commit ed5a86e)
mdelapenya added a commit that referenced this pull request Mar 29, 2022
* chore: remove @Skip:platform tags

* fix: do not use the @Skip:platform tag in pipeline

* fix: quote variables as much as possible

* chore: always quote shell variables

* fix: more double quotes

* fix: more quotes

(cherry picked from commit ed5a86e)

Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
mdelapenya added a commit that referenced this pull request Mar 29, 2022
* chore: remove @Skip:platform tags

* fix: do not use the @Skip:platform tag in pipeline

* fix: quote variables as much as possible

* chore: always quote shell variables

* fix: more double quotes

* fix: more quotes

(cherry picked from commit ed5a86e)

Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
mdelapenya added a commit that referenced this pull request Mar 29, 2022
* chore: remove @Skip:platform tags

* fix: do not use the @Skip:platform tag in pipeline

* fix: quote variables as much as possible

* chore: always quote shell variables

* fix: more double quotes

* fix: more quotes

(cherry picked from commit ed5a86e)

Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
mdelapenya added a commit to mdelapenya/e2e-testing that referenced this pull request Mar 30, 2022
* main: (72 commits)
  fix: always quote variables in shell scripts (elastic#2284)
  chore: bring back system integration scenarios (elastic#2233)
  chore: do not notify build green for fleet nightly builds (elastic#2277)
  chore: include suite, platform and tags in test reports name (elastic#2273)
  bump stack version 8.2.0-dcff22d7 (elastic#2270)
  bump stack version 8.2.0-4509f321 (elastic#2265)
  chore: bump Godog to v0.12.4 (elastic#2245)
  bump stack version 8.2.0-5cc993c1 (elastic#2254)
  bump stack version 8.2.0-82b803a1 (elastic#2250)
  fix: never process elasticsearch CI artifacts (elastic#2246)
  bump stack version 8.2.0-ff67d7b8 (elastic#2242)
  ci: periodic builds for the last two minor versions (elastic#2241)
  bump stack version 8.2.0-9bac538c (elastic#2240)
  fix: add end-of-file line in templates (elastic#2237)
  chore: add Christos to SSH users
  bump stack version 8.2.0-fee3b8d2 (elastic#2234)
  Separate ES workload from Agent spec and make it req for the scenarios (elastic#2203)
  bump stack version 8.2.0-d02c907a (elastic#2228)
  bump stack version 8.2.0-63265ec9 (elastic#2225)
  fix: wrong link (elastic#2220)
  ...
@mdelapenya mdelapenya deleted the fix-skip-tags branch April 11, 2022 15:17
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area:ci Anything related to the CI backport-v7.17.0 Automated backport with mergify backport-v8.0.0 Automated backport with mergify backport-v8.1.0 Automated backport with mergify bug Something isn't working impact:critical Immediate priority; high value or cost to the product. priority:blocker Work is on-hold for a product team, business is at risk until resolution of issue Team:Automation Label for the Observability productivity team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants