Skip to content

build: Enforce shellcheck(1) on all Bats tests#1331

Merged
debarshiray merged 4 commits intocontainers:mainfrom
debarshiray:wip/rishi/test-system-shellcheck
Sep 25, 2023
Merged

build: Enforce shellcheck(1) on all Bats tests#1331
debarshiray merged 4 commits intocontainers:mainfrom
debarshiray:wip/rishi/test-system-shellcheck

Conversation

@debarshiray
Copy link
Copy Markdown
Member

No description provided.

@debarshiray debarshiray requested a review from martymichal as a code owner July 4, 2023 14:22
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 4, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 28:
  run --separate-stderr $TOOLBOX --version
                        ^------^ SC2086 (info): Double quote to prevent
                                 globbing and word splitting.

See: https://www.shellcheck.net/wiki/SC2086

containers#1331
@debarshiray debarshiray force-pushed the wip/rishi/test-system-shellcheck branch from 26e6560 to 7cc4772 Compare July 4, 2023 14:23
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/f9172e0d8b6943e6b564c74818f3ab97

✔️ unit-test SUCCESS in 9m 23s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 23s
✔️ unit-test-restricted SUCCESS in 8m 17s
✔️ system-test-fedora-rawhide SUCCESS in 26m 24s
✔️ system-test-fedora-38 SUCCESS in 25m 05s
✔️ system-test-fedora-37 SUCCESS in 24m 48s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 4, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 28:
  run --separate-stderr $TOOLBOX --version
                        ^------^ SC2086 (info): Double quote to prevent
                                 globbing and word splitting.

See: https://www.shellcheck.net/wiki/SC2086

containers#1331
@debarshiray debarshiray force-pushed the wip/rishi/test-system-shellcheck branch from 7cc4772 to 9e88266 Compare July 4, 2023 14:53
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/29cb2529f82d4b6ba5ba7b18a92c1b07

✔️ unit-test SUCCESS in 9m 07s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 34s
✔️ unit-test-restricted SUCCESS in 8m 22s
✔️ system-test-fedora-rawhide SUCCESS in 25m 27s
✔️ system-test-fedora-38 SUCCESS in 24m 27s
✔️ system-test-fedora-37 SUCCESS in 24m 27s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 4, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 28:
  run --separate-stderr $TOOLBOX --version
                        ^------^ SC2086 (info): Double quote to prevent
                                 globbing and word splitting.

See: https://www.shellcheck.net/wiki/SC2086

containers#1331
@debarshiray debarshiray force-pushed the wip/rishi/test-system-shellcheck branch from 9e88266 to f2efbfb Compare July 4, 2023 17:20
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/b4cd529d6d7b4568a49f1e85d66745da

✔️ unit-test SUCCESS in 9m 04s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 10s
✔️ unit-test-restricted SUCCESS in 8m 10s
system-test-fedora-rawhide FAILURE in 9m 10s
✔️ system-test-fedora-38 SUCCESS in 24m 50s
✔️ system-test-fedora-37 SUCCESS in 24m 29s

@debarshiray
Copy link
Copy Markdown
Member Author

Wow! Looks like a network problem caused podman pull to crash:

fedora-rawhide | not ok 1 setup_suite
fedora-rawhide | # (from function `assert_success' in file test/system/libs/bats-assert/src/assert.bash, line 114,
fedora-rawhide | #  from function `_setup_docker_registry' in file test/system/libs/helpers.bash, line 178,
fedora-rawhide | #  from function `setup_suite' in test file test/system/setup_suite.bash, line 58)
fedora-rawhide | #   `_setup_docker_registry' failed
fedora-rawhide | #
fedora-rawhide | # -- command failed --
fedora-rawhide | # status : 134
fedora-rawhide | # output (1270 lines):
fedora-rawhide | #   Trying to pull quay.io/toolbox_tests/registry:latest...
fedora-rawhide | #   Getting image source signatures
fedora-rawhide | #   Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
fedora-rawhide | #   Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
fedora-rawhide | #   Copying blob sha256:792d5e381e06530261139997a960a2dd7cc771af1d6c8ddde680468c2082fcdd
fedora-rawhide | #   Copying blob sha256:7a854618fb4b3af401a78181e3bf7ca1d9608493f822839f254ac0ec2a36a2df
fedora-rawhide | #   Copying blob sha256:cdee6680187b23f826eb651b125a7f16245aaa0381d7ed46a1b666e606c75d80
fedora-rawhide | #   Copying blob sha256:4980e9f75679a80a546f5d8c386ea8f77360100f2d8e8b0ae8782dab79f5d995
fedora-rawhide | #   Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
fedora-rawhide | #   Copying blob sha256:0e99fe58b59645892107c30b0f25fed2e0777d6d94ce22101d1b8988d1ea06a5
fedora-rawhide | #   Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
fedora-rawhide | #   Copying blob sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
fedora-rawhide | #   runtime: marked free object in span 0x7fd65f59b578, elemsize=16 freeindex=0 (bad use of unsafe.Pointer? try -d=checkptr)
...
...
...

@debarshiray
Copy link
Copy Markdown
Member Author

recheck

@softwarefactory-project-zuul
Copy link
Copy Markdown

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/a463beecb14a4968bacf8fd80e9c7372

✔️ unit-test SUCCESS in 9m 32s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 09s
✔️ unit-test-restricted SUCCESS in 8m 27s
✔️ system-test-fedora-rawhide SUCCESS in 25m 51s
✔️ system-test-fedora-38 SUCCESS in 25m 39s
✔️ system-test-fedora-37 SUCCESS in 24m 25s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 5, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 28:
  run --separate-stderr $TOOLBOX --version
                        ^------^ SC2086 (info): Double quote to prevent
                                 globbing and word splitting.

See: https://www.shellcheck.net/wiki/SC2086

containers#1331
@debarshiray debarshiray force-pushed the wip/rishi/test-system-shellcheck branch from f2efbfb to 8985f12 Compare July 5, 2023 13:01
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/41ff60e4ad6f466e94f606f97b41c148

✔️ unit-test SUCCESS in 10m 25s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 4m 30s
✔️ unit-test-restricted SUCCESS in 9m 08s
✔️ system-test-fedora-rawhide SUCCESS in 28m 09s
✔️ system-test-fedora-38 SUCCESS in 26m 41s
system-test-fedora-37 NODE_FAILURE Node request 200-0006175323 failed in 0s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 7, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 28:
  run --separate-stderr $TOOLBOX --version
                        ^------^ SC2086 (info): Double quote to prevent
                                 globbing and word splitting.

See: https://www.shellcheck.net/wiki/SC2086

containers#1331
@debarshiray debarshiray force-pushed the wip/rishi/test-system-shellcheck branch from 8985f12 to 73e756f Compare July 7, 2023 14:38
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/47945763cb184e65ae9e6d49c0ed63eb

unit-test RETRY_LIMIT in 55s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 2m 59s
unit-test-restricted RETRY_LIMIT in 42s
system-test-fedora-rawhide RETRY_LIMIT in 40s
✔️ system-test-fedora-38 SUCCESS in 24m 47s
✔️ system-test-fedora-37 SUCCESS in 24m 53s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 7, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 28:
  run --separate-stderr $TOOLBOX --version
                        ^------^ SC2086 (info): Double quote to prevent
                                 globbing and word splitting.

See: https://www.shellcheck.net/wiki/SC2086

containers#1331
@debarshiray debarshiray force-pushed the wip/rishi/test-system-shellcheck branch from 73e756f to 66cd03d Compare July 7, 2023 16:02
@debarshiray
Copy link
Copy Markdown
Member Author

This is odd:

TASK [Install RPM packages]
fedora-rawhide | ERROR
fedora-rawhide | {
fedora-rawhide |   "msg": "Could not import the dnf python module using /usr/bin/python3 (3.12.0b3 (main, Jun 21 2023, 00:00:00) [GCC 13.1.1 20230614 (Red Hat 13.1.1-4)]). Please install `python3-dnf` or `python2-dnf` package or ensure you have specified the correct ansible_python_interpreter. (attempted ['/usr/libexec/platform-python', '/usr/bin/python3', '/usr/bin/python2', '/usr/bin/python'])",
fedora-rawhide |   "results": []
fedora-rawhide | }

Doesn't look like anything to do with Toolbx.

@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/d660c18584fb4a708f605ac132e52faf

unit-test RETRY_LIMIT in 1m 00s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 24s
unit-test-restricted RETRY_LIMIT in 41s
system-test-fedora-rawhide RETRY_LIMIT in 41s
✔️ system-test-fedora-38 SUCCESS in 25m 17s
✔️ system-test-fedora-37 SUCCESS in 23m 57s

@debarshiray debarshiray marked this pull request as draft July 9, 2023 09:47
@debarshiray debarshiray changed the title build: Enforce shellcheck(1) on all Bats tests [WIP] build: Enforce shellcheck(1) on all Bats tests Jul 9, 2023
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 11, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 218:
  source <(echo "$output")
         ^---------------^ SC1090 (warning): ShellCheck can't follow
                           non-constant source. Use a directive to
                           specify location.

See: https://www.shellcheck.net/wiki/SC1090

containers#1331
@debarshiray debarshiray force-pushed the wip/rishi/test-system-shellcheck branch from 66cd03d to e5ea6b9 Compare July 11, 2023 18:45
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/e5519ac6d27b4532852e5ed770bc4c76

✔️ unit-test SUCCESS in 10m 03s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 18s
✔️ unit-test-restricted SUCCESS in 9m 07s
✔️ system-test-fedora-rawhide SUCCESS in 26m 09s
✔️ system-test-fedora-38 SUCCESS in 25m 41s
✔️ system-test-fedora-37 SUCCESS in 25m 09s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 12, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 28:
  run --separate-stderr $TOOLBOX --version
                        ^------^ SC2086 (info): Double quote to prevent
                                 globbing and word splitting.

See: https://www.shellcheck.net/wiki/SC2086

containers#1331
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/4d7253c247d84fc98358039e8b6f61b5

✔️ unit-test SUCCESS in 10m 09s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 07s
✔️ unit-test-restricted SUCCESS in 9m 07s
✔️ system-test-fedora-rawhide SUCCESS in 30m 16s
✔️ system-test-fedora-38 SUCCESS in 29m 02s
✔️ system-test-fedora-37 SUCCESS in 27m 51s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Aug 12, 2023
These files aren't marked as executable, and shouldn't be, because they
aren't meant to be standalone executable scripts.  They're meant to be
part of a test suite driven by Bats.  Therefore, it doesn't make sense
for them to have shebangs, because it gives the opposite impression.

The shebangs were actually being used by external tools like Coverity to
deduce the shell when running shellcheck(1).  Shellcheck's inline
'shell' directive is a more obvious way to achieve that.

containers#1331
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Aug 12, 2023
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/604dfebb97764019b79dd876bedc3e90

unit-test RETRY_LIMIT in 36s
unit-test-migration-path-for-coreos-toolbox FAILURE in 3m 32s
unit-test-restricted RETRY_LIMIT in 32s
system-test-fedora-rawhide RETRY_LIMIT in 34s
✔️ system-test-fedora-38 SUCCESS in 27m 20s
✔️ system-test-fedora-37 SUCCESS in 26m 50s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Sep 14, 2023
The setup_suite.bash file is meant to be written in Bash, and is not
supposed to have any Bats-specific syntax.  That's why it has the *.bash
suffix, not *.bats.  If Bats finds a setup_suite.bash file, when running
the test suite, it uses Bash's source(1) builtin to read the file.

This is a cosmetic change.  The Bats syntax is a superset of the Bash
syntax.  Therefore, it didn't make a difference to external tools like
Coverity that use the shebang to deduce the shell for shellcheck(1).
Secondly setup_suite.bash isn't meant to be an executable script and,
hence, the shebang has no effect on how the file is used.  However, it's
still a commonly used hint about the contents of the file, and it's
better to be accurate than misleading.

A subsequent commit will replace the shebangs in the test suite with
ShellCheck's 'shell' directives.

Fallout from 7a387dc

containers#1331
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Sep 14, 2023
These files aren't marked as executable, and shouldn't be, because they
aren't meant to be standalone executable scripts.  They're meant to be
part of a test suite driven by Bats.  Therefore, it doesn't make sense
for them to have shebangs, because it gives the opposite impression.

The shebangs were actually being used by external tools like Coverity to
deduce the shell when running shellcheck(1).  Shellcheck's inline
'shell' directive is a more obvious way to achieve that.

containers#1331
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Sep 19, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 28:
  run --separate-stderr $TOOLBOX --version
                        ^------^ SC2086 (info): Double quote to prevent
                                 globbing and word splitting.

See: https://www.shellcheck.net/wiki/SC2086

containers#1331
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Sep 19, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 339:
  pull_distro_image $(get_system_id) $(get_system_version)
                    ^--------------^ SC2046 (warning): Quote this to
                                     prevent word splitting.

See: https://www.shellcheck.net/wiki/SC2046

containers#1331
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Sep 19, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 143:
  assert_line --index 0 "~/.bash_profile read"
                        ^------------------^ SC2088 (warning): Tilde
                                             does not expand in quotes.
                                             Use $HOME.

See: https://www.shellcheck.net/wiki/SC2088

This is a false positive.  There's no need for the tilde to be expanded
because it's not being used for any file system operation.  It's merely
a human-readable string.

However, it's easier to change the string to use $HOME than littering
the file with ShellCheck's inline 'disable' directives.

containers#1331
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Sep 19, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 33:
  assert [ ${#stderr_lines[@]} -eq 0 ]
           ^-----------------^ SC2154 (warning): stderr_lines is
                               referenced but not assigned.

See: https://www.shellcheck.net/wiki/SC2154

containers#1331
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Sep 20, 2023
Otherwise https://www.shellcheck.net/ would complain:
  Line 624:
  local system_id="$(get_system_id)"
        ^-------^ SC2155 (warning): Declare and assign separately to
                  avoid masking return values.

See: https://www.shellcheck.net/wiki/SC2155

containers#1331
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Sep 22, 2023
@debarshiray debarshiray force-pushed the wip/rishi/test-system-shellcheck branch from ed80022 to c5ab2cf Compare September 22, 2023 16:28
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/669cb3b2e92c43d6bb17854c013ee228

unit-test FAILURE in 8m 46s
unit-test-migration-path-for-coreos-toolbox FAILURE in 4m 18s
unit-test-restricted FAILURE in 6m 55s
system-test-fedora-rawhide FAILURE in 35m 20s
✔️ system-test-fedora-38 SUCCESS in 26m 50s
✔️ system-test-fedora-37 SUCCESS in 27m 14s

Bats' 'run' helper is not necessary to merely check if a command
succeeded or not [1].  In this case, it's idiomatic to use the command
as the condition for an 'if' branch.

[1] https://bats-core.readthedocs.io/en/stable/writing-tests.html

containers#1378
This removes any ambiguities and makes it clear what value is being
returned.

containers#1378
Otherwise https://www.shellcheck.net/ would complain:
  Line 343:
  if [ "$status" -ne 0 ]; then
        ^-----^ SC2154 (warning): status is referenced but not assigned.

See: https://www.shellcheck.net/wiki/SC2154

containers#1378
@debarshiray debarshiray force-pushed the wip/rishi/test-system-shellcheck branch from c5ab2cf to 4ebaea6 Compare September 25, 2023 16:19
@debarshiray debarshiray changed the title [WIP] build: Enforce shellcheck(1) on all Bats tests build: Enforce shellcheck(1) on all Bats tests Sep 25, 2023
@debarshiray debarshiray marked this pull request as ready for review September 25, 2023 16:22
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/30722bbb48b7485baedb4e9d2be754cc

✔️ unit-test SUCCESS in 8m 27s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 37s
✔️ unit-test-restricted SUCCESS in 7m 20s
system-test-fedora-rawhide FAILURE in 31m 57s
✔️ system-test-fedora-38 SUCCESS in 24m 52s
✔️ system-test-fedora-37 SUCCESS in 24m 33s

@debarshiray
Copy link
Copy Markdown
Member Author

There are still some test failures on Fedora Rawhide. For example:

fedora-rawhide | not ok 3 help: Run command 'help' in 145ms
fedora-rawhide | # (from function `assert_line' in file test/system/libs/bats-assert/src/assert.bash, line 479,
fedora-rawhide | #  in test file test/system/002-help.bats, line 45)
fedora-rawhide | #   `assert_line --index 0 --partial "toolbox(1)"' failed
fedora-rawhide | # /usr/bin/man
fedora-rawhide | #
fedora-rawhide | # -- line does not contain substring --
fedora-rawhide | # index     : 0
fedora-rawhide | # substring : toolbox(1)
fedora-rawhide | # line      : troff:<standard input>:33: warning: cannot select font 'C'
fedora-rawhide | # --

I believe these are because of changes in various other components in Fedora 39, which we need to track down one by one and work out a fix.

In the mean time, I am going to temporarily override these failures.

@debarshiray debarshiray merged commit 4ebaea6 into containers:main Sep 25, 2023
@debarshiray debarshiray deleted the wip/rishi/test-system-shellcheck branch September 25, 2023 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant