Skip to content

test/system: Consolidate teardown() to remove both containers & images#1526

Merged
debarshiray merged 1 commit intocontainers:mainfrom
debarshiray:wip/rishi/test-system-clean-containers-versus-all
Aug 29, 2024
Merged

test/system: Consolidate teardown() to remove both containers & images#1526
debarshiray merged 1 commit intocontainers:mainfrom
debarshiray:wip/rishi/test-system-clean-containers-versus-all

Conversation

@debarshiray
Copy link
Copy Markdown
Member

@debarshiray debarshiray commented Aug 14, 2024

It's far more consistent and understandable if all tests start with a
clean state without any containers or images present. Otherwise, the
subtle side-effects of having some image left behind from a previous
test can lead to surprises, and there's no need to spend time wondering
whether some tests should only clean up the containers or both
containers and images.

@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/ad998fd64f4041bba8d7c8bca27bd10e

✔️ unit-test SUCCESS in 6m 17s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 04s
✔️ unit-test-restricted SUCCESS in 5m 48s
system-test-fedora-rawhide FAILURE in 7m 11s
✔️ system-test-fedora-40 SUCCESS in 37m 29s
✔️ system-test-fedora-39 SUCCESS in 37m 06s
system-test-fedora-38 NODE_FAILURE Node request 200-0007534976 failed in 0s

@debarshiray
Copy link
Copy Markdown
Member Author

This is odd:

1..368
# test suite: Set up
# test suite: Tear down
not ok 1 setup_suite
# (from function `setup_suite' in test file test/system/setup_suite.bash, line 48)
#   `_pull_and_cache_distro_image rhel 8.10 || false' failed
# Failed to cache image registry.access.redhat.com/ubi8/toolbox:8.10 to /var/tmp/bats-run-kpL047/suite/image-cache/rhel-toolbox-8.10
# time="2024-08-14T19:15:23Z" level=fatal msg="copying system image from manifest list: writing blob: write /var/tmp/bats-run-kpL047/suite/image-cache/rhel-toolbox-8.10/dir-put-blob3332192469: no space left on device"
# bats warning: Executed 1 instead of expected 368 tests

no space left on device

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Aug 14, 2024
@debarshiray debarshiray force-pushed the wip/rishi/test-system-clean-containers-versus-all branch from 368f910 to bb08f47 Compare August 14, 2024 22:16
@debarshiray
Copy link
Copy Markdown
Member Author

recheck

@debarshiray
Copy link
Copy Markdown
Member Author

This is odd:

1..368
# test suite: Set up
# test suite: Tear down
not ok 1 setup_suite
# (from function `setup_suite' in test file test/system/setup_suite.bash, line 48)
#   `_pull_and_cache_distro_image rhel 8.10 || false' failed
# Failed to cache image registry.access.redhat.com/ubi8/toolbox:8.10 to /var/tmp/bats-run-kpL047/suite/image-cache/rhel-toolbox-8.10
# time="2024-08-14T19:15:23Z" level=fatal msg="copying system image from manifest list: writing blob: write /var/tmp/bats-run-kpL047/suite/image-cache/rhel-toolbox-8.10/dir-put-blob3332192469: no space left on device"
# bats warning: Executed 1 instead of expected 368 tests

no space left on device

Earlier this was happening only on Fedora Rawhide, but now it's happening on Fedora 40 too.

@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/07e4fee2ec4c4e57abbc7238f1a76cae

✔️ unit-test SUCCESS in 5m 57s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 26s
✔️ unit-test-restricted SUCCESS in 5m 45s
system-test-fedora-rawhide FAILURE in 7m 09s
system-test-fedora-40 FAILURE in 8m 53s
✔️ system-test-fedora-39 SUCCESS in 52m 17s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Aug 15, 2024
It's far more consistent and understandable if all tests start with a
clean state without any containers or images present.  Otherwise, the
subtle side-effects of having some image left behind from a previous
test can lead to surprises, and there's no need to spend time wondering
whether some tests should only clean up the containers or both
containers and images.

containers#1526
@debarshiray debarshiray force-pushed the wip/rishi/test-system-clean-containers-versus-all branch from bb08f47 to 475c693 Compare August 15, 2024 16:39
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Aug 15, 2024
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/bdcdf5c70ca24438915df4b7cfa2d794

✔️ unit-test SUCCESS in 6m 55s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 12s
✔️ unit-test-restricted SUCCESS in 5m 45s
system-test-fedora-rawhide FAILURE in 8m 06s
system-test-fedora-40 FAILURE in 8m 30s
system-test-fedora-39 TIMED_OUT in 1h 00m 25s

@debarshiray
Copy link
Copy Markdown
Member Author

recheck

@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/122dadec97d6436aa3b861a3db24d2d2

✔️ unit-test SUCCESS in 5m 58s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 07s
✔️ unit-test-restricted SUCCESS in 5m 45s
system-test-fedora-rawhide FAILURE in 7m 16s
system-test-fedora-40 FAILURE in 8m 27s
system-test-fedora-39 TIMED_OUT in 1h 00m 20s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Aug 16, 2024
It's far more consistent and understandable if all tests start with a
clean state without any containers or images present.  Otherwise, the
subtle side-effects of having some image left behind from a previous
test can lead to surprises, and there's no need to spend time wondering
whether some tests should only clean up the containers or both
containers and images.

This additional work of cleaning up the images for all tests makes it
necessary to increase the timeout for all Fedora nodes to prevent the CI
from timing out.

containers#1526
@debarshiray debarshiray force-pushed the wip/rishi/test-system-clean-containers-versus-all branch from 475c693 to aa46462 Compare August 16, 2024 16:56
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/db1ec688df3c4a978faf1cada8ed46c1

✔️ unit-test SUCCESS in 6m 49s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 24s
✔️ unit-test-restricted SUCCESS in 5m 46s
system-test-fedora-rawhide FAILURE in 8m 03s
system-test-fedora-40 FAILURE in 8m 43s
system-test-fedora-39 TIMED_OUT in 1h 20m 19s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Aug 16, 2024
It's far more consistent and understandable if all tests start with a
clean state without any containers or images present.  Otherwise, the
subtle side-effects of having some image left behind from a previous
test can lead to surprises, and there's no need to spend time wondering
whether some tests should only clean up the containers or both
containers and images.

This additional work of cleaning up the images for all tests makes it
necessary to increase the timeout for all Fedora nodes to prevent the CI
from timing out.

containers#1526
@debarshiray debarshiray force-pushed the wip/rishi/test-system-clean-containers-versus-all branch from aa46462 to f5ba81f Compare August 16, 2024 18:31
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/dcb457d4bc354951aacf0e12c4cf6607

✔️ unit-test SUCCESS in 6m 45s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 22s
✔️ unit-test-restricted SUCCESS in 5m 38s
system-test-fedora-rawhide FAILURE in 8m 03s
system-test-fedora-40 FAILURE in 8m 44s
system-test-fedora-39 TIMED_OUT in 1h 40m 19s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Aug 16, 2024
It's far more consistent and understandable if all tests start with a
clean state without any containers or images present.  Otherwise, the
subtle side-effects of having some image left behind from a previous
test can lead to surprises, and there's no need to spend time wondering
whether some tests should only clean up the containers or both
containers and images.

This additional work of cleaning up the images for all tests makes it
necessary to increase the timeout for all Fedora nodes to prevent the CI
from timing out.

containers#1526
@debarshiray debarshiray force-pushed the wip/rishi/test-system-clean-containers-versus-all branch from f5ba81f to 4264173 Compare August 16, 2024 20:34
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build failed.
https://softwarefactory-project.io/zuul/t/local/buildset/a497551f831945ddb567ac5c6edcfa2e

✔️ unit-test SUCCESS in 6m 52s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 00s
✔️ unit-test-restricted SUCCESS in 5m 34s
system-test-fedora-rawhide FAILURE in 8m 10s
system-test-fedora-40 FAILURE in 8m 40s
system-test-fedora-39 TIMED_OUT in 1h 50m 22s

It's far more consistent and understandable if all tests start with a
clean state without any containers or images present.  Otherwise, the
subtle side-effects of having some image left behind from a previous
test can lead to surprises, and there's no need to spend time wondering
whether some tests should only clean up the containers or both
containers and images.

This additional work of cleaning up the images for all tests makes it
necessary to increase the timeout for all Fedora nodes to prevent the CI
from timing out.

containers#1526
@debarshiray debarshiray force-pushed the wip/rishi/test-system-clean-containers-versus-all branch from 4264173 to 67d4002 Compare August 29, 2024 19:15
@softwarefactory-project-zuul
Copy link
Copy Markdown

Build succeeded.
https://softwarefactory-project.io/zuul/t/local/buildset/2fd3c64e9c974520b9462e8caea3d727

✔️ unit-test SUCCESS in 5m 37s
✔️ unit-test-migration-path-for-coreos-toolbox SUCCESS in 3m 38s
✔️ unit-test-restricted SUCCESS in 5m 28s
✔️ system-test-fedora-rawhide SUCCESS in 1h 26m 57s
✔️ system-test-fedora-40 SUCCESS in 1h 25m 39s
✔️ system-test-fedora-39 SUCCESS in 1h 28m 14s

@debarshiray
Copy link
Copy Markdown
Member Author

Seems like Software Factory somehow fixed itself. Yay!

@debarshiray debarshiray merged commit 67d4002 into containers:main Aug 29, 2024
@debarshiray debarshiray deleted the wip/rishi/test-system-clean-containers-versus-all branch August 29, 2024 21:41
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jan 26, 2026
It's far more consistent and understandable if all tests start with a
clean state without any containers or images present.  Otherwise, the
subtle side-effects of having some image left behind from a previous
test can lead to surprises, and there's no need to spend time wondering
whether some tests should only clean up the containers or both
containers and images.

This additional work of cleaning up the images for all tests makes it
necessary to increase the timeout for all Fedora nodes to prevent the CI
from timing out.

containers#1526
containers#1740
(backported from commit 67d4002)
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jan 26, 2026
It's far more consistent and understandable if all tests start with a
clean state without any containers or images present.  Otherwise, the
subtle side-effects of having some image left behind from a previous
test can lead to surprises, and there's no need to spend time wondering
whether some tests should only clean up the containers or both
containers and images.

This additional work of cleaning up the images for all tests makes it
necessary to increase the timeout for all Fedora nodes to prevent the CI
from timing out.

containers#1526
containers#1741
(backported from commits 67d4002 and
 55c0e63)
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Feb 2, 2026
It's far more consistent and understandable if all tests start with a
clean state without any containers or images present.  Otherwise, the
subtle side-effects of having some image left behind from a previous
test can lead to surprises, and there's no need to spend time wondering
whether some tests should only clean up the containers or both
containers and images.

This additional work of cleaning up the images for all tests makes it
necessary to increase the timeout for all Fedora nodes to prevent the CI
from timing out.

containers#1526
containers#1741
containers#1750
(backported from commits 67d4002 and
 55c0e63)
(backported from commit 23b4a34)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant