Skip to content

Fix apparent typos in zstd:chunked tests#24686

Merged
openshift-merge-bot[bot] merged 3 commits intocontainers:mainfrom
mtrmac:test-typo
Nov 28, 2024
Merged

Fix apparent typos in zstd:chunked tests#24686
openshift-merge-bot[bot] merged 3 commits intocontainers:mainfrom
mtrmac:test-typo

Conversation

@mtrmac
Copy link
Copy Markdown
Contributor

@mtrmac mtrmac commented Nov 26, 2024

Does this PR introduce a user-facing change?

None

@openshift-ci openshift-ci bot added release-note-none approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Nov 26, 2024
Copy link
Copy Markdown
Member

@giuseppe giuseppe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Member

@Luap99 Luap99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this change did not cause the test to fail are we testing the right thing? Clearly this option does not seem to matter so why are we even setting it?

@giuseppe
Copy link
Copy Markdown
Member

If this change did not cause the test to fail are we testing the right thing? Clearly this option does not seem to matter so why are we even setting it?

we force to force the pull to the "partial pulls" code path even with traditional images.

We can improve the test to really check that it is what happens (and we could have caught the issue earlier doing so), but we need that flag to test the conversion code path.

@Luap99
Copy link
Copy Markdown
Member

Luap99 commented Nov 27, 2024

We can improve the test to really check that it is what happens (and we could have caught the issue earlier doing so)

Well we must do that, without it the test seems like a NOP. If you want to test the "partial pulls" code path then the test MUST check that this code path is used. Passing the right cli option is not good enough. What if the option passing has a bug or we do not pass the option along the right code paths, etc...? If the test didn't fail before it will not fail when a regression causes the code path to not be executed.

@giuseppe
Copy link
Copy Markdown
Member

@mtrmac could you please amend something like?

diff --git a/test/system/155-partial-pull.bats b/test/system/155-partial-pull.bats
index 961354bc7..e1673eb4a 100644
--- a/test/system/155-partial-pull.bats
+++ b/test/system/155-partial-pull.bats
@@ -84,9 +84,10 @@ EOF
 
     run_podman $globalargs rmi $image2 $image3
 
-    run_podman $globalargs pull \
+    run_podman --log-level debug $globalargs pull \
                $pushpullargs \
                $image2
+    assert "$output" =~ "Retrieved partial blob"
 
     run -0 skopeo inspect containers-storage:$image2
     assert "$output" =~ "application/vnd.oci.image.layer.v1.tar\+zstd" "pulled image must be zstd-compressed"
@@ -182,9 +183,10 @@ EOF
 
     run_podman $globalargs rmi $image
 
-    run_podman $globalargs pull \
+    run_podman --log-level debug $globalargs pull \
                $pushpullargs \
                $image
+    assert "$output" =~ "Retrieved partial blob"
 
     # expect that the image contains exactly the same data as before
     mount_image_and_take_digest $image

@mtrmac mtrmac marked this pull request as draft November 27, 2024 18:15
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 27, 2024
@mtrmac
Copy link
Copy Markdown
Contributor Author

mtrmac commented Nov 27, 2024

we force to force the pull to the "partial pulls" code path even with traditional images.

That’s not what this option does, FWIW (that would be convert_images). This one enables partial pulls at all, they are now disabled by default.

@mtrmac
Copy link
Copy Markdown
Contributor Author

mtrmac commented Nov 27, 2024

For the record: As of commit 466e1af , adding the checks for Retrieved partial blob but not fixing the typo in the option, tests fail (and the logs say Failed to retrieve partial blob: partial images are disabled).

@mtrmac
Copy link
Copy Markdown
Contributor Author

mtrmac commented Nov 27, 2024

Totally unrelated, but: https://cirrus-ci.com/task/6587238856785920 reports success, but the detailed logs https://api.cirrus-ci.com/v1/task/6587238856785920/logs/main.log show 4 failed tests (all in the setup phase). Is that expected???

@Luap99 any idea?

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Copy link
Copy Markdown
Member

@Luap99 Luap99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally unrelated, but: https://cirrus-ci.com/task/6587238856785920 reports success, but the detailed logs https://api.cirrus-ci.com/v1/task/6587238856785920/logs/main.log show 4 failed tests (all in the setup phase). Is that expected???

Uhm no, like this is bad, very bad... I need to do some tests there if bats didn't exit > 0 (bats problem) or if our script that runs bats twice ignores the exit code somehow (unlikely in my eyes as I have be seen many test failures since we changed that)


function setup() {
skip_if_remote "none of these tests work with podman-remote"
skip_if_remote "zstd:chunked tests depend on start_registry, which does not work with podman-remote"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not just that,
--storage-driver, --pull-option enable_partial_pulls=true, --cert-dir are all gloab podman options that are not supported by the remote client as they need to be set on the server. But in sys test there is only one server for all tests.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found “does not work with podman-remote” 100% unhelpful, so I’m trying to capture reasons.

Is the current version better?

If Podman maintainers know what this is about and “does not work with podman-remote” is sufficient, I’m happy to leave things alone.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I agree skips must have a meaningful reason. skip_if_remote "doesn't work with remote" is pointless as skip_if_remote said as much. It should list a proper reason why not, i.e. what you did although the actual reason why start_registry doe snot work is that it uses gloabal storage options.

Basically any test using global podman options will not work correctly in a remote context.

}

@test "zstd chunked does not modify image content" {
skip_if_remote "need to mount an image"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the first skip is part of setup() which is a magic bats function that will be run for each test case so this here is redundant

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes… I was trying to capture “this is one more reason why the test won’t work” but I appreciate that having a redundant skip here might be confusing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now with a comment.

@mtrmac
Copy link
Copy Markdown
Contributor Author

mtrmac commented Nov 27, 2024

Totally unrelated, but: https://cirrus-ci.com/task/6587238856785920 reports success, but the detailed logs https://api.cirrus-ci.com/v1/task/6587238856785920/logs/main.log show 4 failed tests (all in the setup phase). Is that expected???

Uhm no, like this is bad, very bad...

I have split an attempt to reproduce that, + a bit of debug logging, into #24697 . I’d be perfectly happy to leave investigating this to an expert.

@Luap99
Copy link
Copy Markdown
Member

Luap99 commented Nov 27, 2024

Totally unrelated, but: https://cirrus-ci.com/task/6587238856785920 reports success, but the detailed logs https://api.cirrus-ci.com/v1/task/6587238856785920/logs/main.log show 4 failed tests (all in the setup phase). Is that expected???

Uhm no, like this is bad, very bad...

I have split an attempt to reproduce that, + a bit of debug logging, into #24697 . I’d be perfectly happy to leave investigating this to an expert.

I will chase this down, don't worry about it. Running this through several CI cycles is pretty pointless for me I can easily run the test locally.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
@mtrmac
Copy link
Copy Markdown
Contributor Author

mtrmac commented Nov 27, 2024

Updated:

  • Added the sanity check that we are truly using the chunked pull code path, thanks to @giuseppe (and 466e1af shows that this does cause failures without the original typo fix)
  • … except on VFS, where we can’t be doing that (but keep the test running, to confirm that the fallback code path is not broken for chunked images)
  • Tried to better document the reasons to skip the test
  • Manually confirmed the remote tests are not failing now; a true fix will be make remotesystem: fail early if serial tests fail #24698 .

Ready for review.

@mtrmac mtrmac marked this pull request as ready for review November 27, 2024 21:43
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 27, 2024
Copy link
Copy Markdown
Member

@Luap99 Luap99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci bot commented Nov 28, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: giuseppe, Luap99, mtrmac

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [Luap99,giuseppe,mtrmac]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@giuseppe
Copy link
Copy Markdown
Member

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 28, 2024
@openshift-merge-bot openshift-merge-bot bot merged commit 18191d6 into containers:main Nov 28, 2024
@mtrmac mtrmac deleted the test-typo branch November 28, 2024 17:49
@stale-locking-app stale-locking-app bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Feb 27, 2025
@stale-locking-app stale-locking-app bot locked as resolved and limited conversation to collaborators Feb 27, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. release-note-none

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants