Skip to content

Fix unexpected results when filtering images#2413

Merged
openshift-merge-bot[bot] merged 2 commits into
containers:mainfrom
Honny1:list-images
Jun 26, 2025
Merged

Fix unexpected results when filtering images#2413
openshift-merge-bot[bot] merged 2 commits into
containers:mainfrom
Honny1:list-images

Conversation

@Honny1

@Honny1 Honny1 commented Apr 7, 2025

Copy link
Copy Markdown
Member

The filtered image contains all Names of image. This causes the podman to list images that are not expected.

For example:
If an image box:latest is taged as example:latest and then images are filtered with reference=example, box:latest and example:latest will be output because the image has multiple names.

Fixes: podman-container-tools/podman#25725
Fixes: https://issues.redhat.com/browse/RUN-2726

@mtrmac mtrmac left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The imageMatchesReferenceFilter logic LGTM, that makes sense. The design of the upper parts of the call stack needs a bit of cleanup.

Comment thread libimage/filters.go
Comment thread libimage/filters.go
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters_test.go Outdated
Comment thread libimage/filters_test.go Outdated
Comment thread libimage/image.go Outdated
@Honny1 Honny1 force-pushed the list-images branch 4 times, most recently from 3e00335 to 8dd85f6 Compare April 8, 2025 17:37
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
if lookedUp != nil {
if lookedUp.ID() == img.ID() {
return true, nil
resolvedName, _, _ := strings.Cut(resolvedName, "@")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this doing?

In the later loop, when we are making various imprecise matches, the return value is the full refString. This seems to be dropping digests from the returned values. Why?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still have no idea what this is doing. AFAICS LookupImage can easily be a (precise) repo:tag or repo@digest match. And the range refs loop below always collects the precise ref.String() values that match. Why is this ignoring the digest part (if that is what it is) and returning all tags (if that is what it is)?

And, even apart from the again, HasPrefix just looks implausible. Why should example.com/a match example.com/ab?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only want to return names that match the name if the name@digest reference format is used.

So, for example, for an image with the following names:

  • quay.io/libpod/alpine:latest
  • quay.io/libpod/al:latest
  • quay.io/libpod/al:tag

I expect the result for filtering with the al@digest reference, these names:

  • quay.io/libpod/al:latest
  • quay.io/libpod/al:tag

Usage HasPrefix is wrong. I fixed that.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK… It’s very unclear to me that a filter for reference=name@digest should return name:tag (and I’ll not even comment on the ignoreme thing, I guess that’s what we deserve), but it’s a conversation we can have, or maybe a non-negotiable backward compatibility constraint.

Where do short names, and hostname-less paths, come in? For ordinary tags we have the 1)2)3) candidates for matching, and that doesn’t happen here.

I think it’s a fair point to recognize that if value contains a digest, we probably go through this code path and not the later path.Match code, so we only truly need to handle digests on this path (is that the case???). But I still think that the two should be as consistent as possible.

Also consider inputs like alpine@digest matching all four of quay.io/libpod/alpine@digest, quay.io/libpod/alpine:latest, example.com/mirror/alpine@digest, example.com/mirror/alpine@latest. I don’t know whether the output should contain tags but I think it definitely should contain both host names, because that happens for tags in the 3) case.


IOW, conceptually, I think we should first decide whether the image matches, and then build resolvedNames, considering each entry of NamesReference, individually. And if that does need a digest special case, oh well, so be it.

(I don’t immediately know whether the code should be a “match” loop + ”build names” loop, or whether we should preserve the current 2 matching situations, with a “match a name to input” helper. Maybe the latter.)


Uh… what about *pine@digest? … Let’s pretend I didn’t ask, I think.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also consider inputs like alpine@digest matching all four of quay.io/libpod/alpine@digest, quay.io/libpod/alpine:latest, example.com/mirror/alpine@digest, example.com/mirror/alpine@latest. I don’t know whether the output should contain tags but I think it definitely should contain both host names, because that happens for tags in the 3) case.

Oh, I hadn't thought of that case at all. I think for the alpine@digest case, filterReferences() should return all names and thus keep the original behaviour. I checked the Podman documentation, and the reference filter accepts the pattern of an image reference <image-name>[:<tag>]. Is it okay to do that way?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“all” as in “not filter at all”, or “the ones that match alpine”?


I don’t know what are the constraints here. Podman documentation is definitely relevant, but also Docker’s API seems to be.

And Docker code, from a very quick check, seems to possibly match the input against repo@digest values. (If a reference.Named value matches a "reference=" filter, it is assigned to either RepoDigests or to RepoTags in the output. That’s for the traditional implementation; the containerd one is doing something else with RepoDigests, I can’t investigate the details now.) I’d also need to investigate further whether this always includes any exiting digest of an image, or only digests that were used to explicitly pull repo@digest.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“all” as in “not filter at all”, or “the ones that match alpine”?

Not filtered at all.

Comment thread libimage/filters.go Outdated
Comment thread libimage/image.go Outdated
@Honny1 Honny1 force-pushed the list-images branch 7 times, most recently from f644901 to c1d4722 Compare April 9, 2025 21:18
@Honny1 Honny1 requested a review from mtrmac April 9, 2025 21:31
@Honny1

Honny1 commented Apr 14, 2025

Copy link
Copy Markdown
Member Author

PTAL @mtrmac

@Honny1 Honny1 requested a review from Luap99 April 16, 2025 08:21

@Luap99 Luap99 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
if lookedUp != nil {
if lookedUp.ID() == img.ID() {
return true, nil
resolvedName, _, _ := strings.Cut(resolvedName, "@")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still have no idea what this is doing. AFAICS LookupImage can easily be a (precise) repo:tag or repo@digest match. And the range refs loop below always collects the precise ref.String() values that match. Why is this ignoring the digest part (if that is what it is) and returning all tags (if that is what it is)?

And, even apart from the again, HasPrefix just looks implausible. Why should example.com/a match example.com/ab?

@Honny1 Honny1 force-pushed the list-images branch 3 times, most recently from 3d1f37e to 289f353 Compare April 17, 2025 13:08
@Honny1

Honny1 commented Apr 17, 2025

Copy link
Copy Markdown
Member Author

@mtrmac PTAL, I have implemented your feedback.

@Honny1 Honny1 requested a review from mtrmac April 17, 2025 13:11
Comment thread libimage/runtime.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
@Honny1 Honny1 force-pushed the list-images branch 3 times, most recently from 73e55be to ba9f9e1 Compare April 30, 2025 17:18
Comment thread libimage/filters.go Outdated
@Honny1

Honny1 commented May 16, 2025

Copy link
Copy Markdown
Member Author

Podman prints the same list of all image names for all imputes that were given to Docker.

… Are you saying that Docker has the same behavior that containers/podman#25725 is complaining about? I was assuming that the 5 tags in the samples are point at the same deduplicate image, in which case the test cases do show references being filtered. Is the test using 5 different images?

No, I meant to say that I also tested Podman with the same inputs I used for the Docker examples. And the result was always the same list of 5 image tags. The test does not use 5 different images.

@Honny1

Honny1 commented May 16, 2025

Copy link
Copy Markdown
Member Author

That would mean giving up for Podman 5… and either documenting things, or… just… saying “it’s complicated, best to use precise image names, we are not making any promises for substrings and wildcards”.

I used a fully qualified image name, and Podman listed all the image tags/names.

$ podman images docker.io/library/alpine:latest
REPOSITORY                TAG         IMAGE ID      CREATED       SIZE
host1.example.com/foo     bar         8d591b0b7dea  3 months ago  8.47 MB
host2.example.com/foo     bar         8d591b0b7dea  3 months ago  8.47 MB
host1.example.com/test    foo         8d591b0b7dea  3 months ago  8.47 MB
host1.example.com/test    bar         8d591b0b7dea  3 months ago  8.47 MB
docker.io/library/alpine  latest      8d591b0b7dea  3 months ago  8.47 MB

I think we could say that if a fully qualified image name is used, the output should only show the corresponding names. And in the case of short names, wildcards, and digests. Podman should display all the names of the image.

If we agree on this behavior, I'm not sure how to check if the user input is a fully qualified image name, and if we should wait for Podman 6 to make this behavior change.

@mtrmac

mtrmac commented May 16, 2025

Copy link
Copy Markdown
Contributor

I think we could say that if a fully qualified image name is used, the output should only show the corresponding names. And in the case of short names, wildcards, and digests. Podman should display all the names of the image.

That could work. I’d include fully-qualified names with digests in the first case: “if the input 100% unambiguously specifes precisely one image, it is a query; if it is any more ambiguous, it is a search”.

But users with such a precise input can use podman image inspect, so… this clear case might not be that important for users of podman images.

If we agree on this behavior, I'm not sure how to check if the user input is a fully qualified image name

reference.ParseNamed (instead of the usual reference.ParseNormalizedNamed), or the equivalent check that ParseNormalizedNamed().String() matches the original input, would reject short names, and fail on wildcards.

Then !reference.IsNameOnly() to require a tag and/or digest to be present.


, and if we should wait for Podman 6 to make this behavior change.

That’s still a topic — I’m leaning towards Podman 6 but that’s a very weak opinion.

@Honny1

Honny1 commented May 19, 2025

Copy link
Copy Markdown
Member Author

I've updated the behavior so that the output only shows the corresponding names if a fully qualified image name is used, including name@digest. And wildcards and short names output all image names.

I'm not sure about this case:
Current status:

$ podman images --digests
REPOSITORY                TAG         DIGEST                                                                   IMAGE ID      CREATED       SIZE
docker.io/library/image   v2          sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c  8d591b0b7dea  3 months ago  8.47 MB
docker.io/library/image   v1          sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c  8d591b0b7dea  3 months ago  8.47 MB
docker.io/library/alpine  latest      sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c  8d591b0b7dea  3 months ago  8.47 MB

What should be the correct output of this command?

$ podman images docker.io/library/image@sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c 

Displays all names.

REPOSITORY                TAG         IMAGE ID      CREATED       SIZE
docker.io/library/image   v2          8d591b0b7dea  3 months ago  8.47 MB
docker.io/library/image   v1          8d591b0b7dea  3 months ago  8.47 MB
docker.io/library/alpine  latest      8d591b0b7dea  3 months ago  8.47 MB

Displays all tags that match the image name. (PR implementation)

REPOSITORY                TAG         IMAGE ID      CREATED       SIZE
docker.io/library/image   v2          8d591b0b7dea  3 months ago  8.47 MB
docker.io/library/image   v1          8d591b0b7dea  3 months ago  8.47 MB

WDYT? @mtrmac

@mtrmac mtrmac left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What should be the correct output [repo@digest]?

Yeah… I think the PR’s implementation is reasonable. In effect, we have 3 separate behaviors:

  • fqRepo:tag = print only that repo:tag
  • fqRepo@digest = print all matching tags (if any), and possibly the digest if it is in image.Names
  • anything else (short names, wildcards) = no filtering.

and we’d have to document them as such (with a caveat that this might change in Podman 6??).

@Luap99 / @mheon WDYT?


Procedurally, if we go one of these routes and don’t give up entirely: could you, split the filters_test.go refactor that shows matched names as a separate first commit, please? It’s not just cleaner as a matter of formal tidiness, it would mean that a second commit, actually changing behavior and updating the tests, would be explicit and self-documenting about the changes.

Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/runtime.go Outdated

@Luap99 Luap99 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I think it is fine to move ahead with this without a 6.0.
There might be a risk here that someone expects multiple names but I don't see why.
Overall since this changes the behaviour it might be best to to make a test PR against podman with this change and make sure all tests still pass first before merging. If it turns out this breaks a lot of our tests than maybe it is not a good change for a minor release

@Honny1

Honny1 commented Jun 11, 2025

Copy link
Copy Markdown
Member Author

Test PR: podman-container-tools/podman#26344

@mheon

mheon commented Jun 14, 2025

Copy link
Copy Markdown
Member

Code LGTM

@mtrmac mtrmac left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code, implementing a single-image filter, LGTM — but, still, #2413 (comment) .

Alternatives:

  • Only do the name filtering for the special case of a single filter in wantedReferenceMatches (and empty unwantedReferenceMatches?). It is a special case anyway, so let’s narrow down the case where we are changing behavior.
  • Treat each reference= separately, and build a set of accepted image names. (I.e. if anything in wantedReferenceMatches is a wildcard or similarly generic, include all names; if all wantedReferenceMatches elements are fully-qualified !NameOnly names, include only matches for those names
  • Something else?

I think it would be useful to think what the end-user documentation of the behavior will be (write a draft?). We can implement any of these behaviors, but documenting them in a way that can be understood seems harder than that.

Personally, I weakly lean towards the first option (with an explanation similar to #2413 (comment) “it is a query” vs. “it is a search”) — that would change the effect of podman images $singleInput, but any users doing something more complex with --filter would not see a change.


BTW I think Podman should eventually get a man page update documenting this.

Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
Comment thread libimage/runtime.go
Signed-off-by: Jan Rodák <hony.com@seznam.cz>
@Honny1

Honny1 commented Jun 24, 2025

Copy link
Copy Markdown
Member Author

@mtrmac I have modified the implementation so that name stripping is only done for the special case of a single reference in wantedReferenceMatches and empty unwantedReferenceMatches, i.e., when it is a query.

@Honny1 Honny1 requested a review from mtrmac June 24, 2025 13:57

@mtrmac mtrmac left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

Comment thread libimage/filters.go Outdated
Comment thread libimage/filters.go Outdated
The filtered image contains all Names of image. This causes the podman to list images that are not expected.

Fixes: podman-container-tools/podman#25725
Fixes: https://issues.redhat.com/browse/RUN-2726

Signed-off-by: Jan Rodák <hony.com@seznam.cz>
@Honny1

Honny1 commented Jun 25, 2025

Copy link
Copy Markdown
Member Author

I fixed the non-blocking nits. Can we merge this PR? @mtrmac

@Honny1 Honny1 requested a review from mtrmac June 25, 2025 09:57
@mtrmac

mtrmac commented Jun 25, 2025

Copy link
Copy Markdown
Contributor

@Luap99 or @mheon , PTANL.

@Luap99 Luap99 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci

openshift-ci Bot commented Jun 26, 2025

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Honny1, Luap99

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

podman images <image> can show duplicate results

4 participants