Skip to content

Fix multiarch image push tag for containerd snapshotter#49949

Merged
thaJeztah merged 1 commit intomoby:masterfrom
henry118:pushtag
May 19, 2025
Merged

Fix multiarch image push tag for containerd snapshotter#49949
thaJeztah merged 1 commit intomoby:masterfrom
henry118:pushtag

Conversation

@henry118
Copy link
Contributor

@henry118 henry118 commented May 9, 2025 •
edited by vvoland
Loading

- What I did
When the "containerd-snapshotter" feature is enabled and pushing multiarch images, I would expect all platform specific image manifest to be pushed by digest, but only the top level manifest index being pushed by tag. But in the current implementation in moby, all of them are pushed by tag.

- How I did it
Replicate the similar logic in containerd here: https://github.com/containerd/containerd/blob/main/client/client.go#L485-L488

- How to verify it
Turn on debug level and verify the logs.

Steps:

$ docker images
REPOSITORY                      TAG       IMAGE ID       CREATED         SIZE
localhost:5000/multi-platform   latest    e3456080ddef   5 hours ago     15.9MB

$ docker push localhost:5000/multi-platform:latest

Logs before fix:

    May 09 20:51:06 ip-172-31-53-148.us-west-2.compute.internal dockerd[350657]: time="2025-05-09T20:51:06.466443875Z" level=debug msg="checking and pushing to" digest="sha256:bee6572fc9601b636b4c9ca47f3242394c9acc16a2cafbe2b9d70863fa8bf68a" mediatype=application/vnd.oci.image.manifest.v1+json size=668 url="https://localhost:5000/v2/multi-platform/manifests/latest"
    May 09 20:51:06 ip-172-31-53-148.us-west-2.compute.internal dockerd[350657]: time="2025-05-09T20:51:06.466682422Z" level=debug msg="checking and pushing to" digest="sha256:26de95632909aec43e64b57756169187e4c9ad5b839dedde93d3ad1b38e40179" mediatype=application/vnd.oci.image.manifest.v1+json size=566 url="https://localhost:5000/v2/multi-platform/manifests/latest"
    May 09 20:51:06 ip-172-31-53-148.us-west-2.compute.internal dockerd[350657]: time="2025-05-09T20:51:06.466849174Z" level=debug msg="checking and pushing to" digest="sha256:4276f635f76d8ce2a58e74895718839ad845ee7c0616a7411253cfcc2b482e1b" mediatype=application/vnd.oci.image.manifest.v1+json size=668 url="https://localhost:5000/v2/multi-platform/manifests/latest"
    May 09 20:51:06 ip-172-31-53-148.us-west-2.compute.internal dockerd[350657]: time="2025-05-09T20:51:06.467165117Z" level=debug msg="checking and pushing to" digest="sha256:61a91a053385e6639c7d6c5619a9955574df7ba788bdcbe130778380b9c5649e" mediatype=application/vnd.oci.image.manifest.v1+json size=566 url="https://localhost:5000/v2/multi-platform/manifests/latest"
    May 09 20:51:06 ip-172-31-53-148.us-west-2.compute.internal dockerd[350657]: time="2025-05-09T20:51:06.487842936Z" level=debug msg="checking and pushing to" digest="sha256:e3456080ddefc6caf948b68daa8ee8ea0dda5ec46f272dcb55e0d6b9f1ed2e64" mediatype=application/vnd.oci.image.index.v1+json size=1607 url="https://localhost:5000/v2/multi-platform/manifests/latest"

Logs after fix:

May 09 20:48:38 ip-172-31-53-148.us-west-2.compute.internal dockerd[350106]: time="2025-05-09T20:48:38.810289068Z" level=debug msg="checking and pushing to" digest="sha256:bee6572fc9601b636b4c9ca47f3242394c9acc16a2cafbe2b9d70863fa8bf68a" mediatype=application/vnd.oci.image.manifest.v1+json size=668 url="https://localhost:5000/v2/multi-platform/manifests/sha256:bee6572fc9601b636b4c9ca47f3242394c9acc16a2cafbe2b9d70863fa8bf68a"
May 09 20:48:38 ip-172-31-53-148.us-west-2.compute.internal dockerd[350106]: time="2025-05-09T20:48:38.810358707Z" level=debug msg="checking and pushing to" digest="sha256:61a91a053385e6639c7d6c5619a9955574df7ba788bdcbe130778380b9c5649e" mediatype=application/vnd.oci.image.manifest.v1+json size=566 url="https://localhost:5000/v2/multi-platform/manifests/sha256:61a91a053385e6639c7d6c5619a9955574df7ba788bdcbe130778380b9c5649e"
May 09 20:48:38 ip-172-31-53-148.us-west-2.compute.internal dockerd[350106]: time="2025-05-09T20:48:38.810367373Z" level=debug msg="checking and pushing to" digest="sha256:26de95632909aec43e64b57756169187e4c9ad5b839dedde93d3ad1b38e40179" mediatype=application/vnd.oci.image.manifest.v1+json size=566 url="https://localhost:5000/v2/multi-platform/manifests/sha256:26de95632909aec43e64b57756169187e4c9ad5b839dedde93d3ad1b38e40179"
May 09 20:48:38 ip-172-31-53-148.us-west-2.compute.internal dockerd[350106]: time="2025-05-09T20:48:38.810552604Z" level=debug msg="checking and pushing to" digest="sha256:4276f635f76d8ce2a58e74895718839ad845ee7c0616a7411253cfcc2b482e1b" mediatype=application/vnd.oci.image.manifest.v1+json size=668 url="https://localhost:5000/v2/multi-platform/manifests/sha256:4276f635f76d8ce2a58e74895718839ad845ee7c0616a7411253cfcc2b482e1b"
May 09 20:48:38 ip-172-31-53-148.us-west-2.compute.internal dockerd[350106]: time="2025-05-09T20:48:38.815839022Z" level=debug msg="checking and pushing to" digest="sha256:e3456080ddefc6caf948b68daa8ee8ea0dda5ec46f272dcb55e0d6b9f1ed2e64" mediatype=application/vnd.oci.image.index.v1+json size=1607 url="https://localhost:5000/v2/multi-platform/manifests/latest"

containerd image store: Fix duplicate `PUT` requests when pushing a multi-platform image

@thaJeztah thaJeztah requested a review from dmcgowan May 9, 2025 23:10
@austinvazquez austinvazquez self-assigned this May 9, 2025
@thaJeztah
Copy link
Member

thaJeztah commented May 12, 2025

cc @vvoland @dmcgowan - looks like this relates to;

@thaJeztah thaJeztah added area/distribution Image Distribution status/2-code-review containerd-integration Issues and PRs related to containerd integration labels May 12, 2025
@thompson-shaun thompson-shaun requested a review from vvoland May 12, 2025 14:45
vvoland
vvoland reviewed May 13, 2025
View reviewed changes
Copy link
Contributor

@vvoland vvoland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good! I left a suggestion on the implementation though.

@henry118
Fix multiarch image push tag for containerd snapshotter
a7ef4a2 
Signed-off-by: Henry Wang <henwang@amazon.com>
@thaJeztah thaJeztah mentioned this pull request May 15, 2025
Closed
@thaJeztah thaJeztah requested a review from vvoland May 15, 2025 09:53
@thaJeztah
Copy link
Member

thaJeztah commented May 15, 2025

@vvoland @dmcgowan PR was updated; PTAL. Also if this PR is indeed addressing docker/cli#5874 or if I'm very off 🙈 😅

@thaJeztah thaJeztah added this to the 28.2.0 milestone May 15, 2025
@vvoland
Copy link
Contributor

vvoland commented May 15, 2025

Oh, thought I review it yesterday!

Yeah I think it might help, although initially I considered the cause for this issue to be different 🤔

vvoland
vvoland approved these changes May 15, 2025
View reviewed changes
Copy link
Contributor

@vvoland vvoland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Would appreciate a 2nd look from @dmcgowan though

@dmcgowan
Copy link
Member

dmcgowan commented May 16, 2025

Also if this PR is indeed addressing docker/cli#5874 or if I'm very off 🙈 😅

I'm not sure about that specific registry, but with this change Moby should no longer be pushing to the same manifest endpoint multiple times, solving any immutable tag issues or PUT request duplication.

@thaJeztah thaJeztah merged commit 94e8416 into moby:master May 19, 2025
153 checks passed
@vvoland vvoland added the kind/bugfix PR's that fix bugs label May 19, 2025
@henry118 henry118 deleted the pushtag branch May 22, 2025 04:25
@vdamery vdamery mentioned this pull request Jun 4, 2025
Closed
@psviderski psviderski mentioned this pull request Jun 13, 2025
Closed
@KavyaSree2610 KavyaSree2610 mentioned this pull request Jan 22, 2026
Open
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dmcgowan dmcgowan dmcgowan approved these changes

@vvoland vvoland vvoland approved these changes

Assignees

@vvoland vvoland

@austinvazquez austinvazquez

Labels

area/distribution Image Distribution containerd-integration Issues and PRs related to containerd integration impact/changelog kind/bugfix PR's that fix bugs status/2-code-review

Projects

None yet

Milestone

28.2.0

Development

Successfully merging this pull request may close these issues.

5 participants

@henry118 @thaJeztah @vvoland @dmcgowan @austinvazquez