Skip to content

[release/1.7 backport] import/export: Support references to missing content #9600

Merged
mxpv merged 5 commits intocontainerd:release/1.7from
vvoland:archive-sparse-1.7
Jan 4, 2024
Merged

[release/1.7 backport] import/export: Support references to missing content #9600
mxpv merged 5 commits intocontainerd:release/1.7from
vvoland:archive-sparse-1.7

Conversation

@vvoland
Copy link
Contributor

@vvoland vvoland commented Jan 3, 2024

Allow importing/exporting archives which doesn't have all the referenced blobs.
This allows to export/import an image with only some of the platforms available locally while still persisting the full index.

The blobs directory MAY be missing referenced blobs, in which case the missing blobs SHOULD be fulfilled by an external blob store.

https://github.com/opencontainers/image-spec/blob/v1.0/image-layout.md#blobs

neersighted and others added 3 commits January 3, 2024 16:07
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 377c5ca)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Allow importing/exporting archives which doesn't have all the referenced
blobs. This allows to export/import an image with only some of the
platforms available locally while still persisting the full index.

> The blobs directory MAY be missing referenced blobs, in which case the missing blobs SHOULD be fulfilled by an external blob store.

https://github.com/opencontainers/image-spec/blob/v1.0/image-layout.md#blobs

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 61a7c49)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Persist manifest/indexes distribution source labels as annotations in
the index.json. This could allow the importer to fetch the missing blobs
from the external repository.

These can't really be persisted directly in blob descriptors because
that would alter the digests.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit b9af453)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
@k8s-ci-robot
Copy link

Hi @vvoland. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@vvoland vvoland force-pushed the archive-sparse-1.7 branch 2 times, most recently from 6578029 to c8a5c13 Compare January 3, 2024 15:34
@thaJeztah
Copy link
Member

@vvoland looks like something is failing in this branch; can you also check if the "cherry-picked from" commit-id's match the one from the PR against main?

Also refactor tests to use the t.Run and run each test concurrently in a
separate namespace.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 0d3c3ef)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Don't make the `Export` signature more restrictive in a cherry-pick.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
@vvoland vvoland force-pushed the archive-sparse-1.7 branch from 39da4bc to 6089b05 Compare January 4, 2024 11:16
@vvoland vvoland marked this pull request as ready for review January 4, 2024 12:21
Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoop! looks like it's green now 🎉

LGTM

@thaJeztah
Copy link
Member

/cc @laurazard @cpuguy83

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants