Move the tar-split digest into the TOC#1902
Move the tar-split digest into the TOC#1902openshift-merge-bot[bot] merged 5 commits intocontainers:mainfrom
Conversation
pkg/chunked/internal/compression.go
Outdated
| ManifestChecksumKey = "io.github.containers.zstd-chunked.manifest-checksum" | ||
| ManifestInfoKey = "io.github.containers.zstd-chunked.manifest-position" | ||
| TarSplitChecksumKey = "io.github.containers.zstd-chunked.tarsplit-checksum" | ||
| TarSplitChecksumKey = "io.github.containers.zstd-chunked.tarsplit-checksum" // Deprecated: Use the TOC.TarSplitDigest field instead, this one is not authenticated by ManifestChecksumKey. |
There was a problem hiding this comment.
I think it is fine if we just break now, and not have to worry about it
There was a problem hiding this comment.
The alternative would be to ignore the tar-split. That would make the current images consumable, but they would IIRC break on re-pushing the layers.
a2ea4cc to
710533e
Compare
ff1e003 to
791869f
Compare
|
#1888 must have a solution. This is one possible solution; it also breaks all currently-existing zstd:chunked images, which might be fine or very bad, I don’t know which one it is. |
It is fine to break these images now. The tarsplit data must not be mandatory. If the tarsplit data is missing, we should just skip fetching it (and deal with errors to recreate the original image later) |
This is a microptimization, we call strings.ToLower only once, but more importantly it will make it easier to add more fields. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Other TOC formats don't fill the data in. For now, this only increases memory usage, but we will need the data soon. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... so that we can uniquely identify partially-pulled layers by the TOC digest. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Include more details in the returned error text. Don't continue in tests when we fail to obtain a TOC. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
|
With this PR:
Alternatively, in the last case, we could accept the layer and ignore the tar-split data. If we do that, the images will continue to pull fine; but previously they also pushed fine and that would now break. Marking ready for review now. |
could we just drop it from the image generation and always ignore it? So we don't have to worry about it in future |
I’m not sure which part you mean should be dropped. We need:
We can’t drop the tar-split offset. Do you mean to stop adding the |
yes, to stop doing that and use only the information in the TOC |
We are already not reading it, so simplify the code. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
|
Done. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: giuseppe, mtrmac The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/lgtm |
A minimal prototype for a fix of #1888 .
Note that this would immediately break pulling all currently-existing layers.
Also this probably increases memory requirements because the TOC exists as individual Go values longer.
Cc: @giuseppe