Edit the manifest when pushing uncompressed data from c/storage#2273
Merged
mtrmac merged 5 commits intocontainers:mainfrom Feb 7, 2024
Merged
Conversation
Collaborator
Author
mtrmac
added a commit
to mtrmac/buildah
that referenced
this pull request
Feb 1, 2024
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
mtrmac
added a commit
to mtrmac/skopeo
that referenced
this pull request
Feb 2, 2024
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
mtrmac
added a commit
to mtrmac/skopeo
that referenced
this pull request
Feb 2, 2024
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
mtrmac
added a commit
to mtrmac/skopeo
that referenced
this pull request
Feb 3, 2024
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
mtrmac
added a commit
to mtrmac/skopeo
that referenced
this pull request
Feb 3, 2024
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Collaborator
Author
|
I have manually tested that in Podman with containers/podman#21461 , this fixes the original reproducer of #2182. Also, Skopeo’s tests demonstrate the same behavior — and rely on it, which is why they are failing here. containers/skopeo#2213 will update Skopeo. |
b31f2b0 to
f02a40f
Compare
mtrmac
added a commit
to mtrmac/skopeo
that referenced
this pull request
Feb 5, 2024
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
mtrmac
added a commit
to mtrmac/buildah
that referenced
this pull request
Feb 5, 2024
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Collaborator
Author
|
Ready for review. Downstream tests are mostly passing. The Skopeo test failures here expose the previous incorrect behavior; containers/skopeo#2213 contains test updates, and passes. Cc: @giuseppe |
mtrmac
added a commit
to mtrmac/buildah
that referenced
this pull request
Feb 6, 2024
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
mtrmac
added a commit
to mtrmac/buildah
that referenced
this pull request
Feb 7, 2024
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
mtrmac
added a commit
to mtrmac/libpod
that referenced
this pull request
Feb 7, 2024
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Member
|
Tests are failing? |
Collaborator
Author
|
Yes, as documented in detail above. |
Member
|
LGTM |
In particular, it is OK to use CompressionOperation: Decompress on already-decompressed layers. TestUpdatedMIMEType already ensures this works correctly. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Make it clearer that it is intended for manifest edits, and is not indicative of what we actually did. Should not change behavior. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... describing in more granularity what we are doing. Use it in recordValidatedDigestData, adding a case where we can record that a layer is uncompressed when we determine that while copying it without changes. Adding this separate .operation field will also allow us to set .uploadedOperation to Decompress in the bpcOpPreserveUncompressed case, without changing what recordValidatedDigestData does. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
…d on upload
When copying an uncompressed layer without modification, arrange
types.BlobInfo.CompressionOperation to be set to types.Decompress,
so that we edit the manifest to use an uncompressed MIME type
(and not, e.g. a zstd:chunked MIME type which might not be representable
in the destination at all).
This is all still gated by the
> if srcInfosUpdated || layerDigestsDiffer(srcInfos, destInfos) {
condition in copyLayers, so we don't do these manifest updates
frivolously.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
…d on reuse Signed-off-by: Miloslav Trmač <mitr@redhat.com>
f02a40f to
278b324
Compare
mtrmac
added a commit
to mtrmac/skopeo
that referenced
this pull request
Feb 7, 2024
This is necessary so that c/image tests can pass. Signed-off-by: Miloslav Trmač <mitr@redhat.com>
mtrmac
added a commit
to containers/skopeo
that referenced
this pull request
Feb 7, 2024
vexxhost-ci bot
pushed a commit
to vexxhost/atmosphere
that referenced
this pull request
Apr 21, 2024
…1114) This PR contains the following updates: Package Change Age Adoption Passing Confidence github.com/containers/image/v5 v5.29.2 -> v5.30.0 WarningSome dependencies could not be looked up. Check the warning logs for more information. Release Notes containers/image (github.com/containers/image/v5) v5.30.0 Compare Source What's Changed A fair number of improvements when working with zstd and zstd:chunked-compressed images. Note that make install now installs policy.json and registries.d/default.yaml. Refuse compression to zstd when using schema1 by @mtrmac in containers/image#2196 Don't expose local account details in oci-archive tar files by @mtrmac in containers/image#2202 Trigger a conversion to OCI when compressing to Zstd by @mtrmac in containers/image#2204 Add buildtags to avoid fulcio and rekor dependencies by @siretart in containers/image#2180 copy: do not fail if digest mismatches by @giuseppe in containers/image#1980 Moving policy.json and default.yaml from containers/skopeo by @rahilarious in containers/image#2215 Embrace codespell: config, workflow (to alert when new typos added) and get typos fixed by @yarikoptic in containers/image#2214 Fix raspberry pi zero cpu variant recognition by @lstolcman in containers/image#2086 storage: validate images converted to zstd:chunked by @giuseppe in containers/image#2243 Make blob reuse choices manifest-format-sensitive, and allow conversions when writing to format-agnostic transports by @mtrmac in containers/image#2213 Edit the manifest when pushing uncompressed data from c/storage by @mtrmac in containers/image#2273 Random storage-related cleanups by @mtrmac in containers/image#2287 Improve storage transport documentation, primarily about locking by @mtrmac in containers/image#2291 Fix c/storage destination with partial pulls by @mtrmac in containers/image#2288 Fix manifest updates when we match a layer by TOC digest by @mtrmac in containers/image#2294 Cleanly fail when trying to obtain a DiffID of a non-OCI image by @mtrmac in containers/image#2295 Beautify TOC-related parts of storageImageSource by @mtrmac in containers/image#2296 storage: use the new ApplyStagedLayer interface by @giuseppe in containers/image#2301 Also annotate image instances using zstd:chunked as using zstd by @mtrmac in containers/image#2302 Support editing ArtifactType, preserve it in lists by @nalind in containers/image#2304 Provide data to correctly report throughput on partial pulls by @mtrmac in containers/image#2308 Add validation error to digesting reader by @saschagrunert in containers/image#2312 Fix handling of errors when fetching layers by URLs by @mtrmac in containers/image#2310 Improve handling of zstd vs. zstd:chunked matching by @mtrmac in containers/image#2317 New Contributors @rahilarious made their first contribution in containers/image#2215 @yarikoptic made their first contribution in containers/image#2214 @lstolcman made their first contribution in containers/image#2086 @bainsy88 made their first contribution in containers/image#2260 Full Changelog: containers/image@v5.29.2...v5.30.0 Configuration 📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied. ♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 Ignore: Close this PR and you won't be reminded about this update again. If you want to rebase/retry this PR, check this box This PR has been generated by Mend Renovate. View repository job log here.
doanac
pushed a commit
to lmp-mirrors/meta-virtualization
that referenced
this pull request
Jul 16, 2024
Bumping skopeo to version v1.15.1-8-g80626759, which comprises the following commits:
0002d9ca [release-1.15] Bump to v1.15.2
260f444b [release-1.15] Fixes an interoperability issue while listing tags
cda43af4 [skip-ci] Packit: enable c10s downstream sync
7e81106f Refer to registry.k8s.io instead of k8s.gcr.io
e2ea4269 [release-1.15] Bump to v1.15.1
5267baab [release-1.15] Freeze the fedora-minimal image reference
cd5f7090 [release-1.15] CVE-2024-3727
99e3c448 Bump to v1.15.0
6d6d5b7b chore(deps): update module google.golang.org/protobuf to v1.33.0 [security]
78ddfd9d fix(deps): update module github.com/containers/ocicrypt to v1.1.10
e46c71c3 chore(deps): update module gopkg.in/go-jose/go-jose.v2 to v2.6.3 [security]
549fc862 chore(deps): update module github.com/go-jose/go-jose/v3 to v3.0.3 [security]
cca855ac [CI:DOCS] Migrate skopeo container image
2ffc4ec3 fix(deps): update module github.com/containers/common to v0.58.0
086701bd fix(deps): update module github.com/containers/image/v5 to v5.30.0
96443048 fix(deps): update module github.com/containers/storage to v1.53.0
a80f634f fix(deps): update module golang.org/x/term to v0.18.0
4d80bf8c fix(deps): update github.com/containers/image/v5 digest to faa4f4f
2db09313 Move a go.mod dependency entry to the right section
95f6eac4 Avoid a warning by gopls / VSCode
dbf465d6 fix typo
55abe69d update docs based on review
439dd5f4 docs: make sure that CGO_ENABLED='1'
f0b07848 [CI:DOCS] Update dependency golangci/golangci-lint to v1.56.2
8b6774b4 fix(deps): update module github.com/opencontainers/image-spec to v1.1.0
e294a380 [CI:DOCS] Update dependency golangci/golangci-lint to v1.56.1
c3e2b202 Update c/image after containers/image#2273
8f2a7c65 Also change the MIME type in manifest when decompressing a layer
812a02af Replace decompress-dirs.sh with Go code
09f282e4 Split decompressDir from runDecompressDirs
9fd98112 [CI:DOCS] Update dependency golangci/golangci-lint to v1.56.0
122bf968 fix(deps): update module github.com/containers/common to v0.57.4
149fec0a fix(deps): update golang.org/x/exp digest to 1b97071
d77f7153 fix(deps): update module github.com/containers/common to v0.57.3
c941d813 Update github.com/containers/image/v5 to post-v5.29.2 main
b412ed6e Run codespell on code
58ff9fdb fix(deps): update module github.com/containers/storage to v1.52.0
177d4adb add kubasobon:semver changes
4baf4fe9 fix(deps): update module github.com/opencontainers/image-spec to v1.1.0-rc6
e32dfefb Bump to v1.15.0-dev
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #2182.
At this point completely untested.