Skip to content

Compute manifest metadata when not provided#3245

Merged
dmcgowan merged 1 commit intocontainerd:masterfrom
msg555:master
May 7, 2019
Merged

Compute manifest metadata when not provided#3245
dmcgowan merged 1 commit intocontainerd:masterfrom
msg555:master

Conversation

@msg555
Copy link
Contributor

@msg555 msg555 commented Apr 27, 2019

Adds a fallback to manually compute the manifest digest when it is not presented as the Docker-Content-Digest header by the registry server. This is to address #3238.

@msg555
Copy link
Contributor Author

msg555 commented Apr 27, 2019

I think the digest calculation might depend on the content type of the manifest. Probably only want to do special processing on the signed content types rather than always. I think the code I have now works just for MediaTypeDockerSchema1Manifest. The awkward thing is the registry I was testing against registry.access.redhat.com seems to give text/plain as the Content-Type.

@msg555 msg555 force-pushed the master branch 3 times, most recently from 4abc424 to 4317ad0 Compare April 28, 2019 04:05
@msg555
Copy link
Contributor Author

msg555 commented Apr 28, 2019

This change makes it so that

  1. When Docker-Content-Digest is not present it computes the digest of the manifest
  2. Implements the preprocessing required to do the digest calculation on schema1
  3. Defaults to schema1 if the content type is not understood

@msg555
Copy link
Contributor Author

msg555 commented Apr 28, 2019

Will probably need to wait for #3249 before this will be ready and passing CI

@codecov-io
Copy link

codecov-io commented Apr 28, 2019

Codecov Report

Merging #3245 into master will decrease coverage by 0.09%.
The diff coverage is 17.39%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #3245     +/-   ##
=========================================
- Coverage   44.63%   44.54%   -0.1%     
=========================================
  Files         113      113             
  Lines       12164    12198     +34     
=========================================
+ Hits         5430     5434      +4     
- Misses       5899     5930     +31     
+ Partials      835      834      -1
Flag Coverage Δ
#linux 48.53% <18.91%> (-0.13%) ⬇️
#windows 39.78% <17.39%> (-0.1%) ⬇️
Impacted Files Coverage Δ
remotes/docker/resolver.go 54.48% <17.39%> (-5.68%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a17c809...ee902af. Read the comment docs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

V1 manifest should not be the default, rather check for the specific anomalous conditions and add a comment why that condition is there. The normal format would be stating the registry which this block is correcting for and the date at which it was last tested to still have that issue. So in this case, just check for the text/plain.

Also this block can be moved higher so that the digest calculation can be done inside the other block you added, rather than splitting the logic across 2 blocks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright made that change.

As for the two block thing I was originally just trying to avoid duplicating the media type parsing code. I've just moved it to a separate function now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libtrust doesn't meet our standards for vendoring as it is deprecated and the library is not split apart to just import packages we need.

Look at the github.com/containerd/containerd/remotes/docker/schema1 package, that has pulled out just the logic which is needed. It is ok to create a function there which just takes a schema1 blob and returns a digest for it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool thanks, added a utility method there that does what I need.

@msg555 msg555 changed the title Compute Docker-Content-Digest when not provided Compute manifest metadata when not provided May 1, 2019
@msg555 msg555 force-pushed the master branch 4 times, most recently from 9113232 to a4a0145 Compare May 1, 2019 04:08
This closes containerd#3238

Signed-off-by: msg555 <msg555@gmail.com>
@estesp
Copy link
Member

estesp commented May 1, 2019

Seems Windows CI is hitting #2835 a lot these days; restarted

Copy link
Member

@estesp estesp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; assigning to @dmcgowan to verify his review comments are resolved. Thanks.

Copy link
Member

@dmcgowan dmcgowan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dmcgowan
Copy link
Member

@msg555 as this was your first contribution to containerd, can you please provide your real name here for use in our release notes. Thanks!

@msg555
Copy link
Contributor Author

msg555 commented Jul 26, 2019

Mark Gordon

@artch
Copy link

artch commented Aug 28, 2019

@dmcgowan Is it possible to include this to 1.2.x branch? Recently released GitHub Package Registry doesn't provide Docker-Content-Digest, and stable 1.2.x cannot be used with it.

@artch artch mentioned this pull request Aug 28, 2019
@thaJeztah
Copy link
Member

@artch I can open a backport for consideration by the maintainers; there's another ticket tracking issues with the GitHub Package Registry here: #3291 (so there may still be issues remaining)

@thaJeztah
Copy link
Member

Opened #3591 for consideration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants