Skip to content

Docker Registry - Fix Authorizer logic#3052

Closed
simonferquel wants to merge 3 commits intocontainerd:masterfrom
simonferquel:multi-scope-token
Closed

Docker Registry - Fix Authorizer logic#3052
simonferquel wants to merge 3 commits intocontainerd:masterfrom
simonferquel:multi-scope-token

Conversation

@simonferquel
Copy link
Copy Markdown

@simonferquel simonferquel commented Feb 28, 2019

This PR fixes two issues in dockerAuthorizer logic:

First, the scope deduplication was a bit too naive. This brings a scope-gramar aware deduplication logic, featuring:

  • smarter deduplication (repository:foo/bar:pull,push + repository:foo/bar:push,pull => repository:foo/bar:pull,push)
  • sort-based normalization (actions on a resources are always sorted
    alphabethically, returned string slice is always sorted
    alphabethically)
  • merging of scopes concerning the same resource (repository:foo/bar:pull + repository:foo/bar:push => repository:foo/bar:pull,push)

The Deduplication code comes with a suite of test cases checking this behavior.

Second, the token cache takes into account token expiration, and associated scopes. Previously, as soon as a token was present in cache for a given host, it was used unconditionally for subsequent requests to this host. In some cases, if the subsequent requests required a broader scope than the original one, it would fail, To fix that, the PR adds:

  • a fully tested token cache component
  • that evicts expired tokens
  • and that checks that a token satisfy the current requested scopes before using it

@codecov-io
Copy link
Copy Markdown

codecov-io commented Feb 28, 2019

Codecov Report

Merging #3052 into master will decrease coverage by 6.76%.
The diff coverage is 77.4%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3052      +/-   ##
==========================================
- Coverage   47.57%   40.81%   -6.77%     
==========================================
  Files          93       75      -18     
  Lines        8590     9989    +1399     
==========================================
- Hits         4087     4077      -10     
- Misses       3776     5343    +1567     
+ Partials      727      569     -158
Flag Coverage Δ
#linux ?
#windows 40.81% <77.4%> (?)
Impacted Files Coverage Δ
remotes/docker/tokencache.go 100% <100%> (ø)
remotes/docker/authorizer.go 66.82% <67.39%> (-3.6%) ⬇️
remotes/docker/scope.go 69.49% <70.37%> (-5.51%) ⬇️
snapshots/native/native.go 1.79% <0%> (-51.25%) ⬇️
archive/tar.go 16.99% <0%> (-33.86%) ⬇️
metadata/snapshot.go 21.53% <0%> (-33.23%) ⬇️
cio/io.go 1.52% <0%> (-25.75%) ⬇️
content/local/writer.go 56.86% <0%> (-7.34%) ⬇️
metadata/containers.go 47.97% <0%> (-6.62%) ⬇️
content/local/store.go 48.51% <0%> (-5.03%) ⬇️
... and 94 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ceba568...4a232ed. Read the comment docs.

@dmcgowan
Copy link
Copy Markdown
Member

The docs are a bit confusing, but the "specified multiple times if there is more than one scope" is part of the deprecated "GET" approach. Refer to https://docs.docker.com/registry/spec/auth/oauth/ for the POST approach which is designed to be compatible with the OAuth2 specification. containerd always tries the oauth method if credentials are provided first, then it will try the GET if that is not supported or the request is anonymous (OAuth is not designed for this kind of anonymous use).

@dmcgowan dmcgowan added this to the 1.3 milestone Mar 13, 2019
@simonferquel
Copy link
Copy Markdown
Author

From a previous discussion with people in the hub team, I think the doc is not aligned with what is actually working... because my manual tests with both oauth and basic auth fails if scopes are not split in multiple form entries:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should treat empty and nil map the same way, using len(other) instead.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

Change the token scope merging to be grammar-aware.
This features:
 - smarter deduplication (repository:foo/bar:pull,push+repository:foo/bar:push,pull -> repository:foo/bar:pull,push)
 - sort-based normalization (actions on a resources are always sorted
 alphabethically, returned string slice is always sorted
 alphabethically)
 - merging of scopes concerning the same resource (repository:foo/bar:pull+repository:foo/bar:push => repository:foo/bar:pull,push)

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
@simonferquel
Copy link
Copy Markdown
Author

@dmcgowan seems like you are write (at least on hub, I managed to mount blobs from one repo to the other with a single "scope" form value). I removed the incriminated commit

@simonferquel simonferquel changed the title Docker Registry - Fix multi-auth scopes logic Docker Registry - Fix Authorizer logic Mar 15, 2019
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
Also, extracted token caching logic for better testability

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
@estesp
Copy link
Copy Markdown
Member

estesp commented Jun 13, 2019

Given the merge of #3218, there are now conflicts and the scope/token code has significant changes. Can you validate if there are still deficiencies that need to be addressed?

@dmcgowan
Copy link
Copy Markdown
Member

No longer necessary after #3218

@dmcgowan dmcgowan closed this Jun 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants