fix(pipelines): DockerCredential.dockerHub() silently fails auth#18313
Merged
mergify[bot] merged 3 commits intomasterfrom Jan 10, 2022
Merged
fix(pipelines): DockerCredential.dockerHub() silently fails auth#18313mergify[bot] merged 3 commits intomasterfrom
DockerCredential.dockerHub() silently fails auth#18313mergify[bot] merged 3 commits intomasterfrom
Conversation
rix0rrr
approved these changes
Jan 10, 2022
Contributor
rix0rrr
left a comment
There was a problem hiding this comment.
I like the new code, reads a lot more readably to me! Thanks
Contributor
|
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
Collaborator
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
Contributor
|
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
TikiTDO
pushed a commit
to TikiTDO/aws-cdk
that referenced
this pull request
Feb 21, 2022
…ws#18313) ### Problem: `DockerCredential.dockerHub()` silently failed to authenticate users, resulting in unexpected and intermittent throttling due to docker's policy for unauthenticated users. ### Reason: `.dockerHub()` added `index.docker.io` to the domain credentials, but the actual docker command [authenticated](https://github.com/moby/moby/blob/1e71c6cffedb79e3def696652753ea43cdc47b99/registry/config.go#L35) with `https://index.docker.io/v1/` which it was unable to find as a domain credential, thus failing to trigger `docker-credential-cdk-assets` during the `docker --config build` call. Furthermore, the credential `DockerCredential.customRegistry('https://index.docker.io/v1/', secret)` alone does not work. This would successfully trigger `docker-credential-cdk-assets` but fail to authenticate because of how `cdk-assets` handles credential lookup. The command strips the endpoint into just a hostname so in this case we try `fetchDockerLoginCredentials(awsClient, config, 'index.docker.io')` which fails: https://github.com/aws/aws-cdk/blob/4fb0309e3b93be276ab3e2d510ffc2ce35823dcd/packages/cdk-assets/bin/docker-credential-cdk-assets.ts#L32-L38 So the workaround for this bug was to specify both domains as credentials, each to satisfy a separate step of the process: ```ts dockerCredentials: [ pipelines.DockerCredential.dockerHub(secret), pipelines.DockerCredential.customRegistry('https://index.docker.io/v1/', secret), ], ``` ### Solution: This PR introduces two separate changes to address both problems. First, we change the hardcoded domain in `DockerCredential.dockerHub()` to be `https://index.docker.io/v1/`. This allows us to successfully trigger `docker-credential-cdk-assets` when the `docker --config build` command is called. Next, to make sure the credential lookup succeeds, we check for both the complete endpoint and the domain name. In this case, we will check for both `https://index.docker.io/v1/` as well as `index.docker.io`. Since `https://index.docker.io/v1/` exists in the credentials helper, authentication will succeed. Why do we still check for the domain `index.docker.io`? I don't know how custom registries or ecr works in this context and believe it to be beyond the scope of the PR. It's possible that they require the domain only for lookup. ### Testing: The change to credential lookups is unit tested in `docker-credentials.test.ts`. I confirmed that the change to `DockerCredential.dockerHub()` is successful by configuring a mock `cdk-docker-creds.json` file and successfully `cdk deploy`ing a docker image that depends on a private repository. This isn't a common use case but ensures that failure to authenticate results in failure every time. Thanks @james-mathiesen for the suggestion. ### Contributors: Thanks to @nohack for the code in `cdk-assets`. Fixes aws#15737. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem:
DockerCredential.dockerHub()silently failed to authenticate users, resulting inunexpected and intermittent throttling due to docker's policy for unauthenticated users.
Reason:
.dockerHub()addedindex.docker.ioto the domain credentials, but the actual dockercommand authenticated with
https://index.docker.io/v1/which it was unable to findas a domain credential, thus failing to trigger
docker-credential-cdk-assetsduring the
docker --config buildcall.Furthermore, the credential
DockerCredential.customRegistry('https://index.docker.io/v1/', secret)alone does not work. This would successfully trigger
docker-credential-cdk-assetsbut fail to authenticate because of how
cdk-assetshandles credential lookup.The command strips the endpoint into just a hostname so in this case we try
fetchDockerLoginCredentials(awsClient, config, 'index.docker.io')which fails:aws-cdk/packages/cdk-assets/bin/docker-credential-cdk-assets.ts
Lines 32 to 38 in 4fb0309
So the workaround for this bug was to specify both domains as credentials, each to
satisfy a separate step of the process:
Solution:
This PR introduces two separate changes to address both problems. First, we change
the hardcoded domain in
DockerCredential.dockerHub()to behttps://index.docker.io/v1/.This allows us to successfully trigger
docker-credential-cdk-assetswhen thedocker --config buildcommand is called.Next, to make sure the credential lookup succeeds, we check for both the complete
endpoint and the domain name. In this case, we will check for both
https://index.docker.io/v1/as well as
index.docker.io. Sincehttps://index.docker.io/v1/exists in the credentials helper,authentication will succeed.
Why do we still check for the domain
index.docker.io? I don't know how custom registries orecr works in this context and believe it to be beyond the scope of the PR. It's possible that they
require the domain only for lookup.
Testing:
The change to credential lookups is unit tested in
docker-credentials.test.ts. I confirmed thatthe change to
DockerCredential.dockerHub()is successful by configuring a mockcdk-docker-creds.jsonfile and successfullycdk deploying a docker image that depends ona private repository. This isn't a common use case but ensures that failure to authenticate
results in failure every time. Thanks @james-mathiesen for the suggestion.
Contributors:
Thanks to @nohack for the code in
cdk-assets.Fixes #15737.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license