Skip to content

feat(assets): support networking mode for DockerImageAsset#18114

Merged
mergify[bot] merged 9 commits intoaws:masterfrom
kirintwn:feat-docker-assets-network-mode
Feb 7, 2022
Merged

feat(assets): support networking mode for DockerImageAsset#18114
mergify[bot] merged 9 commits intoaws:masterfrom
kirintwn:feat-docker-assets-network-mode

Conversation

@kirintwn
Copy link
Copy Markdown
Contributor

@kirintwn kirintwn commented Dec 21, 2021

As we are not allowed to specify networking mode for DockerImageAsset, users deploying cdk on containerized environment like Kubernetes will not be able to bundle assets without the build option --network host.

With this support, we are allowed to:

  • bundle image assets on a specific networking mode with the new networkMode property of DockerImageAsset from aws-ecr-assets.
  • bundle DockerImageFunction from aws-lambda on a specific networking mode.
  • bundle container images for AWS Fargate from on a specific networking mode.

Close #15516.


The possible values of --network

According to Docker CLI, the default value for --network will be default if omitted.

$ docker build --help
Usage:  docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile

Options:
...
--network string          Set the networking mode for the RUN instructions during build (default "default")
...

According to the Docker Official Docs- API 1.25:

supported standard values are: bridge, host, none, and container:<name|id>. Any other value is taken as a custom network's name to which this container should connect to.

But according to Source Code - docker/engine BuildKit, the value bridge is not accepted by BuildKit & should use the value default instead.

Therefore, the static values for NetworkMode are default, host & none, with 2 static functions NetworkMode.fromContainer() to construct a container:<name|id> & NetworkMode.custom() to construct a custom networking mode.

$ DOCKER_BUILDKIT=1 docker build --network=bridge .
Error response from daemon: network mode "bridge" not supported by buildkit

References:


Builder experience with aws-ecr-assets

Specify networkMode with DEFAULT or HOST for docker image assets

new assets.DockerImageAsset(stack, 'DockerImage', {
  directory: path.join(__dirname, 'demo-image'),
  networkMode: NetworkMode.HOST,
});

Builder experience with aws-ecs

Specify networkMode with DEFAULT or HOST for container image

taskDefinition.addContainer('web', {
  image: ecs.ContainerImage.fromAsset(path.join(__dirname, '../demo-image'), {
    networkMode: NetworkMode.DEFAULT,
  }),
  portMappings: [{
    containerPort: 8000,
  }],
});

Builder experience with aws-lambda

Specify networkMode with DEFAULT or HOST from docker image assets

new DockerImageFunction(this, 'MyLambda', {
  code: DockerImageCode.fromImageAsset(path.join(__dirname, 'docker-arm64-handler'), {
    networkMode: NetworkMode.DEFAULT,
  }),
});

@gitpod-io
Copy link
Copy Markdown

gitpod-io bot commented Dec 21, 2021

@github-actions github-actions bot added the @aws-cdk/assets Related to the @aws-cdk/assets package label Dec 21, 2021
eladb
eladb previously approved these changes Jan 25, 2022
Copy link
Copy Markdown
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mergify mergify bot dismissed eladb’s stale review January 25, 2022 10:12

Pull request has been modified.

eladb
eladb previously approved these changes Feb 2, 2022
@mergify mergify bot dismissed eladb’s stale review February 2, 2022 08:36

Pull request has been modified.

eladb
eladb previously approved these changes Feb 2, 2022
@mergify mergify bot dismissed eladb’s stale review February 2, 2022 08:40

Pull request has been modified.

eladb
eladb previously approved these changes Feb 7, 2022
@mergify mergify bot dismissed eladb’s stale review February 7, 2022 10:40

Pull request has been modified.

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Feb 7, 2022

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit a7b39f5 into aws:master Feb 7, 2022
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Feb 7, 2022

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Copy Markdown
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject89A8053A-LhjRyN9kxr8o
  • Commit ID: 979e5e1
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mergify bot pushed a commit that referenced this pull request Feb 10, 2022
…8905)

A recent PR, #18114, updated the cloud assembly schema, but parts of the descriptions were missing. Re-running `yarn update-schema` to get them back in sync. I am skipping actually bumping the version of the cloud assembly schema here, because the API is not changing. 


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
moelasmar pushed a commit to moelasmar/aws-cdk that referenced this pull request Feb 15, 2022
…s#18905)

A recent PR, aws#18114, updated the cloud assembly schema, but parts of the descriptions were missing. Re-running `yarn update-schema` to get them back in sync. I am skipping actually bumping the version of the cloud assembly schema here, because the API is not changing. 


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
TikiTDO pushed a commit to TikiTDO/aws-cdk that referenced this pull request Feb 21, 2022
As we are not allowed to specify networking mode for DockerImageAsset, users deploying cdk on containerized environment like Kubernetes will not be able to bundle assets without the build option `--network host`.

With this support, we are allowed to:
* [x] bundle image assets on a specific networking mode with the new `networkMode` property of `DockerImageAsset` from `aws-ecr-assets`.
* [x] bundle DockerImageFunction from aws-lambda on a specific networking mode.
* [x] bundle container images for AWS Fargate from on a specific networking mode.

Close aws#15516.

---

## The possible values of `--network`

According to Docker CLI, the default value for `--network` will be `default` if omitted.
```shell
$ docker build --help
Usage:  docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile

Options:
...
--network string          Set the networking mode for the RUN instructions during build (default "default")
...
```


According to the [Docker Official Docs- API 1.25](https://docs.docker.com/engine/api/v1.25/#operation/ImageBuild):
> supported standard values are: `bridge`, `host`, `none`, and `container:<name|id>`. Any other value is taken as a custom network's name to which this container should connect to.

But according to [Source Code - docker/engine BuildKit](https://github.com/docker/engine/blob/8955d8da8951695a98eb7e15bead19d402c6eb27/builder/builder-next/builder.go#L308-L314), the value `bridge` is not accepted by BuildKit & should use the value `default` instead.

Therefore, the static values for `NetworkMode` are `default`, `host` & `none`, with 2 static functions `NetworkMode.fromContainer()` to construct a `container:<name|id>` & `NetworkMode.custom()` to construct a custom networking mode.

```
$ DOCKER_BUILDKIT=1 docker build --network=bridge .
Error response from daemon: network mode "bridge" not supported by buildkit
```

References:
* [Docker Official Docs- API 1.25](https://docs.docker.com/engine/api/v1.25/#operation/ImageBuild)
* [Docker Official Docs - Use the default bridge network](https://docs.docker.com/network/bridge/#use-the-default-bridge-network)
* [Source Code - docker/engine BuildKit](https://github.com/docker/engine/blob/8955d8da8951695a98eb7e15bead19d402c6eb27/builder/builder-next/builder.go#L308-L314)


---
## Builder experience with `aws-ecr-assets`

Specify `networkMode` with `DEFAULT` or `HOST` for docker image assets
```ts
new assets.DockerImageAsset(stack, 'DockerImage', {
  directory: path.join(__dirname, 'demo-image'),
  networkMode: NetworkMode.HOST,
});
```

## Builder experience with `aws-ecs`

Specify `networkMode` with `DEFAULT` or `HOST` for container image
```ts
taskDefinition.addContainer('web', {
  image: ecs.ContainerImage.fromAsset(path.join(__dirname, '../demo-image'), {
    networkMode: NetworkMode.DEFAULT,
  }),
  portMappings: [{
    containerPort: 8000,
  }],
});
```

## Builder experience with `aws-lambda`

Specify `networkMode` with `DEFAULT` or `HOST` from docker image assets
```ts
new DockerImageFunction(this, 'MyLambda', {
  code: DockerImageCode.fromImageAsset(path.join(__dirname, 'docker-arm64-handler'), {
    networkMode: NetworkMode.DEFAULT,
  }),
});
```
TikiTDO pushed a commit to TikiTDO/aws-cdk that referenced this pull request Feb 21, 2022
…s#18905)

A recent PR, aws#18114, updated the cloud assembly schema, but parts of the descriptions were missing. Re-running `yarn update-schema` to get them back in sync. I am skipping actually bumping the version of the cloud assembly schema here, because the API is not changing. 


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

@aws-cdk/assets Related to the @aws-cdk/assets package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

(aws-ecr-assets): cannot build image correctly without docker option --network host

3 participants