Skip to content

Proposal: Change --publish=SPEC to allow binding to custom host port ranges#12927

Merged
thaJeztah merged 1 commit intomoby:masterfrom
lindenlab:custom-host-port-ranges
Aug 19, 2015
Merged

Proposal: Change --publish=SPEC to allow binding to custom host port ranges#12927
thaJeztah merged 1 commit intomoby:masterfrom
lindenlab:custom-host-port-ranges

Conversation

@dkjer
Copy link
Contributor

@dkjer dkjer commented May 1, 2015

This PR is similar to #12622, but is more narrow and focuses purely on the publish SPEC. This change would allow specifying a range for the hostPort of a publish SPEC (iff containerPort is not a range), which would override the default binding to ephemeral ports and instead bind within the specified hostPort range:

Example:

$ docker run -d -p 8000-9000:5000 training/webapp python app.py

This would bind port 5000 in the container to a randomly available port between 8000 and 9000 on the host.

@bru7us
Copy link

bru7us commented May 3, 2015

It looks like there are a few conflicts between this and #12622, I think the better solution would be some combination of both PRs (see: #10052 (comment)) - I just never got around to implementing this format.

A couple of questions for you:

  • What does the hostConfig look like on a container that is started with this feature?
  • What happens on restart of the container started this way? (try to keep the same allocated port, or always re-allocate from the range?)

I love what you've done with the port range keys in the global map. I started out doing something similar, but with multiple port maps so I came across an issue where you can have potentially overlapping maps that will conflict with each other. This is a much more elegant approach.

@dkjer
Copy link
Contributor Author

dkjer commented May 4, 2015

@bru7us

Here is an example session:

$ docker run -d -p 8000-9000:80 --name porttest nginx
$ docker port porttest
80/tcp -> 0.0.0.0:8003
$ docker inspect porttest
[{
  ...
    "HostConfig": {
  ...
        "PortBindings": {
            "80/tcp": [
                {
                    "HostIp": "",
                    "HostPort": "8000-9000"
                }
            ]
        },
  ...
    },
  ...
    "NetworkSettings": {
  ...
        "Ports": {
            "443/tcp": null,
            "80/tcp": [
                {
                    "HostIp": "0.0.0.0",
                    "HostPort": "8003"
                }
            ]
        }
  ...
    },
  ...
}
]
$ docker restart porttest
$ docker port porttest
80/tcp -> 0.0.0.0:8004

The restart does not attempt to keep the same port, but it does re-allocate from within the same requested port range.

@bru7us
Copy link

bru7us commented May 4, 2015

Thanks for the follow up - I'm struggling to understand the benefit of using this instead of just any ephemeral port (outside of the scope of the other PRs). If every restart auto-allocates a port - why not just use the ephemeral range?

I think my latest comment (#12622 (comment)) sums up the ultimate solution (a combination of your work, mine and that in #10052), do you have any feedback on that?

@dkjer
Copy link
Contributor Author

dkjer commented May 4, 2015

My use-case is that I'm attempting to use AWS Security Groups to differentiate between 'public' vs 'private' port ranges to allow external traffic. I need to be able to assign container ports in a more controlled manner than just using ephemeral ports.

I'll take a look at #12622 and provide some feedback

@LK4D4
Copy link
Contributor

LK4D4 commented May 18, 2015

@docker/core-maintainers for design review. I think it is smaller and less-breaking than #12622

@bru7us
Copy link

bru7us commented May 18, 2015

I think it is smaller and less-breaking than #12622

It's definitely smaller, but it's also different than #12622 as it doesn't provide persistence for dynamic allocations (which is the primary goal of that PR - I just absorbed the changes from this one into it for better use case coverage).

@dkjer
Copy link
Contributor Author

dkjer commented Jun 12, 2015

Rebased off master. Updated vendor/src/github.com/docker/libnetwork to match moby/libnetwork#285

@icecrime icecrime added the status/needs-attention Calls for a collective discussion during a review session label Jun 26, 2015
@icecrime
Copy link
Contributor

@mavenugo @mrjana @aboch Thoughts?

@dkjer
Copy link
Contributor Author

dkjer commented Jul 8, 2015

@mavenugo @mrjana @aboch @icecrime Any word on this?

@icecrime icecrime added status/2-code-review area/networking Networking and removed status/1-design-review status/needs-attention Calls for a collective discussion during a review session labels Jul 23, 2015
@icecrime
Copy link
Contributor

Group review session

Design LGTM assuming that the container doesn't persist the elected port (which was the reason that the alternative implementation got previously closed).

Ping @mrjana @mavenugo @aboch: please review!

@dkjer
Copy link
Contributor Author

dkjer commented Jul 29, 2015

@mrjana @mavenugo @aboch @icecrime : Rebased off master

@mavenugo
Copy link
Contributor

@dkjer have a comment / question for your PR in libnetwork moby/libnetwork#285. Can you please address that so that we can get this PR moving ?

@dkjer
Copy link
Contributor Author

dkjer commented Aug 12, 2015

@mavenugo Sure, apologies for the delay

clnperez pushed a commit to clnperez/libnetwork that referenced this pull request Aug 13, 2015
See moby/moby#12927 for docker portion.

Signed-off-by: Don Kjer <don.kjer@gmail.com>
mavenugo added a commit to mavenugo/docker that referenced this pull request Aug 14, 2015
Notable changes include :
- moby#285 : Fix required for moby#12927
- moby#283 : Code re-architecture/tech-debt in bridge driver
- Upgraded to latest Netlink library
- Fixed certain race-conditions

Signed-off-by: Madhu Venugopal <madhu@docker.com>
@mavenugo
Copy link
Contributor

@dkjer latest libnetwork with your patch is in docker master now. Can you please rebase to the latest and update this PR ?

@dkjer
Copy link
Contributor Author

dkjer commented Aug 14, 2015

@mavenugo Rebased

pkg/nat/nat.go Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bru7us
Copy link

bru7us commented Aug 14, 2015

@mavenugo, did you see moby/libnetwork#453 yet? If we can catch it before this PR gets merged we can easily maintain full backward compatibility in libnetwork with a minor update to this PR to include separate start port and specified port.

If this gets merged first, there will be a dependency on docker and libnetwork to be updated together to get persistence policies in.

@dkjer
Copy link
Contributor Author

dkjer commented Aug 14, 2015

@mavenugo Error string updated / commit squashed

pkg/nat/nat.go Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason why we are trying to ignore the error (other than reason mentioned here?).
I guess we could break this assumption with a simple Unit-Test.
Though it is not something we will see today, this assumption can lead to issues down the line.
Hence it would be better to not ignore the error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mavenugo I agree. This was just following the pattern given in Port.Int(), which ignores errors with the same assumption. I will add the error case

@aboch
Copy link
Contributor

aboch commented Aug 15, 2015

@dkjer Could you please cover this in current integration cli test for port specs: docker_cli_port_test.go
You can just integrate the existing test functions with few more cases where the host port range is specified (correctly and incorrectly) and verify it.

@dkjer
Copy link
Contributor Author

dkjer commented Aug 15, 2015

@mavenugo @aboch Addressed code review comments and added integration test. Squashed commit

Signed-off-by: Don Kjer <don.kjer@gmail.com>

Changing vendor/src/github.com/docker/libnetwork to match lindenlab/libnetwork custom-host-port-ranges-1.7 branch
@aboch
Copy link
Contributor

aboch commented Aug 15, 2015

LGTM

@mavenugo
Copy link
Contributor

@bru7us sorry not yet. we have start with design review of moby/libnetwork#453 and make sure the features align with all the upcoming features. We have to also look @ all the design review comments of #12622. I think it is okay to keep 2 separate.

@bru7us
Copy link

bru7us commented Aug 15, 2015

Ok, thanks for the response.

@mavenugo
Copy link
Contributor

@dkjer Thanks for addressing the comments.
The implementation LGTM and is inline with the use-case that was discussed in this PR during the design review.

But, I think we need some inputs from the docs team especially because the -p host-port-range:container-port-range behaves a bit different from the newly introduced -p host-port-range:container-port and also the persistence expectations are a bit different.

ping @moxiegirl @thaJeztah

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkjer Thank you for your contribution. This is great information and I think expanding it will help readers get this information more easily. Also, I'd like it if we could put this information in a table. The table is beyond your scope, if you are willing to add it, good. If not I can carry the work for you.

--> https://gist.github.com/moxiegirl/8fa6820db70a011a8414

Option Description
--expose=[] Expose a port or a range of ports from the container,without publishing it to your host
-P=false Publish all exposed ports to the host interfaces.
-p=[]

Publish a container᾿s port or a range of ports to the host. The value's format can be any of:

  • ip:hostPort:containerPort
  • ip::containerPort
  • hostPort:containerPort
  • containerPort

Use the ip:hostPort:containerPort or ip::containerPort formats if you want to be sure to bind a specific interface on the host side.

You can specify both the hostPort and containerPort as port ranges. When specifying ranges for both, the number of values in the ranges must match, for example:-p 2000-3000:2500-3500/tcp The total number of values are both 1000.

You can also specify a range for hostPort alone and a single value for the container:-p 1234-1236:2000/tcp In this case, the containerPort is mapped to a value somewhere within the given hostPort range. Use the docker port command to see the actual mapping.

--link="" Add link to another container (id:alias, name:alias, name, id)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@moxiegirl I won't have time for a couple days to update this; if you could carry the doc changes that would be excellent

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkjer shall I suggest a minor tweak to this Doc and get it merged ?
@moxiegirl can update the doc in a later commit with the table structure that she proposed ?

The proposed documentation is confusing

When specifying a range for hostPort only, the containerPort must not be a range.

Confusing and it is conflicting with the other option of providing both hostPort range and containerPort range.

In this case the container port is published somewhere within the specified hostPort range.

I would reword it as `In this case the container port is mapped to a host port somewhere within the specified hostPort range``

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mavenugo I'll pick up your change wording when I carry the docs.

@moxiegirl
Copy link
Contributor

@mavenugo I'll LGTM and carry the change after the merge if you are cool with it.

@thaJeztah
Copy link
Member

sgtm, agreed to update after merging so LGTM

@moxiegirl I'll merge this one, then I'll rebase #15675 when I'm home, does that work for you?

thaJeztah added a commit that referenced this pull request Aug 19, 2015
Proposal: Change --publish=SPEC to allow binding to custom host port ranges
@thaJeztah thaJeztah merged commit 59e49e1 into moby:master Aug 19, 2015
@dkjer
Copy link
Contributor Author

dkjer commented Aug 19, 2015

@moxiegirl Thanks for carrying the doc changes!

@thaJeztah
Copy link
Member

Thank you @dkjer for contributing, and being so patient!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants