Yay! Can't wait to play with this tomorrow. Thanks Guillaume.

On Fri, Jun 7, 2013 at 1:51 AM, Guillaume J. Charmes <
notifications@github.com> wrote:

• Implement the ADD with remote URL as soruce
• Allow to use docker build with remote URL
• Allow to use docker build with a git/github repository
• Minor bugfixes

---

You can merge this Pull Request by running

git pull https://github.com/dotcloud/docker builder_server-3

Or view, comment on, or merge it at:

#848
Commit Summary

• Allow remote url to be passed to the ADD instruction within the
  builder
• Fix issue EXPOSE override CMD within builder
• Move isUrl to utils.IsURL
• Allow to docker build URL
• Move the docker build URL form client to server, prepare for GIT
  support
• Implement build from git
• Update documentation for docker build

File Changes

• M api.go https://github.com/dotcloud/docker/pull/848/files#diff-0(79)
• M buildfile.gohttps://github.com/Improve Docker build #848/files#diff-1(72)
• M commands.gohttps://github.com/Improve Docker build #848/files#diff-2(33)
• M docs/sources/api/docker_remote_api.rsthttps://github.com/Improve Docker build #848/files#diff-3(1)
• M docs/sources/commandline/command/build.rsthttps://github.com/Improve Docker build #848/files#diff-4(4)
• M docs/sources/use/builder.rsthttps://github.com/Improve Docker build #848/files#diff-5(2)
• M utils/utils.gohttps://github.com/Improve Docker build #848/files#diff-6(22)

Patch Links:

• https://github.com/dotcloud/docker/pull/848.patch
• https://github.com/dotcloud/docker/pull/848.diff

This is really minor, but could you please add to that list support for more than one tab stop in a build file?

Does build: echo -e "FROM\tubuntu:precise" | docker build -
Doesn't build (errors 404): echo -e "FROM\t\tubuntu:precise" | docker build -

How is this supposed to be used with git?

I just tried docker build git://git@github.com:niclashoyer/somerepo.git but I got

2013/06/10 22:24:53 error: http: no such file

you can either use docker build git://github.com/niclashoyer/somerpo or docker build github.com/niclashoyer/somerepo.
The ssh url format is not supported.

I think we're going to deprecate the 1st version (with git:// scheme). Only the second version (no scheme) will be guaranteed to work. This is the same as golang packages.
—
@solomonstre
@getdocker

On Mon, Jun 10, 2013 at 1:38 PM, Guillaume J. Charmes
notifications@github.com wrote:

you can either use docker build git://github.com/niclashoyer/somerpo or docker build github.com/niclashoyer/somerepo.

The ssh url format is not supported.

Reply to this email directly or view it on GitHub:
#848 (comment)

Is it also possible to use private repositories with docker build?

@shykes, I inspired myself from 'git clone' in order to allow arbitrary git repository. If I remove git://, then we limit git to github. Is that OK?

I see what you mean. I was looking at 'go get' for inspiration. I don't
know how they detect git vs. plain http. Does it try each protocol in turn,
or does it have a hardcoded url-to-protocol mapping?

On Tue, Jun 11, 2013 at 11:15 AM, Guillaume J. Charmes <
notifications@github.com> wrote:

@shykes https://github.com/shykes, I inspired myself from 'git clone'
in order to allow arbitrary git repository. If I remove git://, then we
limit git to github. Is that OK?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/848#issuecomment-19281726
.

@niclashoyer Yes, you can use private repository with docker build. However, remote private repositories are not yet implemented and you could not share the Dockerfile if you do this.

@solomonstre: regexps: http://golang.org/src/cmd/go/vcs.go#L534, VCS lookup: http://golang.org/src/cmd/go/vcs.go#L55.
It is quite complicated for that few. Maybe we can start with github support only and implement more later?

@creack thanks for the info! The use cas I had in mind was a using specific Dockerfiles customer projects. At the moment I use a public Dockerfile that just sets up a lamp server. Then for each private project I added a Dockerfile to the private repository that installs some packages on top of the lamp server and adds the files related to the project.

Is it possible to leave the git:// syntax as a fallback for any git repository and just hand the url over to git clone <url>? That way private repos could also be cloned based on ssh authentication.

I thought a bit about it. I think if we want to support different VCS in the future we've got the the following options:

1. Use RegExp Magic

Use the code mentioned by @creack and maybe package it as a go module, so that not every other project out there has to reimplement this magic.

docker build github.com/niclashoyer/somerepo        # ~> git clone https://github.com...

2. Use an URI schema for each VCS

This way we would determine the VCS to use by looking at the prefix of the string given on command line.

docker build git://github.com/niclashoyer/somerepo    # ~> git clone https://github.com...
docker build svn://example.com/svnrepo                # ~> svn co https://example.com...

3. Use a Command Line Switch

Use a command line switch to determine the VCS and hand the argument over to the VCS command.

docker build -r git github.com/niclashoyer/somerepo           # ~> git clone https://github.com...
docker build -r git https://github.com/niclashoyer/somerepo   # ~> git clone https://github.com...
docker build -r git ssh://git@github.com/niclashoyer/somrepo  # ~> git clone ssh://git@github.c...
docker build -r svn example.com/somerepo                      # ~> svn co https://example.com...

There is an issue, not related to this change, but I think it should be solved by this PR.
docker build uses cache, to do so it compares ConfigFiles.
It can sometimes be a problem, I spent quite some time helping a user with this.
In short:

Let's say there is 4 RUN in your Dockerfile.
And the 3rd is apt-get...

If your /etc/resolv.conf in your host is empty, the build will fail on step 2, so the 2 first layouts will be created.
If you fix /etc/resolv.conf in your host and start the build again, it'll still fail because it will get the layouts 1 and 2 from cache => with etc/resolv.conf empty.

To solve this you have to manually remove the layers 1 and 2

@vieux this will be solved by #832

@vieux @creack this is caused by the fact that images embed their DNS configuration, right? That's a good example of breaking Separation of Concerns. The DNS setup should be re-computed at each run, and the image should have no way to influence it (just like host mounts).

@shykes It is already the case. Right now, there is no DNS instruction within the builder, so there is no way to use docker run -dns.
Images do not embed their DNS configuration, it is handle 100% by docker at each run.

First, just to clarify what I mean, try typing this on your machine:

docker build -t shykes/inject-dns:result - <<EOF
from shykes/inject-dns
run touch /whatever
cmd echo My dns is: && cat /etc/resolv.conf
EOF

Then type this:

docker run shykes/inject-dns:result

This is because there is a "Dns" field in Container.Config. So even if the UI isn't very practical, it's possible for an image to leak its DNS configuration to the host, and it shouldn't be possible.

@shykes if the source image has been configured with custom DNS, this is expected. You want to remove this feature?

Mostly I want to avoid the situation where a build with 'docker -dns FOO' results in an image with {"Dns":["foo"]}, if you know what I mean.

@metalivedev is that better or do you want me to develop a bit more?

@creack I'm merging this with #900, since they overlap a lot, and testing the result.

Boom! You're welcome :)

Ok, so to summarize, in this pull request we have:

• ADD of remote URLs
• docker build can use a remote text file over HTTP as Dockerfile
• docker build can fetch and build a remote git repository
• remote API 1.3: build context is streamed instead of buffering+multipart. Simpler and faster.
• ADD behavior is more logical, and well documented
• More regression tests
• It's easier to add build tests

From the documentation I (falsely) inferred that running,
ADD http://example.com/archive.tar.gz /opt/
would actually extract the archives contents to /opt, which would have been quite convenient.

Have you considered adding this functionality, or would this break peoples scripts?

/opt and /opt/ will do 2 different things. Did you check that neither does what you want?

@solomonstre
@docker

On Thu, Oct 17, 2013 at 2:10 AM, jottr notifications@github.com wrote:

From the documentation I (falsely) inferred, that running,
ADD http://example.com/archive.tar.gz /opt/"
would actually extract the contents to /opt.

Have you considered adding this functionality, or would this break peoples scripts?

Reply to this email directly or view it on GitHub:
#848 (comment)

Yes, I tried both forms, but neither does the trick.
Although, to my understanding it should be /opt/ when I would like to extract the contents to that directory.