WIP: Use dockerswarm/dind image instead of doing docker-in-docker ourselves by aanand · Pull Request #1783 · docker/compose

aanand · 2015-07-29T15:19:59Z

This simplifies the test suite, removing a layer of nesting and allowing us to abstract away the philosophical terror of Docker-in-Docker. Unfortunately I get intermittent connection errors:

$ script/test

(...build output...)

Running lint checks
Running tests against Docker 1.7.1
++ docker run -d --privileged --volume=/var/lib/docker --expose=2375 dockerswarm/dind:1.7.1 docker -d -H tcp://0.0.0.0:2375
+ daemon_container_id=ebebf491e47a54e41b9221d072720a1cfc7f3c892c91079981e19f695c12c6b7
+ docker run --rm --link=ebebf491e47a54e41b9221d072720a1cfc7f3c892c91079981e19f695c12c6b7:docker --env=DOCKER_HOST=tcp://docker:2375 --entrypoint=nosetests docker-compose:2759ab5
.........stty: standard input: Inappropriate ioctl for device
.stty: standard input: Inappropriate ioctl for device
.stty: standard input: Inappropriate ioctl for device
........................................................................................................................E.........E.........................................................................................................................................................................................................
======================================================================
ERROR: test_start_container_builds_images (tests.integration.service_test.ServiceTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/code/tests/integration/service_test.py", line 431, in test_start_container_builds_images
    self.assertIn('success', container.logs())
  File "/code/compose/container.py", line 158, in logs
    return self.client.logs(self.id, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/docker/utils/decorators.py", line 20, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/docker/client.py", line 561, in logs
    return self._get_result(container, stream, res)
  File "/usr/local/lib/python2.7/site-packages/docker/clientbase.py", line 242, in _get_result
    cont = self.inspect_container(container)
  File "/usr/local/lib/python2.7/site-packages/docker/utils/decorators.py", line 20, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/docker/client.py", line 482, in inspect_container
    self._get(self._url("/containers/{0}/json".format(container))),
  File "/usr/local/lib/python2.7/site-packages/docker/clientbase.py", line 86, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 477, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', ResponseNotReady())
-------------------- >> begin captured stdout << ---------------------
Step 0 : FROM busybox:latest
 ---> 8c2e06607696
Step 1 : LABEL com.docker.compose.test_image true
 ---> Running in 14e9136f232f
 ---> eccfd4744bb9
Removing intermediate container 14e9136f232f
Step 2 : CMD echo "success"
 ---> Running in 6f96b87e40b4
 ---> 555009043f69
Removing intermediate container 6f96b87e40b4
Successfully built 555009043f69

--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
requests.packages.urllib3.connectionpool: DEBUG: "GET /v1.18/images/composetest_test/json HTTP/1.1" 404 32
compose.service: INFO: Building test...
requests.packages.urllib3.connectionpool: DEBUG: "POST /v1.18/build?pull=0&nocache=False&q=False&t=composetest_test&forcerm=False&rm=True HTTP/1.1" 200 None
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): docker
requests.packages.urllib3.connectionpool: DEBUG: "GET /v1.18/containers/json?all=1&limit=-1&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dcomposetest%22%2C+%22com.docker.compose.service%3Dtest%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D&size=0 HTTP/1.1" 200 3
requests.packages.urllib3.connectionpool: DEBUG: "GET /v1.18/images/composetest_test/json HTTP/1.1" 200 1583
compose.service: DEBUG: Added config hash: d4a67bf2c286487bedd93971dada61be95e35846b7732f866edd06076cdc3851
compose.service: INFO: Creating composetest_test_1...
requests.packages.urllib3.connectionpool: DEBUG: "POST /v1.18/containers/create?name=composetest_test_1 HTTP/1.1" 201 90
requests.packages.urllib3.connectionpool: DEBUG: "GET /v1.18/containers/7aacea8a037273d801189bbd0e175ceecb4eef775aee8051c55607989499366b/json HTTP/1.1" 200 None
requests.packages.urllib3.connectionpool: DEBUG: "POST /v1.18/containers/7aacea8a037273d801189bbd0e175ceecb4eef775aee8051c55607989499366b/start HTTP/1.1" 204 0
requests.packages.urllib3.connectionpool: DEBUG: "POST /v1.18/containers/7aacea8a037273d801189bbd0e175ceecb4eef775aee8051c55607989499366b/wait HTTP/1.1" 200 17
requests.packages.urllib3.connectionpool: DEBUG: "GET /v1.18/containers/7aacea8a037273d801189bbd0e175ceecb4eef775aee8051c55607989499366b/logs?timestamps=0&tail=all&stdout=1&stderr=1&follow=0 HTTP/1.1" 200 None
--------------------- >> end captured logging << ---------------------

======================================================================
ERROR: test_start_container_uses_tagged_image_if_it_exists (tests.integration.service_test.ServiceTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/code/tests/integration/service_test.py", line 444, in test_start_container_uses_tagged_image_if_it_exists
    self.assertIn('success', container.logs())
  File "/code/compose/container.py", line 158, in logs
    return self.client.logs(self.id, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/docker/utils/decorators.py", line 20, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/docker/client.py", line 561, in logs
    return self._get_result(container, stream, res)
  File "/usr/local/lib/python2.7/site-packages/docker/clientbase.py", line 242, in _get_result
    cont = self.inspect_container(container)
  File "/usr/local/lib/python2.7/site-packages/docker/utils/decorators.py", line 20, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/docker/client.py", line 482, in inspect_container
    self._get(self._url("/containers/{0}/json".format(container))),
  File "/usr/local/lib/python2.7/site-packages/docker/clientbase.py", line 86, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 477, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', ResponseNotReady())
-------------------- >> begin captured logging << --------------------
requests.packages.urllib3.connectionpool: DEBUG: "POST /v1.18/build?pull=0&nocache=False&q=False&t=composetest_test&forcerm=False&rm=True HTTP/1.1" 200 None
requests.packages.urllib3.connectionpool: DEBUG: "GET /v1.18/images/composetest_test/json HTTP/1.1" 200 1583
requests.packages.urllib3.connectionpool: DEBUG: "GET /v1.18/containers/json?all=1&limit=-1&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dcomposetest%22%2C+%22com.docker.compose.service%3Dtest%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D&size=0 HTTP/1.1" 200 3
requests.packages.urllib3.connectionpool: DEBUG: "GET /v1.18/images/composetest_test/json HTTP/1.1" 200 1583
compose.service: DEBUG: Added config hash: 529b4b4fc05cc619e6df0fe07854150c191179c9a0a33f793739c7dc49e9595b
compose.service: INFO: Creating composetest_test_1...
requests.packages.urllib3.connectionpool: DEBUG: "POST /v1.18/containers/create?name=composetest_test_1 HTTP/1.1" 201 90
requests.packages.urllib3.connectionpool: DEBUG: "GET /v1.18/containers/3d30602efde333baea2714e2693bd4fb542955fffce2ce15e3a58d16e7ffbece/json HTTP/1.1" 200 None
requests.packages.urllib3.connectionpool: DEBUG: "POST /v1.18/containers/3d30602efde333baea2714e2693bd4fb542955fffce2ce15e3a58d16e7ffbece/start HTTP/1.1" 204 0
requests.packages.urllib3.connectionpool: DEBUG: "POST /v1.18/containers/3d30602efde333baea2714e2693bd4fb542955fffce2ce15e3a58d16e7ffbece/wait HTTP/1.1" 200 17
requests.packages.urllib3.connectionpool: DEBUG: "GET /v1.18/containers/3d30602efde333baea2714e2693bd4fb542955fffce2ce15e3a58d16e7ffbece/logs?timestamps=0&tail=all&stdout=1&stderr=1&follow=0 HTTP/1.1" 200 None
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 343 tests in 134.628s

FAILED (errors=2)

Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>

dnephin · 2015-07-29T15:27:27Z

I was thinking about this just the other day. I like it

dnephin · 2015-08-31T20:06:48Z

I just tried using docker-machine on my desktop (with virtualbox driver) and I saw exactly the same tests fail with the same errors.

I noticed the tests ran quite a bit faster though (probably because the image graph is empty).

According to https://github.com/kennethreitz/requests/issues/2568 it's an issue with the version of requests we're using. I'm going to try upgrading to see if it fixes it.

dnephin · 2015-08-31T20:45:38Z

It also looks like there is no tag for 1.8.1, I've opened a PR aluzzardi/dind#1

…docker#1783) Signed-off-by: Daniel Nephin <dnephin@gmail.com>

dnephin · 2015-09-17T21:58:00Z

Replaced by #2065

Use dockerswarm/dind image instead of doing docker-in-docker ourselves

1290ee1

Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>

GordonTheTurtle added the status/0-triage label Jul 29, 2015

This was referenced Aug 6, 2015

Abort tests if daemon fails to start #1815

Merged

Rewrite the toolchain a bit docker-archive-public/docker.libcompose#11

Merged

dnephin added the kind/cleanup label Aug 26, 2015

dnephin added a commit to dnephin/compose that referenced this pull request Sep 1, 2015

Upgrade requests to latest version to fix dind connection problem (see …

f4a8251

…docker#1783) Signed-off-by: Daniel Nephin <dnephin@gmail.com>

This was referenced Sep 3, 2015

Bump requests to 2.7 to fix the ResponseNotReady() error #1974

Merged

Find a better way to test against dev Docker #1144

Closed

Containerized Integration docker/docker-py#757

Merged

dnephin self-assigned this Sep 9, 2015

dnephin added area/tests and removed status/0-triage labels Sep 9, 2015

dnephin added a commit to dnephin/compose that referenced this pull request Sep 10, 2015

Upgrade requests to latest version to fix dind connection problem (see …

ab1e7e3

…docker#1783) Signed-off-by: Daniel Nephin <dnephin@gmail.com>

dnephin mentioned this pull request Sep 17, 2015

Use dockerswarm/dind image for ci #2065

Merged

dnephin closed this Sep 17, 2015

sirlatrom mentioned this pull request Mar 28, 2019

Compose schema version 3.8 support #6530

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Use dockerswarm/dind image instead of doing docker-in-docker ourselves#1783

WIP: Use dockerswarm/dind image instead of doing docker-in-docker ourselves#1783
aanand wants to merge 1 commit intodocker:masterfrom
aanand:use-dind-image

aanand commented Jul 29, 2015

Uh oh!

dnephin commented Jul 29, 2015

Uh oh!

dnephin commented Aug 31, 2015

Uh oh!

dnephin commented Aug 31, 2015

Uh oh!

dnephin commented Sep 17, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

aanand commented Jul 29, 2015

Uh oh!

dnephin commented Jul 29, 2015

Uh oh!

dnephin commented Aug 31, 2015

Uh oh!

dnephin commented Aug 31, 2015

Uh oh!

dnephin commented Sep 17, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants