Adjust `--platform` tests for changes in docker engine by thaJeztah · Pull Request #2382 · docker/docker-py

thaJeztah · 2019-07-15T13:05:26Z

These tests started failing on recent versions of the engine because the error string changed,
and due to a regression, the status code for one endpoint changed from a 400 to a 500.

On Docker 18.03:

The docker build case properly returns a 400, and "invalid platform" as error string;

docker build --platform=foobar -<<EOF
FROM busybox
EOF

Sending build context to Docker daemon  2.048kB
Error response from daemon: invalid platform: invalid platform os "foobar"

DEBU[2019-07-15T12:17:22.745511870Z] Calling GET /_ping
DEBU[2019-07-15T12:17:22.748224796Z] Calling POST /session
DEBU[2019-07-15T12:17:22.748692282Z] Calling POST /v1.37/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&platform=foobar&rm=1&session=d7b6ceba9d8d0aed67a805528554feb5285781fe888a4bf4e0c15cb09bffd614&shmsize=0&target=&ulimits=null

The docker pull --platform=foobar hello-world:latest case incorrectly returns a 500

DEBU[2019-07-15T12:16:08.744827612Z] Calling POST /v1.37/images/create?fromImage=hello-world&platform=foobar&tag=latest
DEBU[2019-07-15T12:16:08.745594874Z] FIXME: Got an API for which error does not match any expected type!!!: invalid platform: invalid platform os "foobar"  error_type="*errors.errorString" module=api
ERRO[2019-07-15T12:16:08.745916686Z] Handler for POST /v1.37/images/create returned error: invalid platform: invalid platform os "foobar"
DEBU[2019-07-15T12:16:08.746191172Z] FIXME: Got an API for which error does not match any expected type!!!: invalid platform: invalid platform os "foobar"  error_type="*errors.errorString" module=api

On Docker 18.09;

docker build --platform=foobar -<<EOF
FROM busybox
EOF

Error response from daemon: "foobar": unknown operating system or architecture: invalid argument

Which incorrectly returns a 500 status;

DEBU[2019-07-15T11:59:20.687268380Z] Calling POST /v1.39/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&platform=foobar&rm=1&session=jko7kejjvs93judyfnq7shoda&shmsize=0&target=&ulimits=null&version=1
DEBU[2019-07-15T11:59:20.687282279Z] Calling POST /session
INFO[2019-07-15T11:59:20.687761392Z] parsed scheme: ""                             module=grpc
INFO[2019-07-15T11:59:20.687833668Z] scheme "" not registered, fallback to default scheme  module=grpc
INFO[2019-07-15T11:59:20.688017578Z] ccResolverWrapper: sending new addresses to cc: [{ 0  <nil>}]  module=grpc
INFO[2019-07-15T11:59:20.688270160Z] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2019-07-15T11:59:20.688353083Z] pickfirstBalancer: HandleSubConnStateChange: 0xc4209b0630, CONNECTING  module=grpc
INFO[2019-07-15T11:59:20.688985698Z] pickfirstBalancer: HandleSubConnStateChange: 0xc4209b0630, READY  module=grpc
DEBU[2019-07-15T11:59:20.812700550Z] client is session enabled
DEBU[2019-07-15T11:59:20.813139288Z] FIXME: Got an API for which error does not match any expected type!!!: invalid argument
github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs.init
	/go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs/errors.go:40
github.com/docker/docker/vendor/github.com/containerd/containerd/content.init
	<autogenerated>:1
github.com/docker/docker/builder/builder-next.init
	<autogenerated>:1
github.com/docker/docker/api/server/backend/build.init
	<autogenerated>:1
main.init
	<autogenerated>:1
runtime.main
	/usr/local/go/src/runtime/proc.go:186
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:2361  error_type="*errors.fundamental" module=api
ERRO[2019-07-15T11:59:20.813210677Z] Handler for POST /v1.39/build returned error: "foobar": unknown operating system or architecture: invalid argument
DEBU[2019-07-15T11:59:20.813276737Z] FIXME: Got an API for which error does not match any expected type!!!: invalid argument
github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs.init
	/go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs/errors.go:40
github.com/docker/docker/vendor/github.com/containerd/containerd/content.init
	<autogenerated>:1
github.com/docker/docker/builder/builder-next.init
	<autogenerated>:1
github.com/docker/docker/api/server/backend/build.init
	<autogenerated>:1
main.init
	<autogenerated>:1
runtime.main
	/usr/local/go/src/runtime/proc.go:186
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:2361  error_type="*errors.fundamental" module=api

Same for the docker pull --platform=foobar hello-world:latest case:

docker pull --platform=foobar hello-world:latest
Error response from daemon: "foobar": unknown operating system or architecture: invalid argument

DEBU[2019-07-15T12:00:18.812995330Z] Calling POST /v1.39/images/create?fromImage=hello-world&platform=foobar&tag=latest
DEBU[2019-07-15T12:00:18.813229172Z] FIXME: Got an API for which error does not match any expected type!!!: invalid argument
github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs.init
	/go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs/errors.go:40
github.com/docker/docker/vendor/github.com/containerd/containerd/content.init
	<autogenerated>:1
github.com/docker/docker/builder/builder-next.init
	<autogenerated>:1
github.com/docker/docker/api/server/backend/build.init
	<autogenerated>:1
main.init
	<autogenerated>:1
runtime.main
	/usr/local/go/src/runtime/proc.go:186
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:2361  error_type="*errors.fundamental" module=api
ERRO[2019-07-15T12:00:18.813365546Z] Handler for POST /v1.39/images/create returned error: "foobar": unknown operating system or architecture: invalid argument
DEBU[2019-07-15T12:00:18.813461428Z] FIXME: Got an API for which error does not match any expected type!!!: invalid argument
github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs.init
	/go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs/errors.go:40
github.com/docker/docker/vendor/github.com/containerd/containerd/content.init
	<autogenerated>:1
github.com/docker/docker/builder/builder-next.init
	<autogenerated>:1
github.com/docker/docker/api/server/backend/build.init
	<autogenerated>:1
main.init
	<autogenerated>:1
runtime.main
	/usr/local/go/src/runtime/proc.go:186
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:2361  error_type="*errors.fundamental" module=api

thaJeztah · 2019-07-15T13:05:51Z

I'll also have a look at the regression in the engine (status changing from 400 to 500)

thaJeztah · 2019-07-15T13:06:08Z

ping @shin- @ulyssessouza PTAL

thaJeztah · 2019-07-15T13:11:05Z

Ah, booh ... 79 characters. Thought it was 80

tests/integration/api_build_test.py:451:80: E501 line too long (81 > 79 characters)
tests/integration/api_build_test.py:453:80: E501 line too long (80 > 79 characters)
tests/integration/api_image_test.py:72:80: E501 line too long (81 > 79 characters)
tests/integration/api_image_test.py:74:80: E501 line too long (80 > 79 characters)
ERROR: InvocationError for command /home/travis/build/docker/docker-py/.tox/flake8/bin/flake8 docker tests setup.py (exited with code 1)
___________________________________ summary ____________________________________
ERROR:   flake8: commands failed
The command "tox" exited with 1.

rumpl · 2019-07-15T13:33:55Z

tests/integration/api_build_test.py

-        assert excinfo.value.status_code == 400
-        assert 'invalid platform' in excinfo.exconly()
+        # Some API versions incorrectly returns 500 status; assert 400 or 500
+        assert excinfo.value.is_error()


If I understand correctly this will check for any error, the comment is misleading, either we change the comment or we test for the code to be 400 or 500

Well, it tests for 400 or 500;

docker-py/tests/unit/errors_test.py

Lines 26 to 38 in 219c521

def test_status_code_400(self):

"""The status_code property is present with 400 response."""

resp = requests.Response()

resp.status_code = 400

err = APIError('', response=resp)

assert err.status_code == 400

def test_status_code_500(self):

"""The status_code property is present with 500 response."""

resp = requests.Response()

resp.status_code = 500

err = APIError('', response=resp)

assert err.status_code == 500

Whoops; wrong link:

docker-py/docker/errors.py

Lines 66 to 77 in 219c521

def is_error(self):

return self.is_client_error() or self.is_server_error()

def is_client_error(self):

if self.status_code is None:

return False

return 400 <= self.status_code < 500

def is_server_error(self):

if self.status_code is None:

return False

return 500 <= self.status_code < 600

Can change it to 4xx or 5xx

Isn't this that is called when you check for is_error()?

docker-py/docker/errors.py

Lines 66 to 77 in 1073b73

def is_error(self):

return self.is_client_error() or self.is_server_error()

def is_client_error(self):

if self.status_code is None:

return False

return 400 <= self.status_code < 500

def is_server_error(self):

if self.status_code is None:

return False

return 500 <= self.status_code < 600

@rumpl Yes, sorry, linked to the wrong part initially (see comments above)

changed it to 4xx or 5xx in the comment

These tests started failing on recent versions of the engine because the error string changed, and due to a regression, the status code for one endpoint changed from a 400 to a 500. On Docker 18.03: The `docker build` case properly returns a 400, and "invalid platform" as error string; ```bash docker build --platform=foobar -<<EOF FROM busybox EOF Sending build context to Docker daemon 2.048kB Error response from daemon: invalid platform: invalid platform os "foobar" ``` ``` DEBU[2019-07-15T12:17:22.745511870Z] Calling GET /_ping DEBU[2019-07-15T12:17:22.748224796Z] Calling POST /session DEBU[2019-07-15T12:17:22.748692282Z] Calling POST /v1.37/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&platform=foobar&rm=1&session=d7b6ceba9d8d0aed67a805528554feb5285781fe888a4bf4e0c15cb09bffd614&shmsize=0&target=&ulimits=null ``` The `docker pull --platform=foobar hello-world:latest` case incorrectly returns a 500 ``` DEBU[2019-07-15T12:16:08.744827612Z] Calling POST /v1.37/images/create?fromImage=hello-world&platform=foobar&tag=latest DEBU[2019-07-15T12:16:08.745594874Z] FIXME: Got an API for which error does not match any expected type!!!: invalid platform: invalid platform os "foobar" error_type="*errors.errorString" module=api ERRO[2019-07-15T12:16:08.745916686Z] Handler for POST /v1.37/images/create returned error: invalid platform: invalid platform os "foobar" DEBU[2019-07-15T12:16:08.746191172Z] FIXME: Got an API for which error does not match any expected type!!!: invalid platform: invalid platform os "foobar" error_type="*errors.errorString" module=api ``` On Docker 18.09; ```bash docker build --platform=foobar -<<EOF FROM busybox EOF Error response from daemon: "foobar": unknown operating system or architecture: invalid argument ``` Which incorrectly returns a 500 status; ``` DEBU[2019-07-15T11:59:20.687268380Z] Calling POST /v1.39/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&platform=foobar&rm=1&session=jko7kejjvs93judyfnq7shoda&shmsize=0&target=&ulimits=null&version=1 DEBU[2019-07-15T11:59:20.687282279Z] Calling POST /session INFO[2019-07-15T11:59:20.687761392Z] parsed scheme: "" module=grpc INFO[2019-07-15T11:59:20.687833668Z] scheme "" not registered, fallback to default scheme module=grpc INFO[2019-07-15T11:59:20.688017578Z] ccResolverWrapper: sending new addresses to cc: [{ 0 <nil>}] module=grpc INFO[2019-07-15T11:59:20.688270160Z] ClientConn switching balancer to "pick_first" module=grpc INFO[2019-07-15T11:59:20.688353083Z] pickfirstBalancer: HandleSubConnStateChange: 0xc4209b0630, CONNECTING module=grpc INFO[2019-07-15T11:59:20.688985698Z] pickfirstBalancer: HandleSubConnStateChange: 0xc4209b0630, READY module=grpc DEBU[2019-07-15T11:59:20.812700550Z] client is session enabled DEBU[2019-07-15T11:59:20.813139288Z] FIXME: Got an API for which error does not match any expected type!!!: invalid argument github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs.init /go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs/errors.go:40 github.com/docker/docker/vendor/github.com/containerd/containerd/content.init <autogenerated>:1 github.com/docker/docker/builder/builder-next.init <autogenerated>:1 github.com/docker/docker/api/server/backend/build.init <autogenerated>:1 main.init <autogenerated>:1 runtime.main /usr/local/go/src/runtime/proc.go:186 runtime.goexit /usr/local/go/src/runtime/asm_amd64.s:2361 error_type="*errors.fundamental" module=api ERRO[2019-07-15T11:59:20.813210677Z] Handler for POST /v1.39/build returned error: "foobar": unknown operating system or architecture: invalid argument DEBU[2019-07-15T11:59:20.813276737Z] FIXME: Got an API for which error does not match any expected type!!!: invalid argument github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs.init /go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs/errors.go:40 github.com/docker/docker/vendor/github.com/containerd/containerd/content.init <autogenerated>:1 github.com/docker/docker/builder/builder-next.init <autogenerated>:1 github.com/docker/docker/api/server/backend/build.init <autogenerated>:1 main.init <autogenerated>:1 runtime.main /usr/local/go/src/runtime/proc.go:186 runtime.goexit /usr/local/go/src/runtime/asm_amd64.s:2361 error_type="*errors.fundamental" module=api ``` Same for the `docker pull --platform=foobar hello-world:latest` case: ```bash docker pull --platform=foobar hello-world:latest Error response from daemon: "foobar": unknown operating system or architecture: invalid argument ``` ``` DEBU[2019-07-15T12:00:18.812995330Z] Calling POST /v1.39/images/create?fromImage=hello-world&platform=foobar&tag=latest DEBU[2019-07-15T12:00:18.813229172Z] FIXME: Got an API for which error does not match any expected type!!!: invalid argument github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs.init /go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs/errors.go:40 github.com/docker/docker/vendor/github.com/containerd/containerd/content.init <autogenerated>:1 github.com/docker/docker/builder/builder-next.init <autogenerated>:1 github.com/docker/docker/api/server/backend/build.init <autogenerated>:1 main.init <autogenerated>:1 runtime.main /usr/local/go/src/runtime/proc.go:186 runtime.goexit /usr/local/go/src/runtime/asm_amd64.s:2361 error_type="*errors.fundamental" module=api ERRO[2019-07-15T12:00:18.813365546Z] Handler for POST /v1.39/images/create returned error: "foobar": unknown operating system or architecture: invalid argument DEBU[2019-07-15T12:00:18.813461428Z] FIXME: Got an API for which error does not match any expected type!!!: invalid argument github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs.init /go/src/github.com/docker/docker/vendor/github.com/containerd/containerd/errdefs/errors.go:40 github.com/docker/docker/vendor/github.com/containerd/containerd/content.init <autogenerated>:1 github.com/docker/docker/builder/builder-next.init <autogenerated>:1 github.com/docker/docker/api/server/backend/build.init <autogenerated>:1 main.init <autogenerated>:1 runtime.main /usr/local/go/src/runtime/proc.go:186 runtime.goexit /usr/local/go/src/runtime/asm_amd64.s:2361 error_type="*errors.fundamental" module=api ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

ulyssessouza

LGTM

thaJeztah · 2019-07-15T20:33:28Z

ping @shin- 🤗

shin- · 2019-07-19T05:49:51Z

Thank you! :)

thaJeztah mentioned this pull request Jul 15, 2019

Update to python 3.7 (buster) and use build-args #2379

Merged

thaJeztah force-pushed the fix_platform_tests branch from 08ee40b to 57770e9 Compare July 15, 2019 13:13

rumpl reviewed Jul 15, 2019

View reviewed changes

thaJeztah mentioned this pull request Jul 15, 2019

Bump docker-py to 4.0.2, and run tests from upstream repository moby/moby#39068

Merged

thaJeztah force-pushed the fix_platform_tests branch from 57770e9 to bc46490 Compare July 15, 2019 13:48

ulyssessouza self-requested a review July 15, 2019 14:15

ulyssessouza approved these changes Jul 15, 2019

View reviewed changes

thaJeztah mentioned this pull request Jul 15, 2019

Fix error handling of incorrect --platform values moby/moby#39527

Merged

shin- merged commit 17c8642 into docker:master Jul 19, 2019

shin- added this to the 4.1.0 milestone Jul 19, 2019

thaJeztah deleted the fix_platform_tests branch July 20, 2019 14:29

This was referenced Aug 9, 2019

Jenkinsfile: remove "experimental" stage moby/moby#39709

Merged

docker-py: skip PullImageTest::test_pull_invalid_platform moby/moby#39714

Merged

thaJeztah mentioned this pull request Sep 16, 2019

[19.03 backport] Fix error handling of incorrect --platform values docker-archive/engine#365

Merged

This was referenced Jan 3, 2020

docker-py: skip broken ImageCollectionTest::test_pull_multiple, and re-enable fixed tests moby/moby#40343

Merged

[19.03 backport] docker-py: skip broken ImageCollectionTest::test_pull_multiple, and re-enable fixed tests docker-archive/engine#437

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adjust `--platform` tests for changes in docker engine#2382

Adjust `--platform` tests for changes in docker engine#2382
shin- merged 1 commit intodocker:masterfrom
thaJeztah:fix_platform_tests

thaJeztah commented Jul 15, 2019

Uh oh!

thaJeztah commented Jul 15, 2019

Uh oh!

thaJeztah commented Jul 15, 2019

Uh oh!

thaJeztah commented Jul 15, 2019

Uh oh!

rumpl Jul 15, 2019

Uh oh!

thaJeztah Jul 15, 2019 •

edited

Loading

Uh oh!

thaJeztah Jul 15, 2019

Uh oh!

thaJeztah Jul 15, 2019

Uh oh!

rumpl Jul 15, 2019

Uh oh!

thaJeztah Jul 15, 2019

Uh oh!

ulyssessouza left a comment

Uh oh!

thaJeztah commented Jul 15, 2019

Uh oh!

shin- commented Jul 19, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	def test_status_code_400(self):
	"""The status_code property is present with 400 response."""
	resp = requests.Response()
	resp.status_code = 400
	err = APIError('', response=resp)
	assert err.status_code == 400

	def test_status_code_500(self):
	"""The status_code property is present with 500 response."""
	resp = requests.Response()
	resp.status_code = 500
	err = APIError('', response=resp)
	assert err.status_code == 500

	def is_error(self):
	return self.is_client_error() or self.is_server_error()

	def is_client_error(self):
	if self.status_code is None:
	return False
	return 400 <= self.status_code < 500

	def is_server_error(self):
	if self.status_code is None:
	return False
	return 500 <= self.status_code < 600

Conversation

thaJeztah commented Jul 15, 2019

Uh oh!

thaJeztah commented Jul 15, 2019

Uh oh!

thaJeztah commented Jul 15, 2019

Uh oh!

thaJeztah commented Jul 15, 2019

Uh oh!

rumpl Jul 15, 2019

Choose a reason for hiding this comment

Uh oh!

thaJeztah Jul 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah Jul 15, 2019

Choose a reason for hiding this comment

Uh oh!

thaJeztah Jul 15, 2019

Choose a reason for hiding this comment

Uh oh!

rumpl Jul 15, 2019

Choose a reason for hiding this comment

Uh oh!

thaJeztah Jul 15, 2019

Choose a reason for hiding this comment

Uh oh!

ulyssessouza left a comment

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Jul 15, 2019

Uh oh!

shin- commented Jul 19, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

thaJeztah Jul 15, 2019 •

edited

Loading