Add npipe support by friism · Pull Request #1186 · portainer/portainer

friism · 2017-09-10T20:11:02Z

I hadn't noticed @StefanScherer had also started this work here: develop...StefanScherer:npipe_support

The current code is ~~~not~~~ working. ~~~It seems like the named-pipe dial doesn't work correctly. I've checked that the right named pipe url is fed in.~~~

deviantony

I did a quick review, what are the issues with this implementation? Any errors in the logs?

deviantony · 2017-09-10T20:16:14Z

api/http/proxy/manager.go

-	} else {
+	}
+	case "npipe":
+		var namedPipePath = strings.Replace(endpoint.URL, "npipe:", "", 1)


I think that you can use endpoint.URL.Path here instead of strings.Replace(endpoint.URL, "npipe:", "", 1)

deviantony · 2017-09-10T20:19:59Z

api/http/proxy/transport.go

+func newNamedPipeTransport(namedPipePath string) *http.Transport {
+	return &http.Transport{
+		Dial: func(proto, addr string) (conn net.Conn, err error) {
+			return npipe.Dial(namedPipePath)


Issue might come from here... Looks like @StefanScherer is using github.com/Microsoft/go-winio here. You might want to give it a try.

See: develop...StefanScherer:npipe_support#diff-d678b9df77b49ad5bb4195768e75b3afR45

deviantony · 2017-09-10T20:21:01Z

app/components/endpoint/endpointController.js

      $('#loadingViewSpinner').hide();
      $scope.endpoint = data;
-      if (data.URL.indexOf('unix://') === 0) {
+      if (data.URL.indexOf('unix://') === 0 || data.URL.indexOf('npipe:') === 0) {


Shouldn't that be data.URL.indexOf('npipe://') instead?

deviantony · 2017-09-10T20:23:12Z

app/components/initEndpoint/initEndpointController.js

    $('#createResourceSpinner').show();
    var name = 'local';
-    var URL = 'unix:///var/run/docker.sock';
+    var URL = 'npipe:\\\\.\\pipe\\docker_engine';


Of course we won't replace the socket by the npipe when merged. At the moment, this is still acceptable for tests.

Regarding the UX during the first endpoint initialization, we have multiple choices here:

Detect the platform where Portainer is running and automatically adapt the URL value with either the socket path or named pipe

Replace the two options ('local' and 'remote') by Local (Linux), Local (Windows) and Remote.

Agreed, this was just a testing hack - I'll rebase out that commit

deviantony · 2017-09-10T20:24:19Z

app/services/api/endpointService.js

    var endpoint = {
      Name: 'local',
-      URL: 'unix:///var/run/docker.sock',
+      URL: 'npipe:\\\\.\\pipe\\docker_engine',


Same thing here, I think that we'll need to have createLocalLinuxEndpoint and createLocalWindowsEndpoint.

@deviantony what do you want to do here? Is there a way to extricate the server host OS with angular?

deviantony · 2017-09-10T20:26:57Z

api/http/proxy/named_pipe.go

@@ -0,0 +1,35 @@
+package proxy


Is this file really useful? Looks similar to api/http/proxy/socket.go.

friism · 2017-09-10T20:28:27Z

Ah, using @StefanScherer's changes to use winio makes it work for me too.

deviantony · 2017-09-10T20:29:40Z

Ok, great ! Please have a look at my comments, there are a few things I'd like to change.

StefanScherer

Thanks @friism for pushing this forward 👍. Hadn't the time to open a PR. So we can work on this one.

StefanScherer · 2017-09-10T21:36:16Z

app/components/initEndpoint/initEndpoint.html

-                      This feature is not yet available for <u>native Docker Windows containers</u>.
-                    </p>
                    <p class="text-primary">
                      Please ensure that you have started the Portainer container with the following Docker flag <code>-v "/var/run/docker.sock:/var/run/docker.sock"</code> in order to connect to the local Docker environment.


The user should also see some hints how to do this with Windows <code>-v \\.\pipe\\docker_engine:\\.\pipe\\docker_engine</code>.

StefanScherer · 2017-09-10T21:36:42Z

api/http/proxy/local.go

 package proxy

-// unixSocketHandler represents a handler to proxy HTTP requests via a unix:// socket
+// represents a handler to proxy HTTP requests via a unix:// socket and npope:// named pipes


a typo -> s/npope/npipe/

StefanScherer · 2017-09-10T21:38:10Z

app/components/initEndpoint/initEndpointController.js

    $('#createResourceSpinner').show();
    var name = 'local';
-    var URL = 'unix:///var/run/docker.sock';
+    var URL = 'npipe:////./pipe/docker_engine';


I also thought how we get the server OS here. Maybe this should be left empty and the EndponitService uses the defaults for the correct platform? @deviantony WDYT

StefanScherer · 2017-09-10T21:38:39Z

app/services/api/endpointService.js

    var endpoint = {
      Name: 'local',
-      URL: 'unix:///var/run/docker.sock',
+      URL: 'npipe:////./pipe/docker_engine',


same here, keep it empty and use default in Go server side?

friism · 2017-09-11T04:59:34Z

@deviantony @StefanScherer ready for review

StefanScherer

Awesome. Will test this later today.
Do we need a version check or notice when someone tries this on current Windows OS?

StefanScherer · 2017-09-11T05:39:08Z

app/components/initEndpoint/initEndpoint.html

-                    </p>
                    <p class="text-primary">
-                      Please ensure that you have started the Portainer container with the following Docker flag <code>-v "/var/run/docker.sock:/var/run/docker.sock"</code> in order to connect to the local Docker environment.
+                      Please ensure that you have started the Portainer container with the following Docker flag <code>-v /var/run/docker.sock:/var/run/docker.sock</code> (MacOS/Linux) or <code>-v \\.\pipe\docker_engine:\\.\pipe\docker_engine</code> (Native Windows containers) in order to connect to the local Docker environment.


I suggest to remove MacOS and instead only mention Linux containers.
I'm working from a Mac to start Portainer on a Windows Docker host as well.
This reminds me of LCOW, let's see what combinations we can use there? 🤓

~For now, I'd rather not see the information message for the Windows platform. As this feature is not available yet for Windows container, it may confuse our current Windows users.

I'd go for merging the support of the named pipe, without really being able to use it before the ability to mount the named pipe is released (no changes on the front-end for now).~

EDIT: We'll merge this PR when the ability to mount the named pipe in a container is available.

As @StefanScherer said, I'd rather see the MacOS mention removed as well.

StefanScherer · 2017-09-11T06:52:30Z

I compiled the PR and tested it on Windows.
For my tests I don't use a mapped volume, so I have to enter the admin password. After logging in for the first time I see this dialog to select where to connect.

Then I select local and see the new message with the -v option for Linux and Windows.

Then I sometimes saw error messages that the local connection couldn't be established, so I pressed again.

In this case only the dashboard shows an error:

It seems that the dashboard directly communicates with the local connection?

When I open the endpoints I see the primary (from the -H option) and the connection I had to select before seeing the dashboard.

So maybe the named pipe is not available directly after starting the container and portainer.exe is accessing it too early? I'll try more in a different Azure VM.

deviantony

Good work !

Two things here:

Regarding the API, please see my comment about introducing a Local field in the postEndpointsRequest struct
Regarding the front-end, I would rather not see any change in the initEndpoint.html view yet. The ability to mount the named pipe inside a container is not available yet and this may confuse our Windows users.

EDIT: Disregard my comment regarding the front-end, we can keep this PR open and only merge it when the ability to mount the named pipe in a container is available.

deviantony · 2017-09-11T07:34:17Z

app/components/initEndpoint/initEndpoint.html

-                    </p>
                    <p class="text-primary">
-                      Please ensure that you have started the Portainer container with the following Docker flag <code>-v "/var/run/docker.sock:/var/run/docker.sock"</code> in order to connect to the local Docker environment.
+                      Please ensure that you have started the Portainer container with the following Docker flag <code>-v /var/run/docker.sock:/var/run/docker.sock</code> (MacOS/Linux) or <code>-v \\.\pipe\docker_engine:\\.\pipe\docker_engine</code> (Native Windows containers) in order to connect to the local Docker environment.


~For now, I'd rather not see the information message for the Windows platform. As this feature is not available yet for Windows container, it may confuse our current Windows users.

I'd go for merging the support of the named pipe, without really being able to use it before the ability to mount the named pipe is released (no changes on the front-end for now).~

EDIT: We'll merge this PR when the ability to mount the named pipe in a container is available.

As @StefanScherer said, I'd rather see the MacOS mention removed as well.

deviantony · 2017-09-11T07:43:22Z

api/http/handler/endpoint.go

 type (
 	postEndpointsRequest struct {
 		Name      string `valid:"required"`
-		URL       string `valid:"required"`


I'd rather introduce a boolean in postEndpointsRequest, something like Local to specify if the creation of the endpoint should rely on the platform check. This field would be required.

I don't really like the fact that the creation of a local endpoint relies on the Name field and an empty URL. Could you update the api/swagger.yaml file as well?

deviantony · 2017-09-11T07:44:01Z

app/services/api/endpointService.js

  service.createLocalEndpoint = function(name, URL, TLS, active) {
    var endpoint = {
      Name: 'local',
-      URL: 'unix:///var/run/docker.sock',


Based on my comment above, the payload for the endpoint creation should contain the new Local field.

deviantony · 2017-09-11T07:48:18Z

@StefanScherer

I compiled the PR and tested it on Windows.
For my tests I don't use a mapped volume, so I have to enter the admin password. After logging in for the first time I see this dialog to select where to connect.

This is strange, if you used the -H field on the CLI, you should not see the endpoint init. view. I'll have a look at this.

EDIT: ~~I was able to reproduce this, I'll push a fix ASAP~~. It was introduced in #1160
and has been fixed via be4f3ec

Could you test again after the rebase? We need to ensure that the npipe error cannot be reproduced.

@friism could you rebase on the latest version of the develop branch?

friism · 2017-09-11T14:26:55Z

rebased, @StefanScherer feel free to give a whirl.

Still need to do the API-related changes

deviantony · 2017-10-27T12:28:34Z

@StefanScherer can I use the latest version of microsoft/nanoserver image?

Also, could any of you confirm that the container console feature is working?

StefanScherer · 2017-10-27T16:31:12Z

@deviantony If you are running Windows Server 2016 (10.0.14393.x) then you cannot run the microsoft/nanoserver:1709 container.

To be able to run this you need Windows Server 1709 (10.0.16299.x). The kernel version of host and container must match. Only with Server 1709 + HyperV feature enable you are also able to run older 10.0.14393 images with docker run --isolation hyperv ....

And yes, the console (cmd.exe into the portainer container as nano has no powershell.exe) works:

Only the xterm.js (right?) shows some window shade effect, but this also happens in older versions.

BTW: The next PowersShell 6 beta 9 renames powershell.exe to pwsh.exe.

PatrickLang · 2017-10-27T16:48:28Z

Thanks! Giving it a look

PatrickLang · 2017-10-27T17:39:12Z

Here's what I'm getting right now. I had to update to the latest docker/dockerd.exe from master.dockerproject.org first.

Process isolation (default on Windows Server 1709) - works

with --isolation=hyperv - verified both on Windows Server version 1709 and Windows 10 version 1709

 Connect Failure
open //./pipe/docker_engine: message readmode pipes not supported

Is that consistent with what you're seeing @friism and @StefanScherer ?

I also double checked that it was being handled by running with dockerd.exe -D:

DEBU[2017-10-27T17:39:56.789671000Z] HCSShim::CreateContainer id=f967aa300ef7d9884c1d6c606f708e835cf27f73d86ed5850fe64f1126120383 config={"SystemType":"Container","Name":"f967aa300ef7d9884c1d6c606f708e835cf27f73d86ed5850fe64f1126120383","Owner":"docker","IgnoreFlushesDuringBoot":true,"LayerFolderPath":"C:\\ProgramData\\docker\\windowsfilter\\f967aa300ef7d9884c1d6c606f708e835cf27f73d86ed5850fe64f1126120383","Layers":[{"ID":"68de52ca-5622-5c24-baea-708358f023c1","Path":"C:\\ProgramData\\docker\\windowsfilter\\945ea8554f9fe65fecbc006179f0038e4b8288bc67f5df692fd2c2a624e7e2df"},{"ID":"d3f12bc6-6abd-5ca7-b984-961589f70759","Path":"C:\\ProgramData\\docker\\windowsfilter\\7e9c7a371c6efdab8d14bab592f19f0143f92ed06717a8ce326a0239d4a3c804"},{"ID":"af336f59-35c1-517c-808d-81dd6cc8428d","Path":"C:\\ProgramData\\docker\\windowsfilter\\77d9dc52b021d7b659430800a40cd93b5bf7045fc4a5ebb4b0db4255c744c51e"},{"ID":"4f18bbae-1111-5d0e-bb5b-88371b2a6ad7","Path":"C:\\ProgramData\\docker\\windowsfilter\\7d314064c55d0f8148e123b0d3d493982ff0925a0f8d7e630af48185f8b7cd5a"},{"ID":"c3bd3eb5-7e5b-5353-8ed0-994e7cc72cec","Path":"C:\\ProgramData\\docker\\windowsfilter\\d4efa9c6742564402e7c2dd344d86977e61264572921b2198f9b32e6224ed251"},{"ID":"768b0c9c-d4c6-5625-ac20-e2ebc27cf6d9","Path":"C:\\ProgramData\\docker\\windowsfilter\\9ed561c60ebef7bf7f919a5e1136af7641e5e1b1e3f09069ff38471dced51518"},{"ID":"bc74a93e-df9c-5a77-b2f3-3e6c8d7c0981","Path":"C:\\ProgramData\\docker\\windowsfilter\\8fd36097680b1c28f2c93fbe9754ab0ca683bf9b6a2fb13c76896d12808516e2"}],"HostName":"f967aa300ef7","MappedDirectories":[{"HostPath":"C:\\ProgramData\\docker\\volumes\\e121b90864c40c674df5cb9bd64b19ede560d857f3cf9c6982df9106b3003218\\_data","ContainerPath":"c:\\data","ReadOnly":false,"BandwidthMaximum":0,"IOPSMaximum":0,"CreateInUtilityVM":false}],"MappedPipes":[{"HostPath":"\\\\.\\pipe\\docker_engine","ContainerPipeName":"docker_engine"}],"HvPartition":true,"EndpointList":["bdceaebb-3c12-49ff-bdf5-bb9340f45356"],"HvRuntime":{"ImagePath":"C:\\ProgramData\\docker\\windowsfilter\\9ed561c60ebef7bf7f919a5e1136af7641e5e1b1e3f09069ff38471dced51518\\UtilityVM"},"AllowUnqualifiedDNSQuery":true}
DEBU[2017-10-27T17:39:57.233108800Z] HCSShim::CreateContainer succeeded id=f967aa300ef7d9884c1d6c606f708e835cf27f73d86ed5850fe64f1126120383 handle=76718496

"MappedPipes":[{"HostPath":"\\\\.\\pipe\\docker_engine","ContainerPipeName":"docker_engine"}] is there as it should be

StefanScherer · 2017-10-27T18:01:06Z

@PatrickLang I also saw message readmode pipes not supported as I accidentally called portainer with --isolation=hyperv.

deviantony · 2017-10-30T08:38:20Z

If you are running Windows Server 2016 (10.0.14393.x) then you cannot run the microsoft/nanoserver:1709 container.

To be able to run this you need Windows Server 1709 (10.0.16299.x). The kernel version of host and container must match. Only with Server 1709 + HyperV feature enable you are also able to run older 10.0.14393 images with docker run --isolation hyperv ....

So, let's say I would like this to be part of our next Portainer release, what information message would I need to show to the users? Can I still use microsoft/nanoserver:latest or is microsoft/nanoserver:1709 mandatory?

If I understand correctly, this feature woud only be available for Windows Server 1709 (10.0.16299.x) users?

StefanScherer · 2017-10-30T14:43:44Z

@deviantony Yes, microsoft/nanoserver:1709 is mandatory at the moment and will only work on Windows Server 1709 without the HyperV isolation mode.

Please ensure that you have started the Portainer container with the following Docker flag -v /var/run/docker.sock:/var/run/docker.sock (Linux) or -v \.\pipe\docker_engine:\.\pipe\docker_engine (Native Windows containers) in order to connect to the local Docker environment.

Maybe this:

Please ensure that you have started the Portainer container with the following Docker flag -v /var/run/docker.sock:/var/run/docker.sock (Linux) or -v \.\pipe\docker_engine:\.\pipe\docker_engine (Native Windows Server 1709 containers) in order to connect to the local Docker environment.

Maybe there will be a fix for HyperV isolation in the future, then Windows 10 users could use that.

I've used the manifest-tool today to provide multi-os-version manifest list to support both Windows versions. This works fine, so the docker engine pulls the image with the matching os.version number.
I can provide you more details after some further tests how to provide two Windows variants for the next release.

deviantony · 2017-10-30T16:00:47Z

@StefanScherer would there be any con to use microsoft/nanoserver:1709 instead of microsoft/nanoserver:latest ? I'd like to avoid having too many images to maintain. Will latest point to 1709 at some point?

We could also merge this, or keep it open, until it can affect a wider userbase.

StefanScherer · 2017-10-30T16:31:47Z

@deviantony yes, if you only use microsoft/nanoserver:1709 image then Windows Server 2016 LTS users that don't update to the Semi-annual channel cannot run Portainer, even with HyperV isolation it's not possible to run newer container images on older Windows Docker host.

I was able to produce a manifest list for multiple Windows OS versions today with manifest-tool push from-spec with a manifest YAML file. I guess the manifest-tool push from-args won't help us here.

Creating a second Windows image for 1709 would work even on an older Windows Docker host. I did that eg. for a curl image with a manifest.yml and these steps in AppVeyor.

At least having this PR merged and available with only one Windows image would be better than nothing. But as portainer/portainer was the first best use-case for multi-arch as we just can docker run portainer/portainer without thinking of any platform, it would be consequent to add 1709 support as well :-)

I'll explain my steps in a blog post soon to get more feedback on that approach.

deviantony · 2017-10-30T16:49:59Z

Well, there is still some work to do before this PR can be merged but I agree with you, it'd be neat to have this feature implemented even if it will be only available to a specific set of users.

Ping me when your blog post is ready, I'm keen to add 1709 support :)

StefanScherer · 2017-11-01T02:14:13Z

@deviantony https://stefanscherer.github.io/poc-build-images-for-1709-without-1709/ 🎃
For the COPY deployed portainer this way should work fine. That's what I did a few days ago with your provided image for this PR.

StefanScherer · 2017-11-29T09:46:04Z

We still have no answer in microsoft/go-winio#67

This PR waits for the refactoring to make it work for Linux again, @friism ping me if I should take care of the remaining steps.

aiminickwong · 2018-05-24T06:19:34Z

that's a great goal!
For windows socket I found a little interesting article：
https://blogs.msdn.microsoft.com/commandline/2017/12/19/af_unix-comes-to-windows/
https://blogs.msdn.microsoft.com/commandline/2018/02/07/windowswsl-interop-with-af_unix/

StefanScherer · 2018-05-24T07:50:55Z

There is no AF_UNIX support in Docker engine on Windows yet. See moby/moby#36442 for first ideas. Also Golang has to support it first.

StefanScherer · 2018-05-24T18:12:56Z

npipe support is very theoretical at the moment.

The Docker 17.10.0-preview3 should no longer be used due to security reasons.
The current Docker EE 17.06.2-ee-10 does not have npipe support.
The Docker for Windows Stable and Edge binaries have npipe support, eg. D4W Edge with 18.05.0-ce binaries.

I'm using the dockerd.exe from D4W Edge in a Windows Server 1803 on Azure. With this setup I can run docker run -v //./pipe/docker_engine://./pipe/docker_engine stefanscherer/docker-cli-windows docker version which works fine.

But on Windows 10 there is only hyperv isolation mode and this is not supported for npipe mounts.

olljanat · 2018-07-01T21:28:47Z

@friism, @StefanScherer 18.03.1-ee-1 is out so can we now get forward with this one?

carlfischer1 · 2018-07-02T15:56:02Z

@olljanat named pipes into process isolated containers will work with 18.03.1. use with hyper-v isolated containers requires PRs that landed too late to make it into 18.03.1-ee-1:

See docker/cli#1157 (comment)

olljanat · 2018-07-02T19:29:10Z

@carlfischer1 based on my experience I would not recommend Hyper-V isolation mode for anyone anyway because of much bigger overhead and worse performance.

So can we get this one merged so we can start using it on environments which are using process isolation mode?

deviantony · 2018-07-03T05:55:18Z

@olljanat The implementation of this POC relied on the microsoft/go-winio library. We've spotted an issue during our tests which has still not been fixed: microsoft/go-winio#67

I believe that we should wait for an update on this topic before going further.

deviantony · 2018-07-04T09:29:13Z

Closed in favor of #2018

…1186)

deviantony reviewed Sep 10, 2017

View reviewed changes

StefanScherer reviewed Sep 10, 2017

View reviewed changes

friism changed the title ~~[WIP] Add npipe support~~ Add npipe support Sep 11, 2017

StefanScherer reviewed Sep 11, 2017

View reviewed changes

deviantony added contrib/func-review-approved labels Sep 11, 2017

deviantony requested changes Sep 11, 2017

View reviewed changes

deviantony added changes-required Waiting for an update of the contributor rebase-required Indicates that the PR must be rebased on the latest development branch and removed contrib/tech-review-in-progress labels Sep 11, 2017

friism added 12 commits September 11, 2017 07:25

add npipe support

40b2f00

[scratch] test hacks

4792f5b

steal correct npipe handling from stefan

31152b6

simplify

032a362

address feedbacks

cf5a708

hacks on hacks

906ab93

move default value to backend to account for runtime os

77dc988

typo

31440cc

add windows instructions

661e3a0

extra newline

1135248

gofmt

4beb64d

remove "MacOS"

f0f4815

friism force-pushed the add-npipe-support branch from 24293a7 to f0f4815 Compare September 11, 2017 14:26

deviantony removed the rebase-required Indicates that the PR must be rebased on the latest development branch label Sep 11, 2017

StefanScherer mentioned this pull request Dec 4, 2017

Support mapped named pipe endpoint for Windows container #1179

Closed

deviantony mentioned this pull request May 24, 2018

[Feature Request] Windows agent support #1929

Closed

olljanat mentioned this pull request Jul 3, 2018

feat(api): Add npipe support #2018

Merged

deviantony removed contrib/func-review-approved feedback-wanted Indicates that the PR can be tested in its current state labels Jul 4, 2018

deviantony closed this Jul 4, 2018

deviantony removed the work-in-progress Indicates that the PR is a work-in-progress and not ready yet label Jul 20, 2018

xAt0mZ pushed a commit that referenced this pull request Aug 25, 2022

fix(fdo): change 'http' to 'https' in the placeholder text EE-2479 (#…

4415ea3

…1186)

portainer-bot bot pushed a commit that referenced this pull request Sep 13, 2025

Fix typo (#1186)

2f2e70b

Conversation

friism commented Sep 10, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

deviantony left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

friism commented Sep 10, 2017

Uh oh!

deviantony commented Sep 10, 2017

Uh oh!

StefanScherer left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

StefanScherer Sep 10, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

friism commented Sep 11, 2017

Uh oh!

StefanScherer left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

deviantony Sep 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

StefanScherer commented Sep 11, 2017

Uh oh!

deviantony left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

deviantony Sep 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

deviantony commented Sep 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

friism commented Sep 11, 2017

Uh oh!

deviantony commented Oct 27, 2017

Uh oh!

StefanScherer commented Oct 27, 2017

Uh oh!

PatrickLang commented Oct 27, 2017

Uh oh!

PatrickLang commented Oct 27, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

StefanScherer commented Oct 27, 2017

friism commented Sep 10, 2017 •

edited

Loading

StefanScherer Sep 10, 2017 •

edited

Loading

deviantony Sep 11, 2017 •

edited

Loading

deviantony left a comment •

edited

Loading

deviantony Sep 11, 2017 •

edited

Loading

deviantony commented Sep 11, 2017 •

edited

Loading

PatrickLang commented Oct 27, 2017 •

edited

Loading

StefanScherer commented Nov 1, 2017 •

edited

Loading

olljanat commented Jul 1, 2018 •

edited

Loading