Skip to content

daemon/restart: Don't mount with c8d snapshotters#47093

Closed
vvoland wants to merge 1 commit intomoby:masterfrom
vvoland:c8d-windows-restart-nomount
Closed

daemon/restart: Don't mount with c8d snapshotters#47093
vvoland wants to merge 1 commit intomoby:masterfrom
vvoland:c8d-windows-restart-nomount

Conversation

@vvoland
Copy link
Contributor

@vvoland vvoland commented Jan 17, 2024

- What I did

- How I did it

- How to verify it

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

@vvoland vvoland added platform/windows containerd-integration Issues and PRs related to containerd integration labels Jan 17, 2024
@vvoland vvoland self-assigned this Jan 17, 2024
@vvoland
Copy link
Contributor Author

vvoland commented Jan 17, 2024

Looks like this helped these two:

-=== FAIL: github.com/docker/docker/integration-cli TestDockerCLIRestartSuite/TestRestartContainerwithRestartPolicy (12.24s)
-    docker_cli_restart_test.go:293: assertion failed: 
-        Command:  D:\a\moby\moby\out\docker.exe restart 1bc60617e320cd5c23a4c90fbc01760b0cf4458f75c4b5f657ab7688cf558803
-        ExitCode: 1
-        Error:    exit status 1
-        Stdout:   
-        Stderr:   Error response from daemon: Cannot restart container 1bc60617e320cd5c23a4c90fbc01760b0cf4458f75c4b5f657ab7688cf558803: failed to mount C:\Users\runneradmin\AppData\Local\Temp\moby-root\rootfs\windows\1bc60617e320cd5c23a4c90fbc01760b0cf4458f75c4b5f657ab7688cf558803: failed to activate layer C:\Users\RUNNER~1\AppData\Local\Temp\ctn-root\io.containerd.snapshotter.v1.windows\snapshots\104: hcsshim::ActivateLayer failed in Win32: The process cannot access the file because it is being used by another process. (0x20)
-        
-        
-        Failures:
-        ExitCode was 1 expected 0
-        Expected no error
-
-=== FAIL: github.com/docker/docker/integration-cli TestDockerCLIRestartSuite/TestRestartStoppedContainer (3.24s)
-    docker_cli_restart_test.go:38: assertion failed: 
-        Command:  D:\a\moby\moby\out\docker.exe restart 0a31f0c99e4109084b2970acaedae4335c3b71d93edc8de235217ee4558fcded
-        ExitCode: 1
-        Error:    exit status 1
-        Stdout:   
-        Stderr:   Error response from daemon: Cannot restart container 0a31f0c99e4109084b2970acaedae4335c3b71d93edc8de235217ee4558fcded: failed to mount C:\Users\runneradmin\AppData\Local\Temp\moby-root\rootfs\windows\0a31f0c99e4109084b2970acaedae4335c3b71d93edc8de235217ee4558fcded: failed to activate layer C:\Users\RUNNER~1\AppData\Local\Temp\ctn-root\io.containerd.snapshotter.v1.windows\snapshots\120: hcsshim::ActivateLayer failed in Win32: The process cannot access the file because it is being used by another process. (0x20)
-        
-        
-        Failures:
-        ExitCode was 1 expected 0
-        Expected no error
-
-=== FAIL: github.com/docker/docker/integration-cli TestDockerCLIRestartSuite (39.97s)

+=== RUN   TestDockerCLIRestartSuite/TestRestartContainerwithRestartPolicy
+--- PASS: TestDockerCLIRestartSuite/TestRestartContainerwithRestartPolicy (23.78s)
+=== RUN   TestDockerCLIRestartSuite/TestRestartStoppedContainer
+--- PASS: TestDockerCLIRestartSuite/TestRestartStoppedContainer (5.43s)

Will rerun the CI a couple more times to see if they pass consistently.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
@vvoland vvoland force-pushed the c8d-windows-restart-nomount branch from 813a2a6 to f083dce Compare January 17, 2024 17:27
Comment on lines 51 to +52
// VM.
if !containertypes.Isolation.IsHyperV(actualIsolation) {
if !daemon.UsesSnapshotter() && !containertypes.Isolation.IsHyperV(actualIsolation) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably update the comment above to describe why we're not doing it with containerd snapshotters as well now.

Copy link
Contributor

@TBBle TBBle Jan 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I following this logic correctly and daemon.Mount is incrementing a reference count if the given container's storage is already mounted to the host, i.e. the current Linux case. The only way I understand this logic is if that's the case, and daemon.containerStop does an implicit daemon.Unmount, and daemon.containerStart does an implicit daemon.Mount.

The current change proposed here, as far as I understand, will reintroduce the needless (but otherwise harmless?) unmount/mount for Linux containerd-storage modes, as they still currently rely on host-mounting to function.

Would it be better to simply say "Only take a reference if the container is already mounted", or is that not knowable/thread-safe here? That would isolate this logic from host-mount-related flow-changes made elsewhere, e.g. moving to WithSnapshot.

That said, I'm not clear why this would fail on Windows in process isolation, since we currently also use a host-mount in that case, so this should be the same simple ref-count-only change, and not lead to calling ActivateLayer as seen in the failures... Is it perhaps elsewhere doing a path-name comparison, and getting a false-negative over something like what we use getBackingDeviceForContainerdMount to fix, and hence trying to create a second mount for the same snapshot?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it looks like it could be caused by our ref-counted mount having a whitelist of filesystems. I opened a PR to reverse the logic and use the refcounted mounter by default (which we already talked about in the past): #47105

If these tests pass on the mentioned PR, we can close this one.

@vvoland
Copy link
Contributor Author

vvoland commented Jan 18, 2024

Closing in favor of: #47105

@vvoland vvoland closed this Jan 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

containerd-integration Issues and PRs related to containerd integration platform/windows

Projects

Development

Successfully merging this pull request may close these issues.

3 participants