Description

With docker 26.1 we saw the following logs being repeated so often that disk space literally filled up (syslog):

2024-05-08T10:49:19.381166+00:00 hostnamereplaced dockerd[25865]: time="2024-05-08T10:49:19.381077536Z" level=info msg="attempting to unpublish volume" attempt=436364316 module=node/agent/csi volume.id=jjn8qz56mg3852jdllefpz21l
2024-05-08T10:49:19.381171+00:00 hostnamereplaced dockerd[25865]: time="2024-05-08T10:49:19.381115531Z" level=info msg="upublishing volume failed" attempt=436364316 error="cannot get plugin csi-smb:latest: plugin does not implement PluginAddr interface" module=node/agent/csi volume.id=jjn8qz56mg3852jdllefpz21l 

I think this is possibly related to moby/swarmkit#3171, which seems to have altered the logic that decides to log this error.

The cluster volume driver we use is the kubernetes SMB CSI driver, built for swarm according to information from https://github.com/olljanat/csi-plugins-for-docker-swarm.

The nodes that we would see this error message on were not supposed to have any containers running with the volume mounted, so we don't actually understand why it is attempting to unpublish it.

Reproduce

We can not trigger reproduction actively, it just "seems to happen" after some time. See description above.

Expected behavior

First of all it seems that even if some error occurs, it should not proceed to busy-loop and log every iteration.

Second, the log seems to indicate a programming error - some incompatibility between swarmkit and docker itself?

docker version

It was docker 26.1. We have since downgraded to 26.0 / some nodes updated to 27, which are not relevant to this issue.

docker info

See docker version info: We have since downgraded and i did not get the info while we were on the bad version.

Additional Info

No response