Problem

Container bind mounts cannot be intercepted to perform additional setup and teardown (e.g. in the case of cross-platform Docker products like Docker for Mac).

Proposal: Mount Point Plugins

Mount point plugins enable end-users, systems developers, and Docker Inc to develop plugins that interpose on mount point setup and teardown in containers. In particular, a number of mount point plugins can be installed in the engine that are consulted, in order, to confirm or deny mounts and potentially change their source location. On container shutdown or stop, relevant mount point plugins would also be called, in reverse order, to teardown the mounts. These teardown or detachment transactions block shutdown until they complete and may change the container exit code if the teardown fails (e.g. synchronizing state fails). Mount point plugins will be able to consume the new consistency flags.

Initially, only bind mounts and volume mounts will be supported (i.e. not tmpfs, network mounts, secrets, or container layers). Mount point plugins will register filters on initialization so that only
applicable plugins are consulted for any given mount. This improves performance by reducing the number of plugin round trips for un-interposed mounts.

This functionality is necessary to fix bind mount inotify events not delivered after container
restart and fs events not working for services as well as Cannot add, remove, add overlapping directory. It will remove one of the use cases of the current Docker API proxy in Docker for Mac and make componentization of that product more tractable.

Finally, by enabling container file system virtualization, mount point plugins will enable a number of interesting use cases like:

• Seamless cross-platform bind mounts
• Sensitive data write auditing
• Unused file analysis
• Access pattern recording
• Unix domain socket file snooping
• Copy-on-write mount points
• Process-based access control
• Non-swappable tmpfs mounts (with tmpfs support)
• Generic file system caching (e.g. deferring flush until container shutdown)
• Special path-dispatched file systems
• Arbitrary ownership mapping and JIT ownership transfer

I've drafted a(n unpublished) patchset implementing this functionality as a way to explore the design space and familiarize myself with the relevant Docker subsystems. I'm interested in your thoughts.

/cc @yallop @cpuguy83 @dnephin