Summary

There are numerous high-value Docker use cases that could be satisfied if the Docker Volume engine supported volume snapshots and clones. Examples include:

• Rapidly create identical copies of Docker volumes

  • Test & training environments
  • Experimentation on temporary copies of data sets

• Create periodic snapshots of Docker volumes

  • Best practice of keeping regular snapshots of critical data
  • Backup tools that copy snapshots to a separate location

• Create a Docker volume initialized to the contents of an earlier snapshot

  • Recovery of deleted files or earlier file versions

• Rollback a Docker volume to an earlier snapshot

  • Rapid recovery from accidental or malicious deletion or corruption

Plan

We propose to modify Docker (and have already done the work!) to add support for volume snapshots and clones. The following Docker CLI commands are needed and are designed to “feel” identical to the existing volume commands:

• docker volume clone

  • Creates a new volume by cloning an existing volume, either directly or from a specified snapshot
  • volume clone [OPTIONS] VOLUME SOURCE_VOLUME [SOURCE_SNAPSHOT]

• docker volume snapshot create

  • Creates a new snapshot of a volume
  • volume snapshot create [OPTIONS] VOLUME SNAPSHOT

• docker volume snapshot inspect

  • Returns details of one or more snapshots, including volume name, snapshot name, and snapshot creation time
  • volume snapshot inspect [OPTIONS] VOLUME SNAPSHOT [SNAPSHOT...]

• docker volume snapshot ls

  • Lists all the snapshots of a specific volume in tabular form, including snapshot name and creation time
  • volume snapshot ls [OPTIONS] VOLUME

• docker volume snapshot rename

  • Renames an existing snapshot
  • volume snapshot rename [OPTIONS] VOLUME SNAPSHOT NEW_NAME

• docker volume snapshot restore

  • Reverts the contents of a volume (in place) to an earlier snapshot
  • volume snapshot restore [OPTIONS] VOLUME SNAPSHOT

• docker volume snapshot rm

  • Deletes one or more volume snapshots
  • volume snapshot rm [OPTIONS] VOLUME SNAPSHOT [SNAPSHOT...]

Adding this feature requires modification of several components: the Docker client/CLI, the engine, and the volume plugin library.

Not only have we developed this feature, but we even demonstrated it on stage during a DockerCon 2017 session. Please see a video demo here: (link).

Notes

We're posting this proposal to gauge interest, and we are glad to follow up with code, unit tests, etc.

It's worth noting that the Kubernetes community is actively building support for volume snapshots, and volume clones are also a roadmap item, so supporting such basic storage features will soon become table stakes for container orchestrators.

Note also that multiple filesystems used by Docker today, such as btrfs and zfs, support snapshots and could be enhanced to support this feature.

We can leverage extensive experience developing similar functionality in OpenStack (link). For example, when restoring a volume to a snapshot, some storage systems automatically delete all snapshots newer than the one being restored. That may be considered a form of data loss, so the safest approach is to only allow restoring a volume to the newest snapshot available.