Ceph » CephFS

Take the example of removing a subvolume which involes moving the subvolume uuid directory to staging (trash) followed by updating the subvolume metadata to reflect its state (e.g.: snapshot-retained). Now, what has been seen in a cluster full case is that the move to stating directory, which essentially is a rename operation succeeds, however, the metadata update operation fails (due to lack to cluster free size to store the update operation). Result is an invalid subvolume state - missing uuid directory (due to removal), but the path component in the subvolume metadata is still pointing to the uuid directory, causing subsequent subvolume operations to fail.

In theory, we could simply reverse the order of operation (update metadata and then remove the subvolume), thereby catching -ENOSPC failure first and erroring out, however, we should also assess if doing some sort of multi-phase commit could mitigate or recover from the intermediate state.