Skip to content

[diff/applier plugin] support sync-fs for data persistence #9497

@fuweid

Description

@fuweid

What is the problem you're trying to solve

When containerd unpacks OCI image v1 tar file into specific filesystems, diff/applier should provide sync-fs option for strong requirement for data persistence.

diff --git a/api/services/diff/v1/diff.proto b/api/services/diff/v1/diff.proto
index 62b308cd902e..990e82ceb1ca 100644
--- a/api/services/diff/v1/diff.proto
+++ b/api/services/diff/v1/diff.proto
@@ -44,6 +44,8 @@ message ApplyRequest {
 	repeated containerd.types.Mount mounts = 2;
 
 	map<string, google.protobuf.Any> payloads = 3;
+	// SyncFs is to synchronize the underlying filesystem containing files.
+	bool sync_fs = 4;
 }

This feature is used to fix data loss after power failure. Existing issue reports:

For CRI users, both image store and container writable rootfs share same disk currently.
SyncFS could impact existing running processes who consuming that disk, including containerd itself.
The CRI config provides a experimental setting image_pull_with_sync_fs to fix data loss issue.
Users can set it based on their scenerios.

For image_pull_with_sync_fs = false cases, containerd should provide a debug tool to check which layer has data loss issue and how to mitigate it, which is like a surgery.

  • ctr image support data loss check

Reference

  1. Darwin SYNC: https://developer.apple.com/documentation/xcode/reducing-disk-writes#Minimize-explicit-storage-synchronization
  2. FreeBSD SYNC: https://man.freebsd.org/cgi/man.cgi?query=sync&sektion=2&apropos=0&manpath=FreeBSD+14.0-RELEASE+and+Ports (It seems it syncs all the filesystems)
  3. Windows SYNC: https://learn.microsoft.com/en-us/sysinternals/downloads/sync

Describe the solution you'd like

N/A

Additional context

Why not using a general solution which is to sync regular files and its parent? Please check that description #9401

Metadata

Metadata

Assignees

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions