-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Description
What is the problem you're trying to solve
When containerd unpacks OCI image v1 tar file into specific filesystems, diff/applier should provide sync-fs option for strong requirement for data persistence.
diff --git a/api/services/diff/v1/diff.proto b/api/services/diff/v1/diff.proto
index 62b308cd902e..990e82ceb1ca 100644
--- a/api/services/diff/v1/diff.proto
+++ b/api/services/diff/v1/diff.proto
@@ -44,6 +44,8 @@ message ApplyRequest {
repeated containerd.types.Mount mounts = 2;
map<string, google.protobuf.Any> payloads = 3;
+ // SyncFs is to synchronize the underlying filesystem containing files.
+ bool sync_fs = 4;
}- Linux platform:
- when using bind-mount or unpack to overlay-dir directly: https://github.com/containerd/containerd/pull/9401/files
- Other filesystems or devices
- Darwin
- FreeBSD
- Windows
This feature is used to fix data loss after power failure. Existing issue reports:
- container not starting in few nodes "standard_init_linux.go:219: exec user process caused: exec format error" #5854
- Cannot pull the image: */snapshots/112/fs: no such file or directory #3369 (comment)
For CRI users, both image store and container writable rootfs share same disk currently.
SyncFS could impact existing running processes who consuming that disk, including containerd itself.
The CRI config provides a experimental setting image_pull_with_sync_fs to fix data loss issue.
Users can set it based on their scenerios.
For image_pull_with_sync_fs = false cases, containerd should provide a debug tool to check which layer has data loss issue and how to mitigate it, which is like a surgery.
-
ctr imagesupport data loss check
Reference
- Darwin SYNC: https://developer.apple.com/documentation/xcode/reducing-disk-writes#Minimize-explicit-storage-synchronization
- FreeBSD SYNC: https://man.freebsd.org/cgi/man.cgi?query=sync&sektion=2&apropos=0&manpath=FreeBSD+14.0-RELEASE+and+Ports (It seems it syncs all the filesystems)
- Windows SYNC: https://learn.microsoft.com/en-us/sysinternals/downloads/sync
Describe the solution you'd like
N/A
Additional context
Why not using a general solution which is to sync regular files and its parent? Please check that description #9401
Metadata
Metadata
Assignees
Labels
Type
Projects
Status