Skip to content

bugfix: updatedAt timestamp file may be empty#2580

Merged
crosbymichael merged 3 commits intocontainerd:masterfrom
HusterWan:zr/fix-read-empty-timestamp
Aug 31, 2018
Merged

bugfix: updatedAt timestamp file may be empty#2580
crosbymichael merged 3 commits intocontainerd:masterfrom
HusterWan:zr/fix-read-empty-timestamp

Conversation

@HusterWan
Copy link
Copy Markdown
Contributor

@HusterWan HusterWan commented Aug 27, 2018

Signed-off-by: Michael Wan zirenwan@gmail.com

we may restart containerd when pulling a image. After containerd restarted, there may occur an error like below when we try to re-pull the image:

could not parse timestamp file /var//lib/containerd/root/io.containerd.content.v1.content/ingest/995d4daf892822f4098d1f9bb585f11b5d716808f067eadca215aa5be0e06bd3/updatedat: parsing time  as "2006-01-02T15:04:05Z07:00"

The reason why occurred this error is because the updatedat file is empty, this PR is to fix this problem: we should allow the timestamp file is empty, and when finding the updatedat file is empty, we should set the updatedat equals to startedat

@HusterWan HusterWan force-pushed the zr/fix-read-empty-timestamp branch from 6718bf4 to caa9572 Compare August 27, 2018 07:26
@codecov-io
Copy link
Copy Markdown

codecov-io commented Aug 27, 2018

Codecov Report

Merging #2580 into master will not change coverage.
The diff coverage is 66.66%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #2580   +/-   ##
=======================================
  Coverage   44.05%   44.05%           
=======================================
  Files          98       98           
  Lines       10130    10130           
=======================================
  Hits         4463     4463           
  Misses       4945     4945           
  Partials      722      722
Flag Coverage Δ
#linux 47.81% <66.66%> (+0.02%) ⬆️
#windows 40.97% <33.33%> (ø) ⬆️
Impacted Files Coverage Δ
snapshots/btrfs/btrfs.go 57.39% <100%> (ø) ⬆️
content/local/store.go 48.76% <100%> (ø) ⬆️
snapshots/native/native.go 43.3% <50%> (ø) ⬆️
snapshots/overlay/overlay.go 52.52% <50%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7cb847b...92243ff. Read the comment docs.

@estesp
Copy link
Copy Markdown
Member

estesp commented Aug 27, 2018

I'm curious about the second fix: returning no error when the file contents are empty. Seems like the caller should be fixing that on error rather than have a function readFileTimestamp hide the underlying problem? I don't have much experience with this particular code, so maybe this is reasonable?

However, I would assume the right solution is to re-write a partially or incorrectly written file w/timestamp when it is discovered to be empty or interrupted during the initial write/create.

@dmcgowan
Copy link
Copy Markdown
Member

However, I would assume the right solution is to re-write a partially or incorrectly written file w/timestamp when it is discovered to be empty or interrupted during the initial write/create.

Ingests should be temporary and we should avoid read side-effects which write. Treating an empty file as an empty timestamp seems reasonable. However I think if this problem is manifesting itself in the first place, we should not be using ioutil.WriteFile and use an atomic implementation which does the write to temp name + rename approach. There is one in continuity which is not exposed (atomicWriteFile) but we could expose one from there (do not use github.com/docker/docker/pkg/ioutils).

@dmcgowan dmcgowan added this to the 1.2 milestone Aug 27, 2018
@HusterWan
Copy link
Copy Markdown
Contributor Author

HusterWan commented Aug 28, 2018

@dmcgowan @estesp thanks for your opinions

If i should first create a pr to expose the atomicWriteFile function for github.com/containerd/continuity; then update the containerd/containerd vendor and use the exposed AtomicWriteFile function ?

@HusterWan
Copy link
Copy Markdown
Contributor Author

add a pr in containerd/continuity @dmcgowan PTLA , thanks

@HusterWan HusterWan force-pushed the zr/fix-read-empty-timestamp branch from caa9572 to 831b155 Compare August 29, 2018 09:28
Signed-off-by: Michael Wan <zirenwan@gmail.com>
@HusterWan
Copy link
Copy Markdown
Contributor Author

Encounter an error when update containerd/continuity package, fs.DiskUsage has changed params: add a context.Context. so all functions used fs.DiskUsage are not worked now.

Signed-off-by: Michael Wan <zirenwan@gmail.com>
@HusterWan HusterWan force-pushed the zr/fix-read-empty-timestamp branch 2 times, most recently from 27d2c64 to 6a08168 Compare August 29, 2018 14:01
@HusterWan
Copy link
Copy Markdown
Contributor Author

HusterWan commented Aug 29, 2018

CI failed: seems to docker.io/library/nginx:latest image changed, i pull this image local also encounter an error:

#pouch pull docker.io/library/nginx:latest
docker.io/library/nginx:latest:                                                resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:db196c86553472e3e6a0b2ef1a6220f1c353bf4f92799c0534e6442bf17188fe: done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 5.7 s                                                                 total:   0.0 B (0.0 B/s)
Error: failed to display progress: failed to pull image: manifest sha256:db196c86553472e3e6a0b2ef1a6220f1c353bf4f92799c0534e6442bf17188fe: not found

E0829 14:16:55.615921   10579 remote_image.go:108] PullImage "nginx:latest" from image service failed: rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/nginx:latest": failed to unpack image on snapshotter overlayfs: no match for platform in manifest sha256:db196c86553472e3e6a0b2ef1a6220f1c353bf4f92799c0534e6442bf17188fe: not found
Aug 29 14:16:55.615: INFO: Unexpected error occurred: rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/nginx:latest": failed to unpack image on snapshotter overlayfs: no match for platform in manifest sha256:db196c86553472e3e6a0b2ef1a6220f1c353bf4f92799c0534e6442bf17188fe: not found

@dmcgowan
Copy link
Copy Markdown
Member

Signed-off-by: Michael Wan <zirenwan@gmail.com>
@HusterWan HusterWan force-pushed the zr/fix-read-empty-timestamp branch from 6a08168 to 92243ff Compare August 30, 2018 01:52
@HusterWan
Copy link
Copy Markdown
Contributor Author

So the CI is happy now, PTAL @estesp @dmcgowan thanks

Copy link
Copy Markdown
Member

@estesp estesp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crosbymichael
Copy link
Copy Markdown
Member

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants