Optimization WritePidFile And WriteAddress#8151
Conversation
|
Hi @helen-frank. Thanks for your PR. I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
| tempPath := filepath.Join(filepath.Dir(path), fmt.Sprintf(".%s", filepath.Base(path))) | ||
| f, err := os.OpenFile(tempPath, os.O_RDWR|os.O_CREATE|os.O_EXCL|os.O_SYNC, 0666) | ||
|
|
||
| f, err := os.CreateTemp(filepath.Dir(path), "."+filepath.Base(path)) |
There was a problem hiding this comment.
We lose O_SYNC (for both functions) with this change. https://cs.opensource.google/go/go/+/refs/tags/go1.20.1:src/os/tempfile.go;l=44
| } | ||
| tempPath := filepath.Join(filepath.Dir(path), fmt.Sprintf(".%s", filepath.Base(path))) | ||
| f, err := os.OpenFile(tempPath, os.O_RDWR|os.O_CREATE|os.O_EXCL|os.O_SYNC, 0666) | ||
| f, err := os.CreateTemp(filepath.Dir(path), "."+filepath.Base(path)) |
There was a problem hiding this comment.
Sorry. I don't get your point here. The tempPath has temp prefix, but it doesn't mean it is named randomly.
The WriteAddress is to create temp file named .X and then rename it to X, just in case that the X content is corrupted after the node's power outage or out of disk space.
I can't tell any optimization from your patch. The original OpenFile is very straightforward. But this patch will pick up a unused name and open it with 0600 permission. And then it files other syscall to chmod to 0666....
One syscall -> More than two syscalls.
Please point it out if I miss somethings. Thanks
There was a problem hiding this comment.
Oop, I never submitted my review besides my stray comment. I mostly have the same feedback here, I don't get what this is optimizing. Do you have numbers to share/could you put in the description what the optimization is?
There was a problem hiding this comment.
-
use
os.CreateTemp()to ensure that the temporary file name does not conflict with the existing file name, more secure and reliable -
use
f.Write()to write the pid instead offmt.Fprintf()because the former does not require string formatting and is therefore faster -
use
defer f.Close()to guarantee that the file will be closed if opened in any case
There was a problem hiding this comment.
Thanks for the quick reply.
Basically, if Write{PidFile,Address} fails, it won't retry and the bundle will be deleted. So, it won't be conflict issue.
For the part 2, it can be accepted. Thanks
And for part 3 , it's based on the statements. Since the Close after write, it is safe (at least to me).
There was a problem hiding this comment.
My idea is that temporary data may be written to an existing file, resulting in the original file data being coverage, although the probability is not high.
There was a problem hiding this comment.
It's based on the usecase. In shim, the temporary file is .X. The prefix . means temporary.
But in common case, your change is good. We've already provided it as AtomicWriteFile. I think we can handle the close in defer.
https://github.com/containerd/continuity/blob/v0.3.0/ioutils.go#L28
There was a problem hiding this comment.
got it
(Sorry, I am a containerd novice and do not know this. I am currently learning containerd. I hope I can participate in community development through some of my own ideas.😀
There was a problem hiding this comment.
Neven mind! Welcome to contribute to container project. The code review can recall memory for the code. :p
1300dc9 to
0d4d1ea
Compare
|
/retest |
|
@helen-frank: Cannot trigger testing until a trusted user reviews the PR and leaves an DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
| } | ||
| return os.Rename(tempPath, path) | ||
|
|
||
| return continuity.AtomicWriteFile(path, []byte(strconv.Itoa(pid)), 0666) |
There was a problem hiding this comment.
Sorry for the unclear comment in #8151 (comment)
I don't want to change the original atomic write here, because we don't need random filename here and it won't be called twice for a container.
But we can use WriteString(strconv.Itoa(pid)) here.
There was a problem hiding this comment.
@helen-frank: Cannot trigger testing until a trusted user reviews the PR and leaves an
/ok-to-testmessage.
Just want to ask a question, what thresholds do you need to become a member of the containerd community? @fuweid
( I may have been studying containerd recently, so I may make some contributions and hope to pass the online test faster.😀
There was a problem hiding this comment.
please checkout this one https://github.com/containerd/project/blob/main/GOVERNANCE.md#adding-maintainers~
There was a problem hiding this comment.
got it, It seems that I still have a long way to go.😀
0d4d1ea to
a20bb0a
Compare
|
/retest |
|
@helen-frank: Cannot trigger testing until a trusted user reviews the PR and leaves an DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/retest |
8df33ee to
2880b3a
Compare
| }() | ||
|
|
||
| if _, err = f.WriteString(strconv.Itoa(pid)); err != nil { | ||
| return err |
There was a problem hiding this comment.
Why not just have an f.Close() before this line, adding a defer and extra variable increases code complexity just to handle a single condition. I don't think there is ever a need to add more logic in between the write and close that would necessitate using the defer pattern.
There was a problem hiding this comment.
+1. Since we need to call f.Close() anyway, why don't we keep it the same as before:
_, err = f.WriteString(strconv.Itoa(pid))
closeErr := f.Close()
if err != nil {
return err
}
if closeErr != nil {
return closeErr
}
closeErr is only added to align with the behavior of this PR. We don't even need it if we want to keep the behavior the same as the original code (i.e. ignoring the error from f.Close()).
There was a problem hiding this comment.
got it
(Sorry, there are some other things being dealt with recently.)
| return err | ||
| } | ||
| tempPath := filepath.Join(filepath.Dir(path), fmt.Sprintf(".%s", filepath.Base(path))) | ||
| tempPath := filepath.Join(filepath.Dir(path), "."+filepath.Base(path)) |
There was a problem hiding this comment.
nice. I didn't know this was faster than fmt.Sprintf
Signed-off-by: helen <haitao.zhang@daocloud.io>
2880b3a to
4cc9ea7
Compare
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
This PR is stale because it has been open 90 days with no activity. This PR will be closed in 7 days unless new comments are made or the stale label is removed. |
|
This PR was closed because it has been stalled for 7 days with no activity. |
use os.CreateTemp and defer f.Close, close the file before changing the name