Skip to content

Fixes inotify not notifying on deletion of open files.#205

Closed
vladlosev wants to merge 5 commits intofsnotify:masterfrom
vladlosev:fix-delete-open-files
Closed

Fixes inotify not notifying on deletion of open files.#205
vladlosev wants to merge 5 commits intofsnotify:masterfrom
vladlosev:fix-delete-open-files

Conversation

@vladlosev
Copy link
Copy Markdown

What does this pull request do?

Fixes issue #194: no notification arrives when a file with open handles is deleted. The change essentially stops ignoring the Chmod notification when the file in question does not exist. Such situation can arise on Linux, when a process holds an open handle for the file and the file is being deleted. In that case the watcher receives an IN_ATTRIB notification to indicate the changed count of hard links to the file. But the file will not be actually deleted from disk until all open handles are closed, and correspondingly IN_DELETE_SELF notifications will be postponed until then. With this change, a watcher process will receive a Chmod notification and have a chance to respond.

Where should the reviewer start?

The test explains the scenario well and can be a good place to understand it better.

How should this be manually tested?

A unit test is included and is executed as part of the CI build. The test fails without the code change.

@carlpett
Copy link
Copy Markdown

Is there something blocking the merging of this? It fixes an issue in a utility I'm maintaining, and seems to pass all tests etc?

@nhooyr
Copy link
Copy Markdown
Contributor

nhooyr commented Feb 23, 2018

Just no one has reviewed yet.

@carlpett
Copy link
Copy Markdown

carlpett commented Feb 23, 2018 via email

Copy link
Copy Markdown
Contributor

@nhooyr nhooyr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is sending the Chmod event what we want to do here? How could it be used by apps to circumvent the issue described #205? E.g. how would you know the Chmod is because the file has been deleted? Would the caller be have to check whether the file exists on every Chmod?

inotify.go Outdated
// event was sent after the DELETE. This ignores that MODIFY and
// assumes a DELETE will come or has come if the file doesn't exist.
if !(e.Op&Remove == Remove || e.Op&Rename == Rename) {
// A CHMOD event (triggeded by IN_MODIFY) can arrive when an open file is
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo, should be triggered

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets also link to this issue in the comment.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the issue link and fixed the typo.

@vladlosev vladlosev force-pushed the fix-delete-open-files branch from 3da4b91 to 877935a Compare March 3, 2018 01:43
@vladlosev
Copy link
Copy Markdown
Author

@nhooyr I have outlined the alternatives to fix the issue in #194 (comment). Sending the Chmod event is the simplest solution. Yes, it will require checking for file existence upon receiving the Chmod event if there is a possibility of the file being kept open. But it seems to be the least disruptive option.

@nhooyr
Copy link
Copy Markdown
Contributor

nhooyr commented Mar 13, 2018

#194 (comment)

  1. Convert the IN_ATTRIB notification into Remove for files which do not show up on disk. This will cause fsnotify to always deliver two Remove notifications. That may break some existing clients.

Why don't we go with this but ignore the extra Remove notification? Then fsnotify will behave the way it does everywhere else.

@vladlosev
Copy link
Copy Markdown
Author

It's possible but then we start to get into corner cases such as when the file is moved and then has attributes changed. Handling that will either require more complex (and thus error prone) logic to track file renames or we'll be losing the remove notification.

@nhooyr
Copy link
Copy Markdown
Contributor

nhooyr commented May 12, 2018

After a file is moved, the watcher will not receive any events for it under the old path afaik. So we should be good.

@nhooyr
Copy link
Copy Markdown
Contributor

nhooyr commented May 12, 2018

@vladlosev thank you for the contribution. It's greatly appreciated. I need this in my own project so do you mind if I take this over?

@nhooyr
Copy link
Copy Markdown
Contributor

nhooyr commented May 12, 2018

Actually my solution would not work as if a rename happens following a chmod, we would potentially replace the chmod with a delete whereas we should not have done that.

I'm not convinced there is a good solution for this. I think this is a limitation/mistake in the inotify API.

However, your solution is better than completely ignoring it as, as you said, it at least notifies the client that something has happened.

Copy link
Copy Markdown
Contributor

@nhooyr nhooyr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets add a comment onto the Watcher struct to document this.

}
}

func TestInotifyDeleteOpenFile(t *testing.T) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should have t.Parallel()

@nhooyr
Copy link
Copy Markdown
Contributor

nhooyr commented May 12, 2018

Otherwise, LGTM.

@nhooyr nhooyr mentioned this pull request May 13, 2018
@vladlosev
Copy link
Copy Markdown
Author

@nhooyr feel free to finish this the best way you see fit.

@nathany
Copy link
Copy Markdown
Contributor

nathany commented Oct 5, 2019

@vladlosev Thanks very much for your work on this. It looks like #260 is based on what you've done. Please take a look at that if you have time.

@nathany nathany closed this Oct 5, 2019
@sunxunkang
Copy link
Copy Markdown

thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants