Inotify: fix watcher.watches and paths leaks#73
Inotify: fix watcher.watches and paths leaks#73chamaken wants to merge 3 commits intofsnotify:masterfrom
Conversation
Watcher.watches and paths are not deleted when entry was removed automatically. Deleting those entry at ignoreLinux() to fix it. This changes removing invalid watch name, it (should) always returns ``can't remove non-existent...''.
|
Sure thing, I'll take a look at this tonight or tomorrow. Thanks. |
|
Don't have much time (I'm on holiday!) but I've got a few minutes. |
inotify.go
Outdated
There was a problem hiding this comment.
Not a big fan that all these comments were removed.
|
Sorry, I don't have a ton of time :( I'm a bit confused about what the patch is meant to fix. Could you elaborate? |
|
Thank you for taking your time. |
|
Thanks. There was some stuff in there I thought might save some time looking stuff up in the future :) |
|
LGTM. I don't have a linux machine handy so I can't play around with it, but it looks good. |
|
Thanks @chamaken. I can take a look sometime over the next few days. |
|
Sorry, I still haven't checked this. :-( |
|
Merged. Thanks. |
|
This change e253224 is making some of my unit tests fail non-deterministically. After adding some tracing to the code, here's what I'm seeing (both on Linux 3.4.43 and 4.1.18): When
However I'm seeing that in two of my unit tests sometimes (more than 50% of the time) there is no Has anybody seen this before? |
|
OK I think I got it... With this change it's no longer safe to consume Events from the same goroutine that calls func (shared *InotifyTracker) run() {
// ...
for {
select {
case winfo := <-shared.watch:
shared.error <- shared.addWatch(winfo.fname)
case fname := <-shared.remove:
shared.removeWatch(fname) // <-------- no longer legit
// ...
}
}Removing the watch leads to a deadlock: we're trying to call |
|
Thanks for investigating this @tsuna. Would you mind opening a new issue so this doesn't get lost? |
Calling watcher.Remove() from the run() goroutine is now problematic, because with the change made in fsnotify/fsnotify#73 Remove() can now take an arbitrary amount of time, which means we can deadlock if run() is waiting for fsnotify to acknowledge the removal and fsnotify is trying to send an unrelated Event. So instead we now do part of the cleanup, including calling Remove(), synchronously, in the goroutine trying to unsubscribe. This fixes hpcloud#75. Change-Id: I346c9eecc34b2378312b07b3c3efc41616b95380
Calling watcher.Remove() from the run() goroutine is now problematic, because with the change made in fsnotify/fsnotify#73 Remove() can now take an arbitrary amount of time, which means we can deadlock if run() is waiting for fsnotify to acknowledge the removal and fsnotify is trying to send an unrelated Event. So instead we now do part of the cleanup, including calling Remove(), synchronously, in the goroutine trying to unsubscribe. This fixes hpcloud#75. Thanks to Aaron Beitch for the fix. Change-Id: I346c9eecc34b2378312b07b3c3efc41616b95380
|
@nathany I'm happy to open an issue if you deem that this new code creates an undesirable side effect that needs to be fixed. As you can see above, I've submitted a PR to workaround this quirk in the |
|
Yes. I think we should take another look at this code change. Adding and removing watches from the same goroutine should not cause deadlocks. |
|
I filed bug #123. Thanks. |
Calling watcher.Remove() from the run() goroutine is now problematic, because with the change made in fsnotify/fsnotify#73 Remove() can now take an arbitrary amount of time, which means we can deadlock if run() is waiting for fsnotify to acknowledge the removal and fsnotify is trying to send an unrelated Event. So instead we now do part of the cleanup, including calling Remove(), synchronously, in the goroutine trying to unsubscribe. This fixes #75. Thanks to Aaron Beitch for the fix. Change-Id: I346c9eecc34b2378312b07b3c3efc41616b95380
This partially reverts pull request fsnotify#73, but helps prevent bug fsnotify#123, which is a lot worse than the problem that was fixed by fsnotify#73.
Thanks to @PieterD Watcher.Close() can stop goroutine cleanly.
But I think watches and paths map still leaks, and feel a bit strange
Remove() invalid filename returns differently between at first time and
after. I tried to fix those, would you revew the patches?