Skip to content

[macOS] new hard link to file outside watched root not reported #858

@chrisd8088

Description

@chrisd8088

I realize this may not be the ideal place to report this issue, as it might be better addressed to Apple, but I thought I would describe it here first for the benefit of anyone else who comes across the problem.

On macOS Catalina, it appears that Watchman often does not report the creation within a watched directory of a hard link to an existing file outside the watched directory.

I'm currently testing on macOS Catalina 10.15.17 but saw the issue on prior Catalina versions as well, and in a CI context using Catalina, but notably did not see this issue in some CI jobs which used macOS Mojave. I've tried with both Watchman v4.9.0 and with a fresh build of Watchman master.

Here is a short shell script which demonstrates the issue; this reproduces it pretty consistently for me:

#!/bin/bash
mkdir foo
WATCH_PATH="$PWD/foo"

printf "Setting watch:\n"
watchman watch "$WATCH_PATH"
CLOCK=$(watchman --no-pretty clock "$WATCH_PATH" | sed 's/.*"clock":"//' | sed 's/".*//')

touch bar
cd foo
printf "\nShould report no changes yet:\n"
watchman since "$WATCH_PATH" "$CLOCK"

printf "\nCreating hard link.\n"
ln ../bar

printf "\nMay report no changes, despite hard link creation:\n"
watchman since "$WATCH_PATH" "$CLOCK"

printf "\nCreating new file.\n"
touch baz

printf "\nShould report new file, but may not report hard link:\n"
watchman since "$WATCH_PATH" "$CLOCK"

printf "\nCurrent watched directory contents:\n"
ls

printf "\nStopping watch:\n"
watchman watch-del "$WATCH_PATH"
watchman shutdown-server

Here's the output on my system using Watchman v4.9.0, but the results are the same with a fresh build of Watchman from master and using watchman -S to set the watch with the built binary:

Setting watch:
{
    "version": "4.9.0",
    "watch": "/Users/chrisd8088/foo",
    "watcher": "fsevents"
}

Should report no changes yet:
{
    "version": "4.9.0",
    "is_fresh_instance": false,
    "clock": "c:1601347496:85726:1:3",
    "files": []
}

Creating hard link.

May report no changes, despite hard link creation:
{
    "version": "4.9.0",
    "is_fresh_instance": false,
    "clock": "c:1601347496:85726:1:5",
    "files": []
}

Creating new file.

Should report new file, but may not report hard link:
{
    "version": "4.9.0",
    "is_fresh_instance": false,
    "clock": "c:1601347496:85726:1:7",
    "files": [
        {
            "cclock": "c:1601347496:85726:1:6",
            "nlink": 1,
            "dev": 16777220,
            "ctime": 1601347496,
            "new": true,
            "mtime": 1601347496,
            "gid": 20,
            "mode": 33188,
            "size": 0,
            "oclock": "c:1601347496:85726:1:6",
            "ino": 37925764,
            "uid": 501,
            "exists": true,
            "name": "baz"
        }
    ]
}

Current watched directory contents:
bar	baz

Stopping watch:
{
    "version": "4.9.0",
    "watch-del": true,
    "root": "/Users/chrisd8088/foo"
}
{
    "version": "4.9.0",
    "shutdown-server": true
}

One interesting note is that dumping the corresponding data file under my home volume's .fseventsd directory (in this case, /System/Volumes/Data/.fseventsd/) using the handy fse_dump utility, I see two events recorded, one for the creation of the bar file outside the watched directory and one for the creation of the hard link inside the watched directory. Both events share a common node ID, which makes sense.

But looking at a Watchman log generated during a run of the above test with a parallel invocation of watchman --server-encoding=json --persistent log-level debug, there is no fsevents: got /Users/chrisd8088/foo/bar ... entry for the hard link's creation. An initial skim through the code in watcher/fsevents.cpp suggests to me that no events are being inappropriately ignored, and I didn't see any obvious missing registration for link-related events or anything like that.

So I'm a bit suspicious that possibly the FSEvents API is treating these as events on the link source path and not the link target path, and hence not delivering them to Watchman, which has only registered for event notifications on the watched path and its descendants. But I admit that is pure speculation on my part and I haven't dug any further into the issue than I've described above, and I'm also not a macOS expert or anything, either.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions