fsevents

package module
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 14, 2024 License: BSD-3-Clause Imports: 10 Imported by: 24

README

FSEvents allows an application to monitor a whole file system or portion of it. FSEvents is only available on macOS.

Godoc: https://pkg.go.dev/github.com/fsnotify/fsevents

Warning: This API should be considered unstable.

Caveats

Known caveats of the macOS FSEvents API which this package uses under the hood:

  • FSEvents returns events for the named path only, so unless you want to follow updates to a symlink itself (unlikely), you should use filepath.EvalSymlinks to get the target path to watch.

  • There is an internal macOS limitation of 4096 watched paths. Watching more paths will result in an error calling Start(). Note that FSEvents is intended to be a recursive watcher by design, it is actually more efficient to watch the containing path than each file in a large directory.

Contributing

FSEvents is currently not well maintained or well tested. Patches will generally get merged if there's a decent signal they're correct. Merely "it compiles" it not enough, because so much of this relies on runtime behaviour. Either a new test should be added, or patches should be tested manually.

Test Scripts

Support for test scripts akin to ones in fsnotify have been added.

All the scripts are in the testdata folder, and test can be ran individually using the pattern:

go test -run TestScript/<path to test script file>

Scripts are written using a simple DSL which runs a few available shell commands, and also allows setting/unsetting the watcher, as well as verifying the output.

Available commands:

    watch path [ops]    # Watch the path, reporting events for it. Nothing is
                        # watched by default. 
    unwatch path        # Stop watching the path.
    watchlist n         # Assert watchlist length.

    stop                # Stop running the script; for debugging.

    touch path
    mkdir [-p] dir
    ln -s target link   # Only ln -s supported.
    mv src dst
    rm [-r] path
    chmod mode path     # Octal only
    sleep time-in-ms

    cat path            # Read path (does nothing with the data; just reads it).
    echo str >>path     # Append "str" to "path".
    echo str >path      # Truncate "path" and write "str".

Output can be verified in the scripts using the events and event flags emitted by FSEvents. Assertions are defined in the Output section of the test script. All the flags in test script assertions are equivalent to the ones defined in wrap.go with the type EventFlags.

The output section format:

Output:
    # Comment
    EventFlag1|EventFlag2  path  # Comment

Really quick FSEvents overview

For those new to FSEvents itself (the Apple technology), here's a really quick primer:

Introduced in macOS v10.5, File System Events is made up of three parts:

  • kernel code that passes raw event data to user space through a special device
  • a daemon that filters this stream and sends notifications
  • a database that stores a record of all changes

The API provides access to this system allowing applications to be notified of changes to directory hierarchies, and also to retrieve a history of changes. File System Events are path / file name based, not file descriptor based. This means that if you watch a directory (path) and it is moved elsewhere your watch does not automatically move with it.

The API uses the concept of a Stream (FSEventStream) which is a configurable stream of notifications. A Stream can recursively monitor many directories. Options like latency which allow the coalescing of events are specific to one stream.

Persistent Monitoring

The database kept by the File System Events API allow a program that is run periodically to check for changes to the file system between runs. This has a real impact on the API and why there are parameters like since. This feature should be considered advisory according to Apple's docs - a full scan should still be run periodically. If an older version of macOS modifies the file system (say, by removing the drive and putting it in another computer) it would not update the database.

The since parameter must be an EventID for the host or device that the stream is for (or the special ALL or NOW values).

To discover the EventID at a specific time the function LastEventBefore can be used (calls FSEventsGetLastEventIdForDeviceBeforeTime); provide dev==0 for a host EventID. Current() (calls FSEventsGetCurrentEventId) returns the most recent EventID.

Apple Docs

Device Streams vs Host Streams

A Host Stream (dev == 0) can monitor directories through the entire system (assuming appropriate permissions). EventIDs increase across all devices (i.e. an older event on any device has a smaller EventID than a newer one except when EventID's rollover). This means that if another disk is added / mounted then there is the potential for a historical EventID conflict.

A Device Stream (pass a Device to New) can monitor directories only on the given device. Because the EventIDs refer only to that device there is no chance of conflict.

For real-time monitoring there aren't any notable advantages to a Device Stream over a Host stream. For persistent monitoring (run a program today to see what changed yesterday) Device Streams are more robust since there can be no EventID conflict.

File Events

Is macOS v10.7 Apple introduced File Events (kFSEventStreamCreateFlagFileEvents aka CF_FILEEVENTS). Prior to this events are delivered for directories only, i.e. if /tmp/subdir/testfile is modified and an FSEventStream is monitoring /tmp then an event would be delivered for the path /tmp/subdir. This tells the application that it should scan /tmp/subdir for changes. It was also possible (in corner cases) that an event for the path /tmp could be created with MUSTSCANSUBDIRS set. This should only happen if events are being dropped, and had to be coalesced to prevent the loss of information.

With CF_FILEEVENTS set (>= macOS v10.7) events are generated with the path specifying the individual files that have been modified. I haven't found explicit mention, but it seems likely that the same caveats apply with respect to coalescing if events are being dropped.

Apple warns that using File Events will cause many more events to be generated (in part, I expect, because they don't coalesce as easily as directory-level events).

Temporal Coalescing

If two files in the same directory are changed in a short period of time (e.g. /tmp/test1 and /tmp/test2) (assuming CF_FILEEVENTS=0) a single event could be delivered to the application specifying that the path /tmp contains changes. There is an efficiency boost when scanning /tmp only once looking for all changes versus scanning it once for test1 and once for test2. The latency parameter enables this kind of temporal coalescing. If latency is set to, say, 1 second then the application will not be notified more than once a second about changes. The flag kFSEventStreamCreateFlagNoDefer aka CF_NODEFER specifies whether the application is notified on the leading or lagging edge of changes.

Structure of fsevents.go

Creating a Stream

EventStream encapsulates an FSEventStream, and allows an arbitrary number of paths to be monitored.

For real-time monitoring an EventStream is created with Resume == false. This means it will not deliver historical events. If Resume == true then all recorded events for the supplied paths since EventId would be supplied first, then realtime events would be supplied as they occur.

The Latency parameter is passed on to the API, and used to throttle / coalesce events - '0' means deliver all events.

Device can be used to create streams specific to a device, (See Device Streams vs Host Streams). Use '0' to create a host stream.

Instantiating an EventStream just allocates memory to store the configuration of the stream. Calling EventStream.Start creates the FSEventStream, and a channel that will be used to report events EventStream.Events (unless one has already been supplied). The EventStream is stored (via an unsafe.Pointer) in the FSEventStream context (so the OS callback has access to it).

Running a Stream

In macOS an application must run a RunLoop (see: [Run Loop Management][https://developer.apple.com/library/mac/documentation/cocoa/Conceptual/Multithreading/RunLoopManagement/RunLoopManagement.html#//apple_ref/doc/uid/10000057i-CH16-SW1]) for each thread that wishes to receive events from the operating system.

An FSEventStream needs to be started, and must be supplied with a Runloop reference to deliver events to.

To accomplish this EventStream.Start creates a goroutine that calls CFRunLoopRun(). The call in to CFRunLoopRun does not return until the runloop is stopped by EventStream.Stop. I believe that the current implementation will work correctly because of the following:

  • There is a many->one mapping from goroutines to os threads (i.e. goroutines do not switch os-threads).

  • The os thread that runs the Run Loop will not be available for other goroutines because CFRunLoopRun does not return.

  • This therefore means that each call to EventStream.Start will consume an OS thread until the runloop is stopped by EventStream.Stop calling CFRunLoopStop. Each Stream will get it's own Run Loop (on it's own OS thread).

Receiving Events

The File System Events API causes a C-callback to be called by delivering an event. The callback extracts / converts the supplied data then posts an array of Events on the channel EventStream.Events. The EventStream is stored in the context parameter of the FSEventStream (supplied back to the callback by the File System Events API).

Stopping a Stream

EventStream.Stop stops and invalidates the stream (as per the File System Events API requirements), then releases the Stream memory, and stops the runloop that was started by EventStream.Start. This returns the EventStream to (almost) the same state as before EventStream.Start was called. The difference is that EventStream.EventID is now set to the last event ID received before the stream was stopped, and EventStream.Events will now reference an instantiated channel (it may have been nil before the call to EventStream.Start).

Memory / Callback Safety

EventStream manages some OS resources (an OS thread, an FSEventStream, an OS runloop, etc). If the EventStream is garbage collected without the being stopped then a panic is likely. This is caused by an FSEventStream callback attempting to reference the EventStream via an unsafe.Pointer. To guard against this situation a finalizer is attached to the EventStream when it is started. When the finalizer triggers (through GC of the EventStream) it stops the stream so that the OS resources can be freed. This prevents the panic.

Documentation

Rendered for darwin/amd64

Overview

Package fsevents provides file system notifications on macOS.

Index

Constants

View Source 
const (
	// NoDefer sends events on the leading edge (for interactive applications).
	// By default events are delivered after latency seconds (for background tasks).
	//
	// Affects the meaning of the EventStream.Latency parameter. If you specify
	// this flag and more than latency seconds have elapsed since
	// the last event, your app will receive the event immediately.
	// The delivery of the event resets the latency timer and any
	// further events will be delivered after latency seconds have
	// elapsed. This flag is useful for apps that are interactive
	// and want to react immediately to changes but avoid getting
	// swamped by notifications when changes are occurring in rapid
	// succession. If you do not specify this flag, then when an
	// event occurs after a period of no events, the latency timer
	// is started. Any events that occur during the next latency
	// seconds will be delivered as one group (including that first
	// event). The delivery of the group of events resets the
	// latency timer and any further events will be delivered after
	// latency seconds. This is the default behavior and is more
	// appropriate for background, daemon or batch processing apps.
	NoDefer = CreateFlags(C.kFSEventStreamCreateFlagNoDefer)

	// WatchRoot requests notifications of changes along the path to
	// the path(s) you're watching. For example, with this flag, if
	// you watch "/foo/bar" and it is renamed to "/foo/bar.old", you
	// would receive a RootChanged event. The same is true if the
	// directory "/foo" were renamed. The event you receive is a
	// special event: the path for the event is the original path
	// you specified, the flag RootChanged is set and event ID is
	// zero. RootChanged events are useful to indicate that you
	// should rescan a particular hierarchy because it changed
	// completely (as opposed to the things inside of it changing).
	// If you want to track the current location of a directory, it
	// is best to open the directory before creating the stream so
	// that you have a file descriptor for it and can issue an
	// F_GETPATH fcntl() to find the current path.
	WatchRoot = CreateFlags(C.kFSEventStreamCreateFlagWatchRoot)

	// IgnoreSelf doesn't send events triggered by the current process (macOS 10.6+).
	//
	// Don't send events that were triggered by the current process.
	// This is useful for reducing the volume of events that are
	// sent. It is only useful if your process might modify the file
	// system hierarchy beneath the path(s) being monitored. Note:
	// this has no effect on historical events, i.e., those
	// delivered before the HistoryDone sentinel event.
	IgnoreSelf = CreateFlags(C.kFSEventStreamCreateFlagIgnoreSelf)

	// FileEvents sends events about individual files, generating significantly
	// more events (macOS 10.7+) than directory level notifications.
	FileEvents = CreateFlags(C.kFSEventStreamCreateFlagFileEvents)
)
View Source 
const (
	// MustScanSubDirs indicates that events were coalesced hierarchically.
	//
	// Your application must rescan not just the directory given in
	// the event, but all its children, recursively. This can happen
	// if there was a problem whereby events were coalesced
	// hierarchically. For example, an event in /Users/jsmith/Music
	// and an event in /Users/jsmith/Pictures might be coalesced
	// into an event with this flag set and path=/Users/jsmith. If
	// this flag is set you may be able to get an idea of whether
	// the bottleneck happened in the kernel (less likely) or in
	// your client (more likely) by checking for the presence of the
	// informational flags UserDropped or KernelDropped.
	MustScanSubDirs EventFlags = EventFlags(C.kFSEventStreamEventFlagMustScanSubDirs)

	// KernelDropped or UserDropped may be set in addition
	// to the MustScanSubDirs flag to indicate that a problem
	// occurred in buffering the events (the particular flag set
	// indicates where the problem occurred) and that the client
	// must do a full scan of any directories (and their
	// subdirectories, recursively) being monitored by this stream.
	// If you asked to monitor multiple paths with this stream then
	// you will be notified about all of them. Your code need only
	// check for the MustScanSubDirs flag; these flags (if present)
	// only provide information to help you diagnose the problem.
	KernelDropped = EventFlags(C.kFSEventStreamEventFlagKernelDropped)

	// UserDropped is related to UserDropped above.
	UserDropped = EventFlags(C.kFSEventStreamEventFlagUserDropped)

	// EventIDsWrapped indicates the 64-bit event ID counter wrapped around.
	//
	// If EventIdsWrapped is set, it means
	// the 64-bit event ID counter wrapped around. As a result,
	// previously-issued event ID's are no longer valid
	// for the EventID field when using EventStream.Resume.
	EventIDsWrapped = EventFlags(C.kFSEventStreamEventFlagEventIdsWrapped)

	// HistoryDone is a sentinel event when retrieving events with EventStream.Resume.
	//
	// Denotes a sentinel event sent to mark the end of the
	// "historical" events sent as a result of specifying
	// EventStream.Resume.
	//
	// After sending all the "historical" events that occurred before now,
	// an event will be sent with the HistoryDone flag set. The client
	// should ignore the path supplied in that event.
	HistoryDone = EventFlags(C.kFSEventStreamEventFlagHistoryDone)

	// RootChanged indicates a change to a directory along the path being watched.
	//
	// Denotes a special event sent when there is a change to one of
	// the directories along the path to one of the directories you
	// asked to watch. When this flag is set, the event ID is zero
	// and the path corresponds to one of the paths you asked to
	// watch (specifically, the one that changed). The path may no
	// longer exist because it or one of its parents was deleted or
	// renamed. Events with this flag set will only be sent if you
	// passed the flag WatchRoot when you created the stream.
	RootChanged = EventFlags(C.kFSEventStreamEventFlagRootChanged)

	// Mount for a volume mounted underneath the path being monitored.
	//
	// Denotes a special event sent when a volume is mounted
	// underneath one of the paths being monitored. The path in the
	// event is the path to the newly-mounted volume. You will
	// receive one of these notifications for every volume mount
	// event inside the kernel (independent of DiskArbitration).
	// Beware that a newly-mounted volume could contain an
	// arbitrarily large directory hierarchy. Avoid pitfalls like
	// triggering a recursive scan of a non-local filesystem, which
	// you can detect by checking for the absence of the MNT_LOCAL
	// flag in the f_flags returned by statfs(). Also be aware of
	// the MNT_DONTBROWSE flag that is set for volumes which should
	// not be displayed by user interface elements.
	Mount = EventFlags(C.kFSEventStreamEventFlagMount)

	// Unmount event occurs after a volume is unmounted.
	//
	// Denotes a special event sent when a volume is unmounted
	// underneath one of the paths being monitored. The path in the
	// event is the path to the directory from which the volume was
	// unmounted. You will receive one of these notifications for
	// every volume unmount event inside the kernel. This is not a
	// substitute for the notifications provided by the
	// DiskArbitration framework; you only get notified after the
	// unmount has occurred. Beware that unmounting a volume could
	// uncover an arbitrarily large directory hierarchy, although
	// macOS never does that.
	Unmount = EventFlags(C.kFSEventStreamEventFlagUnmount)

	// ItemCreated indicates that a file or directory has been created.
	ItemCreated = EventFlags(C.kFSEventStreamEventFlagItemCreated)

	// ItemRemoved indicates that a file or directory has been removed.
	ItemRemoved = EventFlags(C.kFSEventStreamEventFlagItemRemoved)

	// ItemInodeMetaMod indicates that a file or directory's metadata has has been modified.
	ItemInodeMetaMod = EventFlags(C.kFSEventStreamEventFlagItemInodeMetaMod)

	// ItemRenamed indicates that a file or directory has been renamed.
	// TODO is there any indication what it might have been renamed to?
	ItemRenamed = EventFlags(C.kFSEventStreamEventFlagItemRenamed)

	// ItemModified indicates that a file has been modified.
	ItemModified = EventFlags(C.kFSEventStreamEventFlagItemModified)

	// ItemFinderInfoMod indicates the the item's Finder information has been
	// modified.
	// TODO the above is just a guess.
	ItemFinderInfoMod = EventFlags(C.kFSEventStreamEventFlagItemFinderInfoMod)

	// ItemChangeOwner indicates that the file has changed ownership.
	ItemChangeOwner = EventFlags(C.kFSEventStreamEventFlagItemChangeOwner)

	// ItemXattrMod indicates that the files extended attributes have changed.
	ItemXattrMod = EventFlags(C.kFSEventStreamEventFlagItemXattrMod)

	// ItemIsFile indicates that the item is a file.
	ItemIsFile = EventFlags(C.kFSEventStreamEventFlagItemIsFile)

	// ItemIsDir indicates that the item is a directory.
	ItemIsDir = EventFlags(C.kFSEventStreamEventFlagItemIsDir)

	// ItemIsSymlink indicates that the item is a symbolic link.
	ItemIsSymlink = EventFlags(C.kFSEventStreamEventFlagItemIsSymlink)
)

Variables

This section is empty.

Functions

func DeviceForPath 

func DeviceForPath(path string) (int32, error)

DeviceForPath returns the device ID for the specified volume.

func EventIDForDeviceBeforeTime 

func EventIDForDeviceBeforeTime(dev int32, before time.Time) uint64

EventIDForDeviceBeforeTime returns an event ID before a given time.

func GetDeviceUUID 

func GetDeviceUUID(deviceID int32) string

GetDeviceUUID retrieves the UUID required to identify an EventID in the FSEvents database

func LatestEventID 

func LatestEventID() uint64

LatestEventID returns the most recently generated event ID, system-wide.

Types

type CreateFlags 

type CreateFlags uint32

CreateFlags specifies what events will be seen in an event stream.

type Event 

type Event struct {
	// Path holds the path to the item that's changed, relative
	// to its device's root.
	// Use DeviceForPath to determine the absolute path that's
	// being referred to.
	Path string

	// Flags holds details what has happened.
	Flags EventFlags

	// ID holds the event ID.
	//
	// Each event ID comes from the most recent event being reported
	// in the corresponding directory named in the EventStream.Paths field
	// Event IDs all come from a single global source.
	// They are guaranteed to always be increasing, usually in leaps
	// and bounds, even across system reboots and moving drives from
	// one machine to another. If you were to
	// stop processing events from this stream after this event
	// and resume processing them later from a newly-created
	// EventStream, this is the value you would pass for the
	// EventStream.EventID along with Resume=true.
	ID uint64
}

Event represents a single file system notification.

type EventFlags 

type EventFlags uint32

EventFlags passed to the FSEventStreamCallback function. These correspond directly to the flags as described here: https://developer.apple.com/documentation/coreservices/1455361-fseventstreameventflags

type EventStream 

type EventStream struct {

	// Events holds the channel on which events will be sent.
	// It's initialized by EventStream.Start if nil.
	Events chan []Event

	// Paths holds the set of paths to watch, each
	// specifying the root of a filesystem hierarchy to be
	// watched for modifications.
	Paths []string

	// Flags specifies what events to receive on the stream.
	Flags CreateFlags

	// Resume specifies that watching should resume from the event
	// specified by EventID.
	Resume bool

	// EventID holds the most recent event ID.
	//
	// NOTE: this is updated asynchronously by the
	// watcher and should not be accessed while
	// the stream has been started.
	EventID uint64

	// Latency holds the number of seconds the service should wait after hearing
	// about an event from the kernel before passing it along to the
	// client via its callback. Specifying a larger value may result
	// in more effective temporal coalescing, resulting in fewer
	// callbacks and greater overall efficiency.
	Latency time.Duration

	// When Device is non-zero, the watcher will watch events on the
	// device with this ID, and the paths in the Paths field are
	// interpreted relative to the device's root.
	//
	// The device ID is the same as the st_dev field from a stat
	// structure of a file on that device or the f_fsid[0] field of
	// a statfs structure.
	Device int32
	// contains filtered or unexported fields
}

EventStream is the primary interface to FSEvents You can provide your own event channel if you wish (or one will be created on Start). 

es := &EventStream{Paths: []string{"/tmp"}, Flags: 0}
es.Start()
es.Stop()
...

func (*EventStream) Flush 

func (es *EventStream) Flush(sync bool)

Flush flushes events that have occurred but haven't been delivered. If sync is true, it will block until all the events have been delivered, otherwise it will return immediately.

func (*EventStream) Restart 

func (es *EventStream) Restart() error

Restart restarts the event listener. This can be used to change the current watch flags.

func (*EventStream) Start 

func (es *EventStream) Start() error

Start listening to an event stream. This creates es.Events if it's not already a valid channel.

func (*EventStream) Stop 

func (es *EventStream) Stop()

Stop stops listening to the event stream.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.