Skip to content

Implement SunOS FileSystemWatcher using portfs event ports#124716

Open
gwr wants to merge 9 commits intodotnet:mainfrom
gwr:illumos-fswatch
Open

Implement SunOS FileSystemWatcher using portfs event ports#124716
gwr wants to merge 9 commits intodotnet:mainfrom
gwr:illumos-fswatch

Conversation

@gwr
Copy link
Contributor

@gwr gwr commented Feb 22, 2026

This implementation uses SunOS portfs (event ports) to watch for directory changes. Unlike Linux inotify, portfs can only detect WHEN a directory changes, not WHAT changed. Therefore, this implementation:

  1. Maintains a snapshot of each watched directory's contents
  2. When portfs signals a change, re-reads the directory
  3. Compares new vs old snapshots to detect creates/deletes/modifications
  4. Generates appropriate FileSystemWatcher events

Key implementation details:

  • Uses pinned GC.AllocateArray for file_obj structures required by portfs
  • When watching attributes or timestamps watches directory contents too
  • Each watched objct gets a unique cookie for identification
  • port_get returns the cookie to indicate which object changed
  • Optimized snapshot comparison with sorted single-pass algorithm
  • Supports IncludeSubdirectories by recursively watching child directories
  • Track mtime of the watched directory to avoid missing changes
  • Implement graceful cancellation using PortSend

Native changes (pal_io.c):

  • SystemNative_PortCreate: Opens event port file descriptor
  • SystemNative_PortAssociate: Associates directory with port using file_obj
  • SystemNative_PortGet: Waits for events, returns cookie identifying directory
  • SystemNative_PortSend: Send an event (used in cancellation)
  • SystemNative_PortDissociate: Removes directory association

Performance notes:

  • Less efficient than inotify (must re-read directories on each change)
  • Better than polling (blocks until change occurs)
  • Memory overhead for directory snapshots

Passes all System.IO.FileSystem.Watcher.Tests


Copilot AI review requested due to automatic review settings February 22, 2026 01:54
@dotnet-policy-service dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label Feb 22, 2026
@dotnet-policy-service
Copy link
Contributor

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements FileSystemWatcher support for SunOS (Solaris/illumos) using the portfs (event ports) API. Unlike Linux's inotify which reports specific file changes, portfs only signals that a directory has changed, requiring the implementation to maintain snapshots and perform comparisons to determine what actually changed.

Changes:

  • Native interop layer for portfs operations (port_create, port_associate, port_get, port_send, port_dissociate)
  • SunOS-specific FileSystemWatcher implementation using snapshot-based change detection
  • Hybrid monitoring mode that watches both directories (for name changes) and individual files (for attribute changes)
  • Support for recursive subdirectory monitoring with resource limits (50 subdirs, 1000 files per directory)

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 18 comments.

Show a summary per file
File Description
src/native/libs/configure.cmake CMake configuration for portfs feature detection (has critical bug)
src/native/libs/System.Native/pal_io.h Header declarations for portfs native functions (has spelling errors)
src/native/libs/System.Native/pal_io.c Native implementation of portfs wrapper functions (has critical string lifetime bug)
src/native/libs/System.Native/entrypoints.c Export table entries for new portfs functions
src/native/libs/Common/pal_config.h.in Configuration flag for HAVE_SUNOS_PORTFS
src/libraries/System.IO.FileSystem.Watcher/tests/System.IO.FileSystem.Watcher.Tests.csproj Added illumos and solaris to test target frameworks
src/libraries/System.IO.FileSystem.Watcher/src/System/IO/FileSystemWatcher.SunOS.cs Main SunOS implementation with snapshot comparison and event generation (multiple issues)
src/libraries/System.IO.FileSystem.Watcher/src/System.IO.FileSystem.Watcher.csproj Added SunOS-specific source files and target frameworks
src/libraries/Common/src/Interop/SunOS/portfs/Interop.portfs.cs Managed P/Invoke declarations for portfs (has hardcoded struct size issue)

@gwr gwr force-pushed the illumos-fswatch branch 2 times, most recently from cbe385d to 04c54a8 Compare February 22, 2026 02:19
Copilot AI review requested due to automatic review settings February 22, 2026 02:19
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 13 comments.

Copilot AI review requested due to automatic review settings February 22, 2026 02:54
@gwr gwr force-pushed the illumos-fswatch branch 2 times, most recently from 8b4f6e2 to 3c5ee5f Compare February 22, 2026 03:00
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.

Copilot AI review requested due to automatic review settings February 22, 2026 03:15
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

@jkotas jkotas added the os-SunOS SunOS, currently not officially supported label Feb 22, 2026
Copilot AI review requested due to automatic review settings February 22, 2026 05:40
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

This implementation uses SunOS portfs (event ports) to watch for directory
changes. Unlike Linux inotify, portfs can only detect WHEN a directory
changes, not WHAT changed. Therefore, this implementation:

1. Maintains a snapshot of each watched directory's contents
2. When portfs signals a change, re-reads the directory
3. Compares new vs old snapshots to detect creates/deletes/modifications
4. Generates appropriate FileSystemWatcher events

Key implementation details:

- Uses pinned GC.AllocateArray for file_obj structures required by portfs
- When watching attributes or timestamps watches directory contents too
- Each watched objct gets a unique cookie for identification
- port_get returns the cookie to indicate which object changed
- Optimized snapshot comparison with sorted single-pass algorithm
- Supports IncludeSubdirectories by recursively watching child directories
- Track mtime of the watched directory to avoid missing changes
- Implement graceful cancellation using PortSend

Native changes (pal_io.c):
- SystemNative_PortCreate: Opens event port file descriptor
- SystemNative_PortAssociate: Associates directory with port using file_obj
- SystemNative_PortGet: Waits for events, returns cookie identifying directory
- SystemNative_PortSend: Send an event (used in cancellation)
- SystemNative_PortDissociate: Removes directory association

Performance notes:
- Less efficient than inotify (must re-read directories on each change)
- Better than polling (blocks until change occurs)
- Memory overhead for directory snapshots

Passes 99% of System.IO.FileSystem.Watcher.Tests

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@gwr
Copy link
Contributor Author

gwr commented Feb 23, 2026

Squashed the last dozen updates to simplify local testing.

@gwr
Copy link
Contributor Author

gwr commented Feb 26, 2026

Can anyone help me interpret the CI build analysis?
Is there anything there I need to fix?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, I wondered if I should add functions like this to pal_io.c or create a new pal_portfs.c or pal_sunos.c or something for the SunOS-specific additions. Guidance? Thanks.

@gwr
Copy link
Contributor Author

gwr commented Mar 3, 2026

Still needs a reviewer. Thanks.

@am11
Copy link
Member

am11 commented Mar 3, 2026

To me it looks good. I had a question from @tmds about the overhead #124716 (comment). If the plan is to switch to inotify in the future when it gets stable upstream, we can treat it as a stopgap and merge this. Thanks for pushing it forward @gwr!

@gwr
Copy link
Contributor Author

gwr commented Mar 3, 2026

The only test failure is name="System.IO.Tests.FileSystemWatcherTests_netstandard17.EndInit_ResumesPausedEnableRaisingEvents(setBeforeBeginInit: False)" which has active issue #103584 so I think that's OK.

@tmds
Copy link
Member

tmds commented Mar 3, 2026

This complexity is probably unavoidable given the lack of per-file events, but (perhaps?) still better than not having any watcher support.

It sounds like it is implemented as good as the kernel allows.

Without this, it's likely an app is broken in some way, so I also consider it better to have this than to throw PNSE.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 9 comments.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Comment on lines +424 to +430
// Share parent's cancellation token so entire tree can be cancelled together
var childWatcher = new RunningInstance(watcher, subdirPath, subdirRelativePath, true, _notifyFilters, _cancellationToken);
childWatcher.Start();
lock (_subdirectoryWatchers)
{
_subdirectoryWatchers.Add(childWatcher);
}
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IncludeSubdirectories spawns a new RunningInstance per subdirectory, and each RunningInstance creates its own dedicated background thread (Start() uses new Thread(ProcessEvents)). On large directory trees this can create hundreds/thousands of threads and ports, risking severe resource exhaustion. Consider redesigning to share a single port and event-loop thread across the whole watcher tree (associating multiple directories/files with unique cookies) or otherwise limiting thread creation (e.g., central dispatcher + ThreadPool).

Copilot uses AI. Check for mistakes.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I previously took feeback like this and implemented some resource limits, but was then advised to remove those. I think we can live with this potentially running out of resources.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This copilot feedback is not asking for hard resource limits. It is asking for more efficient implementation that creates fewer threads.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'm working on an revised design. This may take some time.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Comment on lines +445 to +449
new Thread(ProcessEvents)
{
IsBackground = true,
Name = ".NET File Watcher"
}.Start();
Copy link

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IncludeSubdirectories creates a RunningInstance per subdirectory and each RunningInstance.Start() spins up a dedicated Thread. On directory trees with many subdirectories this can create hundreds/thousands of threads and event ports, risking severe resource exhaustion (threads, FDs, memory) and degraded performance. Consider consolidating subdirectory watches onto a shared event port and a single event-processing thread per root watcher (using cookies to demux), or otherwise limiting thread creation.

Suggested change
new Thread(ProcessEvents)
{
IsBackground = true,
Name = ".NET File Watcher"
}.Start();
ThreadPool.UnsafeQueueUserWorkItem(_ => ProcessEvents(), null);

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-System.IO community-contribution Indicates that the PR has been added by a community member os-SunOS SunOS, currently not officially supported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants