Relax RandomAccess type requirements: make Read/Write methods work with non-seekable files#125512
Relax RandomAccess type requirements: make Read/Write methods work with non-seekable files#125512
Conversation
…seekable files - Add native readv/writev sys-call wrappers in pal_io.c and pal_io.h - Add native entrypoints for ReadV/WriteV in entrypoints.c - Add managed interop files Interop.ReadV.cs and Interop.WriteV.cs - Add interop references to System.Private.CoreLib.Shared.projitems - Modify RandomAccess.cs - relax ValidateInput to allow unseekable handles - Update XML docs for Read/Write methods (remove NotSupportedException, note offset ignored) - Modify RandomAccess.Unix.cs - use readv/writev for non-seekable scatter/gather - Update Base.cs - remove ThrowsNotSupportedExceptionForUnseekableFile test - Add NonSeekable.cs tests using SafeFileHandle.CreateAnonymousPipe - Add NonSeekable_AsyncHandles.cs tests - Update test .csproj to include new test files Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…d of AnonymousPipeServerStream Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
… consistency Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
adamsitnik
left a comment
There was a problem hiding this comment.
@copilot please address my feedback
src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Unix.cs
Outdated
Show resolved
Hide resolved
…ndling - ReadScatterAtOffset: try PReadV first, fall back to ReadV on ENXIO/ESPIPE - WriteGatherAtOffset: try PWriteV first, fall back to WriteV on ENXIO/ESPIPE - SystemNative_ReadV: handle EAGAIN/EWOULDBLOCK with poll loop (like ReadFromNonblocking) - SystemNative_WriteV: handle EAGAIN/EWOULDBLOCK with poll loop (like WriteToNonblocking) Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
adamsitnik
left a comment
There was a problem hiding this comment.
@copilot please address my feedback
@copilot use the RandomAccess.Write in following places (respect the current EPIPE handling by swallowing specific exception):
runtime/src/libraries/System.Console/src/System/ConsolePal.Browser.cs
Lines 40 to 61 in 9a2c40b
runtime/src/libraries/System.Console/src/System/ConsolePal.Unix.cs
Lines 981 to 1030 in 9a2c40b
use the RandomAccess.Read in following places:
runtime/src/libraries/System.Console/src/System/ConsolePal.Wasi.cs
Lines 251 to 259 in 9a2c40b
runtime/src/libraries/System.Console/src/System/ConsolePal.Unix.cs
Lines 946 to 954 in 9a2c40b
If it does not break the build, remove related references from the project file (the list below is not complete, please verify it on your own):
runtime/src/libraries/System.Console/src/System.Console.csproj
Lines 84 to 89 in 9a2c40b
runtime/src/libraries/System.Console/src/System.Console.csproj
Lines 236 to 239 in 9a2c40b
src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/NonSeekable.cs
Outdated
Show resolved
Hide resolved
…ack, use RandomAccess in Console - Extract NeedsNonOffsetFallback helper method in RandomAccess.Unix.cs - Improve tests: deduplicate cancellation checks, handle Read returning 0 in loops, use AssertExtensions.SequenceEqual - Use RandomAccess.Read/Write in ConsolePal.Browser.cs, ConsolePal.Unix.cs, ConsolePal.Wasi.cs - Remove unused Interop.Read.cs, Interop.Write.cs, Interop.Poll.cs references from System.Console.csproj Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
Done in commit a5d8cb2:
All 9741 |
adamsitnik
left a comment
There was a problem hiding this comment.
@copilot please address my feedback
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/NonSeekable.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/NonSeekable.cs
Outdated
Show resolved
Hide resolved
638b94b to
38df64c
Compare
There was a problem hiding this comment.
Pull request overview
This PR relaxes System.IO.RandomAccess to support non-seekable handles (pipes/sockets) by ignoring fileOffset where seeking isn’t supported, adds Unix readv/writev support for multi-buffer operations, and updates affected call sites and tests across the repo.
Changes:
- Update
RandomAccessvalidation/docs to allow unseekable handles forRead*/Write*(while keeping seek-only behavior forGetLength/SetLength). - Add
SystemNative_ReadV/SystemNative_WriteVexports + managed interop, and wire Unix scatter/gather fallbacks to use them. - Add/adjust tests for non-seekable handles and update console/process code paths to use
RandomAccess.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/native/libs/System.Native/pal_io.h | Declares new SystemNative_ReadV / SystemNative_WriteV APIs. |
| src/native/libs/System.Native/pal_io.c | Implements readv/writev wrappers with EINTR + EAGAIN/EWOULDBLOCK polling behavior. |
| src/native/libs/System.Native/entrypoints.c | Exposes SystemNative_ReadV / SystemNative_WriteV for managed P/Invoke. |
| src/libraries/Common/src/Interop/Unix/System.Native/Interop.ReadV.cs | Adds managed Interop.Sys.ReadV P/Invoke. |
| src/libraries/Common/src/Interop/Unix/System.Native/Interop.WriteV.cs | Adds managed Interop.Sys.WriteV P/Invoke. |
| src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.cs | Relaxes validation defaults for unseekable handles; doc updates for offset ignoring. |
| src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Unix.cs | Uses ReadV/WriteV for non-offset fallback in scatter/gather paths. |
| src/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems | Includes new interop source files in CoreLib build. |
| src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/System.IO.FileSystem.Tests.csproj | Adds new non-seekable test sources to the test project. |
| src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/NonSeekable.cs | New coverage for non-seekable pipe handles across sync/async + multi-buffer cases. |
| src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/NonSeekable_AsyncHandles.cs | Adds async-handle-specific tests (notably cancellation behavior). |
| src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/Base.cs | Removes old “unseekable throws” expectation and exposes canceled-token helper. |
| src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/FlushToDisk.cs | Switches unseekable flush test to SafeFileHandle.CreateAnonymousPipe. |
| src/libraries/System.Console/src/System/ConsolePal.Unix.cs | Routes console writes through RandomAccess.Write and updates cursor-position tracking logic. |
| src/libraries/System.Console/src/System/ConsolePal.Wasi.cs | Routes console writes through RandomAccess.Write and keeps broken-pipe suppression behavior. |
| src/libraries/System.Console/src/System/ConsolePal.Unix.ConsoleStream.cs | Uses RandomAccess.Read for console input stream reads. |
| src/libraries/System.Console/src/System.Console.csproj | Removes now-unneeded Unix interop sources for non-browser Unix builds; keeps browser-specific interop. |
| src/libraries/System.Diagnostics.Process/tests/ProcessWaitingTests.cs | Replaces Read with ReadBlock to avoid partial reads in synchronization tests. |
You can also share your feedback on Copilot code review. Take the survey.
...ies/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/NonSeekable_AsyncHandles.cs
Outdated
Show resolved
Hide resolved
| /// <exception cref="T:System.ArgumentNullException"><paramref name="handle" /> is <see langword="null" />.</exception> | ||
| /// <exception cref="T:System.ArgumentException"><paramref name="handle" /> is invalid.</exception> | ||
| /// <exception cref="T:System.ObjectDisposedException">The file is closed.</exception> | ||
| /// <exception cref="T:System.NotSupportedException">The file does not support seeking (pipe or socket).</exception> |
There was a problem hiding this comment.
These should be kept in docs for earlier versions.
There was a problem hiding this comment.
@jkotas are you sure? Does updating XML doc comments in main branch forces an update of all the docs that describe previous releases?
There was a problem hiding this comment.
We have one copy of the API docs that covers all .NET versions. Ideally, we would be in a place where the source of truth for the API docs is in dotnet/runtime. We are experimenting with whether AI can help with that (see #124227).
Until we get there, the next best thing we can do is to make it easy to port docs in dotnet/runtime to dotnet/dotnet-api-docs . If you include all necessary doc changes in this PR, you can then just tell copilot to create PR in dotnet-api-docs to port the changes.
Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…c completion Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
This reverts commit a674e0f.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
adamsitnik
left a comment
There was a problem hiding this comment.
LGTM! @stephentoub the PR is ready for review
Original prompt
This section details on the original issue you should resolve
<issue_title>Relax RandomAccess type requirements, make all Read*|Write* methods work with non-seekable files</issue_title>
<issue_description>In .NET 6 we have introduced new type called
RandomAccessthat allows for reading and writing to specific file offset.As of today, all it's
Read*andWrite*methods throw when given handle points to a non-seekable file like socket or pipe:runtime/src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.cs
Line 251 in d033e1a
But it's not a problem for it's internal implementation (used by
FileStream):runtime/src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Unix.cs
Lines 33 to 38 in d033e1a
And we use it's internal API surface to workaround this limitation is
CoreLib(mind the call to RandomAccess.WriteAtOffset (internal), not RandomAccess.Write (public):runtime/src/libraries/System.Private.CoreLib/src/System/IO/File.cs
Line 382 in d033e1a
We should relax the public requirements and make
RandomAccesswork with non-seekable files.This is going to require:
Everything motioned above should be a single PR. In the same or separate PR, the Unix implementation of overloads that accept multiple buffers should start using
readvandwritevsys-calls. This should be relatively easy (just search forpreadvandpwritevand reuse the patterns)But it's going to allow to:
FileStreamto a very lightweightRandomAccesseverywhere where we don't need buffering in BCL (and ofc outside of BCL). Sample perf gains: File.*AllText* optimizations #58167 (comment)</issue_description>
<agent_instructions>1. Do exactly what I did in #96711, you can even cherry-pick the git commits if you want.
2. For testing, in
RandomAccess_NonSeekable.GetNamedPipeHandlesAsyncdon't useNamedPipeClientStreamorNamedPipeServerStreamjust use the bran newSafeFileHandle.CreateAnonymousPipe(out readHandle, out writeHandle, bool asyncRead = false, bool asyncWrite = false)API.3. Search the repo for places where we have
SafeFileHandleand could use theRandomAccessAPI instead of performing direct sys-calls. Use the API there.4. Make sure you run all the
src/libraries/System.Runtime/tests/System.IO.FileSystem.Teststest and they are passing before you send the PR.</agent_instructions>Comments on the Issue (you are @copilot in this section)
@ Tagging subscribers to this area: @dotnet/area-system-io See info in [area-owners.md](https://github.com/dotnet/runtime/blob/main/docs/area-owners.md) if you want to be subscribed.Issue Details
In .NET 6 we have introduced new type called
RandomAccessthat allows for reading and writing to specific file offset.As of today, all it's
Read*andWrite*methods throw when given handle points to a non-seekable file like socket or pipe:runtime/src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.cs
Line 251 in d033e1a
But it's not a problem for it's internal implementation (used by
FileStream):runtime/src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Unix.cs
Lines 33 to 38 in d033e1a
And we use it's internal API surface to workaround this limitation is
CoreLib(mind the call to RandomAccess.WriteAtOffset (internal), not RandomAccess.Write (public):runtime/src/libraries/System.Private.CoreLib/src/System/IO/File.cs
Line 382 in d033e1a
We should relax the public requirements and make
RandomAccesswork with non-seekable files.This is going to require:
Everything motioned above should be a single PR. In the same or...
🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.