Skip to content

Relax RandomAccess type requirements: make Read/Write methods work with non-seekable files#125512

Open
Copilot wants to merge 14 commits intomainfrom
copilot/relax-randomaccess-requirements
Open

Relax RandomAccess type requirements: make Read/Write methods work with non-seekable files#125512
Copilot wants to merge 14 commits intomainfrom
copilot/relax-randomaccess-requirements

Conversation

Copy link
Contributor

Copilot AI commented Mar 12, 2026

  • Address @adamsitnik feedback: extract NeedsNonOffsetFallback helper method
  • Address @adamsitnik feedback: handle EAGAIN/EWOULDBLOCK in ReadV/WriteV
  • Address @adamsitnik feedback: apply @jozkee's test improvements from PR Make RandomAccess.Read*|Write* methods work with non-seekable files #96711
  • Address @adamsitnik feedback: replace ConsolePal.Read with RandomAccess.Read
  • Address @adamsitnik feedback: merge PartialReads tests into Theory, fix SkipOnPlatform text
  • Fix ProcessWaitingTests: use ReadBlock instead of Read
  • Address @adamsitnik feedback: revert ConsolePal.Browser.cs changes (WASM DllNotFoundException)
  • Address @adamsitnik feedback: restore Interop.Write.cs for browser in csproj
  • Address @jkotas feedback: doc updates should say "In .NET 11 and later versions, ..."
  • Address @adamsitnik feedback: use GetAllowedVectorCount in ReadV/WriteV to cap vector count for IOV_MAX
  • Fix doc comment grammar: "an" → "and" in pal_io.h
  • Address @adamsitnik feedback: fill pipe buffer before write cancellation test to avoid flaky sync completion
Original prompt

This section details on the original issue you should resolve

<issue_title>Relax RandomAccess type requirements, make all Read*|Write* methods work with non-seekable files</issue_title>
<issue_description>In .NET 6 we have introduced new type called RandomAccess that allows for reading and writing to specific file offset.

As of today, all it's Read* and Write* methods throw when given handle points to a non-seekable file like socket or pipe:

ThrowHelper.ThrowNotSupportedException_UnseekableStream();

But it's not a problem for it's internal implementation (used by FileStream):

// The Windows implementation uses ReadFile, which ignores the offset if the handle
// isn't seekable. We do the same manually with PRead vs Read, in order to enable
// the function to be used by FileStream for all the same situations.
int result = handle.CanSeek ?
Interop.Sys.PRead(handle, bufPtr, buffer.Length, fileOffset) :
Interop.Sys.Read(handle, bufPtr, buffer.Length);

And we use it's internal API surface to workaround this limitation is CoreLib (mind the call to RandomAccess.WriteAtOffset (internal), not RandomAccess.Write (public):

RandomAccess.WriteAtOffset(sfh, bytes, 0);

We should relax the public requirements and make RandomAccess work with non-seekable files.

This is going to require:

  • a breaking change doc
  • implementation change (stop throwing exception)
  • test changes (this test shows how to easily create a handle that points to non-seekable pipe)

Everything motioned above should be a single PR. In the same or separate PR, the Unix implementation of overloads that accept multiple buffers should start using readv and writev sys-calls. This should be relatively easy (just search for preadv and pwritev and reuse the patterns)

But it's going to allow to:

<agent_instructions>1. Do exactly what I did in #96711, you can even cherry-pick the git commits if you want.
2. For testing, in RandomAccess_NonSeekable.GetNamedPipeHandlesAsync don't use NamedPipeClientStream or NamedPipeServerStream just use the bran new SafeFileHandle.CreateAnonymousPipe(out readHandle, out writeHandle, bool asyncRead = false, bool asyncWrite = false) API.
3. Search the repo for places where we have SafeFileHandle and could use the RandomAccess API instead of performing direct sys-calls. Use the API there.
4. Make sure you run all the src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests test and they are passing before you send the PR.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@ Tagging subscribers to this area: @dotnet/area-system-io See info in [area-owners.md](https://github.com/dotnet/runtime/blob/main/docs/area-owners.md) if you want to be subscribed.
Issue Details

In .NET 6 we have introduced new type called RandomAccess that allows for reading and writing to specific file offset.

As of today, all it's Read* and Write* methods throw when given handle points to a non-seekable file like socket or pipe:

ThrowHelper.ThrowNotSupportedException_UnseekableStream();

But it's not a problem for it's internal implementation (used by FileStream):

// The Windows implementation uses ReadFile, which ignores the offset if the handle
// isn't seekable. We do the same manually with PRead vs Read, in order to enable
// the function to be used by FileStream for all the same situations.
int result = handle.CanSeek ?
Interop.Sys.PRead(handle, bufPtr, buffer.Length, fileOffset) :
Interop.Sys.Read(handle, bufPtr, buffer.Length);

And we use it's internal API surface to workaround this limitation is CoreLib (mind the call to RandomAccess.WriteAtOffset (internal), not RandomAccess.Write (public):

RandomAccess.WriteAtOffset(sfh, bytes, 0);

We should relax the public requirements and make RandomAccess work with non-seekable files.

This is going to require:

  • a breaking change doc
  • implementation change (stop throwing exception)
  • test changes (this test shows how to easily create a handle that points to non-seekable pipe)

Everything motioned above should be a single PR. In the same or...


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Copilot AI and others added 3 commits March 12, 2026 22:49
…seekable files

- Add native readv/writev sys-call wrappers in pal_io.c and pal_io.h
- Add native entrypoints for ReadV/WriteV in entrypoints.c
- Add managed interop files Interop.ReadV.cs and Interop.WriteV.cs
- Add interop references to System.Private.CoreLib.Shared.projitems
- Modify RandomAccess.cs - relax ValidateInput to allow unseekable handles
- Update XML docs for Read/Write methods (remove NotSupportedException, note offset ignored)
- Modify RandomAccess.Unix.cs - use readv/writev for non-seekable scatter/gather
- Update Base.cs - remove ThrowsNotSupportedExceptionForUnseekableFile test
- Add NonSeekable.cs tests using SafeFileHandle.CreateAnonymousPipe
- Add NonSeekable_AsyncHandles.cs tests
- Update test .csproj to include new test files

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…d of AnonymousPipeServerStream

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
… consistency

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please address my feedback

…ndling

- ReadScatterAtOffset: try PReadV first, fall back to ReadV on ENXIO/ESPIPE
- WriteGatherAtOffset: try PWriteV first, fall back to WriteV on ENXIO/ESPIPE
- SystemNative_ReadV: handle EAGAIN/EWOULDBLOCK with poll loop (like ReadFromNonblocking)
- SystemNative_WriteV: handle EAGAIN/EWOULDBLOCK with poll loop (like WriteToNonblocking)

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 13, 2026

@copilot please address my feedback

All four review items addressed in commit 098316d. All 9741 System.IO.FileSystem.Tests pass.

Copilot AI requested a review from adamsitnik March 13, 2026 08:01
Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please address my feedback

@copilot use the RandomAccess.Write in following places (respect the current EPIPE handling by swallowing specific exception):

private static unsafe void Write(SafeFileHandle fd, byte* bufPtr, int count)
{
while (count > 0)
{
int bytesWritten = Interop.Sys.Write(fd, bufPtr, count);
if (bytesWritten < 0)
{
Interop.ErrorInfo errorInfo = Interop.Sys.GetLastErrorInfo();
if (errorInfo.Error == Interop.Error.EPIPE)
{
return;
}
else
{
throw Interop.GetIOException(errorInfo);
}
}
count -= bytesWritten;
bufPtr += bytesWritten;
}
}

private static unsafe void Write(SafeFileHandle fd, ReadOnlySpan<byte> buffer, bool mayChangeCursorPosition = true)
{
fixed (byte* p = buffer)
{
byte* bufPtr = p;
int count = buffer.Length;
while (count > 0)
{
int cursorVersion = mayChangeCursorPosition ? Volatile.Read(ref s_cursorVersion) : -1;
int bytesWritten = Interop.Sys.Write(fd, bufPtr, count);
if (bytesWritten < 0)
{
Interop.ErrorInfo errorInfo = Interop.Sys.GetLastErrorInfo();
if (errorInfo.Error == Interop.Error.EPIPE)
{
// Broken pipe... likely due to being redirected to a program
// that ended, so simply pretend we were successful.
return;
}
else if (errorInfo.Error == Interop.Error.EAGAIN) // aka EWOULDBLOCK
{
// May happen if the file handle is configured as non-blocking.
// In that case, we need to wait to be able to write and then
// try again. We poll, but don't actually care about the result,
// only the blocking behavior, and thus ignore any poll errors
// and loop around to do another write (which may correctly fail
// if something else has gone wrong).
Interop.Sys.Poll(fd, Interop.PollEvents.POLLOUT, Timeout.Infinite, out Interop.PollEvents triggered);
continue;
}
else
{
// Something else... fail.
throw Interop.GetExceptionForIoErrno(errorInfo);
}
}
else
{
if (mayChangeCursorPosition)
{
UpdatedCachedCursorPosition(bufPtr, bytesWritten, cursorVersion);
}
}
count -= bytesWritten;
bufPtr += bytesWritten;
}
}
}

use the RandomAccess.Read in following places:

private static unsafe int Read(SafeFileHandle fd, Span<byte> buffer)
{
fixed (byte* bufPtr = buffer)
{
int result = Interop.CheckIo(Interop.Sys.Read(fd, bufPtr, buffer.Length));
Debug.Assert(result <= buffer.Length);
return result;
}
}

private static unsafe int Read(SafeFileHandle fd, Span<byte> buffer)
{
fixed (byte* bufPtr = buffer)
{
int result = Interop.CheckIo(Interop.Sys.Read(fd, bufPtr, buffer.Length));
Debug.Assert(result <= buffer.Length);
return result;
}
}

If it does not break the build, remove related references from the project file (the list below is not complete, please verify it on your own):

<Compile Include="$(CommonPath)Interop\Unix\System.Native\Interop.Poll.cs"
Link="Common\Interop\Unix\Interop.Poll.cs" />
<Compile Include="$(CommonPath)Interop\Unix\Interop.Poll.Structs.cs"
Link="Common\Interop\Unix\Interop.Poll.Structs.cs" />
<Compile Include="$(CommonPath)Interop\Unix\System.Native\Interop.Read.cs"
Link="Common\Interop\Unix\Interop.Read.cs" />

<Compile Include="$(CommonPath)Interop\Unix\System.Native\Interop.Poll.cs"
Link="Common\Interop\Unix\Interop.Poll.cs" />
<Compile Include="$(CommonPath)Interop\Unix\Interop.Poll.Structs.cs"
Link="Common\Interop\Unix\Interop.Poll.Structs.cs" />

…ack, use RandomAccess in Console

- Extract NeedsNonOffsetFallback helper method in RandomAccess.Unix.cs
- Improve tests: deduplicate cancellation checks, handle Read returning 0 in loops, use AssertExtensions.SequenceEqual
- Use RandomAccess.Read/Write in ConsolePal.Browser.cs, ConsolePal.Unix.cs, ConsolePal.Wasi.cs
- Remove unused Interop.Read.cs, Interop.Write.cs, Interop.Poll.cs references from System.Console.csproj

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 13, 2026

@copilot please address my feedback

@copilot use the RandomAccess.Write in following places (respect the current EPIPE handling by swallowing specific exception): ...

Done in commit a5d8cb2:

  • ConsolePal.Browser.cs: Replaced Interop.Sys.Write loop with RandomAccess.Write, EPIPE handled via catch (IOException) when (ConvertErrorPlatformToPal(ex.HResult) == EPIPE)
  • ConsolePal.Unix.cs: Read uses RandomAccess.Read, Write uses RandomAccess.Write with EPIPE catch and cursor tracking after the full write. UpdatedCachedCursorPosition now takes ReadOnlySpan<byte> instead of byte*.
  • ConsolePal.Wasi.cs: Both Read and Write use RandomAccess.Read/Write with EPIPE handling
  • Removed Interop.Read.cs, Interop.Write.cs, Interop.Poll.cs, Interop.Poll.Structs.cs from Browser, WASI, and Unix sections of System.Console.csproj

All 9741 System.IO.FileSystem.Tests pass, and System.Console builds for all 8 targets (unix, windows, browser, wasi, android, ios, tvos, and the default).

Copilot AI requested a review from adamsitnik March 13, 2026 09:42
Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot please address my feedback

@adamsitnik adamsitnik force-pushed the copilot/relax-randomaccess-requirements branch from 638b94b to 38df64c Compare March 14, 2026 09:19
@adamsitnik adamsitnik marked this pull request as ready for review March 14, 2026 09:20
Copilot AI review requested due to automatic review settings March 14, 2026 09:20
@adamsitnik adamsitnik closed this Mar 14, 2026
@adamsitnik adamsitnik reopened this Mar 14, 2026
Copilot stopped work on behalf of adamsitnik due to an error March 14, 2026 09:21
Copilot stopped work on behalf of adamsitnik due to an error March 14, 2026 09:22
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR relaxes System.IO.RandomAccess to support non-seekable handles (pipes/sockets) by ignoring fileOffset where seeking isn’t supported, adds Unix readv/writev support for multi-buffer operations, and updates affected call sites and tests across the repo.

Changes:

  • Update RandomAccess validation/docs to allow unseekable handles for Read*/Write* (while keeping seek-only behavior for GetLength/SetLength).
  • Add SystemNative_ReadV/SystemNative_WriteV exports + managed interop, and wire Unix scatter/gather fallbacks to use them.
  • Add/adjust tests for non-seekable handles and update console/process code paths to use RandomAccess.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/native/libs/System.Native/pal_io.h Declares new SystemNative_ReadV / SystemNative_WriteV APIs.
src/native/libs/System.Native/pal_io.c Implements readv/writev wrappers with EINTR + EAGAIN/EWOULDBLOCK polling behavior.
src/native/libs/System.Native/entrypoints.c Exposes SystemNative_ReadV / SystemNative_WriteV for managed P/Invoke.
src/libraries/Common/src/Interop/Unix/System.Native/Interop.ReadV.cs Adds managed Interop.Sys.ReadV P/Invoke.
src/libraries/Common/src/Interop/Unix/System.Native/Interop.WriteV.cs Adds managed Interop.Sys.WriteV P/Invoke.
src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.cs Relaxes validation defaults for unseekable handles; doc updates for offset ignoring.
src/libraries/System.Private.CoreLib/src/System/IO/RandomAccess.Unix.cs Uses ReadV/WriteV for non-offset fallback in scatter/gather paths.
src/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems Includes new interop source files in CoreLib build.
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/System.IO.FileSystem.Tests.csproj Adds new non-seekable test sources to the test project.
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/NonSeekable.cs New coverage for non-seekable pipe handles across sync/async + multi-buffer cases.
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/NonSeekable_AsyncHandles.cs Adds async-handle-specific tests (notably cancellation behavior).
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/Base.cs Removes old “unseekable throws” expectation and exposes canceled-token helper.
src/libraries/System.Runtime/tests/System.IO.FileSystem.Tests/RandomAccess/FlushToDisk.cs Switches unseekable flush test to SafeFileHandle.CreateAnonymousPipe.
src/libraries/System.Console/src/System/ConsolePal.Unix.cs Routes console writes through RandomAccess.Write and updates cursor-position tracking logic.
src/libraries/System.Console/src/System/ConsolePal.Wasi.cs Routes console writes through RandomAccess.Write and keeps broken-pipe suppression behavior.
src/libraries/System.Console/src/System/ConsolePal.Unix.ConsoleStream.cs Uses RandomAccess.Read for console input stream reads.
src/libraries/System.Console/src/System.Console.csproj Removes now-unneeded Unix interop sources for non-browser Unix builds; keeps browser-specific interop.
src/libraries/System.Diagnostics.Process/tests/ProcessWaitingTests.cs Replaces Read with ReadBlock to avoid partial reads in synchronization tests.

You can also share your feedback on Copilot code review. Take the survey.

/// <exception cref="T:System.ArgumentNullException"><paramref name="handle" /> is <see langword="null" />.</exception>
/// <exception cref="T:System.ArgumentException"><paramref name="handle" /> is invalid.</exception>
/// <exception cref="T:System.ObjectDisposedException">The file is closed.</exception>
/// <exception cref="T:System.NotSupportedException">The file does not support seeking (pipe or socket).</exception>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be kept in docs for earlier versions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jkotas are you sure? Does updating XML doc comments in main branch forces an update of all the docs that describe previous releases?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have one copy of the API docs that covers all .NET versions. Ideally, we would be in a place where the source of truth for the API docs is in dotnet/runtime. We are experimenting with whether AI can help with that (see #124227).

Until we get there, the next best thing we can do is to make it easy to port docs in dotnet/runtime to dotnet/dotnet-api-docs . If you include all necessary doc changes in this PR, you can then just tell copilot to create PR in dotnet-api-docs to port the changes.

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…c completion

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
adamsitnik and others added 2 commits March 14, 2026 16:22
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! @stephentoub the PR is ready for review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Relax RandomAccess type requirements, make all Read*|Write* methods work with non-seekable files

4 participants