Background and motivation
This is meant to bundle #46663, #27156, and #22838.
There's the usual need of wrapping memory and text-based types in a Stream and given that there are multiple implementations spread across different libraries, it would be ideal to have a standardized API for it directly on .NET.
A few examples of the current options available:
CommunityToolkit.HighPerformance provides AsStream() extension methods for the following:
Memory<byte>
ReadOnlyMemory<byte>
IMemoryOwner<byte>
IBufferWriter<byte>
Nerdbank.Streams also provides AsStream() extensions for the following:
IDuplexPipe
PipeReader/PipeWriter
WebSocket
ReadonlySequence<byte>
IBufferWriter<byte>
While I'm not certain of providing support for all the types listed above, I think a good starting point would be the following based on the evidence shown in the issues I bundled.
API Proposal
Originally proposed by @bartonjs on #82801 (comment).
Static factory methods on System.IO.Stream for CoreLib types, with a C#14 extension type in System.Memory.dll for ReadOnlySequence<byte>.
// In System.Private.CoreLib (System.Runtime.dll)
namespace System.IO
{
public partial class Stream
{
public static Stream FromText(string text, Encoding? encoding = null);
public static Stream FromText(ReadOnlyMemory<char> text, Encoding? encoding = null);
public static Stream FromReadOnlyData(ReadOnlyMemory<byte> data);
public static Stream FromWritableData(Memory<byte> data);
}
}
// In System.Memory.dll (C#14 extension type)
namespace System.IO
{
public static class ReadOnlySequenceStreamExtensions
{
extension(Stream)
{
public static Stream FromReadOnlyData(ReadOnlySequence<byte> sequence);
}
}
}
Pros:
- Unified developer experience - all methods appear as
Stream.From*() via IntelliSense, regardless of underlying assembly
- Solves layering cleanly - C#14 extensions let System.Memory.dll contribute statics to
Stream without modifying CoreLib
- Maximum discoverability (FDG: "frameworks must be usable without the need for documentation")
Cons:
- Not all .NET languages support C#14 extension syntax; callers in those languages would use
ReadOnlySequenceStreamExtensions.FromReadOnlyData(...) instead of the projected Stream.FromReadOnlyData(...)
- Statics on
Stream in CoreLib are only available to .NET 10+ consumers; multi-targeting libraries (netstandard2.0) cannot use them
API Usage
Example 1:
static HttpClient client = new HttpClient();
static void SendString(string str)
{
var request = new HttpRequestMessage(HttpMethod.Post, "contoso.com");
request.Content = new StreamContent(Stream.FromText(str));
client.Send(request);
}
Example 2:
static unsafe void DoStreamOverSpan(ReadOnlySpan<byte> span)
{
fixed (byte* ptr = &MemoryMarshal.GetReference(span))
{
using (MemoryManager<byte> manager = new PointerMemoryManager<byte>(ptr, span.Length))
{
using Stream s = Stream.FromReadOnlyData(manager.Memory);
// Do something with it...
}
}
}
Example 3:
// Pipeline protocol deserialization with System.IO.Pipelines
ReadResult result = await pipeReader.ReadAsync();
ReadOnlySequence<byte> buffer = result.Buffer;
// Stream-only deserializer (e.g., XmlSerializer, legacy APIs)
using var stream = Stream.FromReadOnlyData(buffer);
var config = (AppConfig)new XmlSerializer(typeof(AppConfig)).Deserialize(stream);
pipeReader.AdvanceTo(buffer.End);
Alternative Design 1
Static factory methods on System.IO.Stream for CoreLib types, with a classic extension method on ReadOnlySequence<byte> in System.Memory.dll.
// In System.Private.CoreLib (System.Runtime.dll)
namespace System.IO
{
public partial class Stream
{
public static Stream FromText(string text, Encoding? encoding = null);
public static Stream FromText(ReadOnlyMemory<char> text, Encoding? encoding = null);
public static Stream FromReadOnlyData(ReadOnlyMemory<byte> data);
public static Stream FromWritableData(Memory<byte> data);
}
}
// In System.Memory.dll (classic extension method)
namespace System.Buffers
{
public static class ReadOnlySequenceExtensions
{
public static Stream AsStream(this ReadOnlySequence<byte> sequence);
}
}
Pros:
- Uses only well-established API patterns
- The use of C#14 static extensions in BCL public API is a new pattern still being evaluated by the review board.
AsStream() follows existing precedent (IEnumerable<T>.ToList(), HttpContent.ReadFromJsonAsync() - FDG 5.6)
- Works in all .NET languages (no extension member syntax dependency)
Cons:
- Inconsistent API pattern:
Stream.FromReadOnlyData(memory) vs sequence.AsStream() for the same conceptual operation
AsStream() is less discoverable — requires knowing to look on ReadOnlySequence<byte> instead of Stream
- In case we want to go all-in with
AsStream() for all types, we cannot extend string [FDG 5.6]: Besides injecting an unrelated concept into a fundamental type, since string is one of the most common types, the pollution in IntelliSense would be massive.
Alternative Design 2
Move ReadOnlySequence<T> to System.Private.CoreLib to enable adding the stream wrapper as a static method directly on Stream, consistent with the other CoreLib types.
From #82801 (comment) and ViveliDuCh#1 (comment)
// In System.Private.CoreLib (System.Runtime.dll)
// Note: This requires moving ReadOnlySequence<T> from System.Memory.dll to System.Private.CoreLib
namespace System.IO
{
public partial class Stream
{
public static Stream FromText(string text, Encoding? encoding = null);
public static Stream FromText(ReadOnlyMemory<char> text, Encoding? encoding = null);
public static Stream FromReadOnlyData(ReadOnlyMemory<byte> data);
public static Stream FromReadOnlyData(ReadOnlySequence<byte> sequence);
public static Stream FromWritableData(Memory<byte> data);
}
}
Pros:
- Fully consistent — all methods are real statics on
Stream, single namespace and entry point
- Maximum discoverability via IntelliSense (FDG: "frameworks must be usable without the need for documentation")
Cons:
- Requires moving
ReadOnlySequence<T> to CoreLib - too much change for a single API scenario
- Breaks
ReadOnlySequence<T> availability on older targets (currently available via NuGet on netstandard2.0)
Alternative Design 3
Place all factory methods in a StreamFactory class in System.Memory.dll (which depends on CoreLib where System.IO.Stream lives).
As suggested by @adamsitnik in this comment.
// In System.Memory.dll
namespace System.IO
{
public static class StreamFactory
{
public static Stream FromText(string text, Encoding? encoding = null);
public static Stream FromText(ReadOnlyMemory<char> text, Encoding? encoding = null);
public static Stream FromReadOnlyData(ReadOnlyMemory<byte> data);
public static Stream FromReadOnlyData(ReadOnlySequence<byte> sequence);
public static Stream FromWritableData(Memory<byte> data);
}
}
Pros:
- No layering issues — all methods in one class in System.Memory.dll, which can reference both CoreLib and System.Memory types
- Broader reach potential if System.Memory.dll ships as a NuGet for older frameworks
Cons:
- Lower discoverability — developers look for stream creation on
Stream, not on StreamFactory (FDG: "Factories sacrifice discoverability, usability, and consistency for implementation flexibility")
- No IntelliSense guidance when typing
Stream. - users must know to look for a separate class
- Generally less preferred when constructors/static methods are viable
Locations
The following dotnet/runtime production code could consume this API:
Current Limitation: Libraries multi-targeting netstandard2.0/net462 cannot use the CoreLib-based API for older targets.
Notes
-
ReadOnlySequence<byte> wrapper need has evolved: Since #27156 was filed (2018), most major serializers (System.Text.Json, MessagePack, protobuf-net) now accept ReadOnlySequence<byte> directly. The remaining gap is Stream-only APIs (XmlSerializer, legacy deserializers) and frameworks like HybridCache (#100434) where Stream remains the most common serializer fallback.
PipeReader.AsStream() does not cover this gap — PipeReader.Create(sequence).AsStream() produces a non-seekable stream, while consumers like XmlSerializer and DataContractSerializer use Length/Seek for buffer sizing and format detection. A purpose-built wrapper provides seekable, zero-copy access matching in-memory stream semantics.
-
Partial coverage of #100434: This proposal addresses the ReadOnlySequence<byte> → Stream portion of that issue. The IBufferWriter<byte> → Stream wrapper is not included here and would be a separate proposal.
Open questions
- Layering consensus for
Stream.From vs To/AsStream: Is the overall consensus, regarding the layering interpretation, that string, ReadOnlyMemory<char>, Memory<byte>, and ReadOnlyMemory<byte> get Stream.From* (as types more primitive than Stream) and that ReadOnlySequence<byte> gets To/AsStream (as a type at the same level or higher)?
- Naming consistency: Does the
Stream.From* naming present a coherent mental model for stream creation across text, memory, and sequences?
- ReadOnlySequence layering consensus: What's the consensus on the layering discussion for
ReadOnlySequence<byte>? See remaining concerns from review.
- Should the
ReadOnlySequence<byte> overload use a C#14 static extension on Stream or a classic extension method in System.Memory.dll? See PR discussion, preliminary review, and subsequent discussion.
- If C#14 static extension approach is used, the fallback invocation for non-C#14 languages becomes
ReadOnlySequenceStreamExtensions.FromReadOnlyData(...), does the extension class name need to follow a naming convention that preserves discoverability alongside Stream.FromReadOnlyData(...)?
- If classic extension method is preferred, is the naming asymmetry (
Stream.FromReadOnlyData(ROM<byte>) vs sequence.AsStream()) acceptable?
- [Based on preliminary api review] Parameter naming: Should the
encoding parameter in FromText be renamed to streamEncoding to disambiguate its purpose? i.e. Stream.FromText(myString, streamEncoding: Encoding.Latin1) - The parameter specifies the encoding used for the bytes in the resulting stream, not the encoding of the source text.
- Encoding default (UTF-8 vs UTF-16): Is the
encoding parameter as nullable with a UTF-8 default correct in FromText(string text, Encoding? encoding = null)?
- Seekability of
ReadOnlyTextStream: Should the text stream returned by FromText be seekable? The current prototype is seekable with the tradeoff.
- Exposing custom stream types: Do we want to expose the custom stream types used to back the static factory methods being proposed in this API (e.g.,
ReadOnlyTextStream), or should they remain internal?
- General
Stream.FromText() / MemoryStream.FromText(): Is there a need or desire for [1] a general Stream.FromText() on Stream that returns a stream without specifying the concrete type (not generic), and [2] something like MemoryStream.FromText() which explicitly returns a MemoryStream, or similar for other derived types?
- Should
FromText expose an optional bufferSize parameter (following StreamReader's constructor pattern), or keep the encoding buffer size as a fixed internal detail?
Risks
No response
Background and motivation
This is meant to bundle #46663, #27156, and #22838.
There's the usual need of wrapping memory and text-based types in a Stream and given that there are multiple implementations spread across different libraries, it would be ideal to have a standardized API for it directly on .NET.
A few examples of the current options available:
CommunityToolkit.HighPerformance provides
AsStream()extension methods for the following:Memory<byte>ReadOnlyMemory<byte>IMemoryOwner<byte>IBufferWriter<byte>Nerdbank.Streams also provides
AsStream()extensions for the following:IDuplexPipePipeReader/PipeWriterWebSocketReadonlySequence<byte>IBufferWriter<byte>While I'm not certain of providing support for all the types listed above, I think a good starting point would be the following based on the evidence shown in the issues I bundled.
API Proposal
Originally proposed by @bartonjs on #82801 (comment).
Static factory methods on
System.IO.Streamfor CoreLib types, with a C#14 extension type in System.Memory.dll forReadOnlySequence<byte>.Pros:
Stream.From*()via IntelliSense, regardless of underlying assemblyStreamwithout modifying CoreLibCons:
ReadOnlySequenceStreamExtensions.FromReadOnlyData(...)instead of the projectedStream.FromReadOnlyData(...)Streamin CoreLib are only available to .NET 10+ consumers; multi-targeting libraries (netstandard2.0) cannot use themAPI Usage
Example 1:
Example 2:
Example 3:
Alternative Design 1
Static factory methods on
System.IO.Streamfor CoreLib types, with a classic extension method onReadOnlySequence<byte>in System.Memory.dll.Pros:
AsStream()follows existing precedent (IEnumerable<T>.ToList(),HttpContent.ReadFromJsonAsync()- FDG 5.6)Cons:
Stream.FromReadOnlyData(memory)vssequence.AsStream()for the same conceptual operationAsStream()is less discoverable — requires knowing to look onReadOnlySequence<byte>instead ofStreamAsStream()for all types, we cannot extendstring[FDG 5.6]: Besides injecting an unrelated concept into a fundamental type, sincestringis one of the most common types, the pollution in IntelliSense would be massive.Alternative Design 2
Move
ReadOnlySequence<T>to System.Private.CoreLib to enable adding the stream wrapper as a static method directly onStream, consistent with the other CoreLib types.From #82801 (comment) and ViveliDuCh#1 (comment)
Pros:
Stream, single namespace and entry pointCons:
ReadOnlySequence<T>to CoreLib - too much change for a single API scenarioReadOnlySequence<T>availability on older targets (currently available via NuGet on netstandard2.0)Alternative Design 3
Place all factory methods in a
StreamFactoryclass in System.Memory.dll (which depends on CoreLib whereSystem.IO.Streamlives).As suggested by @adamsitnik in this comment.
Pros:
Cons:
Stream, not onStreamFactory(FDG: "Factories sacrifice discoverability, usability, and consistency for implementation flexibility")Stream.- users must know to look for a separate classLocations
The following dotnet/runtime production code could consume this API:
Current Limitation: Libraries multi-targeting
netstandard2.0/net462cannot use the CoreLib-based API for older targets.Notes
ReadOnlySequence<byte> wrapper need has evolved: Since #27156 was filed (2018), most major serializers (
System.Text.Json,MessagePack,protobuf-net) now acceptReadOnlySequence<byte>directly. The remaining gap is Stream-only APIs (XmlSerializer, legacy deserializers) and frameworks like HybridCache (#100434) whereStreamremains the most common serializer fallback.PipeReader.AsStream()does not cover this gap —PipeReader.Create(sequence).AsStream()produces a non-seekable stream, while consumers likeXmlSerializerandDataContractSerializeruseLength/Seekfor buffer sizing and format detection. A purpose-built wrapper provides seekable, zero-copy access matching in-memory stream semantics.Partial coverage of #100434: This proposal addresses the
ReadOnlySequence<byte>→Streamportion of that issue. TheIBufferWriter<byte>→Streamwrapper is not included here and would be a separate proposal.Open questions
Stream.FromvsTo/AsStream: Is the overall consensus, regarding the layering interpretation, thatstring, ReadOnlyMemory<char>, Memory<byte>, and ReadOnlyMemory<byte>getStream.From*(as types more primitive than Stream) and thatReadOnlySequence<byte>getsTo/AsStream(as a type at the same level or higher)?Stream.From*naming present a coherent mental model for stream creation across text, memory, and sequences?ReadOnlySequence<byte>? See remaining concerns from review.ReadOnlySequence<byte>overload use a C#14 static extension on Stream or a classic extension method in System.Memory.dll? See PR discussion, preliminary review, and subsequent discussion.ReadOnlySequenceStreamExtensions.FromReadOnlyData(...), does the extension class name need to follow a naming convention that preserves discoverability alongsideStream.FromReadOnlyData(...)?Stream.FromReadOnlyData(ROM<byte>)vssequence.AsStream()) acceptable?encodingparameter inFromTextbe renamed tostreamEncodingto disambiguate its purpose? i.e.Stream.FromText(myString, streamEncoding: Encoding.Latin1)- The parameter specifies the encoding used for the bytes in the resulting stream, not the encoding of the source text.encodingparameter as nullable with a UTF-8 default correct inFromText(string text, Encoding? encoding = null)?ReadOnlyTextStream: Should the text stream returned byFromTextbe seekable? The current prototype is seekable with the tradeoff.ReadOnlyTextStream), or should they remain internal?Stream.FromText()/MemoryStream.FromText(): Is there a need or desire for [1] a generalStream.FromText()onStreamthat returns a stream without specifying the concrete type (not generic), and [2] something likeMemoryStream.FromText()which explicitly returns aMemoryStream, or similar for other derived types?FromTextexpose an optionalbufferSizeparameter (followingStreamReader's constructor pattern), or keep the encoding buffer size as a fixed internal detail?Risks
No response