Background and motivation
System.Diagnostics.Process has plenty of design, usability, performance and reliablity issues. If you are curious, please check this gist with an example of how easy it is to get into deadlock with the current APIs when you need to capture process output.
I would like to introduce a set of new APIs to address the most painful problems. The key goals are:
- Being able to avoid deadlocks when reading process output and error, by providing APIs that do all the heavy lifting:
- draining both stdout and stderr at a same time,
- using multiplexing to use a single thread for synchronous overloads,
- waiting for process exit or EOF,
- draining available output and error pipe buffers when process exited, but EOF was not reported (because some granchild process keeps the pipe opened).
- Being able to natively redirect process standard handles to any file (
NUL, pipe, regular file, terminal):
- improved performance,
- fewer reasons to use custom P/Invoke code to launch processes,
- ability to implement process piping.
- Being able to limit process inheritance to a specific set of handles instead of inheriting all handles in the process to avoid accidental handle leaks (and very hard to diagnose bugs).
- Ensuring that the child process does not outlive the parent process to avoid resource leaks ("file in use" etc).
- Exposing process exit status to be able to distinguish between normal exit, kill and signal termination.
API Proposal
I would like to introduce changes to the approved new Process APIs.
The full list of approved APIs is available here.
namespace System.Diagnostics
{
- public sealed class ProcessStartOptions
- {
- public string FileName { get; }
- public IList<string> Arguments { get; set; }
- public IDictionary<string, string?> Environment { get; }
- public string? WorkingDirectory { get; set; }
-
- public IList<SafeHandle> InheritedHandles { get; set; }
- public bool KillOnParentExit { get; set; }
- public bool CreateNewProcessGroup { get; set; }
-
- public ProcessStartOptions(string fileName);
- }
public sealed ProcessStartInfo
{
+ public SafeFileHandle? StandardInputHandle { get; set; }
+ public SafeFileHandle? StandardOutputHandle { get; set; }
+ public SafeFileHandle? StandardErrorHandle { get; set; }
// approved in 2020, but not implemented yet https://github.com/dotnet/runtime/issues/13943#issuecomment-663652191
public bool InheritHandles { get; set; }
+ public IList<SafeHandle> InheritedHandles { get; set; } // name approved for ProcessStartOptions
+ public bool LeaveHandlesOpen { get; set; } = false; // applies to Standard*Handles and InheritedHandles
+ public bool KillOnParentExit { get; set; } // name approved for ProcessStartOptions
+ public TimeSpan? PostExitDrainTimeout { get; set; } // when null, default is used (exact value may change)
+ public bool StartDetached { get; set; }
}
}
namespace Microsoft.Win32.SafeHandles
{
public partial class SafeProcessHandle
{
public int ProcessId { get; }
public static SafeProcessHandle Open(int processId);
- public static SafeProcessHandle Start(ProcessStartOptions options, SafeFileHandle? input, SafeFileHandle? output, SafeFileHandle? error);
- public static SafeProcessHandle StartSuspended(ProcessStartOptions options, SafeFileHandle? input, SafeFileHandle? output, SafeFileHandle? error);
+ public static SafeProcessHandle Start(ProcessStartInfo startInfo);
- public bool Kill();
+ public void Kill();
- public bool KillProcessGroup();
- public void Resume();
- public void Signal(PosixSignal signal);
+ public bool Signal(PosixSignal signal);
- public void SignalProcessGroup(PosixSignal signal);
public ProcessExitStatus WaitForExit();
public bool TryWaitForExit(TimeSpan timeout, [NotNullWhen(true)] out ProcessExitStatus? exitStatus);
public ProcessExitStatus WaitForExitOrKillOnTimeout(TimeSpan timeout);
public Task<ProcessExitStatus> WaitForExitAsync(CancellationToken cancellationToken = default);
public Task<ProcessExitStatus> WaitForExitOrKillOnCancellationAsync(CancellationToken cancellationToken);
}
public partial class SafeFileHandle
{
+ public bool IsInheritable();
}
}And introduce brand new APIs:
namespace System.Diagnostics
{
public readonly struct ProcessOutputLine // struct on purpose (LOTs of allocated instances)
{
public string Content { get; } // it's not nullable, on EOF enumeration just ends
public bool StandardError { get; }
public ProcessOutputLine(string content, bool standardError);
}
public sealed class ProcessTextOutput
{
public ProcessExitStatus ExitStatus { get; }
public string StandardOutput { get; }
public string StandardError { get; }
public int ProcessId { get; }
public ProcessTextOutput(ProcessExitStatus exitStatus, string standardOutput, string standardError, int processId);
}
public class Process : System.ComponentModel.Component, System.IDisposable
{
// All Read* methods wait for either EOF or process exit (then .
public IEnumerable<ProcessOutputLine> ReadAllLines(TimeSpan? timeout = default);
public IAsyncEnumerable<ProcessOutputLine> ReadAllLinesAsync(CancellationToken cancellationToken = default);
public (string standardOutput, string standardError) ReadAllText(TimeSpan? timeout = default);
public Task<(string standardOutput, string standardError)> ReadAllTextAsync(CancellationToken cancellationToken = default);
public (byte[] standardOutput, byte[] standardError) ReadAllBytes(TimeSpan? timeout = default);
public Task<(byte[] standardOutput, byte[] standardError)> ReadAllBytesAsync(CancellationToken cancellationToken = default);
// The Process and its entire tree are killed when the timeout expires.
public static ProcessTextOutput CaptureTextOutput(ProcessStartInfo startInfo, TimeSpan? timeout = default);
public static ProcessTextOutput CaptureTextOutput(string fileName, IList<string>? arguments = null, TimeSpan? timeout = default);
public static Task<ProcessTextOutput> CaptureTextOutputAsync(ProcessStartInfo startInfo, CancellationToken cancellationToken = default);
public static Task<ProcessTextOutput> CaptureTextOutputAsync(string fileName, IList<string>? arguments = null, CancellationToken cancellationToken = default);
public static ProcessExitStatus Run(ProcessStartInfo startInfo, TimeSpan? timeout = default);
public static ProcessExitStatus Run(string fileName, IList<string>? arguments = null, TimeSpan? timeout = default);
public static Task<ProcessExitStatus> RunAsync(ProcessStartInfo startInfo, CancellationToken cancellationToken = default);
public static Task<ProcessExitStatus> RunAsync(string fileName, IList<string>? arguments = null, CancellationToken cancellationToken = default);
// Starts the process, does not wait for its completion, returns process id and cleans up the resources.
public static int StartAndForget(ProcessStartInfo startInfo);
}
}Alternative design
public sealed ProcessStartInfo
{
+ public SafeFileHandle? StandardInputHandle { get; set; }
+ public SafeFileHandle? StandardOutputHandle { get; set; }
+ public SafeFileHandle? StandardErrorHandle { get; set; }
// approved in 2020, but not implemented yet https://github.com/dotnet/runtime/issues/13943#issuecomment-663652191
- public bool InheritHandles { get; set; }
+ public IList<SafeHandle>? AdditionalInheritedHandles { get; set; } = null; // null means inherit everything
+ public bool LeaveHandlesOpen { get; set; } = false; // applies to Standard*Handles and InheritedHandles
}API Usage
Streaming process output lines
This example demonstrates how to read process output line by line, synchronously by using a single thread and draining both stdout and stderr without risking a deadlock.
ProcesStartInfo startInfo = new("ping", "localhost -n 5")
{
RedirectStandardOutput = true,
RedirectStandardError = true
};
using Process process = Process.Start(startInfo);
foreach (ProcessOutputLine line in process.ReadAllLines())
{
if (line.StandardError)
{
Console.ForegroundColor = ConsoleColor.Red;
}
Console.WriteLine(line.Content);
Console.ResetColor();
}Process piping
This example demonstrates piping output from one process to another using anonymous pipes, providing empty input, redirecting to file and discarding error
SafeFileHandle.CreateAnonymousPipe(out SafeFileHandle readPipe, out SafeFileHandle writePipe);
// Start producer with output redirected to the write end of the pipe, no input and discarding error.
ProcessStartInfo producer = new("cmd", ["/c", "echo hello world & echo test line & echo another test" ])
{
StandardInputHandle = File.OpeNullHandle(),
StandardOutputHandle = writePipe,
StandardErrorHandle = File.OpeNullHandle()
}
// Start consumer with input from the read end of the pipe, writing output to file and discarding error.
ProcessStartInfo consumer = new("findstr", "test")
{
StandardInputHandle = readPipe,
StandardOutputHandle = File.OpenHandle("output.txt", FileMode.Create, FileAccess.Write, FileShare.ReadWrite),
StandardErrorHandle = File.OpeNullHandle()
};
using Process producerProcess = Process.Start(producer);
using Process consumerProcess = Process.Start(consumer);
await producerProcess.WaitForExitAsync();
await consumerProcess.WaitForExitAsync();Preventing resource leaks
Following example demonstrates how to start a background process that automatically terminates when the parent process exits to avoid resource leaks. It also releases all the resources associated with the child process (e.g. Process, SafeProcessHandle etc).
SafeFileHandle.CreateAnonymousPipe(out SafeFileHandle readPipe, out SafeFileHandle writePipe, asyncReads: true, asyncWrites: true);
ProcessStartInfo startInfo = new("myBackgroundProcess.exe")
{
KillOnParentExit = true,
InheritedHandles = [writePipe]
};
_ = Process.StartAndForget(startInfo);
// readPipe can be used for IPC
Background and motivation
System.Diagnostics.Processhas plenty of design, usability, performance and reliablity issues. If you are curious, please check this gist with an example of how easy it is to get into deadlock with the current APIs when you need to capture process output.I would like to introduce a set of new APIs to address the most painful problems. The key goals are:
NUL, pipe, regular file, terminal):API Proposal
I would like to introduce changes to the approved new Process APIs.
The full list of approved APIs is available here.
namespace System.Diagnostics { - public sealed class ProcessStartOptions - { - public string FileName { get; } - public IList<string> Arguments { get; set; } - public IDictionary<string, string?> Environment { get; } - public string? WorkingDirectory { get; set; } - - public IList<SafeHandle> InheritedHandles { get; set; } - public bool KillOnParentExit { get; set; } - public bool CreateNewProcessGroup { get; set; } - - public ProcessStartOptions(string fileName); - } public sealed ProcessStartInfo { + public SafeFileHandle? StandardInputHandle { get; set; } + public SafeFileHandle? StandardOutputHandle { get; set; } + public SafeFileHandle? StandardErrorHandle { get; set; } // approved in 2020, but not implemented yet https://github.com/dotnet/runtime/issues/13943#issuecomment-663652191 public bool InheritHandles { get; set; } + public IList<SafeHandle> InheritedHandles { get; set; } // name approved for ProcessStartOptions + public bool LeaveHandlesOpen { get; set; } = false; // applies to Standard*Handles and InheritedHandles + public bool KillOnParentExit { get; set; } // name approved for ProcessStartOptions + public TimeSpan? PostExitDrainTimeout { get; set; } // when null, default is used (exact value may change) + public bool StartDetached { get; set; } } } namespace Microsoft.Win32.SafeHandles { public partial class SafeProcessHandle { public int ProcessId { get; } public static SafeProcessHandle Open(int processId); - public static SafeProcessHandle Start(ProcessStartOptions options, SafeFileHandle? input, SafeFileHandle? output, SafeFileHandle? error); - public static SafeProcessHandle StartSuspended(ProcessStartOptions options, SafeFileHandle? input, SafeFileHandle? output, SafeFileHandle? error); + public static SafeProcessHandle Start(ProcessStartInfo startInfo); - public bool Kill(); + public void Kill(); - public bool KillProcessGroup(); - public void Resume(); - public void Signal(PosixSignal signal); + public bool Signal(PosixSignal signal); - public void SignalProcessGroup(PosixSignal signal); public ProcessExitStatus WaitForExit(); public bool TryWaitForExit(TimeSpan timeout, [NotNullWhen(true)] out ProcessExitStatus? exitStatus); public ProcessExitStatus WaitForExitOrKillOnTimeout(TimeSpan timeout); public Task<ProcessExitStatus> WaitForExitAsync(CancellationToken cancellationToken = default); public Task<ProcessExitStatus> WaitForExitOrKillOnCancellationAsync(CancellationToken cancellationToken); } public partial class SafeFileHandle { + public bool IsInheritable(); } }And introduce brand new APIs:
Alternative design
public sealed ProcessStartInfo { + public SafeFileHandle? StandardInputHandle { get; set; } + public SafeFileHandle? StandardOutputHandle { get; set; } + public SafeFileHandle? StandardErrorHandle { get; set; } // approved in 2020, but not implemented yet https://github.com/dotnet/runtime/issues/13943#issuecomment-663652191 - public bool InheritHandles { get; set; } + public IList<SafeHandle>? AdditionalInheritedHandles { get; set; } = null; // null means inherit everything + public bool LeaveHandlesOpen { get; set; } = false; // applies to Standard*Handles and InheritedHandles }API Usage
Streaming process output lines
This example demonstrates how to read process output line by line, synchronously by using a single thread and draining both stdout and stderr without risking a deadlock.
Process piping
This example demonstrates piping output from one process to another using anonymous pipes, providing empty input, redirecting to file and discarding error
Preventing resource leaks
Following example demonstrates how to start a background process that automatically terminates when the parent process exits to avoid resource leaks. It also releases all the resources associated with the child process (e.g.
Process,SafeProcessHandleetc).