This is a regression for as we started seeing this behavior in 17.6 preview, but we did not see this behavior in VS 17.5.
We have a large repo.
Expect that we can complete restore build with less than 64GB of memory, we do not see a memory spike with VS 17.5.
We are seeing a huge 30GB memory spike after the build completes and an out of memory exception.
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:09:43.47
MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
Microsoft.Build.Framework.InternalErrorException: MSB0001: Internal MSBuild Error: Throwing from logger shutdown
=============
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.Compression.ZipArchiveEntry.WriteCentralDirectoryFileHeader()
at System.IO.Compression.ZipArchive.WriteFile()
at System.IO.Compression.ZipArchive.Dispose(Boolean disposing)
at Microsoft.Build.Logging.ProjectImportsCollector.Close(Boolean closeStream)
at Microsoft.Build.Logging.ProjectImportsCollector.GetAllBytes()
at Microsoft.Build.Logging.BinaryLogger.Shutdown()
at Microsoft.Build.BackEnd.Logging.LoggingService.ShutdownLogger(ILogger logger)
at Microsoft.Build.BackEnd.Logging.LoggingService.ShutdownComponent()
at Microsoft.Build.Evaluation.ProjectCollection.ShutDownLoggingService()
---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.Compression.ZipArchiveEntry.WriteCentralDirectoryFileHeader()
at System.IO.Compression.ZipArchive.WriteFile()
at System.IO.Compression.ZipArchive.Dispose(Boolean disposing)
at Microsoft.Build.Logging.ProjectImportsCollector.Close(Boolean closeStream)
at Microsoft.Build.Logging.ProjectImportsCollector.GetAllBytes()
at Microsoft.Build.Logging.BinaryLogger.Shutdown()
at Microsoft.Build.BackEnd.Logging.LoggingService.ShutdownLogger(ILogger logger)
at Microsoft.Build.BackEnd.Logging.LoggingService.ShutdownComponent()
at Microsoft.Build.Evaluation.ProjectCollection.ShutDownLoggingService()
--- End of inner exception stack trace ---
at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Exception innerException, Object[] args)
at Microsoft.Build.Evaluation.ProjectCollection.ShutDownLoggingService()
at Microsoft.Build.Evaluation.ProjectCollection.Dispose(Boolean disposing)
at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Boolean needToValidateProject, String schemaFile, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, TextWriter targetsWriter, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsNotAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler, Boolean interactive, ProjectIsolationMode isolateProjects, GraphBuildOptions graphBuildOptions, Boolean lowPriority, String[] inputResultsCaches, String outputResultsCache, String commandLine)
at Microsoft.Build.CommandLine.MSBuildApp.Execute(String commandLine)
Unhandled Exception: Microsoft.Build.Framework.InternalErrorException: MSB0001: Internal MSBuild Error: Throwing from logger shutdown
=============
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.Compression.ZipArchiveEntry.WriteCentralDirectoryFileHeader()
at System.IO.Compression.ZipArchive.WriteFile()
at System.IO.Compression.ZipArchive.Dispose(Boolean disposing)
at Microsoft.Build.Logging.ProjectImportsCollector.Close(Boolean closeStream)
at Microsoft.Build.Logging.ProjectImportsCollector.GetAllBytes()
at Microsoft.Build.Logging.BinaryLogger.Shutdown()
at Microsoft.Build.BackEnd.Logging.LoggingService.ShutdownLogger(ILogger logger)
at Microsoft.Build.BackEnd.Logging.LoggingService.ShutdownComponent()
at Microsoft.Build.Evaluation.ProjectCollection.ShutDownLoggingService()
---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.Compression.ZipArchiveEntry.WriteCentralDirectoryFileHeader()
at System.IO.Compression.ZipArchive.WriteFile()
at System.IO.Compression.ZipArchive.Dispose(Boolean disposing)
at Microsoft.Build.Logging.ProjectImportsCollector.Close(Boolean closeStream)
at Microsoft.Build.Logging.ProjectImportsCollector.GetAllBytes()
at Microsoft.Build.Logging.BinaryLogger.Shutdown()
at Microsoft.Build.BackEnd.Logging.LoggingService.ShutdownLogger(ILogger logger)
at Microsoft.Build.BackEnd.Logging.LoggingService.ShutdownComponent()
at Microsoft.Build.Evaluation.ProjectCollection.ShutDownLoggingService()
--- End of inner exception stack trace ---
at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Exception innerException, Object[] args)
at Microsoft.Build.Evaluation.ProjectCollection.ShutDownLoggingService()
at Microsoft.Build.Evaluation.ProjectCollection.Dispose(Boolean disposing)
at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Boolean needToValidateProject, String schemaFile, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, TextWriter targetsWriter, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsNotAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler, Boolean interactive, ProjectIsolationMode isolateProjects, GraphBuildOptions graphBuildOptions, Boolean lowPriority, String[] inputResultsCaches, String outputResultsCache, String commandLine)
at Microsoft.Build.CommandLine.MSBuildApp.Execute(String commandLine)
at Microsoft.Build.CommandLine.MSBuildApp.Main()
Issue Description
When running
msbuild /t:restore /blon a large repo we are running out of memory at the end of the build on a 64GB machine. We see a large spike in memory after the console output shows the build as complete, before the process terminates with an out of memory exception.This is a regression for as we started seeing this behavior in 17.6 preview, but we did not see this behavior in VS 17.5.
Steps to Reproduce
We have a large repo.
Expected Behavior
Expect that we can complete restore build with less than 64GB of memory, we do not see a memory spike with VS 17.5.
MSBuild version 17.5.1+f6fdcf537 for .NET Framework
Actual Behavior
We are seeing a huge 30GB memory spike after the build completes and an out of memory exception.
MSBuild version 17.6.0-preview-23164-01+18fe51042 for .NET Framework
Analysis
Passing the additional parameter
/bl:ProjectImports=Noneis a workaround for the issue, but not ideal as we prefer to collect complete binlogs from our build agents.Versions & Configurations
MSBuild version 17.6.0-preview-23164-01+18fe51042 for .NET Framework
17.6.0.16401