Skip to content

Create ProjectImports.zip in memory instead of on disk#5718

Merged
rainersigwald merged 4 commits intodotnet:masterfrom
asklar:master
Sep 11, 2020
Merged

Create ProjectImports.zip in memory instead of on disk#5718
rainersigwald merged 4 commits intodotnet:masterfrom
asklar:master

Conversation

@asklar
Copy link
Copy Markdown
Contributor

@asklar asklar commented Sep 5, 2020

Fixes #5383

The behavior of /bl with Embed (which is the default) is such that it will create a zip file named msbuild.ProjectImports.zip next to the binlog, and when the build completes it will embed the zip into the binlog and delete the binlog.

This poses a problem for some processes (which can be called as part of a build) that expect all files in the local directory to be accessible. Since MSBuild is holding open this ProjectImports.zip file, a process like React Native's react-native bundle command will fail the build due to sharing violations.

How tested: Validated with a simple setup of 2 csproj, the first includes the second. Verified that the resulting binlog still includes the imported projects. I changed the closing logic a little because ZipArchive requires to be closed before the data gets flushed to the stream.

@dnfadmin
Copy link
Copy Markdown

dnfadmin commented Sep 5, 2020

CLA assistant check
All CLA requirements met.

@dnfadmin
Copy link
Copy Markdown

dnfadmin commented Sep 5, 2020

CLA assistant check
Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

❌ asklar sign now
You have signed the CLA already but the status is still pending? Let us recheck it.

@asklar
Copy link
Copy Markdown
Contributor Author

asklar commented Sep 5, 2020

CC @KirillOsenkov

Copy link
Copy Markdown
Member

@KirillOsenkov KirillOsenkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@KirillOsenkov
Copy link
Copy Markdown
Member

I did some research on binlogs of various sizes and the zip archive size didn't exceed 2.5 MB, so I think there's no need to worry about increased memory consumption, it's negligible.

When I initially wrote this I had no idea how large this archive is going to be in practice for people's builds, but fortunately it looks like those fears were unfounded now that we have a large collection of real-life binlogs (largest I have is 3.2GB with 2.5 MB archive size).

@KirillOsenkov
Copy link
Copy Markdown
Member

@asklar you'll need to sign the agreement here:

image

@asklar
Copy link
Copy Markdown
Contributor Author

asklar commented Sep 5, 2020

thanks @KirillOsenkov I signed it yesterday, it looks like the check has a race condition where it shows unsigned and signed too? :)

image

Copy link
Copy Markdown
Member

@rainersigwald rainersigwald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! @KirillOsenkov disarmed my size concerns.

Co-authored-by: Rainer Sigwald <raines@microsoft.com>
Copy link
Copy Markdown
Member

@benvillalobos benvillalobos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks solid to me. Thanks for the contribution!

@rainersigwald rainersigwald merged commit ec61b6f into dotnet:master Sep 11, 2020
@asklar
Copy link
Copy Markdown
Contributor Author

asklar commented Oct 24, 2021

@rainersigwald I'm trying to find which version of msbuild / VS included my change, I couldn't find it in the notes for any release, could you help me?

@rainersigwald
Copy link
Copy Markdown
Member

From the tags on the commit ec61b6f it looks like it first appeared in 16.8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Default ProjectImports still creates a temporary zip

5 participants