Skip to content

Persist command line strings to temporary storage to prevent excess in-memory use.#41955

Merged
9 commits merged intodotnet:masterfrom
CyrusNajmabadi:dumpCommandLine
Feb 27, 2020
Merged

Persist command line strings to temporary storage to prevent excess in-memory use.#41955
9 commits merged intodotnet:masterfrom
CyrusNajmabadi:dumpCommandLine

Conversation

@CyrusNajmabadi
Copy link
Copy Markdown
Contributor

Fixes #37212

@CyrusNajmabadi CyrusNajmabadi requested a review from a team as a code owner February 26, 2020 05:02
Comment on lines +367 to +370
//if (stream.Length == 0)
//{
// throw new ArgumentOutOfRangeException();
//}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 Can we avoid writing to temporary storage if the string is empty?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question! i'd prefer to avoid the extra complexity for what seems like a very small savings.

Are you ok with that?

@CyrusNajmabadi CyrusNajmabadi requested review from a team and sharwell February 26, 2020 07:05
{
return;
}
_commandLineStorage.Dispose();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the first call from ctor, do we dispose the storage that we just allocated and allocate a new one? Would it make sense to keep _commandLineStorage uninitialized until this point?

Copy link
Copy Markdown
Member

@tmat tmat Feb 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, creating a storage for an empty string seems unnecessary, so maybe the storage can be allocated only when the string becomes non-empty?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, i wanted to avoid that complexity :) However, you @sharwell and @ryzngard all seem to want that, so i'll see if i can do it with low complexity :)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to always avoid writing an empty string, but the initialization in the constructor could easily be removed. I'll let others speak for what they want though :)

{
public static void WriteString(this ITemporaryStreamStorage storage, string value)
{
using var memoryStream = new MemoryStream();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the MemoryStream(byte[]) constructor to avoid having to allocate our own StreamWriter?

new MemoryStream(Encoding.UTF8Encoding.GetBytes(value))

Not sure how much that saves us, but it seems desirable to allocate the correct size stream if we know it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i switched to a pooled approach so we can avoid some intermediary garbage. note, we still need the StreamWriter, but that's a tiny object (compared the byte[]s we save with pooling).

var service = new TemporaryStorageServiceFactory.TemporaryStorageService(textFactory);
var storage = service.CreateTemporaryStreamStorage(CancellationToken.None);

// 0 length streams are allowed
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I would hope this comment is true given the name of the test :P

Copy link
Copy Markdown
Contributor

@ryzngard ryzngard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good. Would like to see us avoiding the initial storage allocation in the constructor and to investigate if we get savings using MemoryStream(byte[])

@CyrusNajmabadi
Copy link
Copy Markdown
Contributor Author

Feedback addressed. ptal again. Thanks!

Copy link
Copy Markdown

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-approval

@ghost ghost merged commit fd997c7 into dotnet:master Feb 27, 2020
@CyrusNajmabadi CyrusNajmabadi deleted the dumpCommandLine branch February 27, 2020 02:30
@jinujoseph jinujoseph modified the milestone: 16.6.P1 Feb 28, 2020
@jinujoseph jinujoseph added this to the 16.6.P2 milestone Feb 28, 2020
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

VisualStudioOptionsProcessor holds on to command-line string causing heap growth

5 participants