Skip to content

Reduce memory required for document change processing queues#45477

Merged
sharwell merged 4 commits intodotnet:masterfrom
sharwell:touch-document-lite
Jun 27, 2020
Merged

Reduce memory required for document change processing queues#45477
sharwell merged 4 commits intodotnet:masterfrom
sharwell:touch-document-lite

Conversation

@sharwell
Copy link
Contributor

Substantially improves the "robustness" of the IDE in handling branch switches in large solutions (e.g. Roslyn.sln).

Fixes #45452

@sharwell sharwell requested a review from a team as a code owner June 26, 2020 08:05
Copy link
Member

@tmat tmat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@tmat
Copy link
Member

tmat commented Jun 26, 2020

@CyrusNajmabadi PTAL

// Ensure the recoverable text is saved in temporary storage
result.GetValue();

return result;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the implications of this are not clear to me. @jasonmalinowski do you know this part any better?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we use memory mapped files for this right? is there a concern about what this does for using up address space?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The call to GetValue is just forcing this line to run:

return InitRecoverable(_initialSource!.GetValue(cancellationToken));

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure. but there are two things that aren't clear to me:

  1. how expensive all those .GetValue calls will be...
  2. how expensive this is:
            _initialSource = null;
            _version = textAndVersion.Version;
            _filePath = textAndVersion.FilePath;
            _loadDiagnostic = textAndVersion.LoadDiagnostic;
            _text = new RecoverableText(this, textAndVersion.Text);
            _text.GetValue(CancellationToken.None); // force access to trigger save
            return textAndVersion;

i don't have a good sense of what the tradeoff might be ehre.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how expensive all those .GetValue calls will be...

In this case, the call to _initialSource.GetValue is negligible because _initialSource is a ConstantValueSource<TextAndVersion>. The overall cost of the new code is essentially the cost of:

var storage = _parent._storageService.CreateTemporaryTextStorage(cancellationToken);
await storage.WriteTextAsync(text, cancellationToken).ConfigureAwait(false);

We already perform this operation as part of the TouchDocumentAction processing, so there are two fundamental differences with this approach:

  1. Saving to temporary storage occurs earlier, though it still occurs exactly one time in total.
  2. For the case where processing queue depth triggers a Gen 2 GC, it may cause the weak reference to go away which then needs to be loaded from temporary storage during subsequent event processing. This case is unlikely to be slower than the previous approach because this is the scenario that most often led to crashes prior to this pull request.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the case where processing queue depth triggers a Gen 2 GC

Just to check, are we seeing cases where we have CompilationTrackers with multiple TouchDocumentActions queued to the same document?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the heaps I have suggests yes.

@sharwell sharwell merged commit 4e102e1 into dotnet:master Jun 27, 2020
@ghost ghost added this to the Next milestone Jun 27, 2020
@sharwell sharwell deleted the touch-document-lite branch June 27, 2020 07:11
@dibarbet dibarbet modified the milestones: Next, 16.7.P4 Jun 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TouchDocumentAction sequences have unbounded size

5 participants