Skip to content

Improvement to completion for unimported extension methods#45947

Merged
genlu merged 8 commits intodotnet:masterfrom
genlu:mtCompletion
Jul 15, 2020
Merged

Improvement to completion for unimported extension methods#45947
genlu merged 8 commits intodotnet:masterfrom
genlu:mtCompletion

Conversation

@genlu
Copy link
Member

@genlu genlu commented Jul 13, 2020

  1. Run the majority of the computation in parallel (1 task per reference which might contain applicable extension methods)

Some quick measurement using telemetry events shows a 70% improvement over unparalleled version in average (on a 4 core machine). The test is done by repeatedly trigger and commit completion for members in EditorFeatures project. Note this is only based on the actual computation time, i.e. time spent in 'GetUnimportedExtensionMethodsInCurrentProcessAsync', which doesn't include the seemingly random overhead of OOP communication.

All the parallelized computation runs in OOP.

  1. Show overload info in description. This also avoid serializing items which would be de-duplicated away later.

image

genlu added 4 commits July 10, 2020 14:14
Also returns partial results when not all indices are available
by aggregating overloads, also reduce the number of items returned
@genlu genlu added the Area-IDE label Jul 13, 2020
@genlu genlu requested a review from a team as a code owner July 13, 2020 22:57
@CyrusNajmabadi
Copy link
Contributor

  1. What sort of improvement do we see here?
  2. is this in-proc or oop?

{
}

private const string NonBreakingSpaceString = "\x00A0";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the char used in the product, took me some debugging to figure out why my tests were failing.

builder.Add(MethodKey, extensionMethodData.Value.methodSymbolKey);
builder.Add(ReceiverKey, extensionMethodData.Value.receiverTypeSymbolKey);

if (extensionMethodData.Value.overloadCount > 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would maybe have been nice to have this in another pr.

Copy link
Member Author

@genlu genlu Jul 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All changes related to this is in the last commit, skipping it would make it easier to review


var indicesComplete = true;
using var _1 = PooledDictionary<INamespaceSymbol, string>.GetInstance(out var namespaceNameCache);
using var _2 = PooledDictionary<(string containingNamespace, string methodName, bool isGeneric), (IMethodSymbol bestSymbol, int overloadCount)>.GetInstance(out var overloadMap);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seriously on the fence about asking for some named structs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ValueTuple make it easier to use them as dictionary key

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

understood. still on the fence :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When will we start dogfooding records? :)

@CyrusNajmabadi
Copy link
Contributor

my brain got tired 3/4ths through. i'll have to review the rest later. lmk if you're making more changes.

@genlu
Copy link
Member Author

genlu commented Jul 14, 2020

@CyrusNajmabadi

What sort of improvement do we see here?

Some quick measurement using telemetry events shows a 70% improvement over unparalleled version in average (on a 4 core machine). The test is done by repeatedly trigger and commit completion for members in EditorFeatures project. Note this is only based on the actual computation time, i.e. time spent in 'GetUnimportedExtensionMethodsInCurrentProcessAsync', which doesn't include the seemingly random overhead of OOP communication.

is this in-proc or oop?

everything here is OOP

lmk if you're making more changes.

No more changes planned.

@CyrusNajmabadi
Copy link
Contributor

Some quick measurement using telemetry events shows a 70% improvement over unparalleled version in average (on a 4 core machine). The test is done by repeatedly trigger and commit completion for members in EditorFeatures project. Note this is only based on the actual computation time, i.e. time spent in 'GetUnimportedExtensionMethodsInCurrentProcessAsync', which doesn't include the seemingly random overhead of OOP communication.

Awesome. in terms of actual wallclock time, can you put that in perspective? i.e. MS before and MS after? Thanks!

@CyrusNajmabadi
Copy link
Contributor

everything here is OOP

Great. No concerns about multithreading here then.

@genlu
Copy link
Member Author

genlu commented Jul 14, 2020

in terms of actual wallclock time, can you put that in perspective? i.e. MS before and MS after?

In the scenario I tested, it's roughly 70 ms vs 210ms (average per completion session)

@CyrusNajmabadi
Copy link
Contributor

In the scenario I tested, it's roughly 70 ms vs 210ms (average per completion session)

That's fantastic!

@genlu
Copy link
Member Author

genlu commented Jul 14, 2020

@CyrusNajmabadi Please let me know if you have more comments. Thanks!

@CyrusNajmabadi
Copy link
Contributor

looking now.

ISet<string> namespaceFilter,
ConcurrentDictionary<ITypeSymbol, bool> checkedReceiverTypes,
StatisticCounter counter,
CancellationToken cancellationToken)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i would reocmmend in a future pr breaking this into a class. I think 10 params is def the trigger point that indicates things have gone too far.

// queue a new task only if the previous task is completed. This is to avoid
// queueing calculation for the same set of references repeatedly while
// index is being constrcuted, which might take some time.
if (s_indexingTask.IsCompleted)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't get the relation between teh indicesComplete flag and the s_indexingTask.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indicesComplete is true means we have indices available for all references. When it's false, we would try to queue a background task to force create them, which is s_indexingTask, however, if s_indexingTask from a previous completion session is not completed, we will just let it finish w/o attaching another task to it.


if (index.ContainsExtensionMethod && parameters.SemanticModel.Compilation.GetAssemblyOrModuleSymbol(peReference) is IAssemblySymbol assembly)
{
var filter = CreateAggregatedFilter(parameters.ReceiverTypeNames, index);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the terms "filter" or "aggregated filter" aresn't really conveying to me what thsi is for.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added some comments

@genlu
Copy link
Member Author

genlu commented Jul 15, 2020

Last commit 310e391 is to fix a unrelated test. See #45990

@genlu genlu merged commit 4cd6a01 into dotnet:master Jul 15, 2020
@ghost ghost added this to the Next milestone Jul 15, 2020
@genlu genlu deleted the mtCompletion branch July 15, 2020 06:16
@JoeRobich JoeRobich modified the milestones: Next, 16.8.P1 Jul 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants