Skip to content

Batch metadata updates for better performance#8098

Merged
rokonec merged 1 commit intodotnet:mainfrom
marcin-krystianc:marcink-20221027-gathertaskoutputs
Nov 5, 2022
Merged

Batch metadata updates for better performance#8098
rokonec merged 1 commit intodotnet:mainfrom
marcin-krystianc:marcink-20221027-gathertaskoutputs

Conversation

@marcin-krystianc
Copy link
Copy Markdown
Contributor

Improves dotnet/sdk#27738

Context

I was looking for a reason why "design-time-builds" are slow (see dotnet/sdk#27738).
One method that stands out in the performance profiler is a method called GatherTaskOutputs.
It copies the output of the executed task into a new ProjectItemInstance. Copying the output metadata involves creating and populating the ImmutableDicitionary in the newly created ProjectItemInstance. The copying process turns out to be a noticeably slow operation.

Changes Made

Instead of copying metadata properties one by one, all properties will be copied with a single SetItems operation.
According to BenchmarkDotNet using a single operation to populate the ImmutableDictionary is twice as fast.

Testing

Benchmarking using scenario from dotnet/sdk#27738 (comment).

  • before:
    Time Elapsed 00:01:23.18
    Time Elapsed 00:01:23.36
    Time Elapsed 00:01:23.91
    Time Elapsed 00:01:23.18
    Time Elapsed 00:01:23.25

  • after:
    Time Elapsed 00:01:20.87
    Time Elapsed 00:01:20.21
    Time Elapsed 00:01:20.71
    Time Elapsed 00:01:20.49
    Time Elapsed 00:01:20.60

This change improves performance of the tested scenario by 3-4%!

Notes

Copy link
Copy Markdown
Member

@rainersigwald rainersigwald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks awesome, thanks!

Copy link
Copy Markdown
Contributor

@Forgind Forgind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your solution looks pretty big, so I could see where this would have benefits, and I'm assuming the test case you used in Benchmark.NET was similarly big. Can you try with something small? I'm wondering if this change might hurt in that case, since it adds Linq and KeyValuePair allocations.

@marcin-krystianc
Copy link
Copy Markdown
Contributor Author

Your solution looks pretty big, so I could see where this would have benefits, and I'm assuming the test case you used in Benchmark.NET was similarly big. Can you try with something small? I'm wondering if this change might hurt in that case, since it adds Linq and KeyValuePair allocations.

My BenchamrkDotNet tests didn't cover extra allocations from Linq and KeyValuePair allocations, just pure comparison of Add vs AddRange. Anyway, I've tried my changes on the msbuild solution itself and I don't see any noticeable difference:

before:
Time Elapsed 00:00:06.64
Time Elapsed 00:00:06.72
Time Elapsed 00:00:06.72
Time Elapsed 00:00:06.72
Time Elapsed 00:00:06.65

after:
Time Elapsed 00:00:06.95
Time Elapsed 00:00:06.53
Time Elapsed 00:00:06.64
Time Elapsed 00:00:06.53
Time Elapsed 00:00:06.75

My gut feeling is that for small solutions the extra allocations are not noticeable and for large solutions the benefit of improved performance of ImmutableDictionary is much greater than the cost of extra allocations.

@Forgind
Copy link
Copy Markdown
Contributor

Forgind commented Oct 28, 2022

Looks like more variance after, but that might just be a product of small sample size. Thanks for checking that!

@marcin-krystianc
Copy link
Copy Markdown
Contributor Author

"license/cla" status is still pending, what do I need to do to make it go green?

@rainersigwald
Copy link
Copy Markdown
Member

"license/cla" status is still pending, what do I need to do to make it go green?

We are bothering people internally, but it's going frustratingly slowly. Sorry!

@Forgind Forgind added the merge-when-branch-open PRs that are approved, except that there is a problem that means we are not merging stuff right now. label Nov 2, 2022
@rokonec rokonec merged commit 3bcada9 into dotnet:main Nov 5, 2022
@marcin-krystianc marcin-krystianc deleted the marcink-20221027-gathertaskoutputs branch November 7, 2022 08:37
rainersigwald added a commit that referenced this pull request Nov 16, 2022
_directMetadata ??= new CopyOnWritePropertyDictionary<ProjectMetadataInstance>();

var metadata = items
.Where(item => !FileUtilities.ItemSpecModifiers.IsDerivableItemSpecModifier(item.Value))
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should have been item.Key instead of item.Value - this mistake introduced the problem reported in #8153. I'm working on covering this scenario with unit tests.

rainersigwald pushed a commit that referenced this pull request Mar 28, 2023
Improves dotnet/sdk#27738
It is another attempt to introduce changes proposed in #8098, which was reverted due to a bug.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-when-branch-open PRs that are approved, except that there is a problem that means we are not merging stuff right now.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants