[Crossgen2] Deduplicate identical IL method bodies in R2R composite image by kotlarmilos · Pull Request #126047 · dotnet/runtime

kotlarmilos · 2026-03-24T18:04:39Z

Description

This PR adds content-based deduplication of IL method bodies when emitting R2R composite component assemblies.

CopiedMethodIL on the node factory uses a ConcurrentDictionary<byte[], CopiedMethodILNode> keyed by effective body bytes. When a method's body matches an already-seen body, the existing node is reused, causing multiple MethodDef RVA entries to point to the same IL body blob.

Deduplication operates on post-strip bytes: ReadBodyBytes returns the stripped stub for strippable methods, so all stripped non-generic methods collapse into a single shared node. This builds on top of the --strip-il-bodies option added in #125647.

Impact (MAUI HelloWorld iOS arm64)

Metric	Value
Total IL bodies	105,152
Deduplicated	12,874 (12.2%)
Bytes saved	~155 KB

…bodies in R2R composite images Add a --dedup-il-bodies flag to crossgen2 that enables content-based deduplication of identical IL method bodies when emitting R2R composite component assemblies. When enabled, crossgen2 uses a content-keyed dictionary per component factory. When a method's IL body bytes match an already-seen body, the existing CopiedMethodILNode is reused, causing multiple MethodDef RVA entries to point to the same IL body blob. Enable by default on Apple mobile platforms (ios, tvos, iossimulator, tvossimulator, maccatalyst) via MSBuild targets, matching the pattern used for --strip-inlining-info and --strip-debug-info. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR adds an opt-in crossgen2 optimization (--dedup-il-bodies) to content-deduplicate identical IL method bodies when producing R2R composite component assemblies, and wires it through the CLI and Apple mobile MSBuild defaults to reduce output size.

Changes:

Add --dedup-il-bodies CLI option and plumb it into NodeFactoryOptimizationFlags.
Implement IL-body content deduplication in the ReadyToRun node factory using a content-keyed cache.
Enable the flag by default for Apple mobile RIDs via Microsoft.NET.CrossGen.targets.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/tasks/Crossgen2Tasks/Microsoft.NET.CrossGen.targets	Enables `--dedup-il-bodies` by default for iOS/tvOS simulators and maccatalyst publish R2R.
src/coreclr/tools/aot/crossgen2/Properties/Resources.resx	Adds localized description string for the new CLI option.
src/coreclr/tools/aot/crossgen2/Program.cs	Wires the CLI option into `NodeFactoryOptimizationFlags.DedupILBodies`.
src/coreclr/tools/aot/crossgen2/Crossgen2RootCommand.cs	Declares and registers the new `--dedup-il-bodies` option.
src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRunCodegenNodeFactory.cs	Adds the dedup cache and gates dedup behavior behind `DedupILBodies`.
src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRun/CopiedMethodILNode.cs	Adds helper to read raw method body bytes for content-based dedup keys.

.../tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRunCodegenNodeFactory.cs

jkotas · 2026-03-24T19:02:39Z

Can this be on by default? I do not think it needs to be an option.

jkoritzinsky

Could we instead use/enhance ObjectDataInterner from ILC (and enable it in crossgen2) to do the deduplicating for us during emit?

Remove the DedupILBodies flag from NodeFactoryOptimizationFlags, the --dedup-il-bodies CLI option, the MSBuild PublishReadyToRunDedupILBodies properties, and the resource string. Deduplication of identical copied IL method bodies is now unconditional. Restructure CopiedMethodIL to use a factory function in the dedup dictionary to avoid creating unused per-method nodes when bodies deduplicate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

kotlarmilos · 2026-03-25T09:23:31Z

/azp run runtime-coreclr crossgen2,runtime-coreclr crossgen2-composite

azure-pipelines · 2026-03-25T09:23:46Z

Azure Pipelines successfully started running 2 pipeline(s).

kotlarmilos · 2026-03-25T09:27:49Z

Could we instead use/enhance ObjectDataInterner from ILC (and enable it in crossgen2) to do the deduplicating for us during emit?

My understanding of the ObjectDataInterner: it is designed for folding compiled native method bodies where it compares code + relocations + other info and runs in iterations because folding can enabling further folds. For the dedup none of that applies.

Enabling it in crossgen2 would require more changes. I think it can be moved, I just want to check if that was the intention.

.../tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRunCodegenNodeFactory.cs

jkotas · 2026-03-25T14:04:18Z

ObjectDataInterner

I think it would be overkill for what we need here.

Remove the intermediate NodeCache<MethodDesc, CopiedMethodILNode> and use only the content-keyed ConcurrentDictionary for IL body dedup, as suggested in code review. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

.../tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRunCodegenNodeFactory.cs

...tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRun/CopiedMethodILNode.cs

.../tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRunCodegenNodeFactory.cs

MichalStrehovsky · 2026-03-31T00:32:04Z

Looks like this will have an interaction with #125647 that might push this to having to do this later.

kotlarmilos · 2026-03-31T09:56:46Z

Yes, let's first land #125647.

MichalPetryka · 2026-03-31T10:40:36Z

Should this not be done by Roslyn instead? Are there any drawbacks to deduplicating like this?

jkotas · 2026-03-31T18:12:09Z

Should this not be done by Roslyn instead? Are there any drawbacks to deduplicating like this?

This is de-duplicating across multiple assemblies in a composite R2R image.

ReadBodyBytes now accounts for stripping: when a method would be stripped, it returns s_minimalILBody instead of original IL bytes. This makes the dedup key reflect the effective (post-strip) body, so all stripped methods share a single CopiedMethodILNode. GetData() reuses ReadBodyBytes to eliminate duplicated stripping logic. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

.../tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRunCodegenNodeFactory.cs

...tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRun/CopiedMethodILNode.cs

kotlarmilos · 2026-04-08T13:33:04Z

/azp run runtime-coreclr crossgen2,runtime-coreclr crossgen2-composite

azure-pipelines · 2026-04-08T13:33:20Z

Azure Pipelines successfully started running 2 pipeline(s).

jkotas · 2026-04-08T18:46:45Z

kotlarmilos requested review from jkoritzinsky and jkotas

Either this PR or #126112 is fine with me. I will let you figure out which one we want to go with.

MichalStrehovsky · 2026-04-09T01:06:40Z

.../tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRunCodegenNodeFactory.cs

+            byte[] bodyBytes = CopiedMethodILNode.ReadBodyBytes(method, OptimizationFlags);
+            return _copiedMethodIL.GetOrAdd(bodyBytes, _ => new CopiedMethodILNode(method));


This may not work. If first time someone asks for CopiedMethodIL and optimizationFlags.CompiledMethodDefs is still null, we'll get a node for the real method body bytes. This node will become part of the dependency graph and will get written to the output no matter if there's still a reference to it at the writing phase (if the node is marked, it will get written to the output).

If this is not a concern, we should remove the question mark from optimizationFlags.CompiledMethodDefs?.Contains(method) so that this is a hard crash if anyone wants to use this before we do the analysis.

If it is a concern, there are ways to address this. Probably the easiest would be to delete the CopiedMethodILNode completely and instead emit the bodies as part of CopiedMetadataBlobNode. We don't need a separate symbol for each method body - the builder.EmitReloc(factory.CopiedMethodIL(method), RelocType.IMAGE_REL_BASED_ADDR32NB); can be builder.EmitReloc(this, RelocType.IMAGE_REL_BASED_ADDR32NB, someOffsetFromStart); (where someOffsetFromStart is _sourceModule.PEReader.GetMetadata().Length + cumulativeSizeOfMethodBodiesEmittedSoFar).

MichalStrehovsky · 2026-04-09T01:08:30Z

kotlarmilos requested review from jkoritzinsky and jkotas

Either this PR or #126112 is fine with me. I will let you figure out which one we want to go with.

I feel similar about this. If we have a use case for deduplication beyond the method bodies, it might be worth it to take Jeremy's change. If we don't see where else we would use this, I'd not add more complexity to the ILC deduplication.

kotlarmilos · 2026-04-09T10:32:20Z

Jeremy's change allows for more flexibility going forward. @jkoritzinsky How do you feel? Do you want to proceed with your PR instead?

kotlarmilos · 2026-04-09T20:28:39Z

Closing this PR in favour of #126112

Copilot AI review requested due to automatic review settings March 24, 2026 18:04

github-actions bot added the area-crossgen label Mar 24, 2026

dotnet-policy-service bot assigned kotlarmilos Mar 24, 2026

Copilot started reviewing on behalf of kotlarmilos March 24, 2026 18:05 View session

kotlarmilos changed the title ~~[Crossgen2] Add --dedup-il-bodies to deduplicate identical IL method bodies in R2R composite images~~ [Crossgen2] Add --dedup-il-bodies to deduplicate identical IL method bodies in R2R composite image Mar 24, 2026

kotlarmilos mentioned this pull request Mar 24, 2026

[clr-ios] App size improvements on iOS for CoreCLR with R2R configuration #124212

Open

10 tasks

Copilot AI reviewed Mar 24, 2026

View reviewed changes

kotlarmilos added this to the 11.0.0 milestone Mar 24, 2026

jkoritzinsky reviewed Mar 24, 2026

View reviewed changes

jkotas reviewed Mar 25, 2026

View reviewed changes

.../tools/aot/ILCompiler.ReadyToRun/Compiler/DependencyAnalysis/ReadyToRunCodegenNodeFactory.cs Outdated Show resolved Hide resolved

jkoritzinsky mentioned this pull request Mar 25, 2026

Refactor ObjectDataInterner to support pluggable deduplicators and enable for ReadyToRun #126112

Open

Use single byte[] -> Node mapping for IL body dedup

27a15ce

Remove the intermediate NodeCache<MethodDesc, CopiedMethodILNode> and use only the content-keyed ConcurrentDictionary for IL body dedup, as suggested in code review. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 26, 2026 10:36

Copilot started reviewing on behalf of kotlarmilos March 26, 2026 10:37 View session

Copilot AI reviewed Mar 26, 2026

View reviewed changes

kotlarmilos marked this pull request as ready for review March 26, 2026 11:31

kotlarmilos marked this pull request as draft April 7, 2026 13:11

Merge branch 'main' into r2r-dedup-il-bodies-measure

c235831

Copilot AI review requested due to automatic review settings April 8, 2026 13:13

kotlarmilos force-pushed the r2r-dedup-il-bodies-measure branch from 2af15a7 to 28021fa Compare April 8, 2026 13:13

Copilot started reviewing on behalf of kotlarmilos April 8, 2026 13:15 View session

Copilot AI reviewed Apr 8, 2026

View reviewed changes

kotlarmilos changed the title ~~[Crossgen2] Add --dedup-il-bodies to deduplicate identical IL method bodies in R2R composite image~~ [Crossgen2] Deduplicate identical IL method bodies in R2R composite image Apr 8, 2026

kotlarmilos marked this pull request as ready for review April 8, 2026 13:33

kotlarmilos requested review from jkoritzinsky and jkotas April 8, 2026 13:33

kotlarmilos requested a review from MichalStrehovsky April 8, 2026 13:33

build-analysis bot mentioned this pull request Apr 8, 2026

System.Net.NameResolution.Tests DNS failures: Name or service not known #126641

Open

MichalStrehovsky reviewed Apr 9, 2026

View reviewed changes

kotlarmilos closed this Apr 9, 2026

		byte[] bodyBytes = CopiedMethodILNode.ReadBodyBytes(method, OptimizationFlags);
		return _copiedMethodIL.GetOrAdd(bodyBytes, _ => new CopiedMethodILNode(method));

Conversation

kotlarmilos commented Mar 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Impact (MAUI HelloWorld iOS arm64)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jkotas commented Mar 24, 2026

Uh oh!

jkoritzinsky left a comment

Choose a reason for hiding this comment

Uh oh!

kotlarmilos commented Mar 25, 2026

Uh oh!

azure-pipelines bot commented Mar 25, 2026

Uh oh!

kotlarmilos commented Mar 25, 2026

Uh oh!

Uh oh!

jkotas commented Mar 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MichalStrehovsky commented Mar 31, 2026

Uh oh!

kotlarmilos commented Mar 31, 2026

Uh oh!

MichalPetryka commented Mar 31, 2026

Uh oh!

jkotas commented Mar 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kotlarmilos commented Apr 8, 2026

Uh oh!

azure-pipelines bot commented Apr 8, 2026

Uh oh!

jkotas commented Apr 8, 2026

Uh oh!

MichalStrehovsky Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

MichalStrehovsky commented Apr 9, 2026

Uh oh!

kotlarmilos commented Apr 9, 2026

Uh oh!

kotlarmilos commented Apr 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

kotlarmilos commented Mar 24, 2026 •

edited

Loading