Skip to content

[NativeAOT] ConcurrentDictionary is slower #68891

@adamsitnik

Description

@adamsitnik

Most of the ConcurrentDictionary micro benchmarks are few times slower compared to .NET.

Examples:

Details

System.Collections.CreateAddAndClear.ConcurrentDictionary(Size: 512)

Result Base Diff Ratio Alloc Delta Operating System Bit Processor Name Modality
Slower 64614.59 254817.14 0.25 +16385 Windows 10 Arm64 Microsoft SQ1 3.0 GHz
Slower 67615.17 475852.15 0.14 +49154 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 38845.50 414977.53 0.09 +49153 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores bimodal
Slower 78981.98 438806.96 0.18 +49152 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 79174.15 241955.10 0.33 +16384 macOS Monterey 12.2.1 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

System.Collections.CtorFromCollection.ConcurrentDictionary(Size: 512)

Result Base Diff Ratio Alloc Delta Operating System Bit Processor Name Modality
Slower 84175.23 240287.50 0.35 +16385 Windows 10 Arm64 Microsoft SQ1 3.0 GHz
Slower 55556.09 222005.69 0.25 +24577 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz bimodal
Slower 50111.01 388393.75 0.13 +49153 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower 62874.58 227398.68 0.28 +24576 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 69288.84 128176.10 0.54 +8192 macOS Monterey 12.2.1 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_ExpirationTokens

Result Base Diff Ratio Alloc Delta Operating System Bit Processor Name Modality
Slower 77274.94 222586.06 0.35 +11263 Windows 10 Arm64 Microsoft SQ1 3.0 GHz
Slower 40825.59 158493.50 0.26 +28294 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz several?
Slower 38473.82 166670.27 0.23 -2699 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower 43664.37 180698.83 0.24 +29846 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz several?
Slower 70694.75 160708.25 0.44 +11849 macOS Monterey 12.2.1 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_AbsoluteExpiration

Result Base Diff Ratio Alloc Delta Operating System Bit Processor Name Modality
Slower 62336.17 141002.87 0.44 +6193 Windows 10 Arm64 Microsoft SQ1 3.0 GHz
Slower 38369.18 70234.58 0.55 -7994 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 30147.18 136249.59 0.22 +32694 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower 44836.63 86477.55 0.52 -9434 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Slower 58801.04 85427.09 0.69 +14189 macOS Monterey 12.2.1 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

Repro:

git clone https://github.com/dotnet/performance.git
cd performance
py .\scripts\benchmarks_ci.py -f net7.0 --filter "System.Collections.CreateAddAndClear<Int32>.ConcurrentDictionary" --bdn-arguments "--keepFiles true --runtimes net7.0 nativeaot7.0 --ilCompilerVersion 7.0.0-preview.5.22254.9 --invocationCount 3488"
BenchmarkDotNet=v0.13.1.1786-nightly, OS=Windows 10 (10.0.18363.2158/1909/November2019Update/19H2)
Intel Xeon CPU E5-1650 v4 3.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET SDK=7.0.100-preview.5.22254.18
  [Host]     : .NET 7.0.0 (7.0.22.25401), X64 RyuJIT
  Job-ERDEPN : .NET 7.0.0 (7.0.22.25401), X64 RyuJIT                                                                                                                                                                                             
  Job-LHIWER : .NET 7.0.0-preview.5.22254.9, X64 NativeAOT  
Method Runtime Mean Ratio Gen 0 Gen 1 Gen 2 Allocated Alloc Ratio
ConcurrentDictionary .NET 7.0 71.56 us 1.00 16.0550 4.0138 - 124.44 KB 1.00
ConcurrentDictionary NativeAOT 7.0 468.09 us 6.51 21.4286 19.6429 1.7857 172.44 KB 1.39

I took a quick look at numbers reported by VTune and it seems that it might be caused by #67805, but I am not 100% sure so I am reporting a new issue (locking itself might just be slower).

NativeAOT

image

JIT

image

cc @jkotas @MichalStrehovsky

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions