Performance regression with JsonObject creation by +70% #107869
Description
Description
I have a number of .NET applications I maintain, which I've also recently started maintaining some basic continuous benchmarks for. A number of these applications include an endpoints that's used as a health check for deployment/debugging and is implemented something like this:
app.MapGet("/version", static () =>
{
return new JsonObject()
{
["applicationVersion"] = GitMetadata.Version,
["frameworkDescription"] = RuntimeInformation.FrameworkDescription,
["operatingSystem"] = new JsonObject()
{
["description"] = RuntimeInformation.OSDescription,
["architecture"] = RuntimeInformation.OSArchitecture.ToString(),
["version"] = Environment.OSVersion.VersionString,
["is64Bit"] = Environment.Is64BitOperatingSystem,
},
["process"] = new JsonObject()
{
["architecture"] = RuntimeInformation.ProcessArchitecture.ToString(),
["is64BitProcess"] = Environment.Is64BitProcess,
["isNativeAoT"] = !RuntimeFeature.IsDynamicCodeSupported,
["isPrivilegedProcess"] = Environment.IsPrivilegedProcess,
},
["dotnetVersions"] = new JsonObject()
{
["runtime"] = GetVersion<object>(),
["aspNetCore"] = GetVersion<HttpContext>(),
},
};
static string GetVersion<T>()
=> typeof(T).Assembly.GetCustomAttribute<AssemblyInformationalVersionAttribute>()!.InformationalVersion;
});As it's fairly trivial (e.g. has no external dependencies) I've been including it in the benchmarks as a sort of "canary" as a proxy for the overall performance of ASP.NET Core, JSON, the runtime etc.
I've over the last week updated a number of applications from .NET 8 to .NET 9 RC1 and was surprised to find that I'm consistently seeing a regression from this endpoint in terms of the memory allocated compared to .NET 8.
Below is an example chart from my tracking dashboard. The point at which the memory goes up and the duration goes down is the commit that applied the .NET 9 upgrade:
I've pulled this out into its own dedicated benchmark that just creates the JsonObject, and I see the same regression in terms of memory usage (and also duration). I've made an assumption here that the regression is in JsonObject as the properties being serialized are ones I would have thought wouldn't changed during the lifetime of a process and thus be cached.
Admittedly this isn't a critical path in terms of performance, but as it went backwards against what I would have thought the trend would be for the upgrade, and that it was noticeable, I figured I'd raise an issue so that if it's not an expected trade-off somewhere it can be fixed.
These are the results I get on my laptop for the simplified benchmark:
BenchmarkDotNet v0.14.0, Windows 11 (10.0.22621.4169/22H2/2022Update/SunValley2)
12th Gen Intel Core i7-1270P, 1 CPU, 16 logical and 12 physical cores
.NET SDK 9.0.100-rc.1.24452.12
[Host] : .NET 8.0.8 (8.0.824.36612), X64 RyuJIT AVX2
Job-WBWTQA : .NET 8.0.8 (8.0.824.36612), X64 RyuJIT AVX2
Job-MBMPCJ : .NET 9.0.0 (9.0.24.43107), X64 RyuJIT AVX2
| Method | Runtime | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio |
|---|---|---|---|---|---|---|---|---|---|---|
| CreateJsonObject | .NET 8.0 | 400.3 ns | 8.02 ns | 20.71 ns | 391.1 ns | 1.00 | 0.07 | 0.0067 | 1.24 KB | 1.00 |
| CreateJsonObject | .NET 9.0 | 769.2 ns | 14.13 ns | 19.35 ns | 771.6 ns | 1.93 | 0.11 | 0.0572 | 2.12 KB | 1.70 |
Compared to my higher-level benchmarks, it also appears to be slower than .NET 8 as well - I think it's masked there as there's much more going on and improvements elsewhere outweigh the increase in the runtime to create the object.
Reproduction Steps
dotnet run -c Release -f net8.0 -- --runtimes net8.0 net9.0<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<ServerGarbageCollection>true</ServerGarbageCollection>
<TargetFrameworks>net9.0;net8.0</TargetFrameworks>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="BenchmarkDotNet" Version="0.14.0" />
</ItemGroup>
</Project>using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Text.Json.Nodes;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using BenchmarkDotNet.Diagnosers;
BenchmarkRunner.Run<JsonNodeBenchmarks>(args: args);
[MemoryDiagnoser]
public class JsonNodeBenchmarks
{
[Benchmark]
public JsonObject CreateJsonObject()
{
return new JsonObject()
{
["applicationVersion"] = "1.2.3",
["frameworkDescription"] = RuntimeInformation.FrameworkDescription,
["operatingSystem"] = new JsonObject()
{
["description"] = RuntimeInformation.OSDescription,
["architecture"] = RuntimeInformation.OSArchitecture.ToString(),
["version"] = Environment.OSVersion.VersionString,
["is64Bit"] = Environment.Is64BitOperatingSystem,
},
["process"] = new JsonObject()
{
["architecture"] = RuntimeInformation.ProcessArchitecture.ToString(),
["is64BitProcess"] = Environment.Is64BitProcess,
["isNativeAoT"] = !RuntimeFeature.IsDynamicCodeSupported,
["isPrivilegedProcess"] = Environment.IsPrivilegedProcess,
},
};
}
}Expected behavior
Performance using .NET 9 is on par, or better, with .NET 8.
Actual behavior
Duration and memory allocations are greater running under .NET 9, by ~+80% for duration and ~+70% for allocations.
Regression?
Yes.
Known Workarounds
None.
Configuration
.NET SDK 9.0.100-rc.1.24452.12.
Other information
No response