-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Add Benchmark project and fix boxing issue in ConsoleRowHighlightingRule #5967
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- This PR adds a BenchmarkDotNet (BDN) benchmark to the solution to make it easy to microbenchmark. It also adds in the VisualStudio profiler BDN integration so we can get profiler traces and optimize code.
- This PR removes boxing that was occuring in ColoredConsoleSystemPrinter by evaluating the condition and returning a cached boxed bool from ConditionExpression. This reduces allocations in the benchmark by ~8%
|
Thanks for opening this pull request! |
WalkthroughAdds a new NLog.Benchmarks project with a BenchmarkDotNet runner and a ColoredConsoleTarget benchmark that exercises word-highlighting. Updates the solution to include the project and configuration mappings. Adds a new overload in ConditionMethodExpression to accept Func<LogEventInfo, bool> for no-parameter condition methods. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant User
participant Program as Program.Main
participant Switch as BenchmarkSwitcher
participant Runner as Benchmark Runner
participant Bench as ColoredConsoleTargetBenchmark
participant NLog as NLog Logger
participant Target as ColoredConsoleTarget
participant Console as Console.Out (Memory)
User->>Program: start(args)
Program->>Switch: FromAssembly(...).Run(args)
Switch->>Runner: Discover & execute benchmarks
rect rgb(235,245,255)
note right of Bench: GlobalSetup
Runner->>Bench: Setup()
Bench->>Console: Redirect Console.Out -> MemoryStream
Bench->>NLog: Configure logger, target, rule
end
rect rgb(240,255,240)
note right of Bench: Benchmark iteration
Runner->>Bench: WordHighlightingTextIgnoreCase()
Bench->>Console: Clear memory buffer
loop 1000 times
Bench->>NLog: Logger.Debug("... at ...")
NLog->>Target: Apply layout + highlighting
Target->>Console: Write colored output
end
end
rect rgb(255,245,235)
note right of Bench: GlobalCleanup
Runner->>Bench: Cleanup()
Bench->>Console: Restore original Console.Out
end
sequenceDiagram
autonumber
participant Rule as Condition Rule
participant CME as ConditionMethodExpression
participant Pred as Func<LogEventInfo,bool>
participant Log as LogEventInfo
Note over Rule,CME: New overload for no-parameter boolean predicates
Rule->>CME: CreateMethodNoParameters(name, Pred)
CME->>Pred: Invoke(LogEventInfo)
Pred-->>CME: bool
CME-->>Rule: return boxed true/false (object)
Rule->>Rule: Use existing object-returning evaluation path
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. 📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 💡 Knowledge Base configuration:
You can enable these sources in your CodeRabbit configuration. 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (5)
src/NLog/Targets/ColoredConsoleSystemPrinter.cs (1)
104-110: Good allocation win by returning cached boxed booleansUsing ConditionExpression.BoxedTrue/BoxedFalse via static lambdas avoids repeated boxing and the static captures nothing. Nice and targeted fix.
Optional cleanups for consistency/readability:
- Use the same namespace qualifier for both types (either NLog.Conditions.* or Conditions.*) to avoid mixed styles on the same line.
- If you prefer shorter lines, consider a using static import for ConditionExpression and then just return BoxedTrue/BoxedFalse.
Example diff to unify the qualifier (purely cosmetic):
- new ConsoleRowHighlightingRule(Conditions.ConditionMethodExpression.CreateMethodNoParameters("level == LogLevel.Fatal", static (evt) => evt.Level == LogLevel.Fatal ? NLog.Conditions.ConditionExpression.BoxedTrue : NLog.Conditions.ConditionExpression.BoxedFalse), ConsoleOutputColor.Red, ConsoleOutputColor.NoChange), + new ConsoleRowHighlightingRule(Conditions.ConditionMethodExpression.CreateMethodNoParameters("level == LogLevel.Fatal", static (evt) => evt.Level == LogLevel.Fatal ? Conditions.ConditionExpression.BoxedTrue : Conditions.ConditionExpression.BoxedFalse), ConsoleOutputColor.Red, ConsoleOutputColor.NoChange),src/NLog.Benchmarks/Program.cs (1)
11-19: Remove unreachable code or guard it with conditional compilationThe explicit return makes the in-process debug runner unreachable, requiring pragma suppression. Wrap the two paths with #if DEBUG to keep both without disabling CS0162.
Suggested refactor:
- BenchmarkSwitcher.FromAssembly(System.Reflection.Assembly.GetExecutingAssembly()).Run( - args, - DefaultConfig.Instance - .AddJob(BenchmarkDotNet.Jobs.Job.Default.WithIterationCount(5).WithWarmupCount(1).WithEnvironmentVariables(Microsoft.VSDiagnostics.VSDiagnosticsConfigurations.GetDotNetObjectAllocEnvironmentVariables(1)).AsMutator()) - .AddDiagnoser(new Microsoft.VSDiagnostics.DotNetObjectAllocDiagnoser()) - ); return; -#pragma warning disable CS0162 // Unreachable code detected - BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args, new DebugInProcessConfig()); -#pragma warning restore CS0162 // Unreachable code detected +#if DEBUG + BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly) + .Run(args, new DebugInProcessConfig()); +#else + BenchmarkSwitcher.FromAssembly(System.Reflection.Assembly.GetExecutingAssembly()).Run( + args, + DefaultConfig.Instance + .AddJob(BenchmarkDotNet.Jobs.Job.Default + .WithIterationCount(5) + .WithWarmupCount(1) + .WithEnvironmentVariables(Microsoft.VSDiagnostics.VSDiagnosticsConfigurations.GetDotNetObjectAllocEnvironmentVariables(1)) + .AsMutator()) + .AddDiagnoser(new Microsoft.VSDiagnostics.DotNetObjectAllocDiagnoser()) + ); +#endifsrc/NLog.Benchmarks/ColoredConsoleTargetBenchmark.cs (2)
38-41: Ensure predictable writes to the redirected Console outputSet AutoFlush to true on the StreamWriter so Console writes are flushed promptly to the underlying memory stream. It reduces buffering variability across runs.
- streamWriter = new StreamWriter(memoryStream); + streamWriter = new StreamWriter(memoryStream) { AutoFlush = true };
46-49: Tidy up global state after the benchmarkConsider resetting LogManager.Configuration to avoid leaking configuration to other benchmarks/processes. Flush before restoring Console.Out for a clean hand-off.
public void Cleanup() { - Console.SetOut(oldConsoleOutWriter); - streamWriter.Dispose(); - memoryStream.Dispose(); + streamWriter.Flush(); + Console.SetOut(oldConsoleOutWriter); + streamWriter.Dispose(); + memoryStream.Dispose(); + LogManager.Configuration = null; }src/NLog.Benchmarks/NLog.Benchmarks.csproj (1)
13-15: Avoid depending on the UnitTests project; reference NLog directlyReferencing the test project drags test-only dependencies and increases build time. The benchmark only uses NLog types; depend directly on the NLog project instead.
- <ItemGroup> - <ProjectReference Include="..\..\tests\NLog.UnitTests\NLog.UnitTests.csproj" /> - </ItemGroup> + <ItemGroup> + <ProjectReference Include="..\NLog\NLog.csproj" /> + </ItemGroup>
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (5)
src/NLog.Benchmarks/ColoredConsoleTargetBenchmark.cs(1 hunks)src/NLog.Benchmarks/NLog.Benchmarks.csproj(1 hunks)src/NLog.Benchmarks/Program.cs(1 hunks)src/NLog.sln(4 hunks)src/NLog/Targets/ColoredConsoleSystemPrinter.cs(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (2)
src/NLog.Benchmarks/ColoredConsoleTargetBenchmark.cs (1)
src/NLog/LogManager.cs (1)
LogManager(52-411)
src/NLog/Targets/ColoredConsoleSystemPrinter.cs (2)
src/NLog/Targets/ConsoleRowHighlightingRule.cs (2)
ConsoleRowHighlightingRule(48-50)ConsoleRowHighlightingRule(58-63)src/NLog/Targets/ColoredConsoleTarget.cs (2)
ConsoleRowHighlightingRule(463-471)ConsoleRowHighlightingRule(473-482)
🔇 Additional comments (3)
src/NLog.Benchmarks/Program.cs (1)
11-16: Config looks solid for allocation-focused runsThe Job mutation and DotNetObjectAllocDiagnoser align with the PR goal to track object allocations. Running from the executing assembly is fine for discovering all benchmarks.
src/NLog.Benchmarks/ColoredConsoleTargetBenchmark.cs (1)
51-59: Benchmark scenario exercises the targeted pathLooping 1000 Debug events with word highlighting and default row rules will hit the DefaultConsoleRowHighlightingRules and validate the boxing fix in a realistic path. Looks good.
src/NLog.sln (1)
84-85: Solution integration looks correctProject added with build mappings for Debug/Release and nested into the solution. No issues spotted.
Also applies to: 204-207, 239-239
|
I'll fix the unreachable code nit, I'm just waiting for a new fix in our profiling agent that found this. If folks are interested this talk shows the agent that found this fix: https://www.youtube.com/watch?v=zhXhUMkrJDQ |
|
Thank you for this pull-request, very nice find. Some observations:
|
|
Possible you can make co-pilot rewrite this pull-request to match #5968 ? I like the goal of having an official NLog-Benchmark-project depending on Benchmark.net, which makes it easy to profile common usage-patterns. But not sure it should be mixed into this pull-request. Possible that you can update this pull-request to only include the code-fix ? |
|
Yep, happy to reformat to #5968. Would you like me to split this PR into 2 separate PRs, one for the benchmarks and one for the fix? I did it as 2 separate commits with the first being the benchmark addition and the second being the perf fix.
Yeah, its pretty incredible for being able to run a whole bunch of different experiments really quickly and finding clever optimizations, I can't wait to release it! It doesn't always hit the standards right and sometimes creates / extends public interfaces breaking things so you need to keep it in line / touch things up afterwards. I tried to do that with this PR, feel free to point out anything you are not happy with, I'm more than happy to change stuff. Just trying to find successes with the project to make sure it really helps folks and do some good in the community as well :) |
Happy to change this around, was just trying to leave existing APIs as is and this seemed to be the best way to do that. I can add another constructor to |
After further consideration, then just keep this in a single pull-request (Code-fix + new benchmark-project). But please update code-fix to match #5968
Maybe easier that you just commit the NLog.Benchmark-project in its current state. Then I can adjust to my liking (Think it will become an internal-constructor only visible to the unit-test-projects, that takes a method-delegate that returns the TextWriter, where the default delegate will be a method that returns StdOut / StdErr depending on target-configuration). |
Maybe the "Profiler Agent" should rely on a "Code Improvement"-agent that have been prompted differently (Like the "Code Review" agent follows project-guidelines-md). Instead of making the surgical code-fix directly (based on profiler-red-path-discovery), then give the task of implementing the surgical code-fix to a "Code Improvement"-agent. The "Code Improvement"-agent can then consider how the initial change would affect the overall code-base, and matches the project-code-style, and whether other caller-locations should also be considered. But maybe that is only for enterprise-customers, while free-riders just have to do their own code-review of the surgical code-fix. |
- This commit address PR feedback, moving the boxing expression into ConditionMethodExpression. This should help with other areas in the code that use no param boolean funcs for ConditionMethodExpressions. - This also updates the benchmarking package to use the latest BenchmarkDotNetDiagnosers package for VS. This leaves the Program.cs benchmark launcher as is an uses a non-intrusive global config when run through the VS Profiler Agent.
It's a really interesting thought. I looked at the change myself and touched things up as I don't want to be inundating maintainers with bad AI contributions and I missed bringing this up level. It's been fascinating to see it try different perf experiments and what it comes up with. I'm guessing we could have a "Code Improvement"-agent do a few different versions of a performance fix so the dev can chose which one they want (or which is the closest).
Well the Profiler Agent will be in all SKUs of Visual Studio, and if we do a "Code Improvement"-agent hopefully that one can be as well. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
🧹 Nitpick comments (5)
src/NLog.Benchmarks/NLog.Benchmarks.csproj (3)
4-6: Benchmark project: consider multi-targeting, mark as non-packable, and force x64 for stable results
- Benchmarks are often run on multiple runtimes. Recommend multi-targeting to include a modern .NET runtime (e.g., net8.0) alongside net462.
- Mark the project as non-packable to avoid accidental packaging.
- Force x64 for more stable/realistic BDN runs and to avoid 32-bit JIT noise on some dev machines.
Apply this diff:
- <PropertyGroup> - <TargetFramework>net462</TargetFramework> - <OutputType>Exe</OutputType> - </PropertyGroup> + <PropertyGroup> + <!-- Keep net462, but also allow running on a modern runtime --> + <TargetFrameworks>net462;net8.0</TargetFrameworks> + <OutputType>Exe</OutputType> + <IsPackable>false</IsPackable> + <PlatformTarget>x64</PlatformTarget> + <!-- Optional: keep source simple and modern --> + <LangVersion>latest</LangVersion> + </PropertyGroup>
9-11: Diagnostics package choice: prefer BDN’s platform-specific diagnosers and gate them by OSThe Visual Studio DiagnosticsHub diagnoser package tends to be Windows/VS-specific and heavy. For cross-platform CI and fewer restore issues, prefer BenchmarkDotNet.Diagnostics.Windows (Windows ETW) or keep diagnosers optional/conditional.
Option A (Windows-only diagnoser):
<ItemGroup> <PackageReference Include="BenchmarkDotNet" Version="0.15.2" /> - <PackageReference Include="Microsoft.VisualStudio.DiagnosticsHub.BenchmarkDotNetDiagnosers" Version="18.0.36422.2" /> </ItemGroup> + <ItemGroup Condition="'$(OS)' == 'Windows_NT'"> + <PackageReference Include="BenchmarkDotNet.Diagnostics.Windows" Version="0.15.2" /> + </ItemGroup>Please verify:
- Whether you actually need DiagnosticsHub-specific features over BDN’s built-in/Windows diagnosers.
- That 0.15.2 is the intended BDN version across the repo (pinning a newer/older patch may be preferable for consistency).
14-15: ProjectReference is fine as-is; consider excluding this project from default CI buildsThe ProjectReference is appropriate for local benchmarking. To keep CI lean and avoid failing pipelines due to benchmarks, consider excluding the benchmarks project from the default solution build in CI (solution-level config) rather than changing this csproj.
If helpful, I can provide a separate PR to:
- Add a solution configuration where NLog.Benchmarks is not built by default on CI.
- Document how to run benchmarks locally (dotnet run -c Release -f net8.0).
src/NLog/Conditions/ConditionMethodExpression.cs (2)
90-93: Document the non-boxing behavior for discoverabilityAdd an XML summary to call out that this overload returns cached boxed booleans to avoid allocations. It helps future contributors choose this overload for boolean predicates.
Proposed doc (add above the method):
/// <summary> /// Creates a condition method (no parameters) backed by a boolean predicate. /// The result is converted to a cached boxed boolean (BoxedTrue/BoxedFalse) to avoid per-evaluation boxing. /// </summary>
85-99: Ensure the parser actually invokes the bool-overload to avoid boxingVerification shows that ConditionParser (at
src/NLog/Conditions/ConditionParser.cs:151–154) only callsvar method = _configurationItemFactory.ConditionMethodFactory.TryCreateInstanceWithNoParameters(functionName); if (method != null) return ConditionMethodExpression.CreateMethodNoParameters(functionName, method);and has no path for
Func<LogEventInfo,bool>, so all parser-resolved methods still box their bool result. Meanwhile, the new bool overload is only used by hand-written callers (e.g. inSetupLoadConfigurationExtensions.csandColoredConsolePrinter*).To make the zero-parameter bool overload effective for parser-created conditions, pick one integration approach:
- Factory-first
Extend the MethodFactory API to preferentially return aFunc<LogEventInfo,bool>when available:// In MethodFactory public Func<LogEventInfo,bool>? TryCreateInstanceWithNoParametersBool(string methodName) { … } // In ConditionParser.CreateMethodExpression: var methodBool = _configurationItemFactory.ConditionMethodFactory.TryCreateInstanceWithNoParametersBool(functionName); if (methodBool != null) return ConditionMethodExpression.CreateMethodNoParameters(functionName, methodBool); // Fallback to object-returning: var methodObj = _configurationItemFactory.ConditionMethodFactory.TryCreateInstanceWithNoParameters(functionName); if (methodObj != null) return ConditionMethodExpression.CreateMethodNoParameters(functionName, methodObj);- Parser adapter
If the existing factory can indicate the target method’s return type (e.g. via metadata on the delegate), detect boolean-returning methods in ConditionParser and call the new bool overload directly.Please adjust
ConditionParser(around line 151) to route boolean delegates intoCreateMethodNoParameters(string, Func<LogEventInfo,bool>).
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (3)
src/NLog.Benchmarks/NLog.Benchmarks.csproj(1 hunks)src/NLog.Benchmarks/Program.cs(1 hunks)src/NLog/Conditions/ConditionMethodExpression.cs(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- src/NLog.Benchmarks/Program.cs
🧰 Additional context used
🧬 Code graph analysis (1)
src/NLog/Conditions/ConditionMethodExpression.cs (2)
src/NLog/Conditions/ConditionParser.cs (2)
ConditionMethodExpression(113-145)ConditionMethodExpression(147-183)src/NLog/Config/MethodFactory.cs (5)
Func(352-361)Func(363-372)Func(374-383)Func(385-394)Func(396-443)
|
I fixed up the code to align with #5968, let me know if there are any other changes you would like. I'm going to try and see if I can find other fixes, for future PRs do you want each benchmark separate from the fix or new benchmark and fix in the same PR? I typically think of benchmarks as unit tests for perf which is why I include them with the fix. Also, any areas / APIs in particular you want me to have the Profiler Agent try and review? |
I'm happy with benchmark being added together with code-fix (Just needed to adjust my world view, since few NLog contributors focus on isolated benchmark, and if creating benchmarks then making them non-generic and closely matching their own local environment/config). Was thinking to have a dedicated benchmark for the different NLog-features / NLog-use-case.
I was thinking that one should have at least one benchmark for each NLog-target / NLog-Target-wrapper / NLog-Layout / NLog-LayoutRenderer. But maybe also a benchmark for the common NLog Logger use-cases:
|
|
Think you should mark the NLog.Benchmark-csproj, as "test"-project, then Sonar will ignore the missing code-coverage: <SonarQubeTestProject>true</SonarQubeTestProject>
<IsPackable>false</IsPackable> |
|
Looks like most of the other projects have: |
I prefer But I guess at some point then dotnet-analyzers / dotnet-coverage will take over everything :) |
|
@karpinsn Just curious. Why did you choose net462 for the benchmark-project instead of net8 / net9 ? I'm currently treating .NET Framework as mostly legacy. Is it because the JIT-compiler is not so good with net462, while the JIT-compiler in net8 / net9 can hide/fix less optimal code? (Along with more aggressive GC because of DATAS) |
|
@snakefoot I based it off the NLog.RegEx.Tests project since the |
The new benchmark measures the performance of using
ColoredConsoleTargetto log to a console. The performance fix is to remove the boxing that is occurring in the condition passed toConsoleRowHighlightingRule. It now checks the condition of the expression and returns a cached boxed boolean value fromConditionExpression. The current benchmarks results in saving ~8% allocations (from 1.33Mb to 1.22Mb). This doesn't result in a noticable CPU performance saving but should help with GC overhead.Before - 1.33Mb
After - 1.22Mb