Merged
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
da05244 to
143d7d7
Compare
BenchmarksBenchmarks Report for benchmark platform 🐌Benchmarks for #7868 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️Raw results
|
| # Branch containing 1. scripts to launch Windows benchmarks on ephemeral | ||
| # instances (to be used by GitLab CI runners) and 2. scripts to run Windows | ||
| # benchmarks (to be used by the ephemeral instances). | ||
| BP_INFRA_BENCHMARKING_PLATFORM_BRANCH: "dd-trace-dotnet/micro" |
Collaborator
There was a problem hiding this comment.
Guess that this change should be reverted after merging https://github.com/DataDog/benchmarking-platform/pull/215
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (7868) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7868) - mean (76ms) : 72, 79
master - mean (76ms) : 70, 83
section Bailout
This PR (7868) - mean (81ms) : 77, 86
master - mean (79ms) : 73, 85
section CallTarget+Inlining+NGEN
This PR (7868) - mean (1,083ms) : 980, 1186
master - mean (1,067ms) : 976, 1158
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7868) - mean (119ms) : 114, 125
master - mean (118ms) : 111, 125
section Bailout
This PR (7868) - mean (121ms) : 115, 127
master - mean (118ms) : 111, 125
section CallTarget+Inlining+NGEN
This PR (7868) - mean (775ms) : 736, 813
master - mean (758ms) : 720, 796
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7868) - mean (107ms) : 101, 114
master - mean (105ms) : 98, 111
section Bailout
This PR (7868) - mean (108ms) : 102, 113
master - mean (107ms) : 100, 114
section CallTarget+Inlining+NGEN
This PR (7868) - mean (715ms) : 680, 749
master - mean (719ms) : 677, 762
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7868) - mean (104ms) : 99, 109
master - mean (106ms) : 98, 114
section Bailout
This PR (7868) - mean (106ms) : 101, 111
master - mean (105ms) : 98, 112
section CallTarget+Inlining+NGEN
This PR (7868) - mean (694ms) : 662, 726
master - mean (690ms) : 661, 720
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7868) - mean (205ms) : 194, 215
master - mean (195ms) : 192, 198
section Bailout
This PR (7868) - mean (205ms) : 196, 214
master - mean (198ms) : 196, 201
section CallTarget+Inlining+NGEN
This PR (7868) - mean (1,186ms) : 1113, 1259
master - mean (1,129ms) : 1071, 1188
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7868) - mean (293ms) : 271, 315
master - mean (280ms) : 276, 285
section Bailout
This PR (7868) - mean (296ms) : 268, 324
master - mean (281ms) : 276, 287
section CallTarget+Inlining+NGEN
This PR (7868) - mean (1,006ms) : crit, 954, 1058
master - mean (924ms) : 880, 969
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7868) - mean (302ms) : 286, 318
master - mean (274ms) : 267, 280
section Bailout
This PR (7868) - mean (300ms) : crit, 289, 311
master - mean (275ms) : 270, 279
section CallTarget+Inlining+NGEN
This PR (7868) - mean (1,005ms) : crit, 955, 1054
master - mean (902ms) : 860, 944
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7868) - mean (299ms) : 289, 309
master - mean (273ms) : 267, 278
section Bailout
This PR (7868) - mean (301ms) : crit, 287, 316
master - mean (273ms) : 269, 276
section CallTarget+Inlining+NGEN
This PR (7868) - mean (987ms) : crit, 871, 1104
master - mean (839ms) : 813, 865
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
This comment has been minimized.
This comment has been minimized.
This reverts commit 143d7d7.
8fb05f9 to
d209ae2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
[AgentFilter]Reason for change
The
[AgentFilter]is left over from before we moved to benchmarking platform, so delete it.Additionally some benchmarks don't seem like they need to be run every PR, so added separate categories to allow controlling this.
Important
This PR doesn't change which benchmarks we run on PR vs master, it just allows us to do that easily by adding/removing a category
Implementation details
RunBenchmarksNuke target to work with thisTest coverage
This has the same coverage as today