Reapply "Add Google Protobuf Instrumentation"#6647
Conversation
Datadog ReportBranch report: ✅ 0 Failed, 570476 Passed, 5469 Skipped, 45h 58m 33.39s Total Time |
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6647) - mean (71ms) : 62, 81
. : milestone, 71,
master - mean (69ms) : 65, 73
. : milestone, 69,
section CallTarget+Inlining+NGEN
This PR (6647) - mean (1,015ms) : 956, 1074
. : milestone, 1015,
master - mean (1,001ms) : 978, 1024
. : milestone, 1001,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6647) - mean (103ms) : 101, 105
. : milestone, 103,
master - mean (102ms) : 100, 105
. : milestone, 102,
section CallTarget+Inlining+NGEN
This PR (6647) - mean (679ms) : 659, 698
. : milestone, 679,
master - mean (676ms) : 659, 693
. : milestone, 676,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6647) - mean (90ms) : 87, 92
. : milestone, 90,
master - mean (89ms) : 87, 91
. : milestone, 89,
section CallTarget+Inlining+NGEN
This PR (6647) - mean (631ms) : 611, 651
. : milestone, 631,
master - mean (632ms) : 616, 648
. : milestone, 632,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6647) - mean (191ms) : 185, 197
. : milestone, 191,
master - mean (191ms) : 187, 195
. : milestone, 191,
section CallTarget+Inlining+NGEN
This PR (6647) - mean (1,105ms) : 1074, 1135
. : milestone, 1105,
master - mean (1,108ms) : 1072, 1144
. : milestone, 1108,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6647) - mean (271ms) : 266, 276
. : milestone, 271,
master - mean (270ms) : 264, 276
. : milestone, 270,
section CallTarget+Inlining+NGEN
This PR (6647) - mean (862ms) : 832, 892
. : milestone, 862,
master - mean (863ms) : 834, 892
. : milestone, 863,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6647) - mean (263ms) : 259, 267
. : milestone, 263,
master - mean (263ms) : 259, 267
. : milestone, 263,
section CallTarget+Inlining+NGEN
This PR (6647) - mean (839ms) : 810, 868
. : milestone, 839,
master - mean (844ms) : 806, 882
. : milestone, 844,
|
bouwkast
left a comment
There was a problem hiding this comment.
Looks like some merge conflicts
Benchmarks Report for tracer 🐌Benchmarks for #6647 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.SpanBenchmark - Slower
|
| Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 | 1.126 | 557.60 | 627.92 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartFinishSpan |
net6.0 | 391ns | 0.461ns | 1.79ns | 0.00817 | 0 | 0 | 576 B |
| master | StartFinishSpan |
netcoreapp3.1 | 557ns | 0.82ns | 3.18ns | 0.00784 | 0 | 0 | 576 B |
| master | StartFinishSpan |
net472 | 584ns | 1.27ns | 4.91ns | 0.0915 | 0 | 0 | 578 B |
| master | StartFinishScope |
net6.0 | 515ns | 0.487ns | 1.89ns | 0.00983 | 0 | 0 | 696 B |
| master | StartFinishScope |
netcoreapp3.1 | 742ns | 1.68ns | 6.07ns | 0.00932 | 0 | 0 | 696 B |
| master | StartFinishScope |
net472 | 839ns | 2.27ns | 8.78ns | 0.105 | 0 | 0 | 658 B |
| #6647 | StartFinishSpan |
net6.0 | 391ns | 0.637ns | 2.47ns | 0.00817 | 0 | 0 | 576 B |
| #6647 | StartFinishSpan |
netcoreapp3.1 | 629ns | 1.08ns | 4.19ns | 0.0079 | 0 | 0 | 576 B |
| #6647 | StartFinishSpan |
net472 | 626ns | 1.4ns | 5.26ns | 0.0917 | 0 | 0 | 578 B |
| #6647 | StartFinishScope |
net6.0 | 502ns | 0.829ns | 3.1ns | 0.00992 | 0 | 0 | 696 B |
| #6647 | StartFinishScope |
netcoreapp3.1 | 698ns | 1.69ns | 6.09ns | 0.00936 | 0 | 0 | 696 B |
| #6647 | StartFinishScope |
net472 | 778ns | 1.96ns | 7.58ns | 0.104 | 0 | 0 | 658 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Faster 🎉 Same allocations ✔️
Faster 🎉 in #6647
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472
1.239
1,162.68
938.31
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0
1.175
731.18
622.02
| Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 | 1.239 | 1,162.68 | 938.31 | |
| Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 | 1.175 | 731.18 | 622.02 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunOnMethodBegin |
net6.0 | 733ns | 1.3ns | 5.05ns | 0.00951 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
netcoreapp3.1 | 907ns | 1.48ns | 5.73ns | 0.00936 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
net472 | 1.16μs | 1.52ns | 5.88ns | 0.105 | 0 | 0 | 658 B |
| #6647 | RunOnMethodBegin |
net6.0 | 623ns | 1.2ns | 4.66ns | 0.00979 | 0 | 0 | 696 B |
| #6647 | RunOnMethodBegin |
netcoreapp3.1 | 955ns | 1.97ns | 7.64ns | 0.00915 | 0 | 0 | 696 B |
| #6647 | RunOnMethodBegin |
net472 | 940ns | 3.23ns | 11.6ns | 0.104 | 0 | 0 | 658 B |
ac641f8 to
d45d715
Compare
This reverts commit d7349fe.
bouwkast
left a comment
There was a problem hiding this comment.
LGTM just reviewed the new changes as the rest was already reviewed previously
## Summary of changes Attempts to remove the Protobuf Schema tagging instrumentation that was recently added in #6647 Recommended to revert this PR when we do the fix. ## Reason for change Appears to be throwing errors due to MissingMethodExceptions on the DuckTypes. We need time to investigate it and fix. This seems to be the simplest approach as there were multiple PRs. I don't see any code that hits this now so should be good. Additionally, the Sample project is very flaky I attempted to remove that code generating the proto file. ## Implementation details - Comment out instrumented method attributes to disable instrumentations and leave them there for less confusion (maybe?) - Remove Proto file generation in sample which I think is the cause of the CI build error - Skip tests in CI. ## Test coverage - Skipped now. ## Other details <!-- Fixes #{issue} --> <!--⚠️ Note: where possible, please obtain 2 approvals prior to merging. Unless CODEOWNERS specifies otherwise, for external teams it is typically best to have one review from a team member, and one review from apm-dotnet. Trivial changes do not require 2 reviews. -->
## Summary of changes same as #6166 except with: - fixes by @bouwkast from #6640 - gave up on generating the proto object file on the fly - fixed a couple inconsistency issues I discovered when system-testing vs the Java instrumentation (depth calculation was one-off, int value of types were not in sync, and we were not using the full name for references) --------- Co-authored-by: Steven Bouwkamp <steven.bouwkamp@datadoghq.com>
## Summary of changes same as #6166 except with: - fixes by @bouwkast from #6640 - gave up on generating the proto object file on the fly - fixed a couple inconsistency issues I discovered when system-testing vs the Java instrumentation (depth calculation was one-off, int value of types were not in sync, and we were not using the full name for references) --------- Co-authored-by: Steven Bouwkamp <steven.bouwkamp@datadoghq.com>
## Summary of changes Attempts to remove the Protobuf Schema tagging instrumentation that was recently added in #6647 Recommended to revert this PR when we do the fix. ## Reason for change Appears to be throwing errors due to MissingMethodExceptions on the DuckTypes. We need time to investigate it and fix. This seems to be the simplest approach as there were multiple PRs. I don't see any code that hits this now so should be good. Additionally, the Sample project is very flaky I attempted to remove that code generating the proto file. ## Implementation details - Comment out instrumented method attributes to disable instrumentations and leave them there for less confusion (maybe?) - Remove Proto file generation in sample which I think is the cause of the CI build error - Skip tests in CI. ## Test coverage - Skipped now. ## Other details <!-- Fixes #{issue} --> <!--⚠️ Note: where possible, please obtain 2 approvals prior to merging. Unless CODEOWNERS specifies otherwise, for external teams it is typically best to have one review from a team member, and one review from apm-dotnet. Trivial changes do not require 2 reviews. -->
Summary of changes
same as #6166 except with:
Samples.GoogleProtobuferrors #6640