Attempt to fix Samples.GoogleProtobuf errors#6640
Conversation
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6640) - mean (69ms) : 66, 73
. : milestone, 69,
master - mean (69ms) : 66, 73
. : milestone, 69,
section CallTarget+Inlining+NGEN
This PR (6640) - mean (1,001ms) : 977, 1025
. : milestone, 1001,
master - mean (996ms) : 976, 1016
. : milestone, 996,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6640) - mean (103ms) : 100, 106
. : milestone, 103,
master - mean (103ms) : 101, 105
. : milestone, 103,
section CallTarget+Inlining+NGEN
This PR (6640) - mean (672ms) : 657, 686
. : milestone, 672,
master - mean (673ms) : 655, 691
. : milestone, 673,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6640) - mean (89ms) : 87, 91
. : milestone, 89,
master - mean (90ms) : 88, 92
. : milestone, 90,
section CallTarget+Inlining+NGEN
This PR (6640) - mean (633ms) : 617, 648
. : milestone, 633,
master - mean (630ms) : 616, 644
. : milestone, 630,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6640) - mean (190ms) : 187, 194
. : milestone, 190,
master - mean (191ms) : 186, 196
. : milestone, 191,
section CallTarget+Inlining+NGEN
This PR (6640) - mean (1,104ms) : 1074, 1134
. : milestone, 1104,
master - mean (1,104ms) : 1070, 1138
. : milestone, 1104,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6640) - mean (269ms) : 265, 273
. : milestone, 269,
master - mean (270ms) : 265, 274
. : milestone, 270,
section CallTarget+Inlining+NGEN
This PR (6640) - mean (861ms) : 839, 883
. : milestone, 861,
master - mean (865ms) : 832, 897
. : milestone, 865,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (6640) - mean (262ms) : 257, 267
. : milestone, 262,
master - mean (261ms) : 257, 265
. : milestone, 261,
section CallTarget+Inlining+NGEN
This PR (6640) - mean (850ms) : 816, 883
. : milestone, 850,
master - mean (845ms) : 814, 877
. : milestone, 845,
|
Datadog ReportBranch report: ✅ 0 Failed, 151909 Passed, 583 Skipped, 2h 31m 25.13s Total Time New Flaky Tests (1)
|
Benchmarks Report for tracer 🐌Benchmarks for #6640 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations
|
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 | 41.66 KB | 41.89 KB | 234 B | 0.56% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 585μs | 3.02μs | 15.4μs | 0.576 | 0 | 0 | 41.58 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 652μs | 3.57μs | 22.6μs | 0.332 | 0 | 0 | 41.66 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 839μs | 3.27μs | 12.7μs | 8.13 | 2.57 | 0.428 | 53.26 KB |
| #6640 | WriteAndFlushEnrichedTraces |
net6.0 | 574μs | 3.04μs | 19μs | 0.539 | 0 | 0 | 41.66 KB |
| #6640 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 699μs | 3.96μs | 28.8μs | 0.331 | 0 | 0 | 41.89 KB |
| #6640 | WriteAndFlushEnrichedTraces |
net472 | 864μs | 3.78μs | 16.5μs | 8.25 | 2.6 | 0.434 | 53.3 KB |
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteNonQuery |
net6.0 | 1.3μs | 1.6ns | 6.19ns | 0.0142 | 0 | 0 | 1.02 KB |
| master | ExecuteNonQuery |
netcoreapp3.1 | 1.73μs | 2.25ns | 8.73ns | 0.0131 | 0 | 0 | 1.02 KB |
| master | ExecuteNonQuery |
net472 | 2.08μs | 2.41ns | 9.32ns | 0.156 | 0.00103 | 0 | 987 B |
| #6640 | ExecuteNonQuery |
net6.0 | 1.29μs | 1.31ns | 5.05ns | 0.0142 | 0 | 0 | 1.02 KB |
| #6640 | ExecuteNonQuery |
netcoreapp3.1 | 1.83μs | 1.29ns | 4.81ns | 0.0138 | 0 | 0 | 1.02 KB |
| #6640 | ExecuteNonQuery |
net472 | 2.09μs | 2.83ns | 11ns | 0.156 | 0.00105 | 0 | 987 B |
Benchmarks.Trace.ElasticsearchBenchmark - Faster 🎉 Same allocations ✔️
Faster 🎉 in #6640
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0
1.116
1,285.29
1,151.82
| Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 | 1.116 | 1,285.29 | 1,151.82 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | CallElasticsearch |
net6.0 | 1.29μs | 0.399ns | 1.49ns | 0.0139 | 0 | 0 | 976 B |
| master | CallElasticsearch |
netcoreapp3.1 | 1.55μs | 0.7ns | 2.71ns | 0.0133 | 0 | 0 | 976 B |
| master | CallElasticsearch |
net472 | 2.53μs | 1.8ns | 6.98ns | 0.158 | 0 | 0 | 995 B |
| master | CallElasticsearchAsync |
net6.0 | 1.32μs | 0.589ns | 2.21ns | 0.0132 | 0 | 0 | 952 B |
| master | CallElasticsearchAsync |
netcoreapp3.1 | 1.69μs | 0.576ns | 2.23ns | 0.0136 | 0 | 0 | 1.02 KB |
| master | CallElasticsearchAsync |
net472 | 2.61μs | 1.17ns | 4.55ns | 0.167 | 0 | 0 | 1.05 KB |
| #6640 | CallElasticsearch |
net6.0 | 1.15μs | 0.505ns | 1.89ns | 0.0133 | 0 | 0 | 976 B |
| #6640 | CallElasticsearch |
netcoreapp3.1 | 1.48μs | 1.16ns | 4.64ns | 0.0133 | 0 | 0 | 976 B |
| #6640 | CallElasticsearch |
net472 | 2.65μs | 2.07ns | 8.01ns | 0.158 | 0 | 0 | 995 B |
| #6640 | CallElasticsearchAsync |
net6.0 | 1.31μs | 0.876ns | 3.28ns | 0.0132 | 0 | 0 | 952 B |
| #6640 | CallElasticsearchAsync |
netcoreapp3.1 | 1.65μs | 0.487ns | 1.82ns | 0.0141 | 0 | 0 | 1.02 KB |
| #6640 | CallElasticsearchAsync |
net472 | 2.65μs | 1.74ns | 6.74ns | 0.166 | 0 | 0 | 1.05 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteAsync |
net6.0 | 1.35μs | 0.732ns | 2.74ns | 0.0136 | 0 | 0 | 952 B |
| master | ExecuteAsync |
netcoreapp3.1 | 1.55μs | 0.752ns | 2.61ns | 0.0133 | 0 | 0 | 952 B |
| master | ExecuteAsync |
net472 | 1.89μs | 0.828ns | 3.1ns | 0.145 | 0 | 0 | 915 B |
| #6640 | ExecuteAsync |
net6.0 | 1.25μs | 0.403ns | 1.51ns | 0.0131 | 0 | 0 | 952 B |
| #6640 | ExecuteAsync |
netcoreapp3.1 | 1.64μs | 0.95ns | 3.68ns | 0.0124 | 0 | 0 | 952 B |
| #6640 | ExecuteAsync |
net472 | 1.82μs | 0.669ns | 2.59ns | 0.144 | 0 | 0 | 915 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendAsync |
net6.0 | 4.37μs | 1.22ns | 4.71ns | 0.0327 | 0 | 0 | 2.31 KB |
| master | SendAsync |
netcoreapp3.1 | 5.28μs | 1.56ns | 5.85ns | 0.0397 | 0 | 0 | 2.85 KB |
| master | SendAsync |
net472 | 7.32μs | 2.38ns | 8.89ns | 0.494 | 0 | 0 | 3.12 KB |
| #6640 | SendAsync |
net6.0 | 4.39μs | 2.13ns | 8.23ns | 0.033 | 0 | 0 | 2.31 KB |
| #6640 | SendAsync |
netcoreapp3.1 | 5.2μs | 3.11ns | 11.2ns | 0.0391 | 0 | 0 | 2.85 KB |
| #6640 | SendAsync |
net472 | 7.32μs | 2.35ns | 9.09ns | 0.494 | 0 | 0 | 3.12 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 1.48μs | 1.44ns | 5.4ns | 0.0229 | 0 | 0 | 1.64 KB |
| master | EnrichedLog |
netcoreapp3.1 | 2.17μs | 1.22ns | 4.57ns | 0.0229 | 0 | 0 | 1.64 KB |
| master | EnrichedLog |
net472 | 2.52μs | 1.17ns | 4.53ns | 0.249 | 0 | 0 | 1.57 KB |
| #6640 | EnrichedLog |
net6.0 | 1.46μs | 1.14ns | 4.26ns | 0.023 | 0 | 0 | 1.64 KB |
| #6640 | EnrichedLog |
netcoreapp3.1 | 2.07μs | 1.09ns | 4.08ns | 0.0218 | 0 | 0 | 1.64 KB |
| #6640 | EnrichedLog |
net472 | 2.6μs | 0.813ns | 3.04ns | 0.249 | 0 | 0 | 1.57 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 111μs | 57.1ns | 206ns | 0.0556 | 0 | 0 | 4.28 KB |
| master | EnrichedLog |
netcoreapp3.1 | 116μs | 161ns | 625ns | 0.0578 | 0 | 0 | 4.28 KB |
| master | EnrichedLog |
net472 | 150μs | 124ns | 482ns | 0.671 | 0.224 | 0 | 4.46 KB |
| #6640 | EnrichedLog |
net6.0 | 112μs | 147ns | 569ns | 0.0559 | 0 | 0 | 4.28 KB |
| #6640 | EnrichedLog |
netcoreapp3.1 | 116μs | 220ns | 852ns | 0.0574 | 0 | 0 | 4.28 KB |
| #6640 | EnrichedLog |
net472 | 150μs | 166ns | 643ns | 0.678 | 0.226 | 0 | 4.46 KB |
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 3.03μs | 1.03ns | 3.72ns | 0.0303 | 0 | 0 | 2.2 KB |
| master | EnrichedLog |
netcoreapp3.1 | 4.16μs | 1.72ns | 6.68ns | 0.0294 | 0 | 0 | 2.2 KB |
| master | EnrichedLog |
net472 | 4.78μs | 1.26ns | 4.89ns | 0.319 | 0 | 0 | 2.02 KB |
| #6640 | EnrichedLog |
net6.0 | 3.04μs | 1.33ns | 5.15ns | 0.0305 | 0 | 0 | 2.2 KB |
| #6640 | EnrichedLog |
netcoreapp3.1 | 4.25μs | 1.97ns | 7.62ns | 0.0277 | 0 | 0 | 2.2 KB |
| #6640 | EnrichedLog |
net472 | 4.89μs | 1.42ns | 5.49ns | 0.319 | 0 | 0 | 2.02 KB |
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendReceive |
net6.0 | 1.42μs | 0.757ns | 2.83ns | 0.0156 | 0 | 0 | 1.14 KB |
| master | SendReceive |
netcoreapp3.1 | 1.84μs | 0.592ns | 2.29ns | 0.0149 | 0 | 0 | 1.14 KB |
| master | SendReceive |
net472 | 2.06μs | 0.608ns | 2.35ns | 0.183 | 0 | 0 | 1.16 KB |
| #6640 | SendReceive |
net6.0 | 1.47μs | 0.529ns | 1.98ns | 0.0162 | 0 | 0 | 1.14 KB |
| #6640 | SendReceive |
netcoreapp3.1 | 1.78μs | 0.436ns | 1.63ns | 0.0152 | 0 | 0 | 1.14 KB |
| #6640 | SendReceive |
net472 | 2.14μs | 0.871ns | 3.37ns | 0.183 | 0 | 0 | 1.16 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 2.83μs | 0.849ns | 3.29ns | 0.0212 | 0 | 0 | 1.6 KB |
| master | EnrichedLog |
netcoreapp3.1 | 3.87μs | 1.73ns | 6.72ns | 0.021 | 0 | 0 | 1.65 KB |
| master | EnrichedLog |
net472 | 4.51μs | 1.83ns | 6.83ns | 0.322 | 0 | 0 | 2.04 KB |
| #6640 | EnrichedLog |
net6.0 | 2.8μs | 0.868ns | 3.36ns | 0.0225 | 0 | 0 | 1.6 KB |
| #6640 | EnrichedLog |
netcoreapp3.1 | 3.73μs | 2.49ns | 9.33ns | 0.0224 | 0 | 0 | 1.65 KB |
| #6640 | EnrichedLog |
net472 | 4.39μs | 2.42ns | 9.06ns | 0.323 | 0 | 0 | 2.04 KB |
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️
Slower ⚠️ in #6640
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1
1.127
664.68
749.06
| Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 | 1.127 | 664.68 | 749.06 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartFinishSpan |
net6.0 | 392ns | 0.494ns | 1.91ns | 0.00802 | 0 | 0 | 576 B |
| master | StartFinishSpan |
netcoreapp3.1 | 548ns | 0.609ns | 2.36ns | 0.00776 | 0 | 0 | 576 B |
| master | StartFinishSpan |
net472 | 598ns | 1.66ns | 6.42ns | 0.0917 | 0 | 0 | 578 B |
| master | StartFinishScope |
net6.0 | 467ns | 0.605ns | 2.34ns | 0.00987 | 0 | 0 | 696 B |
| master | StartFinishScope |
netcoreapp3.1 | 665ns | 1.21ns | 4.71ns | 0.00939 | 0 | 0 | 696 B |
| master | StartFinishScope |
net472 | 873ns | 2.22ns | 8.61ns | 0.104 | 0 | 0 | 658 B |
| #6640 | StartFinishSpan |
net6.0 | 420ns | 0.289ns | 1.12ns | 0.00798 | 0 | 0 | 576 B |
| #6640 | StartFinishSpan |
netcoreapp3.1 | 537ns | 0.37ns | 1.43ns | 0.0078 | 0 | 0 | 576 B |
| #6640 | StartFinishSpan |
net472 | 587ns | 0.337ns | 1.17ns | 0.0916 | 0 | 0 | 578 B |
| #6640 | StartFinishScope |
net6.0 | 487ns | 0.283ns | 1.1ns | 0.00974 | 0 | 0 | 696 B |
| #6640 | StartFinishScope |
netcoreapp3.1 | 749ns | 0.568ns | 2.2ns | 0.00942 | 0 | 0 | 696 B |
| #6640 | StartFinishScope |
net472 | 792ns | 0.827ns | 3.1ns | 0.104 | 0 | 0 | 658 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunOnMethodBegin |
net6.0 | 722ns | 1.46ns | 5.66ns | 0.00987 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
netcoreapp3.1 | 932ns | 2.05ns | 7.96ns | 0.0089 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
net472 | 1.06μs | 2.4ns | 9.29ns | 0.104 | 0 | 0 | 658 B |
| #6640 | RunOnMethodBegin |
net6.0 | 685ns | 0.57ns | 2.21ns | 0.00989 | 0 | 0 | 696 B |
| #6640 | RunOnMethodBegin |
netcoreapp3.1 | 964ns | 0.68ns | 2.64ns | 0.00915 | 0 | 0 | 696 B |
| #6640 | RunOnMethodBegin |
net472 | 1.07μs | 0.515ns | 1.93ns | 0.104 | 0 | 0 | 658 B |
|
took those changes onboard in #6647 |
## Summary of changes same as #6166 except with: - fixes by @bouwkast from #6640 - gave up on generating the proto object file on the fly - fixed a couple inconsistency issues I discovered when system-testing vs the Java instrumentation (depth calculation was one-off, int value of types were not in sync, and we were not using the full name for references) --------- Co-authored-by: Steven Bouwkamp <steven.bouwkamp@datadoghq.com>
## Summary of changes same as #6166 except with: - fixes by @bouwkast from #6640 - gave up on generating the proto object file on the fly - fixed a couple inconsistency issues I discovered when system-testing vs the Java instrumentation (depth calculation was one-off, int value of types were not in sync, and we were not using the full name for references) --------- Co-authored-by: Steven Bouwkamp <steven.bouwkamp@datadoghq.com>
## Summary of changes same as #6166 except with: - fixes by @bouwkast from #6640 - gave up on generating the proto object file on the fly - fixed a couple inconsistency issues I discovered when system-testing vs the Java instrumentation (depth calculation was one-off, int value of types were not in sync, and we were not using the full name for references) --------- Co-authored-by: Steven Bouwkamp <steven.bouwkamp@datadoghq.com>
Summary of changes
Ran:
GeneratePackageVersions -IncludePackages Google.ProtobufAdds the SampleHelper to the
Samples.GoogleProtobufinslnReason for change
Builds are throwing:
tracer\test\test-applications\integrations\Samples.GoogleProtobuf\Program.cs(4,14): error CS0246: The type or namespace name 'Sample' could not be found (are you missing a using directive or aLocally I couldn't compile the
Samples.GoogleProtobuf, but that seemed to go away.Appears that
GeneratePackageVersionswasn't run for it.Implementation details
Ran:
GeneratePackageVersions -IncludePackages Google.ProtobufWent through files and staged the stuff that looked like it was the missing Protobuf stuff
Test coverage
Hoping that the build doesn't throw the error anymore
Other details