[ASM][IAST] Support manual JSON deserialisation (Newtonsoft.Json) by e-n-0 · Pull Request #5238 · DataDog/dd-trace-dotnet

e-n-0 · 2024-02-26T10:12:05Z

Summary of changes

Taint strings deserialised with a tainted Json from the Newtonsoft.Json assembly.

Reason for change

Add support of Json in IAST for the Newtonsoft library.

Implementation details

Go Recursively through the whole parsed JsonDocument to identify the tokens that are strings, and then taint it.
The implementation is almost the same for the 3 type of json object where the Parse method is available. These are the object where we support the tainting:
- JObject
- JArray
- JToken

Test coverage

Unit tests on the Parse() method for each JObject, JArray and JToken
Integration test /Iast/NewtonsoftJsonParseTainting with a Command Injection vulnerability to test the Json tainting

datadog-ddstaging · 2024-02-26T10:27:03Z

Datadog Report

Branch report: flavien/iast/newtonsoft_json
Commit report: 64ca995
Test service: dd-trace-dotnet

✅ 0 Failed, 337348 Passed, 1571 Skipped, 51m 51.53s Wall Time

andrewlock · 2024-02-26T11:02:30Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5238) - mean (74ms)  : 65, 82
     .   : milestone, 74,
    master - mean (74ms)  : 66, 82
     .   : milestone, 74,

    section CallTarget+Inlining+NGEN
    This PR (5238) - mean (987ms)  : 968, 1005
     .   : milestone, 987,
    master - mean (995ms)  : 967, 1024
     .   : milestone, 995,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5238) - mean (111ms)  : 107, 116
     .   : milestone, 111,
    master - mean (111ms)  : 107, 115
     .   : milestone, 111,

    section CallTarget+Inlining+NGEN
    This PR (5238) - mean (714ms)  : 693, 734
     .   : milestone, 714,
    master - mean (716ms)  : 695, 736
     .   : milestone, 716,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5238) - mean (94ms)  : 91, 97
     .   : milestone, 94,
    master - mean (94ms)  : 91, 97
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (5238) - mean (669ms)  : 643, 696
     .   : milestone, 669,
    master - mean (675ms)  : 648, 701
     .   : milestone, 675,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5238) - mean (188ms)  : 185, 191
     .   : milestone, 188,
    master - mean (188ms)  : 183, 192
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (5238) - mean (1,056ms)  : 1035, 1076
     .   : milestone, 1056,
    master - mean (1,050ms)  : 1026, 1074
     .   : milestone, 1050,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5238) - mean (271ms)  : 266, 276
     .   : milestone, 271,
    master - mean (269ms)  : 262, 275
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (5238) - mean (862ms)  : 842, 882
     .   : milestone, 862,
    master - mean (860ms)  : 829, 891
     .   : milestone, 860,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5238) - mean (259ms)  : 256, 263
     .   : milestone, 259,
    master - mean (257ms)  : 252, 263
     .   : milestone, 257,

    section CallTarget+Inlining+NGEN
    This PR (5238) - mean (846ms)  : 823, 868
     .   : milestone, 846,
    master - mean (842ms)  : 817, 867
     .   : milestone, 842,

andrewlock · 2024-02-26T13:46:28Z

Benchmarks Report 🐌

Benchmarks for #5238 compared to master:

3 benchmarks are faster, with geometric mean 1.220
3 benchmarks are slower, with geometric mean 1.260
1 benchmarks have fewer allocations
2 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.75μs	48.7ns	312ns	0.0211	0.00845	0	7.49 KB
master	`StartStopWithChild`	netcoreapp3.1	10.5μs	38.7ns	145ns	0.0248	0.00992	0	7.58 KB
master	`StartStopWithChild`	net472	16.9μs	61.1ns	237ns	1.33	0.342	0.125	7.96 KB
#5238	`StartStopWithChild`	net6.0	8.71μs	47.3ns	263ns	0.0337	0.0168	0	7.49 KB
#5238	`StartStopWithChild`	netcoreapp3.1	10.6μs	56ns	291ns	0.0199	0.00997	0	7.59 KB
#5238	`StartStopWithChild`	net472	17.1μs	73ns	283ns	1.32	0.352	0.101	7.95 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	466μs	656ns	2.54μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	605μs	242ns	937ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	800μs	421ns	1.57μs	0.398	3.3 KB
#5238	`WriteAndFlushEnrichedTraces`	net6.0	481μs	505ns	1.96μs	0	2.7 KB
#5238	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	621μs	290ns	1.05μs	0	2.7 KB
#5238	`WriteAndFlushEnrichedTraces`	net472	808μs	183ns	684ns	0.403	3.3 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5238

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net472	1.345	170.65	229.55
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody‑net6.0	1.230	2,995.22	3,685.25

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	40.8μs	14.1ns	54.7ns	0.0203	0	2.36 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	43.9μs	53.9ns	209ns	0.0218	0	2.34 KB
master	`AllCycleSimpleBody`	net472	46.8μs	22.7ns	88ns	0.373	0	2.41 KB
master	`AllCycleMoreComplexBody`	net6.0	213μs	325ns	1.17μs	0.106	0	9.84 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	225μs	87ns	326ns	0.112	0	9.73 KB
master	`AllCycleMoreComplexBody`	net472	241μs	88.7ns	343ns	1.57	0	9.91 KB
master	`ObjectExtractorSimpleBody`	net6.0	144ns	0.0908ns	0.328ns	0.00395	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	196ns	0.124ns	0.464ns	0.00375	0	272 B
master	`ObjectExtractorSimpleBody`	net472	171ns	0.0749ns	0.29ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	2.99μs	1.71ns	6.39ns	0.0529	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.97μs	1.83ns	6.83ns	0.0496	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	3.92μs	2.44ns	9.47ns	0.601	0.00586	3.8 KB
#5238	`AllCycleSimpleBody`	net6.0	40.1μs	17.7ns	66.2ns	0.0206	0	2.36 KB
#5238	`AllCycleSimpleBody`	netcoreapp3.1	44.5μs	27.9ns	105ns	0.0221	0	2.34 KB
#5238	`AllCycleSimpleBody`	net472	47.2μs	13.6ns	52.7ns	0.378	0	2.41 KB
#5238	`AllCycleMoreComplexBody`	net6.0	215μs	50.5ns	189ns	0.107	0	9.84 KB
#5238	`AllCycleMoreComplexBody`	netcoreapp3.1	226μs	265ns	1.03μs	0.112	0	9.73 KB
#5238	`AllCycleMoreComplexBody`	net472	242μs	54.9ns	205ns	1.57	0	9.91 KB
#5238	`ObjectExtractorSimpleBody`	net6.0	145ns	0.0854ns	0.319ns	0.00389	0	280 B
#5238	`ObjectExtractorSimpleBody`	netcoreapp3.1	211ns	0.401ns	1.55ns	0.00371	0	272 B
#5238	`ObjectExtractorSimpleBody`	net472	230ns	0.0944ns	0.353ns	0.0446	0	281 B
#5238	`ObjectExtractorMoreComplexBody`	net6.0	3.68μs	1.52ns	5.68ns	0.0534	0	3.78 KB
#5238	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.02μs	1.58ns	6.13ns	0.0502	0	3.69 KB
#5238	`ObjectExtractorMoreComplexBody`	net472	3.78μs	1.95ns	7.55ns	0.603	0.00567	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EncodeArgs`	net6.0	52μs	85.6ns	332ns	0.416	0	30.94 KB
master	`EncodeArgs`	netcoreapp3.1	69.4μs	77.4ns	300ns	0.417	0	31.47 KB
master	`EncodeArgs`	net472	84.1μs	38.8ns	145ns	5.09	0.0841	32.27 KB
master	`EncodeLegacyArgs`	net6.0	125μs	211ns	730ns	0.438	0	33.89 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	153μs	202ns	755ns	0.455	0	34.09 KB
master	`EncodeLegacyArgs`	net472	211μs	130ns	470ns	5.46	0.42	34.99 KB
#5238	`EncodeArgs`	net6.0	52.6μs	60.9ns	228ns	0.442	0	30.94 KB
#5238	`EncodeArgs`	netcoreapp3.1	69.1μs	110ns	425ns	0.412	0	31.47 KB
#5238	`EncodeArgs`	net472	84.4μs	47.5ns	184ns	5.09	0.0841	32.27 KB
#5238	`EncodeLegacyArgs`	net6.0	123μs	220ns	793ns	0.426	0	33.89 KB
#5238	`EncodeLegacyArgs`	netcoreapp3.1	157μs	516ns	2μs	0.466	0	34.09 KB
#5238	`EncodeLegacyArgs`	net472	215μs	654ns	2.53μs	5.53	0.417	34.99 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunWafRealisticBenchmark`	net6.0	190μs	200ns	774ns	0.0943	6.51 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	203μs	325ns	1.21μs	0	6.49 KB
master	`RunWafRealisticBenchmark`	net472	222μs	84ns	314ns	0.999	6.59 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	127μs	50.2ns	194ns	0.0626	4.15 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	134μs	184ns	713ns	0	4.14 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	146μs	38.8ns	145ns	0.659	4.19 KB
#5238	`RunWafRealisticBenchmark`	net6.0	193μs	110ns	398ns	0.095	6.51 KB
#5238	`RunWafRealisticBenchmark`	netcoreapp3.1	206μs	423ns	1.64μs	0	6.49 KB
#5238	`RunWafRealisticBenchmark`	net472	223μs	129ns	498ns	1	6.59 KB
#5238	`RunWafRealisticBenchmarkWithAttack`	net6.0	126μs	127ns	491ns	0	4.15 KB
#5238	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	133μs	66.8ns	241ns	0	4.14 KB
#5238	`RunWafRealisticBenchmarkWithAttack`	net472	147μs	84.9ns	329ns	0.661	4.19 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	170μs	135ns	525ns	0.254	18.26 KB
master	`SendRequest`	netcoreapp3.1	189μs	316ns	1.23μs	0.191	20.42 KB
master	`SendRequest`	net472	0.000679ns	0.000335ns	0.0013ns	0	0 b
#5238	`SendRequest`	net6.0	172μs	307ns	1.19μs	0.172	18.26 KB
#5238	`SendRequest`	netcoreapp3.1	191μs	188ns	678ns	0.192	20.42 KB
#5238	`SendRequest`	net472	0.000797ns	0.000258ns	0.001ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	551μs	1.09μs	4.23μs	0.543	0	0	41.55 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	652μs	1.95μs	7.55μs	0.322	0	0	41.76 KB
master	`WriteAndFlushEnrichedTraces`	net472	845μs	4.19μs	17.8μs	8.08	2.55	0.425	53.24 KB
#5238	`WriteAndFlushEnrichedTraces`	net6.0	539μs	812ns	3.15μs	0.534	0	0	41.54 KB
#5238	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	660μs	1.11μs	4.3μs	0.332	0	0	41.66 KB
#5238	`WriteAndFlushEnrichedTraces`	net472	881μs	2.63μs	9.48μs	8.25	2.6	0.434	53.36 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.09μs	0.689ns	2.67ns	0.0104	776 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.57μs	0.871ns	3.37ns	0.0103	776 B
master	`ExecuteNonQuery`	net472	1.78μs	1.47ns	5.5ns	0.117	738 B
#5238	`ExecuteNonQuery`	net6.0	1.13μs	0.393ns	1.52ns	0.0107	776 B
#5238	`ExecuteNonQuery`	netcoreapp3.1	1.45μs	0.745ns	2.79ns	0.0102	776 B
#5238	`ExecuteNonQuery`	net472	1.76μs	3.24ns	12.5ns	0.117	738 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.17μs	0.65ns	2.43ns	0.0134	944 B
master	`CallElasticsearch`	netcoreapp3.1	1.52μs	0.937ns	3.63ns	0.0129	944 B
master	`CallElasticsearch`	net472	2.41μs	0.879ns	3.4ns	0.152	963 B
master	`CallElasticsearchAsync`	net6.0	1.24μs	0.758ns	2.83ns	0.0125	920 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.56μs	1.4ns	5.25ns	0.0132	992 B
master	`CallElasticsearchAsync`	net472	2.7μs	1.08ns	4.17ns	0.161	1.02 KB
#5238	`CallElasticsearch`	net6.0	1.28μs	0.6ns	2.32ns	0.0135	944 B
#5238	`CallElasticsearch`	netcoreapp3.1	1.58μs	1.36ns	4.92ns	0.0126	944 B
#5238	`CallElasticsearch`	net472	2.53μs	1.33ns	5.14ns	0.152	963 B
#5238	`CallElasticsearchAsync`	net6.0	1.22μs	0.569ns	2.13ns	0.0129	920 B
#5238	`CallElasticsearchAsync`	netcoreapp3.1	1.65μs	0.489ns	1.83ns	0.0133	992 B
#5238	`CallElasticsearchAsync`	net472	2.59μs	1.23ns	4.77ns	0.161	1.02 KB

Benchmarks.Trace.GraphQLBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5238

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0	1.204	1,402.61	1,164.69

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.4μs	1.4ns	5.24ns	0.0125	920 B
master	`ExecuteAsync`	netcoreapp3.1	1.72μs	1.76ns	6.58ns	0.0119	920 B
master	`ExecuteAsync`	net472	1.82μs	0.669ns	2.5ns	0.14	883 B
#5238	`ExecuteAsync`	net6.0	1.16μs	0.689ns	2.58ns	0.0128	920 B
#5238	`ExecuteAsync`	netcoreapp3.1	1.6μs	0.74ns	2.77ns	0.012	920 B
#5238	`ExecuteAsync`	net472	1.88μs	0.929ns	3.48ns	0.14	883 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.04μs	1.59ns	5.95ns	0.0283	2.1 KB
master	`SendAsync`	netcoreapp3.1	4.86μs	1.78ns	6.65ns	0.0341	2.64 KB
master	`SendAsync`	net472	7.87μs	4.39ns	17ns	0.524	3.31 KB
#5238	`SendAsync`	net6.0	3.97μs	1.5ns	5.39ns	0.0298	2.1 KB
#5238	`SendAsync`	netcoreapp3.1	4.88μs	1.33ns	4.97ns	0.0343	2.64 KB
#5238	`SendAsync`	net472	7.62μs	1.75ns	6.54ns	0.525	3.31 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #5238

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0	1.257	63,300.00	50,350.00	multimodal

More allocations ⚠️ in #5238

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	203.4 KB	213.06 KB	9.66 KB	4.75%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	204.12 KB	205.99 KB	1.87 KB	0.92%

Fewer allocations 🎉 in #5238

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	60.95 KB	60.49 KB	-464 B	-0.76%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	64.6μs	973ns	9.68μs	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	52.7μs	238ns	857ns	42.64 KB
master	`StringConcatBenchmark`	net472	37.5μs	69.7ns	261ns	60.95 KB
master	`StringConcatAspectBenchmark`	net6.0	267μs	1.24μs	8.03μs	204.12 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	286μs	1.5μs	7.78μs	203.4 KB
master	`StringConcatAspectBenchmark`	net472	239μs	2.96μs	28.1μs	221.18 KB
#5238	`StringConcatBenchmark`	net6.0	50.7μs	250ns	1.22μs	43.44 KB
#5238	`StringConcatBenchmark`	netcoreapp3.1	52.6μs	217ns	783ns	42.64 KB
#5238	`StringConcatBenchmark`	net472	37.7μs	100ns	375ns	60.49 KB
#5238	`StringConcatAspectBenchmark`	net6.0	281μs	1.47μs	6.88μs	205.99 KB
#5238	`StringConcatAspectBenchmark`	netcoreapp3.1	297μs	1.47μs	9.88μs	213.06 KB
#5238	`StringConcatAspectBenchmark`	net472	226μs	1.02μs	3.82μs	221.18 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.5μs	0.622ns	2.33ns	0.0225	1.58 KB
master	`EnrichedLog`	netcoreapp3.1	2.18μs	5.22ns	20.2ns	0.0218	1.58 KB
master	`EnrichedLog`	net472	2.59μs	1.6ns	5.98ns	0.239	1.51 KB
#5238	`EnrichedLog`	net6.0	1.52μs	0.506ns	1.89ns	0.0222	1.58 KB
#5238	`EnrichedLog`	netcoreapp3.1	2.31μs	4.96ns	19.2ns	0.0207	1.58 KB
#5238	`EnrichedLog`	net472	2.56μs	2.69ns	10.4ns	0.239	1.51 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	115μs	125ns	485ns	0.0571	0	4.22 KB
master	`EnrichedLog`	netcoreapp3.1	117μs	90.3ns	350ns	0.0579	0	4.22 KB
master	`EnrichedLog`	net472	147μs	101ns	376ns	0.658	0.219	4.4 KB
#5238	`EnrichedLog`	net6.0	113μs	91.9ns	356ns	0.0564	0	4.22 KB
#5238	`EnrichedLog`	netcoreapp3.1	118μs	115ns	447ns	0	0	4.22 KB
#5238	`EnrichedLog`	net472	146μs	122ns	458ns	0.659	0.22	4.4 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.91μs	1.89ns	7.33ns	0.0291	2.14 KB
master	`EnrichedLog`	netcoreapp3.1	4.19μs	2.03ns	7.87ns	0.0273	2.14 KB
master	`EnrichedLog`	net472	4.83μs	1.91ns	7.14ns	0.309	1.95 KB
#5238	`EnrichedLog`	net6.0	3.1μs	1.3ns	5.02ns	0.0294	2.14 KB
#5238	`EnrichedLog`	netcoreapp3.1	4.18μs	2.03ns	7.84ns	0.0291	2.14 KB
#5238	`EnrichedLog`	net472	4.95μs	7.15ns	27.7ns	0.31	1.95 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.32μs	2.17ns	8.41ns	0.0154	1.11 KB
master	`SendReceive`	netcoreapp3.1	1.72μs	0.806ns	3.02ns	0.0145	1.11 KB
master	`SendReceive`	net472	2.19μs	2.68ns	10.4ns	0.178	1.12 KB
#5238	`SendReceive`	net6.0	1.29μs	0.537ns	2.08ns	0.0155	1.11 KB
#5238	`SendReceive`	netcoreapp3.1	1.82μs	0.51ns	1.97ns	0.0154	1.11 KB
#5238	`SendReceive`	net472	2.08μs	2.27ns	8.78ns	0.178	1.12 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.73μs	0.705ns	2.64ns	0.0219	1.54 KB
master	`EnrichedLog`	netcoreapp3.1	3.88μs	0.787ns	3.05ns	0.0213	1.58 KB
master	`EnrichedLog`	net472	4.32μs	2.39ns	9.26ns	0.313	1.97 KB
#5238	`EnrichedLog`	net6.0	2.83μs	0.82ns	2.96ns	0.0212	1.54 KB
#5238	`EnrichedLog`	netcoreapp3.1	3.93μs	0.913ns	3.53ns	0.0197	1.58 KB
#5238	`EnrichedLog`	net472	4.17μs	2.6ns	10.1ns	0.313	1.97 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5238

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1	1.199	802.09	668.88

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	590ns	0.848ns	3.28ns	0.00749	544 B
master	`StartFinishSpan`	netcoreapp3.1	803ns	1.17ns	4.54ns	0.00725	544 B
master	`StartFinishSpan`	net472	764ns	2.17ns	8.39ns	0.0864	546 B
master	`StartFinishScope`	net6.0	533ns	0.882ns	3.41ns	0.0093	664 B
master	`StartFinishScope`	netcoreapp3.1	827ns	1.5ns	5.82ns	0.00866	664 B
master	`StartFinishScope`	net472	985ns	2.43ns	9.42ns	0.0994	626 B
#5238	`StartFinishSpan`	net6.0	552ns	1.14ns	4.42ns	0.00752	544 B
#5238	`StartFinishSpan`	netcoreapp3.1	668ns	1.67ns	6.46ns	0.00736	544 B
#5238	`StartFinishSpan`	net472	747ns	2.5ns	9.68ns	0.0866	546 B
#5238	`StartFinishScope`	net6.0	517ns	1.18ns	4.58ns	0.0093	664 B
#5238	`StartFinishScope`	netcoreapp3.1	826ns	1.22ns	4.72ns	0.00882	664 B
#5238	`StartFinishScope`	net472	953ns	2.14ns	8.28ns	0.099	626 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5238

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.210	601.81	727.90

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	602ns	1.45ns	5.62ns	0.00943	664 B
master	`RunOnMethodBegin`	netcoreapp3.1	910ns	1.78ns	6.89ns	0.00909	664 B
master	`RunOnMethodBegin`	net472	1.08μs	3.27ns	12.7ns	0.0994	626 B
#5238	`RunOnMethodBegin`	net6.0	729ns	1.78ns	6.91ns	0.00942	664 B
#5238	`RunOnMethodBegin`	netcoreapp3.1	923ns	1.71ns	6.62ns	0.00873	664 B
#5238	`RunOnMethodBegin`	net472	1.06μs	1.79ns	6.93ns	0.099	626 B

andrewlock · 2024-02-27T11:50:10Z

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5238) (10.716M)   : 0, 10715799
    master (10.925M)   : 0, 10925303
    benchmarks/2.9.0 (10.620M)   : 0, 10620397

    section Automatic
    This PR (5238) (7.408M)   : 0, 7408232
    master (7.496M)   : 0, 7496085
    benchmarks/2.9.0 (7.702M)   : 0, 7701617

    section Trace stats
    This PR (5238) (7.683M)   : 0, 7682898
    master (7.872M)   : 0, 7871947

    section Manual
    This PR (5238) (9.387M)   : 0, 9386931
    master (9.635M)   : 0, 9635035

    section Manual + Automatic
    This PR (5238) (7.017M)   : 0, 7017184
    master (7.114M)   : 0, 7113688

    section Version Conflict
    This PR (5238) (6.306M)   : 0, 6306310
    master (6.292M)   : 0, 6292435

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5238) (9.561M)   : 0, 9560502
    benchmarks/2.9.0 (9.526M)   : 0, 9525834

    section Automatic
    This PR (5238) (6.600M)   : 0, 6599510

    section Trace stats
    This PR (5238) (6.975M)   : 0, 6975377

    section Manual
    This PR (5238) (8.447M)   : 0, 8446507

    section Manual + Automatic
    This PR (5238) (6.194M)   : 0, 6193757

    section Version Conflict
    This PR (5238) (5.553M)   : 0, 5552874

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5238) (10.590M)   : 0, 10590171
    master (9.856M)   : 0, 9856447
    benchmarks/2.9.0 (10.533M)   : 0, 10532864

    section Automatic
    This PR (5238) (7.490M)   : 0, 7489975
    master (7.076M)   : 0, 7076232
    benchmarks/2.9.0 (7.815M)   : 0, 7814762

    section Trace stats
    This PR (5238) (7.811M)   : 0, 7811485
    master (7.404M)   : 0, 7404111

    section Manual
    This PR (5238) (9.056M)   : 0, 9055606
    master (8.658M)   : 0, 8657799

    section Manual + Automatic
    This PR (5238) (7.165M)   : 0, 7165391
    master (6.832M)   : 0, 6831716

    section Version Conflict
    This PR (5238) (6.453M)   : 0, 6452996
    master (6.173M)   : 0, 6172929

gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5238) (7.413M)   : 0, 7412521
    master (7.559M)   : 0, 7559179
    benchmarks/2.9.0 (7.859M)   : 0, 7859139

    section No attack
    This PR (5238) (1.842M)   : 0, 1841879
    master (1.844M)   : 0, 1844300
    benchmarks/2.9.0 (3.196M)   : 0, 3196117

    section Attack
    This PR (5238) (1.452M)   : 0, 1451655
    master (1.451M)   : 0, 1451050
    benchmarks/2.9.0 (2.519M)   : 0, 2519149

    section Blocking
    This PR (5238) (3.204M)   : 0, 3203640
    master (3.102M)   : 0, 3101736

    section IAST default
    This PR (5238) (6.450M)   : 0, 6449837
    master (6.434M)   : 0, 6433840

    section IAST full
    This PR (5238) (5.591M)   : 0, 5590738
    master (5.626M)   : 0, 5625514

    section Base vuln
    This PR (5238) (0.928M)   : 0, 927984
    master (0.917M)   : 0, 917115

    section IAST vuln
    This PR (5238) (0.841M)   : 0, 841394
    master (0.881M)   : 0, 881291

andrewlock

LGTM in general, but I think you should test for some of the more esoteric features that isn't "normal" json, just to make sure. It's probably fine, because you're not deserializing to objects, but might be worth testing to be sure

...ns/integrations/Samples.InstrumentedTests/Vulnerabilities/Json/Newtonsoft.Json/ParseTests.cs

tracer/src/Datadog.Trace/Iast/Aspects/Newtonsoft.Json/NewtonsoftJsonAspects.cs

daniel-romano-DD

LGTM. Thank you

commit 832de4b Author: Flavien Darche <11708575+e-n-0@users.noreply.github.com> Date: Tue Mar 12 20:24:21 2024 +0000 [ASM][IAST] Configure maximum IAST Ranges (#5292) * Add configuration key * Use a RangeList in some case to not exceed the max number * Revert some code + implem correct merge * Fix + Add unit and integration tests * Usual macos fix for snapshot * Fix snapshots hashs * Update snapshots (remove other tests as they can't apply different env var values in same run) * Apply comment * Re-integrate integration tests with multiple processes (new fixture) * Add test case for setting MaxRangeCount to zero commit 83f6ab1 Author: Tony Redondo <tony.redondo@datadoghq.com> Date: Tue Mar 12 21:20:39 2024 +0100 [CI Visibility] - Enable snapshot testing of current testing framework implementations (#5226) commit 233695a Author: Daniel Romano <108014683+daniel-romano-DD@users.noreply.github.com> Date: Tue Mar 12 17:06:06 2024 +0100 [IAST] Vulnerability and Evidence truncation (#5302) * Initial implementation * Updated test bundle * Fix test compilation error * Fix snapshot (from rebase) * Fix typo in config value. Updated tests * Fix typo * Refactor converters initialization commit ea31cf5 Author: Anna <anna.yafi@datadoghq.com> Date: Tue Mar 12 16:39:09 2024 +0100 Deactivate benchmark for legacy encoder (#5299) commit d0d713a Author: NachoEchevarria <53266532+NachoEchevarria@users.noreply.github.com> Date: Tue Mar 12 09:25:27 2024 +0100 Set big regex timeouts for tests (#5297) commit d5388d6 Author: Lucas Pimentel <lucas.pimentel@datadoghq.com> Date: Mon Mar 11 15:20:58 2024 -0400 [Tracing] Support configuring `DD_TRACE_ENABLED` remotely (#5181) * add support for remote TraceEnabled setting * fix unrelated typo * add ApmTracingEnabled capability 19 * add missing RCM capability 18 * add mapping * add unit test * add comments to unit test * rename property to match RCM constant * include config in integration tests * fix test json * rewrite tests to use raw values instead of strings commit 2b95f46 Author: Flavien Darche <11708575+e-n-0@users.noreply.github.com> Date: Mon Mar 11 17:47:55 2024 +0100 [ASM][IAST] Support manual JSON deserialisation (Newtonsoft.Json) (#5238) * Add Newtonsoft.Json (non -working yet) * Refactor the tainting proces + add tests * Add the JToken Parse aspect + test * Rename Aspects class + Duck orignal method call * Add integration test * Fix nullability * Fix compilation issue for netfx * Change JSON formatting in ParseTests * Fix a test json format * Refactor NewtonsoftJsonAspects to static constructor commit 0d511f9 Author: Igor Kravchenko <21974069+kr-igor@users.noreply.github.com> Date: Mon Mar 11 09:35:23 2024 -0500 [DSM] - Fixes for IbmMq instrumentation (#5271) * Use byte properties instead of strings * Fixed nullability files * Added some debug info * Fixed lint issues * Added a bit more logs * Using slow byte->sbyte conversion * Added noop headers adapter * Fixed nullability files * Added more logs * Cleaned up debug logs * Removed symlink * Update tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation/IbmMq/IbmMqHeadersAdapter.cs Removed debug code Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com> * Update tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation/IbmMq/IbmMqHeadersAdapter.cs Using Unsafe.As instead of BlockCopy Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com> * Update tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation/IbmMq/IbmMqHeadersAdapter.cs Use Unsafe.As instead of BlockCopy Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com> * Addressed some of the comments * Removed context propagation options --------- Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com> commit 5684a72 Author: Zach Montoya <zach.montoya@datadoghq.com> Date: Fri Mar 8 20:56:30 2024 -0800 [Tracing] Update instrumentation point for DD_TRACE_DELAY_WCF_INSTRUMENTATION_ENABLED=true (#5206) Updates the instrumentation point for `DD_TRACE_DELAY_WCF_INSTRUMENTATION_ENABLED=true` so that now a server span is created immediately before IDispatchMessageInspector implementations are run, so application code can access the root span from inside a IDispatchMessageInspector.AfterReceiveRequest callback. This PR also does some cleanup to remove unused Wcf files and it makes the entire Wcf instrumentation use nullable reference types. commit ca1bb6e Author: Andrew Lock <andrew.lock@datadoghq.com> Date: Fri Mar 8 17:43:57 2024 +0000 Fix errors identified from telemetry (#5279) * Try to avoid MongoDb exception We're seeing exceptions like this: ``` System.FieldAccessException at REDACTED at Datadog.Trace.ClrProfiler.AutoInstrumentation.MongoDb.MongoDbIntegration.CreateScope[TConnection](Object wireProtocol, TConnection connection) at REDACTED at MongoDB.Driver.Core.WireProtocol.CommandWireProtocol`1.ExecuteAsync(IConnection connection, CancellationToken cancellationToken) ``` and the only explanation I can think of is a duck-chaining issue, so stopped doing duck chaining and being explicit instead * Add local functions to try to isolate problems * Fix ArgumentNullException in AWS SQS integration

…#5238 (#5251) * add possible aspect * Working JSON but not integration test * Add integration test * Revert adding new reference to main project * Reuse tainted objects * Fix error handling in JavaScriptSerializerAspects * Update hash

e-n-0 self-assigned this Feb 26, 2024

e-n-0 requested a review from a team as a code owner February 26, 2024 10:12

github-actions bot added the area:asm label Feb 26, 2024

e-n-0 added 5 commits February 26, 2024 11:21

Add Newtonsoft.Json (non -working yet)

6ef323a

Refactor the tainting proces + add tests

c2e7a36

Add the JToken Parse aspect + test

d84023f

Rename Aspects class + Duck orignal method call

bd989b0

Add integration test

0a8a8aa

Fix nullability

8ea6dbe

e-n-0 force-pushed the flavien/iast/newtonsoft_json branch from 5c6b3ac to 8ea6dbe Compare February 26, 2024 10:34

e-n-0 marked this pull request as draft February 26, 2024 12:27

Fix compilation issue for netfx

4827619

e-n-0 marked this pull request as ready for review February 26, 2024 15:47

andrewlock approved these changes Feb 28, 2024

View reviewed changes

...ns/integrations/Samples.InstrumentedTests/Vulnerabilities/Json/Newtonsoft.Json/ParseTests.cs Outdated Show resolved Hide resolved

...ns/integrations/Samples.InstrumentedTests/Vulnerabilities/Json/Newtonsoft.Json/ParseTests.cs Outdated Show resolved Hide resolved

Change JSON formatting in ParseTests

c6b43da

daniel-romano-DD reviewed Mar 4, 2024

View reviewed changes

tracer/src/Datadog.Trace/Iast/Aspects/Newtonsoft.Json/NewtonsoftJsonAspects.cs Outdated Show resolved Hide resolved

daniel-romano-DD reviewed Mar 4, 2024

View reviewed changes

tracer/src/Datadog.Trace/Iast/Aspects/Newtonsoft.Json/NewtonsoftJsonAspects.cs Outdated Show resolved Hide resolved

daniel-romano-DD reviewed Mar 4, 2024

View reviewed changes

tracer/src/Datadog.Trace/Iast/Aspects/Newtonsoft.Json/NewtonsoftJsonAspects.cs Outdated Show resolved Hide resolved

e-n-0 added 2 commits March 4, 2024 15:11

Fix a test json format

7eb6142

Refactor NewtonsoftJsonAspects to static constructor

64ca995

e-n-0 requested a review from daniel-romano-DD March 5, 2024 16:19

daniel-romano-DD approved these changes Mar 11, 2024

View reviewed changes

e-n-0 merged commit 2b95f46 into master Mar 11, 2024

e-n-0 deleted the flavien/iast/newtonsoft_json branch March 11, 2024 16:47

github-actions bot added this to the vNext milestone Mar 11, 2024

Conversation

e-n-0 commented Feb 26, 2024

Summary of changes

Reason for change

Implementation details

Test coverage

Uh oh!

datadog-ddstaging bot commented Feb 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Datadog Report

Uh oh!

andrewlock commented Feb 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

Uh oh!

andrewlock commented Feb 26, 2024

Benchmarks Report 🐌

Benchmark details

Raw results

Raw results

Slower ⚠️ in #5238

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5238

Raw results

Raw results

Faster 🎉 in #5238

More allocations ⚠️ in #5238

Fewer allocations 🎉 in #5238

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5238

Raw results

Slower ⚠️ in #5238

Raw results

Uh oh!

andrewlock commented Feb 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Throughput/Crank Report:zap:

Uh oh!

andrewlock left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

daniel-romano-DD left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

datadog-ddstaging bot commented Feb 26, 2024 •

edited

Loading

andrewlock commented Feb 26, 2024 •

edited

Loading

andrewlock commented Feb 27, 2024 •

edited

Loading