[Perf] Changes at 12/26/2021 10:58:11 AM #63456
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 71192eacd5194e8afce9686e701aa5f072972b8d |
| Compare | 7ab969c84ef05ba948c0075392716ce335b47744 |
| Diff | Diff |
Regressions in PerfLabTests.GetMember
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| GetMethod10 - Duration of single invocation | 867.00 μs | 1.15 ms | 1.33 | 0.11 | True | |||||
| GetMethod5 - Duration of single invocation | 540.78 μs | 589.19 μs | 1.09 | 0.12 | True | |||||
| GetMethod12 - Duration of single invocation | 1.04 ms | 1.41 ms | 1.35 | 0.08 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.GetMember*'Details
Payloads
Histogram
PerfLabTests.GetMember.GetMethod10
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1516062156593405 > 908.7084953399124.
IsChangePoint: Marked as a change because one of 12/26/2021 4:30:45 AM, 1/5/2022 6:46:20 PM falls between 12/6/2021 2:46:54 PM and 1/5/2022 6:46:20 PM.
IsRegressionStdDev: Marked as regression because -32.249640928760144 (T) = (0 -1165738.8169220851) / Math.Sqrt((2417218554.266879 / (83)) + (1465316332.3535087 / (31))) is less than -1.9813718148759274 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (83) + (31) - 2, .025) and -0.31890306372968186 = (883869.9742083632 - 1165738.8169220851) / 883869.9742083632 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod5
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 589.1945524691358 > 457.07850260416666.
IsChangePoint: Marked as a change because one of 12/26/2021 4:30:45 AM, 1/5/2022 6:46:20 PM falls between 12/6/2021 2:46:54 PM and 1/5/2022 6:46:20 PM.
IsRegressionStdDev: Marked as regression because -26.17071255316827 (T) = (0 -595822.2093277765) / Math.Sqrt((973207935.473748 / (83)) + (692759957.5964096 / (31))) is less than -1.9813718148759274 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (83) + (31) - 2, .025) and -0.3447906618513562 = (443059.4487528011 - 595822.2093277765) / 443059.4487528011 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### PerfLabTests.GetMember.GetMethod12
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4062647023809525 > 1.0935164550781251.
IsChangePoint: Marked as a change because one of 12/26/2021 4:30:45 AM, 1/5/2022 6:46:20 PM falls between 12/6/2021 2:46:54 PM and 1/5/2022 6:46:20 PM.
IsRegressionStdDev: Marked as regression because -33.97323270602115 (T) = (0 -1388667.1066422376) / Math.Sqrt((6334608012.472459 / (83)) + (377140135.0807543 / (31))) is less than -1.9813718148759274 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (83) + (31) - 2, .025) and -0.29892392718979804 = (1069090.4044293014 - 1388667.1066422376) / 1069090.4044293014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
category:performance
theme:benchmarks