Open
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 5108757b997c59ab8ba1fc5309ab0d4e730e2b77 |
| Compare | ecf677d2a596b537a6b237e51a65c5a718a4dbc7 |
| Diff | Diff |
Regressions in System.Text.Json.Tests.Perf_Basic
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| WriteBasicUtf16 - Duration of single invocation | 979.36 ns | 1.03 μs | 1.06 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Basic*'Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 10)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.0336284474718538 > 1027.8616268539915.
IsChangePoint: Marked as a change because one of 9/23/2022 2:49:56 PM, 10/7/2022 4:12:34 PM, 10/25/2022 8:54:59 AM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -61.40601114385213 (T) = (0 -1035.1884216418835) / Math.Sqrt((3.2161243834611812 / (31)) + (4.95623426113491 / (7))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (7) - 2, .025) and -0.05646311540247545 = (979.8623411925867 - 1035.1884216418835) / 979.8623411925867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | ubuntu 20.04 |
| Baseline | 5108757b997c59ab8ba1fc5309ab0d4e730e2b77 |
| Compare | ecf677d2a596b537a6b237e51a65c5a718a4dbc7 |
| Diff | Diff |
Regressions in System.Text.Json.Tests.Perf_Deep
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| WriteDeepUtf16 - Duration of single invocation | 5.36 ms | 5.93 ms | 1.11 | 0.00 | False | |||||
| WriteDeepUtf8 - Duration of single invocation | 5.07 ms | 5.64 ms | 1.11 | 0.00 | False | |||||
| WriteDeepUtf8 - Duration of single invocation | 5.37 ms | 5.90 ms | 1.10 | 0.00 | False | |||||
| WriteDeepUtf16 - Duration of single invocation | 5.07 ms | 5.62 ms | 1.11 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Deep*'Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.932006456910569 > 5.636536617917875.
IsChangePoint: Marked as a change because one of 9/23/2022 2:49:56 PM, 10/25/2022 8:54:59 AM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -103.09702837645301 (T) = (0 -5911444.991333167) / Math.Sqrt((219653226.93222767 / (31)) + (139476596.74883065 / (7))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (7) - 2, .025) and -0.09967486253079402 = (5375629.827282362 - 5911444.991333167) / 5375629.827282362 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.63593612868217 > 5.3247345794270835.
IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/25/2022 8:54:59 AM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -58.12187629480479 (T) = (0 -5593454.027599919) / Math.Sqrt((847916533.0728942 / (31)) + (378357483.8892741 / (7))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (7) - 2, .025) and -0.10345065620429936 = (5069056.77761844 - 5593454.027599919) / 5069056.77761844 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: False)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.897058604878049 > 5.639199849111112.
IsChangePoint: Marked as a change because one of 9/1/2022 1:51:24 AM, 9/23/2022 2:49:56 PM, 10/25/2022 8:54:59 AM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -81.13787542026836 (T) = (0 -5896938.258186877) / Math.Sqrt((271983877.6849514 / (31)) + (240434488.6133558 / (7))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (7) - 2, .025) and -0.09932784052601532 = (5364130.735891544 - 5896938.258186877) / 5364130.735891544 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 5.615225030303029 > 5.326227882812501.
IsChangePoint: Marked as a change because one of 9/23/2022 2:49:56 PM, 10/25/2022 8:54:59 AM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -21.21429871732625 (T) = (0 -5622035.0751970345) / Math.Sqrt((138052045.11067542 / (31)) + (4619772723.853835 / (7))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (7) - 2, .025) and -0.10774484050791204 = (5075207.637725738 - 5622035.0751970345) / 5075207.637725738 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository