Description
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.25094 |
Queue | AmpereWindows |
Baseline | dd715a0747ebeb85a234c8abec7e01bb6652c8ce |
Compare | 9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Collections.Tests.DictionarySequentialKeys
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryGetValue_17_Int_Int - Duration of single invocation | 5.88 ns | 7.40 ns | 1.26 | 0.72 | False | |||||
TryGetValue_3k_Int_Int - Duration of single invocation | 5.61 ns | 7.64 ns | 1.36 | 0.68 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'
Payloads
Histogram
System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_Int
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.395976936880073 > 5.7943076452128945.
IsChangePoint: Marked as a change because one of 3/31/2023 7:02:21 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -7.695170562714825 (T) = (0 -8.018789895969261) / Math.Sqrt((0.18774556681341403 / (12)) + (1.3012597196780862 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.484735265675485 = (5.40082133249599 - 8.018789895969261) / 5.40082133249599 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_Int
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.641963550983901 > 5.879509724413551.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -13.113548564576806 (T) = (0 -8.743435290828566) / Math.Sqrt((0.17092856038841 / (12)) + (0.6791435685309158 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.6305911085842962 = (5.362126191415179 - 8.743435290828566) / 5.362126191415179 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.25094 |
Queue | AmpereWindows |
Baseline | dd715a0747ebeb85a234c8abec7e01bb6652c8ce |
Compare | 9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Buffers.Text.Tests.Utf8ParserTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryParseDateTimeOffset - Duration of single invocation | 5.97 ns | 7.64 ns | 1.28 | 0.83 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
Payloads
Histogram
System.Buffers.Text.Tests.Utf8ParserTests.TryParseDateTimeOffset(value: 12/30/2017 3:45:22 AM -08:00)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.640911329696776 > 6.302415903491371.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -7.262938294741985 (T) = (0 -9.710078430474258) / Math.Sqrt((0.9973057405368995 / (12)) + (2.751057774974143 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.6836997855265233 = (5.7671079570980295 - 9.710078430474258) / 5.7671079570980295 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.25094 |
Queue | AmpereWindows |
Baseline | dd715a0747ebeb85a234c8abec7e01bb6652c8ce |
Compare | 9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_String
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Replace_String - Duration of single invocation | 73.13 ns | 103.03 ns | 1.41 | 0.58 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
Payloads
Histogram
System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 103.02587738588925 > 79.04393196166279.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -6.95275929703907 (T) = (0 -93.03306473253484) / Math.Sqrt((15.875856829350282 / (12)) + (64.28241464298715 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.23016574351153538 = (75.62644726796724 - 93.03306473253484) / 75.62644726796724 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.25094 |
Queue | AmpereWindows |
Baseline | dd715a0747ebeb85a234c8abec7e01bb6652c8ce |
Compare | 9bcf52342b5a6573ecd2c81fbfd541df0ebc42f4 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Text.Json.Tests.Perf_Get
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetUInt16 - Duration of single invocation | 1.28 μs | 1.42 μs | 1.11 | 0.67 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'
Payloads
Histogram
System.Text.Json.Tests.Perf_Get.GetUInt16
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4243274439721787 > 1.1817496321292034.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/14/2023 3:35:53 PM falls between 6/5/2023 10:43:22 AM and 6/14/2023 3:35:53 PM.
IsRegressionStdDev: Marked as regression because -7.208603559815226 (T) = (0 -1474.2133425890993) / Math.Sqrt((6162.145617042773 / (12)) + (15447.056533549852 / (13))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (13) - 2, .025) and -0.25268554569773194 = (1176.8423030442 - 1474.2133425890993) / 1176.8423030442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository