[Perf] Windows/x64: 6 Regressions on 6/8/2023 9:27:53 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	79effd2c09a7a3049d0b29c998e444ba51864131
Compare	9b53f4b5238e4994cc65ecee7e3d9023c7197770
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.DictionarySequentialKeys

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryGetValue_3k_Int_32ByteValue - Duration of single invocation	2.82 ns	4.79 ns	1.70	0.54	False
TryGetValue_3k_Int_32ByteRefsValue - Duration of single invocation	3.08 ns	4.73 ns	1.54	0.57	False
TryGetValue_3k_Int_Int - Duration of single invocation	2.74 ns	5.01 ns	1.83	0.31	False
TryGetValue_17_Int_Int - Duration of single invocation	2.85 ns	5.87 ns	2.06	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteValue

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.785861335125447 > 3.119743871477311.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -28.645332841307425 (T) = (0 -4.937609658750378) / Math.Sqrt((0.034297034216741795 / (24)) + (0.05600889042826007 / (18))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (18) - 2, .025) and -0.641840231328533 = (3.007363057948091 - 4.937609658750378) / 3.007363057948091 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteRefsValue

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.733374490093241 > 3.1945438964243222.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -26.550451395878657 (T) = (0 -4.943757162154572) / Math.Sqrt((0.019989756600105577 / (24)) + (0.07798437490194476 / (18))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (18) - 2, .025) and -0.6286132628993867 = (3.0355623859732686 - 4.943757162154572) / 3.0355623859732686 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_Int

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.013220025715204 > 3.0121390875901715.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -33.185932863915276 (T) = (0 -4.934534147486396) / Math.Sqrt((0.015657222393748563 / (24)) + (0.05214850353368014 / (18))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (18) - 2, .025) and -0.6685454346271671 = (2.9573867424167606 - 4.934534147486396) / 2.9573867424167606 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_Int

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.871274656367327 > 3.000785990491406.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/9/2023 2:21:24 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -27.767219898526523 (T) = (0 -5.022809400591073) / Math.Sqrt((0.0028050865923364733 / (29)) + (0.0745441291032477 / (13))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (13) - 2, .025) and -0.7305132220287068 = (2.902496979885977 - 5.022809400591073) / 2.902496979885977 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	79effd2c09a7a3049d0b29c998e444ba51864131
Compare	9b53f4b5238e4994cc65ecee7e3d9023c7197770
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Parse - Duration of single invocation	1.48 μs	1.75 μs	1.18	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7531331363721627 > 1.5537864687412255.
IsChangePoint: Marked as a change because one of 5/26/2023 5:44:43 PM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -6.629831448557072 (T) = (0 -1675.9675926593177) / Math.Sqrt((4478.21832570024 / (24)) + (8895.153695175779 / (18))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (18) - 2, .025) and -0.11509255678240062 = (1502.9851849207225 - 1675.9675926593177) / 1502.9851849207225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	79effd2c09a7a3049d0b29c998e444ba51864131
Compare	9b53f4b5238e4994cc65ecee7e3d9023c7197770
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Depth

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadSpanEmptyLoop - Duration of single invocation	16.79 μs	19.98 μs	1.19	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Depth*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.97753456221198 > 17.58328112533566.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/26/2023 5:44:43 PM, 6/8/2023 8:02:22 AM, 6/12/2023 11:18:39 PM falls between 6/4/2023 12:53:30 AM and 6/12/2023 11:18:39 PM.
IsRegressionStdDev: Marked as regression because -26.863501788502163 (T) = (0 -20059.33136039175) / Math.Sqrt((225248.61818694748 / (24)) + (134432.59094540466 / (18))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (18) - 2, .025) and -0.21044617748532593 = (16571.84906979057 - 20059.33136039175) / 16571.84906979057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kunalspathak]

by design dotnet/runtime#87115