Regressions in System.Tests.Perf_Decimal on 2/17/2023

[performanceautofiler]

Run Information

Architecture	x64
OS	Windows 10.0.19042
Baseline	3f22bec109fe43dc11b5c82f8263eec369571d31
Compare	5ec100a94d16e2e1529468a436ef17b40cab911c
Diff	Diff

Regressions in System.Tests.Perf_Decimal

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	57.96 ns	62.74 ns	1.08	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.ToString(value: 123456.789)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.74319939434997 > 60.33358420278738.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/13/2023 4:18:46 AM, 2/17/2023 5:12:50 PM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsRegressionStdDev: Marked as regression because -24.936368731626935 (T) = (0 -62.833223611316924) / Math.Sqrt((0.7971718006610916 / (45)) + (0.4129762599690063 / (17))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (17) - 2, .025) and -0.08854287135709848 = (57.722323359650545 - 62.833223611316924) / 57.722323359650545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[dakersnar]

@MihaZupan seems to be caused by #82264. Not a super important regression, but it might be worth looking into why your PR would cause this to regress. We are guessing that another JIT optimization was reliant on the previous range check somehow.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	x64
OS	Windows 10.0.19042
Baseline	3f22bec109fe43dc11b5c82f8263eec369571d31
Compare	5ec100a94d16e2e1529468a436ef17b40cab911c
Diff	Diff

Regressions in System.Tests.Perf_Decimal

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString - Duration of single invocation	57.96 ns	62.74 ns	1.08	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.ToString(value: 123456.789)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.74319939434997 > 60.33358420278738.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/13/2023 4:18:46 AM, 2/17/2023 5:12:50 PM, 2/21/2023 5:46:41 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 5:46:41 AM.
IsRegressionStdDev: Marked as regression because -24.936368731626935 (T) = (0 -62.833223611316924) / Math.Sqrt((0.7971718006610916 / (45)) + (0.4129762599690063 / (17))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (17) - 2, .025) and -0.08854287135709848 = (57.722323359650545 - 62.833223611316924) / 57.722323359650545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	dakersnar
Labels:	area-System.Runtime, tenet-performance, tenet-performance-benchmarks, untriaged
Milestone:	-

[tannergooding]

Closing. The regression was since fixed and we're now faster than before (50ns, rather than 57ns)