Skip to content

Regressions in System.Threading.Tests.Perf_Timer #79409

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline aa8fe36e105a1c0498c733b69685b12f820175df
Compare 843441ecc0831b0621bdf7561e73c5d77cd4301b
Diff Diff

Regressions in System.Threading.Tests.Perf_Timer

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScheduleManyThenDisposeMany - Duration of single invocation 595.44 ms 779.50 ms 1.31 0.22 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Timer*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_Timer.ScheduleManyThenDisposeMany


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 779.496296 > 631.6900120588236.
IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsRegressionStdDev: Marked as regression because -27.275364789733967 (T) = (0 -760402180.7058823) / Math.Sqrt((287760363016342.4 / (18)) + (291508008401776.56 / (17))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (17) - 2, .025) and -0.2601983022924083 = (603398829.6307381 - 760402180.7058823) / 603398829.6307381 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.25094
Baseline aa8fe36e105a1c0498c733b69685b12f820175df
Compare 843441ecc0831b0621bdf7561e73c5d77cd4301b
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 2.54 ns 4.42 ns 1.74 0.90 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.ToString(value: 0)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.421718289376452 > 2.6488778258769807.
IsChangePoint: Marked as a change because one of 10/3/2022 4:48:37 PM, 12/1/2022 7:29:59 PM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsRegressionStdDev: Marked as regression because -11.30156067733412 (T) = (0 -4.295665548960617) / Math.Sqrt((0.3113295496751816 / (18)) + (0.2531362416139169 / (17))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (17) - 2, .025) and -0.8939492296978264 = (2.2680996309737282 - 4.295665548960617) / 2.2680996309737282 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline aa8fe36e105a1c0498c733b69685b12f820175df
Compare 843441ecc0831b0621bdf7561e73c5d77cd4301b
Diff Diff

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 2.33 ns 5.05 ns 2.17 0.84 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.ToString(value: 0)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.051131508520759 > 2.868795317741063.
IsChangePoint: Marked as a change because one of 9/12/2022 8:20:58 PM, 10/3/2022 10:42:38 PM, 12/1/2022 7:29:59 PM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsRegressionStdDev: Marked as regression because -11.318325125981792 (T) = (0 -4.894545995643823) / Math.Sqrt((0.16085245528718428 / (18)) + (0.5745113894330254 / (17))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (17) - 2, .025) and -0.9157635919576466 = (2.554879952928989 - 4.894545995643823) / 2.554879952928989 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions