-
Notifications
You must be signed in to change notification settings - Fork 5.1k
Closed as not planned
Labels
area-System.Threading.Taskstenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark
Description
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.25094 |
Baseline | aa8fe36e105a1c0498c733b69685b12f820175df |
Compare | 843441ecc0831b0621bdf7561e73c5d77cd4301b |
Diff | Diff |
Regressions in System.Threading.Tests.Perf_Timer
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ScheduleManyThenDisposeMany - Duration of single invocation | 595.44 ms | 779.50 ms | 1.31 | 0.22 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Timer*'
Payloads
Histogram
System.Threading.Tests.Perf_Timer.ScheduleManyThenDisposeMany
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 779.496296 > 631.6900120588236.
IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsRegressionStdDev: Marked as regression because -27.275364789733967 (T) = (0 -760402180.7058823) / Math.Sqrt((287760363016342.4 / (18)) + (291508008401776.56 / (17))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (17) - 2, .025) and -0.2601983022924083 = (603398829.6307381 - 760402180.7058823) / 603398829.6307381 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | arm64 |
---|---|
OS | Windows 10.0.25094 |
Baseline | aa8fe36e105a1c0498c733b69685b12f820175df |
Compare | 843441ecc0831b0621bdf7561e73c5d77cd4301b |
Diff | Diff |
Regressions in System.Tests.Perf_UInt32
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToString - Duration of single invocation | 2.54 ns | 4.42 ns | 1.74 | 0.90 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'
Payloads
Histogram
System.Tests.Perf_UInt32.ToString(value: 0)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.421718289376452 > 2.6488778258769807.
IsChangePoint: Marked as a change because one of 10/3/2022 4:48:37 PM, 12/1/2022 7:29:59 PM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsRegressionStdDev: Marked as regression because -11.30156067733412 (T) = (0 -4.295665548960617) / Math.Sqrt((0.3113295496751816 / (18)) + (0.2531362416139169 / (17))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (17) - 2, .025) and -0.8939492296978264 = (2.2680996309737282 - 4.295665548960617) / 2.2680996309737282 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.25094 |
Baseline | aa8fe36e105a1c0498c733b69685b12f820175df |
Compare | 843441ecc0831b0621bdf7561e73c5d77cd4301b |
Diff | Diff |
Regressions in System.Tests.Perf_UInt64
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToString - Duration of single invocation | 2.33 ns | 5.05 ns | 2.17 | 0.84 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'
Payloads
Histogram
System.Tests.Perf_UInt64.ToString(value: 0)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.051131508520759 > 2.868795317741063.
IsChangePoint: Marked as a change because one of 9/12/2022 8:20:58 PM, 10/3/2022 10:42:38 PM, 12/1/2022 7:29:59 PM, 12/8/2022 3:49:24 AM falls between 11/29/2022 2:55:01 PM and 12/8/2022 3:49:24 AM.
IsRegressionStdDev: Marked as regression because -11.318325125981792 (T) = (0 -4.894545995643823) / Math.Sqrt((0.16085245528718428 / (18)) + (0.5745113894330254 / (17))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (17) - 2, .025) and -0.9157635919576466 = (2.554879952928989 - 4.894545995643823) / 2.554879952928989 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
area-System.Threading.Taskstenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark