-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 46f9533d9b0663aa652b6dc7001417d4924343d2 |
Compare | 2b185ac8c5c1302804a3336c62cb33782d9d4916 |
Diff | Diff |
Improvements in System.Tests.Perf_Single
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryParse - Duration of single invocation | 112.96 ns | 98.81 ns | 0.87 | 0.01 | False | |||||
Parse - Duration of single invocation | 113.01 ns | 99.75 ns | 0.88 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'
Payloads
Histogram
System.Tests.Perf_Single.TryParse(value: "3.4028235E+38")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 98.81289939689104 < 107.48052990300704.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 1/27/2023 3:43:27 AM, 2/16/2023 5:39:51 AM, 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 37.875697112233276 (T) = (0 -98.90223887294428) / Math.Sqrt((13.361767592509388 / (299)) + (0.0955403054653846 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.08352230461187501 = (107.91559835077005 - 98.90223887294428) / 107.91559835077005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Single.Parse(value: "3.4028235E+38")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 99.74702683103696 < 107.64371695799338.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 1/27/2023 3:43:27 AM, 2/16/2023 5:39:51 AM, 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 31.630867502929267 (T) = (0 -99.88386576574007) / Math.Sqrt((15.239184156309493 / (299)) + (0.10067235089413894 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.07393012592000937 = (107.85780702019949 - 99.88386576574007) / 107.85780702019949 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 46f9533d9b0663aa652b6dc7001417d4924343d2 |
Compare | 2b185ac8c5c1302804a3336c62cb33782d9d4916 |
Diff | Diff |
Improvements in StoreBlock.AnyLocation
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
CopyBlock128 - Duration of single invocation | 8.70 ns | 3.62 ns | 0.42 | 0.00 | False | |||||
InitBlockAllZeros128 - Duration of single invocation | 6.78 ns | 2.65 ns | 0.39 | 0.00 | False | |||||
InitBlockAllOnes128 - Duration of single invocation | 6.81 ns | 2.65 ns | 0.39 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'StoreBlock.AnyLocation*'
Payloads
Histogram
StoreBlock.AnyLocation.CopyBlock128
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.6176493169229724 < 8.267658307410352.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 498.98997860797977 (T) = (0 -3.6083714532829116) / Math.Sqrt((0.018686869102303386 / (299)) + (0.0003062070404929029 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.5812758881717188 = (8.61753921341579 - 3.6083714532829116) / 8.61753921341579 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### StoreBlock.AnyLocation.InitBlockAllZeros128
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.648042047725526 < 6.451410187953502.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 5436.481015378895 (T) = (0 -2.648645087010155) / Math.Sqrt((0.00014272700429543447 / (299)) + (8.472171019274126E-07 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.6105236900385083 = (6.800529375642981 - 2.648645087010155) / 6.800529375642981 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### StoreBlock.AnyLocation.InitBlockAllOnes128
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.6493692531260584 < 6.468435884166999.
IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 851.8995408169569 (T) = (0 -2.6437543160258046) / Math.Sqrt((0.00014536900775572733 / (299)) + (0.00018666799123831813 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.611297495753127 = (6.801485164465781 - 2.6437543160258046) / 6.801485164465781 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 46f9533d9b0663aa652b6dc7001417d4924343d2 |
Compare | 2b185ac8c5c1302804a3336c62cb33782d9d4916 |
Diff | Diff |
Improvements in System.Tests.Perf_Double
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Parse - Duration of single invocation | 154.47 ns | 144.68 ns | 0.94 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'
Payloads
Histogram
System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 144.68021806029012 < 146.45615809059993.
IsChangePoint: Marked as a change because one of 11/30/2022 2:05:04 AM, 3/13/2023 2:14:21 PM, 3/16/2023 2:30:29 AM falls between 3/7/2023 7:00:18 AM and 3/16/2023 2:30:29 AM.
IsImprovementStdDev: Marked as improvement because 71.60807427737574 (T) = (0 -144.8493115912621) / Math.Sqrt((2.374877877383479 / (299)) + (0.032747671147605295 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.051446067404310285 = (152.70540410378803 - 144.8493115912621) / 152.70540410378803 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository