Skip to content

[Perf] Linux/x64: 13 Regressions on 12/28/2022 6:20:56 PM #11270

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf8 - Duration of single invocation 24.25 ns 27.38 ns 1.13 0.06 False
EncodeUtf8 - Duration of single invocation 86.99 ns 98.66 ns 1.13 0.03 False
EncodeUtf8 - Duration of single invocation 56.38 ns 60.28 ns 1.07 0.04 False
EncodeUtf8 - Duration of single invocation 68.70 ns 75.82 ns 1.10 0.04 False
EncodeUtf8 - Duration of single invocation 54.40 ns 66.48 ns 1.22 0.04 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",16)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.38339939908008 > 25.32808934198907.
IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/7/2022 4:10:19 PM, 11/10/2022 9:13:35 AM, 11/18/2022 8:48:12 PM, 12/1/2022 7:29:59 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.058001012376725 (T) = (0 -27.46168470771202) / Math.Sqrt((0.8326968207866183 / (29)) + (0.013254001035322508 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.12771116366632132 = (24.351700676998586 - 27.46168470771202) / 24.351700676998586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 98.66323554506643 > 91.24919194338443.
IsChangePoint: Marked as a change because one of 10/31/2022 10:23:25 PM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.80649573019915 (T) = (0 -98.59663915861687) / Math.Sqrt((8.486130171467286 / (29)) + (0.0009985656060789638 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.12193298874261922 = (87.88104115658173 - 98.59663915861687) / 87.88104115658173 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.27949161899527 > 59.25339992121415.
IsChangePoint: Marked as a change because one of 10/17/2022 12:47:34 PM, 11/7/2022 4:10:19 PM, 11/10/2022 9:13:35 AM, 12/1/2022 7:29:59 PM, 12/3/2022 7:24:45 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -16.995514013921436 (T) = (0 -59.752643845121426) / Math.Sqrt((0.4719556899203906 / (29)) + (0.2326512657276228 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.054390844842166344 = (56.67029843574363 - 59.752643845121426) / 56.67029843574363 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.82173553085313 > 72.94025391330943.
IsChangePoint: Marked as a change because one of 10/17/2022 12:47:34 PM, 11/7/2022 4:10:19 PM, 11/10/2022 9:13:35 AM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.63914200299813 (T) = (0 -75.77183580670749) / Math.Sqrt((2.936184503467602 / (29)) + (0.15829679708616626 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.0899511845616439 = (69.51855906939664 - 75.77183580670749) / 69.51855906939664 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,16)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.48226516794898 > 57.10550034286644.
IsChangePoint: Marked as a change because one of 10/17/2022 12:47:34 PM, 11/7/2022 4:10:19 PM, 11/10/2022 9:13:35 AM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.74017308510893 (T) = (0 -66.51670872735204) / Math.Sqrt((10.325223086645083 / (28)) + (0.004032309356583842 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.2064944704777867 = (55.13221183766437 - 66.51670872735204) / 55.13221183766437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Collections.TryAddGiventSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 4.11 μs 4.63 μs 1.13 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddGiventSize<Int32>.Dictionary(Count: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.62625227239137 > 4.331658172567962.
IsChangePoint: Marked as a change because one of 10/17/2022 8:10:07 PM, 10/19/2022 7:27:15 PM, 10/25/2022 2:34:53 PM, 11/3/2022 3:03:31 PM, 11/18/2022 8:48:12 PM, 12/7/2022 8:33:47 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -29.883741928896654 (T) = (0 -4640.507786964781) / Math.Sqrt((7330.286335941202 / (29)) + (415.0575826401282 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.12135473611799896 = (4138.304889164409 - 4640.507786964781) / 4138.304889164409 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in Burgers

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test3 - Duration of single invocation 130.63 ms 185.03 ms 1.42 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Burgers*'

Payloads

Baseline
Compare

Histogram

Burgers.Test3


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 185.02650623076923 > 137.19255053269234.
IsChangePoint: Marked as a change because one of 12/16/2022 6:44:06 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -1112.7765880908323 (T) = (0 -185122771.44463634) / Math.Sqrt((13240655093.700161 / (29)) + (27129338344.55862 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.41669925035144206 = (130671892.00439878 - 185122771.44463634) / 130671892.00439878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 2.71 μs 3.89 μs 1.44 0.12 False
Stack - Duration of single invocation 11.18 μs 14.28 μs 1.28 0.15 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.HashSet(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.8907520246405722 > 2.8081831122808456.
IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 11/7/2022 4:10:19 PM, 11/16/2022 6:51:59 PM, 12/6/2022 2:12:43 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -20.382220831488578 (T) = (0 -3887.1581556971505) / Math.Sqrt((89606.66516106357 / (29)) + (39.38660082669739 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.41163258525157526 = (2753.6613962509214 - 3887.1581556971505) / 2753.6613962509214 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.279416201891715 > 11.746494870154343.
IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/17/2022 12:47:34 PM, 10/25/2022 12:27:43 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -8.432930007529418 (T) = (0 -13658.81911015653) / Math.Sqrt((301396.32639466535 / (29)) + (415571.30843547493 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.14102872541050823 = (11970.61809749136 - 13658.81911015653) / 11970.61809749136 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 17.56 μs 21.17 μs 1.21 0.38 False
Stack - Duration of single invocation 16.14 μs 21.66 μs 1.34 0.21 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.List(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.173356838880416 > 18.43591835033121.
IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 2:34:53 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -10.460864814769154 (T) = (0 -20488.20646486722) / Math.Sqrt((1031305.6060377982 / (29)) + (791933.3435009287 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.18339694871477663 = (17313.046553921195 - 20488.20646486722) / 17313.046553921195 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.662782607192256 > 18.488819196403437.
IsChangePoint: Marked as a change because one of 11/3/2022 3:03:31 PM, 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -13.008394141928974 (T) = (0 -20825.254297429554) / Math.Sqrt((330277.48204226414 / (29)) + (818953.8689956036 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.19779192267618312 = (17386.37062345557 - 20825.254297429554) / 17386.37062345557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Numerics.Tests.Perf_Plane

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateFromVerticesBenchmark - Duration of single invocation 4.12 ns 5.16 ns 1.25 0.13 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Plane.CreateFromVerticesBenchmark


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.161945325220399 > 4.32516936202548.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/15/2022 1:59:35 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -39.59038821839044 (T) = (0 -5.1753305683102795) / Math.Sqrt((0.017429968384272555 / (29)) + (0.0008638337964258109 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.24522849534533744 = (4.156129246684973 - 5.1753305683102795) / 4.156129246684973 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 4.28 μs 4.63 μs 1.08 0.13 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrueComparer<Int32>.HashSet(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.634883012398908 > 4.494167167146445.
IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 12/3/2022 7:24:45 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -8.383402882087836 (T) = (0 -4590.878715862526) / Math.Sqrt((2824.678779357185 / (29)) + (13342.592213360631 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.06290815065136753 = (4319.167853825526 - 4590.878715862526) / 4319.167853825526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions