[Perf] Regressions in System.Collections.Tests.Perf_BitArray

[performanceautofiler]

Run Information

Architecture	x86
OS	Windows 10.0.18362
Baseline	daf25da8e67857fde603b16d92192cd1368bafc7
Compare	a29a33456bd4df703e4ff675f36fe0f22903953d
Diff	Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BitArrayLengthCtor - Duration of single invocation	11.66 ns	13.55 ns	1.16	0.18	False
BitArrayIntArrayCtor - Duration of single invocation	144.79 ns	153.79 ns	1.06	0.00	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayLengthCtor(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.552456654774215 > 12.311515625815773.
IsChangePoint: Marked as a change because one of 4/10/2022 11:44:05 PM, 4/11/2022 5:01:30 PM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -3.1934874249260923 (T) = (0 -12.868695907343728) / Math.Sqrt((1.8689032056672892 / (53)) + (0.20798193329018388 / (4))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (53) + (4) - 2, .025) and -0.07910354378561829 = (11.925357841193696 - 12.868695907343728) / 11.925357841193696 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 153.78919173026836 > 152.37757986958002.
IsChangePoint: Marked as a change because one of 4/8/2022 10:19:59 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -22.810390319926345 (T) = (0 -153.674493616133) / Math.Sqrt((0.9275447022264903 / (37)) + (1.740421529380291 / (20))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (20) - 2, .025) and -0.05229391205141096 = (146.0376154001973 - 153.674493616133) / 146.0376154001973 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[ghost]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[DrewScoggins]

Seems related to #67160

[danmoseley]

The first one appears to have recovered.

[DrewScoggins]

Windows x86 regressions dotnet/perf-autofiling-issues#4591

[DrewScoggins]

Win-x64 Improvements: dotnet/perf-autofiling-issues#4582

[mangod9]

@kouvel, is this related to your tiering change as mentioned above?

[kouvel]

The first regression doesn't look like a clear regression to me from the history, it just looks like the newer measurement used is a higher one within the regular fluctuations. I can repro the second regression, but it's reproing even with slot backpatching disabled (which doesn't use the changed code paths). It's probably something external that's causing the difference, like layout or inlining differences in coreclr.dll or something else. Optimizing the before and after dlls with PGO instrumented against the benchmark eliminated the perf difference.