.NET 8 per-preview performance overview

[naricc]

Description

This issue provides an overview of the all performance regressions from microbenchmark automated runs in WASM, Mono AOT, and Interpreter during the timeframe of .NET 8 preview releases. WASM regressions are tracked separately at dotnet/perf-autofiling-issues#9652.

List of queries for autofilling issues:

• Mono AOT autofilling issues: https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aissue+is%3Aopen+label%3Amono-aot
• Interpreter autofilling issues: https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aopen+is%3Aissue+label%3Amono-interpreter
• WASM autofilling issues: https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aopen+is%3Aissue+label%3Aarch-wasm
• WASM AOT autofilling issues: https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aopen+is%3Aissue+label%3Awasm-aot
• Mono JIT autofilling issues: https://github.com/dotnet/perf-autofiling-issues/issues?q=is%3Aopen+is%3Aissue+label%3Amono-jit

Mono AOT

Within the .NET 8 timeframe about 100 microbenchmarks regressed more than 10%. Summary of the most impactful issues to be resolved:

• Vector2/3/4 intrinsics - [Perf][Mono] Linux/x64 Regressions due to rewritten of Matrix3x2 and Matrix4x4 #80569, [Perf] Linux/x64: 26 Regressions on 2/2/2023 8:18:56 AM perf-autofiling-issues#12546, [Perf] Linux/x64: 26 Regressions on 2/2/2023 8:18:56 AM perf-autofiling-issues#12660, [Perf] Linux/x64: 26 Regressions on 2/2/2023 8:18:56 AM perf-autofiling-issues#12660
• Further investigation is needed - [Perf] Linux/x64: 28 Regressions on 3/9/2023 2:48:42 AM perf-autofiling-issues#13978
• PackedSimd API not supported - [Perf] Linux/x64: 81 Regressions on 3/8/2023 4:46:51 PM perf-autofiling-issues#13992
• Deduplicated binary logic - [Perf] Linux/x64: 52 Regressions on 4/17/2023 11:29:15 PM perf-autofiling-issues#16604, [Perf] Linux/x64: 30 Regressions on 4/17/2023 11:29:15 PM perf-autofiling-issues#16945
• BitArray with Vector512 - [Perf] Linux/x64: 4 Regressions on 9/21/2023 8:08:31 PM perf-autofiling-issues#22179

Details

RC2

Issue	Status
dotnet/perf-autofiling-issues#22179	Resolved
dotnet/perf-autofiling-issues#21818, dotnet/perf-autofiling-issues#21820	Resolved
dotnet/perf-autofiling-issues#22174	Resolved

RC 1

Issue	Status
#91361, dotnet/perf-autofiling-issues#20290	Resolved
dotnet/perf-autofiling-issues#20971	Resolved

Preview 7

Issue	Status
dotnet/perf-autofiling-issues#19469, dotnet/perf-autofiling-issues#19517	Resolved

Preview 6

Issue	Status

Preview 5

Issue	Status
dotnet/perf-autofiling-issues#17874, dotnet/perf-autofiling-issues#17870	Resolved
dotnet/perf-autofiling-issues#17202, dotnet/perf-autofiling-issues#17311	Resolved
dotnet/perf-autofiling-issues#18385, dotnet/perf-autofiling-issues#18384	Resolved
dotnet/perf-autofiling-issues#18023	Resolved
dotnet/perf-autofiling-issues#17208, dotnet/perf-autofiling-issues#17312	Resolved
dotnet/perf-autofiling-issues#17505, dotnet/perf-autofiling-issues#17553	Resolved
dotnet/perf-autofiling-issues#17201, dotnet/perf-autofiling-issues#17310	Resolved
dotnet/perf-autofiling-issues#17200	Resolved

Preview 4

Issue	Status
dotnet/perf-autofiling-issues#17004, dotnet/perf-autofiling-issues#16946, dotnet/perf-autofiling-issues#16945, dotnet/perf-autofiling-issues#16604	Resolved
dotnet/perf-autofiling-issues#16359	Resolved
dotnet/perf-autofiling-issues#16481	Resolved
dotnet/perf-autofiling-issues#16361	Resolved
dotnet/perf-autofiling-issues#14907	Resolved
dotnet/perf-autofiling-issues#14523	Resolved
dotnet/perf-autofiling-issues#15689, dotnet/perf-autofiling-issues#15694, dotnet/perf-autofiling-issues#15769, dotnet/perf-autofiling-issues#15794, dotnet/perf-autofiling-issues#15764, dotnet/perf-autofiling-issues#15659	Resolved
dotnet/perf-autofiling-issues#16943, dotnet/perf-autofiling-issues#16433, dotnet/perf-autofiling-issues#16603	Resolved
dotnet/perf-autofiling-issues#15660	Resolved
dotnet/perf-autofiling-issues#17003, dotnet/perf-autofiling-issues#15697, dotnet/perf-autofiling-issues#15795, dotnet/perf-autofiling-issues#15774, dotnet/perf-autofiling-issues#15798	Resolved

Preview 3

Issue	Status
dotnet/perf-autofiling-issues#13978	Resolved
dotnet/perf-autofiling-issues#12060	Resolved
dotnet/perf-autofiling-issues#14017, dotnet/perf-autofiling-issues#13977, dotnet/perf-autofiling-issues#14016, dotnet/perf-autofiling-issues#13992	Resolved
dotnet/perf-autofiling-issues#14569	Resolved
dotnet/perf-autofiling-issues#14286	Resolved
dotnet/perf-autofiling-issues#14570	Resolved
dotnet/perf-autofiling-issues#14520	Resolved

Preview 2

Issue	Status
dotnet/perf-autofiling-issues#12660	Resolved
dotnet/perf-autofiling-issues#12546	Resolved
#80569	Resolved
dotnet/perf-autofiling-issues#12957	Resolved
dotnet/perf-autofiling-issues#13187	Resolved
dotnet/perf-autofiling-issues#13500	Resolved
dotnet/perf-autofiling-issues#13770	Resolved

.NET 7 / Preview 1

Issue	Status
dotnet/perf-autofiling-issues#14015	Resolved
dotnet/perf-autofiling-issues#11536	Resolved
dotnet/perf-autofiling-issues#11539	Resolved
dotnet/perf-autofiling-issues#13980	Resolved
dotnet/perf-autofiling-issues#13774	Resolved
dotnet/perf-autofiling-issues#13771	Resolved
dotnet/perf-autofiling-issues#13499	Resolved
dotnet/perf-autofiling-issues#13497	Resolved
dotnet/perf-autofiling-issues#13495	Resolved
dotnet/perf-autofiling-issues#13494	Resolved
dotnet/perf-autofiling-issues#13772	Resolved
dotnet/perf-autofiling-issues#13984	Resolved
dotnet/perf-autofiling-issues#12058	Resolved
dotnet/perf-autofiling-issues#11780	Resolved
dotnet/perf-autofiling-issues#11812	Resolved
dotnet/perf-autofiling-issues#12054	Resolved
dotnet/perf-autofiling-issues#12073	Resolved
dotnet/perf-autofiling-issues#12484	Resolved
dotnet/perf-autofiling-issues#12307	Resolved
dotnet/perf-autofiling-issues#10951	Resolved
dotnet/perf-autofiling-issues#11779	Resolved
dotnet/perf-autofiling-issues#11811	Resolved
dotnet/perf-autofiling-issues#12056	Resolved
dotnet/perf-autofiling-issues#12057	Resolved
dotnet/perf-autofiling-issues#8834	Resolved
dotnet/perf-autofiling-issues#9173	Resolved
dotnet/perf-autofiling-issues#9599	Resolved
#77900	Resolved
#78804	Resolved

Mono Interpreter

Within the .NET 8 timeframe about 250 microbenchmarks regressed more than 10%. Summary of the most impactful issues to be resolved:

• PackedSimd API not supported - [Perf] Linux/x64: 81 Regressions on 3/8/2023 4:46:51 PM perf-autofiling-issues#13992
• Vector128.Widen intrinsic - [Perf] Linux/x64: 28 Regressions on 6/19/2023 11:51:49 PM perf-autofiling-issues#19225
• ActivatorUtilitiesBenchmark - [Perf] Linux/x64: 1 Regression on 4/24/2023 6:11:43 PM perf-autofiling-issues#17235
• IUtf8SpanParsable - [Perf] Linux/x64: 14 Regressions on 7/12/2023 7:15:00 AM perf-autofiling-issues#19919
• DependencyInjection - [Perf] Linux/x64: 23 Regressions on 7/13/2023 8:36:41 PM perf-autofiling-issues#19921
• Utf8Encoding - [Perf] Linux/arm64: 10 Regressions on 8/12/2023 11:28:52 PM perf-autofiling-issues#20737
• FrozenDictionary missing optimizations - [Perf] Linux/x64: 28 Regressions on 12/11/2023 8:56:10 AM perf-autofiling-issues#26185
• SSA improvements - [Perf] Linux/x64: 8 Regressions on 4/17/2023 11:29:15 PM perf-autofiling-issues#16642

Details

RC2

Issue	Status
dotnet/perf-autofiling-issues#20737	Resolved
dotnet/perf-autofiling-issues#21828, dotnet/perf-autofiling-issues#21829, dotnet/perf-autofiling-issues#21830, dotnet/perf-autofiling-issues#21872, dotnet/perf-autofiling-issues#22250	Resolved
dotnet/perf-autofiling-issues#21832	Resolved

Preview 7

Issue	Status
dotnet/perf-autofiling-issues#19921	Resolved
dotnet/perf-autofiling-issues#19919	Resolved
dotnet/perf-autofiling-issues#19225	Resolved
dotnet/perf-autofiling-issues#19923	Resolved
dotnet/perf-autofiling-issues#19918	Resolved
dotnet/perf-autofiling-issues#19756	Resolved
dotnet/perf-autofiling-issues#19486	Resolved
dotnet/perf-autofiling-issues#19484	Resolved
dotnet/perf-autofiling-issues#19227	Resolved
dotnet/perf-autofiling-issues#19228	Resolved

Preview 6

Issue	Status
dotnet/perf-autofiling-issues#19016	Resolved
dotnet/perf-autofiling-issues#19018	Resolved
dotnet/perf-autofiling-issues#18528	Resolved

Preview 5

Issue	Status
dotnet/perf-autofiling-issues#18318	Resolved
dotnet/perf-autofiling-issues#17235	Resolved
dotnet/perf-autofiling-issues#18040	Resolved
dotnet/perf-autofiling-issues#17237	Resolved

Preview 4

Issue	Status
dotnet/perf-autofiling-issues#16642	Resolved
dotnet/perf-autofiling-issues#15719	Resolved
dotnet/perf-autofiling-issues#17018	Resolved
dotnet/perf-autofiling-issues#16546	Resolved
dotnet/perf-autofiling-issues#15724	Resolved
dotnet/perf-autofiling-issues#15626	Resolved
dotnet/perf-autofiling-issues#15154	Resolved
dotnet/perf-autofiling-issues#14530	Resolved

Preview 3

Issue	Status
dotnet/perf-autofiling-issues#13992	Resolved
dotnet/perf-autofiling-issues#13993	Resolved
dotnet/perf-autofiling-issues#13747	Resolved
dotnet/perf-autofiling-issues#13749	Resolved

Preview 2

Issue	Status
dotnet/perf-autofiling-issues#12707, dotnet/perf-autofiling-issues#11453	Resolved
#79973	Resolved
dotnet/perf-autofiling-issues#12299	Resolved
dotnet/perf-autofiling-issues#13746	Resolved
dotnet/perf-autofiling-issues#13199	Resolved

.NET 7 / Preview 1

Issue	Status
#79876	Resolved
#79336	Resolved
dotnet/perf-autofiling-issues#11449	Resolved
dotnet/perf-autofiling-issues#11147	Resolved
dotnet/perf-autofiling-issues#12501	Resolved
dotnet/perf-autofiling-issues#12965	Resolved
dotnet/perf-autofiling-issues#12081	Resolved
dotnet/perf-autofiling-issues#12962	Resolved
dotnet/perf-autofiling-issues#12080	Resolved
dotnet/perf-autofiling-issues#11791	Resolved
dotnet/perf-autofiling-issues#11790	Resolved
#78807	Resolved
dotnet/perf-autofiling-issues#12047	Resolved
dotnet/perf-autofiling-issues#11451	Resolved
#79604	Resolved
#78820	Resolved
dotnet/perf-autofiling-issues#11293	Resolved
#79606	Resolved
dotnet/perf-autofiling-issues#10933	Resolved
#79877	Resolved
#78815	Resolved
#79605	Resolved
#79607	Resolved
#78838	Resolved
#78826	Resolved
#78821	Resolved
#78980	Resolved
#78809	Resolved
#78812	Resolved
#78813	Resolved
#78814	Resolved
dotnet/perf-autofiling-issues#8641	Resolved
#79337	Resolved
#78810	Resolved
#78836	Resolved
#78825	Resolved
#78824	Resolved
#78823	Resolved
#78822	Resolved
#78837	Resolved
#78827	Resolved
#78835	Resolved
#78805	Resolved
dotnet/perf-autofiling-issues#10158	Resolved
#78804	Resolved
#78811	Resolved
#78808	Resolved
#78806	Resolved
#78819	Resolved
dotnet/perf-autofiling-issues#9346	Resolved
dotnet/perf-autofiling-issues#8642	Resolved
#77900	Resolved
dotnet/perf-autofiling-issues#8870	Resolved
dotnet/perf-autofiling-issues#9029	Resolved
dotnet/perf-autofiling-issues#9030	Resolved
dotnet/perf-autofiling-issues#9199	Resolved

Mono On Disk Size - iOS

Details

.NET 8

Date	Diff	Change	Size increase (bytes)	Size increase (%)	Tracking issue	Status
06 Dec 2022	b399eed...fcf0a09	62f3eb2	-	-	#79285
12 Apr 2022	e7ed417...7526a4c	#84289	590688	2.11	#85450
14 Apr 2022	2d833f4...9c02b25	#84587	351776	1.25	#85451
24 Apr 2022	223d152...9aefa9d	#84582	385449	1.32	#85452

[lewing]

cc @artl93 @stevisok @jeffschwMSFT

[kotlarmilos]

The .NET 9 issues have been moved to #98209. This issue will be closed after remaining regressions are addressed.

[lewing]

@DrewScoggins @kotlarmilos why aren't the wasm interpreter runs being labeled as interpreter runs and being tracked in #98209 ?

[kotlarmilos]

They are tracked separately in #96444.

[kotlarmilos]

@matouskozak Do we plan to triage tracked issues?

[matouskozak]

@matouskozak Do we plan to triage tracked issues?

I took a look at some MonoAOT ones, closed the ones that are no-longer regressed in .NET 9. We should align on what to do with the rest. Let's sync on Wednesday call and decide.

[kotlarmilos]

All issues have been transferred/resolved to .NET 9+.