Huge performance drop in opening projects using MSBuildWorkspace

[jzielnik]

Version Used: 4.12.0 (versus 4.8.0)

Steps to Reproduce:

1. Open project using MSBuildWorkspace (we use loop to iterate over all projects in solution)

var workspace = MSBuildWorkspace.Create(additionalProperties);
Project project = workspace.OpenProjectAsync(projectPath).Result;

2. Measure performance of opening projects in both versions of Microsoft.CodeAnalysis nugets.
3. We have used following open-sourced projects:

• SharpDevelop: https://github.com/icsharpcode/SharpDevelop/tree/master
• SimplCommerce: https://github.com/simplcommerce/SimplCommerce
• CAP: https://github.com/dotnetcore/CAP/tree/master/src

Expected Behavior: Time of opening projects between versions should not be that different.

Actual Behavior: Performance of the opening projects has decreased significantly. I suspect this could be related to moving Roslyn engine to separate exe which is BuildHost.

This performance issue could be introduced with 4.9.0 when BuildHost was introduced, but we have updated directly from 4.8.0 to 4.12.0 so we didn't observe performance between these versions.

version 4.8.0

Solution      | Number of projects | Total time [s] | MAX [s] | AVG [s]
SharpDevelop  |         44         |      14.40     |  5.80   | 0.33
SimplCommerce |         34         |      13.97     |  6.34   | 0.41
CAP           |         31         |      11.66     |  5.35   | 0.38

version 4.12.0 (in brackets there is percentage increase)

Solution      | Number of projects | Total time [s] | MAX [s] | AVG [s]
SharpDevelop  |         44         |  119.24 (828%) |  10.24  | 2.71
SimplCommerce |         34         |   80.36 (575%) |   8.22  | 2.36
CAP           |         31         |   48.85 (419%) |   9.18  | 1.58

[jasonmalinowski]

Not that we can't look at places to improve performance, but this jumped out to me:

Open project using MSBuildWorkspace (we use loop to iterate over all projects in solution)

Is there a reason this wouldn't be opening the solution directly and looking at each of the projects that came out there?

[jzielnik]

Not that we can't look at places to improve performance, but this jumped out to me:

Open project using MSBuildWorkspace (we use loop to iterate over all projects in solution)

Is there a reason this wouldn't be opening the solution directly and looking at each of the projects that came out there?

Thanks for suggestion. We've switched to opening the entire solution at once. The performance is still worse than original 4.8.0 version (about 20-40% rather than 400-800%) but acceptable.

[jasonmalinowski]

We can still dig for further improvements here, but there will always be some extra cost to the build host process no matter what. And given the accuracy that brings...that's going to be worth it long term.