Theme: Improve startup and throughput using runtime execution information (PGO)

[richlander]

Profile guided optimization is a standard feature of many development platforms. We would like to make it a standard feature of .NET. It is already a feature of .NET in that the platform is PGO-optimized, but it doesn't extend beyond that. PGO for .NET is a multi-release journey, with the goal of significantly improved startup and throughput performance and working set reduction using both automatic and opt-in mechanisms/scenarios.

Today, both the native runtime and managed libraries are PGO-compiled using training data that we produce ourselves. The native runtime uses standard C++ compiler tools. The managed libraries use a technology called "Instruction Block Count" (IBC). IBC is notoriously hard to use (only a few people on the .NET team can use it effectively). Going forward, we intend to produce new tools that are straightforward to use, for both training (that product PGO data) and PGO-optimized compilation (that consume PGO data).

A key split in the PGO domain is static versus dynamic PGO. We are interested in both, with the following prioritization/staging:

• Enable optimizing ready-to-run images based on static PGO data.
• Enable optimizing JITed code based on static PGO data, as a new feature of tiered compilation.
• Enable optimizing JITed code based on PGO data collected (and not persisted) at runtime.

More context:

• MPGO is a past attempt at achieving similar goals.
• Other platforms, like Rust support PGO.

User stories:

• PGO collection tools can be used to collect PGO data from apps in a variety of scenarios: local machine, VMs and containers. (Developers can use PGO collection tools to collect PGO data from apps in a variety of scenarios #5492)
• People that build .NET from source can use existing PGO data (stored in a repo in the dotnet org) to optimize the .NET runtime and libraries.
• .NET applications startup and run faster and use less memory, using static PGO data
  • Ready to run images are optimized using PGO data, including when assemblies are trimmed (which currently breaks the existing PGO system).
  • The JIT can use static PGO data, either when read-to-run images are not available, or when tiering from tier 0 to tier 1. The JIT should produce better code than what is available in ready-to-run images.
• .NET applications are progressively optimized at runtime, using dynamic PGO data, for example by inlining devirtualized virtual methods. (Developers can progressively optimize their .NET apps at runtime, using dynamic PGO data #5494)

[danmoseley]

@billwert @DrewScoggins @davidwrighton will our microbenchmark data continue to be gathered on pre-PGO bits, or on PGOized bits? My assumption is the former, because (1) PGO may introduce unexpected "regressions" and "improvements" when training data is updated, which would hide real regressions (2) PGO is not designed to improve microbenchmarks, in general. cc @stephentoub

I recall working on VS that when optimization data changed (or grew stale), perf moved around - but in that case, we were measuring E2E scenarios, which is what the optimization was specifically for, and it was valuable to know that e.g, the data had become stale.

[DrewScoggins]

My understanding is that when you do a normal release build you get PGO data applied. This is the pattern that we follow. I know that we do not use the JIT PGO opt-out currently. I know that in the past we have been able to see the signs of stale PGO data in microbenchmark numbers, but I will admit I am not an expert in this space.